handle shape as a polygon
convert your points (each line) into a set of lines (length,angle) , as in this image:

this provides rotation / translation invariance. If you see more lines with angle=PI , combine them together to avoid the coincidence of the same shapes with a different selection, also try to match the same CW / CCW rule of the polygonal winding for both shapes.
find starting point
May be the largest or smallest angle, length ... or a specific order of angles+lengths . Therefore, reorder the lines of one polygon (cyclic shift) so that your shapes compare with the "same point" if they can.
comparison - for exact match
- the number of lines should be the same
- perimeters should be the same +/- some accuracy
for example:
fabs (sum of all lengths of poly1 - sum of all lengths of poly2) <= 1e-3
If not different shapes. Then compare all the lengths and angles. If any value differs more than the value of accuracy, then the shapes are different.
comparison - size doesn't matter
calculate the perimeter of both polygons l1,l2 and change the size of all lengths of the compared poly2 to match the perimeter of poly1 so that all lengths of poly2 multiplied by value = l1/l2; . After that use comparison with bullet # 3
comparison - shape deviations can still have a positive result (the size should be the same)
try setting the number of rows to the same value (connect all rows with an angle close to PI ). Then the perimeters must "correspond" ... fabs(l1-l2)<=1e-3*l1 . You can use bullet comparison # 4
comparison - deviations in size and shape may still correspond
just resize poly2 to fit the perimeter of poly1 like in pool # 4 and then use bullet # 5
If you cannot find the starting point in the polygons of the stand (bullet # 2 )
Then you need to check all the shifts of the starting point, so if your polygons have a stand of 5 lines:
poly1: l11,l12,l13,l14,l15 poly2: l21,l22,l23,l24,l25
Then you need to compare all 5 combinations (unless you have found a match before):
cmp (l11,l12,l13,l14,l15),(l21,l22,l23,l24,l25) cmp (l11,l12,l13,l14,l15),(l22,l23,l24,l25,l21) cmp (l11,l12,l13,l14,l15),(l22,l23,l24,l25,l21) cmp (l11,l12,l13,l14,l15),(l23,l24,l25,l21,l22) cmp (l11,l12,l13,l14,l15),(l24,l25,l21,l22,l23) cmp (l11,l12,l13,l14,l15),(l25,l21,l22,l23,l24)
[Note]
There are also faster comparison methods, but in some cases they may skip
- you can compare histograms of rows, angles
- you can use a neural network (I donβt like them, but they are ideal for such classifications)
if your figures should be oriented in the same way (without rotation invariance)
then instead of the vertex angle use the line direction angle
if you cannot provide the same winding rule for both compared polygons
then you should check their stand:
cmp (l11,l12,l13,l14,l15),(l21,l22,l23,l24,l25) cmp (l11,l12,l13,l14,l15),(l25,l24,l23,l22,l21)
I know this is a slightly vague answer, but still hope this helps at least a little ...