Computer vision for calculating the ratio of numbers (fingers)

Question

If someone scans their right hand, pressed to the scanner glass, the result will look like this:

(without orange and white annotations). How can we identify someone with a 2D: 4D ratio from their hand image?

Answer 1

You already noted this opencv, which is great - I would highly recommend looking at openFrameworks and addCV openCV, as the main examples there will give you great starting blocks for this.

The general approach to this, which I would like to do, is to first transfer the image to light and dark areas, detect the edges of the hands and fingers, and then simplify your data until you have lines representing the edges and fingertips. Finally, grab the bottom nest between the 2nd and 3rd fingers, stop at the tip of the 2nd and the second of the 3rd and 4th, stop at the tip of the 4th, which should give you a 2D: 4D ratio.

First you need to process your images to get black and white images that openCV can handle easily. You may need to play with different thresholds to get both the outline of the hand and the fingers that you need to detect. (You may need two passes to discover both the contour and inseams)

While there are many approaches to detecting a function, OpenCV usually returns arrays of "blobs". With the correct thresholds, I believe that you can reliably and simply find adjacent horizontal drops (or almost adjacent, allowing some distance between adjacent drops) for the inner fingers of each finger.

A simple algorithm for detecting inseams would be to go through the detected blobs, starting from the upper left corner and go from left to right through the image, as if reading a page. Gather an array of detected horizontal lines from droplets in the image and play with different image processing thresholds, minimum acceptable line lengths and acceptable distances between detected drops, which you still consider part of the same line, until you are satisfied that you are good at detecting the edges of the fingers .

Once you find the horizontal lines, you can process the drops again by looking for the vertical lines that represent the fingertips (stopping when you hit the previously detected horizontal lines).

Finally, find the lines that represent the correct inseams, measure them until they intersect with the corresponding fingertips, and you should have your own ratio!

Answer 2

Interest Ask. I would do this:

First, align the image with the Otsu threshold. Then find the image skeleton using the Medial-Axis Transform (MAT). This would mean doing a distance conversion on the image, and then using an adaptive threshold value to get local maxima in the distance conversion. This gives a rough and finished skeleton of your image. Sample code from here.

The resulting manual skeleton may be slightly disabled, in which case the OpenCV function "CLOSE" morphology (not "open") function may connect it to one skeleton. Then, checking for defects in the bulge of the arm obtained should give an assessment.