Finding shapes in an image using opencv

Question

I am trying to search for shapes in an image using OpenCV. I know the forms that I want to compare (there are some forms that I do not know about, but I do not need to find them) and their orientation. I do not know their size (scale) and location.

My current approach:

• Edge detection
• For each path, calculate the maximum bounding box.
• Align each bounding box with one of the famous shapes separately. In my real project, I scale the area to the size of the template and calculate the differences in the Sobel gradient, but for this demonstration I just use the aspect ratio.

Where this approach is canceled, a touch of figures. Contour detection picks up two adjacent shapes as a single contour (single bounding box). Accordingly, the match step will be unsuccessful.

Is there a way to change my approach for handling contiguous forms separately? Also, is there a better way to complete step 3?

For example: (color green, color blue Ys)

Bad case: (unknown figure in red)

Source:

import cv import sys E = cv.LoadImage('e.png') E_ratio = float(E.width)/E.height Y = cv.LoadImage('y.png') Y_ratio = float(Y.width)/Y.height EPSILON = 0.1 im = cv.LoadImage(sys.argv[1], cv.CV_LOAD_IMAGE_GRAYSCALE) storage = cv.CreateMemStorage(0) seq = cv.FindContours(im, storage, cv.CV_RETR_EXTERNAL, cv.CV_CHAIN_APPROX_SIMPLE) regions = [] while seq: pts = [ pt for pt in seq ] x, y = zip(*pts) min_x, min_y = min(x), min(y) width, height = max(x) - min_x + 1, max(y) - min_y + 1 regions.append((min_x, min_y, width, height)) seq = seq.h_next() rgb = cv.LoadImage(sys.argv[1], cv.CV_LOAD_IMAGE_COLOR) for x,y,width,height in regions: pt1 = x,y pt2 = x+width,y+height if abs(float(width)/height - E_ratio) < EPSILON: color = (0,255,0,0) elif abs(float(width)/height - Y_ratio) < EPSILON: color = (255,0,0,0) else: color = (0,0,255,0) cv.Rectangle(rgb, pt1, pt2, color, 2) cv.ShowImage('rgb', rgb) cv.WaitKey(0) 

e.png:

y.png:

well:

badly:

Before anyone asks, no, I’m not trying to break the captcha :) OCD by itself does not really matter here: the actual figures in my real project are not characters - I'm just lazy and characters are the easiest thing to draw (and still detected by trivial methods).

Answer 1

Since your shapes can vary in size and ratio, you should look at scaling invariant descriptors. The letter of such descriptors would be ideal for your application.

Process these descriptors on a test pattern, and then use some kind of simple classification to retrieve them. It should give good results with the simple shapes that you show.

I used Zernike and Hu in the past, the last of which was the most famous. Here you can find an example implementation: http://www.lengrand.fr/2011/11/classification-hu-and-zernike-moments-matlab/ .

Other: given your problem, you should look at OCR technologies (means optical character recognition: http://en.wikipedia.org/wiki/Optical_character_recognition ;)).

Hope this helps a bit.

Julien

Answer 2

Whether you try to copy the match or contour match (match) using CCH as a handle.

Bevel Alignment uses remote transformation of the target image and the outline of the template. not exactly large-scale, but fast.

The latter is rather slow, since the complexity is at least quadratic for the bipartite comparison problem. on the other hand, this method is invariant to scale, rotation, and probably local distortion (for approximate correspondence, which IMHO is good for the bad example above).