How to find letter in image using python

Question

---

I have an image that contains some values ​​in meters written by the manuscript, I would like to find the position of the letter m so that I can crop it and leave only the number.

This is an example:

Original Image: Input images are as follows: this is actually the best handwriting I can get, usually it’s much worse.

Image of a train: I have a list of many types of letters m, cut from different handwritten images that I have.

Resulting image: The result I want to get

I already tried using the opencv pattern matching functions, but it did not work, also found this github , but it also uses pattern matching. I would like to know if there is another way to solve this problem.

Answer 1

It seems that the letter is always at the end of the number. If so, you can follow a much simpler approach:

• Find all contours;

2. Create a list of bounding rectangles (i.e. one field for each path);

3. Determine which one is the most bounding rectangle;

4. Use the information (x, y, width, height) of all other fields to create an ROI and crop only numbers;

Source code for Python 2.7 and OpenCV 2.4 :

import cv2 ### load input image and convert it to grayscale img = cv2.imread("input.png") print("img shape=", img.shape) gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #### extract all contours _, contours, _ = cv2.findContours(gray.copy(), cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) # debug: draw all contours #cv2.drawContours(img, contours, -1, (0, 0, 255), 2) #cv2.imwrite("all_contours.jpg", img) #### create one bounding box for every contour found bb_list = [] for c in contours: bb = cv2.boundingRect(c) # save all boxes except the one that has the exact dimensions of the image (x, y, width, height) if (bb[0] == 0 and bb[1] == 0 and bb[2] == img.shape[1] and bb[3] == img.shape[0]): continue bb_list.append(bb) # debug: draw boxes #img_boxes = img.copy() #for bb in bb_list: # x,y,w,h = bb # cv2.rectangle(img_boxes, (x, y), (x+w, y+h), (0, 0, 255), 2) #cv2.imwrite("boxes.jpg", img_boxes) #### sort bounding boxes by the X value: first item is the left-most box bb_list.sort(key=lambda x:x[0]) # debug: draw the last box of the list (letter M) #print("letter M @ ", bb_list[-1]) #x,y,w,h = bb_list[-1] #cv2.rectangle(img, (x, y), (x+w, y+h), (0, 0, 255), 2) #cv2.imwrite("last_contour.jpg", img) ### remove the last item from the list, ie remove box for letter M bb_list = bb_list[:-1] ### and now the fun part: create one large bounding box to rule them all x_start, _, _, _ = bb_list[0] x_end, _, w_end, _ = bb_list[-1] x = x_start w = (x_end + w_end) - x_start bb_list.sort(key=lambda y:y[1]) # sort by Y value: the first item has the smallest Y value _, y, _, _ = bb_list[0] bb_list.sort(key=lambda y:y[3]) # sort by Height value: the last item has the largest Height value _, _, _, h = bb_list[-1] print("x=", x, "y=", y, "w=", w, "h=", h) # debug: draw the final region of interest roi_img = img.copy() cv2.rectangle(roi_img, (x, y), (x+w, y+h), (0, 0, 255), 2) cv2.imwrite("roi.jpg", roi_img) # crop to the roi crop_img = img[y:y+h, x:x+w] cv2.imwrite("crop.jpg", crop_img) 

Answer 2

How about calculating adjacent sets of points and removing the last “right” in coordinates from the image? which gets rid of the letter m

Usually reads this to understand this idea: Labeling of connected components

A related SO post with Python on a connected component in terms of pixels