How to programmatically find the pixel arrangement of certain objects in an image?

Question

I am creating an automated electricity / gas reader using OpenCV and Python. I got to the webcam:

Then I can use the afine transform to reject the image (adaptation of this example ):

def unwarp_image(img): rows,cols = img.shape[:2] # Source points left_top = 12 left_bottom = left_top+2 top_left = 24 top_right = 13 bottom = 47 right = 180 srcTri = np.array([(left_top,top_left),(right,top_right),(left_bottom,bottom)], np.float32) # Corresponding Destination Points. Remember, both sets are of float32 type dst_height=30 dstTri = np.array([(0,0),(cols-1,0),(0,dst_height)],np.float32) # Affine Transformation warp_mat = cv2.getAffineTransform(srcTri,dstTri) # Generating affine transform matrix of size 2x3 dst = cv2.warpAffine(img,warp_mat,(cols,dst_height)) # Now transform the image, notice dst_size=(cols,rows), not (rows,cols) #cv2.imshow("crop_img", dst) #cv2.waitKey(0) return dst 

.. which gives me an image something like this:

I still need to extract the text using some kind of OCR procedure, but first I would like to automate the part that identifies which pixel locations are used for affinity conversion. Therefore, if someone knocks on a webcam, this does not stop the software.

Answer 1

Since your image is quite flat, you can find homography between the image you get from the webcam and the desired image (in vertical position).

Edit: this will rotate the image vertically. After you registered your image (lifted it upright), you could make predictions by rows or columns (sum all the pixels along the columns to get one vector, sum all the pixels along the lines to get one vector). You can use these vectors to find out where you have the color jump, and crop it there.

Alternatively, you can use the Hough transform, which gives you lines in the image. You may be able to avoid registering the image if you do.