How to find a subimag using the PIL library?

Question

I want to find a sub-image from a large image using the PIL library. I also want to know the coordinates, where is it found?

Answer 1

import cv2 import numpy as np image = cv2.imread("Large.png") template = cv2.imread("small.png") result = cv2.matchTemplate(image,template,cv2.TM_CCOEFF_NORMED) print np.unravel_index(result.argmax(),result.shape) 

It works perfectly and efficiently for me.

Answer 2

I only managed to do this with PIL.

Some reservations:

• This is a perfect pixel search. It just searches for matching RGB pixels.
• For simplicity, I remove the alpha / transparency channel. I am only looking for RGB pixels.
• This code loads the entire array of pixels in pixels into memory, storing a large image out of memory. On my system, Python supported about 26 megabytes of memory for a tiny 40mm image viewing a 1920x1200 screenshot.
• This simple example is not very effective, but increasing efficiency will add complexity. Here I keep everything straight and easy to understand.
• This example works on Windows and OSX. Not tested on Linux. It takes a screenshot of only the main display (for settings of multiple monitors).

Here is the code:

 import os from itertools import izip from PIL import Image, ImageGrab def iter_rows(pil_image): """Yield tuple of pixels for each row in the image. From: http://stackoverflow.com/a/1625023/1198943 :param PIL.Image.Image pil_image: Image to read from. :return: Yields rows. :rtype: tuple """ iterator = izip(*(iter(pil_image.getdata()),) * pil_image.width) for row in iterator: yield row def find_subimage(large_image, subimg_path): """Find subimg coords in large_image. Strip transparency for simplicity. :param PIL.Image.Image large_image: Screen shot to search through. :param str subimg_path: Path to subimage file. :return: X and Y coordinates of top-left corner of subimage. :rtype: tuple """ # Load subimage into memory. with Image.open(subimg_path) as rgba, rgba.convert(mode='RGB') as subimg: si_pixels = list(subimg.getdata()) si_width = subimg.width si_height = subimg.height si_first_row = tuple(si_pixels[:si_width]) si_first_row_set = set(si_first_row) # To speed up the search. si_first_pixel = si_first_row[0] # Look for first row in large_image, then crop and compare pixel arrays. for y_pos, row in enumerate(iter_rows(large_image)): if si_first_row_set - set(row): continue # Some pixels not found. for x_pos in range(large_image.width - si_width + 1): if row[x_pos] != si_first_pixel: continue # Pixel does not match. if row[x_pos:x_pos + si_width] != si_first_row: continue # First row does not match. box = x_pos, y_pos, x_pos + si_width, y_pos + si_height with large_image.crop(box) as cropped: if list(cropped.getdata()) == si_pixels: # We found our match! return x_pos, y_pos def find(subimg_path): """Take a screenshot and find the subimage within it. :param str subimg_path: Path to subimage file. """ assert os.path.isfile(subimg_path) # Take screenshot. with ImageGrab.grab() as rgba, rgba.convert(mode='RGB') as screenshot: print find_subimage(screenshot, subimg_path) 

Speed:

 $ python -m timeit -n1 -s "from tests.screenshot import find" "find('subimg.png')" (429, 361) (465, 388) (536, 426) 1 loops, best of 3: 316 msec per loop 

During the execution of the above command, I moved the window containing the prototype diagonally when timeit was timeit .

Answer 3

It looks like you want to perform object discovery, perhaps through pattern matching. This is not a trivial problem if you are not looking for an exact match in half, and PIL is not designed for this kind of thing.

Ian is right that you should try OpenCV. It is a robust computer vision library with good Python connections.

Here is a good example in Python that draws a rectangle around a consistent area: https://github.com/jungilhan/Tutorial/blob/master/OpenCV/templateMatching.py