Digitizing crosswords using image processing

Question

I am new to image processing. I need to convert a crossword to an image file in the corresponding binary equivalent. The output should be an array with 1 for black squares and 0 for white squares. Also, all other extraneous information in the image, such as text, etc., should be ignored and digitized only the grid.

Can the corner detection algorithm help us determine the crossword puzzle angles and then digitize the blocks of pixels accordingly using brute force? Is this the best way to do this or are there effective methods to accomplish the task? I would prefer a python based solution.

Answer 1

I think you do not need to use angular detection here. Just using the outlines, you can solve it (if your images are straightforward). Below is the code that prints the array for your image above. Codes commented on:

import numpy as np import cv2 img = cv2.imread('cross.jpg') gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY) ret, thresh = cv2.threshold(gray,127,255,cv2.THRESH_BINARY_INV) thresh2 = cv2.bitwise_not(thresh) contours,hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, 1) max_area = -1 # find contours with maximum area for cnt in contours: approx = cv2.approxPolyDP(cnt, 0.02*cv2.arcLength(cnt,True), True) if len(approx) == 4: if cv2.contourArea(cnt) > max_area: max_area = cv2.contourArea(cnt) max_cnt = cnt max_approx = approx # cut the crossword region, and resize it to a standard size of 130x130 x,y,w,h = cv2.boundingRect(max_cnt) cross_rect = thresh2[y:y+h, x:x+w] cross_rect = cv2.resize(cross_rect,(130,130)) # you need to uncomment these lines if your image is rotated #new_pts = np.float32([[0,0], [0,129],[129,129],[129,0]]) #old_pts = max_approx.reshape(4,2).astype('float32') #M = cv2.getPerspectiveTransform(old_pts,new_pts) #cross_rect = cv2.warpPerspective(thresh2,M,(130,130)) cross = np.zeros((13,13)) # select each box, if number of white pixels is more than 50, it is white box for i in xrange(13): for j in xrange(13): box = cross_rect[i*10:(i+1)*10, j*10:(j+1)*10] if cv2.countNonZero(box) > 50: cross.itemset((i,j),1) print cross 

I have this output for your image above:

 [[ 0. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1.] [ 1. 0. 1. 0. 1. 0. 1. 0. 0. 0. 1. 0. 1.] [ 1. 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1.] [ 1. 0. 1. 0. 1. 0. 1. 0. 1. 0. 1. 0. 1.] [ 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 1.] [ 1. 0. 0. 0. 1. 0. 1. 0. 1. 0. 1. 0. 1.] [ 1. 1. 1. 1. 0. 0. 0. 0. 0. 1. 1. 1. 1.] [ 1. 0. 1. 0. 1. 0. 1. 0. 1. 0. 0. 0. 1.] [ 1. 1. 1. 1. 1. 1. 1. 0. 1. 1. 1. 1. 1.] [ 1. 0. 1. 0. 1. 0. 1. 0. 1. 0. 1. 0. 1.] [ 1. 1. 1. 1. 0. 1. 1. 1. 1. 1. 1. 1. 1.] [ 1. 0. 1. 0. 0. 0. 1. 0. 1. 0. 1. 0. 1.] [ 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0.]] 

I wrote code to extract sudoku from an image with a detailed explanation from the links below (the textbook is incomplete). You can contact them for more information:

http://opencvpython.blogspot.com/2012/06/sudoku-solver-part-1.html

http://opencvpython.blogspot.com/2012/06/sudoku-solver-part-2.html

http://opencvpython.blogspot.com/2012/06/some-common-questions.html

Answer 2

Corner detection can find corners of text, etc. - you have to try it.

I would start by searching for strings using the Hough transform http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/hough_lines/hough_lines.html . Once you find all the straight lines, you can find the corners of the squares by finding the intersections of the lines. Once you know where the squares are, it would be easy to determine if the square is black or white.

You can also write a Hough transform to detect a rectangle, but this is probably an unnecessary complication.

Whether you are doing this in Python, C ++, Java is not a big deal.