OpenCV - C ++ for Java - Pattern Matching

Question

I am trying to detect more than a square (marker) in my image, as I ask here Detecting marker position in a 2D image

There is one guy who showed me a solution in C ++, here it is:

#include <iostream> #include <opencv2/highgui/highgui.hpp> #include <opencv2/imgproc/imgproc.hpp> //See: http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html //See: http://answers.opencv.org/question/60382/detect-markers-position-in-2d-images/ int main() { cv::Mat img, templateImg, result; cv::VideoCapture capture("http://answers.opencv.org/upfiles/14297307634571599.png"); if(capture.isOpened()) { capture >> img; } else { return -1; } capture = cv::VideoCapture("http://answers.opencv.org/upfiles/14297308125543022.png"); if(capture.isOpened()) { capture >> templateImg; } else { return -1; } /// Reduce the size of the image to display it on my screen cv::resize(img, img, cv::Size(), 0.5, 0.5); /// Reduce the size of the template image /// (first to fit the size used to create the image test, second to fit the size of the reduced image) cv::resize(templateImg, templateImg, cv::Size(), 0.25, 0.25); cv::Mat img_display; img.copyTo(img_display); // Create the result matrix int result_cols = img.cols - templateImg.cols + 1; int result_rows = img.rows - templateImg.rows + 1; result.create(result_rows, result_cols, CV_32FC1); /// Do the Matching and Normalize cv::matchTemplate(img, templateImg, result, CV_TM_CCORR_NORMED); cv::normalize(result, result, 0, 1, cv::NORM_MINMAX, -1, cv::Mat()); /// Localizing the best match with minMaxLoc double minVal; double maxVal; cv::Point minLoc; cv::Point maxLoc; cv::Point matchLoc; for(;;) { cv::minMaxLoc(result, &minVal, &maxVal, &minLoc, &maxLoc, cv::Mat()); matchLoc = maxLoc; std::cout << "Max correlation=" << maxVal << std::endl; if(maxVal < 0.8) { break; } /// Show me what you got cv::rectangle(img_display, matchLoc, cv::Point(matchLoc.x + templateImg.cols , matchLoc.y + templateImg.rows), cv::Scalar::all(0), 2, 8, 0); cv::rectangle(result, cv::Point(matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2), cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), 2, 8, 0); cv::imshow("result", result); cv::waitKey(0); /// Fill the detected location with a rectangle of zero cv::rectangle(result, cv::Point( matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2), cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), -1); } while (maxVal > 0.9); cv::imshow("result", result); cv::imshow("img_display", img_display); cv::waitKey(0); return 0; } 

The for loop is responsible for detecting several markers and detecting them, I try to adapt it to my java code, and I get an infinite loop here, this is my code:

 public void run(String inFile, String templateFile, String outFile, int match_method) { System.out.println("\nRunning Template Matching"); Mat img = Highgui.imread(inFile); Mat templ = Highgui.imread(templateFile); // / Create the result matrix int result_cols = img.cols() - templ.cols() + 1; int result_rows = img.rows() - templ.rows() + 1; Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1); // / Do the Matching and Normalize Imgproc.matchTemplate(img, templ, result, match_method); Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat()); Point matchLoc; Point maxLoc; Point minLoc; MinMaxLocResult mmr; boolean iterate = true; while(iterate){ // / Localizing the best match with minMaxLoc mmr = Core.minMaxLoc(result); matchLoc = mmr.maxLoc; if(mmr.maxVal < 0.8) { iterate = false; } // / Show me what you got Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()), new Scalar(0, 255, 0)); } // Save the visualized detection. System.out.println("Writing "+ outFile); Highgui.imwrite(outFile, img); } 

I noticed that the minMaxLoc function has more arguments in C ++ than in java, maybe the problem? Why can't I get the same behavior in java, can someone help me?

Thank you so much in advance

Answer 1

As Kiriel said, you forgot to fill in the zeros for the maximum location , and therefore you got an infinite loop. Maybe he forgot to explain to you that,

  for each iteration find the max location check if max value is greater than desired threshold if true show me what is max else break // not found anything that matches make the existing max to be zero and continue to search for other max// you forgot this and hence infinite loop end 

For a deeper understanding of the problem, you can comment on the following lines in C ++ code and run it, you will have a similar problem. (Here cv :: Scalar :: all (0), - 1 args says to fill the existing max / found area with 0 and continue)

 // Fill the detected location with a rectangle of zero cv::rectangle(result, cv::Point(matchLoc.x - templateImg.cols / 2, matchLoc.y - templateImg.rows / 2), cv::Point(matchLoc.x + templateImg.cols / 2, matchLoc.y + templateImg.rows / 2), cv::Scalar::all(0), -1); 

Hope this helps.

Answer 2

You forgot to draw a second rectangle. Source:

  cv::rectangle(result, cv::Point(matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2), cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), 2, 8, 0); ... /// Fill the detected location with a rectangle of zero cv::rectangle(result, cv::Point( matchLoc.x - templateImg.cols/2 , matchLoc.y - templateImg.rows/2), cv::Point(matchLoc.x + templateImg.cols/2 , matchLoc.y + templateImg.rows/2 ), cv::Scalar::all(0), -1); 

however, in your code there is only the following:

 Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()), new Scalar(0, 255, 0)); 

The second part of the source code, most likely, is crucial - it draws a completed, and not just a form similar to the first line.

By the way, this line is Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()), new Scalar(0, 255, 0)); also probably incorrect - 1) You should draw a rectangle on the result layout, and not on the img mat. 2) The source code contains Core.rectangle(img, matchLoc, new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()), new Scalar(0, 255, 0)); and in your code new Point(matchLoc.x + templ.cols(), matchLoc.y + templ.rows()) . Are you sure everything is in order?

Answer 3

Although this may help, you do not need OpenCV / JavaCV for this relatively simple task.

Basically, you want to find images within you that:

 a: all black b: square c: a percentage of size of the total image 

Firstly, a threshold image so that all pixels are either black or white.

My approach would be to find connected components by scanning the image pixel by pixel until you reach the black pixel. Then "fill fill" until you get all the black pixels. Mark them (color them differently) and save them in some structure (list of points) and repeat until you reach the last pixel of the image.

For each component:

• define a bounding box (min and max x and y values).
• Calculate the window size and determine if it is large enough / large.
• Calculate the percentage of comparable pixels in this field. If it's about 100%, you have a square shape.

You can map the corresponding bounding fields to the original image.