OpenCV 2.2 SURF Feature Match Issues

Question

I changed the OpenCV demo application "matching_to_many_images.cpp" to request an image (on the left) for frames from a webcam (on the right). What went wrong in the upper right corner of the first image?

We believe that this is due to another problem that we have. We start with an empty database and add only unique functions that do not correspond to the functions of our database, but after adding only three functions we get the correspondence to all new functions ....

we use: SurfFeatureDetector surfFeatureDetector (400,3,4); SurfDescriptorExtractor surfDescriptorExtractor; FlannBasedMatcher flannDescriptorMatcher;

The full code can be found at: http://www.copypastecode.com/71973/

Answer 1

I think this is due to boundary key points. The detector detects key points, but for the SURF descriptor to return consistent values, it needs pixel data in a block of pixels around it, which is not available in border pixels. You can use the following snippet to remove boundary points after detecting key points, but before the descriptors are computed. I suggest using borderSize 20 or more.

removeBorderKeypoints( vector<cv::KeyPoint>& keypoints, const cv::Size imageSize, const boost::int32_t borderSize ) { if( borderSize > 0) { keypoints.erase( remove_if(keypoints.begin(), keypoints.end(), RoiPredicatePic((float)borderSize, (float)borderSize, (float)(imageSize.width - borderSize), (float)(imageSize.height - borderSize))), keypoints.end() ); } } 

Where RoiPredicatePic is implemented as:

 struct RoiPredicatePic { RoiPredicatePic(float _minX, float _minY, float _maxX, float _maxY) : minX(_minX), minY(_minY), maxX(_maxX), maxY(_maxY) {} bool operator()( const cv::KeyPoint& keyPt) const { cv::Point2f pt = keyPt.pt; return (pt.x < minX) || (pt.x >= maxX) || (pt.y < minY) || (pt.y >= maxY); } float minX, minY, maxX, maxY; }; 

In addition, approximate indexing of the nearest neighbor is not the best way to map functions between pairs of images. I suggest you try other simple helpers.

Answer 2

Your approach works flawlessly, but it shows incorrect results due to an incorrect call to the drawMatches function.

Your wrong call was something like this:

 drawMatches(image2, image2Keypoints, image1, image1Keypoints, matches, result); 

The correct call should be:

 drawMatches(image1, image1Keypoints, image2, image2Keypoints, matches, result); 

Answer 3

I ran into the same problem. Surprisingly, the solution has nothing to do with the boundary points or with the KNN connector. You just need a different matching strategy to filter out “good matches” from multiple matches.

Use 2 NN search and the following condition

if the distance (1st match) 0.6 * distance (2nd match) 1st match - “a good match”.

Filter out all matches that do not satisfy the above condition, and call drawMatches only for "good matches." Voila!