Image Detail Using OpenCV

Question

I'm trying to get into the seams. I am using cv::detail .

I am trying to follow this example :

I roughly understand the cross-linking pipe.

there is a function matchesGraphAsString() that returns a graph. I am wondering how he calculates this graph. In addition, what is the dfination of the confidence interval in this case.

The output is in DOT format, and the approximate schedule looks like

 graph matches_graph{ "15.jpg" -- "13.jpg"[label="Nm=75, Ni=50, C=1.63934"]; "15.jpg" -- "12.jpg"[label="Nm=47, Ni=28, C=1.26697"]; "15.jpg" -- "14.jpg"[label="Nm=149, Ni=117, C=2.22011"]; "11.jpg" -- "13.jpg"[label="Nm=71, Ni=52, C=1.77474"]; "11.jpg" -- "9.jpg"[label="Nm=46, Ni=37, C=1.69725"]; "11.jpg" -- "10.jpg"[label="Nm=87, Ni=73, C=2.14076"]; "9.jpg" -- "8.jpg"[label="Nm=122, Ni=99, C=2.21973"]; } 

What does label , Nm and Ni mean here? It appears that the official document is missing this data.

Answer 1

This is a very interesting question. As @hatboyzero noted, the meaning of variables is quite simple:

• Nm is the number of matches (in the overlap area, so obvious outliers have already been removed).
• Ni is the number of rulers after finding homography with Ransac .
• C is the confidence that two images match.

Background to match

A panorama is constructed by searching for points of interest in all images and computation descriptors for them. These descriptors, such as SIFT, SURF, and ORB, were designed to detect the same parts of the image. This is just a medium-sized vector (typical 64 or 128 dimensions). By calculating L2 or another distance between two descriptors, we can find matches. The number of matches in a pair of images is described by the term Nm .

Note that so far, matching has only been done through the appearance of image areas around points of interest. As a rule, many of these coincidences are erroneous. This may be due to the fact that the descriptor looks the same (think: a repeating object, for example, window sills in a multi-line building or leaves on a tree) or because the descriptor is simply too uninformative.

A common solution is to add geometric constraints: a pair of images was taken from the same position with the same camera, so points close to one image should be close to another image. In particular, all points had to go through the same transformation. In the panoramic case where the camera was rotated around the nodal point of the camera lens, this transformation should be 2D homography.

Ransac is the gold standard algorithm for finding the best conversion and all matches matching that transformation. The number of these matched matches is called Ni . Ransac works by randomly selecting 4 matches in this case (see Section "Section 3.1") and fitting homography to these four matches. Then count how many matches out of all possible matches will agree with this homography. Repeat 500 times (see Paper) and at the end take the model that mattered the most. Then recount the model with all the values. The name of the algorithm comes from RANdom SAmple Consensus: RanSaC.

Confidence term

The question for me was about this mysterious certainty. I quickly found where it was designed.

From stitching/sources/matches.cpp :

 // These coeffs are from paper M. Brown and D. Lowe. "Automatic Panoramic Image Stitching // using Invariant Features" matches_info.confidence = matches_info.num_inliers / (8 + 0.3 * matches_info.matches.size()); // Set zero confidence to remove matches between too close images, as they don't provide // additional information anyway. The threshold was set experimentally. matches_info.confidence = matches_info.confidence > 3. ? 0. : matches_info.confidence; 

the mentioned article has in section 3.2 ("A probabilistic model for checking the conformity of images") in more detail what this means.

This section has highlighted several things.

• Their model has many variables (mostly probabilities). These values ​​are defined in the document without any justification. The following is a key suggestion:

Although in practice we have chosen values ​​for p0, p1, p (m = 0), p (m = 1) and pmin, they could, in principle, be extracted from the data.

So, this is just a theoretical exercise, because the parameters were torn from the air. Please note that in principle one could find out.

2. Equation 13 provides a confidence calculation. If you read correctly, this means that matches_info.confidence indicates the correct match between two images if its value is above 1.

3. I see no excuses for eliminating compliance (setting confidence to 0) when confidence is above 3. This simply means that there are very few outliers. I think that programmers thought that the large number of matches that turn outlier means that the images overlap a lot, but this is not provided by the algorithms behind it. (Simply, comparisons are based on the appearance of symptoms.)

Answer 2

Looking at the OpenCV source code available on the Internet, I understand that they mean the following:

• Nm - the number of pairwise matches
• Ni - Number of geometrically consistent matches
• C. Confidence. Two images from one panorama.

I base my assumptions on a fragment from the matchesGraphAsString body in modules / stitching / src / motion_estimators.cpp from version 2.4.2 of the OpenCV source code. I.e.

  str << "\"" << name_src << "\" -- \"" << name_dst << "\"" << "[label=\"Nm=" << pairwise_matches[pos].matches.size() << ", Ni=" << pairwise_matches[pos].num_inliers << ", C=" << pairwise_matches[pos].confidence << "\"];\n"; 

In addition, I also look at the documentation for detail :: MatchesInfo for information on Ni and C.