Crop a panoramic image in OpenCV

Question

I am trying to find a simple algorithm to crop (remove black areas) a panoramic image created using the OpenCV Stitcher module.

My idea is to calculate the innermost black dots in the image that will define the cropping area, as shown in the following image:

Expected trimmed result:

I tried the following two approaches, but they do not crop the image as expected:

First approach:

void testCropA(cv::Mat& image) { cv::Mat gray; cvtColor(image, gray, CV_BGR2GRAY); Size size = gray.size(); int type = gray.type(); int left = 0, top = 0, right = size.width, bottom = size.height; cv::Mat row_zeros = Mat::zeros(1, right, type); cv::Mat col_zeros = Mat::zeros(bottom, 1, type); while (countNonZero(gray.row(top) != row_zeros) == 0) { top++; } while (countNonZero(gray.col(left) != col_zeros) == 0) { left++; } while (countNonZero(gray.row(bottom-1) != row_zeros) == 0) { bottom--; } while (countNonZero(gray.col(right-1) != col_zeros) == 0) { right--; } cv::Rect cropRect(left, top, right - left, bottom - top); image = image(cropRect); } 

Second approach:

 void testCropB(cv::Mat& image) { cv::Mat gray; cvtColor(image, gray, CV_BGR2GRAY); int minCol = gray.cols; int minRow = gray.rows; int maxCol = 0; int maxRow = 0; for (int i = 0; i < gray.rows - 3; i++) { for (int j = 0; j < gray.cols; j++) { if (gray.at<char>(i, j) != 0) { if (i < minRow) {minRow = i;} if (j < minCol) {minCol = j;} if (i > maxRow) {maxRow = i;} if (j > maxCol) {maxCol = j;} } } } cv::Rect cropRect = Rect(minCol, minRow, maxCol - minCol, maxRow - minRow); image = image(cropRect); } 

Answer 1

I never used stapler blanks, but I think that you can get an estimated homography matrix on each pair of images, if you could easily get it, then you can multiply it by the corners of the first source image and, therefore, for the corner of the last original, you will get their stitched coordinate, then get the minimum coordinate on the left and right x and the minimum y-coordinates of each image. You can get the coordinates of each stitched image, which you need to do in some cropping cases.