OpenCV Speed ​​Road Sign Detection

Question

I have a problem with detecting speed road signs with opencv 2.4 for Android. I do the following:

"capture frame → convert it to HSV → extract red areas → detect signs with ellipse detection"

While ellipse detection works perfectly, so far the image is good. But, as you see in the figures below, red extraction does not work properly, due to the poor quality of the frames, in my opinion.

Convert source image to HSV:

Imgproc.cvtColor(this.source, this.source, Imgproc.COLOR_RGB2HSV, 3); 

Extraction of red flowers:

 Core.inRange(this.source, new Scalar(this.h,this.s,this.v), new Scalar(230,180,180), this.source); 

So my question is another way to detect a traffic sign like this, or to extract red areas from it, which, incidentally, can be very weak, like in the last image?

This is the original image:

This is a conversion to HSV, as you can see that the red areas look the same as the nearby trees. Thats how I suppose to know this is red, but I can’t.

Converted to HSV:

It is with highlighted red flowers. If the colors are correct, I should get an almost perfect circle / ellipse around the mark, but it is incomplete due to false colors.

Result after extraction:

Ellipse Method:

 private void findEllipses(Mat input){ Mat thresholdOutput = new Mat(); int thresh = 150; List<MatOfPoint> contours = new ArrayList<MatOfPoint>(); MatOfInt4 hierarchy = new MatOfInt4(); Imgproc.threshold(source, thresholdOutput, thresh, 255, Imgproc.THRESH_BINARY); //Imgproc.Canny(source, thresholdOutput, 50, 180); Imgproc.findContours(source, contours, hierarchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE); RotatedRect minEllipse[] = new RotatedRect[contours.size()]; for(int i=0; i<contours.size();i++){ MatOfPoint2f temp=new MatOfPoint2f(contours.get(i).toArray()); if(temp.size().height > minEllipseSize && temp.size().height < maxEllipseSize){ double a = Imgproc.fitEllipse(temp).size.height; double b = Imgproc.fitEllipse(temp).size.width; if(Math.abs(a - b) < 10) minEllipse[i] = Imgproc.fitEllipse(temp); } } detectedObjects.clear(); for( int i = 0; i< contours.size(); i++ ){ Scalar color = new Scalar(180, 255, 180); if(minEllipse[i] != null){ detectedObjects.add(new DetectedObject(minEllipse[i].center)); DetectedObject detectedObj = new DetectedObject(minEllipse[i].center); Core.ellipse(source, minEllipse[i], color, 2, 8); } } 

}

Problem Sign:

Answer 1

You can find an overview of traffic sign detection methods here and here .

You will see that there are two ways to achieve this:

• Based on color (e.g. what you are doing now)
• Shape based

In my experience, I have found that form-based methods work very well, because color can change dramatically under different lighting conditions, camera quality, etc.

Since you need to detect speed signs that I think are always circular, you can use an ellipse detector to search for all circular objects in your image, and then apply some checking to determine if it is a traffic sign or not.

Why ellipse detection?

Well, since you're looking for promising distorted circles, you're actually looking for ellipses. Real-time ellipse detection is an interesting (albeit limited) research topic. I will show you 2 articles with C ++ source code (which you can use in your application through your own JNI calls):

• l Libuda, I. Grothues, K.-F. Kreis, ellipse detection in digital image data using geometric features, in: J. Braz, A. Ranchordas, H. Arajo, J. Jorge (Eds.), Advances in Computer Graphics and Computer Vision, Volume 4 Communications in Computer and Information Technology, Springer Berlin Heidelberg, 2007, p. 229-239. link , code

• M

  . Fornaciari, A. Prati, R. Cucchiara, "Fast and efficient ellipse detector for applications with built-in vision", Pattern Recognition, 2014 link , code

---

UPDATE

I tried method 2) without preprocessing. You can see that at least the sign with the red border is detected very well:

Answer 2

Link to your text:

This is a conversion to HSV, as you can see that the red areas look the same as nearby trees. Thats how I suppose to know this is red, but I can’t.

I want to show you my result mainly about what you did (simple operations should be easily ported to android openCV):

  // convert to HSV cv::Mat hsv; cv::cvtColor(input,hsv,CV_BGR2HSV); std::vector<cv::Mat> channels; cv::split(hsv,channels); // opencv = hue values are divided by 2 to fit 8 bit range float red1 = 25/2.0f; // red has one part at the beginning and one part at the end of the range (I assume 0° to 25° and 335° to 360°) float red2 = (360-25)/2.0f; // compute both thresholds cv::Mat thres1 = channels[0] < red1; cv::Mat thres2 = channels[0] > red2; // choose some minimum saturation cv::Mat saturationThres = channels[1] > 50; // combine the results cv::Mat redMask = (thres1 | thres2) & saturationThres; // display result cv::imshow("red", redMask); 

These are my results:

From your result, please note that findContours changes the input image, so you may have extracted an ellipse, but you can no longer see it in the image if you saved the image AFTER findContours.

Answer 3

 private void findEllipses(Mat input){ Mat thresholdOutput = new Mat(); int thresh = 150; List<MatOfPoint> contours = new ArrayList<MatOfPoint>(); MatOfInt4 hierarchy = new MatOfInt4(); Imgproc.threshold(source, thresholdOutput, thresh, 255, Imgproc.THRESH_BINARY); //Imgproc.Canny(source, thresholdOutput, 50, 180); Imgproc.findContours(source, contours, hierarchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE); // source = thresholdOutput; RotatedRect minEllipse[] = new RotatedRect[contours.size()]; for(int i=0; i<contours.size();i++){ MatOfPoint2f temp=new MatOfPoint2f(contours.get(i).toArray()); if(temp.size().height > minEllipseSize && temp.size().height < maxEllipseSize){ double a = Imgproc.fitEllipse(temp).size.height; double b = Imgproc.fitEllipse(temp).size.width; if(Math.abs(a - b) < 10) minEllipse[i] = Imgproc.fitEllipse(temp); } } detectedObjects.clear(); for( int i = 0; i< contours.size(); i++ ){ Scalar color = new Scalar(180, 255, 180); if(minEllipse[i] != null){ detectedObjects.add(new DetectedObject(minEllipse[i].center)); DetectedObject detectedObj = new DetectedObject(minEllipse[i].center); Core.ellipse(source, minEllipse[i], color, 2, 8); } } } 

Answer 4

Have you tried using opencv ORB? It works very well. I created a cascade of hara for a road sign (circular motion in my case) and used opencv ORB to map functions and remove any false positives. Google's tensor was used for image recognition and the results were impressive.