OpenCV / FFMpeg Image Capture Issues

Question

I am trying to capture images from an IP camera in real time. The thread works fine in VLC, but OpenCV cvQueryFrame() seems to shuffle and corrupt incoming images to a point without recognition.

Again, capturing from a file works fine, but not live. If that matters, I use the rtsp connection url; I also tried this with two different cameras (different brands) and the problem remains.

In addition, the codec (I assume) displays several errors of the following type: Error at MB: 1746 and concealing 6000 DC, 6000 AC, 6000 MV errors .

What can I do?

Update: The first error in the sequence always cannot parallelize deblocking type 1, decoding such frames in sequential order

Update 2: Well, it seems that the problem with OpenCV / FFMPEG is related to rtsp / h264 streams. I tried the Qt Phonon library, which also does not work, and I quickly provided the Live555 library. This last one seems to work, in the sense that everyone says that it is, and the sample application (OpenRTSP) really works well with my stream. However, to be honest, getting into hands with the Live555 code seems like a long affair that I can barely afford now. Prohibition of any other alternative, I think I will have to go along this route.

Is there any other solution that comes to mind?

Update 3: I got the RTSP test client from Live555 code to work, so I know how to extract h264 frame information from the stream, but now I need to recombine this frame information into the actual displayed frames, which do not seem simple! Anyone familiar with Live555 know how to do this? Thanks.

Answer 1

It seems you need an additional software layer to capture stream packets and reconstruct frames locally, and then send them to openCV. You can easily achieve this with libVLC. It will also avoid codec problems, since you can parse almost all codecs using libVLC and then load the raw frames into openCV.

Answer 2

I do not know if this helps (since I am not an experienced C ++ developer), but I recently managed to get the stream from the IP camera . Here is a quick test:

 #include <opencv2/core/core.hpp> #include <opencv2/highgui/highgui.hpp> #include <opencv2/imgproc/imgproc.hpp> #include <iostream> #include <stdio.h> using namespace cv; using namespace std; int main(int, char**) { VideoCapture ipCam; Mat frame; const string LOCATION = "rtsp://192.168.0.200:554/rtsph264vga"; if(!ipCam.open(LOCATION)) { cout << "Error opening video stream or file" << endl; return -1; } for(;;) { if(!ipCam.read(frame)) { cout << "No frame" << endl; waitKey(); } imshow("cam", frame); if(waitKey(1) >= 0) break; } return 0; } 

Before switching to C ++, I installed the camera for export to H264 VGA (since it was not enabled by default on the cam I work with) and make sure that I have a stream running in VLC. I am using OpenCV 2.4.1 with fffmpeg enabled. As far as I understand, ffmpeg integration with OpenCV is available with OpenCV 2.0 up.

I ran into several problems when I had to integrate cv-code merging with other C ++ code, since I have OpenCV and ffmpeg + dependencies created for a 64-bit arch. and other code relied on many 32-bit libraries. The VideoCapture class is part of the highgui lib and basically the one you need to worry about. If it is not compiled with ffmpeg support, you will receive an error message or warning because VideoCapture will not be able to transcode the contents.

Not sure if this is the best option, but you can try to transfer / transcode the stream from VLC (by ticking Streaming / Saving in the Open Source / Network tab)

Answer 3

Here is a piece of code that I used to capture frames from WebCam. It worked for me, hope it works for you too ...

 int main(int argc, char* argv[]) { CvCapture *capture = NULL; IplImage* frame=NULL; int key =0; capture = cvCaptureFromCAM(0); if (!capture) { printf("Cannot initailize webcam"); return 1; } cvNamedWindow("result",CV_WINDOW_AUTOSIZE); while(key != 'q') { frame=cvQueryFrame(capture); if(!frame) break; cvShowImage("result",frame); key=cvWaitKey(10); frame=NULL; } cvDestroyWindow("result"); cvReleaseCapture(&capture); return 0; } 

Answer 4

For OpenCV 2.3.1, I wrote this code and it works fine, i.e. I get images from the camera feed.

 VideoCapture cap(0); if(!cap.isOpened()) { cout<<"Camera is not connected"<<endl; getchar(); } namedWindow("Camera Feed",1); for(;;) { Mat frame; cap >> frame; imshow("Camera Feed", frame); if(!frame.empty()) detectAndDisplay(frame); else cout<<"No frame as input"<<endl; int c=waitKey(10); if(c==27) break; } return 0; 

As you can see, it takes an input and displays it continuously and exits if you press ESC on the keyboard. Here is the documentation for CV 2.1, which has the same set of commands as CV 2.3. It looks like the teams have changed from 2.4, although I'm not too sure about that. Hope this helps. :)