QT GUI freezes when capturing video from a webcam using OpenCV

Question

I use Opencv to process real-time video.

As an external interface, I use the QT infrastructure.

In my GUI, I have an input image window (displayed on a shortcut) and an output image window (mapped to another label) and 3 buttons. One to start entering video capture, the second to process the video (code has not been written yet), and the third to exit.

Currently, I can broadcast my video and display it on the Front-end. But this blocks my GUI and cannot exit.

I tried using QTimers (using suggestions from this and on the QT forum), but my GUI still remains locked.

Would appreciate if someone could point me in the right direction.

Below is the code:

mainwindow.h #ifndef MAINWINDOW_H #define MAINWINDOW_H #include <QMainWindow> #include <opencv2/highgui/highgui.hpp> #include <opencv2/core/core.hpp> #include <opencv2/imgproc/imgproc.hpp> // for cvtColor #include <iostream> #include <QTimer> namespace Ui { class MainWindow; } class MainWindow : public QMainWindow { Q_OBJECT public: explicit MainWindow(QWidget *parent = 0); ~MainWindow(); private slots: void on_buttonCaptureVideo_clicked(); void on_buttonExit_clicked(); public slots: virtual void doNextFrame() {repaint();} private: Ui::MainWindow *ui; CvCapture *capture; // OpenCV Video Capture Variable IplImage *frame; // Variable to capture a frame of the input video cv::Mat source_image; // Variable pointing to the same input frame cv::Mat dest_image; // Variable to output a frame of the processed video QTimer *imageTimer; }; #endif // MAINWINDOW_H 

mainwindow.cpp

 #include "mainwindow.h" #include "ui_mainwindow.h" MainWindow::MainWindow(QWidget *parent) : QMainWindow(parent), ui(new Ui::MainWindow) { ui->setupUi(this); } MainWindow::~MainWindow() { delete ui; cvReleaseImage(&frame); cvReleaseCapture(&capture); } void MainWindow::on_buttonCaptureVideo_clicked() { // Set to 25 frames per second const int imagePeriod = 1000/25; // ms imageTimer = new QTimer(this); imageTimer->setInterval(imagePeriod); connect(imageTimer, SIGNAL(timeout()), this, SLOT(doNextFrame())); // Use the default camera capture = cvCreateCameraCapture(-1); while(capture) { // Capture a frame frame = cvQueryFrame(capture); // Point to the same frame source_image = frame; // Resize Image cv::resize(source_image, source_image, cv::Size(128,128) , 0, 0); // Change to RGB format cv::cvtColor(source_image,source_image,CV_BGR2RGB); // Convert to QImage QImage qimg = QImage((const unsigned char*) source_image.data, source_image.cols, source_image.rows, QImage::Format_RGB888); // convert to QImage // Display on Input Label ui->labelInputVideo->setPixmap(QPixmap::fromImage(qimg)); // Resize the label to fit the image ui->labelInputVideo->resize(ui->labelInputVideo->pixmap()->size()); } } void MainWindow::on_buttonExit_clicked() { connect(ui->buttonExit, SIGNAL(clicked()), qApp, SLOT(closeAllWindows())); } 

Answer 1

When you press the button,

 while(capture) { ... } 

the loop will run forever since capture will never be set to NULL. This means that the code stream never leaves your loop, and thus the main thread cannot handle anything else, such as repainting, for example.

QTimer will emit its timeout signals (), but they will be placed as Events in the Qt Event Queue. While your on_buttonCaptureVideo_clicked() method is running, these events will not be processed.

Here are my suggestions on how to make it work:

---

This code:

 // Set to 25 frames per second const int imagePeriod = 1000/25; // ms imageTimer = new QTimer(this); imageTimer->setInterval(imagePeriod); connect(imageTimer, SIGNAL(timeout()), this, SLOT(doNextFrame())); // Use the default camera capture = cvCreateCameraCapture(-1); 

belongs to the MainWindow constructor, because you want to install it only once. There is no need to do this again when the user presses a button on the second, third, etc.

The code that is inside your while should go to the doNextFrame() slot (without the while construct).

Then your button will do

 imageTimer->start(); 

and then for example do

 imageTimer->stop(); 

when pressed again.

Code example:

 void MainWindow::on_buttonCaptureVideo_clicked() { if( imageTimer->isActive() ) { imageTimer->stop(); } else { imageTimer->start(); } } 

---

What happens if you do this?

When you press the button, your slot with on_buttonCaptureVideo_clicked() will be called from the GUI thread, the timer will start, and the method will return almost instantly.
Now the GUI thread is free and able to process reviews, etc.
From now on, the timer will send timeout signals () every 40 ms. Whenever the GUI thread is free, it will process this signal and call your doNextFrame slot.
This slot will capture the next frame and return when it is done. When this is done, the GUI thread will be able to process other events again (for example, redraw).
As soon as you press the button again, the timer will stop and no new timeout () events will be dispatched. If you still see a couple of frames after clicking the button, this may mean that timer events were sent faster than they could be processed.

Answer 2

Preface: I am not strong in C ++, so I cannot offer specific code, but I am experienced in PyQt

This is a common trap for newbies to Qt. It seems that your on_buttonCaptureVideo_clicked is on_buttonCaptureVideo_clicked in your main GUI thread to do the work. In QT, you want to not do anything busy in your main thread. Qt eventloop should be able to constantly process and clean your GUIs when they come in. What you do is blocking the event loop.

Here you can do two different things. The first is a more basic approach, but allows you to see more immediate results. You can "pump" eventloop. Depending on how quickly the while repeats, you can call qApp->processEvents(); . This will allow Qt to handle pending GUI events and make your application more responsive. Its main exchange time between the while loop and the main loop. Perhaps you want to call it on every nth frame. Depends on how often you want to make sure the GUI is updated.

Another option, which is more preferable, is to place a capture loop in QThread. When a new frame is available, you can emit a signal with frame data. The signal will be placed in the Qt event loop, which will be processed with everything else and will not delay your GUI. This is usually the approach you want to use with any heavy crunchy or long triggering messages.

Edit

I just realized that you are starting QTimer in addition to looping in the main thread. If you want to use QTimer and your image processing is not too heavy (it does not take much time per loop), you should move everything to doNextFrame and completely remove the while . If your doNextFrame is a heavy process, then you should use QThread and signals.