Display on screen using QAbstractVideoSurface

Question

I am trying to display a camera image on the screen using a subclass of QAbstractVideoSurface, and I have no experience with this.

I would appreciate it if someone could explain how to do this.

Answer 1

QAbstractVideoSurface is an interface between a producer and a consumer of video frames. First you need to implement only two functions:

• supportedPixelFormats so that the manufacturer can choose the appropriate format for QVideoFrame
• present , which is a more general wording for show \ display this frame

Suppose you want to use a classic QWidget for display. In this case, you can use QImage to draw the widget.

The first Qt is expected to draw a QImage , which is RGB24 (or BGR24) on most platforms. So,

 QList<QVideoFrame::PixelFormat> LabelBasedVideoSurface::supportedPixelFormats( QAbstractVideoBuffer::HandleType handleType) const { if (handleType == QAbstractVideoBuffer::NoHandle) { return QList<QVideoFrame::PixelFormat>() << QVideoFrame::Format_RGB24; } else { return QList<QVideoFrame::PixelFormat>(); } } 

Now, to present a QVideoFrame, you map your data to a QImage and draw a QImage for the widget. For simplicity, I will use QLabel , which I refer directly to (without a signal there is no slot).

 bool LabelBasedVideoSurface::present(const QVideoFrame &frame) { if (notMyFormat(frame.pixelFormat())) { setError(IncorrectFormatError); return false; } else { QVideoFrame frametodraw(frame); if(!frametodraw.map(QAbstractVideoBuffer::ReadOnly)) { setError(ResourceError); return false; } //this is a shallow operation. it just refer the frame buffer QImage image( frametodraw.bits(), frametodraw.width(), frametodraw.height(), frametodraw.bytesPerLine(), QImage::Format_RGB444); mylabel->resize(image.size()); //QPixmap::fromImage create a new buffer for the pixmap mylabel->setPixmap(QPixmap::fromImage(image)); //we can release the data frametodraw.unmap(); mylabel->update(); return true; } } 

This example is obviously not optimal.

• He does not take money for what can be saved in video memory, because we draw with pixmap.
• Converting from image to pixmap is an unnecessary hit.

You can write your own widget and implement paintEvent to improve performance. In addition, you have several design rights for present() behavior. For instance:

• Is a non-blocking surface i.e. the frame is already shown when it ends. Above, this will mean using mylabel->repaint() instead of mylabel->update()
• What happens when you cannot complete a presentation. You might want to draw a blank frame rather than return an error that might stop the music.