All geek questions in one place

How to efficiently process and process video streams using a GPU?

I plan to develop a tool for real-time video processing using C ++, Qt and OpenGL. Video overlay is not an option since shaders should be used to process frames. Currently, I present the following sequence of steps:

  • Video Decoding (CPU)
  • Preprocessing (optional, CPU)
  • Move it to video memory (GPU using DMA)
  • Then process it using the vertex and fragment shaders (GPU)
  • Render it (GPU)

I am looking for some general tips explaining which extensions or technique you can use here. Is there a good reason to use Direct3D?

+3
source share
5 answers

First, there is no explicit way to use DMA on a PC. The driver can use it or use something else.

In any case, step 3 will “change the texture data on the graphics card”. In OpenGL, the PBO extension (Pixel Buffer Object) or the nice old glTexSubImage * function. In D3D9, this is LockRect on a texture or in other ways (for example, LockRect on a scratch texture, and then blit into the GPU texture). Any of them can potentially use DMA, but you cannot be sure.

Then the data is in the texture. You can render it to the screen with some shaders (for example, by making YCbCr conversion), or render it to other textures (s) to make more complex processing effects (for example, blur / glow / ...).

Direct3D , " wast" . OpenGL , - , - , ( ).

+2

Linux, NVIDIA 180.xx VDPAU api ( ). api, mplayer, vlc, ffmpeg mythtv. , api , - .

, CUDA (, , VDPAU)

+2

GPU, , s. .

, , , (YCbCr RGB).

" " "" , . , , FBO .

+1

GPU (GPGPU), CUDA NVIDIA Stream SDK ATI. , , . , .

, , , , , . , - , , ( CUDA, , , ).

I believe that if you have decent experience working with a shader, it may not be easy for you to learn the new platform.

0
source

The following steps should do this:

  • decode video in yuv

    This is usually what libs decoders do.

  • Download to OpenGL as a texture

  • convert yuv to rgb

    Since you do not want to use an overlay, you need to convert manually. Here is an example of using shaders.

  • place the transformed texture on a square and display

0
source

Source: https://habr.com/ru/post/1702251/

More articles:


All Articles