How to efficiently scale a video frame using NDK

Question

I am doing an Android project about working with video frames, I need to process each frame before displaying it. The process includes scaling frames with a resolution of 1920x1080 to 2560x1440, color space conversion, and some necessary RGB-based image processing, and all these work must be completed within 33 ms ~ 40 ms.

I optimized yuv-> rgb and other processing with the neon arm, they worked well. But I need to scale the frame first from 1080p to 2k, this is a performance bottleneck.

My question is how to efficiently scale an image with a resolution of 1080p to 2k for 20 ms, I do not have much experience with the scaling algorithm, so any suggestions are useful. Can I use arm neon to optimize an existing algorithm?

Hardware environment:

• Processor: Samsung Exynos 5420
• Memory: 3 GB
• Display: 2560X1600 px

Update:

I will talk about my decoding process, I use MediaCodec to decode regular video (H.264) to YUV (NV12), the default decoder is hardware, it is very fast. Then I use arm neon to convert NV12 to RGBW, and then send the RGBW frame to the surface flasher for display. I just use a regular SurfaceView rahter than GLSurfaceView.

The bottleneck is increasing YUV from 1080p to 2K fast.

Answer 1

, , , OpenGL YUV → RGB: http://www.fourcc.org/source/YUV420P-OpenGL-GLSLang.c

:

• H.264 → YUV
• YUV OpenGL; 3 : Y, U V
• , Y, U V RGB (W); .
• , ,

, , . : OpenGL, Y , Y-, , upsampler , upsampler .

Answer 2

opengls, - , , . 1920 x 1080, , , 2048 x 1024. 35 7.

GLSurfaceView . ffmpeg, , , sws_scale , opengl. /, fps , gpu.

. ffmpeg Android .

Answer 3

, . , . , OGL. OGL ; . , OGL 2048x2048. , . , N72013 LG2. , NDK OGL, . , .