Extract / change video frames on Android

Question

I have a video file. I want to get every frame of the video and make some changes to the frame, for example, draw another bitmap, putting text, etc.

Is there any API / framework for receiving frames from video on Android? I did something similar in iOS using their AVFramework.

If possible with ffmpeg, I will use some of the available open source NDKs.

Answer 1

Option A:

You can create a SurfaceTexture and attach it to a MediaPlayer as follows

myPlayer = new MediaPlayer ... myRedirectionSurface = new Surface(mySurfaceTexture); myPlayer->setSurface(myRedirectionSurface); 

Thus, the decoded player stream is redirected to a SurfaceTexture, not a SurfaceView. OnFrameAvailableListener is called whenever you have a decoded frame. To access / change the image, you can use the surface lock / unlock methods on myRedirectionSurface.

IMPORTANT NOTICE. To get this working, you need to have API level 14 support.

Option B:

As you indicated the likelihood of using ffmpeg, you can achieve what you are going for, since you have full access to the output frames of the decoder. You can get started with the fpmpeg portal for RockPlayer or MoboPlayer. But in this option, the rendering of the video output from the NDK is not stellar.