The fastest way to draw a video fragment decoded by MediaCodec onto the screen?

Question

I am looking for the fastest way to take an image frame received from a MediaCodec decoder and draw it on the screen of an Android device. Important limitations and explanations are:

• You cannot use MediaPlayer. The middleware is not allowed.

• It is necessary as quickly as possible to output the output frames from the MediaCodec decoder to the screen (to minimize the delay).

• The available decoder output formats are as follows:
  ColorFormat [0] 0x00000013 COLOR_FormatYUV420Planar
  ColorFormat [1] 0x00000015 COLOR_FormatYUV420SemiPlanar
  ColorFormat [2] 0x7F000001 OMX_SEC_COLOR_FormatNV12TPhysicalAddress
  ColorFormat [3] 0x7FC00002 OMX_SEC_COLOR_FormatNV12Tiled

• The resolution of the video, and therefore the resolution of each output frame, is 960x720.

• The target platform is Galaxy Note II, and this approach may be specific to this platform (for example, take advantage of the available hardware functionality). This should not work on other platforms or be a common solution.

An approach that takes less than 66 ms will be good. Less than 33 ms would be great. My current approach takes 80-90 ms, which sucks. (I will not describe this since I do not want to distort the answers in any particular direction.)

Answer 1

It’s best to decode directly to Surface . ByteBuffer decoding ByteBuffer slow you down a bit. A number of bigflake examples (such as ExtractMpegFramesTest ) send decoder output to an external surface and examine it with GLES, but this is a simple change to make it work with an on-screen SurfaceView .

Update: Grafika has two different MediaCodec players that send SurfaceView and TextureView output, respectively.