MediaCodec with surface input: create output

Question

I am trying to create short sequential mp4 files from CameraPreview data via MediaCodec.createInputSurface() . However, to recreate MediaCodec and its associated surface, you need to stop the camera to allow another call to mCamera.setPreviewTexture(...) . This delay results in an unacceptable number of dropped frames.

Therefore, I need to periodically generate CODEC_CONFIG and END_OF_STREAM without recreating the input surface and, therefore, to call mCamera.setPreviewTexture(...) . Is it possible that the value of MediaFormat not changed?

(I am adapting fadden CameraToMpegTest . My full code is here )

Failed Attempts:

Calling MediaCodec.signalEndOfInputStream() , dropping MediaCodec , and then calling MediaCodec.flush() between the chunks IllegalStateException an IllegalStateException on the second call to MediaCodec.signalEndOfInputStream() .

Call MediaCodec.signalEndOfInputStream() , reset MediaCodec , and then call MediaCodec.stop(); MediaCodec.configure(...), MediaCodec.start() MediaCodec.stop(); MediaCodec.configure(...), MediaCodec.start() between fragments without calling MediaCodec.createInputSurface() again causes the following error:

  09-30 13:12:49.889 17638-17719/x.xx.xxxx E/Surface﹕ queueBuffer: error queuing buffer to SurfaceTexture, -19 09-30 13:12:49.889 17638-17719/x.xx.xxxx E/IMGSRV﹕ :0: UnlockPostBuffer: Failed to queue buffer 0x592e1e70 09-30 13:12:49.889 17638-17719/x.xx.xxxx E/CameraToMpegTest﹕ Encoding loop exception! 09-30 13:12:49.889 17638-17719/x.xx.xxxx W/System.err﹕ java.lang.RuntimeException: eglSwapBuffers: EGL error: 0x300b 09-30 13:12:49.896 17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.ChunkedHWRecorder$CodecInputSurface.checkEglError(ChunkedHWRecorder.java:731) 09-30 13:12:49.896 17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.ChunkedHWRecorder$CodecInputSurface.swapBuffers(ChunkedHWRecorder.java:713) 09-30 13:12:49.896 17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.ChunkedHWRecorder.startRecording(ChunkedHWRecorder.java:164) 09-30 13:12:49.896 17638-17719/x.xx.xxxx W/System.err﹕ at x.xx.xxxx.HWRecorderActivity$CameraToMpegWrapper.run(HWRecorderActivity.java:76) 09-30 13:12:49.896 17638-17719/x.xx.xxxx W/System.err﹕ at java.lang.Thread.run(Thread.java:841) 

Solved Thanks fadden. The full source of the solution is here .

Answer 1

The signalEndOfInputStream() call updates the state of the various layers in the MediaCodec stack. You can understand what operations are valid from the comments of the tests above in MediaCodecTest , but in general the behavior of MediaCodec is simply not defined for "unusual" purposes.

So you need to look at the code. The lifetime of the input surface is tied to the lifetime of OMXNodeInstance ; It is presented by GraphicBufferSource . After you signal EOS, GraphicBufferSource will ignore extra frames (see line 426 ). There is no way to reset the EOS flag without breaking the GraphicBufferSource, but when you do this, it disables the buffer queue underlying the surface.

Therefore, I do not think that you can stop / restart MediaCodec and continue to use Surface.

However ... you do not need to. CameraToMpegTest directs the camera preview to SurfaceTexture, and then maps the texture to the input surface of the encoder using GLES. SurfaceTexture is separate from the encoder and does not need to be changed. I think what needs to be changed is a CodecInputSurface that calls eglCreateWindowSurface() using Surface from MediaCodec to tell GLES where to draw. If you add a new Surface Upgrade API (destroy the old EGLSurface, create a new EGLSurface, eglMakeCurrent) and call it whenever you create a new MediaCodec, I think it will all work.

Update for comments :

It is important that you change only EGLSurface . The checkAndUpdateEglStateLocked() function in GLConsumer.cpp checks that the EGLDisplay and EGLContext do not change after they are installed, you cannot call release() / eglSetup() in CodecInputSurface because it changes the EGLContext . You just want to destroy and recreate the EGLSurface .