DJI drone video stream decoding on Android

Question

Hi, I would like to do some image processing using OpenCv on a video stream from DJI phantom 3 pro. Unfortunately, for this you need to do your own video decoding. I know this should work using the Media Codec Android class, but I don't know how to do this. I saw several examples for decoding video from a video file, but I could not change this code for my purpose. Can someone show an example or tutorial on how to do this? thanks for the help

mReceivedVideoDataCallBack = new DJIReceivedVideoDataCallBack(){ @Override public void onResult(byte[] videoBuffer, int size){ //recvData = true; //DJI methods for decoding //mDjiGLSurfaceView.setDataToDecoder(videoBuffer, size); } }; 

This is a method that sends an encoding stream from drone, and I need to send a videoBuffer to decode, and then change to Mat for OpenCV.

Answer 1

Initialize a video callback like this

 mReceivedVideoDataCallBack = new DJICamera.CameraReceivedVideoDataCallback() { @Override public void onResult(byte[] videoBuffer, int size) { if(mCodecManager != null){ // Send the raw H264 video data to codec manager for decoding mCodecManager.sendDataToDecoder(videoBuffer, size); }else { Log.e(TAG, "mCodecManager is null"); } } } 

Do your work by implementing TextureView.SurfaceTextureListener and for TextureView mVideoSurface call this line after its initialization:

 mVideoSurface.setSurfaceTextureListener(this); 

and then do:

  @Override public void onSurfaceTextureAvailable(SurfaceTexture surface, int width, int height) { Log.v(TAG, "onSurfaceTextureAvailable"); DJICamera camera = FPVDemoApplication.getCameraInstance(); if (mCodecManager == null && surface != null && camera != null) { //Normal init for the surface mCodecManager = new DJICodecManager(this, surface, width, height); Log.v(TAG, "Initialized CodecManager"); } } @Override public void onSurfaceTextureSizeChanged(SurfaceTexture surface, int width, int height) { Log.v(TAG, "onSurfaceTextureSizeChanged"); } @Override public boolean onSurfaceTextureDestroyed(SurfaceTexture surface) { Log.v(TAG, "onSurfaceTextureDestroyed"); if (mCodecManager != null) { mCodecManager.cleanSurface(); mCodecManager = null; } return false; } @Override public void onSurfaceTextureUpdated(SurfaceTexture surface) { final Bitmap image = mVideoSurface.getBitmap(); //Do whatever you want with the bitmap image here on every frame } 

Hope this helps!