Orient the raw camera data correctly.

Question

My camera application processes raw camera frames with onPreviewFrame() using OpenCV and then displays them on the screen. However, the raw frames are oriented the way the camera is installed on the phone, which is usually not right. To fix this, I rotate them manually using OpenCV, which is time consuming.

I reviewed the use of setDisplayOrientation , but the documentation states that

This does not affect the order of the byte array passed to onPreviewFrame

and I need the data to actually be right side up, and not just display the right side up. How to properly orient the raw camera data? If this is not possible, can I effectively rotate the byte array passed to me in onPreviewFrame() , say using OpenGL?

Answer 1

No, there is no way to force the API to do this for you, but to find out how you need to rotate the data, see the sample code on Camera.Parameters.setRotation . Although setRotation () only affects JPEGs, you want to apply the same amount of rotation to the data you get in onPreviewFrame () as you do with JPEG files.

Reproducing the sample code here with minor changes:

 public void onOrientationChanged(int orientation) { if (orientation == ORIENTATION_UNKNOWN) return; android.hardware.Camera.CameraInfo info = new android.hardware.Camera.CameraInfo(); android.hardware.Camera.getCameraInfo(cameraId, info); orientation = (orientation + 45) / 90 * 90; int rotation = 0; if (info.facing == CameraInfo.CAMERA_FACING_FRONT) { rotation = (info.orientation - orientation + 360) % 360; } else { // back-facing camera rotation = (info.orientation + orientation) % 360; } mCameraOrientation = rotation; // Store rotation for later use } ... void onPreviewFrame(byte[] data, Camera camera) { switch(mCameraOrientation) { case 0: // data is correctly rotated break; case 90: // rotate data by 90 degrees clockwise case 180: // rotate data upside down case 270: // rotate data by 90 degrees counterclockwise } } 

So, you need to inherit from OrientationEventListener and override onOrientationChanged as above and then use the calculated orientation value from there to rotate the preview frames when they enter.

Answer 2

setDisplayOrientation only rotates the camera against the surface you are painting on, if I remember correctly. I believe that you want to rotate the interpretation of the camera, which you could achieve with setRotation