How to handle individual camera frames when using the Android mobile viewing library

Question

I am trying to make a camera app that detects faces using Google Mobile Vision Api with a custom camera instance. Not the same in the "CameraSource" in google Api, because I do frame processing to detect colors too, and with the help of camera sources I was not allowed to get camera frames. after I searched for it, and all the results that I got relate to using mobile vision with CameraSource, and not using any Api custom camera1. I tried to do the frame processing, and then determine the output photo, as here.

camera.setPreviewCallback(new Camera.PreviewCallback() {
            @Override
            public void onPreviewFrame(byte[] data, Camera camera) {
                Log.d("onPreviewFrame", "" + data.length);
                Camera.Parameters parameters = camera.getParameters();
                int width = parameters.getPreviewSize().width;
                int height = parameters.getPreviewSize().height;
                ByteArrayOutputStream outstr = new ByteArrayOutputStream();
                Rect rect = new Rect(0, 0, width, height);
                YuvImage yuvimage = new YuvImage(data, ImageFormat.NV21, width, height, null);
                yuvimage.compressToJpeg(rect, 20, outstr);
                Bitmap bmp = BitmapFactory.decodeByteArray(outstr.toByteArray(), 0, outstr.size());
                detector = new FaceDetector.Builder(getApplicationContext())
                        .setTrackingEnabled(true)
                        .setClassificationType(FaceDetector.ALL_LANDMARKS)
                        .setMode(FaceDetector.FAST_MODE)
                        .build();

                detector.setProcessor(
                        new MultiProcessor.Builder<>(new GraphicFaceTrackerFactory())
                                .build());

                if (detector.isOperational()) {
                    frame = new Frame.Builder().setBitmap(bmp).build();
                    mFaces = detector.detect(frame);
//                    detector.release();
                }
            }
        });

, - ? , : https://github.com/etman55/FaceDetectionSampleApp

** NEW UPDATE

CameraSource , , , , → github.

Answer 1

.

• FaceDetector , , - , . , .

• YUV_420_SP ( NV21), YUV, Bitmap, Frame.Builder() . Frame.Builder Documentation, , NV21 Preview Camera. :

  @override public void onPreviewFrame(byte[] data, Camera camera) {detector.detect(new Frame.Builder().setImageData(ByteBuffer.wrap(data), previewW, previewH, ImageFormat.NV21));}