FFMPEG and JNI - transfer AVFrame data to Java and vice versa

Question

I have a C code that decodes video frame by frame. I get to the point where I have an AVFrame in BGR32, and I would like to send it back to Java for editing.

I have a ByteBuffer object in my C code that was created in Java using AllocateDirect, but I'm struggling to write the contents of AVFrame-> data [0] (like uint8_t) and read it. I tried memcpy with no luck. Does anyone have an idea how to achieve this?

UPDATE The following comment will comment below and wrote this in C

char *buf = (*pEnv)->GetDirectBufferAddress(pEnv, byteBuffer);
memcpy(buf, rgb_frame->data[0], output_width*output_height*4);

The buffer contains some data in Java, but doing the following returns a null bitmap

BufferedImage frame = ImageIO.read(bitmapStream);

Where bitmapStream is a ByteBufferInputStream defined here: https://code.google.com/p/kryo/source/browse/trunk/src/com/esotericsoftware/kryo/io/ByteBufferInputStream.java?r=205

Not sure what I am writing wrong in this buffer

UPDATE 2

It turned out pretty close, thanks to the last fragment. I use the BGR32 format in my C code, i.e. 4 bytes per pixel. So I modified Java a bit:

final byte[] dataArray = new byte[width*height*4];
imageData.get(dataArray);
final BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_BGR);
final DataBuffer buffer = new DataBufferByte(dataArray, dataArray.length);
Raster raster = Raster.createRaster(sampleModel, buffer, null);
image.setData(raster);

I am drawing the image correctly, but there seems to be a problem with color channels

Tried different formats without luck

Answer 1

From Oracle JNI documentation at
https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html#GetDirectBufferAddress

GetDirectBufferAddress

void* GetDirectBufferAddress(JNIEnv* env, jobject buf);

java.nio.Buffer.

, Java . LINKAGE:

230 JNIEnv. :

env: JNIEnv

buf: java.nio.Buffer( NULL) RETURNS:

, . NULL, undefined, java.nio.Buffer, JNI . :

JDK/JRE 1.4

++:

static char framebuf[100];

JNIEXPORT void JNICALL Java_javaapplication45_UseByteBuffer_readBuf
  (JNIEnv *env, jobject usebb, jobject bb) {
    void *addr = env->GetDirectBufferAddress(bb);
    framebuf[0] = 77;
    memcpy(addr,framebuf,100);
}

Java:

public class UseByteBuffer {
    public native void readBuf(ByteBuffer bb);
}

...

public static void main(String[] args) {
    System.load("/home/shackle/NetBeansProjects/usebb/dist/Debug/GNU-Linux-x86/libusebb.so");
    ByteBuffer bb = ByteBuffer.allocateDirect(100);
    new UseByteBuffer().readBuf(bb);
    byte first_byte = bb.get(0);
    System.out.println("first_byte = " + first_byte);
}

first_byte = 77, , .

ImageIO.read() , , ImageReader, JPEG PNG.

raw (3 r, g, b)

int width = 256;
int height = 256;
ByteBuffer bb = ByteBuffer.allocateDirect(height*width*3);

byte[] raw = new byte[width * height * 3];
bb.get(raw);
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_3BYTE_BGR);
DataBuffer buffer = new DataBufferByte(raw, raw.length);
SampleModel sampleModel = new ComponentSampleModel(DataBuffer.TYPE_BYTE, width, height, 3, width * 3, new int[]{0,1,2});
Raster raster = Raster.createRaster(sampleModel, buffer, null);
image.setData(raster);

2

BGR32 , :

ByteBuffer imageData = ByteBuffer.allocateDirect(height * width * 4);
byte[] raw = new byte[width * height * 4];
imageData.get(raw);
BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_BGR);
DataBuffer buffer = new DataBufferByte(raw, raw.length);
SampleModel sampleModel = new ComponentSampleModel(
        DataBuffer.TYPE_BYTE, width, height, 4, width * 4,
        new int[]{2,1,0} // Try {1,2,3}, {3,2,1}, {0,1,2}
);
Raster raster = Raster.createRaster(sampleModel, buffer, null);
image.setData(raster);

, , , , bandOffsets ComponentSampleModel, .

3

sampleModel BufferedImage.copyData() WritableRaster getRaster().

SampleModel sampleModel = new ComponentSampleModel(
        DataBuffer.TYPE_BYTE, width, height, 4, width * 4,
        new int[]{2, 1, 0} 
);

...

BufferedImage newImage = ImageIO.read(new File("test.png"));
byte newRaw[] = new byte[height*width*4];
DataBuffer newBuffer = new DataBufferByte(newRaw, newRaw.length);
WritableRaster newRaster = Raster.createWritableRaster(sampleModel, newBuffer, null);
newImage.copyData(newRaster);