What is the most efficient way to transfer a file (read from Java) to a native method?

Question

I have approx. 30,000 files (1 MB each) that I want to put in my own method, which requires only an array of bytes and its size as arguments.

I looked through some examples and tests (e.g. http://nadeausoftware.com/articles/2008/02/java_tip_how_read_files_quickly ), but they all do some other bizarre things.

In principle, I am not interested in the contents of the file, I do not want to access anything in this file or byte array or do anything else with it. I just want to put the file in my own method, which takes an array of bytes as quickly as possible.

I am currently using RandomAccessFile, but it is terribly slow (10 MB / s).

Is there something like

byte[] readTheWholeFile(File file){ ... } 

which I could put in

 native void fancyCMethod(readTheWholeFile(myFile), myFile.length()) 

What would you suggest?

Answer 1

Using regular arrays can be inefficient, since the virtual machine can copy the array when transferring it to its own code and can also use intermediate memory during I / O.

For fast I / O, use ByteBuffer.allocateDirect to allocate a byte buffer. The main array is “special” in the sense that it is not part of the regular JVM heap. Source code and I / O can directly access the array.

To read data into the buffer,

 ByteBuffer byteBuffer = ByteBuffer.allocateDirect(randomAccessFile.length()); RandomAccessFile.getChannel().read(byteBuffer, 0); 

To get the base array to go to JNI, use

 byte[] byteArray = byteBuffer.array(); 

Then you can pass this array and file length to JNI.

Direct buffers are extremely difficult to create, since all your files are 1 MB (or so), you should be able to reuse the same buffer for multiple files.

Hope this helps!

Answer 2

I'm not quite sure if this is what you are asking for, but it looks like you want to efficiently pass the contents of the file as a byte array into your own method.

If so, I suggest you read the contents of the file in Java using a BufferedInputStream and store them in a ByteBuffer that was allocated through ByteBuffer#allocateDirect , so that it can be passed to the JNI side and available in general. Now, in your own method, you can call GetDirectByteBufferAddress to directly access the buffer.

Answer 3

Here is an example readFileFully that can be implemented

  public static byte[] readFileFully(String aFileName) throws IOException { byte[] retData = null; File inputFile = new File(aFileName); if (inputFile == null || !inputFile.exists() || !inputFile.canRead()) { throw new IOException("INVALID FILE : " + aFileName); } // Read in the file data BufferedInputStream iStream = null; try { iStream = new BufferedInputStream(new FileInputStream(inputFile)); int size = (int)inputFile.length(); retData = new byte[size]; int bytes_read = 0; // read stuff in here while (bytes_read < size) { bytes_read += iStream.read(retData,bytes_read,size - bytes_read); } } finally { if (iStream != null) { try { iStream.close(); } catch(IOException e) { } } inputFile = null; } return retData; }