Any code hints to speed up random reads from the Java FileChannel?

I have a large (3Gb) binaries binary, which I access (more or less) randomly during an iterative algorithm that I wrote for data clustering. Each iteration is about half a million reads from a file and about 100 thousand records of new values.

I am creating a FileChannel like this ...

f = new File(_filename);
_ioFile = new RandomAccessFile(f, "rw");
_ioFile.setLength(_extent * BLOCK_SIZE);
_ioChannel = _ioFile.getChannel();

Then I use the closed ByteBuffer double size to read from it

private ByteBuffer _double_bb = ByteBuffer.allocate(8);

and my reading code is as follows

public double GetValue(long lRow, long lCol) 
{
    long idx = TriangularMatrix.CalcIndex(lRow, lCol);
    long position = idx * BLOCK_SIZE;
    double d = 0;
    try 
    {
        _double_bb.position(0);
        _ioChannel.read(_double_bb, position);
        d = _double_bb.getDouble(0);
    } 

    ...snip...

    return d;
}

and I write to him like this ...

public void SetValue(long lRow, long lCol, double d) 
{
    long idx = TriangularMatrix.CalcIndex(lRow, lCol);
    long offset = idx * BLOCK_SIZE;
    try 
    {
        _double_bb.putDouble(0, d);
        _double_bb.position(0);
        _ioChannel.write(_double_bb, offset);
    } 

    ...snip...

}

, , . , , , , , , , , .

, : - / JVM, , ? , , , , .

+3
5

, , , , .. get/set .

, , HD - , , , - ​​ 10 000 , - .

, ( , - ) , , .

, SSD () . - .

+4

ByteBuffer, , . FileChannel.map().

, , GetValue(row, col) SetValue(row, col) . row col ? , , : , , row + 1, row - 1, col - 1, col + 1 ; , 8 x 8 16 x 16 . , , (, , ).

( ): , GetValue(row, col) SetValue(row, col) , .

+4

, , .

3Gb 64- JVM, .

, "", . , . , , .

, , , .

+1
  • ( Java). / (, ).

  • ? .

...

+1

, , , .

HDF . Java API, Java. Apache.

+1

Source: https://habr.com/ru/post/1726322/


All Articles