How to work with large arrays in Java

Question

I am reading a file containing 10,000 int values, and then trying to store them in an array. An exception occurs, which says that the value of the array is too large.

It was interesting to me, rather than writing this array to a variable, I could just save it in memory and read it from there. Would this be a suitable way to solve this problem?

change

After a more detailed study, it turns out that the error caused by the error is a "code for big for try" error. I read every element of the array and add it to the string, maybe this is what causes the error?

Answer 1

Instead, you can use an ArrayList, but the array should be in order with 10,000 values. Can you post more details? Code, full stack trace, etc. Theoretically, this should be fine with Integer.MAX_VALUE elements (LOT greater than 10 KB), but of course you may not have memory at first!

In terms of “just store it in memory and read it from there”, well variables are simply stored in memory, so no matter if you use an array or list (or any other data structure), you will always read it from memory!

EDIT: based on your further explanation, this is not a problem with the size of the array at all, it is a problem with the fact that you create 10,000 lines of code to fit in one block, which is too much, and therefore it complains. Modify your code to generate code that uses a loop instead, and everything should be fine, however you have many elements there (before Integer.MAX_VALUE, of course.)

Answer 2

An array of 10,000 int values ​​is about 40 KB.

You can try to reduce the memory used, but I suspect that this is not your problem.

Can you give us the actual error message? An array value is too large if its a long for example. let's say you used File.length()/4 to determine the size of the array, in which case you need to give it to int

Answer 3

It is strange that you cannot create an array with a length of 10,000 elements. I believe that your problem is not in the length of the array, but in the value of a specific element of the array. In any case, if you need large arrays, use Lists. In particular java.util.LinkedList .

Answer 4

Your problem is that you write every array or row assignment in full, something like this:

  array[0] = 0; array[1] = 1; array[2] = 2; // all the way up to 9999! 

or that:

  String s = ""; s += array[0]; s += array[1]; s += array[2]; // all the way up to 9999! 

instead of a loop. This way you create more code than Java allows in the method .

This results in a compilation error that you describe:

 $ javac Test.java Test.java:7: code too large for try statement try { ^ Test.java:4: code too large public static void main(String[] args) { ^ 2 errors 

After discussing in the comments, the code that you say produces this compiler error does not have a lot of lines. Something does not make sense - the error you are reporting is not consistent with the code you specified. At this late stage, I strongly recommend that you post some code and error so that others can try to figure out what might be causing this.

(Also, your question is unlikely to attract much attention because you accepted the answer. You might want to reconsider this if your question has no answer.)

Answer 5

An array of 10,000 ints is not very large. I cannot understand why you will have a problem with storing data in memory (i.e. it is assigned to a variable).

Answer 6

It seems strange to me that 10,000 ints take up too much memory. It may be something else if you eat your memory. Have you tried to increase the available memory for Java? (I.e.-Xmx512m). If this is not possible, you can always try using shorts or bytes if the numbers are small enough.

An array will take up as much space as a piece of memory (e.g. c).

Answer 7

This is a known bug in the JVM. It prevents you from creating an array of integers of size 10,000 (as well as 16,384 on Mac OS X). This is due to the way Java translates byte code into machine code. Resizing the array to 10.001 will solve the problem.