Line Printing, Java Heap Space

Question

I want to print each line from a huge text file (over 600,000 MB).

But when I try to execute the code below, I get "... OutOfMemoryError: Java heap space" right before I reach the line number of 1,000,000.

Is there a better way to handle input than FileReader and LineNumberReader?

FileReader fReader = new FileReader(new File("C:/huge_file.txt")); LineNumberReader lnReader = new LineNumberReader(fReader); String line = ""; while ((line = lnReader.readLine()) != null) { System.out.println(lnReader.getLineNumber() + ": " + line); } fReader.close(); lnReader.close(); 

Thanks in advance!

---

Thank you all for your answers!

I finally found a memory leak, an unused instance of the Java class that duplicated it for each iteration of the lines. In other words, it had nothing to do with the file upload part.

Answer 1

LineNumberReader extends BufferedReader . Perhaps the buffered reader is buffering too much. Running the program through the profiler should prove this without any doubt.

One of the BufferedReader constructors accepts the size of the buffer; this constructor is also available in the line reader.

replace:

 LineNumberReader lnReader = new LineNumberReader(fReader); 

from:

 LineNumberReader lnReader = new LineNumberReader(fReader, 4096); 

Answer 2

Perhaps you should try setting the maximum heap size for the Java virtual machine? Or check out this link:

http://www.techrepublic.com/article/handling-large-data-files-efficiently-with-java/1046714

Answer 3

use this class to read the file: RandomAccessFile , after which you will no longer have a memory problem