Why does Perl take my memory (RAM) when printing a file?

Question

This question arises when I notice that my script starts to stop and shuts down when it writes contents to a file in a while loop. First, I check my memory and see a trend that this happens when my operating system has decreased to about 2-3%. Then I look at my block of code that consumes RAM, and I figured that as the script continues to feed the contents to a file, it grows in size, which also takes up memory. Here is the code that creates the file:

open (my $fh, '>>', $filePath); while (my $row = $sth->fetchrow_hashref) { print $fh join ('|', $row->{name}, $row->{address}) "\n"; } 

If I uncomment the print statement and run the script, my RAM will not decrease. So, I am sure that the RAM memory is occupied by FILEHANDLE / something else behind Perl that takes up memory. This leads me to the question, if there is a way to write a file from a Perl script to disk, instead of using RAM memory?

• I tried flushing FILEHANDLE on each line and it still didn't work.
• One thing, which is also very strange, is that after I finish my script, I look at my memory and still process the files. And when I delete files, it frees up my memory. I use free on linux to check my memory.

Answer 1

So, the answer to your question "Why does Perl take my RAM when printing a file?" that Perl does not use memory. The operating system cache uses memory.

Your Perl script wrote a large file. First, it enters the cache file in RAM, the operating system bypasses it, writing it to disk when it has time or when it runs out of RAM. It does not delete it from RAM unless you delete the file, or it needs RAM for something else.

The file cache is not bad, and you should not worry about how much of it your RAM uses. When a program requires more memory, the OS can quickly clear the file cache and reuse memory for your program.

Answer 2

Flushing the file descriptor buffer to disk does not free the buffer, so it does not free memory. It would be useless to free the buffer, since the buffer had to be allocated again the next time the program was launched into a file.

If you're interested, each file descriptor has a 4 KiB or 8 KiB buffer, depending on your version of Perl.