Why is% MEM increasing gradually, although I use Archive :: Tar: Streamed?

Question

I use Archive :: Tar :: Streamed to archive a 4 gigabyte directory containing very small small source code files and some jars. I use a system with 4 GB of RAM and use Archive :: Tar :: Streamed because I do not want my entire directory to be in memory, as this leads to a lack of memory status. I read the files one by one from the directory in DFS mode using File :: Find and writing it to a tar file using the add method in Archive :: Tar :: Streamed. At any given moment in time, only one file remains in memory, even then the% MEM of my perl process gradually increases. It has grown to 20%. There is no 800 MB file in my directory. I have two questions here 1) why does MEM increase gradually? 2) Why it has grown to 20%. This is because Archive :: Tar does not free memory afterhow did she write the contents to a tar file?

Answer 1

This is not normal when your VSZ memory envelope grows, when you process your data in a thread-oriented manner and, moreover, for RES (% MEM), which usually should stabilize in your application.

Therefore, I would strongly suspect that cross-references to data are stored around (for example, in a hash) or (which is less likely) that circular links are generated.

See this post, “are there any tools for finding memory leaks in my perl program” for tips on tracking variables (whether in your code or in the module you use) increase memory or keep cross-references.