Using Perl memory profiles and leak detection?

Question

I wrote a persistent network service in Perl that runs on Linux.

Unfortunately, as it works, its stack size (RSS) just grows, grows and slowly grows, but grows steadily.

This, despite the diligent efforts on my part, to exclude all unnecessary hash keys and remove all references to objects, which otherwise cause the reference count to remain in place and prevent garbage collection.

Are there any good tools for profiling memory usage related to various source data primitives, hash-referenced object blocks, etc. in a Perl program? What do you use to track memory leaks?

I'm not used to wasting time in the Perl debugger or on any of the various interactive profilers, so a warm, gentle, non-esoteric answer would be appreciated. :-)

Answer 1

You may have a circular link to one of your objects. When the garbage collector comes to free this object, the circular link means that everything that is mentioned in this link will never be freed. You can check the circular links of Devel :: Cycle and Test :: Memory :: Cycle . One thing to try (although it can become expensive in production code, so I would disable it when the debug flag is not set) checks for circular references inside the destructor for all of your objects:

# make this be the parent class for all objects you want to check; # or alternatively, stuff this into the UNIVERSAL class destructor package My::Parent; use strict; use warnings; use Devel::Cycle; # exports find_cycle() by default sub DESTROY { my $this = shift; # callback will be called for every cycle found find_cycle($this, sub { my $path = shift; foreach (@$path) { my ($type,$index,$ref,$value) = @$_; print STDERR "Circular reference found while destroying object of type " . ref($this) . "! reftype: $type\n"; # print other diagnostics if needed; see docs for find_cycle() } }); # perhaps add code to weaken any circular references found, # so that destructor can Do The Right Thing } 

Answer 2

You can use Devel :: Leak to search for memory leaks. However, the documentation is quite rare ... for example, where to get the $ handle link to go to Devel::Leak::NoteSV() ? f I will find the answer, I will edit this answer.

Well, it turns out that using this module is pretty simple (the code is shamelessly stolen from Apache :: Leak ):

 use Devel::Leak; my $handle; # apparently this doesn't need to be anything at all my $leaveCount = 0; my $enterCount = Devel::Leak::NoteSV($handle); print STDERR "ENTER: $enterCount SVs\n"; # ... code that may leak $leaveCount = Devel::Leak::CheckSV($handle); print STDERR "\nLEAVE: $leaveCount SVs\n"; 

I would put as much code as possible in the middle section, while checking leaveCount as close to the end of execution (if you have one) - after most of the variables have been freed as much as possible (if you can't get the variable from the scope apparently, you can assign undef to it to free everything that it points to).

Answer 3

What else should I try (I'm not sure that this will be best placed in the comments after the aforementioned question by Alex): what would I try next (except for Devel :: Leak):

Try to eliminate the "unnecessary" parts of your program or segment it into separate executable files (they can use signals to communicate or, possibly, with command line arguments) - the goal is to collapse the executable file in the least amount of code, which is still demonstrates bad behavior . If you are sure that this is not your code that does this, reduce the number of external modules that you use, especially those that have an XS implementation. If maybe this is your own code, find something potentially suspicious:

• definitely any use of Inline :: C or XS code
• direct use of links, for example. \@list or \%hash , and not predefined links, such as [qw (foo bar)] (the first creates another link that may be lost, and the second only one link that you need to worry about, which is usually stored in the local lexical scalar
• manipulating variables indirectly, for example. $$foo , where $foo changed, which may cause automatic passing of variables (although you need to disable strict 'refs' checking)

Answer 4

A good reference on this is in the Perl manual: Debugging Perl Memory Usage

Answer 5

I recently used NYTProf as a profiler for a large Perl application. It does not track memory usage, but it tracks all executable code codes that help you figure out where the leaks are coming from. If what you are leaking is limited resources, such as database connections, tracking where they are allocated and closed, is a long way to finding leaks.