How to find the amount of physical memory used by a hash in Perl?

Question

I have a Perl script where I maintain a very simple cache using a hash table. I would like to clear the hash if it takes more than n bytes in order to avoid memory overflow (32-bit version) and memory failure.

I can check the number of key-value pairs:

if (scalar keys %cache > $maxSize) { %cache = (); } 

But is it possible to check the actual memory occupied by the hash?

Answer 1

Devel :: Size - the answer to your question. (Note that Devel :: Size temporarily allocates a significant amount of memory when processing a large data structure, so it is not well suited for this purpose.)

However, Cache :: SizeAwareMemoryCache and Tie :: Cache already implement what you are looking for (with several different interfaces) and can save you from rethinking the wheel.

Memoize is a module that makes it easy to cache the return value from a function. It does not implement a size-based cache limit, but it should be possible to use Tie :: Cache as the backend for Memoize.

Answer 2

Are you looking for Devel :: Size

NAME

Devel :: Size - Perl extension for finding memory usage of Perl variables

SYNTAX

 use Devel::Size qw(size total_size); my $size = size("A string"); my @foo = (1, 2, 3, 4, 5); my $other_size = size(\@foo); my $foo = {a => [1, 2, 3], b => {a => [1, 3, 4]} }; my $total_size = total_size($foo); 

Answer 3

You can set Devel :: Size to find out the memory received by any construct in Perl. However, keep in mind that this will require a lot of intermediate memory, so I would not use it against a large data structure. Of course, I would not do this if you think that you may have a lack of memory.

By the way, in CPAN there are some good modules for caching in memory and otherwise. Instead of rolling on my own, I would suggest using one of them. For example, try Tie :: Cache :: LRU for a cache in memory that will go only up to the specified number of keys.

Answer 4

You can use Devel :: Size to determine the memory used, but usually you cannot return the RAM. It looks like you're just trying to clean and reuse, although this should work fine.

If the cache is intended for a function, consider using the Memoize module rather than saving the cache. It supports caching (via Memoize :: Expire), so you can limit the size of the cache without destroying it completely.

Answer 5

Cache :: memory

 use Cache::Memory; my $cache = Cache::Memory->new( namespace => 'MyNamespace', default_expires => '600 sec' ); my $size = $cache->size() my $limit = $cache->size_limit(); 

Answer 6

If you are worried about managing the amount of memory used by Perl, you should probably take a look at an alternative approach. Why do you need so much in RAM at once? Should you use some kind of perseverance system?

Answer 7

As others have said, caching is not a wheel that you need to reinvent; there are many simple caching solutions on CPAN that do a great job.

Cache :: SizeAwareMemoryCache you can specify the maximum size that you want to use, then you can leave it to care for the cache for you.