Why does a small pickled nested dictionary use a lot of memory?

Question

If I have a nested dictionary, the cPickle'd file size is much smaller than the memory used by the python process when I load it into memory. (I mean the "Memory" column in Windows 7 Task Manager.)

For example, a single 28 MB file seems to be turning into 202 MB of memory. Another 155 MB file occupies 1.50 GB in memory.

Any indication of why this is happening, and how can I reduce this inflation?

A nested dictionary contains only integer keys, and values ​​contain integers or other nested integer dictionaries.

Answer 1

You can try objgraph : it allows you to check what happens in your program memory; It can also generate visual output.

See http://mg.pov.lt/objgraph/ for more details.