All geek questions in one place

Overview / History of Resident Memory Usage

I have a rather complicated program (Python with SWIG'ed C ++ code, long running server) that shows the ever-increasing memory usage of resident memory. I dug the usual leak tools (valgrind, Pythons gc module, etc.), but so far nothing has worked. I'm a little afraid that the actual problem is memory fragmentation in Python and / or libc managed memory.

In any case, my question is more specific right now: is there a tool to visualize the use of resident memory and ideally show how it develops over time? I think the source data is in / proc / $ PID / smaps, but I was hoping there would be some tool that would show me a good graph of the sums used by mmap's files against mmap's anonymous memory against heap over time, it’s easier to see (literally) that which is changing. I could not find anything, though.

Does anyone know of a tool that intuitively graphically displays a memory card (memory type and quantity) of a specific process in space and time?

Update: I found the "pmap" tool, but the version on my system does not seem to process RSS and does not allow the ability to merge sizes for all associated files, respectively. displayed "anon" area. As a result, I hacked a little script that parses / proc / $ PID / smaps every two minutes while the original program starts, and prints lines like this:

12:00:28 {'_TOTAL': 729.20703125, 'file': 53.609375, 'heap': 22.08984375,
          'anon': 653.5, 'stack': 0.0078125}
...
15:42:47 {'_TOTAL': 940.16015625, 'file': 53.484375, 'heap': 22.2109375,
          'anon': 864.45703125, 'stack': 0.0078125}

There is nothing good in the graph, but after several hours of work, I think it’s safe to bet on the fact that I need to take a closer look at the anon memory segments :-)

:. valgrind ( "" ) --pages-as-heap=yes. ! , Massif Visualizer , , , mmap.\/

+3
3

vmstat, .., :

[~]> vmstat -S K 1
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
1  0    172 663244 410016 187756    0    0     6    12   14    4  0  0 99  0
0  0    172 663228 410016 187756    0    0     0    68   20   66  0  0 100  0
0  0    172 663228 410016 187756    0    0     0     0   12   54  0  0 100  0
0  0    172 663228 410016 187756    0    0     0     0   20   54  0  0 100  0
^C
0

This is rare, but you can check for a file (or socket) leak, I mean, when a program opens files and never closes them. In my desktop configuration, there were no signs of a socket leak until it exceeded 1000+. Of course they were open. 1 / sec, so he appeared a few days later. This evil!

0
source

Source: https://habr.com/ru/post/1782114/

More articles:


All Articles