Memory leak observed in Erlang application

Question

Let me ask my question as simple as shown below. Mine is the network router software built into erlang, but under a certain scenario I can see very high memory growth as shown by VM.

I have one process that receives a binary package from another process from a socket.

This process parses the binary package and passes the binary package gen_server (handle_cast is called)

Gen_server again saves some information in the ETS table and sends the packet to the peer server.

When the peer server responds, the record from the ETS is deleted and gen_server responds to the first process

Also, if the first process (which sent the gen_server packet) receives a timeout after 5 seconds of waiting for a response from gen_server, it also deletes the ETS record in gen_server and exits.

Now I observe a high memory growth when many events fail (due to the inaccessibility of the peer server) and from the fact that I examined its "** binary **" and "** process_used **" given by the erlang: memory command using most of the memory.

but this is not the case when events are successfully processed.

Answer 1

Lost memory can be basically only in three places:

• The state of your gen_server

  • look at yourself, find out if there is large or growing material.

• Process Mailboxes

  • make sure that there is a way to disable unsurpassed messages (for gen_server handle_info callback) in the usual receive a Any -> .

  • if the mailbox is only temporarily full, probably due to a too slow receiving process for the speed of the generated messages. This is usually a problem for asynchronous communication. If his only temporary bursts that do not break anything, this may be intended.

    • In this case, you can optimize the process of obtaining

    • or fix the protocol to use less messages

    • If you have several functions that receive some messages, make sure that all receiving parts are called regularly. Do not forget about the sentences Any -> .

  • Keep in mind that during processing in the gen_servers callback, messages will not be received, therefore, if you need more time in the callback that will be required, asynchronous messages can accumulate (for example, random message occurrence + fixed processing time builds an unlimited growing queue, for details see Queue Theory

• In the ETS table

  • Perhaps the information in the ETS is not completely deleted? Forgot to delete something in certain cases?

Answer 2

start the GC manually and see what happens to the memory.

 [garbage_collect(Pid) || Pid <- processes()] 

Answer 3

Most likely, you are leaving running processes that have links to binary files. If a process dies, all memory associated with this process will be cleared (including any binaries belonging only to this process).

If you still have binary leaks, it means that you have some kind of lengthy process (server, singleton, etc.) that maintains links to binary files, both in process state and in irregular recursive ones functions. Make sure you clear your condition as soon as the communication process expires or they die. Also, make sure that you do not leave references to binary files on the heap using irregular non-recursive calls.