Calculate the memory used by a process and its descendants

Photo by Heath Cajandig

Is there a way of calculating -more or less- how much memory a process is using, including large chunks of binaries and child processes??

An approach to the problem would be:

• Find out what the “process tree” associated to the process we’re studying is.
• Sum the memory erlang tells us each one of the process of the “process tree” is consuming.
• Watch out for the binary memory consumption of every process inside that “process tree”.

“Process tree”, what “process tree”??

Keep in mind that in elixir is extremely easy to create/generate processes to help out. Say for example …

{:ok, counter_pid} = Agent.start(fn -> 0 end) 
 
some_iterable = [1, 2, 3, 4, 5] 
 
some_iterable |> Enum.map(fn element -> Agent.update(counter_pid, fn count -> count + element end)) 
 
IO.puts "final count: #{inspect(Agent.get(counter_pid, fn count -> count end))}"

I know, I know it’s a worthless piece of code, but it illustrates the point, which is: creating process is extremely cheap & easy in elixir. If, at any given moment, we wanted to know how much memory this code is using, we would have to know:

• How much memory the process where the code is running take.
• How much memory the process where the agent is running take (what is called counter_pid)

So the first thing to know when studying a process is to know which processes -if any- are “derived” from the process we’re studying. Say our code is being executed inside some process “A”, and part of our code is the use of Task.async/1; that would launch another process — let’s call it process “B”. So now, if we want to know how memory is our code consuming, we should consider the memory consumption of process “A” and also the memory consumption of process “B”.

So we can talk about some “process tree” derived from the main process. Let’s say our “main” process is called “A” and spawns three process, “A1”, “A2” and “A3”. Now suppose “A1” spawns two process “A11” and “A12” and “A3” spawns one “A31” process. Now, if we were to find out how much memory is consumed because process “A” we would have to know:

• The memory used by process “A”
• The memory used by processes “A1”, “A2” and “A3”
• The memory used by processes “A11”, “A12” and “A31”

You can clearly see how a “process tree” is derived.

How erlang VM manages binaries

How to measure how much memory a process is using?? For that we need to know that the memory used to store binaries may not be in what erlang considers to be the memory “used” by the process:

• If the code within a process have a binary that uses less than 64Kb of memory, then that binary is “stored” in the process memory, so when erlang says that process uses some amount of memory, that includes the memory used to store the binary.
• But if the binary uses more than 64Kb of memory, then the binary is NOT “stored” in the process memory, but in the general vm memory, therefore when erlang says that the process uses some amount of memory, that doesn’t include the memory used to store the binary. That information has to be looked up elsewhere.

Why erlang does this? I guess it’s because that way other processes can access those “external” binaries by their reference, let’s say process “A” and process “B” both uses exactly the same binary and that binary takes 2Mb. The both could use the same binary, accessing it by reference, and the memory used to store that is used only once, not twice.

The Tool

The basic tool we use to obtain info from a process is:

Process.info(pid, <some_atom>)

where <some_atom> refers to the “kind of info” we want to retrieve from the process identified by pid. Let’s see what kind of info we can get for a process, depending on the value of <some_atom> :

Basic information about a process

> Process.info(pid, :current_function)
 {:current_function, {SomeModule, :some_function, 2}}
> Process.info(pid, :status)
 {:status, :running}
> Process.info(pid, :memory)
 {:memory, 426552}

Where

• Process.info(pid, :current_function) tells us about **what function is running in this process right now**, it provides, the module name, the function name within that module, and the arity of the function.
• Process.info(pid, :status) tells us about in which status is the process right now, so far, I’ve seen two status and I can reasonably guess what they mean: :running when the process is actually running some code and :waiting when the process is doing nothing, waiting from some signal (see the`receivemacro about that)
• Process.info(pid,:memory) tells us about what memory erlang says this process is using (remember, this does not include the memory used by any binary > 64Kb that the process may be using)

What are “linked process”?

> Process.info(pid, :links)
{:links, [#PID<0.805.0>, #PID<0.806.0>, #PID<0.807.0>, #PID<0.782.0>]}

The :links atom tells us about processes or ports linked to this process. That is, processes that will die when this process die (somehow).

In elixir, when two process are “linked” mean that if one of them dies of non-natural causes, the other one dies, too. What are “natural causes”? when a process dies, all its linked process know about its dead, and also knows about an atom called :reason which the “reason of death”. If it’s {:shutdown, whatever} or :normal or :shutdown, then it’s considered a “natural death” and therefore the linked process doesn’t have to die.

P.S.: The only “natural causes” reason that pure elixir/erlang accepts is :normal, but if you’re creating you process following the OTP Design principles, then you’re using (even If you don’t know), proc_lib for that, and then :shutdown and {:shutdown, whatever} are also accepted as “natural causes”.

What is the “process dictionary”?

> Process.info(pid, :dictionary)
{:dictionary,
  [
    "$initial_call": {IEx.Evaluator, :init, 5},
    …,
    "$ancestors": [#PID<0.2599.0>],
    "$callers": [#PID<0.2599.0>],
    …
  ]}

The :dictionary atom tells us about the process dictionary, That is, a map that every process have, and that we can add key/value pairs to … It can be completely empty. (It’s not guaranteed that it has any value at all)

When you use proc_lib to create process (again, maybe you don’t know you’re using it), then …

Some useful information is initialized when a process starts. The registered names, or the process identifiers, of the parent process, and the parent ancestors, are stored together with information about the function initially called in the process.

That means you can expect to find a process dictionary, and you expect it to has the :”$ancestors” or :”$callers” key …

Am I using :proc_lib ???

All this cooked-in implementations use it “under the hood”:

• GenServer functions
• Agent functions
• Task.async
• Task.async_stream (for the processes used to run every element of the enumerable provided)

So yeah, unless you’re using spawn directly (and thinks like that), I’d say it’s very probable that you’re using :proc_lib everywhere.

There’s a good discussion about it in the elixir forums, though.

The “binaries” of a process

> Process.info(pid, :binary)
{:binary, [{139843198370240, 256, 1}, {94479811247752, 256, 1}]}

As said before, if a binary a process uses is larger than 64Kb, the it’s not stored in the process’s memory, but outside. According to that, Process.info(pid, :binary) returns a list of tuples with the following structure: {reference, memory_used, how_many_times_used}, so in order to know how much “binary” memory is used, we’d had to sum memory info from this list, but, remember, only once!
Say process “A” has in ther binary list {:binary, [{111111, 256, 2}, …]} and process “B”: {:binary, […, {111111, 256, 1}, …}, and the memory calculations we have to do suppose process “B” belongs to the “process tree” derived from “A”. Then the memory of {111111, 256, _} is only to be counted once, even if two process are using it.

The Task.async_stream quirk

When Task.async_stream is launched, a single, monitor process without parents or callers is launched just to collect results of the processes responsible for each element of the enumerable provided
Let’s see with an example, assume we invoke Task.async_stream([1, 2, 3, 4], &some_function/1)
and the process we invoke from is the 111 (#PID<0.111.0>).
Well then:

• A “monitor process” with nothing is its dictionary :”$ancestors” or :”$callers” entries is called. This process is linked to #PID<0.111.0>, though.
• 4 processes (one for every element of the enumerable) are called. Each of these processes run some_function(element). These processes are linked to the “monitor process” and in the :”$ancestors” entry of each, there’s only one element, which is the monitor process and in the :”$callers” entry of each, is [#PID<0.111.0>], the original process that called Task.async_stream in the first place.

Show me the code!

With all this hard-won knowledge, I put together some code than I hope helps you …