A static pointer to a dynamically allocated buffer inside a function

Question

I have a function in C that dynamically allocates a buffer that is passed to another function to store the return value. Something like the following dummy example:

void other_function(float in, float *out, int out_len) { /* Fills 'out' with 'out_len' values calculated from 'in' */ } void function(float *data, int data_len, float *out) { float *buf; int buf_len = 2 * data_len, i; buf = malloc(sizeof(float) * buf_len); for (i = 0; i < data_len; i++, data++, out++) { other_function(*data, buf, buf_len); /* Do some other stuff with the contents of buf and write to *out */ } free buf; } 

function is called by an iterator over a multidimensional array (this, of course, is the core of NumPy gufunc), so it is called millions of times with the same value for data_len . It seems wasteful to create and destroy a buffer over and over again. Usually I move the buffer allocation to the function that calls the function and pass it a pointer, but I do not control it directly, so this is not possible. Instead, I plan to do the following:

 void function(float *data, int data_len, float *out) { static float *buf = NULL; static int buf_len = 0; int i; if (buf_len != 2 * data_len) { buf_len = 2 * data_len; buf = realloc(buf, sizeof(float) * buf_len); /* same as malloc if buf == NULL */ } for (i = 0; i < data_len; i++, data++, out++) { other_function(*data, buf, buf_len); /* Do some other stuff with the contents of buf and write to *out */ } } 

This means that I never directly free the memory that I allocate: it is used again in subsequent calls, and then lingers there until the exit of my program. This is not like what needs to be done, but not so bad as the amount of allocated memory will always be small. I'm worry? Is there a better approach to this?

Answer 1

This approach is legal (but see below), although tools like valgrind will incorrectly mark it as a “leak”. (This is not a leak, since a leak is an unlimited increase in memory usage.) You might want to determine exactly how much time is wasted on malloc and free compared to other functions that the function performs.

If you can use C99 or gcc, and if your buffer is not too large, you should also consider variable-length arrays that are just as fast (or faster) as a static buffer and do not create fragmentation. If you are on a different compiler, you can explore the non-standard (but wide ) alloca extension.

You need to know that using a static buffer makes your function:

• Thread-unsafe - if it is called from several threads at the same time, it will destroy the data of another instance. If Python is called from numpy, this is probably not a problem since the threads will be effectively serialized by the GIL.

• Non-reentrant - if other_function calls some Python code that ends with a function call - for some reason - ends before the function ends, your function will destroy its own data again.

Unless you need true parallel execution and repositioning, this use of static variables is great, and a lot of C code uses them that way.

Answer 2

This is a great approach, and something like this is probably used inside many libraries. When you exit the program, memory is automatically freed.

You might want to round buf_len to a multiple of some block size, so you don't realloc() every time data_len changes a small bit. But if data_len almost always the same size, this is optional.