Python ctype - How to pass data between C functions

Question

I have a makeshift C library with which I want to access using python. The problem is that the code consists essentially of two parts, initialization for reading in data from several files and several calculations that need to be performed only once. The other part is called in a loop and uses the data generated earlier several times. To this function, I want to pass parameters from python.

My idea was to write two C shell functions, "init" and "loop" - "init" reads the data and returns a void pointer to a structure that loop can use along with additional parameters that I can pass from python . Sort of

void *init() { struct *mystruct ret = (mystruct *)malloc(sizeof(mystruct)); /* Fill ret with data */ return ret; } float loop(void *data, float par1, float par2) { /* do stuff with data, par1, par2, return result */ } 

I tried calling init from python as c_void_p, but since the loop changes part of the contents of the data and the void void pointers are immutable, this did not work.

Other solutions to similar problems that I have seen seem to require knowing how much memory "init" will use, and I don't know that.

Is there a way to pass data from one C function to another via python without telling python exactly what and how it is? Or is there another way to solve my problem?

---

I tried (and could not) write a minimal emergency example, and after some debugging it turned out that there was an error in my C code. Thanks to all who responded! Hoping this can help other people, here is something like a minimal working version (still without a separate “free” - sorry):

pybug.c:

 #include <stdio.h> #include <stdlib.h> typedef struct inner_struct_s { int length; float *array; } inner_struct_t; typedef struct mystruct_S { int id; float start; float end; inner_struct_t *inner; } mystruct_t; void init(void **data) { int i; mystruct_t *mystruct = (mystruct_t *)malloc(sizeof(mystruct_t)); inner_struct_t *inner = (inner_struct_t *)malloc(sizeof(inner_struct_t)); inner->length = 10; inner->array = calloc(inner->length, sizeof(float)); for (i=0; i<inner->length; i++) inner->array[i] = 2*i; mystruct->id = 0; mystruct->start = 0; mystruct->end = inner->length; mystruct->inner = inner; *data = mystruct; } float loop(void *data, float par1, float par2, int newsize) { mystruct_t *str = data; inner_struct_t *inner = str->inner; int i; inner->length = newsize; inner->array = realloc(inner->array, newsize * sizeof(float)); for (i=0; i<inner->length; i++) inner->array[i] = par1 + i * par2; return inner->array[inner->length-1]; } 

compile as

 cc -c -fPIC pybug.c cc -shared -o libbug.so pybug.o 

Running in python:

 from ctypes import * sl = CDLL('libbug.so') # What arguments do functions take / return? sl.init.argtype = c_void_p sl.loop.restype = c_float sl.loop.argtypes = [c_void_p, c_float, c_float, c_int] # Init takes a pointer to a pointer px = c_void_p() sl.init(byref(px)) # Call the loop a couple of times for i in range(10): print sl.loop(px, i, 5, 10*i+5) 

Answer 1

You must have the appropriate function for a free data buffer when the caller is executed. Otherwise, I do not see the problem. Just pass the loop pointer you get from init .

 init.restype = c_void_p loop.argtypes = [c_void_p, c_float, c_float] loop.restype = c_float 

I'm not sure what you mean by "ctypes" void pointers are unchangeable "unless you talk about c_char_p and c_wchar_p . The problem is that if you pass a Python string as an argument, it uses the Python private pointer for the string buffer. If the function can change the string, you must first copy it to the c_char or c_wchar .

Here is a simple example showing the problem of passing a Python string (byte string 2.x) as an argument to a function that modifies it. In this case, it changes the index 0 to '\ x00':

 >>> import os >>> from ctypes import * >>> open('tmp.c', 'w').write("void f(char *s) {s[0] = 0;}") >>> os.system('gcc -shared -fPIC -o tmp.so tmp.c') 0 >>> tmp = CDLL('./tmp.so') >>> tmp.f.argtypes = [c_void_p] >>> tmp.f.restype = None >>> tmp.f('a') >>> 'a' '\x00' >>> s = 'abc' >>> tmp.f(s) >>> s '\x00bc' 

This refers to passing Python strings as arguments. It is not a problem to pass pointers to data structures that must be mutable, or ctypes data objects such as Structure , or pointers returned by libraries.

Answer 2

Is your C code in a DLL? If possible, you might want to create a global pointer there. init () will do any required initialization and set the pointer to the newly allocated memory, and loop () will work in that memory. Also remember to free it with the close () function