SWIG C ++ Python polymorphism and multithreading

Question

I am integrating a third-party C ++ package into a python application using SWIG . The package connects to the proprietary API over the network and receives updates. The general thread is that python instantiates a C ++ object, calls its functions to configure it, and then waits for updates.

I implemented a callback mechanism for updates using the functions of SWIG directors , and during testing using python or from C ++ functions called by python, this works well. Namely, I can inherit the C ++ class in Python, call its virtual functions from C ++ and see that the python code takes precedence and is executed.

Problem:
When I get updates from the network, I get:

The thread 'Win32 Thread' (0x1f78) has exited with code 0 (0x0). Unhandled exception at 0x1e0650cb in python.exe: 0xC0000005: Access violation writing location 0x0000000c. 

This exception is thrown from python27.dll when the callback function is called.
My suspicion is this: I violated the GIL
AFAIU updates come from another thread and invoke python code using this thread.

At this moment I am at a loss. The SWIG director function is limited only to threads initiated in python (i.e. from python-based managed threads)?
How do I get around this? How can I induce updates from C ++ to python? Is SWIG possible?
Should I use a completely different approach?

I am open to any suggestions on this ...

Answer 1

If your SWIG-wrapped C ++ code calls the in-thread reverse procedure, then there is probably no problem with the GIL. The generated SWIG code does not perform any GIL control that I saw, which means that when you call Python code in C ++ code, you save the GIL during the call.

However, if your C ++ code cancels the callback to another thread, you most likely violated the GIL. It's simple enough to get around: before calling the callback, call PyGILState_Ensure (), and when the callback finishes, call PyGILState_Release. See http://docs.python.org/c-api/init.html , section "Threads Generated Other Than Python". (If you use C ++ exception handling here, you might need extra care so that you can release the GIL.)

If you haven't looked at the stack trace yet, it's worth checking that this NULL deref pointer is not something stupid in your code. (You can connect to the Python process running your code using VS / GDB / WinDBG, Python execution will remain incomprehensible, but you can trace your C ++ code this way.)