If I have a link to a related method in Python, will only this object stay alive?

Question

I wrote something like this today (not much different from the mpl_connect documentation:

class Foo(object): def __init__(self): print 'init Foo', self def __del__(self): print 'del Foo', self def callback(self, event=None): print 'Foo.callback', self, event from pylab import * fig = figure() plot(randn(10)) cid = fig.canvas.mpl_connect('button_press_event', Foo().callback) show() 

It looks reasonable, but it will not work - as if matplotlib is losing track of the function that I gave it. If instead of passing Foo().callback pass it to lambda e: Foo().callback(e) , it works. Similarly, if I say x = Foo() and then pass it to x.callback , it works.

My presumption is that the nameless instance of Foo created by Foo() is immediately destroyed after the mpl_connect line, that matplotlib, having the Foo.callback link, does not save Foo live. It's right?

In the non-toy code I came across, the solution x = Foo() does not work, apparently because in this case show() was in a different place, so x went out of scope.

In general, Foo().callback is a <bound method Foo.callback of <__main__.Foo object at 0x03B37890>> . My main surprise is that it looks like the related method does not actually support an object reference. It is right?

Answer 1

Yes, the associated method refers to an object - the object is the value of the attribute of the object of the attached method .im_self .

So, I am wondering if matplotlib mpl_connect() remembers the increase in reference counts for the arguments passed to it. If not (and this is a common mistake), then there is nothing to keep the anonymous Foo().callback active when mpl_connect() returned.

If you have easy access to the source code, take a look at the mpl_connect() implementation? You want the C code to execute Py_INCREF() ; -)

EDIT This looks relevant, from the documentation here :

The canvas only retains weak references to callbacks. Therefore, if the callback is an instance method of a class, you need to keep a reference to that instance. Otherwise, the instance will contain garbage and the callback will disappear.

So this is your mistake - LOL; -)

Answer 2

Here's the rationale from matplotlib.cbook.CallbackRegistry.__doc__ :

In practice, you always need to disable all callbacks when they are no longer needed in order to avoid dangling links (and, consequently, Leak memory). However, the actual code in matplotlib rarely does this and it is quite difficult to design such code by its design. To get around this and prevent this class of memory leaks, we instead store weak references only to related methods, so when the target must die, CallbackRegistry will not keep it alive. The Python slide module module stdlib cannot create weak links to related methods directly, so we need to create a proxy object to handle weak links to related methods (or regular free functions). This method was shared by Peter Parente on his "Mindtrove" blog <http://mindtrove.info/articles/python-weak-references/> _.

It is a shame that there is no official way around this behavior.

Here's a clue to get around it, which is dirty, but might be OK for an unproductive test / diagnostic code: attach the Foo number:

 fig._dont_forget_this = Foo() cid = fig.canvas.mpl_connect('button_press_event', fig._dont_forget_this.callback) 

This still leaves the question of why lambda e: Foo().callback(e) works. Obviously, it makes a new Foo on every call, but why doesn't the lambda get garbage collection? Is the fact that it only works if the behavior is undefined?