Python threading.Thread, scope and garbage collection

Question

Say what I get from threading.Thread:

from threading import Thread class Worker(Thread): def start(self): self.running = True Thread.start(self) def terminate(self): self.running = False self.join() def run(self): import time while self.running: print "running" time.sleep(1) 

Any instance of this class with a running thread must have a thread that is actively shutting down before it can garbage collect (the thread contains a link). So this is a problem because it completely ignores the goal of garbage collection. In this case, if some object encapsulates the stream, and with the last instance of the object leaving the scope, the destructor receives a call to complete and clear the stream. Thuss destructor

  def __del__(self): self.terminate() 

won't do the trick.

The only way I see encapsulating threads nicely is to use the built-in thread module with low level and weakref weak links. Or maybe I'm missing something fundamental. So better than weakref things up in weakref spaghetti code?

Answer 1

What about using a wrapper class (which has-a Thread , not is-a Thread )?

eg:

 class WorkerWrapper: __init__(self): self.worker = Worker() __del__(self): self.worker.terminate() 

And then use these wrapper classes in client code, not the threads directly.

Or maybe I missed something (:

Answer 2

To add an answer inspired by @datenwolf's comment, here is another way to do this, related to the object being deleted or the parent thread terminating:

 import threading import time import weakref class Foo(object): def __init__(self): self.main_thread = threading.current_thread() self.initialised = threading.Event() self.t = threading.Thread(target=Foo.threaded_func, args=(weakref.proxy(self), )) self.t.start() while not self.initialised.is_set(): # This loop is necessary to stop the main threading doing anything # until the exception handler in threaded_func can deal with the # object being deleted. pass def __del__(self): print 'self:', self, self.main_thread.is_alive() self.t.join() def threaded_func(self): self.initialised.set() try: while True: print time.time() if not self.main_thread.is_alive(): print('Main thread ended') break time.sleep(1) except ReferenceError: print('Foo object deleted') foo = Foo() del foo foo = Foo() 

Answer 3

I assume that you are converting from C ++, where a lot of meaning can be attached to areas of variables equal to the lifetimes of the variables. This does not apply to Python, and the garbage collector in general. Scope! = Lifetime simply because garbage collection occurs whenever the interpreter approaches it, and not along the boundaries of the area. Especially since you are trying to make asynchronous material with it, the raised hair on your neck should vibrate until all the warning bells in your head scream! You can do things with the lifetime of objects using del . (In fact, if you read the sources in the cpython garbage collection module, the obvious (and somewhat ridiculous) contempt for objects with finalizers ( del ) expressed there should tell everyone to use even the object's lifetime only if necessary).

You can use sys.getrefcount (self) to find out when to leave the loop in your thread. But I can hardly recommend it (just try what numbers will return. You will not be happy. To find out who holds what just check gc.get_referrers (self)). The reference count may / will depend on garbage collection.

In addition, linking the execution time of the execution thread to the areas / lifetimes of objects is a 99% error. Even Boost does not. From your RAII method, you can define something called a “disconnected” thread. http://www.boost.org/doc/libs/1_55_0/doc/html/thread/thread_management.html