Python background function

Question

I have a Python script that sometimes displays images to the user. Images can be quite large at times, and they are often reused. Displaying them is not critical, but displaying the message associated with it. I have a function that loads the required image and saves it locally. Now it runs in a row with code that displays a message to the user, but sometimes it can take more than 10 seconds for non-local images. Is there a way that I could call this function when it was necessary, but run it in the background while the code continues to run? I would just use the default image until it becomes available.

Answer 1

Do something like this:

def function_that_downloads(my_args): # do some long download here 

then inline, do something like this:

 import threading def my_inline_function(some_args): #do some stuff download_thread = threading.Thread(target=function_that_downloads, args=my_args) download_thread.start() #continue doing stuff 

You can check if the thread has completed before moving on to other things by calling download_thread.isAlive()

Answer 2

Typically, the way to do this is to use a thread pool and load the queue up, which will result in a signal, aka event, this task has completed processing. You can do this as part of the stream module that Python provides.

To perform these steps, I would use event objects and a Queue module .

However, a brief and dirty demonstration of what you can do using a simple threading.Thread implementation can be seen below:

 import os import threading import time import urllib2 class ImageDownloader(threading.Thread): def __init__(self, function_that_downloads): threading.Thread.__init__(self) self.runnable = function_that_downloads self.daemon = True def run(self): self.runnable() def downloads(): with open('somefile.html', 'w+') as f: try: f.write(urllib2.urlopen('http://google.com').read()) except urllib2.HTTPError: f.write('sorry no dice') print 'hi there user' print 'how are you today?' thread = ImageDownloader(downloads) thread.start() while not os.path.exists('somefile.html'): print 'i am executing but the thread has started to download' time.sleep(1) print 'look ma, thread is not alive: ', thread.is_alive() 

It might be wise not to question how I am doing the above. In this case, I would change the code to this:

 import os import threading import time import urllib2 class ImageDownloader(threading.Thread): def __init__(self, function_that_downloads): threading.Thread.__init__(self) self.runnable = function_that_downloads def run(self): self.runnable() def downloads(): with open('somefile.html', 'w+') as f: try: f.write(urllib2.urlopen('http://google.com').read()) except urllib2.HTTPError: f.write('sorry no dice') print 'hi there user' print 'how are you today?' thread = ImageDownloader(downloads) thread.start() # show message thread.join() # display image 

Please note that the daemon flag is not set here.

Answer 3

I prefer to use gevent for this:

 import gevent from gevent import monkey; monkey.patch_all() greenlet = gevent.spawn( function_to_download_image ) display_message() # ... perhaps interaction with the user here # this will wait for the operation to complete (optional) greenlet.join() # alternatively if the image display is no longer important, this will abort it: #greenlet.kill() 

Everything works in a single thread, but whenever a kernel lock is blocked, gevent switches contexts when other green ones start. Concerns about blocking, etc. Significantly reduced, since only one thing works at a time, but the image will continue to load whenever a lock operation is performed in the "main" context.

Depending on how much and what you want to do in the background, it can be better or worse than thread-based solutions; Of course, this is much more scalable (i.e. you can do many other things in the background), but this may not bother the current situation.