Trigger function after returning HttpResponse from django view

Question

I am developing a django web server on which another machine (with a known IP) can upload a spreadsheet to my web server. After the spreadsheet has been updated, I want to call some processing / verification / analysis in the spreadsheet (which may take> 5 minutes --- too long for another server to wait for a response) and then send another computer (with known IP) HttpResponse. indicating that data processing is complete.

I understand that you cannot do processing.data() after returning HttpResponse , but functionally I want the code to look something like this:

 # processing.py def spreadsheet(*args, **kwargs): print "[robot voice] processing spreadsheet........." views.finished_processing_spreadsheet() # views.py def upload_spreadsheet(request): print "save the spreadsheet somewhere" return HttpResponse("started processing spreadsheet") processing.data() def finished_processing_spreadsheet(): print "send good news to other server (with known IP)" 

I know how to write each function individually, but how can I effectively call processing.data() after views.upload_spreadsheet returned the answer?

I tried using the django request_finished signal structure , but this does not call the processing.spreadsheet() method after returning the HttpResponse . I tried using the decorator on views.upload_spreadsheet with the same problem.

I have a suspicion that this may have something to do with writing middleware or perhaps custom view classes , none of which I have experience with, so I thought I would pose a question to the universe in search of some help .

Thanks for your help!

Answer 1

In fact, Django has a synchronous model. If you want to do true asynchronous processing, you need a message queue. The most used with django is celery, it may look a little "outwit", but this is a good answer.

Why do we need this? because in a wsgi application, apache passes the request to the executable, and the executable returns the text. This is only once when the executable completes its execution, that apache introduces the completion of the request.

Answer 2

The problem with your implementation is that if the number of tables in the process is equal to the number of workers: your site will no longer respond.

You should use the background task queue , basically have 2 processes: your server and background task manager . The server must delegate the processing of the spreadsheet to the background task manager . When the background task is completed, it must somehow inform the server. For example, it can execute model_with_spreadsheet.processed = datetime.datetime.now ().

You should use a background job manager , such as django-ztask (very simple setup), celery (very powerful, perhaps brute force in your case) or even uwsgi spooler (which clearly requires uwsgi deployment).