Real-time data processing in Django / Python

Question

We are working on a project related to real-time data processing. We plan to use Django / Python. Actual process:

• Tens of thousands of devices take 4 samples per second (0, 0.25, 0.5, 0.75) and continuously send them back to our Django server, basically these are time series with time stamp and value
• We need to align the samples from all devices in accordance with the time stamp (it is necessary to have millisecond accuracy) and make a simple average of all time series
• All this must be done in real time (maximum delay for 1 second) and sent using another stream

We study RRDTool and scikits.timeseries, but they do not have millisecond accuracy, so they cannot align our time series.

Just wondering if there are any tools / data structure that we can use with Django / Python for this type of real-time data processing. And thread safety is important, as sending the result will be done in another thread.

Thanks in advance.

Answer 1

Your options for real-time web services in python: Twisted , Tornado, and Eventlet

You can integrate all this to work with Python / Django. A tutorial on this .

Answer 2

Short answer: None. Django will not help you with this.

: - , -. , python script wsgi apache, C!

Answer 3

Tornado, -, epoll. , , Django.

Answer 4

. Django , , . django Brubeck Mongrel2 ZeroMQ.