All geek questions in one place

How to read continuous HTTP streaming data in Python?

How to read binary streams from a streaming http server server in python. I did a search and someone said that urllib2 can do the job, but you have problems with blocking. Someone suggested Twisted framework.

My questions:

  • If only the streaming client reads the data in the background, can I ignore the blocking problems caused by urllib2?

  • What happens if urllib2 doesn't catch up with the streamer server? Will the data be lost?

  • If the streamer server requires user authentication via GET or POST, some parameters on the server before receiving data, can this be done using urllib2?

  • Where can I find an example urllib2 and twisted thread client?

Thanks.

Jack

+3
source share
1 answer

To defeat urllib2's internal buffering, you can do:

import socket
socket._fileobject.default_bufsize = 0

because it is the actual socket._fileobjectone that buffers under it. In any case, the data will not be lost, but with default buffering (8192 bytes at a time), the data may turn out to be too interleaved for real-time streaming (completely removing buffering may hurt performance, but you can try smaller chunks).

For Twisted, see twisted.web2.stream and many links from them.

+6
source

Source: https://habr.com/ru/post/1720630/

More articles:


All Articles