All geek questions in one place

Processing Asynchronous Threads in Python

Let's start with a simple example. The HTTP data stream arrives in the following format:

MESSAGE_LENGTH, 2 bytes
MESSAGE_BODY, 
REPEAT...

I am currently using urllib2 to extract and process streaming data, as shown below:

length = response.read(2)
while True:
    data = response.read(length)
    DO DATA PROCESSING

It works, but since all messages are 50-100 bytes in size, the above method limits the size of the buffer each time it reads, which can hurt performance.

Can separate streams be used to retrieve and process data?

+3
source share
2 answers

, , . , , , , . -, , , .

, "", "", , , .: -)

, , , , Python , . , , , , , , .

, , URL-, , , , URL- , "". , URL-, -, URL- "outqueue", , .

, NFS, ( ), , ( ).

.

0

, , .

httplib Python 2.2.3 , , ( select() httplib).

, , ( , httplib- chunked http-, read()).

statemachine, , , Queue.Queue, .

, (zlib.ADLER32) . 40 / HTTP/chunked.

+1

Source: https://habr.com/ru/post/1720652/

More articles:


All Articles