Python sends an HTTP response

Question

I am trying to write a very simple HTTP server that sends streaming video on the server side. When the client connects, the get_video program (fictitious) starts in another process and its stdout is passed to us (assuming that get_video sends / transmits video to stdout). I am using subprocess.Popen() for this.

 import subprocess, socket def send_video(sock, programme_id): p = subprocess.Popen(["get_video","--pid",programme_id], stdout=subprocess.PIPE) sock.send("HTTP/1.1 200 OK\nContent-type: application/octet-stream\n\n") while True: chunk = p.stdout.read(1024) if chunk: try: sock.send(chunk) except Exception: pass else: break def main(): s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('',8080)) s.listen(5) while True: client, address = s.accept() data = client.recv(1024) send_video(client, "123456") client.close() if __name__ == "__main__": main() 

It works. If I issue an HTTP request using wget http://localhost:8080/blah.mp4 , everything will look as expected - the video is transferred to the client socket and added to the new blah.mp4 file.

However, if I create a dummy HTML page with <a href="http://localhost:8080/blah/mp4">download</a> , and then try to save target / link as ... 'from this link, get_video is called twice. The second time the video is really sent to the client socket. There must be some kind of buffering.

Note the try/except block in send_video() . I used to get a “broken pipe” error (using the HTML layout page method), indicating that the client socket was not written. I set try/except to try to ignore it.

I'm pretty confused. HTTP requests look the same, I'm not sure what my browser (Firefox) does differently to trigger this.

Any ideas?

Edit:

Header from wget:

 GET /blah.mp4 HTTP/1.0 User-Agent: Wget/1.12 (linux-gnu) Accept: */* Host: 192.168.1.2:8080 Connection: Keep-Alive 

Title from html dummy:

 GET /blah.mp4 HTTP/1.1 Host: 192.168.1.2:8080 User-Agent: Mozilla/5.0 (blah) Gecko/blah Firefox/3.6.16 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-gb,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 115 Connection: keep-alive Pragma: no-cache Cache-Control: no-cache 

The python server reported only one of them, but Wireshark reported 2 GET requests for the dummy html link method. I don’t see anything obvious here ...

Answer 1

It became clear to me that since the browser sends a GET and then FIN, ACK after the "Save Target As ..." link was clicked, then this will cause a broken channel error. This is when you choose where you want to save the file , and click OK in your browser so that another GET is released. What was happening, I was still stuck in this while True when I discovered a broken pipe. Therefore, I continued to read from the Popen object and did not send it to the client every time. Once this is over, the next GET was able to continue, and the transfer was successful. Phew

In short, working code:

 import subprocess, socket def send_video(sock, programme_id): p = subprocess.Popen(["get_video","--pid",programme_id], stdout=subprocess.PIPE) sock.send("HTTP/1.1 200 OK\nContent-type: application/octet-stream\n\n") while True: chunk = p.stdout.read(1024) if chunk: try: sock.send(chunk) except socket.error, e: sock.close() break else: break def main(): s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) s.bind(('',8080)) s.listen(5) while True: client, address = s.accept() data = client.recv(1024) send_video(client, "123456") client.close() if __name__ == "__main__": main() 

Thanks.