Python connector is closed before all data is deleted

Question

I am writing a Python module that interacts with the go program through unix sockets. The client (python module) writes data to the socket, and the server consumes it.

# Simplified version of the code used outputStream = socket.socket(socketfamily, sockettype, protocol) outputStream.connect(socketaddress) outputStream.setblocking(True) outputStream.sendall(message) .... outputStream.close() 

My problem is that the Python client tends to end and close the socket before the data has been read efficiently by the server, which leads to a "broken pipe, connection reset by peer" server-side. No matter what I do, everything is sent for the Python code, and so the calls to send () sendall () select () completed successfully ...

Thank you in advance

EDIT: I cannot use shutdown due to Mac OS

EDIT2: I also tried removing the timeout and calling setblocking (True), but nothing changes

EDIT3: after you prepared this release of http://bugs.python.org/issue6774 , it seems the documentation is not needed, so I restored the shutdown, but I still have the same question:

 # Simplified version of the code used outputStream = socket.socket(socketfamily, sockettype, protocol) outputStream.connect(socketaddress) outputStream.settimeout(5) outputStream.sendall(message) .... outputStream.shutdown(socket.SHUT_WR) outputStream.close() 

Answer 1

IHMO is best done with the Asynchornous I / O library / framework. Here's a solution using circuits :

The echos server receives what it receives in stdout, and the client opens the file and sends it to the server, waiting for it to complete before the socket is closed and completed. This is done with a mixture of Async I / O and Coroutines.

server.py:

 from circuits import Component from circuits.net.sockets import UNIXServer class Server(Component): def init(self, path): UNIXServer(path).register(self) def read(self, sock, data): print(data) Server("/tmp/server.sock").run() 

client.py:

 import sys from circuits import Component, Event from circuits.net.sockets import UNIXClient from circuits.net.events import connect, close, write class done(Event): """done Event""" class sendfile(Event): """sendfile Event""" class Client(Component): def init(self, path, filename, bufsize=8192): self.path = path self.filename = filename self.bufsize = bufsize UNIXClient().register(self) def ready(self, *args): self.fire(connect(self.path)) def connected(self, *args): self.fire(sendfile(self.filename, bufsize=self.bufsize)) def done(self): raise SystemExit(0) def sendfile(self, filename, bufsize=8192): with open(filename, "r") as f: while True: try: yield self.call(write(f.read(bufsize))) except EOFError: break finally: self.fire(close()) self.fire(done()) Client(*sys.argv[1:]).run() 

In my testing, this behaves exactly as I expect, errors and servers receive the full file before the client clsoes socket on and off.

Answer 2

After a discussion with a colleague aware of C sockets (in cpython, the socket module is a wrapper for C sockets), he talked about this http://ia600609.us.archive.org/22/items/TheUltimateSo_lingerPageOrWhyIsMyTcpNotReliable/the-ultimate-so_linger- page-or-why-is-my-tcp-not-reliable.html (how this is done in PHP for writing)

TL & DR: shutdown + quick polling + shutdown or ioctl (SIOCOUTQ) on linux