Previous Warning : I am hacking out of curiosity here. I have no particular reason to do what I do below!
The following is done Python 2.7.13 on MacOS 10.12.5
I hacked python and I thought it would be interesting to see what happened if I did stdout non-blocking
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK)
The fcntl call is definitely successful. Then I try to write a large amount of data (more than the maximum channel buffer size on OSX, which is 65536 bytes). I do it in different ways and get different results, sometimes an exception, sometimes what seems like a difficult failure.
Case 1
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) try: sys.stdout.write("A" * 65537) except Exception as e: time.sleep(1) print "Caught: {}".format(e) # Safety sleep to prevent quick exit time.sleep(1)
This always raises a Caught: [Errno 35] Resource temporarily unavailable exception Caught: [Errno 35] Resource temporarily unavailable . I think I think. A wrapper of a higher level file object tells me that the write call failed.
Case 2
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) try: sys.stdout.write("A" * 65537) except Exception as e: print "Caught: {}".format(e) # Safety sleep to prevent quick exit time.sleep(1)
This sometimes raises a Caught: [Errno 35] Resource temporarily unavailable exception Caught: [Errno 35] Resource temporarily unavailable , or sometimes an exception is thrown, and I see the following output:
close failed in file object destructor: sys.excepthook is missing lost sys.stderr
Case 3
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) try: sys.stdout.write("A" * 65537) except Exception as e: print "Caught: {}".format(e) # Safety sleep to prevent quick exit time.sleep(1) print "Slept"
This sometimes throws a Caught: [Errno 35] Resource temporarily unavailable exception Caught: [Errno 35] Resource temporarily unavailable , or sometimes an exception is thrown, and I see "Slept". It seems that print ing "Slept" I am not getting the error message from Case 2.
Case 4
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) try: os.write(sys.stdout.fileno(), "A" * 65537) except Exception as e: print "Caught: {}".format(e) # Safety sleep to prevent quick exit time.sleep(1)
Always good!
Case 5
fcntl.fcntl(sys.stdout.fileno(), fcntl.F_SETFL, os.O_NONBLOCK) try: print os.write(sys.stdout.fileno(), "A" * 65537) except Exception as e: print "Caught: {}".format(e) # Safety sleep to prevent quick exit time.sleep(1)
This is sometimes good or sometimes it prints the error message close failed in file object destructor .
My question is: why is this happening in python? Am I doing something fundamentally bad here - either with python or at the system level?
It seems that too soon to write too often when the recording has failed, it causes an error message. Error is no exception. I don’t know where he is from.
NB I can write an equivalent program in C and it works fine:
#include <stdio.h> #include <stdlib.h> #include <memory.h> #include <sys/fcntl.h> #include <unistd.h> int main(int argc, const char * argv[]) { const size_t NUM_CHARS = 65537; char buf[NUM_CHARS]; // Set stdout non-blocking fcntl(fileno(stdout), F_SETFL, O_NONBLOCK); // Try to write a large amount of data memset(buf, 65, NUM_CHARS); size_t written = fwrite(buf, 1, NUM_CHARS, stdout); // Wait briefly to give stdout a chance to be read from usleep(1000); // This will be written correctly sprintf(buf, "\nI wrote %zd bytes\n", written); fwrite(buf, 1, strlen(buf), stdout); return 0; }