Poor performance with large write operations on O_NONBLOCK fifo on mac os

I have a reader and writer on FIFO, where the reader should not be blocked indefinitely. To do this, I open the end for reading with O_NONBLOCK.

The recording content may be blocked, so I open it as a regular file. Large writes are unacceptably awful - reading / writing a 4 MB block takes minutes instead of the expected fraction of a second (expected because the same code takes a split second on Linux).

Sample code in Python replicating a problem. First create fifo using mkfifofor example. mkfifo some_fifo, then start the end of the read, then the end of the write.

End of reading:

import os, time
# mkfifo some_fifo before starting python
fd = os.open('some_fifo',os.O_RDONLY | os.O_NONBLOCK)
while True:
    try:
        read = len(os.read(fd, 8192)) # read up to 8kb (FIFO buffer size in mac os)
        print(read)
        should_block = read < 8192 # linux
    except BlockingIOError:
        should_block = True # mac os
    if should_block:
        print('blocking')
        time.sleep(0.5)

Recording ends:

import os
fd = os.open('some_fifo',os.O_WRONLY)
os.write(fd, b'aaaa'*1024*1024) # 4MB write

. , , - - Java-, Linux. , , kqueue kevent data, , - epoll/kqueue, . , fd à la this answer .

: kqueue ,

2: Linux os.read() BlockingIOError , , , , , ( ( 0), errno EAGAIN). , Linux.

3: macOS:

import select, os
# mkfifo some_fifo before starting python
fd = os.open('some_fifo',os.O_RDONLY | os.O_NONBLOCK)
kq = select.kqueue()
ke = select.kevent(fd)
while True:
    try:
        read = len(os.read(fd, 8192)) # read up to 8kb (FIFO buffer size in mac os)
    except BlockingIOError:
        evts = kq.control([ke], 1, 10) # 10-second timeout, wait for 1 event
        print(evts)

, , , -.

+4

Source: https://habr.com/ru/post/1693628/


All Articles