Resource of temporarily unavailable error with subprocess module in Python

Question

In Python, I run a process gnuplotto create gifs from a dataset.

from subprocess import Popen, PIPE
def gnuplotter(...)
    p = Popen([GNUPLOT], shell=False, stdin=PIPE, stdout=PIPE)
    p.stdin.write(r'set terminal gif;')
    ...
    p.stdin.write(contents)
    p.stdout.close()

It works fine when I use it gnuplotter()once, but when I run the process several times, I got an error Resource temporarily unavailable.

for i in range(54):
    gnuplotter(i, ... 

  File "/Users/smcho/code/PycharmProjects/contextAggregator/aggregation_analyzer/aggregation_analyzer/gnuplotter.py", line 48, in gnuplotter
    p = Popen([GNUPLOT], shell=False, stdin=PIPE, stdout=PIPE)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 711, in __init__
    errread, errwrite)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1205, in _execute_child
    self.pid = os.fork()
OSError: [Errno 35] Resource temporarily unavailable

What happened, and how can I close the gnuplot process before another erupts?

Answer 1

pid numbers, open file descriptors, memory limited by resources.

fork (2) manual says when errno.EAGAINshould happen:

[EAGAIN] The system-imposed limit on the total number of processes under
          execution would be exceeded.  This limit is configuration-dependent.

[EAGAIN]  The system-imposed limit MAXUPRC () on the total number of processes
          under execution by a single user would be exceeded.

, :

import resource

resource.setrlimit(resource.RLIMIT_NPROC, (20, 20))

, , p.stdin.close(), gnuplot stdin , .. gnuplot . / ( Python 2.7), .

, , .communicate():

from subprocess import Popen, PIPE, STDOUT

p = Popen("gnuplot", stdin=PIPE, stdout=PIPE, stderr=PIPE,
          close_fds=True, # to avoid running out of file descriptors
          bufsize=-1, # fully buffered (use zero (default) if no p.communicate())
          universal_newlines=True) # translate newlines, encode/decode text
out, err = p.communicate("\n".join(['set terminal gif;', contents]))

.communicate() (, ), p.stdin, p.stdout, p.stderr( , gnuplot , EOF flushes ) ( ).

Popen _cleanup() , , .. p.wait() (, ).

Answer 2

p.wait(), , , .

( N ), p.poll() , .

, p.communicate(), . . .