OSError: [Errno 12] Cannot allocate memory from python subprocess.call

Question

I read several similar posts on this subject, but no one seems to be helping me directly. If this is actually a duplicate of the message, please direct me to the stream containing the solution!

I save a bunch of images and then call ffmpeg on them using subprocess.call. I do this several times for collections of different images. This is basically what I am doing:

from subprocess import call for video in videos: call(['ffmpeg', ..., '-i', video, video+'.mp4')]) 

In isolation, this works great. However, when I also have some other processing done before these calls (not in a loop, literally just storing the values ​​in memory until the start of the loop), it crashes with a memory error after I made some videos (actually, making the last one) . According to this comment , subprocess.call forks / clones the current process, which apparently means that it is requesting a memory allocation equal to how much I currently have in memory, which seems to be too complicated for what I want to do when calling ffmpeg.

How can I call ffmpeg from within python without asking to allocate unnecessary amounts of memory?

Answer 1

Although it is true that subprocess.call makes the fork process, and this child process has its own memory space, which is initially identical to the parent process (your python program), modern operating systems will use write-to-copy . The overhead of ramified process memory is initially relatively small, and only a few KB of memory is required to account for processes. Only until this child process begins to make changes to its memory does additional memory be required.

After formatting, the child process spawned by subprocess.call will call one of the exec system calls, which loads ffmpeg into memory and starts to execute it.

Furthermore, fork is usually the only mechanism for creating a new process on a POSIX system ( see here ), so I don't think there is an alternative to fork-then-exec that is implemented by the .call subprocess.

You can try running your program through strace or Valgrind to see which system call does not receive the requested memory. This can help you determine how to reduce memory requirements.

Answer 2

I had the same problem today and I just worked on it using os:

 import os for video in videos: job = ' ffmpeg ' + ... + ' -i ' + video + '.mp4' os.system( job )