The usual way on a Posix system would be to use the preexec_fn parameter in subprocess.Popen to call the function before running the command (see preexec_fn in this answer for details). Unfortunately, this must happen between the system calls fork and exec and Windows does not create processes this way.
On Windows, the system call (WinAPI) used to create CreateProcess is CreateProcess . The MSDN page says:
BOOL WINAPI CreateProcess( ... _In_ DWORD dwCreationFlags, ... );
dwCreationFlags [in]
Flags controlling the priority class and process creation ... This parameter also controls the new process priority class, which is used to prioritize scheduling process flows.
Unfortunately, the Python interface does not provide for setting a child priority, because it is explicitly said:
The creation flag, if specified, can be CREATE_NEW_CONSOLE or REATE_NEW_PROCESS_GROUP. (Windows only)
But the documentation for dwCreationFlags on MSDN also states:
... If none of the flags of the priority class is specified, the default priority class is NORMAL_PRIORITY_CLASS, if the priority class of the creation process is not IDLE_PRIORITY_CLASS or BELOW_NORMAL_PRIORITY_CLASS . In this case, the child process receives the default priority class of the calling process .
This means that the priority can simply be inherited, since the way to control the child priority in Python for Windows is to set the priority before starting the subprocess and reset it immediately after:
def run(command):
The remainder of this answer will relate to a Posix system such as Linux or Unix.
The preexec_fn parameter in Popen is what you need. This allows you to call the called object (for example, a function) between the creation of the child process and the execution of the command. You could do:
def set_low_pri(): psutil.Process().nice(psutil.BELOW_NORMAL_PRIORITY_CLASS)
then use it to have a low priority baby:
def run(command):
In this way, Python ensures that low priority is set before executing your command.
Link: The documentation for the subprocess module reads:
17.5.1.2. Popen constructor
...
class subprocess.Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, stderr=None, preexec_fn=None, close_fds=True, shell=False, cwd=None, env=None, universal_newlines=False, startupinfo=None, creationflags=0, restore_signals=True, start_new_session=False, pass_fds=(), *, encoding=None, errors=None)
...
If preexec_fn is set for the called object, this object will be called in the child process immediately before the execution of the child process. (POSIX only)
But the above method is not thread safe! If two threads work simultaneously, we can get into the following race condition:
- thread A lowers priority
- thread A starts its child (low priority)
- thread B lowers priority (no operation)
- thread A resets normal priority
- thread B starts its descendant with normal priority
- thread B resets normal priority (no operation)
The problem is that multiprocessing.dummy is a wrapper for threading . The Python standard library documentation (3.6) is written in 17.2.2.13. Multiprocessing.dummy module
multiprocessing.dummy copies the multiprocessing API, but is nothing more than a wrapper for a thread module.
Once the problem is discovered, the fix becomes trivial: just use Lock to protect the critical section:
lock = mp.Lock() def run(command): try : print(command[0]) lock.acquire() psutil.Process().nice(psutil.BELOW_NORMAL_PRIORITY_CLASS)