Multiprocessing.Pool.apply_async on Windows

Question

I am trying to use a pool to process some subprocess calls in parallel. Everything works fine, if I build a whole iterability pool and use the imap, map, imap_unordered, and so on, but I can not get apply_asyncto work.

For example, this works correctly:

from subprocess import check_call
from multiprocessing import Pool

def dispatch_call(file_name):
    return check_call(...)

if __name__ == '__main__':
    files = (constructed file list)
    pool = Pool()
    pool.imap(dispatch_call, files)
    pool.close()
    pool.join()

This, however, does not mean:

from subprocess import check_call
from multiprocessing import Pool

def dispatch_call(file_name):
    return check_call(...)

if __name__ == '__main__':
    files = (constructed file list)
    pool = Pool()
    for f in files:
        pool.apply_async(dispatch_call, f)
    pool.close()
    pool.join()

I checked the docs for multiprocessing, and none of the problems associated with Windows seem relevant. Am I just out of luck trying to get this to work on Windows?

Answer 1

You pass the callback arguments as a sequence (list / tuple).

str object , ( ):

pool.apply_async(dispatch_call, [f])

pool.apply_async(dispatch_call, (f,))