Multiprocessing: execution order

Question

I am trying to execute the following code:

from multiprocessing import Pool

def f(x):    
    return x

if __name__ == '__main__':
    p = Pool(5)
    print(p.map(f, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13]))

As I understand it, 5 processors get 0, 1, 2, 3, 4to perform operations. If processor 1 shuts down, does it receive 5immediately while the rest of the processors are busy 1,2,3,4, or does the code end for all processors, so the next batch will receive together 5, 6, 7, 8, 9and so on. If later happens, how to implement the above code so that when the processors are idle, it will be assigned a new task?

How to check implementation?

Answer 1

Threadpool ( ). , 4 , 12- .

PS , 10.

from multiprocessing import Pool
import time
import random

def f(x):
    print "Enter %s" % x
    time.sleep( random.randrange(1,100,1)/10.0 )
    print "Exit %s" % x
    return x

if __name__ == '__main__':
    p = Pool(5)
    print(p.map(f, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13]))

Enter 0
Enter 1
Enter 2
Enter 3
Enter 4
Exit 0
Enter 5
Exit 3
Enter 6
Exit 2
Enter 7
Exit 5
Enter 8
Exit 1
Enter 9
Exit 6
Enter 11
Exit 11
Enter 12
Exit 4
Enter 13
Exit 7
Exit 12
Exit 9
Exit 8
Exit 13
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 12, 13]

Answer 2

, . . , ( ), , .

:

(func, iterable [, chunksize])

, . () , chunksize .

, f(x), x .

from multiprocessing import Pool
import time
import threading

def f(x):
    print('x: ' + str(x) + '\tThread ID: ' + str(threading.get_ident()))
    time.sleep(x)

if __name__ == '__main__':
    chunksize = 3
    with Pool(2) as p:
        p.map(f, [10, 1, 1, 1, 1, 1], chunksize)

[10, 1, 1, 1, 1, 1] len(arr) / chunksize = 2:

[10, 1, 1]  # For thread 1, takes 12 seconds to finish
[ 1, 1, 1]  # For thread 2, takes 3 seconds to finish

, 2 3 , 1 9 .

:

x: 10   Thread ID: 8556
x: 1    Thread ID: 59180
x: 1    Thread ID: 59180
x: 1    Thread ID: 59180
x: 1    Thread ID: 8556
x: 1    Thread ID: 8556

, chunksize. 1 .