How to track status with multiprocessing and pool.map?

Question

I am creating a multiprocessor module for the first time, and basically, I plan to do something in accordance with

from multiprocessing import pool
pool = Pool(processes=102)
results = pool.map(whateverFunction, myIterable)
print 1

As I understand it, it 1will be printed as soon as all processes return and the results are completed. I would like to receive some status update. What is the best way to implement this?

I hesitate to make a seal whateverFunction(). Especially if there are about 200 values, I'm going to print something like "process done" 200 times, which is not very useful.

I expect the conclusion to be

10% of myIterable done
20% of myIterable done

Answer 1

pool.map , . pool.apply_async . , callback . log_result foo. , foo.

from __future__ import division
import multiprocessing as mp
import time

def foo(x):
    time.sleep(0.1)
    return x

def log_result(retval):
    results.append(retval)
    if len(results) % (len(data)//10) == 0:
        print('{:.0%} done'.format(len(results)/len(data)))

if __name__ == '__main__':
    pool = mp.Pool()
    results = []
    data = range(200)
    for item in data:
        pool.apply_async(foo, args=[item], callback=log_result)
    pool.close()
    pool.join()
    print(results)

10% done
20% done
30% done
40% done
50% done
60% done
70% done
80% done
90% done
100% done
[0, 1, 2, 3, ..., 197, 198, 199]

---

log_result results data. log_result, , pool.apply_async, , foo.

, , , , log_result :

from __future__ import division
import multiprocessing as mp
import time

def foo(x):
    time.sleep(0.1)
    return x

def make_log_result(results, len_data):
    def log_result(retval):
        results.append(retval)
        if len(results) % (len_data//10) == 0:
            print('{:.0%} done'.format(len(results)/len_data))
    return log_result

if __name__ == '__main__':
    pool = mp.Pool()
    results = []
    data = range(200)
    for item in data:
        pool.apply_async(foo, args=[item], callback=make_log_result(results, len(data)))
    pool.close()
    pool.join()
    print(results)