What is the best way in Python to call the same function on separate threads?

Question

What is the best way to call the same function in separate threads and have a separate list with return values ​​for each instance without a duplicate function?

Example:

import threading


def function(a):

    returned_values = []

    ct = threading.currentThread() 
    while getattr(ct, "do_run", True):
        ret = do_something(a)
        returned_values.append(ret)


t1 = threading.Thread(target=function, args=("AAA",))
t2 = threading.Thread(target=function, args=("BBB",))
t3 = threading.Thread(target=function, args=("CCC",))

t1.start()
t2.start()
t3.start()

import time;time.sleep(10)
t1.do_run = t2.do_run = t3.do_run = False

EDIT: Forgot to mention that I'm using Python 2.7

Answer 1

Use ThreadPool

Something like that

from multiprocessing.pool import ThreadPool

pool = ThreadPool()
pool.map(function, list_containing_args)

PS it works similar to multiprocess map.Each argument is given a new thread .You can specify the number of threads you want to spawn if you have limited resources or a big list

from multiprocessing.pool import ThreadPool
import subprocess
def func(ip):
    c=subprocess.Popen("ping -c 3 "+ip, shell=True, stdout=subprocess.PIPE)
    output, error= c.communicate()
    return output

pool = ThreadPool()
for i in  pool.map(func,["127.0.0.1", "www.google.com", "www.facebook.com"]):
    print i

Answer 2

Wouldn't it be better ProcessPool, since threads are best suited for network I / O problems, where they are ProcessPoolbest suited for memory intensive tasks.

from concurrent.futures import ProcessPoolExecutor

with futures.ProcessPoolExecutor(max_workers=n) as executor:
    executor.map(fn, args)

Answer 3

threading, :

• n_thread, args_set = 3, [('AAA',), ('BBB',), ('CCC',)]
  

• threads = [threading.Thread(target=function, args=args_set[i])
             for i in range(n_thread)]
  [t.start() for t in threads]
  

• t1, t2 ..

  for i in range(n_thread):
      var_thread = locals()['t%d' % i]
      var_thread = threading.Thread(target=function, args=args_set[i])
      var_thread.start()
  print t1, t2