Python single-function multiprocessor

Question

I have a simulation that currently works, but the ETA is about 40 hours - I am trying to speed it up using multiprocessing.

It essentially repeats the three values ​​of one variable (L) and more than 99 values ​​of the second variable (a). Using these values, it essentially performs complex modeling and returns 9 different standard deviations. This way (although I haven't encoded it yet), it is essentially a function that takes two values ​​as inputs (L, a) and returns 9 values.

Here is the gist of the code I have:

STD_1 = []
STD_2 = []
# etc.

for L in range(0,6,2):
    for a in range(1,100):
        ### simulation code ###
        STD_1.append(value_1)
        STD_2.append(value_2)
        # etc.

Here is what I can change:

master_list = []

def simulate(a,L):
    ### simulation code ###
    return (a,L,STD_1, STD_2 etc.)

for L in range(0,6,2):
    for a in range(1,100): 
        master_list.append(simulate(a,L))

Since each of the simulations is independent, it seems like an ideal place to implement a kind of multithreading / processing.

How exactly will I encode this?

EDIT: , , ?

EDIT 2: , . , .

import multiprocessing

data = []

for L in range(0,6,2):
    for a in range(1,100):
        data.append((L,a))

print (data)

def simulation(arg):
    # unpack the tuple
    a = arg[1]
    L = arg[0]
    STD_1 = a**2
    STD_2 = a**3
    STD_3 = a**4
    # simulation code #
    return((STD_1,STD_2,STD_3))

print("1")

p = multiprocessing.Pool()

print ("2")

results = p.map(simulation, data)

3: . , OS X. ?

Answer 1

• .
• data
• f
• p = multiprocessing.Pool().
• results = p.map(f, data)

f, .

Edit1: :

from multiprocessing import Pool

data = [('bla', 1, 3, 7), ('spam', 12, 4, 8), ('eggs', 17, 1, 3)]

def f(t):
    name, a, b, c = t
    return (name, a + b + c)

p = Pool()
results = p.map(f, data)
print results

Edit2:

UNIX- , OSX. , os.fork ( , MS Windows). . . .

Answer 2

:

import threading

L_a = []

for L in range(0,6,2):
    for a in range(1,100):
        L_a.append((L,a))
        # Add the rest of your objects here

def RunParallelThreads():
    # Create an index list
    indexes = range(0,len(L_a))
    # Create the output list
    output = [None for i in indexes]
    # Create all the parallel threads
    threads = [threading.Thread(target=simulate,args=(output,i)) for i in indexes]
    # Start all the parallel threads
    for thread in threads: thread.start()
    # Wait for all the parallel threads to complete
    for thread in threads: thread.join()
    # Return the output list
    return output

def simulate(list,index):
    (L,a) = L_a[index]
    list[index] = (a,L) # Add the rest of your objects here

master_list = RunParallelThreads()