Python: how to change a global variable in a function using pprocess

Question

it seems that I cannot change the global variable in Python when using the function called from pprocess. Here is my example:

import pprocess import time numbers=[0,0,0,0,0,0,0,0,0,0] # find system time and store in global variable def find_time(index): global numbers x=time.time() print "Setting element %s of numbers to %f" % (index, x) numbers[index]=x return x # parallel execution of the function results=pprocess.pmap(find_time, [0,1,2,3,4,5,6,7,8,9], limit=6) for y in results: print '%f' % y # this list is unchanged print numbers # serial execution of the function for x in [0,1,2,3,4,5,6,7,8,9]: find_time(x) # now it seems to work print numbers 

"numbers" is just a list of zeros, and for demonstration, I am trying to set each element of the list to the current system time. When called using pprocess, this will not work, but when I use a simple loop to call a function, the global variable changes.

I spent some time reading global variables and sincerely hope this is not a trivial problem. Can someone explain to me what is going on?

Thank you very much,

Enno

Answer 1

I understand that pprocess uses subprocessing under the hood. If so, then every time the function starts, it is actually a separate process. And therefore, these changes are not displayed when your function returns.

You will probably need to make a multiprocessing.Manager list.

eg.

 numbers = multiprocessing.Manager().list([0]*10) 

Answer 2

pprocess creates another process. This means that it does not exchange memory with the calling code. Everything that changes the parallel process will be changed in its own memory space, so the memory space of the calling code will remain unchanged. That is, they do not share global variables.

You will need to make all of your message between the two explicitly, through channels or any pprocess sentences or sockets, etc.