Anything behind 'if __name__ == "__main__":' is not running

Question

So here is my situation.

I use PyDev in Eclipse, the Python interpreter version 2.7.2 on Windows.

I use the built-in multiprocessing library, trying to unlock a bunch of processes in order to parallelize a cycle with very intensive computation. The tutorials that I looked at can use,

if __name__ == "__main__": 

to prevent the occurrence of almost endless processes and essentially put my system on its knees.

The problem is that I am calling this from a module, not from my main script; as such, nothing after this will NEVER be done. There is no chance for parallelism at all. Of course, if I delete it, I get spam infiniprocess, which kills the machine that runs the code.

For reference, the corresponding code is here:

 from tribe import DataCache from tribe import WorldThread from tribe import Actor from time import sleep import multiprocessing class World: def __init__(self,numThreads,numActors,tickRate): print "Initalizing world..." self.cache = DataCache.DataCache() self.numThreads = numThreads self.numActors = numActors self.tickRate = tickRate self.actors = [] self.processes = [] for i in range(numActors): self.actors.append(Actor.Actor("test.xml",self.cache)) print "Actors loaded." def start_world(self): print "Starting world" run_world = True; while run_world: self.world_tick() sleep(2) def world_tick(self): if __name__ == '__main__': print "World tick" actor_chunk = len(self.actors)/self.numThreads if len(self.processes)==0: for _ in range(self.numThreads): new_process = multiprocessing.Process(WorldThread.WorldProcess.work, args=(_, self.actors[_*actor_chunk,(_+1)*actor_chunk])) 

And the class that it calls:

 class WorldProcess(): def __init__(self): print "World process initilized." ''' Really, I'm not sure what kind of setup we'll be doing here yet. ''' def work(self, process_number, actors): print "World process" + str(process_number) + " running." for actor in actors: actor.tick() print "World process" + str(process_number) + " completed." 

I proceed in my assessment that the whole is if name == "main": does the check only work if it is in the executable script itself? If so, how do you safely refuse processes from modules? If not, why doesn't he work here?

Answer 1

To control the number of processes, use the Pool class from multiprocessing :

 from multiprocessing import Pool p = Pool(5) def f(x): return x*x p.map(f, [1,2,3]) 

(Edit: as per the comment, it's just howto for the Pool class. See more)

Using __name__ not required as you explicitly pass Process executable virtual python function .

It:

 def world_tick(self): if __name__ == '__main__': print "World tick" actor_chunk = len(self.actors)/self.numThreads if len(self.processes)==0: for _ in range(self.numThreads): new_process = multiprocessing.Process(WorldThread.WorldProcess.work, args=(_, self.actors[_*actor_chunk,(_+1)*actor_chunk])) 

very bad. Simplify this.

The best template would be:

 class WorkArgs(object): ... many attributes follow ... def proc_work(world_thread, work_args): world_thread.WorldProcess.work(work_args.a, work_args.b, ... etc) p = Pool(5) p.map(proc_work, [(world_thread, args0), (world_thread, args1), ...]) 

Hope this helps!

As a side note, etching your arguments and passing them to other processes will import your module. Thus, it is best to make sure that the module does not transform any forking / magic / work function if it is not said (for example, only function / class definitions or magic __name__ , and not the actual code blocks).

Answer 2

Adding this as an answer as it was in the comments:

if __name__ == "__main__" is what you do at the root level of the script, which will be the entry point. Its a way to do only what the script is executing.

If you have a script that is an entry point, you use the name == main. And in the module that you want to multiprocess, you just loop and start your processes in the same way as a loop and start threads.