Multiprocessor pool initializer not working

I am trying to use multiprocessing.Poolto implement a multi-threaded application. To share some variables, I use Queueas outlined here :

def get_prediction(data):
    #here the real calculation will be performed
    ....


def mainFunction():
    def get_prediction_init(q):
        print("a")
        get_prediction.q = q

    queue = Queue()
    pool = Pool(processes=16, initializer=get_prediction_init, initargs=[queue,])

if __name__== '__main__':
    mainFunction()

This code works fine on a Debian machine, but doesn’t work at all on another Windows 10 device. Error with error

AttributeError: Can't pickle local object 'mainFunction.<locals>.get_prediction_init'

I really don't know what exactly is causing the error. How can I solve the problem so that I can run the code on a Windows device?

EDIT: The problem is resolved if I create the function get_predediction_initat the same level as mainFunction. It only failed when I defined it as an internal function. Sorry for the confusion in my post.

+4
1

, . , , "mainFunction" AttributeError, .

, , , . Windows 10 , Python 3.6.1 ( , Python 3 print), "a" 16 :

import multiprocessing as mp

def get_prediction(data):
    #here the real calculation will be performed
    pass

def get_prediction_init(q):
    print("a")
    get_prediction.q = q

if __name__ == "__main__":
    queue = mp.Queue()
    pool = mp.Pool(processes=16, initializer=get_prediction_init, initargs=[queue,])
    pool.close()
    pool.join()

, , :

import multiprocessing as mp

def get_prediction(data):
    #here the real calculation will be performed
    pass

def get_prediction_init(q):
    print("a")
    get_prediction.q = q

def mainFunction():
    queue = mp.Queue()
    pool = mp.Pool(processes=16, initializer=get_prediction_init, initargs=[queue,])
    pool.close()
    pool.join()

if __name__ == "__main__":
    mainFunction()

2

get_prediction_init() mainFunction. : -)

, . . , - , ; -)

+3

Source: https://habr.com/ru/post/1674992/


All Articles