Doing multiple parallel jobs inside threads in python3

I have two functions func1()and func2()that are independent of each other and can be executed in two threads. I use the library threadingin python3to run these two threads. But inside, func1()I also work in parallel forusing joblib. So using threading as well as joblib gives the following warning -

/usr/local/lib/python3.6/dist-packages/joblib/parallel.py:547: UserWarning: Multiprocessing-backed parallel loops cannot be nested below threads, setting n_jobs=1,

and then the for loops inside func1()just execute sequentially.

The following is an example code snippet:

from joblib import Parallel, delayed
import threading
from math import sqrt

class MyThread(threading.Thread):
    def __init__(self, sample, type):
        threading.Thread.__init__(self)
        self.type = type
        self.sample = sample
    def run(self):
        if self.type=='func1':
            self.sample.func1()
        else:
            self.sample.func2()

class Sample:
   def func1(self):
       print('this function runs for loop in parallel.')
       ans = Parallel(n_jobs=8)(delayed(sqrt)(i**2) for i in range(1000)) 

   def func2(self):
       print('this function runs sequentially.')

   def try(self):    
       thread1 = MyThread(self, 'func1')
       thread2 = MyThread(self, 'func2')

       thread1.start()
       thread2.start()

       thread1.join()
       thread2.join()

       print('Done...')

if __name__ == '__main__':
    s = Sample()
    s.try()

How can I solve this problem? Or if this is a joblib problem, is there another library that can be used instead of joblib?

+4

Source: https://habr.com/ru/post/1690921/


All Articles