I have two functions func1()
and func2()
that are independent of each other and can be executed in two threads. I use the library threading
in python3
to run these two threads. But inside, func1()
I also work in parallel for
using joblib
. So using threading as well as joblib gives the following warning -
/usr/local/lib/python3.6/dist-packages/joblib/parallel.py:547: UserWarning: Multiprocessing-backed parallel loops cannot be nested below threads, setting n_jobs=1
,
and then the for loops inside func1()
just execute sequentially.
The following is an example code snippet:
from joblib import Parallel, delayed
import threading
from math import sqrt
class MyThread(threading.Thread):
def __init__(self, sample, type):
threading.Thread.__init__(self)
self.type = type
self.sample = sample
def run(self):
if self.type=='func1':
self.sample.func1()
else:
self.sample.func2()
class Sample:
def func1(self):
print('this function runs for loop in parallel.')
ans = Parallel(n_jobs=8)(delayed(sqrt)(i**2) for i in range(1000))
def func2(self):
print('this function runs sequentially.')
def try(self):
thread1 = MyThread(self, 'func1')
thread2 = MyThread(self, 'func2')
thread1.start()
thread2.start()
thread1.join()
thread2.join()
print('Done...')
if __name__ == '__main__':
s = Sample()
s.try()
How can I solve this problem? Or if this is a joblib problem, is there another library that can be used instead of joblib?