I will learn how to make my script faster. I think parallel is a good way. So I try gevent and multiprocessing. But I am confused by this other result. Let me show you two examples that I met,
ex 1:
a=np.zeros([3])
def f(i):
a[i]=1
print a
def f_para():
p=multiprocessing.Pool()
p.map(f, range(3))
def f_asy():
threads = [gevent.spawn(f, i) for i in xrange(3)]
gevent.joinall(threads)
f_para()
[ 0. 1. 0.]
[ 0. 0. 1.]
[ 1. 0. 0.]
f_asy()
[ 1. 0. 0.]
[ 1. 1. 0.]
[ 1. 1. 1.]
I found that when using multiprocessing, the global object anever changes in fat, and after starting f_para(), ait is still the original array. During operation, f_asy()he has changed a.
ex 2:
def f2(i):
subprocess.call(['./a.out', str(i)])
time.sleep(0.2)
def f2_loop():
for i in xrange(20):
f2(i)
def f2_para():
p=multiprocessing.Pool()
p.map(f2, range(20))
def f2_asy():
threads = [gevent.spawn(f2, i) for i in xrange(20)]
gevent.joinall(threads)
%timeit -n1 f2_loop()
1 loop, best of 3: 4.22 s per loop
%timeit -n1 f2_asy()
1 loop, best of 3: 4.22 s per loop
%timeit -n1 f2_para()
1 loop, best of 3: 657 ms per loop
I find that f2_asy()does not reduce runtime. And the conclusion is f2_asy()one by one, the same as f2_loop(), therefore, there is no parallel in f2_asy(), I think.
a.out is a simple C ++ code:
#include <iostream>
int main(int argc, char* argv[])
{
std::cout<<argv[1]<<std::endl;
return 0;
}
So my question is:
ex 1, f_para a?
ex 2, f2_asy ?
gevent ? , .