Preventing rewriting of modules in parallelizing Julia

I wrote a Julia module with various functions that I call to analyze data. Some of these functions depend on the packages that are included at the beginning of the "NeuroTools.jl."

module NeuroTools

using MAT, PyPlot, PyCall;

function getHists(channels::Array{Int8,2}...

Many of the functions that I have are useful for running in parallel, so I wrote a script driver to map functions to different threads using remotecall / fetch. To load functions in each thread, I run Julia with the -L option to load my module on every worker.

julia -p 16 -L NeuroTools.jl parallelize.jl

To load loaded functions into scope, the "parallelize.jl" script has a line

@everywhere using NeuroTools

My parallel function works and runs correctly, but every worker thread spills out a bunch of warnings from rewritable modules.

WARNING: replacing module MAT
WARNING: Method definition read(Union{HDF5.HDF5Dataset, HDF5.HDF5Datatype, HDF5.HDF5Group}, Type{Bool}) in module MAT_HDF5...
(contniues for many lines)

- , ? .

+4
2

(rd,wr) = redirect_stdout()

,

remotecall_fetch(worker_id, redirect_stdout)

,

,

out = STDOUT
(a,b) = redirect_stdout()
#then to turn it back on, do:
redirect_stdout(out)
+4

, @everywhere using ... , . GitHub .

Julia, , using NeuroTools NeuroTools.jl @everywhere using NeuroTools. Parallel Computing Julia 0.5 :

using DummyModule ; , .

@everywhere using NeuroTools , , replacing module.

0

Source: https://habr.com/ru/post/1650965/


All Articles