Parallel r with foreach and mclapply at the same time

Question

I am implementing a parallel processing system, which will eventually be deployed in a cluster, but I have problems with developing the interaction of various parallel processing methods.

I need to use a for loop to run a large block of code that contains several large matrix operations. To speed this up, I want to parallelize the for loop with foreach () and do list operations in parallel with mclapply.

pseudo code example:

cl<-makeCluster(2)
registerDoParallel(cl)

outputs <- foreach(k = 1:2, .packages = "various packages") {

    l_output1 <- mclapply(l_input1, function, mc.cores = 2)
    l_output2 <- mclapply(l_input2, function, mc.cores = 2)
    return = mapply(cbind, l_output1, l_output2, SIMPLIFY=FALSE)
}

It seems to work. My questions:

1) - a reasonable approach? They seem to be working together on my small-sized tests, but this is a bit unpleasant.

2) ? , , (foreach 7 , mclapply 70 ). , 6 "", ( 2 foreach 2 mclapply.

Answer 1

, , , mclapply . - ( cbind), .

10 : makeCluster, mclapply (2 + 2 (2 + 2)). . , mclapply, mclapply foreach, .

, node . , .

, mclapply , MPI. MPI , fork, mclapply. , , PSOCK, ssh MPI.

---

Update

, , mclapply , "parallel" "snow". . .