Submission of Scala code to a cluster

Question

Is it possible to run some akka code in an oracle grid engine using multiple nodes?

So, if I use the actor model, which is the "messaging model", can I use Scala and akka framework to run my code in a distributed memory system, for example, in a cluster or grid?

If so, is there something similar, such as mpirun in mpi -c , to run my program on different nodes? Can you provide an example feed using the oracle kernel engine?

How do I know inside Scala on which node am I and how many nodes the job was sent to?

Is it possible to communicate with other nodes through the actor model?

Answer 1

mpirun or ( mpiexec on some systems) can run any executable files (even if they do not use MPI). I am currently using it to run java and scala codes on clusters. It can be difficult to pass arguments to the executable when mpirun called, so you can use an intermediate script.

We use Torque / Maui scripts that are not compatible with GridEngine, but here is a script my colleague currently uses:

 #!/bin/bash #PBS -l walltime=24:00:00 #PBS -l nodes=10:ppn=1 #PBS -l pmem=45gb #PBS -q spc # Find the list of nodes in the cluster id=$PBS_JOBID nodes_fn="${id}.nodes" # Config file config_fn="human_stability_article.conf" # Java command to call java_cmd="java -Xmx10g -cp akka/:EvoProteo-assembly-0.0.2.jar ch.unige.distrib.BuildTree ${nodes_fn} ${config_fn} ${id}" # Create a small script to pass properly the parameters aktor_fn="./${id}_aktor.sh" echo -e "${java_cmd}" >> $aktor_fn # Copy the machine file to the proper location rm -f $nodes_fn cp $PBS_NODEFILE $nodes_fn # Launch the script on 10 notes mpirun -np 10 sh $aktor_fn > "${id}_human_stability_out.txt"