Spark - Akka Disassociated Remote Client

Question

I configure Spark 0.9 on AWS and find that when I launch the Pyspark interactive shell, my artists / remote workers first register:

14/07/08 22:48:05 INFO cluster.SparkDeploySchedulerBackend: Registered executor: Actor[akka.tcp:// [e-mail] :54110/user/ Executor#-862786598] with ID 0 

and then disassociated almost immediately before I have a chance to run something:

 14/07/08 22:48:05 INFO cluster.SparkDeploySchedulerBackend: Executor 0 disconnected, so removing it 14/07/08 22:48:05 ERROR scheduler.TaskSchedulerImpl: Lost an executor 0 (already removed): remote Akka client disassociated 

Any idea what could be wrong? I tried setting the JVM parameters spark.akka.frameSize and spark.akka.timeout, but I am sure that this is not a problem, because (1) I do not start anything to start, and (2) my artists disconnect a few seconds after the start. which corresponds to a timeout of 100 seconds by default.

Thanks!

Jack

Answer 1

I had a very similar problem, if not the same. He began to work for me as soon as the workers connected with the master, using the same name as the owner whom he knew.

My log messages were something like this:

ERROR remote.EndpointWriter: AssociationError [akka.tcp:// [e-mail] :7078] -> [akka.tcp:// [e-mail] :7077]: Error [Association failed with [akka.tcp:// [e-mail] :7077]]. ERROR remote.EndpointWriter: AssociationError [akka.tcp:// sparkWorker@192.168.121.127 :7078] -> [akka.tcp:// [e-mail] :7077]: Error [Association failed with [akka.tcp:// [e-mail] :7077]]

WARN util.Utils: Your hostname, idc1-hrm1 resolves to a loopback address: 127.0.0.1; using 192.168.121.187 instead (on interface eth0)

So, check the wizard’s log and see what name it counts. Then use the same name for the workers.