EMR Spark - TransportClient: Failed to send RPC

Question

I get this error, I tried to increase the memory on the cluster instances, as well as in the executor and driver parameters without success.

17/05/07 23:17:07 ERROR TransportClient: Failed to send RPC 6465703946954088562 to ip-172-30-12-164.eu-central-1.compute.internal/172.30.12.164:34706: java.nio.channels.ClosedChannelException

Does anyone have the opportunity to fix this error?

BTW I use YARN as a cluster manager

Thanks in advance

Answer 1

Finally I solved the problem. This was due to lack of disk space. One column of Hadoop logs showed:

Hadoop YARN: 1/1 local directories are bad: / var / lib / hadoop-yarn / cache / yarn / nm-local-dir; 1/1 log directories are bad: / var / log / hadoop-yarn / container

http://gethue.com/hadoop-yarn-11-local-dirs-are-bad-varlibhadoop-yarncacheyarnnm-local-dir-11-log-dirs-are-bad-varloghadoop-yarncontainers/

" , !"

, EMR.

http://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-web-interfaces.html

ec2,

:

: 172.30.12.84:8088 : 172.30.12.21:8042

, , etl.py :

: 3.2
: c3.4xlarge