All geek questions in one place

What does the exitCode spark mean: 12 means?

I am trying to run a spark application written in scala 11.8, spark 2.1 on an EMR cluster version 5.3.0. I configured a cluster with the following json:

[ { "Classification": "hadoop-env", "Configurations": [ { "Classification": "export", "Configurations": [], "Properties": { "JAVA_HOME": "/usr/lib/jvm/java-1.8.0" } } ], "Properties": {} }, { "Classification": "spark-env", "Configurations": [ { "Classification": "export", "Configurations": [], "Properties": { "JAVA_HOME": "/usr/lib/jvm/java-1.8.0" } } ], "Properties": {} } ]

If I try to start client mode, everything works fine. when trying to run the application in cluster mode, it failed with status code 12.

Here is the part of the main log where I see the status code:

17/02/01 10:08:26 INFO TaskSetManager: Completed task 79.0 in stage 0.0 (TID 79) in 293 ms on ip-10-234-174-231.us-west-2.compute.internal (executor 2) (78/11102) 02/17/01 10:08:27 INFO YarnAllocator: the driver requested from 19290 performers. 02/17/01 10:08:27 INFO ApplicationMaster: Final application status: FAILED, exitCode: 12, (reason: An exception was thrown 1 time from the Reporter thread). 02/17/01 10:08:27 INFO SparkContext: calling stop () from shutdown

UPDATE:

As part of the assignment, I need to read some data from s3, something like this: sc.textFile( "s3n://stambucket/impressions/*/2017-01-0[1-9]/*/impression_recdate*) If I I’ll take only one day, there will be no errors, but from 9 I get this exit code 12. This is even strange, given the fact that 9 days in client mode are just wonderful.

+5
source share
2 answers

Exit code 12 is the standard exit code in Linux to get out of memory.

Spark sets the default memory size for each executing process to 1gb. EMR will not override this value regardless of the amount of memory available on the cluster / master nodes. One possible fix is ​​to set the maximizeResourceAllocation flag to true.

+2
source

Try increasing ApplicationMaster Java heap spark.yarn.am.memory = 2G or set maxExecutors to a reasonable spark.dynamicAllocation.maxExecutors = 400

https://issues.apache.org/jira/browse/SPARK-19226

+1
source

Source: https://habr.com/ru/post/1263673/

More articles:


All Articles