I am new to EMR, and whenever I run a task, I get this error in syslog:
2014-05-27 03:59:57,220 INFO org.apache.hadoop.mapreduce.Job (main): Job job_1401143947501_0009 failed with state FAILED due to: Application application_1401143947501_0009 failed 2 times due to AM Container for appattempt_1401143947501_0009_000002 exited with exitCode: 1 due to: Exception from container-launch:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
at org.apache.hadoop.util.Shell.run(Shell.java:379)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
I have no idea what this means and what I should study. I looked around a bit and there seemed to be a lot of different opinions. Some of them suggested removing yarn.application.classpath from yarn-site.xml and / or changing yarn.nodemanager.resource.memory-mb in the same file. But so far nothing has happened. I feel like shooting in the dark. If someone can tell me what I should look for, then it will be really helpful.
Thanks in advance
source
share