Applications not shown in the yarn user interface when starting mapreduce hadoop?

Question

I am using Hadoop2.2 . I see that my work has been completed with success. I can scan the file system to find a way out. However, when I look through http://NNode:8088/cluster/apps , I cannot see any applications that have been executed so far (I have completed 3 wordcount tasks, but none of them are covered here).

Are there any configurations that need to be considered?

Here is yarn-site.xml

 <property> <name>yarn.resourcemanager.hostname</name> <value>NNode</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <!-- <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> --> 

Here is mapred-site.xml :

 <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> 

I also have a job history server running:

 jps 4422 NameNode 5452 Jps 4695 SecondaryNameNode 4924 ResourceManager 72802 Jps 5369 JobHistoryServer 

Answer 1

After the applications are completed, their responsibility can be transferred to the Job History Server . So check the Job History Server URL. Usually it listens on port 19888 . For instance.

http://<job_history_server_address>:19888/jobhistory

Log directories and log duration are configured in yarn-site.xml . With YARN you can even aggregate logs in one (custom) location.

Sometimes, although the application is listed, the logs are not available (I'm not sure due to an error in YARN ). However, almost every time I managed to get the logs using the command line:

Yarn Logs -applicationId the_application_id

Although there are several options. Use the help for more information:

log threads --help

Answer 2

you can refer Hadoop does not show my work in the job tracker, even if it works

  conf.set("fs.defaultFS", "hdfs://master:9000"); conf.set("mapreduce.jobtracker.address", "master:54311"); conf.set("mapreduce.framework.name", "yarn"); conf.set("yarn.resourcemanager.address", "master:8032"); 

I tested in my cluster. He works!