Where hasoop mapreduce framework sends my System.out.print () instructions? (Standard output)

Question

I want to debug a mapreduce script, and without getting into it, I tried to put some printing instructions into my program. But I can not find them in any of the magazines.

Answer 1

Actually stdout only displays System.out.println() of non-map classes.

System.out.println() for the display and decrease phases can be seen in the logs. Easy way to access magazines -

http: // localhost: 50030 / jobtracker.jsp- > click on the completed task-> click on the map or reduce the task-> click on the task number-> task logs-> stdout.

Hope this helps

Answer 2

Another way is through the terminal:

1) Go to your Hadoop_Installtion directory and then to "logs / userlogs".
2) Open the job_id directory.
3) Check directories with _m_ if you want mapper or _r_ output if you are looking for reducers.

Example : In Hadoop-20.2.0:

 > ls ~ / hadoop-0.20.2 / logs / userlogs / attempt_201209031127_0002_m_000000_0 /
 log.index stderr stdout syslog

The above means:
Hadoop_Installation: ~ / hadoop-0.20.2
job_id: job_201209031127_0002
_m_: task card, "card number": _000000_

4) open stdout if you used "system.out.println" or stderr if you used "system.err.append".

PS. other hadoop versions may have different hierarchies of vision, but they should all be under $ Hadoop_Installtion / logs / userlogs.

Answer 3

In a Hadoop cluster with yarn you can get logs, including stdout, using

 yarn logs -applicationId application_1383601692319_0008 

For some reason, I found this more complete than what I see in the webinterface. The web interface did not display the output of System.out.println() for me.

Answer 4

To get your stdout and log message in the console, you can use the apache commons framework for your mapper and reducer.

 public class MyMapper extends Mapper<..,...,..,...> { public static final Log log = LogFactory.getLog(MyMapper.class) public void map() throws Exception{ // Log to stdout file System.out.println("Map key "+ key); //log to the syslog file log.info("Map key "+ key); if(log.isDebugEanbled()){ log.debug("Map key "+ key); } context.write(key,value); } }