Cannot start Hadoop remotely

I want to remotely run the hadoop job from a windows machine. The cluster runs on Ubuntu.

Basically, I want to do two things:

  • Remote job execution hadoop.
  • Get the result from the output directory hadoop.

I do not know how to achieve this. I am using version 1.1.2 for adoop

I tried passing the jobtracker / namenode url in Job setup, but it fails.

I tried the following example: Running java hasoop on a local / remote cluster

Result: getting the error sequentially because the directory cannot be loaded. It is similar to this post: Exception when sending mapreduce job from remote system

+4
source share
2 answers

. , Hadoop 2.2 ( ), .

, :

  • Hadoop 2.2 source tarball Linux temp.
  • , Windows Linux.
  • , . , 64- , 64- Linux. , , , . , protobuf 2.5 sudo ldconfig, . .
  • dist tar hadoop-2.2.0-src/hadoop-dist/target node . , .
  • Java Windows. , , . c:\java\jdk1.7.
  • hadoop dist tar, Windows. Windox/Linux.
  • winutils Windows, fooobar.com/questions/73750/.... , Hadoop Windows.
  • JAVA_HOME, HADOOP_HOME PATH,
  • unix2dos ( Cygwin ), .cmd bin etc\hadoop, .
  • XML , fs.default.name, mapreduce.jobtracker.address, yarn.resourcemanager.hostname .
  • , 2. . .

, Linux Hadoop- Windows. !

+3

, , , .

-2

Source: https://habr.com/ru/post/1530181/


All Articles