All geek questions in one place

Invalid jar when starting Hadoop

I want to run the WordCount Example.

In eclipse, it works correctly. There is an output file in the output folder.

I made a jar WordCount file and want to run it with the command

hadoop jar WordCount.jar /Projects/input /Projects/output

it gives me an error

 Not a valid JAR: /Projects/WordCount.jar

result hdfs dfs -ls /Projects

 Found 3 items -rw-r--r-- 1 hduser supergroup 3418 2014-11-02 15:38 /Projects/WordCount.jar drwxr-xr-x - hduser supergroup 0 2014-11-02 14:13 /Projects/input drwxr-xr-x - hduser supergroup 0 2014-11-02 14:16 /Projects/output

he also gives me the same error

  hadoop jar /Projects/WordCount.jar wordPackage.WordCount /Projects/input /Projects/output Not a valid JAR: /Projects/WordCount.jar

how to solve this error.

I ran the tvf command, it gives this output

  jar -tvf /home/hduser/Desktop/Files/WordCount.jar 60 Sun Nov 02 16:10:10 PKT 2014 META-INF/MANIFEST.MF 1895 Sun Nov 02 14:02:38 PKT 2014 wordPackage/WordCount.class 1295 Sun Nov 02 14:02:38 PKT 2014 wordPackage/WordCount.java 2388 Sun Nov 02 14:02:06 PKT 2014 wordPackage/WordReducer.class 707 Sun Nov 02 14:02:06 PKT 2014 wordPackage/WordReducer.java 2203 Sun Nov 02 14:02:08 PKT 2014 wordPackage/WordMapper.class 713 Sun Nov 02 14:02:06 PKT 2014 wordPackage/WordMapper.java 16424 Sun Nov 02 13:50:00 PKT 2014 .classpath 420 Sun Nov 02 13:50:00 PKT 2014 .project
+7
source share
3 answers

You cannot keep the jar in HDFS when executing the same hadoop command, the jar must be available in the local path

If jar does not start, try the following (specify package.mainclass)

 hadoop jar /home/hduser/Desktop/Files/WordCount.jar wordPackage.WordCount /Projects/input /Projects/output

If a bit can be started, the next one can be used.

 hadoop jar /home/hduser/Desktop/Files/WordCount.jar /Projects/input /Projects/output

If the problem still persists, you need to restore this jar (WordCount.jar) in eclipse again

+8
source

despite reassembling the jar, if the problem still persists, make sure you give + x privileges (running chmod 755) before running the command in my case this was the cause of the problem. Command help : chmod + x jarname.jar

0
source

I ran into the same problem. But in my case, I wrote java code referencing the hadoop 1.x libraries and tried to execute it using 2.x. Initially, I got the same error in the terminal. Then I tried to go to the path where I had the jar file. It worked.

Maybe you can try the following: (after going to the jar file path)

 hadoop jar WordCount.jar wordPackage.WordCount /Projects/input /Projects/output
-1
source

Source: https://habr.com/ru/post/977606/

More articles:


All Articles