Running the Flink Sample Flink Program in Local

Question

I am trying to run a sample program in Apache Flink in local mode.

import org.apache.flink.api.common.functions.FlatMapFunction; import org.apache.flink.api.java.DataSet; import org.apache.flink.api.java.ExecutionEnvironment; import org.apache.flink.api.java.tuple.Tuple2; import org.apache.flink.util.Collector; public class WordCountExample { public static void main(String[] args) throws Exception { final ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); DataSet<String> text = env.fromElements( "Who there?", "I think I hear them. Stand, ho! Who there?"); //DataSet<String> text1 = env.readTextFile(args[0]); DataSet<Tuple2<String, Integer>> wordCounts = text .flatMap(new LineSplitter()) .groupBy(0) .sum(1); wordCounts.print(); env.execute(); env.execute("Word Count Example"); } public static class LineSplitter implements FlatMapFunction<String, Tuple2<String, Integer>> { @Override public void flatMap(String line, Collector<Tuple2<String, Integer>> out) { for (String word : line.split(" ")) { out.collect(new Tuple2<String, Integer>(word, 1)); } } } } 

This gives me an exception:

  Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/InputFormat at WordCountExample.main(WordCountExample.java:10) Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.InputFormat at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 1 more 

What am I doing wrong?

I also used the right cans. Flink-Java-0.9.0-milestone-1.jar FLiNK-clients-0.9.0-milestone-1.jar Flink-core-0.9.0-milestone-1.jar

Answer 1

Adding three Flink Jar files depending on dependencies in your project is not enough, because they have other transitive dependencies, for example, on Hadoop.

The easiest way to get a working setup for developing (and running locally) Flink programs is to follow the quick start guide , which uses the Maven archetype to set up the Maven project. This Maven project can be imported into your IDE.

Answer 2

NoClassDefFoundError extends LinkageError

It is thrown if an instance of the Java virtual machine or an instance of ClassLoader tries to load into the class definition (as part of a regular method call or as part of creating a new instance using a new expression), and no class definition can be found. Class search definition existed while compiling the current executable class, but the definition could no longer be found.

Your code / flask depends on hadoop. Found here download the jar file and add it to your classpath org.apache.hadoop.mapreduce.InputFormat