Spark Streaming does not read files from a local directory in Windows

public class StreamingWordCount implements Serializable { 

public static void main(String[] args) {    

    JavaStreamingContext jssc = new JavaStreamingContext("local[2]", "JavaWordCount",
            new Duration(1000));
    JavaDStream<String> data = jssc.textFileStream("D:/krishna/").cache();
    data.foreach(new Function<JavaRDD<String>, Void>() {

        public Void call(JavaRDD<String> rdd) throws Exception {
            List<String> output = rdd.collect();
            System.out.println("Sentences Collected from files " + output);
            return null;
        }
    });

    data.print();
    jssc.start();
    jssc.awaitTermination();
  }
}

I use Spark standalone in Windows 8.

Does JavaStreamingContext support only HDFS directories?

Click to display: This is displayed on the console, Sentence, assembled from files, is empty. I tried changing the directory files. But still, the code does not collect files. This is my first code in Spark Streaming. Please help.

+4
source share
1 answer

5 10 , - , , jssc.awaitTermination() .

0

Source: https://habr.com/ru/post/1614868/


All Articles