Storm spout - How to read all lines from a text file only once using> 1 stream?

Question

The storm topology contains the Spout component, which is launched using> 1 thread. eg.

 builder.setSpout("lines", new TestLineSpout(), 2);

The Spout function (an open function) opens and reads all lines of a text file and nextTuple emits each line to a bolt.

When 2 threads are executed, for the nose each line of the file is read twice.

I'm new to the assault, and I wonder how best to handle this? I could reduce the number of threads to 1 or change the nose so that each thread reads different lines - or how (how) do I need to use the TopologyContext parameter? I'm not sure I missed the “stormy” way of tricking this?

Answer 1

,

, , . , , , , Storm.

:

• Storm , . , -, , Storm, . - , Spark, .

• , (VM) , node () . " ". , , , . Storm, Kafka 0mq.

• , , - HDFS. , , , , , , . , , , .

, : D

Answer 2

:

mq kafka java-, . .