How to bundle several different InputStreams into one InputStream

Question

I am wondering if there is any ideomatic way to associate multiple InputStreams with one continuous InputStream in Java (or Scala).

To do this, I need to parse the flat files that I upload over the network from an FTP server. I want to make a file [1..N], open the streams, and then combine them into one stream. Therefore, when file1 comes to an end, I want to start reading from file2, and so on, until I get to the end of fileN.

I need to read these files in a certain order, the data comes from an inherited system that creates files in bars, so the data in one depends on the data in the other file, but I would like to process them as one continuous stream to simplify my logical domain interface.

I searched and found PipedInputStream, but I'm not sure if this is what I need. An example will be helpful.

Answer 1

This is right in the JDK! Quoting JavaDoc SequenceInputStream :

A SequenceInputStream represents the logical concatenation of other input streams. It begins with an ordered set of input streams and is read from the first until the end of the file is reached, after which it will be read from the second and so on until the end of the file is reached in the last of the input streams.

You want to combine an arbitrary number of InputStream , and SequenceInputStream accepts only two. But since SequenceInputStream also an InputStream , you can apply it recursively (nest them):

 new SequenceInputStream( new SequenceInputStream( new SequenceInputStream(file1, file2), file3 ), file4 ); 

... you get the idea.

see also

• How to combine two input streams in Java? (dup?)

Answer 2

This is done using SequencedInputStream, which is directly in Java, as Tomasz Nurkevich’s answer shows. I had to do this several times in the project recently, so I added a bit of Scala -y kindness through the "pimp my library" template.

 object StreamUtils { implicit def toRichInputStream(str: InputStream) = new RichInputStream(str) class RichInputStream(str: InputStream) { // a bunch of other handy Stream functionality, deleted def ++(str2: InputStream): InputStream = new SequenceInputStream(str, str2) } } 

With this, I can execute a sequence of sequence as follows

 val mergedStream = stream1++stream2++stream3 

or even

 val streamList = //some arbitrary-length list of streams, non-empty val mergedStream = streamList.reduceLeft(_++_) 

Answer 3

Another solution: first create a list of input stream, and then create a sequence of input streams:

 List<InputStream> iss = Files.list(Paths.get("/your/path")) .filter(Files::isRegularFile) .map(f -> { try { return new FileInputStream(f.toString()); } catch (Exception e) { throw new RuntimeException(e); } }).collect(Collectors.toList()); new SequenceInputStream(Collections.enumeration(iss))) 

Answer 4

Here is a more elegant solution using Vector, it is specifically for Android, but use a vector for any Java

  AssetManager am = getAssets(); Vector v = new Vector(Constant.PAGES); for (int i = 0; i < Constant.PAGES; i++) { String fileName = "file" + i + ".txt"; InputStream is = am.open(fileName); v.add(is); } Enumeration e = v.elements(); SequenceInputStream sis = new SequenceInputStream(e); InputStreamReader isr = new InputStreamReader(sis); Scanner scanner = new Scanner(isr); // or use bufferedReader