Read large command output using clojure

Question

I use the sh function from the clojure.java.shell command to read a very large output from the command. The output is about 60 million data.

I keep getting java.lang.OutOfMemoryError. Is there a way to open some kind of pipe that will allow me to read the result and analyze it into a vector. How lazy sequence to output the command?

Basically, the data is a large array of bytes, which I want to convert to just numbers and put into a vector.

Answer 1

clojure.java.shell/sh will always return a non-lazy string

( , ), lazy line-seq BufferedReader:

(->> (.exec (Runtime/getRuntime) "YOUR_LONG_RUNNING_COMMAND ARG ...")
    .getInputStream
    clojure.java.io/reader
    line-seq
    (map YOUR-FUNCTION))