Unknown-sized AbstractSpliterator implementation throws OutOfMemoryError: Java heap space

Question

I have java.lang.OutOfMemoryError: Java heap space when I use an AbstractSpliterator implementation that reports an unknown size.

In this case, I defined a StreamCollapse class that extends AbstractSpliterator and combines a number of contiguous elements in the tryAdvance() implementation. Its constructor calls the super constructor as super(Long.MAX_VALUE, source.characteristics()) .

Regarding the API documentation , I expected that using Long.MAX_VALUE indicate an unknown size. However, it seems he is trying to allocate memory with this size.

Why is he trying to highlight this space? What value should be used for the estimated size?

Here is a test example:

 Stream<Integer> nrs = Stream.of(3, 3, 5, 5, 3, 3, 3, 4, 4, 4 ,5 , 5); Integer [] expected = {3, 5, 3, 4, 5}; Object[] actual = collapse(nrs).toArray(); assertEquals(actual, expected); 

And the implementation of the collapse() method:

 static <T> Stream<T> collapse(Stream<T> source) { return StreamSupport.stream( new StreamCollapse<T>(source.spliterator()), false); } class StreamCollapse<T> extends AbstractSpliterator<T> implements Consumer<T> { private final Spliterator<T> source; private T curr = null; StreamCollapse(Spliterator<T> source) { super(Long.MAX_VALUE, source.characteristics()); this.source = source; } @Override public boolean tryAdvance(Consumer<? super T> action) { T prev = curr; boolean hasNext; while ((hasNext = source.tryAdvance(this)) && curr.equals(prev)) { } if(hasNext) action.accept(curr); return hasNext; } @Override public void accept(T item) { curr = item; } } 

Answer 1

You must remove the characteristics from your composite spliterator, for example:

 // an unknown spliterator shouldn't having SIZED | SUBSIZED characteristics // v super(Long.MAX_VALUE, source.characteristics() & (~(SIZED | SUBSIZED))); 

WHEN Spliteartor is a SIZED Spliteartor, Spliterator # getExactSizeIfKnown will be used by the stream to create the array.

A characteristic value, meaning that the value returned from estimateSize() before traversing or splitting is the final size, which, in the absence of a structural modification of the source, is an accurate count of the number of elements that will occur during a full traverse.

IF stream runs in parallel Stream # toArray will raise an IllegalArgumentException if, using the parameter scaleSize> = Long.MAX_VALUE - 8 .

IF stream is a sequential stream. Stream # toArray will increase its internal array capacity to the size of the estimate.