How to remove duplicate in <T> JAVA 8 list

Question

In practice, I know ways to reduce the duplicate value of t20> or assign List to Set , but I have a slightly different problem. How to solve smart way below problem in JAVA 8 using stream or maybe StreamEx ?

Say we have objects in a List

A, A, A, B, B, A, A, A, C, C, C, A, A, B, B, A

Now i need

A, B, A, C, A, B, A

So the duplicate was deleted, but only if it appears as the next, but should remain if there is another object next to it. I tried several solutions, but was ugly and not readable.

Answer 1

Option 1: Filter

You can write a filter with a state, but you should never do this because it violates the filter(Predicate<? super T> predicate) contract filter(Predicate<? super T> predicate) :

predicate - a non-interfering , stateless , to apply to each element to determine whether to include it

 public class NoRepeatFilter<T> implements Predicate<T> { private T prevValue; @Override public boolean test(T value) { if (value.equals(this.prevValue)) return false; this.prevValue = value; return true; } } 

Test

 List<String> result = Stream .of("A", "A", "A", "B", "B", "A", "A", "A", "C", "C", "C", "A", "A", "B", "B", "A") // .parallel() .filter(new NoRepeatFilter<>()) .collect(Collectors.toList()); System.out.println(result); 

Output

[A, B, A, C, A, B, A]

The reason it should be stateless is because it will fail if the thread is parallel, for example. check again with .parallel() uncommented:

[A, A, B, B, A, C, C, C, A, B, B, A]

---

Option 2: Collector

An important decision is to create your own Collector using of(...) :

 public class NoRepeatCollector { public static <E> Collector<E, ?, List<E>> get() { return Collector.of(ArrayList::new, NoRepeatCollector::addNoRepeat, NoRepeatCollector::combineNoRepeat); } private static <E> void addNoRepeat(List<E> list, E value) { if (list.isEmpty() || ! list.get(list.size() - 1).equals(value)) list.add(value); } private static <E> List<E> combineNoRepeat(List<E> left, List<E> right) { if (left.isEmpty()) return right; if (! right.isEmpty()) left.addAll(left.get(left.size() - 1).equals(right.get(0)) ? right.subList(1, right.size()) : right); return left; } } 

Test

 List<String> result = Stream .of("A", "A", "A", "B", "B", "A", "A", "A", "C", "C", "C", "A", "A", "B", "B", "A") // .parallel() .collect(NoRepeatCollector.get()); System.out.println(result); 

Exit (with and without .parallel() )

[A, B, A, C, A, B, A]

---

Option 3: Loop

If your input is List (or another Iterable ), you can remove duplicate values ​​using a simple loop:

 public static <E> void removeRepeats(Iterable<E> iterable) { E prevValue = null; for (Iterator<E> iter = iterable.iterator(); iter.hasNext(); ) { E value = iter.next(); if (value.equals(prevValue)) iter.remove(); else prevValue = value; } } 

Test

 List<String> list = new ArrayList<>(Arrays.asList( "A", "A", "A", "B", "B", "A", "A", "A", "C", "C", "C", "A", "A", "B", "B", "A")); removeRepeats(list); System.out.println(list); 

Output

[A, B, A, C, A, B, A]

Answer 2

It is pretty simple without using threads .. Something like this:

 public List<T> noConsecutiveDuplicates(final List<T> input) { final List<T> output = new ArrayList<>(); for (final T element : input) { if (!element.equals(lastElement(output))) { output.add(element); } } return output; } private T lastElement(final List<T> list) { if (list.size() == 0) { return null; } return list.get(list.size() - 1); } 

Answer 3

I would give StreamEx snapshot and use StreamEx::collapse :

 List<String> strings = Arrays.asList("A", "A", "A", "B", "B", "A", "A", "A", "C", "C", "C", "A", "A", "B", "B", "A"); List<String> collect = StreamEx.of(strings) .collapse(Objects::equals) .collect(Collectors.toList()); 

You can also use vanilla Java and use the idea of "edge detection" :

 List<String> collect = IntStream.range(0, strings.size()) .filter(i -> i == 0 || !Objects.equals(strings.get(i - 1), strings.get(i))) .mapToObj(strings::get) .collect(Collectors.toList()); 

Answer 4

 List<String> lst = Arrays.asList("A", "A", "A", "B", "B", "A", "A", "A", "C", "C", "C", "A", "A", "B", "B", "A"); List<String> result = IntStream.range(0, lst.size()) .filter(index->index ==0 || !lst.get(index).equals(lst.get(index-1))) .mapToObj(i->lst.get(i)).collect(Collectors.toList()); result.stream().forEach(System.out::print); 

You can simply iterate over indexes from the data source and filter those elements that do not match the previous element.

Answer 5

This may not be the cleanest solution, but you can use a filter that remembers the previous value of the stream.

 class noDuplicateFilter implementsd Function<T>{ private T previous=null; public boolean test(T input){ boolean distinct= !Objects.equals(input, previous); this.previous = input; return distinct; } } 

Then use it inside your thread.

There is probably a glitter of solutino in JavaRx.

There are also some solutions here.

Answer 6

I think the most concise way is to use the reduction method, as shown below:

 import java.util.ArrayList; import java.util.Arrays; import java.util.List; import java.util.Stack; import java.util.function.BiFunction; import java.util.function.BinaryOperator; public class Main { public static void main(String[] args) { List<String> ss =Arrays.asList("A","A","A","B","B", "A","A","A", "C", "C", "C","A","A","B","B","A"); BiFunction<ArrayList<String>, String, ArrayList<String>> acc = new BiFunction<ArrayList<String>, String, ArrayList<String>>() { @Override public ArrayList<String> apply(ArrayList<String> strings, String s) { if(strings.isEmpty() || !strings.get(strings.size()-1).equals(s)){ strings.add(s); } return strings; } }; BinaryOperator<ArrayList<String>> combiner = new BinaryOperator<ArrayList<String>>() { @Override public ArrayList<String> apply(ArrayList<String> strings, ArrayList<String> strings2) { strings.addAll(strings2); return strings; } }; ss.stream().reduce(new ArrayList<String>(), acc, combiner).forEach(System.out::println); } }