How to check if exists any duplicate in Java 8 Streams?

JavaJava 8DuplicatesJava Stream

Java Problem Overview


In java 8, what's the best way to check if a List contains any duplicate?

My idea was something like:

list.size() != list.stream().distinct().count()

Is it the best way?

Java Solutions


Solution 1 - Java

Your code would need to iterate over all elements. If you want to make sure that there are no duplicates simple method like

public static <T> boolean areAllUnique(List<T> list){
	Set<T> set = new HashSet<>();

	for (T t: list){
		if (!set.add(t))
			return false;
	}
	
	return true;
}

would be more efficient since it can give you false immediately when first non-unique element would be found.

This method could also be rewritten as (assuming non-parallel streams and thread-safe environment) using Stream#allMatch which also is short-circuit (returns false immediately for first element which doesn't fulfill provided condition)

public static <T> boolean areAllUnique(List<T> list){
	Set<T> set = new HashSet<>();
	return list.stream().allMatch(t -> set.add(t));
}

or as @Holger mentioned in comment

public static <T> boolean areAllUnique(List<T> list){
	return list.stream().allMatch(new HashSet<>()::add);
}

Solution 2 - Java

I used the following:
1. return list.size() == new HashSet<>(list).size();.

I'm not sure how it compares to:
2. return list.size() == list.stream().distinct().count();
and
3. return list.stream().sequential().allMatch(new HashSet<>()::add);
in terms of performance.

The last one (#3) has possibility to handle not only collections (e.g. lists), but also streams (without explicitly collecting them).

Upd.: The last one (#3) seems to be the best not only because it can handle pure streams, but also because it stops on the first duplicate (while #1 and #2 always iterate till the end) — as @Pshemo said in comment.

Solution 3 - Java

You can use the counting collector.

Stream.of(1, 3, 4, 6, 7, 5, 6)
            .collect(Collectors.groupingBy(
                    Function.identity(), Collectors.counting()))
            .entrySet().stream().anyMatch(e -> e.getValue() > 1)

Solution 4 - Java

Started this class as a StreamTool, but I think there must be an even better way with reduce or similar:

public class StreamTool {

	/**
	 * Whether stream records are unique in that stream.
	 * @param <T> Type of records
	 * @param records
	 * @return true if there are no duplicates, false otherwise
	 */
	public static <T> boolean isUnique(Stream<T> records) {
		return records.allMatch(new HashSet<>()::add);
	}
}

Solution 5 - Java

Given array arr,

arr.length != Arrays.stream(arr).distinct().count()

will help check for duplicates

Solution 6 - Java

Use set.add() it is faster.

Set<T> items = new HashSet<>();
list.stream().filter(n -> !items.add(n)) 
            .collect(Collectors.toSet());

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
Questionpedrorijo91View Question on Stackoverflow
Solution 1 - JavaPshemoView Answer on Stackoverflow
Solution 2 - JavaSashaView Answer on Stackoverflow
Solution 3 - JavaWill HumphreysView Answer on Stackoverflow
Solution 4 - JavageekdenzView Answer on Stackoverflow
Solution 5 - JavaEdor LinusView Answer on Stackoverflow
Solution 6 - JavaitroView Answer on Stackoverflow