Adding two Java 8 streams, or an extra element to a stream

I can add streams or extra elements, like this:

Stream stream = Stream.concat(stream1, Stream.concat(stream2, Stream.of(element));

And I can add new stuff as I go, like this:

Stream stream = Stream.concat(
                       Stream.concat(
                              stream1.filter(x -> x!=0), stream2)
                              .filter(x -> x!=1),
                                  Stream.of(element))
                                  .filter(x -> x!=2);

But this is ugly, because concat is static. If concat were an instance method, the above examples would be much easier to read:

 Stream stream = stream1.concat(stream2).concat(element);

And

 Stream stream = stream1
                 .filter(x -> x!=0)
                 .concat(stream2)
                 .filter(x -> x!=1)
                 .concat(element)
                 .filter(x -> x!=2);

My question is:

1. Is there any good reason why concat is static? Or is there some equivalent instance method I'm missing?

2. In any case, is there a better way of doing this?

Unfortunately this answer is probably of little or no help whatsoever, but I did a forensics analysis of the Java Lambda Mailing list to see if I could find the cause of this design. This is what I found out.

In the beginning there was an instance method for Stream.concat(Stream)

In the mailing list I can clearly see the method was originally implemented as an instance method, as you can read in this thread by Paul Sandoz, about the concat operation.

In it they discuss the issues that could arise from those cases in which the stream could be infinite and what concatenation would mean in those cases, but I do not think that was the reason for the modification.

You see in this other thread that some early users of the JDK 8 questioned about the behavior of the concat instance method when used with null arguments.

This other thread reveals, though, that the design of the concat method was under discussion.

Refactored to Streams.concat(Stream,Stream)

But without any explanation, suddenly, the methods were changed to static methods, as you can see in this thread about combining streams. This is perhaps the only mail thread that sheds a bit of light about this change, but it was not clear enough for me to determine the reason for the refactoring. But we can see they did a commit in which they suggested to move the concat method out of Stream and into the helper class Streams.

Refactored to Stream.concat(Stream,Stream)

Later, it was moved again from Streams to Stream, but yet again, no explanation for that.

So, bottom line, the reason for the design is not entirely clear for me and I could not find a good explanation. I guess you could still ask the question in the mailing list.

Some Alternatives for Stream Concatenation

This other thread by Michael Hixson discusses/asks about other ways to combine/concat streams

> 1. To combine two streams, I should do this: > > Stream.concat(s1, s2) > > not this: > > Stream.of(s1, s2).flatMap(x -> x) > > ... right? > > 2. To combine more than two streams, I should do this: > > Stream.of(s1, s2, s3, ...).flatMap(x -> x) > > not this: > > Stream.of(s1, s2, s3, ...).reduce(Stream.empty(), Stream::concat) > > ... right?

If you add static imports for Stream.concat and Stream.of, the first example could be written as follows:

Stream<Foo> stream = concat(stream1, concat(stream2, of(element)));

Importing static methods with generic names can result in code that becomes difficult to read and maintain (namespace pollution). So, it might be better to create your own static methods with more meaningful names. However, for demonstration I will stick with this name.

public static <T> Stream<T> concat(Stream<? extends T> lhs, Stream<? extends T> rhs) {
    return Stream.concat(lhs, rhs);
}
public static <T> Stream<T> concat(Stream<? extends T> lhs, T rhs) {
    return Stream.concat(lhs, Stream.of(rhs));
}

With these two static methods (optionally in combination with static imports), the two examples could be written as follows:

Stream<Foo> stream = concat(stream1, concat(stream2, element));

Stream<Foo> stream = concat(
                         concat(stream1.filter(x -> x!=0), stream2).filter(x -> x!=1),
                         element)
                     .filter(x -> x!=2);

The code is now significantly shorter. However, I agree that the readability hasn't improved. So I have another solution.

---

In a lot of situations, Collectors can be used to extend the functionality of streams. With the two Collectors at the bottom, the two examples could be written as follows:

Stream<Foo> stream = stream1.collect(concat(stream2)).collect(concat(element));

Stream<Foo> stream = stream1
                     .filter(x -> x!=0)
                     .collect(concat(stream2))
                     .filter(x -> x!=1)
                     .collect(concat(element))
                     .filter(x -> x!=2);

The only difference between your desired syntax and the syntax above is, that you have to replace concat(...) with collect(concat(...)). The two static methods can be implemented as follows (optionally used in combination with static imports):

private static <T,A,R,S> Collector<T,?,S> combine(Collector<T,A,R> collector, Function<? super R, ? extends S> function) {
    return Collector.of(
        collector.supplier(),
        collector.accumulator(),
        collector.combiner(),
        collector.finisher().andThen(function));
}
public static <T> Collector<T,?,Stream<T>> concat(Stream<? extends T> other) {
    return combine(Collectors.toList(),
        list -> Stream.concat(list.stream(), other));
}
public static <T> Collector<T,?,Stream<T>> concat(T element) {
    return concat(Stream.of(element));
}

---

Of course there is a drawback with this solution that should be mentioned. collect is a final operation that consumes all elements of the stream. On top of that, the collector concat creates an intermediate ArrayList each time it is used in the chain. Both operations can have a significant impact on the behaviour of your program. However, if readability is more important than performance, it might still be a very helpful approach.

Just do:

Stream.of(stream1, stream2, Stream.of(element)).flatMap(identity());

where identity() is a static import of Function.identity().

Concatenating multiple streams into one stream is the same as flattening a stream.

However, unfortunately, for some reason there is no flatten() method on Stream, so you have to use flatMap() with the identity function.

My StreamEx library extends the functionality of Stream API. In particular it offers methods like append and prepend which solve this issue (internally they use concat). These methods can accept either another stream or collection or varargs array. Using my library your problem can be solved this way (note that x != 0 look strange for non-primitive stream):

Stream<Integer> stream = StreamEx.of(stream1)
             .filter(x -> !x.equals(0))
             .append(stream2)
             .filter(x -> !x.equals(1))
             .append(element)
             .filter(x -> !x.equals(2));

By the way there's also a shortcut for your filter operation:

Stream<Integer> stream = StreamEx.of(stream1).without(0)
                                 .append(stream2).without(1)
                                 .append(element).without(2);

You can use Guava's Streams.concat(Stream<? extends T>... streams) method, which results in a flattened stream:

Stream stream = Streams.concat(stream1, stream2, Stream.of(element));

If you don't mind using 3rd Party Libraries cyclops-react has an extended Stream type that will allow you to do just that via the append / prepend operators.

Individual values, arrays, iterables, Streams or reactive-streams Publishers can be appended and prepended as instance methods.

Stream stream = ReactiveSeq.of(1,2)
                           .filter(x -> x!=0)
                           .append(ReactiveSeq.of(3,4))
                           .filter(x -> x!=1)
                           .append(5)
                           .filter(x -> x!=2);

[Disclosure I am the lead developer of cyclops-react]

How about writing your own concat method?

public static <T> Stream<T> concat(Stream<? extends T> a, 
                                   Stream<? extends T> b, 
                                   Stream<? extends T>... args)
{
    Stream<T> concatenated = Stream.concat(a, b);
    for (Stream<? extends T> stream : args)
    {
        concatenated = Stream.concat(concatenated, stream);
    }
    return concatenated;
}

This at least makes your first example a lot more readable.

As @Legna pointed out, this could very fast result in a StackOverflowError due to nested calls to Stream::concat.

So here is another version that should fix the problem and looks quite neat:

public static <T> Stream<T> concat(final Stream<? extends T>... args)
{
	return args == null ? Stream.empty()
	                    : Stream.of(args).flatMap(Function.identity());
}

At the end of the day I'm not interested in combining streams, but in obtaining the combined result of processing each element of all those streams.

While combining streams might prove to be cumbersome (thus this thread), combining their processing results is fairly easy.

The key to solve is to create your own collector and ensure that the supplier function for the new collector returns the same collection every time (not a new one), the code below illustrates this approach.

package scratchpad;

import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collector;
import java.util.stream.Stream;

public class CombineStreams {
    public CombineStreams() {
        super();
    }

    public static void main(String[] args) {
        List<String> resultList = new ArrayList<>();
        Collector<String, List<String>, List<String>> collector = Collector.of(
                () -> resultList,
                (list, item) -> {
                    list.add(item);
                },
                (llist, rlist) -> {
                    llist.addAll(rlist);
                    return llist;
                }
        );
        String searchString = "Wil";

        System.out.println("After processing first stream\n"
                + createFirstStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

        System.out.println("After processing second stream\n"
                + createSecondStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

        System.out.println("After processing third stream\n"
                + createThirdStream().filter(name -> name.contains(searchString)).collect(collector));
        System.out.println();

    }

    private static Stream<String> createFirstStream() {
        return Arrays.asList(
                "William Shakespeare",
                "Emily Dickinson",
                "H. P. Lovecraft",
                "Arthur Conan Doyle",
                "Leo Tolstoy",
                "Edgar Allan Poe",
                "Robert Ervin Howard",
                "Rabindranath Tagore",
                "Rudyard Kipling",
                "Seneca",
                "John Donne",
                "Sarah Williams",
                "Oscar Wilde",
                "Catullus",
                "Alfred Tennyson",
                "William Blake",
                "Charles Dickens",
                "John Keats",
                "Theodor Herzl"
        ).stream();
    }

    private static Stream<String> createSecondStream() {
        return Arrays.asList(
                "Percy Bysshe Shelley",
                "Ernest Hemingway",
                "Barack Obama",
                "Anton Chekhov",
                "Henry Wadsworth Longfellow",
                "Arthur Schopenhauer",
                "Jacob De Haas",
                "George Gordon Byron",
                "Jack London",
                "Robert Frost",
                "Abraham Lincoln",
                "O. Henry",
                "Ovid",
                "Robert Louis Stevenson",
                "John Masefield",
                "James Joyce",
                "Clark Ashton Smith",
                "Aristotle",
                "William Wordsworth",
                "Jane Austen"
        ).stream();
    }

    private static Stream<String> createThirdStream() {
        return Arrays.asList(
                "Niccolò Machiavelli",
                "Lewis Carroll",
                "Robert Burns",
                "Edgar Rice Burroughs",
                "Plato",
                "John Milton",
                "Ralph Waldo Emerson",
                "Margaret Thatcher",
                "Sylvie d'Avigdor",
                "Marcus Tullius Cicero",
                "Banjo Paterson",
                "Woodrow Wilson",
                "Walt Whitman",
                "Theodore Roosevelt",
                "Agatha Christie",
                "Ambrose Bierce",
                "Nikola Tesla",
                "Franz Kafka"
        ).stream();
    }
}