Is it possible to merge iterators in Java?

JavaIteratorIteration

Java Problem Overview


Is it possible to merge iterators in Java? I have two iterators and I want to combine/merge them so that I could iterate though their elements in one go (in same loop) rather than two steps. Is that possible?

Note that the number of elements in the two lists can be different therefore one loop over both lists is not the solution.

Iterator<User> pUsers = userService.getPrimaryUsersInGroup(group.getId());
Iterator<User> sUsers = userService.getSecondaryUsersInGroup(group.getId());

while(pUsers.hasNext()) {
  User user = pUsers.next();
  .....
}

while(sUsers.hasNext()) {
  User user = sUsers.next();
  .....
}

Java Solutions


Solution 1 - Java

Guava (formerly Google Collections) has Iterators.concat.

Solution 2 - Java

Also the Apache Commons Collection have several classes for manipulating Iterators, like the IteratorChain, that wraps a number of Iterators.

Solution 3 - Java

You could create your own implementation of the Iterator interface which iterates over the iterators:

public class IteratorOfIterators implements Iterator {
    private final List<Iterator> iterators;

    public IteratorOfIterators(List<Iterator> iterators) {
        this.iterators = iterators;
    }

    public IteratorOfIterators(Iterator... iterators) {
        this.iterators = Arrays.asList(iterators);
    }


    public boolean hasNext() { /* implementation */ }

    public Object next() { /* implementation */ }

    public void remove() { /* implementation */ }
}

(I've not added generics to the Iterator for brevity.) The implementation is not too hard, but isn't the most trivial, you need to keep track of which Iterator you are currently iterating over, and calling next() you'll need to iterate as far as you can through the iterators until you find a hasNext() that returns true, or you may hit the end of the last iterator.

I'm not aware of any implementation that already exists for this.

Update:
I've up-voted Andrew Duffy's answer - no need to re-invent the wheel. I really need to look into Guava in more depth.

I've added another constructor for a variable number of arguments - almost getting off topic, as how the class is constructed here isn't really of interest, just the concept of how it works.

Solution 4 - Java

I haven't written Java code in a while, and this got me curious to whether I've still "got it".

First try:

import java.util.Iterator;
import java.util.Arrays; /* For sample code */

public class IteratorIterator<T> implements Iterator<T> {
    private final Iterator<T> is[];
    private int current;

    public IteratorIterator(Iterator<T>... iterators)
    {
            is = iterators;
            current = 0;
    }

    public boolean hasNext() {
            while ( current < is.length && !is[current].hasNext() )
                    current++;

            return current < is.length;
    }

    public T next() {
            while ( current < is.length && !is[current].hasNext() )
                    current++;

            return is[current].next();
    }

    public void remove() { /* not implemented */ }

    /* Sample use */
    public static void main(String... args)
    {
            Iterator<Integer> a = Arrays.asList(1,2,3,4).iterator();
            Iterator<Integer> b = Arrays.asList(10,11,12).iterator();
            Iterator<Integer> c = Arrays.asList(99, 98, 97).iterator();

            Iterator<Integer> ii = new IteratorIterator<Integer>(a,b,c);

            while ( ii.hasNext() )
                    System.out.println(ii.next());
    }
}

You could of course use more Collection classes rather than a pure array + index counter, but this actually feels a bit cleaner than the alternative. Or am I just biased from writing mostly C these days?

Anyway, there you go. The answer to you question is "yes, probably".

Solution 5 - Java

public class IteratorJoin<T> implements Iterator<T> {
	private final Iterator<T> first, next;
	
	public IteratorJoin(Iterator<T> first, Iterator<T> next) {
		this.first = first;
		this.next = next;
	}

	@Override
	public boolean hasNext() {
		return first.hasNext() || next.hasNext();
	}

	@Override
	public T next() {
		if (first.hasNext())
			return first.next();
		return next.next();
	}
}

Solution 6 - Java

Starting with Java 8 and later this can be done without external dependencies using Stream API. This also allows concatenation of iterator with other types of streams.

Streams.concat(StreamSupport.stream(<iter1>, false), StreamSupport.stream(<iter2>, false));

Solution 7 - Java

move your loop to a method and pass the iterator to method.

void methodX(Iterator x) {
    while (x.hasNext()) {
        ....
    }
}

Solution 8 - Java

an iterator comes FROM a collection or a set.
why not use the method already available
Collection.addAll(Collection c);
and then create your iterator from the last object.
this way, your iterator will iterate all the contents of both collection.

Solution 9 - Java

You can use my version of an extendable iterator. It uses a double-ended queue of iterators which to me makes sense:

import java.util.Deque;
import java.util.Iterator;
import java.util.concurrent.ConcurrentLinkedDeque;

public class ExtendableIterator<T> implements Iterator<T> {
	
	public Deque<Iterator<T>> its = new ConcurrentLinkedDeque<Iterator<T>>();

	public ExtendableIterator() {
	
	}

	public ExtendableIterator(Iterator<T> it) {
		this();
		this.extend(it);
	}
	
	@Override
	public boolean hasNext() {
		// this is true since we never hold empty iterators
		return !its.isEmpty() && its.peekLast().hasNext();
	}
	
	@Override
	public T next() {
		T next = its.peekFirst().next();
		if (!its.peekFirst().hasNext()) {
			its.removeFirst();
		}
		return next;
	}
	
	public void extend(Iterator<T> it) {
		if (it.hasNext()) {
			its.addLast(it);
		}
	}
}

Solution 10 - Java

I would refactor the original design from:

Iterator<User> pUsers = userService.getPrimaryUsersInGroup(group.getId());
Iterator<User> sUsers = userService.getSecondaryUsersInGroup(group.getId());

To something like:

Iterator<User> users = userService.getUsersInGroup(group.getId(), User.PRIMARY, User.SECONDARY, ...);

Solution 11 - Java

The Merged Iterator:

import static java.util.Arrays.asList;

import java.util.Iterator;
import java.util.LinkedList;
import java.util.List;
import java.util.NoSuchElementException;


public class ConcatIterator<T> implements Iterator<T> {

    private final List<Iterable<T>> iterables;
    private Iterator<T> current;

    @SafeVarargs
    public ConcatIterator(final Iterable<T>... iterables) {
        this.iterables = new LinkedList<>(asList(iterables));
    }

    @Override
    public boolean hasNext() {
        checkNext();
        return current != null && current.hasNext();
    }

    @Override
    public T next() {
        checkNext();
        if (current == null || !current.hasNext()) throw new NoSuchElementException();
        return current.next();
    }

    @Override
    public void remove() {
        if (current == null) throw new IllegalStateException();
        current.remove();
    }

    private void checkNext() {
        while ((current == null || !current.hasNext()) && !iterables.isEmpty()) {
            current = iterables.remove(0).iterator();
        }
    }

}

The concat method to create an Iterable:

@SafeVarargs
public static <T> Iterable<T> concat(final Iterable<T>... iterables) {
    return () -> new ConcatIterator<>(iterables);
}

Simple JUnit test:

@Test
public void testConcat() throws Exception {
    final Iterable<Integer> it1 = asList(1, 2, 3);
    final Iterable<Integer> it2 = asList(4, 5);
    int j = 1;
    for (final int i : concat(it1, it2)) {
        assertEquals(j, i);
        j++;
    }
}

Solution 12 - Java

You can try ConcatIterator from Cactoos:

Iterator<String> names = new ConcatIterator<>(
  Arrays.asList("Sarah", "Mary").iterator(),
  Arrays.asList("Jeff", "Johnny").iterator(),
);

Also check ConcatIterable, which concatenates Iterables.

Solution 13 - Java

In the Apache Commons Collections there is public static <E> Iterator<E> org.apache.commons.collections4.IteratorUtils.chainedIterator(Collection<Iterator<? extends E>> iterators) that says

>Gets an iterator that iterates through a collections of Iterators one after another.

which should be what you want.

import java.util.Arrays;
import java.util.Iterator;

import org.apache.commons.collections4.IteratorUtils;
//also works: import org.apache.commons.collections.IteratorUtils;

class Scratch {
	public static void main( String[] args ) {
		final Iterator<String> combinedIterator = IteratorUtils.chainedIterator(
				Arrays.asList( "a", "b", "c" ).iterator(),
				Arrays.asList( "1", "2", "3" ).iterator()
			);
		while( combinedIterator.hasNext() ){
			System.out.println( combinedIterator.next() );
		}
        // "abc123" will have been printed out
	}
}

Solution 14 - Java

every Iterator object holds own memory location (adress), so you can't simply "merge" them. except if you extend iterator class and write your own implementation there.

If you are dealing with the same number of objects in both iterators an alternative solution would be to process two iterators in one loop like this :

   while (iterator1.hasNext() && iterator2.hasNext()) {
      // code
    }

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionJahanzeb FarooqView Question on Stackoverflow
Solution 1 - JavaAndrew DuffyView Answer on Stackoverflow
Solution 2 - JavaItherView Answer on Stackoverflow
Solution 3 - JavaNoel MView Answer on Stackoverflow
Solution 4 - JavaChristofferView Answer on Stackoverflow
Solution 5 - JavaclelsonView Answer on Stackoverflow
Solution 6 - JavaTaras AleninView Answer on Stackoverflow
Solution 7 - JavamhshamsView Answer on Stackoverflow
Solution 8 - Javap01ntbl4nkView Answer on Stackoverflow
Solution 9 - JavaReut SharabaniView Answer on Stackoverflow
Solution 10 - JavaBillworth VandoryView Answer on Stackoverflow
Solution 11 - JavabenezView Answer on Stackoverflow
Solution 12 - Javayegor256View Answer on Stackoverflow
Solution 13 - JavaSledView Answer on Stackoverflow
Solution 14 - JavaYoussefView Answer on Stackoverflow