How to combine two python iterators?

Question

I have two iterators, a list object and itertools.count (i.e. an infinite value generator). I would like to combine these two into a resulting iterator, which will alternate the values ​​of profitability between them:

 >>> import itertools >>> c = itertools.count(1) >>> items = ['foo', 'bar'] >>> merged = imerge(items, c) # the mythical "imerge" >>> merged.next() 'foo' >>> merged.next() 1 >>> merged.next() 'bar' >>> merged.next() 2 >>> merged.next() Traceback (most recent call last): ... StopIteration 

What is the easiest and most concise way to do this?

Answer 1

The generator will solve your problem well.

 def imerge(a, b): for i, j in itertools.izip(a,b): yield i yield j 

Answer 2

You can do something that is almost impossible that @Pramod suggested.

 def izipmerge(a, b): for i, j in itertools.izip(a,b): yield i yield j 

The advantage of this approach is that you will not run out of memory if both a and b are infinite.

Answer 3

I would do something like this. This will be the most effective time and space, since you will not have overhead costs for fastening objects. This will also work if both a and b infinite.

 def imerge(a, b): i1 = iter(a) i2 = iter(b) while True: try: yield i1.next() yield i2.next() except StopIteration: return 

Answer 4

I also agree that itertools is not required.

But why stop at 2?

  def tmerge(*iterators): for values in zip(*iterators): for value in values: yield value 

handles any number of iterators from 0 up.

UPDATE: DOH! One commenter noted that this would not work if all iterators do not have the same length.

The correct code is:

 def tmerge(*iterators): empty = {} for values in itertools.izip_longest(*iterators, fillvalue=empty): for value in values: if value is not empty: yield value 

and yes, I just tried it with lists of unequal length and a list containing {}.

Answer 5

You can use zip as well as itertools.chain . This will only work if the first list is finite :

 merge=itertools.chain(*[iter(i) for i in zip(['foo', 'bar'], itertools.count(1))]) 

Answer 6

I'm not sure what your application is, but you may find the enumerate () function more useful.

 >>> items = ['foo', 'bar', 'baz'] >>> for i, item in enumerate(items): ... print item ... print i ... foo 0 bar 1 baz 2 

Answer 7

I prefer this other way, which is much more concise:

 iter = reduce(lambda x,y: itertools.chain(x,y), iters) 

Answer 8

One of the lesser-known features of Python is that you can have more sentences in the generator expression. Very useful for smoothing nested lists, for example from zip () / izip ().

 def imerge(*iterators): return (value for row in itertools.izip(*iterators) for value in row) 

Answer 9

Here is an elegant solution:

 def alternate(*iterators): while len(iterators) > 0: try: yield next(iterators[0]) # Move this iterator to the back of the queue iterators = iterators[1:] + iterators[:1] except StopIteration: # Remove this iterator from the queue completely iterators = iterators[1:] 

Using the actual queue to improve performance (as David suggested):

 from collections import deque def alternate(*iterators): queue = deque(iterators) while len(queue) > 0: iterator = queue.popleft() try: yield next(iterator) queue.append(iterator) except StopIteration: pass 

It works even if some iterators are finite and others are infinite:

 from itertools import count for n in alternate(count(), iter(range(3)), count(100)): input(n) 

Print

 0 0 100 1 1 101 2 2 102 3 103 4 104 5 105 6 106 

It also stops correctly if / when all iterators have been exhausted.

If you want to process iterators without an iterator, like lists, you can use

 def alternate(*iterables): queue = deque(map(iter, iterables)) ... 

Answer 10

Use izip and chain together:

 >>> list(itertools.chain.from_iterable(itertools.izip(items, c))) # 2.6 only ['foo', 1, 'bar', 2] >>> list(itertools.chain(*itertools.izip(items, c))) ['foo', 1, 'bar', 2] 

Answer 11

Why do I need itertools?

 def imerge(a,b): for i,j in zip(a,b): yield i yield j 

In this case, at least one of a or b must have a finite length, then zip will return a list, not an iterator. If you need an iterator as output, you can go for a Claudiu solution.

Answer 12

Using itertools.izip () instead of zip (), as in some other answers, will improve performance:

As "pydoc itertools.izip" shows: "Works like a zip () function, but consumes less memory, returning an iterator instead of a list."

Itertools.izip will also work fine, even if one of the iterators is infinite.

Answer 13

A concise method is to use a generator expression with itertools.cycle (). It avoids creating a long chain () of tuples.

 generator = (it.next() for it in itertools.cycle([i1, i2]))