Grouper with chunk size sequence in python?

Question

I have a generator that I want to repeat on two levels. The first level is unevenly distributed, then I want to say the next level in groups of 5. I need the memory to be efficient and work at the input of the generator, so I do something like the following. I have to think what could be a better way? In particular, I do not want the trailing Nones to be received with uneven length.

import itertools def dynamic_grouper(iterable, intervals): for i in intervals: inner_iter = list(itertools.islice(iterable, i)) # this is a "group" yield inner_iter iterable = iter(xrange(100)) chunk_sizes = [22,30,38,10] for i,group in enumerate(dynamic_grouper(iterable, chunk_sizes)): args = [iter(group)] * 5 for item in itertools.izip_longest(fillvalue=None, *args): print "Group %i" % i print "Items %s" % list(item) 

Answer 1

To avoid None s, you can use snippets :

 def chunks(seq, n): # /questions/1021/how-do-you-split-a-list-into-evenly-sized-chunks/15485#15485 (Ned Batchelder) """ Yield successive n-sized chunks from seq.""" for i in xrange(0, len(seq), n): yield seq[i:i + n] for i,group in enumerate(dynamic_grouper(iterable, chunk_sizes)): for item in chunks(group, 5): print "Group %i" % i print "Items %s" % list(item)