Python groupby doesn't work as expected

Question

I am trying to read an Excel spreadsheet containing some columns in the following format:

column1__
column1__AccountName
column1__SomeOtherFeature
column2__blabla
column2_SecondFeat

I already saved the values ​​of one row as a list of tuples, where tuple (column_name, column_value) in a variable x.

Now I would like to group it as follows:

result = { 
    'column__1': [list of (k,v) tuples, which keys start with 'column__1'],
    'column__2': [list of (k,v) tuples, which keys start with 'column__2']
}

But this does not give the expected result:

>>> from itertools import groupby

>>> x
[(u'My field one__AccountName', u'Lorem ipsum bla bla bla'),
 (u'My field one__AccountNumber', u'1111111222255555'),
 (u'My field two__Num', u'Num: 612312345'),
 (u'My field two', u'asdasdafassda'),
 (u'My field three__Beneficiary International Bank Account Number IBAN',
  u'IE111111111111111111111'),
 (u'My field one__BIC', u'BLEAHBLA1'),
 (u'My field three__company name', u'Company XYZ'),
 (u'My field two__BIC', u'ASDF333')]

>>> groups = groupby(x ,lambda (field, val): field.split('__')[0])

>>> grouped_fields = {key: list(val) for key, val in groups}

>>> grouped_fields

{u'My field one': [(u'My field one__BIC', u'BLEAHBLA1')],
 u'My field three': [(u'My field three__company name', u'Company XYZ')],
 u'My field two': [(u'My field two__BIC', u'ASDF333')]}

 >>> x[0]
(u'My field one__AccountName', u'Lorem ipsum bla bla bla')

>>> x[1]
(u'My field one__AccountNumber', u'1111111222255555')

>>> x[0][0].split('__')[0] == x[1][0].split('__')[0]
True

However, it works with another instance of the original list:

>>> y = [(u'x a b__2', 3), (u'x a b__', 1), (u'x a b__1', 2), (u'y a__1', 1), (u'y a__2', 2)]

>>> y
[(u'x__2', 3), (u'x__', 1), (u'x__1', 2), (u'y__1', 1), (u'y__2', 2)]

>>> groupes_y = groupby(y, lambda (k,v): k.split('__')[0])

>>> grouped_y = {key:list(val) for key, val in groupes_y}

>>> grouped_y

{u'x': [(u'x__2', 3), (u'x__', 1), (u'x__1', 2)],
 u'y': [(u'y__1', 1), (u'y__2', 2)]}

I don’t know what I am doing wrong.

Answer 1

As the docs say , you should apply groupbyto a list that is already sorted using the same keyas groupby:

key = lambda fv: fv[0].split('__')[0]
groups = groupby(sorted(x, key=key), key=key)

Then grouped_fields:

{u'My field one': [(u'My field one__AccountName', u'Lorem ipsum bla bla bla'),
  (u'My field one__AccountNumber', u'1111111222255555'),
  (u'My field one__BIC', u'BLEAHBLA1')],
 u'My field three': [(u'My field three__Beneficiary International Bank Account Number IBAN',
   u'IE111111111111111111111'),
  (u'My field three__company name', u'Company XYZ')],
 u'My field two': [(u'My field two__Num', u'Num: 612312345'),
  (u'My field two', u'asdasdafassda'),
  (u'My field two__BIC', u'ASDF333')]}

, y :

>>> y == sorted(y, key=key)
True