Pandas: find the index of the row with the second highest value

Question

I am trying to get the row index with the second highest value after groupby is done, but I am not getting the correct result

df = pd.DataFrame({'Sp':['a','b','c','d','e','f'], 'Mt':['s1', 's1', 's2','s2','s2','s3'], 'Value':[1,2,3,4,5,6], 'count':[3,2,5,10,10,6]})

Doing this

df.iloc[df.groupby(['Mt'])['Value'].apply(lambda x: (x!=max(x)).idxmax())]

returns

    Mt  Sp  Value   count
0   s1  a   1   3
2   s2  c   3   5
5   s3  f   6   6

For group s2, it is necessary to return index 3 of the original data block.

Answer 1

Since the "Value" is already sorted, you can use nth:

In [11]: g = df.groupby("Mt", as_index=False)

In [12]: g.nth(-2)
Out[12]:
   Mt Sp  Value  count
0  s1  a      1      3
3  s2  d      4     10

Otherwise, I would first sort by value df = df.sort_values("Value").

If you want the latter (if there are less than two in this group), you can also capture it

In [21]: g = df.groupby("Mt")

In [22]: res = g.nth(-1)

In [23]: res.update(g.nth(-2))

In [24]: res
Out[24]:
   Sp  Value  count
Mt
s1  a      1      3
s2  d      4     10
s3  f      6      6

---

Associated function tail(to get the last two elements):

In [31]: g.tail(2)
Out[31]:
   Mt Sp  Value  count
0  s1  a      1      3
1  s1  b      2      2
3  s2  d      4     10
4  s2  e      5     10
5  s3  f      6      6

Answer 2

. , .

df.iloc[df.groupby(['Mt'])['Value'].apply(lambda x: (x!=max(x)).order(ascending=False).head(1).index[0])]

, , . , x!=max(x) .