Select rows in a DataFrame where columns have more values ​​in the series

Question

I have a dataframe of values,

df1 = pd.DataFrame(np.random.rand(5*4).reshape(5,4),columns=['a','b','c','d'])
         a        b        c        d 
0 0.346137 0.537688 0.984077 0.809581
1 0.644753 0.363966 0.617507 0.114848
2 0.495147 0.014281 0.780733 0.579303
3 0.393447 0.108278 0.255716 0.318466
4 0.718629 0.789863 0.217532 0.891606

and a series of highs.

s = pd.Series(np.random.rand(4),index=['a','b','c','d'])

a    0.005678
b    0.419059
c    0.511721
d    0.322693

I am trying to identify all rows in df1, where the value in the df1 columns is greater than the corresponding value in s.

I have a way to do this one column at a time, but would like to do it all at once.

df1[df1.a > s.a].index,df1[df1.b > s.b].index,df1[df1.c > s.c].index,df1[df1.d > s.d].index

(Int64Index([0, 1, 2, 3, 4], dtype='int64'),
 Int64Index([0, 4], dtype='int64'),
 Int64Index([0, 1, 2], dtype='int64'),
 Int64Index([0, 2, 4], dtype='int64'))

in the end, I would like the result to be [0, 1, 2, 3, 4]

Answer 1

Here's the approach -

r,c = np.where((df1 > s).T)
out = np.split(df1.index[c], np.flatnonzero(r[1:] > r[:-1])+1 )

Run Example -

In [141]: df1
Out[141]: 
          a         b         c         d
0  0.346137  0.537688  0.984077  0.809581
1  0.644753  0.363966  0.617507  0.114848
2  0.495147  0.014281  0.780733  0.579303
3  0.393447  0.108278  0.255716  0.318466
4  0.718629  0.789863  0.217532  0.891606

In [142]: s
Out[142]: 
a    0.005678
b    0.419059
c    0.511721
d    0.322693
dtype: float64

In [143]: r,c = np.where((df1 > s).T)

In [144]: np.split(df1.index[c], np.flatnonzero(r[1:] > r[:-1])+1 )
Out[144]: 
[Int64Index([0, 1, 2, 3, 4], dtype='int64'),
 Int64Index([0, 4], dtype='int64'),
 Int64Index([0, 1, 2], dtype='int64'),
 Int64Index([0, 2, 4], dtype='int64')]

Answer 2

I found

df1.loc[(df1 > s).any(axis=1) == True].index.tolist()

work correctly and concise.