How to count the number of indexes or null values ​​in a Pandas data group

Question

Its always what seems easy, that torment me. I am trying to get a count of the number of non-zero values ​​of some variables in a Dataframe, grouped by month and year. So I can do it that works great

counts_by_month=df[variable1, variable2].groupby([lambda x: x.year,lambda x: x.month]).count() 

But I REALLY want to know how many of these values ​​in each group are NaN. Therefore, I want to count Nans in each variable too, so that I can calculate the percentage data that is not in each group. I can not find a function for this. or maybe I could get to the same end by counting the total number of elements in the group. Then NaNs will be Total - "Non-Null values"

I tried to find out if I can somehow calculate the index values, but I could not do it. Any help on this is greatly appreciated. Best regards jason

Answer 1

 In [279]: df Out[279]: ABCDE a foo NaN 1.115320 -0.528363 -0.046242 b bar 0.991114 -1.978048 -1.204268 0.676268 c bar 0.293008 -0.708600 NaN -0.388203 d foo 0.408837 -0.012573 1.019361 1.774965 e foo 0.127372 NaN NaN NaN In [280]: def count_missing(frame): return (frame.shape[0] * frame.shape[1]) - frame.count().sum() .....: In [281]: df.groupby('A').apply(count_missing) Out[281]: A bar 1 foo 4 dtype: int64 

Answer 2

 df.isnull().sum() 

Faster and no user function needed :)