Handling Missing Data in Pandas

Question

I have 200,000 x 500 dataframe loaded in Pandas. Is there a function that can automatically tell me which columns are missing data? Or do I need to iterate over each column and check element by element?

As soon as I found a missing item, how can I define a user-defined function (based both on the column name and some other data on the same line) for automatic replacement. I see the fillna () method, but I don’t think that a function (lambda) is required as input.

Thanks!

Answer 1

sort of:

 import pandas as pd pd.isnull(frame).any() 

You are probably looking to search for missing data.

fillna does not currently accept lambda functions, although this works like an open issue on github.

You can use DataFrame.apply to do custom settings. Although can you be more specific about what you need to do to fill in the data? Just curious what a use case is.