Pandas Dataframe with NA Values ​​Throwing a ValueError

Question

I have a dataframe in pandas that looks like this

df.head(2)
Out[25]: 
                                   CompanyName Region MachineType
recvd_dttm                                                    
2014-07-13 12:40:40     Company1    NA    Machine1
2014-07-13 15:31:39     Company2    NA    Machine2

First I take the data in a specific date range, and then I try to get the data that is in the NA area, and this is MachineType Machine1.

However, I keep getting this error: ValueError: Length mismatch: Expected axis has 4 elements, new values have 3 elements

This code worked until I added a region column and used this row: df = df[(df['Region']=='NA') & (df['CallType']=='Optia')]

Since at first the data for NA (NorthAmerica) was read as NaN, I used keep_default_na=Falseread_csv in my command.

However, I did pivot_table this way

result = df.groupby([lambda idx: idx.month, 'CompanyName']).agg(len).reset_index()
result.columns = ['Month', 'CompanyName', 'NumberCalls']

pivot_table = result.pivot(index='Month', columns='CompanyName', values='NumberCalls').fillna(0)

And the error occurs in the result.columns line, although I won’t be surprised if the fillna (0) command is possible, since there are other values NAthat should have been NaN, and not NorthAmerica.

ValueError NA?

Answer 1

NaN :

import pandas as pd
import numpy as np

df = pd.read_clipboard()
print df

# I created a test column
           recvd_dttm CompanyName  Region MachineType  Test
2014-07-13   12:40:40    Company1     NaN    Machine1   NaN
2014-07-13   15:31:39    Company2     NaN    Machine2   NaN

df['Region'] = df['Region'].replace(np.NaN, 'NorthAm')
print df

           recvd_dttm CompanyName   Region MachineType  Test
2014-07-13   12:40:40    Company1  NorthAm    Machine1   NaN
2014-07-13   15:31:39    Company2  NorthAm    Machine2   NaN