Pandas ".convert_objects (convert_numeric = True)" is deprecated

Question

I have this line in my code that converts my data to numeric ...

data["S1Q2I"] = data["S1Q2I"].convert_objects(convert_numeric=True) 

The fact is that now the new version of pandas (0.17.0) says that this function is deprecated. This is mistake:

 FutureWarning: convert_objects is deprecated. Use the data-type specific converters pd.to_datetime, pd.to_timedelta and pd.to_numeric. data["S3BD5Q2A"] = data["S3BD5Q2A"].convert_objects(convert_numeric=True) 

So, I went to the new documentation, and I could not find any examples of how to use the new function to convert my data ...

He only says this:

"DataFrame.convert_objects is deprecated in favor of the type-specific functions pd.to_datetime, pd.to_timestamp and pd.to_numeric (new in 0.17.0) (GH11133)."

Any help would be enjoyable!

Answer 1

As @EvanWright explained in the comments,

 data['S1Q2I'] = pd.to_numeric(data['S1Q2I']) 

is now the preferred type conversion method. A detailed explanation of the changes can be found in github PR GH11133 .

Answer 2

You can make a replacement by applying as done here . An example is:

 >>> import pandas as pd >>> a = pd.DataFrame([{"letter":"a", "number":"1"},{"letter":"b", "number":"2"}]) >>> a.dtypes letter object number object dtype: object >>> b = a.apply(pd.to_numeric, errors="ignore") >>> b.dtypes letter object number int64 dtype: object >>> 

But this sucks in two ways:

• You should use the apply method, not the non-native dataframe method
• You need to copy to another data frame - cannot be done in place. So much to use with big data.

I do not really like the direction of pandas. I have not used R data.table much, but so far it seems to be excellent.

I think a data table with a natural in-place type conversion is pretty simple for a competitive data analysis framework.

Answer 3

It depends on the version of Pandas ...... if you have Pandas version 0.18.0 this type will work ........

 df['col name'] = df['col name'].apply(pd.to_numeric, errors='coerce') 

other versions ........

 df['col name']=df.col name .astype(float) 

Answer 4

You can force it to correctly apply to a specific variable name in the data framework without having to copy it to another data format as follows:

 >>> import pandas as pd >>> a = pd.DataFrame([{"letter":"a", "number":"1"},{"letter":"b", "number":"2"}]) >>> a.dtypes letter object number object dtype: object >>> a['number'] = a['number'].apply(pd.to_numeric, errors='coerce') >>> a.dtypes letter object number int64 dtype: object 

An example based on the original question above would be something like this:

 data['S1Q2I'] = data['S1Q2I'].apply(pd.to_numeric, errors='coerce') 

This works the same as your original:

 data['S1Q2I'] = data['S1Q2I'].convert_objects(convert_numeric=True) 

in my hands, anyway ....

This does not apply to the paragraph of the point made about the derivation of data types that are slightly above my head. I'm afraid!