How to convert dates in the Pandas framework to the 'date' data type?

Question

I have a Pandas data frame, one of the columns of which contains date strings in the format "YYYY-MM-DD", for example. '2013-10-28'.

The column dtype is currently an "object".

How to convert column values ​​to Pandas date format?

Answer 1

Use astype

In [31]: df Out[31]: a time 0 1 2013-01-01 1 2 2013-01-02 2 3 2013-01-03 In [32]: df['time'] = df['time'].astype('datetime64[ns]') In [33]: df Out[33]: a time 0 1 2013-01-01 00:00:00 1 2 2013-01-02 00:00:00 2 3 2013-01-03 00:00:00 

Answer 2

Essentially equivalent to @waitingkuo, but I would use to_datetime here (it seems to be a little cleaner and offers some additional features, like dayfirst ):

 In [11]: df Out[11]: a time 0 1 2013-01-01 1 2 2013-01-02 2 3 2013-01-03 In [12]: pd.to_datetime(df['time']) Out[12]: 0 2013-01-01 00:00:00 1 2013-01-02 00:00:00 2 2013-01-03 00:00:00 Name: time, dtype: datetime64[ns] In [13]: df['time'] = pd.to_datetime(df['time']) In [14]: df Out[14]: a time 0 1 2013-01-01 00:00:00 1 2 2013-01-02 00:00:00 2 3 2013-01-03 00:00:00 

---

Handling ValueError s
If you encounter a situation when to do

 df['time'] = pd.to_datetime(df['time']) 

Throws

 ValueError: Unknown string format 

This means that you have invalid (not forced) values. If you are pd.NaT with converting them to pd.NaT , you can add the errors='coerce' argument to to_datetime :

 df['time'] = pd.to_datetime(df['time'], errors='coerce') 

Answer 3

I assume that CSV files contain a lot of Pandas data, in which case you can simply convert the date during the initial CSV reading:

dfcsv = pd.read_csv('xyz.csv', parse_dates=[0]) , where 0 refers to the column in which the date is located.
You can also add , index_col=0 if you want the date to be your index.

See http://pandas.pydata.org/pandas-docs/stable/generated/pandas.io.parsers.read_csv.html

Answer 4

Now you can do df['column'].dt.date

Note that for datetime objects, if you do not see the hour when they are all 00:00:00, this is not pandas. This iPython laptop is trying to make everything beautiful.

Answer 5

Perhaps dates should be converted to a different frequency. In this case, I would suggest setting the index by dates.

 #set an index by dates df.set_index(['time'], drop=True, inplace=True) 

After that, you can more easily convert the date format format that you need most. Below, I sequentially convert to a number of date formats, eventually ending with a set of daily dates at the beginning of the month.

 #Convert to daily dates df.index = pd.DatetimeIndex(data=df.index) #Convert to monthly dates df.index = df.index.to_period(freq='M') #Convert to strings df.index = df.index.strftime('%Y-%m') #Convert to daily dates df.index = pd.DatetimeIndex(data=df.index) 

For brevity, I do not show that after each line above I run the following code:

 print(df.index) print(df.index.dtype) print(type(df.index)) 

This gives me the following result:

 Index(['2013-01-01', '2013-01-02', '2013-01-03'], dtype='object', name='time') object <class 'pandas.core.indexes.base.Index'> DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03'], dtype='datetime64[ns]', name='time', freq=None) datetime64[ns] <class 'pandas.core.indexes.datetimes.DatetimeIndex'> PeriodIndex(['2013-01', '2013-01', '2013-01'], dtype='period[M]', name='time', freq='M') period[M] <class 'pandas.core.indexes.period.PeriodIndex'> Index(['2013-01', '2013-01', '2013-01'], dtype='object') object <class 'pandas.core.indexes.base.Index'> DatetimeIndex(['2013-01-01', '2013-01-01', '2013-01-01'], dtype='datetime64[ns]', freq=None) datetime64[ns] <class 'pandas.core.indexes.datetimes.DatetimeIndex'> 

Answer 6

Another way to do this, and it works well if you have multiple columns to convert to date and time. cols = ['date1', 'date2'] df [cols] = df [cols] .apply (pd.to_datetime)