Specify the format of the time phrase when reading a CSV file using pandas?

Question

I have a .csv file with thousands of records created by a data logger.

The format more or less looks like this:

time                | data
01/07/2015 12:25:45 | 356.24
01/07/2015 12:25:50 | 357.24
01/07/2015 12:25:55 | 351.24
01/07/2015 12:26:00 | 357.20
01/07/2015 12:26:05 | 356.32
...

When I read a file using pandas

import pandas as pd
df = pd.read_csv(filename,  parse_dates=True, infer_datetime_format=True)

Some dates are considered erroneous for some reason, so I would like to specify the date format string manually, being a format string format_str = '%d/%m/%Y %H:%M:%S'

How can i do this?

Answer 1

A function pandas.read_csvnot only takes an argument parse_dates=, but also an argument date_parser=. With an argument, date_parser=you can specify your own function for parsing dates.

Like this:

def myparser(x):
    return datetime.strptime(x, '%d/%m/%Y %H:%M:%S')

df = pd.read_csv(filename,  parse_dates=True, date_parser=myparser)

This should ensure that your dates are always processed only with this format.

csv, , , , .

.