Localizing date and time with python / django

Question

I am trying to parse an RSS feed. Feed entries have date elements, for example:

<dc:date>2016-09-21T16:00:00+02:00</dc:date> 

Using feedparser, I am trying to do:

 published_time = datetime.fromtimestamp(mktime(entry.published_parsed)) 

But the problem is that I seem to be misusing the time stored in the database. In this particular case, the datetime time is saved as:

 2016-09-21 13:00:00 

... when I would expect 14:00 - the correct time is UTC.

I assume the problem is in our django settings, where we have:

 TIME_ZONE = 'Europe/Berlin' 

Because when I switch to:

 TIME_ZONE = 'UTC' 

... the datatum is saved as the correct UTC time:

 2016-09-21 14:00:00 

Is there a way to keep django settings as they are, but to properly and correctly store this date-time without changing the django time zone setting?

EDIT: Maybe it's more clear how it is ...

 print entry.published_parsed published_time = datetime.fromtimestamp(mktime(entry.published_parsed)) print published_time localized_time = pytz.timezone(settings.TIME_ZONE).localize(published_time, is_dst=None) print localized_time time.struct_time(tm_year=2016, tm_mon=9, tm_mday=21, tm_hour=14, tm_min=0, tm_sec=0, tm_wday=2, tm_yday=265, tm_isdst=0) 2016-09-21 15:00:00 2016-09-21 15:00:00+02:00 

Answer 1

feedparser entry.published_parsed always a entry.published_parsed temporary tuple regardless of the time input string. To get an object oriented datetime object:

 from datetime import datetime utc_time = datetime(*entry.published_parsed[:6], tzinfo=utc) 

where utc is a tzinfo object such as datetime.timezone.utc , pytz.utc or just your custom tzinfo (for older versions of python) .

You should not pass utc mktime() that is waiting for local time. Same error: Enter the correct time and time with the correct time zone .

Make sure USE_TZ=True so that django uses well-known datetime objects everywhere. Given a datetime object that supports a time zone, django should save it in db correctly, regardless of whether TIME_ZONE or timezone.get_current_timezone() is equal .

Answer 2

Have you tried using datetime.utcfromtimestamp() instead of datetime.fromtimestamp() ?

As a secondary solution, you can get unparsed data (I believe it is available as entry.published ?) And just use python-dateutil to parse the string and then convert it to pytz.utc timezone like this.

 >>> import pytz >>> from dateutil import parser >>> dt = parser.parse('2016-09-21T16:00:00+02:00') >>> dt datetime.datetime(2016, 9, 21, 16, 0, tzinfo=tzoffset(None, 7200)) >>> dt.astimezone(pytz.utc) datetime.datetime(2016, 9, 21, 14, 0, tzinfo=<UTC>) 

Answer 3

Using

 published_time = pytz.utc.localize(datetime.utcfromtimestamp(calendar.timegm(parsed_entry.published_parsed))) 

Feedparser can analyze a wide range of date formats, you can find them here .

As you can see in feedparser/feedparser/datetimes/__init__.py , the built-in function from Feedparser _parse_date does the following:

Parses many date formats into 9-bit set in GMT

This means that in parsed_entry.published_parsed you have a time.struct_time object in GMT time zone.

When you convert it to a datetime object using

 published_time = datetime.fromtimestamp(mktime(parsed_entry.published_parsed)) 

The problem is that mktime assumes that the transmitted tuple is in local time, and this is not GMT / UTC! Also, you are not correctly locating the datetime object at the end of the conversion.

You need to replace this conversion with the following, remembering that Feedparser returns GMT struct_time and localizes it using the time zone you like (UTC for simplicity).

• You use calendar.timegm , which gives the number of seconds between the epoch and the date passed as a parameter, provided that the passed object is in UTC / GMT format (we know from Feedparser).
• You use utcfromtimestamp to get a naive datetime object (which, as we know, is datetime in UTC, but Python is not working at the moment)
• With pytz.utc.localize you correctly localize a datetime object in UTC.

Example:

 import calendar from datetime import datetime import pytz localized_dt = pytz.utc.localize(datetime.utcfromtimestamp(calendar.timegm(parsed_entry.published_parsed))) 

As long as you agree, it doesn't matter if you fromtimestamp or utcfromtimestamp . If you are using fromtimestamp , you need to tell Python that the created datetime object has a local time zone. If you are in Europe / Berlin, this is also great:

 pytz.timezone('Europe/Berlin').localize(datetime.fromtimestamp(calendar.timegm(parsed_entry.published_parsed))) 

If parsed_entry.published_parsed in the local time zone, use mktime instead of calendar.timegm .

Alternatively, you can parse the data string that you get from Feedparser parsed_entry['published']

 from dateutil import parser localized_dt = parser.parse(parsed_entry['published']) 

You can verify that the following returns True :

 parser.parse(parsed_entry['published']) == pytz.utc.localize(datetime.utcfromtimestamp(calendar.timegm(parsed_entry.published_parsed))) 

Django’s TIME_ZONE setting doesn't really matter, as it is used only for visualization purposes or for automatically converting naive datetimes.

When USE_TZ is True, this is the default time zone that Django will use to display data in templates and to interpret dates entered on forms.

It is important to always use the correct localized dates, no matter what time zone is used. Until they are in a naive format, they will be properly handled by Django.