Datetime fields and timezones in Plone 5.2

Plone Support Add-on Development
#1

What should the type of data be that is stored in a Datetime field in Plone 5.2?

  1. Zope DateTime?
  2. Python datetime with timezone information?
  3. Python datetime without timezone information?

Background: I have a site that I am migrating from Plone 4.3 Archetypes to Plone 5.2 Dexterity. In the resulting 5.2 site after migration, I am seeing all three versions of data in Datetime fields.

  1. The standard effective date field of both migrated and new items is Zope DateTime. This is actually what I expected everywhere.
  2. My custom migrated content has Python datetime with timezone information. The data of this field was migrated using migrate_datetimefield, following the documentation from plone.app.contenttypes.
  3. Newly created custom content has Python datetime without timezone information.

Naturally, this leads to various kinds of TypeErrors when I need to compare these three kinds of dates:

TypeError: can't compare datetime.datetime to DateTime
TypeError: can't compare datetime.datetime to long
TypeError: can't compare offset-naive and offset-aware datetimes

My field definition is this:

    start_time = schema.Datetime(
        title=_(u"label_start_time", default=u"Start"),
        required=True,
    )

What I now notice in plone.app.dexterity, is that the field definition of the effective date has this extra line:

directives.widget('effective', DatetimeFieldWidget)

This widget is from plone.app.z3cform.widget.DatetimeFieldWidget.
I guess this is what makes it a Zope DateTime.
[Update: wrong. Adding this directive to my field and creating a new instance, I still get Python datetime without timezone info.]

So is it best to use this widget for all Datetime fields in 5.2? (Except in probably rare edge cases.)
And should I adapt my migration to keep the Zope DateTime objects?

1 Like
#2

Always Python datetime with a timezone information. A datetime w/o timezone does not say anything.

Except for historical reasons sigh we have Zope DateTime around for effective/expired/created and modified. This should be changed but nobody stepped up. Migration is probably some effort.

#3

But how do I do that? With the simple start_time = schema.Datetime(title=u"Start") from above, I get a Python datetime without timezone info:

>>> context.start_time
datetime.datetime(2021, 1, 19, 16, 0)

This is Plone 5.2.3 on Python 2.7, tried both with and without setting a timezone in the dateandtime-controlpanel. I don't suppose on Python 3 I would suddenly get a timezone where on 2.7 I don't.

#4

from a current project (Plone 6 based)

from plone.app.event.base import default_timezone
...
    gesperrt_bis = schema.Datetime(
        title="gesperrt bis", description="behördliche Sperre", required=False
    )
    widget(
        "gesperrt_bis",
        DatetimeFieldWidget,
        default_timezone=default_timezone,
    )
    write_permission(gesperrt_bis="tfv.edit_most_member_data")
2 Likes
#5

Excellent, thanks! With that I get this for a new item:

>>> context.start_time
datetime.datetime(2021, 1, 14, 16, 0, tzinfo=<DstTzInfo 'Europe/Brussels' CET+1:00:00 STD>)

On a related note, what I already found was that you cannot compare this with datetime.now, but need a special function. This fails:

>>> from datetime import datetime
>>> datetime.now()
datetime.datetime(2021, 1, 13, 15, 57, 56, 319656)
>>> datetime.now() < context.start_time
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot compare offset-naive and offset-aware datetimes

This works:

>>> from plone.app.event.base import localized_now
>>> localized_now()
datetime.datetime(2021, 1, 13, 15, 59, 41, tzinfo=<DstTzInfo 'Europe/Brussels' CET+1:00:00 STD>)
>>> localized_now() < context.start_time
True
#6

To compare, the timezone itself should not matter as long both have a valid timezone, right? So a
datetime.datetime.now(datetime.timezone.utc) < context.start_time
should do it as well.

#7

One more opinion, always store datetimes with UTC timezone, and in a separate field store the user's local timezone as a preference for localization. JavaScript solutions (Moment, et al) that localize the stored datetime alone are annoying and fallible, especially while traveling.

#8

@stevepiercy I do not agree here. You can always convert timezones between each other. So it does not make any difference if you store UTC or any other official timezone.

#9

I should clarify that when storing the datetimes, one should use the user's preference timezone to convert the datetime to UTC going in to, as well as going out from, persistent storage.

In the US there are 6 timezones. The app I built uses datetime math to measure durations of events while accounting for daylight savings time (it handles 23-hour or 25-hour days, as well as the ambiguous datetimes that recur when we "fall back" for the 25-hour day—02:00:00-02:59:00—and the invalid datetime that does not occur when we "spring forward"). A customer may have multiple locations across timezones, and the only way to ensure that their computations are consistent is to do this conversion.

Another use case is when a user moves between timezones.

Datetime is a frequent topic at PyCon US. Here's one: Mario Corchero It's time for datetime PyCon 2017.

It is not absolutely necessary to store all datetimes in UTC only, but it is absolutely necessary to choose one consistent timezone for all users.

1 Like
#10

I know the problem. In my programmer life I wrote more than one calendar application spreading multiple timezones. Anyway, I agree with all you write, except the stored timezone does not matter to be UTC and similar for all users. It best just reflects the intended timezone of the editor of the data. Then all computations for output can be done from there. UTC is also just a timezone, so it really does not matter. Thus, a second field is not needed. While creating plone.app.event we had this discussion in depth and in fact this was the conclusion. In between we had indeed an implementation with timezone stored separate, but it made all more difficult so we ditched it.

Now the killer argument: If it comes to recurring events you really need to store a date with the original editors timezone. Imagine a user in some US timezone schedules a daily event at 10am DST. Now it is converted to UTC. UTC has no DST. All calculations are done on the stored date, the information of the intended timezone is lost. Now, if in autumn we no no longer have DST or vice versa in spring, one day need to have 23 or 25 hours added for recurring dates. With UTC it is impossible to know when, the date of change in US is different from Europe or other parts in the world. Thus, timezone is essential information of a date. For the user the output date can be always calculated to reflect the its timezone, so weekly at 10am might mean mostly at 15pm for someone else, but at some weeks at 14pm. This can only be done at output rendering, never by unifying the input to an arbitrary timezone.

4 Likes