Dates and dateranges - core handling, storage, and planning for the future

Drupal 8 has been a difficult experience for any website requiring dates and calendars. And with Drupal 9 being just around corner, we may face an extended period of difficulty unless this is resolved once and for all.

Everyone effected knows this, but efforts to coordinate seem to be fragmented and short of resources.

For now, I am sticking with date_recur and its complementary widget modules, as this is the most usable solution.

Thanks for raising this. I had originally intended for Smart Date to be only a widget, to be used with the core Date Range module, but decided to make it a completely new field type after observing the impact on performance of having the values stored as strings. You may also want to refer to #2366213: Date field doesn't use database-native date storage and the other issues that it, in turn, refers to, as they give a good sense of the discussions that brought us to the current state.

For some reason I can't seem to set the original storage discussion directly as a related issue. I wanted to, because that's where the decision was made to only support dates stored as varchar, in an ISO standard format.

A couple of other clarifications:

I completely agree that core should support additional date storage options, especially the native datetime storage fields. I understand the argument about complexity, but we've pushed the problem downstream, so contrib authors are trying to find their own solutions. I understand and appreciate the idea of trying to keep an option open for sites using an RDBMS that doesn't have a native datetime field, but that's why we should leave the current storage as an option, but also let site builders that don't need the portability choose an option that is specifically designed and optimized for the very problem they're trying to solve: efficient handling of dates and time. Honestly, if there's one thing I would like to see Drupal 9 get right, it's this.

Let's be honest, the argument about simplification is oversold. Most contrib modules that are designed to handle arbitrary date fields already have to anticipate dealing with varchar/ISO date fields, or timestamps, as both are in core. The idea that it's somehow an unreasonable burden to also support a third option simply doesn't ring true for me.

Been out of the loop for a while now, but the thing that has been keeping me from core dev is starting to wrap up.

Datetime storage has a long history, dating back to the Drupal 7 Date module, and the first patch to get datetime into core, as noted above.

Timestamp fields were really supposed to be internal-only fields (or base fields). I am not terribly happy that they got put back into the UI, as it does cause fragmentation and code duplication.

There is a longstanding issue, #2716891: DateTimeIso8601::setDateTime() vs. DateTimeItem::schema(): (at least) one of them is broken, to update storage to reflect the UTC offset as part of storage (note that the +HH:MM portion is not the timezone, it is the offset from UTC for that particular timestamp which takes into account the timezone for a particular place and any seasonal/religious adjustments). The fix itself is easy; the update path is a beast. There is also an issue to store the IANA TZ name in storage along with the ISO8601 string.

Personally, I am against letting users pick storage methods like they could in D7. But, I think looking into backend specific storage is good, as long as the API advertises something consistent. I think there are three parallel paths that can be taken.

1. Look into the DB-specific storage options, with varchar as the default.

2. Improve the FieldItem API for datetime, similar to what the REST normalizers do, so setting values can be flexible and internally convert to the storage format, and getting values will convert to the full ISO8601 string. This is a potentially stopgap until #2716891: DateTimeIso8601::setDateTime() vs. DateTimeItem::schema(): (at least) one of them is broken gets figured out.

3. Encourage people, better document, and potentially improve API and/or internals, to get people to use the computed DDT object instead of the value string.

Adding some related issues:
#3064357: Join forces and reduce duplicate effort? is where I'm discussing smart_date with the maintainer and trying to increase collaboration and reduce fragmentation.
#3064640: Provide alternate storage backends for native DB date fields is about trying to support DB-native date storage for core datetime and daterange fields inside datetime_extras (as proof-of-concept for something we could hopefully move into core).

Cheers,
-Derek

@mandclu: Re: #8 -- that's because the original discussion you're linking is on groups.drupal.org, not an issue on d.o, so it can't be officially listed as a "related issue". At least #2366213: Date field doesn't use database-native date storage is already in the list.

Big + 1

Is this still alive? Did the discussion move somewhere else?

I'd say not much has changed, except for Smart Date rising in popularity and being included in Drupal CMS.

A big hangup is module support for Datetime vs Smart Date vs others like Recurring Dates Field, Year Only, etc. So I doubt core will change any time soon.

Defaulting to using Smart Date is a relatively safe bet. Otherwise, you still have to experiment to find the right combination of date storage, widgets, and module support.

Comment #6 still stands. I don't know if anything has changed on the database front to make a difference on storing timestamps vs native date fields.

If I use Smart Date, what options do I have for producing an iCal feed, or CalDAV?