Problem/Motivation

The DateTimeIso8601::getDateTime() method on the @DataType=datetime_iso8601 plugin incorrectly constructs a \Drupal\Core\Datetime\DrupalDateTime object: it forgot to specify the timezone of the stored value (which is always UTC).

In Drupal core, there are zero calls to DateTimeIso8601::getDateTime(). Only tests are calling it. It's thanks to the API-First Initiative adding a normalizer to correctly and consistently normalize datetime information that we discovered this at all.

Proposed resolution

Start specifying the timezone!
Expand test coverage: the existing test coverage was a low-level kernel test which specifically had a timezone offset specified (which was done explicitly for testing the TypedData aspects), but in reality datetime strings are stored in the database without timezone offsets (see @joelstein in #2926508-132: Add DateTimeNormalizer+TimestampNormalizer, deprecate TimestampItemNormalizer: @DataType-level normalizers are reusable by JSON:API).

Remaining tasks

Review.

User interface changes

None.

API changes

None. The \Drupal\Core\Datetime\DrupalDateTime instance returned by \Drupal\Core\TypedData\Plugin\DataType\DateTimeIso8601::getDateTime() now has the appropriate timezone set.

Data model changes

None.

BC implications

This patch changes nothing about current or future stored values. This only changes the mapping of stored values into DrupalDateTime objects, which almost no code does (in core, only tests do this).

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Wim Leers created an issue. See original summary.

Wim Leers’s picture

Status: Active » Needs review
FileSize
1.97 KB

This should pass tests, but it doesn't.

Wim Leers’s picture

This is the solution.

Wim Leers’s picture

Assigned: Unassigned » mpdonadio

Assigning to @mpdonadio and @jhedstrom for review.

The last submitted patch, 2: 3002164-2.patch, failed testing. View results

mpdonadio’s picture

+++ b/core/lib/Drupal/Core/TypedData/Plugin/DataType/DateTimeIso8601.php
@@ -23,10 +24,10 @@ class DateTimeIso8601 extends StringData implements DateTimeInterface {
-        $datetime = DrupalDateTime::createFromArray($this->value);
+        $datetime = DrupalDateTime::createFromArray($this->value, DateTimeItemInterface::STORAGE_TIMEZONE);
       }
       else {
-        $datetime = new DrupalDateTime($this->value);
+        $datetime = new DrupalDateTime($this->value, DateTimeItemInterface::STORAGE_TIMEZONE);

This couples core to the datetime.module, so maybe explicit 'UTC' w/ a comment?

The new assertion looks good, but I think we also need this in TypedDataTest. The problem there is that

    // Date Time type.
    $value = '2014-01-01T20:00:00+00:00';
    $typed_data = $this->createTypedData(['type' => 'datetime_iso8601'], $value);
    $this->assertTrue($typed_data instanceof DateTimeInterface, 'Typed data object is an instance of DateTimeInterface.');
    $this->assertTrue($typed_data->getValue() == $value, 'Date value was fetched.');
    $this->assertEqual($typed_data->getValue(), $typed_data->getDateTime()->format('c'), 'Value representation of a date is ISO 8601');
    $this->assertEqual($typed_data->validate()->count(), 0);
    $new_value = '2014-01-02T20:00:00+00:00';
    $typed_data->setValue($new_value);
    $this->assertTrue($typed_data->getDateTime()->format('c') === $new_value, 'Date value was changed and set by an ISO8601 date.');
    $this->assertEqual($typed_data->validate()->count(), 0);
    $this->assertTrue($typed_data->getDateTime()->format('Y-m-d') == '2014-01-02', 'Date value was changed and set by date string.');
    $this->assertEqual($typed_data->validate()->count(), 0);
    $typed_data->setValue(NULL);
    $this->assertNull($typed_data->getDateTime(), 'Date wrapper is null-able.');
    $this->assertEqual($typed_data->validate()->count(), 0);
    $typed_data->setValue('invalid');
    $this->assertEqual($typed_data->validate()->count(), 1, 'Validation detected invalid value.');
    // Check implementation of DateTimeInterface.
    $typed_data = $this->createTypedData(['type' => 'datetime_iso8601'], '2014-01-01T20:00:00+00:00');
    $this->assertTrue($typed_data->getDateTime() instanceof DrupalDateTime);
    $typed_data->setDateTime(new DrupalDateTime('2014-01-02T20:00:00+00:00'));
    $this->assertEqual($typed_data->getValue(), '2014-01-02T20:00:00+00:00');
    $typed_data->setValue(NULL);
    $this->assertNull($typed_data->getDateTime());

all of those date strings have a UTC offset in them, which implicitly sets them to UTC (kinda, see https://3v4l.org/jctWg). When you parse a string w/ an offset that will override the time zone in the constructor.

Not totally sure what is best here.

Wim Leers’s picture

This couples core to the datetime.module, so maybe explicit 'UTC' w/ a comment?

That's exactly what I contemplated too. I sort of hoped you would say this :D Not sure what the comment should look like. Whatever you prefer there I'm probably fine with.

all of those date strings have a UTC offset in them, which implicitly sets them to UTC (kinda, see https://3v4l.org/jctWg). When you parse a string w/ an offset that will override the time zone in the constructor.

Indeed. For that reason, that test coverage doesn't reflect the reality of actually executed code.

Not totally sure what is best here.

I think I know: just add additional test coverage with a non-timezone-annotated value. The

$this->assertEqual($typed_data->getValue(), $typed_data->getDateTime()->format('c'), 'Value representation of a date is ISO 8601');

assertion does not pass for this new test coverage, which again suggests that @DataType=datetime_iso8601 was always meant to include timezone information. Solving that is out of scope here though; this issue is about making the bare minimum change to fix the problem at hand; #2716891: DateTimeIso8601::setDateTime() vs. DateTimeItem::schema(): (at least) one of them is broken exists for restoring that assertion.

The last submitted patch, 7: 3002164-7-test_only_FAIL.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 7: 3002164-7.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Wim Leers’s picture

Status: Needs work » Needs review
Issue tags: +Needs reroll

The non-test-only patch did in fact pass tests. It just has some CS violations.

Back to NR.

Wim Leers’s picture

Issue tags: -Needs reroll
FileSize
2.52 KB
8.04 KB

Done.

Wim Leers’s picture

mpdonadio’s picture

Status: Needs review » Reviewed & tested by the community
FileSize
3.1 KB
+++ b/core/tests/Drupal/KernelTests/Core/TypedData/TypedDataTest.php
@@ -153,11 +155,45 @@ public function testGetAndSet() {
+    $typed_data->setDateTime(new DrupalDateTime('2014-01-02T20:00:00'));
+    $this->assertSame('+11:00', $typed_data->getDateTime()->getTimezone()->getName());

This had me scratching my head for a while. The default timezone for tests is 'Australia/Sydney', which happens to have a UTC offset of +11:00 on 2014-01-02.

Let's look closer

class DateTimeIso8601 extends StringData implements DateTimeInterface {

  public function getDateTime() {
    if ($this->value) {
      if (is_array($this->value)) {
        // Data of this type must always be stored in UTC.
        $datetime = DrupalDateTime::createFromArray($this->value, 'UTC');
      }
      else {
        // Data of this type must always be stored in UTC.
        $datetime = new DrupalDateTime($this->value, 'UTC'); // NOTE 2
      }
      return $datetime;
    }
  }

  public function setDateTime(DrupalDateTime $dateTime, $notify = TRUE) {
    $this->value = $dateTime->format('c'); // NOTE 1
    // Notify the parent of any changes.
    if ($notify && isset($this->parent)) {
      $this->parent->onChange($this->name);
    }
  }

}

The test does `$typed_data->setDateTime(new DrupalDateTime('2014-01-02T20:00:00'));`. The TZ of the anonymous object passed to `setDateTime()` will be 'Australia/Sydney', per PHP's rules. Set a breakpoint to see.

At "NOTE 1", `$this->value` gets set to '2014-01-02T20:00:00+11:00', which stores the timezone offset and not the timezone name in the string.

At "NOTE 2", we build a new `DrupalDateTime()` from the string '2014-01-02T20:00:00+11:00'. The timezone name gets set to '+11:00' per PHP's rules, not 'Australia/Sydney'. This then is what is in the assertion.

To be honest, I don't think we can really do anything about this. Using '+00:00' in `getDateTime()` smells bad. If we want to be pedantic, then I think the attached interdiff may be best. I am not wed to that though.

My issues about in #6 are addressed to my satisfaction (and sorry for not being more specific about what "kinda" meant).

Wim Leers’s picture

Woot!

Thank you for being so diligent! 👌

Wim Leers’s picture

Woot!

Thank you for being so diligent! 👌

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs issue summary update, +Needs change record

Does this work have an BC implications. What about existing code?

I think we need a change record to detail the change and the issue summary could be updated to be more explicit about why this change is correct rather than pointing to lots of other issues to work out why.

+++ b/core/tests/Drupal/KernelTests/Core/TypedData/TypedDataTest.php
@@ -153,11 +155,45 @@ public function testGetAndSet() {
+    $typed_data->setDateTime(new DrupalDateTime('2014-01-02T20:00:00'));
+    $this->assertSame('+11:00', $typed_data->getDateTime()->getTimezone()->getName());
+    $this->assertEqual($typed_data->getValue(), '2014-01-02T20:00:00+11:00');

Given this had a datatime maintainer scratching their head maybe we should add a comment for future us.

Wim Leers’s picture

Assigned: mpdonadio » Unassigned
Issue summary: View changes
Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs issue summary update, -Needs change record

Issue summary updated, change record created.

Wim Leers’s picture

Oh, Given this had a datatime maintainer scratching their head maybe we should add a comment for future us. — did that too.

Wim Leers’s picture

Issue summary: View changes
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

I committed this to 8.7.x - as I'm unsure of the implications for contrib if they have used this to manipulate and then store values I'm going to leave there. If there are compelling reasons to backport then we can discuss and do that.

Committed f51bfe1 and pushed to 8.7.x. Thanks!

  • alexpott committed f51bfe1 on 8.7.x
    Issue #3002164 by Wim Leers, mpdonadio: DateTimeIso8601::getDateTime()...
Wim Leers’s picture

#2999438: [upstream] Datetime field shown with wrong timezone offset just landed, which had to effectively simulate/backport this so that JSON:API responses in Drupal 8.6 and 8.5 are also correct. I don't expect core to cherry-pick this patch into the 8.6 and 8.5 branches, although I certainly would not oppose it either :)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.