Problem/Motivation

#2739290: UTC+12 is broken on date only fields changed the way date-only fields work. This broken the default value handling when the UTC date is different than the user's current date.

To reproduce

- add a date-only field to a node, set the default value to now
- create a node when your date is different than the date in London, England.
- note that the default value is tomorrow for you, but correct for UTC

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mpdonadio created an issue. See original summary.

mpdonadio’s picture

Status: Active » Needs review
FileSize
1.92 KB

Here is a fix that works for me now on the node/add page and DateTimeFieldTest passes locally (it is 2016/08/05 locally and 2016/08/06 UTC). Not sure if it will still work in the morning when my date is the same as UTC.

Not sure if this is even really the correct fix.

Not sure if our coverage of this in DateTimeFieldTest::testDefaultValue() is correct all the time. Hesitant to change that w/o being able to do a proper test-only patch, but I don't know how we can do this without properly mocking DrupalDateTime objects in WebTestBase. I did a quick test to see if this would work in BrowserTestBase, but there are still some missing methods for a quick conversion (mainly the xpath stuff).

mpdonadio’s picture

#2 had some cruft from hashing this out. This is better.

mpdonadio’s picture

OK, I was right. Patch appears to still work for me in the morning, but the local DateTimeFieldTest failed when checking default values.

The last submitted patch, 2: 2778083-02.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 3: 2778083-03.patch, failed testing.

gambry’s picture

Patch at #3 works for me too, tested manually with different time zones (Europe/London, Australia/Sydney, Pacific/Auckland) at 00:01, 12:01, 23:01.

Should $expected_date = new DrupalDateTime('now', DATETIME_STORAGE_TIMEZONE); on DateTimeFieldTest.php:712 and 745 be relative to user's time zone?

UPDATE: Or at least - as per #2627512 comment #84 option 3 - keeping DATETIME_STORAGE_TIMEZONE but adding the offset between DATETIME_STORAGE_TIMEZONE and user's TZ.

mpdonadio’s picture

Status: Needs work » Needs review
FileSize
12.32 KB
10.46 KB

Let's adjust the test, and add the timezone looping. Still want to figure out how reliably demonstrate this for a test-only patch.

gambry’s picture

As we can't alter the request time, the only way possible is probably what I've been doing manually and - from what I can see - you are already doing on the test: running the checks with a -12 and a +12 TZs and at least one of them will always fail the assertEqual()s for the default date.

I ran the updated test without the patch and it fails tests for Pacific/Kwajalein, Pacific/Funafuti and Pacific/Tongatapu.
With the patch everything passes.

mpdonadio’s picture

#9, it's little worse than request time. If you read DateTimeFieldItemList::processDefaultValue(), the string 'now' or what the user enters gets passed as the first argument to the DrupalDateTime() constructor. If you continue to trace this out, that value eventually gets passed to the \DateTime constructor. This means it is current time, not request time, is the basis for that.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

gambry’s picture

@mpdonadio I didn't say (neither mean) the problem is the request time. What I meant is the most reliable way to demonstrate this properly would be to manipulate the time values to force the test to fail where and how we want, but this is impossible at the moment (it will be when/if introducing Clock Mocking).

So the only way possible I see is to test a wider range of TZs. Without the patch test will always fail.

Do you think there could be a better solution?

jhedstrom’s picture

If you continue to trace this out, that value eventually gets passed to the \DateTime constructor. This means it is current time, not request time, is the basis for that.

Since using 'now' goes all the way back to PHP, perhaps we don't need to actually have the test use 'now', but rather the request time plus an offset? Meaning, we can assume that 'now' will always work in a similar fashion as something like REQUEST_TIME + 5 hours, since all that logic is contained in PHP itself.

mpdonadio’s picture

#13, 'now' and REQUEST_TIME aren't always the same thing, especially with integration tests when you are installing modules, rebuilding caches, etc.

I think three followups are needed as we move into 8.3.x,

1. Get #2717207: Add a time service to provide consistent interaction with time()/microtime() and superglobals into core.
2. Weave the time service through core as much as possible so we can have reliable testing, potentially with the test bases using a override so they can hardcode the time.
3. Update as many datetime related tests as possible to use the timezone loop.

jhedstrom’s picture

'now' and REQUEST_TIME aren't always the same thing, especially with integration tests when you are installing modules, rebuilding caches, etc.

For the purposes of this issue though, is it specific to a default value of 'now', or is the bug with all default values? If the later, then there shouldn't be a need to test with a default value of 'now', but something close, which would be REQUEST_TIME, or REQUEST_TIME plus some fixed amount of time...

mpdonadio’s picture

Issue tags: -Needs tests
class DateTimeFieldItemList extends FieldItemList {

  /**
   * Defines the default value as now.
   */
  const DEFAULT_VALUE_NOW = 'now';

Not sure what we can do about that; it will eventually end up going to the \DateTime constructor. The patch in #8 adds the timezone loop, so I am as confident as I can be that we have adequate test coverage; we just can't demonstrate this with a test-only patch b/c the 'now' complication.

Removing the test tag. I think we can do final review on this.

jhedstrom’s picture

Status: Needs review » Reviewed & tested by the community

re #16 aaaah, I get it now. I thought this only referred to a test using 'now', forgetting we had this constant at all.

I think this is good to go. It's easiest to review when applied and do a diff ignoring white space.

YesCT’s picture

just fixed some 80 char comment line wrapping since some comment indents changed.
(it does make git diff 8.3.x --ignore-space-change a little less clean)

Also a test only patch.

YesCT’s picture

Title: Default value for Date-Only fields is broken » Default value for Date-Only fields is broken when UTC date is different than user current date
FileSize
12.34 KB
1 KB

another nit, https://www.drupal.org/coding-standards#array last time in an array, should have trailing comma

(these were already changed lines in the patch)

The last submitted patch, 18: 2778083-18-test-only.patch, failed testing.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 19: 2778083-19.patch, failed testing.

mpdonadio’s picture

Status: Needs work » Needs review

Back to NR b/c spurious fails.

YesCT’s picture

@mpdonadio Does that mean committing this would commit a random fail? Is it depending on the date(line) of the testbot? Looking...
https://www.drupal.org/pift-ci-job/425785
Ah, those look like from StableTemplateOverrideTest and TwigTransTest
So not date related.

mpdonadio’s picture

#23, yeah, the spurious was not date related and was similar to ones that pop up here and there.

One of the things done in this patch is to introduce the "timezone loop" to make sure the new code gets tested at a range of zones, including extremes at both ends. This method of testing date/time related code has had an impact on tracking down problems, and also should reduce or prevent fails (or passes) that depend on when patch is done. A hope for 8.3.x, is to weave this pattern more through all of date/time testing so we can see if we have any lingering timezone related issues.

jhedstrom’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC. Thanks for the cleanup!

xjm’s picture

Version: 8.3.x-dev » 8.2.x-dev

As a bugfix with no disruption, API additions, or internal BC breaks, this issue should be eligible for 8.2.x I think. (As far as I understand it is not an issue in 8.1.x.)

mpdonadio’s picture

Yeah, this got auto bumped to 8.3.x. This is a followup to #2739290: UTC+12 is broken on date only fields, which went into 8.2.x, and I see no problem with 8.2.x for this. This should not be applied to 8.1.x though. The two RTBC datetime patches that would be 8.2.x eligible should not cause the other to need a re-roll, either.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 19: 2778083-19.patch, failed testing.

The last submitted patch, 18: 2778083-18-test-only.patch, failed testing.

gambry’s picture

Status: Needs work » Reviewed & tested by the community

Back to RTBC after spurious fail.

xjm’s picture

In general rather than just saying something is a "spurious fail" we should document what it is and the existing critical to address it is, because random test failures are critical bugs somewhere in core or with DrupalCI. I get nervous when I see this many random fails in a short time on a patch, because it is possible to introduce failures in places people don't expect.

One fail I see was: https://www.drupal.org/pift-ci-job/435269:

Update.Drupal\views\Tests\Update\FieldHandlersUpdateTest
✗	
runUpdates
fail: [Other] Line 264 of core/modules/system/src/Tests/Update/UpdatePathTestBase.php:
Schema key views.view.files:display.default.display_options.cache failed with: missing schema

That is a potential fail when something happens to the test environment so that data is straight up missing from the bot, but also can happen if something is added without a schema. While this issue does touch some settings that would be reflected in schema, it does not have anything to do with the files view. Fails like this are a common result of #2791163: Random automatic testing failures on SQLite with PHP 5.5 and I think that issue may actually not only be about SQLite. So this issue's patch is almost certainly fine and hopefully fails like these will be less common in the future. :)

alexpott’s picture

Status: Reviewed & tested by the community » Patch (to be ported)

Committed 6992cbe and pushed to 8.3.x. Thanks!

Setting to patch to be ported for eventual cherry-pick to 8.2.x once 8.2.0 is released.

  • alexpott committed 6992cbe on 8.3.x
    Issue #2778083 by mpdonadio, YesCT: Default value for Date-Only fields...
alexpott’s picture

Status: Patch (to be ported) » Fixed

Committed 635c476 and pushed to 8.2.x. Thanks!

  • alexpott committed 635c476 on 8.2.x
    Issue #2778083 by mpdonadio, YesCT: Default value for Date-Only fields...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.