Problem/Motivation

Currently, WebTestBase::setUp() does not set a system timezone. This can cause several problems. Once, it means that Drupal will run tests in the context of the default system timezone. On TestBot, this will be UTC (citation required). On local system, it can vary widely. MAMP sets the default timezone to 'Europe/Berlin', which adds to the confusion, especially for those who don't live in UTC+1. The net results is that integration tests will run in a more consistent way, regardless of environment.

Proposed resolution

Add a timezone near the extreme to catch edge cases in WebTestBase::setUp(), and then fix any tests that break.

Remaining tasks

Do it.

User interface changes

N/A

API changes

N/A

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Task because we choose and change the default timezone used by simpletests, thus improving the test suite.
Issue priority Normal because there is nothing broken at the moment.
Unfrozen changes Unfrozen because it only changes tests, more specific it changes the default timezone that all simpletests use and fixes the test failures the change causes, thus improving the tests.
Prioritized changes While not in the list of prioritized changes, the purpose of this is to prevent TZ related regressions and reduce the fragility of the testing system in general, where there are cases when tests pass or fail depending on when they run of where they run (TestBot vs Local).
Disruption None
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mpdonadio’s picture

Lets see what breaks.

xjm’s picture

Status: Active » Needs review

Whee!

cilefen’s picture

Status: Needs work » Needs review
FileSize
516 bytes
645 bytes

+1 for this for making things make sense. Why not use UTC?

cilefen’s picture

The patch in #1 proved this is a real issue. Some tests rely on being executed close to UTC.

Status: Needs work » Needs review
cilefen’s picture

Ouch!

mpdonadio’s picture

We run TestBot outside of the root directory to make sure we don't have any code that accidentally relies on or only works when the base URL is /. Choosing a TZ at the far edge is similar. Dates in are stored as UTC in the database. By choosing something close to the extreme, you will expose bugs that assume UTC b/c you often have day rollover when the TZ is applied to the value pulled from the database. Thought would need to go into whether a large positive value or a large negative value make more sense for finding potential bugs.

I think the proper solution is to pick a non-UTC zone, and then fix the tests (or code) that relies on UTC.

cilefen’s picture

@mpdonadio Exactly - that makes a lot of sense. I started to understand in #5.

borisson_’s picture

Status: Needs work » Needs review
FileSize
1010 bytes
1.62 KB

This makes the ConfigTranslationUiTest green. I haven't looked at the other tests yet because I'd like to wait and check if this is a good solution.

Status: Needs review » Needs work

The last submitted patch, 12: simpletest_should_set_a-2497585-12.patch, failed testing.

mpdonadio’s picture

+++ b/core/modules/config_translation/src/Tests/ConfigTranslationUiTest.php
@@ -465,8 +465,10 @@ public function testDateFormatTranslation() {
-      $formatted_date = format_date(494015820, $id, NULL, NULL, 'fr');
+      $formatted_date = format_date(494015820, $id, NULL, 'Europe/Berlin', 'fr');
       $this->assertEqual($formatted_date, 'Tue', 'Got the right formatted date using the date format translation pattern.');

Not sure if this is correct? How did you arrive at the TZ for Berlin? Did you pick it or is that what happened to be in the dated b/c is is the default in MAMP? I think that TZ in particular should be avoided b/c it is the MAMP default.

mpdonadio’s picture

Status: Needs work » Needs review
FileSize
2.26 KB
1.63 KB

Based off of the comments in ConfigTranslationUiTest and LocaleConfigTranslationTest, I think this is the correct fix. However, I am not really sure what the intent of those tests are because Tuesday in French is mardi (or at least that is what the Google machine tells me), so I don't know why the French langcode is passed in but the English text is the expectation.

Haven't looked at the other fails yet.

I'm also expecting this to fail periodically in the Datetime tests, depending on when the test actually runs. Later today, I am also going to post two testonly patches: one with a TZ with a large positive offset and one with a large negative one to see what happens. But, I am hoping to find a table that matches up the PHP zone names with their UTC offsets instead of trying to guess which will be closest to the international date line.

Status: Needs review » Needs work

The last submitted patch, 15: simpletest_should_set_a-2497585-15.patch, failed testing.

mpdonadio’s picture

Ok, did some poking around. It looks like the dates in the RDF tests are being created in the host system's time zone (for me, UTC-4). The expectation in the test is using the zone we set in WebTestBase::setUp (UTC+9). Didn't figure out if it is an Easy_RDF problem bypassing the zone that WebTestBase creates, or if some comment code is actually executing in the host system's container and not the test system's container.

Also note that #15 had a fail in the datetime tests that wasn't present in others. My work in #2399211: Support all options from views fields in DateTime formatters has a fix for this.

davidhernandez’s picture

Aha! The Drupal\datetime\Tests\DateTimeFieldTest fail is exactly the problem I was having on another issue. The date rolls over/under, so it thinks it is 12-30 instead of 12-31 or vice versa. The format itself is correct.

tstoeckler’s picture

  1. In general this patch is awesome, yay!
  2. bootstrap.php that is used for PHPUnit tests sets the timezone for similar reasons except that it sets it to UTC. I get the reasoning here why you went with a different value but perhaps we should open a follow-up then to change bootstrap.php to the same value.
  3. I don't think it's a great idea to use an explicit time zone in the actual tests which do not actually test anything time-related. Was that done because the date strings (e.g. "Tue") get translated? In that case I think it would be better to add a call to t() or similar.
jhodgdon’s picture

This is fascinating.

One small note: Regarding your question about the 'fr' translations, probably the result is 'Tue' because there are not French translation files being used, so there is no translation for the days of the week in the Drupal database.

Anyway, carry on! I'll be interested to see whether a time zone like Tokyo (one of the first to get to the new day) or western Alaska (one of the last) gives more interesting test failures. Obviously we should make sure to fix all the failures associated with either one... One interesting thing to do would be: Pick the time zone that causes the most fails. For any test that passes in that time zone and fails in the other time zone, override the time zone setting in that test's setUp() method, so it fails too. [hope that makes sense] ... maybe there could be a member variable for tihs setting, similar to $modules and other ones that the default class uses in its setUp(), to make this easy to override?

mpdonadio’s picture

#19-2: just made #2498619: Unit tests should use a default timezone other that UTC

#19-3: I agree. In the tests I fixed, I have no idea what that test is trying to achieve, though, other than it expecting what is stated in the comment above the lines that changed.

mpdonadio’s picture

mpdonadio’s picture

This needed a reroll. Also changed the TZ to Australia/Sydney, to mirror #2498619: Unit tests should use a default timezone other that UTC. This is an extreme, but not too extreme, TZ that participates in DST.

mpdonadio’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 23: simpletest_should_set_a-2497585-23.patch, failed testing.

Anonymous’s picture

FileSize
454.96 KB

After debugging the failure in CommentAttributesTest a bit, I noticed that the comment posted by the anonymous user gets the wrong timezone. Or it's not the one we provide (and thus expect) anyway.

That shouldn't be too hard to reproduce as to find the root cause, but I ran out of time for now.

jhodgdon’s picture

As on #2498619-37: Unit tests should use a default timezone other that UTC, when you set the time zone to Sydney, you should add a comment explaining why that was chosen.

mpdonadio’s picture

Status: Needs work » Needs review
FileSize
5.6 KB
3.91 KB

This should come up green.

The problem had to do with calling PHP's date() directly. In some of the fails, it was grabbing the simpletest time zone, in others it was grabbing the system time zone.

I updated the tests to use format_date, and Drupal\rdf\CommonDataConverter() to use \Drupal::service('date.formatter'), and both to force the timestamp to UTC (which shouldn't matter for this purpose, since the TZ offset it part of the output and will be interpreted by anything using it). Not sure if the CommonDataConverter() part would really be considered in scope, but we have a service for formatting timestamps, so we should use it instead of the low level function.

jhodgdon’s picture

Status: Needs review » Reviewed & tested by the community

It did come up green. All of the changes to tests and code look very reasonable to me, and the code comment looks good too. Let's get this in!

Anonymous’s picture

Issue summary: View changes

Added beta evaluation. I borrowed a lot from the one in #2498619: Unit tests should use a default timezone other that UTC.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed 794cf89 and pushed to 8.0.x. Thanks!

Thanks for adding the beta evaluation to the issue summary.

  • alexpott committed 794cf89 on 8.0.x
    Issue #2497585 by mpdonadio, cilefen, borisson_, pjonckiere: Simpletest...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Berdir’s picture

This doesn't work as expected, the problem is that user configurable timezone is enabled by default and the first users are created with timezone UTC, so the timezone is switching all the time: #2611082: Inconsistent time zones in web tests