Comments

Pancho created an issue. See original summary.

pancho’s picture

Status: Active » Needs work
StatusFileSize
new3.89 KB
new730 bytes
new374 bytes

A hot candidate might be #3051865: Email handler ignores selected theme.

Let's try three patches:

pancho’s picture

What do we learn from this?

  1. The placebo failed, so indeed our branch is currently broken on PHP5.
  2. #3051865: Email handler ignores selected theme fixes a test so it actually works (and fails, because the underlying functionality is broken), but it obviously didn't add any new regression.

We obviously need to dig deeper.

pancho’s picture

StatusFileSize
new1.63 KB

I traced this back to 5.0.0-rc27 (November 2018) where WebformBrowserTestBaseTest was introduced. So it seems this bug has existed for quite some time, and only came to the light with the new test coverage.

Fortunately it seems this is mostly a test artefact, as the tests are hardcoding a particular order of emails, while for some reason, emails are sent in a divergent order:

  • On PHP 7.2, the 'webform_contact_email_confirmation' is always sent first, the 'webform_contact_email_notification' comes last
  • On PHP 5.6, the two emails are always sent just the other way around.

I tweaked WebformBrowserTestBase::getLastEmail() a bit, so I got to see more than just the very last email.

I'm also fixing the argument order of two assertEquals() calls. The expected value needs to be first.

This however doesn't fix the test failure here, and I'm not trying to hide it. We still have to fix the order emails are sent out.

pancho’s picture

Status: Needs work » Needs review
StatusFileSize
new5.17 KB
new2.96 KB
new3.67 KB

OK, I think I found it, and it was very simple:
The email_confirmation and email_notification handlers in the yml files both have a weight of 1, so while they are not called in an outright arbitrary order, the order seems to be determined by hardly predictable circumstances.

Enclosed patch tests green locally, both under PHP 5.6 and PHP 7.2.
tests-only patch should fail.

pancho’s picture

Title: 8.x-5.x-dev is broken for PHP5 » Unpredictable order of confirmation and notification mails breaks tests on PHP5
StatusFileSize
new7 KB
new4.09 KB
new5.5 KB

Yay!

Here's just another iteration avoiding code duplication on WebformHandlerEmailRenderingTest::testEmailRendering(). I think it's easier to grasp like that, but feel free to pick whatever you want from here.

pancho’s picture

jrockowitz’s picture

There is too much going on in this patch. We should not have reorder how emails are being sent to pass a test. I am going to turn on automatted tests for PHP5.

pancho’s picture

We should not have reorder how emails are being sent to pass a test.

In that case, the simple alternative is:
rather than only reading the very last email when testing, read the last two emails, no matter in which order they were sent, figure our which is which, and run the test on the correct one.

However, from my point of view, this is not just about fixing our tests. We need to ensure predictability, otherwise not only our tests will fail, but also custom or contrib code's expectations.

There is too much going on in this patch.

Sorry about that. Once we figured whether to fix our default config or just fix our tests, I will provide a patch that is stripped down to the minimum necessary.

jrockowitz’s picture

StatusFileSize
new18.3 KB

I think fixing the handler weight in the webform.contact.yml should fix the issue.

jrockowitz’s picture

StatusFileSize
new1.29 KB

Sorry please ignore the patch from #10.

pancho’s picture

Since recently, D8.8 doesn't support PHP 5.5 anymore. Only D8.6 and D8.7 do.

  • jrockowitz committed 76bd896 on 8.x-5.x
    Issue #3058899 by Pancho, jrockowitz: Unpredictable order of...
jrockowitz’s picture

I committed the patch from #11 which should fix the PHP5 issue.

jrockowitz’s picture

Status: Needs review » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.