Problem/Motivation

Our javascript tests don't work when using the latest Chromedriver versions (version 75 and higher) because it defaults to using the w3c protocol which is not supported by the packages that Behat/Mink require.

In phpunit.xml we can pass { "chromeOptions": { "w3c": false } } instead of null and the latest version of Chromedriver works.

Original debugging done by @tedbow and @zrpnr:
1. $page->find() never works for using css or xpath.
2. Somehow xml doesn’t parse drupal generate html.
3. $doc->loadXML($content); fails if this a request for a drupal page.
4. $doc->loadXML($content); does not fail if this is a request for static html file we know is valid

Optional workaround:
Don't use the recommended Chromedriver version, but use the 2.x version for now (it seems to work fine with new Chrome versions)

Proposed resolution

Update our instructions for running JS tests.

Remaining tasks

Update our instructions.

Release notes snippet

If you update to the latest Chromedriver (75+) version to run tests, you need to add { "chromeOptions": { "w3c": false } } to your MINK_DRIVER_ARGS_WEBDRIVER settings.

CommentFileSizeAuthor
#4 3073342-3.patch951 bytesLendude
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Lendude created an issue. See original summary.

Lendude’s picture

Lendude
Primary language Dutch
Location Amsterdam
CreditAttribution: Lendude as a volunteer and at ezCompany commented
Issue tags: +Needs manual testing

Haven't checked yet but per @zrpnr on slack:

in chrome 75 they changed "w3c standard" to default to true
https://bugs.chromium.org/p/chromedriver/issues/detail?id=2536&q=&colspe...

in phpunit.xml you can pass { "chromeOptions": { "w3c": false } } instead of null and the latest version of chromedriver should work

<env name="MINK_DRIVER_ARGS_WEBDRIVER" value='["chrome", { "chromeOptions": { "w3c": false } }, "http://drupal.test:4444/wd/hub"]' />

Adding the 'needs manual testing' tag since we can't force Drupal CI to update these things

Lendude credited zrpnr.

Lendude’s picture

Lendude
Primary language Dutch
Location Amsterdam
CreditAttribution: Lendude as a volunteer and at ezCompany commented
Status: Active » Needs review
FileSize
3073342-3.patch951 bytes

Updated my Chromedriver version from 2.x to 76.x.

This fails all JS tests when not adding the chromeOptions. Everything works as expected when adding the options.

Background:
https://medium.com/@alex.designworks/chromedriver-75-enforces-w3c-standa...

TL;DR
instaclick/php-webdriver does not support “w3c” protocol.

Adding a patch to update the phpunit.xml.dist to give the correct new setup. Also changed the default to Chrome with the explanation of how to change to Firefox because that looked clearer to me then adding more instructions about what you need to change when using the other browser.

Lendude’s picture

Lendude
Primary language Dutch
Location Amsterdam
CreditAttribution: Lendude as a volunteer and at ezCompany commented
Issue summary: View changes

Updated the IS.

Wim Leers’s picture

Wim Leers
Location Ghent 🇧🇪🇪🇺
CreditAttribution: Wim Leers at Acquia commented
Title: Javascript tests don't work with Chromedriver 75 and higher » JavaScript tests don't work with Chromedriver 75 and higher

At this point this is only a documentation change. I think the only thing this is blocked on (before reaching RTBC) is manual testing by somebody who uses Chromedriver 75 or newer, right?

zrpnr’s picture

zrpnr
he/him
Primary language English
Location UTC-7
CreditAttribution: zrpnr at Acquia commented
Status: Needs review » Reviewed & tested by the community

Is there a way to see what settings and type of webdriver the Drupal CI uses?

Looks like @mixologic would know: https://www.drupal.org/project/drupal/issues/2775653#comment-12394474

geckodriver might not be completely webdriver api compliant. I've installed chromedriver on the testbots, and am using that as the baseline for now

I can confirm that this config change works and is necessary for running FunctionalJavascript tests using the latest version of chromedriver- which you will have installed if you follow the instructions at:

https://www.drupal.org/node/2941464

I also agree with recommending chromedriver over firefox (geckodriver) here - especially since I wasn't able to get geckodriver working at all locally. It could be I'm using out of date config for it as well.

@mixologic again: https://www.drupal.org/project/drupal/issues/2941560#comment-12491205

Im not entirely sure what we should keep in scope for the docs - i.e the testbots are running *solely* chromedriver, in headless mode, with no selenium server at all. There's geckodriver for the folks who want to see if the tests run on FF, but it looks like theres some bugs with it

If the testbots are using chromedriver that seems like enough reason to set that as the default in this file.
This patch applies cleanly, I think it good to mark RTBC

larowlan credited tedbow.

larowlan’s picture

larowlan
Location 🇦🇺🏝.au GMT+10
CreditAttribution: larowlan at PreviousNext commented
larowlan’s picture

larowlan
Location 🇦🇺🏝.au GMT+10
CreditAttribution: larowlan at PreviousNext commented
Status: Reviewed & tested by the community » Fixed

Committed ad3be82 and pushed to 8.8.x. Thanks!

  • larowlan committed ad3be82 on 8.8.x 
    Issue #3073342 by Lendude, zrpnr, tedbow: JavaScript tests don't work...
larowlan’s picture

larowlan
Location 🇦🇺🏝.au GMT+10
CreditAttribution: larowlan at PreviousNext commented
Version: 8.8.x-dev » 8.7.x-dev
Status: Fixed » Patch (to be ported)

Cherry-picked this to 8.7.x

larowlan’s picture

larowlan
Location 🇦🇺🏝.au GMT+10
CreditAttribution: larowlan at PreviousNext commented
Status: Patch (to be ported) » Fixed

  • larowlan committed a2adf87 on 8.7.x 
    Issue #3073342 by Lendude, zrpnr, tedbow: JavaScript tests don't work...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.