
Problem/Motivation
Nightwatch tests have proven to be too shaky for my liking and may not be a good long-term choice.
Details:
- Random fails, such as: https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/3888711 and https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/3888757
- Outputs errors, but tests still run normally such as: https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/3888793
Error Error while running .clearElementValue() protocol action: invalid element state (Session info: headless chrome=106.0.5249.103) (Driver info: chromedriver=106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}),platform=Linux 5.10.230-223.885.amzn2.x86_64 x86_64) Error Error while running .clearElementValue() protocol action: invalid element state (Session info: headless chrome=106.0.5249.103) (Driver info: chromedriver=106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}),platform=Linux 5.10.230-223.885.amzn2.x86_64 x86_64)
- Difficulty running/maintaining tests that work across Drupal 9.5.x, 10.x, and 11.x in part due to differing Nightwatch versions
Although Drupal core uses some Nightwatch tests, we don't need to restrict ourselves to the testing tools shipped with core.
Also, Drupal core might adopt Playwright.
Proposed resolution
Replace Nightwatch with Playwright.
Remaining tasks
- Set up Playwright for testing
- Port existing tests to Playwright
User interface changes
n/a
API changes
n/a
Data model changes
n/a
Comments
Comment #2
star-szrComment #3
star-szrMore random fails:
Comment #4
star-szrI did a quick search today and found that core is looking to go in the direction of Playwright, which based on my recent evaluations for a client project would be very high on my list as well.
Tools to check out:
https://github.com/Lullabot/ddev-playwright
https://github.com/julienloizelet/ddev-playwright
Comment #5
star-szrAt this point the path seems clear to use Playwright, of course things can change but converting to a task for now.
I am hoping to have some time available in the next week or two to continue this work.