Problem/Motivation

Move performance tests to selenium. Follow-up to #3463534: Performance test gitlab job is broken - revert to use chromedriver and blocks #3462682: Deprecate non-W3C compliant testing

Steps to reproduce

Move job to use selenium and all tests are skipped.

Proposed resolution

Fix container. I'm pretty sure it has something to do with 

  tags:
    - performance-test-runner

As the tests run successfully on selenium as part of the main job.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3463696

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

alexpott created an issue. See original summary.

alexpott opened merge request !8923

catch made their first commit to this issue’s fork.

catch’s picture

catch
he/him
Primary language English
commented

Added verbose logging, the logging is more verbose but it's not giving me any more hints about what's going wrong.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

From the verbose we see:

[0;37m[service:selenium__standalone-chrome-selenium-standalone-chrome-selenium] 2024-07-27T08:33:55.799363620Z 08:33:55.799 DEBUG [DriverService.start] - Starting driver at /usr/bin/chromedriver with [--port=6662, --log-path=/builds/chromedriver.log][0;m
[0;37m[service:selenium__standalone-chrome-selenium-standalone-chrome-selenium] 2024-07-27T08:33:55.806694897Z 08:33:55.806 DEBUG [UrlChecker.waitUntilAvailable] - Waiting for [http://localhost:6662/status][0;m
[0;37m[service:selenium__standalone-chrome-selenium-standalone-chrome-selenium] 2024-07-27T08:33:55.807341040Z 08:33:55.807 DEBUG [UrlChecker.lambda$waitUntilAvailable$1] - Polling http://localhost:6662/status[0;m
[0;37m[service:selenium__standalone-chrome-selenium-standalone-chrome-selenium] 2024-07-27T08:33:55.807568985Z 08:33:55.807 DEBUG [ExternalProcess$Builder.lambda$start$0] - completed to copy the output of process 64[0;m
[0;37m[service:selenium__standalone-chrome-selenium-standalone-chrome-selenium] 2024-07-27T08:33:55.810179587Z 08:33:55.809 WARN [DriverServiceSessionFactory.apply] - Error while creating session with the driver service. Stopping driver service: Driver server process died prematurely.[0;m

So the key part seems to be that selenium fails to start up chromedriver - it'd be great to see what is in /builds/chromedriver.log

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Active » Needs review

Needed to open a branch on core to actually test this. This MR is now working https://git.drupalcode.org/project/drupal/-/jobs/8348192

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

@maxwhitehead did the debugging that lead to the solution - I added them to the contribution list.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

So two things...

1. The chromedriver.log problem happens because there already is a chromedriver.log file in /build
2. This is not done right because we're not getting all the logs back from chrome....

There were 3 skipped tests:
1) Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryAuthenticatedPerformanceTest::testAuthenticatedPerformance
Incomplete log from chromedriver, giving up.
2) Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryFrontPagePerformanceTest::testFrontPagePerformance
Incomplete log from chromedriver, giving up.
3) Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryNodePagePerformanceTest::testNodePage
Incomplete log from chromedriver, giving up.
OK, but some tests were skipped!
Tests: 3, Assertions: 5, Skipped: 3.

The latest pushes to this branch address the latter issue. Something we thought was legacy turns out not to be.

I think this is ready now.

catch’s picture

catch
he/him
Primary language English
commented
Status: Needs review » Reviewed & tested by the community

OK this looks great. I'm not sure why I thought one of those chrome configuration keys was legacy, maybe got confused with https://issues.chromium.org/issues/42322034. Let's just remove the comment.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

I think some of tracing categories are legacy but still supported so maybe that's it. But moving to the non legacy versions eg disabled-by-default-devtools.timeline would result in more and different data so we shouldn't move to that here. Maybe it is something to consider when we move to a more modern version of Chrome.

longwave’s picture

longwave
he/him
Primary language English
Location UK
commented
Version: main » 11.x-dev
Status: Reviewed & tested by the community » Fixed

Committed and pushed f1884efb4f9 to main and fec981c61ad to 11.x. Thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • longwave committed fec981c6 on 11.x 
    test: #3463696 Move performance test to selenium

By: alexpott
By: catch...

  • longwave committed f1884efb on main 
    test: #3463696 Move performance test to selenium

By: alexpott
By: catch...

longwave closed merge request !8923

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.