Add Nightwatch Test Coverage for Olivero

The current version of the merge request installs the standard profile. I think we prefer to test with one of the smaller profiles: minimal, testing, or nightwatch_testing.

I did some manual testing, installing Drupal with the minimal and testing profiles.

I installed Drupal with the testing profile, then tried to install Olivero. I got the message

Unable to install olivero due to unmet dependencies: block.block.olivero_account_menu (block)

After installing the block module, I was able to install Olivero and set it as the default theme.

Then I added drupal:block as a dependency in olivero.info.yml and reinstalled Drupal with the testing profile. Now, when I visit the Appearance page, I see the message

Olivero 9.2.0-dev (experimental theme) A clean, accessible, and flexible Drupal front-end theme. Requires:

Block (disabled)

This theme requires the listed modules to operate correctly. They must first be enabled via the Extend page.

The text "Extend page" is a link to /admin/modules.

That seems like better behavior, so I think we should add block as a dependency. Maybe that should be a separate issue.

With both the testing and minimal profiles, an anonymous user sees the Login block on the home page, not the home page we see with the standard profile.

I see that the usual home page comes from a template included when Views renders the home page. If we enable views as well as block, then that view has the path /node, which is the default front page with the standard profile. That suggests two ways to get the tests to pass with the minimal profile. Both require enabling the block and views modules.

1. In the test, load /node instead of /.
2. Set system.site.page.front to "/node", as in the standard profile.

I tried (1), and the first few assertions passed. The test fails to find #block-olivero-main-menu. I think that comes from standard.links.menu.yml in the standard profile.

I am not sure what the right way is to add a Home link to the page. One option is to create a test module that copies standard.links.menu.yml, then enable that module in the test setup. We could add that test module to core/modules/system/tests/modules. Another option is to enable the menu_link_content module and then create a menu link entity.

Another way to make sure the Main menu is not empty is to add something like this to the install script:

$link_properties = ['title' => 'Home', 'route_name' => '<front>', 'menu_name' => 'main'];
\Drupal::service('plugin.manager.menu.link')
  ->addDefinition('olivero.front_page', $link_properties);

I have updated the MR with this option.

I haven't tested this yet, but I'm so excited. I have a feature request.

When you add a link to the main menu, can you add a submenu below it? The submenus contain complicated functionality that differs between mobile and desktop, and it'd be great to add tests for those.

Creating links the way I did, with Drupal::service('plugin.manager.menu.link'), is awkward. If you want more than one, then I think a test module is the right way to do it.

I added olivero_test under core/modules/system/tests/modules, defining two menu links. If there is a better place for the test module, then it should be easy to move.

You should be able to experiment locally by installing Drupal with the minimal profile, then enabling the views and olivero_test modules. Remember to visit /node instead of /.

I think that is all I will have time to do today.

I am leaving the status at NW. Now that we have a second menu item, we should use it in the tests.

Thanks for the help @benjifisher!

Adding child issue #3206015: Olivero: Additional wide menu Nightwatch tests

I'm working/helping out reviewing and debugging nightwatch tests at MidCamp contribution day.

We looked at jsCookieTest.js as a pattern to using browser.execute() to make the navigation trap test.

This is ready for review IMO

Thanks for the review @mradcliffe! Changes are made and pushed :)

Good catch. Resolved!

The integration of Nightwatch tests with the Drupal testbot seems to be incomplete. The message "Build Successful" on the latest test seems to mean that one of the Nightwatch tests failed.

Look at the console output and jump to the bottom of the page. The oliveroMobileMenuTest test fails, and it fails when I run it locally. I will leave a comment on the MR.

On second thought, I will not comment on the MR. I thought I saw where the problem was, but I was wrong.

When I run the test locally, sometimes it passes and sometimes it does not. When it fails, it is always on Line 37, the bottom line of this snippet:

      browser.click(`[aria-controls="${linkSubMenuId}"]`, () => {
        browser
          .pause(200)
          .assert.visible(`#${linkSubMenuId}`)

The pause does not make a difference: the test tries for 5s (5000 ms) before giving up, if I read the test results correctly.

I am afraid that I do not know nearly enough about Nightwatch to help explain why a test fails sporadically.

Yeah, the "Build Successful" message was confusing me too.

I pulled the latest version and ran the test several times. It passed 7 times, then failed on the 8'th try.

 Error while running .clickElement() protocol action: An element command could not be completed because the element is not visible on the page. – element not visible

✖ Timed out while waiting for element <#home-submenu-1> to be visible for 1000 milliseconds. - expected "visible" but got: "not visible" (1043ms)
    at NightwatchAPI.<anonymous> (/app/core/tests/Drupal/Nightwatch/Tests/oliveroMobileMenuTest.js:35:19)
    at processTicksAndRejections (internal/process/task_queues.js:97:5) 

well 💩. I was just about to post here that it passed the tests on Drupal CI. I will see if I get it to fail on my local.

I'm also able to get this to successfully fail on my local

@justafish helped work through some of the random test failures! I think we're good now.

Pushed some more slight refactors based on what I learned from @justafish. This should be good to go

Committed ff49ffc and pushed to 9.2.x. Thanks!

Exciting to see new Nightwatch tests in core. This seems a great start for Olivero JS test coverage.