Problem/Motivation

In Settings Tray tests we have OutsideInJavascriptTestBase::waitForNoElement. This is used when an element will be removed from the page but we can't just check to see if it removed becase the Javascript to remove might take time and this could cause random failures.

Proposed resolution

Create \Drupal\FunctionalJavascriptTests\JSWebAssert::assertNoElementAfterWait()

This will be similar to \Drupal\FunctionalJavascriptTests\JSWebAssert::waitForElement but course there is no point in returning the element which should not be there.

Remaining tasks

Write patch

User interface changes

None

API changes

New test assert

Data model changes

None

CommentFileSizeAuthor
#48 2892440-48.patch37.05 KBalexpott
#48 43-48-interdiff.txt3.64 KBalexpott
#43 2892440-43.patch35.75 KBtedbow
#43 interdiff-40-43.txt648 bytestedbow
#40 2892440-40.patch36.72 KBbnjmnm
#40 interdiff_37-40.txt3.61 KBbnjmnm
#37 2892440-37.patch36.2 KBbnjmnm
#34 2892440-34.patch36.08 KBbnjmnm
#34 interdiff_31-34.txt562 bytesbnjmnm
#31 2892440-31.patch35.99 KBbnjmnm
#31 interdiff_29--31.txt738 bytesbnjmnm
#27 2892440-27.patch35.91 KBbnjmnm
#27 interdiff_22-27.txt22.77 KBbnjmnm
#22 2892440-21.patch13.9 KBbnjmnm
#20 interdiff-16-20.txt2.9 KBAnonymous (not verified)
#20 2892440-20.patch13.75 KBAnonymous (not verified)
#16 interdiff-14-16.txt1.88 KBAnonymous (not verified)
#16 2892440-16.patch12.75 KBAnonymous (not verified)
#14 interdiff-10-14.txt5.87 KBAnonymous (not verified)
#14 2892440-14.patch12.74 KBAnonymous (not verified)
#12 interdiff-10-12.txt1.66 KBAnonymous (not verified)
#12 2892440-12.patch3.62 KBAnonymous (not verified)
#10 interdiff-5-10.txt2.7 KBAnonymous (not verified)
#10 2892440-10.patch10.17 KBAnonymous (not verified)
#5 2892440-5.patch10.69 KBtedbow

Comments

tedbow created an issue. See original summary.

dawehner’s picture

I know https://www.drupal.org/u/marcoscano had the same usecase on the weekend. I'm not sure right now what he ended up with.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

tedbow’s picture

Status: Active » Needs review
StatusFileSize
new10.69 KB

Ok here is patch. It does:

  1. Adds the new assert
  2. Updates \Drupal\FunctionalJavascriptTests\Tests\JSWebAssertTest to test the assert. It test that it passes, and fails for a too short wait.
  3. Updates off-canvas and settings tray tests to remove the waitForNoElement() function and replace it with the new assert.
  4. Adds 1 seemingly unrelated which is actually related:
    -    $web_assert->elementTextContains('css', '#drupal-live-announce', 'Exited edit mode.');
    +    $web_assert->waitForElement('css', '#drupal-live-announce:contains(Exited edit mode)');

    This just waits for element instead of assuming it is already there. My assumption as to why this is needed because the new assert checks if he conditions pass more frequently then \Drupal\FunctionalJavascriptTests\JavascriptTestBase::assertJsCondition did which was used before so it get to this code a little bit faster and the JS that did Drupal.announce has not fire yet.

Anonymous’s picture

Looks nice! But why assertNoElementAfterWait() doesn't use waitFor() API? Probably because it will lead to a kind of duplication of the header function inside the function. But I think it's better than duplicating the function body. We also can not affect the operation of this function, if one day we decide to change the waitFor().

tedbow’s picture

@vaplas, we can't use \Behat\Mink\Element\Element::waitFor because it calls a user function until it returns a non-empty result. Where as in this case we have to wait for a empty result.

Anonymous’s picture

But we can determine the condition by which the user function returns TRUE with empty result, and FALSE with non-empty result:

$result = $page->waitFor($timeout / 1000, function () use ($selector_type, $selector, $page) {
  return empty($page->find($selector_type, $selector));
});

$this-assertTrue($result, $message);
tedbow’s picture

We could but the docs for this method on \Behat\Mink\Element\ElementInterface::waitFor() specifically say

Waits for an element(-s) to appear and returns it.

The implementation of it though on \Behat\Mink\Element\Element::waitFor has nothing to do with elements in the code.

But still we would be calling this method for the opposite of its intended purpose. If we were going to use it in this case should we use every time we need to wait on
$result = call_user_func($callback, $this);
regardless of whether it had anything to do with what the method on the interface docs say?

It would work in all kinds of situations but I don't think we should use outside of its intended purpose as describe on the interface.

Anonymous’s picture

Then maybe there is a sense to make a universal JTB::waitFor() method like c/p from Behat\Mink\Element\Element::waitFor(). But this can be done later of course. RTBC++

Just reroll #5 + few nit fixes.

+++ b/core/modules/settings_tray/tests/src/FunctionalJavascript/SettingsTrayBlockFormTest.php
@@ -139,7 +139,7 @@ protected function doTestBlocks($theme, $block_plugin, $new_page_text, $element_
-    $web_assert->waitForElement('css', '#drupal-live-announce:contains(Exited edit mode)');
+    $web_assert->elementTextContains('css', '#drupal-live-announce', 'Exited edit mode.');

Revert, because out of scope. This will be fixed like part of #2946294: Fix race conditions in OffCanvasTestBase.


Additional issues, that also wait assertNoElementAfterWait:

Status: Needs review » Needs work

The last submitted patch, 10: 2892440-10.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Anonymous’s picture

Status: Needs work » Needs review
StatusFileSize
new3.62 KB
new1.66 KB

#11: Random fails?

Also after setExpectedException() the test is stoped, it should be used only for the last assert. Replaced on try/catch.

Status: Needs review » Needs work

The last submitted patch, 12: 2892440-12.patch, failed testing. View results

Anonymous’s picture

Status: Needs work » Needs review
StatusFileSize
new12.74 KB
new5.87 KB

#5:

+++ b/core/modules/settings_tray/tests/src/FunctionalJavascript/SettingsTrayBlockFormTest.php
@@ -158,7 +158,7 @@ public function testBlocks($theme, $block_plugin, $new_page_text, $element_selec
-    $web_assert->elementTextContains('css', '#drupal-live-announce', 'Exited edit mode.');
+    $web_assert->waitForElement('css', '#drupal-live-announce:contains(Exited edit mode)');

It is really related. Because assertNoElementAfterWait() faster than waitForNoElement(). As a result, a test flaw is detected. So reverted this part from #2946294: Fix race conditions in OffCanvasTestBase again.


OffCanvasTest uses waitForElement(). So will be better just deprecated this method, but not remove.

Status: Needs review » Needs work

The last submitted patch, 14: 2892440-14.patch, failed testing. View results

Anonymous’s picture

Status: Needs work » Needs review
StatusFileSize
new12.75 KB
new1.88 KB
dawehner’s picture

Status: Needs review » Reviewed & tested by the community

This will be similar to \Drupal\FunctionalJavascriptTests\JSWebAssert::waitForElement but course there is no point in returning the element which should not be there.

Haha, yeah that's certainly a true statement.

Then maybe there is a sense to make a universal JTB::waitFor() method like c/p from Behat\Mink\Element\Element::waitFor(). But this can be done later of course. RTBC++

Do you want to open up a follow up for that?

Anonymous’s picture

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/system/tests/src/FunctionalJavascript/OffCanvasTestBase.php
    @@ -149,4 +134,21 @@ public function themeDataProvider() {
    +  protected function waitForNoElement($selector, $timeout = 10000) {
    +    $this->assertSession()->assertNoElementAfterWait('css', $selector, $timeout);
    +  }
    

    We need a @trigger_error('some mesage', E_USER_DEPRECATED) to inform other tests that this is going away.

  2. +++ b/core/tests/Drupal/FunctionalJavascriptTests/JSWebAssert.php
    @@ -367,4 +367,37 @@ function t(r, lx, ly) {
    +    if ($node) {
    +      throw new ElementHtmlException($message, $this->session->getDriver(), $node);
    +    }
    

    I think the if ($node) { is unnecessary. Less conditions makes things less complicated.

  3. +++ b/core/tests/Drupal/FunctionalJavascriptTests/Tests/JSWebAssertTest.php
    @@ -29,6 +30,22 @@ public function testJsWebAssert() {
    +    $assert_session->elementExists('css', '[data-drupal-selector="edit-test-assert-no-element-after-wait-pass"]');
    +    $page->findButton('Test assertNoElementAfterWait: pass')->press();
    +    $assert_session->assertNoElementAfterWait('css', '[data-drupal-selector="edit-test-assert-no-element-after-wait-pass"]', 1000);
    +
    +    $assert_session->elementExists('css', '[data-drupal-selector="edit-test-assert-no-element-after-wait-fail"]');
    +    $page->findButton('Test assertNoElementAfterWait: fail')->press();
    +    try {
    +      $assert_session->assertNoElementAfterWait('css', '[data-drupal-selector="edit-test-assert-no-element-after-wait-fail"]', 500, 'Element exists on page after too short wait.');
    +      $this->fail('Element not exists on page after too short wait.');
    +    }
    +    catch (ElementHtmlException $e) {
    +      $this->assertSame('Element exists on page after too short wait.', $e->getMessage());
    +    }
    +
    +    $assert_session->assertNoElementAfterWait('css', '[data-drupal-selector="edit-test-assert-no-element-after-wait-fail"]', 2500, 'Element remove after another wait.ss');
    

    Nice tests!

Anonymous’s picture

Status: Needs work » Needs review
StatusFileSize
new13.75 KB
new2.9 KB

#19: thanks for nice points! Done. Also remove FieldTest::waitForNoElement(). In contrast to OffCanvasTestBase, FieldTest is not *Base test.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

bnjmnm’s picture

StatusFileSize
new13.9 KB

Reroll + a few coding standards fixes

phenaproxima’s picture

Status: Needs review » Reviewed & tested by the community
+++ b/core/tests/Drupal/FunctionalJavascriptTests/JSWebAssert.php
@@ -367,4 +367,37 @@ function t(r, lx, ly) {
+  public function assertNoElementAfterWait($selector_type, $selector, $timeout = 10000, $message = 'Element exists on the page.') {
+
+    $start = microtime(TRUE);

There's an extra blank line here, but that is fixable on commit.

Otherwise, I think this looks fine. Good to go.

The last submitted patch, 20: 2892440-20.patch, failed testing. View results

tedbow’s picture

Status: Reviewed & tested by the community » Needs work

I can't wait to get this is but if you search for "2892440" you will find 3 more @todo's we need to take care of in this issue. Should just be able to swap out the calls.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

bnjmnm’s picture

Status: Needs work » Needs review
StatusFileSize
new22.77 KB
new35.91 KB

Took care of the 10 remaining @todos

lendude’s picture

Status: Needs review » Reviewed & tested by the community

Thank you @bnjmnm. All feedback has been addressed, back to RTBC.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 27: 2892440-27.patch, failed testing. View results

lendude’s picture

+++ b/core/modules/layout_builder/tests/src/FunctionalJavascript/MoveBlockFormTest.php
@@ -90,7 +90,6 @@ protected function setUp() {
-    $this->waitForNoElement('#drupal-off-canvas');

The fail might be related to the removal of this line. It seems that removing this might cause a random fail.

bnjmnm’s picture

Status: Needs work » Needs review
StatusFileSize
new738 bytes
new35.99 KB

Thanks @lendude, looks like that line was inadvertently removed instead of replaced with the new method. Glad you & the tests caught that!

Status: Needs review » Needs work

The last submitted patch, 31: 2892440-31.patch, failed testing. View results

bnjmnm’s picture

The new test failure is most likely due to css transitions changing the location of the element the test is attempting to access. There's a test module in Layout Builder that disables css transitions that will likely fix the issue (and would add right now if I was on the machine with my dev environment)

bnjmnm’s picture

Status: Needs work » Needs review
StatusFileSize
new562 bytes
new36.08 KB

Added the test module that disables CSS transitions to MoveBlockFormTest. Hoping that addresses the unexpected test failure.

dww’s picture

Status: Needs review » Reviewed & tested by the community

Interdiffs look clean and right. Bot is happy green. RTBC.

Thanks!
-Derek

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs reroll

For some reason this hasn;t been set back to needs work for the re-roll.

bnjmnm’s picture

Status: Needs work » Needs review
StatusFileSize
new36.2 KB

Reroll

tedbow’s picture

Status: Needs review » Needs work

@bnjmnm thanks for getting this going again.

  1. +++ b/core/modules/layout_builder/tests/src/FunctionalJavascript/ContentPreviewToggleTest.php
    @@ -78,8 +78,7 @@ public function testContentPreviewToggle() {
    -    // Wait for preview content hide() to complete.
    -    $this->waitForNoElement('[data-layout-content-preview-placeholder-label] .field--name-body:visible');
    +    // Confirm that block content is not on page.
    

    Is there a reason we are not replacing this with a call to assertNoElementAfterWait()?

  2. +++ b/core/modules/layout_builder/tests/src/FunctionalJavascript/InlineBlockTestBase.php
    @@ -167,26 +167,11 @@ protected function configureInlineBlock($old_body, $new_body, $block_css_locator
    -  protected function waitForNoElement($selector, $timeout = 10000) {
    -    $condition = "(typeof jQuery !== 'undefined' && jQuery('$selector').length === 0)";
    -    $this->assertJsCondition($condition, $timeout);
    -  }
    -
    

    Can we actually just remove this or do we have to deprecate it like the occurence in OffCanvasTestBase? Because they are both base test classes.

    I think before we could just remove it because Layout Builder was experimental but now it is stable.

  3. +++ b/core/modules/layout_builder/tests/src/FunctionalJavascript/MoveBlockFormTest.php
    @@ -86,7 +87,6 @@ protected function setUp() {
    -    $this->waitForNoElement('#drupal-off-canvas');
    

    We should repace this call like the others.

  4. +++ b/core/modules/system/tests/src/FunctionalJavascript/OffCanvasTestBase.php
    @@ -103,25 +103,12 @@ protected function getOffCanvasDialog() {
    +   * @deprecated in Drupal 8.5.x, will be removed before Drupal 9.0.0. Use
    +   *   Drupal\FunctionalJavascriptTests\JSWebAssert::assertNoElementAfterWait()
    

    I guess this has taken a while 😉
    Need update this to 8.8.x

  5. +++ b/core/modules/system/tests/src/FunctionalJavascript/OffCanvasTestBase.php
    @@ -103,25 +103,12 @@ protected function getOffCanvasDialog() {
    +    @trigger_error('::waitForNoElement is deprecated in Drupal 8.5.0 and will be removed before Drupal 9.0.0. Use \Drupal\FunctionalJavascriptTests\JSWebAssert::assertNoElementAfterWait() instead.', E_USER_DEPRECATED);
    +    $this->assertSession()->assertNoElementAfterWait('css', $selector, $timeout);
    

    Need to update to 8.8.0

vacho’s picture

Issue tags: -Needs reroll

Patch applies well to 8.8.x

bnjmnm’s picture

Status: Needs work » Needs review
StatusFileSize
new3.61 KB
new36.72 KB

#38.1

Is there a reason we are not replacing this with a call to assertNoElementAfterWait()?

. After discovering that assertNoElementAfterWait would not work due to the element still being present (just hidden), I also realized the hide() transition is now instantaneous, so the wait is no longer neccessary. It had been added in an earlier iteration of the toggle issue when hide/show were animated

Items 2-5 of #38 are also addressed in this patch.

krzysztof domański’s picture

--- a/core/tests/Drupal/FunctionalJavascriptTests/JSWebAssert.php
+++ b/core/tests/Drupal/FunctionalJavascriptTests/JSWebAssert.php
@@ -387,4 +387,35 @@ function t(r, lx, ly) {
     return $this->session->evaluateScript($full_javascript_visibility_test);
   }
 
+  /**
+   * Asserts that no matching element exists on the page after a wait.
+   *
+   * @param string $selector_type
+   *   The element selector type (CSS, XPath).

The element selector type is case-sensitive. CSS and XPath should be lowercase.

See #3041900: The element selector type "CSS, XPath" in JSWebAssert should be lowercase.

tedbow’s picture

StatusFileSize
new648 bytes
new35.75 KB

reroll plus #41

Status: Needs review » Needs work

The last submitted patch, 43: 2892440-43.patch, failed testing. View results

krzysztof domański’s picture

krzysztof domański’s picture

#3056536: LayoutBuilderDisableInteractionsTest randomly fails

The assertNoElementAfterWait has been changed in core/modules/layout_builder/tests/src/FunctionalJavascript/LayoutBuilderDisableInteractionsTest.php.

alexpott’s picture

Priority: Critical » Normal

This is no longer critical.

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new3.64 KB
new37.05 KB

Rerolled and fixed fail.

lendude’s picture

Status: Needs review » Reviewed & tested by the community

Can't find anything to add to this, all feedback had been addressed again, so back to RTBC

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed 9a11de6 and pushed to 8.8.x. Thanks!
Committed 6ad73cb and pushed to 8.7.x. Thanks!

  • alexpott committed 9a11de6 on 8.8.x
    Issue #2892440 by bnjmnm, tedbow, alexpott, Krzysztof Domański, Lendude...

  • alexpott committed 6ad73cb on 8.7.x
    Issue #2892440 by bnjmnm, tedbow, alexpott, Krzysztof Domański, Lendude...

  • alexpott committed 05d8368 on 8.7.x
    Issue #2892440 follow-up by alexpott: Provide helper test method to wait...
alexpott’s picture

Didn't quite merge to 8.7.x correctly so had to commit a quick follow-up afterwards... Add block vs Add Block - fun https://www.drupal.org/commitlog/commit/2/6ad73cb21868ac8fba291aef6de5a6...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.