The WebTestBase provides a family of assertion-methods capable of examining form fields and their values. Regrettably the method signatures are not too consistent and that is probably the reason that they are invoked with wrong parameters every one and then. Grepping through the code the following cases turn up:

core/modules/aggregator/lib/Drupal/aggregator/Tests/ImportOpmlTest.php:

$this->assertField('refresh', '', 'Found Refresh field.');

core/modules/language/lib/Drupal/language/Tests/LanguageBrowserDetectionUnitTest.php:

$this->assertField('edit-mappings-xx-browser-langcode', 'xx', 'Browser language code found.');
$this->assertField('edit-mappings-xx-browser-langcode', 'xx', 'Browser language code found.');
$this->assertField('edit-mappings-xx-drupal-langcode', 'en', 'Drupal language code found.');
$this->assertField('edit-mappings-xx-drupal-langcode', 'zh-hans', 'Drupal language code found.');
$this->assertField('edit-mappings-zh-cn-browser-langcode', 'zh-cn', 'Chinese browser language code found.');
$this->assertField('edit-mappings-zh-cn-drupal-langcode', 'zh-hans-cn', 'Chinese Drupal language code found.');

assertField has no $value parameter. Name should be first and the message second.

core/modules/system/lib/Drupal/system/Tests/Form/ValidationTest.php:

$this->assertNoFieldByName('name', 'Form element was hidden.');
$this->assertNoFieldByName('name', 'Form element was hidden.');

core/modules/user/lib/Drupal/user/Tests/UserLanguageCreationTest.php:

$this->assertNoFieldByName('language[fr]', 'Language selector is not accessible.');
$this->assertNoFieldByName($relationship_name, 'Make sure that no relationship option is available');

assertNoFieldByName has $value parameter. Name should be first, value second and the message third.

Comments

sun’s picture

Status: Active » Needs review
StatusFileSize
new5.17 KB

Well spotted!

Attached patch fixes the offending assertions.

For D8, I wonder whether we shouldn't change the signature of assertField() and assertNoField() to also have a second $value argument - like all of the other assertField* methods - and simply ignore the passed $value? → Consistency appears to be more important here than an unused parameter?

Status: Needs review » Needs work

The last submitted patch, 1: drupal8.test-assertfield.1.patch, failed testing.

sun’s picture

Priority: Normal » Major

Worst possible result: The tests do not pass with the corrected assertions.

I think that makes this issue at least major, if not even critical.

znerol’s picture

Status: Needs work » Needs review
StatusFileSize
new5.17 KB
znerol’s picture

StatusFileSize
new5.94 KB

The last failing test was introduced with #642702: Form validation handlers cannot alter $form structure, commit b60848. I'm not so sure whether it was the idea that form-alterations would survive multiple rebuilds. Therefore I propose to fix the test and do not touch the implementation.

The last submitted patch, 4: 2171939-test-assert-field-d8-3.patch, failed testing.

mr.baileys’s picture

Status: Needs review » Needs work
  1. Another instance of assertNoFieldByName() that needs to be changed:
    core/modules/views/lib/Drupal/views/Tests/Handler/HandlerTest.php:274:
        $this->assertNoFieldByName($relationship_name, 'Make sure that no relationship option is available');
    
  2. +++ b/core/modules/system/tests/modules/form_test/lib/Drupal/form_test/Callbacks.php
    @@ -35,6 +35,8 @@ public function validateName(&$element, &$form_state) {
    +      $element['#access'] = FALSE;
    

    I *think* this is the correct approach and the validation handler is responsible for setting access/hiding the element on successive form builds. Would be great to get a Form API Guru to confirm this though.

mr.baileys’s picture

Status: Needs work » Needs review
StatusFileSize
new6.79 KB

Changed the incorrect invocation of assertNoFieldByName() in core/modules/views/lib/Drupal/views/Tests/Handler/HandlerTest.php

sun’s picture

Note that #2105617: False pass with WebTestBase::assertFieldByName with select element just landed, which seems to have fixed just a single of these instances, but at the same time, it also fixed some form handling logic and added test coverage for the select form handling.

znerol’s picture

The patch from #8 still applies cleanly to head. The thing we are still missing in this issue is a decision whether the change in core/modules/system/tests/modules/form_test/lib/Drupal/form_test/Callbacks.php is justifiable or not.

Status: Needs review » Needs work

The last submitted patch, 8: 2171939-8-assert-fields.patch, failed testing.

dawehner’s picture

Issue tags: +Needs reroll

.

rpayanm’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new6.61 KB

dawehner queued 14: 2171939-14.patch for re-testing.

mile23’s picture

Version: 8.0.x-dev » 8.1.x-dev

Applies cleanly to 8.1.x. Imagine that.

Will try to start up the testbot.

Mile23 queued 14: 2171939-14.patch for re-testing.

mile23’s picture

StatusFileSize
new6.61 KB

Re-uploading for drupalci.

Note that this is just a re-upload of #14. No credit to me, please.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Status: Needs review » Needs work

The last submitted patch, 18: 2171939-14.patch, failed testing.

mile23’s picture

Version: 8.2.x-dev » 8.1.x-dev
Status: Needs work » Needs review
StatusFileSize
new6.61 KB

Moving back to 8.1.x since this is a bug and is a bunch of test improvements.

Rerolling #18 which is a reroll of #14.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

quietone’s picture

Project: Drupal core » SimpleTest
Version: 8.9.x-dev » 8.x-3.x-dev
Component: simpletest.module » Code

Triaging issues in simpletest.module as part of the Bug Smash Initiative to determine if they should be in the Simpletest Project or core.

This looks like it belongs in the Simpletest project.