Problem/Motivation

The test coverage for theme output across core is patchy, and in some places not well implemented.

Tests that compare one string of HTML output to another are fragile, and failures are not necessarily representitive of a failure to the user (perhaps because an elements attributes are in a different order.

Proposed resolution

Improve coverage of themed output
Test core themes separately
Use superior analysis tools such as CSS Selectors or XPath/SimpleXml to perform assertions with rendered output.
More - TBD

Remaining tasks

Everything!

User interface changes

N/A

API changes

TBD

#1971860: Document that Twig debug mode breaks tests
#1959660: Replace xpath() with WebTestBase::cssSelect() by leveraging Symfony CssSelector

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

shanethehat’s picture

To get the ball rolling, here is an XPath test approach that allows a rendered string to be compared against an element name and array of attributes. Attributes that are missing will fail the test, as will unexpected additional attributes.

 /**
   * Constructs an xpath query string
   * 
   * @param string $name
   *   Name of the element to search for
   * @param array $attributes
   *   List of attributes the element must contain
   * 
   * @return string
   *   Xpath expression
   */
  protected function buildXPath($name, array $attributes) {
    $expression = '/' . $name . '[';
    $attribs = '';
    foreach ($attributes as $attr => $value) {
      $attribs .= ' and @' . $attr . '=\'' . $value . '\'';
    }
    $expression .= substr($attribs,5) . ']';
    return $expression;
  }
  
  /**
   * Compare a string of HTML to an element definition.
   * 
   * An element can be defined by its name and attributes. This compares a
   * provided string of HTML containing a target element with an XPath
   * expression constructed from the expected name and attributes. Comparison
   * fails if the XPath expression does not match an element, or if it matches
   * an element that contains unexpected attributes. Example:
   * @code
   * $this->compareElement($rawHtml, 'img', array(
   *   'src' => 'some-uri.jpg',
   *   'width => 50,
   *   'alt' => '',
   * );
   * @endcode
   * 
   * @param string $element
   *   Raw HTML to search for an element.
   * @param string $name
   *   The name of the element to match.
   * @param array $attributes
   *   Array of expected attributes.
   */
  protected function compareElement($element, $name, array $attributes) {
    $expression = $this->buildXPath($name, $attributes);
    
    $xml = new \SimpleXMLElement($element);
    $found = $xml->xpath($expression);
    $number_of_elements = count($found);
    // Assert that the xpath matches an element
    $this->assertEqual($number_of_elements, 1, htmlentities($element) . " matches the expected expression: $expression.");
    if ($number_of_elements) {
      // Assert that the matched element doesn't contain unexpected attributes.
      foreach ($found[0]->attributes as $attribute => $value) {
        if (!array_key_exists($attribute, $attributes)) {
          $this->fail("Unexpected attribute $attribute found.");
        }
      }
    }
  }
shanethehat’s picture

Modified (shortened) version of the above code as a patch to WebtestBase

shanethehat’s picture

Issue summary: View changes

Using issue summary template

shanethehat’s picture

Issue summary: View changes

wrong issue number

star-szr’s picture

Issue summary: View changes
Issue tags: +Twig
joelpittet’s picture

Status: Active » Needs review
Issue tags: +Needs tests

This looks kinda neat. Probably could use some tests to give it a try. Leaving this at 8.0.x because it's test improvements and can make it into a patch release.

Status: Needs review » Needs work

The last submitted patch, 2: improve-theme-tests-1972786-2.patch, failed testing.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.