Follow-up to #2297711: Fix HTML escaping due to Twig autoescape

Follow-up to #1825952: Turn on twig autoescape by default

Non-technical explanation

After #2264041: Add a test to ensure title callbacks are not vulnerable to XSS have proven that even battle hardened core developers can't write XSS free code we have introduced #1825952: Turn on twig autoescape by default to fix a torrent of security holes already present in core known and unknown and to avoid the most frequent kind of sechole(Security Hole) in the history of Drupal contrib. However, this has broken some places that were already securely written, resulting in broken layout and HTML tags shown to users. We need to find those places and update them to be compatible with the new method.

Problem/Motivation

Can be tricky to discover the double escaping.

Proposed resolution

@dawehner's and @joelpittet's idea about testing existing checked routes that are already tested for double escaping.
Duckpatch drupalGet/drupalPost methods in simpletest to check for double escaping on pages.
Potentially a more permanent fixture could be possible.

Remaining tasks

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

joelpittet created an issue. See original summary.

joelpittet’s picture

Status: Needs review » Needs work

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

dawehner’s picture

I know that I had an issue about that too, well, no idea where it is now :)

There have been just a few actual valid failures, most the other ones seems to be real bugs ... wow

Status: Needs work » Needs review
joelpittet’s picture

160 was the count, retesting to see where we are at so far.

Status: Needs review » Needs work

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

Status: Needs work » Needs review
joelpittet’s picture

Issue tags: +rc target triage

Would be nice to have no unintentional escaping with D8 release.

alexpott’s picture

Status: Needs review » Needs work

The last submitted patch, 11: 2571065.11.patch, failed testing.

joelpittet’s picture

32 to 6 is quite nice!

alexpott’s picture

@joelpittet it is even better than that! I'd argue we only really have to investigate 2 of the fails...

  • Drupal\Tests\user\Unit\Plugin\Action\RemoveRoleUserTest - not related solved by latest patch on the other issue
  • Drupal\filter\Tests\FilterAdminTest - expected fail the filter admin page has intentional double escaping
  • Drupal\search\Tests\SearchNodePunctuationTest - needs investigation
  • Drupal\simpletest\Tests\SimpleTestTest (this accounts for 2 of the fails) - needs investigation - but Simpletest UI so very very unimportant and this test is evil
  • Drupal\system\Tests\Theme\EntityFilteringThemeTest - needs investigation
alexpott’s picture

Drupal\search\Tests\SearchNodePunctuationTest is fine - it is searching for & so it is an expected fail.

alexpott’s picture

Created #2598502: Double escaping in views attachment titles to fix Drupal\system\Tests\Theme\EntityFilteringThemeTest

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

xjm’s picture

Issue tags: -rc target triage

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

joelpittet’s picture

Status: Needs work » Needs review
FileSize
12.72 KB

Reroll

Status: Needs review » Needs work

The last submitted patch, 23: 2571065.11-reroll.patch, failed testing.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.