Follow-up to #2297711: Fix HTML escaping due to Twig autoescape

Follow-up to #1825952: Turn on twig autoescape by default

Non-technical explanation

After #2264041: Add a test to ensure title callbacks are not vulnerable to XSS have proven that even battle hardened core developers can't write XSS free code we have introduced #1825952: Turn on twig autoescape by default to fix a torrent of security holes already present in core known and unknown and to avoid the most frequent kind of sechole(Security Hole) in the history of Drupal contrib. However, this has broken some places that were already securely written, resulting in broken layout and HTML tags shown to users. We need to find those places and update them to be compatible with the new method.

Problem/Motivation

Can be tricky to discover the double escaping.

Proposed resolution

@dawehner's and @joelpittet's idea about testing existing checked routes that are already tested for double escaping.
Duckpatch drupalGet/drupalPost methods in simpletest to check for double escaping on pages.
Potentially a more permanent fixture could be possible.

Remaining tasks

Files: 

Comments

joelpittet created an issue. See original summary.

joelpittet’s picture

Issue summary: View changes
Status: Active » Needs review
Issue tags: -Perfomance, -Needs issue summary update
FileSize
2.41 KB
FAILED: [[SimpleTest]]: [PHP 5.5 MySQL] 146,825 pass(es), 161 fail(s), and 0 exception(s). View

Status: Needs review » Needs work

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

dawehner’s picture

I know that I had an issue about that too, well, no idea where it is now :)

There have been just a few actual valid failures, most the other ones seems to be real bugs ... wow

Status: Needs work » Needs review
joelpittet’s picture

160 was the count, retesting to see where we are at so far.

Status: Needs review » Needs work

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

Status: Needs work » Needs review
joelpittet’s picture

Issue tags: +rc target triage

Would be nice to have no unintentional escaping with D8 release.

alexpott’s picture

Status: Needs review » Needs work

The last submitted patch, 11: 2571065.11.patch, failed testing.

joelpittet’s picture

32 to 6 is quite nice!

alexpott’s picture

@joelpittet it is even better than that! I'd argue we only really have to investigate 2 of the fails...

  • Drupal\Tests\user\Unit\Plugin\Action\RemoveRoleUserTest - not related solved by latest patch on the other issue
  • Drupal\filter\Tests\FilterAdminTest - expected fail the filter admin page has intentional double escaping
  • Drupal\search\Tests\SearchNodePunctuationTest - needs investigation
  • Drupal\simpletest\Tests\SimpleTestTest (this accounts for 2 of the fails) - needs investigation - but Simpletest UI so very very unimportant and this test is evil
  • Drupal\system\Tests\Theme\EntityFilteringThemeTest - needs investigation
alexpott’s picture

Drupal\search\Tests\SearchNodePunctuationTest is fine - it is searching for & so it is an expected fail.

alexpott’s picture

Created #2598502: Double escaping in views attachment titles to fix Drupal\system\Tests\Theme\EntityFilteringThemeTest

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

The last submitted patch, 2: find_escaping_due_to-2571065-2.patch, failed testing.

xjm’s picture

Issue tags: -rc target triage

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

joelpittet’s picture

Status: Needs work » Needs review
FileSize
12.72 KB

Reroll

Status: Needs review » Needs work

The last submitted patch, 23: 2571065.11-reroll.patch, failed testing.