Problem/Motivation

There are at least 2 aspects that prove that this functionality has never worked as intended (it's broken in such a way that it is very unlikely that anyone has ever used it):

  1. FilterFormat::getHTMLRestrictions() computes the combined HTML restrictions across all filters, but doesn't correctly handle the case of using both the allowed (allowlist) and forbidden_tags (blocklist) aspects of FilterInterface::getHTMLRestrictions:
    👉 #3271698: getHtmlRestrictions() ignores 'allowed_tags' settings if the first filter checked only provides 'forbidden_tags'
  2. The one filter that uses the forbidden_tags functionality does not actually work and is part of a test module but no test actually uses it to forbid tags:
    👉 #3271706: FilterFormat getHtmlRestrictions forbidden_tags expects different structure than the one provided by FilterTestRestrictTagsAndAttributes + #3231331: [PP-1] Generate test data compatible with interface in Drupal\filter_test\Plugin\Filter\FilterTestRestrictTagsAndAttributes::getHTMLRestrictions

Furthermore, a key security best practice to our knowledge is to use allowlists instead of blocklists.

Marked Major because it impacts the upgrade path from ckeditor.module (CKEditor 4) to ckeditor5.module (CKEditor 5): #3231336: Simplify HtmlRestrictions and FundamentalCompatibilityConstraintValidator now that "forbidden tags" are deprecated.

Proposed resolution

Because of the three points above, @bnjmnm and I wondered if it even makes sense to fix the two three bugs. Clearly nobody is using it.

Therefore we propose to deprecate this (obscure) functionality in Drupal 9.4 and remove it in Drupal 10.0.

This would allow us to close two bugs and be a net win for Drupal's security.

Remaining tasks

  1. Ask backend framework manager and release manager @catch for his input.

    👉 He responded:

    @wimleers (he/him) I can't ever remember a filter that only disallows tags, if we want to remove, it's easy to deprecate in 9.4.x and remove in 10.0.x - can't really see a reason not to.

  2. Deprecate the returning of forbidden_tags by a filter's getHTMLRestrictions() in 9.4.
    👉 #5
  3. Remove the "forbidden tags" functionality from FilterTestRestrictTagsAndAttributes in both Drupal 9.4 and 10.0, because it is not used anyway.
    👉 #6

User interface changes

None.

API changes

  1. Drupal 9.4: any filter plugin's getHTMLRestrictions() that returns a top-level forbidden_tags key will result in a deprecation notice.
  2. Drupal 10.0: any filter plugin's getHTMLRestrictions() that returns a top-level forbidden_tags key will result in an exception.

Data model changes

None.

Release notes snippet

TBD

Comments

Wim Leers created an issue. See original summary.

Wim Leers credited bnjmnm.

Wim Leers credited catch.

wim leers’s picture

Paired with @bnjmnm in writing the issue summary! 🚀

wim leers’s picture

Status: Active » Needs review
StatusFileSize
new1.71 KB

This is the baseline patch — should work for both 9.4 and 10.0. But for 10.0, we need more changes. That will happen in the next patch.

wim leers’s picture

StatusFileSize
new14.6 KB
new14.08 KB

This is what we expect to happen for 10.0.x.

wim leers’s picture

Issue summary: View changes
wim leers’s picture

Issue summary: View changes

Language clarifications.

wim leers’s picture

Found another bug that we could close thanks to this: #3231331: [PP-1] Generate test data compatible with interface in Drupal\filter_test\Plugin\Filter\FilterTestRestrictTagsAndAttributes::getHTMLRestrictions — this was also filed thanks to the work on CKEditor 5, nobody else had discovered this for nearly a decade. So: doing this will close three bugs now!

yogeshmpawar’s picture

StatusFileSize
new13.92 KB
new1.51 KB

Reroll patch with reroll diff.

Status: Needs review » Needs work

The last submitted patch, 10: 3272516-10.patch, failed testing. View results

yogeshmpawar’s picture

Status: Needs work » Needs review
StatusFileSize
new14.73 KB
new833 bytes

Updated FilterAPITest.php as per FilterFormat.php changes.

catch’s picture

9.4.x patch:

+++ b/core/modules/filter/src/Entity/FilterFormat.php
@@ -307,6 +307,7 @@ public function getHtmlRestrictions() {
           // Track the union of forbidden tags.
           if (isset($new_restrictions['forbidden_tags'])) {
+            @trigger_error('forbidden_tags for FilterInterface::getHTMLRestrictions() is deprecated in drupal:9.4.0 and is removed from drupal:10.0.0', E_USER_DEPRECATED);
             if (!isset($restrictions['forbidden_tags'])) {
               $restrictions['forbidden_tags'] = $new_restrictions['forbidden_tags'];

Are there any docs we need to update too or is this an undocumented feature? Also could use a @grou

+++ b/core/modules/filter/tests/filter_test/src/Plugin/Filter/FilterTestRestrictTagsAndAttributes.php
@@ -54,11 +54,6 @@ public function getHTMLRestrictions() {
       }
     }
-    if (isset($restrictions['forbidden_tags'])) {
-      foreach ($restrictions['forbidden_tags'] as $tag => $bool) {
-        $restrictions['forbidden_tags'][$tag] = (bool) $bool;
-      }
-    }
 

Is this completely untested then apart from this test code that doesn't look excercised? Was going to say we should add @group legacy coverage, but that might be optional if it's completely untested and undocumented.

catch’s picture

  1. +++ b/core/modules/filter/src/Plugin/FilterInterface.php
    @@ -198,7 +198,6 @@ public function process($text, $langcode);
        *             the possible values are TRUE or FALSE: to mark the attribute
        *             value as allowed or forbidden, respectively
    -   *     -  'forbidden_tags': (optional) the forbidden tags
        *
    

    Needs a (deprecated) note in the 9.4.x patch.

wim leers’s picture

#13:

  • RE: "undocumented feature?" — it is documented on \Drupal\filter\Plugin\FilterInterface::getHTMLRestrictions(). Update docs. And added a deprecation test.
  • RE "completely untested?" → yes. In fact, for the test I wrote here, I needed to restore this code, otherwise I would not have been able to write a deprecation test.
    And if I'd also enable filter_html for this test, you'd get an Undefined index: allowed PHP error too 😅
wim leers’s picture

StatusFileSize
new3.12 KB
new4.43 KB
new997 bytes
new15.59 KB

👆 d.o decided to post my comment even though I was just attaching files. 😬

wim leers’s picture

And if I'd also enable filter_html for this test, you'd get an Undefined index: allowed PHP error too 😅

That's now been proven by 3272516-deprecation-9.4-16.patch above 🤓

The "removal-10.0" patch from #16 is still relevant, no update needed.

Status: Needs review » Needs work

The last submitted patch, 17: 3272516-deprecation-9.4-17.patch, failed testing. View results

bnjmnm’s picture

Feedback on the D10 patch 1 of 2. My next step will be applying the patch to core locally and seeing if there's anything additional to remove or rephrase.

  1. +++ b/core/modules/ckeditor5/src/HTMLRestrictions.php
    @@ -38,7 +38,6 @@
      * NOTE: Currently only supports the 'allowed' portion.
    

    This "only supports" clarification is misleading now that 'allowed' is the only supported key.

  2. +++ b/core/modules/filter/src/Entity/FilterFormat.php
    @@ -376,28 +366,6 @@ public function getHtmlRestrictions() {
    -      if (isset($restrictions['allowed'])) {
    -        if (count($restrictions['allowed']) === 1 && array_key_exists('*', $restrictions['allowed']) && !isset($restrictions['forbidden_tags'])) {
    -          $restrictions['allowed'] = [];
    -        }
    -      }
    

    I noticed FilterAPITest was updated to reflect this removal, but should this actually be removed? It's a condition that is true when $restrictions['forbidden_tags'] is not present, so wouldn't we just remove the <code>isset($restrictions['forbidden_tags']) in the condition chain, but otherwise keep this the same

  3. +++ b/core/modules/filter/src/Plugin/FilterInterface.php
    @@ -184,7 +184,7 @@ public function process($text, $langcode);
        *     - 'allowed': (optional) the allowed tags as keys, and for each of those
    

    It could be argued 'allowed' is still *optional*, because there won't be an error if it isn't present. That's not particularly helpful, though. It should communicate that 'allowed' needs to be present for the filter to be of any use.

bnjmnm’s picture

D10 patch review 2 of 2, while installed on local core:

  1. \Drupal\ckeditor\Plugin\CKEditorPlugin\Internal has forbidden tags logic
       if (isset($html_restrictions['forbidden_tags'])) {
            foreach ($html_restrictions['forbidden_tags'] as $tag) {
              $disallowed[$tag] = TRUE;
            }
          }
    

    It looks like this might suggest an even larger LOC-removal opportunity in this class? generateACFSettings might have it's return structure changed?

  2. \Drupal\editor\EditorXssFilter\Standard::filterXss references 'forbidden' in a manner that is no longer accurate since a text format does not forbid tags anymore:
     // Therefore, we want to be smarter still. We want to take into account
        // which HTML tags are allowed and forbidden by the text format we're
        // filtering for, and if we're switching from another text format, we want
        // to take that format's allowed and forbidden tags into account as well.
        // In other words: we only expect markup allowed in both the original and
        // the new format to continue to exist.
    
  3. I notice \Drupal\filter_test\Plugin\Filter\FilterTestRestrictTagsAndAttributes is still there. It seems like it (and its corresponding schema) is quite removable since it is not actually used by tests in 9.x either.
wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new1.88 KB
new4.92 KB

#17 failed because there's a test case in ValidatorsTest triggering the deprecation. I think it's reasonable to just remove it.

bnjmnm’s picture

Status: Needs review » Needs work

Setting to "Needs work" for visibility #21 fixes a test failure, but the feedback in #19-#20 needs to be addressed. (at least according to the interdiff)

wim leers’s picture

wim leers’s picture

Assigned: wim leers » Unassigned
Status: Needs work » Needs review
StatusFileSize
new5.18 KB
new18.63 KB

#19:

  1. ✅ Removed it since it is no longer relevant.
  2. ✅ Interesting! I think that makes sense: if no tags are allowed, the global attribute restrictions also are meaningless. But … it could also be argued that they're harmless. Still, I think your proposal is closer to the original behavior, so … yep, went with that!
  3. ✅ Updated.

#20:

  1. 👎 While you're right in principle … that is code in the CKEditor 4 module, and that will be removed in Drupal 10 🤓 So rather pointless to update it.
  2. ✅ Good catch, updated!
  3. 👎 That's not true — this filter is being used, just not the forbidden_tags aspect of it. Removed those aspects. The allowed aspect of it is being used in the following tests: \Drupal\Tests\ckeditor\Kernel\CKEditorTest::testGetJSSettings(), \Drupal\Tests\filter\Kernel\FilterAPITest::testFilterFormatAPI(), \Drupal\Tests\filter\Kernel\FilterAPITest::testDependencyRemoval()
bnjmnm’s picture

Status: Needs review » Reviewed & tested by the community

#19 and #20 are either addressed or theres a good explanation as to why not. Lets move this to the next stage.

  • catch committed 35b8d4f on 10.0.x
    Issue #3272516 by Wim Leers, yogeshmpawar, bnjmnm, catch: Deprecate...
catch’s picture

Version: 10.0.x-dev » 9.4.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed the respective patches to to 10.x and 9.4.x, thanks!

  • catch committed c7a6942 on 9.4.x
    Issue #3272516 by Wim Leers, yogeshmpawar, bnjmnm, catch: Deprecate...
martijn de wit’s picture

Think this can also be commit in 9.5.x :)

wim leers’s picture

wim leers’s picture

@Martijn: 9.5.x has not yet been opened yet!

  • catch committed 7ef5ad6 on 9.5.x
    Issue #3272516 by Wim Leers, yogeshmpawar, bnjmnm, catch: Deprecate...
catch’s picture

@Wim Leers: yes it has! But I forgot - first commit since it opened and the muscle memory for 10.0.x -> 9.4.x is still there. Cherry-picked there too.

wim leers’s picture

:O I actually went and checked the Gitlab UI and it didn't list it 🤔

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.