Problem/Motivation

It appears that using the "ToC Chunker" may interfere with content output, specifically if the content does not pass DomDocument's HTML validation.

Steps to reproduce

  1. Install and enable Sector ToC along with its dependencies
  2. Configure a content type to use "ToC Chunker" on the Body field at /admin/structure/types/manage/<type>/display
  3. Create a piece of content which uses an unrecognised HTML tag, such as <drupal-media>, <drupal-entity-embed>, or (unconfirmed) HTML5 tags such as <command>, <details> or <figure>
  4. View the content; when chunked, unrecognised tags will be stripped from the HTML source before the filters are applied

Proposed resolution

  1. Verify the reported problem, ideally by introducing test coverage
  2. Agree on a solution
  3. Implement the solution

Remaining tasks

User interface changes

API changes

Data model changes

Comments

xurizaemon created an issue. See original summary.

xurizaemon’s picture

xurizaemon
he/him
Location Ōtepoti, Aotearoa 🏝
commented
Issue summary: View changes
xurizaemon’s picture

xurizaemon
he/him
Location Ōtepoti, Aotearoa 🏝
commented

Reproduction notes:

Configured the "Full HTML" format with CKEditor5, "Manually editable HTML tags" permitting additional tags ` `. It was not necessary to add ` ` as these tags were already provided by a CKEditor plugin.

Input HTML

The Body of the node was set to:

<h2>Apples</h2><h3>Granny Smith</h3><p>Use&nbsp;<command>sudo gimme apple --type=granny-smith</command> to retrieve a Granny Smith from /dev/urandom.</p><h3>Fuji</h3><p>Not a picture of an apple at all.</p><drupal-media data-entity-type="media" data-entity-uuid="3c2d60eb-31a8-4c66-a96a-4e772b0493ff" data-view-mode="full_width">&nbsp;</drupal-media><h3>Gala</h3><p>Secrets about Gala applesThese apples are cuboid if grown in space.</p><h2>Oranges</h2><h3>Mandarins</h3><h3>Tangelos</h3><h2>Melons</h2><h3>Watermelons</h3><h3>Rockmelons</h3><p>&nbsp;</p>

Output when Body display is set to "ToC Chunker"

Excerpted, the first paragraph outputs as:

<p>Use sudo gimme apple --type=granny-smith to retrieve a Granny Smith from /dev/urandom.</p>

The media in the second section is not displayed at all.

Output when Body display is set to "Chunker"

<p>Use&nbsp;<command>sudo gimme apple --type=granny-smith to retrieve a Granny Smith from /dev/urandom.</command></p>

This also appears wrong - the closing tag of the <command> container has moved to the end of the line?!

Output when Body display is set to "Default"

Use&nbsp;<command>sudo gimme apple --type=granny-smith to retrieve a Granny Smith from /dev/urandom.</command></p>

The closing tag of the <command> container has moved to the end of the line again. For this reason I am not (yet) moving the issue from Sector ToC to Chunker; the removal of unrecognised tags is the issue being reported here.

(I expect that moving the closing </command> tag is either my own mistake, some other local configuration, or perhaps a CKEditor behaviour. Most likely the first!)

Analysis

I see that some of the HTML modification is coming from Chunker, but also that the behaviour is worse (removes Drupal media embeds) in Sector ToC.

Chunker below is processing the source HTML - so non-HTML markup such as <drupal-*> may be visible in the source, as may be HTML5-valid tags such as <command>. It may be that Chunker should run later in the process - especially if the filters might impact the heading structure of the document.

When we're in Chunker's \Drupal\chunker\Plugin\Field\FieldFormatter\ChunkerFormatter::viewElements(), inside the foreach() on $items we have a single item, where the value is:

<h2>Apples</h2><h3>Granny Smith</h3><p>Use&nbsp;<command>sudo gimme apple --type=granny-smith</command> to retrieve a Granny Smith from /dev/urandom.</p><h3>Fuji</h3><p>Not a picture of an apple at all.</p><drupal-media data-entity-type="media" data-entity-uuid="3c2d60eb-31a8-4c66-a96a-4e772b0493ff" data-view-mode="full_width">&nbsp;</drupal-media><h3>Gala</h3><p>Secrets about Gala applesThese apples are cuboid if grown in space.</p><h2>Oranges</h2><h3>Mandarins</h3><h3>Tangelos</h3><h2>Melons</h2><h3>Watermelons</h3><h3>Rockmelons</h3><p>&nbsp;</p>

The `#text` value of `$elements` after this foreach is set to:

<div class="chunker-section"><h2 id="section-1">Apples</h2><h3>Granny Smith</h3><p>Use&nbsp;<command>sudo gimme apple --type=granny-smith to retrieve a Granny Smith from /dev/urandom.</p><h3>Fuji</h3><p>Not a picture of an apple at all.</p><drupal-media data-entity-type="media" data-entity-uuid="3c2d60eb-31a8-4c66-a96a-4e772b0493ff" data-view-mode="full_width">&nbsp;</drupal-media><h3>Gala</h3><p>Secrets about Gala applesThese apples are cuboid if grown in space.</p></div><div class="chunker-section"><h2 id="section-2">Oranges</h2><h3>Mandarins</h3><h3>Tangelos</h3></div><div class="chunker-section"><h2 id="section-3">Melons</h2><h3>Watermelons</h3><h3>Rockmelons</h3><p>&nbsp;</p></div>

In debugger output we see warnings such as:

PHP Warning: DOMDocument::loadHTML(): Tag drupal-media invalid in Entity, line: 1 in /app/web/ on line 294
PHP Warning: DOMDocument::loadHTML(): Tag details invalid in Entity, line: 1 in /app/web/ on line 294
PHP Warning: DOMDocument::loadHTML(): Tag summary invalid in Entity, line: 1 in /app/web/ on line 294

Notes

I did investigate whether applying the change from #2441811: Upgrade filter system to HTML5 to core modified this behaviour, but it did not seem to have any impact.

jonathan_hunt’s picture

jonathan_hunt commented

I was not able to reproduce this using Chunker formatter alone, as output format for basic page body. However, with Sector ToC the <drupal-media> present in the body text area is not rendered. This is on a text area that is not using CKEditor5. So it's either Sector ToC or toc_api at issue...

xurizaemon’s picture

xurizaemon
he/him
Location Ōtepoti, Aotearoa 🏝
commented

Possibility this is related to #2986763: FIX for html stripped by Xss:filterAdmin() in TocBuilder::buildContent() as well, not yet investigated (thanks @jonathan_hunt).

ericgsmith’s picture

ericgsmith commented

I think the debugger warnings mentioned are fine - they are coming from https://git.drupalcode.org/project/drupal/-/blob/10.1.x/core/lib/Drupal/... which is suppressed for a reason.

I have tested using patch from #2986763: FIX for html stripped by Xss:filterAdmin() in TocBuilder::buildContent() and was not able to reproduce the issue - I strongly suspect applying the patch from 2986763 will resolve this issue. That should perhaps be added somewhere as a needed step (project page?) if confirmed.

xurizaemon’s picture

xurizaemon
he/him
Location Ōtepoti, Aotearoa 🏝
commented
Status: Active » Closed (duplicate)
Related issues: +#2986763: FIX for html stripped by Xss:filterAdmin() in TocBuilder::buildContent()

Yep, it appears that this was a manifestation of #2986763: FIX for html stripped by Xss:filterAdmin() in TocBuilder::buildContent(), thanks Eric. Applying the patch from that issue appears to have addressed this behaviour.