Problem/Motivation
Drupal\ckeditor5\SmartDefaultSettings::addToolbarItemsToMatchHtmlAttributesInFormat isn't necessarily handling use cases where CKEditor plugin only supports specific attribute value in the best possible way. Currently it will enable plugins that allow ALL values for an attribute. This means the attribute+value is now allowed but additional attribute values are permitted as well. This may need to be more selective.
If we don't do that, we could end up allowing <p class> instead of only allowing <p class="text-align-center">, for example.
Steps to reproduce
Proposed resolution
Remaining tasks
User interface changes
API changes
Data model changes
| Comment | File | Size | Author |
|---|---|---|---|
| #63 | 3231328-63.patch | 50.06 KB | wim leers |
| #63 | interdiff.txt | 4.91 KB | wim leers |
| #60 | 3231328-60.patch | 50.04 KB | wim leers |
| #60 | interdiff.txt | 16.98 KB | wim leers |
| #59 | 3231328-59.patch | 47.78 KB | wim leers |
Comments
Comment #2
wim leersThis should wait for #3228334: Refactor HTMLRestrictionsUtilities to a HtmlRestrictions value object to finish; that will make this simpler.
Comment #4
wim leersOnce #3259174: Add missing CKE5 SmartDefaultSettings test coverage (wildcard tag with unsupported attribute) lands, the impact of this will be much clearer thanks to us being able to refine that test coverage in this issue! 😊
Comment #5
wim leers#3259174: Add missing CKE5 SmartDefaultSettings test coverage (wildcard tag with unsupported attribute) seems to already solve this … but it does not. It still picks the first CKEditor 5 plugin which addresses a need.
This issue is basically saying we should pick the most specific one. That would avoid (to the degree this is possible) that the upgrade path results in more tags/attributes/attribute values being allowed after the upgrade than before.
Comment #6
wim leersComment #7
wim leers#3228334: Refactor HTMLRestrictionsUtilities to a HtmlRestrictions value object is in!
This adds the necessary test coverage: this should fail.
(And actually, the test coverage should expect the center and justify alignment buttons to appear, not the generic one. But this initial patch demonstrates the problem most clearly. A different set of buttons being selected is what will help meet this expectation.)
Comment #8
wim leersAnd actually … if #3259593: Alignment being available as separate buttons AND in dropdown is confusing lands first, then that part will remain the same, and then the patch in #7 is perfect in terms of updated expectations!
Comment #10
wim leersAs of #3263384-10: Add ckeditor5-code-block package and CodeBlock plugin, this issue blocks #3263384, and since that is , I have to bump this up too. Fortunately we already have a test case: #7.
Comment #11
wim leersThis updates
\Drupal\ckeditor5\SmartDefaultSettings::addToolbarItemsToMatchHtmlAttributesInFormat()to make the test in #7 pass.Note: #3263384: Add ckeditor5-code-block package and CodeBlock plugin needs us to minimize the superset of tags, not attributes. So more work is needed:
\Drupal\ckeditor5\SmartDefaultSettings::addToolbarItemsToMatchHtmlTagsInFormat()should also be updated. But this is already a key step forward.Comment #12
wim leersWill tackle #11 next, but would love a review in the meantime — hoping @bnjmnm has time, since he was the original architect of this algorithm 🤓😊
Comment #13
wim leerscspell…
Comment #14
wim leersTo help reviewing #11:
This has just been moved into a new helper, that computes exactly
$net_newas it exists in HEAD, but also$superset(which is basically how much adding this plugin results in "overshooting": which HTML elements it adds that we didn't actually want/need).Rather than selecting a plugin, this merely lists all candidates, and provides a score to select one later.
Because we didn't select, only found candidates, this no longer needs to be updated in every loop.
This now makes a selection. It chooses the plugin which adds what we needs with the smallest possible superset.
Tada! 🤩
Comment #15
wim leersComment #16
wim leersTest coverage for what I described as still remaining in #11.
Comment #18
bnjmnmThe explanation for $potential_future is very clear, I think it would be worth doing the same for $superset.
Also, wouldn't $superset be an inaccurate name since it diffs against baseline? Perhaps something like $surplus_additions would be more accurate and descriptive?
Would it be better to move $superset_score up a few levels of nesting? The performance benefits of avoiding the repeat calls are likely negligible. The biggest benefit to me is the move makes it more apparent to code viewers that the score is not dependent on which iteration within the loop(s) it is being determined.
I think the superset terminology could be changed to more accurate represent what is happening - find the candidate with the lowest [surplus elements?] score to avoid allowing excess tags/attributes when possible.
Using the broader mathematic terms in HTMLRestrictions makes sense since it's responsible for making the actual comparisons. When dealing with the data processed by HTMLRestrictions, shifting the descriptions to be more context specific could be helpful.
Is there any risk of giving a more permissive superset a lower score due to it configuring attributes/attribute values as TRUE instead of an array of allowed values?
I was going to point mention this test depends on the redundant (and likely to be removed) alignment plugin, but more tests were added as I was reviewing this 🙂
Comment #19
wim leers\Drupal\ckeditor5\Plugin\CKEditor5PluginManager::findPluginSupportingElement()is too simplistic: it returns the first match it founds, without any consideration for whether it's the best (minimal!) match. That's what the test coverage added in #16 is proving through its failure.That means that even though that's a public API, we should deprecate it; its only use is being dropped in this patch. I'll leave that for the next patch iteration.
This patch iteration drastically simplifies
SmartDefaultSettings. Rather than having lots of complex logic to deal with the "HTML restrictions" array structure, we can now leverage the infrastructure that #3228334: Refactor HTMLRestrictionsUtilities to a HtmlRestrictions value object introduced! That removes the need for the twoaddToolbarItemsToMatchHtmlTagsInFormat()+addToolbarItemsToMatchHtmlAttributesInFormat()methods — just one will do now! :)P.S.: Can I just point out that
diffstatfor the interdiff? 🤓 🤩→ fixing things by having less code, and that's even without deleting the now unused
\Drupal\ckeditor5\Plugin\CKEditor5PluginManager::findPluginSupportingElement()! 🥳 Rare occasion, but so nice when it happens!Comment #20
wim leers#18:
Comment #21
nod_This patch makes the MR in #3263384: Add ckeditor5-code-block package and CodeBlock plugin pass
Comment #22
wim leersGreat! :D Thanks for checking that :)
Comment #23
wim leersComment #24
alisonDo you think the issue described over on #2710427 is related to what y'all are working on here? (#2710427: Broken "Allowed Tags" updating: after all values for an attribute are allowed, it should not be overridden to allow only certain attribute values)
I tried the patch on #20, and it didn't fix the issue we're having on that other issue, but holy heck does
SmartDefaultSettingslook and feel related, so I figured it can't hurt to ask.Thank you!
Comment #25
wim leersThis issue definitely cannot be related, since this issue only affects the CKEditor 5 module!
Comment #26
nod_<unspecified:true>and<unspecified:false>, we add a bunch of sprintf and then do a preg_match to see what's the value. Why the "<>". We can do a TRUE/FALSE key in the array maybe? then the pregmatch can be replace by is_bool. It's a magic key either way, not like people will see what's in there.This makes it so that
<code class data-language>has the same score of 3 as<code class="language">. I would expect the first to have 3 and the second to have 2 as the score, which means having the count to stop at depth 2 (if we start depth at 1) and ignore the rest no?In the case of codeblock, if the code plugin has
<code data-language>for theelementsconfig, then this doesn't work as expected and codeBlock is used instead because the 2 plugins share the same score and codeBlock is the last plugin to be processed (which overrides the plugin_id for the surplus_score.It's fine if we assume that the most appropriate plugin has the least amount of additional attributes
I would use
...[$plugin_id] = $surplus_score;to avoid overwrites.
Comment #27
nod_Was a problem previously but this is not translatable like this.
Comment #28
wim leers#26:
[$tag => TRUE]and[$tag => FALSE](as opposed to[$tag => […]]). I wanted to make 100% sure that it could not be mistaken/interpreted for an attribute name. By using this weird syntax, it's very obvious that this is NOT an attribute name 🤓We cannot use
TRUEorFALSEas the key names, sadly. That's what I had at first — it gets cast to integers automatically: https://3v4l.org/k2BKZ. That's why I made these string representations. But … you're right, it does work 99% of the time, and it's for internal use only. The only thing that's kinda tricky is the interpretation of what is a boolean key when iterating over them, because they're no longer booleans. But we can deal with that.Implemented your suggestion! 😊
… and just prior to uploading the patch, I noticed that it fails. So there is a subtle problem caused by relying on the PHP typecasting after all. But please do double-check the changes I made!
<code class data-language>would need a higher score than<code class="language">. But I disagree with stopping the count at depth 2 — because<code class="language foo bar">should have a higher score than<code class="language">. Will tackle that next.#27: yep, definitely out of scope here. I think it makes sense to tackle this as part of #3245967: Messages upon switching to CKEditor 5 are overwhelming.
Comment #29
wim leersNow addressing #26.2+3.
Comment #30
nod_For #4 it's not real syntax, just didn't want to copy paste the code... This is trying to avoid overwrites when 2 plugin have the same score.
For #2 stopping at depth 2 isn't the right solution you're right.
Comment #31
alisonWhoops! Thanks for explaining for me!
Comment #32
nod_Yeah this coupled with the implicit casting to number is not great for readability/long term maintainance. Maybe we can find a couple of words, like
attributes-noneandattributes-anyI think it's the preg_match that bothered me the most here. We could also do a in_array for the two possible values instead of a regex. That would be less bad (for me at least)
Comment #33
wim leers#32:
AFAICT #32 is saying that the previous approach kind of was okay/clearer in some contexts, but that the
preg_match()was especially weird/confusing. I'm then also assuming that the use of angular brackets made it look like HTML and made it extra confusing. SO! Simpler proposal: rather than<unspecific:true>and<unspecific:false>(mine in #20) or the problematicTRUE/FALSE(#28), I'm also not going to go withattributes-noneorattributes-anyas proposed in #32 because that too can lead to edge cases. I've got something far simpler:-true-and-false-, or-any-and-none-, because leading dashes are not allowed in attributes. No confusion with HTML tags possible :)I'm gonna go with
-attributes-any-and-attributes-none-, since that stays closer to the spirit of #32! 😊Comment #34
wim leersThis addresses #26.4 — this indeed is far clearer/better/simpler! I'm really glad you caught that! 😄
Comment #35
nod_Very nice solution to prefix the names by "-" making sure they are not valid attributes. +1 to the new names, it's clearer and more… specific than "unspecific" :)
Comment #36
wim leersThis addresses #26.2+3. Still needs tests.
Comment #37
nod_that's supposed to be 1000 here I think
Missing a line, comment not finished here.
technically it's language-* that is allowed here, that's what most people use, not "lang" since it's a test pretty minor comment though.
Comment #38
wim leersRefactor to make it possible to write tests, and make it possible to write better docs.
Comment #39
wim leers(Marking for the tests.)
#37.1 & #37.2: yep, and the tests I'm working on will should confirm/reveal those bugs :)
#37.3: Well, it was just meant to be a placeholder equivalent for #3263384: Add ckeditor5-code-block package and CodeBlock plugin anyway — the
codeBlockPlaceholdertoolbar item will also need to be changed in #3263384!Comment #40
wim leersTests written. This was tricky, because no unit tests existed yet. Writing kernel tests for this would be extremely unwieldy. Fortunately there was one precedent already in the CKEditor 5 module's unit tests 👍
Tests should fail, because of #37.1 at the very least.
Comment #41
wim leersAnd here's the fix. This fixes #37.1 + #37.2. #40 added tests to show that #26.2 + #26.3 have been addressed, but tests were failing; now they should pass, which means that it is indeed proven.
If @nod_ finds more nuance he wants to add to the surplus scores, we can now easily do that, because expanding test coverage is now trivial 👍
Comment #43
nod_I would move the surplus count in a separate method it'll get complex pretty fast.
<code data-dada data-tata><code data-*>Attributes values
<code class="language1 language2"><code class="language*">*I guess what I expect the score to be is something like:
<pre>1000000<code *>0100000<code data-*>0010000<code class>0001000<code class>0000100<code class="lang-*">0000010<code class="lang-js">0000001<code>0000000That would give something like (when looking at the support for the code tag):
10010101 new attribute,
1 attributes with wildcard value restriction
00010022 allowed values
00020044 allowed values
010000000011001 attribute without value restriction
10221102 new attributes wildcards,
2 new attributes,
1 attributes without value restriction,
1 attributes with wildcard restriction
We have to stop somewhere though, so I don't know how much effort should go into this upgrade path given it's a one-off thing with a low effort to fix problems if they show up (interface is on a single page, with fancy ajax UI, the set of buttons available is limitted).
Comment #44
wim leers#43:
I agree. But better to get it right now, while both your head and mine are in this space, rather than having to revisit. Tackling.
This just addresses #43.1. And in doing so, I realized that we've been computing the surplus score far too often all this time 🙈 yay for refactoring!
Comment #45
nod_Just realized that we can't have more than 9 additional thing otherwise it'll break down. maybe an array of integers instead of an integer would be better?
Comment #46
wim leers🤩🤩🤩🤩
The tables with your perspective and the samples were both super helpful! There's only one bug in them, in the very last row, it says — but there's actually three of those. So the expected score is actually
1022310instead of1022110Comment #47
wim leersOops, this is the right interdiff for #46 — I just re-uploaded the #45 interdiff in #46 🙈
Comment #48
wim leers#45: >9 tags is not a problem (because it's already the biggest number). Only >9 attributes etc matter. I do not think it's going to happen very often that a single plugin explicitly enables that many. And even if that happens, the results will still be sorted by decreasing surplus score. If we're entering that territory, we're seeing some absurdly exotic CKEditor 5 plugins. So … yeah … I'm fine with that letting happen — this issue is a huge leap forward already, especially now that we've implemented your #43 proposal 🤓
Comment #49
nod_yeah that's fair enough, +1
Comment #50
wim leerscspell😬Comment #52
wim leersD'oh, I only implemented #43 in #46, and added
\Drupal\Tests\ckeditor5\Unit\SmartDefaultSettingsTest::testSurplusScore()… but forgot to update the expected scores in\Drupal\Tests\ckeditor5\Unit\SmartDefaultSettingsTest::providerComputeCandidates()🙈🙈🙈🙈I was a little bit overenthusiastic obviously.
Comment #53
nod_This looks great to me. Spend a lot of time on this so the comment seems enough for me.
Since it's so smart now we can even simplify things so that #3268174: Bug in CKE 4 → 5 upgrade path "format" does not always map to "heading", it could map to "codeBlock" too, or both, or neither is not actually a blocker for #3263384: Add ckeditor5-code-block package and CodeBlock plugin.
As far as review goes, took me some time to get into it, adding a dump/breakpoint in
computeCandidatesandaddToolbarItemsToMatchHtmlElementsInFormathelps (I like the dump because it makes the ajax call fail and we can just retrigger it as necessary without reloading the page).Comment #54
wim leersRight, but #3268174: Bug in CKE 4 → 5 upgrade path "format" does not always map to "heading", it could map to "codeBlock" too, or both, or neither already landed, so it's already not a blocker anyway :P
Hah, this is similar to one of my debugging productivity tricks: I just do a
exit;before a response is sent, then I can simply re-run an XHR request from the browser console :)Comment #55
lauriiiShould we link to https://www.drupal.org/project/drupal/issues/3231328#comment-14444987 for explanation on the score? It does a really good job at explaining it.
Infinity could be exaggeration given that we can't handle ints above
PHP_INT_MAX.I don't think we usually use compute as the verb in method names. I think we could rename this to
getCandidates.How does this work in cases where plugin depends on another plugin, like
media_mediaAlign?I see three values listed here 🤔
This causes some problems with the logic below because
$missing_attributesis now always truthy.s/HtmlRestrictions/HTMLRestrictions/
This method is a beast 🤯 I need to take a break and come back to this later.
Comment #56
wim leersComment #57
wim leersRerolled after #3268174: Bug in CKE 4 → 5 upgrade path "format" does not always map to "heading", it could map to "codeBlock" too, or both, or neither landed, which conflicted in
\Drupal\Tests\ckeditor5\Kernel\SmartDefaultSettingsTest. Both were adding new test cases in the same place. Easy fix.Comment #58
wim leers#55:
🤓 But I also don't think this is an issue in 99.99% of cases. There always is an explicit toolbar item to support a particular tag.
ckeditor5_imageCaption,ckeditor5_imageAlign,ckeditor5_drupalMediaCaptionandmedia_mediaAlignare all examples of this kind of dependent plugin, where it not only is automatically enabled based on some conditions but also adds support for additional elements (<img data-caption>,<img data-align>,<drupal-media data-caption>,<drupal-media data-align>respectively). But all four of these only do that if a filter is enabled and a toolbar item is present. All other plugins that are conditionally enabled do not add more elements:ckeditor5_linkImage,ckeditor5_linkMediaandckeditor5_imageResizedepend on other plugins but do not add more elements.ckeditor5_htmlSupportandckeditor5_pasteFromOfficeare enabled unconditionally but don't add more elements either.👉 But … how do we guarantee this? Well, the good news is that we can. We can simply add more validation to
\Drupal\ckeditor5\Plugin\CKEditor5PluginDefinition::validateDrupalAspects()to prevent "risky conditions". Shall I open a follow-up to address that 0.01% case?HTMLRestrictionsvalue object, but didn't update these if-tests. (The other one wasif (!empty($unsupported)) {). Fixed both.selectCandidate()helper method. Will do that in the next iteration.Comment #59
wim leersI just realized after posting #58 that I forgot to update the unit test manually, because PHPStorm's refactoring functionality isn't smart enough for that just yet … 🙈
Comment #60
wim leersThis addresses #55.8 as described in #58.8, and expands the unit test coverage accordingly.
Self-RTBC'ing because:
Comment #61
wim leersFYI: the order in the "selected plugins" array is weird (as you can see: the "attribute name" reason is in a higher level in the tree structure than the "tag name" reason 🙃). But this already is the case in HEAD.
Changing that is trivial, but I was just trying to minimize change.
I think this contributed to .
Comment #62
lauriii#58: I don't know what's the likelihood that we ever run into that but I guess we can deal with that later if we figure out a case where that would be needed.
Can we just add
@see \Drupal\filter\Plugin\Filter\FilterHtml::getHTMLRestrictionshere since otherwise this is really confusing 😅Can we just add a brief explanation here that this is critical information later so that additional plugins are not enabled for no reason if the goal was to add support for an additional attribute?
So much sadness in the comments of this method 🥲
Can we add explanation to why we want to minimize the reasons? Also would be good to explain why we think supporting
<a href>is less relevant than<a>since that seems confusing statement?What is the purpose of the intermediary
$selected_pluginsvariable?Comment #63
wim leers$plugins_to_enable_to_support_attribute_configto$selected_plugins. So … did that.(The confusing remaining part is described in #61. We could change that here too if you prefer. I don't feel strongly either way. #3245967: Messages upon switching to CKEditor 5 are overwhelming will probably have to change things in this area anyway.)
Comment #67
lauriiiCommitted 4d02e19 and pushed to 10.0.x. Also cherry-picked to 9.4.x and 9.3.x since CKEditor 5 is experimental. Thanks!