Problem/Motivation

Update to CKEditor 5 v34.1.0 in core when this URL no longer 404s: https://github.com/ckeditor/ckeditor5/releases/tag/v34.1.0

v34.1 unblocks:

  1. #3271418 — see #3271418-8: [upstream] [drupalMedia] Linked media wrapped with <div> doesn't upcast correctly
  2. #3273552 — see #3273552-6: [upstream] codeBlock breaks when changing the `language-*` class
  3. #3247634 — see #3247634-7: [upstream] [drupalImage] Unlinking linked inline images while GHS is enabled: wrapping <a> is impossible to remove → this requires test updates because they were asserting the bug existed, see #21
  4. #3274635 — see #3274635-6: [upstream] Use CKEditor 5's native <ol type> and <ul type> UX

Note that this will also fix #3277405-3: Update @ckeditor/ckeditor5-list to v34.0.1.

Proposed resolution

  1. core/package.json
  2. cd core
  3. yarn install
  4. yarn run vendor-update
  5. yarn run build:ckeditor5
  6. yarn run build:ckeditor5-types

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

TBD

Comments

Wim Leers created an issue. See original summary.

Wim Leers credited Reinmar.

wim leers’s picture

Title: Update to CKEditor 5 v35.0.0 » Update to CKEditor 5 v34.1.0/v35.0.0
Issue summary: View changes

Per @Reinmar:

Also, we’re not yet sure whether it will be a major release – I don’t think there were BCs on major APIs so it may also be 34.1.0
wim leers’s picture

wim leers’s picture

Title: Update to CKEditor 5 v34.1.0/v35.0.0 » Update to CKEditor 5 v34.1.0
Assigned: Unassigned » wim leers
Issue summary: View changes
Status: Postponed » Active
Issue tags: +9.4.0 release notes, +9.3.14 release notes
wim leers’s picture

Assigned: wim leers » Unassigned
Status: Active » Needs review
StatusFileSize
new1.07 MB
new1.07 MB

Because #3266912: Review version constraints for production yarn dependencies did not land in 9.3.x, we need a different patch here for that branch. All 3 other branches that need to be updated can use the same patch (9.4.x, 9.5.x, 10.0.x) — it applies cleanly to all three 👍

The only differences between the two patches are in core/package.json and consequently also in core/yarn.lock

wim leers’s picture

😬😬😬

$ git pull
warning: redirecting to https://git.drupalcode.org/project/drupal.git/
Already up to date.

$ git apply -3v 3277438-7-others.patch
Checking patch core/assets/vendor/ckeditor5/alignment/translations/af.js...
Applied patch to 'core/assets/vendor/ckeditor5/alignment/translations/af.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/basic-styles/basic-styles.js...
Applied patch to 'core/assets/vendor/ckeditor5/basic-styles/basic-styles.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/basic-styles/translations/af.js...
Applied patch to 'core/assets/vendor/ckeditor5/basic-styles/translations/af.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/block-quote/block-quote.js...
Applied patch to 'core/assets/vendor/ckeditor5/block-quote/block-quote.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/block-quote/translations/af.js...
Applied patch to 'core/assets/vendor/ckeditor5/block-quote/translations/af.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/code-block/code-block.js...
Applied patch to 'core/assets/vendor/ckeditor5/code-block/code-block.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/editor-classic/editor-classic.js...
Applied patch to 'core/assets/vendor/ckeditor5/editor-classic/editor-classic.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/heading/heading.js...
Applied patch to 'core/assets/vendor/ckeditor5/heading/heading.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/heading/translations/lv.js...
Applied patch to 'core/assets/vendor/ckeditor5/heading/translations/lv.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/horizontal-line/horizontal-line.js...
Applied patch to 'core/assets/vendor/ckeditor5/horizontal-line/horizontal-line.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/html-support/html-support.js...
Applied patch to 'core/assets/vendor/ckeditor5/html-support/html-support.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/image/image.js...
Applied patch to 'core/assets/vendor/ckeditor5/image/image.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/image/translations/lv.js...
Applied patch to 'core/assets/vendor/ckeditor5/image/translations/lv.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/language/language.js...
Applied patch to 'core/assets/vendor/ckeditor5/language/language.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/link/link.js...
Applied patch to 'core/assets/vendor/ckeditor5/link/link.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/link/translations/lv.js...
Applied patch to 'core/assets/vendor/ckeditor5/link/translations/lv.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/list/list.js...
Applied patch to 'core/assets/vendor/ckeditor5/list/list.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/list/translations/lv.js...
Applied patch to 'core/assets/vendor/ckeditor5/list/translations/lv.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/paste-from-office/paste-from-office.js...
Applied patch to 'core/assets/vendor/ckeditor5/paste-from-office/paste-from-office.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/remove-format/translations/nl.js...
Applied patch to 'core/assets/vendor/ckeditor5/remove-format/translations/nl.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/source-editing/source-editing.js...
Applied patch to 'core/assets/vendor/ckeditor5/source-editing/source-editing.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/special-characters/special-characters.js...
Applied patch to 'core/assets/vendor/ckeditor5/special-characters/special-characters.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/table/table.js...
Applied patch to 'core/assets/vendor/ckeditor5/table/table.js' cleanly.
Checking patch core/assets/vendor/ckeditor5/table/translations/lv.js...
Applied patch to 'core/assets/vendor/ckeditor5/table/translations/lv.js' cleanly.
Checking patch core/core.libraries.yml...
Applied patch to 'core/core.libraries.yml' cleanly.
Checking patch core/modules/ckeditor5/js/build/ckeditor5.types.jsdoc...
Applied patch to 'core/modules/ckeditor5/js/build/ckeditor5.types.jsdoc' cleanly.
Checking patch core/package.json...
Applied patch to 'core/package.json' cleanly.
Checking patch core/yarn.lock...
Applied patch to 'core/yarn.lock' cleanly.
Applied patch core/assets/vendor/ckeditor5/alignment/translations/af.js cleanly.
Applied patch core/assets/vendor/ckeditor5/basic-styles/basic-styles.js cleanly.
Applied patch core/assets/vendor/ckeditor5/basic-styles/translations/af.js cleanly.
Applied patch core/assets/vendor/ckeditor5/block-quote/block-quote.js cleanly.
Applied patch core/assets/vendor/ckeditor5/block-quote/translations/af.js cleanly.
Applied patch core/assets/vendor/ckeditor5/code-block/code-block.js cleanly.
Applied patch core/assets/vendor/ckeditor5/editor-classic/editor-classic.js cleanly.
Applied patch core/assets/vendor/ckeditor5/heading/heading.js cleanly.
Applied patch core/assets/vendor/ckeditor5/heading/translations/lv.js cleanly.
Applied patch core/assets/vendor/ckeditor5/horizontal-line/horizontal-line.js cleanly.
Applied patch core/assets/vendor/ckeditor5/html-support/html-support.js cleanly.
Applied patch core/assets/vendor/ckeditor5/image/image.js cleanly.
Applied patch core/assets/vendor/ckeditor5/image/translations/lv.js cleanly.
Applied patch core/assets/vendor/ckeditor5/language/language.js cleanly.
Applied patch core/assets/vendor/ckeditor5/link/link.js cleanly.
Applied patch core/assets/vendor/ckeditor5/link/translations/lv.js cleanly.
Applied patch core/assets/vendor/ckeditor5/list/list.js cleanly.
Applied patch core/assets/vendor/ckeditor5/list/translations/lv.js cleanly.
Applied patch core/assets/vendor/ckeditor5/paste-from-office/paste-from-office.js cleanly.
Applied patch core/assets/vendor/ckeditor5/remove-format/translations/nl.js cleanly.
Applied patch core/assets/vendor/ckeditor5/source-editing/source-editing.js cleanly.
Applied patch core/assets/vendor/ckeditor5/special-characters/special-characters.js cleanly.
Applied patch core/assets/vendor/ckeditor5/table/table.js cleanly.
Applied patch core/assets/vendor/ckeditor5/table/translations/lv.js cleanly.
Applied patch core/core.libraries.yml cleanly.
Applied patch core/modules/ckeditor5/js/build/ckeditor5.types.jsdoc cleanly.
Applied patch core/package.json cleanly.
Applied patch core/yarn.lock cleanly.
wim leers’s picture

StatusFileSize
new1.07 MB

Ah … if I don't do -3v but just -v, I can reproduce it.

I really wish DrupalCI applied patches with -3 (for 3-way merge). It'll fail if it cannot resolve conflicts with absolute confidence. Ah well 😔

10.0.x-specific patch attached.

nod_’s picture

Status: Needs review » Needs work

missing the update to the `ckeditor5` package :)

wim leers’s picture

So I forgot

"ckeditor5": "34.0.x",

in devDependencies 😬

But … metadata detail doesn't even matter because it looks like this caused consistent failures in ImageTest and MediaTest functional JS tests. Those will need to be investigated.

nod_’s picture

It does impact the ckeditor-dll.js file so might be relevant?

wim leers’s picture

Status: Needs work » Needs review
StatusFileSize
new2.51 MB
new2.51 MB
new2.51 MB

D'oh, right!

This was pretty painful — can't wait to not have to generate patches for four branches 🤪

Looks like I made not one mistake (#10) but two: I hadn't noticed previously that new plugin translations were available! 🙈

$ git status
On branch 9.5.x
Your branch is ahead of 'origin/9.5.x' by 5 commits.
  (use "git push" to publish your local commits)

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	core/assets/vendor/ckeditor5/alignment/translations/ur.js
	core/assets/vendor/ckeditor5/basic-styles/translations/ur.js
	core/assets/vendor/ckeditor5/block-quote/translations/ur.js
	core/assets/vendor/ckeditor5/ckeditor5-dll/translations/gu.js
	core/assets/vendor/ckeditor5/ckeditor5-dll/translations/ms.js
	core/assets/vendor/ckeditor5/ckeditor5-dll/translations/ur.js
	core/assets/vendor/ckeditor5/code-block/translations/af.js
	core/assets/vendor/ckeditor5/code-block/translations/ur.js
	core/assets/vendor/ckeditor5/heading/translations/ur.js
	core/assets/vendor/ckeditor5/horizontal-line/translations/ur.js
	core/assets/vendor/ckeditor5/html-support/translations/lv.js
	core/assets/vendor/ckeditor5/html-support/translations/ur.js
	core/assets/vendor/ckeditor5/image/translations/ur.js
	core/assets/vendor/ckeditor5/indent/translations/ur.js
	core/assets/vendor/ckeditor5/language/translations/lv.js
	core/assets/vendor/ckeditor5/link/translations/ur.js
	core/assets/vendor/ckeditor5/list/translations/ur.js
	core/assets/vendor/ckeditor5/source-editing/translations/lv.js
	core/assets/vendor/ckeditor5/special-characters/translations/lv.js
	core/assets/vendor/ckeditor5/table/translations/ur.js

👆 These are all new! Mostly ur aka Urdu 👍

nod_’s picture

Status: Needs review » Reviewed & tested by the community

All good for me :)

The last submitted patch, 13: 3277438-13-10.0.patch, failed testing. View results

The last submitted patch, 13: 3277438-13-9.4_and_9.5.patch, failed testing. View results

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 13: 3277438-13-9.3.patch, failed testing. View results

wim leers’s picture

#11:

But … metadata detail doesn't even matter because it looks like this caused consistent failures in ImageTest and MediaTest functional JS tests. Those will need to be investigated.

… still happening. So: genuine failures. 😬

wim leers’s picture

I got a message from CKEditor 5 developer https://github.com/niegowski:

hey, CKE5 34.1.0 just landed on npm and it might affect your plugin integration with GHS DataFilter#_consumeAllowedAttributes() is now public (was protected but AFAIR you were using it in one of your plugins) and is now known as DataFilter#processViewAttributes()  https://github.com/ckeditor/ckeditor5/issues/10827

Maybe that's causing the failures 🤞

bnjmnm’s picture

#19 takes care of the failing Media tests, but not the fails in \Drupal\Tests\ckeditor5\FunctionalJavascript\ImageTest::testLinkability. looking into that now...

bnjmnm’s picture

The image test is because of a no-longer-needed workaround that is specifically targeted here #3277438: Update to CKEditor 5 v34.1.0. I'm going to close that issue as it has to solved for the update to be applied at all.

Status: Needs review » Needs work

The last submitted patch, 21: 3277438-20-10x.patch, failed testing. View results

lauriii’s picture

Status: Needs work » Needs review
StatusFileSize
new2.58 MB
new2.58 MB
new2.58 MB
new1.35 KB
xjm’s picture

+++ b/core/modules/ckeditor5/tests/src/FunctionalJavascript/ImageTest.php
@@ -386,27 +386,14 @@ public function testLinkability(string $image_type, bool $unrestricted) {
     $assert_session->elementExists('css', '.ck-content .ck-widget.' . $expected_widget_class);
-    // @todo Remove the different assertion for the "inline, unrestricted" case when https://www.drupal.org/project/ckeditor5/issues/3247634 is fixed.
-    if ($image_type === 'inline' && $unrestricted) {
-      $assert_session->elementNotExists('css', '.ck-content a[href]');
-      $assert_session->elementExists('css', '.ck-content a.trusted');
-    }
-    else {
-      $assert_session->elementNotExists('css', '.ck-content a');
-    }
+    $assert_session->elementNotExists('css', '.ck-content a');
     $assert_session->elementExists('css', '.ck-content .ck-widget.' . $expected_widget_class . ' > img[src*="image-test.png"][alt="drupalimage test image"]');
 
     // Assert the "dataDowncast" HTML after making changes.
     $xpath = new \DOMXPath($this->getEditorDataAsDom());
     $this->assertCount(0, $xpath->query('//a[@href="http://www.drupal.org/association"]/img[@alt="drupalimage test image"]'));
     $this->assertCount(1, $xpath->query('//img[@alt="drupalimage test image"]'));
-    // @todo Remove the different assertion for the inline cases when https://www.drupal.org/project/ckeditor5/issues/3247634 is fixed.
-    if ($image_type === 'inline') {
-      $this->assertCount(1, $xpath->query('//a'));
-    }
-    else {
-      $this->assertCount(0, $xpath->query('//a'));
-    }
+    $this->assertCount(0, $xpath->query('//a'));

Just getting this into the issue comments since the patch is 2.5 MB of mostly-automated updates. This is the test discussed in #21 and the only thing that needs actual code review.

lauriii’s picture

Status: Needs review » Reviewed & tested by the community

Reviewed changes since #14 and all looks good. I realize that I authored #23 but I think the changes in #23 and #21 are simple enough that it shouldn't prevent from moving this issue to RTBC.

xjm’s picture

Soo I'm not getting the same results as the patch. Steps for the 10.0.x patch:

  1. cd core
  2. emacs package.json replace all 34.0.x with 34.1.x throughout the file.
  3. rm -rf node_modules; yarn install
  4. yarn run vendor-update
  5. yarn run build:ckeditor5
  6. yarn run build:ckeditor5-types

I would expect the diff between this and #23 to only be the image test, but the actual diffstat is:

[ayrton:core | Thu 14:51:54] $ git diff 10.0.x-cke5 --stat
 .../vendor/ckeditor5/alignment/translations/ur.js       |  1 -
 .../vendor/ckeditor5/basic-styles/translations/ur.js    |  1 -
 .../vendor/ckeditor5/block-quote/translations/ur.js     |  1 -
 .../vendor/ckeditor5/ckeditor5-dll/translations/gu.js   |  1 -
 .../vendor/ckeditor5/ckeditor5-dll/translations/ms.js   |  1 -
 .../vendor/ckeditor5/ckeditor5-dll/translations/ur.js   |  1 -
 .../vendor/ckeditor5/code-block/translations/af.js      |  1 -
 .../vendor/ckeditor5/code-block/translations/ur.js      |  1 -
 core/assets/vendor/ckeditor5/heading/translations/ur.js |  1 -
 .../vendor/ckeditor5/horizontal-line/translations/ur.js |  1 -
 .../vendor/ckeditor5/html-support/translations/lv.js    |  1 -
 .../vendor/ckeditor5/html-support/translations/ur.js    |  1 -
 core/assets/vendor/ckeditor5/image/translations/ur.js   |  1 -
 core/assets/vendor/ckeditor5/indent/translations/ur.js  |  1 -
 .../assets/vendor/ckeditor5/language/translations/lv.js |  1 -
 core/assets/vendor/ckeditor5/link/translations/ur.js    |  1 -
 core/assets/vendor/ckeditor5/list/translations/ur.js    |  1 -
 .../vendor/ckeditor5/source-editing/translations/lv.js  |  1 -
 .../ckeditor5/special-characters/translations/lv.js     |  1 -
 core/assets/vendor/ckeditor5/table/translations/ur.js   |  1 -
 core/core.libraries.yml                                 |  3 ++-
 core/modules/ckeditor5/js/build/drupalMedia.js          |  2 +-
 .../drupalMedia/src/drupalmediageneralhtmlsupport.js    |  2 +-
 .../tests/src/FunctionalJavascript/ImageTest.php        | 17 +++++++++++++++--
 24 files changed, 19 insertions(+), 25 deletions(-)
xjm’s picture

StatusFileSize
new2.5 MB

In #26 I just forgot to stage the new files; the actual diffstat against 23 is:

 core/core.libraries.yml                                 |  3 ++-
 core/modules/ckeditor5/js/build/drupalMedia.js          |  2 +-
 .../drupalMedia/src/drupalmediageneralhtmlsupport.js    |  2 +-
 .../tests/src/FunctionalJavascript/ImageTest.php        | 17 +++++++++++++++--
 4 files changed, 19 insertions(+), 5 deletions(-)

...whereas it should just be ImageTest. Not sure why core.libraries.yml doesn't match either.

Here's the full patch I would get, minus ImageTest changes.

xjm’s picture

StatusFileSize
new869 bytes

OK here is the missing step 0: Apply attached one-line change.

Then follow steps in #26. Result:

[ayrton:core | Thu 15:20:14] $ git diff 10.0.x-cke5 --stat
 core/core.libraries.yml                                 |  3 ++-
 .../tests/src/FunctionalJavascript/ImageTest.php        | 17 +++++++++++++++--
 2 files changed, 17 insertions(+), 3 deletions(-)

...Which is what I expected, plus or minus a todo in the libraries file that should be manually removed.

xjm’s picture

Here are the changes to the built assets in 10.0.x:

diff --git a/core/modules/ckeditor5/js/build/drupalMedia.js b/core/modules/ckeditor5/js/build/drupalMedia.js
index 744fde4cf4..1bbb80584a 100644
--- a/core/modules/ckeditor5/js/build/drupalMedia.js
+++ b/core/modules/ckeditor5/js/build/drupalMedia.js
@@ -3142,7 +3142,7 @@ function viewToModelDrupalMediaAttributeConverter(dataFilter) {
       'element:drupal-media',
       (evt, data, conversionApi) => {
         function preserveElementAttributes(viewElement, attributeName) {
-          const viewAttributes = dataFilter._consumeAllowedAttributes(
+          const viewAttributes = dataFilter.processViewAttributes(
             viewElement,
             conversionApi,
           );

That is identical to the change in the interdiff from #28, in the built file. So that is reassuring.

xjm’s picture

Confirmed the 9.5.x/9.4.x change is the same as #29.

  • xjm committed 3b3972e on 10.0.x
    Issue #3277438 by Wim Leers, bnjmnm, lauriii, xjm, nod_, Reinmar: Update...

  • xjm committed 90085a8 on 9.5.x
    Issue #3277438 by Wim Leers, bnjmnm, lauriii, xjm, nod_, Reinmar: Update...

  • xjm committed 7ab0af4 on 9.4.x
    Issue #3277438 by Wim Leers, bnjmnm, lauriii, xjm, nod_, Reinmar: Update...

  • xjm committed 6a67477 on 9.3.x
    Issue #3277438 by Wim Leers, bnjmnm, lauriii, xjm, nod_, Reinmar: Update...
xjm’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: -9.3.14 release notes +9.3.15 release notes

Just as validation for the RTBC in #25, @bnjmnm confirmed that he was about to upload an identical fix as #23 and practically crossposted with it.

Committed to 10.0.x, 9.5.x, 9.4.x, and 9.3.x. Phew! There is a new adventure almost every time we update CKE5.

Updating release tag because there was a sec release yesterday, so this will be a new fix next week on the June bugfix window.

Thanks everyone!

wim leers’s picture

Issue summary: View changes

#21: hah — I even said that in the issue summary: it's the 3rd unblocked issue! 😄 But I forgot that it would break tests, because our tests were literally designed to break if the fix would land upstream 🙈 Thanks for sorting that out!

"The 4 issues from the issue summary that are unblocked:

  1. 🥳 #3271418: [upstream] [drupalMedia] Linked media wrapped with <div> doesn't upcast correctly is now marked fixed after I manually confirmed it!
  2. 🥳 #3273552: [upstream] codeBlock breaks when changing the `language-*` class is now marked fixed after I manually confirmed it!
  3. 🥳 #3247634: [upstream] [drupalImage] Unlinking linked inline images while GHS is enabled: wrapping <a> is impossible to remove is now marked fixed after @bnjmnm updated the test coverage in #21 (see above) and I manually tested it.
  4. 🥳 #3274635: [upstream] Use CKEditor 5's native <ol type> and <ul type> UX now has one less blocker

That means three 2 fewer stable blockers! 🥳 (EDIT: 2, not 3, because #3273552: [upstream] codeBlock breaks when changing the `language-*` class was not a stable blocker.)

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.