Problem/Motivation
When creating a paste filter to replace special space characters like Thin and En (U+2009 and U+2002) with regular spaces (U+0020), the Replacement does not respect escaped values like: \u0020 or \x20.
Although the Replacement field can be populated by typing a single space, this appears no different than an empty field and can be a point of confusion in long term management of a site.
Steps to reproduce
1. Install the CKEditor 5 Paste Filter module
2. Add or edit a text format (/admin/config/content/formats)
3. Set the Text editor of the text format to CKEditor 5
4. Under CKEditor 5 plugin settings select the Paste filter vertical tab
5. Enable the plugin by checking the Filter pasted content checkbox
6. Add a custom filter
7. Save the text format: Scroll to the bottom and click Save configuration
8. Add a new node using the configured text format (/node/add)
9. Paste the rich content into the editor
Search expression: [\u{2000}-\u{200A}]
Replacement: \u0020
Markup samples
<p>Veritas sapientia scientia natura fortuna,</p>
<p>Lorem ipsum vita terra firma officia</p>Markup result (pasting without filtering)
<p>Veritas sapientia scientia natura fortuna,</p>
<p>Lorem ipsum vita terra firma officia</p>Markup result (pasting with filtering)
<p>Veritas\u0020sapientia scientia\u0020natura\u0020fortuna,</p>
<p>Lorem ipsum vita\u0020terra firma\u0020officia</p>Expected markup result
<p>Veritas sapientia scientia natura fortuna,<p>
<p>Lorem ipsum vita terra firma officia,</p>
Comments
Comment #2
star-szrOh hi @littlecoding!
I might be missing something, but it seems like for this use case you could instead replace with an HTML character reference such as
 for a regular space. I don't think CKEditor 5 knows what to do with the JavaScript-style character escape within an HTML context.Comment #3
littlecodingIn my use case, the goal is to replace the plain old space and avoid introducing HTML entities.
Other use cases could be:
In the admin UI, the filter Replacement value for all of these cases all look the same. And could be confused with being empty like a filter that removes matches.
Here are the Search Expression and Replacement value in action (via regex101) https://regex101.com/r/rthrOB/1
Comment #4
star-szr@littlecoding have you tried with HTML entities as replacements? CKEditor 5 should convert those to the plain characters anyway, so they won't end up as HTML entities in your markup.
Search expression:
[\u{2000}-\u{200A}]Replacement:
 An approach along these lines should work for all the use cases you mentioned unless I'm missing something, and accomplish your goal of being able to see the replacement on the Paste filter config form when editing the text format.
Comment #5
littlecodingThank you, @star-szr. This has helped me identify a quirk of the DEV server in use on a project. Replacement with an HTML entity is working fine on my LOCAL stack (DDEV) and the client's STAGING server.
And with some digging into the code a little:
web/modules/contrib/ckeditor5_paste_filter/js/ckeditor5_plugins/pasteFilter/src/pastefilter.js
lines 34-37
html = html.replace(
new RegExp(filter.search, 'gimsu'),
filter.replace,
);
According to Mozilla's reference document on String.prototype.replace(), the use of escaped characters is supported among the special replacement patterns when the replacement parameter is a string type. But this is not the behaviour we are observing.
But using HTML element(s) as the replacement string (field value), a later line of code kicks in.
line 51
data.content = this.editor.data.htmlProcessor.toView(html);
From the CKEditor 5 API, the HTML data processor method toView() parses and converts the pasted string into a `ViewDocumentFragment`. The parsing does replace some of the base HTML elements with UFT8 characters. In the case here
 is replace with the space character (U+0020). And as we can all guess now, the parsing ignores HTML elements like No-Break Space .Hope the info helps someone else down the road.
Closing the issue.