Non-UTF-8 characters in query parameters break json/javascript

Problem/Motivation

This has originally been reported as a possibly security issue and cleared for public reporting.

json_encode() only supports UTF-8.

system_js_settings_alter() puts the path and query arguments into drupalSettings, if those contain non-UTF-8 characters, the drupal-settings-json section is empty and parsing it causes a JS error.

This in turn causes most other JS to fail entirely.

I don't think you can do XSS or anything like it, at best you can confuse users, the more a site relies on JS, the more broken it is.

A client discovered this due to a newsletter tool not using UTF-8 and putting such characters in utm_content query arguments.

Steps to reproduce

Visit /?foo=%F6 on an Drupal 8+ site.

Proposed resolution

Explicitly instruct json_encode to ignore non-utf8 characters.

Remaining tasks

From @larowlan in the security issue:

+++ b/core/lib/Drupal/Core/Asset/JsCollectionRenderer.php
@@ -83,7 +83,7 @@ public function render(array $js_assets) {
-          $element['#value'] = Json::encode($js_asset['data']);
+          $element['#value'] = json_encode($js_asset['data'], JSON_HEX_TAG | JSON_HEX_APOS | JSON_HEX_AMP | JSON_HEX_QUOT | JSON_INVALID_UTF8_IGNORE);

Should we be adding JSON_INVALID_UTF8_IGNORE to \Drupal\Component\Serialization\Json::encode too?

response @berdir:

Not sure, I think as an API, it shouldn't silently drop that. Maybe check the status, trigger a warning in the log and plan to throw an exception in D10/11? (JSON_THROW_ON_ERROR)

User interface changes

API changes

Data model changes

Release notes snippet