[no patch] Consolidate change records relating to safe markup and filtering/escaping to ensure cross references exist

a group of us are sprinting on this issue as part of NHDevDays2 D8 Accelerate sprint

I'm pretty sure https://www.drupal.org/core/issue-priority defines minor as typos. And this is more than that. So changing to normal.

See also #2280965-20: [meta] Remove every SafeMarkup::set() call about the reordering of the recommended replacements in the CR. Or here are notes on it from our planning meeting:

Update "4 options" so that SafeMarkup::format() is first, document use-cases of the inline templates and calling the service
When to use SafeMarkup::format() vs. #type inline templates
Inline templates are alterable by modules, themes, overrides

Evaluated the following change records and updated as indicated:

1. https://www.drupal.org/node/2296163 - updated description to be clearer
2. https://www.drupal.org/node/2302363
3. https://www.drupal.org/node/2311123 - removed option 3, changed option 4 to option 2, added link to safemarkup class description located at https://www.drupal.org/node/2296163
4. https://www.drupal.org/node/2445441

https://www.drupal.org/node/2489900 is a draft change record that will possibly need to be combined once published

we will evaluate #2280965-20 again based on your recommendation

Thanks @leslieg! Marking for review.

https://www.drupal.org/node/2311123 should be merged into the main change record in https://www.drupal.org/node/2296163, and should also mention that using render arrays is also a preferred option in contexts that support them, with #markup used as a best practice for applying XSS admin filtering.

Notes from #2554889: Remove SafeMarkup::set() from the codebase:

https://www.drupal.org/node/2311123 includes nothing about render arrays or the item list template, but leveraging render arrays was one of the ways we were able to get rid a lot of SafeMarkup::set() usage in core. Also the links in https://www.drupal.org/node/2549395 are not accessible; we should add actual link text (unlike what I did here) ;). Finally, we need a g.d.o/core post announcing this as well given the impact.

Edit: Also, we need to cover how to do things that are incompatible with Xss::filterAdmin() somewhere.

Also the SafeMarkup::format() example in https://www.drupal.org/node/2311123 arguably should be a Twig template. Anyway, I updated it with stubs for how it should be updated. I also think it should actually stop to exist as a separate thing entirely and be merged into https://www.drupal.org/node/2296163 because its title makes no sense now (I had filed an issue for this to happen several months ago but looks like it hasn't happened).

I can work on these things later if someone else doesn't get to them first.

Working on those two parts of it to start, and also on https://www.drupal.org/node/2549395.

• I deleted https://www.drupal.org/node/2489900 because the change it proposed is not included in core.
• I retitled https://www.drupal.org/node/2296163/revisions/view/8473230/8837793 and updated it to de-emphasize SafeMarkup and remove a couple safe explanations.
• I updated https://www.drupal.org/node/2302363/revisions/view/8787151/8837711 for #2552579: Remove SafeMarkup::placeholder(), deprecate drupal_placeholder() and stop drupal_placeholder() from marking safe..
• I retitled https://www.drupal.org/node/1312890/revisions/view/8559613/8837809.
• I made https://www.drupal.org/node/2445441/revisions/view/8367305/8837825 more human-readable and suggested that it should be updated for #2558791: "!"-prefixed tokens should Xss::filterAdmin() but not affect safeness on that issue.

Lots of CRs need to be checked over for the checkPlain() deprecation as well. These should have gotten fixed before that patch went in probably, but we should definitely do so now:

https://www.drupal.org/list-changes/published/drupal?keywords_descriptio...

https://www.drupal.org/list-changes/published/drupal?keywords_descriptio...

Also see:

• #2545972-84: Remove all code usages SafeMarkup::checkPlain() and rely more on Twig autoescaping
• #2545972-95: Remove all code usages SafeMarkup::checkPlain() and rely more on Twig autoescaping
• #2555473: Deprecate SafeMarkup::checkPlain() for Drupal 8.0.x

https://www.drupal.org/node/2392803 might also want an update; I believe render arrays are now supported for the link generator as well.

Updated: https://www.drupal.org/node/2457593/revisions/view/8302341/8838229

Added: https://www.drupal.org/node/2296163/revisions/view/8837793/8838257

And I'll update all of those to reference that.

This message is added to all the other CRs (copy-pastable for your future SafeMarkup changing fun!)

<h3>Related change records</h3>

See <a href="https://www.drupal.org/node/2296163">Twig autoescape enabled and text sanitization APIs updated</a> for a full list of related change records.

I also reduced the inline_template CR back to cover what it actually covers:
https://www.drupal.org/node/2311123/revisions/view/8828245/8838343

And began consolidating the concatenation strategies onto the main CR:
https://www.drupal.org/node/2296163/revisions/view/8838269/8838345

From #2555473-32: Deprecate SafeMarkup::checkPlain() for Drupal 8.0.x:

Alright, @alexpott, @effulgentsia, and I reworked the main change record for the method removals in SafeMarkup methods are removed. There are still a couple outstanding things that I think should be improved (copying and expanding notes from our google docs draft):

1. If the string is used in a Twig template, rely on Twig's auto-escaping feature. Simply remove the call to SafeMarkup::checkPlain(). For example, most SafeMarkup::checkPlain() calls in template preprocess functions can be removed. Another common case is the construction of a render array that uses a Twig template. For example, data that is being themed using an item list or table does not need to be escaped as Twig will do this for you. If text that has already been escaped using Html::escape() ends up in a Twig template variable, it will be double-escaped.

   This is clearer now, but as a reader of this I still would not entirely understand "is used in a template". First of all, there are ways to use strings unsafely in a template. Do we have a Twig resource we can reference about how output ends up in templates and how it is used safely? Secondly, there are many cases where a developer is creating output that will be used in an existing template without the developer knowing anything about it. (Maybe we need a chart somewhere indicating which strings will be escaped, which strings will be filtered, and which will not be sanitized. That doesn't need to block this though.)

2. For example, most SafeMarkup::checkPlain() calls in template preprocess functions can be removed

   We should clarify when they can't be removed. For example, if the string is XSS-filtered, this will be more permissive than escaping and bypass it because the resulting string will be safe.

3. Another common case is the construction of a render array that uses a Twig template. For example, data that is being themed using an item list or table does not need to be escaped as Twig will do this for you.

   This example is helpful. A code snippet might make it more so. However, some render array elements, like #prefix, #suffix, #value, and #markup are not escaped (only admin filtered) so this sentence is somewhat too vague.

I am all for improving documentation of all kinds but this is for change records for a now unsupported version of Drupal. There has been work here for 9 years as well. I suggest it would be better to focus our time on improving documentation and change records for supported versions. By this time, people are using internet searched to find support on these topics.

Although this was not 100% complete, I am going to close it as fixed to allow credit for those who worked on the change records. If you disagree with this decision, re-open and add a comment.

Thanks!