Problem/Motivation

Quoting #2545972-71: Remove all code usages SafeMarkup::checkPlain() and rely more on Twig autoescaping.1, which is a review of a patch that made me discover this long-standing problem:

+++ b/core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/BasicStringFormatter.php
@@ -37,7 +37,7 @@ public function viewElements(FieldItemListInterface $items) {
-      $elements[$delta] = array('#markup' => nl2br(SafeMarkup::checkPlain($item->value)));
+      $elements[$delta] = array('#markup' => nl2br(Html::escape($item->value)));

It's a pre-existing problem, but this will cause Markup's filtering safe strategy to still be applied. Even though that's absolutely unnecessary.

The only tag that is going to be in there is <br>, by definition. And that is allowed by \Drupal\Component\Utility\Xss::getAdminTagList().

However, using SafeString::create() here or creating another SafeStringInterface implementation for this also feels kinda overkill.

Not sure what's best.

Proposed resolution

TBD

Remaining tasks

TBD

User interface changes

None.

API changes

None.

Data model changes

None.

Comments

Wim Leers created an issue. See original summary.

joelpittet’s picture

Status: Active » Needs review
StatusFileSize
new926 bytes

Many ways to skin a escaped string.
http://twig.sensiolabs.org/doc/filters/nl2br.html

The value will be escaped normally by twig and the new lines will be converted to br tags. BOOM!

wim leers’s picture

Status: Needs review » Reviewed & tested by the community

Perfect!

andypost’s picture

StatusFileSize
new1.19 KB
new1.22 KB

+1 rtbc, just clean-up "use" and inified array syntax

alexpott’s picture

Is there a test for this?

andypost’s picture

Yep, there's
\Drupal\field\Tests\String\RawStringFormatterTest::testStringFormatter
\Drupal\field\Tests\String\StringFormatterTest::testStringFormatter

also \Drupal\Core\Field\Plugin\Field\FieldFormatter\StringFormatter::viewValue() does the same, so maybe override that too?

andypost’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new1.37 KB
new2.6 KB

I'm sure better re-title

Status: Needs review » Needs work

The last submitted patch, 7: basicstringformatter_s-2554755-7.patch, failed testing.

The last submitted patch, 7: basicstringformatter_s-2554755-7.patch, failed testing.

andypost’s picture

Status: Needs work » Needs review
StatusFileSize
new1.81 KB
new4.41 KB
joelpittet’s picture

Status: Needs review » Needs work
+++ b/core/modules/field/src/Tests/String/UuidFormatterTest.php
@@ -47,11 +47,11 @@ public function testUuidStringFormatter() {
-    $this->assertIdentical($render_array[0]['#markup'], $entity->uuid(), 'The rendered UUID matches the entity UUID.');
+    $this->assertIdentical($render_array[0]['#context']['value'], $entity->uuid(), 'The rendered UUID matches the entity UUID.');
...
-    $this->assertIdentical($render_array[0]['#title']['#markup'], $entity->uuid());
+    $this->assertIdentical($render_array[0]['#title']['#context']['value'], $entity->uuid());

Can we just render the array and get the value so we don't have to keep changing this test if the way we build the data changes later?

$this->render()

Thanks @andypost, that makes all the other things consistent. I think it's good to go here too, it's a bit scope creepy but tolerable I think.

lauriii’s picture

Status: Needs work » Needs review
StatusFileSize
new4.43 KB
new1.05 KB

Fixed #11

joelpittet’s picture

Status: Needs review » Reviewed & tested by the community

Thanks setting back to RTBC.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 12: basicstringformatter_s-2554755-12.patch, failed testing.

joelpittet’s picture

Status: Needs work » Reviewed & tested by the community

Looks like unrelated failure.

Drupal\system\Tests\Update\UpdatePathTestBaseTest	388	1	0
									
Message	Group	Filename	Line	Function	Status
GET http://ec2-54-191-113-78.us-west-2.compute.amazonaws.com/checkout/update.php/results returned 0 (0 bytes).
andypost’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new2.63 KB
new5.91 KB

I'm sure we need to extend formatters tests
- UuidFormatterTest tests build of StringFormatter
- RawStringFormatterTest tests build of BasicStringFormatter

Now tests are the same

joelpittet’s picture

Status: Needs review » Needs work
+++ b/core/modules/field/src/Tests/String/UuidFormatterTest.php
@@ -46,13 +46,23 @@ public function testUuidStringFormatter() {
+    $this->assertIdentical($render_array[0]['#title']['#type'], 'inline_template');
+    $this->assertIdentical($render_array[0]['#title']['#template'], '{{ value|nl2br }}');
+    $this->assertIdentical($render_array[0]['#title']['#context']['value'], $entity->uuid());

@andypost the reason I asked @lauriii to change is it is because this is testing how we built something and not what we built. Though I appreciate more tests I think this the wrong kind of test. It should test the output is as expected not the "how we built the output".

andypost’s picture

@joelpittet as you see I extend tests same way as they are. Mean testing how formatter returns build, also checking how that rendered.

+++ b/core/modules/field/src/Tests/String/UuidFormatterTest.php
@@ -46,13 +46,23 @@ public function testUuidStringFormatter() {
-    $this->assertIdentical($render_array[0]['#markup'], $entity->uuid(), 'The rendered UUID matches the entity UUID.');
+    $this->assertIdentical($render_array[0]['#type'], 'inline_template');
+    $this->assertIdentical($render_array[0]['#template'], '{{ value|nl2br }}');
...
     $this->assertIdentical($render_array[0]['#type'], 'link');

for me that exactly a test of the formatter build

joelpittet’s picture

Yes, that's what I'm saying we shouldn't be testing "how it's built" but rather we should test "what is the end result of the render of the array". Does that make sense?

Put in other words: Right now it's testing the input is there instead of testing the output we get from that input.

dawehner’s picture

Does someone mind trying to profile how much this makes for example /admin/content slower, with let's say 50 nodes?

joelpittet’s picture

Scenario

xdebug off, twig C extension off. xhprof on.

  1. Added a plaintext long field to article,
  2. set the field on teaser display view mode,
  3. generated 50 nodes,
  4. ensured nl2br was being rendered from twig template and back to 8.0.x branch for the xhprof.
  5. Turned of render cache bins (render and dynamic_page)
=== 8.0.x..8.0.x compared (55fa31db35b03..55fa3a49350d5):

ct  : 252,217|252,217|0|0.0%
wt  : 616,541|616,601|60|0.0%
mu  : 29,857,784|29,857,784|0|0.0%
pmu : 30,011,168|30,011,168|0|0.0%

http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=55fa31db35b03&...

=== 8.0.x..2554755 compared (55fa31db35b03..55fa3a641876b):

ct  : 252,217|249,109|-3,108|-1.2%
wt  : 616,541|610,817|-5,724|-0.9%
mu  : 29,857,784|29,929,568|71,784|0.2%
pmu : 30,011,168|30,083,776|72,608|0.2%

http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?run1=55fa31db35b03&...

dawehner’s picture

Oh this is really nice. As we no longer use #markup, we no longer have to Xss::filter() the result, which saves us quite some time, see http://www.lionsad.de/xhprof-kit/xhprof/xhprof_html/?symbol=Drupal%5CCom...

joelpittet’s picture

Oh I forgot to mention but I'm only showing 10 nodes at a time, should have changed it to 50 but I'm sure the % would be similar.

joelpittet’s picture

@andypost re #20 is there any reason why we should be testing the input?

andypost’s picture

Status: Needs work » Needs review
StatusFileSize
new1.28 KB
new4.8 KB

reverted #17 changes and added more render tests, should be ready now

joelpittet’s picture

Status: Needs review » Reviewed & tested by the community

Thank you @andypost. Still not sure why some of this needs to test the input but part of that was there before this patch so as long as it's testing the output I'm happy:)

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Twig making things quicker... now there is a turn up for the books! Nice work all. Committed 3137a14 and pushed to 8.0.x. Thanks!

  • alexpott committed 3137a14 on 8.0.x
    Issue #2554755 by andypost, lauriii, joelpittet: BasicStringFormatter's...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.