Follow up from #2492839-116: Views replacement token bc layer allows for Twig template injection via arguments. Anytime you see the same code three times, it needs to be abstracted!

+++ b/core/modules/views/src/Plugin/views/PluginBase.php
@@ -357,34 +353,44 @@ protected function viewsTokenReplace($text, $tokens) {
         assert('preg_match(\'/^[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*$/\', $token) === 1', 'Tokens need to be valid Twig variables.');
...
+        assert('preg_match(\'/^[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*$/\', $top) === 1', 'Tokens need to be valid Twig variables.');
...
+          assert('preg_match(\'/^[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*$/\', $key) === 1', 'Tokens need to be valid Twig variables.');

We should either provide this regex as a constant or provide a Twig::isValidVariable() function.

CommentFileSizeAuthor
#21 2567269-21.patch6.72 KBrgoodine
#11 FireShot Screen Capture #069 - 'Подробности I SKITOURS' - skitours_com_ua_test_admin_reports_dblog_event_15842.png44.43 KBfomenkoandrey
#8 2567269-7-8.interdiff.txt1.76 KBmikeker
#8 2567269-8.Twig-utilities.patch5.96 KBmikeker
#7 interdiff-4-7.txt680 bytessnehi
#7 2567269-7.patch5.61 KBsnehi
#4 2567269-3-4.interdiff.txt2.46 KBmikeker
#4 2567269-4-Twig-utilities.patch5.61 KBmikeker
#3 2567269-3-Twig-utilities.patch3.14 KBmikeker
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

mikeker created an issue. See original summary.

joelpittet’s picture

joelpittet
Primary language English
Location Vancouver
CreditAttribution: joelpittet commented
Title: [Follow-up] Create a Twig regex constant or function that validates a Twig variable » Create a Twig regex constant or function that validates a Twig variable
Issue tags: +rc eligible

This seems like improve testing.

mikeker’s picture

mikeker CreditAttribution: mikeker as a volunteer commented
Status: Active » Needs review
FileSize
2567269-3-Twig-utilities.patch3.14 KB

How about this as a starting point?

mikeker’s picture

mikeker CreditAttribution: mikeker as a volunteer commented
FileSize
2567269-4-Twig-utilities.patch5.61 KB
2567269-3-4.interdiff.txt2.46 KB
mikeker’s picture

mikeker CreditAttribution: mikeker as a volunteer commented

Sorry, meant to say in the previous comment:

Adds a few more tests and cleans up the incorrect naming of an existing test (the "...WithTokens" test was testing replacement without tokens).

joelpittet’s picture

joelpittet
Primary language English
Location Vancouver
CreditAttribution: joelpittet commented
Status: Needs review » Needs work

@mikeker thanks for pushing this along. Here's a review:

  1. +++ b/core/lib/Drupal/Component/Utility/Twig.php
@@ -0,0 +1,43 @@
+   * @param  string $variable

    nit: extra space between @param and string. 

  2. +++ b/core/lib/Drupal/Component/Utility/Twig.php
@@ -0,0 +1,43 @@
+    return preg_match('/^[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*$/', $variable) === 1;
...
+    if (empty($message)) {
+      $message = '"' . Html::escape($variable) . '" is not a valid Twig variable.';
+    }
+    assert(Twig::isValidVariable($variable), $message);

    Before the preg match was happening in a string in the assert. We discussed in another issue about performance and asserts are not called unless in dev and turned on. But this setup will call preg_match() and empty() and Html::escape(). Which will all have a negative performance impact. Can we get away without this? 

  3. +++ b/core/lib/Drupal/Component/Utility/Twig.php
@@ -0,0 +1,43 @@
+   * Asserts if the specified string is not a valid Twig variable
+
+   * @param  string $variable

    nit: Missing a * on the next line and a period. Extra space between @param and string.

snehi’s picture

snehi CreditAttribution: snehi as a volunteer commented
Status: Needs work » Needs review
FileSize
2567269-7.patch5.61 KB
interdiff-4-7.txt680 bytes

As mentioned in #6.
1. Fixed
2. Don't know which part to remove only preg_match or all the functions.
3. Fixed.

mikeker’s picture

mikeker CreditAttribution: mikeker as a volunteer commented
FileSize
2567269-8.Twig-utilities.patch5.96 KB
2567269-7-8.interdiff.txt1.76 KB

@joelpittet, Thanks for the review! From #6:

Before the preg match was happening in a string in the assert. We discussed in another issue about performance and asserts are not called unless in dev and turned on. But this setup will call preg_match() and empty() and Html::escape(). Which will all have a negative performance impact. Can we get away without this?

Good point. If assert() is passed a string, then it's not eval'ed if assertions are turned off. That will save us the preg_match() call in production. However, I don't think there is a way to avoid Html::escape() unless we know that the token keys are safe (I don't believe they are) or that assert messages are escaped before being displayed (I have no idea...). I suppose we could check ASSERT_ACTIVE?

+++ b/core/modules/views/src/Tests/Plugin/PluginBaseTest.php
@@ -44,13 +44,59 @@ public function testViewsTokenReplace() {
+    try {
+      $text = '{{ foo-bar }}';
+      $tokens = [
+        '{{ foo-bar }}' => 'bogus',
+      ];
+
+      \Drupal::service('renderer')->executeInRenderContext(new RenderContext(), function () use ($text, $tokens) {
+        return $this->testPluginBase->viewsTokenReplace($text, $tokens);
+      });
+    }
+    catch(\Exception $e) {
+      $this->assertIdentical('"foo-bar" is not a valid Twig variable.', $e->getMessage());
+    }

Added a $this->fail() at the end of the try block. Otherwise the test never fails and that's not a very good test... :)

mikeker’s picture

mikeker CreditAttribution: mikeker as a volunteer commented

Also, this issue should always be tested against both PHP 5.5 and PHP 7 as the assert function changes substantially between the two versions.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

fomenkoandrey’s picture

fomenkoandrey CreditAttribution: fomenkoandrey commented
FileSize
FireShot Screen Capture #069 - 'Подробности I SKITOURS' - skitours_com_ua_test_admin_reports_dblog_event_15842.png44.43 KB

PHP7
drupal 8.1.3

Warning: assert(): Tokens need to be valid Twig variables.: "preg_match('/^[a-zA-Z_\x7f-\xff][a-zA-Z0-9_\x7f-\xff]*$/', $key) === 1" failed в Drupal\views\Plugin\views\PluginBase->viewsTokenReplace() (строка 376 файла /home/skitours/public_html/core/modules/views/src/Plugin/views/PluginBase.php).

mikeker’s picture

mikeker CreditAttribution: mikeker as a volunteer commented

@fomenkoandrey: Can you provide some more details about the view causing the assert. Specifically any tokens used in field rewrites.

Thanks.

fomenkoandrey’s picture

fomenkoandrey CreditAttribution: fomenkoandrey commented

view: http://savepic.net/8212030.png

Field of date, when node was changed.
override output: {{ changed }}, {{ uid }}
http://savepic.net/8192574.png

Field body, cropped to 600 characters, then overriden to 300 and with link with token {{ path }}
http://savepic.net/8209982.png
http://savepic.net/8216126.png
http://savepic.net/8213054.png

3 filter:
author not admin http://savepic.net/8202814.png
changed <720 hours http://savepic.net/8203838.png
author - opened for users http://savepic.net/8207934.png

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

markhalliwell’s picture

markhalliwell
Primary language English
CreditAttribution: markhalliwell at Tag1 Consulting commented
Version: 8.5.x-dev » 8.6.x-dev
Status: Needs review » Needs work
Issue tags: -rc eligible +Needs reroll

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

rgoodine’s picture

rgoodine CreditAttribution: rgoodine as a volunteer commented

Working on this for Global Sprint Weekend.

rgoodine’s picture

rgoodine CreditAttribution: rgoodine as a volunteer commented
FileSize
2567269-21.patch6.72 KB

Attached is the re-rolled patch.

Minor conflicts in `/core/modules/views/tests/src/Kernel/Plugin/PluginBaseTest.php` & `PluginBase.php` were resolved.

rgoodine’s picture

rgoodine CreditAttribution: rgoodine as a volunteer commented
Status: Needs work » Needs review
Issue tags: -Needs reroll

Status: Needs review » Needs work

The last submitted patch, 21: 2567269-21.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.