Problem/Motivation

Twig 2 was released in early 2017. While Drupal 8 should keep Twig 1 for backwards compatibility, we should optionally support running with Twig 2 in preparation to Drupal 9. #2600154: Update our Twig code to be ready for Twig 2.x made most of the required changes to support Twig 2 but not all of them.

Proposed resolution

Fix all remaining issues with Twig 2 compatibility. Prove with tests that it works.

Remaining tasks

Review. Commit.

User interface changes

N/A.

API changes

None.

Data model changes

None.

Release notes snippet

Drupal 8.7 core now supports optional use of Twig 2 in place of Twig 1. This does not by itself mean contributed modules and themes will work outright. See https://twig.symfony.com/doc/1.x/deprecated.html for a complete list of deprecated functionality from Twig 1, in case you need to make changes. Drupal core's use of Twig did not change APIs on Drupal's API surface.

CommentFileSizeAuthor
#121 2572605-121-twig2.patch10.25 KBandypost
#121 2572605-121.patch7.79 KBandypost
#121 interdiff.txt915 bytesandypost
#120 2572605-120.patch7.78 KBGábor Hojtsy
#120 2572605-120-with-twig2.patch10.24 KBGábor Hojtsy
#120 interdiff.txt973 bytesGábor Hojtsy
#114 2572605-113.patch8.73 KBalexpott
#114 2572605-twig2-113.patch11.19 KBalexpott
#114 112-113-interdiff.txt1.82 KBalexpott
#112 2572605-112.patch9.48 KBGábor Hojtsy
#112 2572605-112-with-twig2.patch11.94 KBGábor Hojtsy
#112 interdiff.txt1.48 KBGábor Hojtsy
#109 2572605-109.patch8 KBGábor Hojtsy
#109 2572605-twig2-109.patch10.46 KBGábor Hojtsy
#105 2572605-105.patch8.77 KBalexpott
#105 2572605-twig2-105.patch11.23 KBalexpott
#105 98-105-interdiff.txt1.72 KBalexpott
#101 2572605-98.patch7.65 KBGábor Hojtsy
#101 2572605-99-only-for-testing.patch10.12 KBGábor Hojtsy
#98 2572605-98.patch7.65 KBGábor Hojtsy
#98 interdiff.txt2.23 KBGábor Hojtsy
#95 2572605-89-code-only.patch7.87 KBGábor Hojtsy
#91 2572605-89-with-composer-twig2-from-81.patch10.33 KBGábor Hojtsy
#92 twig-2572605-2.x-91.patch10.33 KBjibran
#89 twig-2572605-2.x.patch60.65 KBlarowlan
#81 interdiff-2572605-79-81.txt761 bytesstar-szr
#81 2572605-81-1.x-bc.patch7.72 KBstar-szr
#81 2572605-81-2.x.patch10.19 KBstar-szr
#79 2572605-79-wip.patch9.44 KBandypost
#79 interdiff.txt2 KBandypost
#78 2572605-78.patch7.4 KBandypost
#66 2572605-66.patch19.58 KBjofitz
#64 2572605-64.patch0 bytesjofitz
#62 2572605-62.patch0 bytesjofitz
#60 interdiff-2572605-59-60.txt1.61 KBmfernea
#60 2572605-60.patch19.71 KBmfernea
#59 2572605-59.patch19.69 KBjofitz
#59 interdiff-57-59.txt2.55 KBjofitz
#57 2572605-57.patch19.95 KBjofitz
#54 2572605-54.patch63.32 KBaleevas
#50 2572605-50.patch23.13 KBjofitz
#50 interdiff-48-50.txt4.17 KBjofitz
#48 2572605-48.patch23.62 KBjoelpittet
#42 update_twig2-php7_only-2572605-42.patch23.76 KBjofitz
#32 interdiff.txt2.29 KBslasher13
#32 update_twig2-php7_only-2572605-32.patch24.3 KBslasher13
#30 interdiff.txt859 bytesslasher13
#30 update_twig2-php7_only-2572605-30.patch24.21 KBslasher13
#28 update_twig2-php7_only-2572605-28.patch24.28 KBslasher13
#28 interdiff.txt7.47 KBslasher13
#25 interdiff.txt7.11 KBslasher13
#25 update_twig2-php7_only-2572605-25.patch31.75 KBslasher13
#23 update_twig2-php7_only-2572605-23.patch21.04 KBslasher13
#23 interdiff-18-23.txt470 bytesslasher13
#21 interdiff-18-21.txt2.11 KBAnonymous (not verified)
#21 2572605-21.patch22.59 KBAnonymous (not verified)
#18 interdiff.txt5.3 KBslasher13
#18 update_twig2-php7_only-2572605-18.patch21.06 KBslasher13
#16 update_twig2-php7_only-2572605-16.patch19.2 KBslasher13
#11 update_to_twig_2_0_0-2572605-11.patch38.49 KBskyredwang
#9 update_twig2-php7_only-2572605-6.patch38.49 KBskyredwang
#6 update_twig2-php7_only-2572605-5.patch38.53 KBslasher13
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Cottser created an issue. See original summary.

joelpittet’s picture

Version: 8.0.x-dev » 8.1.x-dev

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

joelpittet’s picture

Note to self, this is postponed on the release of Twig 2.x

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

slasher13’s picture

Status: Needs review » Needs work

The last submitted patch, 6: update_twig2-php7_only-2572605-5.patch, failed testing.

joelpittet’s picture

Looks like the error was whitespace related give it another try @slasher13?

skyredwang’s picture

Removed the unnecessary trailing whitespace in the previous patch.

Status: Needs review » Needs work

The last submitted patch, 9: update_twig2-php7_only-2572605-6.patch, failed testing.

skyredwang’s picture

Anonymous’s picture

all 'time' properties need update too :)

Status: Needs review » Needs work

The last submitted patch, 11: update_to_twig_2_0_0-2572605-11.patch, failed testing.

slasher13’s picture

Is testbot incompatible with composer 1.3? Got these time property changes after update to composer 1.3

Anonymous’s picture

Looks like this patch based on #2840596: Update Symfony components to ~2.8.16, but it revert now.

slasher13’s picture

Status: Needs work » Needs review
FileSize
19.2 KB

rebased & composer self-update --rollback

Status: Needs review » Needs work

The last submitted patch, 16: update_twig2-php7_only-2572605-16.patch, failed testing.

slasher13’s picture

slasher13’s picture

+ /usr/local/bin/composer install --no-progress --no-suggest
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
Nothing to install or update

What's wrong with composer.lock file? Twig should be updated on PHP7 environments.

Status: Needs review » Needs work

The last submitted patch, 18: update_twig2-php7_only-2572605-18.patch, failed testing.

Anonymous’s picture

Status: Needs review » Needs work

The last submitted patch, 21: 2572605-21.patch, failed testing.

slasher13’s picture

Status: Needs review » Needs work

The last submitted patch, 23: update_twig2-php7_only-2572605-23.patch, failed testing.

slasher13’s picture

Status: Needs review » Needs work

The last submitted patch, 25: update_twig2-php7_only-2572605-25.patch, failed testing.

joelpittet’s picture

Thanks for tackling this @slasher13! It looks like there are some changes that are not needed in here. Can you tell my why the changes to core/lib/Drupal/Core/Field/BaseFieldDefinition.php were made?

PHP 7 tests seem to get closer to 0

slasher13’s picture

Status: Needs review » Needs work

The last submitted patch, 28: update_twig2-php7_only-2572605-28.patch, failed testing.

slasher13’s picture

Status: Needs review » Needs work

The last submitted patch, 30: update_twig2-php7_only-2572605-30.patch, failed testing.

slasher13’s picture

alexpott’s picture

@slasher13 this change is not going to work :(

We can't update the composer.lock whilst still supporting PHP 5.5.9

slasher13’s picture

But I don't know how to test PHP 7 only. After PHP 7 is green I will update to twig 1.31. So we are compatible with twig 2 and have no problems with composer.lock.

alexpott’s picture

@slasher13 the problem with changing the requirements to ^1.23.1|^2 is that we can't change the composer.lock only for PHP7 so we have no idea of whether all the bridge code works.

joelpittet’s picture

@alexpott couldn't we have the testbot build the composer per environment?

slasher13’s picture

But we fix it in composer.lock. If someone uses composer to handle dependencies. You have to verify it yourself or you have to wait until Drupal drops support of PHP 5.5. and PHP 5.6. So I think it's worth trying to be compatible with twig 2.

You can stay with ^1.23.1, too. Then I have to change it myself.

Status: Needs review » Needs work

The last submitted patch, 32: update_twig2-php7_only-2572605-32.patch, failed testing.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jibran’s picture

star-szr’s picture

Issue tags: +Needs reroll
jofitz’s picture

Status: Needs review » Needs work

The last submitted patch, 42: update_twig2-php7_only-2572605-42.patch, failed testing.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

xjm’s picture

Title: Update to Twig 2.0.0 » Prepare Twig 2.0.0 update (while supporting Twig 1.x if possible)
Issue summary: View changes
Related issues: +#2842431: [policy] Remove PHP 5.5, 5.6 support in Drupal 8.7

Someone understood this issue to be an expectation that 8.4.x or 8.5.x would actually require Twig 2 and PHP 7. Retitling the issue to be less scary and clarifying in the summary.

joelpittet’s picture

Thanks @xjm, nice title change. We are hoping to have both work for BC.

jibran’s picture

joelpittet’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
23.62 KB

Here's a re-roll of the last patch, I explicitly changed semver to ^2.4.0, hope that's ok?

Mostly applied, except for the composer.lock file and Drupal\twig_extension_test\TwigExtension\TestExtension which I fixed the conflict manually.

Status: Needs review » Needs work

The last submitted patch, 48: 2572605-48.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

jofitz’s picture

Status: Needs work » Needs review
FileSize
4.17 KB
23.13 KB

Corrected the test failures and rectified some coding standards errors.

Status: Needs review » Needs work

The last submitted patch, 50: 2572605-50.patch, failed testing. View results

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

jibran’s picture

Issue tags: +Needs reroll
aleevas’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
63.32 KB

I'm rerolled patch from #50
Please review

alexpott’s picture

@aleevas your patch contains the patch - core/2572605-50.patch - can you generate the patch without that file? Thanks.

andypost’s picture

Status: Needs review » Needs work
jofitz’s picture

Status: Needs work » Needs review
FileSize
19.95 KB

Remove .patch from patch in #54.

joelpittet’s picture

Status: Needs review » Needs work

I'm

+++ b/core/tests/Drupal/Tests/Core/Template/TwigExtensionTest.php
@@ -177,10 +193,11 @@ public function testActiveThemePath() {
-    $loader = new \Twig_Loader_String();
+    $name = sprintf('__string_template__%s', hash('sha256', uniqid(mt_rand(), TRUE), FALSE));
+    $loader = new \Twig_Loader_Array([$name => '{{ active_theme_path() }}']);

These look a bit sketch, I wonder if we can get away without having to generate a name like this.

jofitz’s picture

Status: Needs work » Needs review
FileSize
2.55 KB
19.69 KB

Use a hard-coded string for $name (in multiple locations).

mfernea’s picture

Just a little help with cs issues.

markhalliwell’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll
jofitz’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
0 bytes

Re-roll.

Status: Needs review » Needs work

The last submitted patch, 62: 2572605-62.patch, failed testing. View results

jofitz’s picture

Status: Needs work » Needs review
FileSize
0 bytes

Re-roll without redundant file(!)

Status: Needs review » Needs work

The last submitted patch, 64: 2572605-64.patch, failed testing. View results

jofitz’s picture

Status: Needs work » Needs review
FileSize
19.58 KB

After 2(!) empty patches, perhaps this re-roll will work.

alexpott’s picture

@Jo Fitzgerald perhaps you can review #2600154: Update our Twig code to be ready for Twig 2.x which takes care of some of the test stuff and deprecations. It'll make doing this one much easier.

alexpott’s picture

  1. +++ b/core/lib/Drupal/Core/Template/TwigNodeTrans.php
    @@ -17,13 +17,21 @@ class TwigNodeTrans extends \Twig_Node {
    +    if (NULL !== $body) {
    +      $nodes['body'] = $body;
    +    }
    

    Body can't be NULL it's not optional.

  2. +++ b/core/lib/Drupal/Core/Template/TwigNodeTrans.php
    @@ -32,29 +40,27 @@ public function __construct(\Twig_Node $body, \Twig_Node $plural = NULL, \Twig_N
    -    $compiler->write('echo ' . (empty($plural) ? 't' : '\Drupal::translation()->formatPlural') . '(');
    +    $compiler->write('echo ' . ($this->hasNode('plural') ? '\Drupal::translation()->formatPlural' : 't') . '(');
     
         // Move the count to the beginning of the parameters list.
    -    if (!empty($plural)) {
    -      $compiler->raw('abs(')->subcompile($this->getNode('count'))->raw('), ');
    +    if ($this->hasNode('plural')) {
    +      $compiler->raw('abs(')->subcompile($this->hasNode('count') ? $this->getNode('count') : NULL)->raw('), ');
         }
     
         // Write the singular text parameter.
         $compiler->subcompile($singular);
     
         // Write the plural text parameter, if necessary.
    -    if (!empty($plural)) {
    +    if ($this->hasNode('plural')) {
    

    These changes are unnecessary aren't needed.

  3. +++ b/core/lib/Drupal/Core/Template/TwigTransTokenParser.php
    @@ -21,10 +21,10 @@ class TwigTransTokenParser extends \Twig_TokenParser {
    -    $body = NULL;
    -    $options = NULL;
    -    $count = NULL;
         $plural = NULL;
    +    $count = NULL;
    +    $options = NULL;
    +    $body = NULL;
    

    This change looks unnecessary.

In some ways I think it'd be good to split the TwigNodeTrans into a separate issue and remove the related deprecations in \Drupal\Tests\Listeners\DeprecationListenerTrait::getSkippedDeprecations()

jibran’s picture

+++ b/core/composer.json
@@ -31,7 +31,7 @@
-        "twig/twig": "^1.35.0",
+        "twig/twig": "^1.35.0|^2.4.0",

It is important to test the patch on both php5 and php7 to make sure tests are passing on twig 1 and 2.

alexpott’s picture

Fixing the deprecations should be the aim of #2600154: Update our Twig code to be ready for Twig 2.x. This issue should only be about the possible change to composer.json to support twig 2.0.

alexpott’s picture

This issue has never actually tested Twig 2.0 it has just permitted a project to install it. Before we change the constraint we need a patch on this issue that forces testbot to run a test with Twig 2. All the other work to stop us triggering deprecated code should take place in #2600154: Update our Twig code to be ready for Twig 2.x

alexpott’s picture

alexpott’s picture

Status: Needs review » Postponed

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

voleger’s picture

Status: Postponed » Needs work
fgm’s picture

Since we're no longer going to be supporting PHP 5.6 in 8.7, is there any reason at all to keep the Twig 1 support now that this issue targets 8.7 instead of 8.6 ?

effulgentsia’s picture

To not force BC breaks onto contrib modules/themes that use Twig 1.x features that were removed in 2.x.

andypost’s picture

Check for upgrade

Status: Needs review » Needs work

The last submitted patch, 79: 2572605-79-wip.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

star-szr’s picture

I'm rusty but let's try this.

Attaching patches for 2.x and 1.x (untested).

Interdiff code grabbed from our Twig string loader.

The last submitted patch, 81: 2572605-81-2.x.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Gábor Hojtsy’s picture

Are the new problems found new in Twig 2.x since #2600154: Update our Twig code to be ready for Twig 2.x or are we finding them because we did not have these tests to prove it worked?

Either way it sounds like we'll need a testing issue at least ongoing to prove we keep compatibility since we cannot just include a test in core to keep it tested (patch composer files and then build a test environment) can we?

catch’s picture

I think we should open an issue like #3020303: Consider using Symfony flex to allow switching between major Symfony versions to make an install with Twig 2 possible, this will then allow regular core tests against Twig 2 once we have min/max composer testing. Although an ongoing issue with a patch we can run tests against would be good until then.

andypost’s picture

Ir looks like most of them are new, not clear why testloader fails and why it was not updated early

Gábor Hojtsy’s picture

Issue tags: +Twig 2

FYI #2976394: Allow Symfony 4.4 to be installed in Drupal 8 found deprecated API use for removal in Twig 3. I am surprised that did not come up here as fails since they are already marked deprecated without updating twig at all.

Berdir’s picture

@Gabor: That is because those deprecation messages are again from the debug classloader that's new in Symfony 4 and not triggered by Twig itself. Also I guess those are more examples that we just need to ignore, as I assume that whatever's the replacement for that interface isn't available yet in Twig 1.x?

Gábor Hojtsy’s picture

Status: Needs review » Needs work

@berdir: yeah the concrete issue is #3030494: [Twig 3] The "Drupal\Core\Template\Loader\StringLoader" class implements "Twig_ExistsLoaderInterface" and "Twig_SourceContextLoaderInterface" that are deprecated since 1.12 and 1.27 (to be removed in 3.0). where I proposed to ignore them but that did not work :/ There are still genuine fails here that would be great to fix as soon as we can so we can say people can try out Drupal 8.7 with Twig 2 :)

larowlan’s picture

Status: Needs work » Needs review
FileSize
60.65 KB

Trying to fix the fails

Gábor Hojtsy’s picture

Status: Needs review » Needs work

Yay, thanks @larowlan.

Since you did not post an interdiff, I compared the patches and looks like the effective change you made is this:

Patch in #81:

  public function getSourceContext($name) {
    $name = (string) $name;
    return new Source($name, $name);
  }

Patch in #89:

  public function getSourceContext($name) {
    $name = (string) $name;
    $value = $this->getSource($name);
    return new Source($value, $name);
  }

On the other hand you added 50k worth of additional composer file changes, so not only updating twig anymore but also a bunch of other things, including phpunit and adding various new vendors. We should cut back the testing patch to only updating to twig 2 unless some dependency updates are required to do alongside it.

Gábor Hojtsy’s picture

Ok I removed the composer changes from #89 and rolled a patch with just that. Also rolled in the composer changes from #81 that were actually limited to twig2 updates for testing. This should better represent that #89 is working (or not).

I merely spliced together patches, did not change anything :)

jibran’s picture

Here we go. No interdiff cuz it only reverts composer.lock changes.

Status: Needs review » Needs work

The last submitted patch, 91: 2572605-89-with-composer-twig2-from-81.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Gábor Hojtsy’s picture

Status: Needs work » Needs review

Hm, there is zero diff between @jibran's twig-2572605-2.x-91.patch and my 2572605-89-with-composer-twig2-from-81.patch and yet mine failed in Drupal\Tests\system\Functional\Module\InstallUninstallTest which seemed suspiciously unrelated anyway. Random fail I assume. Ok now we need reviews so we can get this small fix in ASAP :)

I opened the Twig 2 counterpart of #3020303: Consider using Symfony flex to allow switching between major Symfony versions at #3031639: [PP-1] Relax composer.json requirements to allow Drupal 8 to be installed with Twig 2 but marked it postponed on the Symfony 4 one as the mechanics and concerns may be the same and are still under discussion there.

Gábor Hojtsy’s picture

Reuploading the patch to review for clarity. It prepares core to Twig 2 but makes no changes to which twig we ship with. That is the goal of the issue. The patches with composer changes were merely to test.

andypost’s picture

+++ b/core/tests/Drupal/Tests/Core/Template/TwigExtensionTest.php
@@ -10,6 +10,20 @@
 use Drupal\Tests\UnitTestCase;
+/*
+if (!function_exists(__NAMESPACE__ . 't')) {
+
+  function t($string, array $args = []) {
+    return strtr($string, $args);
+  }
+
+}
+if (!function_exists(__NAMESPACE__ . 'file_create_url')) {
+
+  function file_create_url() {
+  }
+
+}*/

any reason to add this commented?

alexpott’s picture

  1. This is looking good
  2. +++ b/core/lib/Drupal/Core/Template/TwigEnvironment.php
    @@ -38,6 +38,8 @@ class TwigEnvironment extends \Twig_Environment {
    +  protected $templateClassPrefix = '__TwigTemplate_';
    

    This variable should have a comment.

  3. +++ b/core/lib/Drupal/Core/Template/TwigTransTokenParser.php
    @@ -55,14 +55,14 @@ public function parse(\Twig_Token $token) {
    -  public function decideForFork($token) {
    +  public function decideForFork(\Twig_Token $token) {
    ...
    -  public function decideForEnd($token) {
    +  public function decideForEnd(\Twig_Token $token) {
    

    As we are changing the argument we should also fix the missing docs.

  4. It's nice that the code in #96 is not needed!
Gábor Hojtsy’s picture

Re #96/@andypost:

Removed debug code.

Re #97/@alexpott:

1. Yay!
2a. I don't think that variable is needed at all, it is inherited either way from Twig_Environment and it existed prior to this patch as well with the same value. Based on https://github.com/twigphp/Twig/blob/2.x/lib/Twig/Environment.php this property or its value did not change in Twig 2 either. Just removed it.
2b. While I was there I added docs to the single undocumented variable above it which is our own code. I copied the text verbatim from TwigPhpStorageCache
3. We are not changing the argument, Twig 1 was/is also passing $token as the same type. I noticed we copied the method names from the 'for' tag, and Twig otherwise uses different method naming based on the tag name, so interesting... Either way, not a good opportunity to change now.
4. Removed as per above.

alexpott’s picture

Status: Needs review » Reviewed & tested by the community

@Gábor Hojtsy thanks for addressing the feedback. This looks good to go now.

The last submitted patch, 66: 2572605-66.patch, failed testing. View results

Gábor Hojtsy’s picture

Here is a version with the twig2 composer changes only for testing. Also uploading #98 again for sake of committer clarity.

Also updating issue summary heavily.

Gábor Hojtsy’s picture

Title: Prepare Twig 2.0.0 update (while supporting Twig 1.x if possible) » Make Drupal 8 optionally working with Twig 2, while keeping its dependency on Twig 1
Issue tags: +Drupal 9

Adding Drupal 9 tag and further clarify the title.

The last submitted patch, 101: 2572605-99-only-for-testing.patch, failed testing. View results

Gábor Hojtsy’s picture

Status: Reviewed & tested by the community » Needs work

Glad we had the testing patch :) TwigEnvironment::$templateClassPrefix became private (from protected) between 1.x and 2.x

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.72 KB
11.23 KB
8.77 KB

Patch attached addresses the fails in #101 by calling parent::getTemplateClass() so we no longer need to access the private.

Whilst making this change I noticed that \Drupal\Core\Template\TwigEnvironment::invalidate() was trying to set \Twig_Environment::$loadedTemplates to an empty array. This will no longer work because it is also private. PHP does not complain because it just creates a public property of the same name. However, setting this to an empty array has always been bogus because this property is filled by doing:

$this->loadedTemplates[$cls] = new $cls($this);

As there is no way of unloading code in PHP once for the same value of $cls PHP will never return something different. So I think we should remove the $this->loadedTemplates = []; line.

Wim Leers’s picture

+++ b/core/tests/Drupal/Tests/Core/Template/AttributeTest.php
@@ -280,7 +280,7 @@ public function testTwigAddRemoveClasses($template, $expected, $seed_attributes
-      ["{{ attributes.class }}", ''],
+      ["{{ attributes }}", ''],

Isn't this a BC break?

alexpott’s picture

@Wim Leers I think Twig is allowed to break its APIs between version 1 and 2.

larowlan’s picture

Isn't this a BC break?

This change is in the 1.x patch, so perhaps needs further investigation?

Gábor Hojtsy’s picture

Well let's see what happens without that change. The following tests after where that change was still test the .class so that should work as well as before. The other test changes made only affect how the test template loading happens which should not affect this.

There is existing use of attributes.class in tests and some modules, neither of which needed to change so I am hopeful that this was not necessary to change in the test.

core/modules/system/tests/modules/twig_theme_test/templates/twig_theme_test.filter.html.twig:<div><span class="{{ attributes.class }}"{{ attributes|without('class') }}>Class attributes in front, remainder at the back:</span></div>
core/modules/system/tests/modules/twig_theme_test/templates/twig_theme_test.filter.html.twig:<div><span{{ attributes|without('class') }} data-class="{{ attributes.class }}">Class attributes in back, remainder at the front:</span></div>
core/modules/system/tests/modules/twig_theme_test/templates/twig_theme_test.filter.html.twig:<div><span class="{{ attributes.class }}">Class attributes only:</span></div>
core/modules/update/templates/update-version.html.twig:<div class="{{ attributes.class }} project-update__version"{{ attributes|without('class') }}>
core/themes/stable/templates/admin/update-version.html.twig:<div class="{{ attributes.class }} project-update__version"{{ attributes|without('class') }}>

The last submitted patch, 109: 2572605-twig2-109.patch, failed testing. View results

Gábor Hojtsy’s picture

Status: Needs review » Needs work

Ok this fails with the following error:

Drupal\Tests\Core\Template\AttributeTest::testTwigAddRemoveClasses with data set #0 ('{{ attributes.class }}', '')
ArgumentCountError: Too few arguments to function Drupal\Core\Template\Attribute::hasClass(), 0 passed in /var/www/html/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1626 and exactly 1 expected

Directly this is because Drupal\Core\Template\Attribute::hasClass() expects a class name and returns true if that class is set on the attributes object, otherwise returns FALSE. But why is that called in the first place?

From https://twig.symfony.com/doc/2.x/templates.html

For convenience's sake foo.bar does the following things on the PHP layer:

  • check if foo is an array and bar a valid element;
  • if not, and if foo is an object, check that bar is a valid property;
  • if not, and if foo is an object, check that bar is a valid method (even if bar is the constructor - use __construct() instead);
  • if not, and if foo is an object, check that getBar is a valid method;
  • if not, and if foo is an object, check that isBar is a valid method;
  • if not, and if foo is an object, check that hasBar is a valid method;
  • if not, return a null value.

Why is this a new problem with Twig 2? Because Twig 1 does not have the hasBar() part. See https://twig.symfony.com/doc/1.x/templates.html:

For convenience's sake foo.bar does the following things on the PHP layer:

  • check if foo is an array and bar a valid element;
  • if not, and if foo is an object, check that bar is a valid property;
  • if not, and if foo is an object, check that bar is a valid method (even if bar is the constructor - use __construct() instead);
  • if not, and if foo is an object, check that getBar is a valid method;
  • if not, and if foo is an object, check that isBar is a valid method;
  • if not, return a null value.

So a method we have on Drupal\Core\Template\Attribute is now in the middle of Twig's expected extension API.

Removing testing of this is certainly not the solution :D It would also not be a solution to make this method return TRUE when there was some class (and make the $class argument optional) because that would change the twig output from the empty output we expect (based on an expected null return value).

The sole reason we have this hasClass() method seems to be to test attributes. At least according to this grep:

$ git grep "\->hasClass"
core/modules/quickedit/tests/src/FunctionalJavascript/QuickEditJavascriptTestBase.php:    $this->assertFalse($toolbar_edit_button->hasClass('is-active'), 'The "Edit" button in the toolbar is not yet marked as active.');
core/modules/quickedit/tests/src/FunctionalJavascript/QuickEditJavascriptTestBase.php:      $this->assertTrue($dom_node->hasClass('visually-hidden'), 'The contextual links trigger "' . $dom_node->getParent()->getAttribute('data-contextual-id') . '" is hidden.');
core/modules/quickedit/tests/src/FunctionalJavascript/QuickEditJavascriptTestBase.php:    $this->assertTrue($toolbar_edit_button->hasClass('is-active'), 'The "Edit" button in the toolbar is marked as active.');
core/modules/quickedit/tests/src/FunctionalJavascript/QuickEditJavascriptTestBase.php:      $this->assertFalse($dom_node->hasClass('visually-hidden'), 'The contextual links trigger "' . $dom_node->getParent()->getAttribute('data-contextual-id') . '" is visible.');
core/modules/quickedit/tests/src/FunctionalJavascript/QuickEditJavascriptTestBase.php:    $this->assertFalse($entity_contextual_links_container->hasClass('open'));
core/modules/quickedit/tests/src/FunctionalJavascript/QuickEditJavascriptTestBase.php:    $this->assertTrue($entity_contextual_links_container->hasClass('open'));
core/modules/system/tests/src/Functional/Pager/PagerTest.php:    $this->assertTrue($element->hasClass($class) !== FALSE, $message);
core/modules/system/tests/src/Functional/Pager/PagerTest.php:    $this->assertTrue($element->hasClass($class) === FALSE, $message);
core/modules/toolbar/tests/src/FunctionalJavascript/ToolbarIntegrationTest.php:    $this->assertFalse($tray->hasClass('toolbar-tray-vertical'), 'Toolbar tray is not vertically oriented by default.');
core/modules/toolbar/tests/src/FunctionalJavascript/ToolbarIntegrationTest.php:    $this->assertTrue($tray->hasClass('toolbar-tray-vertical'), 'After toggling the orientation the toolbar tray is now displayed vertically.');
core/modules/toolbar/tests/src/FunctionalJavascript/ToolbarIntegrationTest.php:    $this->assertTrue($tray->hasClass('toolbar-tray-horizontal'), 'After toggling the orientation a second time the toolbar tray is displayed horizontally again.');
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertEquals(TRUE, $machine_name_1_wrapper->hasClass('visually-hidden'), 'The ID field must not be visible');
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertEquals(TRUE, $machine_name_2_wrapper->hasClass('visually-hidden'), 'The ID field must not be visible');
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertEquals(FALSE, $machine_name_1_wrapper->hasClass('visually-hidden'), 'The ID field must now be visible');
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertEquals(TRUE, $machine_name_2_wrapper->hasClass('visually-hidden'), 'The ID field must not be visible');
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertTrue($assert->fieldExists('id')->hasClass('error'));
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertFalse($assert->fieldExists('id2')->hasClass('error'));
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertTrue($assert->fieldExists('id')->hasClass('error'));
core/tests/Drupal/FunctionalJavascriptTests/Core/MachineNameTest.php:    $this->assertTrue($assert->fieldExists('id2')->hasClass('error'));
core/tests/Drupal/FunctionalJavascriptTests/TableDrag/TableDragTest.php:      return $handle->hasClass($class);
core/tests/Drupal/FunctionalJavascriptTests/TableDrag/TableDragTest.php:      return !$handle->hasClass($this::DRAGGING_CSS_CLASS);
core/tests/Drupal/Tests/Core/Template/AttributeTest.php:    $this->assertFalse($attribute->hasClass('a-class-nowhere-to-be-found'));
core/tests/Drupal/Tests/Core/Template/AttributeTest.php:    $this->assertTrue($attribute->hasClass('we-totally-have-this-class'));

So the question is how dangerous would it be to rename this method at this point on Drupal\Core\Template\Attribute or make some ugly workaround to make it work like an offsetGet('class') if no argument was given and add a todo to remove this ugly thing in D9 :)

Gábor Hojtsy’s picture

Status: Needs work » Needs review
FileSize
1.48 KB
11.94 KB
9.48 KB

Here is the ugly workaround solution, let's discuss.

lauriii’s picture

Status: Needs review » Needs work
+++ b/core/lib/Drupal/Core/Template/Attribute.php
@@ -264,13 +264,27 @@ public function removeClass() {
+   * @todo Rename this method in Drupal 9. Maybe deprecate it now?
...
+    if (!isset($class)) {
+      return $this->offsetGet('class');
+    }

I don't think changing this API to specifically support Twig 2 in this way is nice. I also don't think we should be deprecating this method since it was added for a reason and I don't think that has changed. I assume this problem will occur in other places as well, and we don't want to suggest this being an ideal solution for the problem. I suggest we try to solve this by an if clause that would allow us to specifically check if ->class has been set.

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.82 KB
11.19 KB
8.73 KB

I think we can fix this in a more elegant way. I'm not sure that I agree with hassers being getters especially with respect to Twig - it was added in https://github.com/twigphp/Twig/pull/1969 - bascially to make Twig more like https://symfony.com/doc/current/components/property_access.html#using-ha... - but making a has* return anything other than a bool is really really odd so I don't think twig_get_attribute or \Twig_Template::getAttribute() should use them. Will open an issue against Twig later.

The more elegant fix it to implement getClass() to get the class attribute and call offsetGet() in it. That way it will take precedence over our hasClass() implementation which I don't think we should be changing - hasClass() is used a lot in contrib - see http://grep.xnddx.ru/search?text=-%3EhasClass%28&filename=

Gábor Hojtsy’s picture

Duh, I should have just added a getClass() method based on the twig lookup list, yeah :D

alexpott’s picture

Opened https://github.com/twigphp/Twig/issues/2833 to address the hassers as getters.

jibran’s picture

+++ b/core/lib/Drupal/Core/Template/TwigTransTokenParser.php
@@ -54,15 +54,21 @@ public function parse(\Twig_Token $token) {
-  public function decideForFork($token) {
+  public function decideForFork(\Twig_Token $token) {
...
-  public function decideForEnd($token) {
+  public function decideForEnd(\Twig_Token $token) {

As per #2887179-5: Fix the typehinting in CurrentPathStack

So adding a typehint can break BC if a passed-in value does not match the typehint.

As this method doesn't exist on an interface an can be used directly I agree this is a BC break.

andypost’s picture

+++ b/core/tests/Drupal/KernelTests/Core/Theme/TwigEnvironmentTest.php
@@ -216,7 +216,7 @@ public function testTemplateInvalidation() {
+    $reflection = new \ReflectionClass('\Twig_Environment');

I'd better used \Twig_Environment::class instead of string for better code navigation

Gábor Hojtsy’s picture

@jibran: these are callbacks passed onto the Twig parser, I don't believe there is any meaningful result if called otherwise, they are merely public so the Twig parser can invoke them. They also already requires the argument to be an object instance with a test method on it that takes a string or array even if we don't add the typehint.

Gábor Hojtsy’s picture

re @jibran, I looked through Twig 2.6.2's Parser class code and there only seems to be an expectation that the callable's class be an instanceof Twig_TokenParserInterface and even that only in an error condition. That said, Twig 2 does typehint its callables this way but there may not be a reason to force us to do so. (Again I did not write this patch, and I am contributing by poking at things :D).

So it does not hurt to try to not to add the typehint for safety sake. And see what happens then.

andypost’s picture

jibran’s picture

Status: Needs review » Reviewed & tested by the community

This looks good now.

  • catch committed 9ec3433 on 8.7.x
    Issue #2572605 by Gábor Hojtsy, slasher13, Jo Fitzgerald, alexpott,...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Agreed this looks really solid now, good to see all the iterations.

Committed 9ec3433 and pushed to 8.7.x. Thanks!

Gábor Hojtsy’s picture

Next step is to #3031639: [PP-1] Relax composer.json requirements to allow Drupal 8 to be installed with Twig 2 (which is dependent on some process discussions in another issue, see there) and #2976407: Use drupalci.yml for composer dependency min/max testing - DO NOT COMMIT to ensure there are no regressions in the future.

I opened #3032695: Manually test Drupal 8 with Twig 2 for occassional manual testing for now.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

webchick’s picture

Issue tags: +8.7.0 highlights

Seems worthy of calling this out!