Issue #1975462 by thedavidmeister, alexpott, alexrayu, chx, joelpittet: Fixed Allow and test for NULL and integer 0 values in Twig templates.

Problem description

NULL variables being passed to Twig templates lead to exceptions - eg. #1961872: Convert html.tpl.php to Twig, #1843746: Convert views/templates/views-view-field.tpl.php to Twig . NULL variables can simply be the result of calling drupal_render() on a renderable array with '#access' => FALSE within a preprocess/process function - #1975442: drupal_render() should always return a string., so it's very likely that this will turn up more in contrib than we're currently seeing in core.

Additionally, Drupal's Twig implementation does not print anything for the following: {% set var = 0 %}{{ var }}. This should print '0'.

Solution

Fix Drupal's twig implementation to

  • not produce exceptions when passed a null value
  • print 0 when passed an integer 0
  • add tests for PHP's scalar variables.

Commit message

git commit -m "Issue #1975462 by thedavidmeister, alexrayu, alexpott, chx, joelpittet: Fixed Allow and test for NULL and integer 0 values in Twig templates."

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

star-szr’s picture

Issue tags: +Twig

tagging.

thedavidmeister’s picture

Status: Active » Needs review
FileSize
2.4 KB

First, a test to demonstrate the exceptions. Setting needs review just to get the testbots running.

thedavidmeister’s picture

FileSize
3.16 KB

Here's the proposed fix + test.

Fabianx’s picture

Status: Needs review » Needs work

I was about to RTBC this, then I noticed that is_null also gives back TRUE if the array index does not exist. Therefore this fix is not right as the following script proves:

<?php

function gC_a($context, $item) {
  if (!isset($context[$item])) {
    echo "Error: $item does not exist";
  }
  return $context[$item];
}

function gC_b($context, $item) {
  if (!isset($context[$item]) && !is_null($context[$item])) {
    echo "Error: $item does not exist";
  }
  return $context[$item];
}



$context = array(
  'page_top' => NULL,
/*  'page_undefined' => NULL,*/
);

var_dump(gC_a($context, 'page_top'));
var_dump(gC_b($context, 'page_top'));

var_dump(gC_a($context, 'page_undefined'));
var_dump(gC_b($context, 'page_undefined'));

Expected result: 3 error messages; Actual result: 2 error messages.

Proposed Fix

I checked what Twig uses and they use a much slower:

if (!array_key_exists($item, $context)) {

So as a compromise and because isset() is really fast, I suggest we do:

if (!isset(...) && !array_key_exists($item, $context)) {

The new tests should still pass.

thedavidmeister’s picture

what about !isset($context[$item]) && $context[$item] !== NULL?

thedavidmeister’s picture

Status: Needs work » Needs review
FileSize
2.27 KB

I actually don't know how to write an "assert exception" test in SimpleTest, otherwise I would.

thedavidmeister’s picture

bah, !== NULL throws a notice when used on an undefined variable but then again, so are we so that might actually be ok? I dunno. I'll have a look at the array_key_exists() thing too.

Status: Needs review » Needs work

The last submitted patch, 1975462-6.patch, failed testing.

thedavidmeister’s picture

FileSize
2.27 KB

I just did some simple benchmarking like this:

  $i = 0;
  $var = NULL;
  $array = array_fill(0, 50, 'foo');
  timer_start('foo');
  while($i < 10000) {
    ($var !== NULL); // 0.40
    @($var !== NULL); // 2.9
    ($undefined !== NULL); // 7188
    @($undefined !== NULL); // 93
    array_key_exists('foo', $array); // 4.4
    array_key_exists(50, $array); // 3.5
    $i++; // 0.63
  }

The numbers in comments next to each line represent the timer values for each of those lines. The notice errors thrown by ($undefined === NULL) are crazy slow (the testbots hate it anyway)! Even with @ they're still about 20x slower than the next closest example.

The thing is though, that if we're optimising for the case that there are no errors (production sites) rather than when there are errors (dev sites), @($var !== NULL) is about 20% faster than array_key_exists().

(also, this is all silly micro-optimisation and @Fabianx told me in IRC this class is getting deleted soon and replaced with something else.)

Patch attached.

thedavidmeister’s picture

Status: Needs work » Needs review
thedavidmeister’s picture

FileSize
2.28 KB

eeexcept that print $undefined === NULL; prints 1.

isset() scores 0.5 on the benchmarking for #9 btw. I forgot to put it in there.

Here's something that actually works. Thanks @Fabianx, I got there eventually ;)

Status: Needs review » Needs work

The last submitted patch, 1975462-11.patch, failed testing.

thedavidmeister’s picture

Status: Needs work » Needs review
FileSize
681 bytes
3.16 KB

Failed because I forgot to include the preprocess in that patch. oops.

joelpittet’s picture

Status: Needs review » Reviewed & tested by the community

Gave this a run through and it worked great. Thanks @thedavidmeister

Also thanks for adding the test in there too.

There was a new line whitespace issue in the patch, I think it is in "core/modules/system/tests/themes/test_theme_twig/theme_test.template_test.html.twig" but it got corrected when I applied it the patch or my editor autofixed it because it wasn't there on opening it... :S

chx’s picture

FileSize
599 bytes
3.17 KB

Whitespace fix.

joelpittet’s picture

FileSize
3.16 KB

I think the one I am seeing is line 65 of the patch, EOF whitespace. Maybe it's a patch format discrepancy but thanks @chx for finding a real whitespace issue:)

I'll post the patch file, that git diff is providing me. No interdiff because the extra line ending is fixed automatically when the patch is applied.

star-szr’s picture

+++ b/core/modules/system/lib/Drupal/system/Tests/Theme/ThemeTestTwig.phpundefined
@@ -69,4 +69,15 @@ function testFindThemeTemplates() {
+   * Tests that Twig templates handle PHP various data types correctly.

I think this should be 'various PHP data types', right?

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/system/lib/Drupal/system/Tests/Theme/ThemeTestTwig.phpundefined
@@ -69,4 +69,15 @@ function testFindThemeTemplates() {
+   * Tests that Twig templates handle PHP various data types correctly.

Also... is this really various types... is not just testing whether or not twig handles nulls in the expected fashion?

thedavidmeister’s picture

+ // @todo: Coverage for all PHP data types. ;)

I'll fix up the docs.

thedavidmeister’s picture

Assigned: Unassigned » thedavidmeister
thedavidmeister’s picture

Status: Needs work » Needs review
FileSize
1.35 KB
3.18 KB

Hopefully these docs are better.

thedavidmeister’s picture

Assigned: thedavidmeister » Unassigned
Fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC per #14.

I second this RTBC also:

* Fixes a bug
* Has tests to show the bug
* Is performance optimized

=> RTBC (plus #14 RTBC)

alexpott’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
3.65 KB
4.8 KB

I don't think we should be adding @todo's at this stage. Let's test all the php scalar variable types.

This will fail because at the moment...

    'twig_int_0' => array(
      'value' => 0,
      'expected' => '0',
    ),

Will result in <li>twig_int_0: </li> whereas I think we should expect <li>twig_int_0: 0</li>

Fabianx’s picture

This is fixed here: #1970960: Twig implementation does not print int(0)

So maybe wait till that is committed, then commit this here?

thedavidmeister’s picture

#24 - I don't see any reason not to add the @todo. I was planning on making a followup issue to flesh that test out if this lands, but didn't want to make the issue straight away in case this patch doesn't get committed.

In this issue I'd really like to focus just on NULL because a bunch of conversion patches ran into it. Tests for the variables making it through Twig are important in general though, so I wanted to make something that will be easy to extend beyond NULL.

alexpott’s picture

Title: Allow NULL values in Twig templates. » Allow and test for NULL and integer 0 values in Twig templates.
FileSize
5.24 KB
360 bytes

@thedavidmeister since we're now in code slush each commit should make us closer to release and therefore we should avoid adding @todo especially if the patch is small and well contained. I see no reason not to merge #1970960: Twig implementation does not print int(0) and press on with your good idea to test all scalar variable types.

The commit message should additionally attribute alexrayu.

thedavidmeister’s picture

I guess I'll close the other issue as a duplicate then..

alexpott’s picture

Issue summary: View changes

Updated issue summary.

alexpott’s picture

And now removing the @todo :) doh!

Fabianx’s picture

Still RTBC per #14.

I second this RTBC also:

* Fixes two bugs
* Unblocks some twig conversions
* Has tests to show the bugs, adds extensive coverage for rendering via twig template.
* Is performance optimized

=> RTBC

Fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC per #30 - ahem ...

star-szr’s picture

Something to consider: it might be worth creating a new template/callback for these additional tests. Maybe it's just me but it feels a bit like we are commandeering theme_test.template_test.html.twig which was previously a simple template override test:

+++ b/core/modules/system/tests/themes/test_theme_twig/theme_test.template_test.html.twigundefined
@@ -1,2 +1,8 @@
 {# Output for Theme API test #}
 Success: Template overridden.
+
+<ul>
+{% for key, php_value in php_values %}
+  <li>{{ key }}: {{ php_value.value }}</li>
+{% endfor %}
+</ul>
alexpott’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
6.44 KB
5.34 KB

@Cottser yep I agree... let's not do that... added a new twig_theme_test module so we don't add any twigness to the theme_test module.

alexpott’s picture

Remove unnecessary use...

star-szr’s picture

Ah, that looks much better. Thanks @alexpott! :)

+++ b/core/modules/system/tests/modules/twig_theme_test/twig_theme_test.moduleundefined
@@ -0,0 +1,69 @@
+
+/**
+ * Implements hook_menu().
+ */
+function twig_theme_test_menu() {
+  $items['twig-theme-test/php-variables'] = array(
+    'page callback' => 'twig_theme_test_php_variables_callback',
+    'access callback' => TRUE,
+    'type' => MENU_CALLBACK,
+  );
+  return $items;
+}

Does this need a routing.yml now if we're adding a new page callback? Does the page callback need to be a controller? Not sure what the policy is around this now :)

Nitpickery:

  1. +++ b/core/modules/system/tests/modules/twig_theme_test/templates/twig_theme_test.php_variables.html.twigundefined
    @@ -0,0 +1,6 @@
    +{# Output for Twig Theme PHP variables test #}
    

    End this comment in a period per http://drupal.org/node/1354#drupal.

  2. +++ b/core/modules/system/tests/modules/twig_theme_test/twig_theme_test.moduleundefined
    @@ -0,0 +1,69 @@
    + * Menu callback for testing php variables in a twig template.
    

    All caps for PHP.

  3. +++ b/core/modules/system/tests/modules/twig_theme_test/twig_theme_test.moduleundefined
    @@ -0,0 +1,69 @@
    + * Menu callback for testing php variables in a twig template.
    ...
    +  // Prefix each variable with "twig_" so that twig doesn't get confused
    

    Twig should be capitalized (not the "twig_" one of course).

Fabianx’s picture

Issue tags: +Quick fix

Adding quick fix tag once this is RTBC again :).

alexpott’s picture

Moved to new routing system and fixed everything in #35 plus a few other coding standards related things.

star-szr’s picture

Status: Needs review » Needs work

Thanks @alexpott, looks good. I went over the whole patch again and this is all I could find:

+++ b/core/modules/system/tests/modules/twig_theme_test/lib/Drupal/twig_theme_test/TwigThemeTestController.phpundefined
@@ -0,0 +1,32 @@
+/**
+ * Controller routines for book routes.
+ */
+class TwigThemeTestController implements ControllerInterface {

Docblock needs updating :)

+++ b/core/modules/system/tests/modules/twig_theme_test/lib/Drupal/twig_theme_test/TwigThemeTestController.phpundefined
@@ -0,0 +1,32 @@
+   * Menu callback for testing php variables in a Twig template.

All caps PHP please.

star-szr’s picture

Ah I think there's some discrepancies with newlines too:

+++ b/core/modules/system/tests/modules/twig_theme_test/lib/Drupal/twig_theme_test/TwigThemeTestController.phpundefined
@@ -0,0 +1,32 @@
+<?php
+/**
+ * @file
+ * Contains \Drupal\twig_theme_test\TwigThemeTestController.
+ */

http://drupal.org/node/1918356 shows a blank line before the @file, http://drupal.org/node/1353118 doesn't. I'm guessing the former is accurate.

+++ b/core/modules/system/tests/modules/twig_theme_test/lib/Drupal/twig_theme_test/TwigThemeTestController.phpundefined
@@ -0,0 +1,32 @@
+use Drupal\Core\ControllerInterface;
+use Symfony\Component\DependencyInjection\ContainerInterface;
+
+
+/**
+ * Controller routines for book routes.
+ */
+class TwigThemeTestController implements ControllerInterface {

Extra blank line after use statements can be removed.

alexpott’s picture

Status: Needs work » Needs review
FileSize
7.57 KB
1.08 KB

Damn c&p from Drupal\book\Controller\BookController ... at least I didn't commit it :)

star-szr’s picture

Status: Needs review » Reviewed & tested by the community

Looks great. Thanks everyone!

star-szr’s picture

Issue summary: View changes

Updated issue summary.

star-szr’s picture

Issue summary: View changes

Add commit message

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.x, thanks!

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Whitespace