Problem/Motivation

When developing #2907810: [PP-1] Add $entity->toUrl() and $entity->toLink() methods to allowed methods list in Twig sandbox policy it was identified that the current sandbox method matching is very broad, allowing for methods to match _any_ object. This means it could possibly lead to unintended calls to possibly unsafe methods.

By hardening this is also makes rationalizing new additions to the allowed methods like ::toUrl and ::toLink if its possible to restrict the methods to a specific interface instead of trying to guess at any place the method might exist and how safe or unsafe its usage might be.

Steps to reproduce

Proposed resolution

Modify the format of the allowed_method settings to allow targeting methods on specific interfaces or classes. Something like:

    $allowed_methods = Settings::get('twig_sandbox_allowed_methods', [
      // Only allow idempotent methods.
      EntityInterface::class . '::id',
      EntityInterface::class . '::label',
      EntityInterface::class . '::bundle',
      // Globally allowed methods.
      '::get',
      '::__toString',
      '::toString',
    ]);

Remaining tasks

Finalize how much we harden this.

User interface changes

n/a

API changes

Data model changes

Release notes snippet

Twig sandboxing
Drupal's Twig default sandboxing has been hardened. It now allows access to the id, label, and bundle methods only on entities, not on any object. If you have custom templates that make use of these methods, you will need to customise the twig_sandbox_allowed_methods setting.

Issue fork drupal-3239105

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

neclimdul created an issue. See original summary.

neclimdul’s picture

neclimdul credited seanB.

neclimdul’s picture

adding some credits to people that participated in generating the current patch.

neclimdul’s picture

  1. +++ b/core/lib/Drupal/Core/Template/TwigSandboxPolicy.php
    @@ -55,15 +56,25 @@ public function __construct() {
    +      [$class, $name] = explode('::', $method);
    +      $this->allowed_methods[$name] = $class;
    

    This is going to fail for methods of the same name on different interfaces.

    A good example that we might actually want to include is AccountInterface::id

  2. +++ b/core/lib/Drupal/Core/Template/TwigSandboxPolicy.php
    @@ -94,7 +105,8 @@ public function checkMethodAllowed($obj, $method) {
    -      return TRUE;
    +      $allowed_class = $this->allowed_methods[$method];
    +      return empty($allowed_class) || $obj instanceof $allowed_class;
    

    Returning false doesn't behave like you might expect for this method. There actual is _no_ return type for this method, failures are suppose to throw and exception and calls in twig ignore any return value. I guess we don't have any failure case tests currently. Yikes.

neclimdul’s picture

Status: Active » Needs review
StatusFileSize
new10.37 KB
new11.18 KB

We do have tests for "dangerous methods" but they didn't cover the new case where the method matched but the interface didn't.

More tests and address the two points above.

longwave’s picture

Issue tags: +Needs change record

> There actual is _no_ return type for this method

Twig 3 declares the return type as void so any return statement would error, but we are still on Twig 2 :(

+++ b/core/lib/Drupal/Core/Template/TwigSandboxPolicy.php
@@ -55,15 +56,32 @@ public function __construct() {
+        @trigger_error('Not specifying a fully-qualified method name is deprecated in drupal:9.3.0 and will throw an error in drupal:10.0.0.', E_USER_DEPRECATED);

This message should probably explain that it's part of twig_sandbox_allowed_methods. Also needs a test and change record.

Status: Needs review » Needs work

The last submitted patch, 7: 3239105-7.patch, failed testing. View results

chi’s picture

+ if (strpos($method, $prefix) === 0) {
As Drupal core requires "symfony/polyfill-php80" it should be safe to use str_starts_with.

neclimdul’s picture

Issue summary: View changes
Status: Needs work » Needs review
Related issues: +#3094493: Upgrade to Twig 3
StatusFileSize
new5.01 KB
new5.01 KB

Good catch on the return, we probably shouldn't be writing new code we know will fail on twig 3 so I've remove the return value.

Also modified the deprecation and added the test. Turns out there was a bug in the BC that this caught!

Good eye on that polyfill. Technically, that code didn't change it as just moved, so I'd generally argue against touching it. But that's both a better method and the polyfill has a better implementation so might as well make the obvious fix. Bonus, the code is self documenting now so we don't need to explain the reason for it anymore :)

Looks like we've got some fun bugs. Looks like that suspicion about some breaking changes here are confirmed:

Twig\Sandbox\SecurityError: Calling "id" method on a "Drupal\Core\Layout\LayoutDefinition" object is not allowed. in Drupal\Core\Template\TwigSandboxPolicy->checkMethodAllowed() (line 10 of core/themes/stable/templates/layout/layout.html.twig). 

I added LayoutDefinition::id because its a plugin not an entity. Maybe we add PluginDefinitionInterface? Its probably safe but I just added LayoutDefinition in this pass to see how things go. Thoughts?

neclimdul’s picture

StatusFileSize
new12.98 KB

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

jonathanshaw’s picture

Very sensible. Implementation looks complete, tests look good.

Twig\Sandbox\SecurityError: Calling "id" method on a "Drupal\Core\Layout\LayoutDefinition" object is not allowed. in Drupal\Core\Template\TwigSandboxPolicy->checkMethodAllowed() (line 10 of core/themes/stable/templates/layout/layout.html.twig). 

I added LayoutDefinition::id because its a plugin not an entity. Maybe we add PluginDefinitionInterface? Its probably safe but I just added LayoutDefinition in this pass to see how things go. Thoughts?

I'm concerned about contrib here. It's easy for sites to add to twig_sandbox_allowed_methods. But how do contrib modules and themes do that if they need to?

The only clue I could find was https://stackoverflow.com/questions/16070602/how-to-properly-enable-the-...

Even if there's no supported way for contrib to currently add methods, it could be frustrating if we removed global support for id etc and there was no easy fix for contrib to restore it.

I wonder it's best to keep id and label global for now, and discuss making them more specific in a follow-up. This issue would add good infrastructure for more precise and harder policies, even if it didn't do much hardening itself.

The best hardening I have thought of would be to create a DrupalObjectInterface or TwigSandboxableInterface (without any methods on it itself), and have EntityInterface, PluginInterface, Url, etc extend from it. By extending from it these child interfaces would be contracting that they shared the generic Drupal understanding of id, label, get, toString as safe operations.

If we wanted to be really radical, we could give DrupalObjectInterface an isSafeMethodCall($method) {} and therefore delegate responsibility for twig sandboxing to the classes themselves. After all, they are the ones who know what their own methods mean.

  1. +++ b/core/lib/Drupal/Core/Template/TwigSandboxPolicy.php
    @@ -55,15 +57,32 @@ public function __construct() {
    +      if (isset($this->allowed_methods[$name])) {
    +        $this->allowed_methods[$name][] = $class;
    

    Worth commenting this condition is for BC and can be removed in 10.0?

  2. +++ b/core/tests/Drupal/Tests/Core/Template/TwigSandboxTest.php
    @@ -107,54 +116,190 @@ public function testEntitySafePrefixes() {
    +      public function bundle() {
    +        return 'id';
    

    Nit: should return "bundle" for consistency?

neclimdul’s picture

StatusFileSize
new13.04 KB
new2.53 KB
new12 KB
new2.71 KB

Thanks!

If tightening some of these globals is holding the issue back from getting committed we could make them global again. Since this is about that hardening, it seems worth going as far as possible though and I haven't found anything this would effect yet.

I think you're right about adding extension for contrib being the method for adding their own security exceptions. That's actually exactly how core's is doing it are a couple contrib modules adding extensions with tweaks and features so this doesn't seem a big deal.

re: points
1) That's actually not BC, all those ifs are used to combine the global and qualified formats. The only BC is the part with the deprecation message. I'll include a 10.0 patch with the BC removed.
2) Sure. that's definitely a nit since it doesn't even get used but it looks better I guess :)

Here's a reroll with an updated 9.4 deprecation message and 10.x patch with the removal. Might be late to do it but the patches are useful on their own.

jonathanshaw’s picture

If tightening some of these globals is holding the issue back from getting committed we could make them global again. Since this is about that hardening, it seems worth going as far as possible

It's not only about hardening. It's also about making it easier to add other much needed exceptions like entity::toUrl and URL:: setAbsolute.

I'm happy to RTBC as is one we have a CR, but I suspect there will be more discussion of the BC.

adding extension for contrib being the method for adding their own security exceptions. That's actually exactly how core's is doing it are a couple contrib modules adding extensions with tweaks and features so this doesn't seem a big deal.

I tried a grep at http://grep.xnddx.ru and could not find anything altering the sandbox policy. We should have a working example of this for the CR if we are going to break BC for id and label. I'm happy to write the CR if you can provide the snippet.

neclimdul’s picture

I guess I meant this was spit to discuss that hardening complexity from the Url issue. Its titled and tagged specifically to discuss that. I'll tag it for maintainer review though because that's a decision they'll have to make a decision about the impact.

If it helps, I've been running this in production for about 6 months without issue. Everything that could have run into this was cleared by the EntityInterface.

As far as searching, I would not have expected to see it because generally this sort of thing would probably be handled in a preprocess. I also assumed adding an extension(like twig_tweak) meant adding another policy would be easy. Assumptions did their thing though and turns out there are problems.
1) Twig extensions are keyed by class name so you can only have one SandboxExtension.
2) SandboxExtension is handled special so you can't add a sandbox-like extension.
3) Not matching throws an exception(makes sense, there can only be one) so you can't pass off handling to another exception.

That's kinda a mess and means you'd actually have to document sites adding to the setting. Probably a better method though would be to a preprocess method or adding filters like twig_tweak does to work around some of these link building problems on entities.

neclimdul’s picture

jonathanshaw’s picture

Ah, of course. Preprocess is the right solution for the CR.

How about we deprecate calling id & label from twig outside of EntityInterface? i.e. trigger a deprecation error if they are used in the sandbox on an object that doesn't extend EntityInterface.

That would progress the hardening but respect our usual BC approach.

neclimdul’s picture

Created the PR and included that as part of the notes.

I'm going to think through the more conservative approach to id and label but I'm

jonathanshaw’s picture

The CR looks good, I've tweaked slightly.

Stepping back to look at this problem space more broadly I come to think that:

1. If being class-specific is good for twig_sandbox_allowed_methods then probably it's good for twig_sandbox_allowed_prefixes too. The present patch only does methods.
2. It would be good to have this sandbox logic handled by a service so that sites or contrib could decorate it.
3. It's unfortunate that methods added in settings.php replace the defaults rather than add to them; this means that if you wish to add a method, you have to take maintenance responsibility for the whole list of methods.
4. I dislike the way that this central policy, which lacks intimate knowledge of the objects, has the job of making decisions about which are safe. I like the idea of creating a TwigSafeMethodInterface with an isTwigSafeMethod($function) method, that can be used on classes like EntityBase to allow decentralising the logic for this.

@neclimundul I'd appreciate your opinion on which of these are relevant to this issue, which are worth a follow-up, and which I should forget about.

I'm not going to drag the issue down with unwanted scope-creep, we'll get this to RTBC soon. But the _prefixes question above deserves at least a reason to be given here.

+++ b/core/lib/Drupal/Core/Template/TwigSandboxPolicy.php
@@ -88,20 +107,27 @@ public function checkPropertyAllowed($obj, $property) {}
+        if (str_starts_with($method, $prefix)) {

str_starts_with requires PHP 8, but we're still supporting php 7.4.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new148 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

neclimdul’s picture

Status: Needs work » Needs review
StatusFileSize
new11.96 KB

Reroll. The conflict was with core converting to starts_with I believe. No useful interdiff, the patch is the same.

smustgrave’s picture

What are the odds this could cause a breaking change for existing sites?

neclimdul’s picture

Status: Needs review » Needs work

Looking at this, it was targeted originally at deprecation in 9 and removal in 10. The current patch is the removal, not the deprecation so... likely to break things.

This needs to be re-rolled to deprecate this in 10 like the earlier patches did for 9.

neclimdul’s picture

Status: Needs work » Needs review
StatusFileSize
new12.03 KB
new1.06 KB

Tiny change but I've been I've been running in circles on some other things... This should be the non-breaking version. It just throws that deprecation message if your site doesn't match the hardened format.

Status: Needs review » Needs work

The last submitted patch, 28: 3239105-28.patch, failed testing. View results

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

neclimdul’s picture

Status: Needs work » Needs review

Quick reroll on MR

needs-review-queue-bot’s picture

Status: Needs review » Needs work

The Needs Review Queue Bot tested this issue.

While you are making the above changes, we recommend that you convert this patch to a merge request. Merge requests are preferred over patches. Be sure to hide the old patch files as well. (Converting an issue to a merge request without other contributions to the issue will not receive credit.)

andypost’s picture

neclimdul’s picture

neclimdul’s picture

Status: Needs work » Needs review

Also conflicted with #3487031: Performance Degraded after update to twig 3.14.2

merge request made and passing so back to NR.

The performance regression changes where a bit disruptive. They make this change kinda BC breaking because now the settings are directly exposed in the less secure expression format. Which is really frustrating and I don't really know how to resolve it.

larowlan’s picture

This looks good to me.
The main issue here is the disruption to existing sites who don't have an existing entry in settings.php who will now see their default change.
I think the only way we can handle that is with a release note, so tagging for needing a release note in the issue summary.
The change notice also looks good, but needs some version number updates. I think this is most likely 11.2 territory given we're in RC for 11.1, so I think it would be best to use that as the version number.
I'd be keen to see this go into 11.x (and therefore 11.2) early in the minor cycle.

Tagging as NW for the CR/Release note updates

dww made their first commit to this issue’s fork.

dww’s picture

I pushed a commit to fix the version number in the deprecation message, and also updated the CR. Leaving NW for the release note.

I'm not clear if we should be adding a default version of this to default.settings.php.

neclimdul’s picture

I was confused why we hadn't changed default.settings.php and the answer is that its not currently there. This is one of those power user settings we don't document and only set if you're running into an edge case.

I suspect that will continue and changing that would be its own issue but I understand how its tied up in this. I think we can mitigate this by linking to the documentation in the changelog and maybe release notes so in the unlikely case this does push someone into the edge case they know how to resolve it.

jonathanshaw’s picture

Issue summary: View changes
Status: Needs work » Reviewed & tested by the community

Added release note snippet to IS

dww’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: -Needs release note

Removing the release note tag thanks to #41.

However, I opened a new MR thread that should be addressed before merging (either renaming the function, or at least fixing the docs, or possibly refactoring this code again).

dww’s picture

Assigned: Unassigned » dww

Via Slack, @alexpott confirmed:
“ I think adding another setting is a good idea - twig_sandbox_allowed_class_methods seems best”
“ It is a bit least worst possible option - but sometimes that’s what you need.”

I’ll work on a version of this later today.

dww’s picture

Assigned: dww » Unassigned

Ugh, sorry. Got busy with too many other things. If anyone else wants to run with this, please do. I'll reassign if I pick it back up.

mvonfrie’s picture

Not directly related to the topic of this issue, but related:

Why is ::uuid not allowed as well for the types where ::id is allowed? At least this would make sense in relation to #1726734: Replace serial entity IDs with UUIDs in URLs, or even globally?, #2353611: Make it possible to link to an entity by UUID and related issues.

dww’s picture

@mvonfrie, re #45: Feel free to open another postponed child issue about that, much like #2907810: [PP-1] Add $entity->toUrl() and $entity->toLink() methods to allowed methods list in Twig sandbox policy. Once this issue is merged and fixed, it'll be much easier and safer to expand the allowed methods and target specific interfaces where we want to allow other methods.

Thanks,
-Derek

prudloff made their first commit to this issue’s fork.

rgpublic’s picture

It's probably too late for this idea, but nevertheless - perhaps for a future iteration: I think it'd make much more sense to use PHP attributes for this and specifically mark the methods that are really intended for consumption in Twig with #[TwigAllow]. Currently it's IMHO also a problem that contrib modules can't easily extend this.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

graber made their first commit to this issue’s fork.

neclimdul’s picture

There was a bug where this broke the __toString optimization listener. This didn't have any functional effects but twig 3.27.0 changed its sandbox and now the stringable sandbox logic has some pretty big side effects for certain objects. https://github.com/twigphp/Twig/issues/4820 This affects a bunch of Drupal objects so this bug in this patch could cause some pretty big breakage for sites using certain twig functions.

If you see something like Argument #1 ($node) must be of type Drupal\Core\Entity\ContentEntityInterface, array given this is the likely culprit.

I've fixed the bug and rebased to cleanly apply to main.

rgpublic’s picture

But we now have this?

https://www.drupal.org/node/3551699

Shouldnt this be used instead? Just wondering... Or perhaps I'm confusing something (sorry in advance if that's the case)?

neclimdul’s picture

I'll be honest its frustrating that happened and this is still stalled.

While it does seem to address a similar goal, I think no for a couple reasons.

1. Annotations aren't controllable by site administrators. Arguably the annotation removes control from site admins as previously they had full control and there is no method for excluding other then I guess maybe adding your own policy if that's possible.
2. This has control over methods that couldn't be annotated. Notably __toString but possibly others.