Problem/Motivation
When developing #2907810: [PP-1] Add $entity->toUrl() and $entity->toLink() methods to allowed methods list in Twig sandbox policy it was identified that the current sandbox method matching is very broad, allowing for methods to match _any_ object. This means it could possibly lead to unintended calls to possibly unsafe methods.
By hardening this is also makes rationalizing new additions to the allowed methods like ::toUrl and ::toLink if its possible to restrict the methods to a specific interface instead of trying to guess at any place the method might exist and how safe or unsafe its usage might be.
Steps to reproduce
Proposed resolution
Modify the format of the allowed_method settings to allow targeting methods on specific interfaces or classes. Something like:
$allowed_methods = Settings::get('twig_sandbox_allowed_methods', [
// Only allow idempotent methods.
EntityInterface::class . '::id',
EntityInterface::class . '::label',
EntityInterface::class . '::bundle',
// Globally allowed methods.
'::get',
'::__toString',
'::toString',
]);
Remaining tasks
Finalize how much we harden this.
User interface changes
n/a
API changes
Data model changes
Release notes snippet
Twig sandboxing
Drupal's Twig default sandboxing has been hardened. It now allows access to the id, label, and bundle methods only on entities, not on any object. If you have custom templates that make use of these methods, you will need to customise the twig_sandbox_allowed_methods setting.
| Comment | File | Size | Author |
|---|
Issue fork drupal-3239105
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
neclimdulHardening component of #2907810: [PP-1] Add $entity->toUrl() and $entity->toLink() methods to allowed methods list in Twig sandbox policy
Comment #5
neclimduladding some credits to people that participated in generating the current patch.
Comment #6
neclimdulThis is going to fail for methods of the same name on different interfaces.
A good example that we might actually want to include is AccountInterface::id
Returning false doesn't behave like you might expect for this method. There actual is _no_ return type for this method, failures are suppose to throw and exception and calls in twig ignore any return value. I guess we don't have any failure case tests currently. Yikes.
Comment #7
neclimdulWe do have tests for "dangerous methods" but they didn't cover the new case where the method matched but the interface didn't.
More tests and address the two points above.
Comment #8
longwave> There actual is _no_ return type for this method
Twig 3 declares the return type as void so any return statement would error, but we are still on Twig 2 :(
This message should probably explain that it's part of
twig_sandbox_allowed_methods. Also needs a test and change record.Comment #10
chi commented+ if (strpos($method, $prefix) === 0) {
As Drupal core requires "symfony/polyfill-php80" it should be safe to use str_starts_with.
Comment #11
neclimdulGood catch on the return, we probably shouldn't be writing new code we know will fail on twig 3 so I've remove the return value.
Also modified the deprecation and added the test. Turns out there was a bug in the BC that this caught!
Good eye on that polyfill. Technically, that code didn't change it as just moved, so I'd generally argue against touching it. But that's both a better method and the polyfill has a better implementation so might as well make the obvious fix. Bonus, the code is self documenting now so we don't need to explain the reason for it anymore :)
Looks like we've got some fun bugs. Looks like that suspicion about some breaking changes here are confirmed:
I added LayoutDefinition::id because its a plugin not an entity. Maybe we add PluginDefinitionInterface? Its probably safe but I just added LayoutDefinition in this pass to see how things go. Thoughts?
Comment #12
neclimdulComment #14
jonathanshawVery sensible. Implementation looks complete, tests look good.
I'm concerned about contrib here. It's easy for sites to add to twig_sandbox_allowed_methods. But how do contrib modules and themes do that if they need to?
The only clue I could find was https://stackoverflow.com/questions/16070602/how-to-properly-enable-the-...
Even if there's no supported way for contrib to currently add methods, it could be frustrating if we removed global support for id etc and there was no easy fix for contrib to restore it.
I wonder it's best to keep id and label global for now, and discuss making them more specific in a follow-up. This issue would add good infrastructure for more precise and harder policies, even if it didn't do much hardening itself.
The best hardening I have thought of would be to create a DrupalObjectInterface or TwigSandboxableInterface (without any methods on it itself), and have EntityInterface, PluginInterface, Url, etc extend from it. By extending from it these child interfaces would be contracting that they shared the generic Drupal understanding of id, label, get, toString as safe operations.
If we wanted to be really radical, we could give DrupalObjectInterface an
isSafeMethodCall($method) {}and therefore delegate responsibility for twig sandboxing to the classes themselves. After all, they are the ones who know what their own methods mean.Worth commenting this condition is for BC and can be removed in 10.0?
Nit: should return "bundle" for consistency?
Comment #15
neclimdulThanks!
If tightening some of these globals is holding the issue back from getting committed we could make them global again. Since this is about that hardening, it seems worth going as far as possible though and I haven't found anything this would effect yet.
I think you're right about adding extension for contrib being the method for adding their own security exceptions. That's actually exactly how core's is doing it are a couple contrib modules adding extensions with tweaks and features so this doesn't seem a big deal.
re: points
1) That's actually not BC, all those ifs are used to combine the global and qualified formats. The only BC is the part with the deprecation message. I'll include a 10.0 patch with the BC removed.
2) Sure. that's definitely a nit since it doesn't even get used but it looks better I guess :)
Here's a reroll with an updated 9.4 deprecation message and 10.x patch with the removal. Might be late to do it but the patches are useful on their own.
Comment #16
jonathanshawIt's not only about hardening. It's also about making it easier to add other much needed exceptions like entity::toUrl and URL:: setAbsolute.
I'm happy to RTBC as is one we have a CR, but I suspect there will be more discussion of the BC.
I tried a grep at http://grep.xnddx.ru and could not find anything altering the sandbox policy. We should have a working example of this for the CR if we are going to break BC for id and label. I'm happy to write the CR if you can provide the snippet.
Comment #17
neclimdulI guess I meant this was spit to discuss that hardening complexity from the Url issue. Its titled and tagged specifically to discuss that. I'll tag it for maintainer review though because that's a decision they'll have to make a decision about the impact.
If it helps, I've been running this in production for about 6 months without issue. Everything that could have run into this was cleared by the EntityInterface.
As far as searching, I would not have expected to see it because generally this sort of thing would probably be handled in a preprocess. I also assumed adding an extension(like twig_tweak) meant adding another policy would be easy. Assumptions did their thing though and turns out there are problems.
1) Twig extensions are keyed by class name so you can only have one
SandboxExtension.2) SandboxExtension is handled special so you can't add a sandbox-like extension.
3) Not matching throws an exception(makes sense, there can only be one) so you can't pass off handling to another exception.
That's kinda a mess and means you'd actually have to document sites adding to the setting. Probably a better method though would be to a preprocess method or adding filters like twig_tweak does to work around some of these link building problems on entities.
Comment #18
neclimdultag
Comment #19
jonathanshawAh, of course. Preprocess is the right solution for the CR.
How about we deprecate calling id & label from twig outside of EntityInterface? i.e. trigger a deprecation error if they are used in the sandbox on an object that doesn't extend EntityInterface.
That would progress the hardening but respect our usual BC approach.
Comment #20
neclimdulCreated the PR and included that as part of the notes.
I'm going to think through the more conservative approach to id and label but I'm
Comment #21
jonathanshawThe CR looks good, I've tweaked slightly.
Stepping back to look at this problem space more broadly I come to think that:
1. If being class-specific is good for twig_sandbox_allowed_methods then probably it's good for twig_sandbox_allowed_prefixes too. The present patch only does methods.
2. It would be good to have this sandbox logic handled by a service so that sites or contrib could decorate it.
3. It's unfortunate that methods added in settings.php replace the defaults rather than add to them; this means that if you wish to add a method, you have to take maintenance responsibility for the whole list of methods.
4. I dislike the way that this central policy, which lacks intimate knowledge of the objects, has the job of making decisions about which are safe. I like the idea of creating a TwigSafeMethodInterface with an isTwigSafeMethod($function) method, that can be used on classes like EntityBase to allow decentralising the logic for this.
@neclimundul I'd appreciate your opinion on which of these are relevant to this issue, which are worth a follow-up, and which I should forget about.
I'm not going to drag the issue down with unwanted scope-creep, we'll get this to RTBC soon. But the _prefixes question above deserves at least a reason to be given here.
str_starts_with requires PHP 8, but we're still supporting php 7.4.
Comment #24
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.
Comment #25
neclimdulReroll. The conflict was with core converting to starts_with I believe. No useful interdiff, the patch is the same.
Comment #26
smustgrave commentedWhat are the odds this could cause a breaking change for existing sites?
Comment #27
neclimdulLooking at this, it was targeted originally at deprecation in 9 and removal in 10. The current patch is the removal, not the deprecation so... likely to break things.
This needs to be re-rolled to deprecate this in 10 like the earlier patches did for 9.
Comment #28
neclimdulTiny change but I've been I've been running in circles on some other things... This should be the non-breaking version. It just throws that deprecation message if your site doesn't match the hardened format.
Comment #31
neclimdulQuick reroll on MR
Comment #33
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue.
While you are making the above changes, we recommend that you convert this patch to a merge request. Merge requests are preferred over patches. Be sure to hide the old patch files as well. (Converting an issue to a merge request without other contributions to the issue will not receive credit.)
Comment #34
andypostComment #35
neclimdulbroken by #3470890: Remove misc usage of whitelist in tests and comments. reroll soon.
Comment #36
neclimdulAlso conflicted with #3487031: Performance Degraded after update to twig 3.14.2
merge request made and passing so back to NR.
The performance regression changes where a bit disruptive. They make this change kinda BC breaking because now the settings are directly exposed in the less secure expression format. Which is really frustrating and I don't really know how to resolve it.
Comment #37
larowlanThis looks good to me.
The main issue here is the disruption to existing sites who don't have an existing entry in settings.php who will now see their default change.
I think the only way we can handle that is with a release note, so tagging for needing a release note in the issue summary.
The change notice also looks good, but needs some version number updates. I think this is most likely 11.2 territory given we're in RC for 11.1, so I think it would be best to use that as the version number.
I'd be keen to see this go into 11.x (and therefore 11.2) early in the minor cycle.
Tagging as NW for the CR/Release note updates
Comment #39
dwwI pushed a commit to fix the version number in the deprecation message, and also updated the CR. Leaving NW for the release note.
I'm not clear if we should be adding a default version of this to
default.settings.php.Comment #40
neclimdulI was confused why we hadn't changed default.settings.php and the answer is that its not currently there. This is one of those power user settings we don't document and only set if you're running into an edge case.
I suspect that will continue and changing that would be its own issue but I understand how its tied up in this. I think we can mitigate this by linking to the documentation in the changelog and maybe release notes so in the unlikely case this does push someone into the edge case they know how to resolve it.
Comment #41
jonathanshawAdded release note snippet to IS
Comment #42
dwwRemoving the release note tag thanks to #41.
However, I opened a new MR thread that should be addressed before merging (either renaming the function, or at least fixing the docs, or possibly refactoring this code again).
Comment #43
dwwVia Slack, @alexpott confirmed:
“ I think adding another setting is a good idea -
twig_sandbox_allowed_class_methodsseems best”“ It is a bit least worst possible option - but sometimes that’s what you need.”
I’ll work on a version of this later today.
Comment #44
dwwUgh, sorry. Got busy with too many other things. If anyone else wants to run with this, please do. I'll reassign if I pick it back up.
Comment #45
mvonfrie commentedNot directly related to the topic of this issue, but related:
Why is
::uuidnot allowed as well for the types where::idis allowed? At least this would make sense in relation to #1726734: Replace serial entity IDs with UUIDs in URLs, or even globally?, #2353611: Make it possible to link to an entity by UUID and related issues.Comment #46
dww@mvonfrie, re #45: Feel free to open another postponed child issue about that, much like #2907810: [PP-1] Add $entity->toUrl() and $entity->toLink() methods to allowed methods list in Twig sandbox policy. Once this issue is merged and fixed, it'll be much easier and safer to expand the allowed methods and target specific interfaces where we want to allow other methods.
Thanks,
-Derek
Comment #47
mvonfrie commentedCreated #3506428: [PP-1] Add $entity->uuid() methods to allowed methods list in Twig sandbox policy.
Comment #49
rgpublicIt's probably too late for this idea, but nevertheless - perhaps for a future iteration: I think it'd make much more sense to use PHP attributes for this and specifically mark the methods that are really intended for consumption in Twig with #[TwigAllow]. Currently it's IMHO also a problem that contrib modules can't easily extend this.
Comment #52
neclimdulThere was a bug where this broke the __toString optimization listener. This didn't have any functional effects but twig 3.27.0 changed its sandbox and now the stringable sandbox logic has some pretty big side effects for certain objects. https://github.com/twigphp/Twig/issues/4820 This affects a bunch of Drupal objects so this bug in this patch could cause some pretty big breakage for sites using certain twig functions.
If you see something like
Argument #1 ($node) must be of type Drupal\Core\Entity\ContentEntityInterface, array giventhis is the likely culprit.I've fixed the bug and rebased to cleanly apply to main.
Comment #53
rgpublicBut we now have this?
https://www.drupal.org/node/3551699
Shouldnt this be used instead? Just wondering... Or perhaps I'm confusing something (sorry in advance if that's the case)?
Comment #54
neclimdulI'll be honest its frustrating that happened and this is still stalled.
While it does seem to address a similar goal, I think no for a couple reasons.
1. Annotations aren't controllable by site administrators. Arguably the annotation removes control from site admins as previously they had full control and there is no method for excluding other then I guess maybe adding your own policy if that's possible.
2. This has control over methods that couldn't be annotated. Notably __toString but possibly others.