While working on #2031473: Convert menu local actions to plugins so that we can generate dynamic titles and paths we realized that the attributes array contains just the upcasted value, but there are serveral use cases where we really need the original value. The most obvious one is generating a Drupal path for a route with a dynamic pattern like /node/{node}/edit

Just replace the value won't fly, so the new value has to be stored somewhere else. The idea is to use a property bag
stored in $request->attributes->get('_raw_variables')

In order to still allow nice injection into controllers/whatever else, the controller resolver (which is a class which puts arguments from the request and together with the information about the controller, figures out the needed values).

Follow-up

#2034857: What to do with raw request attributes vs. converted (upcast) attributes

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner’s picture

Status: Active » Needs review
FileSize
15.67 KB
fubhy’s picture

I did the same on my typed data resolver sandbox as part of the ParamConverter refactoring/cleanup (including some other things). So yeah, we _really_ don't want to override our request attributes like we do now and instead put the upcast values somewhere else so we can still retrieve the original values.

+1

We can either commit that here (if green and after a good review) or together with the param converter refactoring.

Status: Needs review » Needs work

The last submitted patch, drupal-2031487-1.patch, failed testing.

dawehner’s picture

List of access checkers which deal with upcasted values:

  • EntityAccessCheck
  • LinkDeleteAccessCheck
  • RoleAccessCheck (account object)
  • LoginStatusCheck (account object)
  • FilterAccessCheck
  • FormatDisableCheck
  • DeleteMenuAccessCheck
  • EntityCreate (will maybe dissapear)
  • DeleteLInkAccessCheck (WTF?)
fubhy’s picture

Status: Needs work » Needs review
FileSize
9.13 KB
18.52 KB

We discussed this during the WSCCI meeting yesterday and agreed that it is necessary to solve this somehow. I am still wondering if it's better to keep a copy of the raw values somewhere and continue to do the upcasting by overriding in the actual attributes array or do what this patch does now (adding another ParameterBag to the attributes where we put the converted values).

Status: Needs review » Needs work

The last submitted patch, 2031487-5.patch, failed testing.

fubhy’s picture

Status: Needs work » Needs review
FileSize
18.6 KB
790 bytes
dawehner’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests
+++ b/core/includes/menu.incundefined
@@ -985,7 +985,10 @@ function menu_item_route_access(Route $route, $href, &$map) {
+      if ($request->attributes->get('converted')->has($matches['placeholder'])) {
+        $map[$index] = $request->attributes->get('converted')->get($matches['placeholder']);
+      }
+      elseif ($request->attributes->has($matches['placeholder'])) {
         $map[$index] = $request->attributes->get($matches['placeholder']);
       }

You could also just use $map[$index] = $request->get('converted')->get($matches['placeholder'], $request->attributes->get($matches['placeholder']));

dawehner’s picture

Status: Needs work » Needs review
Issue tags: +PHPUnit
FileSize
25.3 KB

Use proper unit tests.

fubhy’s picture

Adding a follow-up for discussing whether it might be better to instead store the raw values somewhere else and keep doing these overrides.

#2034857: What to do with raw request attributes vs. converted (upcast) attributes

Let's proceed with this issue though as it helps to unblock/cleanup other things that rely on UrlGenerator (local actions, tasks, etc).

Status: Needs review » Needs work

The last submitted patch, drupal-2031487-9.patch, failed testing.

dawehner’s picture

Priority: Normal » Major

As this blocks a proper local actions/local tasks api I consider this as major.

In general I also like the raw approach, whatever works, but to be honest we should just agree on one approach.

tim.plunkett’s picture

Title: Don't replace the upcasted values in the request attributes array. » Don't replace the upcasted values in the request attributes array
Issue tags: +WSCCI
fubhy’s picture

Assigned: Unassigned » fubhy

Judging by the latest replies on the other issue I guess we now agreed on storing a copy of the raw values instead. I like that much better. I have got a patch for that on my office pc waiting to be uploaded. Will post it here in the morning.

fubhy’s picture

Assigned: fubhy » Unassigned
Status: Needs work » Needs review
FileSize
6.61 KB

Storing a backup of the raw values seems much simpler indeed. What about something like this? I believe the ParamConverter is the right place to store the backup as it also knows which things were converted.

I made sure this patch touches as few files as possible and thus also reverted the architectural changes made to the ParamConverterManager previously in this issue. We are doing those in the DX issue anyways.

Status: Needs review » Needs work

The last submitted patch, 2031487-15.patch, failed testing.

fubhy’s picture

Status: Needs work » Needs review
FileSize
815 bytes
6.65 KB
dawehner’s picture

+++ b/core/lib/Drupal/Core/Controller/ControllerResolver.phpundefined
@@ -98,4 +99,21 @@ protected function createController($controller) {
+   */
+  protected function doGetArguments(Request $request, $controller, array $parameters) {
+    $arguments = parent::doGetArguments($request, $controller, $parameters);
+    if ($request->attributes->has('_raw') && $raw = $request->attributes->get('_raw')->all()) {
+      foreach ($parameters as $parameter) {
+        // Use the raw value if a parameter has no typehint.
+        if (!$parameter->getClass() && isset($raw[$parameter->name])) {
+          $position = $parameter->getPosition();
+          $arguments[$position] = $raw[$parameter->name];
+        }
+      }
+    }
+    return $arguments;
+  }

Some more documentation what goes on and why we are doing it should be added.

+++ b/core/modules/system/tests/modules/paramconverter_test/lib/Drupal/paramconverter_test/TestControllers.phpundefined
@@ -8,13 +8,14 @@
-  public function testUserNodeFoo($user, $node, $foo) {
+  public function testUserNodeFoo(UserInterface $user, $node, $foo) {
     $retval = "user: " . (is_object($user) ? $user->label() : $user);
     $retval .= ", node: " . (is_object($node) ? $node->label() : $node);
     $retval .= ", foo: " . (is_object($foo) ? $foo->label() : $foo);
diff --git a/core/tests/Drupal/Tests/Core/Controller/ControllerResolverTest.php b/core/tests/Drupal/Tests/Core/Controller/ControllerResolverTest.php

Any reason why there is still is_object on there?

Status: Needs review » Needs work

The last submitted patch, 2031487-17.patch, failed testing.

fubhy’s picture

Status: Needs work » Needs review
FileSize
3.72 KB
8.37 KB

Thanks for the review.

Should be green now.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

Thank you very much.

Crell’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: +Needs tests
+++ b/core/lib/Drupal/Core/Controller/ControllerResolver.php
@@ -98,4 +99,27 @@ protected function createController($controller) {
+        // Use the raw value if a parameter has no typehint.

Are you sure about that? Why wouldn't I want the object just because I have no type hint? It could be a highly variable object in some cases.

(You're free to push back and say that in that case you can bloody well upcast it yourself if you feel that's appropriate.)

Crell’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

Eh, never mind. :-) Crosspost.

fubhy’s picture

Are you sure about that? Why wouldn't I want the object just because I have no type hint? It could be a highly variable object in some cases.

(You're free to push back and say that in that case you can bloody well upcast it yourself if you feel that's appropriate.)

Oh, we are still doing upcasting, even if there is no type hint. It's just that we do not pass the upcasted object in the argument if there is no typehint. You can still pull it from the request if you wanted to:

public function mySuperGenericController($foo) {
  $object = is_object($foo); // This is FALSE
  $object = is_object($request->attributes->get('foo')); // This MAY be TRUE
}

So yeah, I am pretty sure it's okay to do this. What are you going to do in your controller if you don't even know that it's a object of some sort? How are you going to interact with it? There is just a limited number of things you can do without knowing what object you are actually working with (if we are speaking about realistic, real-world use-cases).

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs issue summary update

The issue summary says...

@TODO Better summary.

Also...

+++ b/core/tests/Drupal/Tests/Core/Controller/ControllerResolverTest.phpundefined
@@ -0,0 +1,76 @@
+    $this->assertEquals($mock_entity, $arguments[0], 'Type hinted variables used no upcasted values.');
+    $this->assertEquals(1, $arguments[1], 'Not type hinted variables used upcasted values.');

The messages here are incorrect. They should be saying what we expect to happen because when it fails, the statement fails to be true.

alexpott’s picture

Issue summary: View changes

Updated issue summary.

dawehner’s picture

Issue summary: View changes

blub

pwolanin’s picture

Title: Don't replace the upcasted values in the request attributes array » When replacing the upcasted values in the request attributes array, retain the original raw value too
Issue tags: -Needs issue summary update

fixing title

pwolanin’s picture

Issue summary: View changes

blub

pwolanin’s picture

Is _raw really the right key to use here?

Also, would it make sense to add the UID from the user object as the _raw 'account' value (the new global user object)?

dawehner’s picture

Status: Needs work » Needs review
FileSize
896 bytes
8.38 KB

So there are quite a lot of places which applied the opposite logic when switching to phpunit ...

pwolanin’s picture

per later discussion - maybe not sensible to sick account in there globally - individual consumers can handle that.

fubhy’s picture

FileSize
8.38 KB
2.38 KB

Peter suggested we should go with a colon instead. I have no strong preference. I just feel that our request attributes are getting quite messy quickly and we seem to dump unrelated stuff in there all the time. We really need to find a solution for that.

Also @see #2043781: Drupal::request()->attributes->get('account') may conflict with an account object loaded from the path ... Or think about HtmlFormController (form_state / form). Ugh

Let's find a convention. Or even better, find some other place for this stuff.

pwolanin’s picture

Status: Needs review » Needs work

Motivation for colon is that is something that should never appear in a variable taken from the URI http://tools.ietf.org/html/rfc6570

+++ b/core/lib/Drupal/Core/ParamConverter/ParamConverterManager.php
@@ -68,17 +69,23 @@ public function enhance(array $defaults, Request $request) {
 
+    // Keep the copies the raw values of things that were converted.
+    $backup = array_intersect_key($backup, array_flip($converted));
+    $defaults[':raw'] = new ParameterBag($backup);

I don't think the logic is quite right here - we should keep the value in :raw even if it was not converted, otherwise we have to go hunting for it later.

pwolanin’s picture

Status: Needs work » Needs review
FileSize
2.17 KB
8.46 KB

per discussion with fubhy in IRC - this keeps a copy of all the raw values that are names of variables in the route path pattern.

This should make it much simpler to supply all the needed parameters for the route e.g. for the URL generator.

jibran’s picture

Can we use _raw instead of :raw?

pwolanin’s picture

@jibran - why? We should use something that can never be a path variable name.

Status: Needs review » Needs work
Issue tags: -PHPUnit, -WSCCI

The last submitted patch, 2031487-32.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review
Issue tags: +PHPUnit, +WSCCI

#32: 2031487-32.patch queued for re-testing.

dawehner’s picture

Another possible prefix "#" would be at least more common in Drupal world.

fubhy’s picture

We should go back to _ actually now that I thought about it more. It gives us the possibility to get it passed as $_raw.

dawehner’s picture

What is the usecase for that? If someone really wants the raw values, it is fine to get it manually from the request object.

pwolanin’s picture

@dawhhner - the assertion is that some controller would want that instead of the whole request. I can see both sides of that argument - and I dont' really liek the symfony _ prefix habbit.

Crell’s picture

Nevertheless, _-prefixing for "Special" is the Symfony convention and the convention we're already using in a bunch of places. I am very -1 on introducing more special characters. We're not Perl. :-)

dawehner’s picture

FileSize
2.52 KB
8.46 KB

I agree, let's not assume that the developer is stupid.

fubhy’s picture

Agreed.

Crell’s picture

Status: Needs review » Reviewed & tested by the community

Let's get on with it then.

Dries’s picture

Status: Reviewed & tested by the community » Needs work

Patch no longer applies after the parameter router patch. Needs a quick re-roll.

fubhy’s picture

Assigned: Unassigned » fubhy
fubhy’s picture

Assigned: fubhy » Unassigned
Status: Needs work » Needs review
FileSize
7.23 KB

Re-roll

pwolanin’s picture

I think "_raw" is too vague. This converts it to "_raw_variables" since these are the raw values corresponding to the Route variables (i.e. path slugs like {node})

pwolanin’s picture

Issue summary: View changes

elaborate use case

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Updated the summary.

Status: Reviewed & tested by the community » Needs work
Issue tags: -PHPUnit, -WSCCI

The last submitted patch, 2031487-48.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review

#48: 2031487-48.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, 2031487-48.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review

#48: 2031487-48.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +PHPUnit, +WSCCI

The last submitted patch, 2031487-48.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review
FileSize
798 bytes
7.28 KB

hmm this was a real fail:

( ! ) Fatal error: Call to a member function compile() on a non-object in /Users/Shared/www/drupal-8/core/lib/Drupal/Core/ParamConverter/ParamConverterManager.php on line 157

Looks like fubhy's last re-roll removed the $route variable by mistake. This puts it back.

jibran’s picture

Status: Needs review » Reviewed & tested by the community

RTBC as per #49

catch’s picture

Title: When replacing the upcasted values in the request attributes array, retain the original raw value too » Change notice: When replacing the upcasted values in the request attributes array, retain the original raw value too
Priority: Major » Critical
Status: Reviewed & tested by the community » Active
Issue tags: +Needs change record

Not 100% about raw variables, but seems better than _raw. Committed/pushed to 8.x

Will need a change notice.

dawehner’s picture

Added one paragraph to the main route system change notice: https://drupal.org/node/1800686/revisions/view/2764405/2780643

dawehner’s picture

Status: Active » Needs review

.

larowlan’s picture

Title: Change notice: When replacing the upcasted values in the request attributes array, retain the original raw value too » When replacing the upcasted values in the request attributes array, retain the original raw value too
Priority: Critical » Major
Status: Needs review » Fixed
Issue tags: -Needs change record

Change notice is fine

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

blub