During development on Drupal 8 many people realized that the amount of possible variables is very limited. So for examples once view_modes got entities a lot of controllers
broke, because they still assumed the pure string.
The idea of this issue is to just do automatic upcasting based upon the variable_name == entity_type_id idea, if you have some sort of typhinting, like the SpecificEntityInterface.
aggregator_feed_refresh:
pattern: '/admin/config/services/aggregator/update/{aggregator_feed}'
defaults:
_controller: '\Drupal\aggregator\Controller\AggregatorController::feedRefresh'
requirements:
_permission: 'administer news feeds'
And a controller that looks something like this:
/**
* Refreshes a feed, then redirects to the overview page.
*
* @param \Drupal\aggregator\FeedInterface $aggregator_feed
* An object describing the feed to be refreshed.
* @param \Symfony\Component\HttpFoundation\Request $request
* The current request object containing the search string.
*
* @return \Symfony\Component\HttpFoundation\RedirectResponse
* A redirection to the admin overview page.
*
* @throws \Symfony\Component\HttpKernel\Exception\AccessDeniedHttpException
* If the query token is missing or invalid.
*/
public function feedRefresh(FeedInterface $aggregator_feed, Request $request) {
// @todo CSRF tokens are validated in page callbacks rather than access
// callbacks, because access callbacks are also invoked during menu link
// generation. Add token support to routing: http://drupal.org/node/755584.
$token = $request->query->get('token');
if (!isset($token) || !drupal_valid_token($token, 'aggregator/update/' . $aggregator_feed->id())) {
throw new AccessDeniedHttpException();
}
// @todo after https://drupal.org/node/1972246 find a new place for it.
aggregator_refresh($aggregator_feed);
return new RedirectResponse(url('admin/config/services/aggregator', array('absolute' => TRUE)));
}
Now, due to the fact that FeedInterface
extends EntityInterface
we already know that what we have here is an entity. Now, since all entities should define a unique interface like FeedInterface
or NodeInterface
we can iterate over the $entityManager->getDefinitions()
and check (with Reflection) which entity type uses an entity class that implements FeedInterface
. This is the entity type we want to upcast to. This obviously only works if the typehinted interface is unique for a given entity type. If it is ambiguous like an EntityInterface
typehint or some other, custom SomethingBaseInterface
the reflection discovery skips the given route.
Comment | File | Size | Author |
---|---|---|---|
#209 | 1906810-209.patch | 57.71 KB | damiankloip |
#200 | interdiff-1906810-200.txt | 4.61 KB | damiankloip |
Comments
Comment #1
EclipseGc CreditAttribution: EclipseGc commentedI think this would also be really really helpful to scotch and contrib built around scotch. We are defining plugin contexts via typed_data so if the routes were defining their parameter upcasting via typed_data as well (and I would vote for a more explicit defining here) then we could seamlessly work together, and I think that was always the point.
Eclipse
Comment #2
fubhy CreditAttribution: fubhy commentedOkay, giving this a try... Pretty simple actually. Note that the syntax for entity upcasting is currently a bit weird with the constraints there but that will improve once #1868004: Improve the TypedData API usage of EntityNG lands.
I didn't touch any of the existing Enhancers so Views et al still use the EntityParamConverter. But the tests should pass.
Comment #3
fubhy CreditAttribution: fubhy commentedTagging.
Comment #4
Crell CreditAttribution: Crell commentedThis would imply no auto-detection at all, requiring module developers to explicitly define all conversions. That's a no. We can't regress DX like that.
We can't force all module developers to do this on every route.
Comment #5
fubhy CreditAttribution: fubhy commentedAuto-detection magic on routes does not work with SCOTCH. We can't do any Condition validation, Variant matching (however we end up doing that? NestedMatcher (RouteFilter)? RouteEnhancer?) without metadata about the {slugs}.
As mentioned in #2 once #1868004: Improve the TypedData API usage of EntityNG lands the syntax is going to be incredibly simple for entities. The current .yml syntax does not look nice, granted. This is what it would look like with #1868004: Improve the TypedData API usage of EntityNG:
Comment #6
Crell CreditAttribution: Crell commentedWe put a lot of effort into not causing a DX regression in the original param converter issue. I cannot get behind regressing that. I've mentioned that to EclipseGc multiple times.
Comment #7
EclipseGc CreditAttribution: EclipseGc commented%node is documentation, {node} is not. one tells me how to load, the other is magic that guesses. (well ok %node can be magicky too when it doesn't work, but the fact remains) This is not a regression, and buys us MUCH. This approach is very similar to any situation in which you might have two of the same thing in the url you need upcast. magic vs non-magic in that instance seems like a real dx wtf to me. Explicit documentation of how to upcast parameters isn't really weird, aren't there other benefits to this as well?
If you aren't open to that idea, then I'm out of ideas on how to prevent a plugin type like page_manager's tasks from existing. We'll have to do a lot of really dirty guessing of parameters when we're not actually on the route and it will be messy. I'd encourage us to really consider this further before declaring it a "no go". As always I'm happy to chat about this and illustrate my reasoning further.
Eclipse
Comment #8
Crell CreditAttribution: Crell commented%node triggers automagic conversion of an into to a stdClass. Using {node} does the same. "One is documentation, the other tells me how to load" is completely false. The only difference is that the "magic guess" of {node} is, currently, more involved than "guess for a function with a magic name".
Admittedly, another potential benefit is being able to use {account} instead of {user}, because {user} means a variable named $user that is not the same as the global $user variable, which is a security hole BEGGING to happen, but the fix to that is to get rid of the crime against nature that is global $user.
If we were to require explicit declaration and require that ALL parameters were typed data only, then at that point we probably just rip out the ParamConverter system entirely and make it a single route enhancer (or bridged from a single route enhancer). Anyone trying to upcast something that's not a typed data object then is on their own. That would likely make Sam happy, because we could trivially do that upcasting earlier during a Route Filter. However, that then requires explicit declaration on the route. That's a regression from "%node is magic, it just works". There's no way to whitewash that.
Maybe that trade-off is worth it. But we can't pretend it's not a trade-off. And... I'm not going to make that call myself. :-)
Comment #9
fubhy CreditAttribution: fubhy commentedExactly what I'd say we should do. And exactly what this patch does, except for the part where we remove the ParamConverters ;-). And yes, it would make Sam I am happy. That I am sure of... Okay, we gotta admit that it takes slightly more effort to create a route with typed data upcasting. But in most cases it's just one additional line of .yml which is absolutely fine if you ask me. However, while it might decrease DX when writing the route definitions it increases readability of the code / .yml. With the metadata on the route it's much more obvious what's going to happen during the request.
So, who can we assign this to to make the call? :P
Comment #10
EclipseGc CreditAttribution: EclipseGc commentedFWIW, a LOT of stuff is typed data already. every entity can be upcast that way (config or content) strings, ints, dates, durations, fields and more. This is one of the big reasons I think page_manager has had such a long road to community acceptance, the "context plugins" which is essentially what typed data is doing in D8, are a pain to write, and no one understands them and the only system using them is page_manager, so when people run into a situation where they need one, rather than invest in it, they just move on. Typed Data already covers a lot more than page_manager's context system ever tried to cover, and should solve I think 98% of the parameter upcasting use case, and then if we are forcing this, it prevents any "writing around" it that's likely to happen and gives contrib a consistent tool for solving the problem. Most typed_data plugins are actually pretty simple, it's really just entity that is confusing and that's being worked on.
Indeed we should probably figure out who can make this call.
Eclipse
Comment #11
Anonymous (not verified) CreditAttribution: Anonymous commentedFrom today's meeting, it sounds like this is major for SCOTCH.
It seems like it's blocked on figuring out who can make a call?
Comment #12
webchickI can throw in an opinion, if not make a call... I find the DX impact of #4 absolutely repulsive. It seems like the argument for foisting this extra overhead onto 100% of module developers is to make it easier for core developers and ~5% of module developers who define their own unique type casting logic. I do not find that an acceptable trade-off, since it seems like defining a foo.routing.yml file will happen about 20 bazillion times more often than defining a new way to typecast slugs.
"So, the gain from this would be: Less repetition of loading logic plus encouraging people to implement their objects as typed data :P." ... "This is important because I don't want to have to do all those checks in my controllers in D8."
I don't see how this makes this major, seems like just a nice to have? Maybe the issue summary needs an update?
Comment #13
Anonymous (not verified) CreditAttribution: Anonymous commentedOK, I'm going to bump this back down to normal until the issue summary makes it clear what functionality in SCOTCH this blocks.
Comment #14
EclipseGc CreditAttribution: EclipseGc commentedNo, this is not at all a good summary of why we need this, or what it buys us.
When you upcast something currently, I have exactly 0 idea of what that thing you're upcasting is. Also whatever it is, it likely has no layer around it which I can use when the upcast item is NOT available.
Let's take a simple example like node/%node (going to stay as close to D7 stuff as possible to illustrate this as clearly as possible).
The menu system in D7 knows that %node means node_load(). The system in D8 is going to check if 'node' is an entity name and upcast it. if not, there are some fallback guesses it will also make. From a metadata perspective this is completely useless. In terms of generating a "display" for this, I am immediately sunk because I know nothing about this page beyond what I can code/guess. And even if I know it's a node, I need it as typed data anyway, because when working in the UI I won't have a legit node object, I'll have something representing a node that is not actually (which typed data can do).
To put this into D7 terms, if you've ever looked at page_manager's task plugins, this is EXACTLY why they exist. The system has no clue what is on a page, so it needs a layer to do that interpretation. If I don't have some sort of metadata layer telling me what all the parameters in a route are, we will be in exactly the same place for D8 and no page except pages with explicit layers built for them will ever be able to have a display take over their output.
Embedding this metadata layer directly into the routing system will bring global display coverage to not only core, but also all of contrib. That is a HUGE win, and the fact that we're asking people to explicitly define how their parameters from the URL are loaded does not seem unreasonable, nor is it really a DX loss since there won't be any mystery about how the system works. This has the beneficial byproduct of forcing all future modules to define typed data handlers for anything they ever want to pass across the URL (there are many of these already for simple strings, integers, etc already) which means contrib continues to stay compatible with the core approach here.
I hope this clarifies why I think this is such a huge issue. I though I had outlined this once already, but apparently not, so my apologies.
Eclipse
Comment #15
fagoMy POV on the DX pain issue is that something along the lines of #5 is not a DX regression, but improvement. It tells me that the system is going to interpret that part of the path using the specified typed data plugin. That saves one from looking up and understanding the guessing approach. In short: less magic -> DX++. Then, I don't care much about having to provide those 3 LoC to define how it will load the object.
Comment #16
effulgentsia CreditAttribution: effulgentsia commentedI think #5 is fine when you have a slug named {entity} that you want to constrain to be a node of type 'page'. But I agree with webchick and disagree with fago that forcing extra lines on route declarations for slugs named {node} that you want to be any node makes for worse DX. I also don't see why it's necessary to do that to achieve the goals of this issue. I think it's possible to have TypedData support, to allow for introspection from SCOTCH, and also to allow for sane default metadata of slug names that correspond to entity types. Maybe we just need to move where the code that applies the defaults lives? #1943846: Improve ParamConverterManager and developer experience for ParamConverters contains one such move, though maybe we want to move it somewhere even different to be available to SCOTCH.
Comment #17
EclipseGc CreditAttribution: EclipseGc commentedI don't seen how separating the explicit loading declarations from the route buys us anything but yet more files/hooks/something. I want this on the route if possible, as this will ensure that we have a workflow that is not broken just for the sake of expediency. Likewise, both route parameters and typed data can do more than just entity loading. String, date, etc handling is a hugely powerful thing.
As a side note, I've got a d8 contrib I'm working through right now, and the number of files I'm creating is a little staggering. I like a lot of my workflow stuff doing it thus far, but that aspect has been a little daunting, and I already know most of the files I need to create... so, I'm "--" to separating the loading declaration from the route declaration.
Eclipse
Comment #18
effulgentsia CreditAttribution: effulgentsia commentedI'm certainly not suggesting that the loading declarations be specified manually somewhere other than in the routing.yml file. For ones that need loading declarations that can't be easily inferred from the slug name, I agree with putting them within the
options
portion of each route. All I'm saying is that there can be code somewhere along the lines of:This allows for routes like
/node/{node}
to not require options be manually specified within routing.yml files.Ideally, the code that provides this defaulting should be somewhere centralized, so that both routing code and SCOTCH can just call $route->getOptions() (or something like that, if not that exactly) and receive a $options array that includes a merge of manually specified and default options, and therefore, not need to care what was manually specified and what was defaulted.
Whether it makes sense to define defaults for slug names that aren't entity types remains to be seen. For example, {entity_type} and {langcode} seem like string-valued slug names that we should be able to define some default validation constraints around. But whether we do that or not doesn't change the desirability of doing it for {node}, {user}, and all other slug names that are entity types, since we know that those are very common slug names used in very many routes.
Comment #19
EclipseGc CreditAttribution: EclipseGc commentedwell, if we're going to suggest that (and based upon you're explanation I'm not entirely opposed) why not just expect slugs like:
{string}
{entity:node}
[language]
{entity:user}
{integer}
This would ensure that the slugs are in typed data order, and we'd just need a way to put constraints on it further in the options if necessary. So long as I can request from the route object a definition of all it's variables and get either definitions or full typed data objects, I'll be happy.
Eclipse
Comment #20
fubhy CreditAttribution: fubhy commentedAll right, this might sound cool at first. In simple cases AND if we can massage the Symfony route controller reflection factory so that it does not break when special characters like ":" are used it MIGHT work (Currently, I think, it's not possible due to the incapability of the reflection factory to map "entity:node" to $entity_node or $node or something). Additionally, how would this work if you got 2x {node} on the same route? Or, what if you have a route in module "A" that defines a slug named {foo} which is supposed to be a simple string and then module "B" comes along and defines an entity type called "foo"? ===> BOOM! The whole thing blows up.
In Drupal 7 you had some kind of protection against that because %node was reserved due to the existence of node_load() as a function which obviously fails anyways if a second module defines the same argument name and loader callback. Things like "%" (any value) did not have to be explicitly named and were simply passed to the 'page callback' by position, not by reflection. Now that we do this with reflection we have to be more specific I am afraid. Any other solution, as cool as it might look initially, is not failure proof.
That being said, I absolutely doubt that requiring module developers to write a single extra line of code/config to define the typed data type of {node} => 'entity:node' is asking for too much.
Anyways, there is one more possible solution that I can think of which would be both simple and failure proof but would require us to extend the controller factory to work with the custom {slug} syntax. I am thinking of allowing a {slug} syntax that would read something like this: {foo{entity:node}} and thereby declare the route constructor argument $foo to be of type "entity:node". Again, this would require some additional code to allow this syntax.
Comment #21
effulgentsia CreditAttribution: effulgentsia commentedOk, I think that's a pretty compelling argument actually. Can you open a new issue for removing the magic from the existing EntityConverter in HEAD? We need to get webchick, catch, and others to weigh in on that.
Comment #22
fubhy CreditAttribution: fubhy commentedYai! However, I think we need to fix this in one go (as in, put typed data upcasting in place while removing the EntityConverter) because some routes already rely on the new upcasting / param converter system and I doubt that we want to roll back those. It should not increase the size of this patch that much if we do it right here. I am open for better ideas/battleplans though.
Comment #23
catchRemoving the old stuff while adding the new one seems OK here - if it doesn't massively bloat the patch it'll make it easier to review the change.
I haven't reviewed the patch properly here so not commenting on the DX issue at all.
Comment #24
fubhy CreditAttribution: fubhy commentedI opened a related/blocking issue (at least for entities once the EntityWrapper is gone): #1983100: Provide a LoadableInterface for Typed Data objects
Comment #25
fubhy CreditAttribution: fubhy commentedThis patch ...
Note:
Also:
In #20 I said this:
I took another look at how possible ways of implementing such an alternative, simplified syntax. While it's technically absolutely possible to do this syntax parsing (either in \Drupal\Core\ControllerResolver or in the TypedDataEnhancer itself by replacing the $defaults array keys with the actual parameter name after upcasting) I am pretty sure we would regret that in the long run. Alternative syntax are usually a bad idea and usually cause even more confusion. Oh, and we would make life harder for SCOTCH too because they would have to do that same parsing themselves as well.
Hence, I would vote for simply going with the route options syntax. Once #1868004: Improve the TypedData API usage of EntityNG lands it's going to be much simpler anyways.
Comment #26
fubhy CreditAttribution: fubhy commentedOh and once #1983100: Provide a LoadableInterface for Typed Data objects is resolved we can also get rid of the ViewsUIEnhancer entirely by overriding 'class' in the typed data definition on the affected routes and use that to run upcasting for the Views objects through the Views UI Cache loader. Getting rid of the ViewsUIEnhancer is a good thing as we should make sure that we run as few enhancers as possible as they are evaluated for every route regardless of whether they apply or not (only the enhancer itself knows).
Comment #28
fubhy CreditAttribution: fubhy commentedForgot Feeds, Contact categories and Field UI.
Comment #29
fubhy CreditAttribution: fubhy commentedI opened #1983368: Remove ViewsUIConverter (Views UI Route Enhancer) to take care of removing the ViewsUIEnhancer once this (and #1983100: Provide a LoadableInterface for Typed Data objects) lands.
Comment #30
tim.plunkettThe DX regression on Views routes is really really unfortunate.
Seriously?
Also, ConfigEntities aren't typed data and I don't think we want them to be...
Comment #31
fubhy CreditAttribution: fubhy commentedAs mentioned previously, it will look like this soon:
And we can still talk about a simplified syntax like:
Technically absolutely possible as mentioned previously as well. We can either do that in our ControllerResolver or directly in the TypedDataEnhancer by replacing 'view[entity:view]' with simply 'view' in the $defaults array while upcasting it to the entity at the same time. I don't see a problem with that other than the potential confusion caused by multiple syntax for the same thing + possibly more effort required for SCOTCH to parse the route. But it's definitely surmountable.
Comment #32
fubhy CreditAttribution: fubhy commentedI spent some more time looking into possibilities for how to implement that simplified syntax...
It's actually not as easy as I thought due to the Symfony RouteCompiler being pretty restrictive in terms of extracting the {slugs} from the route. The regex currently only allows #\{\w+\}# which normally makes sense as they have to be mapped to constructor arguments through reflection.
Sadly, the Symfony RouteCompiler is a bit weird due to all the methods being marked as private so we can't easily override it's behavior without implementing it from scratch. It wouldn't be too hard to do exactly that but I wonder if we should hard-wire the route compiler with assumptions closely-related to typed data.
On the other hand, it sounds much better to evaluate the type syntax during route compilation and compile that simplified syntax into the $options array using the proper typed data definitions syntax so we don't have to do that on run-time.
I quickly hacked /vendor to do just that and it seems to work nicely... Again, technically this is all possible but first we should find out what we really want. Personally, I am fine with simply allowing this:
That syntax is easy enough for me and I personally don't see it as a DX regression at all.
Comment #33
tim.plunkettIt is still a DX regression, because if I want to load a View ConfigEntity, why do I have to know what typeddata is?
That's needlessly exposing internals, and frankly has nothing to do with Views, Routes, or ConfigEntities.
Comment #34
Crell CreditAttribution: Crell commented1) Removing the intermediary ParamConverter level is absolutely not OK. We MUST be able to do ParamConversion outside of the routing process to be able to generate access-restricted links. Removing that decoupling is a won't fix.
2) Hacking the placeholder (dude, really, slug? That's a stupid name; even Symfony doesn't use it generically like that) with an alternate syntax is also not cool. Right now, you can use existing Symfony documentation to document how our routing API works, about 80% of the way. The URI template syntax is an IETF standard:
http://tools.ietf.org/html/rfc6570
No, we are not breaking an IETF standard we've already adopted. That is not happening. The explicit options/typeddata construct is better than that in every possible way, even if I am still firmly against it on the grounds that "Typed Data ALL THE THINGS!" is not a concept that anyone has managed to sell me on yet in the first place.
3) The one and only compelling argument I've heard from anyone so far for why we can't just have, duh, multiple converters (one for entities, one for your own random crap, etc.) is avoiding Task plugins for SCOTCH (which is one of the most obtuse parts of Panels in D7, and that's some pretty steep competition). OK, so how do we address that problem in a way OTHER THAN "every object in all of Drupal must be part this big reimplementation of PHP syntax called Typed Data that all of 4 people understand"?
That's the question. And it's nominally separate from param conversion in the first place.
Comment #35
effulgentsia CreditAttribution: effulgentsia commentedIn #1943846: Improve ParamConverterManager and developer experience for ParamConverters, I've been suggesting a syntax like this:
That removes the words
typeddata
andconverter
from the routing file, both of which are internal concepts. It retains the idea that you need to explicitly provide information about your parameters, at a minimum, their "type", if you want to do anything with them other than pass through their string values.Not sure how relevant that question is to what's being discussed here, but the compelling argument for needing to explicitly state the type even for entity types, and even if that was implemented without TypedData, is in #21.
Comment #36
yched CreditAttribution: yched commentedRight, automatically recognizing entity type names is kind of handy but seems to bring more problems when modules get enabled and the magic semantics of a given placeholder suddenly changes.
Explicit argument declarations ++
Comment #37
tim.plunkettAbove, I said one of my biggest problems with this typeddata approach:
I've rerolled #1943846: Improve ParamConverterManager and developer experience for ParamConverters, I really like that approach over this.
So we'd end up with
For the routes that need a view entity upcast to a ViewUI from tempstore.
Can we retitle this to remove the TypedData parts, or should we just do that in the other issue and close this one?
Comment #38
fubhy CreditAttribution: fubhy commentedHad a long chat with @timplunkett as well as some input from @Crell and we found some common ground for this issue (+related issues). Assigning back to me. Upcoming patch is going to be built around #1943846: Improve ParamConverterManager and developer experience for ParamConverters. More about the conceptual idea behind that also tomorrow.
Comment #39
fubhy CreditAttribution: fubhy commentedComment #40
benjifisher#28: typed-data-upcasting-1906810-28.patch queued for re-testing.
Comment #43
jrglasgow CreditAttribution: jrglasgow commentedComment #44
jrglasgow CreditAttribution: jrglasgow commentedrerolled the patch in #28
Comment #45
jrglasgow CreditAttribution: jrglasgow commentedComment #47
jrglasgow CreditAttribution: jrglasgow commentedI apparently had a problem with that patch... let's try that again
Comment #49
fubhy CreditAttribution: fubhy commentedPostponing for #1943846: Improve ParamConverterManager and developer experience for ParamConverters. Let's first see where we end up with that issue.
Comment #50
disasm CreditAttribution: disasm commentedThe reroll introduced some no new line EOF errors. Please reroll again without introducing those errors (or modify the patch, if that's easier for you).
Comment #51
socketwench CreditAttribution: socketwench commentedReroll. Should eliminate the whitespace errors.
Comment #52
fubhy CreditAttribution: fubhy commentedI've been playing with this around in my head for the past couple of days and after discussions with Kris, Sam, Larry, etc. I came back to the conclusion that we -really- need this. And before the end of next week too. I am not going to go into the argument about why this is so fundamentally important as that has been explained in previous comments already. To summarize: A unified concept for explicitly describing the parameters of a route with metadata buys us generic implementation with all sorts of page manager features that Kris and Sam are working on in princess. Without that, we are making a huge leap backwards to D7 time where every route needs to be introduced to the pager manager through some sort of plugin. I hope everyone understand how much of a win this would be for sitebuilding...
Anyways... After a quick chat with @berdir about typed data and the planned changes that have been discussed at Portland (especially #2002102: Move TypedData primitive types to interfaces) and the LoadableInterface for typed data objects that I proposed over at #1983100: Provide a LoadableInterface for Typed Data objects there is a great opportunity for us to achieve this goal and even address the DX concerns people were having in this issue together. Let me go back in time and explain how we got to where we are now and then outline the idea that I have in mind:
How we ended up here
One of the first patches around param conversion (I think initially written by @katbailey?!) was built around the concept of reflection. While incredibly simple in terms of DX it did have some conceptual flaws and limitations and only covered entity upcasting back then. So we threw it out the window. After that, @Crell and @g.oechsler came up with the concept of "guessing" route arguments based on their naming pattern (e.g. {node} == Node entity). This looked pretty cool initially but turned out to be rather error-prone/unstable as explained in #1943846: Improve ParamConverterManager and developer experience for ParamConverters. So we decided to throw it out the window just as well and agreed that we need explicit declaration of parameter types. This is where we stand now...
However, when I saw how Routes and ParamConverters play together in SensioFrameworkExtraBundle I drooled and started thinking about this again. Now, after speaking with @EclipseGc and @sdboyer and also seeing that @effulgentsia seems to be more and more aboard with TypedData (as well as some other folks, I guess Portland was a big help with that) and talking to @Crell last night I sense that it is time to start discussing this issue again and ultimately make a decision.
Pre-requisites for the following concept
Why TypedData?
TypedData is the perfect solution for our parameter conversion problem as we can use it as a generic object factory/object factory location with a unified pattern for defining the parameter types so they can be identified by other sub-systems such as a page manager which would then be capable of fully understanding any defined route without requiring an intermediate plugin layer for every single route definition.
Furthermore it becomes very easy to make custom objects "upcastable" by simply implementing them as typed data objects. And that is pretty much a no-brainer. In many cases this is probably going to be a by-product anyways as many objects will want to be typed data anyways as that buys them integration with other systems as well. Implement it once, integrate with all the things.
TypedData upcasting through Reflection (or other means of type discovery) and explicit parameter definition
Now, how to we implement this without turning route definition DX into a nightmare? The latest patch in this issue is solely based on defining the type and constraints of typed data types. However, as #2002102: Move TypedData primitive types to interfaces has been started lately (and there is already a patch) we will be able to take another look at Reflection to provide an absolutely straight-forward concept for benefiting from Upcasting by simply typehinting the controller arguments with the desired typed data interface as an alternative to manually defining the typed data type in the route options.
The ultimate goal is to end up with a nice typed data definition for each parameter that should be upcast (basically everything except for things that should remain the original input values). The typed data definition can either be manually defined in the route options or automatically discovered by Reflection of the controller or other means during RouteBuildEvent (which then writes the discovered definition into the route options). So whatever scenario applies, in the end we will have the typed data definition in the route options.
How would that work?
First scenario - Reflection
Let's assume we have a route definition that looks something like this:
And a controller that looks something like this:
Now, due to the fact that
FeedInterface
extendsEntityInterface
we already know that what we have here is an entity. Now, since all entities should define a unique interface likeFeedInterface
orNodeInterface
we can iterate over the$entityManager->getDefinitions()
and check (with Reflection) which entity type uses an entity class that implementsFeedInterface
. This is the entity type we want to upcast to. This obviously only works if the typehinted interface is unique for a given entity type. If it is ambiguous like anEntityInterface
typehint or some other, customSomethingBaseInterface
the reflection discovery skips the given route.Second scenario - Automatic discovery for ambigious or missing typehints
Things like
HtmlEntityFormController::content()
do not have typehints. However, they make other assumptions. A route definition for an entity form looks something like this:We know that this is an entity form because the controller class is a HtmlEntityFormController. Hence, we can use the same type discovery logic that the controller uses: Retrieve the entity type from
$route->getDefault('_enitty_form')
by reading it from the string (@seeHtmlEntityFormController::getFormObject()
). Once retrieved we can, again, write it's typed data definition into the route options. Different discovery, same outcome.The same process can be applied to basically everything we got. The only criteria we need is that, in the end, we get a nice typed data definition in the route options.
Third scenario - Manual definition in the route options
You might have single, custom routes in your module which do not justify writing a proper parameter type detection service for the RouteBuildEvent. In those cases, it is more convenient to manually define the typed data type for your parameter in the route options. Manual parameter type definitions always overrule the automatic discovery obviously meaning that the
ParamConverterManager
(or whatever we call the thing that subscribes to the RouteBuildEvent to iterate over each parameter and request type detection from the services that have been registered with it) would simply skip those arguments.I am very optimistic that this describes a very nice, reliable and developer-friendly solution to our problem. Thoughts?
I would like to bump this issue to critical if we can agree that this is the way to go. I am also going to push Sam and Kris to comment on this once again to express how important this is for what they are building.
Comment #52.0
fubhy CreditAttribution: fubhy commentedChanging headers.
Comment #53
fubhy CreditAttribution: fubhy commentedI updated the issue summary with the last post.
Comment #54
effulgentsia CreditAttribution: effulgentsia commented+1 to everything in #52, including the priority change. I especially like the idea of applying reflection at route build time and writing out the definition into the route options, rather than attempting any sort of reflection at request resolution time.
1. Is getting this done in a week feasible?
2. Is #1943846: Improve ParamConverterManager and developer experience for ParamConverters a useful interim step?
3. If the answer to #1 is no, and the answer to #2 is yes, then what is the urgency of getting this done before API freeze? In other words, what APIs would you foresee this breaking, once the other issue is done? Or is the urgency because of other core work blocked on this?
Ok, so pondering on my above questions, since the only upcasting we currently have in HEAD is entities, and since those already are reflectable and introspectable as you describe, are you saying we should skip the required explicit type declaration proposed in #1943846: Improve ParamConverterManager and developer experience for ParamConverters, and implement something directly here that maintains the ability of most/all current routes in HEAD to not require any changes to the routing.yml files?
Comment #55
effulgentsia CreditAttribution: effulgentsia commentedFor the record, I have always been on board with TypedData. Sorry if I wasn't clear about that earlier in this issue. I was initially against required explicit type declaration if it were in any way possible to support some kind of smart automagic. I came around to the need for it in #21, but am thrilled that you came up with a reflection-based idea for how to make automagic work robustly. If explicit type declaration is required for every route parameter (that requires upcasting) of every route, then I was and still am against requiring the long syntax in #4, and we don't yet have the ability to shorten it to #5 (though we hopefully will soon). However, if we can reflect and autogenerate the definitions for 90+% of our route parameters, then I'm ok with requiring the longer declarations for the edge cases, especially since it's likely we'll be able to support the shorter syntax eventually.
Anyway, thanks for all the great thinking that led you to #52 and for writing it up so well.
Comment #56
fubhy CreditAttribution: fubhy commentedOh, I am sorry! That meant to read "Now that effulgentsia seems to be on board with the typed data upcasting" (as in, being +1 on this issue) :).
Now that you mention it (especially #3) I believe you are right that (while changing APIs dramatically) that won't be noticed by most routes as they continue to get automatic type-detection.
Depending on what we decide #1943846: Improve ParamConverterManager and developer experience for ParamConverters might still be useful. I am not yet sure what we would end up with though. Do we even need a ParamConverterManager layer once we have this? As in, do we support other concepts for upcasting on that level? I don't see the need, to be honest, especially since that will always be possible with a custom RouteBuildEvent subscriber anyways. And I don't see a case where that would be needed. Simply expose your types as typed data. I don't see an argument for not doing that (and hence requiring a custom parameter converter). I guess we should just go with a TypedDataConverter (RouteEnhancerr) and then have a ParameterTypeDetectionManager (RouteBuildEvent subscriber) with which we register the typed data type detector services with. So instead of having a ParamConverterManager and multiple ParamConverters we just have a single TypedDataConverter (direct RouteEnhancer without the manager layer) and a RouteBuildEvent subscriber that iterates over the registered type detection services and writes the type definitions that those return (if they return one) into the route options. Then, on run-time / request-time the TypedDataConverter kicks in and iterates over the parameters described in the route options and upcasts accordingly through the typed data manager.
So, the architecture is rather straight forward. We just need some code for the reflection type detection services. It should not require that much code to be honest. The harder parts are the reflection type detection services. But that's just a few lines of code too.
Regarding #1: I will be in Dublin next week (Wednesday til Monday). And even though I basically signed myself up for some Drupal\Core\Extension cleanup (#2024083: [META] Improve the extension system (modules, profiles, themes)) I was going to dedicate at least one day to this. So I hope that we can fix this in one week indeed.
Comment #57
effulgentsia CreditAttribution: effulgentsia commented1. Will we make
tempstore:COLLECTION
a TypedData type?2. Do we want to support contrib modules that integrate routes from non-Drupal projects, where they want to upcast to something that uses some other typing system entirely, instead of Drupal's TypedData?
Comment #58
fubhy CreditAttribution: fubhy commentedad #1: Not sure if that should be a generic type. To me that sounds more like e.g. ViewUI should expose this ViewUI decorator that they are using as a typed data type implementing LoadableInterface and then, in the ViewUI::load() static method do what ViewUIConverter currently does.
ad #1: Hmm, can you give an example?
Comment #58.0
fubhy CreditAttribution: fubhy commentedUpdated issue summary.
Comment #59
effulgentsia CreditAttribution: effulgentsia commentedRe 58.1: ok, makes sense.
Re 58.2: Unfortunately, I'm not really very up to speed on the entity/document architectures of other projects. But for example, http://symfony.com/doc/master/cmf/bundles/routing.html#using-the-phpcr-o... mentions using the Symfony CMF router (same as Drupal's router) for routes that work with PHPCR-ODM. While I'm not familiar with the details of that, from what I do understand, Doctrine ODM mapped objects have some conceptual similarities to, but very different implementations than, Drupal's entities and TypedData objects. It seems possible that someone might want to write a Drupal module that provides some integration with something based on PHPCR-ODM. I don't know enough about such a use case to speculate on whether requiring that integration to involve a TypedData bridge makes any sense or not.
Comment #60
fubhy CreditAttribution: fubhy commentedad #58.2 Then that would fall under the responsibility of such a module I'd say. That's not something our TypedDataConverter would care about I guess. That rather looks like a custom RouteEnhancer then. But if possible, I'd vote for keeping the number of layers for this as minimal as possible so I'd opt for not having an intermedia ParamConverter layer if we decide to go for this proposal:
That's about it. And that's very simple actually. I'd prefer to keep it that simple instead of having another layer underneat (ParamConverterManager).
I am going to start writing this patch now.
Comment #61
fubhy CreditAttribution: fubhy commentedOkay, let's keep the parameter converter for now :). It probably actually makes the code in this one cleaner and easier. So let's still work on #1943846: Improve ParamConverterManager and developer experience for ParamConverters. I am going to incorporate the current version of the patch from #1943846: Improve ParamConverterManager and developer experience for ParamConverters into what I am working on now so we get a clean diff. Plus @Crell is not going to scream when he sees this :)
Comment #62
fubhy CreditAttribution: fubhy commentedC'mon d.o ... What's your problem with the stupid tags
Comment #63
fubhy CreditAttribution: fubhy commentedFor god's sake...
Comment #64
Crell CreditAttribution: Crell commentedOK, since much of typed data is still a mystery to me (it still feels like over-engineered over-extending), given that I the following code in my module that looks like the following:
What is the *minimum* amount of work I have to do in order to make this controller Just Work(tm):
And if the answer includes "Thingie extends SomeDrupalClass", then that's a won't fix.
Comment #65
fubhy CreditAttribution: fubhy commentedOnce #1867856: Use annotation discovery for data type plugins lands you'd have to put a @DataType annotation on your class and, if it needs to be loadable, implement LoadableInterface for example. The TypedDataInterface is getting removed in #2002138: Use an adapter for supporting typed data on ContentEntities.
Comment #66
fubhy CreditAttribution: fubhy commented@Crell asked for a code-example so here it is... If I understand @berdir correctly then this is all you have to do once the issues that were agreed upon and opened during and after Portland are resolved (which they are already working on).
Disclaimer: This assumes that we get annotations for typed data types @see #1867856: Use annotation discovery for data type plugins.
Comment #67
effulgentsia CreditAttribution: effulgentsia commentedWell, given the intro paragraph of #60, the answer to #64 is you have a choice:
- Either expose your Thingie as a Drupal @DataType, ala #66.
- Or, implement your own RouteEnhancer.
And if I read #61 right, then maybe "implement your own ParamConverter" will remain as yet another option.
Comment #68
fubhy CreditAttribution: fubhy commentedAbsolutely... So there is a multitude of options. TypedData being the one which would give you integration with page manager, etc. at the same time.
Comment #69
EclipseGc CreditAttribution: EclipseGc commentedSo, after a short skype with fubhy, I am VERY +1 to this. We need to be reflecting from _content/_form as well, but the basic approach is good as far as I'm concerned. This will give us access to the definitions of objects w/o having to have real objects, it will give us magic integration if people follow best practices, and it allows us to manually notate it in a worst case scenario in contrib or whatever and still get the support we want. Scotch REALLY needs this, and I can't express how much I appreciate fubhy applying mental effort to solving this for everyone (DX and Scotch).
TL:DR +10000000000
Eclipse
Comment #70
Gaelan CreditAttribution: Gaelan commentedRerolled.
Comment #71
disasm CreditAttribution: disasm commentedComment #73
fubhy CreditAttribution: fubhy commentedPlease don't waste your time on re-roll atm. I am geoing to rework this entirely ;).
Comment #74
fubhy CreditAttribution: fubhy commentedOver at #2002102: Move TypedData primitive types to interfaces I just suggested that we ensure typed data type interface uniqueness and base discovery on that as well so we can easily map typehints to the right typed data type without any chance of that failing due to the interface being ambigious in typed-data land. This would dramatically decrease the complexity for this issue and ensure that it is absolutely failure-proof.
In any case, the typed data refactoring based on a more interface-based approach is moving forward so I am very optimistic of getting a patch for this issue done this week.
Comment #75
Crell CreditAttribution: Crell commentedTo be honest, I don't fully understand how what fubhy is proposing solves EclipseGc's issue. But I guess I don't need to. I am not a huge fan of it, but I can live with what he has proposed. The caveat is that we do need to keep the ParamConverter layer below RouteEnhancer, because we need ParamConverter to be divorced from routing per se so that we can use it elsewhere.
Addendum: I should still be able to add my own ParamConverter that ignores TypedData and LoadableInterface and so on entirely; if that makes it incompatible with Scotch, that's my problem, but I should be able to roll my own directly if I want to.
Comment #76
sdboyer CreditAttribution: sdboyer commentedadding my princess tag
Comment #77
fubhy CreditAttribution: fubhy commentedI set up a sandbox with the initial implementation. There are still dependencies that need to be resolved:
However, let me explain what I did so far:
On route compile time
There is a
TypedDataSubscriber
which subscribes aTypedData\ResolverManager
to therouting.route_alter
. The ResolverManager gets different types of resolver services registered with it (so far we got aTypedData\ReflectionResolver
and aTypedData\EntityResolver
solver) which each can have their own strategy for interpreting the typed data type of route arguments. Each basically returns a keyed array of typed data definitions representing the definitions of the arguments that it was able to detect. TheTypedData\ResolverManager
then writes them into the route options.The
EntityConverter
is very similiar toEntityRouteEnhancer
in that it reads out the_entity_form
or_entity_list
values from$route->getDefaults()
and then extracts the entity types from then. It then generates the typed data definitions for those.The
ReflectionConverter
reads from the type hints of each callable that it can find in$route->getDefaults()
and tries to map them to typed data types.On request time
There is a
ParamConverter\TypedDataConverter
(old style, since I see #1943846: Improve ParamConverterManager and developer experience for ParamConverters as less critical at the moment which is why I just focused on this part for now) which reads out the generated typed data definitions and then converts them based on that by going through theTypedDataManager::load()
method.Especially the ReflectionResolver absolutely depends on the above list of issues to get resolved in order to function properly.
Comment #78
Crell CreditAttribution: Crell commentedI just realized one potential issue with reflection: That means loading all controllers into memory on dumping in order to reflect on them. That's... unpleasant for performance. That's why we're not using reflection for annotations now.
Comment #79
effulgentsia CreditAttribution: effulgentsia commentedUgh. Yes, performance in terms of CPU time isn't as much a concern since it's only on router rebuild, but it could lead to out of memory errors and a resulting WSOD, which is a deal killer. I wonder if any of the Doctrine code we already have in core can help us reflect typehints from tokenized source, similar to how we currently handle the plugin annotations. If not, there's https://github.com/Andrewsville/PHP-Token-Reflection, and perhaps other libraries to look into.
Comment #80
effulgentsia CreditAttribution: effulgentsia commentedAnother option is instead of trying to do it all during router rebuild, we progressively warm up the router. After a new rebuild, there's no auto-discovered type info written out. Each time we route, we check if the type info is there; if not, we reflect just the controller we need to concern ourselves with, and then update the router with that info to speed up that route for future requests.
Comment #81
fubhy CreditAttribution: fubhy commentedReflection is just a fall-back at the moment and in most cases we don't even need that at all as most things go through the entity system (_entity_form/_entity_list and in the future probably also _entity_view). We do not have the proper _controllers on the route for many routes at the point where we do the reflection which causes it to be skipped in most cases anyways. So I don't really see an issue with performance or WSOD at the moment.
Comment #82
dawehnerWell, this could be measured by loading all current controllers/forms from forminterface and then try a reflection on it.
There are currently ~100 of them, but this number will potentially be 1000 or more on a real site.
Comment #83
fubhy CreditAttribution: fubhy commentedClosing as a duplicate of #1943846: Improve ParamConverterManager and developer experience for ParamConverters. We will solve the typed data converter as part of that patch.
Comment #84
fubhy CreditAttribution: fubhy commentedOkay, re-opening due to comment #106 at #1943846: Improve ParamConverterManager and developer experience for ParamConverters.
Just uploading the bare typed data converter patch. Nothing else in there. Needs tests.
Comment #84.0
fubhy CreditAttribution: fubhy commentedUpdated issue summary.
Comment #85
jibranIs this issue still valid?
Comment #86
Crell CreditAttribution: Crell commentedIt is. fubhy's been swamped with work, though. If you want to pick it up, that would be great. :-)
Comment #87
Berdir@fubhy said in #1983100: Provide a LoadableInterface for Typed Data objects that that interface is no longer needed, so I'm not sure what this means for this issue?
Comment #88
fubhy CreditAttribution: fubhy commentedWell... The whole thing with typed data and upcasting is not entirely off the table. What's off the table though is using the Typed Data API to actually *load* the typed data object. If we even wanted to do that we would have to have those IdentifiableInterface / LoadableInterface and I think we agreed that that is not how we want Typed Data to work (and we don't want to make it any more complex either). So, yes... That is off the table at this point.
HOWEVER! While this means we don't want to upcast things through the Typed Data API it does not mean that we should ignore Typed Data in routing. Knowing the typed data type of a parameter is highly valuable information and will definitely come in handy in contrib too (not only for @sdboyer's and @EclipseGc's stuff ;-)). That still needs to be discussed. As Larry pointed out though I am absolutely drowning in work-work and studies for months now... There is a DrupalCamp in Vienna between 21st-24th November for which I freed up some time. There will be a sprint too so I will have time to look at this again.
Right now the best possible solution I can think of is to allow ParamConverters to return Typed Data information about a parameter when asked for it. Something along those lines...
Comment #89
sun@EclipseGc & @sdboyer: Would be great to get an update from you with regard to this issue; specifically whether the TypedData information (sans upcasting) is useful, or much rather, whether the lack of it presents a hard roadblock for your work.
Comment #90
catchEither way this doesn't look critical to me. Splitting the difference at major.
Comment #91
Crell CreditAttribution: Crell commentedComment #92
EclipseGc CreditAttribution: EclipseGc commented@sun, et. al,
It's imperative that we have some sort of meta data layer on the route definition to tell use what sort of data we'll be getting. I'll attempt to illustrate:
/node/{node}
This tells us nothing, and even if you have some upcasting methodology that says "Oh we have this 'node' thing, check to see if there's an entity of that type... oh there is, great load it." that's not actually useful because if we ever want to administratively override /node/{node} then we absolutely need a definition of what {node} is, instead of figuring that out at run time for the page itself. This is how page_manager in D7 works, but since D7's menu system uses magic loading by *_load() methods, and provides no metadata, merlin invented a custom plugin type to define that metadata and route the requests to that menu item through this plugin which asks page manager if there are overrides for the specific item in question or whether it should farm it out to Drupal's normal callback. This means even though page_manager is smart enough to deal with almost every entity type ever invented (except for rules and search api indexes and servers) it can't hijack the pages those entities define as their view route. Yes we could have written more code to do that, but it's a stop gap measure and having the routing system in D8 just support this sort of behavior via introspection of the menu route, or required manual definition by a developer is a FAR superior methodology as it opens the door to overriding ANY page created anywhere... ever.
In short, yes it's super important that the route, without being executed, be able to tell me that {node} is a node entity and that {string} is a string.
Eclipse
Comment #93
EclipseGc CreditAttribution: EclipseGc commentedAlso, I apologize for taking so long to respond here. Life, work, stuff... you know.
Eclipse
Comment #94
dawehnerJust to inform other people in the issue, this kind of metadata is already available:
so the issue is more about avoiding the unwanted upcasting problem.
Comment #95
kgoel CreditAttribution: kgoel commentedComment #96
kgoel CreditAttribution: kgoel commentedI have looked into the sandbox project and followed this commit - http://drupalcode.org/sandbox/sumsi/2031313.git/commit/d67edc0. Its my initial approach to pull out the only part which takes care of type data upcasting.
Comment #98
kgoel CreditAttribution: kgoel commentedI pulled code from wrong branch, spoke with dawehner and I will be working on correct branch to submit new patch.
Comment #99
kgoel CreditAttribution: kgoel commentedworking on a new patch and will submit tomorrow
Comment #100
kgoel CreditAttribution: kgoel commentedfollowed fubhy's sandbox - https://drupal.org/sandbox/sumsi/2031313. This patch may not be complete since I nitpicked manually and there were some files which I couldn't find in core but its just a initial approach just to get me started.
Comment #101
kgoel CreditAttribution: kgoel commentedComment #103
kgoel CreditAttribution: kgoel commentedComment #105
kgoel CreditAttribution: kgoel commentedComment #107
kgoel CreditAttribution: kgoel commentedComment #109
kgoel CreditAttribution: kgoel commentedMaybe someone else can take a look at fubhy's sandbox. I have manually checked all the commits and found it very confusing.
Comment #110
dawehnerAfter looking at the git repo of fubhy this is basically the functionality we need without all the crazy abstractions.
Comment #112
dawehner.
Comment #113
YesCT CreditAttribution: YesCT commentedtag typo fix
Comment #114
dawehnerWorked a bit on unit, let's see whether I covered all usecases with the tests.
Comment #117
dawehnerGiven that we use that kind of reflection now we initialize all dependencies of all controllers, which requires basically a fullstrapped drupal instead of a DrupalUnitTestCase
Comment #119
dawehnerThis could fix most of the failures.
Comment #120
damiankloip CreditAttribution: damiankloip commentedTHE entity manager.
Needs @param
This could use is_subclass_of() now I think, it works with interfaces too. I don;t really like call_implements...
Yeah, people don't really care, but you know we need one :)
Docblocks please.
Maybe some simple docblocks
Comment #122
dawehnerI don't see why. It would not add a single value and just get it into core allows you to improve things in the longrun.
One thing you should learn from the fixed failures: Never execute code in your constructor. Just assign your dependencies and continue later.
Comment #124
dawehnerSome less failures.
Comment #126
dawehner.
Comment #128
Crell CreditAttribution: Crell commentedPer dawehner's request so it's on his radar for DevDays...
Comment #129
dawehnerLet's see, this should fix quite a couple of the failures.
Comment #131
dawehnerMade a bit of progress on the REST side of it.
Comment #133
dawehner.
Comment #135
dawehnerMaybe this helps already.
Comment #137
dawehnerThis fixes at least a couple of the failures.
Comment #139
dawehnerLet's see ...
Comment #141
dawehnerThere we go.
Comment #142
tim.plunkettComment #143
dawehnerRerolled.
Comment #144
dawehnerupdating the title.
Comment #145
dawehnerComment #146
cosmicdreams CreditAttribution: cosmicdreams commentedgit merge cruft.
Comment #147
dawehnerGood catch.
A couple of additional cleanups got added/unit test coverage got improved.
Comment #148
tim.plunkettIs this all there is? Does this run after things like _entity_form/_entity_list/_entity_view?
What is $cat?
These commas are optional
This should have
, 2
in the explode to prevent future weirdnessEek, can we combine them on one line? Bitwise always looks like assign-by-reference to me, and this is short enough to stay on one line
Nice!
Why this change?
Unless I'm misreading this, there are no actual changes here.
Hm, a comment would help here
Why are these needed here, but not for other entities like node?
Put this back please :)
Comment #149
dawehnerThank you for your review, actually someone who not just talks shit all day long.
There is indeed, but there we don't need reflection but can leverage entity information, see setParametersFromEntityInformation()
It is just yet another variable name, changed that.
Good point.
Because it fixes the test
This was probably triggered by some merge problem.
Views don't actually use the entity upcasted but converts to viewsUI objects, this conflicts with the reflection based approach.
Comment #151
dawehnermeh
Comment #153
Crell CreditAttribution: Crell commentedIsn't the @todo the whole point? Rely on the type hint exclusively, not the variable name?
I am very confused as to why this is necessary.
Splitting this out to a method makes total sense, but we should name it createBaseRoute() or similar instead. getBaseRoute() suggests that we'd get back the same route object on each call, which is not the case.
Comment #154
dawehnerNope, the whole point is to no upcast unless you really meant it.
The reason are tests. Previously route rebuilding might have been triggered but no theme registry needed to be injected, but now they have to.
Comment #155
Crell CreditAttribution: Crell commentedHm. Them I'm very confused, because the last time I looked we wanted to NOT be relying on the placeholder name to avoid name collisions with newly defined entities and use the method signature only, as that's safer. If this is just a step toward that, fine, but I'm pretty sure that was the intent.
Comment #156
dawehnerWell my intension for the issue is to fix an actual problem (accident upcasting), which is fixed by this issue. We rely on the reflection information anyway.
Comment #157
tstoecklerI spent some quality time with this one now and I must say:
Awesome patch! I really like it. Especially the unit test is beautiful to read, nice work!
I have a bunch of remarks, most of them are very minor and you can also ignore them if you disagree.
I just understood the last few comments after reading the patch and per my comment below, I'm pretty sure it's currently not possible to resolve that @todo in any (sane) way.
Super minor, but the one-line description is a little bit ambiguous. At least "parameter" -> "route parameter" would be helpful I think.
The longer description could use some elaboration as well, in my opinion. Maybe something like "If controllers of routes with route parameters, type-hint the parameters with an interface, upcasting is done automatically." ?
Minor: "an" -> "a". Maybe also "out of" -> "using"
With "pattern" you mean stuff like _content,_form, _entity_form? I think that could be clarified.
Perhaps "... if it was possible to instantiate it; NULL otherwise."
Missing () after $form_arg.
Perhaps "The route object to populate with upcasting information."
"...; FALSE otherwise.
I know this is more a matter of personal preference, but would you object to
list($instance, $method) = $controller; new \ReflectionMethod($instance, $method);
?I had to read this a couple times to realize that the foreach is over $parameters and not $parameter_definitions. I know this is super-nitpicky, but could we re-organize those variable initializations a bit?
Yes, but for that we really need Interface == $entity_type_id. Right now NodeInterface is type-hinted all over the place (which is good!) but the entity system itself, doesn't know it. It knows 'node' and it knows \Drupal\node\Entity\Node, which happens to implement NodeInterface. I can swap out the class and then make it implement a different interface. If that interface extends NodeInterface then all is good. But there is no programmatic way to figure out the interface for an entity type.
tl:dr; ++, follow-up
Aloso again very minor and arguable, but I would find the code flow more legible if the
($reflection_class = $parameter->getClass())
would be moved into the if().$parameter_definitions += array($parameter_name => array());
?Suggestion:
"Sets the upcasting information using the _entity_* route defaults.
Supported are the '_entity_view', '_entity_list' and '_entity_form' route defaults."
Minor, but the 2 can be omitted (or kept :-), doesn't matter, just saying)
Perhaps a comment saying "The parameter types are of the form 'entity:$entity_type'." ?
So if I understand the code correctly, the
setParametersFromEntityInformation()
is only needed for _entity_*-style controllers which do not utilize type-hinting. I.e. for those entity view/list/form builders that do utilize type-hinting the call is not needed, right?If that is true, would it make sense to simply remove support for that, and really force people to use type-hinting even for _entity_*-style controllers?
I thought about this because if I'm not mistaken the current code unconditionally sets an upcasting parameter named after the entity type for _entity_*-style controllers, while e.g. for _entity_list controllers this will generally not be needed. Now that's certainly not a big problem, but it's also not really nice.
Yay, ++ for typed catching.
I think the comment could be improved a bit, though. Maybe "Some tests define routes with parameters for non-existant entity types." (Also note that unless I'm mistaken existat is spelled with an a in English).
It seems this is only being called once in this class (and not within a loop), so I'm curious why you introduced a separate method. Is
setRouteOptions()
being called multiple times per request?Edit: Just saw the code below.
Still wanted to note that we do still call EntityManager::getDefinition() in setParametersFromEntityInformation(), i.e. where we don't use the cached entity type list.
This one was mistaken, it should still be @param.
In the EntityResolverManager, you used +=. Is there a specific reason you're using just = here?
Just a thought, feel free to object: Would it make sense to add a processRoute(Route $route) method to ConfigNamesMapper, that is called in each of the get*Route() methods, but is a no-op in ConfigNamesMapper. That way we would not have to override all those methods. The pattern is the same that is used for populateFromRequest() in that class, if I'm not mistaken.
I know this is just copied around, but can we rename this to $mapper_id or something while we are at it. This way it is a little bit confusing.
AFAIK @throws comes after @return, but not really important either way.
I like this change, it makes a lot of sense. It turns the class into a true list builder which itself is stateless.
Wow, that must have been fun to debug...
Can I haz @group Drupal and @group Entity :-)?
"Injection" -> "injection" (lowercase i)
Here and below as well, I think instead of "the setRouteOptions method" you can just say "setRouteOptions()".
Maybe instead instead of "special route/controller", "route parameters" is sufficient?
Doing "the setRouteOptions method" -> "setRouteOptions()" would allow to also do "arg" -> "argument" here.
Also "non entity" -> "non-entity".
Would "controller" -> "default" be more correct? I think controller could be confused with _controller here.
Comment #158
dawehner#2266605: Add the entity specific interfaces to the entity type
Tim will force you to specify all the times.
I am pretty sure that there are places all over the place where it is kind of automatic assumed that the upcasting happends. Just have a look at the node_help() in the interdiff. Given that people
don't really like explicit code I think this would block the patch, if the support would be removed.
Mh, both phpstorm and dict.cc disagree with you: http://www.dict.cc/?s=existant
Good catch.
Not at all, you brain!
Yeah, this looks way simpler.
Oh totally
Sure
Sure
Comment #160
dawehnerMeh
Comment #162
dawehnerAnd back to green.
Comment #163
dawehnerRerolled.
Comment #164
dawehnerReroll
Comment #166
tstoecklerThis needed a re-roll due to #2178795: Allow DiscoveryInterface::getDefinition() to throw an exception for an invalid plugin.
Comment #168
tstoecklerI can't really reproduce this fail. Both with native PHPUnit and with run-tests.sh
EntityUnitTest
passes for me.Comment #169
martin107 CreditAttribution: martin107 commentedPulling this quote from the test results
Output: [PHP Fatal error: Cannot use Drupal\entity_test\Entity\EntityTest as EntityTest because the name is already in use in /var/lib/drupaltestbot/sites/default/files/checkout/core/tests/Drupal/Tests/Core/Entity/EntityUnitTest.php on line 13
PS testbot reports fail with php version 5.4 ... tests all green locally for me 5.4.10
PPS
See below now that tstoeckler is done stabbing in the dark ... you can't see this, but this is me "Dancing in the dark" :-)
Comment #170
tstoecklerStabbing in the dark...
Comment #171
tstoecklerOk, now with a less silly name.
Btw, I have NFI why that caused the testbot to choke on this.
Comment #173
tstoecklerI think the testbot is just playing with me at this point.
Interdiff is still relative to #166. (Previous one was as well, even though it was incorrectly named.)
Comment #175
tstoeckler173: 1906810-173-upcast-type-hint.patch queued for re-testing.
Comment #177
YesCT CreditAttribution: YesCT commented"SystemListingCompatibleTest test executed." found Other InstallationProfileModuleTestsTest.php 63
fail was from #2239969-22: Session of (UI) test runner leaks into web tests which was reverted.
sending for a retest.
Comment #178
YesCT CreditAttribution: YesCT commented173: 1906810-173-upcast-type-hint.patch queued for re-testing.
Comment #179
dawehnerOh that seemed to be a "fun" one ... Rock on tstoeckler!
Comment #180
tstoecklerYeah. RTBC++. Even though I wasted a lot of time on this :-) my actual code contributions to this issue are minimal (an almost trivial merge and the interdiff in #173) so if @dawehner agrees with the interdiff, this is ready to go!
Comment #181
catchI'm a bit confused why none of the core entity modules are touched by the patch i.e. NodeViewController still type hints with EntityInterface with the patch applied.
Needs an issue summary update and/or follow-ups being created?
Comment #182
fubhy CreditAttribution: fubhy commentedAt the moment the patch only checks if there is an entity interface at all on the parameter === definitely entity that wants to be upcasted. The type of the entity is then determined either by the route definition (e.g. special cases like "_entity_view" / "_entity_form", etc.) or by reading the name of the parameter (e.g. $node). That's not perfect, but sufficient for now and eliminates the risk of clashes with random string parameters that coincidentally have the same name as an entity type (e.g. a custom contrib route that has a $node parameter... If it's not type hinted, it doesn't get upcasted... If, however, it IS typehinted with an EntityInterface / More specific interface we can assume that it's desired to have it upcasted.)
Comment #183
dawehner@catch
This would be certainly doable properly once #2028097: Map data types to interfaces to make typed data discoverable is doable/done. Currently there is no simple/relyable way to get the interface from the entity type itself.
Comment #184
alessiofx CreditAttribution: alessiofx commented173: 1906810-173-upcast-type-hint.patch queued for re-testing.
Comment #185
xjmReroll for #2247991: [May 27] Move all module code from …/lib/Drupal/… to …/src/… for PSR-4.
Comment #186
catchSorry still a few things I don't really get in the patch. Overall it doesn't really seem to be reducing the amount of work needed anywhere, maybe I just had wrong expectations of what the patch was going to do.
That issue's in, so should we not just use the clas resolver?
Could we add the issue to the @todo.
When would this happen?
Why do they do that? Shouldn't we throw the exception rather than ignoring it to avoid test failures?
Comment #187
damiankloip CreditAttribution: damiankloip commented- Injected the class resolver into the EntityResolverManager.
- Removed the first @todo - I think we should discuss whether we even want to allow this in a follow up. So on that basic, let's not have a todo and just change it if we decide to do it?
RE #4, Haven't changed yet but wouldn't this need two parameters the same for this to happen?
Comment #189
tstoecklerSeems this is no longer needed and is also not going to work.
We should still retain the
array($controller, 'buildForm')
no?Comment #190
damiankloip CreditAttribution: damiankloip commentedYes, totally right. Sorry - rushing never works out! :)
Comment #192
damiankloip CreditAttribution: damiankloip commentedShould be green again.
RE #186; I think the purpose of this patch is not really to reduce the code needed, but to make sure things do not get accidentally upcasted. Basically make this mechanism more robust.
So that really leaves #186.3 and #186.4.
Comment #193
damiankloip CreditAttribution: damiankloip commentedComment #194
tstoecklerInterdiff looks good. Not RTB Cing as im not sure @catch' s concerns have been adressed fully. (could be, just not sure.)
Comment #195
catchIt'd be good to figure out #186.3 and #186.4
If we're just making this more robust, patch overall seems fine - just think we shouldn't suppress errors while doing that.
Comment #196
dawehnerRe 183.3:
This was fixed in #1906810-124: Require type hints for automatic entity upcasting. The example when this happens is pretty good documented in the interdiff: https://drupal.org/files/issues/interdiff_2560.txt
You don't want to check the upcasting bits in case the developer has manually specific the upcasting already. For example the views UI sets it manually but still define
some kind of interface, but on top views has its own special upcasting code.
Re 186.4:
This was added in #1906810-119: Require type hints for automatic entity upcasting
One classical example is that you have a DUTB test, and menu_link not enabled. The system manager has the following piece of code:
so we have to protect against that.
Comment #197
catchShouldn't the dutb test enable menu then?
Comment #198
damiankloip CreditAttribution: damiankloip commentedOk, let's see how much stuff breaks then.
Comment #200
damiankloip CreditAttribution: damiankloip commentedSo we would just need to make these changes. catch?
Comment #202
catch200: 1906810-200.patch queued for re-testing.
Comment #204
damiankloip CreditAttribution: damiankloip commentedThat's a legit failure. We are just used to always seeing locale with random ones :)
I think that test is currently passing but the actual response is not right. I might dig a bit later on.
Comment #205
damiankloip CreditAttribution: damiankloip commentedSo what's ironic and pretty funny is that the failing test is LocaleLocaleLookupTest, which states (also with a testCircularDependency test method):
The assertion passes falsely normally, with this:
Symfony\Component\DependencyInjection\Exception\ServiceCircularReferenceException: Circular reference detected for service "current_user", path: "exception_listener -> exception_controller -> url_generator -> path_processor_manager -> path_processor_language -> current_user -> authentication -> authentication.early_translation_test". in Symfony\Component\DependencyInjection\Container->get() (line 282 of /Users/damianlee/Sites/D8/core/vendor/symfony/dependency-injection/Symfony/Component/DependencyInjection/Container.php).
So not sure when this was added, but it is a completely useless test in its current form :)
With this patch we now just get a 403 on that page. I think because after the module form is submitted we end up with an anon user again. Haven't dug into it too much yet though.
EDIT: Looks like it was added in #2244447: Translation of low-level info/annotations leads to circular dependencies.
Comment #206
damiankloip CreditAttribution: damiankloip commentedJust opened #2284111: Exceptions caught and printed from index.php should not return a 200 status code
Comment #207
andypostThis test was added to test language manager and translation service dependencies, not sure that useless
Comment #208
damiankloip CreditAttribution: damiankloip commentedRead #205 - I know what the test was added for. I am saying it is currently useless as it does not catch a circular dependency. Which is precisely what it exists for. See the issue I opened in #206 also.
Comment #209
damiankloip CreditAttribution: damiankloip commentedJust a reroll.
Comment #211
damiankloip CreditAttribution: damiankloip commentedOnce we get #2284111: Exceptions caught and printed from index.php should not return a 200 status code in, this failure will be fixed.
Comment #212
damiankloip CreditAttribution: damiankloip commentedComment #213
damiankloip CreditAttribution: damiankloip commented209: 1906810-209.patch queued for re-testing.
Comment #214
dawehnerIt is green.
Comment #215
catchCommitted/pushed to 8.x, thanks!
Comment #217
nod_This commit added stuff under
core/sites/files/
Was that supposed to happen?
Comment #218
catchNope, those sneaked into the patch. Reverted, fixed the patch locally, recommitted/pushed.
Comment #220
damiankloip CreditAttribution: damiankloip commentedSorry! Not sure how they snuck in.
Comment #222
undertext CreditAttribution: undertext commentedSorry. Commited patch breaks possibility to pass function name to '_content' property of the route.
For example :
From here : This could be a PHP 4 style function, but best practice in Drupal 8 is to use a controller class.
It is all because of EntityResolverManager::setRouteOptions method. If u look at it you will see that
$this->getController
call should return array with controller instance and method name as its values. (docblock comment)
Next. Looking at the getController method.
And here is it. getControllerFromDefinition method can return just callback function name instead of array.
This scenario is not covered in implementation and u will get en error : "Argument 1 passed to Drupal\Core\Entity\EntityResolverManager::setParametersFromReflection() must be of the type array, string given, called in /home/yarik/www/drupal8/dev/core/lib/Drupal/Core/Entity/EntityResolverManager.php on line 188"
Comment #223
dawehner@undertext
Please open up a follow up and link it from here. This is the way how to stay sane. I guess it is not that problematic to
Comment #224
undertext CreditAttribution: undertext commentedOk. Here is it #2321393: Unable to pass function name to '_content' property of the route.
Sorry for reopening this. Just lack of my knowledge about issues workflow.