Currently when the RouteBuilder is compiling routes for the routing table it iterates over all the access checkers for every route. This seems unnecessary. It would make more sense to only iterate over the access checkers that apply for that route - this would reduce the amount of iterations and additional function calls by ALOT. This overhead could make a big difference on larger sites in particular (especially if we have to register access checkers for all the things - #2011424: Allow access methods on controllers to be used in routes).
Me and @dawehner discussed this briefly; We could have access checkers declare which requirement keys they apply to. We could then build a map of requirement keys => access checker ID and then just iterate over the requirements for each route and add the checkers that way.
Here is a *very* rough initial patch - Tests will likely fail hard, I've added an appliesTo() method to all the checkers, this is then used to compile a requirement map on the checker. So I have also changed the applies method on the AccessManager to reflect this.
Comment | File | Size | Author |
---|---|---|---|
#55 | 2012382-results.png | 283.65 KB | damiankloip |
#47 | 2012382-47.patch | 37.39 KB | damiankloip |
#47 | interdiff-2012382-47.txt | 1.15 KB | damiankloip |
#44 | 2012382-44.patch | 37.72 KB | damiankloip |
#44 | interdiff-2012382-44.txt | 2.06 KB | damiankloip |
Comments
Comment #1
dawehnerNice, that's exactly what I thought would help here.
Couldn't we even get rid of the applies method, or do we need this flexibility? One question is: how do we deal with possible multiple access checkers which applies to a single requirements key.
I don't see a reason why this should actually be a static method, as we already have the object available.
This change feels really wrong.
Comment #2
damiankloip CreditAttribution: damiankloip commentedOk, here's a new patch.
- Made appliesTo return an array
- Store requirementMap as an array of service_ids, as feasibly two access checkers could apply with the same keys.
- Moved the _method logic in Drupal\rest\Access\CSRFAccessCheck into the access() method
- Reverted the change to RouteBuilder - this is a complete accident
- appliesTo() is not static
- Removed applies() method from everywhere
Would be keen to get some validation on the approach in general, then maybe I can add a few more unit tests.
Comment #3
BerdirHm, that limits access checkers to apply to statically defined routes. It for example doesn't allow me to write more intelligent/dynamic checks, especially when adding checks to routes that I don't define.
Which, I think, was the main reason why we went with applies() before.
Couldn't we combine it? Have appliesTo(), allow it to be an empty array (which would match everything, not nothing) and then call applies() when appliesTo() matches, with a default implementation that says return TRUE. Possibly with different names.
Comment #5
dawehnerSo basically that's what damian proposed in the first version of the patch :) I see that this still improves performance, that is great.
Comment #6
damiankloip CreditAttribution: damiankloip commentedSo something more like this?
Comment #8
damiankloip CreditAttribution: damiankloip commentedLet's try this and see how it gets on. I think there might be some issues with entity access.
I have just returned TRUE from applies() if we have a static requirement in appliesTo(), what do you think? it's pointless to check array_key_exists in that case. This only makes sense for routes that won't use the appliesTo() method. See Drupal\rest\Access\CSRFAccessCheck in the patch.
Comment #10
damiankloip CreditAttribution: damiankloip commentedOops.
Comment #11
dawehnerThis comment should explain why we need both applies and appliesTo. Maybe also mention that return something does lead to a not called apply() method().
Some docs missing :(
Can we have a one liner comment above, please?
maybe we could add a baseclass in a follow up?
{@inheritdoc}
This change seems wrong.
There could be also made some unit tests, for the new functionality.
Comment #12
damiankloip CreditAttribution: damiankloip commentedThanks for the review! I will make those changes and add some tests.
Yes, very wrong. That will result in quite a few failures I imagine.
Yep, good idea.
Comment #14
damiankloip CreditAttribution: damiankloip commentedComment #15
damiankloip CreditAttribution: damiankloip commentedComment #17
damiankloip CreditAttribution: damiankloip commentedThis should fix most of the problems. I am thinking we could also build a list of checkers that have empty appliesTo() methods? Then we wouldn't even have to iterate through all checkers like we are now...
Anyway, Let's see how this patch gets on - Then we can maybe make that change too.
Comment #19
damiankloip CreditAttribution: damiankloip commentedLet's try the dynamic requirements map too, see if we're in the same spot. Then it will just be tests (new/fixing).
Comment #20
dawehnerWe can still iterate over some of the internal details if needed, but for now this is fine.
Does someone know whether it is worth to test this internal behavior (could be used by using mocks).
{@inheritdoc}
Comment #22
damiankloip CreditAttribution: damiankloip commentedJust posting some changes here (after a long day travelling) to add a StaticAccessCheckInterface that uses appliesTo() instead of applies().
Not sure what we should exactly do with the interfaces at the moment, as I've currently just copied AccessCheckInterface but changed the applies() method. Should we just have one common interface they both inherit, like AccessInterface? Then the checkers can extend that add add their methods respectively?
This path will still fail quite a few tests, due to the applies() being used for them.
I also haven't changed the DX for this yet, I know we were dicussing having an 'authorization' options key instead of something. This could get tricky, if we just want to be able to provide a list of 'static' checkers. As they may still want to use the value from the requirements, like DefaultAccessCheck for example.
Comment #23
Crell CreditAttribution: Crell commentedBefore we go any further, let's check with higher authority.
Dries: This is a DX improvement. It is not critical, but it is useful. I'm pretty sure it can be done without breaking an existing API; just expanding it. It does not require any follow-ups, but we could likely simplify/refactor several one-off checkers as a result of this. It MAY involve a change to the route that uses those one-offs, but since they're one-offs by definition no one else is going to be using them so no already-ported contribs should be affected. Go/no go?
Comment #24
damiankloip CreditAttribution: damiankloip commentedYes, it would be good to get some clarification on this.
I think the bigger the site the more important this is, we already have a lot of access checkers in core, many of which are for one off purposes. When contrib gets involved this is going to get crazy.
I agree though, we could live without this if we had to. I think.. We have no examples of large sites, and we wont have for some time until sites are actually using D8, then it might be too late. Hopefully this is OK, as I think we are just extending, rather than changing. I hope to keep the current AccessCheckInterface intact. If any modules are using any of the current access checkers in their routes, this should still work without change.
Comment #25
BerdirThe only thing that would break is existing custom access checkers.
And if we really have to, we could even make that part optional, by making the method optional, but I'd prefer to not do that as many contrib module authors with custom access checkers will then forget to implement that.
Comment #26
damiankloip CreditAttribution: damiankloip commentedI thought the way we are heading now, existing access checkers would still work with the applies() method as before?
I agree that making it optional is not a good idea. I like the idea of having two interfaces for this. I think it even makes more sense than how I initially had this with the optional method. That can easily make things confusing as the behaviour of the applies() method would change slightly if appliesTo() was implemented.
Comment #27
Panchotagging
Comment #28
dawehnerProove me wrong, but as the interface for the AccessCheckInterface does not change at all, people don't even have a need to port their code to the static access check interface.
Comment #29
BerdirYes, I'm a bit confused now, this already seems to be a completely separate interface with only appliesTo().
Are the interfaces really up to date?
Also, why don't you use instanceof here? is_subclass_of() looks strange as it's an interface.
Sounds like this interface should extend from the other and just define this additional method?
But yes, if we have a separate interface, then it's not an API change, the only problem is educating developers to use that interface when they should... which should be possible by a) updating all core access checkers and examples (in change notices, documentation) and b) have good documentation on the interfaces, e.g. document on the non-static interface that in most cases, the static extension should be used.
Comment #30
damiankloip CreditAttribution: damiankloip commentedYeah, the interfaces are kind of up to date...I think :) that's kind of what I was going for.
The patch should also make most of the conversions of the core checkers anyway, and they work fine. It's mainly an issue of the tests being fixed that just use applies() to check the boolean return of that.
I think having two interfaces makes more sense;
- The current interface can be left intact
- The core access checkers will implement the new interface, so we will have easy to see examples in core
- I think it makes the logic easier to understand, we then have a distinct behaviour for each interface. Rather than basing on logic on whether the appliesTo() method returns anything etc.. Which IMO is more likely to cause confusion for developers.
I will reroll a patch today, I'm still not sure how to deal with the tests that call applies() directly though? Any thoughts on that? Do we just check the output of appliesTo(), and check it returns the array we expect? do we remove them? do we change the tests to use the access manager and check they work as expected that way?
Comment #31
dawehnerIt feels kind of wrong to have all the static::ALLOW ... constants in the same interface && the actual access method. Can't they be moved to a base interface?
Comment #32
damiankloip CreditAttribution: damiankloip commentedThat's what I was asking ^^ :) I think a base interface is a good idea, but wanted some thoughts.
Comment #33
BerdirAh, so the interfaces are exclusive, you can't mix the two methods? Works for me if there's no use case for that. But yes, we should have a common interface then I think.
Comment #34
damiankloip CreditAttribution: damiankloip commentedI've added a new AccessInterface that AccessCheckInterface and StaticAccessCheckInterface (better name?) extend. I have changed some tests that were testing applies() directly, no doubt there are a few more as I just grepped for testApplies() methods in Unit tests for now.
Comment #36
damiankloip CreditAttribution: damiankloip commented#34: 2012382-34.patch queued for re-testing.
Comment #38
damiankloip CreditAttribution: damiankloip commented#34: 2012382-34.patch queued for re-testing.
Comment #40
damiankloip CreditAttribution: damiankloip commentedTypo is ruining everything.
Comment #41
dawehnerOh php actually cares about it sometimes, nice! Btw. should we better link to AccessInterface::ALLOW here?
Comment #42
damiankloip CreditAttribution: damiankloip commentedYep, I think so. I will see how these tests get on first.
Comment #43
Crell CreditAttribution: Crell commentedThis is no longer funny, d.o...
Comment #44
damiankloip CreditAttribution: damiankloip commentedComment #45
dawehnerIf you look at the actual code this does not return anything, as it just sets the information onto the object.
Let's keep {@inheritdoc}
Comment #46
dawehnerComment #47
damiankloip CreditAttribution: damiankloip commentedHere we go.
Comment #48
dawehnerThank you.
Comment #50
damiankloip CreditAttribution: damiankloip commented#47: 2012382-47.patch queued for re-testing.
Comment #51
damiankloip CreditAttribution: damiankloip commentedBack to
Comment #52
Crell CreditAttribution: Crell commentedHold on a sec. This
1) Is an API change, based on the number of classes modified.
2) Doesn't address the DX issue of _my_custom_thing: 'TRUE', which is the really dumb part.
I don't think a non-critical-path performance tweak without benchmarks justifies an API change at this point when it doesn't fix the DX issue.
Comment #53
damiankloip CreditAttribution: damiankloip commentedI'm not seeing how this is an API change? It changes the core access checkers, yes, but if you are already using them in a route definition it will be the same, and using applies() on AccessCheckInterface will be the same. So it's an addition.
The DX issue; As this issue is about improving the efficiency of adding applicable access checkers to routes, I don' think here is the place to solve that. Having an array of checkers to check e.g. [one, two, three] would still not solve the issue as we want the value from some of these keys, such as
_permission: 'my permission'
- which i think is what we discussed (or something to that effect). In my opinion that's an issue in itself.Benchmarks around the routing system.....Let's not go there (Benchmarking of the routing system in general, and ..lack of it). This is just common sense, why would we iterate over every access checker on every route when we don't need to do that to compile the router data?
Comment #54
catchRouter building is critical path when you don't have a router, or if you're the person who ended up triggering the router build, but this does need profiling.
Neither non-bc breaking API additions, nor performance improvements need to be assigned to Dries.
Comment #55
damiankloip CreditAttribution: damiankloip commentedI created a small script that created 5 route collections each containing 1000 routes, and then running setChecks() for each collection.
Comment #56
dawehnerTry to also provide the actual test code ... The points from crell got explained. Especially this issue is not about improving the DX so let's add a follow up, if this is still valid. This could be a problem of the route system in general, let's see.
Comment #57
catchThanks for the profiling, that's a very nice improvement.
Committed/pushed to 8.x, thanks!