Currently when the RouteBuilder is compiling routes for the routing table it iterates over all the access checkers for every route. This seems unnecessary. It would make more sense to only iterate over the access checkers that apply for that route - this would reduce the amount of iterations and additional function calls by ALOT. This overhead could make a big difference on larger sites in particular (especially if we have to register access checkers for all the things - #2011424: Allow access methods on controllers to be used in routes).

Me and @dawehner discussed this briefly; We could have access checkers declare which requirement keys they apply to. We could then build a map of requirement keys => access checker ID and then just iterate over the requirements for each route and add the checkers that way.

Here is a *very* rough initial patch - Tests will likely fail hard, I've added an appliesTo() method to all the checkers, this is then used to compile a requirement map on the checker. So I have also changed the applies method on the AccessManager to reflect this.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

dawehner’s picture

Nice, that's exactly what I thought would help here.

+++ b/core/lib/Drupal/Core/Access/AccessManager.phpundefined
@@ -69,13 +75,9 @@ public function setChecks(RouteCollection $routes) {
+      if (isset($this->requirementMap[$key]) && $this->checks[$this->requirementMap[$key]]->applies($route)) {
+        $checks[] = $this->requirementMap[$key];
       }

Couldn't we even get rid of the applies method, or do we need this flexibility? One question is: how do we deal with possible multiple access checkers which applies to a single requirements key.

+++ b/core/lib/Drupal/Core/Access/AccessManager.phpundefined
@@ -134,4 +136,25 @@ protected function loadCheck($service_id) {
+      if ($applies_to = $this->checks[$service_id]->appliesTo()) {

+++ b/core/lib/Drupal/Core/Access/DefaultAccessCheck.phpundefined
@@ -23,6 +23,13 @@ public function applies(Route $route) {
+  public static function appliesTo() {

I don't see a reason why this should actually be a static method, as we already have the object available.

+++ b/core/lib/Drupal/Core/Routing/RouteBuilder.phpundefined
@@ -100,8 +100,8 @@ public function rebuild() {
-      $this->dispatcher->dispatch(RoutingEvents::ALTER, new RouteBuildEvent($collection, $module));
       $this->dumper->addRoutes($collection);
+      $this->dispatcher->dispatch(RoutingEvents::ALTER, new RouteBuildEvent($collection, $module));

This change feels really wrong.

damiankloip’s picture

Status: Active » Needs review
FileSize
17.42 KB
16.8 KB

Ok, here's a new patch.

- Made appliesTo return an array
- Store requirementMap as an array of service_ids, as feasibly two access checkers could apply with the same keys.
- Moved the _method logic in Drupal\rest\Access\CSRFAccessCheck into the access() method
- Reverted the change to RouteBuilder - this is a complete accident
- appliesTo() is not static
- Removed applies() method from everywhere

Would be keen to get some validation on the approach in general, then maybe I can add a few more unit tests.

Berdir’s picture

Hm, that limits access checkers to apply to statically defined routes. It for example doesn't allow me to write more intelligent/dynamic checks, especially when adding checks to routes that I don't define.

Which, I think, was the main reason why we went with applies() before.

Couldn't we combine it? Have appliesTo(), allow it to be an empty array (which would match everything, not nothing) and then call applies() when appliesTo() matches, with a default implementation that says return TRUE. Possibly with different names.

Status: Needs review » Needs work

The last submitted patch, 2012382-2.patch, failed testing.

dawehner’s picture

So basically that's what damian proposed in the first version of the patch :) I see that this still improves performance, that is great.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
14.53 KB
17.07 KB

So something more like this?

Status: Needs review » Needs work

The last submitted patch, 2012382-6.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
17.53 KB
20.79 KB

Let's try this and see how it gets on. I think there might be some issues with entity access.

I have just returned TRUE from applies() if we have a static requirement in appliesTo(), what do you think? it's pointless to check array_key_exists in that case. This only makes sense for routes that won't use the appliesTo() method. See Drupal\rest\Access\CSRFAccessCheck in the patch.

Status: Needs review » Needs work

The last submitted patch, 2012382-8.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
624 bytes
20.8 KB

Oops.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Access/AccessCheckInterface.phpundefined
@@ -41,13 +41,22 @@
+   * Declares the route requirement keys this access checker applies to.
...
+   * @return array
+   *   An array of route requirement keys this access checker applies to. An
+   *   empty array will check all routes using the apply method.
+   */
+  public function appliesTo();

This comment should explain why we need both applies and appliesTo. Maybe also mention that return something does lead to a not called apply() method().

+++ b/core/lib/Drupal/Core/Access/AccessManager.phpundefined
@@ -34,6 +35,13 @@ class AccessManager extends ContainerAware {
+   *
...
+  protected $requirementMap = array();

Some docs missing :(

+++ b/core/lib/Drupal/Core/Access/AccessManager.phpundefined
@@ -71,13 +79,27 @@ public function setChecks(RouteCollection $routes) {
+    foreach ($route->getRequirements() as $key => $value) {
+      if (isset($this->requirementMap[$key])) {
+        foreach ($this->requirementMap[$key] as $service_id) {
+          if (empty($this->checks[$service_id])) {
+            $this->loadCheck($service_id);
+          }
+          if ($this->checks[$service_id]->applies($route)) {
+            $checks[] = $service_id;
+          }
+        }

Can we have a one liner comment above, please?

+++ b/core/lib/Drupal/Core/Access/DefaultAccessCheck.phpundefined
@@ -19,12 +19,19 @@ class DefaultAccessCheck implements AccessCheckInterface {
   public function applies(Route $route) {
-    return array_key_exists('_access', $route->getRequirements());
+    return TRUE;
   }

maybe we could add a baseclass in a follow up?

+++ b/core/lib/Drupal/Core/Access/DefaultAccessCheck.phpundefined
@@ -19,12 +19,19 @@ class DefaultAccessCheck implements AccessCheckInterface {
+   * Implements AccessCheckInterface::access().

{@inheritdoc}

+++ b/core/modules/rest/lib/Drupal/rest/Access/CSRFAccessCheck.phpundefined
@@ -21,7 +21,8 @@ class CSRFAccessCheck implements AccessCheckInterface {
-    if (array_key_exists('_access_rest_csrf', $requirements)) {
+
+    if (array_key_exists('_entity_create_access', $requirements)) {

This change seems wrong.

There could be also made some unit tests, for the new functionality.

damiankloip’s picture

Assigned: Unassigned » damiankloip

Thanks for the review! I will make those changes and add some tests.

This change seems wrong.

Yes, very wrong. That will result in quite a few failures I imagine.

maybe we could add a baseclass in a follow up?

Yep, good idea.

Status: Needs review » Needs work

The last submitted patch, 2012382-10.patch, failed testing.

damiankloip’s picture

Assigned: damiankloip » Unassigned
FileSize
2.33 KB
21.4 KB
damiankloip’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 2012382-13.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
1.74 KB
21.25 KB

This should fix most of the problems. I am thinking we could also build a list of checkers that have empty appliesTo() methods? Then we wouldn't even have to iterate through all checkers like we are now...

Anyway, Let's see how this patch gets on - Then we can maybe make that change too.

Status: Needs review » Needs work

The last submitted patch, 2012382-17.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
2.93 KB
21.69 KB

Let's try the dynamic requirements map too, see if we're in the same spot. Then it will just be tests (new/fixing).

dawehner’s picture

We can still iterate over some of the internal details if needed, but for now this is fine.
Does someone know whether it is worth to test this internal behavior (could be used by using mocks).

+++ b/core/lib/Drupal/Core/Access/DefaultAccessCheck.phpundefined
@@ -16,15 +16,22 @@
+  /**
+   * Implements AccessCheckInterface::access().

{@inheritdoc}

Status: Needs review » Needs work

The last submitted patch, 2012382-19.patch, failed testing.

damiankloip’s picture

FileSize
26.91 KB
28.82 KB

Just posting some changes here (after a long day travelling) to add a StaticAccessCheckInterface that uses appliesTo() instead of applies().

Not sure what we should exactly do with the interfaces at the moment, as I've currently just copied AccessCheckInterface but changed the applies() method. Should we just have one common interface they both inherit, like AccessInterface? Then the checkers can extend that add add their methods respectively?

This path will still fail quite a few tests, due to the applies() being used for them.

I also haven't changed the DX for this yet, I know we were dicussing having an 'authorization' options key instead of something. This could get tricky, if we just want to be able to provide a list of 'static' checkers. As they may still want to use the value from the requirements, like DefaultAccessCheck for example.

Crell’s picture

Assigned: Unassigned » Dries

Before we go any further, let's check with higher authority.

Dries: This is a DX improvement. It is not critical, but it is useful. I'm pretty sure it can be done without breaking an existing API; just expanding it. It does not require any follow-ups, but we could likely simplify/refactor several one-off checkers as a result of this. It MAY involve a change to the route that uses those one-offs, but since they're one-offs by definition no one else is going to be using them so no already-ported contribs should be affected. Go/no go?

damiankloip’s picture

Yes, it would be good to get some clarification on this.

I think the bigger the site the more important this is, we already have a lot of access checkers in core, many of which are for one off purposes. When contrib gets involved this is going to get crazy.

I agree though, we could live without this if we had to. I think.. We have no examples of large sites, and we wont have for some time until sites are actually using D8, then it might be too late. Hopefully this is OK, as I think we are just extending, rather than changing. I hope to keep the current AccessCheckInterface intact. If any modules are using any of the current access checkers in their routes, this should still work without change.

Berdir’s picture

The only thing that would break is existing custom access checkers.

And if we really have to, we could even make that part optional, by making the method optional, but I'd prefer to not do that as many contrib module authors with custom access checkers will then forget to implement that.

damiankloip’s picture

I thought the way we are heading now, existing access checkers would still work with the applies() method as before?

I agree that making it optional is not a good idea. I like the idea of having two interfaces for this. I think it even makes more sense than how I initially had this with the optional method. That can easily make things confusing as the behaviour of the applies() method would change slightly if appliesTo() was implemented.

Pancho’s picture

dawehner’s picture

Proove me wrong, but as the interface for the AccessCheckInterface does not change at all, people don't even have a need to port their code to the static access check interface.

Berdir’s picture

+++ b/core/lib/Drupal/Core/Access/AccessManager.phpundefined
@@ -196,4 +219,38 @@ protected function loadCheck($service_id) {
+      // Empty arrays will not register anything.
+      if (is_subclass_of($this->checks[$service_id], 'Drupal\Core\Access\StaticAccessCheckInterface')) {
+        foreach ((array) $this->checks[$service_id]->appliesTo() as $key) {
+          $this->staticRequirementMap[$key][] = $service_id;
+        }

Yes, I'm a bit confused now, this already seems to be a completely separate interface with only appliesTo().

Are the interfaces really up to date?

Also, why don't you use instanceof here? is_subclass_of() looks strange as it's an interface.

Sounds like this interface should extend from the other and just define this additional method?

But yes, if we have a separate interface, then it's not an API change, the only problem is educating developers to use that interface when they should... which should be possible by a) updating all core access checkers and examples (in change notices, documentation) and b) have good documentation on the interfaces, e.g. document on the non-static interface that in most cases, the static extension should be used.

damiankloip’s picture

Yeah, the interfaces are kind of up to date...I think :) that's kind of what I was going for.

The patch should also make most of the conversions of the core checkers anyway, and they work fine. It's mainly an issue of the tests being fixed that just use applies() to check the boolean return of that.

I think having two interfaces makes more sense;

- The current interface can be left intact
- The core access checkers will implement the new interface, so we will have easy to see examples in core
- I think it makes the logic easier to understand, we then have a distinct behaviour for each interface. Rather than basing on logic on whether the appliesTo() method returns anything etc.. Which IMO is more likely to cause confusion for developers.

I will reroll a patch today, I'm still not sure how to deal with the tests that call applies() directly though? Any thoughts on that? Do we just check the output of appliesTo(), and check it returns the array we expect? do we remove them? do we change the tests to use the access manager and check they work as expected that way?

dawehner’s picture

It feels kind of wrong to have all the static::ALLOW ... constants in the same interface && the actual access method. Can't they be moved to a base interface?

damiankloip’s picture

Issue tags: -Performance, -WSCCI

That's what I was asking ^^ :) I think a base interface is a good idea, but wanted some thoughts.

Berdir’s picture

Ah, so the interfaces are exclusive, you can't mix the two methods? Works for me if there's no use case for that. But yes, we should have a common interface then I think.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
10.9 KB
36.18 KB

I've added a new AccessInterface that AccessCheckInterface and StaticAccessCheckInterface (better name?) extend. I have changed some tests that were testing applies() directly, no doubt there are a few more as I just grepped for testApplies() methods in Unit tests for now.

Status: Needs review » Needs work
Issue tags: -DX (Developer Experience), -access rules

The last submitted patch, 2012382-34.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review

#34: 2012382-34.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, 2012382-34.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review

#34: 2012382-34.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +DX (Developer Experience), +access rules

The last submitted patch, 2012382-34.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
FileSize
633 bytes
36.56 KB

Typo is ruining everything.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Access/AccessManager.phpundefined
@@ -194,7 +194,7 @@ protected function checkAny(array $checks, $route, $request) {
+      if ($service_access === AccessCheckInterface::ALLOW) {

Oh php actually cares about it sometimes, nice! Btw. should we better link to AccessInterface::ALLOW here?

damiankloip’s picture

Yep, I think so. I will see how these tests get on first.

Crell’s picture

Issue tags: +Performance, +WSCCI

This is no longer funny, d.o...

damiankloip’s picture

FileSize
2.06 KB
37.72 KB
dawehner’s picture

+++ b/core/lib/Drupal/Core/Access/AccessManager.phpundefined
@@ -196,4 +219,38 @@ protected function loadCheck($service_id) {
+   * @return array
+   *   An array of requirement keys mapped to access checkers. In the format
+   *   '_access_key' => 'service_id'.

If you look at the actual code this does not return anything, as it just sets the information onto the object.

+++ b/core/lib/Drupal/Core/Access/DefaultAccessCheck.phpundefined
@@ -13,17 +13,17 @@
-   * {@inheritdoc}
+   * Implements AccessCheckInterface::access().

Let's keep {@inheritdoc}

dawehner’s picture

Status: Needs review » Needs work
damiankloip’s picture

Status: Needs work » Needs review
FileSize
1.15 KB
37.39 KB

Here we go.

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Thank you.

Status: Reviewed & tested by the community » Needs work
Issue tags: -Performance, -DX (Developer Experience), -access rules, -WSCCI

The last submitted patch, 2012382-47.patch, failed testing.

damiankloip’s picture

Status: Needs work » Needs review
Issue tags: +Performance, +DX (Developer Experience), +access rules, +WSCCI

#47: 2012382-47.patch queued for re-testing.

damiankloip’s picture

Status: Needs review » Reviewed & tested by the community

Back to

Crell’s picture

Status: Reviewed & tested by the community » Needs review

Hold on a sec. This
1) Is an API change, based on the number of classes modified.
2) Doesn't address the DX issue of _my_custom_thing: 'TRUE', which is the really dumb part.

I don't think a non-critical-path performance tweak without benchmarks justifies an API change at this point when it doesn't fix the DX issue.

damiankloip’s picture

Assigned: Unassigned » Dries
Status: Reviewed & tested by the community » Needs review
Issue tags: -needs profiling +DX (Developer Experience)

I'm not seeing how this is an API change? It changes the core access checkers, yes, but if you are already using them in a route definition it will be the same, and using applies() on AccessCheckInterface will be the same. So it's an addition.

The DX issue; As this issue is about improving the efficiency of adding applicable access checkers to routes, I don' think here is the place to solve that. Having an array of checkers to check e.g. [one, two, three] would still not solve the issue as we want the value from some of these keys, such as _permission: 'my permission' - which i think is what we discussed (or something to that effect). In my opinion that's an issue in itself.

Benchmarks around the routing system.....Let's not go there (Benchmarking of the routing system in general, and ..lack of it). This is just common sense, why would we iterate over every access checker on every route when we don't need to do that to compile the router data?

catch’s picture

Assigned: Dries » Unassigned
Issue tags: -DX (Developer Experience) +needs profiling

Router building is critical path when you don't have a router, or if you're the person who ended up triggering the router build, but this does need profiling.

Neither non-bc breaking API additions, nor performance improvements need to be assigned to Dries.

damiankloip’s picture

FileSize
283.65 KB

I created a small script that created 5 route collections each containing 1000 routes, and then running setChecks() for each collection.

Run #51dffef37f325 Run #51dfff126c6f8 Diff Diff%
Number of Function Calls 67,034 5,146 -61,888 -92.3%
Incl. Wall Time (microsec) 183,300 22,690 -160,610 -87.6%
Incl. CPU (microsecs) 183,331 22,695 -160,636 -87.6%
Incl. MemUse (bytes) 1,989,104 1,925,664 -63,440 -3.2%
Incl. PeakMemUse (bytes) 2,143,120 2,081,000 -62,120 -2.9%
dawehner’s picture

Status: Needs review » Reviewed & tested by the community

Try to also provide the actual test code ... The points from crell got explained. Especially this issue is not about improving the DX so let's add a follow up, if this is still valid. This could be a problem of the route system in general, let's see.

catch’s picture

Assigned: Dries » Unassigned
Status: Needs review » Fixed
Issue tags: -DX (Developer Experience) +needs profiling

Thanks for the profiling, that's a very nice improvement.

Committed/pushed to 8.x, thanks!

Automatically closed -- issue fixed for 2 weeks with no activity.