Problem/Motivation

According to Symfony2 Routing documentation in http://symfony.com/doc/current/book/routing.html#adding-requirements, the same paths can be declared in different routes using regex requirements to set the matching conditions.

If we write a module declaring this routes in YAML:

my_module.zone:
  path: '/news/{zone}'
  defaults:
    _controller: '\Drupal\my_module\Controller\BasicController::newsByZone'
   zone: 'us'
  requirements:
    _access: 'TRUE'
    zone: 'eu|us'

my_module.news:
  path: '/news/{title}'
  defaults:
    _controller: '\Drupal\my_module\Controller\BasicController::newsByTitle'
  requirements:
    _access: 'TRUE'

The expected behavior when entering /news/eu or /news/us would be the response of newsByZone(), but for some reason I don't understand the response is always returned by newsByTitle even if we change the route declaration order. If am not doing something wrong I think this is a bug.

CommentFileSizeAuthor
#47 2120397-nr-bot.txt144 bytesneeds-review-queue-bot
#35 interdiff-31-35.txt3.55 KBAnonymous (not verified)
#35 2120397-35.patch4.44 KBAnonymous (not verified)
#35 2120397-35-test-only.patch3.03 KBAnonymous (not verified)
#31 2120397-31.patch3.47 KBMunavijayalakshmi
#27 2120397-27.patch3.35 KBAnonymous (not verified)
#27 2120397-27-test-only.patch1.75 KBAnonymous (not verified)
#22 2120397-22.patch3.35 KBkostyashupenko
#17 2120397-17.patch3.3 KBjhedstrom
#17 interdiff.txt1.05 KBjhedstrom
#11 2120397-11.patch3.01 KBjhedstrom
#11 2120397-11-TEST-ONLY.patch1.68 KBjhedstrom
#1 2120397-1.patch1.51 KBdawehner

Comments

dawehner’s picture

StatusFileSize
new1.51 KB

Here is a short test which tries to explain existing behavior.

Both routes do match /news/eu so one of them will be chosen. Maybe you have to implement some additional class implement RouteFilterInterface to have the exact controller you need.

shivansunfire’s picture

From http://symfony.com/doc/current/book/routing.html#adding-requirements

The Symfony router will always choose the first matching route it finds.

This is not happening now for some reason. If this is not the criteria to decide the route (earlier routes always win in Symfony2), which crieria is being applied?

The answer to the problem is to add route requirements

Earlier routes always win, except that you declare requirements for the parameters in the first route, so that way the first route will not match and so will do the second route.

I think in the test we shouldn't expect two routes matching the path but only the first route like Symfony does.

dawehner’s picture

The Symfony router will always choose the first matching route it finds.

Well, it finds the first matching route, we don't ensure order when we push and pull routes from the database.

The only problem is that we just sort routes by fit.

    $routes = $this->connection->query("SELECT name, route FROM {" . $this->connection->escapeTable($this->tableName) . "} WHERE pattern_outline IN (:patterns) ORDER BY fit DESC", array(
      ':patterns' => $ancestors,
    ))
      ->fetchAllKeyed()
Crell’s picture

I am open to adding another order by column to help with these cases if we can figure out a good one to use. "Order defined in the file" doesn't quite work in Drupal's case as well as it does in fullstack or Silex. :-/

Crell’s picture

Hm, actually, should we instead integrate the presence of regex requirements into calculating the "fitness" of a route? Right now it's just path placeholder based, but we could certainly adjust that if needed. Adding a regex makes a route "more specific", and we order most specific to least specific, so...

shivansunfire’s picture

I think we should leave fit as it is now, "a numeric representation of how specific the path is", or maybe we should say "how specific the pattern_outline is". Two routes with the "same" path but different parameter names still has the "same" path, equivalent to the same pattern_outline.

Besides, the RouteCompiler::getFit() method does a good job using bitwise operators and binary math. For each different "number_parts" path type there are 2 raised to the power of number_parts, minus 1 fit combinations, depending on the number of parameters and their position. Introducing regex existence checking would raise the number of combinations to 3 raised to the power of number_parts, minus 1, and we would need "ternary math" instead of using bitwise operators, too much complexity.

The query now sorts by fit, and draw is resolved by MySQL sorting the results by the length of the first column: name, which seems weird and random. I think we need a new column.

As a second sort column, we could set another integer that takes into account if the parameter has a requirement for it or not. We could use the same binary approach that in RouteCompiler::getFit(), applying 0 to each bit (path position) where there is a parameter and it hasn't a requirement, and applying 1 otherwise.

We could also add a weight as an option for each route in the .routing.yml, so developers could specify which route has to be considered first. This secondary sorting option would allow much more flexibility defining routes with parameters, and would improve the interaction between modules. The "weight" (or some similar name) column would move the regex parameter requirement as the third sorting option, so if nobody define a weight, the RouteProvider would still returning first the route with the regex parameter. It would be something like "ORDER BY fit DESC, weight, regex_param_rate"

sun’s picture

Priority: Normal » Major
Issue summary: View changes

This bug is at least be major, I think.

IIRC, e.g. in #2068471: Normalize Controller/View-listener behavior with a Page object we were not able to rename a route, because the current matching algo ignores other route aspects, so another definition (of the same routing.yml file) suddenly came first.

I first wanted to agree with simply lumping this as a factor into 'fit', but on a second thought, I don't think that's going to be a reliable and maintainable approach → the final fitness would become a "magic" value and we'd also have to lump 'methods' and possibly other factors into that sum.

dawehner’s picture

I first wanted to agree with simply lumping this as a factor into 'fit', but on a second thought, I don't think that's going to be a reliable and maintainable approach → the final fitness would become a "magic" value and we'd also have to lump 'methods' and possibly other factors into that sum.

An alternative could be to introduce a step / use RouteFilters to just sort the returned routes, so more specific ones comes later.

Crell’s picture

That's more work along the critical path. If we can put this logic into the provider, I'd rather do that.

Of course, the schema of the router table is an internal detail, not public (since the provider is swappable and we don't promise that it's SQL at all), so we can certainly do things like add a column for "has a regex", "has a method", "has a mime type", etc. Or even just a "number of restrictions" column. That's quite easy for us to do, I'd think, and would let us order by fitness, then the number of specifiers/restrictions.

We could also, on selected backends, bake the regex into the query itself. That's probably not worth doing for the default implementation, though; it just needs to work on all SQL backends, not be fully optimized.

dawehner’s picture

@crell
So do you have any recommendations how/if we should move this issue forward? It seems indeed like an issue,
but can't we figure out on route compile time whether we need to do regex checking, which should basically no overhead at all? Maybe one function call by default + the calls to the regexes in case somebody decided to use them.

jhedstrom’s picture

Status: Active » Needs review
StatusFileSize
new1.68 KB
new3.01 KB

Here's an attempt to pick this up. It includes the test, rerolled and amended, from #1. Not sure if this is the appropriate way to do this sorting in the route provider or not.

The last submitted patch, 11: 2120397-11-TEST-ONLY.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 11: 2120397-11.patch, failed testing.

The last submitted patch, 11: 2120397-11-TEST-ONLY.patch, failed testing.

The last submitted patch, 11: 2120397-11.patch, failed testing.

jhedstrom’s picture

The patch in #11 breaks certain local action links (eg, 'Add a custom block', etc) for some reason.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new1.05 KB
new3.3 KB

The fails were because views-provided routes had more specificity than the routes they replace (eg, node.add_page vs views.content.page_1--the latter had _method requirement).

This patch filters out the deprecated _method and _scheme requirements before sorting on requirements.

Status: Needs review » Needs work

The last submitted patch, 17: 2120397-17.patch, failed testing.

Status: Needs work » Needs review

jhedstrom queued 17: 2120397-17.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 17: 2120397-17.patch, failed testing.

tim.plunkett’s picture

Issue tags: +Needs reroll
kostyashupenko’s picture

Issue tags: -Needs reroll
StatusFileSize
new3.35 KB

re-rolled with auto merge

andypost’s picture

Status: Needs work » Needs review

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

tim.plunkett’s picture

Priority: Major » Normal

Still a bug, but has not proven major over time.

Anonymous’s picture

StatusFileSize
new1.75 KB
new3.35 KB

#22 re-roll.

The last submitted patch, 27: 2120397-27-test-only.patch, failed testing.

Anonymous’s picture

Version: 8.2.x-dev » 8.3.x-dev

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Munavijayalakshmi’s picture

StatusFileSize
new3.47 KB

Rerolled the patch.

dawehner’s picture

  1. +++ b/core/lib/Drupal/Core/Routing/RouteProvider.php
    @@ -371,6 +374,16 @@ protected function getRoutesByPath($path) {
    +      // @todo This won't be needed once Symfony 3 is in use.
    

    Do you mind filling a follow up so we don't forget about it?

  2. +++ b/core/tests/Drupal/KernelTests/Core/Routing/RouteProviderTest.php
    @@ -722,6 +722,40 @@ public function testGetRoutesPaged() {
    +  public function testOutlineMatchPathWithRegex() {
    ...
    +    $collection = new RouteCollection();
    +    // Create routes with increasing order of specificity.
    +    $collection->add('test_a', new Route('/example/{foo}/{otherwise}'));
    +    $collection->add('test_c', new Route('/example/{foo}/{something}', [], ['foo' => 'foo']));
    +    $collection->add('test_b', new Route('/example/{foo}/{bar}', [], [
    +      'foo' => 'foo|bar',
    

    For a better test coverage it feels like it would be better to include access checkers as well, given that they have to be ignored in that counting process.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Anonymous’s picture

StatusFileSize
new3.03 KB
new4.44 KB
new3.55 KB

Forgive me, @dawehner. In those days I was not experienced enough to realize your review :)

Done now.

The last submitted patch, 35: 2120397-35-test-only.patch, failed testing. View results

borisson_’s picture

#35 fixes the remarks @dawehner had in #32. I don't really understand the routing system well enough to feel comfortable setting this issue to RTBC, but I couldn't find anything that I'd like to see changed.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new144 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.