Problem/Motivation

In #2075889: Make Drupal handle incoming paths in a case-insensitive fashion for routing I found another routing-related regression from Drupal 7 and before - repeated slashes in the request path cause a 404 instead of being normalized away.

This may affect users upgrading from earlier versions of Drupal if links exist on or to site pages that have accidental repeated slashed, and can cause cache bloat as well.

Steps to Reproduce:

  1. Add a node
  2. Go to /node/[nid]. The node will load
  3. Go to /node//[nid]. 404

Proposed resolution

Normalize (remove) duplicate slashes in the incoming path that's seen when matching a route and in the rest of the request

Remaining tasks

port over the fix puled out from #2075889: Make Drupal handle incoming paths in a case-insensitive fashion for routing and add more test coverage.

User interface changes

none

API changes

none

Data model changes

none

Comments

pwolanin created an issue. See original summary.

mpdonadio’s picture

Issue tags: +Needs tests
StatusFileSize
new3.09 KB

These are the removed hunks from the other patch to turn #291 back into #289, so will be our starting point when we get that in.

And, adding Needs Tests so we can add some more cases to check and prevent regressions.

pwolanin’s picture

Status: Active » Needs review

Patch seems to apply cleanly to 8.4.x

wolffereast’s picture

Issue summary: View changes
Status: Needs review » Reviewed & tested by the community
Issue tags: +Baltimore2017, +Triaged for D8 major current state

The issue still exists in 8.4. Applying the patch fixes the issue (Tested /node//[nid], /node/[nid]/, and /node//[nid]//). The functional tests are passing. Moving to RTBC

Not a blocking issue, just a comment: The update to remove the duplicate and trailing slashes could be changed from

$parts = preg_split('@/+@', $request->getPathInfo(), NULL, PREG_SPLIT_NO_EMPTY);
$path = '/' . implode('/', $parts);

to
$path = preg_replace(['{//+}', '{/^}'], ['/', ''], $request->getPathInfo());
I didn't roll a new patch as I have no idea which one would be faster. It may well just be a personal preference.

pwolanin’s picture

Status: Reviewed & tested by the community » Needs review

We are using the preg_split other places, so this at least matches the other code.

Still needs tests

pwolanin’s picture

Issue tags: -Needs tests
StatusFileSize
new5.64 KB
new2.55 KB

Here's with an added kernel test, maybe gtg?

wolffereast’s picture

Fired off the updated tests with no failures. Makes sense on the preg_split, thanks!

One minor note on the following comment as it doesn't read smoothly. I recommend `Note that no test case is included for paths with multiple leading slashes because they are redirected and cannot be used to create Request objects.` or something similar.

+    // Note that paths with multiple leading slashes get redirected, and also
+    // cannot be used to create a Request object, so multiple leading slashes
+    // are not part of the text cases.

Status: Needs review » Needs work

The last submitted patch, 6: 2852361-6.patch, failed testing.

wolffereast’s picture

StatusFileSize
new5.64 KB

updated the patch. only change is the suggested comment edit. not sure what the failure from the test bot is about

pwolanin’s picture

Status: Needs work » Needs review

Failure might be something sporadic - let's try again (setting to NR for that)

mpdonadio’s picture

Failure was b/c https://www.drupal.org/node/2863416#comment-12062889 so NR is def OK here.

I think this looks good. My involvement was minimal with this change; going to read this applied later today and likely set RTBC.

+++ b/core/lib/Drupal/Core/Routing/RouteProvider.php
@@ -146,22 +146,23 @@ public function __construct(Connection $connection, StateInterface $state, Curre
-    $cid = 'route:' . $request->getPathInfo() . ':' . $request->getQueryString();
+    // routes. We can not yet convert the path to lowercase since wildcard path
+    // portions may be case sensitive if they contain data like a base64 encoded
+    // token. We remove repeated and trailing slashes to normalize the path.
+    $parts = preg_split('@/+@', $request->getPathInfo(), NULL, PREG_SPLIT_NO_EMPTY);
+    $path = '/' . implode('/', $parts);
+    $cid = 'route:' . $path . ':' . $request->getQueryString();
     if ($cached = $this->cache->get($cid)) {

More thinking out loud here, but this will result in some stale data in the cache, but I don't think this is a real problem for an edge-case and the entries will be GCed anyway. IOW, I think a post-update hook to clear is overkill.

mpdonadio’s picture

StatusFileSize
new66.62 KB

Played with this applied.

Two things, not sure if we need to go back to NW.

1. If Internal Page Cache is enabled, and a path like '/node/////1' gets cached as a 404, the patch will still serve up a 404 until a cache clear. I think we may want to consider clearing that bin (if enabled) in a post-update hook.

2. Does anyone recall why we aren't doing a RedirectResponse to 301 to the proper slashified path? We will serve up the proper route, but at the path with the wonky slashes? By itself, this may not be a problem, but there are a few side effects with breadcrumbs:

?

pwolanin’s picture

@mpdonadio - we are doing a redirect for leading slashes. I guess that's an option we could use for any repeated (or trailing?) slashes instead of this.

The \Drupal\system\PathBasedBreadcrumbBuilder constructor takes a request context - and fixing that to normalize slashes might be tricky. Maybe the redirect is the better option? It seems otherwise to handle this consistently we'd need to be normalizing the request path before or in the initial Request object creation in index.php.

john cook’s picture

Status: Needs review » Needs work

While the patch works, I don't think that it's implemented as it should be.

If you got to ///node/[nid] you are redirected to /node/[nid]. I believe the same thing should happen if you visit /node///[nid] as is mentioned by pwolanin in #13. This should also fix mpdonadio's breadcrumb problem in #12.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

xano’s picture

Status: Needs work » Needs review
StatusFileSize
new6.58 KB

Re-roll for 8.6.x. This includes a rather significant change in the form of RouteProvider::normalizePath(), which I introduced to deal with the introduction of RouteProvider::getRouteCollectionCacheId() since the last iteration of this patch.

I will look into redirecting duplicated non-leading slashes as well.

xano’s picture

StatusFileSize
new3.92 KB

This implements the redirect approach as discussed in #12 and #13.

xano’s picture

StatusFileSize
new730 bytes
new3.87 KB
xano’s picture

I think this is a lot cleaner, primarily because it keeps our API pure, and we move processing invalid paths to a layer dedicated to handling input.

runeasgar’s picture

Status: Needs review » Reviewed & tested by the community
  1. Reproducing. Creating an article, landed on /node/1: works. /node//1: 404.
  2. Applying patch: curl -l https://www.drupal.org/files/issues/2018-03-28/drupal_2852361_19.patch | git apply -v
  3. Applied clean.
  4. drush cr
  5. Retry node//1: success, I am redirected to /node/1.
  6. Few more for good measure. /node/1//, /node//1//, both work, but I am redirected to /node/1/ (trailing backslash). I assume this is fine.

Since this issue has successful tests, and has no UI changes (no screenshot needed), I'm moving it to RTBC. Apologies if that's not the correct action.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 19: drupal_2852361_19.patch, failed testing. View results

xano’s picture

Status: Needs work » Reviewed & tested by the community

That looked like an unrelated failure.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 19: drupal_2852361_19.patch, failed testing. View results

xano’s picture

Status: Needs work » Reviewed & tested by the community
alexpott’s picture

Priority: Major » Normal
Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs change record

I don't see how this is a major bug. Also the current fix makes it a minor release only fix and one that needs a change record.

+++ b/core/core.services.yml
@@ -1210,8 +1210,8 @@ services:
-  redirect_leading_slashes_subscriber:
-    class: Drupal\Core\EventSubscriber\RedirectLeadingSlashesSubscriber
+  redirect_successive_slashes_subscriber:
+    class: Drupal\Core\EventSubscriber\RedirectSuccessiveSlashesSubscriber

similarity index 79%
rename from core/lib/Drupal/Core/EventSubscriber/RedirectLeadingSlashesSubscriber.php

rename from core/lib/Drupal/Core/EventSubscriber/RedirectLeadingSlashesSubscriber.php
rename to core/lib/Drupal/Core/EventSubscriber/RedirectSuccessiveSlashesSubscriber.php

Can't do this in bug fix. Imo it'd be better to just do the changes to RedirectLeadingSlashesSubscriber and then do the service rename and class rename in another issue.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

xano’s picture

So keep RedirectLeadingSlashesSubscriber but make it handle trailing slashes as well? Would a docblock update and change record be sufficient, or do we want to document this another way too?

xano’s picture

Status: Needs work » Needs review
Issue tags: -Needs change record
StatusFileSize
new1.49 KB
new3 KB

This patch addresses the feedback given in #26. Thanks! I also added a change record.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

xjm’s picture

Priority: Normal » Major
Issue tags: -Triaged for D8 major current state +Usability

I'm not sure I agree with #26. This is extremely painful UX-wise and has caused a lot of pain since the major changes from D8 menu/routing sprint. It also creates D7 upgrade path problems with old links suddenly becoming broken. There should be some related issues about the leading slashes issues; I didn't find them but they're out there.

Setting major for now.

ranjith_kumar_k_u’s picture

StatusFileSize
new2.89 KB

Rerolled #29

smustgrave’s picture

StatusFileSize
new782.83 KB
new1.12 MB
new1.46 KB

Tested patch #38 and verified it worked see screenshots

This patch I'm uploading is the same as #38 but noticed there was no tests only patch.

smustgrave’s picture

StatusFileSize
new2.9 KB

Not sure why this didn't upload. Again SAME as #38

If tests-only patch fails and this passes I'll mark RTBC

The last submitted patch, 39: 2852361-39-tests-only.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 40: 2852361-39.patch, failed testing. View results

smustgrave’s picture

Status: Needs work » Reviewed & tested by the community

Looks like everything passed and test-only failed.

  • catch committed cad7dfb on 10.0.x
    Issue #2852361 by Xano, smustgrave, pwolanin, mpdonadio, wolffereast,...
  • catch committed 511778a on 10.1.x
    Issue #2852361 by Xano, smustgrave, pwolanin, mpdonadio, wolffereast,...
  • catch committed a4cf727 on 9.5.x
    Issue #2852361 by Xano, smustgrave, pwolanin, mpdonadio, wolffereast,...
catch’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: +Needs followup

Committed/pushed to 10.1.x, cherry-picked to 10.0.x and 9.5.x

We could use a follow-up for the class rename.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

quietone’s picture

I made the followup, change the CR to make it easier to read and published it.