Problem/Motivation

http://8.x.localhost/admin/config/search/path/add

'Existing system path' => node/1

'Path alias' => foo

Then:

'Existing system path' => /node/1

'Path alias' => bar

Expected result: validation error

Actual result:

The website encountered an unexpected error. Please try again later.

Both are saved to the database. The UI is unreachable. Only way to recover is to delete the bad record from the URL alias table.

Critical because this is an unrecoverable fatal error with the core UI.

Upgrade path because existing 8.x installs could have a mix of leading and non leading '/' stored.

Also upgrade path because nearly everywhere else in core is treating system path has having a leading '/' (such as the menu UI) so we might want to change this to be consistent.

Proposed resolution

  • Store the path aliases with a starting slash
  • Provide the UI + validation to start with a slash
  • Adapt the path alias search
  • Now that the path aliases work with slashes also path processing in general got converted to use a slash
  • This results in having slashes for the following variables as well:
    • site_frontpage
    • site_403
    • site_404

Either don't allow paths to be entered with a leading '/'.

Or bring this more into a line with the menu UI and both require and store a leading '/' on entered paths.

Both theoretically could require an upgrade path since we allow 'node/1' and '/node/2' to appear in the database at the moment.

Remaining tasks

  • Fill out \Drupal\system\Tests\Update\UpdatePathSystemSiteConfigSlashTest and \Drupal\system\Tests\Update\UpdatePathSystemSiteConfigSlashTest

User interface changes

Yes, new validation.

API changes

Data model changes

Maybe.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

catch’s picture

Issue tags: +D8 upgrade path

Found this while reviewing #2508037: Clarify PathProcessor path format.

Tagging with upgrade path because currently menu link storage using internal with a leading '/', and path alias storage doesn't store the leading '/'.

I was originally confused because I copy/pasted the same path into the path alias and menu UIs, and got a validation error on the menu UI.

Then I thought 'what if I add the leading slash to the path alias UI, will it trim it'? And LOLsobbed.

If #2508037: Clarify PathProcessor path format goes the way of the current proposal, we should strongly consider having alias storage and lookup require the leading slash too.

This means adding the leading slash to existing aliases.

catch’s picture

Also if we don't block the upgrade path on this, we

catch’s picture

Issue summary: View changes
dawehner’s picture

Status: Active » Needs review
FileSize
10.21 KB

Just a couple of random changes here and there so far.

catch’s picture

Title: Path alias UI allows node/1 and /node/1 as system path and is inconsistent with menu UI » Path alias UI allows node/1 and /node/1 as system path then fatals

Status: Needs review » Needs work

The last submitted patch, 4: 2509300-2.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
10.73 KB
20.2 KB

Let's see

Status: Needs review » Needs work

The last submitted patch, 7: 2509300-7.patch, failed testing.

xjm’s picture

This is very much leftover technical debt from the menu link / path work around the Princeton sprint in January. Adding to the meta. #2430593: Store {url_alias}.source and .alias with a leading / is particularly relevant.

dawehner’s picture

Status: Needs work » Needs review
FileSize
45.85 KB
26.93 KB

Status: Needs review » Needs work

The last submitted patch, 10: 2509300-10.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
71.75 KB
30.75 KB

Status: Needs review » Needs work

The last submitted patch, 12: 2509300-12.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
86.39 KB
16.78 KB

Some more fixes.

Status: Needs review » Needs work

The last submitted patch, 14: 2509300-14.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
93.07 KB
10.44 KB

Some more ...

Status: Needs review » Needs work

The last submitted patch, 16: 2509300-16.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
104.61 KB
18.91 KB

Hopefully less ...

Status: Needs review » Needs work

The last submitted patch, 18: 2509300-18.patch, failed testing.

The last submitted patch, 4: 2509300-2.patch, failed testing.

The last submitted patch, 7: 2509300-7.patch, failed testing.

The last submitted patch, 10: 2509300-10.patch, failed testing.

The last submitted patch, 12: 2509300-12.patch, failed testing.

The last submitted patch, 14: 2509300-14.patch, failed testing.

The last submitted patch, 16: 2509300-16.patch, failed testing.

The last submitted patch, 18: 2509300-18.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
110.46 KB
7.79 KB

A couple of less ...

Status: Needs review » Needs work

The last submitted patch, 27: 2509300-27.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
112.32 KB
3.04 KB

lesss ...

Status: Needs review » Needs work

The last submitted patch, 29: 2509300-29.patch, failed testing.

Fabianx’s picture

Issue tags: +D8 Accelerate

Needs issue summary update, please :)

From what I get form the patch this is forcing '/' for all paths?

Is this overall not the same problem space as link module with base:// and external://, etc. for user entered paths?

Would move path aliases to entities (issue exists in good shape) and re-using the menu link plugins make sense here (while keeping the same storage in the background)?

dawehner’s picture

Issue summary: View changes

Would move path aliases to entities (issue exists in good shape) and re-using the menu link plugins make sense here (while keeping the same storage in the background)?

Well, the minority of the patch is the path UI, so this really does not help, its the fact that also path processing should use a starting slash, which is related to the path alias processor.
And then things get big. Entities are just a different storage layer, kinda an internal implementation detail, to be honest.

@effulgentsia agreed that we should do that as part of the patch.

Explained a bit more in the issue summary, still not perfect.

dawehner’s picture

Status: Needs work » Needs review
FileSize
115.64 KB
5.17 KB

Patch did not applied anymore due to #2480811: Cache incoming path processing and route matching but well, let's fix the failing tests

Status: Needs review » Needs work

The last submitted patch, 33: 2509300-33.patch, failed testing.

dawehner’s picture

Issue summary: View changes
Status: Needs work » Needs review
FileSize
120.49 KB
4.58 KB

Fixed the failures ... as well as added the update path, now we just need to fill out the tests ;)

Status: Needs review » Needs work

The last submitted patch, 35: 2509300-35.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
120.61 KB
1.06 KB

Quick fix

chx’s picture

Status: Needs review » Needs work

The last submitted patch, 37: 2509300-37.patch, failed testing.

catch’s picture

Issue tags: +Triaged D8 critical

We discussed this on the critical triage call with alexpott, xjm and effulgentsia. Not everyone had realised there was no way to recover from the fatal error. The thing that makes this really critical is that once the duplicate paths are in the database, the admin UI fatals, which means you can't delete either record via the UI. Note I've not looked at why exactly the UI fatals yet, that might also be a symptom of something else.

Tagging as triaged critical.

@chx what do you think is missing from the issue summary, the proposed resolution is pretty accurate compared to what's going on in the patch I think?

dawehner’s picture

Well, I totally think that the patch is not the most minimal patch you can write, but on the other hand, we should solve things the right way and not just implement local workarounds without thinking about the big picture.

catch’s picture

Yes I think the patch is the right way to fix this as well.

@xjm last night pointed out we'd had an issue open to fix this since the last round of path refactoring: #2430593: Store {url_alias}.source and .alias with a leading /. Shows what we run into when we don't fix things all the way in the first place.

dawehner’s picture

Shows what we run into when we don't fix things all the way in the first place.

Kinda reminds me of history.

benjy’s picture

+++ b/core/lib/Drupal/Core/PathProcessor/PathProcessorFront.php
@@ -50,8 +50,8 @@ public function processInbound($path, Request $request) {
+    if ($path == '/<front>') {
+      $path = '/';
     }

Do the magic constants like now need a leading slash when used?

dawehner’s picture

Do the magic constants like now need a leading slash when used?

Good question!

Well yeah its what happens if you call \Drupal::urlGenerator()->generateFromPath('<front>')

webchick’s picture

Can we not either subclass or submit an upstream patch to wherever $this->makeSubrequest() is coming from so that it handles the trimming for you? Developers are going to miss this, so while we might fix core, contrib will end up hitting the same issue.

dawehner’s picture

Can we not either subclass or submit an upstream patch to wherever $this->makeSubrequest() is coming from so that it handles the trimming for you? Developers are going to miss this, so while we might fix core, contrib will end up hitting the same issue.

This is an internal method and well ... every path has now "/" in there, so I'd argue its more consistent to have /<front> there. I hope its okay if we don't overcomplicate things. This code path is not even the recommended one, to be honest.

catch’s picture

Let's split the upgrade path out to a follow-up major issue. There's still time to get this in before hook_update_N() required, and even if it doesn't we decided we could commit separately for critical issues to keep things reviewable (and can bump the follow-up to critical then).

There are cases the upgrade in #37 doesn't catch:

+-----+----------+--------+----------+
| pid | source   | alias  | langcode |
+-----+----------+--------+----------+
|   1 | /node/1  | /test1 | en       |
|   2 | /node/2  | /biii  | en       |
|   3 | //node/1 | /bar   | en      |
+-----+----------+--------+----------+
3 rows in set (0.00 sec)

1. Adds forward slash to a source that already has a forward slash.
2. If we fixed #1, we'd additionally have to de-duplicate the /node/1 source when the alias is the same - although apparently the url_alias table has no unique key, so you can have 100% duplicate aliases and that wouldn't fatal at least.

catch’s picture

dawehner’s picture

Status: Needs work » Needs review
Related issues: +#2511962: Upgrade path for 2509300
FileSize
119.12 KB
1.49 KB

here is the upgrade path issue: #2511962: Upgrade path for 2509300

Status: Needs review » Needs work

The last submitted patch, 50: 2509300-49.patch, failed testing.

chx’s picture

Sorry. It was a cross post of sorts: didn't realize <12hrs before I tagged dawehner already fixed it (we talked about it on IRC, I knew I wanted to tag it but he was faster and I didn't realize). Daniel is awesome like that :)

dawehner’s picture

Status: Needs work » Needs review
FileSize
120.29 KB
865 bytes

Let's fix the remaining bits.

Sorry. It was a cross post of sorts: didn't realize <12hrs before I tagged dawehner already fixed it (we talked about it on IRC, I knew I wanted to tag it but he was faster and I didn't realize). Daniel is awesome like that :)

Cool! I had issues with explaining issues in the past :)

Status: Needs review » Needs work

The last submitted patch, 53: 2509300-53.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
118.8 KB
1.49 KB

Forgot to remove the actual update function ... note: this is always reviewable ...

tim.plunkett’s picture

+++ b/core/lib/Drupal/Core/PathProcessor/PathProcessorFront.php
@@ -50,8 +50,8 @@ public function processInbound($path, Request $request) {
-    if ($path == '<front>') {
-      $path = '';
+    if ($path == '/<front>') {
+      $path = '/';

+++ b/core/modules/system/src/PathBasedBreadcrumbBuilder.php
@@ -161,7 +161,7 @@ public function build(RouteMatchInterface $route_match) {
       $links[] = Link::createFromRoute($this->t('Home'), '<front>');

As long as the module-author-facing part of this API is still

<front>

, then I don't mind it being

/<front>

internally.

dawehner’s picture

As long as the module-author-facing part of this API is still

Yeah to be clear, the recommended way: Url::fromRoute('<front>') still works as expected.
For sure, we could special case <front>, but then you could ask yourself, why ...

Crell’s picture

Better question... If the leading / is always required... why does <front> need to exist? It exists to differentiate from the empty string, we don't need to differentiate from the empty string anymore because the front page is /.

/ is synonymous with <front>. Let's just use that and deprecate <front> entirely. (We can't remove it right now, but we can deprecate it.)

Edit: Writing about <front> in HTML is really annoying. :-P

Wim Leers’s picture

+1

dawehner’s picture

Thank you for some feedback!

Better question... If the leading / is always required... why does need to exist? It exists to differentiate from the empty string, we don't need to differentiate from the empty string anymore because the front page is /.

/ is synonymous with . Let's just use that and deprecate entirely. (We can't remove it right now, but we can deprecate it.)

Edit: Writing about in HTML is really annoying. :-P

This code we are talking about is for url outbound processing ... existed though since ever in Drupal and getting rid of it in general should be part of a different issue.

To explain that again, Url::fromRoute('')->toString() would not pass <front> to the path processor.
For an actual incoming request you also end up with passing in '/', this is only for the following codepath $url_generator->generateFromPath | $url_assember->assemble(),
its all about outbound processing

On the other hand there are still a hell lot of instances, for example:

$ grep "drupalGet('<front>')" . -rn | wc -l
38

What about cleaning this up as part of a follow up? Why do we want to get rid of <front> as part of this issue?

effulgentsia’s picture

Why do we want to get rid of <front> as part of this issue?

Because #55 does get rid of it, and replaces it with /<front>, which is nonsensical. What if instead of exposing that nonsense to path processors, we changed generateFromPath() to look for <front> and internally call generateFromRoute('<front>') on that instead, thereby still allowing drupalGet('<front>') to continue working for now? Then in a followup, we can finish removing all traces of <front> as a path?

effulgentsia’s picture

Note, however, that #61 is not worth doing if it means the difference between this patch landing before the beta is tagged vs. after, since it would be wonderful to not need to write a hook_update_N() for this.

dawehner’s picture

Because #55 does get rid of it, and replaces it with /, which is nonsensical

Well, I agree its odd, but not always start the path and special case <front> in that sense would not be sensical either, IMHO.
On the other hand, we do that already in either places, to be fair.

larowlan’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/Path/AliasStorageInterface.php
    @@ -47,8 +50,8 @@ public function save($source, $alias, $langcode = LanguageInterface::LANGCODE_NO
    +   *   - source (string): The internal system path with a starting lash.
    

    /me bats lashes

  2. +++ b/core/lib/Drupal/Core/PathProcessor/PathProcessorFront.php
    @@ -50,8 +50,8 @@ public function processInbound($path, Request $request) {
    +    if ($path == '/<front>') {
    
    +++ b/core/lib/Drupal/Core/Routing/RouteProvider.php
    @@ -140,14 +140,16 @@ public function getRouteCollectionForRequest(Request $request) {
    +      $path = $path == '/' ? $path : rtrim($request->getPathInfo(), '/');
    

    nitpick, should be ===, $path = 0 would evaluate true here - see http://3v4l.org/vf405

  3. +++ b/core/modules/config/src/Tests/ConfigSingleImportExportTest.php
    @@ -27,7 +27,7 @@ class ConfigSingleImportExportTest extends WebTestBase {
    -  public function testImport() {
    +  public function tesImport() {
    
    @@ -134,14 +134,14 @@ public function testImportSimpleConfiguration() {
    -  public function testExport() {
    +  public function tesExport() {
    

    debugging needs fix

  4. +++ b/core/modules/path/src/Form/PathFormBase.php
    @@ -151,8 +163,18 @@ public function validateForm(array &$form, FormStateInterface $form_state) {
    +      $form_state->setErrorByName('source', 'The source path has to start with a slash.');
    ...
    +      $form_state->setErrorByName('alias', 'The alias path has to start with a slash.');
    
    +++ b/core/modules/path/src/Plugin/Field/FieldWidget/PathWidget.php
    @@ -95,6 +95,10 @@ public static function validateFormElement(array &$element, FormStateInterface $
    +      $form_state->setError($element, t('The alias needs to start with a slash.'));
    
    +++ b/core/modules/system/src/Form/SiteInformationForm.php
    @@ -155,13 +165,17 @@ public function validateForm(array &$form, FormStateInterface $form_state) {
    +      $form_state->setErrorByName('site_frontpage', $this->t("The path '%path' has to start with a /.", ['%path' => $form_state->getValue('site_frontpage')]));
    
    @@ -172,6 +186,12 @@ public function validateForm(array &$form, FormStateInterface $form_state) {
    +      $form_state->setErrorByName('site_403', $this->t("The path '%path' has to start with a /.", ['%path' => $form_state->getValue('site_403')]));
    ...
    +      $form_state->setErrorByName('site_404', $this->t("The path '%path' has to start with a /.", ['%path' => $form_state->getValue('site_404')]));
    

    Not seeing any tests for this new validation?

dawehner’s picture

Status: Needs work » Needs review
FileSize
120.43 KB
9.56 KB

Thank you larowlan!

Status: Needs review » Needs work

The last submitted patch, 65: 2509300-65.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
FileSize
120.57 KB
421 bytes

Ups.

catch’s picture

Opened #2513952: Deprecate <front> (as a path, not a route name) to remove/deprecate <front>

There's a beta due today, and I'd like for that to happen without data migration changes pending in the critical queue, of which #2453153: Node revisions cannot be reverted per translation is the other one, so I'm leaning towards committing this soon after it's RTBC (if we can get it there today).

The other option would be to not commit this now, then either try to get #2511962: Upgrade path for 2509300 in before the next beta, or in the release notes of the next beta we'd have to explicitly say there's no upgrade path for the path alias changes and people should help get one into head2head for that (or use it if it's there).

As in #48 the migration path is not straightforward here, we can have a mix of valid and invalid data, which even after fixing can then have duplicate records.

Patch itself looks good to me, but could do with an additional set of eyes before commit, so not marking it RTBC myself.

alexpott’s picture

I've done a superficial review and nothing is jumping out at me.

catch’s picture

Status: Needs review » Reviewed & tested by the community

RTBC for me.

Fabianx’s picture

RTBC + 1

Makes sense and it is good to see /foo in code everywhere :).

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Discussed some of the rtrimming in RouteProvider with @dawehner in IRC - I think we might be doing one too many but there is no harm as far as I can see.

Committed e044adb and pushed to 8.0.x. Thanks!

  • alexpott committed e044adb on 8.0.x
    Issue #2509300 by dawehner, catch, larowlan: Path alias UI allows node/1...
effulgentsia’s picture

Great to see this land. I started reviewing this just prior to that, so here's what I found before getting the notification that it landed, but this is incomplete.

  1. +++ b/core/lib/Drupal/Core/EventSubscriber/CustomPageExceptionHtmlSubscriber.php
    @@ -66,7 +66,7 @@ protected static function getPriority() {
    -    $this->makeSubrequest($event, $path, Response::HTTP_FORBIDDEN);
    +    $this->makeSubrequest($event, trim($path, '/'), Response::HTTP_FORBIDDEN);
    

    Do we have a followup for makeSubrequest() to expect $path to have a leading slash?

  2. +++ b/core/lib/Drupal/Core/Path/AliasWhitelist.php
    @@ -107,7 +107,7 @@ public function get($offset) {
    -    $exists = $this->aliasStorage->pathHasMatchingAlias($root);
    +    $exists = $this->aliasStorage->pathHasMatchingAlias('/' . $root);
    

    Do we have a follow-up for AliasWhitelist to expect $root to have a leading slash?

  3. +++ b/core/lib/Drupal/Core/Path/PathMatcher.php
    @@ -101,7 +101,7 @@ public function isFrontPage() {
    -        $this->isCurrentFrontPage = ($url->getRouteName() && $url->getInternalPath() === $this->getFrontPagePath());
    +        $this->isCurrentFrontPage = ($url->getRouteName() && '/' . $url->getInternalPath() === $this->getFrontPagePath());
    

    getInternalPath() is marked as deprecated. Will we remove it before 8.0.0? If not, do we want to change its return value to have a leading slash?

  4. +++ b/core/lib/Drupal/Core/Path/PathValidator.php
    @@ -155,10 +156,10 @@ protected function getPathAttributes($path, Request $request, $access_check) {
    -    $path = $this->pathProcessor->processInbound($path, $request);
    +    $path = $this->pathProcessor->processInbound('/' . $path, $request);
    

    Do we have a followup for PathValidator to expect $path to have a leading slash?

dawehner’s picture

Good points yeah I tried to restrict the changes to a reasonable level.

Here is a follow up: #2514166: Use starting slash in :: makeSubrequest AliasWhitelist and PathValidator

lauriii’s picture

Atleast site configuration page is broken after this going in. Created follow-up for that: #2514196: AliasManager->getAliasByPath() handles $path in confusing manner.

TR’s picture

Issue tags: +Needs change record

Again, is it too much to ask for a change record when you commit a change that you know will break existing contribs? Just write up the changes you put in, how long does that take? A hell of a lot less time than it took me to track down why my tests started failing ...

effulgentsia’s picture

Title: Path alias UI allows node/1 and /node/1 as system path then fatals » [needs change record] Path alias UI allows node/1 and /node/1 as system path then fatals
Status: Fixed » Needs work
Mile23’s picture

So the solution to $simpletest->drupalGet('/'); throwing a 'path needs a leading slash' exception is $simpletest->drupalGet(''); ??

dawehner’s picture

So the solution to $simpletest->drupalGet('/'); throwing a 'path needs a leading slash' exception is $simpletest->drupalGet(''); ??

Feel free to open up an issue for that as well

Working on a CR now.

dawehner’s picture

Issue tags: -Needs change record

Again, is it too much to ask for a change record when you commit a change that you know will break existing contribs? Just write up the changes you put in, how long does that take? A hell of a lot less time than it took me to track down why my tests started failing ...

Yes I simply forgot about creating a change record, so please remind yourself that there are humans on the other side of the internet.

Created the change record.

Is there anything else we need to close this down again?

Fabianx’s picture

Title: [needs change record] Path alias UI allows node/1 and /node/1 as system path then fatals » Path alias UI allows node/1 and /node/1 as system path then fatals
Status: Needs work » Fixed

Published the CR, back to fixed.

jhedstrom’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.