Problem/Motivation
http://8.x.localhost/admin/config/search/path/add
'Existing system path' => node/1
'Path alias' => foo
Then:
'Existing system path' => /node/1
'Path alias' => bar
Expected result: validation error
Actual result:
The website encountered an unexpected error. Please try again later.
Both are saved to the database. The UI is unreachable. Only way to recover is to delete the bad record from the URL alias table.
Critical because this is an unrecoverable fatal error with the core UI.
Upgrade path because existing 8.x installs could have a mix of leading and non leading '/' stored.
Also upgrade path because nearly everywhere else in core is treating system path has having a leading '/' (such as the menu UI) so we might want to change this to be consistent.
Proposed resolution
- Store the path aliases with a starting slash
- Provide the UI + validation to start with a slash
- Adapt the path alias search
- Now that the path aliases work with slashes also path processing in general got converted to use a slash
- This results in having slashes for the following variables as well:
- site_frontpage
- site_403
- site_404
Either don't allow paths to be entered with a leading '/'.
Or bring this more into a line with the menu UI and both require and store a leading '/' on entered paths.
Both theoretically could require an upgrade path since we allow 'node/1' and '/node/2' to appear in the database at the moment.
Remaining tasks
- Fill out
\Drupal\system\Tests\Update\UpdatePathSystemSiteConfigSlashTest
and\Drupal\system\Tests\Update\UpdatePathSystemSiteConfigSlashTest
User interface changes
Yes, new validation.
API changes
Data model changes
Maybe.
Comment | File | Size | Author |
---|---|---|---|
#67 | interdiff.txt | 421 bytes | dawehner |
#67 | 2509300-67.patch | 120.57 KB | dawehner |
#65 | interdiff.txt | 9.56 KB | dawehner |
#65 | 2509300-65.patch | 120.43 KB | dawehner |
#55 | interdiff.txt | 1.49 KB | dawehner |
Comments
Comment #1
catchFound this while reviewing #2508037: Clarify PathProcessor path format.
Tagging with upgrade path because currently menu link storage using internal with a leading '/', and path alias storage doesn't store the leading '/'.
I was originally confused because I copy/pasted the same path into the path alias and menu UIs, and got a validation error on the menu UI.
Then I thought 'what if I add the leading slash to the path alias UI, will it trim it'? And LOLsobbed.
If #2508037: Clarify PathProcessor path format goes the way of the current proposal, we should strongly consider having alias storage and lookup require the leading slash too.
This means adding the leading slash to existing aliases.
Comment #2
catchAlso if we don't block the upgrade path on this, we
Comment #3
catchComment #4
dawehnerJust a couple of random changes here and there so far.
Comment #5
catchComment #7
dawehnerLet's see
Comment #9
xjmThis is very much leftover technical debt from the menu link / path work around the Princeton sprint in January. Adding to the meta. #2430593: Store {url_alias}.source and .alias with a leading / is particularly relevant.
Comment #10
dawehnerGood point yeah I'm currently doing #2509300: Path alias UI allows node/1 and /node/1 as system path then fatals
Comment #12
dawehnerComment #14
dawehnerSome more fixes.
Comment #16
dawehnerSome more ...
Comment #18
dawehnerHopefully less ...
Comment #27
dawehnerA couple of less ...
Comment #29
dawehnerlesss ...
Comment #31
Fabianx CreditAttribution: Fabianx for Drupal Association commentedNeeds issue summary update, please :)
From what I get form the patch this is forcing '/' for all paths?
Is this overall not the same problem space as link module with base:// and external://, etc. for user entered paths?
Would move path aliases to entities (issue exists in good shape) and re-using the menu link plugins make sense here (while keeping the same storage in the background)?
Comment #32
dawehnerWell, the minority of the patch is the path UI, so this really does not help, its the fact that also path processing should use a starting slash, which is related to the path alias processor.
And then things get big. Entities are just a different storage layer, kinda an internal implementation detail, to be honest.
@effulgentsia agreed that we should do that as part of the patch.
Explained a bit more in the issue summary, still not perfect.
Comment #33
dawehnerPatch did not applied anymore due to #2480811: Cache incoming path processing and route matching but well, let's fix the failing tests
Comment #35
dawehnerFixed the failures ... as well as added the update path, now we just need to fill out the tests ;)
Comment #37
dawehnerQuick fix
Comment #38
chx CreditAttribution: chx commentedComment #40
catchWe discussed this on the critical triage call with alexpott, xjm and effulgentsia. Not everyone had realised there was no way to recover from the fatal error. The thing that makes this really critical is that once the duplicate paths are in the database, the admin UI fatals, which means you can't delete either record via the UI. Note I've not looked at why exactly the UI fatals yet, that might also be a symptom of something else.
Tagging as triaged critical.
@chx what do you think is missing from the issue summary, the proposed resolution is pretty accurate compared to what's going on in the patch I think?
Comment #41
dawehnerWell, I totally think that the patch is not the most minimal patch you can write, but on the other hand, we should solve things the right way and not just implement local workarounds without thinking about the big picture.
Comment #42
catchYes I think the patch is the right way to fix this as well.
@xjm last night pointed out we'd had an issue open to fix this since the last round of path refactoring: #2430593: Store {url_alias}.source and .alias with a leading /. Shows what we run into when we don't fix things all the way in the first place.
Comment #43
dawehnerKinda reminds me of history.
Comment #44
benjy CreditAttribution: benjy at CodeDrop commentedDo the magic constants like now need a leading slash when used?
Comment #45
dawehnerGood question!
Well yeah its what happens if you call
\Drupal::urlGenerator()->generateFromPath('<front>')
Comment #46
webchickCan we not either subclass or submit an upstream patch to wherever $this->makeSubrequest() is coming from so that it handles the trimming for you? Developers are going to miss this, so while we might fix core, contrib will end up hitting the same issue.
Comment #47
dawehnerThis is an internal method and well ... every path has now "/" in there, so I'd argue its more consistent to have
/<front>
there. I hope its okay if we don't overcomplicate things. This code path is not even the recommended one, to be honest.Comment #48
catchLet's split the upgrade path out to a follow-up major issue. There's still time to get this in before hook_update_N() required, and even if it doesn't we decided we could commit separately for critical issues to keep things reviewable (and can bump the follow-up to critical then).
There are cases the upgrade in #37 doesn't catch:
1. Adds forward slash to a source that already has a forward slash.
2. If we fixed #1, we'd additionally have to de-duplicate the /node/1 source when the alias is the same - although apparently the url_alias table has no unique key, so you can have 100% duplicate aliases and that wouldn't fatal at least.
Comment #49
catchFound another bug: #2511968: Path field should fall back to language neutral aliases (also makes this happen for the form widget!).
Comment #50
dawehnerhere is the upgrade path issue: #2511962: Upgrade path for 2509300
Comment #52
chx CreditAttribution: chx commentedSorry. It was a cross post of sorts: didn't realize <12hrs before I tagged dawehner already fixed it (we talked about it on IRC, I knew I wanted to tag it but he was faster and I didn't realize). Daniel is awesome like that :)
Comment #53
dawehnerLet's fix the remaining bits.
Cool! I had issues with explaining issues in the past :)
Comment #55
dawehnerForgot to remove the actual update function ... note: this is always reviewable ...
Comment #56
tim.plunkettAs long as the module-author-facing part of this API is still
, then I don't mind it being
internally.
Comment #57
dawehnerYeah to be clear, the recommended way:
Url::fromRoute('<front>')
still works as expected.For sure, we could special case
<front>
, but then you could ask yourself, why ...Comment #58
Crell CreditAttribution: Crell as a volunteer commentedBetter question... If the leading / is always required... why does <front> need to exist? It exists to differentiate from the empty string, we don't need to differentiate from the empty string anymore because the front page is /.
/ is synonymous with <front>. Let's just use that and deprecate <front> entirely. (We can't remove it right now, but we can deprecate it.)
Edit: Writing about <front> in HTML is really annoying. :-P
Comment #59
Wim Leers+1
Comment #60
dawehnerThank you for some feedback!
This code we are talking about is for url outbound processing ... existed though since ever in Drupal and getting rid of it in general should be part of a different issue.
To explain that again, Url::fromRoute('')->toString() would not pass
<front>
to the path processor.For an actual incoming request you also end up with passing in '/', this is only for the following codepath $url_generator->generateFromPath | $url_assember->assemble(),
its all about outbound processing
On the other hand there are still a hell lot of instances, for example:
What about cleaning this up as part of a follow up? Why do we want to get rid of
<front>
as part of this issue?Comment #61
effulgentsia CreditAttribution: effulgentsia at Acquia commentedBecause #55 does get rid of it, and replaces it with
/<front>
, which is nonsensical. What if instead of exposing that nonsense to path processors, we changedgenerateFromPath()
to look for<front>
and internally callgenerateFromRoute('<front>')
on that instead, thereby still allowingdrupalGet('<front>')
to continue working for now? Then in a followup, we can finish removing all traces of<front>
as a path?Comment #62
effulgentsia CreditAttribution: effulgentsia at Acquia commentedNote, however, that #61 is not worth doing if it means the difference between this patch landing before the beta is tagged vs. after, since it would be wonderful to not need to write a hook_update_N() for this.
Comment #63
dawehnerWell, I agree its odd, but not always start the path and special case
<front>
in that sense would not be sensical either, IMHO.On the other hand, we do that already in either places, to be fair.
Comment #64
larowlan/me bats lashes
nitpick, should be ===, $path = 0 would evaluate true here - see http://3v4l.org/vf405
debugging needs fix
Not seeing any tests for this new validation?
Comment #65
dawehnerThank you larowlan!
Comment #67
dawehnerUps.
Comment #68
catchOpened #2513952: Deprecate <front> (as a path, not a route name) to remove/deprecate
<front>
There's a beta due today, and I'd like for that to happen without data migration changes pending in the critical queue, of which #2453153: Node revisions cannot be reverted per translation is the other one, so I'm leaning towards committing this soon after it's RTBC (if we can get it there today).
The other option would be to not commit this now, then either try to get #2511962: Upgrade path for 2509300 in before the next beta, or in the release notes of the next beta we'd have to explicitly say there's no upgrade path for the path alias changes and people should help get one into head2head for that (or use it if it's there).
As in #48 the migration path is not straightforward here, we can have a mix of valid and invalid data, which even after fixing can then have duplicate records.
Patch itself looks good to me, but could do with an additional set of eyes before commit, so not marking it RTBC myself.
Comment #69
alexpottI've done a superficial review and nothing is jumping out at me.
Comment #70
catchRTBC for me.
Comment #71
Fabianx CreditAttribution: Fabianx for Drupal Association commentedRTBC + 1
Makes sense and it is good to see /foo in code everywhere :).
Comment #72
alexpottDiscussed some of the rtrimming in RouteProvider with @dawehner in IRC - I think we might be doing one too many but there is no harm as far as I can see.
Committed e044adb and pushed to 8.0.x. Thanks!
Comment #74
effulgentsia CreditAttribution: effulgentsia at Acquia commentedGreat to see this land. I started reviewing this just prior to that, so here's what I found before getting the notification that it landed, but this is incomplete.
Do we have a followup for makeSubrequest() to expect $path to have a leading slash?
Do we have a follow-up for AliasWhitelist to expect $root to have a leading slash?
getInternalPath() is marked as deprecated. Will we remove it before 8.0.0? If not, do we want to change its return value to have a leading slash?
Do we have a followup for PathValidator to expect $path to have a leading slash?
Comment #75
dawehnerGood points yeah I tried to restrict the changes to a reasonable level.
Here is a follow up: #2514166: Use starting slash in :: makeSubrequest AliasWhitelist and PathValidator
Comment #76
lauriiiAtleast site configuration page is broken after this going in. Created follow-up for that: #2514196: AliasManager->getAliasByPath() handles $path in confusing manner.
Comment #77
TR CreditAttribution: TR commentedAgain, is it too much to ask for a change record when you commit a change that you know will break existing contribs? Just write up the changes you put in, how long does that take? A hell of a lot less time than it took me to track down why my tests started failing ...
Comment #78
effulgentsia CreditAttribution: effulgentsia at Acquia commentedComment #79
Mile23So the solution to
$simpletest->drupalGet('/');
throwing a 'path needs a leading slash' exception is$simpletest->drupalGet('');
??Comment #80
dawehnerFeel free to open up an issue for that as well
Working on a CR now.
Comment #81
Mile23Filed #2516806: After path requirement change, drupalGet('/') must be replaced with drupalGet('')
Comment #82
dawehnerYes I simply forgot about creating a change record, so please remind yourself that there are humans on the other side of the internet.
Created the change record.
Is there anything else we need to close this down again?
Comment #83
Fabianx CreditAttribution: Fabianx for Drupal Association commentedPublished the CR, back to fixed.
Comment #84
jhedstromAdded #2528414: Block visibility by path docs are missing leading slash.