Problem/Motivation

Part of #2407505: [meta] Finalize the menu links (and other user-entered paths) system. As part of #2411333: Create standard logic to handle a entity: URI scheme and #2412509: Change LinkItem.uri to type 'uri' rather than 'string' and introduce user-path: scheme, we are adding support for entity: and user-path: schemes to unify the storage of internal paths to valid URIs. The // are not used with these schemes because this is actually in violation of the spec and incompatible with PHP's parse_url() function. (See #2407505-39: [meta] Finalize the menu links (and other user-entered paths) system for details.) base:// is invalid for the same reason.

Proposed resolution

Convert Url::fromUri() base:// scheme (used for base-relative non-routes, e.g. /robots.txt) to base:.

Remaining tasks

Patch in progress.

Either as part of this issue or as a followup (depending on other work related to the meta), we should also re-evaluate the uses of base: and identify which of them should perhaps be user-path: from #2412509: Change LinkItem.uri to type 'uri' rather than 'string' and introduce user-path: scheme instead. Additionally, all uses of base: in core should probably have a comment justifying their use (vs. other schemes or methods).

User interface changes

None.

API changes

Url::fromUri() now requires base: instead of base:// for base-relative paths.

Comments

xjm’s picture

Status: Active » Needs review
StatusFileSize
new26.78 KB

Starter patch with a couple @todos. Some of these uses of base: are suspect, but many of them will be removed in #2412509: Change LinkItem.uri to type 'uri' rather than 'string' and introduce user-path: scheme.

xjm’s picture

Issue summary: View changes

Status: Needs review » Needs work

The last submitted patch, 1: allyourbase.patch, failed testing.

wim leers’s picture

Only one failure!

  1. +++ b/core/includes/common.inc
    @@ -499,9 +499,9 @@ function _format_date_callback(array $matches = NULL, $new_langcode = NULL) {
    + *   be prepended with base:. For example:
    
    @@ -604,9 +604,9 @@ function drupal_http_header_attributes(array $attributes = array()) {
    + *   be prepended with base:. For example:
    

    Perhaps slightly out of scope, but "should be prepended with base:" is kind of strange, "should use the base: scheme" would be better.

  2. +++ b/core/lib/Drupal/Core/Utility/UnroutedUrlAssembler.php
    @@ -61,14 +61,15 @@ public function assemble($uri, array $options = []) {
    +    // @todo Can we now use parse_url() here?
    
    @@ -108,8 +109,9 @@ protected function buildLocalUrl($uri, array $options = []) {
    +    // @todo Can we now just use parse_url() here?
    

    AFAIK: yes.

xjm’s picture

So the test failure is because we have an assertion that relies on base:// not being a relevant scheme:

    // Add an invalid path from a random test failure.
    $this->drupalPostForm('admin/structure/views/nojs/display/test_view/page_1/path', array('path' => 'AKI@&hO@'), t('Apply'));
    $this->assertText('Invalid path');

Edit: Which is checking:

   if (!parse_url('base:' . $path)) {
      $errors[] = $this->t('Invalid path. Valid characters are alphanumerics as well as "-", ".", "_" and "~".');
    }

in PathPluginBase::validatePath().

[mandelbrot:~ | Thu 18:06:12] $ php -r "print_r(parse_url('base:AKI@&hO@'));"
Array
(
    [scheme] => base
    [path] => AKI@&hO@
)
[mandelbrot:~ | Thu 18:06:34] $ php -r "print_r(parse_url('base://AKI@&hO@'));"
[mandelbrot:~ | Thu 18:06:46] $ 

which was introduced to resolve the random failure #2353347: Random failure in DisplayPathTest which in turn was introduced by the original change. I wonder if we can remove this assertion?

xjm’s picture

Ah, yep, we can remove the assertion. The exact reason it was introduced was:

[10-Oct-2014 14:24:59 UTC] Uncaught PHP Exception InvalidArgumentException: "The URI "base://AKI@&hO@" is invalid. You must use a valid URI scheme. Use base:// for a path, e.g., to a Drupal file that needs the base path. Do not use this for internal paths controlled by Drupal." at /var/lib/drupaltestbot/sites/default/files/checkout/core/lib/Drupal/Core/Url.php line 207

Thrown in:

    if (!parse_url($uri, PHP_URL_SCHEME)) {
      throw new \InvalidArgumentException(String::format('The URI "@uri" is invalid. You must use a valid URI scheme. Use base: for a path, e.g., to a Drupal file that needs the base path. Do not use this for internal paths controlled by Drupal.', ['@uri' => $uri]));
    }
xjm’s picture

Status: Needs work » Needs review
StatusFileSize
new27.57 KB
new814 bytes

Status: Needs review » Needs work

The last submitted patch, 7: routing-2416763-7.patch, failed testing.

xjm’s picture

Status: Needs work » Needs review
StatusFileSize
new27.82 KB

Less reroll fail. Good night moon.

xjm’s picture

Status: Needs review » Needs work

So a followup from #6, I think if we remove that assertion, we actually need to add an assertion in the Url tests that it is valid. Same possibly for the note elsewhere in the path about colons. Working on that and the other @todo.

xjm’s picture

xjm’s picture

Status: Needs work » Needs review
StatusFileSize
new52.13 KB
new45.65 KB
new9.07 KB
new8.28 KB

Alright, that was fun.

Attached:

I've also attached two -do-not-test.patch, one that contains only the // removals (review with git diff --color-words) and one that includes all the changes after that (for easier rerolling next time).

xjm’s picture

+++ b/core/tests/Drupal/Tests/Core/UnroutedUrlTest.php
@@ -69,20 +69,52 @@ protected function setUp() {
+    $this->assertSame($is_external, $url->isExternal());

@@ -104,118 +136,87 @@ public function testCreateFromRequest() {
-  /**
-   * Tests the isExternal() method.
-   *
-   * @depends testFromUri
-   *
-   * @covers ::isExternal
-   */
-  public function testIsExternal(array $urls) {
-    $this->assertTrue($urls[0]->isExternal());
-    $this->assertFalse($urls[1]->isExternal());
-  }

Oops, I meant to move this back out into its own method. Will do that shortly.

yesct’s picture

Status: Needs review » Needs work

does not apply. things moving fast today.

xjm’s picture

Status: Needs work » Needs review
StatusFileSize
new51.7 KB
new928 bytes

Needed a reroll again already. Also addresses #14.

Status: Needs review » Needs work

The last submitted patch, 16: routing-2416763-15.patch, failed testing.

wim leers’s picture

effulgentsia and I reviewed this; we only found a nitpick and some minor missing test coverage. Once that's fixed, we think it's RTBC.

  1. +++ b/core/lib/Drupal/Core/Utility/UnroutedUrlAssembler.php
    @@ -61,14 +61,14 @@ public function assemble($uri, array $options = []) {
    +    if (parse_url($uri, PHP_URL_SCHEME) == 'base') {
    

    Nit: ===

  2. +++ b/core/tests/Drupal/Tests/Core/UnroutedUrlTest.php
    @@ -69,20 +69,52 @@ protected function setUp() {
    +      ['#foo'],
    +      ['foo?bar'],
    +      ['?bar'],
    ...
    +      ['test'],
    

    These test schemeless URIs with respectively a fragment, a path + querystring, querystring and path.

    We also want to test one with a leading slash and one with a double leading slash (scheme-relative).

yesct’s picture

xjm’s picture

Status: Needs work » Needs review
StatusFileSize
new51.74 KB
new1.14 KB

Addresses #18.

xjm’s picture

Just to clarify, I have reproduced #19 only once out of dozens of test runs, so I've postponed that issue on it happening again and assuming #20 passes as expected we should be good here.

wim leers’s picture

Status: Needs review » Reviewed & tested by the community

RTBC per #18.

xjm’s picture

xjm’s picture

StatusFileSize
new1.49 KB
xjm’s picture

Status: Reviewed & tested by the community » Needs work

Unit test failure.

xjm’s picture

Status: Needs work » Needs review
StatusFileSize
new53.23 KB
new1.29 KB
wim leers’s picture

Status: Needs review » Reviewed & tested by the community

effulgentsia and I reviewed the interdiffs; this is RTBC if it comes back green :)

The last submitted patch, 23: routing-2416763-23.patch, failed testing.

webchick’s picture

Status: Reviewed & tested by the community » Fixed

Gettin' this one in while it's hot!

Committed and pushed to 8.0.x. Thanks!

  • webchick committed e69cd76 on 8.0.x
    Issue #2416763 by xjm, Wim Leers, effulgentsia: Convert Url::fromUri()...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.