Problem/Motivation

The code in \Drupal\Core\Routing\UrlGenerator::processPath() doesn't use the (potentially) altered query string from the path processor processOutbound() call:

    $path = '/' . $this->pathProcessor->processOutbound(trim($actual_path, '/'), $options, $this->requestStack->getCurrentRequest(), $cacheable_metadata);
    $path .= $query_string;

Note, $options['query'] can be changed by any of the outbound path processors.

Proposed resolution

Instead of appending the already-set $query_string, the string should be generated after the outbound processors have had a chance to alter it.

Remaining tasks

  • Fix
  • Figure out what should be done with all the core code that is passing $options['query'] as a string instead of an array

User interface changes

API changes

CommentFileSizeAuthor
#67 interdiff.txt798 bytesdawehner
#67 2507005-67.patch11.95 KBdawehner
#66 interdiff.txt3.08 KBdawehner
#66 2507005-66.patch11.81 KBdawehner
#65 2507005-65.patch808 bytesmpp
#56 2507005-56.patch777 bytesnapche
#51 48-51-interdiff.txt1.87 KBhchonov
#51 2507005-51.patch15 KBhchonov
#48 2507005-48.patch12.75 KBhchonov
#47 2507005-47.patch12.63 KBjhedstrom
#47 interdiff.txt427 bytesjhedstrom
#45 2507005-45.patch12.51 KBjhedstrom
#45 interdiff.txt1.82 KBjhedstrom
#43 2507005-43.patch12.58 KBjhedstrom
#43 interdiff.txt1.49 KBjhedstrom
#36 34-36-interdiff.txt1.7 KBhchonov
#36 2507005-36.patch13.27 KBhchonov
#34 2507005-34.patch14.74 KBhchonov
#30 interdiff.txt3.44 KBdawehner
#30 2507005-30.patch14.78 KBdawehner
#28 interdiff.txt1.87 KBdawehner
#28 2507005-28.patch14.03 KBdawehner
#26 interdiff.txt5.98 KBdawehner
#26 2507005-26.patch13.11 KBdawehner
#24 process-path-altered-query-2507005-24.patch9.21 KBjhedstrom
#24 interdiff.txt930 bytesjhedstrom
#22 process-path-altered-query-2507005-22.patch8.3 KBjhedstrom
#22 interdiff.txt1.09 KBjhedstrom
#20 process-path-altered-query-2507005-20.patch8.3 KBjhedstrom
#20 interdiff.txt1.76 KBjhedstrom
#18 process-path-altered-query-2507005-17.patch7.63 KBjhedstrom
#18 interdiff.txt5.83 KBjhedstrom
#15 process-path-altered-query-2507005-15-TEST-ONLY.patch1.5 KBjhedstrom
#13 process-path-altered-query-2507005-13.patch5.02 KBjhedstrom
#13 interdiff.txt3.83 KBjhedstrom
#12 process-path-altered-query-2507005-12.patch5.48 KBjhedstrom
#12 interdiff.txt627 bytesjhedstrom
#10 process-path-altered-query-2507005-10.patch5.44 KBjhedstrom
#10 interdiff.txt853 bytesjhedstrom
#7 process-path-altered-query-2507005-07.patch5.44 KBjhedstrom
#7 interdiff.txt1.03 KBjhedstrom
#4 process-path-altered-query-2507005-04.patch5.19 KBjhedstrom
#4 process-path-altered-query-2507005-04-TEST-ONLY.patch1.47 KBjhedstrom
#2 process-path-altered-query-2507005-02-TEST-ONLY.patch1.47 KBjhedstrom

Comments

jhedstrom’s picture

Note, this appears to only be an issue when UrlGenerator::generateFromRoute() is used. UrlGenerator::generateFromPath eventually uses the correct query parameters.

jhedstrom’s picture

Status: Active » Needs review
StatusFileSize
new1.47 KB

This test illustrates the failure. Fixing it seems non-trivial though as I think there are some outbound path processors already in place that do not fully expect the query string to be appended globally. Simply making generateFromRoute behave like generateFromPath breaks the batch process, and reworking processPath to use the query string defined by $options['query'] results in this odd exception:

LogicException: A stray drupal_render() invocation with $is_root_call = TRUE is causing bubbling of attached assets to break. in Drupal\Core\Render\Renderer->doRender() (line 485 of core/lib/Drupal/Core/Render/Renderer.php).

Status: Needs review » Needs work

The last submitted patch, 2: process-path-altered-query-2507005-02-TEST-ONLY.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new1.47 KB
new5.19 KB

This gets the test to pass, let's see about the rest of our tests.

The last submitted patch, 4: process-path-altered-query-2507005-04-TEST-ONLY.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 4: process-path-altered-query-2507005-04.patch, failed testing.

jhedstrom’s picture

Assigned: jhedstrom » Unassigned
Status: Needs work » Needs review
StatusFileSize
new1.03 KB
new5.44 KB

The query parameters already attached to the path (from route parameters such as _format) need to be pushed into the $options['query'] array.

Status: Needs review » Needs work

The last submitted patch, 7: process-path-altered-query-2507005-07.patch, failed testing.

jhedstrom’s picture

On the PathAliasTest fails, this alias:

/- ._~!$'"()*@[]?&+%#,;=:%23%25%26%2B%2F%3Féøïвβ中國書۞

is being double-escaped (on the %23%25%26%2B%2F%3F part).

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new853 bytes
new5.44 KB

This fixes a few tests.

Status: Needs review » Needs work

The last submitted patch, 10: process-path-altered-query-2507005-10.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new627 bytes
new5.48 KB

This fixes the issue with the alter test failing on sites running in a subdirectory. Still need to figure out the path alias double-escaping issue.

jhedstrom’s picture

StatusFileSize
new3.83 KB
new5.02 KB

This should address the apparent double-escaping. Since the alias test had a '?' in it, that was parsed out as the query string, and then escaped in UrlHelper::buildQuery().

The last submitted patch, 12: process-path-altered-query-2507005-12.patch, failed testing.

jhedstrom’s picture

StatusFileSize
new1.5 KB

Uploading the test-only patch since it has changed slightly since #4.

The last submitted patch, 13: process-path-altered-query-2507005-13.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 15: process-path-altered-query-2507005-15-TEST-ONLY.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new5.83 KB
new7.63 KB

I added a helper method to merge query strings that may already be attached to the path, with ones specified in the options array. I also needed to update RedirectDestination::get() to properly set $options['query'] to an array instead of a string (not sure if that should be a separate issue, but if it is, this one would depend on that fix).

Status: Needs review » Needs work

The last submitted patch, 18: process-path-altered-query-2507005-17.patch, failed testing.

jhedstrom’s picture

Issue summary: View changes
Status: Needs work » Needs review
StatusFileSize
new1.76 KB
new8.3 KB

This should fix the exceptions. The fatal errors were due to a bunch of core code passing $options['query'] as a string. I added a @todo and update the IS. We can either fix here, or in a follow-up.

Status: Needs review » Needs work

The last submitted patch, 20: process-path-altered-query-2507005-20.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new1.09 KB
new8.3 KB

Using DiffArray actually does a strict comparison, so ajax_form=TRUE and ajax_form=1 caused that parameter to be appended to the url twice, which caused the fails in #20.

Status: Needs review » Needs work

The last submitted patch, 22: process-path-altered-query-2507005-22.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new930 bytes
new9.21 KB

The unit test RedirectDestinationTest had an assumption that $options['query'] would be a string.

jhedstrom’s picture

Priority: Normal » Major

Bumping to major since this is clearly a regression from Drupal 7.

dawehner’s picture

StatusFileSize
new13.11 KB
new5.98 KB

Could we also do something like this?

  1. +++ b/core/lib/Drupal/Core/Routing/UrlGenerator.php
    @@ -299,19 +301,18 @@ public function generateFromRoute($name, $parameters = array(), $options = array
    -      $query = $query_params ? '?' . http_build_query($query_params, '', '&') : '';
    +      $query = !empty($options['query']) && is_array($options['query']) ? '?' . UrlHelper::buildQuery($options['query']) : '';
    

    Why do we need to check for the array bit? It seems to be a clear developer error when query is not an array. What about adding an assertion for that?

  2. +++ b/core/lib/Drupal/Core/Routing/UrlGenerator.php
    @@ -561,4 +562,52 @@ public function getRouteDebugMessage($name, array $parameters = array()) {
    +    // @todo Fix all the instances in core where query is not set to an array.
    +    if (!is_array($query)) {
    

    OH I see, meh

Status: Needs review » Needs work

The last submitted patch, 26: 2507005-26.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new14.03 KB
new1.87 KB

Fixed the failures.

Status: Needs review » Needs work

The last submitted patch, 28: 2507005-28.patch, failed testing.

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new14.78 KB
new3.44 KB

Meh

hchonov’s picture

Issue tags: +blocker

This issue is a blocker for #1810394: Site configuration with domain based language negotiation results in redirecting authenticated users to a different domain when accessing a content entity route for translation language different from the interface language where we define a new outbound processor and have to manually alter the path instead of only adding a new query parameter in the options array.

hchonov queued 30: 2507005-30.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 30: 2507005-30.patch, failed testing.

hchonov’s picture

Status: Needs work » Needs review
StatusFileSize
new14.74 KB

re-rolling patch.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Routing/UrlGenerator.php
@@ -561,4 +605,52 @@ public function getRouteDebugMessage($name, array $parameters = array()) {
+
+  /**
+   * Merge query string from the path and options array.
+   *
+   * @param string $path
+   *   The path. If it contains a query string, these parameters should already
+   *   be encoded.
+   * @param array $query
+   *   An array of key/value pairs. These will be passed to
+   *   \Drupal\Component\Utility\UrlHelper::buildQuery().
+   *
+   * @return string
+   *   The complete path including the query string.
+   */
+  protected function mergeQueryString($path, $query) {
+    // @todo Fix all the instances in core where query is not set to an array.
+    if (!is_array($query)) {
+      $string = $query;
+      $query = [];
+      parse_str($string, $query);
+    }
+
+    $path_query = [];
+    if (($query_pos = strpos($path, '?')) && $query_pos !== FALSE) {
+      // Parse the query that is part of the string into an array.
+      $actual_path = substr($path, 0, $query_pos);
+      $query_string = ltrim(substr($path, $query_pos), '?');
+      parse_str($query_string, $path_query);
+      $query_string = '?' . $query_string;
+    }
+    else {
+      $actual_path = $path;
+      $query_string = '';
+    }
+
+    // Make sure not to include the same parameter more than once.
+    $query = array_diff_key($query, $path_query);
+
+    // Build the path.
+    $full_path = $actual_path . $query_string;
+    if (!empty($query)) {
+      $separator = $query_string ? '&' : '?';
+      $full_path .= $separator . UrlHelper::buildQuery($query);
+    }
+
+    return $full_path;
+  }

Note: We can remove this entirely

hchonov’s picture

StatusFileSize
new13.27 KB
new1.7 KB

@dawehner:
Done.

dawehner’s picture

Cool, thank you. Asked @pwolanin for a review.

pwolanin’s picture

Status: Needs review » Needs work

This looks wrong:

+      parse_str($options['query'], $options['query']);

The query should always be an array until the final rendering.

This method name doesn't make much sense to me - what's it supposed to be doing?

generatePathWithString()
pwolanin’s picture

It looks like fixing this bug properly requires more rewriting of the internals of the generator. In particular, this method call should change and not pass in the query params and should get back out any extra parameters somehow to add to them.

$path = $this->getInternalPathFromRoute($name, $route, $parameters, $query_params);
dawehner’s picture

The query should always be an array until the final rendering.

Drupal sucks, see issue summary.

pwolanin’s picture

So, we should try to fix this properly. In the worst case, we need a BC shim to check the result of outbound processing for a "?" to see if some crufty code appending a string query that needs to be merged with the array data.

jhedstrom’s picture

Issue tags: +rc target triage
jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new1.49 KB
new12.58 KB

This patch forces query parameters to an array--this change used to break the batch process itself, but in local testing doesn't appear to, so let's see where else this is still a string.

Status: Needs review » Needs work

The last submitted patch, 43: 2507005-43.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new1.82 KB
new12.51 KB

It looks like $options['query'] = NULL should be supported. This fixes some of the fails above.

Status: Needs review » Needs work

The last submitted patch, 45: 2507005-45.patch, failed testing.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new427 bytes
new12.63 KB

Not sure how this use statement wasn't present.

hchonov’s picture

StatusFileSize
new12.75 KB

Rerolling, because the patch couldn't be applied anymore as changes in UrlGenerator have been commited.

Status: Needs review » Needs work

The last submitted patch, 48: 2507005-48.patch, failed testing.

hchonov’s picture

hchonov’s picture

Status: Needs work » Needs review
StatusFileSize
new15 KB
new1.87 KB

Status: Needs review » Needs work

The last submitted patch, 51: 2507005-51.patch, failed testing.

hchonov’s picture

Status: Needs work » Needs review
tim.plunkett’s picture

Issue tags: -blocker, -rc target triage

The issue that was blocked on this is closed.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

napche’s picture

Version: 8.1.x-dev » 8.1.2
StatusFileSize
new777 bytes

I added a small patch that partially fixes this issue, as this does not seem fixed in 8.1.2.

Disregard..

napche’s picture

Version: 8.1.2 » 8.1.x-dev

Status: Needs review » Needs work

The last submitted patch, 56: 2507005-56.patch, failed testing.

hchonov’s picture

Status: Needs work » Needs review
Issue tags: +DevDaysMilan

Setting back to "needs review" as only the patch from #51 is to be considered.

The last submitted patch, 51: 2507005-51.patch, failed testing.

The last submitted patch, 51: 2507005-51.patch, failed testing.

dawehner’s picture

@hchonov Does #51 still apply?

hchonov’s picture

@dawehner: no, will work on the reroll.

mpp’s picture

Status: Needs review » Needs work

Subscribing as OutboundPathProcessorInterface currently doesn't allow to change the url query.
Needs a reroll.

mpp’s picture

StatusFileSize
new808 bytes
dawehner’s picture

StatusFileSize
new11.81 KB
new3.08 KB

Tried to reroll and simplify the patch a bit

dawehner’s picture

Status: Needs work » Needs review
StatusFileSize
new11.95 KB
new798 bytes

One bugfix

Status: Needs review » Needs work

The last submitted patch, 67: 2507005-67.patch, failed testing.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dawehner’s picture

ruloweb’s picture

I'm working with 8.2.x and confirm that patch in #2809789-40: UrlGenerator fails to urlencode aliases, incorrectly encodes "system" path fix this issue, and it was committed to 8.3.x in #2809789-46: UrlGenerator fails to urlencode aliases, incorrectly encodes "system" path so maybe we should mark this as it wont fix.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

pwolanin’s picture

Status: Needs work » Fixed
Issue tags: +DrupalCampNJ2017, +Triaged for D8 major current state

ok, so we can mark this as fixed since it was resolved in the other issue

xjm’s picture

Status: Fixed » Closed (duplicate)

Thanks @pwolanin! I think duplicate makes the most sense here.