Problem/Motivation

Blocks #2364157: Replace most existing _url calls with Url objects

$base_url is not a valid option in UrlGenerator::generateFromRoute(). Drupal 7 has this option in url(), and domain-based language negotiation relies on it.

Proposed resolution

Update UrlGenerator::generateFromRoute to support this option.

Remaining tasks

  • Update code in Update UrlGenerator::generateFromRoute
  • Update docblock
  • Update coverage in Drupal\Tests\Core\Routing\UrlGeneratorTest()
Contributor tasks needed
Task Novice task? Contributor instructions Complete?
(after reroll) Create a patch (and interdiff) to address #24 Instructions Yes
Reroll the patch if it no longer applies. Instructions Yes
Add automated tests (see #22) Instructions Yes

User interface changes

None.

API changes

None.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because the language selection block is currently unusable in HEAD
Issue priority Critical
Disruption No disruption, there is no API change.

Original report by @mpdonadio

The links in the Language Switcher block are incorrect for domain based negotiation.

Steps to reproduce:

Spin up drupal-8.0.x from HEAD
Login
Enable Languages
Install Italian from admin/config/regional/language
Set to domain based negotiation, and configure URLs from admin/config/regional/language/detection/url
Place Language Switcher block in a region from admin/structure/block; enable both languages.
Go back to the main site.
Hover over English link in the Language Switcher, see it is correct.
Hover over Italian link, see it is the same as the English one. Snap!

I actually think this is a general problem with generating urls from route names when you specify the language, as I am working on another issue and am having problems with code like

$languages = $this->container->get('language_manager')->getLanguages();
$italian_url = Url::fromRoute('system.admin', [], ['language' => $languages['it'], 'absolute' => TRUE])->toString();

ignoring the language option, but I cannot confirm that, so I am starting with this report.

I will work on this if someone can point me in the right direction.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Gábor Hojtsy’s picture

Issue tags: -i18n +D8MI, +language-base

core/modules/language/src/Tests/LanguageUrlRewritingTest.php has sort of unit tests (it needs to build a request environment itself because the test system may not receive whatever domains, so it needs to simulate an environment) for domain + port based link generation using url().

core/modules/language/src/HttpKernel/PathProcessorLanguage.php is where the inbound (from request) and outbound (to output) URL handling is for languages; you should look at the outbound processing and why it does not work

mpdonadio’s picture

Assigned: Unassigned » mpdonadio

I'll start digging into this base on those leads.

mpdonadio’s picture

Status: Active » Needs review
Issue tags: +WSCCI
FileSize
620 bytes

OK, patch is attached. Added WSCCI as this just got complicated:

Domain based language negotiation sets $options['base_url] as part of Drupal\language\Plugin\LanguageNegotiation\LanguageNegotiationUrl::processOutbound(), which gets passed off for URL generation.

Drupal\Core\Url does not have a valid $option for setting the $base_url. $options['base_url] will be ignored, and the $base_url will be determined by the UrlGenerator() by the context (cf, Drupal\Core\Routing\UrlGenerator::generateFromRoute).

The patch addresses this portion. If this is acceptable, I will adjust the docblocks to add this option back in.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Routing/UrlGenerator.php
@@ -183,7 +183,7 @@ public function generateFromRoute($name, $parameters = array(), $options = array
-    $base_url = $this->context->getBaseUrl();
+    $base_url = isset($options['base_url']) ? $options['base_url'] : $this->context->getBaseUrl();

It would be great to expand the test coverage in Drupal\Tests\Core\Routing\UrlGeneratorTest

mpdonadio’s picture

Title: Links in Language Switcher block are incorrect for domain based negotiation » Add $options['base_url'] to UrlGenerator::generateFromRoute()
Component: language system » base system
Category: Bug report » Task
Issue summary: View changes

Per a conversation with @dawehner, this should be added in as the D7 url() function has this parameter.

I will fill this out with proper test coverage, update docs, etc, to have a proper patch for review.

mpdonadio’s picture

Docs and test coverage.

mfb’s picture

There is some code in the generateFromRoute() method that doesn't really make sense to me:

return $scheme . '://' . $host . $port . $base_url . $path . $fragment; 

Doesn't base_url include the scheme, host and port?

dawehner’s picture

@mfb
Nope, $base_url is $this->context->getBaseUrl() which is something else than global $base_url;

dawehner’s picture

Status: Needs review » Reviewed & tested by the community

+1 for that so far.

mfb’s picture

Right. But this patch has $base_url = isset($options['base_url']) ? $options['base_url'] : $this->context->getBaseUrl();

$options['base_url'] includes the scheme, host, port.

$this->context->getBaseUrl() is the base path.

If $base_url is set to a full URL as found in $options['base_url'] then combining that with scheme, host, port down at the bottom of the function doesn't work.

Contrib modules (and custom modules) do need a way to alter all aspects of the URL, including scheme and domain (e.g. Domain Access module modifying the domain, or Secure Login module forcibly rewriting a URL to HTTPS with mixed-mode sessions disabled) so I think it's correct to allow the base_url to be altered. but the code in this method doesn't allow it - it decides what the scheme and host will be and slaps the base_url on the end of that.

mfb’s picture

Status: Reviewed & tested by the community » Needs work

Some sample problems with this patch applied, and two languages enabled, with domain URL language detection:

Clicking on Log out redirects me to http://d8.localhttp//d8.local/

Clicking the Uninstall button at the module uninstall page redirects me to http://d8.localhttp//d8.local/admin/modules/uninstall/confirm

Running tests redirects me to http://d8.localhttp//d8.local/admin/config/development/testing/results/1

mpdonadio’s picture

Status: Needs work » Needs review
FileSize
3.59 KB
1.18 KB

Actually, this is a bug, and a weird one that results in a happy accident in some cases, and sad problems in others.

When LanguageNegotiationUrl::processOutbound() runs with domain negotiation, it sets both $options['base_url'] and $options['absolute'], eg

array(
  'absolute' => TRUE,
  'base_url' => 'http://it.localhost:8888/drupal-8.0.x,
);

Since $absolute has already been pulled out at the top of UrlGenerator::generateFromRoute(), before outbound processing, it is a happy accident and the function calls the `return $base_url . $path . $fragment;` line. Apparently, the cases that @mfb points out are the sad problems. If I apply this on top of the patch from #2364157: Replace most existing _url calls with Url objects, then those tests also fail with the double-concatenated base.

Since anything that implements OutboundPathProcessorInterface could potentially set $options['absolute'] = TRUE, then that line needs to be moved to be after we have done all that.

This cause more quirks. I think this patch is the best option. I think it addressees everything that @mfb saw.

@dawehner, this approach will also get us much closer to finishing up #2364157: Replace most existing _url calls with Url objects.

dawehner’s picture

+++ b/core/lib/Drupal/Core/Routing/UrlGenerator.php
@@ -183,8 +185,8 @@ public function generateFromRoute($name, $parameters = array(), $options = array
+    $base_url = isset($options['base_url']) ? $options['base_url'] : $this->context->getBaseUrl();
+    if (!$absolute || preg_match('/^http(s)?\:\\/\//', $base_url) === 1 || !$host = $this->context->getHost()) {

Can you explain why we need this here? At least a comment about the regular expression should be necessary. It sounds though not like a good idea, to be honest.

mpdonadio’s picture

FileSize
4.53 KB
2.64 KB

Reworked logic to eliminate regex and added comments; added logic and test coverage to support mixed-mode sessions w/ base_url (generateFromPath() has this).

mfb’s picture

Thanks, this resolves the issue I noticed.

One question: Is it necessary to say that base URL is "Only used internally, to modify the base URL when a language dependent URL requires so." I realize this is how it was documented in Drupal 7, but I believe modifying base URL here may be the only way for contrib or custom modules to modify the base URL (to use a different hostname, domain or scheme)?

mpdonadio’s picture

That language was copied over from `UrlGeneratorInterface::generateFromPath`, which nearly one-to-one from the Drupal 7 language, and is the starting point.

Anything that implements one of the path processors can potentially mess with the options, and setting the base_url is a possibility. In core, the domain-based language negotiation is the only(?) thing that does this. I suspect that an origin-pull CDN module would work this way in Drupal 8 (though I haven't put too much thought into it. I think. This may be better:

'base_url': Only used internally, to modify the base URL when a language dependent URL, or other module, requires so.

mfb’s picture

maybe to be more clear something like

Only used internally by a path processor, e.g. to modify the base URL when a language dependent URL requires so.

?

mpdonadio’s picture

FileSize
5.26 KB
1.54 KB

Ok, updates both places to use that language, but didn't use the abbreviation for ESL reasons.

disasm’s picture

+++ b/core/lib/Drupal/Core/Routing/UrlGenerator.php
@@ -183,7 +183,26 @@ public function generateFromRoute($name, $parameters = array(), $options = array
+          $base_url = str_replace('http://', 'https://', $base_url);
+        }
+        elseif ($options['https'] === FALSE) {
+          $base_url = str_replace('https://', 'http://', $base_url);

Should the regexp specify to only str_replace if it matches http or https at the beginning of the string? I can't think of a scenario where http:// or https:// would ever occur anywhere but the beginning of a url, but it might be safer to specify ^http instead of http.

mpdonadio’s picture

str_replace() doesn't do regular expressions. We are avoiding them in this context for performance reasons.

mfb’s picture

Status: Needs review » Reviewed & tested by the community

This looks good to me, setting to RTBC

alexpott’s picture

Category: Task » Bug report
Status: Reviewed & tested by the community » Needs review

From the issue summary this is a bug. Do we need to add some assertions to LanguageUILanguageNegotiationTest::testLanguageDomain() or create a new test along the lines of the issue summary:

  1. Login
  2. Enable Languages
  3. Install Italian from admin/config/regional/language
  4. Set to domain based negotiation, and configure URLs from admin/config/regional/language/detection/url
  5. Place Language Switcher block in a region from admin/structure/block; enable both languages.
  6. Go back to the main site.
  7. Hover over English link in the Language Switcher, see it is correct.
  8. Hover over Italian link, see it is the same as the English one. Snap!
dawehner’s picture

Priority: Major » Critical
Issue tags: +Needs tests

Given that this blocks #2364157: Replace most existing _url calls with Url objects at the moment, this issue should be considered to be a critical one.

mfb’s picture

Status: Needs review » Needs work

Apparently mixedModeSessions has been removed (as a fan of HTTPS everywhere, yay!) so setting to Needs work.

YesCT’s picture

Issue summary: View changes
Issue tags: +Needs reroll, +blocker
mpdonadio’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
5.54 KB

Just the reroll; conflicts were limited to the docblocks.

Status: Needs review » Needs work

The last submitted patch, 26: add-2369225-26.patch, failed testing.

mpdonadio’s picture

FileSize
5.44 KB
1.78 KB

Updates for mixed-mode sessions being gone. #22 has not been addressed yet; this just gets back a passing patch.

mpdonadio’s picture

Status: Needs work » Needs review
mpdonadio’s picture

Issue tags: -Needs tests
FileSize
8.89 KB
3.32 KB

OK, LanguageSwitchingTest seemed the best place to add a new test, as other tests to the language switcher block are there. Used XPath testing to be double sure the test checks the proper links.

mpdonadio’s picture

Issue summary: View changes
FileSize
8.61 KB
4.54 KB

Missed a merge in a comment.

andypost’s picture

Status: Needs review » Needs work

Just a nitpics,

  1. +++ b/core/modules/language/src/Tests/LanguageSwitchingTest.php
    @@ -159,6 +162,72 @@ protected function doTestLanguageBlockAnonymous($block_label) {
    +    $this->assertText('The domain may not be left blank for English', 'The form does not allow blank domains.');
    +    $this->rebuildContainer();
    

    why rebuild needed here? no data changed

  2. +++ b/core/modules/language/src/Tests/LanguageSwitchingTest.php
    @@ -159,6 +162,72 @@ protected function doTestLanguageBlockAnonymous($block_label) {
    +    /** @var UrlGenerator $generator */
    

    should be a namespaced

  3. +++ b/core/lib/Drupal/Core/Routing/UrlGenerator.php
    @@ -172,7 +172,26 @@ public function generateFromRoute($name, $parameters = array(), $options = array
    +    else {
    ...
    +    }
    

    remove else

andypost’s picture

and assertText() should use t()

mpdonadio’s picture

Re #32-1, everything before the block placement in that test was copied verbatim from LanguageUILanguageNegotiationTest::testLanguageDomain(), which has the untranslated assertions and the container rebuilds. I'll fix, though, in the test I added.

mpdonadio’s picture

Status: Needs work » Needs review
FileSize
8.5 KB
3.03 KB

#32 and #33 are addressed. One container rebuild is necessary; moved to proper place and added comment. Also removed the use of the global and used the current Request object instead, as I think that one is slated to be removed.

znerol’s picture

Can you also post a patch which only contains the test (XX-test-only.patch)? The intention of such a patch is to demonstrate the bug and the effectiveness of the test (by means of failing test assertions). See #2378703-4: Port denial of service fixes from SA-CORE-2014-006 to Drupal 8 for an example.

+++ b/core/lib/Drupal/Core/Routing/UrlGeneratorInterface.php
@@ -59,8 +59,8 @@
-   *   - 'base_url': Only used internally, to modify the base URL when a language
-   *     dependent URL requires so.
+   *   - 'base_url': Only used internally by a path processor, for example, to
+   *     modify the base URL when a language dependent URL requires so.
    *   - 'prefix': Only used internally, to modify the path when a language
    *     dependent URL requires so.
    *   - 'script': Added to the URL between the base path and the path prefix.
@@ -132,6 +132,8 @@ public function getPathFromRoute($name, $parameters = array());

@@ -132,6 +132,8 @@ public function getPathFromRoute($name, $parameters = array());
    *   - 'https': Whether this URL should point to a secure location. If not
    *     defined, the current scheme is used, so the user stays on HTTP or HTTPS
    *     respectively. TRUE enforces HTTPS and FALSE enforces HTTP.
+   *   - 'base_url': Only used internally, to modify the base URL when a language
+   *     dependent URL requires so.

Maybe use the same documentation for both methods, it looks like the second one was copied before the first one was modified.

The rest of the patch looks RTBC to me.

mpdonadio’s picture

#36 has been addressed.

Added a test-only patch. Shows the exact fail against HEAD for me locally (lines 215 and 223).

znerol’s picture

Status: Needs review » Reviewed & tested by the community

Applied #35 on a local instance and manually verified that the language selection block renders the links properly. As per #36 this is RTBC, provided that #37 turns green.

znerol’s picture

Issue summary: View changes

The last submitted patch, 37: interdiff-35-37.patch, failed testing.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 37: 2369225-test-only.patch, failed testing.

mpdonadio’s picture

Status: Needs work » Reviewed & tested by the community

Resetting status from #38, as the actual patch passed.

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.0.x, thanks!

  • catch committed cd0cb0d on 8.0.x
    Issue #2369225 by mpdonadio: Add $options['base_url'] to UrlGenerator::...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

mpdonadio’s picture

Assigned: mpdonadio » Unassigned

Cleaning up the issues that I am assigned to...