Fix mismatch of $base_url with Symfony request object

This affects #1543858: Add a startup configuration for the built-in PHP server that supports clean URLs

Progress! Thank you Fabianx.

The patch from #2 allows me to run d8 from a subdirectory of webroot while masking (hiding) the subdirectory from the URL though an .htaccess file in webroot. See https://www.drupal.org/node/2528988#comment-11313379 for a complete description of the use case, configuration, and associated problems.

I'll report any unexpected side effects.

Unfortunately, this breaks the case where you don't wish to mask the installation subdirectory in the URL.

Scenario A (PASS)
- Webroot is /home/user/sites/www.example.com/htdocs/
- D8 installed in /home/user/sites/www.example.com/htdocs/d8/
- There is not an .htaccess file in webroot
- The .htaccess file in /home/user/sites/www.example.com/htdocs/d8/ has not been modified
- The patch from #2 has not been applied
- Everything works as expected when visiting http://www.example.com/d8/

Scenario B (FAIL)
- Webroot is /home/user/sites/www.example.com/htdocs/
- D8 installed in /home/user/sites/www.example.com/htdocs/d8/
- There is not an .htaccess file in webroot
- The .htaccess file in /home/user/sites/www.example.com/htdocs/d8/ has not been modified
- The patch from #2 has been applied
- When visiting http://www.example.com/d8/ the links are correct, but the paths to css and js files are incorrect:

<link rel="shortcut icon" href="/core/misc/favicon.ico" type="image/vnd.microsoft.icon" />
HREF should be "/d8/core/misc/favicon.ico"

<link rel="stylesheet" href="/sites/default/files/css/css_P31p..." media="all" />
HREF should be "/d8/sites/default/files/css/css_P31p..."

<script src="/core/assets/vendor/modernizr/modernizr.min.js?v=3.3.1"></script>
SRC should be "/d8/core/assets/vendor/modernizr/modernizr.min.js?v=3.3.1">

<script src="/sites/default/files/js/js_BKc..."></script>
SRC should be "/d8/sites/default/files/js/js_BKc..."

Scenario C (PASS)
- Webroot is /home/user/sites/www.example.com/htdocs/
- D8 installed in /home/user/sites/www.example.com/htdocs/d8/
- There is an .htaccess file in webroot

RewriteRule ^$ d8/$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ d8/$1 [L]

- The .htaccess file in /home/user/sites/www.example.com/htdocs/d8/ has not been modified
- The patch from #2 has been applied
- Everything works as expected when visiting http://www.example.com/

This is a mess. From a quick search of open 8.x issues...

#2528988: Remove the option to specify a base_url from within settings.php
#2070185: Unify global $base_path and the symfony base_url from the request context.
#2529170: Remove DrupalKernel::initializeRequestGlobals and replace base_root, base_url and base_path with a service
#1433996: update.php doesn't work when Drupal is installed in a subfolder and RewriteBase is used
#2341553: Use routing to generate update.php url
#2377233: $_SERVER['REQUEST_URI'] reverse proxies, base_urls and Apache2 don't seem to mix well
#2403967: DrupalKernel should setup the request context
#2404601: Figure out how to best deal with RequestContext::setCompleteBaseUrl
#2568773: Add $options['base_url'] support to the unrouted URL assembler
#2488514: D8: trusted_host_patterns should include base URL?
#2548095: Add option to Url() to force the site base_url to be used

As a side note, we also have special handling for install.php and rebuild.php in Drupal root's .htacess file.

Until this mess is sorted, the attached patch will allow you to:

1. Install Drupal in webroot/ (default)
2. Install Drupal in webroot/subdir1/
3. Install Drupal in webroot/subdir1/ and mask (hide) subdir1/ with an .htaccess file in webroot/ (See note 1 below)
4. Install Drupal in webroot/subdir1/subdir2/
5. Install Drupal in webroot/subdir1/subdir2/ and mask (hide) subdir1/subdir2/ with an .htaccess file in webroot/ (See note 2 below)

Notes

(1) Tested with this .htaccess file in webroot (from scenario 3 above):

# Send everything else to Drupal.
RewriteRule ^$ subdir1/$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ subdir1/$1 [L]

(2) Tested with this .htaccess file in webroot (from scenario 5 above):

# Send everything else to Drupal.
RewriteRule ^$ subdir1/subdir2/$1 [L]
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ subdir1/subdir2/$1 [L]

(3) With scenarios 3 and 5...

     - When accessing install.php, the subdirectory(s) are not masked in the URL.
     - When accessing install.php or update.php,viewing source in the browser reveals the install path for all links (css, js, etc.).
     - You must clear the cache (or drush cr ) after the installation.

None of these affect the functionality of the site, and the minor issues with install.php and update.php are expected given the other known, related issues.

I also tested with Pathauto and Redirect installed and configured -- no problems.

My expectations are exceedingly low.

Regarding #13...

Calls to /somepath/core/authorize.php, /somepath/core/install.php, and /somepath/core/rebuild.php are made directly -- they bypass the front controller (/somepath/index.php). In these cases, Symfony reports the base path as /somepath/core instead of /somepath. So, if the last five characters of $request->getBasePath() are '/core' we chomp that bit off to set the the globals ($base_path, $base_url, etc.).

We don't need to special case update.php because it is located in Drupal's base directory, not in /somepath/core.

In the long run we want to remove those globals. But if we need to fix $base_url in the meantime, then let's do it.

1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
   @@ -1028,33 +1028,16 @@ protected function initializeSettings(Request $request) {
      protected function initializeRequestGlobals(Request $request) {
   -    global $base_url;
   -    // Set and derived from $base_url by this function.
   -    global $base_path, $base_root;
   +    global $base_path, $base_root, $base_url;
   

   This looks like you want to introduce a new global on first sight. For the sake of reviewers sanity, please let's not mix functional and cosmetic changes.

2. +++ b/core/lib/Drupal/Core/DrupalKernel.php
   @@ -1028,33 +1028,16 @@ protected function initializeSettings(Request $request) {
   +    if (substr($request->getBasePath(), -5) == '/core') {
   +      $base_path = substr($request->getBasePath(), 0, strlen($request->getBasePath()) - 4);
        }
        else {
   -      $base_path = '/';
   +      $base_path = $request->getBasePath() . '/';
        }
   

   We might want to switch the logic here, first assign $base_path unconditionally and then remove the core suffix if necessary.

3. +++ b/core/lib/Drupal/Core/DrupalKernel.php
   @@ -1509,3 +1492,4 @@ protected function addServiceFiles(array $service_yamls) {
   +
   

   No.

Oh, and thanks for the list in #8, this is really helpful.

Patch modified per znerol's review in #16.

The test failures in shown in comments #19 and #20 are examples of #2749955: Random fails in UpdatePathTestBase tests. A retest of the #18 patch returned no errors.

This still needs test coverage.

+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -1046,32 +1046,15 @@ protected function initializeSettings(Request $request) {
-      // Remove "core" directory if present, allowing install.php,
-      // authorize.php, and others to auto-detect a base path.
-      $core_position = strrpos($dir, '/core');
-      if ($core_position !== FALSE && strlen($dir) - 5 == $core_position) {
-        $base_path = substr($dir, 0, $core_position);
-      }
...
+    $base_path = $request->getBasePath();
+    if (substr($base_path, -5) == '/core') {
+      $base_path = substr($base_path, 0, strlen($base_path) - 5);

Let's keep the existing documentation.

1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
   @@ -1046,32 +1046,15 @@ protected function initializeSettings(Request $request) {
   -    // Set and derived from $base_url by this function.
   ...
   -      // Remove "core" directory if present, allowing install.php,
   -      // authorize.php, and others to auto-detect a base path.
   

   I agree with @dawehner we shouldn't remove the comments.

2. +++ b/core/tests/Drupal/Tests/Core/DrupalKernel/DrupalKernelTest.php
   @@ -13,6 +13,8 @@
   +    protected $backupGlobals = FALSE;
   +
   

   Why do we set this for all the tests? At the very least this needs documenting.

3. The fix looks good and brings some consistency - it's nice to be use a less complex code path here. I think the test coverage tests everything - it'd be nice to see a test-only patch on the issue.

Three of the four comments in initializeRequestGlobals(Request $request) are no longer applicable, and the fourth needed some clarification.

@m1r1k - Can you address points 2 and 3 of #28?

Patch #29 resolves an issue I was having when trying to access "/user" and getting "Redirects to external URLs are not allowed by default, use \Drupal\Core\Routing\TrustedRedirectResponse for it." +1 for getting this issue wrapped up and code committed.

#29 no longer applies.

Also still needs to address #28.

Rerolled.

Small fix for the #29 with notices from #28
Please check

The method name in test was the issue

Fix for provider as well

I though this was causing #2826137, but having tried it in detail (see #2826137-22: CDN is serving files with wrong path), this seems to work fine: $request->getSchemeAndHttpHost() . $request->getBasePath() === $GLOBALS['base_url'], both when D8 is installed in the root of a domain and in a subdirectory.

So it's not clear to me at all what the state of this issue is. Can we get a failing test, i.e. a test that shows what the problem is in HEAD? The steps to reproduce in #7 no longer work because CSS/JS asset URLs are now always root-relative, since #1494670: References to CSS, JS, and similar files should be root-relative URLs: avoids mixed content warnings & fewer bytes to send.

Re-roll, same contents except for line numbers really, but composer was being tetchy about it.
Will try to detail full use case for at least one required situation tomorrow.

Reroll of #49.

We've trying to have a proxy (nginx) that redirects the traffic to several applications:
- mydomain.com => a nextjs app that uses Drupal content to render the frontend application
- mydomian.com/drupal => the D9 site

We do not see any "easy and no hacky" way to tell D9 that it's under a subfolder and behind a proxy.

I think this is the issue that might fix this, is there any chance that these patches go further?

Thank u in any case for the great product and effort.

@jherencia: Is the D9 site within a "drupal/" URL prefix on the origin server? If not, can you make it be using something like the trick in https://medium.com/@lmakarov/drupal-8-and-reverse-proxies-the-base-url-d...? If it is, then it should work just fine without any of the patches that are in this issue, unless I'm misunderstanding your setup.

Thank you @effulgentsia.

We also had to create symlinks inside of the web site document roots to map the suffixes.

...

We made Drupal think it’s installed in a subdirectory. This way, it was including the country suffix in the absolute URLs it generated:

This is what I called "little tricky / hacky" way, and it didn't work either. I prefer not to share the configuration details in this thread as it would make noise to other people.

I think Drupal should have compatibility with "not so common" configurations. Thank you anyway :)

Hi @jherencia, Have you found solution for this #54?

Re-rolled patch against 9.4.x.

Fixed custom command failure errors in #60.

On fixing this 170 | ERROR | [x] Line indented incorrectly; expected 4 spaces, found 2, the following errors popped up:

 172 | ERROR | [ ] If the line declaring an array spans longer than 80 characters, each element should be broken into its own line
 174 | ERROR | [ ] If the line declaring an array spans longer than 80 characters, each element should be broken into its own line
 175 | ERROR | [x] Array closing indentation error, expected 4 spaces but found 2

Thanks!

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Rerolled #61

Using MR and looks really

#45 tagged for an issue summary update which still needs to happen it appears.