Problem/Motivation

Bug:
If you have multiple sites and if your Drupal is behind a reverse proxy, the matching site path is not resolved, the settings are not initialized, and you always end up on the install page (with wrong urls generated).

Why?
1) When the request is handled, one of the first function to be called is DrupalKernel->initializeSettings().
2) To initialize the settings, Drupal needs to determine the site path to read the settings.php file so the functionDrupalKernel::findSitePath() is called.
3) In this function, when you have multiple sites, the function Request->getHttpHost() is called on the request (the http host is needed to match it with the mapping (dir => http host) you have made in the sites.php file).

Problem:
1) The http host depends on the trusted proxies configuration (cf Symfony Request->getHttpHost() code).
2) The trusted proxies are only set by the ReverseProxyMiddleware middleware so they have not yet been set in the request. The container has not yet been initialized.

Note:
This cannot be resolved easily because there is a circular dependency (cf attached schema).
1) To find the site path, Drupal needs the valid http host (so the trusted proxies must have been set on the request).
2) To set the trusted proxies, the settings must have been initiliazed (because currently the reverse proxy settings are in the site settings.php file).
3) To initialize the site settings, Drupal needs the site path.

My two cents:
1) Setting the request trusted proxies in a HttpKernel middleware is a design error.
2) The reverse proxy settings can only be global and not by site. As long as Drupal depends on the http host to find the site, a configuration by site is not possible. I geuss that's ok for the vast majority of project using multiple sites and reverse proxy.

Proposed resolution

Ideas:
1) Set the trusted proxies on the Request as soon as possible. We need to make sure that the trusted proxies are set before any real usage of the Request. The safest solution is to do it at the very beginning of the request handle. If you look at Symfony doc about how to configure Symfony to work behind a reverse proxy (https://symfony.com/doc/current/deployment/proxies.html), they advise the user to set the trusted proxies directly in the front controller (so asap).
2) Deprecate the ReverseProxyMiddleware class.

Proposed patch:
Now:
1) Introduce a new configuration file sites/reverse_proxy_settings.php that behaves exactly like the current reverse proxy settings.
2) Introduce a new method DrupalKernel::setTrustedProxies() that is called as soon as the request is handled.
3) The new method reads the new configuration file and set the trusted proxies on the request.
4) The new method uses the ReverseProxyMiddleware::setSettingsOnRequest() method during the deprecation period (so the same configuration options are used and the behavior is exactly the same).
5) Deprecate the ReverseProxyMiddleware class and trigger a deprecation notice when the reverse proxy settings still come from the site settings.php file (this is the case when you have only one site).

Once the deprecation period is over:
1) Remove the ReverseProxyMiddleware class.
2) Copy the code of the ReverseProxyMiddleware::setSettingsOnRequest() method in the new DrupalKernel::setTrustedProxies() method.
3) Remove the call to ReverseProxyMiddleware::setSettingsOnRequest() in the install.core.inc file as it will not be needed anymore.

Alternatives:
1) Maybe introducing a new configuration file is not the best solution and that the reverse proxy configuration could be in the existing sites.php file directly?
2) Maybe we should remove all reverse proxy configuration from Drupal and and let the users manage the trusted proxy themselves in their front controller (like in Symfony projects).

Remaining tasks

1) Discuss the issue.
2) Review the proposed resolution and patch.
3) Eventually Write tests and update the documentation.

Going further:

Instead of creating a new specific file just for the reverse proxy settings, this could be an opportunity to create a global settings file that all sites settings would inherit. The settings of each site would be merged with the global ones. Those general settings could be used before the site path has been resolved for other purposes. It is already common to create a settings.common.php file that you require in all the sites settings.php files.

Issue fork drupal-2998728

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

fancyweb created an issue. See original summary.

fancyweb’s picture

StatusFileSize
new4.22 KB

Here is a simple patch.

fancyweb’s picture

Issue summary: View changes
fancyweb’s picture

Version: 8.7.x-dev » 8.6.x-dev
beram’s picture

Hi,

FYI your patch does not apply on a vanilla Drupal 8.6.x because it is started from core folder and not the docroot.

The following link could help you to create a patch for a Drupal core issue: https://www.drupal.org/contributor-tasks/create-patch.

mamoot’s picture

Hi,

The patch proposed from this issue #2972310: Reverse proxy settings not used doesn't work with this problematic because the call of ReverseProxyMiddleware::setSettingsOnRequest($request, Settings::getInstance()); is made after the initializeSettings() call.

A very simple solution is to alter or extends DrupalKernel and call Request::setTrustedProxies() with good IP's values.

Ex :
DrupalKernel.php

  /**
   * {@inheritdoc}
   */
  public function handle(Request $request, $type = self::MASTER_REQUEST, $catch = TRUE) {
      ...
      // line 657
      $request::setTrustedProxies([$request->server->get('REMOTE_ADDR')]); // your proxy IP's array
      ...

  }

It's not a viable solution, but it's KISS...

Nicolas

fancyweb’s picture

StatusFileSize
new5.45 KB
fancyweb’s picture

I have just updated the patch starting from the drupal/drupal project. It was also an oppotunity to add an example reverse_proxy_settings.php file.

mamoot’s picture

Hi,

This patch is a simple solution and fix effectively the problem with trusted proxies.

Use case
I applied this patch on 2 projects which are behind Nginx Reverse Proxy + Varnish on AWS with a multisites configuration and everything works as expected.
The load balancer which we use is ALB.

About the patch

I'm not convinced that add a another configuration files is necessary.
sites/reverse_proxy_settings.php is unuseful because, the action to set Trusted Proxies must be apply in the Front Controller, not at the instanciation of DrupalKernel.

Proxy are environmental and their configuration must be apply apart from DrupalKernel.

Symfony 4 front controller apply configuration on proxy before the Kernel instanciation :

<?php
...

if ($trustedProxies = $_SERVER['TRUSTED_PROXIES'] ?? false) {
    Request::setTrustedProxies(explode(',', $trustedProxies), Request::HEADER_X_FORWARDED_ALL ^ Request::HEADER_X_FORWARDED_HOST);
}

if ($trustedHosts = $_SERVER['TRUSTED_HOSTS'] ?? false) {
    Request::setTrustedHosts(explode(',', $trustedHosts));
}

$kernel = new Kernel($env, $debug);
$request = Request::createFromGlobals();
$response = $kernel->handle($request);
$response->send();
$kernel->terminate($request, $response);

Big thank's
Thank's @fancyweb for this patch and your pragmatic way to handle this problem.

beram’s picture

Hi

Thanks for the patch :)
It has been tested on production with a multisite Drupal instance and it works.
I have some comments about the patch and the comment of @mamoot that I'll add later since my comment is about another issue that this patch fixes too.

This patch fixes another issue as well:
if your Drupal is behind a reverse proxy the trusted host mechanism does not works.
When it checks if the host is parts of the trusted hosts, the reverse proxy settings are not set up yet.
The host retrieved by $request->getHost(); inside DrupalKernel::setupTrustedHosts() is the reverse proxy one.
It means that the host is not trustworthy.

Before adding this issue as a proper issue and linked it to this one with a comment saying "The patch from #2998728 comment #8 fix this issue as well":
I'll wanted to ask your opinion about creating a specific issue that will list all the issues about the reverse proxy settings not set up properly at the correct moment.
Its aim could be to set up a roadmap and centralized the discussion about it.

What do you think?

beram’s picture

StatusFileSize
new3.41 KB
new1014 bytes

I just fix a warning Warning: func_get_arg(): Argument 3 not passed to function on the patch provided in comment #8

beram’s picture

StatusFileSize
new5.11 KB
new5.08 KB
new5.08 KB

Reroll for 8.6.x, 8.7.x, 8.8.x
No interdiff needed.

beram’s picture

Issue tags: +DevDaysTransylvania
avpaderno’s picture

Issue tags: -reverse proxy, -drupal kernel, -request handling

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

larowlan’s picture

Status: Active » Needs work
Issue tags: +Needs reroll, +Bug Smash Initiative

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Prem Suthar made their first commit to this issue’s fork.

prem suthar’s picture

Status: Needs work » Needs review

Added the Mr For the #13 for re-roll the patch .

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new4.14 KB

The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

wim leers’s picture

I ran into this as well for https://wimleers.com after upgrading from Drupal 7 to 10. It made me pull my hair out for hours. Although I'm on a very atypical setup: local server at home, via non-standard port (ISP blocks port HTTP(S) ports) as HTTP, served by a CDN as HTTPS. At first, I just thought I was doing something wrong in my exotic (but dead simple!) setup, but it turns out to have been this long-standing Drupal core bug 🙈

Even though it's a single site! Because many years ago, it was considered a best practice to not use sites/default/files, but sites/example.com/files. Because I don't use sites/default, the same code path is triggered as for a multisite.

I didn't really need trusted proxies; I only needed the reverse proxy (Fastly) to be trusted, to ensure that the TLS termination at the Fastly edge was respected.

While I first hacked core (🙈), I then settled on something simpler: running this early enough, aka before Drupal's index.php is executed:

// When any non-local IP acccesses this, serve this as if it's the live site.
// (It should be a Fastly IP address, and it is possible to validate against that, but this suffices.)
// @see https://www.fastly.com/documentation/reference/api/utils/public-ip-list/
if (filter_var($_SERVER['REMOTE_ADDR'], FILTER_VALIDATE_IP, FILTER_FLAG_NO_PRIV_RANGE | FILTER_FLAG_NO_RES_RANGE)) {
  $_SERVER['HTTP_HOST'] = 'wimleers.com';
  // Fastly also does TLS termination for us, so make Drupal think it's accessed via https://.
  // @see https://docs.fastly.com/en/guides/tls-termination#when-using-wordpress
  $_SERVER['HTTPS'] = 'on';
}

(Because doing it in settings.php is too late.)

avpaderno’s picture

Assigned: fancyweb » Unassigned

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.