Problem/Motivation

Symfony (and therefore Drupal 8) supports 5 different proxy headers:

https://github.com/symfony/http-foundation/blob/3.4/Request.php#L660

*  * Request::HEADER_CLIENT_IP:    defaults to X-Forwarded-For   (see getClientIp())
*  * Request::HEADER_CLIENT_HOST:  defaults to X-Forwarded-Host  (see getHost())
*  * Request::HEADER_CLIENT_PORT:  defaults to X-Forwarded-Port  (see getPort())
*  * Request::HEADER_CLIENT_PROTO: defaults to X-Forwarded-Proto (see getScheme() and isSecure())
*  * Request::HEADER_FORWARDED:    defaults to Forwarded         (see RFC 7239)

By default, any and all of these are trusted.

In this context "trusted" means that \Symfony\Component\HttpFoundation\Request's "getter" methods will read values from the headers if they're present in the request.

Drupal has a setting which corresponds to the name of each of these headers, in order that they can be customised e.g. if a CDN uses a different name, e.g.:

https://cgit.drupalcode.org/drupal/tree/sites/default/default.settings.p...

/**
 * Set this value if your proxy server sends the client IP in a header
 * other than X-Forwarded-For.
 */
# $settings['reverse_proxy_header'] = 'X_CLUSTER_CLIENT_IP';

Symfony provides a way of disabling any of the headers that are not being used, and therefore should not be trusted:

https://github.com/symfony/http-foundation/blob/3.4/Request.php#L671

Setting an empty value allows to disable the trusted header for the given key.

This is also true of D8's settings for the header names; setting an empty value effectively tells Symfony to ignore that header, so that it is no longer "trusted".

However, this is not carried through into the D8 documentation yet.

Proposed resolution

The comments in default.settings.php should illustrate how to disable any proxy headers which are not in use, and therefore should not be "trusted" when determining the properties of an incoming request.

Remaining tasks

* Provide a patch for default.settings.php
* Review the patch.
* Commit the patch.
* Create follow-up issue to add more (functional?) tests for these headers: #3025077: Improve testing of Trusted Proxy Headers

User interface changes

None.

API changes

None.

Data model changes

None.

Release notes snippet

tbc (could possibly use some of the problem summary from above)

Comments

mcdruid created an issue. See original summary.

mcdruid’s picture

StatusFileSize
new2.35 KB

Patch which adds details of how to disable unused proxy headers to default.settings.php

mcdruid’s picture

Issue summary: View changes
anavarre’s picture

Status: Active » Needs review
mcdruid’s picture

Just noticed that the description of the settings for the last 3 headers are copy-pasta from the X-Forwarded-Proto one, so they all say:

Set this value if your proxy server sends the client protocol in a header ...

...which we could fix while we're here too. Unless anyone thinks that's worth a separate follow-up issue?

gg4’s picture

Issue tags: +Security improvements
mcdruid’s picture

StatusFileSize
new2.57 KB
new1.24 KB

New patch which fixes the previously mentioned copy pasta error.

You could argue this should be a separate issue, but it's fixing one word in each of the three comments we're already changing here.

gg4’s picture

Is there an opportunity to add additional test coverage for instances where these $settings are set to ''?

mcdruid’s picture

AFAICS there's some basic unit tests of the settings in \Drupal\Tests\Core\StackMiddleware\ReverseProxyMiddlewareTest

However, I don't think there is any test coverage for how these headers actually function, unless I'm missing something.

gg4’s picture

The documentation changes LGTM, though.

mcdruid’s picture

Discussed this with @bonus and we agreed that test coverage in core for disabling trusted proxy headers via setting empty values for these settings would be good, as would more test coverage for the headers in general.

However, we agreed that we wouldn't want the documentation changes in this issue to be held up on tests.

mcdruid’s picture

Issue summary: View changes
jhodgdon’s picture

Status: Needs review » Needs work

Thoughts on this issue/patch:

a) Make a follow-up issue now to add tests as in #9/12, and add it to the issue summary, so that the question doesn't come up again.

b) Grammar nitpick:

 * A selection of different reverse proxy headers are supported (see below), but

A selection ... is supported

This of course sounds weird, so maybe better as:

Various reverse proxy headers are supported...

c) E.g. nitpick:

+ * corresponding setting to an empty value, e.g. ''.

E.g. means "for example", which I think is not what you mean (maybe it is?). Anyway, best to be specific. Why not just say

corresponding setting to ''.

which is shorter, and gets the point across, without using "e.g.", which is often misunderstood and confused with "i.e.", and is also punctuated incorrectly in your text. We actually have a docs writing style somewhere that suggested never using "e.g." and "i.e." in docs, because of them being so commonly confused with each other.

d) Item (c) applies in a bunch of places.

e) Other than those nitpicks, this looks very clear. Thanks for the issue and patch!

mcdruid’s picture

Issue summary: View changes
mcdruid’s picture

Issue summary: View changes
mcdruid’s picture

Status: Needs work » Needs review
StatusFileSize
new2.46 KB
new2.43 KB

Thanks for the review; some really good suggestions.

a) Done: #3025077: Improve testing of Trusted Proxy Headers

b) Done; your simplified wording's better.

c) I _think_ I do mean "for example" here, specifically because there are a number of different ways to assign an "empty value" and two single quotes is only one of them. I was trying to avoid being too prescriptive.

However, I agree that shorter and more to the point is better and that using "e.g." a lot could cause confusion. I've changed the first instance to explicitly use "for example" (because - as mentioned - using '' is one of several valid ways of assigning an empty value), but simplified the subsequent sentences for the sake of brevity and clarity.

d) Done (as detailed above).

e) Thank you!

mcdruid’s picture

... using '' is one of several valid ways of assigning an empty value

Thinking about it, I'm not so sure that there are really "several" without getting very convoluted and straying a long way from our coding standards! I'm pretty comfortable just saying set this to '' as suggested, but personally I'd still stick with the initial "for example" for the reason outlined. (Both are implemented in the most recent patch, so I'm not suggesting any further changes, just expanding on my reasoning a little).

gg4’s picture

Status: Needs review » Reviewed & tested by the community

These changes still LGTM.

jhodgdon’s picture

Status: Reviewed & tested by the community » Needs work

If you're going to use either "for example" or "e.g.", we should get the punctuation correct. It should be:

To do this, set the corresponding setting to an empty value; for example, ''.

But anyway... is it really checked using the PHP empty() function? Or just evaluated as a Boolean? Or is it checked to see if it is == ''? Or ??? Can you guarantee that -- how is it documented and where? Because if you say "setting to an empty value", that implies to me that it is really being checked with empty(), but I'm not sure it really is.

I really think you should just say to set it to '' and leave it at that, since you know that works, and why, really, would you want to provide alternatives?

jhodgdon’s picture

As another note...
https://github.com/symfony/http-foundation/blob/3.4/Request.php#L671
shows the @param for the $key and $value are both string variables. So ... yes, the text in the doc block does say "setting to an empty value", but the fact that the @params are typed as string implies they should be strings, hence '' is really the way to say "don't use this".

mcdruid’s picture

Status: Needs work » Needs review
StatusFileSize
new2.45 KB
new562 bytes

I haven't scrutinised the types all the way along the path from these Drupal settings into Symfony, but anyway - good point, well made.

Looking elsewhere in default.settings.php there is:

 * You can optionally set prefixes for some or all database table names
 * by using the 'prefix' setting. If a prefix is specified, the table
 * name will be prepended with its value. Be sure to use valid database
 * characters only, usually alphanumeric and underscore. If no prefixes
 * are desired, leave it as an empty string ''.

...which seems to be to be a very similar scenario.

Here's a new patch which uses that wording in the initial explanation, and keeps the simple wording for the specific settings.

o'briat’s picture

There's another issue about the wording of the reverse_proxy_headers: https://www.drupal.org/project/drupal/issues/2673572

cilefen’s picture

jhodgdon’s picture

The docs look good here to me! Mostly... Two questions/thoughts:

a)

/**
  * Set this value if your proxy server sends the client IP in a header
- * other than X-Forwarded-For.
+ * other than X-Forwarded-For. If you are using a proxy server but not using
+ * this header, set this to ''.
  */
 # $settings['reverse_proxy_header'] = 'X_CLUSTER_CLIENT_IP';

Is the default here actually x-forwarded-for? Because in other settings, such as

/**
  * Set this value if your proxy server sends the client protocol in a header
- * other than X-Forwarded-Proto.
+ * other than X-Forwarded-Proto. If you are using a proxy server but not using
+ * this header, set this to ''.
  */
 # $settings['reverse_proxy_proto_header'] = 'X_FORWARDED_PROTO';

it looks to me as though the default setting is put into the commented-out code line. But in the reverse_proxy_header line, if the docs are correct, the default isn't the setting in the code line.

I think we should be consistent, and always put the defaults into the code line. So, in this first doc block, either the docs are wrong or the code line is wrong (I have no idea which it would be -- not my area of expertise).

b) We do have a documentation standard that says all 1st-line docs in /** */ doc blocks should be one sentence of less than 80 characters, followed by a blank line, and optionally some more documentation. None of the items in this patch qualify. I realize the docs in this file didn't follow the standard before the patch either, but it is normal if patching an area of docs to also make it comply with the docs standards.

mcdruid’s picture

Assigned: Unassigned » mcdruid
Status: Needs review » Needs work

a) I wonder if you felt a sense of déjà vu when you pondered this?! See #2673572-4: Improve reverse proxy documentation in default.settings.php

I suspect the reason it's different to the others is that reverse_proxy_header has been in Drupal for ages, whereas the other settings came along with Symfony.

I'm all for consistency, and I'd be happy to change this example so that it contains the default like the other examples do.

b) I'll look at reformatting all of the comments we're touching here to adhere to the docs standards you mention.

o'briat’s picture

Don' forget to fix descriptions, they are still copy/past typos in your last patch ("sends the client protocol in a header"), see #2673572 for more detailed descriptions.

mcdruid’s picture

Assigned: mcdruid » Unassigned
Status: Needs work » Needs review
StatusFileSize
new2.92 KB
new1.91 KB

New patch which changes the commented-out example for X-Forwarded-For to the default of 'X_FORWARDED_FOR' to match the other settings.

Also (hopefully) implemented the docs standard of one <80 char sentence, blank line, and then the rest of the explanation for each setting. I've gone with the word "Client" for most of the first lines; we could use "Forwarded" or "Originating" or something else, but I think "Client" works.

@O'Briat thanks, but I think I'd already fixed the "client protocol" which was copy-pasta'd into every setting, so that we now have "client host", "client port" etc...

jhodgdon’s picture

Status: Needs review » Reviewed & tested by the community

This looks very good to me! I think you can see from the patch why our docs standards ask for one < 80 character line to start each documentation header -- it really makes the documentation easier to scan and understand.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 27: 3017957-27.patch, failed testing. View results

jhodgdon’s picture

Status: Needs work » Reviewed & tested by the community

That seems to have been a random test failure. Created issue #3031842: Random fail in OffCanvasTest and will submit for retest.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 27: 3017957-27.patch, failed testing. View results

jhodgdon’s picture

Status: Needs work » Reviewed & tested by the community

Same random test fail again. :(

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 27: 3017957-27.patch, failed testing. View results

mcdruid’s picture

Status: Needs work » Reviewed & tested by the community

OffCanvasTest snafu again; back to RTBC.

o&#039;briat’s picture

It should be add that reverse_proxy_addresses allow subnet notation and IPV6 address.
@see
\Symfony\Component\HttpFoundation\Request::isFromTrustedProxy => \Symfony\Component\HttpFoundation\IpUtils::checkIp/checkIp4/checkIp6 : "IPv4 address or subnet in CIDR notation"

Suggestion:

 /**
+ * Reverse Proxy Addresses.
+ *
  * Specify every reverse proxy IP address in your environment.
+ * String or array of IPV4/6 address or subnet in CIDR notation.
  * This setting is required if $settings['reverse_proxy'] is TRUE.
  */
 # $settings['reverse_proxy_addresses'] = ['a.b.c.d', ...];
jhodgdon’s picture

Adding this information might be a good idea, but I am not sure... What is CIDR? I am not familiar with that acronym. Is it different from the usual IP address notation?

If CIDR is just the normal IP address notation, then I think this added line is probably not needed. The line above says you need to put in every reverse proxy IP address in the environment, and the example setting line shows that it is an array of strings in 'a.b.c.d' notation... ???

mcdruid’s picture

@O'Briat that's interesting!

However I'd vote for a separate follow-up for that change as this issue was really about including extra details about the settings which correspond to the supported proxy headers which are not in use.

o&#039;briat’s picture

@jhodgdon: The array in the example is confusing, it could induce that only a list of IP is a valid input, i.e. : no subnet notation.
Another source of confusion is that Drupal 7 does not allow such subnet notation.

As for CIDR, it's taken from the symfony comment, plus, at this point of the settings the user should have the needed knowledge.

@mcdruid : Done: #3032746

alexpott’s picture

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

anybody’s picture

Added a closely related issue. Furthermore, I guess the documentation should point out that the default checks for HTTP_X variables. The current doc at least didn't make that clear to me.

I checked the $_SERVER variables which showed me "HTTP_X_FORWARDED_FOR" as key for example.

So I tried to override the defaults by:

$settings['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$settings['reverse_proxy_proto_header'] = 'HTTP_X_FORWARDED_PROTO';
$settings['reverse_proxy_host_header'] = 'HTTP_X_FORWARDED_HOST';
$settings['reverse_proxy_port_header'] = 'HTTP_X_FORWARDED_PORT';

which was wrong, instead I should have used the defaults, but the documentation says:

/**
* Set this value if your proxy server sends the client IP in a header
* other than X-Forwarded-For.
*/

So finally I'd suggest to explicitely add the key name to the documentation?:

/**
* Set this value if your proxy server sends the client IP in a header
* other than X-Forwarded-For ("HTTP_X_FORWARDED_FOR").
*/

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

pavlosdan’s picture

Status: Postponed » Needs review

Looks like the ticket this was postponed on has landed. Setting back to needs review in case things changed since.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

jcnventura’s picture

It would be good to show the same info for reverse_proxy_addresses as Symfony have in their docs: https://symfony.com/doc/current/deployment/proxies.html#solution-settrus...

Maybe an option would be document it as

 # $settings['reverse_proxy_addresses'] = ['a.b.c.d', 'e.f.g.h/24', ...];
mfb’s picture

@jcnventura There's a dedicated issue for that at #3032746: Improve documentation for reverse proxy addresses setting - I added your suggestion there as a patch

mfb’s picture

Status: Needs review » Closed (outdated)

I don't see anything in this patch that is still relevant now that $settings['reverse_proxy_trusted_headers'] is used, so closing - please reopen if I missed something