Problem/Motivation
Symfony (and therefore Drupal 8) supports 5 different proxy headers:
https://github.com/symfony/http-foundation/blob/3.4/Request.php#L660
* * Request::HEADER_CLIENT_IP: defaults to X-Forwarded-For (see getClientIp())
* * Request::HEADER_CLIENT_HOST: defaults to X-Forwarded-Host (see getHost())
* * Request::HEADER_CLIENT_PORT: defaults to X-Forwarded-Port (see getPort())
* * Request::HEADER_CLIENT_PROTO: defaults to X-Forwarded-Proto (see getScheme() and isSecure())
* * Request::HEADER_FORWARDED: defaults to Forwarded (see RFC 7239)
By default, any and all of these are trusted.
In this context "trusted" means that \Symfony\Component\HttpFoundation\Request's "getter" methods will read values from the headers if they're present in the request.
Drupal has a setting which corresponds to the name of each of these headers, in order that they can be customised e.g. if a CDN uses a different name, e.g.:
https://cgit.drupalcode.org/drupal/tree/sites/default/default.settings.p...
/**
* Set this value if your proxy server sends the client IP in a header
* other than X-Forwarded-For.
*/
# $settings['reverse_proxy_header'] = 'X_CLUSTER_CLIENT_IP';
Symfony provides a way of disabling any of the headers that are not being used, and therefore should not be trusted:
https://github.com/symfony/http-foundation/blob/3.4/Request.php#L671
Setting an empty value allows to disable the trusted header for the given key.
This is also true of D8's settings for the header names; setting an empty value effectively tells Symfony to ignore that header, so that it is no longer "trusted".
However, this is not carried through into the D8 documentation yet.
Proposed resolution
The comments in default.settings.php should illustrate how to disable any proxy headers which are not in use, and therefore should not be "trusted" when determining the properties of an incoming request.
Remaining tasks
* Provide a patch for default.settings.php
* Review the patch.
* Commit the patch.
* Create follow-up issue to add more (functional?) tests for these headers: #3025077: Improve testing of Trusted Proxy Headers
User interface changes
None.
API changes
None.
Data model changes
None.
Release notes snippet
tbc (could possibly use some of the problem summary from above)
| Comment | File | Size | Author |
|---|---|---|---|
| #27 | interdiff-3017957-21-27.txt | 1.91 KB | mcdruid |
| #27 | 3017957-27.patch | 2.92 KB | mcdruid |
| #21 | 3017957-21.patch | 2.45 KB | mcdruid |
Comments
Comment #2
mcdruid commentedPatch which adds details of how to disable unused proxy headers to default.settings.php
Comment #3
mcdruid commentedComment #4
anavarreComment #5
mcdruid commentedJust noticed that the description of the settings for the last 3 headers are copy-pasta from the X-Forwarded-Proto one, so they all say:
...which we could fix while we're here too. Unless anyone thinks that's worth a separate follow-up issue?
Comment #6
gg4 commentedComment #7
mcdruid commentedNew patch which fixes the previously mentioned copy pasta error.
You could argue this should be a separate issue, but it's fixing one word in each of the three comments we're already changing here.
Comment #8
gg4 commentedIs there an opportunity to add additional test coverage for instances where these
$settingsare set to''?Comment #9
mcdruid commentedAFAICS there's some basic unit tests of the settings in \Drupal\Tests\Core\StackMiddleware\ReverseProxyMiddlewareTest
However, I don't think there is any test coverage for how these headers actually function, unless I'm missing something.
Comment #10
gg4 commentedThe documentation changes LGTM, though.
Comment #11
mcdruid commentedDiscussed this with @bonus and we agreed that test coverage in core for disabling trusted proxy headers via setting empty values for these settings would be good, as would more test coverage for the headers in general.
However, we agreed that we wouldn't want the documentation changes in this issue to be held up on tests.
Comment #12
mcdruid commentedComment #13
jhodgdonThoughts on this issue/patch:
a) Make a follow-up issue now to add tests as in #9/12, and add it to the issue summary, so that the question doesn't come up again.
b) Grammar nitpick:
A selection ... is supported
This of course sounds weird, so maybe better as:
Various reverse proxy headers are supported...
c) E.g. nitpick:
E.g. means "for example", which I think is not what you mean (maybe it is?). Anyway, best to be specific. Why not just say
which is shorter, and gets the point across, without using "e.g.", which is often misunderstood and confused with "i.e.", and is also punctuated incorrectly in your text. We actually have a docs writing style somewhere that suggested never using "e.g." and "i.e." in docs, because of them being so commonly confused with each other.
d) Item (c) applies in a bunch of places.
e) Other than those nitpicks, this looks very clear. Thanks for the issue and patch!
Comment #14
mcdruid commentedComment #15
mcdruid commentedComment #16
mcdruid commentedThanks for the review; some really good suggestions.
a) Done: #3025077: Improve testing of Trusted Proxy Headers
b) Done; your simplified wording's better.
c) I _think_ I do mean "for example" here, specifically because there are a number of different ways to assign an "empty value" and two single quotes is only one of them. I was trying to avoid being too prescriptive.
However, I agree that shorter and more to the point is better and that using "e.g." a lot could cause confusion. I've changed the first instance to explicitly use "for example" (because - as mentioned - using '' is one of several valid ways of assigning an empty value), but simplified the subsequent sentences for the sake of brevity and clarity.
d) Done (as detailed above).
e) Thank you!
Comment #17
mcdruid commentedThinking about it, I'm not so sure that there are really "several" without getting very convoluted and straying a long way from our coding standards! I'm pretty comfortable just saying set this to '' as suggested, but personally I'd still stick with the initial "for example" for the reason outlined. (Both are implemented in the most recent patch, so I'm not suggesting any further changes, just expanding on my reasoning a little).
Comment #18
gg4 commentedThese changes still LGTM.
Comment #19
jhodgdonIf you're going to use either "for example" or "e.g.", we should get the punctuation correct. It should be:
To do this, set the corresponding setting to an empty value; for example, ''.
But anyway... is it really checked using the PHP
empty()function? Or just evaluated as a Boolean? Or is it checked to see if it is== ''? Or ??? Can you guarantee that -- how is it documented and where? Because if you say "setting to an empty value", that implies to me that it is really being checked with empty(), but I'm not sure it really is.I really think you should just say to set it to '' and leave it at that, since you know that works, and why, really, would you want to provide alternatives?
Comment #20
jhodgdonAs another note...
https://github.com/symfony/http-foundation/blob/3.4/Request.php#L671
shows the @param for the $key and $value are both string variables. So ... yes, the text in the doc block does say "setting to an empty value", but the fact that the @params are typed as string implies they should be strings, hence '' is really the way to say "don't use this".
Comment #21
mcdruid commentedI haven't scrutinised the types all the way along the path from these Drupal settings into Symfony, but anyway - good point, well made.
Looking elsewhere in
default.settings.phpthere is:...which seems to be to be a very similar scenario.
Here's a new patch which uses that wording in the initial explanation, and keeps the simple wording for the specific settings.
Comment #22
o'briatThere's another issue about the wording of the reverse_proxy_headers: https://www.drupal.org/project/drupal/issues/2673572
Comment #23
cilefen commentedComment #24
jhodgdonThe docs look good here to me! Mostly... Two questions/thoughts:
a)
Is the default here actually x-forwarded-for? Because in other settings, such as
it looks to me as though the default setting is put into the commented-out code line. But in the reverse_proxy_header line, if the docs are correct, the default isn't the setting in the code line.
I think we should be consistent, and always put the defaults into the code line. So, in this first doc block, either the docs are wrong or the code line is wrong (I have no idea which it would be -- not my area of expertise).
b) We do have a documentation standard that says all 1st-line docs in /** */ doc blocks should be one sentence of less than 80 characters, followed by a blank line, and optionally some more documentation. None of the items in this patch qualify. I realize the docs in this file didn't follow the standard before the patch either, but it is normal if patching an area of docs to also make it comply with the docs standards.
Comment #25
mcdruid commenteda) I wonder if you felt a sense of déjà vu when you pondered this?! See #2673572-4: Improve reverse proxy documentation in default.settings.php
I suspect the reason it's different to the others is that
reverse_proxy_headerhas been in Drupal for ages, whereas the other settings came along with Symfony.I'm all for consistency, and I'd be happy to change this example so that it contains the default like the other examples do.
b) I'll look at reformatting all of the comments we're touching here to adhere to the docs standards you mention.
Comment #26
o'briatDon' forget to fix descriptions, they are still copy/past typos in your last patch ("sends the client protocol in a header"), see #2673572 for more detailed descriptions.
Comment #27
mcdruid commentedNew patch which changes the commented-out example for
X-Forwarded-Forto the default of'X_FORWARDED_FOR'to match the other settings.Also (hopefully) implemented the docs standard of one <80 char sentence, blank line, and then the rest of the explanation for each setting. I've gone with the word "Client" for most of the first lines; we could use "Forwarded" or "Originating" or something else, but I think "Client" works.
@O'Briat thanks, but I think I'd already fixed the "client protocol" which was copy-pasta'd into every setting, so that we now have "client host", "client port" etc...
Comment #28
jhodgdonThis looks very good to me! I think you can see from the patch why our docs standards ask for one < 80 character line to start each documentation header -- it really makes the documentation easier to scan and understand.
Comment #30
jhodgdonThat seems to have been a random test failure. Created issue #3031842: Random fail in OffCanvasTest and will submit for retest.
Comment #32
jhodgdonSame random test fail again. :(
Comment #34
mcdruid commentedOffCanvasTest snafu again; back to RTBC.
Comment #35
o'briatIt should be add that reverse_proxy_addresses allow subnet notation and IPV6 address.
@see
\Symfony\Component\HttpFoundation\Request::isFromTrustedProxy => \Symfony\Component\HttpFoundation\IpUtils::checkIp/checkIp4/checkIp6 : "IPv4 address or subnet in CIDR notation"
Suggestion:
Comment #36
jhodgdonAdding this information might be a good idea, but I am not sure... What is CIDR? I am not familiar with that acronym. Is it different from the usual IP address notation?
If CIDR is just the normal IP address notation, then I think this added line is probably not needed. The line above says you need to put in every reverse proxy IP address in the environment, and the example setting line shows that it is an array of strings in 'a.b.c.d' notation... ???
Comment #37
mcdruid commented@O'Briat that's interesting!
However I'd vote for a separate follow-up for that change as this issue was really about including extra details about the settings which correspond to the supported proxy headers which are not in use.
Comment #38
o'briat@jhodgdon: The array in the example is confusing, it could induce that only a list of IP is a valid input, i.e. : no subnet notation.
Another source of confusion is that Drupal 7 does not allow such subnet notation.
As for CIDR, it's taken from the symfony comment, plus, at this point of the settings the user should have the needed knowledge.
@mcdruid : Done: #3032746
Comment #39
alexpottI'm going to postpone this on #3030501: [Symfony 4] Drupal\Core\StackMiddleware\ReverseProxyMiddleware calls Symfony\Component\HttpFoundation\Request::setTrustedHeaderName() which does not exist as that patch fixes the implementation to accord for Symfony's deprecations and changes this significantly.
Comment #44
anybodyAdded a closely related issue. Furthermore, I guess the documentation should point out that the default checks for HTTP_X variables. The current doc at least didn't make that clear to me.
I checked the $_SERVER variables which showed me "HTTP_X_FORWARDED_FOR" as key for example.
So I tried to override the defaults by:
which was wrong, instead I should have used the defaults, but the documentation says:
So finally I'd suggest to explicitely add the key name to the documentation?:
Comment #47
pavlosdanLooks like the ticket this was postponed on has landed. Setting back to needs review in case things changed since.
Comment #50
jcnventuraIt would be good to show the same info for reverse_proxy_addresses as Symfony have in their docs: https://symfony.com/doc/current/deployment/proxies.html#solution-settrus...
Maybe an option would be document it as
Comment #51
mfb@jcnventura There's a dedicated issue for that at #3032746: Improve documentation for reverse proxy addresses setting - I added your suggestion there as a patch
Comment #52
mfbI don't see anything in this patch that is still relevant now that
$settings['reverse_proxy_trusted_headers']is used, so closing - please reopen if I missed something