Problem/Motivation
Since #942690: Security harden stream wrappers by defaulting them as remote
\Drupal\Core\StreamWrapper\PrivateStream and \Drupal\Core\StreamWrapper\PublicStream
have:
/**
* {@inheritdoc}
*/
public static function getType() {
return StreamWrapperInterface::LOCAL_NORMAL;
}
This means public and private file uploads could be included as PHP files. This is a security weakness that seems to have been added by mistake and shoudl be remediated.
Also, StreamWrapperInterface uses class constants which , before PHP 5.6.0 do not support expressions so unlike D7 they are calculated, and it' not clear they are correct in all cases.
D7 uses defines and so it does for NORMAL:
define('STREAM_WRAPPERS_WRITE_VISIBLE', STREAM_WRAPPERS_READ | STREAM_WRAPPERS_WRITE | STREAM_WRAPPERS_VISIBLE)
We should add documentation and tests to verify these calculations are correct.
Proposed resolution
Add tests to insure that each constant that is a combination is correctly calculated to match the component parts.
Remaining tasks
Add tests
review
User interface changes
none
API changes
change stream wrapper types back to not be "local" to avoid possibly security vulnerabilities.
Data model changes
none
Reported by chx as part of the Drupal 8 security bug bounty
https://tracker.bugcrowd.com/submissions/58ebf88424951cadfcd1c3146d6dfc2...
Beta phase evaluation
| Issue category | Bug because it's restoring more secure defaults (regression) from D7. Also improving documentation and adding tests |
|---|---|
| Issue priority | Major since this is a bug with security implcations |
| Prioritized changes | The main goal of this issue is security |
| Disruption | Not disruptive for core/contributed and custom modules/themes |
| Comment | File | Size | Author |
|---|---|---|---|
| #59 | 2514644-patch-apply-fail.txt | 132 bytes | needs-review-queue-bot |
| #42 | increment.txt | 837 bytes | pwolanin |
| #42 | 2514644-42.patch | 6.69 KB | pwolanin |
| #37 | interdiff-2514644-31to37.txt | 2.19 KB | davidhernandez |
| #37 | file_inclusion_weakness-2514644-37.patch | 6.68 KB | davidhernandez |
Comments
Comment #1
pwolanin commentedComment #2
pwolanin commentedComment #3
pwolanin commentedQuick start on docs patch based on the D7 defines.
Comment #4
pwolanin commentedWith new test coverage.
Comment #5
pwolanin commentedComment #6
pwolanin commentedMoving this to documentation since the only code affected is a test, while the core code only doc is affected
Comment #7
pwolanin commentedAh - discussed with chx and we found the real bug - in the last conversion the public and private stream wrappers were overridden to LOCAL_NORMAL instead of being left as NORMAL
This could indeed allow other bugs to be leveraged into an arbitrary code execution bug
Comment #8
pwolanin commentedThe D7 default for public, private, etc was:
Comment #9
pwolanin commentedComment #10
pwolanin commentedComment #11
pwolanin commentedComment #12
chx commentedSo to clarify, when I began my stream wrapper work I saw that public was registered with a type 29 and that immediately screamed security weakness because it was LOCAL. I then miscalculated (opsie) 1C to be 29 so I filed on bugcrowd saying "constants are wrong". Well the constants are not wrong but documenting and testing them is good.
However, this didn't change the fact that registering public:// was happening as a local wrapper and so after I saw this test pass I looked into PublicStream a bit late and saw the getType methods. I think the "local stream" class name have caused some confusion here just because we store something on a local disk does not mean it should not be registered as a (remote) URL wrapper. Yay for PHP and its clear, easy to use and understand security facilities.
Comment #13
pwolanin commentedThe original change to use LOCAL was actually in #942690: Security harden stream wrappers by defaulting them as remote.
effulgentsia seems to think that improved security, but as best as I can see, it actually didn't achieve that goal
Trying to get his feedback though.
Comment #14
pwolanin commentedSo, further discussion with chx - we should probably consider a backport removal of LOCAL to D7 also.
The PHP option allow_url_fopen is true by default which means public:// should work fine with functions like file_get_contents() even if it's flagged as a "STREAM_IS_URL" , so there doesn't seem to be any real advantage to also allowing file inclusion by for uploaded files.
Comment #15
pwolanin commenteddiscussing with effulgentsia - since we don't currently require PHP to be configured as allow_url_fopen=1 in php.ini, this is not the right solution.
The change is also working against the code in the GD image toolkit:
Comment #16
effulgentsia commentedNope, even before that issue, public, private, and all other wrappers that didn't explicitly specify a type, defaulted to STREAM_WRAPPERS_NORMAL, which before that issue, was defined as
STREAM_WRAPPERS_LOCAL | STREAM_WRAPPERS_WRITE_VISIBLE.What do we do for servers where php.ini defines
allow_url_fopen=0?I'm nervous about that. image_gd_save() checks
file_get_stream_wrappers(STREAM_WRAPPERS_LOCAL), and so I imagine some contrib modules do as well, especially with file_get_stream_wrappers() documentation having that as an example.Comment #17
pwolanin commentedThis might be a more sensible approach? Let's make sure GD is ok.
Comment #18
pwolanin commentedbetter without the debug code...
Comment #20
chx commentedMaybe but that needs a requirement warning because we can't make your site less secure w/o a warning. Also who switches off url fopen...?
Comment #21
pwolanin commentedhow about this? Also more DRY
Comment #22
chx commentedboth need better wording: PHP doesn't allow opening URLs as file. And more importantly it reduces the site's security (because we are forced to register public:// as local yadda, yadda).
Comment #23
pwolanin commentedok, maybe we need to find someone to wordsmith
Comment #24
davidhernandezThis is only visible on the install page, correct?
How about...
Your server's PHP configuration has allow_url_include turned on. This can negatively impact your site's security, and you are strongly encouraged to turn it off. To disable this feature, set allow_url_include to 0 in php.ini.
Your server's PHP configuration has allow_url_fopen turned off. This disables the ability to retrieve files using a URL, and may reduce your site's functionality. To enable this feature, set allow_url_fopen to 1 in php.ini.
Comment #25
pwolanin commentedIt will be visible also in the status report and during updates.
Comment #26
pwolanin commented@chx - is there really a case where an attacker would be able to include public:// and not able to force a simple filepath or file:// ?
Comment #27
chx commentedSite builders find a way. Never underestimate what people can come up with.
Comment #28
pwolanin commented@chx - sure, I'm just wondering if it's worth emphasizing the security element in the UI.
Comment #29
chx commentedIt is. Security is always good to emphasize.
Comment #30
pwolanin commentedDavid's wording with minor tweaks.
Comment #31
davidhernandezIt looks like the only thing that was changed was quoting the value. Was that a mistake?
I added my text from above and quoted the values. (I'm not too sure the quotes are necessary, though.)
Comment #32
pwolanin commentedHmm, maybe I posted the wrong patch or something. For the second one seems chx is saying we should at least mentioned that there can be a security impact.
The values are shown quoted here: http://php.net/manual/en/filesystem.configuration.php, so that seemed reasonable to match.
Comment #33
davidhernandez@pwolanin, do you mean add wording to indicate the suggestion of enabling allow_url_fopen has security implications?
Comment #34
pwolanin commented@davidhernandez - yes, since by treating public files as being accessed by a URL, we cannot ever include them as PHP code via the stream wrapper.
Comment #35
pwolanin commentedLet me try again to clarify:
The patch changes the behavior of the public:// stream wrapper.
The result of the change is that if you have allow_url_fopen enabled we can tell PHP that public:// is a "URL".
The way PHP behaves is if public:// is a "URL" and you have allow_url_include disabled then it's impossible (we hope) to include PHP code via something like
include("public://foo.php");As a result, there is a potential (modest) security gain to having allow_url_fopen on as long as allow_url_include is off. This is what chx would like to see mentioned.
Again, that gain is due to the changes in the patch, it doesn't exist in HEAD
Comment #36
pwolanin commentedwe want to say that you should use the PHP defaults, which is allow_url_fopen enabled but _include NOT enabled
Comment #37
davidhernandezI nested the checks since _include requires _fopen to even function. A warning about _include with _fopen off is not needed. Added a sentence to help clarify. (I think) It is a bit tough to make it succinct, which I think is important given how long this text is getting. Lets just make sure the text is reasonable. The code change is more important than the message.
Comment #38
pwolanin commentedThese settings are pretty hard to explain. I think this is fine unless we want to go as far as setting an ERROR for the _include being enabled.
Comment #39
pwolanin commentedLet's go with this
Comment #40
alexpottNo one seems to have answered @effulgentsia's question from #16
How does this work now?
in core/modules/system/src/Plugin/ImageToolkit/GDToolkit.php
Comment #41
alexpottelse {should be on a new line.Comment #42
pwolanin commented@alexpott - we still mark e.g. public as StreamWrapperInterface::LOCAL_NORMAL so the GD code will work fine. In this case local is meaning really what we expect, that's it's on the same server.
Comment #47
mile23Is this fixed by https://www.drupal.org/SA-CORE-2017-003 ?
Comment #48
wim leers#47: No. See http://cgit.drupalcode.org/drupal/commit/?id=c732355412b84a6f7079d92b402....
Comment #49
wim leersComment #59
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".
Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.