Problem/Motivation

Since #942690: Security harden stream wrappers by defaulting them as remote

\Drupal\Core\StreamWrapper\PrivateStream and \Drupal\Core\StreamWrapper\PublicStream

have:

  /**
   * {@inheritdoc}
   */
  public static function getType() {
    return StreamWrapperInterface::LOCAL_NORMAL;
  }

This means public and private file uploads could be included as PHP files. This is a security weakness that seems to have been added by mistake and shoudl be remediated.

Also, StreamWrapperInterface uses class constants which , before PHP 5.6.0 do not support expressions so unlike D7 they are calculated, and it' not clear they are correct in all cases.

D7 uses defines and so it does for NORMAL:

 define('STREAM_WRAPPERS_WRITE_VISIBLE', STREAM_WRAPPERS_READ | STREAM_WRAPPERS_WRITE | STREAM_WRAPPERS_VISIBLE)

We should add documentation and tests to verify these calculations are correct.

Proposed resolution

Add tests to insure that each constant that is a combination is correctly calculated to match the component parts.

Remaining tasks

Add tests
review

User interface changes

none

API changes

change stream wrapper types back to not be "local" to avoid possibly security vulnerabilities.

Data model changes

none

Reported by chx as part of the Drupal 8 security bug bounty

https://tracker.bugcrowd.com/submissions/58ebf88424951cadfcd1c3146d6dfc2...

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because it's restoring more secure defaults (regression) from D7. Also improving documentation and adding tests
Issue priority Major since this is a bug with security implcations
Prioritized changes The main goal of this issue is security
Disruption Not disruptive for core/contributed and custom modules/themes

Comments

pwolanin’s picture

Issue summary: View changes
pwolanin’s picture

Priority: Major » Normal
Issue summary: View changes
pwolanin’s picture

Status: Active » Needs review
StatusFileSize
new1.87 KB

Quick start on docs patch based on the D7 defines.

pwolanin’s picture

Title: StreamWrapperInterface constants are wrong allowing malicious public:// include » StreamWrapperInterface composite constants are not clearly documented or tested for correctness
Category: Bug report » Task
Issue tags: -Security, -Needs tests
StatusFileSize
new4.25 KB
new2.38 KB

With new test coverage.

pwolanin’s picture

Issue summary: View changes
pwolanin’s picture

Component: file system » documentation

Moving this to documentation since the only code affected is a test, while the core code only doc is affected

pwolanin’s picture

Title: StreamWrapperInterface composite constants are not clearly documented or tested for correctness » File include wekness, and StreamWrapperInterface composite constants are not clearly documented or tested for correctness
Component: documentation » file system
Category: Task » Bug report
Issue summary: View changes
Priority: Normal » Major
Issue tags: +Security improvements

Ah - discussed with chx and we found the real bug - in the last conversion the public and private stream wrappers were overridden to LOCAL_NORMAL instead of being left as NORMAL

This could indeed allow other bugs to be leveraged into an arbitrary code execution bug

pwolanin’s picture

StatusFileSize
new5.83 KB
new1.59 KB

The D7 default for public, private, etc was:

    foreach ($wrappers as $scheme => $info) {
      // Add defaults.
      $wrappers[$scheme] += array('type' => STREAM_WRAPPERS_NORMAL);
    }
pwolanin’s picture

Issue summary: View changes
pwolanin’s picture

Issue summary: View changes
pwolanin’s picture

Title: File include wekness, and StreamWrapperInterface composite constants are not clearly documented or tested for correctness » File inclusion weakness in stream wrappers, and StreamWrapperInterface composite constants are not clearly documented or tested for correctness
chx’s picture

Status: Needs review » Reviewed & tested by the community

So to clarify, when I began my stream wrapper work I saw that public was registered with a type 29 and that immediately screamed security weakness because it was LOCAL. I then miscalculated (opsie) 1C to be 29 so I filed on bugcrowd saying "constants are wrong". Well the constants are not wrong but documenting and testing them is good.

However, this didn't change the fact that registering public:// was happening as a local wrapper and so after I saw this test pass I looked into PublicStream a bit late and saw the getType methods. I think the "local stream" class name have caused some confusion here just because we store something on a local disk does not mean it should not be registered as a (remote) URL wrapper. Yay for PHP and its clear, easy to use and understand security facilities.

pwolanin’s picture

Issue summary: View changes

The original change to use LOCAL was actually in #942690: Security harden stream wrappers by defaulting them as remote.

effulgentsia seems to think that improved security, but as best as I can see, it actually didn't achieve that goal

Trying to get his feedback though.

pwolanin’s picture

Issue tags: +Needs backport to D7

So, further discussion with chx - we should probably consider a backport removal of LOCAL to D7 also.

The PHP option allow_url_fopen is true by default which means public:// should work fine with functions like file_get_contents() even if it's flagged as a "STREAM_IS_URL" , so there doesn't seem to be any real advantage to also allowing file inclusion by for uploaded files.

pwolanin’s picture

Status: Reviewed & tested by the community » Needs work

discussing with effulgentsia - since we don't currently require PHP to be configured as allow_url_fopen=1 in php.ini, this is not the right solution.

The change is also working against the code in the GD image toolkit:

      // If destination is not local, save image to temporary local file.
      $local_wrappers = file_get_stream_wrappers(StreamWrapperInterface::LOCAL);
effulgentsia’s picture

The original change to use LOCAL was actually in #942690: Security harden stream wrappers by defaulting them as remote.

Nope, even before that issue, public, private, and all other wrappers that didn't explicitly specify a type, defaulted to STREAM_WRAPPERS_NORMAL, which before that issue, was defined as STREAM_WRAPPERS_LOCAL | STREAM_WRAPPERS_WRITE_VISIBLE.

The PHP option allow_url_fopen is true by default which means public:// should work fine with functions like file_get_contents()

What do we do for servers where php.ini defines allow_url_fopen=0?

we should probably consider a backport removal of LOCAL to D7 also

I'm nervous about that. image_gd_save() checks file_get_stream_wrappers(STREAM_WRAPPERS_LOCAL), and so I imagine some contrib modules do as well, especially with file_get_stream_wrappers() documentation having that as an example.

pwolanin’s picture

Status: Needs work » Needs review
StatusFileSize
new6.07 KB
new3.41 KB

This might be a more sensible approach? Let's make sure GD is ok.

pwolanin’s picture

StatusFileSize
new6.06 KB
new527 bytes

better without the debug code...

The last submitted patch, 17: 2514644-17.patch, failed testing.

chx’s picture

Maybe but that needs a requirement warning because we can't make your site less secure w/o a warning. Also who switches off url fopen...?

pwolanin’s picture

StatusFileSize
new6.36 KB
new2.68 KB

how about this? Also more DRY

chx’s picture

+      'title' => t('PHP allows opening URLs as files'),
+      'description' => t('PHP has allow_url_fopen set to 0. This reduces your site\'s functionality and is not the default PHP value. You are advised to edit php.ini and set allow_url_fopen to 1'),

both need better wording: PHP doesn't allow opening URLs as file. And more importantly it reduces the site's security (because we are forced to register public:// as local yadda, yadda).

pwolanin’s picture

Status: Needs review » Needs work

ok, maybe we need to find someone to wordsmith

davidhernandez’s picture

This is only visible on the install page, correct?

How about...

+++ b/core/modules/system/system.install
@@ -97,6 +97,24 @@ function system_requirements($phase) {
+      'description' => t('PHP has allow_url_include set to 1. This reduces your site\'s security. You are strongly advised to edit php.ini and set allow_url_include to 0'),

Your server's PHP configuration has allow_url_include turned on. This can negatively impact your site's security, and you are strongly encouraged to turn it off. To disable this feature, set allow_url_include to 0 in php.ini.

+++ b/core/modules/system/system.install
@@ -97,6 +97,24 @@ function system_requirements($phase) {
+      'description' => t('PHP has allow_url_fopen set to 0. This reduces your site\'s functionality and is not the default PHP value. You are advised to edit php.ini and set allow_url_fopen to 1'),

Your server's PHP configuration has allow_url_fopen turned off. This disables the ability to retrieve files using a URL, and may reduce your site's functionality. To enable this feature, set allow_url_fopen to 1 in php.ini.

pwolanin’s picture

It will be visible also in the status report and during updates.

pwolanin’s picture

@chx - is there really a case where an attacker would be able to include public:// and not able to force a simple filepath or file:// ?

chx’s picture

Site builders find a way. Never underestimate what people can come up with.

pwolanin’s picture

@chx - sure, I'm just wondering if it's worth emphasizing the security element in the UI.

chx’s picture

It is. Security is always good to emphasize.

pwolanin’s picture

Status: Needs work » Needs review
StatusFileSize
new6.4 KB
new1.44 KB

David's wording with minor tweaks.

davidhernandez’s picture

It looks like the only thing that was changed was quoting the value. Was that a mistake?

I added my text from above and quoted the values. (I'm not too sure the quotes are necessary, though.)

pwolanin’s picture

Hmm, maybe I posted the wrong patch or something. For the second one seems chx is saying we should at least mentioned that there can be a security impact.

The values are shown quoted here: http://php.net/manual/en/filesystem.configuration.php, so that seemed reasonable to match.

davidhernandez’s picture

@pwolanin, do you mean add wording to indicate the suggestion of enabling allow_url_fopen has security implications?

pwolanin’s picture

@davidhernandez - yes, since by treating public files as being accessed by a URL, we cannot ever include them as PHP code via the stream wrapper.

pwolanin’s picture

Let me try again to clarify:

The patch changes the behavior of the public:// stream wrapper.

The result of the change is that if you have allow_url_fopen enabled we can tell PHP that public:// is a "URL".

The way PHP behaves is if public:// is a "URL" and you have allow_url_include disabled then it's impossible (we hope) to include PHP code via something like include("public://foo.php");

As a result, there is a potential (modest) security gain to having allow_url_fopen on as long as allow_url_include is off. This is what chx would like to see mentioned.

Again, that gain is due to the changes in the patch, it doesn't exist in HEAD

pwolanin’s picture

we want to say that you should use the PHP defaults, which is allow_url_fopen enabled but _include NOT enabled

davidhernandez’s picture

I nested the checks since _include requires _fopen to even function. A warning about _include with _fopen off is not needed. Added a sentence to help clarify. (I think) It is a bit tough to make it succinct, which I think is important given how long this text is getting. Lets just make sure the text is reasonable. The code change is more important than the message.

pwolanin’s picture

These settings are pretty hard to explain. I think this is fine unless we want to go as far as setting an ERROR for the _include being enabled.

pwolanin’s picture

Status: Needs review » Reviewed & tested by the community

Let's go with this

alexpott’s picture

Status: Reviewed & tested by the community » Needs work

No one seems to have answered @effulgentsia's question from #16

How does this work now?

      // If destination is not local, save image to temporary local file.
      $local_wrappers = file_get_stream_wrappers(StreamWrapperInterface::LOCAL);
      if (!isset($local_wrappers[$scheme])) {
        $permanent_destination = $destination;
        $destination = drupal_tempnam('temporary://', 'gd_');
      }

in core/modules/system/src/Plugin/ImageToolkit/GDToolkit.php

alexpott’s picture

+++ b/core/modules/system/system.install
@@ -97,6 +97,26 @@ function system_requirements($phase) {
+  } else {

else { should be on a new line.

pwolanin’s picture

Status: Needs work » Needs review
StatusFileSize
new6.69 KB
new837 bytes

@alexpott - we still mark e.g. public as StreamWrapperInterface::LOCAL_NORMAL so the GD code will work fine. In this case local is meaning really what we expect, that's it's on the same server.

pwolanin queued 42: 2514644-42.patch for re-testing.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

mile23’s picture

wim leers’s picture

wim leers’s picture

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
StatusFileSize
new132 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.