Problem/Motivation

Drupal core doesn't provide any way to defend against file uploads attacks like the one described at https://soroush.secproject.com/blog/2014/05/even-uploading-a-jpg-file-ca...

This attack can most readily be mitigated by serving uploaded files from a different domain or subdomain where the user's session cannot be used to access private data from Drupal.

This issue reported via the Drupal 8 security bug bounty
https://tracker.bugcrowd.com/submissions/b54faf7e3c8a492dc289759fa17e0fc...

Proposed resolution

Provide a setting to override the domain used for public file URL or to replace part of it (e.g. to map to a different domain or subdomain)

This would be more effective if we stop sending session cookies to all subdomains per #2522002: Do not strip www. from cookie domain by default because that leaks session cookies to subdomains

Remaining tasks

patch
review

User interface changes

Additional section on the file system admin page (see screen shot)

API changes

Minor API addition

Data model changes

none

Comments

pwolanin’s picture

Priority: Normal » Major
Issue summary: View changes
effulgentsia’s picture

Because this is a security improvement, I think it's ok to still do for 8.0, though if it doesn't get done in time, it can also go into 8.1.

Provide a config setting...Possibly a configuration option

I disagree with making it config. I think just a settings.php setting is enough. Just like file_public_path. This means it can be shown in the UI, but not edited. See FileSystemForm::buildForm().

effulgentsia’s picture

The decision to use settings instead of config for file_public_path was made in #1856766: Convert file_public_path to the new settings system (make it not configurable via UI or YAML). Not sure that the reasons there necessarily apply to whatever we name this new thing (file_public_host?), but I think the two variables should be in the same system (i.e., either both config or both settings, but not one of each).

pwolanin’s picture

Status: Active » Needs review
StatusFileSize
new1.57 KB

An example of how such a setting might work at the stream wrapper level

pwolanin’s picture

Issue summary: View changes

@effulgentsia - sure, it makes sense they would be in the same system. Would we need a commented-out example in default.settings.php?

Also, this proposes a preg_replace(), but a strtr might actually be more appropriate? e.g. the setting could just be:

array('www.' => 'downloads.');

Instead of like:

array('pattern' => 'www\.(.+)$', 'replacement' => 'downloads.\1');
jplopezy’s picture

Hi,

I have found this security issue.

Thanks Drupal for fix this issue.

regards

pwolanin’s picture

StatusFileSize
new2.28 KB
new1.77 KB
dave reid’s picture

Why not just make this an alter hook instead? It feels like we're limiting ourselves with what can be done.

dave reid’s picture

Couldn't this also be accomplished by using hook_file_url_alter(), especially since the example code in that hook is exactly this use case?

pwolanin’s picture

Issue summary: View changes
StatusFileSize
new3.96 KB
new3.49 KB
new198.59 KB

@Dave Reid - so this is for the narrow use case of core mapping only public files to another domain. In other words, I'd like to be able to provide a doc on how to serve Drupal 8 user-uploaded files more securely without needing any contrib modules or custom code.

Yes, it overlaps with that hook, but generally it doesn't make sense for core to implement its own hooks, and the hook would be used e.g. by the CDN module for more complex use cases.

Here's a patch with a few further tweaks including showing the result in the UI + screenshot.

File system UI change

dave reid’s picture

Hrm, it just feels like we're investing a whole other thing that duplicates something already in core. The UI addition is good, but could also be made to work with the result of hook_file_url_alter().

pwolanin’s picture

The form is actually a bit weird since it could possibly get an instance of what responds to public:// instead of calling those static methods, but maybe the goal is to show just what the core/default implementation will do.

To me, keeping this contained in the class makes it easier to replace via swapping the service, while hook_file_url_alter() is going to act on a lot more things.

effulgentsia’s picture

I think this is different than hook_file_url_alter(), because hook_file_url_alter() does something a little different than its name implies. hook_file_url_alter() is for altering the URI of a file prior to the URI being converted to a URL. So, for example, a CDN module could use the hook to alter public://foo.txt to my-cdn://foo.txt and then let the my-cdn stream wrapper decide how to map that to an external URL (via getExternalUrl()). Of course, http and https URIs are also URLs, so hook_file_url_alter() can also be used to alter public://foo.txt directly to http://.../foo.txt, and you're right that the documented example implementation of hook_file_url_alter() does that, which perhaps isn't ideal.

So I don't think the use-case here should be implemented via hook_file_url_alter(), because then we'd lose the information that it's still a public:// file. For example, suppose a CDN module ran at a higher weight, then our implementation would turn public://foo.txt to http://internal-not-cdn.subdomain/foo.txt and the CDN module wouldn't then recognize this as something that should be further mapped to a CDN. In other words, I think as much as possible, hook_file_url_alter() should keep $uri as a URI and defer conversion to URL to the stream wrapper.

But, I think using a replacement pattern here is overkill. I think the setting should be an exact base URL, like file_public_base_url, and if someone needs it to be dynamic, then they can implement the code to make it so. Note that it can be a protocol-relative base URL, and probably should be in the commented out example we put into default.settings.php.

pwolanin’s picture

@effulgentsia - to me the replacement is more useful since I want to be able to e.g. do different mappings in dev and prod, or map the "www" subdomain in both dev and prod to "downloads"

effulgentsia’s picture

Your settings.php needs to be different on dev and prod anyway, due to $databases. And if you're smart enough to make $databases vary by environment via PHP code in the settings.php or its includes, then you can figure out how to do the same for $settings['file_public_base_url']. Same for multisite.

effulgentsia’s picture

E.g., you can make $settings['file_public_base_url'] an object that implements a __toString() method if you want to make it dependent on $base_url; not sure if default.settings.php should mention that in a comment or not.

pwolanin’s picture

StatusFileSize
new3.84 KB
new4.42 KB

Didn't mention it here, but included a string cast.

wim leers’s picture

Issue tags: +Needs tests

#13: the URI vs URL distinction is technically correct, but Drupal 7 didn't make that distinction. It was just "file URLs", period. The CDN module's hook_file_url_alter() implementation does receive a file URI and then returns a file URL. It doesn't use stream wrappers.

So I don't think the use-case here should be implemented via hook_file_url_alter(), because then we'd lose the information that it's still a public:// file.

This makes a ton of sense. And is the best justification for #10, and answers the concerns in #11 by Dave Reid.


Code review

Overall, +1, thanks to the excellent rationale provided by @effulgentsia.

  1. +++ b/core/lib/Drupal/Core/StreamWrapper/PublicStream.php
    @@ -21,6 +21,20 @@
    +   * The instance base path.
    ...
    +   * Instance public base URL with scheme.
    

    The mention of "Instance" here is very confusing/weird. What instance of what?

  2. +++ b/sites/default/default.settings.php
    @@ -455,6 +455,18 @@
    + * accessing public files. This can be used for a simple CDN integration, or
    + * to improve security by serving user-uploaded files from a different domain
    

    Nit: 80 cols.

  3. +++ b/sites/default/default.settings.php
    @@ -455,6 +455,18 @@
    + * or sub-domain pointing to the same server
    

    Nit: s/sub-domain/subdomain/ (I think?)
    Nit: Missing trailing period.

pwolanin’s picture

StatusFileSize
new3.93 KB
new1.62 KB

All the lines are < 80 col afaict. Fixed other nits.

wim leers’s picture

Still missing test coverage.

pwolanin’s picture

Issue tags: -Needs tests
StatusFileSize
new4.98 KB
new3.76 KB

Ah, sorry - I missed that suggestion.

Here's some test coverage. Also, realized there is little value to storing the values in the instance.

znerol’s picture

Do you think we can get rid of the configurable $base_url global in settings.php with this change?

pwolanin’s picture

@znerol - no, this is limited to public files and doesn't affect general URL generation.

wim leers’s picture

Status: Needs review » Needs work

Tests look good :)

  1. +++ b/core/lib/Drupal/Core/StreamWrapper/PublicStream.php
    @@ -53,7 +53,25 @@ public function getDirectoryPath() {
    +  public static function baseUrl() {
    
    +++ b/core/modules/system/src/Form/FileSystemForm.php
    @@ -89,6 +89,13 @@ public function buildForm(array $form, FormStateInterface $form_state) {
    +      '#markup' => PublicStream::baseUrl(),
    

    This means you're making it an API, but there's no interface. So if somebody provides alternative implementation of the stream_wrapper.public interface, this will break.

    Either needs to not be public, or needs an interface.

  2. +++ b/core/modules/file/src/Tests/DownloadTest.php
    @@ -113,14 +113,25 @@ function testFileCreateUrl() {
    +    // Test public files with a different host name from settings.
    

    s/host name/base URL/

    (For consistency.)

pwolanin’s picture

FileSystemForm.php already uses the PublicStream class directly and calls a static method not on the interface, so I think the basic answer is that this admin page already doesn't give you accurate information if you replace the public stream wrapper.

effulgentsia’s picture

+++ b/core/lib/Drupal/Core/StreamWrapper/PublicStream.php
@@ -53,7 +53,25 @@ public function getDirectoryPath() {
+    return static::baseUrl() . '/' . $this->getDirectoryPath() . '/' . UrlHelper::encodePath($path);
...
+++ b/sites/default/default.settings.php
@@ -455,6 +455,18 @@
+ * accessing public files. This can be used for a simple CDN integration, or

For simple CDN integration, we might not want the directory path automatically appended. Should we make the return value in the first line just baseUrl() . '/' . UrlHelper::encodePath($path), and require the setting, if specified, to include sites/default/files if that's wanted?

wim leers’s picture

#27 is an excellent point.

pwolanin’s picture

Issue summary: View changes
StatusFileSize
new6.52 KB
new3.68 KB
new486.21 KB

Ok, changed that and added some code comments suggested by Wim Leers + screen shot to show it working locally.

pwolanin’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 29: 2522008-29.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review
StatusFileSize
new6.49 KB
new1.37 KB

forgot to fix the test to match.

dawehner’s picture

Isn't that already supported by all the magic in: file_create_url() ?

effulgentsia’s picture

See #13: this does it without needing to implement the alter hook and worry about how it plays with other implementations of the alter. Since it's a good security best practice, it's nice to support as a simple setting.

Status: Needs review » Needs work

The last submitted patch, 32: 2522008-32.patch, failed testing.

pwolanin’s picture

Status: Needs work » Needs review
Issue tags: +Random test failure
StatusFileSize
new6.49 KB

re-posting patch.

pwolanin queued 36: 2522008-32.patch for re-testing.

Status: Needs review » Needs work

The last submitted patch, 36: 2522008-32.patch, failed testing.

pwolanin’s picture

Issue tags: +Needs reroll
nlisgo’s picture

Assigned: Unassigned » nlisgo
nlisgo’s picture

Assigned: nlisgo » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new6.52 KB

wim leers’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Random test failure

AFAICT all feedback has been addressed.

+++ b/core/lib/Drupal/Core/StreamWrapper/PublicStream.php
@@ -53,7 +53,29 @@ public function getDirectoryPath() {
+   *   The external base URL for public://

Nit: Missing trailing period. Can be fixed on commit.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 41: provide_a_setting_to-2522008-41.patch, failed testing.

Status: Needs work » Needs review
pwolanin’s picture

Status: Needs review » Reviewed & tested by the community

Looks like a temporary or sporadic fail that kicked it back.

effulgentsia’s picture

Adding credit to @jplopezy for reporting the original security issue and to @Wim Leers for substantive reviews.

effulgentsia’s picture

Status: Reviewed & tested by the community » Fixed

Pushed to 8.0.x! Thanks for the nice security hardening.

  • effulgentsia committed c1316d6 on 8.0.x
    Issue #2522008 by pwolanin, nlisgo, Wim Leers, jplopezy: Provide a...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Status: Closed (fixed) » Needs work

The last submitted patch, 41: provide_a_setting_to-2522008-41.patch, failed testing.

tim.plunkett’s picture

Status: Needs work » Closed (fixed)

wtf, d.o?