Create secured temporary /files tmp directory (or) create hash on uploaded tmp files

I just tested this theory against a D7 site and D8 site and in both situations, the temporary file location provided right after upload (the link provided in the edit form) does indeed provide full unfettered access to the file for the public. This tells me that a hashed file, won't really work either unless Drupal can somehow mask the hash which makes the whole thumbnail and link a lot more difficult.

IMO, the best method is for the files to be uploaded to a secured folder first, then moved upon a form submit. Without this, websites that allow authenticated users, is still just as susceptible to this problem as a site that deals with this for anonymous with the only security block being the registration of an account on the site. Sites that require email verification have yet an additional layer of security, however we all know the bots / people employing the bots, can get around that as well...

Point is, PSA-2016-003 really only addresses this for anonymous uploads and the proposed fixes, are only mitigating the issue on a module by module basis. Core itself, is still fully susceptible to this attack, even with trusted authenticated users, despite the PSA suggesting that anonymous / untrusted is the culprit. This round it was, but what about the next time? There's still a gaping security hole that needs to be filled.

Another thought is to have Drupal use the private file system by default and push the private file system by default. This becomes more difficult for new Drupalers to setup and point to the private file structure, however it would help mitigate this issue. If this approach is taken, two things I can think of, need to happen:

1) The private field option needs to be moved above the public for the file field with the directions added as such:

• Private File System - Recommended (More secure from attacks)
• Public File System - Security Risks Involved (Only use this if you want your files to be indexed by search engines)

2) The switch between public and private needs to have a DB statement run that will automatically migrate the files from the public location over to the private location or vice versa, along with the links and redirects involved with them.

Unfortunately, unless the temporary file problem is secured and fixed for the public file system, pushing the private file system still won't resolve this issue for public fields created by authenticated users that explicitly need to be indexed by Google. This means the best overall route (I can see) is to use some sort of secured global tmp directory.

@David_Rothstein do you at least acknowledge that since core does not mitigate this security hole, the problem might crop up again in the future through various module?

I totally get that the chances of this scenario is quite low, but "why leave your keys in the truck just because you live in the country?"

@pwolanin That's not a bad idea... Partial mitigation is better than none at all.

@David_Rothstein

If your site is configured to allow untrusted users to upload publicly-viewable files, then by definition they might upload files you don't like.

So, what exactly constitutes an 'untrusted user'? Authenticated? Can't an authenticated user still be an untrusted user? The issue goes deeper than it being a content moderation problem.

Any Drupal website that allows an open or even verified user registration system, is at risk "if" the site has an entity with an image or file field that can be edited by the authenticated user. It narrows the playing field quite a bit, but the problem is still there. Once those sites are found, it wouldn't be hard for someone with a little knowledge to write a bot that would sit out there and authenticate with the site, then upload a file without submitting, then watch to see how long it takes to get removed. If it cleans out with cron and that's every hour, set the bot to automatically re-upload the file every 61 minutes and now the file becomes persistent on the site, "hidden" from the site maintainer. You'd never know it was happening unless you were looking for it.

@David_Rothstein curious, how did the latest Core release dealt with this on the private file side of an anonymous user uploading to the private file system? I honestly haven't looked into the problem or the fix, but when I ran across the release notes, it sounded very similar to what I describe here. Is it similar or completely different?

On a side note, I don't see the need to make it easier for administrators to sift through temporary files when it "could" be mitigated altogether through some sort of scrubbing process performed at upload. Fix it once and for all and forget about it.

@David_Rothstein Thanks for that explanation :-). I had a feeling it was similar, but your statement makes sense on why it isn't easy for public...

Still, something to keep in mind though.

@geek.merlin I think the part you're missing is if the system is set to public files. Private files mitigates this by design.

If the site is set to use public files and a user uploads a file, that file sits out in the uploaded location until it gets scrubbed.

If a fresh install of D8 is used with public files and all uploaded files get dumped at ../files, a bot could automatically login to a site and re-upload a malicious file for distribution every few hours. If the upload path never changes, then it would be pretty easy to have a bad file getting perpetually uploaded to the server with no-one even knowing how or why it's happening since a node isn't being saved to track it.

What started this whole thing was someone uploading PDFs through webform that had phishing links in them. These PDFs were then getting indexed in Google and showing up all over the place. You simply uploaded a file within the webform, copy the link that was graciously provided by drupal and close the page. This created the mad-dash for webform to move all attachments to private instead of actually addressing the problem itself.

Same thing can happen with any drupal site where a user has access to an upload form and the filesystem is set to public. I just tested it on D8 and found that a file is still accessible after uploading it without submitting the node: 8.7.7

Why would someone want files set to public? For indexing. PDFs get indexed by google. Images get indexed by google. TXT files get indexed by google. I have manuals, safety data sheets and brochures on one of my sites which all need to be in the public, index-able space. Moving these to private makes it more difficult for indexing purposes, thus I have less content for exposure.

Clear as mud?? ;-)

Just tested this here on d.o, which uses public files and allows anyone to create an account.

I uploaded a test.txt file: https://www.drupal.org/files/issues/2019-11-13/test.txt

I forgot to check the link, so I uploaded it a second time: https://www.drupal.org/files/issues/2019-11-13/test_0.txt

Never saved the form, yet those files will be publicly accessible until cron scrubs them.

In otherwords, "anyone" can signup on D.O and upload a file without saving it to an issue. Create a bot to do it automatically, records the link, then turns around and posts that link out on a blackhat file sharing website. Bot then checks the link to see if it's dead or alive and if dead, do it all over. Free file hosting upto 50MB on this smorgasboard of extension types! jpg jpeg gif png txt xls pdf ppt pps odt fodt ods fods odp fodp gz tgz patch diff zip test info po pot psd yml mov mp4 avi mkv

The likely hood of this happening are obviously small, but still a security hole and I have now literally written a public manual for someone to take advantage of it.

Update: More than 12 hours later and the files still haven't been scrubbed...

@geek.merlin I would be fine with that, the point of this issue is to mitigate the gaping security hole that is currently wide open.

I just checked and those files I uploaded the other day to this issue and never saved, are still sitting out there...

I think we're on the same page with moving the upload system to a more secured setup all-around. How it happens, I don't really care, just that someone finally takes it serious enough to fix it.

I like the idea of sending the upload to a private area and then moving it from there. One thing to consider is maybe tying the view-able permission to the upload session, meaning the only person who can see the uploaded file before save, is the person who uploaded it. The way Drupal currently works, is to show the link to the file after upload or in the case of an image, the stylized image itself.

"How do we create an upload system that is still user friendly for the up-loader, but still secure enough to be inaccessible from everyone else?"

Then, there's the issue of whether cookies are enabled or not in order to track the session.

Also, how hard would it be to spoof a session? (rhetorical, something to consider)

Comment Removed. Should have read previous comment more closely.