Allow redirect responses for download requests (and automatically redirect external file URLs like s3fs)

I've refactored the patch written by @tnathanjames in #5, to rename the config being created so that users can decide to use redirects vs. file responses in general, since this has many potential use cases and doesn't just apply to external file storage. For examples, using redirects instead of file responses can have many benefits related to performance (re: caching) and security (re: private file access control).

Including interdiff for changes from #5 to #6.

This patch is looking really good, but in order to get the performance benefits (cacheability of the redirect), the redirect would need to be a 301, but TrustedRedirectResponse defaults to a 302. Does it make sense to make the redirect code be configurable here? One way to do it would be to have "redirect_download" be an integer, where 0 = no redirect, and it can be set to 301 or 302.

Re-rolled patch to include redirect configuration options (301, 302, or 307). Thanks, @rbayliss!

Lets add it to 8.x-2.x first and then backport to 8.x-1.x if still needed?

Rerolled and adapted for 8.x-2.x

Need this for Drupal 8.4+ Media in core.

Just tagging this with "security" as it affects content security.

updating status, and assignee based on the last 9 comments.

needs work as tests in #11 failed.

Frankly: I don't like this approach.

To me redirects only really make sense for external media sources. Security wise @vitalie's approach is good and integrates access control for "private://" files. Thinking of custom filename suggestion in forced-download situations (content-disposition: attachment): HTTP redirects to "public://" files would prevent any control of HTTP response header from Drupal/PHP since those files are delivered solely by the HTTP daemon - hence would prevent such features from being implemented.

What do you think of my more general approach to handle needs of external media sources in 3012528?

Remember that the original goal of this module was to provide a stable URL for a file so that editors can link to it without worrying about it changing when files are replaced.

If we keep that goal in mind, then always redirecting to the file URL makes the most sense to me. Editors just need an easy way to link to the vanity path and Drupal handles the redirect to the actual file when someone requests the path.

But now I'm a bit conflicted. If a redirect is done for something like an image or a PDF, then it's possible that by default a user's browser may open that URL directly in the browser. That would expose the actual file URL to the browser, which means that a visitor may copy and paste that link to and give it to someone else... which is what we don't want.

I don't know how much I should really be concerned about that.

Perhaps the bigger concern should be: Do we really want to be bootstrapping Drupal to proxy the download of public file assets? I think the performance implications of doing so are great enough that we shouldn't. Imagine a swath of visitors with very slow internet connections, and they all request to download a large file using this method. Now PHP is stuck handling this slow I/O instead of the web server. This is of course unavoidable for private files.

In the mean time while this issue is fleshed out more, I created a small contrib module that redirects the canonical path for file-based media entities to the actual file source URL: Media Entity File Redirect.

I agree with @bkosborne above:

Remember that the original goal of this module was to provide a stable URL for a file so that editors can link to it without worrying about it changing when files are replaced.

Redirects to a managed file, that is uploaded for a Media entity, come with problems. Particularly around caching on High Availability architectures where Varnish, Cloudflare and other reverse-proxy caches are used. Cachability metadata is not set and/or returned by this module's download route either which I'm planning to address in another issue.

The main question you need to think about is "what happens if the File changes?"
Will your redirect still be cached at the Edge, or anywhere else between the Origin and the Visitor, as pointing to the old file?
If it is cached, will it be cleared from cache because the redirect response returned appropriate cache metadata (ie. cache tags, contexts, max age, etc.)

There are already patterns provided by other modules that can do a redirect from a Media entity to its file quite well. I'd point to the Rabbit Hole module, in tandem with the Tokens module:

- https://dgo.to/rabbit_hole
- https://dgo.to/token

Using these you set-up your Media type with Rabbit Hole so any request for a Media of that type is redirected (HTTP 301), using tokens, to the File URI of the attached file.

Here's the config export:

config/default/rabbit_hole.behavior_settings.media_type_document.yml

uuid: b4c6dbf4-5660-4c86-88bd-0bfa5ad0238a
langcode: en
status: true
dependencies:
  config:
    - media.type.document
id: media_type_document
action: page_redirect
allow_override: 1
redirect: '[media:field_document:entity:url]'
redirect_code: 301

FWIW I think the focus of this module is not really related to doing redirects.

Removing the security concern from the issue motivation, since that is being addressed in #2986486: Check entity access instead of global permission in download controller

The main question you need to think about is "what happens if the File changes?"
Will your redirect still be cached at the Edge, or anywhere else between the Origin and the Visitor, as pointing to the old file?
If it is cached, will it be cleared from cache because the redirect response returned appropriate cache metadata (ie. cache tags, contexts, max age, etc.)

These concerns around caching is not justified - This is easily solvable by adding the proper cache metadata to the redirect response. My alternate module Media Entity File Redirect does this. If the file on a "Document" media entity is replaced, the redirect response will be cache invalidated correctly.

Rerolled and adapted for 8.x-2.x

Need this for Drupal 8.8+ Media in core.

Changing status to request review of patch in #22.

I've opened a MR that takes the patch from #22 and then fixes a couple of bugs.

Seems to work though!

I think this is ready for re-review.

tested this approach with Stage File proxy module, and it works perfectly. Made some minor changes to @dave reid PR.
Overall all works perfectly.
Using this patch for couple our Acquia site factory projects

Have some questions about tests. What kind tests do we need to cover this functionality? And how we can emulate remote file download for BrowserTestBase?
i have only one idea is save dummy file in the project repo and and use it permanent link in test module for download file.

thanks