Toggleable public/private upload

I have created a patch that allows for the choice of file upload location (private/public) for each file uploaded using the file field widget. The patch also supports switching file location of existing files.

This can be used for use cases like private groups: public nodes have public files, and if a private group also uses this node type you can now save private files to to go with them, without using a separate field.

New patch.
Added fix for tests that check for file field config settings.
\Drupal\system\Tests\Installer\StandardInstallerTest::testStandardConfig
\Drupal\Tests\field\Kernel\Migrate\d6\MigrateFieldInstanceTest::testFieldInstanceMigration
These now know about the new uri_scheme_field setting.

Some points I see:

1. +++ b/core/modules/file/config/schema/file.schema.yml
   @@ -38,6 +38,9 @@ field.storage_settings.file:
   +    uri_scheme_field:
   +      type: boolean
   +      label: 'Enable Upload destination field'
   

   This would require an update path for existing file fields.

2. +++ b/core/modules/file/file.field.inc
   @@ -64,8 +70,11 @@ function template_preprocess_file_widget_multiple(&$variables) {
   +    // Delay rendering of the "Display" and "Location" option and the weight selector, so that
   

   longer then 80 characters

3. +++ b/core/modules/file/file.field.inc
   @@ -74,7 +83,14 @@ function template_preprocess_file_widget_multiple(&$variables) {
   -    $display = '';
   

   why is this getting removed?

4. +++ b/core/modules/file/src/Plugin/Field/FieldType/FileItem.php
   @@ -139,6 +140,13 @@ public function storageSettingsForm(array &$form, FormStateInterface $form_state
   +      '#title' => $this->t('Enable <em>Upload destination</em> field'),
   ...
   +      '#description' => $this->t('The upload destination option allows users to choose where the final files should be stored.'),
   

   Not sure about the 'upload destination' name. Isn't it more 'file system used for upload' or something along those lines?

5. +++ b/core/modules/file/src/Plugin/Field/FieldWidget/FileWidget.php
   @@ -404,6 +410,23 @@ public static function process($element, FormStateInterface $form_state, $form)
   +      $scheme_options = \Drupal::service('stream_wrapper_manager')->getNames(StreamWrapperInterface::WRITE_VISIBLE);
   ...
   +        $file = \Drupal::service('entity_type.manager')->getStorage('file')->load($item['target_id']);
   

   These can be injected right?

6. +++ b/core/modules/file/tests/src/Functional/FileFieldUploadDestinationTest.php
   @@ -0,0 +1,137 @@
   +    self::assertCount(1, $fields, 'Upload destination radio "public" input exists.');
   

   core uses $this-> not self:: in tests.

7. +++ b/core/modules/file/tests/src/Functional/FileFieldUploadDestinationTest.php
   @@ -0,0 +1,137 @@
   +    $node_storage = $this->container->get('entity_type.manager')->getStorage('node');
   +    $node_storage->resetCache([$nid]);
   +    $node = $node_storage->load($nid);
   

   why is this needed?

8. +++ b/core/modules/file/tests/src/Functional/FileFieldUploadDestinationTest.php
   @@ -0,0 +1,137 @@
   +    // Test if the newly added file is saved in private://.
   

   it's not newly added now anymore right? It's the same file that was moved?

Re #10:
1) Yes - I would need some pointers on how to implement this.
2) Fixed.
3) Fixed - should not have been removed.
4) Fixed - renamed.
5) Can't use injection because the function is static.
6) Fixed.
7) This was unnecessary - removed.
8) Fixed.

Patch contained some files that should not have been there. Correction uploaded.

Updating status to 'Needs work' for #10.1

Added sanity check to check whether the file is still available in the widget.

#10.1 is not yet addressed.

Rerolled the patch for 8.5.4 and tried to fix the test.

Rerolled the patch for 8.9.6

Hear an updated patch for 9.2.x

Reroll for 9.3.x.

Reroll 9.2 to fix
Error: Call to undefined function Drupal\file\Element\file_prepare_directory() in Drupal\file\Element\ManagedFile::valueCallback() (line 106 of core/modules/file/src/Element/ManagedFile.php).

Reroll 9.3 to fix
Error: Call to undefined function Drupal\file\Element\file_prepare_directory() in Drupal\file\Element\ManagedFile::valueCallback() (line 106 of core/modules/file/src/Element/ManagedFile.php).

Fixed CS error.

Rerolled #38 for 9.5.x

small compatibility fix for Drupal 10 undefined call to file_move

fix stupid issue with previous patch where i passed the plain file object to move instead of calling getFileUri on it.

The #45 patch is a small step forward but some odd things:

When creating a new Media entity with this patch enabled:

First add file, then select private storage.
-Upload a file
-Select Private storage
-Save entity
-Edit media entity again.
-Notice it has been stored as a public file.

First select storage, then upload a file.
-Set storage to private.
-Upload a file
-Notice that private has been reset to public, set to private again
-Save entity
-Edit media entity again.
-Notice is has the correct setting, private storage.

Now when you want to change the file from private to public it is not possible anymore. It will always be private.

Used correct service this time.

Try to fix patch #47.

The Needs Review Queue Bot tested this issue.

While you are making the above changes, we recommend that you convert this patch to a merge request. Merge requests are preferred over patches. Be sure to hide the old patch files as well. (Converting an issue to a merge request without other contributions to the issue will not receive credit.)

Please include interdiffs with all patches.

Issue summary could use some updating.

Updated the patch to be compatible with the latest 10.5.x release.

Main differences:
1. Updated imports in FileWidget.php to match Drupal 10.5.x structure
2. Fixed import ordering to be compatible with current codebase

core/modules/file/src/Plugin/Field/FieldWidget/FileWidget.php:
- Added missing import for StreamWrapperInterface (line 14)
- Added missing import for FileInterface (line 19)
- Adjusted import ordering to match Drupal 10.5.x conventions
- Added use statement for Drupal\Core\Url which was missing in patch #49

Technical details:
------------------
The original patch #49 failed to apply to Drupal 10.5.x because:
1. The import structure in FileWidget.php changed between versions
2. Additional imports (like Drupal\Core\Url) were added to 10.5.x
3. The patch context didn't match the current file structure

These changes make the patch compatible with Drupal 10.5.x while
maintaining all the original functionality for the uri_scheme_field feature.

All other functionality remains identical to patch #49.

Rerolled for Drupal 11.3.x (from #53 which targeted 10.5.x).

The functional changes are identical to #53. The patch needed a reroll because of structural changes in Drupal 11 core:

file.module → FileThemeHooks.php: The template_preprocess_file_widget_multiple() logic was moved to \Drupal\file\Hook\FileThemeHooks::preprocessFileWidgetMultiple() as part of the hook attribute conversion (https://www.drupal.org/node/3504125). The Location column header, hide($widget['uri_scheme']), and Location column rendering are now applied in FileThemeHooks.php instead of file.module.

All other files (ManagedFile.php, FileItem.php, FileWidget.php, file.schema.yml, MigrateFieldInstanceTest.php) have the same functional changes — only context lines shifted due to D11 code changes.

Tested against: Drupal 11.3.3. Patch applies cleanly and is in active use on a production multilingual Drupal site.

This needs an issue summary update before review. Also changes should be in MRs vs patches please.

Hidden the patch files.

Updated the IS with the help of AI.

Created a CR with the help of AI.
I have updated the MR with the help of AI.

The change need manual testing for instance in combination with S3 storage on AWS.

Thanks for turning this into an MR, @davidbekker.

We've been running the patch from #55 (re-rolled for 11.3.x) on a production site with a media "Document" type that has uri_scheme_field enabled. I tested it on 11.3.17: uploading to public and to private both work, and switching an existing file from public to private moves the file correctly. I haven't tested MR !17381 itself yet, since it targets main.

Two things we ran into that I think the MR should still address:

1. Restrict which schemes are offered

The widget lists every StreamWrapperInterface::WRITE_VISIBLE wrapper, so any module that registers a writable stream wrapper (s3fs, flysystem, cloud storage wrappers, …) adds an upload destination for editors. The labels also come from the wrapper, so they expose storage details. With s3fs, for example, editors see:

• Public files (s3fs)
• S3 File System
• Private files (or "Private files (s3fs)" when s3fs also takes over private://)

Storage modules that swap out public:// or private:// are infrastructure choices that can differ per environment, while what editors want to decide is public or private access.

I'd suggest a field storage setting next to uri_scheme_field, e.g. uri_scheme_options, as checkboxes of the write-visible wrappers. The widget would only offer the selected schemes, and the server-side check in ManagedFile::valueCallback() / FileFieldItemList would validate against the same list instead of all WRITE_VISIBLE wrappers. A sensible default could be the field's default scheme plus private, so a newly installed storage module doesn't silently add options to existing fields. We now work around this with a #process callback that limits the options to public and private.

2. Existing files on a scheme that isn't offered

If a file is stored on a scheme that is not (or no longer) in the options, #default_value points to a missing option. With #required => TRUE, the editor gets a "field is required" error on an unrelated edit. It would be better to fall back to the field's default scheme, or to keep the current scheme as an option for that file.

The Needs Review Queue Bot tested this issue. It fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.