How to use an alternate stream wrapper when saving media files?

Subscribe.

I have been getting requests to interface with media module / media field; however it still does not appear to support the D7 core's private file scheme. This is needed for any integration of selling files.
My results from testing
@see http://drupal.org/node/1186162#comment-4607394

i now have private download working.
i couldnt however figure out how to use the widgets settings for defining the scheme in /includes/media.pages.inc so i used a workaround adding another form element to the upload-form using the approach from : http://drupal.org/node/1139266#comment-4532552 instead. (maybe someone can put that right?

all it takes is to change the definition of $scheme in media.pages.inc and implement hook_file_download (which would just check for hook_file_download_access?)

this wouldnt let me sleep! here is the missing link. using this should allow to get at the setting. now i just have to figure out where the field-widget and the browser handle stuff differently to pass the correct arguments.

$info_instance = field_info_instance('node', 'field_media', 'article');

the real problem here seems to be that we want to use the global filefield settings unless we add media through a field. yet the uploadform is not aware which entity/bundle/instance/field it is being called from.

even though the 1139514_media_browser_fat branch has the triggering element (and moved the hardcoded $scheme definition to validate function) the form still cannot discern exactly where it is being called from.

http://drupal.org/node/1163100 might be a hint, but throws an error when not uploading through the field.

In testing this also try using http://drupal.org/project/amazons3. Having Media work well with S3 file storage will be a killer feature, especially for large video files and podcasters.

private filesystem support stub
I am hoping someone will figure out how to handle the scheme array correctly (Instead of my foreach wich will break if more than one scheme is allowed for the field) and how to handle uploads that don't go through a field. (make them use global settings and not throw errors, i just ignored that part for now)
i still consider this to be a success though.

I was able to save private files but had some issues testing...
Since this patch adds a parameter to media_add_upload() it breaks some things when media calls it without that param. There should be a default set for it, ie $schemes = array().

For the media field this should really be handled with the element['#media_options']['global']['schemes']... however it seems the media module uses the same upload function for both entity and field, which that function has the hard coded 'public'.

The media maintainers need to find the best way to handle this for both case.

this is a bug ... core implements file schemes, core's file field implements them, so media field / entity / module should also

its easy enough to take care of the breakage in media_add_upload() by wrapping

if (isset($params['schemes'])){
      $schemes = $params['schemes'];
      }

in media.browser.inc i just dont know how to write else {use default filesystem}

the param is taken from the instance-widget-settings if i am not mistaken. it can have multiple values (private and public) which i do not know how to handle.

subscribe

setting the status. thanks for the work on this; hopefully one of us can take a look soon to help push it along.

here is the patch including comment #11

I believe there's some overlap with the following?

#1096374: Provide administrator with ability to customize upload directory, globally and per-field, with token integration

Getting one of these patches in would make the following fix really useful -
#1166168: Expose a widget for filefield and imagefield, so that media browser can be used for them

i am guessing i misformatted the patch? not really sure how to read those testresults, but i thought i would just try recreating the patch one more time ... or is it because of naming?

hmm...

#19: media-privatefs-1069582-19.patch queued for re-testing.

Testbot seems to be misbehaving here.... No tests are being run.

running tests locally against patched version gives the same output as running tests on the non-patched version. (beta5 & 7x-dev git version)

Do the tests run normally without the patch?

no

here an explanaition of the pretty simple steps i put into the patch:
Problems:
1. the stream wrapper is hardcoded to default.
2. the form is not aware of whether it is a field
3. we want to use the fields settings or the global settings

Questions: what is supposed to happen if i fill a field via the browser with a file previously uploaded to public filesystem if the field only allows private filesystem? what is the expected behaviour? copy? move? dont even show in browser to begin with?
is it ok to pass an aditional parameter to the uploadform?
or is there a better way to get at the fields settings ? maybe using http://drupal.org/node/1163100 ?

Explanaition of my attempt:
part one of the patch says: if schemes are set (which means we are dealing with a field, the allowed schemes are in the params!) pass the scheme to the upload form and otherwise pass the default scheme

part two is supposed to then get the allowed schemes for the field. i am not sure how to handle those multiple values though. (in the settings form allowed values are checkboxes)

part three simply adds a function so that private files can be displayed, this could be easily expanded with code from other modules

just noticed i rerolled the wrong changes so here we go again

same patch with comments

patch #28 worked for me and is a good start. I think there just needs to be some safeguarding and checking per suggestions below. It appears this is the easiest way for now to get scheme propagated down to the saving of the file. Ideally this would be handled in the field but media uses its own upload form for everything - field, adding in content section, etc.

Change this section in the patch

  $scheme = variable_get('file_default_scheme', 'public') . '://';
//define $scheme from
//whats the best way to handle multiple values here? what is the default?
foreach ($form_state['build_info']['args'][1] as $schemes => $key) {
    if (!$key == 0) {
    $scheme = $schemes . '://';
    }
  }
 // $scheme = variable_get('file_default_scheme', 'public') . '://';

TO

  $file = $form_state['values']['upload'];

  // determine scheme for saved file
  $scheme = variable_get('file_default_scheme', 'public') . '://';
  if ($file->status != FILE_STATUS_PERMANENT && !empty($form_state['build_info']['args'][1])) {
    $allowed_schemes = array_filter($form_state['build_info']['args'][1]);
    if (!empty($allowed_schemes)) {
      $scheme = array_shift(array_keys($allowed_schemes));
      $scheme .= '://';
    }
  }

Some caveats with this approach:

• for a newly uploaded file, it selects the first allowed scheme. A future task / feature request could be to provide an option on the widget - OR - only allow a single scheme to be selected on the field/instance settings

here is the updated patch

Do you think this patch will work with a pre Beta5 dev? For the time being I'm using a May 22 version for stability/compatibility reasons.

#30: media-1069582-30.patch queued for re-testing.

it seems like this approach is not valid anymore since media_file_download is not required anymore. yet files in private filesystem are not visible (access denied) with current media & file_entity modules (http://drupal.org/node/1223670) any suggestions or clues?or insights where the current extensive efforts are heading? i am overwhelmed by all the pending patches.

Hi, what is the status on this issue please?
Greetings, Martijn

This should now be possible with #1580820: Provide interfaces for selecting stream wrappers during file upload and converting files between stream wrappers.