Don't filter out non-local stream wrappers

same interest. Filefield_path not funktoniert with AmazonS3

any update on this?

Just fyi, one way to use this module in conjunction with Amazon s3 is to simply mount S3 as a local filesystem using something like s3fs.

I can confirm this issue as well. I have the Amazon S3 module set up correctly. I can set files to save onto my desired bucket, using static directories.

If I try to setup a directory location using tokens it will default and still save using the same static location from above. Taking a hint from the opening post I modified the following line in the "filefield_paths.module"

function filefield_paths_uri_is_local($uri) {
  $scheme = file_uri_scheme($uri);
  $wrappers = file_get_stream_wrappers(STREAM_WRAPPERS_LOCAL);
 #Forced "true" to return and COMMENTED out previous return statement
  return true; #!empty($wrappers[$scheme]) && empty($wrappers[$scheme]['remote']);
}

Making this change will force the file (field) paths module to use tokens on the Amazon bucket. This is by no means a stable solutions and is very error prone. Do not do this on a production system.

I'm curious why FileField Path decided that remote filesystems were to be avoided. That filefield_paths_uri_is_local() function precludes this module's potential compatibility with AmazonS3 and my own module, S3 FIle System.

I've created a patch to allow S3 through. One thing i noticed is that within filefield_paths_filefield_paths_process_file() the while loop will erase all files in the s3 bucket. i've disincluded that in the process. this means the temporary folder on S3 where the file is saved on first upload will not be deleted when moved to it's final destination, which should be fine because that directory will be reused. Please test and review.

Any thought about using this Amazon Module https://drupal.org/project/amazons3

Dear, jcandan, thanks for your patch. Now Filefield Path working with AmazomS3 module, but if delete file and folder after that empty, folder doesn't delete.

Any idea?

@mgifford, the patch works in conjunction with amazons3 module.
@weateus, yes, this was the case for me too. but wasn't high priority to fix for the client project i needed this on. I'll try to return to this after deadline's met.

@jcandal thanks for your reply.

New moment in work module with S3. When changed ImageCache (Core module) presets or try to delete we have a long time prepare and after that time-out error (PHP, Apache and MySQL time out have a high value — more 5 minutes).

After delete image in field and try to prepare imagecache presets we have a error

Fatal error: Call to a member function read() on a non-object in /var/www/weateus.ru/www/includes/file.inc on line 1361

In watchdog additionally information

Warning: dir(s3://styles/music_backstretch): failed to open dir: "AmazonS3StreamWrapper::dir_opendir" call failed in function file_unmanaged_delete_recursive() (строка 1360 в файле /var/www/weateus.ru/www/includes/file.inc).

The patch is not the correct way to fix this 'problem'.

FFP should NOT be patched every time a new remote file-system is created, as this patch intends. FFP works with the core file-system, and it also provides a way for other file-systems to organise themselves too. It does so by invoking hook_filefield_paths_process_file. In fact, the patch is editing 'filefield_paths' implementation of the hook.

Therefore, the proper solution is for Amazon S3 module to implement the hook, OR a third module to bridge the 2. The hook would be something like below. And yes, the majority of the hook will be the same as filefield_paths_filefield_paths_process_file

/**
 * Implements hook_filefield_paths_process_file().
 */
function OTHER-MODULE_filefield_paths_process_file($type, $entity, $field, $instance, $langcode, &$items) {
  if (isset($instance['settings']['filefield_paths'])) {
    $settings = $instance['settings']['filefield_paths'];
    foreach ($items as &$file) {
      if (OTHER-MODULE-URI-CHECK($file['uri']) && ($file['timestamp'] == REQUEST_TIME || $settings['active_updating'])) {
        //Do stuff.
      }
    }
  }
}

Feature request in AmazonS3 module...
#1744610: Add support for File (Field) Paths module

@neRok I disagree. All Drupal registered stream wrappers are made in the same way-extend DrupalStreamWrapperInterface and implement hook_stream_wrappers. The whole point of them is to not have to write glue code/implement hooks to have stuff like this work.

The issue is that instead of checking for local stream wrappers, it should be checking for stream wrappers that are writeable. See #1361884: Remotes files should not be processed

Patch attached. This does currently cause an issue with the amazons3 metadata caching, but we'll fix it over at #1744610: Add support for File (Field) Paths module. I also removed the function being used to check for local wrappers since it was calling file_get_stream_wrappers with every file loop. I've tested it with images locally and remotely as well as with media_youtube.

AWESOME! The patch in #14 worked for me, using the S3 File System Module (https://drupal.org/project/s3fs).

Thanks so much :)

@justafish, you make sense. I have tested the patch, and local files are working (all I am setup for). I will email Deciphered and get him to commit the patch. Thanks.

#14 doesn't apply to latest filefield paths dev.

@jordanmagnuson, is it that the patch doesn't apply because it's been resolved in the dev version, or that the patch doesn't match dev's code?

I applied #14 on the latest dev just now and it applies just fine. Also, the patch seems to work indeed. I use the Rackspace Cloud Files (https://www.drupal.org/project/cloud_files) module. More testing then a commit to dev please! Good job everyone.

#14 doesn't apply to latest filefield paths dev. Patch doesn't match dev's code, has this actually been committed?

Actually the patch doesn't apply as-is with last dev, maybe because there is a file inside a folder mixed with another in root module.
However a shameless-manual-patch does the job, guided with the patch text:
filefield_paths.module : removing last function
modules/filefield_paths.inc : updating a function

2015 resolution: learn how to produce/change a patch

I could apply the patch, but it doesn't work. As a first look this is because of S3 module changes, but I'm not sure yet.

I'll check it out within few days.

Figured out.
file_stream_wrapper_uri_normalize() needs to generate proper S3 url (amazons3 module)
However amazons3 won't work yet with this patch, I'm 99% sure, it's up to amazons3 module. The target url is ok with this patch, now I'm getting error messages from AWS S3 streamwrapper when drupal_unlink is called in file_unmanaged_delete()

Triggering tests.

Triggering tests.

@szantog,

+++ b/modules/filefield_paths.inc
@@ -65,7 +69,8 @@ function filefield_paths_filefield_paths_process_file($type, $entity, $field, $i
-        $file['uri'] = "{$field['settings']['uri_scheme']}://" . ltrim(filefield_paths_process_string($settings['file_path']['value'], $token_data, $settings['file_path']['options']) . "/{$file['filename']}", '/');
+        $path = filefield_paths_process_string($settings['file_path']['value'], $token_data, $settings['file_path']['options']);
+        $file['uri'] = file_stream_wrapper_uri_normalize($scheme . '/' . $path . '/' . $file['filename']);

This is the section that is breaking, $scheme . '/ should be $scheme . '://. Changing this locally makes the tests pass, including an uncommitted test that creates a dummy read-only file stream wrapper.

Updated patch. I'll probably commit as per patch, but it would be nice if someone with an S3 setup can test it (I've yet to successfully setup S3 for testing purposes).

Updated patch to compensate for a recent update.

Still haven't been able to do a test with Amazon S3, and can't commit until I do. I suspect I just haven't gotten the my S3 account configured correctly.

Fixed the failing test.

Late, really should roll patches when it's late :|

Ok, I have officially confirmed that this patch works with AmazonS3. The issue I got stuck with is that it appears the default bucket wasn't being prepended to the file path, so I had to ensure that the bucket was included in the File Path settings; this seems to be an issue with AmazonS3, as File (Field) Paths shouldn't have to do something special to work with one single File Stream Wrapper when it doesn't have to do anything for others.

Thanks all for your patience, persistence and patches. Fixed and committed.

For anyone having a similar issue as this with 7.x-1.x, I tried to check on this patch in the current version but it seems it was replaced by other functionality which did solve my problem. I found a settings page at /admin/config/media/file-system/filefield-paths where I was able to change the temp file directory.