Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
If I use this module alone, everything is fine. When I use with Upload destination: AmazonS3, then there are problems.
It seams that filefield_paths check for Wrapper to be not remote and fallbacks to default File directory, therefore tokens does not work. I do not see any readon why theese modules cannot cooperate, as independently bith works just fine. It does not look like issue is in AmazonS3 module as it acts just as wrapper.
Comment | File | Size | Author |
---|---|---|---|
#34 | writeable_stream_wrappers-2019723-34.patch | 3.13 KB | Deciphered |
#32 | interdiff.txt | 772 bytes | Deciphered |
#30 | writeable_stream_wrappers-2019723-30.patch | 2.38 KB | Deciphered |
#30 | interdiff.txt | 696 bytes | Deciphered |
#29 | writeable_stream_wrappers-2019723-29.patch | 2.38 KB | Deciphered |
Comments
Comment #1
fugazi CreditAttribution: fugazi commentedsame interest. Filefield_path not funktoniert with AmazonS3
Comment #2
cancerian7 CreditAttribution: cancerian7 commentedany update on this?
Comment #3
JordanMagnuson CreditAttribution: JordanMagnuson commentedJust fyi, one way to use this module in conjunction with Amazon s3 is to simply mount S3 as a local filesystem using something like s3fs.
Comment #4
jay_m CreditAttribution: jay_m commentedI can confirm this issue as well. I have the Amazon S3 module set up correctly. I can set files to save onto my desired bucket, using static directories.
If I try to setup a directory location using tokens it will default and still save using the same static location from above. Taking a hint from the opening post I modified the following line in the "filefield_paths.module"
Making this change will force the file (field) paths module to use tokens on the Amazon bucket. This is by no means a stable solutions and is very error prone. Do not do this on a production system.
Comment #5
coredumperror CreditAttribution: coredumperror commentedI'm curious why FileField Path decided that remote filesystems were to be avoided. That filefield_paths_uri_is_local() function precludes this module's potential compatibility with AmazonS3 and my own module, S3 FIle System.
Comment #6
jcandan CreditAttribution: jcandan commentedI've created a patch to allow S3 through. One thing i noticed is that within filefield_paths_filefield_paths_process_file() the while loop will erase all files in the s3 bucket. i've disincluded that in the process. this means the temporary folder on S3 where the file is saved on first upload will not be deleted when moved to it's final destination, which should be fine because that directory will be reused. Please test and review.
Comment #7
jcandan CreditAttribution: jcandan commentedComment #8
mgiffordAny thought about using this Amazon Module https://drupal.org/project/amazons3
Comment #9
weateus CreditAttribution: weateus commentedDear, jcandan, thanks for your patch. Now Filefield Path working with AmazomS3 module, but if delete file and folder after that empty, folder doesn't delete.
Any idea?
Comment #10
jcandan CreditAttribution: jcandan commented@mgifford, the patch works in conjunction with amazons3 module.
@weateus, yes, this was the case for me too. but wasn't high priority to fix for the client project i needed this on. I'll try to return to this after deadline's met.
Comment #11
weateus CreditAttribution: weateus commented@jcandal thanks for your reply.
New moment in work module with S3. When changed ImageCache (Core module) presets or try to delete we have a long time prepare and after that time-out error (PHP, Apache and MySQL time out have a high value — more 5 minutes).
After delete image in field and try to prepare imagecache presets we have a error
Fatal error: Call to a member function read() on a non-object in /var/www/weateus.ru/www/includes/file.inc on line 1361
In watchdog additionally information
Warning: dir(s3://styles/music_backstretch): failed to open dir: "AmazonS3StreamWrapper::dir_opendir" call failed in function file_unmanaged_delete_recursive() (строка 1360 в файле /var/www/weateus.ru/www/includes/file.inc).
Comment #12
neRok CreditAttribution: neRok commentedThe patch is not the correct way to fix this 'problem'.
FFP should NOT be patched every time a new remote file-system is created, as this patch intends. FFP works with the core file-system, and it also provides a way for other file-systems to organise themselves too. It does so by invoking
hook_filefield_paths_process_file
. In fact, the patch is editing 'filefield_paths' implementation of the hook.Therefore, the proper solution is for Amazon S3 module to implement the hook, OR a third module to bridge the 2. The hook would be something like below. And yes, the majority of the hook will be the same as
filefield_paths_filefield_paths_process_file
Comment #13
neRok CreditAttribution: neRok commentedFeature request in AmazonS3 module...
#1744610: Add support for File (Field) Paths module
Comment #14
justafish@neRok I disagree. All Drupal registered stream wrappers are made in the same way-extend DrupalStreamWrapperInterface and implement hook_stream_wrappers. The whole point of them is to not have to write glue code/implement hooks to have stuff like this work.
The issue is that instead of checking for local stream wrappers, it should be checking for stream wrappers that are writeable. See #1361884: Remotes files should not be processed
Patch attached. This does currently cause an issue with the amazons3 metadata caching, but we'll fix it over at #1744610: Add support for File (Field) Paths module. I also removed the function being used to check for local wrappers since it was calling file_get_stream_wrappers with every file loop. I've tested it with images locally and remotely as well as with media_youtube.
Comment #15
haysuess CreditAttribution: haysuess commentedAWESOME! The patch in #14 worked for me, using the S3 File System Module (https://drupal.org/project/s3fs).
Thanks so much :)
Comment #16
neRok CreditAttribution: neRok commented@justafish, you make sense. I have tested the patch, and local files are working (all I am setup for). I will email Deciphered and get him to commit the patch. Thanks.
Comment #17
JordanMagnuson CreditAttribution: JordanMagnuson commented#14 doesn't apply to latest filefield paths dev.
Comment #18
JordanMagnuson CreditAttribution: JordanMagnuson commentedComment #19
jcandan CreditAttribution: jcandan commented@jordanmagnuson, is it that the patch doesn't apply because it's been resolved in the dev version, or that the patch doesn't match dev's code?
Comment #20
klase CreditAttribution: klase commentedI applied #14 on the latest dev just now and it applies just fine. Also, the patch seems to work indeed. I use the Rackspace Cloud Files (https://www.drupal.org/project/cloud_files) module. More testing then a commit to dev please! Good job everyone.
Comment #21
genebobmiller CreditAttribution: genebobmiller commented#14 doesn't apply to latest filefield paths dev. Patch doesn't match dev's code, has this actually been committed?Comment #22
pedrospActually the patch doesn't apply as-is with last dev, maybe because there is a file inside a folder mixed with another in root module.
However a shameless-manual-patch does the job, guided with the patch text:
filefield_paths.module : removing last function
modules/filefield_paths.inc : updating a function
2015 resolution: learn how to produce/change a patch
Comment #23
szantog CreditAttribution: szantog as a volunteer commentedI could apply the patch, but it doesn't work. As a first look this is because of S3 module changes, but I'm not sure yet.
Comment #24
szantog CreditAttribution: szantog as a volunteer commentedI'll check it out within few days.
Comment #25
szantog CreditAttribution: szantog as a volunteer commentedFigured out.
file_stream_wrapper_uri_normalize() needs to generate proper S3 url (amazons3 module)
However amazons3 won't work yet with this patch, I'm 99% sure, it's up to amazons3 module. The target url is ok with this patch, now I'm getting error messages from AWS S3 streamwrapper when drupal_unlink is called in file_unmanaged_delete()
Comment #26
Deciphered CreditAttribution: Deciphered commentedTriggering tests.
Comment #27
Deciphered CreditAttribution: Deciphered commentedTriggering tests.
Comment #29
Deciphered CreditAttribution: Deciphered commented@szantog,
This is the section that is breaking, $scheme . '/ should be $scheme . '://. Changing this locally makes the tests pass, including an uncommitted test that creates a dummy read-only file stream wrapper.
Updated patch. I'll probably commit as per patch, but it would be nice if someone with an S3 setup can test it (I've yet to successfully setup S3 for testing purposes).
Comment #30
Deciphered CreditAttribution: Deciphered commentedUpdated patch to compensate for a recent update.
Still haven't been able to do a test with Amazon S3, and can't commit until I do. I suspect I just haven't gotten the my S3 account configured correctly.
Comment #32
Deciphered CreditAttribution: Deciphered commentedFixed the failing test.
Comment #34
Deciphered CreditAttribution: Deciphered commentedLate, really should roll patches when it's late :|
Comment #36
Deciphered CreditAttribution: Deciphered commentedOk, I have officially confirmed that this patch works with AmazonS3. The issue I got stuck with is that it appears the default bucket wasn't being prepended to the file path, so I had to ensure that the bucket was included in the File Path settings; this seems to be an issue with AmazonS3, as File (Field) Paths shouldn't have to do something special to work with one single File Stream Wrapper when it doesn't have to do anything for others.
Thanks all for your patience, persistence and patches. Fixed and committed.
Comment #38
trrroy CreditAttribution: trrroy as a volunteer commentedFor anyone having a similar issue as this with 7.x-1.x, I tried to check on this patch in the current version but it seems it was replaced by other functionality which did solve my problem. I found a settings page at /admin/config/media/file-system/filefield-paths where I was able to change the temp file directory.