Using File (Field) Paths with an alternate stream wrapper like Amazon S3 can be very inefficient as it will upload the unprocessed file to that stream and then (possibly an issue with the stream wrapper) re-upload/download then upload the processed file back to the stream.

A better solution to both this issue and general use of File (Field) Paths is to have all unprocessed files stored in the Temporary:// file system.

CommentFileSizeAuthor
#1 use_temporary-2383527-1.patch1.33 KBDeciphered
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Deciphered’s picture

Status: Active » Needs review
FileSize
1.33 KB
Deciphered’s picture

Status: Needs review » Needs work

Unfortunately it appears that Image styles don't get applied to files in the temporary file system, which is a pretty big issue for this. Will need to investigate further.

Deciphered’s picture

Issue with temporary:// Image styles reported: #2560139: Image styles in 'temporary://' not generated.

Deciphered’s picture

Status: Needs work » Fixed

Committed with workaround for image style issue.

  • Deciphered committed 9c4d95b on 7.x-1.x
    #2383527: Changed unprocessed destination to temporary://.
    
JordanMagnuson’s picture

Status: Fixed » Needs work

This specific commit causes Filefield Paths to stop working nicely with Filefield Sources for me... now when I try to save a node after uploading a file via filefield sources plupload (the initial upload works fine), I'm getting this error:

The specified file temporary://plupload_test_2mb_d.zip could not be copied, because no file by that name exists. Please check that you supplied the correct filename.

See screenshot: http://i.imgur.com/rWMpbrF.png

When I revert this commit, everything works fine...

Deciphered’s picture

So while File (Field) Paths did introduce this issue by doing something out of the ordinary, the issue isn't necessarily with File (Field) Paths.

The issue is that temporary location and the upload destination is the same, and the FileField Sources Plupload module isn't checking if that is the case, and just deleting the temporary location file, which is the real file.

Given that this is an issue in the Plupload source and the Remote source I will implement a workaround, which is to upload the file to a subdirectory in the temporary file system, but I will also be opening issues in the relevant modules as they really should be checking before they delete the file.

Deciphered’s picture

#2564593: Check temporary file isn't destination file before deleting.
#2564597: Check temporary file isn't destination file before deleting.

Take into account that while there is a patch for FileField Sources Plupload, that integration still doesn't work with File (Field) Paths anyway due to another issue (#1854450: Support for Media).

Workaround for File (Field) Paths will come later.

JordanMagnuson’s picture

Thanks Deciphered! Applied those patches, and everything is working great.

I'm confused when you say that FileField Sources Plupload won't work properly because of the Support for Media issue... are these issues even related? FileField Sources Plupload seems to be working fine at the moment with Filefield Paths with the patch you provided (though currently I'm only using the plupload sources widget to upload single value file fields -- ie not fields that allow multiple files -- if that makes any difference).

Deciphered’s picture

I didn't work for me for the same reason that the "Media" issue doesn't work, which is the file doesn't have the appropriate flag allowing the file to be processed. If it work for you, that's great.

I still intend to look further into the issue.

Deciphered’s picture

Workaround committed.

Deciphered’s picture

Status: Needs work » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.