Using Drupal 7.41.

Problem/Motivation

After updating filefield_paths from beta4 to RC2 I was not able to upload new pictures (I guess it happens the same with files)(Also happens with filefield_paths-RC1)

The message "The file could not be uploaded." is displayed when I try to add new pictures to a node.
Activating the database logging the next message is logged:

The upload directory temporary://filefield_paths for the file field field_imatge could not be created or is not accessible. A newly uploaded file could not be saved in this directory as a consequence, and the upload was canceled.

I guess that servers with just one site or sharing the same group between sites will not have this problem.

I have a server with some websites in it.

In each site I have a /tmp folder and a /web where is placed the drupal.

Each site has its own user and group.
Let's say:

  • web1 has user1 and group1 and
  • web 2 has user2 and group2

Filefield_paths-RC1+ has changed the way or the place the temporary files are stored.

  • When an image is uploaded it's saved in the temp directory, by default "/tmp" in a directory called filefield_paths. This directory has the user and group privileges of the website is uploading the picture.
  • This /tmp folder is the server /tmp folder not the web /tmp folder.
  • In the image uploads also a styles directory is created to place the thumbnails of the picture. This directory has the same user and group owners as the filefield_paths directory.
  • The filefield_paths directory is removed after node save. (but if a picture is uploaded and removed before node saving the filefield_paths directory remains there with the deleted picture)
  • The styles folder remains in "/tmp" directory

So the first website is able to upload pictures but the second website is not able to do it because the filefield_paths folder is owned by the first website.

Scenario 1

  1. Website1 uploads a picture (filefiled_paths and styles directories are created in "/tmp" directory)
  2. Website 2 tries to upload a picture but the message "The file could not be uploaded." is displayed and website2 can't upload it. (this happens because the owner of the filefield_path is user1)

Scenario 2

  1. Website1 uploads a picture (filefiled_paths and styles directories are created in "/tmp" directory)
  2. Website1 saves the node. (filefield_paths directory is deleted, styles directory remains in the directory)
  3. Website2 tries to upload a picture. The pictures is uploaded but the thumbnail does not exist. (website2 is not able to create a new presset in "/tmp/styles" because this folder is owned by user1)
  4. Website2 saves the node and the pictures are there without any problem. If we edit the node the thumbnaila are also there because they are recreated.

Scenario 3

  1. Website1 uploads two pictures(picture1 and picture2) (filefiled_paths and styles directories are created in "/tmp" directory)
  2. website1 deletes picture1 before saving the node
  3. Website1 saves the node. (filefield_paths directory is not deleted it remains in the server because there is picture1 pendding to be saved. Styles directory remains in the directory)
  4. Website2 tries to upload a picture.
  5. Website 2 tries to upload a picture but the message "The file could not be uploaded." is displayed and website2 can't upload it.(this happens because the owner of the filefield_path is user1,and user2 can't access to that folder)

Well its a long explanation but I hope it is clear.

Proposed resolution

beta4 was working without problems so try to figure out whats wrong in RC1 and RC2

A temporary solution is to change the temporary directory path from "/tmp" to :

  • "tmp": but then is in drupalrootdirectory/tmp and is web accessible. Maybe adding an .htaccess with restrictions. Same approach as the used for private files directory placed in "drupalrootdirectory/sites/default/files"
  • /absolute-path-to-the-site/tmp
  • folder placed in "sites/default/files": but has the same problems as first option.

Comments

espurnes created an issue. See original summary.

espurnes’s picture

The best (temporal?) solution I found without changing the filefield_path functionality is to set the temporary directory of my websites to "../tmp". So the temporal files remains in the /tmp folder of each website avoiding permission conflicts between sites.

The Temporary directory can be changed in /admin/config/media/file-system.

Remaining tasks

Check if the module is working as expected considering the problem/motivation.

Deciphered’s picture

@esprnes,

Agreed, if you have access issues with /tmp, then change your temporary directory. This is not necessary an issue with File (Field) Paths, because any module that needs to read from and/or write to the temporary directory would be in a similar situation.

What does need to be done is some documentation, or maybe a dependency check, to make this a bit clearer.

The reason that the temporary file system is used is because it ensures that the unprocessed file doesn't get put into the final destination file system, which is especially an issue for those using Amazon S3 or similar. And while I did consider making this behaviour configurable, I honestly think that identifying configuration/permission issues with the Temporary directory/file system is not a bad thing.

Deciphered’s picture

P.s., Thank you so much for a very well detailed issue!

espurnes’s picture

Priority: Major » Normal

Hi Deciphered,

Thank you for your fast answer. This change of behaviour in RC1 was making all my sites with this module work wrong and it was hard to find the problem.

Right now I solved the problem changing the temporal directory to "../tmp".
I agree more documentation about this behaviour is needed.

Pending tasks

Write documentation explaining the way filefield_paths stores the temporal files and how to solve restriction access in servers with more than one website.

derekw’s picture

I'm on Pantheon and going from B4 to RC2 broke just my production site with the same error "File could not be uploaded". Dev and Test kept working with the RC2 code.

I could not see any errors in the logs that I have access to. Pantheon support was not helpful "It must be something about your configuration."

Dropping back to B4 restored the proper functionality.

Changing the temp folder to "../tmp" or "/tmp" does not work -- they must have it hard coded somewhere in the Pantheon profile because it would reset back to their /srv/bindings/[long-complex-alphanumeric]/tmp setting automatically.

Deciphered’s picture

Can you confirm that the temporary file system is writable? Check the Status report page. Because if it is there should be no problem, and if it isn't it seems like you have bigger problems.

Given the reports I don't doubt this is a valid issue, but I do need to be able to reproduce it before I can determine the appropriate fix. I'll reach out to the big three Drupal hosts to see if something can be arranged for testing purposes.

derekw’s picture

No problems reported on the Status Report page for me.

Looking at the phpinfo() page II note that sys_temp_dir has no value. Don't know if that's related.

JOINSO’s picture

Hi!

Same error on my site.
I am using a temporary folder in S3, and s3fs to link the temporary folder in each instance (/drupal-s3-temp), and I get the error in every upload:

The upload directory temporary://filefield_paths for the file field MY_FIELD could not be created or is not accessible.

To work again, I roll back to previous version.

Deciphered’s picture

So what I'm going to do is to make the temporary location configurable by the UI, and for the sake of not breaking anyones environments I will use the public filesystem, which will still resolve the issue of those using Amazon S3.

While I think it's very concerning the number of people who have un-writable temporary file systems, that's an issue for another day. I'll try to get the fix implemented today and release RC3.

  • Deciphered committed 72ed0b7 on 7.x-1.x
    #2607302: Added configurable temporary file location.
    
Deciphered’s picture

Status: Active » Fixed

Marking as Fixed, I've put the default temporary file location as 'public://filefield_paths' and made it configurable via the UI (and as a variable).

Would appreciate confirmation that this resolves the issues for anyone who reported this issue.

derekw’s picture

RC3 working for me on Pantheon, dev, test, and live!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

piyamirum’s picture

I am facing a different kind of problem,
the site is migrated from staging server. in staging media was managed from aws. the previous developer used s3 modules
now after the migration I have disabled s3 related modules. and selected Default download method as
Public local files served by the webserver.but it is preventing me to upload image from my localhost it is throwing an error

piyamirum’s picture

Got the solution, after removing the s3 modules from local, I had to select upload destination, as the destination became null after removing s3 settings

Bibhab’s picture

Just change the file permission to 777 (sites/default/files). It works for me and hope will work for you as well.

harshadananjaya’s picture

set 775 permissions to "tmp" directory and "/sites/default/file" directory.
set apache:apache user group to "tmp" directory and "/sites/default/file" directory.

chmod -R 775 tmp
chmod -R 775 sites/default/files

chown -R apache:apache tmp
chown -R apache:apache sites/default/files/