Hi,

I've created a simple custom feature that use a custom content type using CCK filefield multiple attachment.

I've just discovered that some attached file just disappear from /files directory while the node content is still there... trying to access it the response is a 404.

Please help, is urgent! I'm in production and I can't trust uploading files.

Thank you,

enrico

CommentFileSizeAuthor
#34 socios.zip31.75 KBenricostn
#10 socios.zip32.33 KBenricostn
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

enricostn’s picture

I've just discovered that the file URL ends with "_0" suffix, while the correct URL is without that suffix.

For example in the node the URL is "/files/socios/6/filename_0.pdf" while in the folder I have "/files/socios/6/filename.pdf"

so this issue could be related with some "same filename" algorithm that add the "_0" suffix?

quicksketch’s picture

FileField will automatically append a _0 if it detects that a file with the same name is already in that place. Usually this works without any problem, but if you have the FileField Paths module installed, that's been known to cause some problems due to the way it moves files around before and after the node is saved.

enricostn’s picture

Hi quicksketch,

thank you for the help. I don't have FileField Paths module installed. I've noticed that the issue could be related to files attached one time and then reloaded another day. I have activated revisions now to have a better control.

In the "files" table for example now I have 2 items with the same filename but with a different URL (with and without "_0" suffix).

quicksketch’s picture

Are you using the private file system (paths that start with "system/files/")? That sounds like it could be causing your problems. If users do not have access to previous revisions where the file was originally attached, it won't allow you to access those files either.

I've noticed that the issue could be related to files attached one time and then reloaded another day. I have activated revisions now to have a better control.

I don't quite understand this statement.

It sounds like this problem only happens some of the time, not on all files or on all pieces of content. Can you start with a new node and try to reproduce the problem? If you can provide steps to recreate the problem from scratch, it may be a bug in the module that we can fix.

enricostn’s picture

Hi,

I'm using Public Download Method in File System configuration and the Transliteration is turned ON.

In that statement I would say that looking at revision history of the nodes I've noticed that the URL fail when an attached file was reloaded one or more times. This type of node contain a lot (~ 20) of attached files that users usually update (doc or pdf files generally). The user is always the same for every node: if a node was created by user A only user A could access and modify that node.

If I create a new node and then update it one and more times removing and uploading the same file with the same filename, in the /files/socios/1 directory (where socios is my content type name and 1 is my uid) I find all the files I've uploaded with "_0", "_1", "_2", etc suffix. That is correct, but if I look in the folder of the users affected by the issue I could find only the original file with the filename without any suffix... while the node still demand (correctly I suppose) the new filename with some suffix.

enricostn’s picture

Hi,

the suffix ("_0", etc.) way of investigation about this problem it's not the correct way, I think. Now a lot of new files uploaded only one time just disappear!

So, the problem It's not related to re-upload same filename files...

Please help me to reproduce the error, give me some hint. Please. I'm in production and with the standard node attachment I don't have the same problem, just with FileFields multiple upload.

I've not written or modified lines of code, just created the new content type and added and defined the CCK fields.

enricostn’s picture

not really sure about that, could anyone explain the "_0" process used by CCK FileField?

enricostn’s picture

if it could help, when I'm trying to display a file affected by this issue in drupal "recent logs" I could see these warnings:

page not found 24/01/2012 - 14:42 sites/default/files/socios/73/20120120_uhtl_unisul_a ... admin
filefield 24/01/2012 - 14:42 FileField was trying to display the file ... admin
filefield 24/01/2012 - 14:42 FileField was trying to display the file ... admin
filefield 24/01/2012 - 14:42 FileField was trying to display the file ... admin

and if I enter one of the filefield type it shows me that:

Type filefield
Date Tuesday, 24 January, 2012 - 14:42
User admin
Location http://www.riaipe3.net/gestion/node/539/revisions/view/834/835
Referrer http://www.riaipe3.net/admin/reports/event/51129
Message FileField was trying to display the file sites/default/files/socios/73/20120120_uhtl_unisul_annex_l_staff_costs_form_christian.doc, but it does not exist.
Severity warning
Hostname 79.157.73.32
Operations

enricostn’s picture

I've exported the feature to post here some files:

socios.module


include_once('socios.features.inc');

/**
 * Implementation of hook_init().
 */
function socios_init() {
  drupal_add_css(drupal_get_path('module', 'socios') .'/socios.css');
}

/**
 * Implementation of hook_perm().
 */
function socios_perm() {
  return array('access any socios content');
}

socios.features.inc


/**
 * Implementation of hook_ctools_plugin_api().
 */
function socios_ctools_plugin_api() {
  list($module, $api) = func_get_args();
  if ($module == "context" && $api == "context") {
    return array("version" => 3);
  }
  elseif ($module == "strongarm" && $api == "strongarm") {
    return array("version" => 1);
  }
}

/**
 * Implementation of hook_node_info().
 */
function socios_node_info() {
  $items = array(
    'socios' => array(
      'name' => t('Socios'),
      'module' => 'features',
      'description' => '',
      'has_title' => '1',
      'title_label' => t('Título'),
      'has_body' => '1',
      'body_label' => t('Cuerpo'),
      'min_word_count' => '0',
      'help' => '',
    ),
  );
  return $items;
}

/**
 * Implementation of hook_views_api().
 */
function socios_views_api() {
  return array(
    'api' => '2',
  );
}

socios.features.content.inc

http://pastebin.com/jv5RTE5A

socios.features.fieldgroup.inc

http://pastebin.com/DUaF8Msg

socios.strongarm.inc

http://pastebin.com/wAwQS3Kr

enricostn’s picture

FileSize
32.33 KB

the complete feature.

quicksketch’s picture

not really sure about that, could anyone explain the "_0" process used by CCK FileField?

The "_0" process is used by all of Drupal to prevent overwriting files with the same file name. FileField calls file_save_upload() (provided by core) which in turn calls file_destination(), which finally calls file_create_filename(), which contains this code:

if (file_exists($dest)) {
    // Destination file already exists, generate an alternative.
    if ($pos = strrpos($basename, '.')) {
      $name = substr($basename, 0, $pos);
      $ext = substr($basename, $pos);
    }
    else {
      $name = $basename;
      $ext = '';
    }

    $counter = 0;
    do {
      $dest = $directory . '/' . $name . '_' . $counter++ . $ext;
    } while (file_exists($dest));
  }

So in other words, Drupal keeps appending _0, _1, _2, _3, etc to file names as long as there is already a file that exists with those names.

API documentation:
http://api.drupal.org/api/drupal/includes--file.inc/function/file_save_u...
http://api.drupal.org/api/drupal/includes--file.inc/function/file_destin...
http://api.drupal.org/api/drupal/includes--file.inc/function/file_create...

I've exported the feature to post here some files:

I'm looking at this feature to see if I can reproduce the problem. This is an excellent way of sending over your field configuration but I can't usually suggest it because of the process of creating a feature is beyond what help I can feasibly provide. Anyway I'm having a look now and we'll see if it's reproducible. The field configuration doesn't seem to have any signs of other upload-related modules. Looking through your modules page, are there any other modules related to File handling that you have installed (you mentioned transliteration already, and that you don't have FileField Paths). You should try disabling any file-related modules and see if it helps remove the problem.

Also I'm concerned that this may be something caused by a server configuration that is causing your site to have troubles. With the number of file fields you have on a single node form, you could be running into limitations put in place by your hosting provider. The Suhosin PHP extension is known to cause some difficulties with FileField. See this handbook page: http://drupal.org/node/423478. You should check if your server has this extension installed.

It'd also be good to export your entire database into a MySQL dump and copy all your files to another server (such as your own laptop with MAMP or WAMP installed) and see if you can reproduce the problem on a different server. If so, that would clarify that this is a problem with code, rather than server configuration.

quicksketch’s picture

Well trying this feature out on my local machine doesn't have any particular problems. Is there a set of steps you can use on your own server to make this problem come up? It's hard to debug any problem if you can't consistently reproduce it.

enricostn’s picture

Thank you for your time!

One question: that feature run in an OpenAtrium context. Could it be related?

I've opened an issue on OA community too (https://community.openatrium.com/issues/node/3779) but they don't think it could be related to some OA "hack".

Suhosin PHP extension doesn't seem to be installed.

enricostn’s picture

hi,

today another file is disappeared, I report that because this case can't be related to renaming "same namefile" process. The user just upload the file with other 10 files attached to a new node and now only this file is just disappeared, the others attached to the node still works fine.

Hope this could help.

thank you

quicksketch’s picture

The user just upload the file with other 10 files attached to a new node and now only this file is just disappeared, the others attached to the node still works fine.

That problem sounds identical to what you'd experience of Suhosin were eating the uploads (especially if it were the last upload the user did that disappeared). If it's not Suhosin, it could be a similar module attempting to tighten security.

enricostn’s picture

I've pasted our phpinfo here:

http://pastebin.com/0gwFhXWU

could you see any module similar to Suhosin?

thank you for the help!

quicksketch’s picture

Looks like the (newish) option for max_file_uploads is probably causing you trouble. Try increasing this beyond 20.

It looks like this problem becomes much more common when multiple upload fields are on the same page, because every upload may count as 5 uploads towards this total if you have 5 upload fields on the page. See http://allinthehead.com/retro/349/the-curse-of-max_file_uploads. This may be fixed in newer versions of PHP according to their bug tracker (https://bugs.php.net/bug.php?id=50692) but I'm not sure how that relates to PHP 5.3 (the report was against 5.2.12).

enricostn’s picture

OMG!

thank you for the hint!

just two doubts:

if the limit of 20 is passed (in our case very possible...):

1. the server should log out some kind of error code, correct? where I could look for it?

2. the node should not be saved at all or should be saved only with "in limit" attachments, what do you think?

thank you!!!

quicksketch’s picture

After some Googling, it looks like you should be seeing a very logical error your httpd.log file (assuming you're using Apache):

http://www.simpleviewer.net/forum/viewtopic.php?id=10191

[29-Mar-2010 15:55:15] PHP Warning: Maximum number of allowable file uploads has been exceeded in Unknown on line 0

Since it's a PHP Warning, it probably also gets logged to Drupal's watchdog and you should be able to find the error under admin/reports/dblog.

enricostn’s picture

Hi,

finally speaking with hosting support I've enabled PHP logs....... so we will see if any warning will be recorded there.

In admin/reports/dblog no PHP warnings were found.

Yesterday another user experimented the issue and now we are running PHP with max_file_uploads = 200....

The node's revision's history tell us nothing about deleting or uploading files, just say that the user saved 2 times the node without modifications... but the user is sure that was deleting and re-uploading the file.

Now in that node we still see the bad url pointing to nowhere.

what do you think?

thank you so much for the help.

enricostn’s picture

today another case similar to #20

this night I was able to access the file, but this morning (after 7h) the file is not there, and the URL give a 404.

I'm doing an hourly based backup for that folder so I could restore the file, but we still have the problem.

no PHP warnings in the log.

quicksketch’s picture

this night I was able to access the file, but this morning (after 7h) the file is not there, and the URL give a 404.

This sounds like a different problem possibly. Previously, it sounded as though files were immediately missing, but this is only after a certain amount of time. This sounds like a file was uploaded, but then the node was never saved. Such files are marked as temporary with a "status" of 0 in the "files" database table. Drupal's cron.php will delete temporary files after 6 hours if they haven't been marked permanent (a status of 1).

dgsiegel’s picture

i can confirm the issue, images are missing randomly on several nodes. this is what we have been able to figure out so far:

- images appear to get removed randomly, even on older nodes
- all of those node types use multiple image attachments
- some of the affected node types use revisions, some not
- the files only get removed on the file system, the db entry in the files table is still there
- all relevant files have file status 1 in the db, which means they are not temporary files and hence won't get deleted by cron
- the max_file_uploads on php 5.3 works fine and does not seem to be the problem here
- file size does not seem to be the problem here, as it affects small to large files

thanks a lot for any hints or insights!

enricostn’s picture

This sounds like a different problem possibly.

Yes, I think that exists more than one problem. Today 9 files "disappeared": 2 of them could be restored from backup (issue type A) and the rest never existed on our backup, but on the node we have the url (404).

I'm still not sure but probably the "max_file_uploads" parameter doesn't affect on this issue, I tell that because there are no PHP Warnings in the logs.

quicksketch’s picture

Well I'm pretty much out of ideas on this one.

- the files only get removed on the file system, the db entry in the files table is still there

This almost entirely eliminates FileField as possibly being responsible for the problem. FileField will *never* delete a file and leave a record behind in the database. Though other modules aren't prevented from deleting directly, I think it's most likely server configuration to be leading to these problems.

enricostn’s picture

I think it's most likely server configuration to be leading to these problems

Ok, but what kind of configuration?

quicksketch’s picture

Ok, but what kind of configuration?

Other than what's listed here http://drupal.org/node/423478 and the new max_file_uploads directive discussed above, I don't have any immediate ideas. I can't provide support on every user's individual server configuration, and unless this is shown to be a bug in FileField (by providing steps to reproduce the problem), I don't think I can offer much more help here.

dgsiegel’s picture

This almost entirely eliminates FileField as possibly being responsible for the problem. FileField will *never* delete a file and leave a record behind in the database. Though other modules aren't prevented from deleting directly, I think it's most likely server configuration to be leading to these problems.

to be honest we are not quite sure whether the files are deleted sometime after the upload or never got uploaded.

deekayen’s picture

Version: 6.x-3.10 » 6.x-3.9

We've been running 3.9 since Jan 11, 2011 and this just now cropped up for us.

user@webserver:~$ php --version
PHP 5.2.10-5hardy~ppa2 with Suhosin-Patch 0.9.7 (cli) (built: Mar 1 2010 15:55:25)

We're running PHP as fastcgi so each home directory vhost will run as its own user. The files dir is correctly chmodded 777. The best workaround we've found so far is to first upload a png file because that works, then remove the png and upload the jpg if we really wanted a jpg, which seems to stick, but we're not so sure that the file won't disappear later from some other seemingly unrelated event. If we start with the jpg, it will preview, but disappears when you hit save on the node. After doing the png first, we can upload and remove jpgs to the same field on the same node all we want so far.

Running cron and clearing cache doesn't seem to have any impact on the disappearance of stickiness of a file.

vabzdad’s picture

This almost entirely eliminates FileField as possibly being responsible for the problem. FileField will *never* delete a file and leave a record behind in the database.

For what it's worth guys, I have seen the following in Drupal 6:
1) Using FileField, upload a file to a node (don't save the node yet).
2) The file is uploaded. But the Files table has two entries. Unique fid's. But the same filename and filepath. Both have a status of 0 - temporary.
3) After saving the node, the status of one of the two entries is changed to 1 - permanent.
4) Approx. 6 hours later at a cron run, the system module deletes the temporary file and the entry in the Files table corresponding to the temporary file.
5) The file is gone, but the entry in the Files table with the status of '1' remains.

I haven't figured out what the bug is with the uploader, if it is with ahah.js or whatever. I do know this only seems to occur in Internet Explorer.

deekayen’s picture

We figured out our issue. Turned out to be an rsync we forgot about which had --delete in the arguments.

enricostn’s picture

here we go!

we have changed our application logic to avoid the use of multiple upload fields, so now we have 1 node = 1 attachment.

All was fine, and seems that the "404 phantom" was dead, until today... one user 2 days ago uploaded 1 attachment and today the file is not there... 404.

now I really don't know what could be the reason, anyway... I was thinking that Filefield upload process rely on JavaScript and I checked if the user was using an old browser or something like that. But he use Safari 5.1.5...

So, the problem is not server side (hard disks, etc.) and is not client side (browser, etc.)....

I think we need to debug this seriously, could quicksketch help us? we're in production and our users during next months will use this feature A LOT.

please, help us... I think there is something really wrong and we need to fix it.

thank you,

enricostn

enricostn’s picture

please,

anyone interested on helping us debugging this issue?

we could pay something to get this fixed.

thank you

enricostn’s picture

FileSize
31.75 KB

In attachment the last version of our feature.

Thank you.

vectoroc’s picture

@enricostn: have you solved your problem?

enricostn’s picture

hi vectoroc,

I've solved this... I hope... I've just upgraded to a newer Drupal version and it seems that all is right now... but I'm not satisfied, I don't know what was failing, I cannot (don't know how to) test/debug this kind of things... so I don't think we could say that this issue is SOLVED... but the problem seems it's gone...

I hope this can help you.

bye enrico

vectoroc’s picture

I'm expecting the same problem on one site

Sometimes drupal creates few {files} rows for one file (as described in #30). What I notices from apache's access.log:

xx.xx.xx.xx - - [24/Aug/2012:17:32:28 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:32:26 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:32:28 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:32:38 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:32:27 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:34:20 +0400] "GET /sites/default/files/imagefield_thumbs/u4/cat-logo/telos600_logo.jpg?1345815259 HTTP/1.1" 200 253 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:31:18 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1224 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:32:27 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:32:27 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:31:58 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1217 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
se.rv.er.ip - - [24/Aug/2012:17:30:01 +0400] "GET /cron HTTP/1.0" 200 436 "-" "Wget/1.12 (linux-gnu)"
xx.xx.xx.xx - - [24/Aug/2012:17:34:56 +0400] "POST /filefield/ahah/example/field_cat_photo/0 HTTP/1.1" 200 1226 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"
xx.xx.xx.xx - - [24/Aug/2012:17:34:59 +0400] "POST /content/js_add_more/example/field_cat_photo HTTP/1.1" 200 2495 "http://example.com/node/add/example?destination=front" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.83 Safari/537.1"

Perhaps it happens on intersection with some cron tasks

| 8752 | 4 | telos600_logo.jpg | sites/default/files/u4/cat-logo/telos600_logo.jpg | image/jpeg | 22971 | 0 | 1345815259 |
| 8753 | 4 | telos600_logo.jpg | sites/default/files/u4/cat-logo/telos600_logo.jpg | image/jpeg | 22971 | 1 | 1345815259 |
| 8754 | 4 | telos600_logo.jpg | sites/default/files/u4/cat-logo/telos600_logo.jpg | image/jpeg | 22971 | 0 | 1345815259 |
... and another 13 copies

From watchdog:

Could not set permissions on destination file: sites/default/files/imagefield_thumbs/u4/cat-logo/telos600_logo.jpg

Related issue: #702822: All site images missing

I see one (unreal) possibility to reproduce the problem:

  1. slow down server
  2. run few uploads with identical filenames
  3. all queries will cross line at ~ same time
  4. move_uploaded_file wont check file existence, it simply overwrites it
quicksketch’s picture

Even in the slow-server situation, I don't think multiple rows would be inserted for the exact same file name. FileField will always rename a file if a file with the same name exists on *disk*, and it renames and saves the file to disk *before* it does the database query to insert the file into the database. So no matter how slow your server was or how fast the requests come in, the files would all get renamed _0, _1, _2, etc. Do you have a module installed that modifies the behavior of file saving, such as FileField Paths?

vectoroc’s picture

> FileField will always rename a file
I explained above my point and I see that filefield checks file existence before saving. But filefield does not lock globally filename.

> Do you have a module installed that modifies the behavior of file saving, such as FileField Paths?
I dont have FileField Paths installed (and I dont remember such modules in my installation).
Problem occurs when user tries to upload file (with ahah), server hangs, then user presses "Save" button.

vectoroc’s picture

I got it!
I've reproduces the problem playing with table locks (e.g. mysqldump locks all tables, right?)

Steps:

  • open page node/add/page
  • select file to be attached
  • lock db tables (i used LOCK TABLES `cache` READ, `files` WRITE, `system` READ; query)
  • press "attach" button
  • press submit on node form (few times, it does not happen every time)
  • unlock tables (UNLOCK TABLES;)

Next check the result select fid, filepath, status, timestamp from files order by fid desc limit 2;
You should see something like this

| 24255 | sites/default/files/u1/cat-photos/catalogue_11.jpg |      0 | 1348829068 |
| 24254 | sites/default/files/u1/cat-photos/catalogue_11.jpg |      0 | 1348829068 |
vectoroc’s picture

Another attempt

| 24261 | sites/default/files/u1/cat-photos/ico_metro_3.png |      0 | 1348834941 |
| 24260 | sites/default/files/u1/cat-photos/ico_metro_2.png |      0 | 1348834941 |
| 24259 | sites/default/files/u1/cat-photos/ico_metro_2.png |      0 | 1348834941 |
| 24258 | sites/default/files/u1/cat-photos/ico_metro_1.png |      0 | 1348834941 |
| 24257 | sites/default/files/u1/cat-photos/ico_metro_0.png |      0 | 1348834891 |
vectoroc’s picture

Also, it happens on my working pc (ubuntu 10.04), but I can't repeat it on my macbook

dhigby’s picture

It seems like progress was being made on this issue six months ago, but nothing since?

I have a client in Africa with poor internet connectivity and the server is in Europe. They are using file attach in Drupal 6 to attach images to a page, and then include those images with IMCE editor on the page. But the images are turning up missing on the server even after they were attached and displaying correctly in the WYSIWYG editor.

I suspect the user was treating the drupal page as a word document and leaving it open for long periods of time, in which case (correct me if I'm wrong) the cron script would treat the page as abandoned and delete the images by the time the save button was clicked hours later.

So I slowed down cron to run every three hours instead of every hour, and I told them to add the photos and save the page quickly. They did so, but this time out of the three pictures uploaded, two were saved and the third one was deleted. The pictures, all three of them were displaying on the page when it was saved, but hours later one went missing.

This is not a very workable solution for the client, and I'd appreciate any insight in to how to stop this behavior. Reading above, I wonder if:

  • The user is clicking multiple times on the upload button (maybe only double-clicking)
  • The browser they are using is playing a role in this (I have to ask which), maybe they should try a different one?
  • Maybe there is a different way to accomplish the adding of pictures to pages that doesn't need the filefield module.

Thanks for any help you can give.

quicksketch’s picture

I suspect the user was treating the drupal page as a word document and leaving it open for long periods of time, in which case (correct me if I'm wrong) the cron script would treat the page as abandoned and delete the images by the time the save button was clicked hours later.

This is possible, but the file cleanup only deletes temporary files that are older than 6 hours, no matter how frequently cron runs, so I don't think that's the exact problem you're running into.

akosipax’s picture

Version: 6.x-3.9 » 6.x-3.13
Issue summary: View changes

I am still experiencing the same issue as well: file just disappear from the files directory.

1) Upload the files in the form.
2) Checked the /files directory and visited the file URL and the file's there as expected.
3) Save the node.
4) After a few seconds, the file is not there in the directory anymore and gives a 404.

This seems like a sporadic issue since not all of our nodes experience this issue [at least not as immediately as this certain node I am dealing with]

I don't have Suhosin installed.

AndyPT’s picture

Hello,
(3 years or 2 months later)
I'm also experiencing this issue with Drupal 6.
Attachments on SAVED nodes (such as PDF and PNG, multiple sizes), fully tested, simply disappear from the filesystem.
Information on the database is apparently unchanged (node appears to have the file attached, but since it's gone from the filesystem, a 404 error occurs on retrieval).
Any idea on how to solve it?
Thanks.

Sivaji_Ganesh_Jojodae’s picture

I'm experiencing same issue as described in #45 and #46. Any advice will be appreciated.

Thank you!

AndyPT’s picture

sivaji, I haven't found a solution for this problem, but im my case I might have found a cause (maybe not all cases).

Steps to replicate:
1. Create a new content (page, etc.), but don't save it;
2. Add a file, and take note of the assigned URI;
3. Verify the URI works;
4. Leave the new content page without saving;
5. The URI will work for a few hours. Then the cron will remove the file. This is by design (the node wasn't saved, so any resources will be deleted as well).

This automatic removal might be the problem.

There is also another case where I added a file, then removed it from disk only (not the reference on the content). Then I tried to add it again (2nd reference - got the same URI) and removed the first reference (these steps in no particular order). A few hours later the cron removed the new file, thinking it was the old one (same URI) for which the reference (1st) was removed.

I think this accounts for most of my lost files (but not all).

Hope this helps.

Sivaji_Ganesh_Jojodae’s picture

AndyPT, thanks for your reply. Mine is different from cases mentioned in #48. However same as in #45 and #46.

vids’s picture

I am having the same issue.

If user upload any file for private location, it disappear after few days. In the configuration, user can upload unlimited files.
Did you find any solution?

pwolanin’s picture

Status: Active » Closed (won't fix)

seems no one has debugged to a root cause, so this won't be fixed