Set file status to permanent.

Sounds like this would solve #2409075: Did not delete temporary file "public://personals/personpic/110623-9611.jpg" during garbage collection, because it is in use by the following modules: file..
Did you notice the error reported in that issue in your log?

Is this happening with a local file, or a remote file? Remote files are saved with file_save_data() which sets the status automatically.

On a website where I encountered this issue, it's about a local file. Actually, I wrote an alternative file mapper for that website that produces a file out of a base64 encoded string, saves that in the file temporary directory and then passes the file url of the result to file_feeds_set_target(). The website in question is running 7.x-2.0-alpha8+71-dev (January 16, 2015), so I'll have to recheck with the latest dev.

Sample code of the mapper:

foreach ($values as $key => $value) {
  // (...)
  $image_data = base64_decode($value);
  $dest = file_directory_temp() . '/' . $file_name;
  $file_dest = file_unmanaged_save_data($image_data, $dest);
  $values[$key] = new FeedsEnclosure($file_dest, file_get_mimetype($file_dest));
}
file_feeds_set_target($source, $entity, $field_name . ':uri', $values, $mapping);

I don't think setting the status on the field has any effect, unless I'm missing something.

Hmm, that's not the issue.

file_field_presave() and image_field_presave() both set the status to permanent automatically.

Marked #2547265: "fail" to import to private:// as a duplicate.

Summary of that issue:

• In addition of the file status, the file owner is also set to 0. Might have the same cause or hint to a possible cause of this issue.
• file_field_presave() and image_field_presave() set the file status to FILE_STATUS_PERMANENT (which equals to 1, see /includes/file.inc).
• The patch from #7 seems to fix the issue, but it may just fix a symptom of the problem.
• Filefield Paths is considered as a suspect, though no prove exists for that (I couldn't reproduce the issue on a clean install with Filefield Paths yet).

The process for trying to find the cause of the issue is:

1. Setup:
   1. Create a content type with a file field.
   2. Create an importer that at least maps to this file field.
   3. Do an import using the default settings.
   4. Check the file_managed table for status and uid.
2. Continues tries:
   1. Change a setting or add a module.
   2. Do an import.
   3. Check the file_managed table for status and uid.
   4. Repeat until the file_managed table shows you a record with status = 0 and/or uid = 0.
3. Write down the list of steps required to reproduce the issue and report back.

@MegaChriz

I did several tests with different configurations.

Without the "Enable File (Field) Paths" option on the field image > No problem
Case 1: single image > uid = 0 & status = 1
Case 2: single image + rewrite path with Feeds Tamper > uid = 0 & status = 1
Case 3: multiple images > uid = 0 & status = 1
Case 4: multiple images + rewrite paths with Feeds Tamper > uid = 0 & status = 1

With the "Enable File (Field) Paths" option, and if I specify a specific destination for the images > Problem
Case 1: single image > uid = 0 & status = 0
Case 2: single image + rewrite path with Feeds Tamper > uid = 0 & status = 0
Case 3: multiple images > uid = 0 & status = 0
Case 4: multiple images + rewrite paths with Feeds Tamper > uid = 0 & status = 0

For me, the problem comes from the File (Field) Paths module. (or an incompatibility betwenn Feeds & File (Field) Paths)

@kumkum29
Ah, this is useful. So my suspicion that File (Field) Paths might have something to do with the issue appears to be right. Did you try to reproduce the issue on a clean install as well (as I said in #9 I wasn't able to reproduce it yet on a clean install)? Can you give an example of what you mean with "a specific destination"? Is it a folder with or without use of tokens? Or is it just renaming a file using tokens?

@MegaChriz

In the "File path" setting I have specified a specific directory ("images") and for the file name I use the default tokens: [file:ffp-name-only-original].[file:ffp-extension-original]. On another site, I get the warnings (in logs) with a specific directory (with tokens), and with tokens in the field name.

In all cases, I get always 0 for the uid in the file_managed table of db.

Still have no success to reproduce this on a clean install, even with File (Field) Paths enabled and directory set to 'images' :(.
I did notice an interesting difference regarding 'uid'. When importing using the UI (and logged in as user 1), the owner of the imported files becomes '1'. But when not logged in, and trigger the import either via cron or drush, the owner of the imported files becomes '0'.

@kumkum29
How did you trigger the import?

Hello MegaChriz,

I reopen this old subject, after having make new tests. I did notice the same behavior during the importation. If we use the UI, the owner of the imported files is 1. But if the importation is making with cron, we get 0 in the uid value.

Perhaps this is a starting point to find the solution?

I think I found the cause of the bug! And the bug seems to only be exposed when filefield_paths is enabled.

See the following line in feeds/mappers/file.inc (± line 187):

$field[$language][$delta] += (array) $v->getFile($destination, $mapping['file_exists']);
// @todo: Figure out how to properly populate this field.

This results into the value on the field to become something like this:

[0] => Array

  [uid] => 0
  [uri] => public://image1.jpg
  [filemime] => image/jpeg
  [filename] => image1.jpg
  [fid] => 163941
  [timestamp] => 1485866549
  [filesize] => 2784
  [origname] => image1.jpg
  [status] => 0
  [display] => 1

Note the status value.

When uploading a file using a node edit form, the field value looks something like this:

[0] => Array

  [alt] => 
  [title] => 
  [fid] => 163939
  [display] => 1
  [width] => 240
  [height] => 240
  [description] => 
  [upload_button] => Upload
  [remove_button] => Remove
  [upload] => 

When filefield_paths is not enabled, this doesn't cause any direct issues. But when it is enabled, then filefield_paths uses the data on the field to build a new file (filefield_paths/modules/filefield_paths.inc, ± line 104, function filefield_paths_filefield_paths_process_file()):

if (file_prepare_directory($dirname, FILE_CREATE_DIRECTORY) && $new_file = file_move((object) $old_file, $file['uri'])) {

filefield_paths casts the data on the field to an object.
Before the code from filefield_paths is executed, the file's status gets set to permanent via file_field_presave() or image_field_presave(), but the status value on the field stays the same: 0. Since filefield_paths uses the data on the field, the file gets the status 0.

What I think that Feeds should do is only save the values relevant for the field instead of casting a file object to an array. See attached patch.
I'm still not sure why I am not able to reproduce the issue on a clean install though, but it probably matters what happens in FeedsEnclosure::getFile. In my case, the line $file = file_copy($file, $destination, $replace); gets executed.

Hm, the tests are broken caused by this issue in Drupal core: #1621356: Pass all the parameters of hook_options_list() to options_list_callback.
Note to myself: calling taxonomy_allowed_values() was added in #1019688: Taxonomy mapper options: term name +tid, term name, tid, guid (avoids mapping error for Numeric taxonomy term, too).

@MegaChriz

I'm happy to see that the problem seems to be identified. I'm not a good developer to help you to resolve this issue with a code. But I can test your patch and follow this issue.

@kumkum29
It would be great if you want to test the patch in #15! I see with the information I have now if I can reproduce the issue on a clean install. This way I perhaps could create an automated test for the bug.

@MegaChriz

I have tested your patch #15:

1- In the "file_managed" table, the new files have "0" for the "uid" & "status" columns.
2- I get another problem with this patch. The files seems to be duplicated in the root of files directory (filename with a "0", e.g. myimage_0.jpg) , before that the "filefield paths" rename & move the files in the good directory.
I badly patched the module. I do a test again. Sorry....

I have tested your patch #15:

In the "file_managed" table, the new files have "0" for the "uid" & "status" columns. So I get always warnings in the logs...

@MegaChriz

For me, the patch #15 don't resolve this problem (maybe I have omitted anything...). Do you think resolve this issue by another way?

Thanks.

@kumkum29
It looks like that it doesn't fix the whole problem for me either: I still get tons of "Could not delete temporary file" error messages in the log on a site where I applied the patch from #15, but these may be of files that aren't re-imported again.

Setting to "Needs Work" as explained in #22.

Hi all,

don't know if this is relevant. But I did notice a value in the exported feeds-importer that I did not set so.

In the UI settings of the Node Processor I set the expiry to "Never". But when I export the feeds-importer I see the expire_period value set to "3600". Does 3600 mean "Never" or does it stand for "1 hour" (60 Seconds x 60)?

 ),
  'content_type' => '',
  'update' => 0,
  'import_period' => '-1',
  'expire_period' => 3600,
  'import_on_create' => 1,
  'process_in_background' => 0,
);

Could this be the reason that the files are not set to "permanent"?

Regards,
Paul

Drupal 7 reached end of life and the D7 version of Feeds is no longer being developed. To keep the issue queue focused on supported versions, we’re closing older D7 issues.

If you still have questions about using Feeds on Drupal 7, feel free to ask. While we won’t fix D7 bugs anymore, we’re happy to offer guidance to help you move forward. You can do so by opening (or reopening) a D7 issue, or by reaching out in the #feeds channel on Drupal Slack.

If this issue is still relevant for Drupal 10+, please open a follow-up issue or merge request with proposed changes. Contributions are always welcome!