Problem/Motivation

Hi everyone!

Since this fix was committed, VDE temp files are not deleted during garbage collection.
The watchdog has entries noting that the temporary file is still used by VDE module.
Looking at the code, it seems that the temp file is added to the file_usage table for better handling, but it never gets deleted from that table (when all OK).

I'm not sure where the best place is for correcting this, maybe in views_data_export_garbage_collect() ?
Just before file_delete is called, we could call file_usage_delete() for cleaning the usage entry, hence releasing the file for delete.

Any comments ?

Thank you all for your great work on this module !

Proposed resolution

@tvasilia So the call to file_delete in views_data_export_garbage_collect does actually pass in the force parameter, so that we don't need to clean up the usage before calling file delete.

However, because of our huge lifetime on the exports, the garbage collection doesn't happen for quite some time.
I think we could reduce this, and then your issues would go away I think.

Remaining tasks

Implement the change to the lifetime.

User interface changes

None

API changes

None

Data model changes

None

Comments

Steven Jones created an issue. See original summary.

steinmb’s picture

steinmb CreditAttribution: steinmb as a volunteer commented

Also notice that they are not getting cleaned up.

SELECT fid, uid, filename, status, timestamp FROM file_managed WHERE uri like 'temporary://views_data_export%' ORDER BY timestamp;

+--------+--------+-------------------------+--------+------------+
| fid    | uid    | filename                | status | timestamp  |
+--------+--------+-------------------------+--------+------------+
| 307342 |    509 | views_data_exportz7L8N2 |      0 | 1478614689 |
| 307399 | 303552 | views_data_exportVJmNSL |      0 | 1479049314 |
| 307400 | 303583 | views_data_exportx2tRsh |      0 | 1479109198 |
| 307401 | 303583 | views_data_exportSsJtD4 |      0 | 1479115815 |
| 307402 | 303583 | views_data_exportYydOKo |      0 | 1479115925 |
| 307403 | 303583 | views_data_exporteLYldF |      0 | 1479116141 |
| 307404 | 303583 | views_data_exporttqwNnk |      0 | 1479116237 |
| 307405 | 303583 | views_data_exportHXImkp |      0 | 1479116349 |
| 307406 | 303583 | views_data_export0d2H3u |      0 | 1479116439 |
| 307407 | 303553 | views_data_exportIlWgIs |      0 | 1479239707 |
| 307408 | 303552 | views_data_exporthAulZb |      0 | 1479240958 |
| 307409 | 303553 | views_data_exportv5w5xC |      0 | 1479240984 |
| 307410 | 303552 | views_data_exportkq58vn |      0 | 1479243111 |
| 307412 | 303583 | views_data_exportOMCI5y |      0 | 1479362074 |
| 307414 | 303583 | views_data_exportZAu7g4 |      0 | 1479387473 |
| 307416 | 303583 | views_data_exportJ0IMiO |      0 | 1479455207 |
| 307417 | 303583 | views_data_exportbvaL7f |      0 | 1479470041 |
| 307418 | 303583 | views_data_exportoJpYsz |      0 | 1479470121 |
+--------+--------+-------------------------+--------+------------+
18 rows in set (0.00 sec)

SELECT fid, module, id FROM file_usage WHERE module = 'views_data_export';

+--------+-------------------+----+
| fid    | module            | id |
+--------+-------------------+----+
| 307342 | views_data_export | 47 |
| 307399 | views_data_export | 52 |
| 307400 | views_data_export | 53 |
| 307401 | views_data_export | 54 |
| 307402 | views_data_export | 55 |
| 307403 | views_data_export | 56 |
| 307404 | views_data_export | 57 |
| 307405 | views_data_export | 58 |
| 307406 | views_data_export | 59 |
| 307407 | views_data_export | 60 |
| 307408 | views_data_export | 61 |
| 307409 | views_data_export | 62 |
| 307410 | views_data_export | 63 |
| 307412 | views_data_export | 64 |
| 307414 | views_data_export | 65 |
| 307416 | views_data_export | 66 |
| 307417 | views_data_export | 67 |
| 307418 | views_data_export | 68 |
+--------+-------------------+----+
18 rows in set (0.01 sec)
steinmb’s picture

steinmb CreditAttribution: steinmb as a volunteer commented
Category: Task » Support request

Looking at views_data_export_garbage_collect() and I think it is not a bug. I choose a week as default if not values is set to variables. In my system drush vget views_data_export_gc_expires came back empty.

Tested views_data_export_garbage_collect() by running:

drush vset views_data_export_gc_expires 200
drush eval ' views_data_export_garbage_collect(); '

SELECT fid, uid, filename, status, timestamp FROM file_managed WHERE uri like 'temporary://views_data_export%' ORDER BY timestamp;

+--------+-----+-------------------------+--------+------------+
| fid    | uid | filename                | status | timestamp  |
+--------+-----+-------------------------+--------+------------+
| 307342 | 509 | views_data_exportz7L8N2 |      0 | 1478614689 |
+--------+-----+-------------------------+--------+------------+

So only one left. I think it is good. Just need to configure it. Time to break out hook_update_N().