Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Hi
One of the sites I have uses views data export. Nothing fancy, about 20 different views pages that expose data in different formats. The db. admin suddenly sendt me an email that the db. was growing out of space. Had a look and found 21 000 tables views_data_export_index%.
Looged at the code and found the garbage colletor function. Tested that one from drush (drush eval 'views_data_export_garbage_collect(0, -1);') and it had no problems cleaning them all out. Have so fare not been able to track the cause of the problem down. The stray tables are coming back and are rather quickly building up again.
Drupal 7.23
Chaos tool suite 7.x-1.3
Views 7.x-3.7
Views Data Export (views_data_export) 7.x-3.0-beta6
SHOW TABLES LIKE 'views_data_export_index%';
+---------------------------------------------+
| Tables_in_elmcip (views_data_export_index%) |
+---------------------------------------------+
| views_data_export_index_39026 |
| views_data_export_index_39027 |
| views_data_export_index_39028 |
| views_data_export_index_39029 |
| views_data_export_index_39031 |
| views_data_export_index_39032 |
| views_data_export_index_39033 |
| views_data_export_index_39034 |
| views_data_export_index_39035 |
| views_data_export_index_39036 |
+---------------------------------------------+
10 rows in set (0.01 sec)
A few minutes after I cleaned them all out, and they just keep growing. Any pointers to what goes wrong? And, yes, I checked, cron is getting triggered every hour.
Comments
Comment #1
steinmb CreditAttribution: steinmb commentedPerhaps the site got stale batch jobs? I'm guessing a bit here then I do not know this module all that well. Removed the display I found by looking at views_data_export and flushed the system with views_data_export_garbage_collect().
Comment #2
MhueD CreditAttribution: MhueD commentedWe have an interesting situation which may throw some light on this bug. We have a content entry site that gets 'pushed' up to Production once a day, with no data entry allowed on the target site. Every night we push again and the whole table basis is replaced. In this way, the Production site is entirely secure, and it is 'thrown away' when the next copy is ready with the day's new nodes/changes.
We do not allow the public to enter any new records, as described above, but we DO have Views Data Export attached displays which allow the public to download data. Yesterday we found our MySQL database suddenly growing from 1.5 Gigabytes to 18 Gigabytes and it brought our site down because of settings on our AWS account limiting disc usage for tables.
When we looked at our DB we found what steinmb found above, that 2000 tables had suddenly been declared using the naming convention 'views_data_export_index_X'. ALL of these tables are created quickly, always within the 24-hour period between 'pushes'. Note: the garbage collection function mentioned above DOES work, once the situation is identified...but that is usually too late when the extra tables have brought the site down.
Earlier this year we had gone through a tuning exercise on the Views in question where we turned on Batch Export in the Views UI and adjusted the 'segment size'. If we did NOT use batching we found that there were actually sections of the data set that were not even making it to the output file. Would it be a good idea to possibly present a config that allows the option to have each download garbage collect after the individual job is complete? I am concerned that the regular Cron-hook is being overwhelmed by these enormous numbers of extra tables.