Problem/Motivation

DB server stopped working, disk full due to large views export cache table 47G .ibd file mariadb
compleet DB export is 300MB

When doing a large view export to csv

Steps to reproduce

Do a lot of exports

Proposed resolution

Limit on some way, not logical the cache is larger than the complete DB itself

Remaining tasks

User interface changes

API changes

Data model changes

CommentFileSizeAuthor
#4 views_data_export-cleanup-cache-3186214-4.patch1.79 KBcsmdgl

Comments

ikeigenwijs created an issue. See original summary.

damienmckenna’s picture

damienmckenna
Location TN, USA
commented

If you look at views_data_export_cron() it runs views_data_export_garbage_collect() which goes through all of the exports and purges data from tables. I haven't looked into it in detail yet, but I wonder if the garbage collection logic is faulty?

littledynamo’s picture

littledynamo commented

Same issue here on a Drupal 9 site. views_data_export_object_cache table size gradually increased to 700MB, which is 70% of the database size and caused the client to hit the 1000MB ceiling on Siteground hosting.

csmdgl’s picture

csmdgl commented
StatusFileSize
new views_data_export-cleanup-cache-3186214-4.patch1.79 KB

In D7, there is a bug in the logic of views_data_export_garbage_collect(). It is calling views_data_export_view_clear() with an array of eids, but views_data_export_view_clear() only works for an individual eid.

Patch attached.

csmdgl’s picture

csmdgl commented
Status: Active » Needs review
steven jones’s picture

steven jones commented
Priority: Normal » Critical

Seems like this is pretty critical tbh, if that's what's going on/is wrong.

steven jones’s picture

steven jones commented
Status: Needs review » Closed (won't fix)

@csmdgl I'm not sure it's a bug, because in Drupal 7 it's actually the case that you can send a scalar or an array to a database condition and Drupal would pick out the right operator:

https://api.drupal.org/api/drupal/includes%21database%21query.inc/functi...

I'm going to close this issue down because Drupal 7 is EOL and I suspect that others have found ways around this in the meantime anyway.

Thanks for the effort everyone though.