Early Bird Registration for DrupalCon Portland 2024 is open! Register by 23:59 PST on 31 March 2024, to get $100 off your ticket.
We ran into an issue today on our university Drupal instance where one site enabled Devel query logging to debug some slow queries in the morning. By the afternoon, /tmp was full (2GB) and other sites could no longer upload images because of /tmp/devel_querylog. I know Devel query logging is not supposed to be used for long, but it didn't seem to clean up after itself and it was necessary to delete files in /tmp/devel_querylog to regain space.
Is Devel supposed to clean up after itself? Thanks in advance.
Comment | File | Size | Author |
---|---|---|---|
#11 | devel_query_log_fills-2091535-11.patch | 1.89 KB | salvis |
|
Comments
Comment #1
ottawadeveloper CreditAttribution: ottawadeveloper commentedWe just encountered the same problem - I think it's simply that the log files are growing too fast (it's only programmed to delete 1 time in 1000 at random... which means there's only a 62.2% chance of it actually wiping the file within a given 1000 requests). I have some really big query log files (>50MB) so it's destroying the disk space really fast. It's essential to store these briefly because of the AJAX requests to them so we can't just delete them as soon as they're requested.
My proposal to fix this would be to add some settings to configure the wiping of the query logs. Personally, I'd like to do this by filesize and I think it makes the most sense for most people (that we limit the filesize available to the logging profile).
So something like building the size of the query log directory, then seeing if it's over 1GB (or pick your value perhaps?), then, if it is, order the files by creation time and delete files older than an hour. Letting them configuring the 1GB and one hour value might be useful (or I guess they can strongarm it as long as you use variables). Alternatively, we could delete files until we're below a certain threshold which would guarantee a certain amount of disk space free.
To lower run time, we can do that at random, but much more frequently (and ideally let them control the frequency as well).
Anyone else have any thoughts? I have to fix this in my project anyways (probably on init() or something in a custom module), but I'm perfectly happy to do a fix to patch into devel.
Comment #2
ottawadeveloper CreditAttribution: ottawadeveloper commentedIf anyone else is looking for a way to handle this in the mean time, I put the following into a module:
Comment #3
moshe weitzman CreditAttribution: moshe weitzman commentedChecking filesize on every request is too expensive. We could simply reduce the current check to once every 200 requests, or so.
Comment #4
okeedoak CreditAttribution: okeedoak commentedSame problem here.
Comment #5
dehacker CreditAttribution: dehacker commentedDevel querylog filled 1GB of server tmp partition on Ubuntu 12.04 and crippled the system. Was unable to run 'drush cc all' afterwards. How do we flush or limit the log?
Comment #6
annya CreditAttribution: annya commentedThis patch add possibility to change this value via administrative interface.
Comment #7
Cogax CreditAttribution: Cogax commentedI had the same problem. My tmp/devel_querylog/ dir was 18GB(!) big before i deleted all. I applied patch #6 and decreased the value in the backend to 100. Then the dir size goes up to 1.9GB and now its on 1.2 GB, so i think it works :)
Comment #8
helmo CreditAttribution: helmo at Initfour websolutions commentedAssuming the random function works ok, this could als compare to 1 instead of $range/2 ... saving a little bit of calculation time.
Comment #9
salvisCould this be an issue that should be fixed in D8?
Comment #10
willzyx CreditAttribution: willzyx commented8.x use webprofiler for the query logging nowadays. Back to 7.x
Comment #11
salvisRe-rolled, integrated #8 (no need to make this overly complicated), adjusted the #description.
Comment #13
salvis