Is anyone else bothered by thousands of rows in the sessions table?

Here is a patch that uses cron to expire session entries for anonymous users. This only happens once a day so as not to load the database unnecessarily, and the expiry period is configurable.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Dries’s picture

Ehm, this is a "won't fix". PHP is responsible for this. You can configure the session lifetime in setting.php. PHP's garbage collector will then periodically call Drupal's sess_gc() function with the proper argument.

kbahey’s picture

In my case, it is set to 1440 (Debian default in php.ini), but it never had any effect.

Are you sure sess_gc() gets called on Drupal.org?

Could it be because the default in php.ini for save_handler is set to files and not user, and overridden in .htaccess?

Anyone has this working at all the way it should?

Also note that my patch limits what it does to anonymous sessions, not all sessions like what is in sess_gc().

pwolanin’s picture

chheck your settings.php- that's probably superceeding the setting you have in php.ini.

At the least, I know that changing session cookie lifeting in settings.php does afffect how sessions are handled.

kbahey’s picture

This is standard drupal stuff that I did not change.

Here is how php is setup:

$ grep lifetime /etc/xxx/php.ini
session.cookie_lifetime = 0
; /etc/cron.d/php5, which uses the session.gc_maxlifetime setting below
session.gc_maxlifetime = 1440

Here is how Drupal is setup:

$ grep lifetime xxx/.htaccess

$ grep lifetime sites/xxx/settings.php
ini_set('session.cookie_lifetime',  2000000);
ini_set('session.gc_maxlifetime',   200000);

The cron stuff is standard Debian fare.

$ cat /etc/cron.d/php5
# Look for and purge old sessions every 30 minutes
09,39 *     * * *     root   [ -d /var/lib/php5 ] && find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 | xargs -r -0 rm


$ cat /usr/lib/php5/maxlifetime
#!/bin/sh -e

max=1440

for ini in /etc/php5/*/php.ini; do
        cur=$(sed -n -e 's/^[[:space:]]*session.gc_maxlifetime[[:space:]]*=[[:space:]]*\([0-9]\+\).*$/\1/p' $ini 2>/dev/null || true);
        [ -z "$cur" ] && cur=0
        [ "$cur" -gt "$max" ] && max=$cur
done

echo $(($max/60))

exit 0

Nothing there handles non-file sessions ...

Dries’s picture

Status: Needs review » Closed (won't fix)

Have you looked at your settings.php?

kbahey’s picture

Yes, and I listed what is in it:

$ grep lifetime sites/xxx/settings.php
ini_set('session.cookie_lifetime', 2000000);
ini_set('session.gc_maxlifetime',  200000);

These are the standard settings that ship with Drupal, and still, no cleanup happens.

My question agains is: does it happen on Drupal.org? Anyone else having sess_gc() called automatically?

kbahey’s picture

Version: x.y.z » 6.x-dev
Status: Closed (won't fix) » Needs review

In view of people complaining of us delegating session cleanup to PHP garbage collection, for example this issue, I like to revive this.

If it has a chance of getting in, let me know, I will reroll and test.

catch’s picture

Category: feature » bug
Status: Needs review » Needs work

If this isn't being handled properly, then it's a bug (or someone else should won't fix it again if it is). Either way it needs a re-roll.

kbahey’s picture

Well, to solve the problem at hand, I created a separate session expire module http://drupal.org/project/session_expire

It is more configurable than this patch can ever be.

David_Rothstein’s picture

Status: Needs work » Needs review

I can definitely confirm that sess_gc() never gets called. This is on a Debian Etch system with a standard PHP5 installation. The reason it doesn't work on Debian/Ubuntu is explained at http://drupal.org/node/160046 (which is marked as a duplicate of this issue).

As a start at fixing this, what about patching settings.php as suggested over two years ago (http://drupal.org/node/35834)?:

ini_set('session.gc_probability',   1);
ini_set('session.gc_divisor',   100);

A patch for 6.x is attached. I can confirm that this works for me (note that for testing purposes you may want to set session.gc_probability to 100 so that garbage collection is triggered on all page loads rather than on 1% of them). I believe this will fix the problem for a very large chunk of Debian/Ubuntu users out there.

However, I agree with the original poster that the best way to solve this problem in the long run is to call the garbage collection from within cron... that way it's guaranteed to happen for everyone regardless of their PHP configuration....

David_Rothstein’s picture

FileSize
829 bytes

This is like the third time this has happened to me on Drupal. I know I attached the patch but it didn't get posted. Anyway....

Samat Jain’s picture

Debian reports this "fixed;" Debian's php.ini now includes a better warning. See Debian bug 388808.

With Debian's fix, Drupal still needs to handle this.

quicksketch’s picture

I like David's fix in #11 (just ran into this on a client site also, 15 million session rows, ugh). That's the exact approach I used to fix for our client. I don't think that properly configuring PHP for Drupal is outside the realm of fixing at all. We already set several other variables to sensible defaults to avoid problems with a plethora of different platforms that Drupal runs on. Unfortunately I only just applied the changes today, so I'll need to let it go for a few weeks to see if it has the intended effect.

So, +1 for #11.

chx’s picture

Version: 6.x-dev » 7.x-dev

This is actually needed but you want a probability of 0 and curtail on cron. Performance++

jvandyk’s picture

FileSize
780 bytes

Well then, we probably want to get something like #9 into core. And we'll need to make sure that cron actually runs. In the meantime, here's a patch for D7. I agree with chx that it would be good to have more fine-grained control of this.

catch’s picture

Automatic/failsafe cron runs are here #331611: Add a poormanscron-like feature to core. I think this patch is a good idea.

c960657’s picture

I suggest adding a brief comment explaining how these settings affect Drupal. default.settings.php currently contains several ini_set() calls whose purpose is far from obvious (see #303154: Document ini_set() calls in default.settings.php).

Status: Needs review » Needs work

The last submitted patch failed testing.

jvandyk’s picture

Status: Needs work » Needs review

I agree that we should document the ini settings, but this issue is not about that. This issue is essentially about making Drupal work as expected on Debian/Ubuntu versions that have default PHP settings that prevent garbage collection.

Dries’s picture

If we're going to force the GC to run all the time, I'd like to better understand the performance impact. It seems like some quick performance tests are in order.

quicksketch’s picture

We're not forcing it to run all the time, in fact we're setting it to the usual default (check the gc_probability and gc_divisor settings on Drupal.org, I bet they're the same thing). It just so happens that Debian and Ubuntu turn off these settings in their out-of-box installations, versus nearly every other installation uses these settings that we're now specifically setting. So for most installations, we're not changing anything at all.

mvc’s picture

It's also possible to solve this on Debian/Ubuntu by explicitly calling sess_gc() in sess_close() in includes/session.inc.

Status: Needs review » Needs work

The last submitted patch failed testing.

quicksketch’s picture

Status: Needs work » Needs review
FileSize
1.22 KB

This is still a problem, but maybe now that docs have been added in #303154: Document ini_set() calls in default.settings.php these changes will be better understood. Here's a reroll.

Dries’s picture

You're right! Committed to CVS HEAD. Thanks.

Dries’s picture

Status: Needs review » Fixed

You're right! Committed to CVS HEAD. Thanks.

quicksketch’s picture

w00t! Thanks Dries... feeling core rejected recently. This helps. :-)

neilnz’s picture

Status: Fixed » Needs review
FileSize
486 bytes
467 bytes

I'd like to revisit the approach taken here and contribute two patches I have for D6 and D7 that approach this in a different way.

We run many Drupal sites on Debian servers that don't run a settings.php derived from settings.default.php, so the fix to this issue doesn't help us. Futhermore, I believe Debian turned off the gc_probability stuff for good reason - this kind of maintenance is properly handled from cron, not during a page request (even if it is quick).

My patches add to the cleanup jobs performed by system_cron() to call either sess_gc() or _drupal_session_garbage_collection() as appropriate.

I don't believe any harm can come from running these from cron in addition to standard session garbage collection, and it will provide a fix for users with existing configuration files who upgrade core on Debian.

Status: Needs review » Needs work

The last submitted patch failed testing.

neilnz’s picture

Status: Needs work » Needs review
FileSize
486 bytes

Resubmitting just the D7 patch. It seems to apply cleanly to HEAD for me, think the patch bot might've got confused by D6 and D7 at the same time.

Status: Needs review » Needs work

The last submitted patch failed testing.

Dane Powell’s picture

+1 for some sort of fix - I just discovered that my sessions table was 1,000,000 rows and over 1 GB in size. Setting session.gc_probability in settings.php worked for me, but I agree that this is better taken care of in cron.

izmeez’s picture

subscribing

neilnz’s picture

Could someone please try applying my patch from #28, at least for D6, and confirm that it solves the problem for them, and if so, mark this issue as RTBC.

We've been running the D6 patch on our local branch of Drupal for almost a year now with no issues.

izmeez’s picture

I am using D6 and cannot tell for sure from my phpinfo if the garbage collection is on but it appears to be on.

The settings.php session.gc_maxlifetime = 200000 which is approx 2.5 days and looking at the sessions table directly with phpMyAdmin it appears that the sessions fit within that timeframe.

I am still surprised to see almost 2,000 sessions in the table but maybe that is reasonable. I wonder if I should just change the settings.php rather than apply the D6 patch.

Thanks.

neilnz’s picture

If your session table doesn't have sessions much older than gc_maxlifetime, then you don't need to make any changes at all. In phpinfo(), look for gc_probability. If it's anything but 0, you should be fine.

You will often have a lot of sessions, since every anonymous visitor is given a session which lasts for maxlifetime.

A simple, one-liner SQL to detect whether your session table is misbehaving:

MySQL:

  SELECT unix_timestamp(now()) - MIN(timestamp) AS effective_gcmaxlifetime FROM sessions;
  

PostgreSQL:

  SELECT extract(epoch FROM now()) - MIN(timestamp) AS effective_gcmaxlifetime FROM sessions;
  

If this number is significantly higher than what phpinfo() returns for gc_maxlifetime, then your sessions table isn't being cleaned up.

izmeez’s picture

@neilnz Thanks for the explanation in #36. Yes, gc_probability = 1 and gc_divisor = 100 so I can leave that alone.

I still wonder why Drupal uses a default maxlifetime of 200000 (2.5) days which seems rather long? Sure, I can change it now that I have an understanding of this matter. Thanks to everyone for the details in this thread and specific answers to my questions.

Cyberwolf’s picture

Subscribing.

neilnz’s picture

Status: Needs work » Needs review
FileSize
662 bytes

Trying again to submit the patch for review, this time using CVS diff...

moshe weitzman’s picture

Lets not pile stuff into system_cron() for sites with non standard configuration. This smells like poormanscron to me (and not in a good way). -1 from me.

neilnz’s picture

This is actually about standard configurations, such as stock Debian and Ubuntu systems which all suffer from this.

Just look at the number of different issues and bug reports about this same problem around the web:

The current solution is to put a gc_probability ini_set() into the default settings.php that ships with Drupal and assume that people will be using that, but there's nothing to guarantee that they are, and therefore Drupal offers no guarantee on Debian or Ubuntu systems (and maybe others) that their sessions table will not grow perpetually until they run out of disk space. The fact that Debian decided to disable gc_probability is a testament to how ugly a solution it is.

The analogy to poormanscron is actually much more relevant to gc_probability itself, which essentially works exactly like poormanscron: by picking a random user's web request as the right time to perform a session garbage collection. It would be more reliable for Drupal to do this itself, in the right place for system maintenance tasks, which is hook_cron().

I chose the system to host this task, as its hook_schema() is where the sessions table is declared in the first place, so it seemed like the right module to also perform maintenance tasks on it.

This patch doesn't break any existing configurations, yet fixes a common problem that many people would otherwise be unaware of until it becomes a major issue.

quicksketch’s picture

The problem has already been "fixed" in the case that sessions will now be cleared out as they would be normally on most installations of PHP.

Drupal offers no guarantee on Debian or Ubuntu systems (and maybe others) that their sessions table will not grow perpetually until they run out of disk space.

The changes that have been made already provide assurance against this behavior (unless they modify the settings.php file). I think this would best be handled in a separate issue. The problem that was originally filed has been solved, the newly suggest approach may be an improvement but it is no longer fixing a bug.

kbahey’s picture

@neilnz

I have given up on having this in core, much as I would like it to be so.

The alternate solution is this module that I wrote: http://drupal.org/project/session_expire

neilnz’s picture

@quicksketch, the reason I reopened it is a change to default.settings.php doesn't help any of the existing Drupal userbase who used this file as a template for their settings.php before this was committed. I just checked and neither Drupal 6 nor Drupal 5 CVS have had this default.settings.php change backported yet, so even new sites launched on D6 today won't avoid this problem, nor the existing 99.9% of people suffering this problem right now on existing D5 and D6 installs, so I can't see how this problem is fixed in any real way for anything except new D7 sites.

My wish was for my alternative solution to get backported to D6 and D5 as a fix for existing sites hosted on Debian. My company alone is now looking after around 50 D5 and D6 sites, all on Debian, and all suffering the same problem, with my patch applied to our local base copy of Drupal. This was simply an attempt to get it upstreamed.

If we are to consider the default.settings.php change as the final fix, it does need to be backported to at least D6 ASAP to avoid any more new sites being affected by this bug until D7 is released.

catch’s picture

Version: 7.x-dev » 6.x-dev
Status: Needs review » Patch (to be ported)

I think the original patch here which was committed to D7 should be backported to D6, changing status/version for that.

On cron, I'd suggest opening a new Drupal 7 issue for that.

Albert Volkman’s picture

Status: Patch (to be ported) » Needs review
FileSize
868 bytes

D6 backport.

pwolanin’s picture

Status: Needs review » Reviewed & tested by the community

this is a trivial backport and should be RTBC, though it's too bad Drupal 6 doesn't have a way to re-write existing settings files. This will maybe need a note in the release notes?

gpk’s picture

Agreed, and also agreed this needs a mention in the release notes.

Yes it's a bit of a bore having to check .htaccess and default.settings.php for changes when doing an upgrade but that's always part of my process (I also check robots.txt if I can be bothered). I think that automatically rewriting settings files and/or .htaccess could be a bit dicey not least because it could interfere with any customisations made to those, but it would be useful if there was a listing somewhere showing in which versions of Drupal changes were made to these files - and whether or not they are substantive (sometimes it's just a docs cleanup). I guess one can always look at the git history e.g. http://drupalcode.org/project/drupal.git/history/HEAD:/sites/default/def... but that's all pretty cumbersome and I usually just end up doing a diff between the different versions of the respective default core files.

lpalgarvio’s picture

looking forward ;)

DamienMcKenna’s picture

FileSize
1.18 KB

Shouldn't the ini settings with the huge comment be listed *after* the other ini values?

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 50: drupal-n72856-50.patch, failed testing.

Status: Needs work » Closed (outdated)

Automatically closed because Drupal 6 is no longer supported. If the issue verifiably applies to later versions, please reopen with details and update the version.