Duplicate entry errors caused by race condition

nonsie, were you ever able to get a fix for this issue. We're seeing the same problem over here.

Our current hypothesis is that this is an issue with running multi-site, or possibly just multiple versions of Drupal on the same machine.

Hello,

On the Forgot Password page, after entering an email and clicking Request new password, I am getting the below error every time:

user warning: Duplicate entry '8e6a107842dc2aab476c1b74f4efa362' for key 'session_id' query: UPDATE session_api SET session_id = '8e6a107842dc2aab476c1b74f4efa362' WHERE sid = 118 in /srv/www/example.com/public_html/includes/common.inc on line 3477.

I would say this issue is critical? Can anyone please assist!

Thank you

time() is not unique enough. perhaps microtime() is better.

ok, here is a patch to test it.

found a better way.

sorry wrong patch ;-)

this is faster than @foo();.

@tobiasb

i looked into your patches, my question is now:

are these pathces to look whether the changed lines are the point of trouble or are they to fix the problem?
the patch in #6 seems to some kind of diagnotic, while #5 looks like an atempt to solve the problem.

I didn't try any of these because i do not know which to take.

I'm having the duplicated entrys and want to get rid of them.

#1: I'm not running multiple Drupal installs or an multisite on the machine where i'm having this error messages

#6 is the cleaner way as #5. and it is a fix to avoid the warning message.

Patch in #6 worked for me to get rid of the errors (Thank you tobiasb!). I manually applied it to 6.x-1.4.

I get the same errors:

Duplicate entry 'c5b78520117fe1ea3cf33fad0e6eac5d' for key 'session_id' query: INSERT INTO session_api (session_id) VALUES ('c5b78520117fe1ea3cf33fad0e6eac5d') in /var/www/clients/client94/site244/docroot/includes/common.inc in Zeile 3538.

I'm not quite sure why and when this is happening. Visitors reported them to appear occasionally and I myself could only reproduce it a few times (while randomly surfin around). Since the provided patches simply turn off the error messages for that case I didn't use the patch – I just put the error_reporting exclusively to the log. Yet this floods the log quite a bit.

I might have a deeper look into this next week.

The duplicate key error comes from the database table session_api which defines the column session_id as unique key which, however, is not generated in unique way in session_api.module line 49:

$session_id = md5(ip_address() . time() . drupal_get_private_key());

None of those components is unique enough, so it's very probable that the session_id will be the equal for two or more visitors. There should actually be a random component which would make the session_id distinct from each other.

EDIT: Let's consider we run this module in a higher traffic site for a while, the log is flooded with error messages, we roll-out a patched version fixing the session-id issue, but why there's still errors coming to the log? That's because there's still plenty of those not-so-unique session-id:s around stored in browsers, and if the cookie life-time wasn't set lower by configuration they will live for 30 days, unless the visitor returns to site more often and the cookie is renewed and the same session-id. It will live until visitor's browser cookies are cleared... To get totally rid of error messages we would need to roll-out a patched version that regenerates the session-id if inserting the cookie-originated session-id fails.

Given that I don't really want visitors sharing sessions, I've patched my copy of the module to use microtime() concatenated with rand() instead of time() for the session ID generating code. Will see if this stops the random stream of "duplicate entry" errors.

In fact, given that I now am using rand(), I probably don't need the drupal_get_private_key() or ip_address() calls...

This didn't in fact reduce the number of errors, so I've reverted to the standard code, that just uses time().

We are also experiencing this error. Do the patches above only disable the error message, or do they actually resolve the problem?

Thanks
Mike

subscribing

I'm guessing that okaalto's Edit in #11 is part of the problem, and why the microtime() patch in #4 doesn't immediately work.

Otherwise, I'd guess that a patch like #4 and/or a randomizing aspect suggested in #12 would do the trick.

That's what I thought, but even with microtime() and rand() to generate the unique string to hash I was still getting duplicates. The only way that could be possible is, I think, if we ended up with duplicate md5 hashes, but that's extremely unlikely.

How can we get to the point of trying to re-insert a session ID that we've already generated? Is there a race condition, where two separate browser requests run almost at the same time?

Perhaps the order of setting cookies, checking cookies, and INSERTing a new session_id needs to be re-arranged?

Here is a question.... What update has caused this? For our site, this error has only just begun filling our logs. I can't remember when/what it was that I updated a module that caused these errors to start.

We are having issues with Views versions 2.14 and 2.15 not working (version 2.16 resolved it) and also content on our site is being blocked from anonymous views (such as our Decisions voting module), all of these issues only coming about over the last few weeks.

Crazy
Mike

Anyone actually resolved this issue yet? I started getting these errors today using 6.x-1.4

@drupal_write_record('session_api', $rec);

does not fix this, but still error log it not filled with messages

Done a search on this issue and found a similar situation with Joomla:

http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&...

Summary
Duplicate entry for session Details
Duplicate entry 'bb9c0983d6afb11b9f125e8ddb2b93d9' for key 1 SQL=INSERT INTO
`joo_session` (`session_id`, `client_id`, `time`) VALUES
('bb9c0983d6afb11b9f125e8ddb2b93d9077068bf', 0, 1303308363)

The following post seems to confirm a solution for the issue there:

Adddate: 2011-09-12 22:14:58
I can confirm, that the duplicate entry issue is solved by extending the column
length for session_id to 64 characters in #__sessions (had the same issue with
1.7.0 recently).

Could someone please try a similar fix and update the community here. Note that in this contrib, session_id is already varchar(64)

Module version: 7.x-1.0-beta2
I believe md5() will never return more than 32characters. So, I dont think increasing session_id column length will fix this issue.
I have applied the patch http://drupal.org/node/1058960#17 for D7 to fix expiration logic in CRON. I have included my patch to fix duplicate entries in this patch itself. Please check it & let me know if any errors are raising.
Duplicate entry patch: Line 83 to 88.

Any patch for module version 6.x-1.x-dev?

Patch in #22 really needs to be rolled as an actual patch not just posting the full contents of a file...

I've rolled a patch to the d7 version that uses the same mechanism as core to generate the session_id which should get rid of any newly created duplicate session_ids. As noted you might still have an issue with session_ids that already exist but with this patch it should be OK going forward. I altered the {session_api}.session_id field to be 128 characters to match the change in core {sessions}.sid from 64 chars to 128 chars as well.

i'm running into this issue on a site. I haven't reviewed the patch but I'll also love to wrap the session api database insert with a try/catch and just logged the error instead of breaking the entire page.

The reason why I bring it up, is because I would rather have a watchdog error regarding duplicate keys then the user received a completely broken page because of a duplicate key on this module.

Thoughts? I'll be reviewing this patch later today.

Patch in #25 looks good to me. I've test and deployed with patch to a production environment and is working flawlessly.

Patch from #25 ported to D6... I just copied the internals of D7's fancy drupal_hash_base64(). From testing, I can confirm it gives unique id's in cases where the previous code returned duplicates. I have had no errors since this patch. I'm hoping it also fixes the access denied errors. ::fingers crossed::

Thanks all, I've committed #25 to 7. I'd like the patch in #29 to have some more testing prior to backporting.

Applied the D6 patch in #29: error stopped, no side effects so far. Thanks mattcasey.

Have applied the D6 patch in #29 (remembering to run update.php or drush updatedb!) and so far all seems OK. Session IDs are now longer and otherwise all is working as before. Will report back tomorrow to say whether the "duplicate entry" problem is fixed for me.

Thanks mattcasey!

Just an update that the patch is still working for me... I still get access denied errors but that's a different issue with caching.

After a day's use on a busy website, I can confirm that the patch in #29 fixes the "duplicate entry" errors in Drupal 6 with no obvious problems. The number of "duplicate entry" errors cause by the session_api module has dropped from several dozen a day to none at all.

Perhaps spoke too soon. I now have one "duplicate entry" error:

Duplicate entry '-rGUJ5jqau26rRJJF_cQFlvVmkLSd2NqyML93K8DjZ8' for key 'session_id'
query: INSERT INTO session_api (session_id) VALUES ('-rGUJ5jqau26rRJJF_cQFlvVmkLSd2NqyML93K8DjZ8') in /var/www/shared/drupal6/includes/common.inc on line 3579.

So I think there's still a race condition somewhere.

But it's a LOT better than it was before.

Thanks everybody, I've committed #29 (see 9cc0805) to 6.x.

Further reports here of "Duplicate entry" errors, a few one-off ones, for a variety of different page request URLs, some Drupal pages and some JavaScript files.

Then yesterday (Sunday) three with exactly the same request time (to the nearest second, at least) and IP address (of course). They were from Ubercart's taxes/calculate request from the cart/checkout page.

Just to re-iterate that the patch in #29 doesn't actually fix the "Duplicate entry" errors, it merely makes them appear less often. I suspect that this means that some site visitors that were previously triggering the "Duplicate entry" errors are now being assigned more than one session ID: whether that's a problem or not I don't know.

I think the duplicate session ids are temporary: when multiple requests hit in a short amount of time, multiple session ids are generated for the same user. However, the last one "wins", that cookie will be used for subsequent requests.

For me, this happens for a site that uses multiple ajax requests on page load. The main page is cached and sets no cookies. On the initial page load, no cookie is present at all and all ajax requests are fired. They each generate the same session ID and try to insert it. This succeeds only for the first one. On subsequent requests, the cookie is present in the browser and is used to access the database.

The problem I see is that after the session id is generated, it is not check if it is already in the database. To fix that, I have moved the $sid = db_query("SELECT sid FROM {session_api} WHERE session_id = :session_id", array(':session_id' => $session_id))->fetchField(); part to line 63, ie., just before the $cookie_domain = ini_get('session.cookie_domain'); part. Now when two requests generate the same sid, only one insert is done and the other request uses the existing session.

I also have a case where I have multiple ajax requests being the first anonymous user contact with the base drupal system and I have been getting the duplicate insert errors as a result, here is a simple patch on the d7 version using the db_merge syntax to avoid this problem.

Re-rolled patch with a few changes.

huh..

#44: fix_concurrency_inserts-483176-43.patch queued for re-testing.

Oops, I inadvertently removed the actual setcookie() statement. Funny that tests passed. I guess the module could use some extra test coverage...