Adding an index on the cas_user table, uid field will prevent slow queries for sites with large numbers of users.

Comments

mpruitt’s picture

StatusFileSize
new583 bytes

Patch for 7.x-1.x.

yalet’s picture

I wonder if we can get some data on the speed improvement offered by this patch. Also, which queries were causing the problem? It could be there are code architecture improvements that could be made to solve the problem, in addition to (or instead of) just slapping an index on it.

mpruitt’s picture

yalet, I'll forward the request to my engineer.

Regards,

Mark

mpruitt’s picture

Tim, here is the slow query report: (apologies for ugly line breaks). I have asked for information regarding performance improvement.

If you have additional questions I'll be happy to get the answers.

# Attribute pct total min max avg 95% stddev median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count 3 8863
# Exec time 53 290s 13ms 363ms 33ms 87ms 21ms 23ms
# Lock time 1 382ms 19us 50ms 43us 44us 511us 33us
# Rows sent 0 7.81k 0 1 0.90 0.99 0.29 0.99
# Rows examine 91 671.60M 56.46k 213.48k 77.59k 211.82k 51.20k 56.74k
# Query size 0 442.81k 48 53 51.16 51.63 2.01 51.63
# Query_time distribution
# 1us
# 10us
# 100us
# 1ms
# 10ms ################################################################
# 100ms #
# 1s
# 10s+
# Tables
# SHOW TABLE STATUS FROM `****` LIKE 'cas_user'\G
# SHOW CREATE TABLE `****`.`cas_user`\G
# EXPLAIN /*!50100 PARTITIONS*/
SELECT aid, cas_name FROM cas_user WHERE uid = ######\G
# *************************** 1. row ***************************
# id: 1
# select_type: SIMPLE
# table: cas_user
# partitions: NULL
# type: ALL
# possible_keys: NULL
# key: NULL
# key_len: NULL
# ref: NULL
# rows: 59767
# Extra: Using where

mpruitt’s picture

StatusFileSize
new617 bytes

Here is the patch back ported for 6.x-3.x.

Status: Needs review » Needs work

The last submitted patch, 5: cas-uid_index-2289871-D6-5.patch, failed testing.

mpruitt’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 5: cas-uid_index-2289871-D6-5.patch, failed testing.

emsearcy’s picture

Currently with our hardware and table size, it is taking an average of 33 ms for the query (it's hitting our slow query due to log-queries-not-using-indexes). This may not sound like a lot but that's an extra .03 seconds on any page load that loads a user object (and of course for pages that load many users, like a blog listing, that's several 30ms hits which adds up fast). As our user base grows, this will get slower, and I'm sure there are organizations like universities that may want to implement logins with far larger user databases than I.

Mark's output above was from pt-query-report, here is the top part with better columns from a different day. You can ignore the pct column, as that is percentage of across all queries in our report that day. Exec time row's min, max, avg, and 95th percentile are the key items there.

# Attribute    pct   total     min     max     avg     95%  stddev  median
# ============ === ======= ======= ======= ======= ======= ======= =======
# Count          3    9317
# Exec time     57    311s    13ms   125ms    33ms    87ms    22ms    23ms
# Lock time      1   360ms    20us    12ms    38us    44us   121us    33us
# Rows sent      0   8.11k       0       1    0.89    0.99    0.31    0.99
# Rows examine  86 721.95M  56.46k 213.48k  79.35k 211.82k  53.11k  56.74k
# Query size     0 464.55k      48      53   51.06   51.63    2.07   51.63
emsearcy’s picture

Oh, and the performance improvement is from the old 13-125ms to a new 0.00 sec.

mpruitt’s picture

StatusFileSize
new617 bytes

Renamed patch following the guidelines for "Patches for Different Versions" at https://www.drupal.org/node/332678#versions.

metzlerd’s picture

I think that this is a reasonable request, however, I noticed that in the patch, you call the index cas_user in one place and UID in another. Also, to improve upgradability it would be best to add a test to see if the index exists prior to adding it in the update hook. Would you mind resubmitting this patch with those mods?

Thanks

Dave

mpruitt’s picture

Dave, no problem, I'll be glad to look into that this week.

Mark

mpruitt’s picture

Changes requested by Dave. I apologize for the duplicate file.

mpruitt’s picture

mpruitt’s picture

yalet’s picture

Status: Needs work » Needs review
metzlerd’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me... I will work on committing these today.

metzlerd’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed to 7.x-1.x and 6.x-3.x branches. I made a small alteration to the latest 6.x patch so that it was using the same index name as the 7.x patch.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.