ints are faster than floats for processors to work with, so addition and sorting should be faster.
We can multiply by a constant to get meaningful values in the index. Should we divide again when getting the results, or present the int scores as-is?
| Comment | File | Size | Author |
|---|---|---|---|
| #2 | 2207205.diff | 5.11 KB | drumm |
Comments
Comment #1
drummFor Drupal.org's index, the min and max scores are 0.01528500020504 and 13912. Our boosts are relatively low, 0.5, 2.0 and 3.0. Rounding down to powers of 10:
I suggest using 1,000 as a multiplier in an unsigned int, leaving plenty of room for higher boosts and addition as results are calculated. This will throw away some precision, but the scores in the DB engine aren't too precise anyway.
To get an idea of the effects, there is a score column on searches linked from #2136119: Text search on issue queues is slow and sometimes WSODs. I would be good with presenting the new ints instead of dividing again on output.
Comment #2
drummAttached is a patch for this.
The update is numbered sequentially after the update for #2170689: Use a (word, field_name) covering index on text tables. They can run in either order.
Comment #3
drunken monkey2: 2207205.diff queued for re-testing.
Comment #4
drunken monkeyThanks a lot for the suggestion and the patch, looks quite good. Since scores (or, more accurately, their absolute values) are completely arbitrary, it would be no problem to just switch to integers and multiply them all by 1000. Dividing again when returning the results might look a little nicer (in case they are shown at all), but that's really optional.
However, do you have some benchmark data to prove/back up your claim that using integers is (significantly) faster than using floats? I would have believed that this advantage is gone since the introduction of floating point units into processors, and might even be slightly reversed in some cases.
Comment #5
drummbasic did some informal benchmarking, but we didn't save any numbers. It was roughly a 2x improvement.
When profiling queries for #2136119: Text search on issue queues is slow and sometimes WSODs, I saw that most of the time, by far is spent in either sorting or copying to tmp tables. This will help the sorts and sum(score). As far as I can tell, the benefits will be from processors being able to do multiple int operations per cycle in the core processor vs less per cycle in a separate unit. It will vary greatly depending on processor architecture, but I think good or a draw for all of them.
Comment #6
drunken monkeyOh, wow, that sounds great then, of course.
As said, the patch itself looks good, too, so: committed.
Thanks again!
Comment #8
tvn commentedComment #9
drumm