An optimization which the Apachesolr module currently employs and we don't is to use the q.alt parameter instead of fq when there is a single filter and no regular search keywords (q parameter).
We should implement this, too, as it can ostensibly lead to a significant performance improvement in such cases.

The only question is how this would work with the index_id:FOO filter we use in all searches. This remains to be seen.

CommentFileSizeAuthor
#1 1773440-1--qalt_optimization.patch2.24 KBdrunken monkey

Comments

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Issue summary: View changes
Status: Active » Needs review
StatusFileSize
new 1773440-1--qalt_optimization.patch2.24 KB

Just realized that we still aren't doing this. Patch attached which should fix that. (It also fixes some slightly related problem when passing empty array parameters, and unsets the qf parameter when no keys are present. (I don't think either makes any difference to Solr or performance, but the URLs look a bit cleaner then which makes debugging easier.))

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Status: Needs review » Fixed

Committed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Anonymous’s picture

Anonymous (not verified) commented
Priority: Normal » Critical
Status: Closed (fixed) » Needs work

Hi,

After upgrading to the latest search_api_solr version I noticed a performance downgrade because of this patch. A performance gain can be acquired by using fq instead of q. Not the other way around. The idea behind q.alt is to set a default query when q is empty.
Filter queries take advantage of the filter cache, whereas doing a normal query it will not. I would strongly advise to remove this "performance improvement" in order to get back the real performance improvement, which is, use filter queries to filter, even when the query is empty.

From https://cwiki.apache.org/confluence/display/solr/Common+Query+Parameters

The fq parameter defines a query that can be used to restrict the superset of documents that can be returned, without influencing score. It can be very useful for speeding up complex queries, since the queries specified with fq are cached independently of the main query. When a later query uses the same filter, there's a cache hit, and filter results are returned quickly from the cache.

Better would be to make q.alt configurable from a view, which should contain a query, not filters.

Best,

Jeroen

nick_vh’s picture

nick_vh
he/him
Location Ghent
commented

q.alt should be used when the q parameter is empty. This way you avoid going through the query parser and you can directly start filtering your content.

Can you prove with some more numbers, in Solr access log format, that the q.alt in your case is actually slower compared to the empty q= and the fq parameters?

This thread mentions that it actually increases performance when used correctly:
https://www.drupal.org/node/1429218

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Priority: Critical » Normal

Yes, could you please provide some benchmarks on this? The Apachesolr module has had this for ages and no-one has ever complained there (as far as I know – or at least not convinced the maintainers). Best make benchmarks with different loads and different index sizes.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Status: Needs work » Fixed

See #2532812: Revert the "q.alt" performance "improvement" for this discussion.
Performance tests would still be great, but I'm pretty much convinced now anyways.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

jordanmagnuson’s picture

jordanmagnuson commented

Has default q.alt handling changed at all in recent versions of solr? I just upgraded from Solr 3.x to 6.x, and was getting strange behavior on my Browse pages (with custom filters) defined by the Apache Solr module.

Specifically, these browse pages started to return all results, rather than only results defined by my custom filters, and it appears that the reason for this is that somehow in switching from Solr 3.x to Solr 6.x, the q.alt parameters got switched from AND processing to OR processing. See screenshot: http://i.imgur.com/zR2OS06.png

I altered apachesolr.module and changed the equivalent of line 916 in the above patch from:

$query->addParam('q.alt', implode(' ', $qalt));

To: 

$query->addParam('q.alt', implode(' AND ', $qalt));

And that solved the issue for me. Any thoughts? Should this be changed in the core module code, or is there another way I should be handling this?

martingz’s picture

martingz commented

Any news on this?

jordanmagnuson’s picture

jordanmagnuson commented

This seems to be fixed in latest builds.