First, thanks for creating this module! I am using it to power our search service on Travel Massive.
I am having trouble setting up a conditional filter on fulltext index where I want to exclude any null/empty fields. The condition works for String indexes, but not for Fulltext. I'd like to use a fulltext so I can can improve the matching. However as the fulltext index prevents me using the "is not null" condition, I can't implement the desired search behavior I need.
ie:
$query_filter_has->condition('user:field_link_twitter:url', null, '<>');
Behaviour:
If the field "user:field_link_twitter:url" is indexed as "String", the condition is applied successfully and filters out all null results.
If the field "user:field_link_twitter:url" is indexed as "Fulltext", the condition is not applied and results with null values are returned by the search query.
It looks like in service.inc line 1682, this logic gets caught by this code when a '<> null' condition is parsed:
$keys = $this->prepareKeys($f[1]);
if (!isset($keys)) {
continue;
}
In this scenario, $f[1] is null, so no filter query is applied. There's no other code that handles things, so the query condition is not correctly generated.
Hope this information helps and that there is a solution to support what I'm trying to do without having to move to a different search service. I like search_api_db :)
Thanks,
Ian
Comment | File | Size | Author |
---|---|---|---|
#17 | 2745655-17--fulltext_field_null_conditions.patch | 1.42 KB | drunken monkey |
| |||
#2 | 2745655-2--fulltext_field_null_condition--search_api.patch | 1.14 KB | drunken monkey |
Comments
Comment #2
drunken monkeyThanks for reporting this problem!
While the documentation isn't completely clear about whether checking fulltext fields for
NULL
values is actually supported (and it kinda sounds like it isn't), Solr already seems to support it and I can't really see any reason why we shouldn't.The attached patch should implement this, please test/review!
Also, we should make the documentation clearer in this regard.
Finally, the D8 version probably has exactly the same problem (at least definitely the documentation).
Comment #5
drunken monkeyOh, forgot the "minimum chars" setting has been changed at that point in the test code.
But testing before that would be even worse (since we'd then not even have an item without title), so just changing the assertion. Should be fine now. (Locally, I get over 30 fails with the clean checkout, so can't really tell.)
Comment #7
travelvc CreditAttribution: travelvc commentedThanks for taking time to create the patch, however unfortunately it doesn't seem to work for me.
My steps were:
1. Applied patch in comment #5
2. Changed index type from string to fulltext
3. Reindexed search db
The results are the same as my original comment.
Comment #8
drunken monkeyAre you sure you applied the correct patch – i.e., the last one, with the green tests?
Comment #10
drunken monkeySince the test demonstrates that this indeed fixes an existing problem, I've now committed this regardless.
If you still have problems with the latest dev version (after this commit), please create a new issue.
Comment #11
travelvc CreditAttribution: travelvc as a volunteer commentedSorry for the delay to respond. The patch required a small modification to make it work.
In your patch you added the following three lines to createFilterCondition():
I had to modify the enclosing if statement as follows:
This is what works for me in the end:
Thanks again for steering me in the right direction, and hope this feedback is helpful.
Ian
Comment #12
drunken monkeyHm, that's curious. From what I see in the code,
$field['column'] === 'value'
should always beTRUE
for text types. Can you find out what$field['column']
is for you in this case? (And, if possible, whether that value is also stored in the server options, or added somewhere else in the code.)In any case, though, it seems like it would be a good idea to set
'column'
to'word'
right ingetFieldInfo()
for all fulltext fields. Currently, it's a bit confusing (and prone to cause errors) that the value is just wrong for text fields.(The "Release blocker" tag is for the D8 port.)
Comment #13
travelvc CreditAttribution: travelvc as a volunteer commentedI put a
print_r($field)
insideif (search_api_is_text_type($field['type'])) {
and here's what came out:My query filters are set up as so:
In the configuration under "Select fields to index" I have these fields set as "Fulltext".
The field is a standard Drupal "Link" type - this is what the underlying table looks like:
Hopefully this additional information helps!
Comment #14
drunken monkeyI think I see what's going wrong there. I've opened #2795245: Fix changing of a field's type to "Fulltext" – could this be the problem? Did you change the types of those two fields from "String" (or anything else) to "Fulltext" at some point?
If so, please check out the patch in that issue and see whether it resolves your problem. To test, please just change the types of those fields again to "String" and then back to "Fulltext" – the change should now be handled correctly by the DB backend class and subsequent "is (not) NULL" filters work correctly.
Comment #15
travelvc CreditAttribution: travelvc as a volunteer commentedSuccess! I applied the patch (2795245-5--switch_to_fulltext_type.patch) and followed your instructions (changed the fulltext to string, then back to fulltext).
Note that I also required the patch in this issue to be applied (2745655-5--fulltext_field_null_condition.patch).
My next question is - how best to apply these fixes in production (given I've got a working implementation via the patch in this issue and my own modifications). Should I hold out until a combined patch or new release is made?
Thanks for your attention to this issue - and let me know if I can test anything else for you.
Comment #16
drunken monkeyOK, great to hear, thanks for testing!
Then I'm moving this back to D8 and we can work on fixing this problem in the other issue. (Maybe also leave a comment there that the patch worked for you, so I remember. Or wait until I post a new patch that fixes the test fail and help me test that one.)
Comment #17
drunken monkeyIt's already working fine in D8 due to other changes, so we just need to add the test, too.
Comment #18
borisson_Just a test? Sure!
Comment #20
drunken monkeyGood, thanks for reviewing!
Committed.
Comment #22
AaronBaumanDoes
regressionTest2745655
cover views behavior as well?I'm working on a custom search api argument, and seems like conditions involving fulltext NULL / NOT NULL are not working.
Not sure whether that merits a new issue, or if there's one open already
Comment #23
Marios Anagnostopoulos CreditAttribution: Marios Anagnostopoulos as a volunteer commentedI stumbled upon this as well. In my case I am trying to add a non empty filter for the default body field of nodes.
My field gets indexed as an empty string (like it should). No custom code applies to this process so far.
I am using elasticsearch and my indexed item looks like
Are the two issues relevant?? IDK.
Also @AaronBauman, if you find a related issue please link it here, thanks.