When creating a search index, you are able to select entities (such as nodes, users etc) and entity fields to add to your index. If these fields are entity reference fields, then you are able to select fields on the referenced entity to add to your index.
These indexed fields are then available on search pages and within views. They can be used to display data, filter and sort.
You are unable to add data from referencing entities.
It would be great if you could also add referencing entities to your search index.
Here is an example case: A search page is required where you need all of the search API goodness (such as attachments, faceting etc) and would like to be able to filter by "my groups". The groups module uses a "group content" entity to manage entity to group relationships (what node is in what group, what user is a member of what group). As there is no field on groups or nodes including its group relationship, the group content entity needs to be used to decide whether a node is within a group. However, the group content entity cannot be used to filter the node results, as there is no way to add "referencing entities" to the index.
This can be visualised below:
References that can be included in the index:
[Group] --references--> [Taxonomy term]
[Node] --references--> [User]
References that cannot be included in the index
[Group] <-- referenced by-- [Group content] -- references--> [Node]
This is a replica of the issue for Drupal 7 here: https://www.drupal.org/project/search_api/issues/1490972
It is possible to create relationships that go both ways with fields on both entities and use modules such as "Corresponding Entity References" to help manage this. However my opinion is that if you have duel relationships within your data, then your data model is not optimum.
| Comment | File | Size | Author |
|---|---|---|---|
| #34 | Screen Shot 2018-11-25 at 13.16.13.png | 70.17 KB | kmajzlik |
| #27 | 2986623-26--reverse_entity_references.patch | 18.4 KB | drunken monkey |
Comments
Comment #2
himynameisseb commentedNote:
I have been pointed to the following to help in my particular case, it may also help some else who stumbles across this issue:
https://www.drupal.org/docs/8/modules/search-api/developer-documentation...
https://www.drupal.org/docs/8/modules/search-api/developer-documentation...
(I have not trialled the solutions yet)
The feature request still stands. :)
Comment #3
drunken monkeyYes, the links in #2 would have been what I'd suggest, too, for implementing this on your custom site.
It's also how I'd solve this in general, I think: add a processor which adds properties for each “reverse reference”. (For performance reasons, it should probably be possible to disable that processor, though, I'd think.) It probably shouldn't be too hard to do that, but will definitely require quite a bit of work – and unfortunately I'm pretty swamped at the moment (or chronically, really).
But it's definitely a very interesting and promising feature request, so I'll keep it in mind and will see if I get a chance to work on it. Or, if someone else would like to take a stab at it, I'd of course be glad to offer support.
If you came up with some custom solution for your site, and it looks like something that could be expanded for a general use case, it would also be awesome if you could post that, so we at least have some starting point. (But probably you just hard-coded the tricky parts, I'd guess.)
Comment #4
drunken monkeyOh, and full agreement with this! Don't be sloppy with the data model!
Comment #5
himynameisseb commentedThanks for the feedback.
Our solution for release was to remove the need to filter search API views this way. It is however still on the longer term roadmap of the project, so when we get to it I will look at options and feedback here.
Thanks again for your input!
Comment #6
drunken monkeyAs luck would have it, I did have some time today, all new issues/comments were looked on and I decided to tackle this.
The attached patch is a WIP, with everything already in place, as far as I can tell, but me having run out of time before I was able to actually test this. (Or even begin adding automated tests, for that matter.) So, it will probably still explode horribly when being used, but it should already be a good base from which to work. Reviews are also already welcome.
I hope I’ll have some time later this week to finish this.
Comment #7
yohanaraujo07@gmail.com commentedHi, I applied the patch but the backreference fields doesn't work, it says on Search UI fields were indexed but, when I go on Solr I can't see the indexed values. I tried to add a Text (Plain) as test.
Comment #8
drunken monkeyOK, good to know.
As always, there came more work than anticipated, so I wasn’t able to work more on this yet. But I’ll get to it as soon as I have some time.
Comment #9
yohanaraujo07@gmail.com commentedI debug the code and first thing I did to solve the indexing issue was to add
on the excerpt
inside addFieldValues()
However my backreferenced entity is multivalued and I'm still trying to figure it out how to make to work that out as the problem is now with the field name of the multivalued field sent to Solr. Solr expects it to be prefixed with sm instead of ss.
Debugging further I can the issue is now on a later phase while building that Solr name and searching for properties on the field Entity (node), the Solr code gets lost because of the prefix search_api_reverse_entity_references_
Comment #10
drunken monkeyDoes it work for you if you switch to the Database backend?
Comment #11
drunken monkeyFound and fixed the bug and added some tests.
This looks like a first working version to me, and might already be RTBC. Please test/review!
Two problems are uncovered/highlighted by this new feature, which can’t be fixed in this issue:
Comment #12
shraker13 commented#11 worked for me. Thanks! Just saved me a major headache.
Comment #13
shraker13 commented#11 worked for me perfectly.
Comment #14
valicSmall adjustment to patch at #11, probably missed.
'node' is hardcoded so causing that on all back referenced non-node entites fields are loaded from node entity
- $property->setEntityTypeId('node');
+ $property->setEntityTypeId($entity_type_id);
Comment #15
kumkum29 commentedHi,
I want to filter my results by a data related to the referenced entity. For this, I tried to create a facet block from a backreference field.When I visit the views page, the facet block seems to be always empty...
I have pached the search_api module with the #14 patch.
Can we create facet block with datas from a backreference field?
Oops, I can create a facet block from a referenced data with a call in the structure of the field, e.g. : My entity > Content > Field
thanks.
Comment #16
valicDoes anyone have this working? Even with the latest patch, seem exists some issues.
If I add a field from the referenced entity, I always get something like this on indexing:
ERROR: [doc=rzjsf2-orders-entity:commerce_order/1000476:und] multiple values encountered for non multiValued field its_payment_payment_id: [308484, 308496]
But this field is a single value, and does not exist any other field with this id on this index?
This issue is always present in any field from back referenced entities.
Comment #17
valicIgnore my last comments :-) Patch is working, the problem which had encountered is more related to structure and inconsistent data.
Assuming that you have a field (back reference entity) which have cardinality 1 (getPropertyPathCardinality in Solr), but in DB exist two items.
So Solr think he needs to get one value, but he gets two values from addFieldValues.
Not sure where to address this issue, but on another side in my case is more issue related to some bad data (backreference should only once link to the original entity)
Comment #18
valicAdding new patch addressing the issue at #17.
We should probably check field cardinality, and then compare it with a count of loaded entity ids.
Think it could be a lot of cases where data could stay in DB for any number of reasons, and without this "hackish" solution
indexing of items will always break.
The idea in last addition to the patch is to get field cardinality. And only in the case where field cardinality is 1, and we have loaded more entities, just to take the last one from an array. It is "hackish", but don't see any other solution currently.
Comment #20
valicSorry, bad patch at #18.
Adding again, proper patch against #14
Comment #21
valicComment #23
valicAgain missed some things :-(
now with interdiff between #12 and last changes from comments #17, #18.
Comment #24
valicComment #26
drunken monkeyAh, right. The Solr backend is known to have problems with multi-valued fields in some cases, because they try to guess the field’s cardinality (and fail in a lot of uncommon cases).
The fix, however, should fortunately be simple – see the attached patch. (Interdiff compared to #14. The patch also includes a bit of refactoring, which I left out of the interdiff for the sake of readability.)
Comment #27
drunken monkeyComment #28
drunken monkeyAny feedback on this?
Comment #29
valicHi,
Sorry, we launched new web a few days back, so haven't had time :-)
Have few tasks lined in upcoming days, Solr related. So then should be some kind of feedback on the last patch
Comment #30
drunken monkey*bump*
Comment #32
drunken monkeyOK, growing tired of waiting for feedback on this, so: committed.
Let’s just hope it works fine.
Comment #33
FloGe commentedThank you for this patch!
Searching for groups whose content contains certain words is now working for our application. I have just another question to this.
Is it possible to limit the search to a certain group content type if more then one group content type is using the same field?
E.g. you have two group content types on for the pros and one for the cons concerning some issue which use the same field (maybe comment or so). If you now are searching for groups having x in the pros, you will also find the groups having x in the cons comments.
As far as I see I have no possibility to add a relationship or so to get the group content type into the filter criteria. Or do I miss something?
For the moment my workaround is: add a field to the group content type which saves the type (default value), add this to the search index and set the filter on this field.
Comment #34
kmajzlik commentedThis looks good except one small thing. Using t() placeholder with % is not working properly.
Comment #35
Deno commentedWorks fine for me - I'm using it to include group content in the group search. In simplified form, this is what I'm adding to my index:
Choosing the entries to include in search is a bit tedious, but this is a standard issue with search api. Other than that, it works as designed.
Comment #37
mulongokato.2000 commentedDoes anyone know how to add "group membership" relationships?
I have tested and proved that the "Content (entity_id:entity)" field under Group content is bridging the gap for node entities, and new fields can be discovered automatically, but I have not seen how the "membership" relation to user accounts/profiles is implemented, thou its mentioned in the issue summary. How can I access group members?
[Group] <-- referenced by-- [Group content] -- references--> [User]
I am not technical, but a field like "Membership" to bridge user accounts to the group_content (membership) type that references them will be great, or help on how to implement this use-case.
Thank you in advance
Comment #38
borisson_@mulongokato.2000 Please open a new issue and link to this one.
Comment #39
jcandan commentedI am not sure I follow how to go about creating an index with this. When building an index of content, how do I get the Group that the content is associated with to show up in the index?
Comment #40
ludo.rI have the same question as mulongokato.2000 and jcandan.
I just created a support request for that, see #3207808: Group: how to get group content for current group in Views?
Thanks!