When creating a search index, you are able to select entities (such as nodes, users etc) and entity fields to add to your index. If these fields are entity reference fields, then you are able to select fields on the referenced entity to add to your index.

These indexed fields are then available on search pages and within views. They can be used to display data, filter and sort.

You are unable to add data from referencing entities.

It would be great if you could also add referencing entities to your search index.

Here is an example case: A search page is required where you need all of the search API goodness (such as attachments, faceting etc) and would like to be able to filter by "my groups". The groups module uses a "group content" entity to manage entity to group relationships (what node is in what group, what user is a member of what group). As there is no field on groups or nodes including its group relationship, the group content entity needs to be used to decide whether a node is within a group. However, the group content entity cannot be used to filter the node results, as there is no way to add "referencing entities" to the index.

This can be visualised below:

References that can be included in the index:
[Group] --references--> [Taxonomy term]
[Node] --references--> [User]

References that cannot be included in the index
[Group] <-- referenced by-- [Group content] -- references--> [Node]

This is a replica of the issue for Drupal 7 here: https://www.drupal.org/project/search_api/issues/1490972

It is possible to create relationships that go both ways with fields on both entities and use modules such as "Corresponding Entity References" to help manage this. However my opinion is that if you have duel relationships within your data, then your data model is not optimum.

Comments

HiMyNameIsSeb created an issue. See original summary.

himynameisseb’s picture

Note:
I have been pointed to the following to help in my particular case, it may also help some else who stumbles across this issue:

https://www.drupal.org/docs/8/modules/search-api/developer-documentation...
https://www.drupal.org/docs/8/modules/search-api/developer-documentation...
(I have not trialled the solutions yet)

The feature request still stands. :)

drunken monkey’s picture

Title: Entitiy reference "referencing entity" or "back referencing" for search API » Entity reference "referencing entity" or "back referencing"
Component: General code » Plugins
Priority: Normal » Major

Yes, the links in #2 would have been what I'd suggest, too, for implementing this on your custom site.

It's also how I'd solve this in general, I think: add a processor which adds properties for each “reverse reference”. (For performance reasons, it should probably be possible to disable that processor, though, I'd think.) It probably shouldn't be too hard to do that, but will definitely require quite a bit of work – and unfortunately I'm pretty swamped at the moment (or chronically, really).
But it's definitely a very interesting and promising feature request, so I'll keep it in mind and will see if I get a chance to work on it. Or, if someone else would like to take a stab at it, I'd of course be glad to offer support.

If you came up with some custom solution for your site, and it looks like something that could be expanded for a general use case, it would also be awesome if you could post that, so we at least have some starting point. (But probably you just hard-coded the tricky parts, I'd guess.)

drunken monkey’s picture

It is possible to create relationships that go both ways with fields on both entities and use modules such as "Corresponding Entity References" to help manage this. However my opinion is that if you have duel relationships within your data, then your data model is not optimum.

Oh, and full agreement with this! Don't be sloppy with the data model!

himynameisseb’s picture

Thanks for the feedback.

Our solution for release was to remove the need to filter search API views this way. It is however still on the longer term roadmap of the project, so when we get to it I will look at options and feedback here.

Thanks again for your input!

drunken monkey’s picture

Priority: Major » Normal
Status: Active » Needs review
Issue tags: +Needs tests
StatusFileSize
new13.29 KB

As luck would have it, I did have some time today, all new issues/comments were looked on and I decided to tackle this.
The attached patch is a WIP, with everything already in place, as far as I can tell, but me having run out of time before I was able to actually test this. (Or even begin adding automated tests, for that matter.) So, it will probably still explode horribly when being used, but it should already be a good base from which to work. Reviews are also already welcome.

I hope I’ll have some time later this week to finish this.

yohanaraujo07@gmail.com’s picture

Hi, I applied the patch but the backreference fields doesn't work, it says on Search UI fields were indexed but, when I go on Solr I can't see the indexed values. I tried to add a Text (Plain) as test.

drunken monkey’s picture

Status: Needs review » Needs work

OK, good to know.
As always, there came more work than anticipated, so I wasn’t able to work more on this yet. But I’ll get to it as soon as I have some time.

yohanaraujo07@gmail.com’s picture

I debug the code and first thing I did to solve the indexing issue was to add

//Remove prefix as the original entity doesn't have it.
$property_name = str_replace($prefix,'' , $property_name);

on the excerpt

   foreach ($to_extract as $property_name => $fields_to_extract) {
      //Remove prefix as the original entity doesn't have it.
      $property_name = str_replace($prefix,'' , $property_name);
      if (!isset($references[$entity_type_id][$property_name])) {
        continue;
      }
      $property_info = $references[$entity_type_id][$property_name];

inside addFieldValues()

However my backreferenced entity is multivalued and I'm still trying to figure it out how to make to work that out as the problem is now with the field name of the multivalued field sent to Solr. Solr expects it to be prefixed with sm instead of ss.

Debugging further I can the issue is now on a later phase while building that Solr name and searching for properties on the field Entity (node), the Solr code gets lost because of the prefix search_api_reverse_entity_references_

drunken monkey’s picture

Does it work for you if you switch to the Database backend?

drunken monkey’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new7.4 KB
new18.25 KB

Found and fixed the bug and added some tests.
This looks like a first working version to me, and might already be RTBC. Please test/review!

Two problems are uncovered/highlighted by this new feature, which can’t be fixed in this issue:

  1. The property labels are, in this case, double-escaped in the “Add fields” form. (See also: #3001259: Remove wrong Html::escape() calls to prevent double escaping.)
  2. It would be neat if we could just index the referenced entity directly. Just indexing the entity ID doesn’t come with the same benefits regarding, e.g., facets and Views filters. (I think there was already an issue for that, too, but I can’t find it atm. This could be related, but I don’t think it’s the one I mean: #2851731: Allow indexing of entity references via the data reference properties.)
shraker13’s picture

#11 worked for me. Thanks! Just saved me a major headache.

shraker13’s picture

#11 worked for me perfectly.

valic’s picture

StatusFileSize
new18.26 KB
new585 bytes

Small adjustment to patch at #11, probably missed.
'node' is hardcoded so causing that on all back referenced non-node entites fields are loaded from node entity
- $property->setEntityTypeId('node');
+ $property->setEntityTypeId($entity_type_id);

kumkum29’s picture

Hi,

I want to filter my results by a data related to the referenced entity. For this, I tried to create a facet block from a backreference field.
When I visit the views page, the facet block seems to be always empty...
I have pached the search_api module with the #14 patch.
Can we create facet block with datas from a backreference field?

Oops, I can create a facet block from a referenced data with a call in the structure of the field, e.g. : My entity > Content > Field

thanks.

valic’s picture

Does anyone have this working? Even with the latest patch, seem exists some issues.
If I add a field from the referenced entity, I always get something like this on indexing:

ERROR: [doc=rzjsf2-orders-entity:commerce_order/1000476:und] multiple values encountered for non multiValued field its_payment_payment_id: [308484, 308496]

But this field is a single value, and does not exist any other field with this id on this index?
This issue is always present in any field from back referenced entities.

valic’s picture

Ignore my last comments :-) Patch is working, the problem which had encountered is more related to structure and inconsistent data.

Assuming that you have a field (back reference entity) which have cardinality 1 (getPropertyPathCardinality in Solr), but in DB exist two items.

So Solr think he needs to get one value, but he gets two values from addFieldValues.
Not sure where to address this issue, but on another side in my case is more issue related to some bad data (backreference should only once link to the original entity)

valic’s picture

Adding new patch addressing the issue at #17.

We should probably check field cardinality, and then compare it with a count of loaded entity ids.
Think it could be a lot of cases where data could stay in DB for any number of reasons, and without this "hackish" solution
indexing of items will always break.

The idea in last addition to the patch is to get field cardinality. And only in the case where field cardinality is 1, and we have loaded more entities, just to take the last one from an array. It is "hackish", but don't see any other solution currently.

Status: Needs review » Needs work

The last submitted patch, 18: 2986623-18--reverse_entity_references.patch, failed testing. View results

valic’s picture

StatusFileSize
new19.55 KB

Sorry, bad patch at #18.
Adding again, proper patch against #14

valic’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 20: 2986623-19--reverse_entity_references.patch, failed testing. View results

valic’s picture

Status: Needs work » Needs review
StatusFileSize
new19.54 KB
new585 bytes

Again missed some things :-(
now with interdiff between #12 and last changes from comments #17, #18.

valic’s picture

StatusFileSize
new2.39 KB

Status: Needs review » Needs work

The last submitted patch, 23: 2986623-19--reverse_entity_references.patch, failed testing. View results

drunken monkey’s picture

Status: Needs work » Needs review

Ah, right. The Solr backend is known to have problems with multi-valued fields in some cases, because they try to guess the field’s cardinality (and fail in a lot of uncommon cases).
The fix, however, should fortunately be simple – see the attached patch. (Interdiff compared to #14. The patch also includes a bit of refactoring, which I left out of the interdiff for the sake of readability.)

drunken monkey’s picture

drunken monkey’s picture

Any feedback on this?

valic’s picture

Hi,

Sorry, we launched new web a few days back, so haven't had time :-)
Have few tasks lined in upcoming days, Solr related. So then should be some kind of feedback on the last patch

drunken monkey’s picture

*bump*

  • drunken monkey committed 646db6b on 8.x-1.x
    Issue #2986623 by drunken monkey, valic: Added the "Reverse entity...
drunken monkey’s picture

Status: Needs review » Fixed

OK, growing tired of waiting for feedback on this, so: committed.
Let’s just hope it works fine.

FloGe’s picture

Thank you for this patch!
Searching for groups whose content contains certain words is now working for our application. I have just another question to this.

Is it possible to limit the search to a certain group content type if more then one group content type is using the same field?

E.g. you have two group content types on for the pros and one for the cons concerning some issue which use the same field (maybe comment or so). If you now are searching for groups having x in the pros, you will also find the groups having x in the cons comments.

As far as I see I have no possibility to add a relationship or so to get the group content type into the filter criteria. Or do I miss something?

For the moment my workaround is: add a field to the group content type which saves the type (default value), add this to the search index and set the filter on this field.

kmajzlik’s picture

StatusFileSize
new70.17 KB

This looks good except one small thing. Using t() placeholder with % is not working properly.

Deno’s picture

Works fine for me - I'm using it to include group content in the group search. In simplified form, this is what I'm adding to my index:

  1. Group => Searchable fields within group
  2. reverse link to group nodes => Searchable fields within group nodes

Choosing the entries to include in search is a bit tedious, but this is a standard issue with search api. Other than that, it works as designed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

mulongokato.2000’s picture

Does anyone know how to add "group membership" relationships?

I have tested and proved that the "Content (entity_id:entity)" field under Group content is bridging the gap for node entities, and new fields can be discovered automatically, but I have not seen how the "membership" relation to user accounts/profiles is implemented, thou its mentioned in the issue summary. How can I access group members?

[Group] <-- referenced by-- [Group content] -- references--> [User]

I am not technical, but a field like "Membership" to bridge user accounts to the group_content (membership) type that references them will be great, or help on how to implement this use-case.

Thank you in advance

borisson_’s picture

@mulongokato.2000 Please open a new issue and link to this one.

jcandan’s picture

I am not sure I follow how to go about creating an index with this. When building an index of content, how do I get the Group that the content is associated with to show up in the index?

ludo.r’s picture

I have the same question as mulongokato.2000 and jcandan.

I just created a support request for that, see #3207808: Group: how to get group content for current group in Views?

Thanks!