Currently, the „Index hierarchy“ data alteration only offers the direct fields of the base entity for creating hierarchies, and not related fields. We should fix that to offer all indexed fields which can create hierarchies.

CommentFileSizeAuthor
#34 Untitled-1.png43.45 KBb-prod
#24 hierarchy_facet.jpg227.99 KBgilsbert

Comments

tnightingale’s picture

tnightingale commented

I was just about to create an issue for this myself. I am happy to work on this as we need it implemented for one of our projects. I am planning to get started tomorrow.

I am aware that Search API already supports indexing related fields. Is this just a matter of applying the same approach in the data alteration?

Any pointers/suggestions/inside knowledge you might have would be greatly appreciated ;-)

tnightingale’s picture

tnightingale commented

It looks like getHierarchicalFields() in SearchApiAlterAddHierarchy would need to be refactored to use a similar property/wrapper traversal as one used in search_api_admin.inc _search_api_admin_get_fields().

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

Regrettably, it's rather more complicated there as we manipulate the entities directly in the data alterations. (Can't remember why I made that decision, seems rather stupid to me now. :-/) So when, e.g., making a node's author's tags (author:field_tags) hierarchical, we should keep in mind that several nodes can have the same author as we have to directly change the user entities. So we should proceed about as follows:
- Collect all authors of the nodes in $items
- Load them and manipulate their hierarchical fields accordingly
The real problem here is that we then have to depend on the static entity cache for retaining these changes until the field values are extracted. Meaning this won't even work for entities that don't have static caching, like comments.
This would be far more easier for a processor, but those can't change the property information, which we must do. So I don't really see any better options there.

It looks like getHierarchicalFields() in SearchApiAlterAddHierarchy would need to be refactored to use a similar property/wrapper traversal as one used in search_api_admin.inc _search_api_admin_get_fields().

Please note that, with #1308638: Reduce size of stored index settings, there is now $index->getFields() for that.
I don't really think we need such a complicated traversal, though. Just check all indexed fields for a entity_type key, remember all encountered entity types and then check each of them for recursive properties. All fields of entity types that have such a property can be made hierarchical.

Anyways, it's great that you take this on, thanks a lot!

dimarick’s picture

dimarick commented

Thanks for work. I am see your code, and I am not find simple way to fix this issue too. Possible, can try to create additional field for current entity, like Aggregated fields data alter, with full hierarchy of taxonomy terms? How difficulty make field with properties like taxonomy_term entity or other entities? Or simply rewrite this as workflow:processor?
P.S. Sorry for my english.

mh86’s picture

mh86 commented

I don't think directly modifying the entity is a good idea. Isn't there a possibility to implement it as post processor and modify the data structure just before it gets indexed?

mh86’s picture

mh86 commented

Furthermore you may receive errors like this (when immediately indexing your entities):
"The content on this page has either been modified by another user, or you have already submitted modifications using this form. As a result, your changes cannot be saved."

This warning disappears once the 'Index hierarchy' settings is deactivated.

steven jones’s picture

steven jones commented

If you can make assumptions about the field type, then you can make some progress here, I've created a module that allows you to index parent taxonomy terms on related fields: http://drupal.org/project/search_api_taxonomy

mh86’s picture

mh86 commented

An alternative to this is the "All parent terms" property, which works on related fields as well.

steven jones’s picture

steven jones commented

Ah right, so you index that field, instead of the actual field. Nice!

mh86’s picture

mh86 commented

and "All parent terms" in this case means: the term itself and its parents. The naming is a bit confusing.

capnut’s picture

capnut commented

So guys, have you been able to find a solution to enable index hierarchy on related fields?
I've tried all the possible combinations of including related field in index and also tried #7 sandbox module, but just cannot see my related field showing up in workflow/index hierarchy.

gilsbert’s picture

gilsbert commented

Hi.

I have the same need. Well, in fact, perhaps it is just a similar need: my taxonomy field is inside a field collection and I can't pick it for index hirarchy.

I tried the sandbox project and the workaround #7. I tried both together.
No success.

Regards,
Gilsberty

user654’s picture

user654 commented

.

user654’s picture

user654 commented

I have indexed both the All parents term and Parent terms of not the base Entity and using these fields to a views exposed filter but I take an error:
An error occurred while trying to search with Solr: "400" Status: null: null<html> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/> <title>Error 400 null</title> </head> <body><h2>HTTP ERROR 400</h2> <p>Problem accessing /solr/select. Reason: <pre> null</pre></p><hr /><i><small>Powered by Jetty://</small></i><br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> <br/> </body> </html> .

How can this be fixed?
Thanks

user654’s picture

user654 commented

.

nick_vh’s picture

nick_vh
he/him
Location Ghent
commented
Version: 7.x-1.x-dev » 8.x-1.x-dev
Issue summary: View changes

Bumping this up to Drupal 8.x

user654’s picture

user654 commented

Could this also be fixed for Drupal 7?

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Component: Framework » Plugins

Could this also be fixed for Drupal 7?

I don't think so. You can work around it with custom code, or by indexing the "All parent terms" property instead of the field itself.
For Drupal 8, though, the fix is trivial since processors will be merged with data alterations to combine the strengths of both.

user654’s picture

user654 commented

.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

Indexing and using "Tags » All parent terms" instead of "Tags" (or whatever the name of your term reference field is) should have the exact same effect as using "Index hierarchy" when indexing "Tags", but is compatible with properties on related entities. There is not much to elaborate on.
Could you elaborate on what you did and what goes wrong?

user654’s picture

user654 commented

Hi drunken monkey.
I have a node and its author.The author field has an "Area" taxonomy term field.The Area can have values :France>Paris ,France>Lyon etc
I want to index the hieararchy so when searching for "France" it returns all nodes that have been tagged with Paris,Lyon etc.
I have tried to index the field Parent all but with no luck.
I hope this is clear.
Thanks again

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

Are you getting at least the normal results, where the "lower level" term is directly searched, when indexing the field itself (not "All parent terms")? If not, please see #2217171: Term reference properties on users are empty which I just discovered while testing for this.
Otherwise, I'm not sure what could cause this problem. What backend are you using? It would be great if you could see what gets indexed for that field.

user654’s picture

user654 commented

.

gilsbert’s picture

gilsbert commented
StatusFileSize
new hierarchy_facet.jpg227.99 KB

Hi @pinkonomy.

I got a similar problem with hierarchy facets. Perhaps my solution can work for you too.

I'm using search_api and search_api_db with a hierarchy taxonomy for local just like what you need.
I have the country the state and the city in this hierarchy.

In the "data alterations setup" I'm using "index hierarchy with the option [parent terms]". I also have the option [all parent terms] but I'm using [parent terms] with success. I really dont know what is the difference between both.

Ok, now lets talk about the facet.
In the facet display's setup there will be an option name "hard limit" and its default is 50.
When I was using the default value my facet was not working and the hierarchy was broken. I can't explain why.

What I did was to change this option to "no limit" and my facet is now working perfectly.

Please, note that there will be two different options: "soft limit" which is implemented by javascript and works great to reduce the lenght of the facet list (the user can click over "show more" or "show fewer") and "hard limit" which I suppose cuts the number of items in the facet's list which might as well cut a parent term and given us a broken hierarchy facet.

And still at the facet display's setup I also use the option "flatten hierarchy" with the value "no".

Bellow an image with the most important steps.

hierarchy facet options

Regards,
Gilsberty

P.S.: this is NOT the solution for the original post/issue and it is just an example of setup to produce a hierarchy facet!

user654’s picture

user654 commented

.

gilsbert’s picture

gilsbert commented

Hi @pinkonomy.

I know two solutions for hierarchical information and view's exposed filters.

You might use https://drupal.org/project/hierarchical_select . I used it in the past and this module will fix your problem but you must understand what it does: all terms (children and parents) will be related with your node! For example if the node will be related with Lyon this module will save the relation with France and Lyon for the same node. This is not what Drupal's default behavior makes on that kind of situation (it saves only the children relation or in another word only the relation between the node and Lyon) and I'm not sure what impacts that it would produce on search_api. Of course it might be changed now... Anyway it is a good module!
I can't remeber but I believe this module will give you a new type of filter criteria to be used inside a view (I did the test 6 months ago).

You might as well use https://drupal.org/project/shs (thats the one I'm using on my projects).
This module will save only the children (just like Drupal's default behavior) and will give you a new type of filter to be used inside a view.

What I'm not sure is if those type of filters will work in a view using a search_api index!
I guess it might not work.

I suggest the facet instead of exposed filter because it will work with a search_api index for sure! Of course we can't use a related field in this case (in my case I could not use a term reference inside a field collection)... But we will be able in the future.

Regards,
Gilsberty

user654’s picture

user654 commented

.

user654’s picture

user654 commented

Question for drunken monkey:Will this be possible to fix with custom code?
If yes,how can I do that?
thanks

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

Yes, this is definitely possible to solve with custom code. However, it would be first good to know why indexing "All parent terms" doesn't work, since it definitely should. The cause for that not working might also keep other, custom solutions from functioning.
If you just want to try it nevertheless, you could, e.g., use a data alteration to just change the field's type to a list type and then use a processor to actually change the field value (adding all the ancestors' TIDs).

user654’s picture

user654 commented

.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

That shouldn't have anything to do with it. But I really don't know what the cause is, so it might also be possible.

b-prod’s picture

b-prod commented

I have created a new data alteration callback, based on the hierarchical one, to handle nested fields.

It is a separate class, that extends SearchApiAlterAddHierarchy. But it could be easily merged to the original class if wanted.

@drunken monkey: do you want a patch that integrates the nested fields hierarchy to the original class? If so, I could work on it. Otherwise where should such feature go?

Note that this may require, for some configurations, the patch in #2090007: EntityDrupalWrapper::getIterator() throws PHP error: EntityMetadataWrapperIterator::__construct() must be an array as the search_api_extract_fields() uses a foreach loop on a list Wrapper that could return NULL instead of an empty array, dur to a bug in the Entity module. But this is not specific to the new class I am talking about.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

@drunken monkey: do you want a patch that integrates the nested fields hierarchy to the original class? If so, I could work on it. Otherwise where should such feature go?

How would you do that, in Drupal 7? I wouldn't have thought that's really possible.
If it is, patching the original class would be preferable.

b-prod’s picture

b-prod commented
StatusFileSize
new Untitled-1.png43.45 KB

Sorry for the delay, I do not have time to connect to d.org on a regular basis.

Actually the main problem is that when dealing with related entities, Search API does not create a separate "item" to which values are added when necessary, but relies on the entity itself... This is not the best way but with that as a starting point, it is just a matter of modifying the related entity as expected.

Here is a screenshot of the "index nested hierarchy" filter (this name is only here to not override the original "index hierarchy"). I think merging with the original alter callback is better than provide a new one (adding confusion).

A better way do deal with related entities (node author, entity reference, relation...) could be to create a separated item that is finally indexed, eventually adding some special properties to retrieve the wrapper of the related entity. But I looked first in this way and it seems to be a heavy lifting with probably no possible upgrade path for contributed modules that deals with Search API...

I will work on the merge in the next weeks and post here for review when ready.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented

Actually the main problem is that when dealing with related entities, Search API does not create a separate "item" to which values are added when necessary, but relies on the entity itself... This is not the best way but with that as a starting point, it is just a matter of modifying the related entity as expected.

The problem about that (if I understand you correctly) is that you then have to completely rely on entity caching, since the related entity isn't passed along via the Search API. I've also explained that already in #3. You'd have to:
- load the related entity;
- modify the value of its hierarchical field(s); and
- count on the Search API to load that entity from the static cache when extracting the field values.

And even if that works, it will result in all other code using that entity in the same page request getting the altered data – with unforeseeable consequences. Really, the solution for the existing "Index hierarchy" alteration is hack-ish enough already.

What could, however, work is adding additional properties representing the related fields which have a getter callback defined that takes care of loading the hierarchical data. This might also be a much cleaner implementation of the existing functionality, I think.

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Version: 8.x-1.x-dev » 7.x-1.x-dev

No idea why this was set to D8.

rominronin’s picture

rominronin commented

Just for the record, the advice in #20 for getting hierarchical taxonomies to display in a facet *when the reference field is in a field collection* works fantastically - this is hard to search for online, took me a while to get to this suggestion: Thanks '@drunken monkey'.