Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
It would be practical, both for the Facet API integration (#1182614: Integrate with Facet API) and generally, to be able to index all parents along with a taxonomy term. This should be easily possible with a simple data alteration, which could also be coded generic enough to work for other hierarchies.
It would also be possible to add this to the "Aggregated fields" data alteration, but I doubt that would be a good idea.
Comment | File | Size | Author |
---|---|---|---|
#20 | 1213698--hierarchies-20.patch | 14.87 KB | drunken monkey |
#12 | 1213698--hierarchies-12.patch | 15.42 KB | drunken monkey |
#8 | 1213698--hierarchies-7.patch | 11.51 KB | drunken monkey |
#6 | search_api_dump.txt | 59.15 KB | zambrey |
#6 | search_api_dump_min.txt | 4.93 KB | zambrey |
Comments
Comment #1
zambrey CreditAttribution: zambrey commentedSubscribe
Comment #2
dmiric CreditAttribution: dmiric commentedsub
Comment #3
drunken monkeyThis one works for me, please test / review.
I've also sneaked in a few other minor fixes, you can just ignore those. It would just be good to know if
search_api_extract_fields()
still works the same with the change (should now just need much less entity loads).Comment #4
zambrey CreditAttribution: zambrey commentedI finally have time to test this feature.
So I have applied patch and enabled Index hierarchy data alteration for one category.
I have this error when trying to reindex data:
As you can see I'm using DB backend with PHP 5.2.6.
Comment #5
drunken monkeyHm, I can't reproduce this. I take it you use the latest version of all modules?
Then please add
debug($item); return FALSE;
at the beginning ofSearchApiDbService::indexItem()
, index a single item manually and post the result.Also: Does this happen for all items, or just for some? (Kind of complicated to tell that, I guess. You can index a single item manually and see whether the error occurs. Then you could also mark items indexed by adding
return TRUE;
as the first line inindexItem()
and see if it also occurs when indexing other items manually. Or callsearch_api_index_specific_items()
directly from code.)Comment #6
zambrey CreditAttribution: zambrey commentedAttaching debug message. It's kinda big but hope it will help a little.
I'm using SearchAPI taken from git with this patch applied, Entity API 7.x-beta10, Views latest dev.
EDIT: enabled only this one category for indexing and the error is gone, attaching debug with that also.
But now only two items are indexed and I have many messages in logs like this:
I will try to find which field causes fatal error mentioned in #4.
Comment #7
zambrey CreditAttribution: zambrey commentedOkay, I can't reproduce the error but still only 2 nodes are indexed.
Comment #8
drunken monkeyAh, I knew I messed something up in
search_api_extract_fields()
. Restored that to the original version, I should just fix it properly, in a different issue.The SQL exception is really weird, this really should not hap— oh, wait. Please go to the Fields tab, save the form and see whether this fixes the error.
The error could also stem from an interaction between the "Aggregated fields" data alteration and this one. See if the error occurs when enabling only those two data alterations and no other fields.
Otherwise, yes, please find out what fields cause the errors.
Comment #9
drunken monkeyAh, of course you can't reproduce – as you just wrote, you changed the fields you index, so you already saved the Fields for mas I suggested. Sorry, didn't think of that.
Anyways, seems like that has to be fixed somehow. The issue here seems to be that when the type is changed, the server won't be notified about it.
Comment #10
drunken monkeyOK, now I'm confusing myself completely. To summarize, the two bugs:
- SQL exception when indexing most items: I'd guess this is due to stale field type data in the index. When activating the alteration and saving the "Fields" tab, this should vanish. (But you say, they didn't, so maybe I need a new theory …) Do the entries that are reported as duplicates correspond to the node nids you try to index?
- Fatal error: This I think could come from the interaction between the two data alterations.
Comment #11
zambrey CreditAttribution: zambrey commentedOkay, it works now BUT you must re-enable field that you want to index with hierarchy.
Here's detailed info:
- after applying latest patch category wasn't indexed (it was enabled - first WTF)
- resaving field configuration (haven't changed any field) and disabling hierarchy caused field to be indexed again (NICE)
- enabling hierarchy for category caused #4 error (second WTF)
- resaving field configuration (haven't changed any field) - only two nodes are indexed (getting #6 exceptions)
- disabling and enabling field finally makes all nodes to be indexed with all hierarchy (Yay!)
Comment #12
drunken monkeyThanks for investigating and explaining the problem! I think I now understand what might be wrong here.
The attached patch hopefully fix the bug. I've also included a note that all fields should be de-selected before disabling the data alteration – otherwise there's hardly a way to keep this from triggering in this case. (A flaw in the current framework which will have to be fixed separately.)
Comment #13
marvil07 CreditAttribution: marvil07 commentedI just wanted to point that maybe is better to rely on a external module for doing conversions with fields. Actually cpliakas and me are starting a discussion about it. So this alteration could be a plugin inside converter module.
I hope I am getting this right :-p
Comment #14
drunken monkeyI heard about the Converter module from Chris already, but from the description it doesn't sound like those kinds of things would fall into its area of expertise. This is, after all, more about changing meta data than changing the data format.
Also, Converter isn't usable yet, so we're gonna need an interim solution anyway. However, once Converter is stable enough, it'd be definitely great to integrate it into the Search API (probably with a generic "Converter" data alteration). After all, I've had only positive experiences with Chris' generic frameworks so far. ;)
(By the way: Are you gonna be in London? We'll probably do a Search API BoF there.)
Comment #15
zambrey CreditAttribution: zambrey commentedI tried to use this on another site and it works brilliantly!
Thank you very much for this! Awesome work.
Comment #16
marvil07 CreditAttribution: marvil07 commented@drunken monkey: I see your point, and I am glad you are open for integration in the future. About London, visa stuff let me out this time :-(, but hopefully I will be tracking activity about search anyway the best I can(this project queue issue queue is growing exponentially, good work!).
Comment #17
dmiric CreditAttribution: dmiric commentedHey thanks for solving this problem. Only cant find how to make this work. Updated search api to latest dev version applied patch. Everything passed ok.
Only problem is I don't know how to use it.
Where are the options?
on admin/config/search/search_api/index/nodes_3/fields
i dont see anything new
Can you please write few steps on how to make this work. Maybe I'm misunderstanding what this patch should be doing ?
What I think it does is when i enable indexing of Nodes it should give me somewhere option to say that in that node this vocabulary should index the complete hierarchy. I looked over every option there is again and cant find anything new.
Comment #18
drunken monkeyGo to the index's "Workflow" tab. There is now a new data alteration, "Index hierarchy", which lets you do that.
Comment #19
dmiric CreditAttribution: dmiric commentedYeah I checked there and theres no such option :( ...
Should I role both patches or just last one ? And does it work on latest dev or only on beta build ?
Comment #20
drunken monkeyAh, my apologies! I didn't remember to re-roll this after committing #1064884: Add support for indexing non-entities – a working patch should be attached.
And thanks for reviewing!
Comment #21
davidseth CreditAttribution: davidseth commentedJust applied patch at #20 and no "Index hierarchy" shows up in the data alteration area. :(
Comment #22
drunken monkeyDo you have the latest module version, and did you clear the cache?
Comment #23
davidseth CreditAttribution: davidseth commentedI am using the facet_api integration branch. Your patches applied cleanly. I assumed I needed to use that branch to get integration with Facet API in the first place. Maybe you can shed some light on that? Thanks.
Comment #24
drunken monkeyYou need the branch for Facet API integration – however, this patch is in principle independent of that development. You can index hierarchies without it – you just won't get the corresponding hierarchical facets.
And I just tested it, it works with both the normal Search API and the Facet API integration branch / fork. If you cleared the cache and it still doesn't appear in the workflow (even though you have taxonomy term references on the indexed entity), maybe check the logs what might go wrong. Or debug the
search_api_admin_index_workflow()
function to see what goes wrong there.Comment #25
davidseth CreditAttribution: davidseth commented@drunken monkey: I have got it to work! I had to change two things though.
First there is no entry in search_api_search_api_alter_callback_info() for the new class: SearchApiAlterAddHierarchy. So I added:
to the end of the definitions. This now meant I could see the option, but then I kept getting errors. I have Field Collection module in place and it was causing errors so I added (at line 211):
Note the $type == 'field_collection_item' addition. Is there any reason that this doesn't simply filter only on taxonomy_vocabary?
After I made those two changes above and re-indexed everything worked perfectly!
Cheers,
David
Comment #26
drunken monkeyThis probably means you made some mistake while applying the patch. See the last-but-one change in the patch linked in #20.
Yes – as the functionality is not really specific to taxonomy terms (by the way, for those, filtering on "taxonomy_term" would be the right thing to do), I figured we could just as well allow all other hierarchical information, too.
What kind of errors did field collection items cause? Hard-coding a special case is definitely not the way to go here, so we should look for why this causes errors.
Comment #27
davidseth CreditAttribution: davidseth commented#26 - This is the error I get when it is parsing 'field_collection_item' field types:
Comment #28
drunken monkeyWhen does the error occur – during indexing, or already when configuring the data alteration?
And are you using the latest version of the Entity API? If I'm not mistaken, a similar bug was recently fixed there, and this does seem like an Entity API problem.
Comment #29
davidseth CreditAttribution: davidseth commentedOkay, updated to latest Entity API dev and it works.
BTW, the original error occurred when configuring the data.
This patch works well, please commit.
Cheers,
David
Comment #30
drunken monkeyAh, thought so.
Great to hear it now works. Thanks for reviewing!
Committed.