Currently, all entity properties up to a certain level of indirection will be available as field handlers for Search API index base tables.
While a nice work-around for me not needing to dive into the code of another Views subsystem, this is quite restricted, doesn't abide to the normal way this works in Views and, most importantly, can lead to serious performance issues for more complex site.
Therefore, I think there is little choice than to finally do this the right, not the easy way, and use relationships. It might be possible that a generic Entity Relationship Handler in the Entity API could help us there – but probably I'd have to write that one myself as well. Right now, I have little to no conception of how relationships work in Views (from a developer's perspective), so … let's see, what might be best there.
In any case, this would probably lead to some data loss on updating, unless there's a feasible way to write an update function. Therefore, this has definitely to happen before a 1.0 release – tagging accordingly.
Comment | File | Size | Author |
---|---|---|---|
#51 | 1231512--views-overhaul-follow-up-51.patch | 1.58 KB | drunken monkey |
#49 | 1231512--views-overhaul-49.patch | 36.99 KB | drunken monkey |
#47 | 1231512--views-overhaul-47.patch | 38.03 KB | drunken monkey |
#44 | 1231512--views-overhaul-44.patch | 37.69 KB | drunken monkey |
#43 | 1231512--views-overhaul-43.patch | 40.21 KB | drunken monkey |
Comments
Comment #1
jziwenchen CreditAttribution: jziwenchen commentedthank your work.
how is going on this feature ?
Comment #2
mh86 CreditAttribution: mh86 commentedsubscribing
Comment #3
BerdirSubscribing
Comment #4
BeaPower CreditAttribution: BeaPower commentedsub... this would mean I could add fivestar to search results right?
Comment #5
drunken monkeyNo, I don't think so (if it isn't possibly now in the way you mean it). This wouldn't actually add (much) functionality, but rather fix the way current functionality is provided.
Comment #6
BeaPower CreditAttribution: BeaPower commentedOk, is there a patch anywhere to include fivestar ratings? I'm creating a reviews site, Thanks.
Comment #7
Shadlington CreditAttribution: Shadlington commentedYou can display fivestar in results already if you use the entity row style rather than fields.
However, you cannot sort or filter by fivestar as it does not implement the entity api.
If you want to go into further detail on this its really more appropriate in its own issue - and would need an issue in the fivestar queue to get anywhere useful with regards to sorting/filtering/etc.
EDIT: And now that I've looked at the queue I can see you've already done that #1256572: Integrate with Entity API
Comment #8
drunken monkeySee #1266036: Add generic Views entity tables with fields and relationships for 80% of the work for this issue.
Then, I'll just have to throw out field handlers for related fields and add the relationships.
Comment #9
drunken monkeyI'm now starting to implement this in parallel to my work in #1266036: Add generic Views entity tables with fields and relationships. Attached is a version adapted to the current state of the Entity API views_integration branch. It doesn't work as such, has one nasty but probably small bug (no field handlers are added) and surely tons of others. Particularly, I have yet to figure out how to support non-entities without extending all the field handlers. Got an idea, though.
Anyways, this is not really suitable for testing, but already shows where this is going (or supposed to be going).
Comment #10
Pedro Lozano CreditAttribution: Pedro Lozano commentedSubscribing. I've been trying to build today a search view that needs this. I'll do some testing as soon as it's working.
Thanks.
Comment #11
drunken monkeyCurrent status. Relationships still not working in any way, but the rest is looking more and more promising.
Comment #12
jakonore CreditAttribution: jakonore commentedsubscribing
Comment #13
jsacksick CreditAttribution: jsacksick commentedsubscribing
Comment #14
tnightingale CreditAttribution: tnightingale commentedsubscribing
Comment #15
drunken monkeyLatest status. As written in #1266036-12: Add generic Views entity tables with fields and relationships, almost everything already seems to work (as far as my bit of testing goes). Relationships are missing, though.
Comment #16
drunken monkeyLittle correction to avoid breaking the entity row plugin (which still uses the magic
$row->entity
property).Comment #17
drunken monkeyBut seems to be done, as far as I can tell. (Some bugs will surely spring up, though.)
Comment #18
marcoka CreditAttribution: marcoka commentedi applied the newest patch with git apply -v patchname.patch somehow git applys not all....testing with pact p1 again.
Comment #19
drunken monkeyDid you apply the patch in #1266036-14: Add generic Views entity tables with fields and relationships as well? The patch here is just an application of the changes in Entity API – without those, this has to fail.
Comment #20
Pedro Lozano CreditAttribution: Pedro Lozano commentedWorks like a charm for a very simple setup, I'm trying now to use it on an existing site but everything breaks when the patch are applied.
Is a upgrade path needed for any of the two patches (entity, search_api)?
Comment #21
drunken monkeyYes, as the field handlers for the Search API tables have changed (as have, consequently, some of their options) and you also have to use relationships to get some fields that were previously included by default, some existing views might break.
Creating update code would probably be possible, but rather tedious as you'd have to hack deep into the views' data structures. Regretable as it is, I think it would be easier if people just updated their existing views by hand. Usually, only a few field handlers should be affected.
Or do you mean something else by „everything breaks“? A little elaboration would be nice here.
Comment #22
Pedro Lozano CreditAttribution: Pedro Lozano commentedI got several PDO sql exceptions but I didn't saved the text to paste it here.
I'm trying another approach now. Uninstalling search_api, upgrading entity, upgrading search_api, enable search_api again and build the view form scratch. I'll report later how things are going.
Comment #23
Pedro Lozano CreditAttribution: Pedro Lozano commentedI got several PDO sql exceptions but I didn't saved the text to paste it here.
I'm trying another approach now. Uninstalling search_api, upgrading entity, upgrading search_api, enable search_api again and build the view form scratch. I'll report later how things are going.
Comment #24
drunken monkeyRevised version adding support for click-sorting. Use together with #1266036-19: Add generic Views entity tables with fields and relationships.
@ Pedro: It's weird, everything just kept on working for me. Did you make sure to properly clear the caches, etc.?
Comment #25
das-peter CreditAttribution: das-peter commentedsubscribing
Comment #26
Pedro Lozano CreditAttribution: Pedro Lozano commentedAfter this patch I would expect that any view that can be build without using a search index as a base table can also be build using the search index as a base, since you can fetch the referenced entity.
It is not true for the view I'm trying to build. It involves entityreference fields. Looks like the relationships support of entityreference fields is not working when I build the view using the search index.
I got the following data structure.
* Entity type: THINGS
* THINGS have an entityreference field pointing to content type A
* There is another content type B that has an entityreference field pointing to entities of type THINGS.
Without using search api, I can build a view that:
* Lists THINGS
* Prints the node title and fields from the referred node (type A) (Using entityreference views relationship support)
* Prints the node title and fields of nodes of type B that are referrencing each THING. (Using entityreference *reverse reference* views support).
When building the same view using the search index I cannot build such a view since the relationships options that Entityreference provides are not available in the views relationships interface.
So when building the view without search api I see the following options in the relationship dialog:
Entity Reference: Referenced Entity (A bridge to the Content entity that is referenced via field_referred_node)
Entity Reference: Referencing entity (A bridge to the Things entity that is referencing Content via field_referred_node)
Entity Reference: Referenced Entity (A bridge to the Things entity that is referenced via field_referred_thing)
Entity Reference: Referencing entity (A bridge to the Content entity that is referencing Things via field_reffered_thing)
None of these options are available when building the same view using the search index.
Let me know if you need more info.
Comment #27
drunken monkeyNone of these? That's weird, I thought that the first of these options should show. Do you get the option of indexing the reference (field_referred_node) on the index's „Fields“ tab? Otherwise, the issue might just be that Entity Reference doesn't provide Entity API integration yet, which is nothing we could fix.
Reverse entity references are not supported right now. I'd have to look into how these work / could be made to work here.
Anyways, the problem here is not with the Search API part but with the Entity API part, so this should probably better be discussed in #1266036: Add generic Views entity tables with fields and relationships.
Comment #28
tnightingale CreditAttribution: tnightingale commentedHere is #24 rerolled to work with 7.x-1.x
Comment #29
drunken monkeySlight adjustment regarding sanitizing of field identifiers.
Comment #30
drunken monkeyAdding preloading of result entities.
Comment #31
das-peter CreditAttribution: das-peter commentedAm I right that the patch contains a "forecast" of this #1260812: Move "Database search" into its own project?
The patch deletes all the contrib module search_api_db.
Next guess, the removed module can be found here soon: http://drupal.org/project/search_api_db
Comment #32
das-peter CreditAttribution: das-peter commentedJust applied latest patch from here and synced to the latest entity views branch.
I discovered that if the search index is out of sync I get this notice:
This further leads to this nice exception - an thus to a broken site:
It looks like this could be fixed in the function
render()
ofentity/views/plugins/entity_plugin_row_entity_view.inc
by changing this condition:if (isset($id)) {
to this:
if (isset($id) && isset($this->entities[$id])) {
But I'm not sure if this is the (only|right) place where this kind of sanitization should take place. Has the search_api a good way to check wheter an entity id is still valid?
All other stuff looks very good - after reindexing :)
Thank you very much!
Comment #33
drunken monkeyAh, you're right, sorry!
Attached is a patch without these unrelated changes.
So the problem occurs when the Search API returns an entity ID of an already deleted entity?
a) This shouldn't happen at all, as we immediately remove deleted entities from indexes.
b) I guess this will, as you say, have to be fixed in the Entity API. You should create an issue there (if you haven't done so already).
Good to hear, thanks for testing!
Comment #34
mh86 CreditAttribution: mh86 commentedHey Thomas,
one of my views still throws some notices which I think are caused by this patch:
Undefined property: SearchApiViewsQuery::$group_operator in SearchApiViewsQuery->build() (Zeile 157 von search_api/contrib/search_api_views/includes/query.inc
Comment #35
drunken monkeyThat's (probably) known and unrelated: see and review #1305736: Notice: Undefined property: SearchApiViewsQuery::$group_operator.
Comment #36
mh86 CreditAttribution: mh86 commentedfrom http://drupal.org/node/1266036#comment-5106500 and according to Thomas, related to this issue
Comment #37
drunken monkeySorry for overlooking this one. Please see if the attached patch fixes the problem.
One difficulty I encountered was that
get_result_entities()
doesn't define what to do if a relationship maps to multiple entities for a row. (To be more precise, it doesn't define anything at all, of course. But for that specific case, even looking at the Views default implementation doesn't help.) I now just return the first entity, as returning an array might screw things up considerably more than just returning too few entities.Comment #38
mh86 CreditAttribution: mh86 commentedhm, no, it still doesn't work.
From what I can see $this->query->get_result_entities() just returns the profiles (now correctly), but misses the entity type.
Comment #39
drunken monkeyOops, sorry!
Should be fixed now, also fixed the error in help texts for some properties (e.g., body text).
Comment #40
drunken monkeyComment #41
mh86 CreditAttribution: mh86 commentedWell done, Thomas. My use cases (which are quite complex) seem to work now. Once the Entity API Views stuff is committed, I would say this is ready as well.
Comment #42
fagofrom the entity api issue:
-> We need to solve this UX problem.
Comment #43
drunken monkeyLittle we can do about this, I think.
I admit it could be confusing for users, but the same goes for the other alternatives. Also, this can be easily changed later, too.
Therefore, let's just leave it like it is and mayb change it if the masses begin to revolt.
Attached is a patch adapting to the latest changes in #1266036-90: Add generic Views entity tables with fields and relationships.
Comment #44
drunken monkeyUpdated.
Comment #45
davidseth CreditAttribution: davidseth commented#44 does not apply cleanly for me with the latest dev code.
Comment #46
mh86 CreditAttribution: mh86 commentedIn order to get the multi load functionality for base entities working, entity_load() must be called before EntityFieldHandlerHelper::extract_property_multiple() is used in get_result_wrappers().
Comment #47
drunken monkeyThen you're doing something wrong.
Ah, yes, thanks. Should be fixed in the attached patch. Also enabled multi-loading for entities with no static cache.
Any other objections?
Comment #48
mh86 CreditAttribution: mh86 commentedSomething goes wrong in #47. It seems to multi load the entities, but doesn't display them any more (in my view only one row out of 3 is shown, in another view all rows are display, but the values of a related field are missing).
Adapting patch from #43 by moving the entity_load() call up in get_result_wrappers() works as expected.
Comment #49
drunken monkeyLike this?
Comment #50
mh86 CreditAttribution: mh86 commentedyep, like this. but I think we can remove the check for the static cache, as done in #47.
Comment #51
drunken monkeyNo, we can't (or shouldn't). In #47 I used the entities in the wrappers, thus making this sensible for non-statically-cached entities, too. Since that didn't seem to work, we are now back to the old way, so it now finally works at least …
Committed this now. Any cosmetical or performance-enhancing changes can wait (in theory even until 1.1), we can't wait forever for RC1.
For making it also work with non-statically-cached entities, please see if the attached patch works for you. It's almost the same as the previous one (#47), though, and I don't really know what might go wrong there for you. It works at least for me.
Comment #52
jaymallison CreditAttribution: jaymallison commentedSo I'm curious about this issue and whether it's related to a something that's "missing" in my available fields in my View.
I've built out a view against a content index.
This view contains a relationship for a Drupal Commerce product reference against the listed node.
After adding this relationship, I can add fields for all of the Drupal Commerce fields except the "Add to cart" form. Under a normal Content view, this field is available and works great through a product reference relationship. It's only when the source is for the view is a Search index that this problem arises.
I'm using all of the latest -dev versions of Search Api and related modules. And I've applied that patch in #51 since it's not included in the current -dev.
I would have created a new issue, but it seems this must be related to this issue. Considering after I read this entire issue I saw someone mention that the bottom line with this patch was that you should be able to recreate any view from the search index that you can build with a normal view - if it's working correctly.
Let me know if there is any technical data I can provide.
Let me just say that this project is epic, and I am "this" close to basically developing a full e-commerce site that rivals Newegg in searching and filtering options contextually for each section thanks to this module. This is the last little piece...lol. Thank you for all that you do, it's amazing.
Comment #53
mh86 CreditAttribution: mh86 commentedGreat to have this finally committed :) Patch from #51 works as well.
Concerning #52, I guess Drupal Commerce doesn't yet use the generic field_entity handler with the view entity tables (see #1270890: provide a way to provide entity-related views fields for more info).
Comment #54
jaymallison CreditAttribution: jaymallison commentedI was afraid it might be an issue with Commerce, although I had hoped that since it works fine under a normal view that it would work with the Search index.
I'm not trying to pull the add to cart form off a search indexed field. I've added a new relationship to the view for the Content: Product and it is off of that relationship that I should be able to add the add to cart field.
I'm sure you already knew that though and I'm just going to have to wait for this to be resolved another way.
Comment #55
drunken monkey@ mh86: Great, committed. Thanks for testing!
@ jaymallison: As Matthias says, this isn't something that should work. For that, the Commerce module would have to provide that Views field in a query plugin-independent way, as introduced in #1270890: provide a way to provide entity-related views fields. Or you could manually add that field to the corresponding
entity_X
Views table. We use those tables for our relationships, not the normal ones provided for Views. Maybe the first two posts in #1266036: Add generic Views entity tables with fields and relationships might help understand the principle.Comment #56
jaymallison CreditAttribution: jaymallison commentedOk, thanks for responding and giving me some clues on where to go with this. Otherwise, this system is working really nicely. Great work!
Comment #57
jaymallison CreditAttribution: jaymallison commentedI hate to post in this issue again, but I'm not sure if this is related or not...
If you add fields to the index via the "Add related fields" dialog in the admin and then have them indexed, you of course can access those fields as normal fields in views... What I'm not seeing though, is a way to allow those fields to be a part of a views Sort. They simply just don't show up in the Sort section of views, even though I've set them as string or integer and they are single value.
Comment #58
drunken monkeyDoes work for me. Are you sure both the field and the relation are single-valued? In the HTML code of the index's „Fields“ tab, what are the „Type“ values you can select for the field?
Also, please start creating new issues for such questions.
Comment #59
jaymallison CreditAttribution: jaymallison commentedI'll create a new issue, sorry.
Comment #61
checker CreditAttribution: checker commentedHave you created a new issue? I can't find it and i have exactly the same problem like #57.
Comment #62
checker CreditAttribution: checker commentedOK, the relation was not single-valued and this is not support i guess.