Currently getPropertyDefinitions() unsets fields that have custom storage set. Computed fields are treated as custom storage, since it calls hasCustomStorage() in ContentEntity::getPropertyDefinitions() (/docroot/modules/search_api/src/Plugin/search_api/datasource/ContentEntity.php):
// Exclude properties with custom storage, since we can't extract them
// currently, due to a shortcoming of Core's Typed Data API. See #2695527.
foreach ($properties as $key => $property) {
if ($property->getFieldStorageDefinition()->hasCustomStorage()) {
unset($properties[$key]);
}
}| Comment | File | Size | Author |
|---|---|---|---|
| #25 | Compatitible-for-Drupal-9_2814925_1.patch | 1.13 KB | anshu raj |
| #21 | 2814925-11--index_computed_properties.patch | 1.48 KB | pwaterz |
| #10 | 2814925-10--index_computed_properties.patch | 1.1 KB | drunken monkey |
Comments
Comment #2
kducharm commentedComment #3
kducharm commented8.x-1.0-beta2 patch version8.x-1.x version worked, this is unnecessaryComment #4
kducharm commentedComment #5
drunken monkeyThanks for reporting this issue!
This seems a bit of an unclean solution, though, accessing the definition directly. I'd just add
&& !$property->isComputed()to the condition instead – much cleaner using the interface provided.However, the larger problem is that, according to a quick test, computed fields cannot be retrieved automatically any more than fields with custom storage can. So showing these properties as options would be just as confusing and pointless as for normal custom storage properties.
Or what's your rationale there?
Comment #6
drunken monkeyComment #7
kducharm commentedMy use case for having Computed Fields available for search indexing is for adding data to the search index that I want to fill in during hook_entity_storage_load(), and want that data available from Views built on the Search API index.
For example, a Person entity (DB table) that doesn't contain in itself any data about what Shirt Size(s) they wear. A ShirtSize entity (DB table) has a unique ID, Shirt Size, and a Person ID - but I want the Person index to include their Shirt Size for creating facets. I can add a computed field of "Shirt Sizes" as an entity reference in the Person entity, and fill in a hook_entity_storage_load() that populates the $Person->setShirtSizes($shirt_sizes_ids). Then when I go to configure fields for Person index, I have full access to the Person Shirt Sizes entity all in one spot.
This comes about where you can't alter your DB table/entity to include a column with the entity reference already populated to the destination, and the need to build a View on a Search Index. On a regular view, I would just add to ViewsData to make a new field with the relationship but I can't do that with Search API (as far as I know) since it's not a DB query.
I'm open to other suggestions too, I know there are various ways to get data directly into the search index, but seem to lose the whole entity reference benefits in Views.
Comment #8
drunken monkeyIt's clear that being able to index computed fields (or other fields with custom storage) would be useful in a lot of cases, that's not the issue. The issue, as said, is that it's just not possible to do generically with the current Drupal Core Entity API (as far as I know). Unless that's possible, making the fields available to be added to the index won't get us anything (except end user confusion).
For your use case, you can create a custom processor that just adds your property to the datasource (see
\Drupal\search_api\Plugin\search_api\processor\AddURLfor an example). That custom property can also be an entity reference, that's no problem – and then you should be able to use it in Search API Views just like any other indexed field (at least once #2789431: Allow the display of processor-generated fields/properties in Views is fixed).If you have any general solution for this problem, please explain it.
Otherwise, I'd say we change the issue to a "Support request" and mark it "Fixed".
Comment #9
cyberwolf commentedI stumbled upon this issue while experimenting with computed fields. Let me describe my use case.
I want to expose the year of publication of nodes as facets on a search delivered by search_api, on an index in Elasticsearch with the elasticsearch_connector module, and (obviously) the facets module. I thought I could accomplish this by indexing the date and then there would be a convenient option in facets to do this, but there doesn't seem a way (yet) to make a facet from a part of the indexed date, or I just couldn't find it :)
So, being pretty new to D8 but quite experienced with D7, I remembered the possibility to add 'extra' fields without any storage requirements in D7, searched for something similar in Drupal 8, and ended up with the computed fields as described on https://www.drupal.org/node/2112677. Although the guidelines on that page did not seem to be entirely correct, with some trial and error I managed to create a created_year property on nodes (derived from the value of the 'created' property), which gets successfully indexed by search_api when the patch from this ticket is applied. The biggest pitfall for me was the example BixProfileCurrentCompany class on the documentation page. search_api's Utility\FieldsHelper::extractFieldValues() expects the $data to be 'iterable' (it loops over it with a foreach) when it's a list, but the example BixProfileCurrentCompany does not properly initialize a list on construct, although it extends FieldItemList, so eventually no values were extracted because there was nothing to iterate through.
The essential pieces to create the computed field for my use case follow below. They still need polishing, and they currently live in a custom module named 'ocmwle_common', please don't pay attention to that.
src/Plugin/YearCreated.php:
ocmwle_common.module:
I am attaching an updated patch for search_api, which follows the advice of drunken monkey in #5.
Regarding the last comments in #8, I am too new to D8 to judge wether it is possible or not to do this generically with the current Drupal Core Entity API. In my specific case, it eventually seemed to work. If you have particular cases you know that do not work correctly, please share and I might contribute some time to look at them.
Comment #10
drunken monkeyHuh, you're right, this does look like it should actually work. I had merely happened to test with the "Path" field (the only computed field in Core, as far as I can see), and that just seems to be not implemented properly (for this use). But it might actually work with most other computed fields – not really a way to be sure.
We should, however, exclude the "Path" field, so it doesn't cause confusion. I don't like to do this in general, but it's in Core after all, and otherwise we're bound to get some bug reports for that.
Also, I reformatted a bit to get rid of the overlong line.
Revised patch attached. Please review whether it works correctly for you and I can commit.
Comment #12
kducharm commentedI didn't actually use the patch #10 but pasted in the changes to my project, I can at least confirm this still allowed all of the computed fields I defined to show up as choices when adding fields to the search index. Would love to see it merged, good find.
Comment #13
drunken monkeyThanks for reporting back with this!
Anyone else want to test the patch?
Comment #14
borisson_I don't like that its two nested if statements but I understand you did this because of readability so RTBC imho.
Comment #16
drunken monkeyGood to hear, thanks for reviewing!
I'm not sure I like it, either, but I guess we can go with it. (Even though I wonder why I didn't have the two
ifs in the other order …)Committed.
Thanks again, everyone!
Comment #17
sukanya.ramakrishnan commentedI cant get this patch to work with views? I cannot add a computed field to a search view successfully. Does this work for anyone else?
Thanks
Sukanya
Comment #18
drunken monkeyThis should work, as far as I can see. Maybe the data type in question is just not supported?
Please create a new issue with your problem description and a bit more details.
(Would be good to hear from others here, though, whether this works for them. I don't have a computed field (other than "Path") available for testing.)
Comment #19
sukanya.ramakrishnan commentedThanks for the response.
This is the error I got
Notice: Undefined index: in Drupal\views\Plugin\views\field\Field->getFieldStorageDefinition() (line 332 of core/modules/views/src/Plugin/views/field/Field.php).
Drupal\views\Plugin\views\field\Field->getFieldStorageDefinition() (Line: 346)
Drupal\views\Plugin\views\field\Field->defineOptions() (Line: 99)
Drupal\search_api\Plugin\views\field\SearchApiEntityField->defineOptions() (Line: 17)
Drupal\views_field_metadata\Plugin\views\field\SearchApiEntityFieldWithMetadata->defineOptions() (Line: 138)
Drupal\views\Plugin\views\PluginBase->init(Object, Object, Array) (Line: 102)
Drupal\views\Plugin\views\HandlerBase->init(Object, Object, Array) (Line: 115)
Drupal\views\Plugin\views\field\FieldPluginBase->init(Object, Object, Array) (Line: 177)
when i check the Drupal\views\Plugin\views\field\Field->getFieldStorageDefinition() method, there is a specific condition to get the definitions for only non-computed fields which is why the view is failing,
Thanks,
Sukanya
Comment #21
pwaterz commentedI would like this issue to be reopened. The approach taken in patch 10 is making too big of assumptions about not being able to index fields that have custom_storage set to TRUE. Here is some example code that does NOT work with the current patch because the field can't be selected:
Execute this php script:
In some_module.module
I've attached my purposed solution as a patch. Now you can add 'some_field_with_custom_storage' field to the index, and if you index the nodes, it will index properly. Thoughts?
The issue really is in Drupal core in that you can create fields where you can not retrieve their values via typeddate api, but we can not rely on checking if something has custom storage. I think we need to just exclude known fields until core api allows us to effectively check if it can be indexed.
Perhaps I should open a new issue for this?
Comment #22
cyberwolf commented@pwaterz: The assumption that fields that have custom storage can not be indexed, was there already. I think it's better to open a new ticket to change something regarding the other assumption.
Comment #23
pwaterz commentedWill do
Comment #24
anshu raj commentedHere is the patch for Drupal 9
Comment #25
anshu raj commentedHere the patch for index_computed_properties