In the WissKI system our data backend has huge amounts of information and it is very costly to load the entity, so I would like to avoid it at all costs. However when an entity gets indexed and a particular field is not filled with data the index does not contain an empty array but it contains simply nothing. This leads to search api doing a full load of the entity because it can not detect the dependency as loaded - see:
/search_api/src/Plugin/views/field/SearchApiFieldTrait.php ~around Line 567
foreach ($dependents as $dependent) {
if (!isset($row->$dependent)) {
$required = TRUE;
...
Index some fields of an entity, fill the fields on some entities and on some not, index that and display that in a view.
Either we must store an empty array for an indexed value that was empty or we need to check for empty fields otherwise.
| Comment | File | Size | Author |
|---|---|---|---|
| #13 | Bildschirmfoto 2022-05-23 um 12.07.07.png | 98.67 KB | mkalkbrenner |
| #9 | 3179643-8--return_empty_result_fields--wip.patch | 2.73 KB | drunken monkey |
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
compare compare compare Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #3
drunken monkeyI assume you are using Solr?
Also, in your case, is
$row->$dependentpresent butNULL, or completely missing? I.e., what doesproperty_exists($row, $dependent)return?Comment #4
hongqing commentedI am using 8.x-1.20 and Solr. Empty field does not lead to full entity load with
Requiredchecked, but it does not show any hint.Comment #5
drunken monkey@ hongqing: Could you please elaborate on that? If you are describing a problem you have, it seems to be the exact opposite to the one in the IS, so not sure this is the right issue.
Comment #6
shubhamporwal14@gmail.com commentedHi @drunken monkey, I am also able to replicate the issue.
If the field indexed to solr is empty it will cause whole entity to load.
property_exists($row, $dependent); is returning "false". Please let me know if more information is needed.
Comment #7
drunken monkeyPlease do not raise the priority of an issue simply because it affects you. The meanings of the different priority levels are clearly defined and this is obviously a “Normal” issue, not a “Critical” one.
Anyways, thanks for the information. This is already good to know, but unfortunate. If that would have been
TRUE, we could simply have used that check instead ofisset().As things stand, however, there is little chance that we will be able to solve this problem just in this module. We’d at least need the backend plugin to explicitly set even the values of missing fields to an empty array when they are sure that the value is missing/empty. (However, I don’t think that Solr even supports storing explicit
NULLor empty values for document fields, so this would probably require some complex logic while extracting the result items inSearchApiSolrBackend::extractResults().)However, the first step would in any case be for us to at least support this behavior from the backend by also copying empty field values from the search result to the Views result row. The attached patch would implement that, please review.
Once the patch is committed, we can create a follow-up issue in the Solr module to see whether the maintainer sees a good way of supporting this functionality.
Also, once this is in place you could just implement a local workaround in a custom module, by altering the search results and manually adding all missing fields with empty values to the items. That should then finally let you avoid an entity load reliably.
Comment #9
drunken monkeyOK, committed. This should at least make it possible for other modules (either custom or contrib) to fix this problem (more easily).
Moving this to the Solr module. The goal here: If possible,
SearchApiSolrBackend::extractResults()should not just create fields for all values that are part of the Solr document, but also ones (without values) that have no values for this specific item.Attached is a patch starting on this, however, one crucial part is missing: making sure that the field in question was even part of
fl, to not report fields as missing which we didn’t even request from Solr.Comment #10
drunken monkeyComment #11
drunken monkeyComment #12
mkalkbrennerThere's already an option to "index empty fields". Doesn't that one help?
Comment #13
mkalkbrennerComment #14
ericgsmith commentedI'm having a different issue but it feels very related to this as the issue stems from the behaviour for empty fields.
In my case the field is a property added by a processor - I'm not sure if that fits in the definition here of a field.
In my case, I have "Retrieve result data from Solr" set to true and I am using the search api attachments module to index document contents. I am using that field in views.
When the field is empty I can see the same outcome as the issue summary when the field is pre rendered -
!isset($row->$dependent))is TRUE so the field becomes required. Rather than loading the entity, this now diverges from the issue summary and loads the processor as callsaddFieldValues.I have tried using the "Index empty full fields" option but when it comes to indexing it gets to the loop where values are added - but at this point
$valuesis an empty array so nothing is added to the index.Applying this patch seems to resolve the issue - the empty field is added to the result set and then when the check is done in the getValuesToExtract the value set is an empty array and no action is taken.
I will do a more thorough test & review in the next few days.
Comment #19
ericgsmith commentedAdded a check to cover the remaining check that the field was requested mentioned in #9
I haven't moved this to Github yet - I think getting buy in for the approach first since this has been open for a while.
While I think conceptually "Index empty fields" is probably a more logical thing to opt in to, this looked to have a few moving parts I didn't understanding while debugging.