Come together with the global Drupal community in Rotterdam, 28 Sept – 1 Oct 2026. Sessions, contribution, connection, and Early Bird savings until 8 June.Problem/Motivation
The new search_api 1.18 Module introduced a very critical bug. Any change on an indexed node will lead to a fatal error if entities are indexed directly after node save.
Steps to reproduce
The issue only appears, if you have an index with mixed entity types. In our case we have an entity type "entity:commerce_product" and "entity:node" indexed in the same index.
Now we have a paragraph field on the "entity:node" index, but this field is not present on the "entity:commerce_product" type.
The error arises in search_api/src/Plugin/search_api/datasource/ContentEntity.php
Line 1062
if (!$original_entity || !$items->equals($original_entity->get($relation_info['field_name']))) {
$query = $this->entityTypeManager->getStorage($this->getEntityTypeId())
->getQuery();
$query->accessCheck(FALSE);
The issue here is, that $this->getEntityTypeId() points to "entity:commerce_product" because it is the first datasource plugin. But on an index you can have multiple datasource plugins. The related field is only existing on the datasource plugin "entity:node" . This will result in a fatal error.
Proposed resolution
ContentEntity plugin should take into account that multiple different datasource plugins with different fields can exist
The current workaround is to revert to search_api 1.17 which works fine.
| Comment | File | Size | Author |
|---|---|---|---|
| #25 | 3178941-25--referenced_entities_reindexing_fatal_error.patch | 10.15 KB | drunken monkey |
Comments
Comment #2
ayalon commentedComment #3
ayalon commentedComment #4
Belialius commentedCan confirm that issue exists, have the same error on node save after the update to version 1.18.
Comment #5
kfritscheWe are running in the same issue. This happens for entity updates, delete and create actions.
This issue was introduced with 1.18 in #2007692: Changes in related entities don't lead to proper re-indexing
Comment #6
paulmckibbenAlso seeing this issue on updating content with paragraphs. I can confirm that reverting search_api to version 1.17 resolves the issue.
Comment #7
ooziedie commentedI have the same issue as paulmckibben. Updating a node with paragraphs results in a fatal error. After downgrading to 1.17 everything works as expected.
Comment #8
drunken monkeySorry for not responding sooner. I have a lot of issues to look at, so unfortunately it sometimes takes longer. Thanks to mkalkbrenner for pointing this out to me! Fatal errors that occur for multiple people (especially in common scenarios like “saving a node”) are good candidates for pinging me on Slack, or even via mail, directly. (Just for the future.)
Anyways, unfortunately the issue description isn’t very helpful at the moment. I tried reproducing the problem, but it works fine for me. The “steps to reproduce” aren’t very clear, or detailed, though, so probably I just missed something.
Could someone try to come up with a minimal set of steps to reproduce, preferrably without unrelated contrib modules? Please also clearly state what action then leads to the fatal error. A stack trace would also be very helpful.
Furthermore, if you can get it, it would be very handy to know the values of
$relation_info,$foreign_entity_relationship_map,$entity(at least the type) and$this->getPluginId().A test reproducing the problem would, of course, be even more helpful – we’ll need one anyways, before committing, to ensure this doesn’t happen again in the future.
Finally, I hope at least some of you have also already tried the latest dev version?
Comment #9
drunken monkeyTried reproducing by just adding the
entity:userdatasource toReferencedEntitiesReindexingTest, but no luck.However, I do see a problem in the reindexing logic, just by looking at the code. The foreign relationships map completely ignores the datasource plugin of a relationship. Maybe that is the problem? Please try out the attached patch (Edit: and make sure to clear the cache after applying it). (Answers to my questions in #8 would still be helpful, though.)
If we’d commit something like the attached patch, we should probably also create a change record, as the structure in the
mapping_foreign_relationshipsevent would (slightly) change.Comment #10
drunken monkeyStill need more info, just wanted to trigger the test bot.
Comment #11
cspitzlayI tested the patch on top of search_api 1.18 with our setup and data.
It does seem to solve the issue.
No idea how to get to a minimal test case, yet.
Note to others testing the patch: Make sure you run drush cr after applying it.
Comment #12
cspitzlayFWIW, here's a backtrace of the crash that happens with stock 1.18 when I delete one of our entities:
drush ev '\Drupal\my_metadata\Entity\MyMetadata::load(2405)->delete();'Comment #13
drunken monkeyThanks a lot, cspitzlay! The stack trace already helps a lot! (Also very good to hear that my patch attempt worked for you.)
This makes it clear, though, that there is definitely also a Core bug involved. There should never just be a fatal error – that’s what exceptions are for. (And we make sure to catch all possible exceptions for that method call, specifically to make sure we don’t break saving entities.) After a quick search, I found the Core issue: #2893747: Unhelpful error message when an entity query contains an invalid field name. So definitely follow that, too.
Anyways, not running pointless entity queries (and breaking sites until the Core bug is fixed) is of course a good idea either way. So as soon as we can get a regression test for this (and maybe a bit more feedback about the fix), I can commit this.
For the test, it might already be sufficient to slightly alter the setup of
ReferencedEntitiesReindexingTest, or maybe add another entity save operation.Comment #14
drunken monkeyComment #15
diegopino commentedHi, we just discovered the same issue and was about to test the patch and I hit a different well know old issue.
ContentEntityTrackingManager::__construct() is expecting Drupal\Core\Entity\EntityTypeManager when it should be expecting an Interface. This breaks, e.g, web profiler... Should I open a different ISSUE for that, maybe it can be fixed In the same Patch? I feel it is a blocker.
Here
https://git.drupalcode.org/project/search_api/-/blob/8.x-1.18/src/Plugin...
Comment #16
ooziedie commented@drunken monkey FYI: the backtrace for saving a node with paragraphs with vanilla 1.18.0:
After installing the patch, saving a node with paragraphs works again.
Comment #17
cspitzlayI didn't succeed to reproduce the issue with variations of ReferencedEntitiesReindexingTest.
So I decided to provide the debug dumps of the variable values in getAffectedItemsForEntityChange() just before things go awry.
$foreign_entity_relationship_map:
$relation_info
$this->getPluginId()
entity:kmm_endpoint$this->getEntityTypeId()
kmm_endpointHere is the base field definition on as_metadata (the prefix 'field_' is there for historical reasons, it is indeed a base field). There is no such field on kmm_endpoint though.
Please note that things go wrong the second time the getAffectedItemsForEntityChange() is called.
The first time the plugin ID and entity type ID are entity:as_metadata and as_metadata, respectively.
Comment #18
drunken monkey@ DiegoPino: That’s unrelated, so please create a new issue for it. (Unless there is one already.)
@ cspitzlay: Thanks a lot again for this important information! I now could confirm in manual testing that the key to triggering this bug is to have the trailing
:entityincluded in the'property_path_to_foreign_entity'key. I’m not sure why it’s there in some cases, but not in others, but by just adding that to the call I could now also reproduce this error – and was therefore able to write an automated test for it (see attached patch revision).Also, I remembered that we are now free to use PHP 7, so it’s also actually possible to catch the fatal error even in this case. I’m not certain whether we should, as the problem should already be resolved by the rest of the changes, but it’s there as an option at least. (The danger is of masking other, actual fatal errors that maybe then get harder to debug, or could even cause data inconsistencies in some cases, I guess.)
I’d like some input/opinions from developers on that one, if possible.
Finally, I also created the change record – reviews are welcome for that, too.
Comment #20
megachrizMaybe it would be good to document why any errors are catched, but not logged? Now it looks like that the code is fine with any errors that pop up during the process.
Something like: "We are not interested in any SQL related errors."? (Though that still doesn't explain why SQL errors are ignored.)
I'm trying to understand the implications of ignoring errors in this case. I think the consequence in this case is that entities may not be reindexed if a referenced entity changes. So when changing a term's name causes a SQL error when Search API tries to find entities in which that term is used, that error will be hidden. People may then notice that the old term name is used in the search results and would probably have a hard time figuring out how that could happen. So perhaps exceptions need to be logged. However, there can exist exceptions that are not necessarily errors (for example: MigrateSkipProcessException), so not sure how to properly filter these out as you cannot know all exception classes that exist in contrib modules.
For the tests only patch I checked if the error message showed up when commenting out
'datasource' => 'entity:node',and it did! So that's good.I installed the patch on a site where I got the error on and I can confirm that the patch fixes the issue.
Comment #21
bakulahluwaliaHi all,
I am facing the same error but with a different scenario.
Problem
Search api attachment configuration page throwing the WSOD when I click "Submit and test extraction" on "http://example.com/admin/config/search/search_api_attachments"
logs
Error: Call to a member function getColumns() on bool in Drupal\Core\Entity\Query\Sql\Tables->addField() (line 253 of /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/Query/Sql/Tables.php)
#0 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/Query/Sql/Condition.php(52): Drupal\Core\Entity\Query\Sql\Tables->addField('field_document....', 'INNER', NULL)
#1 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/Query/Sql/Query.php(172): Drupal\Core\Entity\Query\Sql\Condition->compile(Object(Drupal\Core\Database\Driver\mysql\Select))
#2 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/Query/Sql/Query.php(80): Drupal\Core\Entity\Query\Sql\Query->compile()
#3 /mnt/www/html/sitename/docroot/modules/contrib/search_api/src/Plugin/search_api/datasource/ContentEntity.php(1074): Drupal\Core\Entity\Query\Sql\Query->execute()
#4 /mnt/www/html/sitename/docroot/modules/contrib/search_api/src/Utility/TrackingHelper.php(123): Drupal\search_api\Plugin\search_api\datasource\ContentEntity->getAffectedItemsForEntityChange(Object(Drupal\file\Entity\File), Array, NULL)
#5 /mnt/www/html/sitename/docroot/modules/contrib/search_api/search_api.module(255): Drupal\search_api\Utility\TrackingHelper->trackReferencedEntityUpdate(Object(Drupal\file\Entity\File), true)
#6 [internal function]: search_api_entity_delete(Object(Drupal\file\Entity\File))
#7 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Extension/ModuleHandler.php(403): call_user_func_array('search_api_enti...', Array)
#8 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/EntityStorageBase.php(206): Drupal\Core\Extension\ModuleHandler->invokeAll('entity_delete', Array)
#9 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/ContentEntityStorageBase.php(843): Drupal\Core\Entity\EntityStorageBase->invokeHook('delete', Object(Drupal\file\Entity\File))
#10 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/EntityStorageBase.php(434): Drupal\Core\Entity\ContentEntityStorageBase->invokeHook('delete', Object(Drupal\file\Entity\File))
#11 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php(786): Drupal\Core\Entity\EntityStorageBase->delete(Array)
#12 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Entity/EntityBase.php(403): Drupal\Core\Entity\Sql\SqlContentEntityStorage->delete(Array)
#13 /mnt/www/html/sitename/docroot/modules/contrib/search_api_attachments/src/Form/TextExtractorFormSettings.php(160): Drupal\Core\Entity\EntityBase->delete()
#14 [internal function]: Drupal\search_api_attachments\Form\TextExtractorFormSettings->submitForm(Array, Object(Drupal\Core\Form\FormState))
#15 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Form/FormSubmitter.php(114): call_user_func_array(Array, Array)
#16 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Form/FormSubmitter.php(52): Drupal\Core\Form\FormSubmitter->executeSubmitHandlers(Array, Object(Drupal\Core\Form\FormState))
#17 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Form/FormBuilder.php(593): Drupal\Core\Form\FormSubmitter->doSubmitForm(Array, Object(Drupal\Core\Form\FormState))
#18 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Form/FormBuilder.php(321): Drupal\Core\Form\FormBuilder->processForm('search_api_atta...', Array, Object(Drupal\Core\Form\FormState))
#19 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Controller/FormController.php(91): Drupal\Core\Form\FormBuilder->buildForm(Object(Drupal\search_api_attachments\Form\TextExtractorFormSettings), Object(Drupal\Core\Form\FormState))
#20 [internal function]: Drupal\Core\Controller\FormController->getContentResult(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\RouteMatch))
#21 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)
#22 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/Render/Renderer.php(573): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#23 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#24 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)
#25 /mnt/www/html/sitename/vendor/symfony/http-kernel/HttpKernel.php(151): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}()
#26 /mnt/www/html/sitename/vendor/symfony/http-kernel/HttpKernel.php(68): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#27 /mnt/www/html/sitename/docroot/modules/contrib/simple_oauth/src/HttpMiddleware/BasicAuthSwap.php(67): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#28 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Drupal\simple_oauth\HttpMiddleware\BasicAuthSwap->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#29 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#30 /mnt/www/html/sitename/docroot/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#31 /mnt/www/html/sitename/docroot/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#32 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#33 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#34 /mnt/www/html/sitename/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#35 /mnt/www/html/sitename/docroot/core/lib/Drupal/Core/DrupalKernel.php(708): Stack\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#36 /mnt/www/html/sitename/docroot/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#37 {main}
Is there any patch available that is working for search api attachment?
Comment #22
cspitzlay@bakulahluwalia: Have you tried it with the patch from #18 and it did not work?
@drunken_monkey: As a developer I would prefer to let it crash in case of a fatal error instead of it going unnoticed or harder to debug.
Unless the reasons for "saving of entities on production must never be broken by indexing" are paramount (and trump index correctness on production) ...
Comment #23
bakulahluwalia@cspitzlay, patch #18 worked for me.
Thanks @drunken_monkey for the patch.
Comment #24
Alice Heaton commentedSame issue, I can confirm patch on #18 fixes it. No side effects that I have seen so far.
Comment #25
drunken monkey@ MegaChriz: Thanks a lot for the detailed review! I fixed all your notes – including adding logging for the caught exceptions. You’re right, doesn’t make sense to keep this quiet. Admins should at least be notified that something went wrong and that the index might be out of sync. (I guess I just omitted that initially because this was the default behavior before. But, of course, going forward it would be awesome if people could rely on this functionality.)
I also added logic to re-throw any
\Throwables we don’t actually want to catch – i.e., just catch normal exceptions and this particular type of runtime error, and re-throw everything else. I think this should be a good balance, making it almost impossible that this hides any unrelated problems by accident.@ cspitzlay: I can understand your reasoning (being a developer myself, I’d probably prefer it this way, too), but I do think most users would rather not have their website crash when saving an entity, at the price of a maybe slightly out-of-sync index (which they will now be notified about, and should be relatively easy to fix).
@ everyone: Thanks a lot for testing! Please also make sure that this new revision still works for you, then I think I’ll soon be able to commit it.
Comment #26
Anonymous (not verified) commentedHello, I have encountered the same issue and the patches from #18 and #25 both solve the issue for me. I would recommend using the patch from #25.
Comment #27
mjk73 commentedWe ran into this issue after updating to 8.9.10.
The patch in #25 resolved it. Thanks @drunken-monkey. :)
Comment #29
drunken monkeyOK, great to hear, thanks a lot for testing.
Committed.
Thanks again, everyone!
Comment #31
renrhafHi ! Thanks a lot for this fix.
This patch cause an other issue in my case : the "logException" method expects an Exception object but receive now an Error object.
Maybe we should switch that to a Throwable entity.
Applying core patch https://www.drupal.org/project/drupal/issues/2893747#comment-13886822 fixed the issue for now.