Problem/Motivation

The search_api_task entity's schema is manually declared in search_api_schema(). Under normal conditions, this is not a problem since that schema matches the one defined in \Drupal\search_api\Entity\Task::baseFieldDefinitions(). However, when a module (in this case, Multiversion) wants to alter those fields, and is part of an install profile, due to timing issues, the manual declaration creates the table too soon, and when Multiversion comes along to alter the fields, it doesn't because the table already exists (or is not under the full control of the entity API).

Proposed resolution

Remove the manual schema definition and let the Entity system handle it.

Remaining tasks

Review the patch.

User interface changes

API changes

Data model changes

Initial report

I have a site with an Search API index, and I am unable to install the Deploy module suite along with Search API the usual way (by adding them all to core.extensions.yml).

A few pieces of info:
1) I am using Config Installer to install a custom profile
2) Everything works fine if I install Deploy suite in the profile install hook (so after the core.extensions.yml modules are installed).

Here's the stack trace:

#0 /home/vagrant/docroot/web/core/lib/Drupal/Core/Database/Statement.php(59): PDOStatement->execute(Array)
#1 /home/vagrant/docroot/web/core/lib/Drupal/Core/Database/Connection.php(610): Drupal\Core\Database\Statement->execute(Array, Array)
#2 /home/vagrant/docroot/web/core/lib/Drupal/Core/Database/Driver/mysql/Connection.php(81): Drupal\Core\Database\Connection->query('SELECT COUNT(*)...', Array, Array)
#3 /home/vagrant/docroot/web/core/lib/Drupal/Core/Database/Query/Select.php(493): Drupal\Core\Database\Driver\mysql\Connection->query('SELECT COUNT(*)...', Array, Array)
#4 /home/vagrant/docroot/web/core/lib/Drupal/Core/Entity/Query/Sql/Query.php(252): Drupal\Core\Database\Query\Select->execute()
#5 /home/vagrant/docroot/web/core/lib/Drupal/Core/Entity/Query/Sql/Query.php(84): Drupal\Core\Entity\Query\Sql\Query->result()
#6 /home/vagrant/docroot/web/modules/contrib/search_api/src/Task/TaskManager.php(110): Drupal\Core\Entity\Query\Sql\Query->execute()
#7 /home/vagrant/docroot/web/modules/contrib/search_api/src/Task/IndexTaskManager.php(190): Drupal\search_api\Task\TaskManager->getTasksCount(Array)
#8 /home/vagrant/docroot/web/modules/contrib/search_api/src/Utility.php(454): Drupal\search_api\Task\IndexTaskManager->isTrackingComplete(Object(Drupal\search_api\Entity\Index))
#9 [internal function]: Drupal\search_api\Utility::processIndexTasks(Array, Object(Drupal\Core\Config\ConfigImporter))
#10 /home/vagrant/docroot/web/core/lib/Drupal/Core/Config/ConfigImporter.php(492): call_user_func_array(Array, Array)
#11 /home/vagrant/docroot/web/core/lib/Drupal/Core/Config/ConfigImporter.php(464): Drupal\Core\Config\ConfigImporter->doSyncStep(Array, Array)
#12 /home/vagrant/docroot/web/profiles/contrib/config_installer/config_installer.profile(263): Drupal\Core\Config\ConfigImporter->import()
#13 /home/vagrant/docroot/web/core/includes/install.core.inc(652): config_installer_fix_profile(Array)
#14 /home/vagrant/docroot/web/core/includes/install.core.inc(530): install_run_task(Array, Array)
#15 /home/vagrant/docroot/web/core/includes/install.core.inc(115): install_run_tasks(Array)
#16 /home/vagrant/docroot/vendor/drush/drush/includes/drush.inc(726): install_drupal(Object(Composer\Autoload\ClassLoader), Array)
#17 /home/vagrant/docroot/vendor/drush/drush/includes/drush.inc(711): drush_call_user_func_array('install_drupal', Array)
#18 /home/vagrant/docroot/vendor/drush/drush/commands/core/drupal/site_install.inc(80): drush_op('install_drupal', Object(Composer\Autoload\ClassLoader), Array)
#19 /home/vagrant/docroot/vendor/drush/drush/commands/core/site_install.drush.inc(247): drush_core_site_install_version('config_installe...', Array)
#20 /home/vagrant/docroot/vendor/drush/drush/includes/command.inc(366): drush_core_site_install('config_installe...')
#21 /home/vagrant/docroot/vendor/drush/drush/includes/command.inc(217): _drush_invoke_hooks(Array, Array)
#22 /home/vagrant/docroot/vendor/drush/drush/includes/command.inc(185): drush_command('config_installe...')
#23 /home/vagrant/docroot/vendor/drush/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#24 /home/vagrant/docroot/vendor/drush/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#25 /home/vagrant/docroot/vendor/drush/drush/drush.php(12): drush_main()

The problem seems to be that Search API tasks have acquired the ContentEntityStorageTrait but do not yet have the correct columns.

I am filing this in the Multiversion queue because it seems like this could affect other content entity modules.

CommentFileSizeAuthor
#10 2764947-10.patch13.5 KBjhedstrom
#10 interdiff.txt11.92 KBjhedstrom
#4 2764947-04.patch1.58 KBjhedstrom
#3 NO-TEST.diff524 bytesjhedstrom

Comments

dpolant created an issue. See original summary.

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented

When I install manually (eg, not part of an install profile), these fields are created. However, if I install as part of an install profile, they are not (although they are created for nodes, taxonomies, etc).

The search_api_task_revision table is properly created either way.

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented
StatusFileSize
new NO-TEST.diff524 bytes

This simple change to standard.info.yml is enough. It triggers no errors on a vanilla install, but if one inspects the db table after install, search_api_task is missing _deleted and _rev fields.

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented
Project: Multiversion » Search API
Component: Code » General code
StatusFileSize
new 2764947-04.patch1.58 KB

I found the issue.

The search_api_schema() function declares the table that the entity system attempts to create automatically. Simply removing this declaration allows multiversion to properly add the fields.

Here's a patch against search_api (and I'm moving this issue).

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented
Title: Incompatibility with Search API config: Column not found: 1054 Unknown column 'search_api_task._deleted' ... » Incompatibility with Multiversion
Status: Active » Needs review
dpolant’s picture

dpolant commented

Thanks for the quick followup. I will try this patch.

Status: Needs review » Needs work

The last submitted patch, 4: 2764947-04.patch, failed testing.

dpolant’s picture

dpolant commented

Haven't tried the patch yet but I am curious about something. I was able to sidestep this issue by setting multiversion=FALSE in the search_api_task entity type.

Is there a reason we would ever want multiversion enabled for these entities?

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented

Is there a reason we would ever want multiversion enabled for these entities?

Indeed, since multiversion is at the base of the Deploy stack, having all entities enabled for multiversion is what allows them to be deployed across environments. There are probably other reasons too. However, for 99% of sites at this time, the setting mentioned above is probably a valid workaround for now.

The test failures are happening because some kernel tests are still trying to manually install the schema, whereas with this change they'll need to install the entity schema.

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented
Status: Needs work » Needs review
StatusFileSize
new interdiff.txt11.92 KB
new 2764947-10.patch13.5 KB

This should fix the tests.

jhedstrom’s picture

jhedstrom
Primary language English
Location Portland, OR
commented
Title: Incompatibility with Multiversion » Incompatibility with Multiversion due to manually declared entity schema
Issue summary: View changes

I've updated the issue summary to better explain the root cause.

dpolant’s picture

dpolant commented
Status: Needs review » Reviewed & tested by the community

The patch seems to solve the issue. Thanks!

drunken monkey’s picture

drunken monkey
he/him
Primary language German
Location Vienna, Austria
commented
Component: General code » Framework
Status: Reviewed & tested by the community » Fixed

Thanks a lot for debugging and fixing this! Looks great, and tests check out, too. Also, very good job on the issue summary!
Committed.
Thanks again!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

d0t15t’s picture

d0t15t commented

Is this commited in the dev or in the alpha branch?

borisson_’s picture

borisson_
Primary language Dutch
Location Mechelen, 🇧🇪
commented

@dotist: this is not yet included in the latest alpha release.