Problem/Motivation

All entity types having the content_translation_uid field end up having an index name that is too long to be kept intact so it's hashed. The code computing entity schema data relies on the index name to detect field-level indexes/keys (which is very fragile), so it does not recognise hashed-named-indexes as field-level ones and thus a mismatch is reported, as the field is interpreted as a new entity-level index.

Proposed resolution

Fix comparison.

Remaining tasks

  • Commit

User interface changes

The "Entity/field definition" status report item is always displayed now.

API changes

none

Data model changes

none

Original summary

I was able to enable translation for "custom menu link" under Admin => Configuration => Region and Language => Content language and translation.

However, after that, page "admin/reports/status" give an error:

Entity/field definitions: Mismatch detected. Mismatched entity and/or field definitions.

I think this is related to translation, so put it under translation_entity.module component.

Please help.

Thanks.

screenshot 1

screenshot 2

screenshot 3

CommentFileSizeAuthor
#87 entity-schema_data_fix-2597628-87.patch11.73 KBplach
#87 entity-schema_data_fix-2597628-87.interdiff.txt2.88 KBplach
#81 entity-schema_data_fix-2597628-81.patch13.78 KBplach
#81 entity-schema_data_fix-2597628-81.interdiff.txt3.51 KBplach
#70 test.php_.txt332 byteseffulgentsia
#66 Status_report___Drupal_8_-_Dev.png236.38 KBplach
#61 entity-schema_data_fix-2597628-46_0.patch11.56 KBgábor hojtsy
#60 56-60-interdiff.txt5.58 KBalexpott
#60 2597628-2-60.patch7.75 KBalexpott
#56 entity-schema_data_fix-2597628-46.patch11.56 KBplach
#47 entity-schema_data_fix-2597628-46.test.patch3.42 KBplach
#47 entity-schema_data_fix-2597628-46.patch11.56 KBplach
#44 2597628-44.patch9.07 KBalexpott
#44 42-44-interdiff.txt1.89 KBalexpott
#42 39-42-interdiff.txt3.34 KBalexpott
#42 2597628-42.patch9.06 KBalexpott
#39 2597628-38.patch6.38 KBalexpott
#39 2597628.38.test-only.patch3.81 KBalexpott
#35 entity-schema_data_fix-2597628-35.patch4.39 KBplach
#35 entity-schema_data_fix-2597628-35.interdiff.txt950 bytesplach
#30 entity-schema_data_fix-2597628-30.test.patch1.82 KBplach
#30 entity-schema_data_fix-2597628-30.patch4.46 KBplach
#24 2597628-24.patch5.75 KBjhedstrom
#24 2597628-24-TEST-ONLY.patch1.67 KBjhedstrom
#24 interdiff.txt1.67 KBjhedstrom
#23 2597628-23.patch4.08 KBjhedstrom
#23 interdiff.txt4.13 KBjhedstrom
#19 2597628-19.patch520 bytesjhedstrom
#19 interdiff.txt742 bytesjhedstrom
#10 2597628-10.patch2.17 KBjhedstrom
#10 2597628-10-TEST-ONLY.patch1.49 KBjhedstrom
#9 2597628-09.patch695 bytesjhedstrom
img3.PNG44.88 KBhguo
img2.PNG18.75 KBhguo
img1.PNG25.16 KBhguo

Comments

hguo created an issue. See original summary.

dpacassi’s picture

I have a similar bug when working with translatable taxonomy vocabularies.

I did the following:
1) Install a fresh copy of Drupal 8.0.0-rc1
2) Install all core modules under "Multilingual" and it's required modules
3) Install the "Taxonomy" module
4) Add a new taxonomy vocabulary with the "Show language selector on create and edit pages"-option checked
-> No problems so far!
5) Add another taxonomy vocabulary with the options "Show language selector on create and edit pages" AND "Enable translation" checked
-> Now I get following error message in the status report:
Entity/field definitions, Mismatch detected
Mismatched entity and/or field definitions.

If I remove all taxonomy vocabularies, the error message stays in the status report.
The only way to get rid of it, is by uninstalling the taxonomy module.

Currently running Drupal 8 on PHP 5.5.26 and MySQL 5.5.42.
Let me know if you need any further information!
I would be happy to help you out!

hguo’s picture

@dpacassi

Thanks for providing the details of your case. What you described matches exactly what I have been experiencing.

I'm most concerned whether this would any performance implication. Can you shed some light on that?

Thanks!

gdev2018’s picture

Priority: Normal » Critical
gdev2018’s picture

I have the same error occurred when updating the version beta10 to RC2. Thus I have a pure core without some additional modules and content.
Sorry to change the priority to critical, I also every day to track their number and really looking forward to when they will not be at all.

catch’s picture

Title: "Entity/field definitions" -- "Mismatch detected" on reports/status page » Enabling translation for menu links / taxonomy terms results in "Entity/field definitions" -- "Mismatch detected" on reports/status page
Version: 8.0.0-rc1 » 8.0.x-dev
Component: translation_entity.module » entity system

Moving to entity system.

jhedstrom’s picture

Running entity schema updates resolves the issue, but there is no longer any way to do that through the UI IIRC.

Drush provides drush entity-updates or drush updb --entity-updates.

jhedstrom’s picture

I started looking at writing a test, but found that some of our tests already manually apply entity schema updates after enabling translations. From ContentTranslationOperationsTest::setUp:

    // Enable translation for the current entity type and ensure the change is
    // picked up.
    \Drupal::service('content_translation.manager')->setEnabled('node', 'article', TRUE);
    drupal_static_reset();
    \Drupal::entityManager()->clearCachedDefinitions();
    \Drupal::service('router.builder')->rebuild();
    \Drupal::service('entity.definition_update_manager')->applyUpdates();

So perhaps the fix is to run entity schema updates after enabling translations (then remove those manual runnings from the tests)?

jhedstrom’s picture

Status: Active » Needs review
StatusFileSize
new695 bytes

This fixes the issue in manual testing, and will most likely remove the need for content translation tests to manually run entity schema updates.

jhedstrom’s picture

StatusFileSize
new1.49 KB
new2.17 KB

Let's see if this fails/passes as expected.

The last submitted patch, 9: 2597628-09.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 10: 2597628-10.patch, failed testing.

jhedstrom’s picture

Issue tags: +Needs tests

Well that was the opposite of expected. So we'll probably need an additional test here.

effulgentsia’s picture

Let's create a separate issue to get #10's test-only patch committed, unless there's some other need for those lines. But I suspect those lines are from before we had ContentTranslationManager::setEnabled() calling ContentTranslationUpdatesManager::updateDefinitions() calling EntityDefinitionUpdateManager::installFieldStorageDefinition().

Come to think of it, perhaps that issue could explore whether similar lines can be removed from other tests.

jhedstrom’s picture

jhedstrom’s picture

The failures here are because we can't simply call updateEntityType, as this may be an install or delete operation too. I don't immediately see any public methods that would provide insight into what operation is going on (the protected getChangeList() has the needed information).

gábor hojtsy’s picture

Issue tags: +D8MI, +language-content, +sprint

How fast are those updates? We used to have schema updates ran after translation config changes but that required a batch to execute. We eliminated that due to underlying changes making that possible (not sure of the technicalities though :/).

hguo’s picture

Just tried rc2, this issue still exists as expected.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new742 bytes
new520 bytes

How fast are those updates?

Before we removed the automatic running of these during db updates, they weren't in a batch operation. I think since they explicitly avoid *modifying* existing column data, they are either just adding new fields or removing old ones.

One thought is to just have the update manager call applyUpdates() since that contains the necessary logic for determining if an entity type is being installed, updated, or removed. Attached is a patch with that approach (which also removes the test only patch from #10 which is in a separate issue now).

Status: Needs review » Needs work

The last submitted patch, 19: 2597628-19.patch, failed testing.

dawehner’s picture

In order to make proper progress on this issue I would suggest to put https://www.drupal.org/node/2597628#comment-10474214 into a test.

catch’s picture

I

+++ b/core/modules/content_translation/src/ContentTranslationUpdatesManager.php
@@ -66,6 +66,7 @@ public function updateDefinitions(array $entity_types) {
+      $this->updateManager->applyUpdates();

Applying all updates at once means it'll be easy to run into a race condition with definition/schema updates elsewhere. Although thinking about it, that's also possible just with getting the previous state too - we don't know what some other process is doing / is about to do with that when we get it.

Feels like we can rely on this to handle state A to state B, but only if we're able to add locking around the operation.

Something like:

- acquire lock
- update definitions
- apply updates
- release lock

Also adding some related issues.

jhedstrom’s picture

Status: Needs work » Needs review
StatusFileSize
new4.13 KB
new4.08 KB

This adds locking. The 2 failing tests were due to the 'text' module not being enabled, even though it is a declared dependency of entity_test.

jhedstrom’s picture

StatusFileSize
new1.67 KB
new1.67 KB
new5.75 KB

This adds a test that mimics the underlying steps in #2. Note, the first call to applyUpdates is simpler than manually installing all entity types created by a basic kernel test.

webchick’s picture

Issue summary: View changes

The core committers met and discussed this and other critical issues today in relation to picking a D8 release date.

This definitely feels like the "nuclear critical" one. It's both actively introducing data integrity issues to new D8 sites, and extremely tricky to write an upgrade path for. Unlike the rest of the critical issues in the queue, we also don't really have a viable "plan B" if this isn't resolved by D8.0.0's release date.

Some "plan B" suggestions we discussed but weren't thrilled with were:

- Move Content Translation to a hidden module so it can't be enabled at all from the UI. Obviously, totally un-ideal from the standpoint of selling D8's multilingual capabilities.
- Disable the checkboxes in the UI for any entity type that has content created (e.g. menu links). This would mirror what Field UI does, but catch wasn't 100% convinced that this would solve the problem.

So for this issue, what we should prioritize is: 1) test coverage to show the problem (are we not testing the UI at all or something?!) 2) a "plan b" for what we do if this issue isn't resolved in the next couple of weeks. We also discussed splitting off the upgrade path to a separate issue so we can at least stop new sites from getting this problem before release, even if we can't fix existing sites until 8.0.2 or whatever.

Added a heading for "plan B" and put @todo for now.

The last submitted patch, 24: 2597628-24-TEST-ONLY.patch, failed testing.

catch’s picture

I've opened #2602996: Upgrade path for #2597628 for figuring out the upgrade path for affected sites, we should try to keep any discussion of that over there so this issue can focus on preventing sites getting into that state in the first place.

jhedstrom’s picture

I found another use of ::applyUpdates() in the content translation tests. I think this is why the test-only patch in #10 passed unexpectedly. It's in ContentTranslationTestBase::enableTranslation(), called from ::setUp(). I'm testing now to see if it can be added to #2600138: Remove manual calls to EntityDefinitionUpdateManager::applyUpdates() from test setup, or if it requires the fix in this issue in order to be removed.

plach’s picture

Assigned: Unassigned » plach

Working on a test implementing #2.

plach’s picture

Assigned: plach » Unassigned
StatusFileSize
new4.46 KB
new1.82 KB

This should fix the issue without requiring any update. The patch is quite different from the earlier ones so no interdiff (the test patch is only slightly changed, though). Even if this were red/green as expected, some additional test coverage at API level would be good to have before closing this. I'll work on that tomorrow.

plach’s picture

Issue tags: +D8 Accelerate
effulgentsia’s picture

Some "plan B" suggestions we discussed but weren't thrilled with were:

I took a closer look at this, and the problem only seems to occur with menu links and taxonomy terms, per the issue title. The problem does not occur with nodes. I didn't check the other entity types.

The reason it doesn't occur with nodes is that nodes have their own author field, so ContentTranslationHandler::getFieldDefinitions() doesn't add the 'content_translation_uid' field. This is (one of? the only?) field that creates the problem, because it comes with an index.

So what happens is that ContentTranslationUpdatesManager::updateDefinitions() successfully installs the various new fields provided by ContentTranslationHandler::getFieldDefinitions(), including adding their indexes to the db table, but because it doesn't invoke updateEntityType() per #9, that new index never gets registered with the $entity_type.entity_schema_data KV entry in the entity.storage_schema.sql collection. That is what creates the mismatch.

So if we need to resort to a "plan B", then given the above, it could potentially be to only disable translations for the vulnerable entity types, until a future patch can provide the complete fix.

However, note that the entity database tables are actually in the correct state (whether the translation is enabled with or without existing entities already in the db), just that KV entry is out of date. So a complete fix might not require very much more than #9. Related to that:

The failures here are because we can't simply call updateEntityType, as this may be an install or delete operation too. I don't immediately see any public methods that would provide insight into what operation is going on.

I think those failures in #10 can be fixed. We should take a closer look at what those install/delete conditions are. Perhaps $entity_manager->getLastInstalledDefinition() in addition to the $installed_storage_definitions we're already working with there is sufficient for the code to figure out what to do?

Applying all updates at once means it'll be easy to run into a race condition with definition/schema updates elsewhere.

I agree that applyUpdates() is too broad and that we should go back to something more like #9, assuming we can fix the above referenced failures. We could even be more conservative by doing updateEntityType($entity_manager->getLastInstalledDefinition($entity_type->id())), since we're not reacting to an entity type definition change: the only "update" we need is to get the schema manager to sync up its schema information with the schema that's already installed for the currently installed entity type definition.

A lock might still be a good idea. Though if it is, we should lock the entire ContentTranslationUpdatesManager::updateDefinitions() function, not only whatever new code needs to be added for this issue.

effulgentsia’s picture

#32 was an xpost with #30. If #30 fixes the problem of the KV mismatch without requiring a separate updateEntityType() call, that's even better.

plach’s picture

So what happens is that ContentTranslationUpdatesManager::updateDefinitions() successfully installs the various new fields provided by ContentTranslationHandler::getFieldDefinitions(), including adding their indexes to the db table, but because it doesn't invoke updateEntityType() per #9, that new index never gets registered with the $entity_type.entity_schema_data KV entry in the entity.storage_schema.sql collection. That is what creates the mismatch.

Actually what I found is that all entity types having the content_translation_uid end up having an index name that is too long to be kept intact so it's hashed. The code computing entity schema data relies on the index name to detect field-level indexes/keys (which is very fragile), so it does not recognize hashed-named-indexes as field-level ones and thus a mismatch is reported, as the field is interpreted as a new entity-level index.

Field schema data is correctly registered in the KV store, it's just not used correctly it's just unrelated to the issue.

plach’s picture

StatusFileSize
new950 bytes
new4.39 KB
+++ b/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php
@@ -591,22 +591,36 @@ protected function getEntitySchemaTables() {
+          $data = array_diff_key($table_schema[$key], $field_schema_identifiers);
+          $table_schema[$key] = array_intersect_key($table_schema[$key], $data);

This can be simplified

The last submitted patch, 30: entity-schema_data_fix-2597628-30.test.patch, failed testing.

alexpott’s picture

Status: Needs review » Needs work
+++ b/core/modules/content_translation/src/Tests/ContentTranslationEnableTest.php
@@ -0,0 +1,54 @@
+    \Drupal::entityDefinitionUpdateManager()->applyUpdates();
...
+    \Drupal::entityDefinitionUpdateManager()->applyUpdates();

Am I right in thinking this is only needed because we're doing this in a KernelTestBase? If so I think this is a bad idea. We're fixing the test as we go.

alexpott’s picture

Assigned: Unassigned » alexpott

I'm converting the test to use the UI.

alexpott’s picture

Assigned: alexpott » Unassigned
Status: Needs work » Needs review
StatusFileSize
new3.81 KB
new6.38 KB

This seems to work nicely. I had to change the requirements to always report on the the entity/field update status - which makes sense now they are no longer coupled to database updates.

The test-only patch is basically the interdiff.

The last submitted patch, 39: 2597628.38.test-only.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 39: 2597628-38.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
StatusFileSize
new9.06 KB
new3.34 KB

Fixing the test fail caused by the changes to the system status report.

gábor hojtsy’s picture

Issue summary: View changes
Status: Needs review » Needs work
Issue tags: -Needs tests +Needs issue summary update

Updates make sense to me. It is indeed better to webtest this. It is also fine to display a confirmation about entity status in the reports, not just include it when its broken. Just some small remarks:

  1. +++ b/core/modules/content_translation/src/Tests/ContentTranslationEnableTest.php
    @@ -0,0 +1,58 @@
    +    $requirement_value = $this->cssSelect("tr.system-status-report__entry th:contains('Entity/field definitions') + td");
    +    $this->assertEqual('Up to date', trim((string) $requirement_value[0]));
    ...
    +    $requirement_value = $this->cssSelect("tr.system-status-report__entry th:contains('Entity/field definitions') + td");
    +    $this->assertEqual('Up to date', trim((string) $requirement_value[0]));
    

    While we do not run tests with translations, we have a convention of testing for output text that is t()-ed by t()-ing what we assert as well.

  2. +++ b/core/modules/content_translation/src/Tests/ContentTranslationEnableTest.php
    @@ -0,0 +1,58 @@
    +    $this->drupalPostForm('admin/modules', $edit, 'Install');
    ...
    +    $this->drupalPostForm('admin/config/regional/content-language', $edit, 'Save configuration');
    

    Same for button labels.

Also I think we are really close and this therefore does not need a plan B. An issue summary update is required anyway, the underlying problem / proposed solution is not explained.

Finally is this definition cached and do we need to clear that for this fix on existing sites? That historically required a no-up update function to sure that the caches were cleared (via update.php).

alexpott’s picture

Issue summary: View changes
Status: Needs work » Needs review
StatusFileSize
new1.89 KB
new9.07 KB

Fixed up #43

This is not cached so there is no need even for an empty update function afaics.

jhedstrom’s picture

+++ b/core/modules/content_translation/src/Tests/ContentTranslationEnableTest.php
@@ -0,0 +1,58 @@
+    // No pending updates should be available.
+    $this->drupalGet('admin/reports/status');
+    $requirement_value = $this->cssSelect("tr.system-status-report__entry th:contains('Entity/field definitions') + td");
+    $this->assertEqual(t('Up to date'), trim((string) $requirement_value[0]));
...
+    // No pending updates should be available.
+    $this->drupalGet('admin/reports/status');
+    $requirement_value = $this->cssSelect("tr.system-status-report__entry th:contains('Entity/field definitions') + td");
+    $this->assertEqual(t('Up to date'), trim((string) $requirement_value[0]));

Does this add any advantage over directly checking if updates are needed via the entity.definition_update_manager service? It ties this test fairly closely to the UI/requirement hook.

plach’s picture

Assigned: Unassigned » plach
Status: Needs review » Needs work
Issue tags: +Needs tests
Related issues: +#2554911: Inform users/developers about entity/field mismatch details if it happens

The root issue has nothing to do with the CT module, so we still need some API-level integration tests. Working on those.

One of the reasons to not display the entity definition status except that in problematic conditions is that the entity/field definition terminology was deemed not user-friendly and thus confusing. I don't think we should be making UI changes here, they are completely unnecessary to fix this issue and we already have #2554911: Inform users/developers about entity/field mismatch details if it happens to discuss/fix the UI properly.

plach’s picture

Assigned: plach » Unassigned
Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new11.56 KB
new3.42 KB

And here is the additional test coverage. Test-only patch is the interdiff.

plach’s picture

Issue summary: View changes
Issue tags: -Needs issue summary update

I'd still like to see the UI changes reverted but this is not my decision :)

If we don't do that we need to update the UI changes IS section.

Status: Needs review » Needs work

The last submitted patch, 47: entity-schema_data_fix-2597628-46.test.patch, failed testing.

plach’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 47: entity-schema_data_fix-2597628-46.test.patch, failed testing.

effulgentsia’s picture

Finally is this definition cached and do we need to clear that for this fix on existing sites?

#2602996: Upgrade path for #2597628 was closed as a duplicate, so I think yes, we either need to reopen that one, or else add a hook_update_N() to this patch to clean up all the affected *.entity_schema_data KV entries. Those are not caches, but it should be possible to detect ones that are recording an index that they should no longer be according to this improved code's logic, and correspondingly remove it from that entry.

plach’s picture

Status: Needs work » Needs review

I think there's no cache to be cleared: while coding/manually testing this I just kept refreshing the status report page and I saw the error appearing/disappearing depending on the code correctness.

It would be great if @hguo and @dpacassi could confirm that applying the patch is enough to get rid of the status report error.

Status: Needs review » Needs work

The last submitted patch, 47: entity-schema_data_fix-2597628-46.test.patch, failed testing.

plach’s picture

Status: Needs work » Needs review
plach’s picture

StatusFileSize
new11.56 KB

Re-uploading the last patch, not sure what's happening.

The last submitted patch, 47: entity-schema_data_fix-2597628-46.test.patch, failed testing.

alexpott’s picture

@plach I'm not sure that we should call the change to a line on the status report a UI change - because the user interface has not changed.

plach’s picture

I don't have a better definition for that change, sorry, but it looks unnecessary to me. I was suggesting to revert it because it has UX implications (albeit minimal) that would be better discussed elsewhere, to avoid possibly delaying this issue.

alexpott’s picture

StatusFileSize
new7.75 KB
new5.58 KB

So the problem with removing the changes to the UI means we have no reliable way of testing this through the UI which makes the ContentTranslationEnableTest extremely fragile. Trying to improve that shows we have a significant issue in keeping the parent and site under test in sync since I can't get the test to pass even though I'm whacking it with a hammer..

    drupal_flush_all_caches();
    $this->rebuildContainer();

This raises massive questions about much of the testing we do for multilingual where we do stuff in the test-runner and through the UI - i've said this before and I'll say it again - this is an extremely bad practice only done for quicker running tests but in reality no one is testing what they think they are.

gábor hojtsy’s picture

StatusFileSize
new11.56 KB

I think the UI change is very minimal and the status report is full of developery things anyway. Also the UI can be further improved later. Let's keep the UI change and move forward. The bugfix is a lot more important, it is not worth delaying the D8 release on one line in the status report ;) Reuploading that prior patch from #56, basically undoing #60.

Not RTBC due to the unclear resolution of #52. That is the kind of question that are actually serious IMHO :)

The last submitted patch, 60: 2597628-2-60.patch, failed testing.

alexpott’s picture

Re #52/#61 I've discussed this with @plach in IRC and we are both sure that the values in the KV store are correct and do not need fixing and there is no cache to clear.

alexpott’s picture

Also I'm more than fine with leaving the status report changes in :)

plach’s picture

Spoke with @alexpott, we reached similar conclusions to #61. Funny that I started delaying this issue to avoid delaying it ;)

About #52 I can only re-state that I think KV data is not related to this issue and no cache-clear is needed but maybe I'm missing @eff's point.

plach’s picture

Issue summary: View changes
StatusFileSize
new236.38 KB

Added screenshot

gábor hojtsy’s picture

Issue summary: View changes
Status: Needs review » Reviewed & tested by the community

All righto, looks good to me now :) Thanks for the updated issue summary.

The last submitted patch, 60: 2597628-2-60.patch, failed testing.

effulgentsia’s picture

Status: Reviewed & tested by the community » Needs review

maybe I'm missing @eff's point

Ah, I figured out the discrepancy in what I was seeing vs. what you all were. So, if you do the following:

- Install HEAD, Standard profile.
- Enable Content Translation module.
- "Enable translation" for Tags vocabulary.
- Go to /admin/reports/status and notice the mismatch warning.
- Apply #61.
- Go to /admin/reports/status and notice that entity and field definitions are "up to date". Yay!

But, if you do the following:

  1. Install HEAD, Standard profile.
  2. Enable Content Translation module.
  3. "Enable translation" for Tags vocabulary.
  4. Go to /admin/reports/status and notice the mismatch warning.
  5. Try to fix this by invoking an onEntityTypeUpdate() for the taxonomy_term entity type. One way is with the attached script.
  6. Go to /admin/reports/status and notice no mismatch warning.
  7. Apply #61.
  8. Go to /admin/reports/status and notice the mismatch warning.

The reason is that #5 above results in the entity.storage_schema.sql:taxonomy_term.entity_schema_data KV value getting saved with the taxonomy_term__1a0c187e9a index prior to applying the patch. Then the patch fixes the code to not expect that index there.

Note that the situation can be re-fixed by redoing step #5 after applying the patch, so potentially, we can tell people, hey, if you manually messed with things like that during an earlier RC, then you need to redo that same manual step on the next RC.

But, I'm not sure if there's any code path by which that onEntityTypeUpdate() might have been called without the site owner's manual intervention (e.g., some other update function, whether in core or contrib), in which case, releasing this patch and it resulting in #8 above would be confusing.

If we want to fix this, some options might be:

  • Add an update function that does an $edum->updateEntityType($edum->getEntityType()) for every entity type. However, we'd need to check if that works correctly for all database drivers, in case it tries to delete and recreate indexes that it shouldn't.
  • Or, if we want to be more targeted: add an update function that only removes indexes stored in *.entity_schema_data KV entries that should no longer be there after this patch.

Setting to "Needs review" for more input on whether to add such an update function.

effulgentsia’s picture

StatusFileSize
new332 bytes

One way is with the attached script

Here's that attachment.

effulgentsia’s picture

Or, instead of #70, you can use Drush per #7.

Additionally:

I'm not sure if there's any code path by which that onEntityTypeUpdate() might have been called without the site owner's manual intervention (e.g., some other update function, whether in core or contrib)

Yeah, here's one path:

  1. Install beta12.
  2. Enable CT on Tags vocab.
  3. Run update.php (or if that seems arbitrary because why would you run update.php without a code update, then update to beta13 and then run update.php).
  4. Update to RC2 or HEAD and run update.php.
  5. Go to /admin/reports/status and notice no mismatch warning.
  6. Apply #61.
  7. Go to /admin/reports/status and notice the mismatch warning.

The reason is that prior to beta15 (#2542748: Automatic entity updates can fail when there is existing content, leaving the site's schema in an unpredictable state), every run of update.php would do a full EDUM::applyUpdates(), which would get the taxonomy_term__1a0c187e9a index into the entity.storage_schema.sql:taxonomy_term.entity_schema_data KV entry.

Status: Needs review » Needs work

The last submitted patch, 61: entity-schema_data_fix-2597628-46_0.patch, failed testing.

The last submitted patch, 60: 2597628-2-60.patch, failed testing.

The last submitted patch, 56: entity-schema_data_fix-2597628-46.patch, failed testing.

The last submitted patch, 56: entity-schema_data_fix-2597628-46.patch, failed testing.

plach’s picture

Assigned: Unassigned » plach

Ah, excellent point! Let's see whether I can come up with a sensible update function...

The last submitted patch, 56: entity-schema_data_fix-2597628-46.patch, failed testing.

The last submitted patch, 60: 2597628-2-60.patch, failed testing.

alexpott’s picture

Are we sure that we should do this? There is no functionality in core or Drush (apart from tools that allow you to run custom PHP) to get HEAD in this state. For me this opens up a can of worms - where does this responsibility start and end?

plach’s picture

@alexpott:

The steps in #71 can be performed by any site builder, I reproduced them manually with no difficulty. I'm almost done with the update function, I'll post it and then we can decide whether it's worth including it.

plach’s picture

Status: Needs work » Needs review
StatusFileSize
new3.51 KB
new13.78 KB

And here is the update function, if we end-up including it we'll need additional test coverage.

alexpott’s picture

Well we only supported the upgrade path from beta 15. Harsh but true. I think I'd rather post a script for people to run rather than support an upgrade path we don't have to - especially one that mucks about with entity defintiions

alexpott’s picture

Also I tried adding the following assertions to the UpdatePathTestBaseFilledTest because it was added as part of beta 12 and has content_translation installed...

    $this->drupalGet('admin/reports/status');
    $this->assertNoRaw('Out of date');
    $this->assertRaw('Mismatch detected');
    $this->assertRaw('Mismatched entity and/or field definitions');

But it failed - so I think testing the upgrade path is going to be quite interesting.

catch’s picture

Status: Needs review » Reviewed & tested by the community

Let's tackle the upgrade path in the separate issue open for it (will need to be unduplicated) as a major. I think it's very important to remove this error state for new sites to avoid confusion before the next rc.

gábor hojtsy’s picture

I agree we should not need to support an upgrade path for versions we said we are not going to support an upgrade path for ;)

plach’s picture

Ok, then patch to commit is #61 :)

plach’s picture

StatusFileSize
new2.88 KB
new11.73 KB

Ok, removed the update function per #82 and following. The interdiff contains a minor clean-up I did not post yet. I'm posting the removed update in function in #2602996: Upgrade path for #2597628, in case anyone needs it.

plach’s picture

Assigned: plach » Unassigned
gábor hojtsy’s picture

The changeis minor, still looks RTBC to me :)

effulgentsia’s picture

Status: Reviewed & tested by the community » Fixed

Pushed to 8.0.x. Thanks all!

  • effulgentsia committed c4501b3 on 8.0.x
    Issue #2597628 by plach, jhedstrom, alexpott, Gábor Hojtsy: Enabling...
gábor hojtsy’s picture

Issue tags: -sprint

Thanks all, this was tough! :)

  • effulgentsia committed c4501b3 on 8.1.x
    Issue #2597628 by plach, jhedstrom, alexpott, Gábor Hojtsy: Enabling...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

bryanmanalo’s picture

comment #7 works, thanks!

giorgosk’s picture

#7 works
drush updb --entity-updates