Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.
It seems that translatable content entities need to implement two tables: base_table and data_table. Group and GroupContent currently only implement the former. In order to have translatability function properly, it turns out we need to add this (rather poorly documented) table to the annotations.
However, as https://www.drupal.org/node/2554097 points out, this cannot be done automatically. We'll need to write proper update hooks to fix this.
Major because any new module install will basically get a faulty base table.
Comment | File | Size | Author |
---|---|---|---|
#12 | group patch fail.jpg | 149.18 KB | alan.upstone |
#11 | interdiff-11-9.txt | 3.98 KB | kristiaanvandeneynde |
#11 | group-2715011-11-WIP.patch | 8.1 KB | kristiaanvandeneynde |
Comments
Comment #2
kristiaanvandeneyndeWe should also have a battle plan for people with existing views.
See https://www.drupal.org/node/2433153
Comment #3
joachim CreditAttribution: joachim at Torchbox commented> this (rather poorly documented) table to the annotations.
Is there an issue to improve the documentation of this? (I did notice that this module's entities didn't have a 2nd table, but as I don't fully understand it myself, I didn't say anything. Sorry!)
Comment #4
kristiaanvandeneyndeJust created one: #2715089: Document the type of tables an entity type can define
And it's not your fault :) I copied a lot of logic from Node and other content entity types, building it up from scratch. When I got to a point where it would install correctly, I thought I had all the necessary bits in place. Seeing as the module worked fine without a data_table I figured it was optional.
Guess it's not..
Comment #5
kristiaanvandeneyndeHere's an almost complete patch so I can ask core devs to see if this is the right approach. All it needs is a migration of the old data into the new tables which is why I'm trying to retrieve all of the column names. So I can build queries where I select the old data into the new columns.
Few notes:
Comment #6
ronaldtebrake CreditAttribution: ronaldtebrake as a volunteer and for Open Social commentedNicely spotted and thanks for this great effort!
It seems to work well for me. But I will wait for one of the core developers to set it to RTBC I guess :).
A couple of questions:
1. Am I correct in thinking on a new install, with the 'translatable' annotation and the data table key in place the update_hook / migration is 'redundant' since there is nothing to migrate and the tables should be installed correctly?
If so maybe we can alter the documentation of the update hook a bit to make that more clear.
2. Regarding the examples, I found one over here https://www.drupal.org/node/2641828#comment-10711058 especially the interdiff could be useful: https://www.drupal.org/files/issues/choice-entity-type-2641828-5-interdi...
Comment #7
dawehnerOne problem with that is that this potentially ignores other update functions which want to deal with the same schema. The alternative would be to just set the new entity tables + translatable flag
You need to pass the new
$entity_type
to the getTableMapping method, so you get the latest code one, not the last installed version.Comment #8
kristiaanvandeneyndeJust talked a bit more to dawehner on IRC and he suggested I load the old schema, adjust the necessary keys and then save that one. That way we don't undo any schema alterations someone else might have done during their update hook.
Furthermore, we could then have a variable containing the old schema and one with the new, pass those into $storage->getTableMapping() and be able to read the old column names like that.
All that is left then is to map the old columns' data to the new ones' and remove the temporary table.
Comment #9
kristiaanvandeneyndeRight, so this patch takes care of the data migration properly, but it made me rethink both points made by dawehner. I'll see if I can ask him to weigh in on them once more.
First of all, we are rebuilding the entity type's tables from scratch and from code, so I don't see why we need to load the previous schema definition and adjust it instead of simply overwriting it. After all, any module that wanted to adjust the schema also had to adjust the entity type's definition and we are using the latest copy of that in our update hook.
Secondly, I couldn't for the life of me find a way to retrieve the old storage definition to pass into
getTableMapping()
. Instead I tried to manually load the storage handler for the old entity type definition and the new entity type definition, but it turns outgetTableMapping()
then tries to load the storage definition from the entity type in code. So that didn't work and I had to lift the retrieval for the old columns to the top of the update hook. It feels like I'm doing something wrong here, so I'd love to find a way to retrieve the$storage_definition
to pass ontogetTableMapping()
.Do not use this patch yet please, it works great but I want to have someone review it first.
Comment #10
dawehnerWell, let's assume a user updates from moduleA_8000 to moduleA_8002 and moduleB_8001 to moduleB__8009, while hitting update.php. Update.php will not guarantee you the order of those update functions, so other update functions might come before or after. On the other hand the code version doesn't change during that process, so the definition in code, is up to date already. update.php is tricy.
I think it is something like:
Comment #11
kristiaanvandeneyndeRight, so this works.
All it needs now is exceptions for edge cases where the storage doesn't implement a certain interface.
Also note we are not deleting the migration tables so people can still access their old data should anything go wrong during the migration. They should have back-ups, but this is a nice gesture towards those who can't easily access or restore those.
Comment #12
alan.upstone CreditAttribution: alan.upstone commentedHello Kristiaan
Applied the patch. I got the attached error when running update.php, and my views are all broken.
I get this when clicking on your flurb then selecting a group:
http://dev-mobilized.pantheonsite.io/group/20
The website encountered an unexpected error. Please try again later.
Drupal\Core\Database\DatabaseExceptionWrapper: Exception in Group actions[group_actions]: SQLSTATE[42S22]: Column not found: 1054 Unknown column 'unknown' in 'order clause': SELECT 'group_actions:block_2' AS view_name, MIN(group_content.id) AS group_content_id FROM {group_content} group_content GROUP BY view_name ORDER BY unknown ASC; Array ( ) in Drupal\views\Plugin\views\query\Sql->execute() (line 1452 of core/modules/views/src/Plugin/views/query/Sql.php).
Drupal\views\ViewExecutable->execute(NULL) (Line: 1412)
Drupal\views\ViewExecutable->render() (Line: 117)
Drupal\views\Plugin\views\display\Block->execute() (Line: 1587)
Drupal\views\ViewExecutable->executeDisplay('block_2', Array) (Line: 78)
Drupal\views\Element\View::preRenderViewElement(Array) (Line: 42)
Drupal\views\Plugin\Block\ViewsBlock->build() (Line: 202)
Drupal\block\BlockViewBuilder::preRender(Array)
call_user_func('Drupal\block\BlockViewBuilder::preRender', Array) (Line: 381)
Drupal\Core\Render\Renderer->doRender(Array) (Line: 448)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 462)
Drupal\Core\Template\TwigExtension->escapeFilter(Object, Array, 'html', NULL, 1) (Line: 104)
__TwigTemplate_2617f4b4428068cdd24e54b3cdcc7ae84d6ae2f3e1366ac6b01a2537a9c32694->doDisplay(Array, Array) (Line: 381)
Twig_Template->displayWithErrorHandling(Array, Array) (Line: 355)
Twig_Template->display(Array) (Line: 366)
Twig_Template->render(Array) (Line: 64)
twig_render_template('core/themes/bartik/templates/page.html.twig', Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render('page', Array) (Line: 435)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 462)
Drupal\Core\Template\TwigExtension->escapeFilter(Object, Array, 'html', NULL, 1) (Line: 88)
__TwigTemplate_039606928b737abc794de14b5f41a97b59176f66c9dcc0bb3b6df3a09ecc43a2->doDisplay(Array, Array) (Line: 381)
Twig_Template->displayWithErrorHandling(Array, Array) (Line: 355)
Twig_Template->display(Array) (Line: 366)
Twig_Template->render(Array) (Line: 64)
twig_render_template('core/themes/classy/templates/layout/html.html.twig', Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render('html', Array) (Line: 435)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 147)
Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() (Line: 574)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 148)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, 'kernel.view', Object) (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('kernel.view', Object) (Line: 144)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 62)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 98)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 77)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 50)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 628)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)
Comment #13
kristiaanvandeneyndeHey Alan,
I actually expected Views to break because the underlying table structure has changed. You should be able to recover them by exporting their config (single export), changing 'groups' as the base table to 'groups_field_data' and then re-importing that.
Could you try that and let me know?
Comment #14
alan.upstone CreditAttribution: alan.upstone commentedEven your modules's own view - group/x - is expected to crash?
Comment #15
kristiaanvandeneyndeIt's not crashing for me, you probably have a block with a view on that page?
Comment #16
kristiaanvandeneyndeRight, so I just broke my site on purpose to reproduce this error and fixed it like this:
/admin/config/development/configuration/single/export
and copy over all of the affected views (one txt file per view for instance)/admin/config/development/configuration/single/import
and paste in one view at a timegroups
withgroups_field_data
where the key statesbase_table:
or simplytable:
group_content
andgroup_content_field_data
Again: pay special attention to the keys
base_table:
andtable:
. Changingentity_type: group_content
will break your view.I'll commit the patch now so other issues are no longer blocked and will link to this issue in the next release's notes.
Comment #17
kristiaanvandeneyndeComment #18
alan.upstone CreditAttribution: alan.upstone commentedKristiaan was right re #14. There was a block in the sidebar using a view that was crashing; not the main Group view. Changing the appearance setting to another theme temporarily turns off such blocks and allows you to do the stuff in #16. Then you can reinstate your blocks by reverting the theme.
Thanks for taking the time to do #16.
Comment #19
alan.upstone CreditAttribution: alan.upstone commentedI have had all sorts of problems since doing this.
My status report says
Entity/field definitions
Mismatched entity and/or field definitions
The following changes were detected in the entity type and field definitions.
Group entity
The Group entity entity type needs to be updated.
The Default translation field needs to be uninstalled.
Group content entity
The Group content entity entity type needs to be updated.
The Default translation field needs to be uninstalled.
Running drush entity-updates gets...
The following updates are pending:
group entity type :
The Group entity entity type needs to be updated.
The Default translation field needs to be uninstalled.
group_content entity type :
The Group content entity entity type needs to be updated.
The Default translation field needs to be uninstalled.
Do you wish to run all pending updates? (y/n): y
...then the error:
Drupal\Core\Entity\EntityStorageException: The SQL storage cannot change the schema for an existing entity type (group) with data. in
Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema->onEntityTypeUpdate()
(line 303 of /srv/bindings/96df09bfe929475f9e54ce28f76f5177/code/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php).
Failed: Drupal\Core\Entity\EntityStorageException: !message in Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema->onEntityTypeUpdate()
(line 303 of /srv/bindings/96df09bfe929475f9e54ce28f76f5177/code/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorageSchema.php).
Comment #20
kristiaanvandeneyndeYou should not have to run
drush entity-updates
if you just ran update.php. The update hook should have cleanly migrated your existing data structure into the new data structure, after which the status report should no longer list mismatched entity definitions.Do you still have a DB backup from before the update?
Comment #21
alan.upstone CreditAttribution: alan.upstone commentedHello Kristiaan
drush entity-updates was the last report after trying the usual things like update.php with no luck.
I restored from a backup. I ran update.php again and deleted every view named in the error message (see image in comment #12) as causing a taxonomy_index_tid error - until all errors were gone and I could get update.php to complete. Then I recreated the deleted views.
Comment #22
kristiaanvandeneyndeHi Alan,
When it comes to updates like these, you should steer clear from
drush entity-updates
. Data migrating entity updates should always happen via an update hook. I'm glad you got it to work in the end, but it left me wondering a bit about why it went wrong.Anyway, I've updated the alpha4 release notes to mention the specific steps you need to take to upgrade and linked people to this issue if they had any views that needed updating. On the bright side: with this out of the way there should be no more massive data changes like this one.
Sorry to all who were inconvenienced by this patch, even though it was absolutely necessary to get it in as soon as possible. One roadblock less towards beta!
Comment #26
kristiaanvandeneyndeBad testbot! :)