Error updating to v1.0

Confirming

Same problem. I can no longer update the database since i run the 1.0 version of drupal/entityqueue.

Have this same problem over here

We've also had this problem on at least one of our sites.

Similar issue here - not with creating the tmp tables, but with replacing the existing ones:

>  [notice] Update started: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable
>  [error]  Exception thrown while performing a schema update. Cannot rename tmp_eeaf28entity_subqueue_field_data to entity_subqueue_field_data: table entity_subqueue_field_data already exists. 
>  [error]  Update failed: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable 
 [error]  Update aborted by: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable 
 [error]  Finished performing updates. 

I can delete the following tables to allow the process to get past this error, but then the base tables used to generate the tmp tables aren't present if I do... seems like there is a step missing before renaming the tmp tables?

Note that I had updated to RC1 before updating to 1.0.

I got a different error upgrading:
and it leaves a bunch of tmp tables

 tmp_6a8bddentity_subqueue                        
 tmp_6a8bddentity_subqueue__items                 
 tmp_6a8bddentity_subqueue_field_data             
 tmp_6a8bddentity_subqueue_field_revision         
 tmp_6a8bddentity_subqueue_revision               
 tmp_6a8bddentity_subqueue_revision__items        
 tmp_7e1786entity_subqueue                        
 tmp_7e1786entity_subqueue__items                 
 tmp_7e1786entity_subqueue_field_data            
 tmp_7e1786entity_subqueue_field_revision         
 tmp_7e1786entity_subqueue_revision               
 tmp_7e1786entity_subqueue_revision__items  

 ------------- ------------------------ ------------- ------------------------ 
  Module        Update ID                Type          Description             
 ------------- ------------------------ ------------- ------------------------ 
  entityqueue   make_entity_subqueue_r   post-update   Update subqueues to be  
                evisionable_and_transl                 revisionable and        
                atable                                 translatable.           
  entityqueue   remove_reverse_in_admi   post-update   Remove the obsolete     
                n_setting                              'reverse_in_admin'      
                                                       queue setting.          
 ------------- ------------------------ ------------- ------------------------ 


 Do you wish to run the specified pending updates? (yes/no) [yes]:

>  [notice] Update started: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable
>  [error]  Exception thrown while performing a schema update. SQLSTATE[42S02]: Base table or view not found: 1146 Table 'entity_subqueue__scheduled_transition_date' doesn't exist: SELECT t.*
> FROM
> {entity_subqueue__scheduled_transition_date} t
> WHERE (entity_id IN (:db_condition_placeholder_0)) AND (deleted = :db_condition_placeholder_1) AND (langcode IN (:db_condition_placeholder_2, :db_condition_placeholder_3, :db_condition_placeholder_4, :db_condition_placeholder_5))
> ORDER BY delta ASC; Array
> (
>     [:db_condition_placeholder_0] => front_page
>     [:db_condition_placeholder_1] => 0
>     [:db_condition_placeholder_2] => en
>     [:db_condition_placeholder_3] => fr
>     [:db_condition_placeholder_4] => und
>     [:db_condition_placeholder_5] => zxx
> )
>  
>  [error]  Update failed: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable 
 [error]  Update aborted by: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable 
 [error]  Finished performing updates. 

to work around this problem, I restored my database to a version prior to when I ran updb (because the first updb I tried totally FUBAR the schema),

after restoring my database, I downgraded entityqueue to beta5

1. restore database prior to updb that breaks
2. downgrade to entityqueue beta5
3. *EDIT*visit this page: /admin/modules/uninstall/entity/entity_subqueue
4. visit this page (instead) /admin/structure/entityqueue and delete the entity queue altogether
5. *EDIT*deleted the item(s) from the subqueue (there was only one in my case)
6. restore the current release of entityqueue (1.0?)
7. run an updb

Unfortunately, I have no idea what this module is doing in my clients configuration, have no idea what I just deleted.

but I was able to restore it after re-importing the configuration containing the queue.

I tried to restore my entity queue by recreating it after deleting it (to pass updb without error) but it came back with an error even after updb so I restored my db and downgraded back to 1.0-beta5

Locking in 1.0-beta5

composer require 'drupal/entityqueue:1.0-beta5'

Out of curiosity, has anyone seen issues in 1.0-rc1 as well, or just 1.0? We're keeping our sites on rc1 or earlier to avoid these problems, but if rc1 also has issues, we can specify beta5 instead like @joseph.olstad did.

Also, has anyone been able to to figure out which commit is causing these problems?

Ok I did some more digging, and I went back as far as beta5

as soon as I check out the commit right after beta5 was released, can no longer run updb without errors.

commit e099b13506
Date:   Thu May 30

#3058127: Convert subqueues to be revisionable, translatatable and publishable

Note, we are using a lightning based distro called wxt

Ok for us, 100% confirmed, #3058127: Convert subqueues to be revisionable, translatatable and publishable prevents us from upgrading
This is the first commit after 1.0-beta5 is tagged

Sorry for the problem that some people are having with this update :( This issue is caused by the Lightning workflow module: #3126343: Scheduler needs to maintain its base fields properly

We also do not have the Lightning workflow module or wxt (web experience toolkit), both of which have issues submitted that relate to at least one of the issues we're discussing here. To be fair, I'm not sure which problem my web developer has seen on our sites using 1.0, so I'm not sure how much help I can be past there.

@gordon @hugronaphor, the problem here is that there's nothing to fix in Entityqueue module, we are just using the core API for converting an entity type to revisionable and translatable.

The root cause for these errors is usually some field storage definitions that are not installed or updated correctly, either by a contrib module like Lightning workflow or custom ones, and the fix can only be individual for each site that experiences a problem with an entity type conversion.

If either of you can send me a copy of your codebase and a sanitized database dump (e.g. all the content/users/etc. deleted), feel free to contact me in the Drupal slack or using drupal.org's personal contact form, and I can try to help out.

Thank you @joseph.olstad

I found your steps to work in comment

#8

to work in my situation. (entity queue beta 1.0-beta5, restore db, delete entity queue in admin).

However it does mean that the feature of the site that made use of Entity Queue had to be removed.

For everyone, just a perhaps unkind comment and slightly off-topic from me regarding Lightning distro: I'm now wondering what the value of it is compared with just using core.

Whenever it comes to upgrading, I've had to make extra considerations for what has changed in Lighting. I have come across errors in this process in the past.

Lightning does seem to have the goal of enhancing publishing experiences when using Drupal. But I thought Drupal itself, being a CMS is all about publishing, among many other things - so wouldn't it be better to put lightining features into core, then we only have one place and one set of moving parts to deal with.

I'm perhaps going to move away from lighting and use the various config tools to extract out my site config and content export tools in order to rebuild the site on plain Drupal 8without it. Others may well find more value in Lightning I would guess.

@therobyouknow , that's a bit off topic, for now there's a perfectly valid workaround which is to lock the dependencies to entityqueue 1.0-beta5.

Meanwhile there's an open issue for this.
#3126343: Scheduler needs to maintain its base fields properly

confirming

@cobenash, see #17.

I encountered the same problem...

@phjou, see #17 as well.

It appears I may be having this issue as well? I do not have a lightning workflow module installed.

I attempted to update the Entityqueue module to 8.x-1.0-beta5 and it throws this error:

Failed: Drupal\Core\Entity\EntityStorageException: Exception thrown while performing a schema update. Cannot rename tmp_1e5f30entity_subqueue_field_data to entity_subqueue_field_data: table entity_subqueue_field_data already exists. in Drupal\Core\Entity\Sql\SqlContentEntityStorage->wrapSchemaException() (line 1615 of /var/lib/nginx/websitename/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).

This was while using Drupal core version 8.8.5 - and we've since updated to 8.8.6. The problem remains. I cannot do dbupdate due to this error.

@amateescu Easy when you can do a database rollback, in my case the update has been done a while ago. I had to manually update/delete the tables to rollback. It was a big pain...

@phjou, I understand how update problems like this can be very frustrating, but without having access to something like I described in #17 there's really no way to provide a fix.

@amateecu, I can provide you with tarball of an environment for this
I will send you a private message and include my email address, please respond to my email.

@amateescu, good news, I believe I have found the source of the bug.
the entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable function is somehow trying to load an entity_id that is non-numeric , for instance 'front_page'

I have a node on my front page in a block using a panel of some kind.

front_page is not a valid entity id, why would the upgrade load this?

There needs to be some validation to make sure the entity_id is numeric. However I'm not sure how this sandbox works, any suggestions?

The fix needs to be made in entityqueue.post_update.php in this function:

function entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable(&$sandbox) {

I have prepared an environment for you that exposes the issue with a jailkit shell account, I will send you an email now.
unfortunately I haven't yet configured php/drush for the jailkit account however you will be able to copy the files or download the whole site, I will provide details in email.

Basically the environment I have was installed with entityqueue beta5 , it is unable to upgrade to entityqueue 1.0 or the rc versions

as soon as update.php is run, the site will crash
I recommend downloading the whole site and installing it locally for debugging.

I have a non jailkit shell account also if you want I can provide that.

drush updb
 ------------- ----------------------- --------------- ------------------------ 
  Module        Update ID               Type            Description             
 ------------- ----------------------- --------------- ------------------------ 
  entityqueue   8004                    hook_update_n   Add the publishing status field to subqueues.              
  entityqueue   8005                    hook_update_n   Add the revision metadata fields to subqueues.              
  entityqueue   make_entity_subqueue_   post-update     Update subqueues to be revisionable_and_tran revisionable and slatable translatable.
  entityqueue   remove_reverse_in_adm   post-update     Remove the obsolete in_setting 'reverse_in_admin' queue setting.          
 ------------- ----------------------- --------------- ------------------------ 


 Do you wish to run the specified pending updates? (yes/no) [yes]:
 > yes

>  [notice] Update started: entityqueue_update_8004
>  [notice] The publishing status field has been added to Entity Subqueues.
>  [notice] Update completed: entityqueue_update_8004
>  [notice] Update started: entityqueue_update_8005
>  [notice] The revision metadata fields have been added to Entity Subqueues.
>  [notice] Update completed: entityqueue_update_8005
>  [notice] Update started: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable
>  [error]  Exception thrown while performing a schema update. SQLSTATE[42S02]: Base table or view not found: 1146 Table 'c3wxt.entity_subqueue__scheduled_transition_date' doesn't exist: SELECT t.*
> FROM
> {entity_subqueue__scheduled_transition_date} t
> WHERE (entity_id IN (:db_condition_placeholder_0)) AND (deleted = :db_condition_placeholder_1) AND (langcode IN (:db_condition_placeholder_2, :db_condition_placeholder_3, :db_condition_placeholder_4, :db_condition_placeholder_5))
> ORDER BY delta ASC; Array
> (
>     [:db_condition_placeholder_0] => front_page
>     [:db_condition_placeholder_1] => 0
>     [:db_condition_placeholder_2] => en
>     [:db_condition_placeholder_3] => fr
>     [:db_condition_placeholder_4] => und
>     [:db_condition_placeholder_5] => zxx
> )
>  
>  [error]  Update failed: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable 
 [error]  Update aborted by: entityqueue_post_update_make_entity_subqueue_revisionable_and_translatable 
 [error]  Finished performing updates. 

> [:db_condition_placeholder_0] => front_page

this value is not a valid entity_id , if we can fix this, I believe we can fix the bug.

ok so, more investigation and I think it comes down to this:

Why does this table not exist?

table entity_subqueue__scheduled_transition_date doesn't exist.

@joseph.olstad, subqueues do use strings for their ID, so front_page is a valid subqueue ID. I can take a look and debug once you send over the environment by email :)

did some more digging, this might be related to a core issue
#2332857: Decide how secondary entity handlers (like storage_schema) should be instantiated

Some progress to report, @joel_osc also came accross this problem and decided to implement a workaround
see:
#3130960-14: entityqueue dependency 1.0 or 1.0-rc1 fails updb

So for now, this workaround is probably our best/only solution that will allow us to upgrade.

actually that workaround appears to have been created originally by @clayfreeman here:

#3126343-4: Scheduler needs to maintain its base fields properly

So we have to credit @clayfreeman for that workaround

@joseph.olstad, thanks for sending me that environment, I finally had the time to look into it and posted a patch at #3126343: Scheduler needs to maintain its base fields properly which fixes the problem for everyone who uses the Lightning Scheduler module.

For people who aren't using that module, the only way for me to be able to help is if you can provide me a copy of your codebase and a sanitized database dump (e.g. all the content/users/etc. deleted), like @joseph.olstad did.

@gordon, I don't think we should add a fix in Entityqueue for a problem that's specific to the Lightning Scheduler module. Even more, I think the patches in #40 / #41 are a bit dangerous, because if people apply them and then we add a "real" entityqueue_update_8006() update function in another issue, it won't run for them :/

@gordon, interesting.. but the patch from #41 does something with the scheduled_transition_date and scheduled_transition_state fields, that's why I assumed you were using that module as well.

Is there any chance you could send me a sanitized environment so I can dig into the actual problem in your case?

we're good now, see this commit message if you get stuck on this.

https://github.com/drupalwxt/wxt/commit/4d2d909b5732f1746773f2850651391b...

Confirming Comment#11 at https://www.drupal.org/project/lightning_workflow/issues/3126343 resolves this issue when using Lightning Workflow.

If anyone else gets stuck on this particular issue, I've updated the patch in #45 so that it's compatible with the 8006 update since released.

Sadly, the last patch did not work for me, and I've yet to find a solution to this issue, beyond completely uninstalling and reinstalling the module.

@datawench, you might need to lock on entityqueue 1.0 and apply the patch 11

however according to my notes I had to upgrade lightning_workflow first, run the schema updates and THEN upgrade entityqueue, patch it and do it's schema updates

#3126343-14: Scheduler needs to maintain its base fields properly

I admit though, it was tricky getting past this issue.

Unfortunately, @joseph.olstad, lightning isn't installed on the site, so that can't work for me. Even the #45 patch here didn't help.

please export your site configuration using drush cex --destination="/some/path/to/config/sync"

inside your config export will be a few files ending in *version.yml

inside those version files there'll be something about entityqueue

check what version your entityqueue is showing up in that version.yml it might explain why drush updb either didn't work or hasn't been working.

also check drush wd-show

check your reports logs if you've got dblog installed, if not install it, check for error messages.