The 2026 Drupal Business Survey is open. Your response will help shape the future of the Drupal business ecosystem.We are developing in a team environment with the main DEV site hosted on Pantheon. We (devs) all do everything on our local machines, and use the Configuration Synchronization to push to DEV, then the other devs pull those changes.
The biggest issue we've run into is when we create fields locally, and that change gets cascaded to DEV and other local environs - then subsequently delete a field. I can't pinpoint the exact circumstances, but we regularly get errors when pushing the config changes to DEV after deleting a field stating that the table for the field couldn't be renamed to the field_deleted_data_xxxxx{long-hash-string} because the table already exists. And, to be clear, we don't do any config changes on DEV directly, so that's not the reason for things getting out of sync.
The way I understand it, after deleting a field, there's a cron job that deletes the data contained therein (in relatively small batches to keep from hitting the php timeout), and when the field_deleted_data_xxxxxx table is empty, it is deleted (dropped) as well. That process is not working. This is currently a dev site, which means we rarely have more than 2 or 3 nodes for any given content type. After deleting a field, the data should be deleted during the first cron run. However, I've run it sometimes a dozen times or more, yet the field_deleted_data_xxxxx table(s) remain.
I've marked this critical because, when this situation happens (which it does all too frequently), fixing it is difficult and time-consuming.
| Comment | File | Size | Author |
|---|
Issue fork drupal-2836539
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
changes
Comments
Comment #2
jonathanshawPriorities are described here:
https://www.drupal.org/node/45111
Being time-consuming is not a criterion for 'critical'.
Comment #3
swentel commentedBeen trying to reproduce this, but can't so far.
Does the data remain as well ? If so, then somehow the field.field.deleted doesn't contain the field that needs to be deleted. Could you check field.field.deleted in the key/value storage to see if it's there and what it contains ? And if by any chance it's a field on an entity type that doesn't exist anymore (probably not, but just making sure)
Comment #4
extexan commented@swentel, I had a need to delete a field today, so I tried to document the entire process.
Before deleting the field, I checked the Db…
The node__field_po_sequence table (and its revision table) existed.
An entry for that field existed in the config table as:
field.field.node.sales_contract.field_po_sequence
Sorry, I misread your post at first, so I wasn't looking for the correct key when I looked in the key_value table. I don't know if the field.field.deleted entry existed before I deleted *this* field. I suspect that it did. Farther down, I've posted the current value of that key (after deleting this field).
I then went to /admin/structure/types/manage/sales_contract/fields and deleted that field. I got this error:
.
Note: This is different than the error I was getting previously. It has happened on the two most recent attempts to delete a field. Prior to that, I could delete the field fine on my local machine, but when I exported the configuration, pushed it to Pantheon DEV, and ran the config import there, I would get an error saying it couldn’t rename the field table to the deleted data table because it already existed.
I then checked the Db again…
The node__field_po_sequence table (and its revision table) no longer existed.
The entry for that field in the config table was gone.
Two deleted field tables now exist. Both tables are empty.
field_deleted_data_b870a627e4
field_deleted_revision_b870a627e4
Note that the hash(?) string in the tables above doesn’t match the one in the error message, which is c968a7bda4.
There was a field.field.deleted entry in the key_value table, with this value:
.
I ran cron 10 times, checking for the existence of the field_deleted… tables after each run. They were never deleted.
I don’t know where it gets the string it uses in the deleted data table names, but it seems something is out of sync with that. Any help correcting this issue would be greatly appreciated.
Comment #5
extexan commented@swentel, re: all the data in my field.field.deleted entry... does all that need to be cleared out?
I've continued to get errors when trying to delete fields - though now the error has changed. Previously, it consistently errored out when renaming the table because the field_deleted_data_xxxxxxx table already existed. Now (just as consistently), it tells me a particular field_deleted_data_xxxxxxx DOESN'T exist - just the opposite from before.
Comment #7
cilefen commentedComment #8
extexan commented@cilefen, sorry to say, I have no idea how to reproduce this condition. I suspect that it may be unique to our setup - which is to say, something may have happened early on in the development process (when we were doing lots of syncs and a significant number of field deletions) that has created orphaned or inconsistent data in my/our key_value table(s).
At this point, I'm hoping for assistance in getting the problem data cleaned up so that the errors go away. Then, if it comes back, perhaps we can pinpoint how/why it's happening.
Any help would be greatly appreciated.
Comment #9
swentel commentedlinking #2622160: EntityStorageException after field delete and reimport - going to look at this over the weekend
Comment #10
swentel commentedSo, some good news: I can reproduce 'field_deleted_data_xxx' tables not being deleted after I did a sync which deleted fields and then running cron. So I probably will be able to write a test for this and prove that their are not gone, don't have fix yet though. Stay tuned!
Comment #11
swentel commentedactually, scratch my previous comment, I can't .. so still in the dark here
Comment #12
swentel commented@ExTexan so yes, removing field.field.deleted and field.storage.deleted and then removing the field_deleted_data tables is perfectly ok.
Still haven't been able to reproduce this .. importing, exporting, running cron, without and with data .. all is nicely cleaned up.
Comment #13
extexan commented@swentel, Thanks for letting me know which entries in key_value need to be deleted. I will try that and see if my lingering errors go away.
Comment #14
extexan commentedAn update on my issue...
I deleted a field today. Before deleting it, I looked for field.field.deleted and field.storage.deleted entries in the key_value table. There were none that began with "field" at all. I went ahead with the delete, and got this error:
Comment #15
tacituseu commentedFor those with the same problem, try:
then, to check do:
it will trip if there are more broken fields, rinse and repeat.
Comment #16
joelpittetRelated and may help a bit: #2862308: Log a message when skipping deleted fields during field_purge_batch()
Comment #17
extexan commented@joelpittet, thanks for that link.
In my case, however, all fields that I/we have ever deleted are of types provided by Drupal core, not by contrib modules.
Comment #18
joelpittet@ExTexan I also had this with core's comment module which I uninstalled.
Comment #19
coecks commentedThis may be related. Whenever I create a new content type and delete the body which was added by default, I get a white screen with a message in black letters: "Something went wrong, try again later". In the logs there is this message
Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'field_deleted_data_bbeef2ead6' doesn't exist: SELECT DISTINCT t.entity_id AS entity_id FROM {field_deleted_data_bbeef2ead6} t WHERE bundle = :db_condition_placeholder_0 LIMIT 10 OFFSET 0; Array ( [:db_condition_placeholder_0] => article ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->readFieldItemsToPurge() (regel 1553 van /vagrant/deblauwehond/web/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).
FYI, I deleted all default content types provided by the Standard installation profile.
Comment #21
scuba_flyThis may be related: I got the following error after running a `drush entup`
Saying the table can not be renamed because it already exists.
Comment #22
lukasss commentedI have the same problem, for example when uninstall module commerce_product (entity type commerce_product has default fields such as body, commerce_store). Drupal 8.4
Comment #24
puregin commentedHere is a dynamic MySQL query which may help clean up these unwanted tables:
You may need to modify this if you have a table prefix or something else going on.
Hope this helps.
Comment #25
alan d. commentedAgain, probably difficult to replicate, but we hit this today uninstalling Feature modules today.
The issue appeared to escalated somehow when the process appeared to delete the fields twice(?), with an exception on the second attempt blocking the uninstall as Drupal tried to create a second table with the same name for each field.
Workaround seemed to be to delete via Field UI (painfully, there were a lot of fields), then to uninstall the modules that defined these fields.I had about 25 fields without UUID set. I had to delete individual fields, remove the tables, delete the next field, ....
However, with the forced entity delete, why does Drupal even do this now? Surely there is no overhead deleting an empty table!!
[edit]
\Drupal\field\Entity\FieldStorageConfig::getUniqueStorageIdentifier() depends on the UUID, and the problematic fields didn't have one set so all had the same table name. i.e. hash('sha256', NULL) will always be the same
Drupal\Core\Entity\Sql\DefaultTableMapping
Fields created on D8.3.x using Features. Maybe this will help seeing if this is a common denominator to the issue and guide people into the root cause :)
Comment #26
acbramley commentedI've been pointed to this issue by @amateescu as I may have found the cause of what is going wrong here.
Here's what I found:
At first, I noticed that deleting a field via the UI did not purge any of the entity_type__field_my_field table data or tables. All data remained, and the tables remained as well, even after running cron.
Debugging through this here's what happens:
1) FieldConfig::preDelete adds the field definition to the
entity_field.deleted_fields_repository2) FieldConfigDeleteForm then calls field_purge_batch()
3) field_purge_batch() calls ContentEntityStorageBase::purgeFieldData
4) purgeFieldData calls SqlContentEntityStorage::purgeFieldItems
This is where the data is meant to be purged, but it isn't due to the fact that none of my data in node__field_x or node_revision__field_x have the deleted flag set to 1
5) field_purge_batch() then calls field_purge_field() since $count_purged == 0 and the field definition is removed from
entity_field.deleted_fields_repository.6) Running cron won't do anything at this point as the field definition is no longer in the deleted fields repository.
What I'd expect to happen would be:
1) The data and tables are removed
2) The FieldStorageConfig entity is also removed since this is the only bundle that uses it
But neither of those things happen. I may be wrong in my assumptions, but it does seem to be that the data should be purged at this point.
But wait there's more - It gets even worse when deleting fields via config import!
This may be specific to how we delete config in drush_cmi_tools but I think it's worth noting:
1) None of the above even happens since FieldConfig::preDelete never fires.
2) FieldDefinitionListener::onFieldDefinitionDelete also doesn't fire either which means the
entity.definitions.bundle_field_mapis never updated, leading to persistent notices like:A non-existent config entity name returned by FieldStorageConfigInterface::getBundles()Comment #29
ro-no-lo commentedThis issue seems still to exist.
Drupal\Core\Database\DatabaseExceptionWrapper: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'foobar_live.field_deleted_data_cdadb62988' doesn't exist: SELECT DISTINCT t.entity_id AS entity_id FROM {field_deleted_data_cdadb62988} t WHERE bundle = :db_condition_placeholder_0 LIMIT 50 OFFSET 0; Array ( [:db_condition_placeholder_0] => ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->readFieldItemsToPurge() (Zeile 1558 in /var/www/xxx/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).Just saying. No idea what cases it. We are two developers which use config export and import to sync with live and staging. Therefore it must be happend in doing so.
But as far as I understand I can delete these tables manually without problems (?).
Comment #30
pritam.tiwari commentedI have also faced same issue. While cron run with drush. I have created one dummy empty table just for the requirement. Tried to re-run the cron again. I successfully resolved with this temporary solution. May be it could help someone in urgency case. (Replace here your table name in following query.)
// Dummy table structure same like other tables.
CREATE TABLE `field_deleted_data_97b03c8f15` (
`bundle` varchar(128) CHARACTER SET ascii NOT NULL DEFAULT '' COMMENT 'test',
`deleted` tinyint(4) NOT NULL DEFAULT '0' COMMENT 'A boolean indicating whether this data item has been deleted',
`entity_id` int(10) unsigned NOT NULL COMMENT 'The entity id this data is attached to',
`revision_id` int(10) unsigned NOT NULL COMMENT 'The entity revision id this data is attached to',
`langcode` varchar(32) CHARACTER SET ascii NOT NULL DEFAULT '' COMMENT 'The language code for this data item.',
`delta` int(10) unsigned NOT NULL COMMENT 'The sequence number for this data item, used for multi-value fields',
`test` varchar(2048) DEFAULT NULL COMMENT 'test',
PRIMARY KEY(`entity_id`,`deleted`,`delta`,`langcode`),
KEY `bundle` (`bundle`),
KEY `revision_id` (`revision_id`),) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COMMENT='test';
Comment #31
Lloyd_87 commented#30 Really helped me, was having same issue
Comment #32
andzejsw commentedTo fix this, i used #30 query:
+ after that runed command:
drush cronComment #34
maximpodorov commentedWhat about checking of the table existence?
Comment #36
dom. commentedAside question (for personnal learning): what is the purpose of moving the field data to field_deleted_data_xxx table and delete it on next CRON rather than actuall deletion on time ?
Comment #37
rgpublic#34 is really a life-saver! I wish I had discovered this earlier. Before, I always manually recreated the tables it was complaining about which is really unpleasant tedious work. With those 3 lines, finally the purge job is completed. I think they should be included by default. I also don't know under which unfortunate circumstances this happens, but I'm working with lots of different Drupal sites, and time and again I encounter this bug. Fields are not purged anymore and there's no easy way out until you add those 3 lines.
Comment #38
simeHere is a test that shows the issue where a
_deleted_table has been removed unexpectedly. I'm not sure if it replicates the situation described by @acbramley in #26.The patch in #34 makes this test pass, but it should probably come with a logging warning? It is unexpected state and it should be reported. I'm putting it back to Needs Work.
Comment #39
simeAnd re-reading the issue summary I see there are two issues here. In the IS:
The test in #38 (or the patch in #34) do not cover the IS.
Comment #40
berdirThere are cases where that can happen that we probably aren't interested in fixing. For example, if you delete a field on a development/staging site, then sync the database from production without overwriting extra tables then you end up with those deleted tables that Drupal doesn't know about.
Unless we want to implement code that would scan for such tables that we don't know about and deletes them, which sounds scary to me.
#34 seems like a useful fix to get committed as it's ugly to fix that by hand if you run into that situation. Can you upload a combined patch? We could still decide to split the other thing off into a non-major issue, because breaking cron with that error is major, some extra leftover tables in the database that don't break anything isn't.
Comment #42
guillaumeduveauI just had the same problem when syncing my local DB form the production DB, local containing config changes such as field deletions. After a while I tried to drop my local DB and import it from a prod dump, instead of using drush sql-sync. Problem gone.
Do you think it could be that drush sql-sync does not drop tables that are not in the source DB ?
Importing the config from local on prod should go well but I'm not so sure that this was the correct "fix".
The field_deleted_data* and field_deleted_revision* remain, though.
Comment #43
extexan commentedIn our case, we never sync databases that already have existing tables. We simply wouldn't do that going from local (dev) to production, especially as production is on Pantheon.
Going the other way, we always start with a completely empty DB, then import a downloaded SQL dump from Live.
I don't think that is the cause of this issue.
Comment #44
shashwat purav commentedAdded a combined patch for the patches from #34 and #38 as suggested by @Berdir in #40
Hi @Berdir/ @sime, please suggest regarding the warning that we could log for #34
Comment #45
shashwat purav commentedAdded patch for the warning to log and interdiff with #44
Comment #54
gorkagr commentedHi!
Somehow i had this issue in one of my localhost while running cron, the same error has appeared in my "recent log messages" page.
Moved patch #44 to a MR
Best
Comment #55
malcomio commentedI've observed this in the following scenario:On a site running Drupal core 10.4.1, our local build process imports a sanitised database backup, which does not contain any field_deleted_data tables.After the import it runs the update hooks.One of our update hooks calls
\Drupal::service('module_installer')->uninstall()to uninstall a custom module.That module's hook_uninstall implementation has an array of taxonomy vocabulary names, and deletes them in a loop:foreach ($taxonomy as $vid) {
$vocabulary = Vocabulary::load($vid);
$vocabulary?->delete();
}
The exception is thrown as part of this.The strange thing is that this doesn't seem to be happening on our test environments, which use MySQL 5.7.40-log, or on local development machines using Windows, where we also use MySQL 5.7.
The problem only seems to be reproducible on my M1 Mac, where I'm using mysql 8.0.38.
The difference is because there didn't seem to be an ARM compatible Docker image of 5.7 when I was setting up.
But this may be a clue to what is triggering the problem - is it only reproducible on MySQL 8?
The patch from #44 didn't seem to make any difference to my problem.Updated: discovered that
drush sql:dropin our build process is not working as expected due to Docker TLS/SSL problems - replacing that with the raw MySQL commands solved my problem.Comment #57
jastraat commentedUpdated the MR to include logging without injecting the logger. There were pre-existing calls to the logger service for exceptions so this was consistent with the existing code. Also updated the test for the latest Drupal.
Comment #58
smustgrave commentedGave a super light review but issue summary appears incomplete.
Comment #59
acbramley commentedI think this should be PMNMI as we don't have steps to reproduce the bug.
Comment #60
smustgrave commented@jastraat can you provide steps by chance?
Comment #61
jastraat commentedI would if I could! Unfortunately we inherited a code base with this patch added in 2020 - long before we joined the project. And there's no associated ticket or other documentation explaining what set of circumstances made them add it.
Comment #62
andypostI faced it few years ago when attempted to uninstall the contact_storage module but I bet the issue is related to #2763281: Undeleted field(storage) stays in state or #3039906: Properly uninstall entity types
Comment #63
andypostComment #65
smustgrave commentedI've definitely experienced this before but also don't know why. I'm hesitant to close it..
Comment #66
smustgrave commentedSo what do others think we should do here?