I ran into a problem where field_purge_batch() was called during cron, and expecting to delete all data related to field_xyz. The problem was that the table field_data_field_xyz did not exist on my site, so an exception was thrown.

Here are some symptoms:

(1) field_purge_batch(10) returns an exception

(2) field_read_instances(array('deleted' => 1), array('include_deleted' => 1)) returns field_xyz, but field_xyz's corresponding field_data_field_xyz table does not exist.

(3)

$ drush cron;

WD cron: PDOException: SQLSTATE[42S02]: Base table or view not found:[error]
1146 Table 'field_data_field_xyz' doesn't exist:
SELECT DISTINCT field_data_field_xyz0.entity_type AS
entity_type, field_data_field_xyz0.entity_id AS
entity_id, field_data_field_xyz0.revision_id AS
revision_id, field_data_field_xyz0.bundle AS bundle
FROM 
{field_data_field_xyz}
field_data_field_xyz0
WHERE  (field_data_field_xyz0.deleted =
:db_condition_placeholder_0) AND
(field_data_field_xyz0.bundle =
:db_condition_placeholder_1) 
LIMIT 10 OFFSET 0; Array
(
    [:db_condition_placeholder_0] => 1
    [:db_condition_placeholder_1] => stm_tarif_page
)
 in field_sql_storage_field_storage_query() (line 585 of
/path/to/drupal/modules/field/modules/field_sql_storage/field_sql_storage.module).


Table 'stm.field_data_field_stm_tarifs_lignes' doesn't exist

I have no idea why field_data_field_xyz did not exist, perhaps because of an interrupted previous cleanup attempt, but at this point if the table does not exist, might it be OK to simply assume that there is no data?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

alberto56’s picture

Status: Active » Needs review
FileSize
1.28 KB
alberto56’s picture

The above test just makes sure the table exists before attempting to delete its data, which prevents field_purge_batch(10) from throwing an exception during cleanup.

Status: Needs review » Needs work
finne’s picture

Version: 7.x-dev » 8.4.x-dev

I ran into this same issue in the current 8.x versions of Drupal.
People might delete some of these tables by hand or using another method. If these tables are missing purging deleted tables will fail with an SQL exception. I would propose that we be more lenient and try to continue after a delete call to a missing deleted field table.

See also related issues https://www.drupal.org/node/2862308 , https://www.drupal.org/node/2836539 and https://www.drupal.org/node/2622160 .

finne’s picture

Here's a patch that checks for missing field tables if the field is marked deleted and logs an error in the watchdog and continues afterwards.

finne’s picture

Status: Needs work » Needs review

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

ranjith_kumar_k_u’s picture

Re-rolled the last patch for 9.3

quietone’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

I wonder if this is actually a bug.

This will need a test, adding tag. I though it would be easy to add a test to the existing one but I was not able to find an existing test for purging the field tables.

  1. +++ b/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php
    @@ -1624,6 +1624,13 @@ protected function readFieldItemsToPurge(FieldDefinitionInterface $field_definit
    +    // If the field is deleted and the table has been removed from the db.
    ...
    +      // Warn and nothing more to do.
    

    This can be combined to one comment at the beginning of the if block. Maybe something like this,
    '//If the table does not exist just log a message.'

  2. +++ b/core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php
    @@ -1624,6 +1624,13 @@ protected function readFieldItemsToPurge(FieldDefinitionInterface $field_definit
    +      \Drupal::logger('entity')->error($this->t('The table @name was not found when reading field items to purge.', ['@name' => $table_name]));
    

    Is this an error or a warning?

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

nagy.balint’s picture

FileSize
2.21 KB

It seems that it is also possible that the storage itself is missing and getFieldStorageDefinition throws a FieldException

So I added an extra try catch block there, as if there is no storage, then also nothing to do I guess.

for now the patch is for 9.5.

ricardofaria’s picture

Adding a patch for drupal 10.1.x

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mglaman’s picture

Title: When purging a batch of deleted Field API data, don't fail if the field_data... table does not exist » When purging a field definition, don't fail if field storage no longer exists

This happened to me where the field storage was purged even though the deleted fields repository was tracking a field definition. I don't know the root cause. It may be due to cron running at the same time in the background and processing field_cron while the UI is also handling the field purge as well.

There is no lock used for this process to prevent duplicate purge runs on specific fields.

Edit: Example of adding a lock

  foreach ($fields as $field) {

    // Check if we can lock this field. If not, something else is purging this field.
    $lock_id = "field_purge:{$field->getUniqueIdentifier()}";
    if (!\Drupal::lock()->acquire($lock_id)) {
      continue;
    }
    $entity_type = $field->getTargetEntityTypeId();

    $count_purged = \Drupal::entityTypeManager()->getStorage($entity_type)->purgeFieldData($field, $batch_size);
    if ($count_purged < $batch_size || $count_purged == 0) {
      // No field data remains for the field, so we can remove it.
      field_purge_field($field);
    }

     // Release the lock.
    \Drupal::lock()->release($lock_id);

    $batch_size -= $count_purged;
    // Only delete up to the maximum number of records.
    if ($batch_size == 0) {
      break;
    }
  }
andypost’s picture