Change record status: 
Project: 
Introduced in branch: 
8.0.x
Introduced in version: 
8.0.0
Description: 

Entity (database) schema is generated based on the entity type and base field storage definitions. When there was a schema change required, update.php had an automated system to fix the entity schema. However, this did not make the entity schema state predictable for module updates. It was not possible to write content updates based on known states of the entity schema. Therefore that magic was removed and now changes to the entity schema are to be included as regular update functions. (The user facing change is explained in https://www.drupal.org/node/2554101).

New API methods are available on the EntityDefinitionUpdateManagerInterface to be used in update functions:

  • getEntityType($entity_type_id)
  • installEntityType(EntityTypeInterface $entity_type)
  • updateEntityType(EntityTypeInterface $entity_type)
  • uninstallEntityType(EntityTypeInterface $entity_type)
  • getFieldStorageDefinition($name, $entity_type_id)
  • installFieldStorageDefinition($name, $entity_type_id, $provider, FieldStorageDefinitionInterface $storage_definition)
  • updateFieldStorageDefinition(FieldStorageDefinitionInterface $storage_definition)
  • uninstallFieldStorageDefinition(FieldStorageDefinitionInterface $storage_definition)

For each hook_update_N() that needs to add/remove/change entity types or add/remove/change field storage definitions, use these methods to apply the required changes.

When updating definitions, it's very important to retrieve the definitions to be updated from the update manager via ::getEntityType() and ::getFieldStorageDefinition(), as they allow to retrieve from state instances of the definitions ready to be manipulated. In fact when definitions change in code, the system needs to be notified about that and the definitions stored in state need to be reconciled with the ones living in code. Update API functions need to take the system from a known state to another known state: relying on the definitions living in code might prevent this, as the system might transition directly to the last available state, and thus skipping the intermediate steps. Manipulating the definitions in state allows to avoid this and ensures that the various steps of the update process are predictable and repeatable.

For example a new field being added to node entities:

/**
 * Add 'revision_translation_affected' field to 'node' entities.
 */
function node_update_8001() {
  // Install the definition that this field had in
  // \Drupal\node\Entity\Node::baseFieldDefinitions()
  // at the time that this update function was written. If/when code is
  // deployed that changes that definition, the corresponding module must
  // implement an update function that invokes
  // \Drupal::entityDefinitionUpdateManager()->updateFieldStorageDefinition()
  // with the new definition.
  $storage_definition = BaseFieldDefinition::create('boolean')
      ->setLabel(t('Revision translation affected'))
      ->setDescription(t('Indicates if the last edit of a translation belongs to current revision.'))
      ->setReadOnly(TRUE)
      ->setRevisionable(TRUE)
      ->setTranslatable(TRUE);

  \Drupal::entityDefinitionUpdateManager()
    ->installFieldStorageDefinition('revision_translation_affected', 'node', 'node', $storage_definition);
}

Existing fields promoted to entity keys:

/**
 * Promote 'status' and 'uid' fields to entity keys.
 */
function node_update_8003() {
  // The 'status' and 'uid' fields were added to the 'entity_keys' annotation
  // of \Drupal\node\Entity\Node in https://www.drupal.org/node/2498919, but
  // this update function wasn't added until
  // https://www.drupal.org/node/2542748. In between, sites could have
  // performed interim updates, which would have included automated entity
  // schema updates prior to that being removed (see that issue for details).
  // Therefore, we check for whether the keys have already been installed.
  $manager = \Drupal::entityDefinitionUpdateManager();
  $entity_type = $manager->getEntityType('node');
  $entity_keys = $entity_type->getKeys();
  $entity_keys['status'] = 'status';
  $entity_keys['uid'] = 'uid';
  $entity_type->set('entity_keys', $entity_keys);
  $manager->updateEntityType($entity_type);

  // @todo The above should be enough, since that is the only definition that
  //   changed. But \Drupal\Core\Entity\Sql\SqlContentEntityStorageSchema varies
  //   field schema by whether a field is an entity key, so invoke
  //   onFieldStorageDefinitionUpdate() with an unmodified
  //   $field_storage_definition to trigger the necessary changes.
  //   SqlContentEntityStorageSchema::onEntityTypeUpdate() should be fixed to
  //   automatically handle this.
  //   See https://www.drupal.org/node/2554245.
  foreach (array('status', 'uid') as $field_name) {
    $manager->updateFieldStorageDefinition($manager->getFieldStorageDefinition($field_name, 'node'));
  }
}


Changing cardinality of a field having existing data from single to multiple:

/**
 * Makes the 'user_id' field multiple and migrate its data.
 */
function entity_test_update_8001() {
  // To update the field schema we need to have no field data in the storage,
  // thus we retrieve it, delete it from storage, and write it back to the
  // storage after updating the schema.
  $database = \Drupal::database();

  // Retrieve existing field data.
  $user_ids = $database->select('entity_test', 'et')
    ->fields('et', ['id', 'user_id'])
    ->execute()
    ->fetchAllKeyed();

  // Remove data from the storage.
  $database->update('entity_test')
    ->fields(['user_id' => NULL])
    ->execute();

  // Update definitions and schema.
  $manager = \Drupal::entityDefinitionUpdateManager();
  $storage_definition = $manager->getFieldStorageDefinition('user_id', 'entity_test');
  $storage_definition->setCardinality(FieldStorageDefinitionInterface::CARDINALITY_UNLIMITED);
  $manager->updateFieldStorageDefinition($storage_definition);

  // Restore entity data in the new schema.
  $insert_query = $database->insert('entity_test__user_id')
    ->fields(['bundle', 'deleted', 'entity_id', 'revision_id', 'langcode', 'delta', 'user_id_target_id']);
  foreach ($user_ids as $id => $user_id) {
    $insert_query->values(['entity_test', 0, $id, $id, 'en', 0, $user_id]);
  }
  $insert_query->execute();
}

And vice-versa:

/**
 * Makes the 'user_id' field single and migrate its data.
 */
function entity_test_update_8002() {
  // To update the field schema we need to have no field data in the storage,
  // thus we retrieve it, delete it from storage, and write it back to the
  // storage after updating the schema.
  $database = \Drupal::database();

  // Retrieve existing entity data.
  $query = $database->select('entity_test__user_id', 'et')
    ->fields('et', ['entity_id', 'user_id_target_id']);
  $query->condition('et.delta', 0);
  $user_ids = $query->execute()->fetchAllKeyed();

  // Remove data from the storage.
  $database->truncate('entity_test__user_id')->execute();

  // Update definitions and schema.
  $manager = \Drupal::entityDefinitionUpdateManager();
  $storage_definition = $manager->getFieldStorageDefinition('user_id', 'entity_test');
  $storage_definition->setCardinality(1);
  $manager->updateFieldStorageDefinition($storage_definition);

  // Restore entity data in the new schema.
  foreach ($user_ids as $id => $user_id) {
    $database->update('entity_test')
      ->fields(['user_id' => $user_id])
      ->condition('id', $id)
      ->execute();
  }
}
Impacts: 
Module developers
Updates Done (doc team, etc.)
Online documentation: 
Not done
Theming guide: 
Not done
Module developer documentation: 
Not done
Examples project: 
Not done
Coder Review: 
Not done
Coder Upgrade: 
Not done
Other: 
Other updates done

Comments

wizonesolutions’s picture

Is the example of writing the data back in after changing the field schema a rare example of when to use the database service in Drupal 8? Is it to preserve IDs and such?

FillPDF Service - http://fillpdf-service.com - Hosted solution for FillPDF

plach’s picture

In update functions relying on the Entity CRUD API is dangerous because the schema may not be up to date, as the examples show. In this case relying on the low level storage service is the recommended approach.

Berdir’s picture

In case someone is looking for it, the interdiff in https://www.drupal.org/node/2641828#comment-10711058 is an example that converts the structure by hand while there is still data.

kristiaanvandeneynde’s picture

There's a discussion on how to do this taking place here: https://www.drupal.org/node/2715011.

FluxSauce’s picture

Based on some of the techniques described here, I've created a working example that changes the storage definition of a field that has existing content.

https://coderwall.com/p/uyidlq/updating-the-storage-definition-of-entiti...