How to change type of fields with existing data

I had the same issue when worked on my patch to number field. The solution is to fetch the field data and save it in some variable, then truncate the field table to unlock it for changes, then make all the changes and after that insert earlier saved data in the field. Works pretty good, but the only thing you need to not forget is PRIMARY KEY and may be INDEXES. They also should be dropped before making changes on the field type/settings.

Read how to do it in the docs for db_change_field() function. Or look into the real working example in field_update_8004() here:

#2816859: Allow the 'step' to be configured as a NumberWidget setting

The update adds the unsigned storage setting for any type number fields with existing data. You can adjust the code with the settings you need to change on the field. Note that the field_update_8004() also updates number fields on the entity_type revisions if they exist.

To check if the update worked as expected run this code:

  foreach (\Drupal\field\Entity\FieldStorageConfig::loadMultiple() as $field_storage_config) {
    $field_storage_config->save();
  }

If there is no exception thrown then you are lucky enough and may proceed with you updated fields.

I have posted a Drupal 8 code example here:

http://drupal.stackexchange.com/questions/208946/how-do-i-change-a-field...

Subscribing.

I managed to change field type from plain text (string) to formatted text (text). by forking @eric.chenchao's solution given in #4.

<?php

/**
 * Changing field_my_plaintext field from string(plain text) to text(formatted).
 */
function mymodule_update_8006() {

  $fields = [
    'field_my_plaintext' => [
      'table' => 'paragraph__field_my_plaintext',
      'revision_table' => 'paragraph_revision__field_my_plaintext',
      'format_col' => 'field_my_plaintext_format',
    ],
  ];

  $database = \Drupal::database();

  foreach ($fields as $field_name => $f) {
    $table = $f['table'];
    $revision_table = $f['revision_table'];
    // Entity type here.
    $entity_type = 'paragraph';

    // Step 1: Get field storage.
    $field_storage = FieldStorageConfig::loadByName($entity_type, $field_name);

    // Check if field not found.
    if (is_null($field_storage)) {
      continue;
    }

    // Step 2: Store data.
    $rows = NULL;
    $revision_rows = NULL;
    if ($database->schema()->tableExists($table)) {
      // The table data to restore after the update is completed.
      $rows = $database->select($table, 'n')->fields('n')->execute()
        ->fetchAll();
      $revision_rows = $database->select($revision_table, 'n')->fields('n')->execute()
        ->fetchAll();
    }

    // Step 3: Save new field configs & delete existing fields.
    $new_fields = array();
    foreach ($field_storage->getBundles() as $bundle => $label) {
      $field = FieldConfig::loadByName($entity_type, $bundle, $field_name);
      $new_field = $field->toArray();
      $new_field['field_type'] = 'text';
      $new_fields[] = $new_field;
      // Delete field.
      $field->delete();
    }

    // Step 4: Create new storage configs from existing.
    $new_field_storage = $field_storage->toArray();
    $new_field_storage['type'] = 'text';
    $new_field_storage['module'] = 'text';
    $new_field_storage['settings'] = [
      'max_length' => 255,
    ];

    // Step 5: Purge deleted fields data.
    // This is required to create new fields.
    field_purge_batch(250);

    // Step 6: Create new fieldstorage.
    FieldStorageConfig::create($new_field_storage)->save();

    // Step 7: Create new fields for all bundles.
    foreach ($new_fields as $new_field) {
      $new_field = FieldConfig::create($new_field);
      $new_field->save();
    }

    // Step 8: Restore existing data in fields & revision tables.
    if (!is_null($rows)) {
      foreach ($rows as $row) {
        $row = (array) $row;
        $row[$f['format_col']] = 'static_html';
        $database->insert($table)->fields($row)->execute();
      }
    }
    if (!is_null($revision_rows)) {
      foreach ($revision_rows as $row) {
        $row = (array) $row;
        $row[$f['format_col']] = 'static_html';
        $database->insert($revision_table)->fields($row)->execute();
      }
    }

  }

}

#6 did the trick for me. After I ran the update hooks locally I exported the configuration to code and this completed my fix. Our deploy steps ensured that the migration of content and configuration change could be handled in a single release.

drush updatedb -y
drush config-import -y

Caution: deleting the field in Step 3 meant that some blocks were broken on my site. I'm not sure exactly why, but it's possible the block was from a view that was no longer valid, or some other configuration that was referencing the field. I didn't look in to it further.

Instead, I've written an amended script that will truncate the tables so you can update the field settings. Not sure if this will work for converting from one field type to another, but it worked for increasing the precision of a decimal field. I've also added a try/catch for inserting the rows, because some of the data in my tables was out of range.

<?php

use \Drupal\field\Entity\FieldStorageConfig;

$fields = [
  'field_address_latitude' => [
    'table' => 'node__field_address_latitude',
    'revision_table' => 'node_revision__field_address_latitude',
  ],
  'field_address_longitude' => [
    'table' => 'node__field_address_longitude',
    'revision_table' => 'node_revision__field_address_longitude',
  ],
];

$database = \Drupal::database();

foreach ($fields as $field_name => $f) {
  $table = $f['table'];
  $revision_table = $f['revision_table'];
  // Entity type here.
  $entity_type = 'node';

  // Step 1: Get field storage.
  $field_storage = FieldStorageConfig::loadByName($entity_type, $field_name);
  // Check if field not found.
  if (is_null($field_storage)) {
    continue;
  }

  // Step 2: Store data.
  $rows = NULL;
  $revision_rows = NULL;
  if ($database->schema()->tableExists($table)) {
    // The table data to restore after the update is completed.
    $rows = $database->select($table, 'n')->fields('n')->execute()
      ->fetchAll();
    $revision_rows = $database->select($revision_table, 'n')
      ->fields('n')
      ->execute()
      ->fetchAll();
  }

  // Step 3: Empty the tables.
  db_truncate($table)->execute();
  db_truncate($revision_table)->execute();

  // Step 4: Update the storage config.
  $field_storage->set('settings', ['precision' => 18, 'scale' => 12]);
  $field_storage->save();

  // Step 5: Restore existing data in fields & revision tables.
  if (!is_null($rows)) {
    foreach ($rows as $row) {
      $row = (array) $row;
      try {
        $database->insert($table)->fields($row)->execute();
      }
      catch (Exception $exception) {
        watchdog_exception('MYMODULE', $exception);
      }
    }
  }
  if (!is_null($revision_rows)) {
    foreach ($revision_rows as $row) {
      $row = (array) $row;
      try {
        $database->insert($revision_table)->fields($row)->execute();
      }
      catch (Exception $exception) {
        watchdog_exception('MYMODULE', $exception);
      }
    }
  }
}
?>

Use at your own risk, obviously.

@mstrelan

Basically, you did the same as on the patch from #3 comment.

function field_update_8004() {
  $i = 0;
  $all = [];
  $config = \Drupal::configFactory();
  $key_value = \Drupal::keyValue('entity.definitions.installed');
  $database = \Drupal::database();
  $manager = \Drupal::entityDefinitionUpdateManager();

  foreach ($config->listAll('field.') as $field_id) {
    $data = $config->getEditable($field_id)->getRawData();
    $type = isset($data['field_type']) ? $data['field_type'] : NULL;
    if ($type == 'decimal' || $type == 'float') {
      $entity_type = $data['entity_type'];
      $field_name = $data['field_name'];
      $storage_definition = $manager->getFieldStorageDefinition($field_name, $entity_type);
      $settings = $storage_definition->getSettings();
      $settings['unsigned'] = isset($settings['unsigned']) ? $settings['unsigned'] : FALSE;
      $schema = $key_value->get("{$entity_type}.field_storage_definitions")[$field_name]->getSchema();
      $schema['columns']['value']['not null'] = TRUE;
      $schema['columns']['value']['unsigned'] = FALSE;
      $keys = [
        'primary key' => [
          'entity_id',
          'deleted',
          'delta',
          'langcode',
        ],
      ];

      $tables = ["{$entity_type}_revision__{$field_name}", "{$entity_type}__{$field_name}"];
      $column = "{$field_name}_value";

      foreach ($tables as $table) {
        if (array_key_exists($table, $all)) {
          continue 2;
        }
        if ($database->schema()->tableExists($table)) {
          // The table data to restore after the update is completed.
          $all[$table] = $database->select($table, 'n')
            ->fields('n')
            ->execute()
            ->fetchAll();

          // Truncate the number field table to unlock it for changes.
          $database
            ->truncate($table)
            ->execute();

          db_drop_primary_key($table);

          // Assign all the new schema definitions for the field value.
          $database
            ->schema()
            ->changeField($table, $column, $column, $schema['columns']['value'], $keys);
        }
      }
      $storage_definition
        ->setSetting('unsigned', $settings['unsigned'])
        ->save();
    }
  }

  // Restore earlier saved number fields data.
  foreach ($all as $table => $rows) {
    foreach ($rows as $row) {
      $database->insert($table)
        ->fields((array) $row)
        ->execute();
    }
    $i++;
  }

  return t('The @i number fields were updated.', ['@i' => $i]);
}

See: #2816859: Allow the 'step' to be configured as a NumberWidget setting

I'm trying to update a content type to move a field from text to decimal (to allow proper sorting). I've emptied the tables manually and via phpmyadmin truncate(), but an import of the revised configuration still fails with the same error. Is a PHP hook the only way to accomplish this change (that is, can it not be accomplished via a configuration import)? If so, what is required to run the patch as native PHP? (I'm familiar with PHP, but new to drupal).

Thanks,

Jack

@sbs-jms

You can install https://www.drupal.org/project/devel module and then open http://example.com/devel/php page and paste in the text field your custom PHP code and press Execute button. The code will be executed the same way if it was in the file.php. For example, take the function on the #10, modify it for your needs then do this in the textfield:

function field_update_8004() {
  $i = 0;
  $all = [];
  $config = \Drupal::configFactory();
  $key_value = \Drupal::keyValue('entity.definitions.installed');
  $database = \Drupal::database();
  $manager = \Drupal::entityDefinitionUpdateManager();
// ...
}

field_update_8004();

After creating the new fields and deleting the old ones my form and view display modes are changed and the new fields are set to disabled. Is there a way to preserve the position of the field in the display modes?

To answer myself:

Add a loop in Step 3 to gather all the weights of the field:

      // Save the weights in view and form modes
      foreach (array_keys($view_modes) as $view_mode) {
        $view_display = \Drupal::entityTypeManager()
          ->getStorage('entity_view_display')
          ->load($entity_type . '.' . $bundle . '.' . $view_mode)
          ->getComponent($field_name);
        $weights['entity_view_display'][$entity_type . '.' . $bundle . '.' . $view_mode] = $view_display['weight'];
      }
      $form_display = \Drupal::entityTypeManager()
        ->getStorage('entity_form_display')
        ->load($entity_type . '.' . $bundle . '.default')
        ->getComponent($field_name);
      $weights['entity_form_display'][$entity_type . '.' . $bundle . '.default'] = $form_display['weight'];

Add another loop as Step 9 to set the weights

    // Step 9 Reset the positions in the view and form display modes
    foreach ($weights as $display_mode => $view_modes) {
      foreach ($view_modes as $view_mode => $weight) {
        if ($weight) {
          $display = \Drupal::entityTypeManager()
            ->getStorage($display_mode)
            ->load($view_mode)
            ->setComponent($field_name, [
              'weight' => $weight
            ])
            ->save();
        }
      }
    }

#937442: Field type modules cannot maintain their field schema (field type schema change C(R)UD is needed)

/**
 * Update the length of a text field which already contains data.
 *
 * @param string $entity_type_id
 * @param string $field_name
 * @param integer $new_length
 */
change_every_field_full () {

$entity_type_id='node';
$field_name = 'your_field_name';
$new_length ='100';
  $name = 'field.storage.' . $entity_type_id . "." . $field_name;

  // Get the current settings
  $result = \Drupal::database()->query(
    'SELECT data FROM {config} WHERE name = :name',
    [':name' => $name]
  )->fetchField();
  $data = unserialize($result);
  $data['settings']['max_length'] = $new_length;

  // Write settings back to the database.
  \Drupal::database()->update('config')
    ->fields(['data' => serialize($data)])
    ->condition('name', $name)
    ->execute();

  // Update the value column in both the _data and _revision tables for the field
  $table = $entity_type_id . "__" . $field_name;
  $table_revision = $entity_type_id . "_revision__" . $field_name;
  $new_field = ['type' => 'varchar', 'length' => $new_length]; //setting your type example text, int .....
  $col_name = $field_name . '_value';
  \Drupal::database()->schema()->changeField($table, $col_name, $col_name, $new_field);
  \Drupal::database()->schema()->changeField($table_revision, $col_name, $col_name, $new_field);

  // Flush the caches.
  drupal_flush_all_caches();
}

I used an adaptation of #6 to convert a populated field from string to text_long.

When the field is deleted and then recreated, it is removed from all other contexts - so, as mentioned in #9, this will break all blocks where this field is used.

D8 makes it slightly easier to solve for this by using config export, config import and source control.

- After the drush updb has finished execute drush cex
- Revert all config apart from the fields in question
- Run drush cim

This has worked successfully for me, YMMV.

@JayKandari #6 thanks for your reference code.

I have different requirement,
I have a number(integer) field under a paragraph and i need to change field into number(decimal)
Can you share same code as you did for plain text to plain formatted under comment #6.

My current code.
/**
* Change field_fee_card_late_fee field from int to decimal(float).
*/
function nipr_post_blocks_update_8009() {

$fields = [
'field_fee_card_late_fee' => [
'table' => 'paragraph__field_fee_card_late_fee',
'revision_table' => 'paragraph_revision__field_fee_card_late_fee',
'format_col' => 'field_fee_card_late_fee_format',
],
];

$database = \Drupal::database();

foreach ($fields as $field_name => $f) {
$table = $f['table'];
$revision_table = $f['revision_table'];
// Entity type here.
$entity_type = 'paragraph';

// Step 1: Get field storage.
$field_storage = FieldStorageConfig::loadByName($entity_type, $field_name);

// Check if field not found.
if (is_null($field_storage)) {
continue;
}

// Step 2: Store data.
$rows = NULL;
$revision_rows = NULL;
if ($database->schema()->tableExists($table)) {
// The table data to restore after the update is completed.
$rows = $database->select($table, 'n')->fields('n')->execute()
->fetchAll();
$revision_rows = $database->select($revision_table, 'n')->fields('n')->execute()
->fetchAll();
}

// Step 3: Save new field configs & delete existing fields.
$new_fields = array();
foreach ($field_storage->getBundles() as $bundle => $label) {
$field = FieldConfig::loadByName($entity_type, $bundle, $field_name);
$new_field = $field->toArray();
$new_field['field_type'] = 'numeric';
$new_fields[] = $new_field;
// Delete field.
$field->delete();
}

// Step 4: Create new storage configs from existing.
$new_field_storage = $field_storage->toArray();
$new_field_storage['type'] = 'number_decimal';
$new_field_storage['module'] = 'numeric';
$new_field_storage['settings'] = [
'precision' => 10,
'scale' => 2,
'not null' => FALSE,
];

// Step 5: Purge deleted fields data.
// This is required to create new fields.
// field_purge_batch(250);

// Step 6: Create new fieldstorage.
FieldStorageConfig::create($new_field_storage)->save();

// Step 7: Create new fields for all bundles.
foreach ($new_fields as $new_field) {
$new_field = FieldConfig::create($new_field);
$new_field->save();
}

//Step 8: Restore existing data in fields & revision tables.
if (!is_null($rows)) {
foreach ($rows as $row) {
$row = (array) $row;
// $row[$f['format_col']] = 'static_html';
$database->insert($table)->fields($row)->execute();
}
}
if (!is_null($revision_rows)) {
foreach ($revision_rows as $row) {
$row = (array) $row;
// $row[$f['format_col']] = 'static_html';
$database->insert($revision_table)->fields($row)->execute();
}
}

}

}

thanks in advance

@JayKandari #6,

I want to change a number( integer) field to number(decimal) in a paragraph. Kindly share reference code.

There is a module that can help with this issue:
Change Field Type
https://www.drupal.org/project/change_field_type
Now - it changes number types to other number types (like changing Float to Decimal), there may be more functionality added in subsequent updates.

See https://www.drupal.org/docs/drupal-apis/update-api/updating-entities-and... for best practice.

Also please help to create a full-featured Contrib Module with UI and API (service) to use for such tasks: https://www.drupal.org/project/issues/change_field_type

I'm sure the Drupal community needs that functionality, help with coding and sponsorship!

If others should run into the same issue, please see my comment #26 above for finding help. Also please help to create better support for such tasks in core (and contrib, see #26)

In #80 of #937442: Field type modules cannot maintain their field schema (field type schema change C(R)UD is needed) I started a discussion how to add a helper in core to make such field type conversions easier.
This is based on the "Closed (duplicate)" by @dpi in #17 here and the point that this is only one part of multiple core utilities needed for fields (and especially field updates) in #937442: Field type modules cannot maintain their field schema (field type schema change C(R)UD is needed).