I have a field type in which the schema can be edited by the user in the UI. They can add or remove columns to the field schema, which means there also may be indexes that existed in my hook_field_schema() at one point when the field is created, but later does not exist. The problem I'm experiencing is that when field_update_field() is called which should drop the old field table and re-create with the current schema, it loads the 'old' indexes from the field_read_field() stored in the database. This causes a database exception because field_sql_storage attempts to create an index on a table column that does not exist.

It seems that field_create/update_field() supports passing in additional indexes that are not defined in hook_field_schema(), which frankly seems like a very bad idea.

This will need to be verified if this also is present on Drupal 8.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Dave Reid’s picture

Status: Active » Needs review
FileSize
3.34 KB

Patch to confirm the failure.

Status: Needs review » Needs work

The last submitted patch, 1: 2311095-field-update-remove-index.patch, failed testing.

Dave Reid’s picture

Excellent, this demonstrates the problem exactly.

This is so confusing *why* we support allowing arbitrary indexes to be added to a field that aren't registered with the field's hook_field_schema(). Also it seems completely wrong that we don't separate it into a separate property in the field definition array like 'custom indexes'. It makes it *completely* impossible to determine which indexes are custom, and which indexes are from the field, and when an index should no longer stay around, which is the problem here.

Dave Reid’s picture

Status: Needs work » Needs review
FileSize
6.17 KB

We just need to add a new 'custom indexes' property to the field array.

Status: Needs review » Needs work

The last submitted patch, 4: 2311095-field-update-remove-index.patch, failed testing.

Dave Reid’s picture

Version: 7.x-dev » 8.x-dev
Status: Needs work » Needs review
FileSize
4.18 KB

Looks like this might affect D8 as well. Try a test-only version first.

Dave Reid’s picture

Version: 8.x-dev » 8.0.x-dev

Status: Needs review » Needs work

The last submitted patch, 6: 2311095-field-update-remove-index-test-only.patch, failed testing.

Dave Reid’s picture

Priority: Normal » Major
Issue tags: +Contributed project blocker, +Field API

This blocks multifield from being ported to D8.

Dave Reid’s picture

So frustrated that both D7 and D8 store raw data from hook_field_schema() in the field definition instead of "custom things that override the schema in particular cases."

Dave Reid’s picture

Issue tags: +Field schema
catch’s picture

Status: Needs work » Postponed (maintainer needs more info)

This has likely been fixed by #2555665: When index is added for content_translation_uid, the corresponding stored schema definition is not updated and related issues in the past six months which massively overhauled entity/field schema handling.

If it's still an issue it needs re-confirming.

Status: Postponed (maintainer needs more info) » Needs review
Dave Reid’s picture

Status: Needs review » Needs work
Issue tags: +Needs backport to D7

Still an issue, test is failing.

The last submitted patch, 6: 2311095-field-update-remove-index-test-only.patch, failed testing.

Dave Reid’s picture

Status: Needs work » Needs review
FileSize
3.94 KB

Re-rolling the test to see if it fails.

Status: Needs review » Needs work

The last submitted patch, 16: 2311095-field-update-remove-index-test-only.patch, failed testing.

catch’s picture

Sql-backed field schema updates that add columns fail

The last submitted patch, 16: 2311095-field-update-remove-index-test-only.patch, failed testing.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

xjm’s picture

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

acbramley’s picture

Status: Needs work » Postponed (maintainer needs more info)
Issue tags: +Bug Smash Initiative

7 years since the last comment, we'll definitely need to check if this issue still exists on 11.x, although given the length of time I'm sure all the code looks wildly different so may be hard to know.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.