Problem/Motivation

As a follow-up to #2841291: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field we should remove all manual marking of NOT NULL from various storage schema implementations.

Proposed resolution

  1. Remove various places that set 'not null' => TRUE on fields in storage schema implementations and mark the respective fields as setStorageRequired(TRUE) instead.
  2. Add an update path for the field updates
  3. Deprecate the $not_null parameter of SqlContentEntityStorageSchema::addSharedTableFieldIndex()
CommentFileSizeAuthor
#29 3003586-29--full.patch151.12 KBmkalkbrenner
#28 3003586-28--full.patch151.08 KBmkalkbrenner
#27 3003586-27--full.patch151.12 KBhchonov
#27 3003586-27--for-review.txt57.84 KBhchonov
#25 3003586-25--full.patch145.49 KBhchonov
#25 3003586-25--for-review.txt57.84 KBhchonov
#23 3003586-23--full.patch144.78 KBhchonov
#23 3003586-23--for-review.txt57.84 KBhchonov
#21 3003586-21--full.patch131.7 KBhchonov
#21 3003586-21--for-review.txt57.84 KBhchonov
#21 interdiff-19-21.txt7.83 KBhchonov
#19 3003586-19--full.patch127.5 KBhchonov
#19 3003586-19--for-review.txt53.64 KBhchonov
#19 interdiff-17-19.txt1.61 KBhchonov
#17 3003586-17--full.patch117.14 KBhchonov
#17 3003586-17--for-review.txt52.29 KBhchonov
#17 interdiff-15-17.txt1.37 KBhchonov
#15 interdiff-13-15.txt6.38 KBhchonov
#15 3003586-15--for-review.txt52.3 KBhchonov
#15 3003586-15--full.patch115.78 KBhchonov
#13 interdiff-11-13.txt1.05 KBhchonov
#13 3003586-13--for-review.txt49.21 KBhchonov
#13 3003586-13--full.patch112.69 KBhchonov
#11 3003586-11--for-review.txt49.19 KBhchonov
#11 3003586-11--full.patch112.67 KBhchonov
#9 3003586-9--full.patch89.27 KBgease
#6 3003586-4-interdiff.txt27.31 KBtstoeckler
#4 3003586-4--full.patch90.6 KBtstoeckler
#4 3003586-4--for-review.txt41.96 KBtstoeckler
#4 3003586-2-4-interdiff.txt26.92 KBtstoeckler
#2 3003586-2.patch25.08 KBtstoeckler

Issue fork drupal-3003586

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

tstoeckler created an issue. See original summary.

tstoeckler’s picture

tstoeckler
he/him
Primary language German
Location Essen, Germany
commented
Status: Active » Postponed
StatusFileSize
new 3003586-2.patch25.08 KB

This should apply once #2841291: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field is in.

jibran’s picture

jibran
he/him
Primary language Urdu
Location Toronto, Canada
commented

This is a huge DX improvement IMO but to remeber 'not null' => TRUE to setStorageRequired(TRUE) is not stright forward.

tstoeckler’s picture

tstoeckler
he/him
Primary language German
Location Essen, Germany
commented
Status: Postponed » Needs review
StatusFileSize
new 3003586-2-4-interdiff.txt26.92 KB
new 3003586-4--for-review.txt41.96 KB
new 3003586-4--full.patch90.6 KB

OK, so here's a new patch which builds on the latest approach in #2841291: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field and updates the storage schema version (which is introduced as a concept over there) to 2.

Possibly this will fail due to #2976040: default_revision can be left around in data table due to broken entity type CRUD, but not sure yet, so leaving it out for now.

Status: Needs review » Needs work

The last submitted patch, 4: 3003586-4--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

tstoeckler’s picture

tstoeckler
he/him
Primary language German
Location Essen, Germany
commented
Status: Needs work » Needs review
StatusFileSize
new 3003586-4-interdiff.txt27.31 KB

Here's my WIP interdiff for this issue, didn't get any further today. Will explain some of the details once I finish this.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

gease’s picture

gease
he/him
Primary language Russian
Location Praha
commented
StatusFileSize
new 3003586-9--full.patch89.27 KB

Plain reroll of full patch from #4 against 8.9.x. Though fully porting patch may need additional work due to #3004642: Deprecate SqlContentEntityStorageSchemaConverter in favor of the new API provided by EntityTypeListenerInterface::onFieldableEntityTypeUpdate()

Status: Needs review » Needs work

The last submitted patch, 9: 3003586-9--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new 3003586-11--full.patch112.67 KB
new 3003586-11--for-review.txt49.19 KB

I've updated the patch on top of #2841291-229: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field and #3083131-28: BaseFieldDefinition::isStorageRequired wrongly falls back to ::isRequired.

The current approach

  1. adds an update method to the storage schema
  2. updates the entity storage schema version of all entity types to v2
  3. removes the not null handling for fields in the storage schema handlers and replaces it with ::setStorageRequired(TRUE) or ::setStorageRequired(FALSE) calls on the corresponding base fields

Status: Needs review » Needs work

The last submitted patch, 11: 3003586-11--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new 3003586-13--full.patch112.69 KB
new 3003586-13--for-review.txt49.21 KB
new interdiff-11-13.txt1.05 KB

Status: Needs review » Needs work

The last submitted patch, 13: 3003586-13--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new 3003586-15--full.patch115.78 KB
new 3003586-15--for-review.txt52.3 KB
new interdiff-13-15.txt6.38 KB

Status: Needs review » Needs work

The last submitted patch, 15: 3003586-15--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new interdiff-15-17.txt1.37 KB
new 3003586-17--for-review.txt52.29 KB
new 3003586-17--full.patch117.14 KB

My comment.install update was named wrongly - copy paste error.

Status: Needs review » Needs work

The last submitted patch, 17: 3003586-17--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new interdiff-17-19.txt1.61 KB
new 3003586-19--for-review.txt53.64 KB
new 3003586-19--full.patch127.5 KB

We need to take care explicitly of the overridden base fields of the Workspace entity.

The full patch is based on the updated combined patch from #2841291-231: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field and #3083131-28: BaseFieldDefinition::isStorageRequired wrongly falls back to ::isRequired .

Status: Needs review » Needs work

The last submitted patch, 19: 3003586-19--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new interdiff-19-21.txt7.83 KB
new 3003586-21--for-review.txt57.84 KB
new 3003586-21--full.patch131.7 KB

These fails show the importance of #1894286: System updates are executed without priority. Until it is resolved we need to declare explicitly hook_update_dependencies() multiple times.

Status: Needs review » Needs work

The last submitted patch, 21: 3003586-21--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Title: [PP-2] Use setStorageRequired() instead of overriding the storage schema to mark fields as NOT NULL in the database » [PP-4] Use setStorageRequired() instead of overriding the storage schema to mark fields as NOT NULL in the database
Status: Needs work » Needs review
StatusFileSize
new 3003586-23--for-review.txt57.84 KB
new 3003586-23--full.patch144.78 KB

2 more issues emerged out of the previous patch.

The following full patch is based on:

  1. The combined patch from #2841291-231: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field
  2. #3083131-28: BaseFieldDefinition::isStorageRequired wrongly falls back to ::isRequired
  3. #3098430-6: Automatic index creation/deletion does not work if a field storage definition is installed before the entity type is updated
  4. #3098427-10: Manipulating the revision metadata keys before running the BC layer breaks the BC layer

The current approach has not changed and therefore I am no adding an interdiff.

Status: Needs review » Needs work

The last submitted patch, 23: 3003586-23--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new 3003586-25--for-review.txt57.84 KB
new 3003586-25--full.patch145.49 KB

Now that #3098427: Manipulating the revision metadata keys before running the BC layer breaks the BC layer is green, the following full patch is based on:

  1. The combined patch from #2841291-231: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field
  2. #3083131-28: BaseFieldDefinition::isStorageRequired wrongly falls back to ::isRequired
  3. #3098430-6: Automatic index creation/deletion does not work if a field storage definition is installed before the entity type is updated
  4. #3098427-14: Manipulating the revision metadata keys before running the BC layer breaks the BC layer

The current approach has not changed and therefore I am no adding an interdiff.

Status: Needs review » Needs work

The last submitted patch, 25: 3003586-25--full.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

hchonov’s picture

hchonov
Primary language Bulgarian
Location Burgas
commented
Status: Needs work » Needs review
StatusFileSize
new 3003586-27--for-review.txt57.84 KB
new 3003586-27--full.patch151.12 KB

Thanks to the fails, I've found out a bug in the patch for #3098430-6: Automatic index creation/deletion does not work if a field storage definition is installed before the entity type is updated, which is fixed in the new version.

I've updated the underlying patches:

  1. The combined patch from #2841291-231: Fix NOT NULL handling in the entity storage and 'primary key' changes when updating the storage definition of an identifier field
  2. #3083131-28: BaseFieldDefinition::isStorageRequired wrongly falls back to ::isRequired
  3. #3098430-6: Automatic index creation/deletion does not work if a field storage definition is installed before the entity type is updated
  4. #3098427-14: Manipulating the revision metadata keys before running the BC layer breaks the BC layer

The current approach has not changed and therefore I am no adding an interdiff.

mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented
StatusFileSize
new 3003586-28--full.patch151.08 KB

#27 doesn't apply anymore. I try to re-roll it. Let's wait for the test results.

mkalkbrenner’s picture

mkalkbrenner
Primary language German
Location 🇩🇪
commented
StatusFileSize
new 3003586-29--full.patch151.12 KB

I erroneously re-rolled against 8.8.x. Now for 8.9.x.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

smustgrave commented
Status: Needs review » Needs work
Issue tags: +Needs reroll

Will need a reroll for 10.1

And Change Record.

bhanu951’s picture

bhanu951 commented
Assigned: Unassigned » bhanu951

I will work on re-roll

Bhanu951 opened merge request !3069

Rajeshreeputra made their first commit to this issue’s fork.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.