Follow-up to #2855675: Add orderby key to all sequences in core

Problem/Motivation

Sequences that are not sorted can result in unexpected configuration differences whilst importing configuration. Which confuses users because they are being told something is different when it is not. Also this behaviour can result in random test fails

Third party settings are ordered by the order in which the third party settings are added. This means that the same configuration can be different on depending on the order in which it is was configured.

Proposed resolution

Add orderby key and test it.

Remaining tasks

User interface changes

None

API changes

None.

Data model changes

All sequences in configuration sorted. This patch will introduce the generic upgrade path for all #2855675: Add orderby key to all sequences in core sub-issues.

Issue fork drupal-2860531

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

alexpott created an issue. See original summary.

alexpott’s picture

Status: Active » Needs review
FileSize
1.65 KB
alexpott’s picture

Hmmm... thinking about this some more perhaps we should just introduce the generic update path test here and add to it as we add schema that's sorted.

I have to generate a new dump because I wanted to test with config_test entities and that requires config_test to be installed and its entity schema. Furthermore, we need a new dump so something like system_post_update_recalculate_configuration_entity_dependencies() doesn't come a do all the fixing for us.

alexpott’s picture

Issue tags: +8.4.0 release notes

Tagging for 8.4.0 release notes since exporting configuration after doing the update is going to be important.

The last submitted patch, 2: 2860531-2.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 3: 2860531-3.patch, failed testing.

gaurav.kapoor’s picture

Looks like you included a .gz file as well while creating patch. Leading to such content in the patch.

ÈX�drupal-8.3.x.bare-standard-plus-config_test.php�ì½Û–ÛF–(ø0oõ˜\g–í!…û%åv—,Y¶ºtkKvµûäY\ 	‹$h€””.k­óóógæíÌŸœ/™Øq@€I™’Ù«ZNâˆØ{ÇŽ}ß_ÿëâzñ—û÷µ¿Åy’ͯ^/Ñ<AER>½šç~’Mñ_îÿõ¯Ñþªý-…䏇Z‚–(B%Öyª\f±–¬f-Ím‰Éïù•¶X‹¼Äå9y^zs•Œ ½G¥v…ç¸@Kœhэ¶¼ÆÚãbµ@S-87´$Ò—y>-ÏÉÌ´2.²Å¹ÿ—¿¬Jñàå#2¹ËÇ|Õþò—ÿçó9Ž—Y>×þE7..®ðòQuç˯šOêß”ñ5ž¡/¿Ò¿‰L&öESüåZÆ×_ÜÓPQ ›/ÿ¢i_¤ž&åÚ¿|S_$—£,i_#W—7L/‘Í—_Ü—çùR›¯¦Szë͏?}WÝ)³ßùú34­ßYÍËìjŽ“Ö;_±ÿ|±ÌßâùÚ¼CE|Š	*ã,2—)ž_-¯ÙËžóEë{ÙŒ Í\µøCºïDÓ<êùГ‡Ï^«¾eWògè?_,Šl†Ší-¾Qb¸~2›'øîÒA/èõÎg7åoÓ	àÅK\LJ¼dÓ[-Ó`Ô¿L«Ó<~;!.1€}3Ín Ù™L Ú¿ËJ²¦É맽Ûnñ©ƒìÛjQôjµaÁ{}δ‚Ö÷¦h~Eøû—hZÛí¦ÆfZͳßVžS0V™ª'”’'€ßdòMWª%Qè¶XHc”Ézº“ïvÕ·ñ•óà’QANŸ54~<¶0™Dyr3è@[Í“éA	&¨_IpŠVS¾Æê<ÅK<òéJD”›žÏ­ï¨eË›Ý9Ô]á”ãÌãðœ¤b–›éb‰n°•z™P=%þ°å<Z²‰øT¹š?Ýõc%!ñµ€±ÜÏͯʈ¶\w»3£Þ‰-ÖÐ ˆ6¡`Ý}¬ŽßØf'öœ*ß=âqÀÎ)PçFÿ,<ðÈÂâ±…7ºSÒ|ÈçöÝóä.f~µóñ>HüïÐϲ@órŠ`ŸLPš’3ú$£>°èa6„Ï}$šíDð-yeSš1¼½®ÉºVU¤&™æ•1¥øOGTgË8ñç;$£žæŸ‘av9ʧÁ9ͨ>CuÛç-p©jõ+²¾C2-Áèê¸ìlš_X‰Þž™lÉ<Ö1ˆµrNƒÖ<Mr¹†r²VvŽªöçOÖÊ“µr,NÖÊ[¶V*ÄÇ“sû£"Fä:9òeIԒŐÓ!>¤·µ&Œ�ÀÉæb+t7&µ¾®›ÞþÑ
@ŽY1²
swentel’s picture

+++ b/core/modules/system/tests/fixtures/update/drupal-8.config-resave.php
@@ -0,0 +1,30 @@
+ * Partial database to configuration that will be affected by re-saving.

s/to/with ?

@gaurav.kapoor : that's intented - that's how you can add a large dump of a db which will be imported for upgrade tests.

gaurav.kapoor’s picture

thanks for info @swentel.

alexpott’s picture

Issue summary: View changes
Status: Needs work » Needs review
FileSize
822 bytes
133.07 KB

Fixing the test fail.

alexpott’s picture

Oops forget to fix #8 - thanks for the review @swentel.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Wim Leers’s picture

Status: Needs review » Needs work
Issue tags: +Needs change record

AFAICT this patch was just waiting to be RTBC'd, unfortunately I only discovered it now :(

+++ b/core/modules/system/src/Tests/Update/ResaveConfigurationTest.php
@@ -0,0 +1,36 @@
+ * Tests system_post_update_resave_configuration().
...
+class ResaveConfigurationTest extends UpdatePathTestBase {

+++ b/core/modules/system/system.post_update.php
@@ -65,3 +65,30 @@ function system_post_update_hashes_clear_cache() {
+function system_post_update_resave_configuration(&$sandbox = NULL) {

It's not just about resaving configuration, it's about resaving configuration to ensure correct sorting.

So "ResaveConfigForCorrectSorting" or something like that would make more sense. Because "resave config" is something that's likely to happen again in the future.

Other than that, I think this looks totally ready! I do think this will need a change record though, because it'll cause config managed via git to be changed for existing sites.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

larowlan’s picture

eleonel’s picture

Status: Needs work » Needs review
FileSize
4.33 KB
130.99 KB

Re-rolled patch.

eleonel’s picture

Re-rolled patch (ignore patch from #24)

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

needs-review-queue-bot’s picture

Status: Needs review » Needs work
FileSize
144 bytes

The Needs Review Queue Bot tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.

rpayanm made their first commit to this issue’s fork.

rpayanm’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll

Reeolled.

ameymudras’s picture

smustgrave’s picture

Status: Needs review » Needs work
Issue tags: +Needs Review Queue Initiative

This issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.

moving to NW for change record.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

borisson_’s picture

Status: Needs work » Reviewed & tested by the community

Added a CR. Back to rtbc.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/system/system.post_update.php
@@ -51,3 +51,30 @@ function system_post_update_linkset_settings() {
+    // Resave configuration to ensure any new sorting is applied.
+    $config_factory->getEditable($config_name)->save();

During updates you need to save with trusted data set to TRUE otherwise you can get in the way of other updates. Therefore you need to do sort here. Also you should only same them if they have third party settings.

ameymudras’s picture

Status: Needs work » Needs review
FileSize
4.52 KB
smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

@ameymudras thanks for working on this. Please include interdiffs though to easily see the changes.

+ $config_factory->getEditable($config_name)->save(TRUE);
Seems to address #35

Since this was previously RTBC going to restore but do wonder about

unset($sandbox['config_names'][$key]);

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/system/system.post_update.php
@@ -58,6 +58,35 @@ function system_post_update_linkset_settings() {
+      $config_factory->getEditable($config_name)->save(TRUE);

Trusting the configuration here means that it will not sort. So this update function will not work. See discussions on #3017054: Consistently sort filter formats to simplify config exports for lots about this.

I think here we should change the update to do the sorting and continue to use the trusted data feature as this update should not be fixing other things.

We also need to add a sort to \Drupal\Core\Config\Entity\ConfigEntityBase::preSave() in the same place as we do the dependency sorting.

borisson_’s picture

Issue tags: -Needs change record

Removing the needs CR tag,