Problem/Motivation

We would like to deprecate all legacy code in the file core/includes/module.inc. The function module_config_sort() has only few usages in core and 8 usages in contrib http://grep.xnddx.ru/search?text=module_config_sort

Proposed resolution

Deprecate the function without replacement or provide \Drupal\Component\Utility\SortArray::sortByWeightAndKeys() helper

Remaining tasks

review/commit

User interface changes

None

API changes

The function module_config_sort() will be deprecated.

Issue fork drupal-3262807

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

andypost created an issue. See original summary.

andypost’s picture

Issue summary: View changes
andypost’s picture

Issue summary: View changes
Status: Active » Needs review
StatusFileSize
new12.17 KB

Filed CR and looks it need replacement

voleger’s picture

Status: Needs review » Reviewed & tested by the community

Looks good. The new method replaced all function usages with the appropriate code. A deprecation message was added and is correct. The deprecation test is implemented.

andypost’s picture

Status: Reviewed & tested by the community » Needs work

There's +3583 failures and new method needs extensive testing

Also I'm not sure sure that helper is useful, maybe some config event could be used to apply sorting for modules automatically...
This way the function will become @internal

@voleger any ideas?

voleger’s picture

+++ b/core/lib/Drupal/Component/Utility/SortArray.php
@@ -125,4 +125,37 @@ public static function sortByKeyInt($a, $b, $key) {
+  public function sortByWeightAndKeys(array $array): array {

method has to be static

Let's update the method definition and see what else is left.

andypost’s picture

Status: Needs work » Needs review
StatusFileSize
new780 bytes
new12.17 KB

Thank you! good catch)

andypost’s picture

I still looking for a way to get rid of it without replacement, this method is implementation details and should be "protected in module installer" IMO

+++ b/core/lib/Drupal/Core/Extension/ModuleInstaller.php
@@ -214,7 +215,7 @@ public function install(array $module_list, $enable_dependencies = TRUE) {
         $current_module_filenames = $this->moduleHandler->getModuleList();
         $current_modules = array_fill_keys(array_keys($current_module_filenames), 0);
-        $current_modules = module_config_sort(array_merge($current_modules, $extension_config->get('module')));
+        $current_modules = SortArray::sortByWeightAndKeys(array_merge($current_modules, $extension_config->get('module')));
         $module_filenames = [];
         foreach ($current_modules as $name => $weight) {
           if (isset($current_module_filenames[$name])) {

This is only place where the function used for some sophisticated logic at install time.

All other places is config->set(module, SORTABLE) and tests... are hacking config as well

voleger’s picture

Status: Needs review » Reviewed & tested by the community

Test results looks good now

catch’s picture

I think it's worth seeing if we can do #8 before going ahead here, removing the API altogether would be better.

catch’s picture

Status: Reviewed & tested by the community » Needs review

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

mile23’s picture

Issue tags: +Needs tests
StatusFileSize
new12.18 KB

Rerolled for 10.1.x. I'm also not sure where it gets removed, so I said 10.2, which is probably wrong.

There's no unit test for module_config_sort(), and now there's no unit test for sortByWeightAndKeys(), so I'm adding a needs test tag.

If we can make it go away as per #10 and #8, then no test needed.

andypost’s picture

Status: Needs review » Needs work

Deprecation removals are supposed to happen only in major releases so only few nits

The tests should be
- deprecation test to be sure functions rutuning the same results
- extra unit test like core/tests/Drupal/Tests/Component/Utility/SortArrayTest.php

  1. +++ b/core/includes/module.inc
    @@ -104,8 +106,14 @@ function module_set_weight($module, $weight) {
    + * @deprecated in drupal:10.1.0 and is removed from drupal:10.2.0. Use
    

    it should be removed from 11.0

  2. +++ b/core/lib/Drupal/Component/Utility/SortArray.php
    @@ -125,4 +125,37 @@ public static function sortByKeyInt($a, $b, $key) {
    +    // PHP array sorting functions such as uasort() do not work with both keys and
    +    // values at the same time, so we achieve weight and name sorting by computing
    +    // strings with both information concatenated (weight first, name second) and
    

    line longer 80 chars

  3. +++ b/core/lib/Drupal/Component/Utility/SortArray.php
    @@ -125,4 +125,37 @@ public static function sortByKeyInt($a, $b, $key) {
    +      // The maximum weight is PHP_INT_MAX, so pad all weights to 19 digits.
    +      $sort[] = $prefix . sprintf('%019d', abs($weight)) . $name;
    

    19 means only for 64-bits but I think it's ok for nowadays

mile23’s picture

Status: Needs work » Needs review
StatusFileSize
new12.21 KB
new2.12 KB

Takes care of #15 1 and 2, but I'm not really sure what to do about 3.

Still needs a test to determine that both functions operate identically.

nivethasubramaniyan’s picture

StatusFileSize
new12.21 KB

Fixing CCF in #16

smustgrave’s picture

Status: Needs review » Needs work

For tests and #15.3

andypost’s picture

+++ b/core/includes/module.inc
@@ -104,8 +106,14 @@ function module_set_weight($module, $weight) {
+ * @deprecated in drupal:10.1.0 and is removed from drupal:11.0.0. Use
...
+  @trigger_error(__FUNCTION__ . '() is deprecated in drupal:10.1.0 and is removed from drupal:10.2.0. Use \Drupal\Component\Utility\SortArray::sortByWeightAndKeys() instead. See https://www.drupal.org/node/3262814', E_USER_DEPRECATED);

both should be drupal:11.0.0

spokje’s picture

Status: Needs work » Needs review
StatusFileSize
new13.46 KB
new2.02 KB

Tried to address #18 and #19.

andypost’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -Needs tests

I'm sure #15.3 should not have even comment as 32-bit big-int is known issue and mostly no distros left

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 20: 3262807-20.patch, failed testing. View results

spokje’s picture

Status: Needs work » Reviewed & tested by the community

Known (at least to me...) random JS test failure, back to RTBC and ordered retest.

catch’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/lib/Drupal/Component/Utility/SortArray.php
@@ -125,4 +125,40 @@ public static function sortByKeyInt($a, $b, $key) {
 
+  /**
+   * Sorts a keyed array by weight and keys.
+   *
+   * The list of enabled modules is expected to be ordered by weight and name.
+   * The list is always sorted on write to avoid the overhead on read.
+   *
+   * @param array $array
+   *   A keyed array of names with weight as value.
+   *
+   * @return array
+   *   An array sorted by weight and name.
+   *
+   * @internal
+   */

This is interchanging 'keys' and 'names', given the method name I think it should just be 'keys' everywhere?

Also should we be talking about 'the list of enabled modules' in Component/Utility and especially whether it's sorted on read or write? Could that be moved to an inline comment in the calling code somewhere?

andypost’s picture

Status: Needs work » Needs review
StatusFileSize
new1.44 KB
new13.56 KB

Moved comment

andypost’s picture

probably it could be added to @param description

+++ b/core/lib/Drupal/Component/Utility/SortArray.php
@@ -128,8 +128,7 @@ public static function sortByKeyInt($a, $b, $key) {
    * Sorts a keyed array by weight and keys.
...
-   * The list is always sorted on write to avoid the overhead on read.
+   * Array values are supposed to be casted to integer as weights for keys.
    *
    * @param array $array
    *   A keyed array of names with weight as value.

I think it makes sense to point about cast to in for array values

Ratan Priya’s picture

StatusFileSize
new13.56 KB
new698 bytes

@anypost,
Made changes as per comment #26.
Needs review.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

Believe #27 addresses #26

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs tests
+++ b/core/lib/Drupal/Component/Utility/SortArray.php
@@ -125,4 +125,38 @@ public static function sortByKeyInt($a, $b, $key) {
+  /**
+   * Sorts a keyed array by weight and keys.
+   *
+   * @param array $array
+   *   A keyed array of names with weight as value.
+   *   Array values are supposed to be casted to integer as weights for keys.
+   *
+   * @return array
+   *   An array sorted by weight and name.
+   *
+   * @internal
+   */
+  public static function sortByWeightAndKeys(array $array): array {

If we're moving this functionality to the utility class we should be adding explicit and extensive test coverage of this to \Drupal\Tests\Component\Utility\SortArrayTest.

Also the name of this method is interesting I think it should be something else. The other usage of the word weight in this class refers to a weight property. Here we're sorting an array of integers by the value and then by the key. I think the method name should reflect that. To me this is really sortByIntValueAndKey(). I think the typehint should be @param int[] $array.

andypost’s picture

Related issues: +#3347842: Deprecate the trusted data concept in configuration
StatusFileSize
new11.43 KB
new13.57 KB

Changed method name but still needs test.

Meanwhile I think we can remove this method at all in case of related #3347842: Deprecate the trusted data concept in configuration
The new config class can care about sorting module list before saving the config if I got it right

kim.pepper’s picture

Status: Needs work » Needs review
Issue tags: -Needs tests
StatusFileSize
new17.48 KB
new3.91 KB

Added a test including one that I think should pass, but fails.

kim.pepper’s picture

StatusFileSize
new15.53 KB
new2.49 KB

For some reason the diff was whacky. Changing the order of methods seems to make it behave.

The last submitted patch, 31: 3262807-31.patch, failed testing. View results

kim.pepper’s picture

+++ b/core/tests/Drupal/Tests/Component/Utility/SortArrayTest.php
@@ -304,6 +304,77 @@ public function providerSortByTitleProperty() {
+    $tests['negative_weights'] = [
+      [
+        'aaa' => -2,
+        'bbb' => -1,
+        'ccc' => -3,
+      ],
+      [
+        'bbb' => -3,
+        'aaa' => -2,
+        'ccc' => -1,
+      ],
+    ];

This should sort asc right?

Status: Needs review » Needs work

The last submitted patch, 32: 3262807-32.patch, failed testing. View results

kim.pepper’s picture

Status: Needs work » Needs review
StatusFileSize
new16.26 KB
new13.94 KB

I think this can be greatly simplified to just an array multisort like:

array_multisort(array_values($array), SORT_ASC, array_keys($array), SORT_ASC, $array);

This modifies the original array that gets passed in by reference so calling code will need to change too. I think this is more in line with how usort etc. work though.

smustgrave’s picture

Status: Needs review » Needs work

Seems to have some failures.

Have not re-reviewed yet.

kim.pepper’s picture

Status: Needs work » Needs review
StatusFileSize
new15.33 KB
new3.6 KB

Fixes test fails due to dodgy extension config setting logic.

kim.pepper’s picture

StatusFileSize
new15.41 KB
new1.65 KB

Fix more extension config setting issues.

The last submitted patch, 38: 3262807-38.patch, failed testing. View results

Status: Needs review » Needs work

The last submitted patch, 39: 3262807-39.patch, failed testing. View results

kim.pepper’s picture

Status: Needs work » Needs review
StatusFileSize
new15.24 KB
new1013 bytes

Fixes test fails.

kim.pepper’s picture

I wonder if this should just be sortByWeightAndKey() since that's what we use it for?

kim.pepper’s picture

Just re-read the comments and @alexpott suggested sortByIntValueAndKey() back in #29.

andypost’s picture

+++ b/core/lib/Drupal/Component/Utility/SortArray.php
@@ -125,4 +125,15 @@ public static function sortByKeyInt($a, $b, $key) {
+    array_multisort(array_values($array), SORT_ASC, array_keys($array), SORT_ASC, $array);

Looking at https://git.drupalcode.org/project/drupal/-/blob/10.1.x/core/modules/mig...

I think it should add SORT_NUMERIC to value and SORT_STRING to keys

andypost’s picture

+++ b/core/lib/Drupal/Component/Utility/SortArray.php
@@ -125,4 +125,15 @@ public static function sortByKeyInt($a, $b, $key) {
+   * @param array $array
+   *   key: (string) A key name.
+   *   value: (int) A weight integer.

Missing "A Keyed array..."

kim.pepper’s picture

StatusFileSize
new15.3 KB
new3.4 KB

Fixes for #45 and #46.

I also bumped the deprecation message to 10.2.x in anticipation of it opening soon.

kim.pepper’s picture

StatusFileSize
new15.29 KB
new882 bytes

More docblock feedback from @andypost in slack.

andypost’s picture

Status: Needs review » Reviewed & tested by the community

I find it ready, let's wait for 10.2 opening

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 48: 3262807-48.patch, failed testing. View results

kim.pepper’s picture

Looks like a random fail. Happening in HEAD too apparently.

andypost’s picture

Status: Needs work » Reviewed & tested by the community

Yes, back to RTBC

Ref Drupal\FunctionalTests\Installer\InstallerExistingConfigMultilingualTest

larowlan’s picture

  1. +++ b/core/tests/Drupal/Tests/Component/Utility/SortArrayTest.php
    @@ -304,6 +304,89 @@ public function providerSortByTitleProperty() {
    +    $tests['negative_weights_with_equal'] = [
    +      [
    +        'bbb' => -1,
    +        'aaa' => -1,
    +        'ccc' => -3,
    +      ],
    +      [
    +        'ccc' => -3,
    

    Can we add a test case that uses a string value that is numeric e.g. '3'

  2. +++ b/core/lib/Drupal/Component/Utility/SortArray.php
    @@ -125,4 +125,16 @@ public static function sortByKeyInt($a, $b, $key) {
    +  public static function sortByIntValueAndKey(array &$array): void {
    +    array_multisort(array_values($array), SORT_ASC, SORT_NUMERIC, array_keys($array), SORT_ASC, SORT_NATURAL, $array);
    

    Or alternatively can we add a check that each value is an integer?

larowlan’s picture

Status: Reviewed & tested by the community » Needs work
kim.pepper’s picture

Category: Task » Bug report

Changing this to a bug as the current implementation doesn't correctly order negative weights.

kim.pepper’s picture

Version: 10.1.x-dev » 11.x-dev
Status: Needs work » Needs review
Issue tags: +DrupalSouth
StatusFileSize
new15.51 KB
new672 bytes

#53.1

Added a test for passing string values (of numbers) that can still be sorted numerically.

jibran’s picture

Status: Needs review » Reviewed & tested by the community

Reviewed this patch with @kim.pepper at DrupalSouth. I think going with #53.1 makes sense. The patch looks good now so setting it back to RTBC.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 56: 3262807-56.patch, failed testing. View results

andypost’s picture

Status: Needs work » Reviewed & tested by the community

random failure

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 56: 3262807-56.patch, failed testing. View results

kim.pepper’s picture

Status: Needs work » Reviewed & tested by the community

random failure

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 56: 3262807-56.patch, failed testing. View results

smustgrave’s picture

Status: Needs work » Reviewed & tested by the community

Seems like the random failure popping up today.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 56: 3262807-56.patch, failed testing. View results

andypost’s picture

failed Drupal\Tests\views\Functional\Plugin\DisplayFeedTranslationTest::testFeedFieldOutput(), re-queued

sorlov’s picture

Status: Needs work » Reviewed & tested by the community

Tests are ok now

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 56: 3262807-56.patch, failed testing. View results

andypost’s picture

Status: Needs work » Reviewed & tested by the community
quietone’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new15.82 KB

I'm triaging RTBC issues. I read the IS and the comments. I didn't find any unanswered questions or other work to do.

I noted that #53 suggesting either testing for an integer or testing for numeric. I read the patch and it adds a test for numeric values proving that the method works for integer and numeric values. However, the method name is still, sortByIntValueAndKey, which seems misleading. Shouldn't it be sortByNumericValueAndKey? Although, I guess one could also argue that is should 'Natural' should be in the title as well.

  1. +++ b/core/lib/Drupal/Component/Utility/SortArray.php
    @@ -125,4 +125,16 @@ public static function sortByKeyInt($a, $b, $key) {
    +   * Sorts an associative array by int value then key.
    

    "int" is not a word. But this also accepts numeric values.

  2. +++ b/core/lib/Drupal/Component/Utility/SortArray.php
    @@ -125,4 +125,16 @@ public static function sortByKeyInt($a, $b, $key) {
    +   *   - value: An integer.
    

    The tests include testing numeric values, so is this correct?

  3. +++ b/core/tests/Drupal/Tests/Component/Utility/SortArrayTest.php
    @@ -304,6 +304,101 @@ public function providerSortByTitleProperty() {
    +   *   An array of tests, matching the parameter inputs for
    +   *   sortByIntValueAndKey.
    

    This is rather generic, merely describing what any data provider does. I suppose the test should have a case for alphanumeric keys.

Instead of setting back to NW I opted to make a patch changing the method name and updating the comments. Is this better?

andypost’s picture

Good point about numeric, the only question is a test for floats (as they allowed)

doc-block is fine to keep with int

+++ b/core/lib/Drupal/Component/Utility/SortArray.php
@@ -125,4 +125,16 @@ public static function sortByKeyInt($a, $b, $key) {
+   * Sorts an associative array by numeric value then key.
+   *
+   * @param int[] $array
+   *   An associative array containing:
+   *   - key: A string.
+   *   - value: A numeric value.
+   */
+  public static function sortByNumericValueAndKey(array &$array): void {
+    array_multisort(array_values($array), SORT_ASC, SORT_NUMERIC, array_keys($array), SORT_ASC, SORT_NATURAL, $array);

There's no ability to point (int|float)[]

smustgrave’s picture

There an example of floats being used?

andypost’s picture

technically that's allowed but not sure in a wild

smustgrave’s picture

Is it something that should be allowed? Would it be worth a follow up to look into disallowing or throwing an error if floats are passed?

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Going to mark but do wonder if we should prevent floats.

longwave’s picture

Category: Bug report » Task
Status: Reviewed & tested by the community » Needs work

I think it's fine to document that the values should be integers; if it coincidentally works with floats that is not something we officially support as we have not documented it, but also something that I don't think we should spend time checking for or throwing errors for.

The updated method name has not been correctly replaced in all places, such as comments and the deprecation notice, back to NW for that.

Assigned and saved issue credits.

longwave’s picture

  1. +++ b/core/includes/module.inc
    @@ -104,8 +109,14 @@ function module_set_weight($module, $weight) {
    + *   \Drupal\Component\Utility\SortArray::sortByIntValueAndKey() instead.
    

    Method name needs updating.

  2. +++ b/core/includes/module.inc
    @@ -104,8 +109,14 @@ function module_set_weight($module, $weight) {
    +  @trigger_error(__FUNCTION__ . '() is deprecated in drupal:10.2.0 and is removed from drupal:11.0.0. Use \Drupal\Component\Utility\SortArray::sortByIntValueAndKey() instead. See https://www.drupal.org/node/3262814', E_USER_DEPRECATED);
    

    Method name needs updating.

  3. +++ b/core/tests/Drupal/Tests/Component/Utility/SortArrayTest.php
    @@ -304,6 +304,114 @@ public function providerSortByTitleProperty() {
    +   * @dataProvider providerSortByIntValueAndKey
    ...
    +  public function testSortByIntValueAndKey(array $a, array $expected): void {
    ...
    +   * Data provider for SortArray::sortByIntValueAndKey().
    ...
    +   * @see \Drupal\Tests\Component\Utility\SortArrayTest::testSortByIntValueAndKey()
    ...
    +  public function providerSortByIntValueAndKey(): array {
    

    Method names need updating.

andypost’s picture

Status: Needs work » Needs review

Converted to MR and fixed #76

andypost’s picture

I bet it will be easier to improve docs via MR for #75

smustgrave’s picture

Status: Needs review » Needs work

Could it be as simple as

value: A numeric value. Recommended to use integer values, floats will work but not supported

?

andypost’s picture

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Comment looks good. Gitlab is all green.

xjm’s picture

Please remember to hide patch files from the IS when an issue is converted to MR. Thanks!

xjm’s picture

Component: base system » configuration system
Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs subsystem maintainer review

Posted a number of points on the MR. I have several concerns about the data typing and integrity as well as possible behavior changes from this deprecation.

This will need config subsystem maintainer signoff before it is committed. It is also a significant change that may impact the data integrity of the config sort if done incorrectly, so it should have careful framework or release manager review in its final state.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

catch’s picture

Title: Deprecate module_config_sort() » Add SortArray::sortByWeightAndKeys()

I've just committed #3595652: Deprecate module.inc contents which moves the method to a specific class. I think we need to decide if we want to add a generic sort method still, so retitling. Although if we deprecate module weights as a whole, we wouldn't have any usages in core.

nicxvan’s picture

I'd vote for applying the credit here and closing this.

Are there other cases we need this?