Problem/Motivation

[EA] 'array_merge(...)' is used in a loop and is a resources greedy construction.

See https://github.com/kalessil/phpinspectionsea/blob/master/docs/performanc...

Steps to reproduce

Proposed resolution

Example :


  // @see core/modules/views/src/Plugin/views/query/Sql.php

   public function getWhereArgs() {
-    $args = [];
-    foreach ($this->where as $where) {
-      $args = array_merge($args, $where['args']);
-    }
-    foreach ($this->having as $having) {
-      $args = array_merge($args, $having['args']);
-    }
-    return $args;
+    return array_merge([], ...array_column($this->where, 'args'), ...array_column($this->having, 'args'));
   }

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Comments

jungle created an issue. See original summary.

jungle’s picture

The scope of #3044373: Improve performance by not using array_merge() in a loop in TestSetupTrait::getConfigSchemaExclusions() is too narrow, and the patch there is not ready yet to me, so instead of rescoping it, filed a new one with a large scope covering the whole codebase.

Maybe we could/should turn this one into a meta.

jungle’s picture

Status: Active » Needs review
StatusFileSize
new17.14 KB

The first iteration.

jungle’s picture

Title: Refactor array_merge() usage in loops for performance » Refactor array_merge() usage in loops as possible for performance

It's not easy to find and identify all refactorable usages, so changing the title a little bit.

dww’s picture

Status: Needs review » Postponed

Thanks for proposing this and getting the patch started!

Interesting. A new language feature. Slick! ;)

However, the '...' array "spread" operator only exists in PHP 7.4. We'll have to wait for a version of core where that's the minimum. So far, 9.1.x still allows PHP 7.3. Might have to bump this to 10.0.x, TBD.

Meanwhile, some questions / concerns about how this should work once we can use it:

  1. +++ b/core/lib/Drupal/Core/Authentication/AuthenticationCollector.php
    @@ -72,10 +72,7 @@ public function getSortedProviders() {
    +      $this->sortedProviders = array_merge([], ...$this->providerOrders);
    

    If I'm reading the spec correctly, I believe this could just be:

      $this->sortedProviders = [...$this->providerOrders];
    

    No?

  2. +++ b/core/lib/Drupal/Core/Config/ConfigManager.php
    @@ -249,9 +249,9 @@ public function findConfigEntityDependents($type, array $names, ConfigDependency
    -    return $dependencies;
    +    return array_merge([], ...$dependencies);
    

    Why bother? Why not still return $dependencies and just change what we're doing in the loop?

  3. +++ b/core/lib/Drupal/Core/Config/Schema/SchemaCheckTrait.php
    @@ -58,8 +58,9 @@ public function checkConfigSchema(TypedConfigManagerInterface $typed_config, $co
    +    $errors = array_merge([], ...$errors);
    

    Seems like a no-op. Don't you end up with $errors the same before an after this line? How/why are they different?

  4. +++ b/core/lib/Drupal/Core/Config/Schema/SchemaCheckTrait.php
    @@ -122,11 +123,12 @@ protected function checkValue($key, $value) {
    +      return array_merge($errors, ...$nested_errors);
    

    I don't think we nee this, but if we did, I believe it would be:

    return [...$errors, ...$nested_errors];

    But again, I'd rather we just did:

    $errors[] = %this->checkValue(...);

    and then keep return $errors;

  5. +++ b/core/lib/Drupal/Core/Entity/EntityAccessControlHandler.php
    @@ -340,12 +340,14 @@ public function fieldAccess($operation, FieldDefinitionInterface $field_definiti
    -    $grants = [':default' => $default];
    ...
    +      $grants[] = [$module => $this->moduleHandler()->invoke($module, 'entity_field_access', [$operation, $field_definition, $account, $items])];
    

    These two lines should be enough for this whole hunk.

  6. +++ b/core/lib/Drupal/Core/Extension/ModuleHandler.php
    @@ -492,8 +492,9 @@ public function alter($type, &$data, &$context1 = NULL, &$context2 = NULL) {
    +        $extra_modules = array_merge([], ...$extra_modules);
    

    no-op?

There's lots more of the above in the rest of the patch, but I'll stop here and see what's up. ;)

Thanks!
-Derek

dww’s picture

Status: Postponed » Needs work

Actually, seems that a lot of this could be refactored now without ...$foo at all. E.g.:

  errors = [];
  foreach ($something as $whatever) { 
    $errors[] = $this->checkValue($whatever...);
  }
  return $errors;

We definitely don't need to array_merge() each time inside the loop, but we don't need ...$ to replace it, either.

jungle’s picture

Thanks @dww.

    $options = [];
    foreach ($configurationSources as $source) {
        /* something happens here */
        $options[] = $source->getOptions(); // <- yes, we'll use a little bit more memory
    }
   
    /* PHP below 5.6 */
    $options = call_user_func_array('array_merge', $options + [[]]); // the nested empty array covers cases when no loops were made, must be second operand

    /* PHP 5.6+: more friendly to refactoring as less magic involved */
    $options = array_merge([], ...$options); // the empty array covers cases when no loops were made
    
    /* PHP 7.4+: array_merge now accepts to be called without arguments. It will work even if $options is empty */
    $options = array_merge(...$options);

It's not PHP 7.4 only or 7.4+ available, see the link I added in IS. The above are examples from that page. Queued a testing against PHP 7.3.

... here is variadic since PHP 5.6, not spread operator.

We have variadics usages in core. #3125032: Use variadics in Cache::merge* functions - for example Cache::mergeTags()

jungle’s picture

Status: Needs work » Needs review

The testing in #3 against PHP 7.3 passed, setting back to NR.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

sylus’s picture

StatusFileSize
new18.84 KB
akhildev.cs’s picture

StatusFileSize
new122.86 KB

hi
patch #11 applied successfully
drupal 9.3.x-dev & php 7.3
This seems removed all possible "array_merge" from inside the loop.
thankyou.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

joachim’s picture

Status: Needs review » Reviewed & tested by the community
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

The code is certainly prettier in many cases. Nice work.

Committed and pushed cbcdb8b2d42 to 10.0.x and 3cf6d577f81 to 9.4.x. Thanks!

We should get a follow-up for Drupal 10. Since the minim version of PHP there is 8... we change
return array_merge([], ...$dependencies);
TO
return array_merge(...$dependencies);

  • alexpott committed cbcdb8b on 10.0.x
    Issue #3164210 by jungle, sylus, Akhildev.cs, dww: Refactor array_merge...

  • alexpott committed 3cf6d57 on 9.4.x
    Issue #3164210 by jungle, sylus, Akhildev.cs, dww: Refactor array_merge...
alexpott’s picture

@Akhildev.cs please don't post screenshots of code being applied. DrupalCI tells us this already so it looks like a comment purely make to receive issue credit.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.