Problem/Motivation

Similar to #2885129: [PHP 7.2] create_function() is deprecated, assuming we do plan to support PHP 7.2, we will need to swap out each() calls in core (and eventually contrib too) with alternative foreach() or a key()/current().

There are many such calls, and I'm not sure a single patch would be ideal or several small ones would be (I personally think the latter).

Ref https://wiki.php.net/rfc/deprecations_php_7_2#each

Proposed resolution

Use foreach() or a key()/current().

Remaining tasks

User interface changes

None

API changes

None

Data model changes

None

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Ayesh created an issue. See original summary.

Ayesh’s picture

Title: each() function is deprecated in PHP 7.2 » [PHP 7.2] each() function is deprecated
andypost’s picture

Issue summary: View changes
core/lib/Drupal/Component/Diff/Engine/DiffEngine.php:205:        while (list ($junk, $y) = each($matches)) {
core/lib/Drupal/Component/Diff/Engine/DiffEngine.php:213:        while (list ($junk, $y) = each($matches)) {
core/lib/Drupal/Core/Extension/ModuleInstaller.php:97:      while (list($module) = each($module_list)) {
core/lib/Drupal/Core/Extension/ModuleInstaller.php:334:      while (list($module) = each($module_list)) {
core/lib/Drupal/Core/Extension/ThemeInstaller.php:124:      while (list($theme) = each($theme_list)) {
core/modules/book/src/BookOutline.php:42:    // Assigning the array to $flat resets the array pointer for use with each().
core/modules/book/src/BookOutline.php:47:      list($key, $curr) = each($flat);
core/modules/book/src/BookOutline.php:81:    // Assigning the array to $flat resets the array pointer for use with each().
core/modules/book/src/BookOutline.php:84:      list($key,) = each($flat);
core/modules/language/tests/src/Functional/LanguageNegotiationInfoTest.php:164:        while ($equal && list($id) = each($negotiation)) {
core/modules/language/tests/src/Functional/LanguageNegotiationInfoTest.php:165:          list(, $info_id) = each($info['fixed']);
core/modules/rdf/tests/src/Kernel/Field/FieldRdfaTestBase.php:181:      // to an empty array to allow foreach(...) constructions.
core/modules/simpletest/src/AssertContentTrait.php:231:      // to an empty array to allow foreach(...) constructions.
core/modules/system/src/Form/ModulesListForm.php:389:    while (list($module) = each($modules['install'])) {
core/modules/system/src/Tests/Module/InstallUninstallTest.php:75:    while (list($name, $module) = each($all_modules)) {
core/modules/views/tests/src/Functional/Handler/FieldWebTest.php:184:      // to an empty array to allow foreach(...) constructions.
core/modules/views_ui/src/Form/Ajax/ViewsFormBase.php:107:      list($key, $top) = each($view->stack);
klausi’s picture

Issue tags: +Novice

Replacing those each() calls with proper foreach() loops is a good novice task :)

shashikant_chauhan’s picture

This issue is for drupal 7 or 8??

Ayesh’s picture

I think we better focus on D8 first, and then backport/fix for D7 too.

andypost’s picture

Version: 7.x-dev » 8.4.x-dev
Ayesh’s picture

Assigned: Unassigned » Ayesh

I will submit a patch for review shortly covering the lines listed above. Thanks Andrey for the list.

Ayesh’s picture

Ayesh’s picture

Status: Active » Needs review
klausi’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/Extension/ModuleInstaller.php
    @@ -93,8 +93,8 @@ public function install(array $module_list, $enable_dependencies = TRUE) {
    -      // the while loop continues.
    -      while (list($module) = each($module_list)) {
    +      // the foreach loop continues.
    +	  foreach ($module_list as $module => $value) {
    

    indentation error, make sure this is on the same level as the comment before.

  2. +++ b/core/modules/system/src/Form/ModulesListForm.php
    @@ -386,7 +386,7 @@ protected function buildModuleList(FormStateInterface $form_state) {
         // Add all dependencies to a list.
    -    while (list($module) = each($modules['install'])) {
    +	foreach ($modules['install'] as $module => $value) {
    

    indentation error.

After applying the patch 3 instances are missing:

modules/system/src/Tests/Module/InstallUninstallTest.php
75:    while (list($name, $module) = each($all_modules)) {

modules/language/tests/src/Functional/LanguageNegotiationInfoTest.php
164:        while ($equal && list($id) = each($negotiation)) {
165:          list(, $info_id) = each($info['fixed']);

includes/bootstrap.inc
1024:    while (list($key, $callback) = each($callbacks)) {
Ayesh’s picture

Status: Needs work » Active

Sorry yes I will get into that. I was trying to get drupal.org CI to run the tests (it takes forever on my computer for some reason today).

Ayesh’s picture

Status: Active » Needs work
pk188’s picture

Fixed 1 & 2 of #11. Patch applying cleanly in my system.

klausi’s picture

Status: Needs review » Needs work

Thanks, after applying the patch 3 instances are missing:

modules/system/src/Tests/Module/InstallUninstallTest.php
75:    while (list($name, $module) = each($all_modules)) {

modules/language/tests/src/Functional/LanguageNegotiationInfoTest.php
164:        while ($equal && list($id) = each($negotiation)) {
165:          list(, $info_id) = each($info['fixed']);

includes/bootstrap.inc
1024:    while (list($key, $callback) = each($callbacks)) {
arunkumark’s picture

Status: Needs work » Needs review
FileSize
8.34 KB

Hi All,

I have rerolled the patch #14 with fixing the comment #15.

shashikant_chauhan’s picture

Status: Needs review » Needs work

@arunkumark, Next time onwards kindly add interdiff file also with patch.
Here is the minor change.
foreach ($callbacks as $key => $callbacks) {
should be
foreach ($callbacks as $key => $callback) {

pk188’s picture

naveenvalecha’s picture

Thanks! looks good. checked on local. it's not present in any other PHP file.
$ grep --include=\*.php -rnw 'core/' -e 'each('

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 18: php72-each-function-deprecated-2885309-18.patch, failed testing. View results

drewklein’s picture

The change to DiffEngine.php will not necessarily work correctly, see https://3v4l.org/DZBS0.

Anonymous’s picture

@drewklein, good point! And thanks for this information here and your work in #2887904: remove uses of each() function. Can you provide a patch with a more correct version replacements? And interdiff with last patch of current issue.

Also maybe we can use iterator, to reduce the diff with DiffEngine.php? Something like https://3v4l.org/aj5JZ

andypost’s picture

Issue tags: +needs profiling

This change will happen in contrib so better to profile & file change record with suggestions

drewklein’s picture

dawehner’s picture

@drewklein
Do you think you are able to fix the failures?

Anonymous’s picture

#23:

I can not perform qualitative profiling, but simple time measurement (see test.php) gives the following results for array with 1M items:

iterator > 0.16423201560974
    each > 0.23989009857178
 foreach > 0.016668081283569

Intersting fact, when xdebug enabled each() works faster than iterator (or iterator works very slow):

iterator > 2.4610381126404
    each > 0.94104385375977
 foreach > 0.034947872161865
file change record with suggestions

done (if I understand correctly).

#24: thanks!

#25: done.

andypost’s picture

indeed iterator is slow(

Anonymous’s picture

If we use drewklein's idea about foreach (#2887904-5: remove uses of each() function), but change the injection to the wrapper condition, will this be a good compromise between minimize the amount of changes, performance and compatibility?

Diff with -w flag to avoid noise due to indentations.

drewklein’s picture

What's the status of this change?

drewklein’s picture

Status: Needs review » Needs work

The last submitted patch, 30: remove-each-drupal7.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

drewklein’s picture

vegantriathlete’s picture

Issue tags: +dcco2017

If you do not feel comfortable digging into the code for this, you could at least create a proper issue summary.

andypost’s picture

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

catch’s picture

Please open a separate issue for the 7.x backport, the 8.x patch should ideally be the last one on this issue so DrupalCI can retest it easily etc.

andypost’s picture

@catch d7 patch #30 is hidden in #34, so #28 is lat valid patch

xjm’s picture

Status: Reviewed & tested by the community » Needs review

Thanks for working on this issue!

+++ b/core/lib/Drupal/Component/Diff/Engine/DiffEngine.php
@@ -27,6 +27,14 @@
+
+/**
+ * Note on Drupal 8 porting.
+ * The following changes have been done:
+ *  Changed all calls of each() to alternatives for php 7.2 compatibility.
+ */
+
+
 class DiffEngine {

I'd add this to the docblock above it rather than adding a separate docblock. After all, it's adding a diff either way.

I think we should also add/enable a coder rule for this to ensure that we don't reintroduce each() use.

xjm’s picture

Status: Needs review » Needs work

Meant "Needs work", but we can also postpone the issue on adding the coder rule if one is not already available in the Drupal ruleset.

xjm’s picture

Oh, I forgot to mention, I grepped to confirm this is fixing all instances of each() once the patch is applied:

[ibnsina:drupal | Sun 14:24:49] $ grep -r "\beach(" * | grep -v "vendor" | grep -v ".js"
core/lib/Drupal/Component/Diff/Engine/DiffEngine.php: *  Changed all calls of each() to alternatives for php 7.2 compatibility.
vegantriathlete’s picture

Issue tags: -dcco2017
alexpott’s picture

We have Generic.PHP.DeprecatedFunctions sniff but that is unfortunately not picking this up on PHP7.2. I think that is because each is a built-in and therefore the way in which it detects deprecated functions does not work for this. We can add each to the list of functions in \Drupal_Sniffs_Functions_DiscouragedFunctionsSniff.

alexpott’s picture

Category: Task » Bug report
Priority: Normal » Critical
Issue summary: View changes
Status: Needs work » Needs review
FileSize
522 bytes
9.42 KB

As noted in #42 I think there is a place to make that change in coder that @xjm has requested but, on second thoughts, I'm not sure we should be adding deprecations of inbuilt functions to our coding standards checks. We don't have checks for all the changes in PHP7 from PHP5 - see http://php.net/manual/en/migration70.incompatible.php#migration70.incomp.... I think what we really need is a testbot for PHP7.2 - if we had that we have a gazillion failing tests and this would be one of the causes.

Wrt to the comment format that @xjm asked for in #38. I think what @vaplas was going for was to copy what is in core/lib/Drupal/Core/Archiver/ArchiveTar.php. That said that file has // @codingStandardsIgnoreFile and this does not. There are a couple of other things to add to the list so let's file a followup to address whether a comment needs to be added and just remove the comment altogether. I think there is a argument the comment is unnecessary and we should just be using git for this.

Also as this is producing warnings in PHP7.2 and that will out in November this issue is critical.

alexpott’s picture

Filed a followup for whether we should add a comment - see #2920903: Discuss whether a list of changes for Drupal 8 is needed in Diff class

Anonymous’s picture

Issue tags: -Novice

Looks nice! +1 to RTBC


I think what @vaplas was going for was to copy what is in core/lib/Drupal/Core/Archiver/ArchiveTar.php.

Absolutely true)

mondrake’s picture

Status: Needs review » Reviewed & tested by the community

Tried this patch, works fine :), RTBC

However, while this removes calls to each() in Drupal's codebase, PHP Unit tests still fail on PHP 7.2, apparently because PHP Unit itself has calls to each() in the release we can use, 4.8.36.

See https://github.com/sebastianbergmann/phpunit/blob/4.8/src/Util/Getopt.ph...

I checked the 5.7 branch and it looks like the calls to each() were prefixed with the error suppression operator @, see https://github.com/sebastianbergmann/phpunit/commit/20c70e0b72eb92367d00...

EDIT:

There will be no more releases for the 4.8 series, anyway :(, https://github.com/sebastianbergmann/phpunit/pull/2773

borisson_’s picture

We really should get a php7.2 testbot to verify this, but it looks great. RTBC++

catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed 7206f5d and pushed to 8.5.x. Thanks!

  • catch committed 7206f5d on 8.5.x
    Issue #2885309 by vaplas, drewklein, pk188, alexpott, Ayesh, arunkumark...
mondrake’s picture

Shoudn't this be pushed to 8.4.x, too? (assuming we want to support PHP 7.2 for it)

Triggered a test of #43 on 8.4.x.

catch’s picture

If we have a 7.2 DrupalCI environment passing tests with 8.5.x, then we could backport these patches then, but that seems pretty far off at the moment so not sure about backports individually.

JayKandari’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

Niktopia’s picture

Fresh install of Drupal 8.4.4 showing this issue on PHP 7.2.2 resulting in no styling/themes not working.

-- Install Site --
Warning: is_dir(): open_basedir restriction in effect. File(c:\winnt\temp) is not within the allowed path(s): (C:\Windows\Temp\) in Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory() (line 37 of core\lib\Drupal\Component\FileSystem\FileSystem.php).
Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory() (Line: 595)
system_requirements('install') (Line: 967)
drupal_check_profile('standard') (Line: 1976)
install_check_requirements(Array) (Line: 1028)
install_verify_requirements(Array) (Line: 671)
install_run_task(Array, Array) (Line: 549)
install_run_tasks(Array) (Line: 117)
install_drupal(Object) (Line: 44)

-- Configure Site --
Deprecated function: The each() function is deprecated. This message will be suppressed on further calls in Drupal\Core\Extension\ThemeInstaller->install() (line 124 of core\lib\Drupal\Core\Extension\ThemeInstaller.php).
Drupal\Core\Extension\ThemeInstaller->install(Array, 1) (Line: 164)
Drupal\Core\Extension\ThemeHandler->install(Array) (Line: 1563)
install_profile_themes(Array) (Line: 671)
install_run_task(Array, Array) (Line: 549)
install_run_tasks(Array) (Line: 117)
install_drupal(Object) (Line: 44)

-- Finish --
Error message
Deprecated function: The each() function is deprecated. This message will be suppressed on further calls in Drupal\Core\Extension\ThemeInstaller->install() (line 124 of core\lib\Drupal\Core\Extension\ThemeInstaller.php).
Drupal\Core\Extension\ThemeInstaller->install(Array, 1) (Line: 164)
Drupal\Core\Extension\ThemeHandler->install(Array) (Line: 1563)
install_profile_themes(Array) (Line: 671)
install_run_task(Array, Array) (Line: 549)
install_run_tasks(Array) (Line: 117)
install_drupal(Object) (Line: 44)
Warning: count(): Parameter must be an array or an object that implements Countable in Drupal\Core\Form\FormValidator->doValidateForm() (line 260 of core\lib\Drupal\Core\Form\FormValidator.php).
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object, 'install_configure_form') (Line: 119)
Drupal\Core\Form\FormValidator->validateForm('install_configure_form', Array, Object) (Line: 571)
Drupal\Core\Form\FormBuilder->processForm('install_configure_form', Array, Object) (Line: 314)
Drupal\Core\Form\FormBuilder->buildForm('install_configure_form', Object) (Line: 899)
install_get_form('Drupal\Core\Installer\Form\SiteConfigureForm', Array) (Line: 593)
install_run_task(Array, Array) (Line: 549)
install_run_tasks(Array) (Line: 117)
install_drupal(Object) (Line: 44)
Warning: count(): Parameter must be an array or an object that implements Countable in Drupal\Core\Form\FormValidator->doValidateForm() (line 260 of core\lib\Drupal\Core\Form\FormValidator.php).
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object, 'install_configure_form') (Line: 119)
Drupal\Core\Form\FormValidator->validateForm('install_configure_form', Array, Object) (Line: 571)
Drupal\Core\Form\FormBuilder->processForm('install_configure_form', Array, Object) (Line: 314)
Drupal\Core\Form\FormBuilder->buildForm('install_configure_form', Object) (Line: 899)
install_get_form('Drupal\Core\Installer\Form\SiteConfigureForm', Array) (Line: 593)
install_run_task(Array, Array) (Line: 549)
install_run_tasks(Array) (Line: 117)
install_drupal(Object) (Line: 44)
Warning: count(): Parameter must be an array or an object that implements Countable in Drupal\Core\Form\FormValidator->doValidateForm() (line 260 of core\lib\Drupal\Core\Form\FormValidator.php).
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object, 'install_configure_form') (Line: 119)
Drupal\Core\Form\FormValidator->validateForm('install_configure_form', Array, Object) (Line: 571)
Drupal\Core\Form\FormBuilder->processForm('install_configure_form', Array, Object) (Line: 314)
Drupal\Core\Form\FormBuilder->buildForm('install_configure_form', Object) (Line: 899)
install_get_form('Drupal\Core\Installer\Form\SiteConfigureForm', Array) (Line: 593)
install_run_task(Array, Array) (Line: 549)
install_run_tasks(Array) (Line: 117)
install_drupal(Object) (Line: 44)
Warning: count(): Parameter must be an array or an object that implements Countable in Drupal\Core\Form\FormValidator->doValidateForm() (line 260 of core\lib\Drupal\Core\Form\FormValidator.php).
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object) (Line: 239)
Drupal\Core\Form\FormValidator->doValidateForm(Array, Object, 'install_configure_form') (Line: 119)
Drupal\Core\Form\FormValidator->validateForm('install_configure_form', Array, Object) (Line: 571)
Drupal\Core\Form\FormBuilder->processForm('install_configure_form', Array, Object) (Line: 314)
Drupal\Core\Form\FormBuilder->buildForm('install_configure_form', Object) (Line: 899)
install_get_form('Drupal\Core\Installer\Form\SiteConfigureForm', Array) (Line: 593)
install_run_task(Array, Array) (Line: 549)
install_run_tasks(Array) (Line: 117)
install_drupal(Object) (Line: 44)

bkosborne’s picture

All - I'm helping with the D7 version of this issue and I think I found a bug with this implementation.

This patch changed the following code in ModuleInstaller from:

      // Add dependencies to the list. The new modules will be processed as
      // the while loop continues.
      while (list($module) = each($module_list)) {
        foreach (array_keys($module_data[$module]->requires) as $dependency) {
          if (!isset($module_data[$dependency])) {
            // The dependency does not exist.
            throw new MissingDependencyException("Unable to install modules: module '$module' is missing its dependency module $dependency.");
          }

          // Skip already installed modules.
          if (!isset($module_list[$dependency]) && !isset($installed_modules[$dependency])) {
            $module_list[$dependency] = $dependency;
          }
        }
      }

to

      // Add dependencies to the list. The new modules will be processed as
      // the foreach loop continues.
      foreach ($module_list as $module => $value) {
        foreach (array_keys($module_data[$module]->requires) as $dependency) {
          if (!isset($module_data[$dependency])) {
            // The dependency does not exist.
            throw new MissingDependencyException("Unable to install modules: module '$module' is missing its dependency module $dependency.");
          }

          // Skip already installed modules.
          if (!isset($module_list[$dependency]) && !isset($installed_modules[$dependency])) {
            $module_list[$dependency] = $dependency;
          }
        }
      }

By changing the outer loop to a foreach, I believe that means any dependencies that are added to the module list are not checked to see if they exist or not, because the foreach will never get to them. Foreach operates on a copy of the original array. Instead I think what's needed is to manually iterate:

      // Add dependencies to the list.
      // The array is iterated over manually (instead of using a foreach) because
      // modules may be added to the list within the loop and we need to process
      // them.
      while ($module = key($module_list)) {
        next($module_list);
        foreach (array_keys($module_data[$module]->requires) as $dependency) {
          if (!isset($module_data[$dependency])) {
            // The dependency does not exist.
            throw new MissingDependencyException("Unable to install modules: module '$module' is missing its dependency module $dependency.");
          }

          // Skip already installed modules.
          if (!isset($module_list[$dependency]) && !isset($installed_modules[$dependency])) {
            $module_list[$dependency] = $dependency;
          }
        }
      }

We tried using a foreach for this in the D7 version of the patch but tests were failing.

bkosborne’s picture

@alexpott found that this is not actually a bug due to how ->requires works on modules, but the code should be cleaned up anyway. Created follow up: #2950400: Clean up confusing code in ModuleInstaller that checks for dependencies

aadeshvermaster@gmail.com’s picture

When I installing the drupal 8.3.7 on our local machine that has php 7.2, I got this issue on Configuration Site page. After replacing the each loop with foreach loop and install again then i did not get this issue. I have replaced the file in following manner

- while (list($theme) = each($theme_list)) {
+ foreach ($theme_list as $theme => $value) {