Problem / Motivation

Only some specific static caches are reset before running hook_module_install() and related module-hooks. This means that during the request in which the module was enabled/disabled some of the cached data might be stale. This is particularly cumbersome during site installation via drush where lots of modules are enabled at once.

This has lead to several hard-to-track-down issues in contrib:

Perhaps there are others where the root-cause hasn't been identified or simple workarounds like that for payment are in place.

Affected are install profiles, drush installs and some feature reverts.

Proposed solution

Call drupal_static_reset() at the appropriate place in module_enable() and module_disable().

Possible issues with the proposed solution

The consequences are limited to a slightly (measureable?) degraded performance due to the extra cache flushes. Changes of functionality due to a cache flush would be a bug in itself.

Comments

JvE’s picture

A test patch and a test+fix patch.

JvE’s picture

Status: Active » Needs review

A test patch and a test+fix patch.

Status: Needs review » Needs work

The last submitted patch, drupal-1891356-module_enable_static-2.patch, failed testing.

JvE’s picture

Status: Needs work » Needs review
StatusFileSize
new2.49 KB
new1.48 KB

Line endings..

Status: Needs review » Needs work

The last submitted patch, drupal-1891356-module_enable_static-3.patch, failed testing.

JvE’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, drupal-1891356-module_enable_static-3.patch, failed testing.

JvE’s picture

I do not understand the testbot failure at all and I am unable to reproduce it locally.

Running all tests succeeds here, both through the browser and through run-tests.sh
/usr/bin/php ./core/scripts/run-tests.sh --concurrency 8 --php /usr/bin/php --url 'http://d8.local' --keep-results --all

I'm not sure what to do now.

JvE’s picture

Status: Needs work » Needs review
JvE’s picture

Status: Needs review » Needs work

The last submitted patch, drupal-1891356-module_enable_static-3.patch, failed testing.

JvE’s picture

Issue summary: View changes
Status: Needs work » Needs review
StatusFileSize
new3.17 KB

It's been a while. Let's give this another go.

Status: Needs review » Needs work

The last submitted patch, 13: drupal-1891356-install_uninstall_reset_static-13.patch, failed testing.

JvE’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 13: drupal-1891356-install_uninstall_reset_static-13.patch, failed testing.

JvE’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 13: drupal-1891356-install_uninstall_reset_static-13.patch, failed testing.

The last submitted patch, 13: drupal-1891356-install_uninstall_reset_static-13.patch, failed testing.

JvE’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 13: drupal-1891356-install_uninstall_reset_static-13.patch, failed testing.

JvE’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 13: drupal-1891356-install_uninstall_reset_static-13.patch, failed testing.

rob c’s picture

StatusFileSize
new699 bytes

Adding D7patch. All credits to JvE.

JvE’s picture

Status: Needs work » Needs review
StatusFileSize
new3.4 KB

I was working in branch 8.x rather than 8.0.x

jose reyero’s picture

+1

This causes lots of other apparently random issues when enabling other contrib modules, like #1681414: Enabling i18n modules fail in install profile

torotil’s picture

Category: Feature request » Bug report
Priority: Normal » Major
Issue summary: View changes

Marking this as bug (it is one) and as major since it affects several widely used contrib modules. I'd guess it has already cost thousands of dev-hours.

torotil’s picture

StatusFileSize
new1001 bytes

I'm updating the D7 patch. IMHO the static cache must be flushed before hook_install is invoked.

torotil’s picture

@JvE: would be nicer if you also migrated all the information of the bug that you mark as duplicate.

Status: Needs review » Needs work

The last submitted patch, 30: 1891356-drupal_static_reset-on-module-changes-30-D7.patch, failed testing.

torotil’s picture

Issue summary: View changes
rob c’s picture

@torotil apologies, i linked it in that issue, i should have marked it / added additional info.

I know about more modules that are affected (D7) (rules, feature up to some level, etc), might it be a better idea to create a META issue? Because all these modules need patches and we need some list to track these.

torotil’s picture

Np. Thanks for marking the duplicate. I haven't found the D8 issue when reporting the D7 issue.

… might it be a better idea to create a META issue? Because all these modules need patches and we need some list to track these.

I don't think so. For some instances of the problem there is no easy workaround. So I think this must be solved in core. Also there it would be a waste of dev-time to find every single instance where the stale cache is a problem.

Let's elaborate a bit more on possible work arounds.

At least in D7 the order of hooks called when enabling a module is: hook_install, hook_enable, hook_module_installed, hook_module_enabled. Which workaround is possible depends on when the new information is first used (wrt the hooks), and whether the module being enabled is the one doing the caching.

A common case is that some module provides a hook and caches its results. Another module implements this hook and wants to use some functionality of its dependence in hook_install() (ie. a module that defines field-type and then creates a default field(instance) for this type). This is the worst case. The only way outside of core to solve this is in hook_install() - and that in every single module that implements the hook.

So what have we got?

  • It's a 2 line patch in core, that is unlikely to affect any functionality.
  • It affects multiple widely used contrib modules.
  • It's likely that there are numerous not-yet-found issues out there due to this bug.
  • Workarounds are much harder than the actual fix.
  • In general those issues are rather hard to track down.

This is 5 good reasons to get this into core ASAP. So I'd rather go for marking contrib-bugs as duplicates of this.

rob c’s picture

Status: Needs work » Needs review

I agree 100%, the meta was just an idea to facilitate contrib a bit. A list of commonly used modules + the status of their issue would be handy. With this, developers can see how other modules implement their changes for this patch and site builders have some insight into the status of the projects they are using regarding this patch. (patch available, fixed in dev, released). The meta was not for core, i'm all in favour of this patch, i'm using it, that's how i started to find other modules with related issues.

The testbot will test the D7 patch against D8 HEAD - and fail, because this first needs to be fixed in D8 (the issue is marked as a D8 issue), that's another reason i believe an additional post will come in hand. It might be some time before this gets committed, so people will be running a patched core + contrib for at least a couple months/years (see the initial post date).

#26 is still current, back to needs review.

torotil’s picture

Ah right - So why don't we use this issue as tracker for all contrib-bugs triggered by this?

rob c’s picture

Sure, that will also work for now. Some can always create the page at a later time (if the list explodes).

jose reyero’s picture

Agree with @Rob C, we don't want contrib module issues to be posted not followed up here. I think if we expect to ever get the patch in we should keep it clean, focused and up to the point.

So we may need a meta-issue, that maybe if we mark it for D7 we can also use to post and follow up on the D7 patch. If not, we can just link other issues here using "Related issues"

Also since this may have a wider impact on installation process, it may need some performance assessment, like: How does this change the installation time for Drupal core and/or distributions (install profiles) with many modules? (We can try for D7)

JvE’s picture

We've been using the D7 patch (#25) for over two years now on dozens of sites without any issues (performance or otherwise).

As usual, this needs to go into D8 first, and never without tests.

FreekVR’s picture

Relating #2346035 as it was caused by a similar issue - in that case - was resolved by clearing a specific static. Any generic solution for D8 here would make part of the commit in that issue redundant. (http://cgit.drupalcode.org/drupal/diff/core/lib/Drupal/Core/Extension/Mo...)

Status: Needs review » Needs work

The last submitted patch, 30: 1891356-drupal_static_reset-on-module-changes-30-D7.patch, failed testing.

torotil’s picture

Status: Needs work » Needs review

This issue is impossible to be marked as "Needs review" because the testbot always retests the D7 patch (#30) on D8 instead of the patch for D8 in #25. I try "hiding" the D7 patch for now.

The last submitted patch, 25: drupal-1891356-module_enable_static-D7.patch, failed testing.

torotil’s picture

Stil #26 should be reviewed and committed ASAP so that we can finally fix this in D7.

cilefen’s picture

This issue is impossible to be marked as "Needs review" because the testbot always retests the D7 patch

Re-post the D8 patch.

The last submitted patch, 25: drupal-1891356-module_enable_static-D7.patch, failed testing.

geek-merlin’s picture

@torotil #44: i had success in other issues by
* changing issue version to 7.x and uploading 7.x patch (filename indicating 7.x/8.x gives extra points ;-)
* reverting issue version to 8.x and uploading 8.x patch

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

eelkeblok’s picture

Version: 8.1.x-dev » 7.x-dev
eelkeblok’s picture

Version: 7.x-dev » 8.3.x-dev
eelkeblok’s picture

Apparently it is now possible to configure custom testing parameters per patch. I did so for the D7 patch (#25). Not sure why it won't let me do the same for #30.

eelkeblok’s picture

eelkeblok’s picture

The last submitted patch, 25: drupal-1891356-module_enable_static-D7.patch, failed testing.

JvE’s picture

I think the test system may be depending on the bug so it cannot test the fix :)

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

catch’s picture

Priority: Major » Normal

Downgrading to normal, there are 47 calls to drupal_static in 8.x now and we have no known issues with module install and those caches any more. Better to refactor the code using drupal_static() at this point.

grayle’s picture

I might have found an issue with this patch, but I'm still in the testing phases. But I can say is that when I undo the patch, everything works.

Currently during my install profile, with this patch applied, I get the following warnings:

Invalid argument supplied for foreach() common.inc:7229                                                                                                                                                                                                            [warning]
array_keys() expects parameter 1 to be array, null given common.inc:7249      

Which are drupal_schema_field_types() and drupal_schema_fields_sql(). Essentially, drupal_get_schema returns empty schemas when installing new modules. If I change the calls to drupal_get_schema() in those two functions to include $rebuild = true, everything works too.
But it only happens once my install profile reaches the redirect and entityform modules, so there may be a bug elsewhere (in my install profile or in a contrib module) which is just being brought to light due to this patch.

The result of that warning is fairly severe though, because those tables aren't getting created so a bit down the line everything breaks horrendously.

Anyway, I'll keep testing, but I was wondering why drupal_get_schema's 'fallback' when it has no cache fails at picking up schemas that are in the process of being created.

edit: it's about the patch in #30 for D7

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

torotil’s picture

@catch: There are still 247 calls to drupal_static() in D7 out there and at least some bugs caused by it in popular contrib modules. Do you suggest opening another bug report for D7 with a higher priority?

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

torotil’s picture

Title: Reset drupal static caches when a module is enabled or disabled. » D7: Reset drupal static caches when a module is enabled or disabled.
Version: 9.1.x-dev » 7.x-dev
Priority: Normal » Major

I have been bitten by this again. As this has been almost resolved in D8/D9 by switching away from drupal_static() (#59) I’m resetting the version and priority for D7.

torotil’s picture

Status: Needs review » Closed (outdated)

Automatically closed because Drupal 7 security and bugfix support has ended as of 5 January 2025. If the issue verifiably applies to later versions, please reopen with details and update the version.