Problem / Motivation
Only some specific static caches are reset before running hook_module_install() and related module-hooks. This means that during the request in which the module was enabled/disabled some of the cached data might be stale. This is particularly cumbersome during site installation via drush where lots of modules are enabled at once.
This has lead to several hard-to-track-down issues in contrib:
- #2352739: Reset static caches on module enable/disable.
- #1681414: Enabling i18n modules fail in install profile
- #1376122: Stream wrappers of parent site are leaking into all tests
- Any ctools exportable has this issue (ie. hook_file_default_display).
Perhaps there are others where the root-cause hasn't been identified or simple workarounds like that for payment are in place.
Affected are install profiles, drush installs and some feature reverts.
Proposed solution
Call drupal_static_reset() at the appropriate place in module_enable() and module_disable().
Possible issues with the proposed solution
The consequences are limited to a slightly (measureable?) degraded performance due to the extra cache flushes. Changes of functionality due to a cache flush would be a bug in itself.
| Comment | File | Size | Author |
|---|---|---|---|
| #26 | drupal-1891356-install_uninstall_reset_static-26.patch | 3.4 KB | JvE |
| #25 | drupal-1891356-module_enable_static-D7.patch | 699 bytes | rob c |
| #30 | 1891356-drupal_static_reset-on-module-changes-30-D7.patch | 1001 bytes | torotil |
Comments
Comment #1
JvE commentedA test patch and a test+fix patch.
Comment #2
JvE commentedA test patch and a test+fix patch.
Comment #4
JvE commentedLine endings..
Comment #6
JvE commented#4: drupal-1891356-module_enable_static-3.patch queued for re-testing.
Comment #8
JvE commentedI do not understand the testbot failure at all and I am unable to reproduce it locally.
Running all tests succeeds here, both through the browser and through run-tests.sh
/usr/bin/php ./core/scripts/run-tests.sh --concurrency 8 --php /usr/bin/php --url 'http://d8.local' --keep-results --all
I'm not sure what to do now.
Comment #9
JvE commented#4: drupal-1891356-module_enable_static_testonly-3.patch queued for re-testing.
Comment #10
JvE commented#4: drupal-1891356-module_enable_static-3.patch queued for re-testing.
Comment #12
Yaron Tal commentedIssue also applies here:
#2088375: Subscription types statics in some circumstances cause "Undefined permission" error when applying feature
Comment #13
JvE commentedIt's been a while. Let's give this another go.
Comment #15
JvE commented13: drupal-1891356-install_uninstall_reset_static-13.patch queued for re-testing.
Comment #17
JvE commented13: drupal-1891356-install_uninstall_reset_static-13.patch queued for re-testing.
Comment #20
JvE commented13: drupal-1891356-install_uninstall_reset_static-13.patch queued for re-testing.
Comment #22
JvE commentedComment #25
rob c commentedAdding D7patch. All credits to JvE.
Comment #26
JvE commentedI was working in branch 8.x rather than 8.0.x
Comment #27
jose reyero commented+1
This causes lots of other apparently random issues when enabling other contrib modules, like #1681414: Enabling i18n modules fail in install profile
Comment #28
JvE commentedComment #29
torotil commentedMarking this as bug (it is one) and as major since it affects several widely used contrib modules. I'd guess it has already cost thousands of dev-hours.
Comment #30
torotil commentedI'm updating the D7 patch. IMHO the static cache must be flushed before hook_install is invoked.
Comment #31
torotil commented@JvE: would be nicer if you also migrated all the information of the bug that you mark as duplicate.
Comment #33
torotil commentedComment #34
rob c commented@torotil apologies, i linked it in that issue, i should have marked it / added additional info.
I know about more modules that are affected (D7) (rules, feature up to some level, etc), might it be a better idea to create a META issue? Because all these modules need patches and we need some list to track these.
Comment #35
torotil commentedNp. Thanks for marking the duplicate. I haven't found the D8 issue when reporting the D7 issue.
I don't think so. For some instances of the problem there is no easy workaround. So I think this must be solved in core. Also there it would be a waste of dev-time to find every single instance where the stale cache is a problem.
Let's elaborate a bit more on possible work arounds.
At least in D7 the order of hooks called when enabling a module is: hook_install, hook_enable, hook_module_installed, hook_module_enabled. Which workaround is possible depends on when the new information is first used (wrt the hooks), and whether the module being enabled is the one doing the caching.
A common case is that some module provides a hook and caches its results. Another module implements this hook and wants to use some functionality of its dependence in hook_install() (ie. a module that defines field-type and then creates a default field(instance) for this type). This is the worst case. The only way outside of core to solve this is in hook_install() - and that in every single module that implements the hook.
So what have we got?
This is 5 good reasons to get this into core ASAP. So I'd rather go for marking contrib-bugs as duplicates of this.
Comment #36
rob c commentedI agree 100%, the meta was just an idea to facilitate contrib a bit. A list of commonly used modules + the status of their issue would be handy. With this, developers can see how other modules implement their changes for this patch and site builders have some insight into the status of the projects they are using regarding this patch. (patch available, fixed in dev, released). The meta was not for core, i'm all in favour of this patch, i'm using it, that's how i started to find other modules with related issues.
The testbot will test the D7 patch against D8 HEAD - and fail, because this first needs to be fixed in D8 (the issue is marked as a D8 issue), that's another reason i believe an additional post will come in hand. It might be some time before this gets committed, so people will be running a patched core + contrib for at least a couple months/years (see the initial post date).
#26 is still current, back to needs review.
Comment #37
torotil commentedAh right - So why don't we use this issue as tracker for all contrib-bugs triggered by this?
Comment #38
rob c commentedSure, that will also work for now. Some can always create the page at a later time (if the list explodes).
Comment #39
jose reyero commentedAgree with @Rob C, we don't want contrib module issues to be posted not followed up here. I think if we expect to ever get the patch in we should keep it clean, focused and up to the point.
So we may need a meta-issue, that maybe if we mark it for D7 we can also use to post and follow up on the D7 patch. If not, we can just link other issues here using "Related issues"
Also since this may have a wider impact on installation process, it may need some performance assessment, like: How does this change the installation time for Drupal core and/or distributions (install profiles) with many modules? (We can try for D7)
Comment #40
JvE commentedWe've been using the D7 patch (#25) for over two years now on dozens of sites without any issues (performance or otherwise).
As usual, this needs to go into D8 first, and never without tests.
Comment #41
FreekVR commentedRelating #2346035 as it was caused by a similar issue - in that case - was resolved by clearing a specific static. Any generic solution for D8 here would make part of the commit in that issue redundant. (http://cgit.drupalcode.org/drupal/diff/core/lib/Drupal/Core/Extension/Mo...)
Comment #44
torotil commentedThis issue is impossible to be marked as "Needs review" because the testbot always retests the D7 patch (#30) on D8 instead of the patch for D8 in #25. I try "hiding" the D7 patch for now.
Comment #46
torotil commentedStil #26 should be reviewed and committed ASAP so that we can finally fix this in D7.
Comment #47
cilefen commentedRe-post the D8 patch.
Comment #49
geek-merlin@torotil #44: i had success in other issues by
* changing issue version to 7.x and uploading 7.x patch (filename indicating 7.x/8.x gives extra points ;-)
* reverting issue version to 8.x and uploading 8.x patch
Comment #51
eelkeblokComment #52
eelkeblokComment #53
eelkeblokApparently it is now possible to configure custom testing parameters per patch. I did so for the D7 patch (#25). Not sure why it won't let me do the same for #30.
Comment #54
eelkeblokComment #55
eelkeblokComment #57
JvE commentedI think the test system may be depending on the bug so it cannot test the fix :)
Comment #59
catchDowngrading to normal, there are 47 calls to drupal_static in 8.x now and we have no known issues with module install and those caches any more. Better to refactor the code using drupal_static() at this point.
Comment #60
grayle commentedI might have found an issue with this patch, but I'm still in the testing phases. But I can say is that when I undo the patch, everything works.
Currently during my install profile, with this patch applied, I get the following warnings:
Which are drupal_schema_field_types() and drupal_schema_fields_sql(). Essentially, drupal_get_schema returns empty schemas when installing new modules. If I change the calls to drupal_get_schema() in those two functions to include $rebuild = true, everything works too.
But it only happens once my install profile reaches the redirect and entityform modules, so there may be a bug elsewhere (in my install profile or in a contrib module) which is just being brought to light due to this patch.
The result of that warning is fairly severe though, because those tables aren't getting created so a bit down the line everything breaks horrendously.
Anyway, I'll keep testing, but I was wondering why drupal_get_schema's 'fallback' when it has no cache fails at picking up schemas that are in the process of being created.
edit: it's about the patch in #30 for D7
Comment #63
torotil commented@catch: There are still 247 calls to
drupal_static()in D7 out there and at least some bugs caused by it in popular contrib modules. Do you suggest opening another bug report for D7 with a higher priority?Comment #68
torotil commentedI have been bitten by this again. As this has been almost resolved in D8/D9 by switching away from
drupal_static()(#59) I’m resetting the version and priority for D7.Comment #69
torotil commented