Problem/Motivation
if ($this->state->get('install_task') == 'done') {
$threshold = $this->config->get('threshold.autorun');
if ($threshold > 0) {
$cron_next = $this->state->get('system.cron_last', 0) + $threshold;
if (REQUEST_TIME > $cron_next) {
$this->cron->run();
}
}
}
Whether autocron is enabled or not, we have an i/o hit to check if install has finished (!!!!) then another one for the config object (which is cached, but otherwise not needed at all though).
In Drupal 7, this was two variable_get() so they didn't really get noticed.
Proposed resolution
1. Move the automatic cron subscriber into a module so it can be completely disabled if not needed.
OR
1. There must be a way to avoid the install_task check - this is a query every page to check something that happens literally once in the lifetime of the site.
2. Either way we should check config first before checking the install state, to avoid any further work if automatic cron isn't configured. Checking state when automatic cron is disable anyway is really bad.
3. If for some reason #1 isn't possible, then the two state gets could be combined into a single ->getMultiple()
This will save one database query on every request when automatic cron is configured, and also one when it isn't.
Remaining tasks
User interface changes
API changes
| Comment | File | Size | Author |
|---|---|---|---|
| #154 | interdiff.txt | 357 bytes | dawehner |
| #154 | 2507031-154.patch | 32.83 KB | dawehner |
| #152 | 2507031-cron-152.patch | 32.77 KB | wim leers |
| #93 | automated_cron-2507031-93.patch | 27.96 KB | wim leers |
| #92 | interdiff.txt | 573 bytes | dawehner |
Comments
Comment #1
catchThis might be enough.
Comment #3
dawehnerI was wondering whether it would be a good idea to no register that subscriber, unless you really need it. This could be for example achieved by moving the automatic cron into its own module.
Comment #4
catchThat's a good idea. Then we can probably skip the entire maintenance mode/install check since it'll be added later in the install process.
Comment #5
berdirA module would be interesting, can we still do this? We could also dynamically register this, with the downside that changing the setting needs to rebuild the container. We did move page cache to a module to avoid exactly that (among other reasons).
Comment #6
catchYes it's not an API change, and we can enable it in either standard profile or minimal profile or both, so it wouldn't be a functionality change either. This is a significant i/o improvement, both for sites that disable the module and those that enable it (just as page cache was).
The new module itself might need approval from webchick.
I'd like to avoid dynamically rebuilding the container - I know we do this for languages but I'm still holding out hope that we'll be able to factor that away somehow and restrict it to module install/uninstall.
Comment #7
damienmckennaIf it's a new module, should it be enabled in the standard installation profile?
Comment #8
catchComment #9
catchI am the wrong person to add that tag usually, here's the right one.
Comment #10
catchComment #11
dawehnerI agree that we should try to avoid container rebuilds as much as possible. Moving to a module is an elegant way around that in the sense, that people are more aware what they are doing, when they install a new module.
Comment #12
dawehnerLet's try it out.
interdiff doesn't make sense.
Comment #14
mitrpaka commentedCorrected missing ':'
Comment #16
mitrpaka commentedComment #18
mitrpaka commentedState key returned back in order to pass tests.
Comment #20
dawehnerLet's ensure that we fill this out :)
The only remaining bet in order to fix the test is to enable the new module in
\Drupal\system\Tests\System\CronRunTest::testAutomaticCron.Comment #21
mitrpaka commented@dawehner Thanks for helping out to fix all tests.
Basic help text (or at least start of it) added.
Cron settings page is now defined in core\modules\system\src\Form\CronForm.php. Should we move autocron related parts to newly created module as well or disable cron settings if autocron module is completely disabled?
Comment #22
dawehnerYeah we should move that into a configuration on this module itself ... so we need to migrate the data via hook_update_N() in system and install the module there
Comment #23
mitrpaka commentedHere is first attempt where automatic cron configuration included in the module (with use of hook_form_FORM_ID_alter() and custom submit callback).
System update hook included.
Comment #24
dawehnerNote: you need to also write a test for it.
Comment #26
mitrpaka commentedInitial test added.
Comment #27
claudiu.cristeaIf we moved the functionality to a module that can be disabled, let's drop the "never" option. Disabling the module is the new "Never".
Maybe
array $form?@var \Drupal\Core\State\StateInterface?The system module is always enabled regardless of installation profile.
I don't think we want to enable the module in any circumstance. I would test the config
cron_safe_thresholdand I would enable the module only ifcron_safe_threshold> 0.Comment #28
claudiu.cristeaBecause this is a new module, let's use the short syntax for arrays.
[]instead ofarray().Comment #29
berdirI think a Never option is still useful when you want to disable it temporarily or on some environment. That's where #6 is wrong. We don't save anything for sites that have the module enabled as we still need to check how often we need to do it. Checking for never there is no overhead.
Comment #30
dawehnerI agree with berdir that being able to temporary disable it is not a bad idea.
Comment #31
webflo commentedAdded the new module to
core/composer.json.Comment #32
webflo commentedSmaller patch file with
git diff -M.Comment #33
webflo commentedI think we should the related config from system module to automated_cron.
Comment #34
andypost"@var \" missing
state is passed and used
how could it be lees then 0?
useless variable
Comment #35
claudiu.cristeaAddressed points:
automated_cron).threshold.autorunfromsystem.cronto the newautomated_cronconfig.FormBasebecause withoutautomated_cronmodule enabled, no more handles a config vars.automated_cronmodule is disabled, the Cron Settings form displays a help message that tells admin he can enable the module.@andypost, #34.3: Se #29 — the autorun can take the 0 value.
Comment #37
claudiu.cristeaSorry, patches from #35 were incomplete. Consider the comment from #35together with these patches.
Comment #38
claudiu.cristeaAnd the interdiff :)
Comment #40
claudiu.cristeaComment #41
tstoeckler1. I think we should delete the system.cron config object in the upgrade path.
2. Tagging "Needs documentation" so we can see whether the hook_help() implementation passes @jhodgdon's scrutiny. I thought there was a "Needs documentation review" tag but apparently there isn't, so not sure if that's the right one.
Comment #42
claudiu.cristea@tstoeckler
We only split a key from
system.cronand move it toautomated_cron.cron. We need both.Comment #43
tstoeckler@claudiu.cristea Right, sorry about that. But we should
clearthe obsolete key in system.cron, no?Comment #44
claudiu.cristeaComment #46
claudiu.cristeaNitpick.
Comment #47
berdirconfig has a clear() method to delete stuff.
also $system_config seems to be using \Drupal::config(), which is immutable?
Both this and the one above should pass TRUE to save() to avoid config casting/validation.
Comment #48
dawehnerDo we really need this additional message for the altered in value? I don't see that we remove any usage of drupal_set_message() in this patch so we are effectively adding one. This feels wrong :)
This sounds like a good compromise for site builder experience vs. separation of concerns.
Comment #49
claudiu.cristeaIn fact we are removing that message by downgrading the form from ConfigFormBase to FormBase and the user will see no feedback on his action.
Comment #50
claudiu.cristeaAddressed #47. Responded in #49 on #48 about message.
Comment #51
tstoecklerAwesome work, I have one more minor nitpick:
Good point, that means we should also bring over the:
from
ConfigFormBaseto allow it to be styled consistently with other config forms.Comment #52
claudiu.cristeaOk with #51.
But now, alternatively, I feel that there's a more cleaner way to alter the form, without using
hook_form_FORM_ID_alter(). I would:system.cron_settings('/admin/config/system/cron') to point to a new form callback owned by the new module.\Drupal\Core\Form\ConfigFormBase\Drupal\system\Form\CronForm) injected just to grab all needed elements (e.g.::buildForm()result).This would make unnecessary all the redundancies added from \Drupal\Core\Form\ConfigFormBase into our FormBase form.
Comment #53
claudiu.cristeaRight now I think I like this approach more than #52. Seems more clean to me.
Comment #54
catchThis should be able to follow the same pattern as dblog/syslog. Examples via @alexpott - I couldn't find the right example.
https://api.drupal.org/api/drupal/core%21modules%21dblog%21dblog.module/... / https://api.drupal.org/api/drupal/core%21modules%21dblog%21dblog.module/...
Comment #55
claudiu.cristea@catch, #52 is like that.
But the CronForm becomes a simple FormBase (is losing the config functionality). This is forcing us to add some redundancies to the form. IMO #53 is much more cleaner because is using what ConfigFormBase already provides.
Comment #56
catchLoggingForm extends ConfigFormBase though: https://api.drupal.org/api/drupal/core!modules!system!src!Form!LoggingFo...
Comment #57
catchI didn't realise we'd downgraded the form, from ConfigFormBase because it's no longer a config form at all by default, that does make this more tricky.
However #52 seems like the better option here - we need to allow for multiple modules altering the form to add their own settings, and they can't all alter the route at the same time to extend the form.
Comment #58
claudiu.cristeaHiding solution from #53. #52 needs review.
Comment #61
claudiu.cristeaRerolled #52.
Comment #66
catchGiven this adds a new module it's something I think we need to only do before rc or in a minor version, unless we think the performance improvement makes it worth the minor disruption during rc (upgrade path and new strings to translate).
Comment #67
piyuesh23 commentedPatch doesn't apply to 8.0.x head. Adding needs reroll tag
Comment #68
piyuesh23 commentedUploading the re-rolled patch here.
Comment #71
dawehnerLooking into the test failures.
Comment #72
dawehnerThere we go.
Comment #74
berdirWorth mentioning that you should disable the module if you have a separate cron job running so that you actually benefit from the improved performance?
normally we use ..settings.yml if there's just a single file, not sure? Especially since the module name already contains cron.
Comment #76
dawehnerThis is not it yet.
Comment #77
dawehnerFinally ...
Comment #82
dawehnerComment #85
fabianx commentedRTBC - but there are test fails remaining.
Comment #86
webchickSeems reasonable to me, and is inline with what we did with Page Cache. Just so long as it's enabled by default in Standard (it is) and has hook_help() (it does), works for me.
Comment #89
dawehnerJust a simple reroll
Comment #92
dawehnerIncrease MOAR
Comment #93
wim leersReviewing, and addressing most of my nits immediately.
would be clearer I think. Thoughts?
… wouldn't you actually want to uninstall this module instead?
Fixed.
Wouldn't it be better if this were keyed under
'automated_cron'instead?Fixed.
Missing leading backslash.
Fixed.
Everywhere it's called "automated" but here "automatic".
Fixed.
Wouldn't it be better to only have a single value here, with the name
intervalor something like that?Fixed.
Multiple? There can only ever be one?
If multiple, then the key should also match this?
Fixed by doing the above.
This explains the config structure and naming scheme in the
automated_cronmodule, but it's not a valid justification IMO.s/Automatic/Automated/
Fixed.
Comment #96
dawehnerI don't care
Good suggestion.
Comment #97
wim leersOopsie in schema.
Comment #98
wim leersAddressed #93.2 / #96.2.
Comment #99
wim leersSigh, I uploaded the right interdiff but the wrong patch in #98. Here's what I actually intended to upload.
Comment #101
wim leersI have no idea how to fix that last failure, that line 85 in UpdatePathTestBaseFilledTest.
dawehner and I suspected it's related to running cron triggering some translation stuff. But it's specifically testing the translation of the site slogan. The site slogan lives in the "site branding" block (since September 10, #2005546: Use branding block in place of page template branding variables (site name, slogan, site logo)). But the tests in HEAD are still passing. And cron doesn't seem involved in any way in config translation. And the other blocks (menu block, user login block) are translated just fine.
Furthermore, no slogan at all shows up.
The translation string that we check for (
drupal Spanish) lives in the DB dump. I have no idea at all how this patch could possibly break that. Any ideas?Comment #103
jhodgdonWim asked me to review documentation. Looks mostly good! A few small items to address:
The capitalization should be:
Automated Cron
for a module name. Also I think we normally put multiple-word module names in ' ' quotes in yml files?
This description is used on the Extend (admin/modules) page. It needs to be understandable to non-programmers. I think this probably needs a better description.
Also grammar: "it" does not have a clear antecedent...
So my suggestion would be:
Provides an automated way to run cron jobs, by executing them at the end of a page request.
This needs to start with:
The Automated Cron module ...
And anytime you refer to the module by name, it should say:
... the Automated Cron module ...
(note capitalization and always use the actual module name from the .info.yml file)
Also see note on the yml file. Do not use programming or overly technical terms like "response".
Also we don't normally want to use the word "Drupal" in help or UI text. Refer to "your site" or "the site" or something like that instead.
The end of the About should always be something like:
For more information, see the [a href=https://www.drupal.org/documentation/modules/SHORT_NAME]on-line documentation for the Foo Bar module[/a] as well. This page will need to be created. If this link text is not present, a test will fail once #2488032: Integrate help test into module uninstall test goes in.
nitpick: there are two spaces after the comma in the first sentence.
Also I don't think you need to put in the help what the default setting is... but maybe just say "... you can set the frequency for running cron jobs", since I think that is the only setting?
Also maybe it should document what the "frequency" means, because it might not be clear.
Um... so this is clear enough but it's a bit conversational (e.g., "pointless").
How about a slight rewording:
To disable automated cron, the recommended method is to uninstall the module, to reduce site overhead. If you only want to disable it temporarily, you can set the frequency to Never on the Cron page, and then change the frequency back when you want to start it up again.
I think normally when we document an implementation of this hook, we'd add
for the system_cron_settings() form.
at the end of the sentence?
Construct => Constructs
Is this module check needed? It seems like the test would know whether the automated cron module is enabled or not in its testing profile.
Module name should be
Automated Cron
[note caps]
Comment #105
jhodgdonThe module also probably needs an entry in MAINTAINERS.txt?
Comment #106
wim leers#103:
Many, many thanks for your review! :)
#105: Done!
Comment #107
wim leersUgh this snuck in from another issue, should be removed. Will not cause test failures though, and this patch isn't green yet anyway. The next reroll should remove it.
Comment #115
claudiu.cristeaThe patch was green till #53 (August 5). After that something was committed in core that breaks the update. IMO there's a hidden bug somewhere. Very hard to debug. I tried with no chance.
Comment #116
wim leers#115: indeed, but it must somehow be related to cron, or it'd have failed a long time ago.
Comment #117
jhodgdonRegarding the wording for the module description, calling it "server response" is fine with me. I just wasn't sure if "response" (it didn't have "server" in there) was clear. The proposed text I reviewed was:
Provides an automated cron by executing it after a response is sent.
So maybe:
Provides an automated way to run cron jobs, by executing them after a server request response...
hm....
Provides an automated way to run cron jobs, by executing them after a server response.
maybe that? I don't want to be inaccurate just so that we don't use technical jargon.
Comment #118
wim leers#117: Thanks! :) I'll ensure that happens in one of the next rerolls, as soon as we figure out the root cause of this test failure.
Comment #119
Anonymous (not verified) commentedi'll take a look at this.
here's an updated patch that does nothing but make the patch apply. install failed, but it failed on 8.0.x also :-\
gist of fail in case it rings any bells with anyone else:
https://gist.github.com/beejeebus/13925d7ea2cc2aa5b1cc
Comment #122
Anonymous (not verified) commentedunassigning, because i have NFI how to make progress :-)
i'm getting fatals running this test with D8 head, so i suspect i just have a broken set up. not sure when that happened, because i haven't tried to do any D8 for months.
if i can get a clean d8 to not fatal, i'll come back to this.
Comment #123
swentel commentedBeen trying to dig into the fail as well, but no luck so far either - some remarks though on the current code.
Probably not needed either? ConfigFormBase takes care of that
Maybe this isn't really needed, this is set in ConfigFormBase::submitForm as well.
Unintentional change ? networkErModal doesn't exist
Comment #124
swentel commentedOk, nevermind point 1 and 2 - was looking at the wrong form - kind of weird to see CronForm still in system though, should we just move it ?
Comment #125
gábor hojtsy@dawehner asked on IRC if we even know how is this supposed to be translated. That one is pretty easy. If you look at drupal-8.filled.standard.php.gz it has this snippet:
So the
system.siteconfig has a variant in thelanguage.escollection that has this slogan. The test in UpdatePathTestBaseFilledTest::testUpdatedSite() checks if there is output of that on the page. So it is a pretty integrated test so to say of whether the config override was still there, the block was still there, the block's config still said the slogan should be printed and the block was printed in the right language after all.My suspicion is that this patch introduces a module install in the update path and that clears caches that expose failures with the block update path that should have been there before, but this is just a hunch. There does not seem to be caching data in the dump... However I base that on @wimleers observation that no slogan was displayed at all. Grepping for use_site_slogan, it does not seem like that there is an update path to migrate that setting. Looking at system_update_8006(), it does not set that setting for the newly created block, so I assume old sites did not get the slogan in that block.
Comment #126
alexpottSo the problem is with
locale_modules_installed()and in particular this hunk inlocale_system_update():Comment #127
wim leers@alexpott++
Comment #128
gábor hojtsySo locale_system_update() just updates config (translations) based on data in locale. The database dump for the update tests does not have the slogan translation in the locale tables (locales_source, locales_target). Looking at the dump however, the source slogan is empty. If you look at LocaleConfigManager::processTranslatableData(), the override would be filtered out from config anyway, because the source value is empty and there is no translation for it in locale.
IMHO to fix this test:
1. Set a slogan in the original system.site in the dump.
2. Add that slogan in the locales_source table in the dump.
3. Add the same "drupal Spanish" slogan translation in the locales_target table in the dump.
That way, when the config translation is regenerated from the locale data, it will match what's in config and will not be removed.
Unassigning given Alex Pott said he is going for dinner ATM.
Comment #129
alexpottThis is pretty easy to reproduce:
I think this part of this issue is critical and a different issue. And in fact it does not matter whether the install runs as part of the update.php or just regular installation through the module form.
Comment #130
alexpottJust created #2580575: Installing a module can delete config translations since the problem this issue has unearthed is nothing to do with it. Unfortunately it does need to be postponed on it as there is no way to enable a module in the update path without it.
Comment #131
claudiu.cristeaThis bug was introduced at some point after #53, 2 months ago.
Comment #132
berdir@claudiu.cristea: I'm fairly sure 2 month ago is when we added the test that is now showing that bug.. very likely that existed for a longer time.
Comment #133
wim leersAmazing detective work, @alexpott and @Gábor Hojtsy!
Comment #134
claudiu.cristeaReopened.
Comment #138
catchComment #139
wim leersStraight reroll; conflicted with #2571647: Create DateFormatterInterface.
Comment #140
wim leersFixing #107 and #117, which were the only remaining points to address, everything else was about figuring out that mysterious fatal that we've since fixed.
Comment #141
wim leersIMO this is now ready.
Comment #142
claudiu.cristeaExcept this :)
Comment #147
wim leers#142: we can look for maintainers in a follow-up issue. Plus, many modules don't have active maintainers.
Comment #148
claudiu.cristeasystem.install hell
Comment #149
claudiu.cristeaComment #150
wim leers#2575421: Add a Stable base theme to core and make it the default if a base theme is not specified just got committed and already added
system_update_8012(). Incrementing.(Patch is smaller because this time I created it using
git diff -M10%again, to force the recording of moves.)EDIT: @claudiu.cristea beat me to it :) Removed my files.
Comment #152
wim leersLOL, c/p remnant:
Dynamic Page Cache->Automated Cron. Oopsie.Green.
Comment #154
dawehnerFeel free to use that patch
Comment #156
fabianx commentedRTBC, but now someone needs to ping Dries to get approval on the maintainers change, so maybe #152 is better to commit.
Comment #158
catchThis saves two queries and a config get on sites that aren't using automated cron, and should be very low risk. If this hadn't been ready we could have done in 8.1.x, but given this was RTBC for a long time and mainly blocked on locale data loss, committed/pushed to 8.0.x, thanks!
I'm not 100% on keeping the form and altering it, we could possibly move that to a status report item, then link to the form if automated cron is enabled. But less change at the moment is good.
Comment #160
yesct commentedmade #2581949: Move the cron form to a status report item, then link to the form if automated cron is enabled for discussing the concern in #158