Three efforts #

1. Port Elysia Cron from Drupal 7 to 8 at https://www.drupal.org/node/1442434

2. Port Ultimate Cron from Drupal 7 to 8 at https://www.drupal.org/node/2233283

3. While at the same as above efforts 1 and 2. There is a third effort toward collaboration between both ports. Including shared API between both modules.

Original message #

Hi

Greatly inspired by your module, I eventually made Ultimate Cron. I was particularly fond of the hook_cronapi() functionality, which I also embraced. Originally they we're compatible, but have since begun to divert due to natural reasons.

I was thinking that perhaps we could join forces somehow, if nothing else, then to define a common API for declaring cron jobs, just to begin with? And then perhaps, with combined effort, we can influence Drupal 8 to implement a better cron in core? #154043: Cron should not remain monolithic (Implement a scheduling hook for cron)

What do you think?

CommentFileSizeAuthor
#6 cron-d8.patch21.63 KBgielfeldt

Comments

gotheric’s picture

Hi,

it's good for me to define some "standard" between the two of us in some aspect of cron processing.

Probably a hook_cronapi() standard could be a good starting point.

I've changed it in Elysia Cron 2.0 inspired by Drupal 7 hook_menu and similar hooks (i'm trying to get the "Drupal way", when possibile...), to add some features asked by users (hook_alter, #file), and to develop a much more flexible way to define jobs. I don't know what is your mechanism about it, however if you want to illustrate me some points that are diverging we can discuss them.
(Consider that i released the new api a little time ago, so i'm not intended to completely change elysia_cron system. But i can consider some improvements or try to have a compatibility layer with your module, as a starting point).

About influencing the Drupal 8 core cron... i think that Drupal SHOULD have a much better cron (the cron system is nearly the same in all Drupal versions, since a long time ago...), and should have AT LEAST a better segmentation of tasks (modules should define more than a task...) and a basic frequency management (at least hourly/daily/weekly...).

But it seems to me that Drupal folks are not really interested in this topic, at least by looking at Drupal forums... it seems that cron management is intended only as a "power user feature".

However, if you have some ideas to "influence Drupal 8" i can surely help...

gielfeldt’s picture

I've also been wanting to implement hook_cron() in the same way hook_menu() works. So far I've implemented hook_cron_alter() to work in the same way as hook_menu_alter(). However I was reluctant to make change the way hook_cronapi() like you did. But I'm thinking of doing it anyway, especially if we can find common grounds on the API, it might be easier to gain support for this in core. As I see it, the cron in core doesn't necessarily have to implement the rule-part, but just a way for modules to implement an arbitrary amount of cron jobs, which could easily be extended.

I'll post some diffs between Elysia and Ultimate so we can see we're the similarities and differences lie.

gielfeldt’s picture

Struct of 'menu item' for Ultimate Cron.

(
    [unsafe] =>
    [description] => Default cron handler
    [module] => examplemodule
    [configure] => admin/config/system/examplemodule
    [settings] => Array
        (
            [enabled] => 1
            [rules] => Array
                (
                    [0] => * * * * *
                )

            [catch_up] => 300
        )

    [function] => examplemodule_cron
    [callback] => examplemodule_cron
    [file] => sites/all/modules/examplemodule/examplemodule.cron.inc
)

For the hook_cronapi() part, I've extended this with:
1.) 'rule' can return array of rules, as I needed multiple rules per job.
2.) 'settings' can return default settings (including rules).
3.) 'configure' link to the modules administration for this particular cron job

Differences between Ultimate Cron (UC) and Elysia Cron (EC) (please correct or add if necessary):

1.) UC supports multiple rules, but is defined in the settings section.
2.) UC has 'enabled' setting.
3.) UC has 'configure' link.
4.) UC has 'catch up' setting.
5.) EC has 'weight'
6.) EC has arguments callback
7.) EC has 'file path'

Note: the 'unsafe' is no longer relevant in Drupal 7 due to more robust hijacking of cron run.

The differences are not that great and I think they could easily be made compatible. Only the settings/rule stuff seems to differ.

The reason I put rules into the settings array was to store the job settings in one place in order to easily manipulate them. But 'rule' or 'rules' could very well be argued to be an integral part of the cron hook and therefore deserve a place in the root level. It could also be that the settings level should be eliminated and arbitrary settings could be managed by the 3rd party modules themselves somehow.

If we agree on something, we should post our solution as a proposal to core cron #154043: Cron should not remain monolithic (Implement a scheduling hook for cron). As I see it, if there's too much reluctance within the core team, the core cron COULD leave out the rules part, and just focus on using the weight for serial processing. Then other modules such as UC or EC could implement this, since there would be a healthy foundation for declaring the cron jobs.

Thoughts?

gotheric’s picture

For reference, this is elysia_cron structure:

function hook_cronapi($op, $job = NULL) {
  $items['key'] = array(
    'description' => 'string',
    'rule' => 'string',
    'weight' => 1234,
    'callback' => 'function_name', 
    'arguments' => array(...),
    'file' => 'string', // External file, like hook_menu
    'file path' => 'string',
  );
  return $items;
}

1) About the "hook_cronapi" call and the parameters ($op/$job), i think that a standard hook should have NO parameters, just like D7 hook_menu and similar functions.
Elysia cron has them for backward compatibility, but they should be used no more (and if a future Elysia Cron 3.0 will be release they will be dropped).

2) description and callback are the same between EC and UC: Ok

3) weight and arguments are only in EC: names are taken from hook_menu standards, so i think there is no problem with them. If you ever plan to implements them use these names. I think that at least a "weight" argument should be in an hook_cronapi proposal (how do you manage callback order in EC?)

4) module, function, configure, unsafe ara only in UC. I don't know what are their meaning, however they could be distinctive to UC and if you are not interested in "standardize" them it could be OK.
If you think some of them should be in the standard hook, tell me what are their meaning.

5) The real big difference is between EC "rule" and UC "settings".
I don't know how we can join them.
However probably the best thing to do to define a standard is thinking about a generic way to define this key, without considering too much our particular implementions.
How can be defined a generic drupal cron cron?

I really don't know this... i think that drupal core should implement only a basic timing system, not a precise "crontab like" timing. Something that just understand words like "daily", "monthly", "hourly" and similar...
If a Drupal user needs more precise timings he should use some 3rd party modules (like EC or UC).

Or, like you say, core core should have NO RULES AT ALL, and only the weight system to define run order.
In this case every extension could have its own method to define timings.

6) A note on EC's disabled / UC's enabled: for me the best think is having a "status" key that uses some constants like "CRON_ENABLED" and "CRON_DISABLED" (and maybe, in a future, other values).
(CRON_ENABLED can be "1" and CRON_DISABLED "0", this should be similar to your "enabled" key)

7) file/file_path: there are some difference (you use only "file" with full_path). The reason for EC's file+file_path is to use the hook_menu syntax. However i think that this is not a great problem. UC can continue to support its "file" key even if we put in standard file+file_path.

From this consideration, a starting point for a standard could be this:

function hook_cronapi() {
  $items['key'] = array(
    'description' => 'string',
    'status' => CRON_ENABLED|CRON_DISABLED,
    'weight' => 1234,
    'callback' => 'function_name', 
    'file' => 'string', // External file, like hook_menu
    'file path' => 'string',
  );
  return $items;
}

file+file path is not really relevant, we can drop it if it's a problem.

This is only my idea, tell me your changes...

gielfeldt’s picture

1.) I agree. The old hook_cronapi() should be eliminated.


3.) Everything is running in parallel in UC, so I don't use weight. However for a proposal, I was thinking of a somewhat simpler model, which uses weight. I'll post the patch here later today when/if I find the time.


4a.) 'module' is to inform which module this hook belongs to. This is really only necessary as I see it, when the old hook_cronapi() is used.


4b.) 'function' became a way of passing around the key of $items if you will, but is not needed as such i think.


4c.) 'unsafe' is for whether or not the hook is safe to run by UC in regards to the core cron, as this might run in parallel. Not used in D7 anymore, and definitely not needed in D8, if we get this cron implemented in core.


4d.) 'configure' means the same as the 'configure' an module's .info file. It's a link to a configuration page for that particular cron jobs, should the module provide any.


5.) I'm not sure they can be joined either. We should agree on some sort of standard which doesn't necessarily take EC or UC into consideration, like you say. However regarding rules, I disagree that we shouldn't use crontab style rules. These aren't really that hard to grok, and I think that everytime someone tries to simplify this with a "nice" UI, it falls short, either in user friendlynes or feature-wise. Crontab rules, as I see it, are easy to understand, implement and powerful.


6.) Sounds reasonable. Then it will be extendable.


7.) I agree, it should be like hook_menu(). I think I made it this way in the D8 cron patch as well, and I'll probably rectify it in UC.


gielfeldt’s picture

StatusFileSize
new21.63 KB

Here's my first attempt at a Drupal 8 patch ...

There's no interface yet to change rules etc. It's just a somewhat proof-of-concept for the programmatic part.

There might be some issues using variable_get/set for getting last run time, due to (static) caching, but it should be solvable.

gielfeldt’s picture

Hi

Have you had a chance to look at it yet?

gotheric’s picture

Sorry for my late answers but i'm very busy at work in this period...

However, i see that the main difference in our approach is in crontab rules syntax (point 5).

IMHO, Crontab syntax is difficult to understand to every "non programmer" (and even for a lot of programmers).
Even modern linux distributions tries to hide it with a basic scheduling system that only specify if a job is hourly|daily|monthly|weekly.

When i released the first version of elysia cron, googling for user feedback resulted in a lot of people complaining they aren't able to use crontab syntax.

And, the most important fact, i think that for 99% of drupal installations crontab syntax is totally useless.
A very few sites needs to specify the exact minutes of execution, or have needs for complex rules (and a lot of sites doesn't needs crontab timings at all, the standard "run all jobs every hour" is ok).

Elysia cron is for that 1% of drupal installs (and maybe another 1% for debug purpose).
Drupal core cron should be easy, a basic (but expandable) system to cover the 99% of users...

Another problem in crontab syntax is the specific implementation. It's not very easy, and there are some subtle problems than can be trickier to solve. An example is this:
http://drupal.org/node/1265442
I don't think drupal core cron should mess with this implementations...

However, i think that this could be discussed with other drupal members and mantainers. Maybe drupal community is interested in your approach and want to follow it.
Maybe not.

Before making a lot of work in patching drupal core, we should try to understand if there are chances that the patch can be accepted by drupal folks...

gielfeldt’s picture

Another problem in crontab syntax is the specific implementation. It's not very easy, and there are some subtle problems than can be trickier to solve. An example is this:
http://drupal.org/node/1265442

I see what you mean regarding perception of cron. I would change that ticket to "don't fix" as it IMO is really not a bug, just a "limitation" of crontab expression. Some of these limitations can be mitigated using multiple rules.

I still think crontab rule could be sound to use as an internal scheduling mechanism, and then it could be sugarcoated by presets like Hourly, Daily, etc. and a "Custom", just like the dates&formats in Drupal to re-use a convention.

But you also mention it's possible to use a different scheduler in Elysia Cron. This could also be a good idea. Then schedulers more simple or complex than crontab could be added. But in terms of simplicity the "presets" might be more Drupal'ish? We should investigate this further.

Before making a lot of work in patching drupal core, we should try to understand if there are chances that the patch can be accepted by drupal folks...

Sometimes (actually usually) the best way to present an idea is via an example implementation. It hasn't been that much work for me making the Drupal 8 patch. And the patch does address the nearly 5-year-old issue #154043: Cron should not remain monolithic (Implement a scheduling hook for cron), a thread which haven't received much (enough?) attention.

gielfeldt’s picture

Btw, while looking at the other thread regarding cron rules, I started thinking about implementing actual unit tests in Ultimate Cron for use via the Simpletest module. I then saw that you have some tests in elysia_cron_scheduler.inc. I think I'll snatch these and perhaps add few and then create a .test file. You can snatch it back afterwards and we can share tests if we come up with more. How do you feel about this?

Looking at your tests I found something which I might think is a bug. You have a rule "59 23 */10 * *" which your test assumes should run on "2008-01-10 23:59:00" and "2008-01-20 23:59:00". However as I understand the cron specification the / is a stepping operator, not modulus, meaning that for dates */10 should translate into 1-31/10 which would translate into 1,11,21,31.

What do you think? Should I create a new issue for this? Or do you wanna have the discussion here?

gielfeldt’s picture

Hi

I converted your tests to the Drupal test framework. You can get it here if you want: http://drupalcode.org/project/ultimate_cron.git/commit/5c3b63d

/Thomas

gielfeldt’s picture

Are you by any chance in Munich for the DrupalCon? Then perhaps we could talk some cron.

gielfeldt’s picture

Are you by any chance in Munich for the DrupalCon? Then perhaps we could talk some cron.

gotheric’s picture

Sorry, i wasn't able to go there :(

geerlingguy’s picture

It's good to see this discussion here! I, for one, am interested in seeing how we can really make Drupal's cron functionality better. Right now, depending on the site, I have some drush jobs hit by separate cron tasks, and I have other things scheduled via elysia_cron. I have to do this many times, simply because one module = one cron schedule.

I'd love to see, mainly, (a) the ability to specify a set of cron jobs/tasks in hook_cron(), and then (b) the ability to stick these jobs into separate schedules. It could be simpler than what elysia_cron currently does, but would be a thousand times easier for me than having to manage cron schedules for some things via hook_cron(), and other things via drush scripts and crontab on the server.

berdir’s picture

Related #2169267: Replace drupal_cron_run() with a Cron service.

Core won't work different in any way with that, it's still a all or nothing, but a module like elysia_cron just has to replace that service and then it can implement it's "run what needs to be run" logic in there, transparent to everything else, no more custom cron.php scripts and stuff like that required. And it could extend the service to also offer methods to run a single sensor and so on.

thedavidmeister’s picture

Issue summary: View changes

I really like the idea of the Elysia Cron "crontab like" syntax.

Far more than 1% of the site's I build rely on it as critical functionality. I'd say closer to 10-15% have their own cron schedules that needed careful tweaking at some point.

Don't totally count it out for core!

gielfeldt’s picture

Btw, Ultimate Cron 2.x now has scheduler plugins, like we also discussed @gotheric. It currently supports a simple syntax and a crontab syntax.

thedavidmeister’s picture

This should work the way that Berdir described in #16. That way, even if this doesn't make it into D8, it should be D9 ready :)

miro_dietiker’s picture

I see there's no recent activity here.

Just wanted to mention that we provided a working Ultimate Cron port for Drupal 8 that is about to be merged into the d.o repo and MD Systems will continue to maintain that. I would love if we can collaborate and maintain it as a joint solution together.

We are looking forward to consider Elysia Cron features to make it feature complete.
And we would appreciate if you would mention the port on the project page. Mentionning "similar modules" is a common practice.

francewhoa’s picture

Title: Collaboration and Drupal 8 » Port Elysia Cron to Drupal 8 & Collaboration with Ultimate Cron
Issue tags: +Drupal 8

Thanks all for your contributions :)

We would be happy to contribute testing patch, quality assurance, documentation, and agile project management services if needed

Related pages

I updated this ticket title so it's easier to find it with aggregators and search engines

francewhoa’s picture

That related ticket show the D8 porting status on the module project page under "Recommended releases" section at https://www.drupal.org/project/elysia_cron

Which invites additional contributors to join the efforts

Details at https://www.drupal.org/project/contrib_tracker

miro_dietiker’s picture

@Francewhoa I'm confused about the information that you share here.
I don't see a maintainer responding here since 3 years. The last commit in this (D7) repository is 2 years ago.

@gielfeldt Is more working on ultimate cron project and not elysia cron.

What direction are you heading now?
Ultimate Cron is well progressed and pushed forward step by step. There are not many things remaining until we will move to d.o and release alpha.
You are welcome to join forces to complete the port or test it.

However, if ultimate cron is really the recommended direction, then description at #2615318: [elysia_cron] Elysia Cron needs overhaul.

francewhoa’s picture

Component: Miscellaneous » Code
Category: Support request » Task
Issue summary: View changes
Status: Active » Needs work

Hi miro_dietiker :) Thanks for asking

There are three efforts:

1. Port Elysia Cron from Drupal 7 to 8 at https://www.drupal.org/node/1442434

2. Port Ultimate Cron from Drupal 7 to 8 at https://www.drupal.org/node/2233283

3. While at the same as above efforts 1 and 2. There is a third effort toward collaboration between both ports. Including shared API between both modules.

To clarify and reflect the above I updated some tags and the issue summary

francewhoa’s picture

Issue summary: View changes

Minor edit to clarify further

miro_dietiker’s picture

Title: Port Elysia Cron to Drupal 8 & Collaboration with Ultimate Cron » Do not port Elysia Cron, recommend Ultimate Cron for Drupal 8

After long time of inactivity at Elysia Cron, we should help the users to find its healthy replacement:
Ultimate cron has an alpha release for D8 since 4 months and is about to get beta soon.

Is this title proposal acceptable? :-)

kala4ek’s picture

Status: Needs work » Closed (outdated)