cronjob queue not aggressive enough

17:07:25 <@Vertice> the cron queue should be a batch queue
17:07:27 <@Vertice> not a serial queue
17:07:33 <@Vertice> ie: it should try to get all sites within 1 hour
17:07:39 <@Vertice> figuring out the frequency
17:07:43 <@Vertice> it needs to be called
17:07:45 <@Vertice> and the amount of sites
17:07:49 <@Vertice> it also has a threshold
17:08:03 <@Vertice> so the more sites you have, the more frequently it is run and the more sites it runs per cron job

Basically, I think the current design of the cron queue is wrong: we shouldn't run cron jobs in parallel unless we really have to. Also, the control should be over the frequency of the cron jobs (which could be per site) than the number of cron jobs per run. When a cron job is due, it's due, that's it, it needs to be ran. The frequency should just be obeyed.

To alleviate the load on the server (which is an issue we are witnessing now, see #606072: Cron should be disabled for disabled sites), that frequency can be changed by the administrator, or in the case of per site cron frequencies, a minimum can be enforced.