I can tell that I am going to need to run rebuild. Were this integrated into drush, I could run a rebuild off of atd in the wee hours of the AM.

Comments

dave reid’s picture

Status: Active » Postponed

That would be a good idea. This could probably be a part of #481136: Create PCNTL-compatible generate-sitemaps.sh script but I'd need to investigate. For now, marking as postponed.

dave reid’s picture

Assigned: Unassigned » dave reid
Status: Postponed » Active

Add basic Drush integration just for regenerating the sitemap files from the current data (what normally runs on cron).

dave reid’s picture

Status: Active » Postponed

Bleh, so the rebuild that was carefully and painfully worked into a batch process so that it wouldn't break things is a little harder to run from Drush. I'd rather not write the whole thing again not as a batch process just for Drush and I can't find anything on how to convert/run a batch process within Drush. Anyone have any ideas or know-how?

damienmckenna’s picture

Could it just:

  1. Check which of the submodules are enabled
  2. Execute their hook_cron() functions

?

damienmckenna’s picture

Status: Postponed » Needs review
StatusFileSize
new1.74 KB

Here's an example patch that does what I suggested in #4.

damienmckenna’s picture

I wish there was a simple way to set higher limits on the hook_cron() functions if they're running via drush? Maybe add an optional argument to the functions to control the limit?

damienmckenna’s picture

StatusFileSize
new7.31 KB

Here's a patch that extends the hook_cron() functions to allow an optional argument to override the limit being used. It then also adds a command-line option to control this limit - if a --limit value is given that is used otherwise it sets no limit.

moshe weitzman’s picture

Don't know if folks will run into memory problems, but we have recently solved those in migrate module's drush integration by using drush_backend_invoke(). Thats a possible improvement after you get basic drush integration working.

dave reid’s picture

Ooohhhh.... it actually looks like I might be able to use batch API with $batch['progressive'] = FALSE. Looks like all the code is there to make a complete batch run in one pass.

dave reid’s picture

StatusFileSize
new3.85 KB

Ok here's what I've got so far. We can technically run batch api in a non-redirect-browser way that works with command line, expect there's a minor bug with #638712: Make non-progressive batch operations possible. The batch processing runs but there's something funky with file permissions or not getting the right folder to write files to.

dave reid’s picture

Heh, it was my own damn file permissions. You'll have to be careful since the files/xmlsitemap folder may be owned by your webserver user and your drush user doesn't have write permissions to the same directory.

dave reid’s picture

Status: Needs review » Fixed

Committed to CVS! Thanks everyone!

damienmckenna’s picture

So does it work without the patch from #638712 applied?

dave reid’s picture

Yes it does. Luckily there's a quick workaround for it.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.