In many installations, "Last cron did not complete" messages start to appear in the logs, and all search indexing stops at a specific number of remaining nodes. The problem is that if a cron does not complete with a task of indexing 10 nodes, and does only 9, those 9 are not recorded to have been indexed, and the next cron falls in the same trap, thus stopping all indexing. Crons probably stop because of a timeout, although other causes may exist.

We can record each indexing operation by calling node_update_shutdown() at the end (and inside) of the while loop in node_update_index() of the node module.

I have done it many times and it save the situation; cron runs still do not complete, yet the indexing progresses with each cron run. I recommend this hack for anyone with "cron did not complete" messages!

This applies to drupal 4.x and 5.x.

Comments

robertDouglass’s picture

Version: 5.1 » 7.x-dev

It would be good to revisit this and guarantee that we're not stuck in this trap for D7. I've seen lots of instances where broken nodes will stop indexing. The system should be able to overcome such failures.

robertDouglass’s picture

Title: Porgress Indexing when Cron does not complete » Ability to keep indexing beyond cron failures
marcingy’s picture

Version: 7.x-dev » 8.x-dev
marcingy’s picture

Status: Active » Closed (duplicate)