We have been adding a large number of feeds into our site using the aggregator2 module. So far we have about 55 feeds displaying blog content from other sites.
Everything was running fine and smooth, content was being updated; then all of the sudden the cron jobs stopped completing.
In order to ease the strain on our server, I set each feed to update once a day (a level that I think should be adequate to find new content while minimizing load).
After about the 50th or so feed, the cron job started giving me the couldnt complete message. To verify that cron was still running properly, but just not completing, I added another feed into the site. On the next cron job (set on the :03 of each hour) I noticed that new content had been added from the new feed entered. i still received the message that cron did not complete.
For reference, our feeds are set to NOT update existing posts. We dont need to check each and every feed item if there has been a change. We just want the new content.
I started to assume that this was a PHP memory limitation. So i opened settings.tpl.php and increased the PHP max memory limit from 12M to 30M. Still no luck. Here's the specs of our server:
Dedicated server, AMD Opteron 2ghz, 1GB RAM, Red Hat Enterprise Linux, Apache 2.0.x, PHP4. We are running the path module, but not path_auto.
We have about 650 nodes within the site, not too many images. There are a few modules running, but I wouldnt say an extreme amount.