If I run the feed manually it works as expected. If I let it run off of cron I get the following errors and it sticks at 56%

Warning: fopen(public://feeds/FeedsHTTPFetcherResult1378770113): failed to open stream: "DrupalPublicStreamWrapper::stream_open" call failed in FeedsFetcherResult->sanitizeFile() (line 92 of /sites/all/modules/feeds/plugins/FeedsFetcher.inc).
Warning: fgets() expects parameter 1 to be resource, boolean given in FeedsFetcherResult->sanitizeFile() (line 93 of /sites/all/modules/feeds/plugins/FeedsFetcher.inc).
Warning: fclose() expects parameter 1 to be resource, boolean given in FeedsFetcherResult->sanitizeFile() (line 94 of /sites/all/modules/feeds/plugins/FeedsFetcher.inc).
Warning: fopen(public://feeds/FeedsHTTPFetcherResult1378770113): failed to open stream: "DrupalPublicStreamWrapper::stream_open" call failed in ParserCSVIterator->__construct() (line 21 of /sites/all/modules/feeds/libraries/ParserCSV.inc).
Recoverable fatal error: Argument 2 passed to feeds_tamper_feeds_after_parse() must be an instance of FeedsParserResult, null given in feeds_tamper_feeds_after_parse() (line 19 of /sites/all/modules/feeds_tamper/feeds_tamper.module)

Comments

twistor’s picture

Status:Active» Postponed (maintainer needs more info)

How many items are in your feed?
How often is cron running not including Feeds?
How are you scheduling it to run at 2:30?

nicxvan’s picture

There are about 300 items in the feed, it takes about 2 seconds when I run it manually.
I don't know how often it is running.
I am using Elysia cron to allow me to set the task to run on cron at 230am.
I then use the system cron to wget the cron url at 231 to ensure the cron run.

The last run:
Global disable: no

Last channel executed: default

Last run: 09/20/2013 - 11:23

Channel: default

Last run: 09/20/2013 - 11:23

Last execution time: 83s (Shutdown: 2s) (Avg total: 29.57s, Max total: 132s)

Execution count: 25

Last aborted: On function fetch_faq_http_cron

Abort count: 27

twistor’s picture

What does "I then use the system cron to wget the cron url at 231 to ensure the cron run." mean?

So, Feeds breaks up imports across cron runs. The sceduled time is when it starts, but it will only import a set number of items per cron run. By default, 50. The scheduled time is when it starts, but then it continues on each following cron run until it finishes.

So if you are controlling Feeds cron via Elysia cron then it won't ever be able to finish its import.

nicxvan’s picture

Ah ok, how do I change the default number of items imported.

By: "I then use the system cron to wget the cron url at 231 to ensure the cron run."
There is a url on the status report page that allows you to run the cron from outside the site. I have a cron rule set on the system cron that hits that url to set off the cron. Otherwise you have to wait for an http request to run cron for drupal.

Exploratus’s picture

Would love to be able to change the limit of 50. Where can we do that?

MegaChriz’s picture

Category:bug» support
Status:Postponed (maintainer needs more info)» Fixed

Feeds has some hidden settings (this means that Feeds doesn't provide an user interface for changing these settings), like the limit of items being imported per run. The variable for this setting is called feeds_process_limit. Some processors (in contributed modules) may ignore this setting (I don't know which). 0 means 'no limit' and Feeds will try to import all items in one run in this case.

One of the ways to set the variable 'feeds_process_limit' is by defining it in the site's settings.php file:

<?php
// Process 100 items per run.
$conf['feeds_process_limit'] = 100;
?>

See README.txt from the Feeds module for more hidden settings.

Beware that it's possible Feeds can not complete it's task as it may have only limited time or recourses to complete the task. For example, php's max execution time setting may count on cron run. If that setting is active during import and it is set to 60, for example, then Feeds will only have 60 seconds to do the import. An other show stopper can be php's memory limit, this could be a problem when Feeds has to import a very large amount of items or if the items are large by itself. Anyway, because limits in time or memory may apply, Feeds imports 50 items by default. That value is a bit of guesswork, because even then it can happen that the import fails because the items to import are very very large.

I assume this answers the question and fixes the problem. If this is not the case, feel free to reopen this issue.

Exploratus’s picture

Awesome. Thanks! Will test it out.

Exploratus’s picture

Ok. So that worked. Put in feeds_process_limit in settings.php. I tried with value 0 for unlimited, but that just imported 0 items. As soon as I put a high number, my feed imported every item on cron run, instead of only doing the default limit of 50.

So this works, except for the 0 = no limit.

Status:Fixed» Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

dagomar’s picture

Issue summary:View changes

I believe the patch in this issue may solve this problem:

#2275893: Process for non-periodic imports will get stuck