When crawling a feed using a URL pattern, Feeds Crawler correctly ends when the last index 404's, however the Feeds module reports a 404 error when attempting to access this URL. Here's a patch to check the validity of the iterated URL pattern, and if the page is not found, quit crawling the feed.

CommentFileSizeAuthor
#1 feeds_crawler-404fix-2189555-1.patch906 bytesbjpritch
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

bjpritch’s picture

Jony_Niuqiang’s picture

How to use it?

Nikolay Shapovalov’s picture

Status: Needs review » Needs work
+++ b/FeedsCrawler.inc
@@ -62,7 +62,24 @@ class FeedsCrawler extends FeedsHTTPFetcher {
+    $response = drupal_http_request($url);

I think make request to server, just to check page status is bad idea.