I'm working on importing csv feeds from one drupal site to another using the Feeds HTTP Fetcher. The first site provides remote access to its csv feeds using the HTTP Authentication provided by the securesite module.
When I try to import the feeds from the first site using the url http://username:password@example.com/users/analyse/csv , I run into the following error:
HRCurlException: cURL error (47) Maximum (20) redirects followed for http://username:password@example.com/users/analyse/csv in http_request_get() (line 203 of .../sites/all/modules/feeds/libraries/http_request.inc).
The following message showed up 20 times in the apache access log of the site from which I was trying to import the feeds every time I tried to import.
"GET /users/analyse/csv HTTP/1.1" 302 526 "-" "Drupal (+http://drupal.org/)"
However when I tried using the same url in my browser, I could access the feed.

I did some research and found out that the cause was that I wasn't telling cURL to use cookies, therefore I created a small patch that adds the option to tell cURL to use cookies before you import. The option is provided by way of a checkbox on the source form of the HTTP fetcher.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Joanna_Kisaakye created an issue. See original summary.

MegaChriz’s picture

Status: Active » Needs review

I haven't enough knowlegde about cURL to estimate if the patch could have any negative side effects.

+++ b/libraries/http_request.inc
@@ -197,6 +201,11 @@ function http_request_get($url, $username = NULL, $password = NULL, $accept_inva
+      if ($enable_cookies) {
+        $temp_directory = file_directory_temp();
+        curl_setopt($download, CURLOPT_COOKIEFILE, $temp_directory . "/cookie.txt");
+        curl_setopt($download, CURLOPT_COOKIEJAR, $temp_directory . "/cookie.txt");
+      }

I'm not sure, but this looks like a security issue: anyone that has access to the temporary directory would be able to access cookie.txt.

When searching for the error message, I've found this: http://stackoverflow.com/questions/10792419/php-curl-accessing-url-with-...
There is suggested that the solution should be:

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

I hope someone with more knowlegde about cURL could give this patch a better review.

Status: Needs review » Needs work

The last submitted patch, feeds-fix_for_maximum-redirects-1.patch, failed testing.

The last submitted patch, feeds-fix_for_maximum-redirects-1.patch, failed testing.