The workaround specified in
#2427497: Fetching feed via SSL through proxy doesn't work
checks for a certain version of libcurl using curl_version().

My dev (CentOS) and staging (RHEL) environments both report curl version as 7.29.0 but the feed only functions correctly if I disable the workaround.

I think RHEL repositories (and by extension CentOS) - despite receiving updates - do not report the proper version number.

I am currently looking for a way to get PHP to check which repository provided libcurl so that I can exclude the relevant repo. I will post a patch here if I find anything, and would appreciate any guidance if anyone has done this before.

Or perhaps the workaround is flawed.

Comments

Darvanen created an issue. See original summary.

darvanen’s picture

darvanen
he/him
Primary language English
Location Sydney, Australia
CreditAttribution: darvanen commented
Issue summary: View changes
darvanen’s picture

darvanen
he/him
Primary language English
Location Sydney, Australia
CreditAttribution: darvanen commented
Title: Curl header workaround incompatible with RHEL » Curl header workaround for proxy rendering data incomplete
MegaChriz’s picture

MegaChriz CreditAttribution: MegaChriz at WebCoo commented
Version: 7.x-1.x-dev » 7.x-2.x-dev

I'm not an expert on Curl, so I hope you figure out how to fix your issue.

darvanen’s picture

darvanen
he/him
Primary language English
Location Sydney, Australia
CreditAttribution: darvanen commented

Well, for the site I'm working on I can just comment out that part of the module (lines 222-228 of feeds/libraries/http_request.inc) and it works fine, I'm just struggling to come up with anything useful for the wider community.

pbattino’s picture

pbattino CreditAttribution: pbattino commented

Same here, on CentOS
My curl is: 

curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.28.4 zlib/1.2.7 libidn/1.28 libssh2/1.4.3

If I comment out (293-299 on 7.x-2.0-beta4) everything works.

      if ($proxy_server && $curl_ver['version_number'] < 0x71e00 && _drupal_http_use_proxy($uri['host'])) {
        $http_header_break = "\r\n\r\n";
        $response = explode($http_header_break, $result->data);
        if (count($response) > 2) {
          $result->data = substr($result->data, strlen($response[0] . $http_header_break), strlen($result->data));
        }
      }
MegaChriz’s picture

MegaChriz CreditAttribution: MegaChriz as a volunteer commented

This code was added in #1159806: Add proxy support and slightly modified in #2427497: Fetching feed via SSL through proxy doesn't work.

I don't know how to replicate the issue. Maybe ask VladimirAus for help, as he originally added this piece of code.

pbattino’s picture

pbattino CreditAttribution: pbattino commented

According to curl's documentation the code is correct and yet... this workaround breaks the request, for me.

I can't help but think we are reinventing the wheel: I had to fix this problem of wrong header/body split in a custom module developed by a 3rd party, then I saw the same problem solved in chr module (around line 221 in 7.x-1.8 ).

Soooo.... for me the solution is to use set $conf['feeds_never_use_curl'] = TRUE . Since this makes Feed use drupal_http_request and since I use chr as default override for drupal_http_request, problem solved.

However, as an insight for Feeds' own function, probably the best way forward is not to use CURLINFO_HEADER_SIZE to split header and body. Use the same trick used by chr instead, repeated preg_split("/\r\n\r\n|\n\n|\r\r/...)

MegaChriz’s picture

MegaChriz CreditAttribution: MegaChriz as a volunteer commented

@pbattino
What I know is that it is important that the headers are saved separately (for caching). From my understanding, the advantage of using Feeds own HTTP download method are the following:

  • Option to use username and password for the HTTP source.
  • Option to accept invalid SSL certificates.

For the rest, I'm not fully aware of what exactly happens in Feeds' download code.