Problem/Motivation
Paypal have finally implemented their HTTP/1.1 change first announced in 2011, meant to happen in 2014, and now seemingly actually enforced for sandbox IPN requests. One assumes that they might eventually require it on live IPN sometime too.
Because drupal_http_request is an HTTP/1.0 client, all IPN verification attempts now result in "400 Bad Request" - previously, this would only be caused by the missing "Host:" header.
There is a proposal to change this in Core very simply, #2242123: Make drupal_http_request() support HTTP 1.1, however chunking is required for an HTTP/1.1 client #106506: drupal_http_request() does not handle 'chunked' responses - Make it support HTTP 1.1. D8 has solved this by using the Guzzle.
Paypal say they have switched over already, however live requests do seem to continue to work for existing working sites. https://www.paypal-knowledge.com/infocenter/index?page=content&id=FAQ148...
Proposed resolution
- The IPN client/server interaction is not a fully feature functional HTTP/1.1 transaction. A simple local replacement for drupal_http_request could be written to "fake it" enough to make it. (See my horrible hack below)
- As proposed in #2683537: Paypal's New Warning, the cURL module can be used as both a work around, and a permanent solution by adding it as a dependancy of this module.
Work around: Use cURL module and configure via admin/config/services/chr
to use chr_curl_http_request()
as the default method.
NB: Due to the SSL changes that Paypal have also implemented, if your version of PHP is older, its curl version may not be able to talk the required SSL protocols. This will undoubtedly cause even more annoying support requests, at which point a "hack" of a fake HTTP/1.1 client request does seem like a much nicer alternative.
Remaining tasks
Patch the module to use one of the proposed solutions.
User interface changes
None.
API changes
Update will require installing a new module if going the chr module route.
Data model changes
None.
Original report by https://www.drupal.org/u/ibedoya
As I understood PayPal API will discontinue support for HTTP 1.0 protocol before 1 November 2014.
And an update should be applied quickly in Drupal core (https://drupal.org/node/2242123) or in this module
More info:
https://drupal.org/node/2215527
https://ppmts.custhelp.com/app/answers/detail/a_id/922
Postponed - PayPal API Changes:
https://ppmts.custhelp.com/app/answers/detail/a_id/1070
Comment | File | Size | Author |
---|---|---|---|
#42 | paypal_enforcing-2263585-42.patch | 803 bytes | rbogdan |
#41 | paypal_enforcing-2263585-41.patch | 575 bytes | istavros |
#35 | paypal_enforcing-2263585-35.patch | 2.71 KB | mglaman |
#28 | 2263585-TLS-1_2.patch | 3.85 KB | scottatdrake |
#11 | paypal_IPN_http1.1_and_TLS1_support-2263585-#11.patch | 3.43 KB | gaydamaka |
Comments
Comment #1
ELC CreditAttribution: ELC as a volunteer commentedInstalled a new site, and I am unable to test the workflow using the sandbox. All requests result in 400 Bad Request unless I add all of the changes shown in: https://developer.paypal.com/docs/classic/ipn/gs_IPN/
I am in the boat of being stuck with a server that cannot talk curl due to versions being out of step with Paypal, so this is my HACK.
Since this issue has been open in various fashions since 2011 and simply hasn't been addressed and probably never will, I going the insane route. Maybe when Paypal finally enforces it on live sites something might change. Heck, maybe I'll make it a real module! :P Of course, the Catch-22 is that unless enough sites switch over to using HTTP/1.1, Paypal may never actually switch live to it.
I cannot emphasise enough that this hack is a very bad idea and may break your site. Please know what you're doing using this.
insanehack_ipn.info
insanehack_ipn.install
insanehack_ipn.module
Comment #2
rreiss CreditAttribution: rreiss at Dofinity commentedHi,
I've create a new patch for 7.x-2.x branch.
I've replaced `drupal_http_request` with cURL based implementation (as on the `commerce_paypal_api_request` function).
I've also added the setopt to TLSv1 (required..) - `curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1);`
If you get "Attempt to validate IPN failed with cURL error: SSL connect error", then please make sure that your server has the latest libcurl and openssh packages.
For RedHat (e.g. Centos) use `yum install curl openssh` .
Thanks.
Rotem
Comment #3
torgosPizzaTagging with sprint tag so we can get some more eyeballs on it.
Comment #4
squall3d CreditAttribution: squall3d as a volunteer and commentedFixed my issue
Comment #5
ELC CreditAttribution: ELC as a volunteer commentedThis places a mandatory requirement on using curl, and having a system that is able to be fixed if it has an issue. I'm not sure that a reasonable assumption given shared hosting solutions.
This would prevent normal function of the module that would otherwise work on all installations.
How about instead of forcing this module to use curl, it instead uses a system similar to drupal_http_request. The curl code in your patch could go into a sub-module which uses this variable to configure itself, or to use the curl module:
This would allow anyone to use curl module, a locally included curl wrapper, or some other module. Using curl for a broken HTTP/1.1 implementation does seem like overkill since the hack in #1 does work (I sent it live). Paypal don't appear to be using a full stack 1.1 server, just some home grown solution which works with their load balancers.
Comment #6
rreiss CreditAttribution: rreiss at Dofinity commented@ELC I agree.
I think that maybe we should use another method to make the request (e.g. stream socket as on drupal http rquest), not even sure that we want to use the curl snippet in my patch, but... The module is already somehow dependent on cURL (look at the existing `commerce_paypal_api_request` method).
My suggestions are:
1. Use cURL as default, and allow other modules to override the default implementation
2. Add dependency with chr module
3. Use our own drupal_http_request wrapper
I think that the 1st option is the best one (with the proper documentation)
Comment #7
gdanger CreditAttribution: gdanger commented#2 works for me. Thanks
Comment #8
albany CreditAttribution: albany commentedHi...,
I'm inexperienced with patching; so please forgive my ignorance with my questions as follows:-
I patched 7.x-2.3 with commerce_paypal-provide-extra-options-commerce-ec-2028329-13_0 patch
can I patch the 7.x-2.x-dev with the #2 patch and then patch it again with the above?
Hope this make sense.
Regards
Steve
Comment #9
nicksanta CreditAttribution: nicksanta commentedSolved by using the cURL HTTP Request module - https://www.drupal.org/project/chr. Below is a step-bys-step guide.
Comment #10
ndiSIGN CreditAttribution: ndiSIGN as a volunteer commentedHi,
It's a bit strange because I was getting the log message:
Attempt to validate IPN failed with error 400: Bad Request
So I activated the IPN settings (http://www.mysite.com) on my PayPal sandbox account but that didn't help. Then I tried the solution from #9 which solved the 400 error. And the payment info was being printed in a table on (admin/commerce/orders/11/payment).
From this similar thread: https://www.drupal.org/node/1055390 I understood you should not need to do anything with IPN settings on the PayPal side, so I turned these off again.
The funny thing is, it was still working without error messages.
But before to post my findings on the other thread I wanted to make sure I was exact, so I turned everything off and only activated the cURL HTTP Request module, but then the 400 error returned.
It turns out, in my case I need to activate both the module from #9 and set my website address in the IPN settings (dubbel checked this) in the PayPal sandbox to make it work. After that I can clear the IPN settings from PayPal and it still continues to work.
Comment #11
gaydamaka CreditAttribution: gaydamaka as a volunteer and at FFW commentedHi,
#2 not works form me. But if change
CURLOPT_SSLVERSION
fromCURL_SSLVERSION_TLSv1
toCURL_SSLVERSION_TLSv1_0
it works fine.http://stackoverflow.com/questions/26379773/paypal-ipn-acknowledgements-...
Comment #12
rreiss CreditAttribution: rreiss at Dofinity commentedCan we agree that we can add runtime check on hook_install to make some "connectivity" checks?
My idea is to make dummy request with drupal http request to the sandbox / debugging IPN and see if it's successful.
True: Use the default drupal http requst module (we assume that the sie is using chr or another drupal http request alternative).
False: Use my cURL patch from above as the default method, in order to allow the module to work without the need in another contrib. module (btw the module is already using cURL in its other methods..), and show information message to let the developers know that they may use the chr module or another alternative.
What do you guys think? Can we agree on the way to patch this annoying issue?
Comment #13
rreiss CreditAttribution: rreiss at Dofinity commentedComment #14
AnisAlgeria CreditAttribution: AnisAlgeria commentedHi,
My website is hosting at OVH ( french company of web hosting), I am using commerce paypal. everything is ok but IPN notifiction from paypal was failed and error is ( Attempt to validate IPN failed with error 0: Error opening socket ssl://www.sandbox.paypal.com:443).
My Server is a mutuel server in OVH. although Paypal had alerted to upgrade SSL Version to( OpenSSL/1.0.0 to up )
OVh still at (OpenSSL/0.9.8o ).
I have SSL certificate and I redirect all my links to Https://.
version of PHP: 5.6.21
cURL support :enabled
SSL Version :OpenSSL/0.9.8o
I am using commerce Paypal 7.x-2.3
I trayed the patch but didn't work for me,
I ask if there is a solution to get notification from paypal sanbox even if my server sitll in OpenSSL/0.9.8o ?
thank you for your help
Comment #15
rreiss CreditAttribution: rreiss at Dofinity commentedThe patch won't work without the latest openSSL. At least it didn't work for me.
Comment #16
AnisAlgeria CreditAttribution: AnisAlgeria commentedThank you rreiss,
So there is no solution to get IPN notifiction from paypal with this configuration.
Comment #17
rreiss CreditAttribution: rreiss at Dofinity commentedNo, unless your host will support SHA256 see http://stackoverflow.com/questions/32548246/paypal-changes-for-sha-256-c... for more info.
Comment #18
rsbecker CreditAttribution: rsbecker commentedThe patch at #11 did not work out of the box for me. But replacing lines 104 and 382 with the following fixed my site.
Comment #19
Chalk CreditAttribution: Chalk at DrupalSquad commentedThe solution #9 (cURL HTTP Request module) works for me. Thanks!
Comment #20
AntiNSA CreditAttribution: AntiNSA commentedCurl module not working for me.
Comment #21
rreiss CreditAttribution: rreiss at Dofinity commented@AntiNSA are you getting an error? Curl is enabled on your server? If so, curl & OpenSSL are up to date?
Comment #22
armyofda12mnkeys CreditAttribution: armyofda12mnkeys as a volunteer commentedI opened another ticket related to this and saw this thread later (sorry about that): https://www.drupal.org/node/2815541
#9 worked for me (on Dreamhost php 5.6 if makes a difference since seems to depend on curl being on server).
if Paypal IPN went through, should the status be 'Pending' or change to 'Completed' (or the user needs to ship and then should come back and set the Order manually to Completed in the dropdown?)?
Comment #23
torgosPizza@armyofda12:The status of the order is set via Rules; the default that ships with Commerce sets them to Pending. You can easily override that configuration to set them to Completed if you wish - that is what we've done. (I have another Rule that reacts to "An order is updated" and if it contains physical goods sets the status to a custom "Pending Shipping" status.)
Comment #24
armyofda12mnkeys CreditAttribution: armyofda12mnkeys as a volunteer commentedthanks @torgosPizza for that info.
Comment #25
ericjenkins CreditAttribution: ericjenkins at Exel Digital commented#9 worked for me, as well. Thank you!
Comment #26
LIQUID VISUAL CreditAttribution: LIQUID VISUAL as a volunteer and commentedre: #9 - NickSantos - could anyone tell me how to do step three without drush? Many thanks.
Comment #27
rreiss CreditAttribution: rreiss at Dofinity commentedI think that you can just use the chr module's configuration page, can't you?
If not, use the variables editor via the GUI.
Comment #28
scottatdrake CreditAttribution: scottatdrake commentedThis patch is a re-roll of #11. It updates the TLS version to 1.2, which will be required by June 30, 2017.
It uses
curl_setopt($ch, CURLOPT_SSLVERSION, 6);
overcurl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2);
because the constant in the latter example isn't introduced until PHP7.This also updates commerce_payflow.module because it has the same requirements for TLS 1.2 but was left out of the previous patch.
Comment #29
andyg5000Comment #30
acPatch #28 applies and works. Need to get this committed and released pretty soon if June 30 2017 is the deadline!
Comment #31
andyg5000The patch in #28 works as expected. The constant "CURL_SSLVERSION_TLSv1_2" was added in curl v 7.34.0 (not PHP 7), so we should probably still use it.
We should also check for it as part of the requirements when installing and provide notes in the release about the requirement.
As a side note, I was able to connect to https://tlstest.paypal.com/ on a server that didn't have openssl compiled with tls 1.2 by commenting out the "CURLOPT_SSLVERSION" line. Not sure which version it was defaulting to. ¯\_(ツ)_/¯
Comment #32
mglamanReviewing this and linked issues.
Comment #33
mglamanIt was added in PHP 5.5.19 and 5.6.3. So PHP 5.4 is a no go.
Comment #34
mglamanThere's no need to set the SSL version, either. The underlying library will handle those goodies. Basically you'd only set this if it was outdated and needed to use some legacy setup.
See similar discussion at #2877107: Use TLS 1.2 with cURL
Comment #35
mglamanHere is rerolled #28 with items from #34 addressed.
Comment #37
mglamanThis is now fixed. Most payment gateways for Drupal Commerce use curl. And it resolves issues about drupal_http_request.
Comment #38
newaytech CreditAttribution: newaytech commentedAfter updating the paypal_commerce module - I now get a 500 error and wihite screen when POSTING to https://www.sandbox.paypal.com/cgi-bin/webscr
I did have the cURL module installed - and have tried the tick box on and off to override the httpRequest method.
Anyone else seeing this?
I'll rollback the code to see if issue persists.
Comment #39
mglamanPlease provide a proper error message from your logs. A 500 error message can be from many things.
Comment #41
istavros CreditAttribution: istavros at Netstudio commented#35 patch didn't work for me so I had to add one more line.
I made a patch for it.
Comment #42
rbogdan CreditAttribution: rbogdan commentedI analyzed code from PayPal examples and prepare patch. My patch add 3 curl options for commerce_paypal_process_ipn().