Redirect hash not matching after SHA512 implementation

I note that the error message being logged indicates that the problem is with the redirect hash, not the MD5/SHA-512 hash. These are unrelated, and there were no changes to the redirect hash logic. Did you change the value of your "Redirect validation key"? It's normally a 16-byte randomly-generated string.

The redirect hash isn't part of the SIM/DPM API and has no settings on the Authorize side. There's a browser redirection involved in the completion of a SIM/DPM transaction, and the redirect hash is used by the module to try to prevent forged redirect messages. The module sends the hashed data to Authorize, and Authorize just sends it back. The module then checks to make sure the returned hash is correct before completing the transaction. This has no connection to the MD5/SHA-512 hash used earlier in the transaction.

Your error message shows no hash values at all, which seems problematic. I recommend that you replace whatever you currently have in the "Redirect validation key" setting with a new random 16-byte printable ASCII value, and see if that makes a difference.

Any other messages from the module being logged besides the one about the redirect hash mismatch?

Actually, that is useful. This message:

02/02/2019 - 1:47pm An unauthenticated response from Authorize.Net made it to checkout.

is the key. It means that the first failure wasn't the redirect hash match, but the SHA-512 hash match. The redirect hash failure is incidental to the previous SHA one.

There's likely a mismatch between your Signature Key setting in the module configuration and the one generated and stored by Authorize.Net. Take a close look at the two. Is there an embedded line break in the module's pasted-in version? White space at the beginning and end *should* be ignored, but it's worth checking. Anything characters get truncated?

I'm having some difficulty understanding how a given Signature Key could work intermittently, with no changes to your module configuration, unless there's a problem on the Authorize side. I'm wondering if they might not be consistently returning the SHA hash for your account. Am I correct in understanding that all of the problems that you've observed have been with a single account?

Try adding the second two watchdog() invocations to the existing one around line 507 of commerce_authnet_simdpm.module:

  // Handle response, if valid.

  if (!$auth_successful || !$order) { // invalid response
    $message = t("We're sorry, but due to a technical problem, your order could not be processed. Please try placing the order again later.");
    watchdog('commerce_authnet_simdpm', 'An unauthenticated response from Authorize.Net made it to checkout. Order ID: !order_id',
             array("!order_id" => $order->order_id), WATCHDOG_ERROR);
    watchdog('commerce_authnet_simdpm', 'Returned security hash: !RHASH', array('!RHASH' => $auth_hash), WATCHDOG_DEBUG);
    watchdog('commerce_authnet_simdpm', 'Calculated security hash: !CHASH', array('!CHASH' => $local_hash), WATCHDOG_DEBUG);
  }

That will tell us if you're getting back a blank or missing hash from Authorize, or something silly like that.

Do you have another account, including a sandbox account, that you could test with? No one else has reported this problem, either with this module or its Ubercart equivalent, and I haven't seen it in my own testing or among my own clients. If it's truly account-specific, you might need to get Authorize.Net involved to see if they might be bungling the hash calculation for some reason. If it's not, and there's some data-dependent edge case that triggers the problem, it ought to be possible to collect enough details about the values going into the hash to track it down.

On reflection, it's worth pointing out that the Signature Key is used *twice* in a transaction: first, in the payment request sent to Authorize, the time and a few other values (login ID, order ID, purchase amount) are hashed with the key on the Drupal side and that hash is recalculated as a check by Authorize; and second, Authorize hashes together a bunch of returned transaction information with the key and adds that to the payment response, whereupon Drupal recalculates the hash and compares the result. In your case, the first use is always working, or you'd be seeing a different error entirely. This means that your saved Signature Key is fine. The failure is coming from the second use of the key, and since we already know that the key is OK, there must be some variation in how Drupal and Authorize are hashing the data values together. One thing that occurs to me is that if the address/contact data contains characters that would be encoded by a check_plain(), you might see this kind of failure. I probably should be hashing together raw POST values. That could explain the intermittent nature of this. Is there anything in the contact data that you're testing with that might be encoded by check_plain()?

Since this is a production client, you might need to roll them back to the MD5 implementation in the short term. So long as you didn't remove the MD5 hash or transaction key on the Authorize end, you ought to be able to re-install V1.4 of this module. Both of those values are stored in the variables table in Drupal, and weren't removed by the new module (which just added the Signature Key), so the old version should still be able to access them. Authorize is going to break this at some point in the future by no longer returning the MD5 hash value, but it should be OK for a bit.

Here's a patch against the dev version of the module to test whether check_plain() in the hash calculation is the problem.

check_plain() only affects quotation marks, ampersands, and angle brackets, right?

So says the documentation. Both single and double quotes are included.

(And I'm guessing you're only using the billing information anyway, not shipping.)

Actually, Authorize specifies a long list of returned data to be hashed, including the shipping address:

        $hash_data =  "^" . $_POST['x_trans_id'] .
                      "^" . $_POST['x_test_request'] .
                      "^" . $_POST['x_response_code'] .
                      "^" . $_POST['x_auth_code'] .
                      "^" . $_POST['x_cvv2_resp_code'] .
                      "^" . $_POST['x_cavv_response'] .
                      "^" . $_POST['x_avs_code'] .
                      "^" . $_POST['x_method'] .
                      "^" . $_POST['x_account_number'] .
                      "^" . $_POST['x_amount'] .
                      "^" . $_POST['x_company'] .
                      "^" . $_POST['x_first_name'] .
                      "^" . $_POST['x_last_name'] .
                      "^" . $_POST['x_address'] .
                      "^" . $_POST['x_city'] .
                      "^" . $_POST['x_state'] .
                      "^" . $_POST['x_zip'] .
                      "^" . $_POST['x_country'] .
                      "^" . $_POST['x_phone'] .
                      "^" . $_POST['x_fax'] .
                      "^" . $_POST['x_email'] .
                      "^" . $_POST['x_ship_to_company'] .
                      "^" . $_POST['x_ship_to_first_name'] .
                      "^" . $_POST['x_ship_to_last_name'] .
                      "^" . $_POST['x_ship_to_address'] .
                      "^" . $_POST['x_ship_to_city'] .
                      "^" . $_POST['x_ship_to_state'] .
                      "^" . $_POST['x_ship_to_zip'] .
                      "^" . $_POST['x_ship_to_country'] .
                      "^" . $_POST['x_invoice_num'] .
                      "^";

The CVV difference is interesting and seems worth pursuing. I haven't tested without it in the new version. The response hash includes much more data than it used to, including the CVV2 check response code. This shouldn't matter, as they're supposed to be generating their version of the hash based on the same response data that they're sending us, but accidents happen. If disabling it for the sandbox account breaks the comparison, that would be a good confirmation.

Not really, I'm afraid. The problem does seem to be account-specific, and we have no insight into the account beyond their web interface.

At this point, I'd recommend reporting this to Authorize.Net support and asking them to review your account for any idiosyncrasies. You can tell them that the same software works with your sandbox account, and with other production accounts as well (as reported here by others). Please let us know what you find out. I'll postpone this for now.

I'm going to open a new issue regarding the use of check_plain() when calculating the response hash, and copy the patch in #15 there. It wasn't the cause of your problem (for most cases, anyway), but it still ought to be addressed.

I'm not sure if there were better, more specific questions I should have been asking (if you can think of any let me know and I will contact them again.)

I don't think so. I was hoping that they might spot something unusual about your account upon review, but apparently not.

Since I can't reproduce this now on dev, it seems possible that I could re-enable the new version on production and it would work, but I'm reluctant to find out the hard way that I'm wrong about that!

It might be worth trying it using test mode after-hours, if only to take some of the pressure off of your D8 effort.

As I've still received no similar reports from other users of this module or its Ubercart counterpart, I'm going to assume that your earlier failures were somehow specific to either your Authorize account or your particular environment. I'm going to close this issue as "Can't duplicate", but please let us know if you find out more, and feel free to re-open it if you suspect that there's something here that I can fix. Sorry that we couldn't track it down so far.

I just wanted to add that we are experiencing a similar, intermittent issue where although the payment goes through fine at Authorize, it throws the error below (we don't get any info about hash values, though).

"An unauthenticated response from user after redirection from Authorize.Net made it to checkout!".

This is on a new Authorize account, with a new install of the 7.x-1.7 SIM/DPM module.

The order is then marked as abandoned. I'm not a developer, but I'm a pretty good monkey if you can give me some direction on how to help.

These are the errors we receive, in the order they appear in the logs. Note no mention of the hash values.

It seems to be somewhat random - we'll have an error, then have a different customer checkout with no problem later. We haven't been able to reproduce it on a live server with test transactions ourselves. It always seem to work for us.

We're using the SIM method. Would it make sense to switch to DPM to see if that helps things?

The "abandoned cart" transaction:

Tuesday, April 30, 2019 - 16:05 Receiving redirect response for order 2760 from Authorize.Net
Tuesday, April 30, 2019 - 16:05 An unauthenticated response from Authorize.Net made it to checkout. Cart order ID: 2760
Tuesday, April 30, 2019 - 16:05 Receiving payment response for order 2760 from Authorize.Net

And then later that day (with no changes to the settings), a successful transaction:

Tuesday, April 30, 2019 - 23:46 - Receiving redirect response for order 2765 from Authorize.Net
Tuesday, April 30, 2019 - 23:46 - Receiving payment response for order 2765 from Authorize.Net

Re-opening the issue, as it may indeed be the same problem as originally reported.

This log entry:

Tuesday, April 30, 2019 - 16:05 An unauthenticated response from Authorize.Net made it to checkout. Cart order ID: 2760

indicates that the payment response posted by Authorize didn't pass the verification test, so the hash calculated and sent by Authorize didn't match the one we calculated ourselves. Since the payment request previously sent from your site to Authorize uses the same SHA key and was accepted (or the payment would have failed), the key itself should be fine. This again suggests that there are cases where Authorizes hashes some of the customer data (see #17 above) differently that we do *in some cases*. Have a look at those fields for the orders which failed and see if you can spot any unusual characters or punctuation which is common to them, but wouldn't appear in your test transactions or others which worked as expected.

Switching to DPM isn't likely to be helpful with this.

Sorry, my comments above were meant for the Ubercart version of this module, though I suspect this applies to both. Let me know if I should move the issue to there.

Your intuition was correct - I was able to reproduce a failure on a character used from one of the two customers. If you insert an "Ń" (i.e. with the accent mark) in the name field, it will trigger the issue. However, it left the cart in "in checkout" rather than abandoned, but all the error messages were the same.

Even though Authorize seems to recognize the "Ń" (it displays it on the SIM page), I'm guessing it's stripping it from the hash calculation or something? Pure speculation on my part.

I'm still working on the other customer's order. I'll let you know what I find.

Thanks so much for your help, support, and amazing response time! :) :) :)

Second customer with issue:

I'll do my best to explain this.

This transaction went through on Authorize's side, but showed up as "abandoned" on Ubercart.

The address looks completely normal on Ubercart's side. But for some reason in Authorize's Transaction Detail, it shows a "?" character right before the street name. In other words "1234 �YourStreet Rd." Again, there is nothing that shows up in that position on the Ubercart order page.

On the Merchant Email Receipt that Authorize sends, there is an extra space in place of that question mark character.

Later I will try adding an extra space to the address field and see if that also reproduces an error.

Okay, I verified that that adding two spaces in between the address number and the address street produces the "unauthenticated response" error, puts the cart in "in checkout", but the transaction completes on the Authorize side.

The "�" also appeared only in the transaction detail on Authorize.net, not in Ubercart or in the Merchant Receipt email.

The double space only appeared in the Merchant Receipt email. In ubercart, no indication of a double space exists.

Let me know if I can assist in any other way! :)

Thanks for your testing on this.

I'm guessing that Authorize is doing some kind of text normalization on the raw data before generating their hash, but I don't recall seeing a description of it. I'll have another look at the available documentation over the weekend and make sure that I didn't miss something. If I don't find an explanation, I can try to recreate the problem against their dev server and then try a series of likely normalizations on the raw data before sending it to them in an attempt to deduce what they're up to.

I suppose it's also possible that they're not normalizing the data, but that their hash calculation handles the data differently from what we're doing in PHP for some unusual cases. If so, some kind of compatible pre-normalization on our end is likely still the required solution.

Thanks much, Jerry! I agree with your assessment, though I'm pretty ignorant on such matters.

There's a relevant discussion of it here: https://community.developer.authorize.net/t5/Integration-and-Testing/Rel...

More discussion of the "non-standard ASCII" character hash issue here. Question brought up on message 38, potential solution on message 43. It doesn't address Polish characters, though (which is what my customer was submitting), nor does it address the double space issue.

https://community.developer.authorize.net/t5/Integration-and-Testing/Upg...

Thanks for doing all this legwork.

How Authorize is handling the character encoding during their hash calculation is a bit murky. I'm attaching a patch that converts any data used in the response hash calculation from UTF-8 to ISO8859-1 before sending it to Authorize, which appears to be effective in preventing the hash mismatch. The SIM form will show replacement characters ("?") for any multibyte characters sent, but the original data is retained in the order record. If someone has a better solution, please post it. I have limited time available to refine this myself.

Note that this is a patch for the Commerce version of this module. An equivalent patch will be posted to the Ubercart version's issue queue.

Linking related Ubercart module issue.

Sounds like a good solution to me. Thanks much for your expertise, time, and effort, Jerry!

What about the double space problem? Is that something we can strip out prior to sending to Authorize as well?

Perhaps this code would work?

https://stackoverflow.com/questions/1981349/regex-to-replace-multiple-sp...

Multiple-space replacement is already being done:

/**
 * Convert a string potentially containing UTF-8 characters to ISO8859-1 to
 * prevent Authorize.Net hash verification issues.
 */
function _isoize($str) {
  return utf8_decode(preg_replace('/\s\s+/', ' ', trim($str)));
}

Ha, shows you what I know. Thanks much, Jerry! I'll give it some testing soon. :)

This patch works great on Ubercart; I expect it will on Commerce as well. Thanks much, Jerry!

Hello,
thanks a lot for the patch, it resolve most of the issues. But I still have issue with ampersand.
If there's a '&' in user informations like company name, the error still occurs.
I temporarily change the function _isoize to work around this problem.

function _isoize($str) {
  return utf8_decode(preg_replace(array('/\s\s+/', '/&/'), array(' ', ' '), trim($str)));
}

But :
- it can't be a permanent solution
- if the client set a '&' on the authorize interface to correct the missing one in his informations, the error occurs to.

If someone has an idea...

Thanks for the feedback about the ampersand, @aliod. Anyone aware of any other problematic characters that should be special-cased?

if the client set a '&' on the authorize interface to correct the missing one in his informations, the error occurs to

If you disable their ability to edit the form on the Authorize side, as suggested in the installation notes, this won't be an issue.

I tested this patch by putting an umlaut over the "u" in my last name. It did indeed allow the transaction to go through; however, it did this by stripping my entire last name from the form.

I didn't observe that behavior here. The filtering function operates on characters, not words, so I'm not sure why that would have occurred.

If you disable their ability to edit the form on the Authorize side, as suggested in the installation notes, this won't be an issue.

Thank you @jerry for your suggestion.

Here's a proper patch based on @aliod's input, with the replacement patterns switched so that multiple spaces are still removed if an ampersand is replaced.

Fixed in 7.x-1.8.

Encountering a new hashing issue. This time is on the $message or $_GET[REDIRECT_ARG_MESSAGE].

The URL returned is

cart/authnet_simdpm/order_complete?...&msg=We%27re%20sorry%2C%20but%20due%20to%20a%20technical%20problem%2C%20your%20order%20could%20not%20be%20processed.%20Please%20try%20placing%20the%20order%20again%20later.

But has been parsed as, note the encoded "We're"

We're sorry, but due to a technical problem, your order could not be processed. Please try placing the order again later.

Attached patch fixes it.

Now having a new problem, transaction is captured but order failed at commerce_authnet_simdpm_redirect_form_validate(), called by commerce_payment_redirect_pane_checkout_form() via the drupal_goto() checkout/OID/payment/return/KEY.

Error message is
'Payment failed at the payment server. Please review your information and try again.'

function commerce_authnet_simdpm_redirect_form_validate($order, $payment_method) {
  // Should always fail, as the payment return URL should never be visited externally.
  // Visiting it via drupal_goto() (as is done normally) doesn't trigger the function.
  return FALSE;
}

So this part of the code no longer work? Any idea?

Thanks for your contribution. I'll leave it for the next maintainer to review.

Regarding your validation error: as I haven't observed this, nor has anyone else reported it, I think that it's likely to be a configuration or environmental issue unique to your implementation.

Thanks a million, ckng. Your patch at #55 works wonderfully.