Multiple transactions - Suggestions Needed

I'm experiencing this issue today with a site. Have you been able to identify root cause or get any feedback from other users?

I am experiencing this issue as well. We only experience the issue sometimes. It seems to happen when we get multiple hits to the site that attempt to kick off cron runs. I've started digging into the code, but have not found anything yet.

Has anyone identified the root cause or a solution?

This patch may help.
There is a bug in the commerce_recurring_order_load_recurring_line_items function.
If it's called twice for two different orders then the second call returns the line items for the first call.

In the cron run it is called twice.

This is due to the drupal static call.

Also if the order line items change for the same order, then the unchanged line items are called.

For safety I think it's better that the line items are not cached.

Also this SQL may help identify if you have any customers who have been charged to frequently

-- Get orders for users repeating too frequently.
-- Note the 5*24*60*60 represents 5 days, you may need to adjust this.
SELECT DISTINCT o1.mail, o1.order_id, FROM_UNIXTIME(o1.created, "%d/%m/%Y %H:%i") FROM `commerce_order` o1  
INNER JOIN `commerce_order` o2 
  ON o1.`uid` = o2.`uid`
  AND o1.`order_id` <> o2.`order_id`
WHERE ABS(o1.`created` - o2.`created`) <= (5*24*60*60)
AND 
o1.status = "cardonfile_charged" 
AND
o2.`status`= "cardonfile_charged"
AND 
o1.order_id > 1000
ORDER BY o1.mail, o1.`order_id`;

Another issue could be deleting users with recurring entities, the recurring entities will still trigger transactions.

People being charged more than they should be seems like a Critical to me. Plus there is a patch for review.

Thanks, @Chim!

Thanks torgosPizza I agree.
This is critical, we've been able to demonstrate that if two or more orders are processed on the same cron then only the first orders recurring entity is updated with a new due date.

There's another related ticket https://www.drupal.org/node/2075607, but... I think the patch above is a better solution as the other patch doesn't allow for changed line items on the same order.
That is, it still returns the cached originals.

This means that the next cron run will process all but the first again. An unlucky customer could be charged lots of times, if you run an hourly cron, then this could potentfially empty a users account or max out a user's credit card quite quickly.

This is huge isnt it!? Shouldnt we notify people that the module is unsafe without this patch?

Or can we get this committed asap?

Agreed, this is a critical bug and let's get this in right in. Thanks a lot for reporting and fixing it @Chim!

It would be awesome if someone can produce a test for this.

Thanks pcambra.

I'll have another go.

but I did notice that
testCommerceRecurringPurchaseTwoRecurringProducts passes now on the automated testing

Not sure if we can attribute that pass to this patch though

I have a Drupal site with commerce card on file and authorize.net...

The site charged 3 customers about 7 times last night (once each hour) - it should have only charged them once each. Every time it charged them it kept setting their Due Date to "0 Seconds" and charging again. This seems like a big deal and and any guidance would be appreciated.

Thanks!

Have you applied the patch or updated the the dev version?

My code looks the same as the patch.

I'm running:
Card on File Recurring - 7.x-2.0-Beta5
Commerce Recurring - 7.x-2.x-dev
Authorize.net - 7.x-1.4

Thank you!

Experiencing this issue as well. Multiple charges.

I'm running:
Card on File Recurring - 7.x-2.0-Beta5
Commerce Recurring - 7.x-2.x-dev
Stripe - 7.x-1.2+3-dev

Not sure. May be part of this issue.

I am getting a warning on admin/commerce/recurring-entities/9/edit.

Warning: call_user_func_array() expects parameter 1 to be a valid callback, function 'commerce_recurring_edit_product_form' not found or invalid function name in drupal_retrieve_form() (line 842 of ... /public_html/includes/form.inc).

Thanks!

This should be mentioned in the description of the module as critical. I am upgrading to the dev version asap, but according to #15 and #16, it doesn't seem to be fixing the issue.

I have been authorized by my company to offer a $100 bounty to be split between whoever fixes this issue within the next 1 week. Bounty reduces to $50 if it takes longer.

I have been authorized by my company to offer a $100 bounty to be split between whoever fixes this issue within the next 1 week. Bounty reduces to $50 if it takes longer.

Please provide test cases and ways to reproduce this problem instead.

pcambra, thank you for responding so quickly. It is much appreciated.

I am using these modules:
Commerce Recurring Framework: Version: 7.x-2.x-dev
Card on File - Recurring: Version: 7.x-2.0-beta6+1-dev
Commerce Stripe: Version: 7.x-1.2

Using this patch for Card on File
https://www.drupal.org/node/2000420#comment-11593817

The rule, "Charge pending recurring orders", is the only recurring rule that is overridden in order to select Stripe as the payment method. (see link above)

Please let me know if you need any other information.

I have a testing server with a sandbox payment provider so I can test any patches provided.

@g33kg1rl it'd be very useful if you can give me access to that to see how to reproduce the issue.

@pcambra, I sent you a PM.

Setting back to active

@MichaelSt and @z-digital do either of you have more information about reproducing the issue? Maybe if we can find a common thread between our configuration.

Based on my experience with payment gateways I think the error could be when making multiple requests, without a pause between them, some get time out closing the connection with the server before the response is received, creating a false negative in the payment and provoking that it is tried to make the payment again. As @Chim comment: if more than 1 order is processed, only the first one is updated correctly.

I do not know what method / module you are using to manage the payments, but would it be possible for someone to take the following test and tell us the result?
- Limit the number of payments to be processed to 2/3 every time.
- Add a delay between requests of 5/10 seconds.

This change should probably need to be made either in the payment gateway module or for example in the card on file integration -> https://www.drupal.org/sandbox/jpstrikesback/2260137

g33kg1rl - just updated commerce_cardonfile_recurring to Version: 7.x-2.0-beta6 and am using commerce_recurring Version: 7.x-2.x-dev with commerce_stripe Version: 7.x-1.2+3-dev and 'Integration type' stripe js. I was using a friendly client to run a 'live' recurring monthly bill and his payment was triggered at checkout and then again the following day (same billing amount paid twice).

I applied the 'multiple-transactions-2649890-4.patch' about a year ago. Same result, payment was triggered at checkout and then again the following day. I have not used this recurring payment method since.

I'm not even sure I have the recurring products configured properly. I did watch "Commerce Recurring Framework Commerce Module Tuesday" by Augustin Delaporte (?) and went through the commerce recurring module description overview and issues sections.

I have my site on a dev sub-domain currently and was going to do some further testing. Sorry, I'm not a php coder.

If I can be of any help, please let me know.

Thanks in advance.

@z-digital, what method / module are you using to manage the payments?

facine - short answer is that my 'home' website relies on physical billing for services. I have built several websites that use commerce for products and commerce license for file downloads. I have used stripe as the gateway. I was just trying 'recurring products' and 'recurring billing' so that I can get away from my physical billing.

I have worked with drupal (I like it) to build mostly identity or brochure websites. Being self-taught, usually youtube has enough to get me past the issues I've had. Commerce however, doesn't really have the deeper (step-by-step) build training - not their fault.

Thanks for the follow-up and thanks again for the help.

I believe I was having this issue on a site, but if I remember correctly I was able to fix it by simply limiting the number of fetched entities to 2 in the Fetch entity by property for the "Charge pending recurring orders" rule. It seems like if I allowed more than than that many to process each time the Commerce Recurring cron is executed it would get hung up and wouldn't ever change the order status even if the payment went through. This would cause the same orders to pile up and get charged over and over and over again simply because the server couldn't seem to finish processing the rules. I'm pretty sure once I changed it so that it would only process 2 at a time the problem stopped happening.

Can anybody confirm if the patch here actually fixes this problem? It would be nice if we could process more than 2 orders in a cron run if necessary...

We ran across this issue in a pretty serious way earlier this month where we had hundreds of orders running duplicate charges out of nowhere. I think some of it may have been related to not having enough memory provisioned, so if you do see this then you may want to try upping your memory limit.

I believe that the root cause may have something to do with orders being generate/charged but not saving correctly/being deleted afterward, causing a new pending order to be generated and charged again. While troubleshooting, I hit some out of memory errors while generating the "recurring pending orders" leading me to believe that the issue could be memory related. Unfortunately I didn't notice this until after solving the problem with the below method so YMMV.

We tried applying the patch above, as well as limiting the number of of orders as mentioned by @deggertsen, but that did not solve our problem. What we did to get around the issue was:

1. Disable /admin/config/workflow/rules/reaction/manage/charge_pending_recurring_orders to bypass the default payment processing entirely
2. Update /admin/config/workflow/rules/reaction/manage/commerce_recurring_cron_generate_orders so that the number is guaranteed to be greater than the number of orders to be generated on a cron run
3. Implement a custom module (attached) to process payments on pending orders and ensure that the order has not already been charged

Some notes on the attached module:

1. It sets up a queue and populates it with all pending orders, ensuring that each order is only added to the queue a single time.
2. The queue is populated and orders from it processed on cron.
3. It creates a couple of custom order statuses to apply to orders when they are found to be a duplicate or when the payment fails in an unexpected way (I still need to work out what this failure entails, but I think it was causing the orders to retain their recurring_pending status). This makes it easier to track which orders are causing issues and make sure that no further charges are attempted on them while you investigate them.
4. We are using Stripe for payment processing, and each payment to Stripe holds the order number. As such this module leverages Stripe's API to check the charge history of the order owner prior to processing to ensure that the order has not been previously charged. You'd need to update this section if you are not using Drupal's Stripe implementation for processing.

We've got this module running on Prod now and have not experienced any further duplicate charges. I wish I could have posted an update with a patch or something, but hopefully this is just as functional.

Marking as "Needs Review" to alert maintainers to this workaround in case they'd like to implement something similar anywhere.

I created a case and provided a patch to add the ability to hook into the function that sets the new due date so that I could take action if I saw that the data was suspicious. I recommend anybody who's having problems to look at #2995101: Ability to alter recurring entity or add monitoring code when extending values after successful recursion.

Am I right in thinking this bug is to do with the Commerce Cardonfile Recurring module that comes packaged with Commerce Cardonfile?