Problem/Motivation

I am migrating from Drupal 7 + Commerce 1.x to Drupal 9 + Commerce 2.x. All my payments have the decimal point in the wrong place. This is not a problem with prices.

Steps to reproduce

Migrate from Drupal 7 to Drupal 9. Then check the databases.

I need to skip some values that were not migrated. My D7 database:

MySQL [drupal]> SELECT transaction_id, order_id, payment_method, amount
    -> FROM commerce_payment_transaction                                      
    -> WHERE transaction_id > 171 LIMIT 0,5; 
+----------------+----------+----------------+--------+
| transaction_id | order_id | payment_method | amount |
+----------------+----------+----------------+--------+
|            172 |      215 | authnet_aim    |  22467 |
|            173 |      219 | authnet_aim    |  59500 |
|            174 |      222 | authnet_aim    |  18752 |
|            179 |      231 | authnet_aim    |  14595 |
|            180 |      231 | authnet_aim    |  14595 |
+----------------+----------+----------------+--------+
5 rows in set (0.00 sec)

And my D9 database:

MySQL [default]> SELECT payment_id, order_id, amount__number
    -> FROM commerce_payment LIMIT 0,5;
+------------+----------+----------------+
| payment_id | order_id | amount__number |
+------------+----------+----------------+
|        172 |      215 |     22467.0000 |
|        173 |      219 |   59500.000000 |
|        174 |      222 |   18752.000000 |
|        179 |      231 |   14595.000000 |
|        180 |      231 |   14595.000000 |
+------------+----------+----------------+
5 rows in set (0.000 sec)

In Drupal 9, all those values should have two more digits after the decimal point.

Proposed resolution

Divide by 100 in the prepareRow() method of the migration source.

This works for my use case. We might have to work harder to handle all cases.

Remaining tasks

Add test coverage.

User interface changes

None

API changes

None

Data model changes

None

Comments

benjifisher created an issue. See original summary.

benjifisher’s picture

Assigned: benjifisher » Unassigned
Status: Active » Needs review
StatusFileSize
new952 bytes

The attached patch works for me.

Status: Needs review » Needs work

The last submitted patch, 2: 3233594-2.patch, failed testing. View results

benjifisher’s picture

The failing tests are not a surprise:

1) Drupal\Tests\commerce_migrate_commerce\Kernel\Migrate\commerce1\PaymentTest::testPayment
Failed asserting that two strings are identical.
--- Expected
+++ Actual
@@ @@
-'12000.000000'
+'120.000000'

and

1) Drupal\Tests\commerce_migrate_commerce\Kernel\Plugin\migrate\source\commerce1\PaymentTransactionTest::testSource with data set #0 [snip]
Value at 'array[0][amount]' is not correct.
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'425'
+'4.25'

My site is happier with the patch in #2.

I asked on the #commerce channel in Drupal Slack: am I just not finding the configuration option where you say how many decimal points the payments should be? I was told that I am not missing anything. Based on that, I think I am right and the automated tests are wrong.

tonytheferg’s picture

@benjifisher I am using this as a process plugin as a work around:

<?php

namespace Drupal\my_custom\Plugin\migrate\process;

use Drupal\commerce_price\Calculator;
use Drupal\migrate\MigrateExecutableInterface;
use Drupal\migrate\MigrateSkipRowException;
use Drupal\migrate\ProcessPluginBase;
use Drupal\migrate\Row;

/**
 *
 * @MigrateProcessPlugin(
 *   id = "commerce1_migrate_payment_amount"
 * )
 */
class PaymentAmount extends ProcessPluginBase {

  /**
   * {@inheritdoc}
   */
  public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
    if($value) {
      $new_value = [
        'number' => Calculator::divide($value, bcpow(10, 2)),
        'currency_code' => 'USD',
      ];

      return $new_value;
    }
  }
}
rclemings’s picture

I've seen the same thing but not consistently. I haven't been able to discern any pattern as to when it happens.

travis-bradbury’s picture

To support other currencies, we should use Drupal\commerce_price\MinorUnitConverter::fromMinorUnits instead of assuming it should be divided by 100.

benstallings’s picture

An alternative approach is taken by https://www.drupal.org/project/commerce_migrate/issues/3262612 , which allows the CommercePrice process plugin to be used with these values by passing it array of amount and currency_code.

tkiehne’s picture

damienmckenna’s picture

Status: Needs work » Needs review
StatusFileSize
new2.47 KB

I'm seeing the data showing as it does in the issue summary - commerce_payment.amount_number values show as e.g. "572.0000". I suspect the patch should resolve the problem, but I suspect #8 is probably the best approach.

That said, here's some test coverage.

damienmckenna’s picture

StatusFileSize
new2.65 KB

I thought so - the balance_number needed to be changed too.

damienmckenna’s picture

Version: 3.1.x-dev » 3.2.x-dev
damienmckenna’s picture

StatusFileSize
new4.17 KB

This moves over the amount/currency logic from #3262612 and adds the test coverage from here, but doesn't include Benji's original change.

damienmckenna’s picture

The logic from #3262612 doesn't help :-\

damienmckenna’s picture

Going back to the original solution, this should fix the second payment record.

damienmckenna’s picture

Let's add test coverage for the third successful order that's migrated.

tkiehne’s picture

#16 works on my test sandbox and on my live test data with > 30,000 payments

damienmckenna’s picture

A tests-only version of patch #16.

damienmckenna’s picture

Most of the test failures are because of incompatibilities with PHP 8.1, so I opened #3270534.

joekers’s picture

I've tested the patch in #16 and it's working great for me - thanks!

benstallings’s picture

Status: Needs review » Reviewed & tested by the community

setting to RTBC

quietone’s picture

Status: Reviewed & tested by the community » Needs work

Thanks for finding and fixing this and providing a test. Just one point.

+++ b/modules/commerce/src/Plugin/migrate/source/commerce1/PaymentTransaction.php
@@ -68,6 +68,12 @@ public function prepareRow(Row $row) {
+      $amount = (float) $amount / 100;

This is assuming that the currency uses two fraction digits. It would be more robust to get the actual fraction digits for the currency and use that, like the ShippingFlatRate source plugin.
Something like this,

    if ($amount = $row->getSourceProperty('amount')) {
      $currencyRepository = new CurrencyRepository();
      $currency_code = $row->getSourceProperty('currency_code');
      $fraction_digits = $currencyRepository->get($currency_code)->getFractionDigits();
      $amount = bcdiv($amount, bcpow(10, $fraction_digits), $fraction_digits);
      $row->setSourceProperty('amount', $amount);
    }
brianlp’s picture

Patch in #16 worked for me as well, thanks! (Currencies used here have two fraction digits.)

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new1.28 KB
new3.33 KB
new4.63 KB

Adding in the suggestion from #22

The last submitted patch, 24: 3233594-24-fail.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

rishi.kulshreshtha’s picture

Status: Needs review » Reviewed & tested by the community

The patch provided by @quietone works without any issues.

damienmckenna’s picture

quietone’s picture

I am going to commit #24 because it didn't modify the tests and they still passed. The same way of handling the currency fraction digits is used in the CommercePrice process plugin and the ShippingFlatRate source plugin.

Thanks everyone for working on this problem

quietone’s picture

Status: Reviewed & tested by the community » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.