Problem/Motivation

Migrate the discount line items to an order item adjustment

Asked on Commerce Slack about adjustments and bojanz confirmed that promotions are adjustments on the order item.

quietone [9:11 AM]
Working to understand adjustments. From my testing I see that shipping adjustments are always on  the order and promotion adjustments are always on the order item. Taxes can be on both. Is that correct?

bojanz 2 hours ago
They are always on the order item since 2.7/2.8 causr VAT

Add clarification that bojanz is referring to promotions.

Proposed resolution

Get all the discount on the order and for each one get the discount data and then build an adjustment array. Use the total price as the value. The lock is not explicitly set here but will default to TRUE in commerce_adjustments process plugin.

Remaining tasks

Commit

CommentFileSizeAuthor
#123 3012110-123.patch18.13 KBquietone
#123 interdiff-122-123.txt1.76 KBquietone
#122 interdiff-121-122.txt855 bytesquietone
#122 3012110-122.patch18.69 KBquietone
#121 interdiff-119-121.txt14.15 KBquietone
#121 3012110-121.patch18.8 KBquietone
#119 3012110-119.patch17.11 KBquietone
#114 interdiff.txt1.77 KBquietone
#114 3012110-114.patch930.03 KBquietone
#112 3012110-112.patch930.12 KBquietone
#109 3012110-109.patch930.94 KBquietone
#109 3012110-109-review-only.txt27.44 KBquietone
#108 interdiff.txt554 bytesquietone
#108 3012110-108.patch931.07 KBquietone
#106 interdiff.txt1.22 KBquietone
#106 3012110-106.patch931.12 KBquietone
#104 interdiff-101-104.txt436 bytesquietone
#104 3012110-104.patch930.4 KBquietone
#101 interdiff.txt1.36 KBquietone
#101 3012110-101.patch931.06 KBquietone
#99 interiff.txt1.17 KBquietone
#99 3012110-99.patch930.94 KBquietone
#97 interiff.txt1.17 KBquietone
#97 3012110-97.patch930.35 KBquietone
#95 interdiff.txt1.36 KBquietone
#95 3012110-95.patch931.06 KBquietone
#93 3012110-93.patch930.37 KBquietone
#91 3012110-91.patch927.94 KBquietone
#87 3012110-87.patch931.87 KBquietone
#85 3012110-85.patch995.22 KBquietone
#82 interdiff-80-82.txt6.09 KBquietone
#82 3012110-82.patch997.13 KBquietone
#80 3012110-80.patch995.64 KBquietone
#78 3012110-78.patch998.02 KBquietone
#77 3012110-77.patch997.97 KBquietone
#77 interdiff-75-77.txt930 bytesquietone
#75 interdiff.txt1.11 KBquietone
#75 3012110-75.patch997.97 KBquietone
#74 interdiff-58-74.txt753 bytesquietone
#74 3012110-74.patch997.96 KBquietone
#73 3012110-73.patch997.41 KBquietone
#68 3012110-68.patch998.08 KBquietone
#65 3012110-65.patch998.62 KBquietone
#59 3012110-58-review-only-patch.txt37.16 KBquietone
#58 interdiff-57-58.txt6.22 KBquietone
#58 3012110-58.patch997.86 KBquietone
#57 interdiff-55-57.txt1.56 KBquietone
#57 3012110-57.patch997.19 KBquietone
#55 interdiff-53-55.txt6.62 KBquietone
#55 3012110-55.patch995.77 KBquietone
#53 interdiff-51-53.txt1.76 KBquietone
#53 3012110-53.patch992.79 KBquietone
#51 interdiff.txt5.42 KBquietone
#51 3012110-51.patch991.4 KBquietone
#50 interdiff-48-50.txt3.96 KBquietone
#50 3012110-50.patch993.12 KBquietone
#48 interdiff-42-48.txt1.42 KBquietone
#48 3012110-48.patch993.05 KBquietone
#46 interdiff-42-44.txt2.91 KBquietone
#46 3012110-44.patch994.16 KBquietone
#42 interdiff-33-42.txt3.48 KBquietone
#42 3012110-42.patch992.85 KBquietone
#42 interdiff-39-41.txt1.78 KBquietone
#42 interdiff-38-39.txt4.31 KBquietone
#42 interdiff-33-38.txt10.44 KBquietone
#41 commerce_migrate-commerce_discounts_support-3012110-41.patch995.14 KBcameron prince
#40 commerce_migrate-commerce_discounts_support-3012110-39.patch995.27 KBcameron prince
#38 commerce_migrate-commerce_discounts_support-3012110-38.patch994.88 KBcameron prince
#33 interdiff-31-33.txt4.06 KBquietone
#33 3012110-33.patch989.47 KBquietone
#31 interdiff-30-31.txt6.25 KBquietone
#31 3012110-31.patch988.79 KBquietone
#30 3012110-30.patch993.14 KBquietone
#27 3012110-25-review-only.txt37.87 KBquietone
#25 interdiff.txt723 bytesquietone
#25 3012110-25.patch1000.5 KBquietone
#23 interdiff.txt716 bytesquietone
#23 3012110-23.patch1000.17 KBquietone
#21 interdiff.txt749 bytesquietone
#21 3012110-21.patch1000.47 KBquietone
#19 interdiff.txt709 bytesquietone
#19 3012110-19.patch1000.47 KBquietone
#17 interdiff.txt1.79 KBquietone
#17 3012110-17.patch1000.36 KBquietone
#16 interdiff.txt1.74 KBquietone
#16 3012110-16.patch97.06 KBquietone
#14 interdiff-12-14.txt9.5 KBquietone
#14 3012110-14.patch95.19 KBquietone
#12 3012110-12.patch94.5 KBquietone
#11 interdiff.txt1.23 KBquietone
#11 3012110-11.patch89.42 KBquietone
#9 interdiff.txt5 KBquietone
#9 3012110-9.patch90 KBquietone
#7 interdiff.txt772 bytesquietone
#7 3012110-7.patch92.69 KBquietone
#5 interdiff.txt3.22 KBquietone
#5 3012110-5.patch93.07 KBquietone
#3 3012110-3.patch101.6 KBquietone
#2 3012110-2.patch20.87 KBquietone

Comments

quietone created an issue. See original summary.

quietone’s picture

StatusFileSize
new20.87 KB

This is just a working patch and does not have the test fixture it requires because It is a mess and it is too late to be editing that beast now. So, this is just the code to migrate discounts to adjustments. Think of it as a start.

quietone’s picture

Status: Active » Needs review
StatusFileSize
new101.6 KB

Add in the fixture changes and see what breaks.

Status: Needs review » Needs work

The last submitted patch, 3: 3012110-3.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new93.07 KB
new3.22 KB

Remove test files that are not part of this patch.
Add empty adjustment array to ubercart 6 and 7 order item test.

Status: Needs review » Needs work

The last submitted patch, 5: 3012110-5.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new92.69 KB
new772 bytes

Yes,there are not promotion migrated in this patch

quietone’s picture

Status: Needs review » Needs work
  1. +++ b/modules/commerce/src/Plugin/migrate/process/commerce1/OrderItemAdjustment.php
    @@ -0,0 +1,56 @@
    +      case 'product':
    

    Are there other line item adjustments that need to be handled?

  2. +++ b/modules/commerce/src/Plugin/migrate/process/commerce1/OrderItemAdjustment.php
    @@ -0,0 +1,56 @@
    +        foreach ($unit_price['data']['components'] as $component) {
    

    Should this use unit price or total price?

  3. +++ b/modules/commerce/src/Plugin/migrate/process/commerce1/OrderItemAdjustment.php
    @@ -0,0 +1,56 @@
    +                'locked' => FALSE,
    

    Probably TRUE?

  4. +++ b/modules/commerce/src/Plugin/migrate/source/commerce1/LineItem.php
    @@ -91,7 +93,21 @@ class LineItem extends FieldableEntity {
    \ No newline at end of file
    

    Add new line

  5. +++ b/tests/src/Kernel/CommerceMigrateTestTrait.php
    @@ -660,6 +691,67 @@ trait CommerceMigrateTestTrait {
    +   * Asserts a promotion entity.
    

    Oops, This method isn't part of this patch. remove it

quietone’s picture

Issue summary: View changes
Status: Needs work » Needs review
StatusFileSize
new90 KB
new5 KB

Fixes for 3,4,5 and some coding standard fixes.
Items 1,2 have been moved to questions in the IS

Status: Needs review » Needs work

The last submitted patch, 9: 3012110-9.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new89.42 KB
new1.23 KB

That was silly, made changes in the process plugin and didn't update the test.

quietone’s picture

StatusFileSize
new94.5 KB

Been down a bit of a rabbit hole with adjustments. This has expanded to include taxes and uses the same fixture as the order adjustment issue.

Using interdiff to make the interdiff fails and this is very nearly a rewrite anyway so no interdiff. Not sure it all tests will pass.

Status: Needs review » Needs work

The last submitted patch, 12: 3012110-12.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new95.19 KB
new9.5 KB

Removing taxes (will do those in a new issue) and fixing tests.
And next is to find out why the bag discount is no longer in the orderitemtest.

Status: Needs review » Needs work

The last submitted patch, 14: 3012110-14.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new97.06 KB
new1.74 KB

Fix the ubercart tests

quietone’s picture

StatusFileSize
new1000.36 KB
new1.79 KB

This adds a product discount and a new test fixture. The interdiff does not include the test fixture changes.

Status: Needs review » Needs work

The last submitted patch, 17: 3012110-17.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new1000.47 KB
new709 bytes

Added some logs when adding the order.

Status: Needs review » Needs work

The last submitted patch, 19: 3012110-19.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new1000.47 KB
new749 bytes

Yes, 2 order items were added.

Status: Needs review » Needs work

The last submitted patch, 21: 3012110-21.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new1000.17 KB
new716 bytes

Add added an order

Status: Needs review » Needs work

The last submitted patch, 23: 3012110-23.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new1000.5 KB
new723 bytes

And finally the entity form display.

quietone’s picture

To review this the key changes are in the commerce sub module. Ignore the changes to the fixture and in the Ubercart sub module (modified because the test method changed).

I am tired of removing stuff from the fixture and am very likely to commit the whole thing. It takes too much time and effort to trim it down.

quietone’s picture

StatusFileSize
new37.87 KB

And for easier review here is the patch from #25 without the fixture noise.

heddn’s picture

Status: Needs review » Needs work
  1. +++ b/modules/commerce/src/Plugin/migrate/OrderItemDeriver.php
    @@ -103,34 +103,37 @@ class OrderItemDeriver extends DeriverBase implements ContainerDeriverInterface
    +        if ($line_item_type !== 'shipping') {
    

    The title is discounts. But there are fees that are not shipping. Comments or clarifications on what we are doing would help make this more clear.

  2. +++ b/modules/commerce/src/Plugin/migrate/process/commerce1/CommercePrice.php
    @@ -19,11 +19,32 @@ class CommercePrice extends ProcessPluginBase {
    +      $price['number'] = bcdiv($price['number'], bcpow(10, $value['fraction_digits']), $value['fraction_digits']);
    

    It might make more sense to use's Drupal\commerce_price\Calculator.

  3. +++ b/modules/commerce/src/Plugin/migrate/process/commerce1/OrderItemAdjustment.php
    @@ -0,0 +1,57 @@
    +            // todo: percentage.
    

    should we link to an issue here?

  4. +++ b/modules/commerce/src/Plugin/migrate/process/commerce1/OrderItemAdjustment.php
    @@ -0,0 +1,57 @@
    +              // todo: use fraction digits?.
    

    Yes, fraction digits is important.

  5. +++ b/modules/commerce/tests/src/Kernel/Migrate/commerce1/OrderTest.php
    @@ -148,6 +152,56 @@ class OrderTest extends Commerce1TestBase {
    +    $order = [
    +      'id' => 4,
    +      'type' => 'default',
    +      'number' => '4',
    +      'store_id' => '1',
    +      'created_time' => '1543271966',
    +      'changed_time' => '1543271966',
    +      'completed_time' => '1543271966',
    +      'email' => 'CommerceKickstart@example.com',
    +      'ip_address' => '127.0.0.1',
    +      'customer_id' => '1',
    +      'placed_time' => '1543271966',
    +      'total_price' => '24.000000',
    +      'total_price_currency' => 'USD',
    +      'adjustments' => [],
    

    There's no adjustments in here. Why?

  6. +++ b/modules/commerce/tests/src/Kernel/Plugin/migrate/source/commerce1/OrderTest.php
    @@ -125,6 +125,48 @@ class OrderTest extends SourceTestBase {
    +    $tests[0]['source_data']['commerce_line_item'] = [
    +      [
    +        'line_item_id' => '1',
    +        'order_id' => '2',
    +        'type' => 'product',
    +        'line_item_label' => 'sku 1',
    +        'quantity' => '2',
    +        'data' => 'a:0{}',
    +        'created' => '1492868907',
    +        'changed' => '1498620003',
    +      ],
    +      [
    +        'line_item_id' => '11',
    +        'order_id' => '2',
    +        'type' => 'shipping',
    +        'line_item_label' => 'Express',
    

    I don't see any non-shipping adjustments. Am I reading this wrong?

quietone’s picture

Needs a reroll

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new993.14 KB

Interdiff for the reroll fails.

quietone’s picture

StatusFileSize
new988.79 KB
new6.25 KB

1. Comment added.
2. TODO
3. TODO
4. Reworked to use existing process plugin to scale price.
5. The order adjustments, which seem to be only shipping, are being done in #2978105: Migrating shipping order adjustments
6. You are reading it correctly. And that shouldn't be there those changes to the source plugin test are no longer valid.

Having trouble running tests locally, so not sure if this will pass.

quietone’s picture

Status: Needs review » Needs work

28.3 Created an issue for discount percentage #3020862: Add percentage to discount order item adjustment.

Needs work for #28.2

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new989.47 KB
new4.06 KB

And finally 28.2. Reworked the price plugins and use Calculator.

quietone’s picture

Issue summary: View changes
cameron prince’s picture

I've been testing this today and was a little surprised to find that the D7 commerce_line_item table contains multiple rows for the same discount on the same order. My guess is that a row was being added each time the "Apply" button was pressed or upon transition from pane to pane...

|          752 |   100382 | commerce_discount |                    |     1.00 | 1482288725 | 1482288725 | a:1:{s:13:"discount_name";s:22:"discount_2016_closeout";}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|          753 |   100382 | commerce_discount |                    |     1.00 | 1482288725 | 1482288725 | a:1:{s:13:"discount_name";s:22:"discount_2016_closeout";}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|          754 |   100382 | commerce_discount |                    |     1.00 | 1482288934 | 1482288934 | a:1:{s:13:"discount_name";s:22:"discount_2016_closeout";}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|          756 |   100382 | commerce_discount |                    |     1.00 | 1482288935 | 1482288935 | a:1:{s:13:"discount_name";s:22:"discount_2016_closeout";}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
|          758 |   100382 | commerce_discount |                    |     1.00 | 1482288957 | 1482288957 | a:1:{s:13:"discount_name";s:22:"discount_2016_closeout";}

Not sure if others will have this problem or if this is the best way to fix it, but I wrote a process plugin which does this to resolve it.

    $order_id = $row->getSourceProperty('order_id');

      $count = \Drupal::database()->select('commerce_order_item', 'oi')
        ->condition('oi.order_id', $order_id)
        ->condition('oi.type', 'default')
        ->condition('oi.unit_price__number', 0, '<')
        ->countQuery()
        ->execute()
        ->fetchField();

      if ($count > 0) {
        throw new MigrateSkipRowException('Order already has promotion order item - skipping.', TRUE);
      }
    }
cameron prince’s picture

In thinking about this further, why do we need to create commerce_order_item rows for discounts? When I place a new order with a discount, no row is created like this in commerce_order_item. Could we just create the adjustment in a process plugin and skip all commerce_discount rows?

cameron prince’s picture

Here's what I came up with... It works, but I'm hesitant to create a patch without further discussion. This is only going to work for order discounts (not line item discounts) and I'm not sure if it's necessary for everyone to have the check for duplicate discount rows I'm seeing. Notice I'm popping off the last discount row, not matter how many are found.

The percentage look-up is here, but it's not covering all use cases.

This is added to the OrderItemAdjustment.php process plugin and commerce_discount type line item rows are excluded from the normal line item source, i.e. commerce_discount is not in the static map on the type field.

This prevents the unnecessary discount rows in the order_item table.

      if ($value['name'] === 'base_price') {
        // Check for discount rows.
        $order_id = $row->getSourceProperty('order_id');

        Database::setActiveConnection('migrate');
        $connection = Database::getConnection();
        $query = $connection->select('commerce_line_item', 'li')
          ->fields('li', ['data'])
          ->fields('ct', ['commerce_total_data'])
          ->condition('li.order_id', $order_id)
          ->condition('li.type', 'commerce_discount');

        $query->innerJoin('field_data_commerce_total', 'ct', 'li.line_item_id = ct.entity_id');

	$discount_line_items = $query->execute()->fetchAll();
        Database::setActiveConnection();

        if ($discount_line_items) {
          $discount_line_item = array_pop((array_slice($discount_line_items, -1)));
          if ($discount_line_item) {
            $data = unserialize($discount_line_item->data);
            if ($data && !empty($data['discount_name'])) {

              // Look up the percentage.
              Database::setActiveConnection('migrate');
              $query = $connection->select('commerce_discount', 'cd')
                ->fields('cp', ['commerce_percentage_value'])
                ->condition('cd.name', $data['discount_name']);

              $query->innerJoin('field_data_commerce_discount_offer', 'do', 'do.entity_id = cd.discount_id');
              $query->innerJoin('field_data_commerce_percentage', 'cp', 'cp.entity_id = do.commerce_discount_offer_target_id');

              $commerce_percentage_value = $query->execute()->fetchField();

              Database::setActiveConnection();

              if (!empty($commerce_percentage_value)) {
                $commerce_total = $row->getSourceProperty('commerce_total');

                // Scale the incoming price by the fraction digits.
                $fraction_digits = isset($commerce_total[0]['fraction_digits']) ? $commerce_total[0]['fraction_digits'] : '2';
                $amount = (string) $commerce_total[0]['amount'];
                $currency_code = $commerce_total[0]['currency_code'];
                $input = [
                  'amount' => $amount,
                  'fraction_digits' => $fraction_digits,
                  'currency_code' => $currency_code,
                ];
                $price = new CommercePrice([], 'price', '');
                $price_scaled = $price->transform($input, $migrate_executable, $row, NULL);

                $commerce_total_data = unserialize($discount_line_item->commerce_total_data);
                $label = $commerce_total_data['components'][1]['price']['data']['discount_component_title'];

                $adjustment = [
                  'type' => 'promotion',
                  'label' => !empty($label) ? $label : 'Discount',
                  'amount' => $price_scaled['number'],
                  'currency_code' => $currency_code,
                  'percentage' => $commerce_percentage_value,
                  'source_id' => 'custom',
                  'included' => FALSE,
                  'locked' => TRUE,
                ];
              }
            }
          }
        }

      }
      else {
        $parts = explode('|', $value['name'], -1);
cameron prince’s picture

Decided to do a patch after all... This is combined with the tax adjustments patch as they are really dependent upon each other.

I am seeing two different types of discounts in my data. One where discounts are rows in the line_item table and another where the discount is in the price components. This patch handles both for me.

Status: Needs review » Needs work

The last submitted patch, 38: commerce_migrate-commerce_discounts_support-3012110-38.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

cameron prince’s picture

This fixes an issue with the percentage value being stored and also updates the adjustment about to be a proper negative number for discounts.

cameron prince’s picture

The previous patch was using the discounted subtotal and needed pre-discount.

Not sure what's going on with those tests. It could be because of the tax handing this patch includes.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new10.44 KB
new4.31 KB
new1.78 KB
new992.85 KB
new3.48 KB

I've skimmed the recent changes and am beginning by making a bunch of interdiffs.

Then I started working on the duplication of commerce_discount line items which is also seen in the test fixture. In #35 a process plugin is used to determine if discount data already exists in the destination. Not a good idea, the process plugins are not meant to be looking at the destination. This can be done in the source plugin, where prepareRow() can look at the source and then either skip the row in the plugin itself or create a source property that can be examined in the migration yml.

Generally, I prefer having the skip row visible in the migration yml but this just seems to be a case where doing so in the source plugin is sufficient. A query is used to get the max(line_item_id) for all line items of type 'commerce_discount' for this order and to skip the row if the current line_item_id is not the max. The query also compares the data blob, quantity just in case that information is needed later.

Status: Needs review » Needs work

The last submitted patch, 42: 3012110-42.patch, failed testing. View results

quietone’s picture

Issue summary: View changes
quietone’s picture

Issue summary: View changes

Correct the correction I just made!

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new994.16 KB
new2.91 KB

These adjustments are doing my head in. This changes the line item deriver so that a migration for the commerce_discount line items is not created.

Status: Needs review » Needs work

The last submitted patch, 46: 3012110-44.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new993.05 KB
new1.42 KB

OK, went down the wrong path there. Starting again from #42 here and moving the test to skip to the migration yml because then the counts are correct for the order item migrations and thus the order migration will run.

Status: Needs review » Needs work

The last submitted patch, 48: 3012110-48.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new993.12 KB
new3.96 KB

Hopefully fixing the tests

quietone’s picture

StatusFileSize
new991.4 KB
new5.42 KB

As per #36 the discounts don't become line items. This patch should fix that.

Status: Needs review » Needs work

The last submitted patch, 51: 3012110-51.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new992.79 KB
new1.76 KB

Ah, need to add a no_stub to the field plugin for commerce_line_item_reference.

This will not pass all tests as the pricing needs work and entity counts will be of.

Status: Needs review » Needs work

The last submitted patch, 53: 3012110-53.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new995.77 KB
new6.62 KB

Now add in the work done in #37 but put it in the source plugin where all the 'getting' of data should be. Not sure I like what is now in prepareRow but it is at least a start.

Status: Needs review » Needs work

The last submitted patch, 55: 3012110-55.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new997.19 KB
new1.56 KB

That is encouraging, just the source plugin test is failing. This should fix that.

quietone’s picture

StatusFileSize
new997.86 KB
new6.22 KB

Adding comments, coding standard fixes and improve the test for the discount name in the LineItem source plugin.

Assuming tests are green, the next step here is to look at this for BC breaks.

quietone’s picture

StatusFileSize
new37.16 KB

Adding a review only patch, does not contain the fixture, for your reviewing pleasure.

quietone’s picture

There are two migration changes. The shipping rows are skipped because the various order migrations are using the same test fixture in the hopes of reducing the headache of rerolls that involve the fixture. Thus many of the changes here are the same as in #3021057: Migrate tax line items to order item adjustment.

  • The order_item_type migration yml is changed to skip shipping rows. If the previous version is used then the shipping line type will continue to become an order_item_type. That is wrong, but not a change and will not break anything.
  • The order_item migration yml is changed to skip shipping rows, add an process for adjustments and sets uses_legacy_adjustment to false. If the previous versions is used then shipping line items are processed as before and will be incorrect but nothing broken.

The OrderItemDeriver is changed to NOT make a migration for shipping line items or commerce_discount line items. A previously configured one is still around it will still run as the source / process plugins are still usable.

The LineItem source plugin is changed by adding a block of code in prepareRow that will get discounts for this order.

Process plugin changes, none of which should cause previously configured migrations to break.

  • The common process plugin CommerceAdjustments has quite a few changes to validate the data before putting in the adjustment but neither the input format of return value are changed. The only change is that empty $value is not processed. It is used in Ubercart migrations as well and they won't break.
  • The commerce1_migrate_commerce_price process plugin has additional code to handle a unit_price price component. This is an additional change and the input $value and the transformed output remain the same.
  • There is a new process plugin, OrderItemAdjustment, that is only used in the new order_item migration yml.

The field plugin CommerceLineItemReference modifies the process pipeline with a skip_on_empty.

I could be wrong but I don't see any BC issues here.

heddn’s picture

+++ b/modules/commerce/src/Plugin/migrate/process/commerce1/OrderItemAdjustment.php
@@ -0,0 +1,64 @@
+    if (is_array($value)) {

Should we log an error or throw an exception if this happens?

heddn’s picture

No BC complaints.

quietone’s picture

Status: Needs review » Needs work

Good question. It can't be a skip row exception because this also handles other line item adjustments, so maybe logging and error is the way to go.

heddn’s picture

Yeah, let's log something.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new998.62 KB

Logging seems reasonable until I walked through a test and realized that not having adjustments is valid and rather common. So logging them will just be noise. Many of the product lines items in the fixture do not have adjustments. Therefor, this just adds a comment to the process plugin .

The process plugin does not have tests and I don't want to add that here for two reasons. 1) size of patch and 2) want to look at the adjustment process plugins as a whole and simplify them.

Status: Needs review » Needs work

The last submitted patch, 65: 3012110-65.patch, failed testing. View results

quietone’s picture

Made an issue to clean up at the adjustment process plugins, #3027666: Simplify adjustment process plugins

quietone’s picture

StatusFileSize
new998.08 KB

Remove changes to Commerce1TestBase needed locally for #3020704: Move test Traits to a Traits folder and namespace.

quietone’s picture

Status: Needs work » Needs review

The last submitted patch, 2: 3012110-2.patch, failed testing. View results

The last submitted patch, 40: commerce_migrate-commerce_discounts_support-3012110-39.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Status: Needs review » Needs work

The last submitted patch, 68: 3012110-68.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new997.41 KB

I'm all for third time lucky

quietone’s picture

StatusFileSize
new997.96 KB
new753 bytes

Third time was green but still not 100% correct. This should be better and I am now taking a break.

quietone’s picture

StatusFileSize
new997.97 KB
new1.11 KB

Hmm. that is syntax errors in PostgreSQL;

or near &quot;do&quot;
LINE 4: ...6787420field_data_commerce_discount_offer &quot;do&quot; ON do.entity_...

. Taking a punt and changing the tables from do to fdcdo.

Status: Needs review » Needs work

The last submitted patch, 75: 3012110-75.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new930 bytes
new997.97 KB

I missed a table alias. And my local postgres setup isn't running tests either.

quietone’s picture

StatusFileSize
new998.02 KB

Well, this is looking better. The patch failed to apply for sqlite because there was a commit. Here is the reroll.

Status: Needs review » Needs work

The last submitted patch, 78: 3012110-78.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new995.64 KB

Another commit and this needs another reroll.

Status: Needs review » Needs work

The last submitted patch, 80: 3012110-80.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new997.13 KB
new6.09 KB

Change incorrect property name s/source_id/sourceId/ for adjustments.

quietone’s picture

The work to do was from #62 - #65 about adding a log message when the adjustment was an empty array. That can't be done because having no adjustments on a line item is valid.

The following work were rerolls and fixing the use statement for the CartManagerTestTrait because my local is ahead of the version of Commerce used by testbot. We are now back to passing tests so this should be ready to commit.

quietone’s picture

Issue summary: View changes
quietone’s picture

StatusFileSize
new995.22 KB

No needs a reroll

Status: Needs review » Needs work

The last submitted patch, 85: 3012110-85.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new931.87 KB

A reroll.I expect some failing test because this involved a reroll of the fixture

Status: Needs review » Needs work

The last submitted patch, 87: 3012110-87.patch, failed testing. View results

quietone’s picture

Issue summary: View changes
Status: Needs work » Postponed
quietone’s picture

Issue summary: View changes
Status: Postponed » Needs work

No longer postponed, #3028194: Shipping line item has incorrect amount has been committed.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new927.94 KB

Reroll.

Status: Needs review » Needs work

The last submitted patch, 91: 3012110-91.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new930.37 KB

Bad patch.

Status: Needs review » Needs work

The last submitted patch, 93: 3012110-93.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new931.06 KB
new1.36 KB

Adjust entity counts. Not sure I got them all.

Status: Needs review » Needs work

The last submitted patch, 95: 3012110-95.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new930.35 KB
new1.17 KB

That is very odd. Let's start again.

Status: Needs review » Needs work

The last submitted patch, 97: 3012110-97.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new930.94 KB
new1.17 KB

Right, the counts should be the same as the tax line item patch.

Status: Needs review » Needs work

The last submitted patch, 99: 3012110-99.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new931.06 KB
new1.36 KB

This ought to fix the tests.

Status: Needs review » Needs work

The last submitted patch, 101: 3012110-101.patch, failed testing. View results

quietone’s picture

Puzzling the test passes locally.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new930.4 KB
new436 bytes

Fix the source plugin test, I hope.

Status: Needs review » Needs work

The last submitted patch, 104: 3012110-104.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new931.12 KB
new1.22 KB

The ubercart test is failing because core added a new migration providing a migration path for i18nsync.
Adjust the view entity count.

Status: Needs review » Needs work

The last submitted patch, 106: 3012110-106.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new931.07 KB
new554 bytes

Another entity count adjustment.

quietone’s picture

StatusFileSize
new27.44 KB
new930.94 KB

Make a review only patch. Make one small change, s/TRUE/true, in commerce1_order_item.yml.

quietone’s picture

retesting

Status: Needs review » Needs work

The last submitted patch, 109: 3012110-109.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new930.12 KB

No code changes. Review the review-only patch in #109 is fine.

heddn’s picture

Status: Needs review » Needs work
  1. +++ b/modules/commerce/migrations/commerce1_order_item_type.yml
    @@ -7,7 +7,16 @@ migration_tags:
    +      bypass: TRUE
    
    +++ b/modules/commerce/src/Plugin/migrate/field/commerce1/CommerceLineItemReference.php
    @@ -43,9 +43,16 @@ class CommerceLineItemReference extends FieldPluginBase {
    +        'no_stub' => TRUE,
    

    I think true (lowercase) in yaml is more accurate?

  2. +++ b/modules/commerce/src/Plugin/migrate/OrderItemDeriver.php
    @@ -103,34 +103,38 @@ class OrderItemDeriver extends DeriverBase implements ContainerDeriverInterface
    +                $plugin_id = $this->fieldPluginManager->getPluginIdFromFieldType($field_type, ['core' => 7], $migration);
    +                if (!isset($this->fieldPluginCache[$field_type])) {
    +                  $this->fieldPluginCache[$field_type] = $this->fieldPluginManager->createInstance($plugin_id, ['core' => 7], $migration);
    +                }
    +                $this->fieldPluginCache[$field_type]
    +                  ->defineValueProcessPipeline($migration, $field_name, $info);
    +              }
    +              catch (PluginNotFoundException $ex) {
    

    Do we need to handle CCK or not at this point?

  3. +++ b/modules/commerce/src/Plugin/migrate/process/commerce1/OrderItemAdjustment.php
    @@ -0,0 +1,65 @@
    +class OrderItemAdjustment extends ProcessPluginBase {
    ...
    +            $price = new CommercePrice([], 'price', '');
    

    Could we extend CommercePrice and use inheritance instead of constructing a new object? If extended, this could just call parent::transform.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new930.03 KB
new1.77 KB

1. Oops. That is fixed
2. I don't think so. The change record for the change is dated 2016, back in 8.3.2. https://www.drupal.org/node/2751897
3. Yes, that is better. Fixed.

heddn’s picture

Status: Needs review » Reviewed & tested by the community

  • quietone committed 60904d5 on 8.x-2.x
    Issue #3012110 by quietone, cameronprince, heddn: Discount line item to...

  • quietone committed 32187ba on 8.x-2.x
    Revert "Issue #3012110 by quietone, cameronprince, heddn: Discount line...
quietone’s picture

Reverted because tests failed with SQLite. Added test to the latest patch.

quietone’s picture

Status: Reviewed & tested by the community » Needs review
StatusFileSize
new17.11 KB

Needed a reroll and had to add a new process plugin in commerce1 to combine the discount and line adjustments. I'm not happy with this solution and will look at it again.

Status: Needs review » Needs work

The last submitted patch, 119: 3012110-119.patch, failed testing. View results

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new18.8 KB
new14.15 KB

Adjust the order item entity count. And change the assertion of the percentage number to use the formatnumber method so the values being compared are the same format irregardless of the database being used.

quietone’s picture

StatusFileSize
new18.69 KB
new855 bytes

Cleanup the process plugin I didn't like.

quietone’s picture

StatusFileSize
new1.76 KB
new18.13 KB

Ah, migrate_plus has a merge plugin.

quietone’s picture

For all my mucking around the changes since RTBC are minimal:

  • Adjust the entity order_item entity count
  • Use the formatnumber method for testing percentage so that the expected and actual number are in the same format so the comparisons work with PostgreSQL, MySQL and SQLite.
  • Use the merge process plugin to merge two arrays of Adjustments

So, this can be committed.

  • quietone committed 14696cb on 8.x-2.x
    Issue #3012110 by quietone, cameronprince, heddn: Discount line item to...
quietone’s picture

Status: Needs review » Fixed

Thanks heddn and cameronprince!

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.