Problem/Motivation
Migrate the discount line items to an order item adjustment
Asked on Commerce Slack about adjustments and bojanz confirmed that promotions are adjustments on the order item.
quietone [9:11 AM]
Working to understand adjustments. From my testing I see that shipping adjustments are always on the order and promotion adjustments are always on the order item. Taxes can be on both. Is that correct?
bojanz 2 hours ago
They are always on the order item since 2.7/2.8 causr VAT
Add clarification that bojanz is referring to promotions.
Proposed resolution
Get all the discount on the order and for each one get the discount data and then build an adjustment array. Use the total price as the value. The lock is not explicitly set here but will default to TRUE in commerce_adjustments process plugin.
Remaining tasks
Commit
| Comment | File | Size | Author |
|---|---|---|---|
| #123 | 3012110-123.patch | 18.13 KB | quietone |
| #123 | interdiff-122-123.txt | 1.76 KB | quietone |
| #122 | interdiff-121-122.txt | 855 bytes | quietone |
| #122 | 3012110-122.patch | 18.69 KB | quietone |
| #121 | interdiff-119-121.txt | 14.15 KB | quietone |
Comments
Comment #2
quietone commentedThis is just a working patch and does not have the test fixture it requires because It is a mess and it is too late to be editing that beast now. So, this is just the code to migrate discounts to adjustments. Think of it as a start.
Comment #3
quietone commentedAdd in the fixture changes and see what breaks.
Comment #5
quietone commentedRemove test files that are not part of this patch.
Add empty adjustment array to ubercart 6 and 7 order item test.
Comment #7
quietone commentedYes,there are not promotion migrated in this patch
Comment #8
quietone commentedAre there other line item adjustments that need to be handled?
Should this use unit price or total price?
Probably TRUE?
Add new line
Oops, This method isn't part of this patch. remove it
Comment #9
quietone commentedFixes for 3,4,5 and some coding standard fixes.
Items 1,2 have been moved to questions in the IS
Comment #11
quietone commentedThat was silly, made changes in the process plugin and didn't update the test.
Comment #12
quietone commentedBeen down a bit of a rabbit hole with adjustments. This has expanded to include taxes and uses the same fixture as the order adjustment issue.
Using interdiff to make the interdiff fails and this is very nearly a rewrite anyway so no interdiff. Not sure it all tests will pass.
Comment #14
quietone commentedRemoving taxes (will do those in a new issue) and fixing tests.
And next is to find out why the bag discount is no longer in the orderitemtest.
Comment #16
quietone commentedFix the ubercart tests
Comment #17
quietone commentedThis adds a product discount and a new test fixture. The interdiff does not include the test fixture changes.
Comment #19
quietone commentedAdded some logs when adding the order.
Comment #21
quietone commentedYes, 2 order items were added.
Comment #23
quietone commentedAdd added an order
Comment #25
quietone commentedAnd finally the entity form display.
Comment #26
quietone commentedTo review this the key changes are in the commerce sub module. Ignore the changes to the fixture and in the Ubercart sub module (modified because the test method changed).
I am tired of removing stuff from the fixture and am very likely to commit the whole thing. It takes too much time and effort to trim it down.
Comment #27
quietone commentedAnd for easier review here is the patch from #25 without the fixture noise.
Comment #28
heddnThe title is discounts. But there are fees that are not shipping. Comments or clarifications on what we are doing would help make this more clear.
It might make more sense to use's Drupal\commerce_price\Calculator.
should we link to an issue here?
Yes, fraction digits is important.
There's no adjustments in here. Why?
I don't see any non-shipping adjustments. Am I reading this wrong?
Comment #29
quietone commentedNeeds a reroll
Comment #30
quietone commentedInterdiff for the reroll fails.
Comment #31
quietone commented1. Comment added.
2. TODO
3. TODO
4. Reworked to use existing process plugin to scale price.
5. The order adjustments, which seem to be only shipping, are being done in #2978105: Migrating shipping order adjustments
6. You are reading it correctly. And that shouldn't be there those changes to the source plugin test are no longer valid.
Having trouble running tests locally, so not sure if this will pass.
Comment #32
quietone commented28.3 Created an issue for discount percentage #3020862: Add percentage to discount order item adjustment.
Needs work for #28.2
Comment #33
quietone commentedAnd finally 28.2. Reworked the price plugins and use Calculator.
Comment #34
quietone commentedComment #35
cameron prince commentedI've been testing this today and was a little surprised to find that the D7 commerce_line_item table contains multiple rows for the same discount on the same order. My guess is that a row was being added each time the "Apply" button was pressed or upon transition from pane to pane...
Not sure if others will have this problem or if this is the best way to fix it, but I wrote a process plugin which does this to resolve it.
Comment #36
cameron prince commentedIn thinking about this further, why do we need to create commerce_order_item rows for discounts? When I place a new order with a discount, no row is created like this in commerce_order_item. Could we just create the adjustment in a process plugin and skip all commerce_discount rows?
Comment #37
cameron prince commentedHere's what I came up with... It works, but I'm hesitant to create a patch without further discussion. This is only going to work for order discounts (not line item discounts) and I'm not sure if it's necessary for everyone to have the check for duplicate discount rows I'm seeing. Notice I'm popping off the last discount row, not matter how many are found.
The percentage look-up is here, but it's not covering all use cases.
This is added to the OrderItemAdjustment.php process plugin and commerce_discount type line item rows are excluded from the normal line item source, i.e. commerce_discount is not in the static map on the type field.
This prevents the unnecessary discount rows in the order_item table.
Comment #38
cameron prince commentedDecided to do a patch after all... This is combined with the tax adjustments patch as they are really dependent upon each other.
I am seeing two different types of discounts in my data. One where discounts are rows in the line_item table and another where the discount is in the price components. This patch handles both for me.
Comment #40
cameron prince commentedThis fixes an issue with the percentage value being stored and also updates the adjustment about to be a proper negative number for discounts.
Comment #41
cameron prince commentedThe previous patch was using the discounted subtotal and needed pre-discount.
Not sure what's going on with those tests. It could be because of the tax handing this patch includes.
Comment #42
quietone commentedI've skimmed the recent changes and am beginning by making a bunch of interdiffs.
Then I started working on the duplication of commerce_discount line items which is also seen in the test fixture. In #35 a process plugin is used to determine if discount data already exists in the destination. Not a good idea, the process plugins are not meant to be looking at the destination. This can be done in the source plugin, where prepareRow() can look at the source and then either skip the row in the plugin itself or create a source property that can be examined in the migration yml.
Generally, I prefer having the skip row visible in the migration yml but this just seems to be a case where doing so in the source plugin is sufficient. A query is used to get the max(line_item_id) for all line items of type 'commerce_discount' for this order and to skip the row if the current line_item_id is not the max. The query also compares the data blob, quantity just in case that information is needed later.
Comment #44
quietone commentedComment #45
quietone commentedCorrect the correction I just made!
Comment #46
quietone commentedThese adjustments are doing my head in. This changes the line item deriver so that a migration for the commerce_discount line items is not created.
Comment #48
quietone commentedOK, went down the wrong path there. Starting again from #42 here and moving the test to skip to the migration yml because then the counts are correct for the order item migrations and thus the order migration will run.
Comment #50
quietone commentedHopefully fixing the tests
Comment #51
quietone commentedAs per #36 the discounts don't become line items. This patch should fix that.
Comment #53
quietone commentedAh, need to add a no_stub to the field plugin for commerce_line_item_reference.
This will not pass all tests as the pricing needs work and entity counts will be of.
Comment #55
quietone commentedNow add in the work done in #37 but put it in the source plugin where all the 'getting' of data should be. Not sure I like what is now in prepareRow but it is at least a start.
Comment #57
quietone commentedThat is encouraging, just the source plugin test is failing. This should fix that.
Comment #58
quietone commentedAdding comments, coding standard fixes and improve the test for the discount name in the LineItem source plugin.
Assuming tests are green, the next step here is to look at this for BC breaks.
Comment #59
quietone commentedAdding a review only patch, does not contain the fixture, for your reviewing pleasure.
Comment #60
quietone commentedThere are two migration changes. The shipping rows are skipped because the various order migrations are using the same test fixture in the hopes of reducing the headache of rerolls that involve the fixture. Thus many of the changes here are the same as in #3021057: Migrate tax line items to order item adjustment.
The OrderItemDeriver is changed to NOT make a migration for shipping line items or commerce_discount line items. A previously configured one is still around it will still run as the source / process plugins are still usable.
The LineItem source plugin is changed by adding a block of code in prepareRow that will get discounts for this order.
Process plugin changes, none of which should cause previously configured migrations to break.
The field plugin CommerceLineItemReference modifies the process pipeline with a skip_on_empty.
I could be wrong but I don't see any BC issues here.
Comment #61
heddnShould we log an error or throw an exception if this happens?
Comment #62
heddnNo BC complaints.
Comment #63
quietone commentedGood question. It can't be a skip row exception because this also handles other line item adjustments, so maybe logging and error is the way to go.
Comment #64
heddnYeah, let's log something.
Comment #65
quietone commentedLogging seems reasonable until I walked through a test and realized that not having adjustments is valid and rather common. So logging them will just be noise. Many of the product lines items in the fixture do not have adjustments. Therefor, this just adds a comment to the process plugin .
The process plugin does not have tests and I don't want to add that here for two reasons. 1) size of patch and 2) want to look at the adjustment process plugins as a whole and simplify them.
Comment #67
quietone commentedMade an issue to clean up at the adjustment process plugins, #3027666: Simplify adjustment process plugins
Comment #68
quietone commentedRemove changes to Commerce1TestBase needed locally for #3020704: Move test Traits to a Traits folder and namespace.
Comment #69
quietone commentedComment #73
quietone commentedI'm all for third time lucky
Comment #74
quietone commentedThird time was green but still not 100% correct. This should be better and I am now taking a break.
Comment #75
quietone commentedHmm. that is syntax errors in PostgreSQL;
. Taking a punt and changing the tables from do to fdcdo.
Comment #77
quietone commentedI missed a table alias. And my local postgres setup isn't running tests either.
Comment #78
quietone commentedWell, this is looking better. The patch failed to apply for sqlite because there was a commit. Here is the reroll.
Comment #80
quietone commentedAnother commit and this needs another reroll.
Comment #82
quietone commentedChange incorrect property name s/source_id/sourceId/ for adjustments.
Comment #83
quietone commentedThe work to do was from #62 - #65 about adding a log message when the adjustment was an empty array. That can't be done because having no adjustments on a line item is valid.
The following work were rerolls and fixing the use statement for the CartManagerTestTrait because my local is ahead of the version of Commerce used by testbot. We are now back to passing tests so this should be ready to commit.
Comment #84
quietone commentedComment #85
quietone commentedNo needs a reroll
Comment #87
quietone commentedA reroll.I expect some failing test because this involved a reroll of the fixture
Comment #89
quietone commentedComment #90
quietone commentedNo longer postponed, #3028194: Shipping line item has incorrect amount has been committed.
Comment #91
quietone commentedReroll.
Comment #93
quietone commentedBad patch.
Comment #95
quietone commentedAdjust entity counts. Not sure I got them all.
Comment #97
quietone commentedThat is very odd. Let's start again.
Comment #99
quietone commentedRight, the counts should be the same as the tax line item patch.
Comment #101
quietone commentedThis ought to fix the tests.
Comment #103
quietone commentedPuzzling the test passes locally.
Comment #104
quietone commentedFix the source plugin test, I hope.
Comment #106
quietone commentedThe ubercart test is failing because core added a new migration providing a migration path for i18nsync.
Adjust the view entity count.
Comment #108
quietone commentedAnother entity count adjustment.
Comment #109
quietone commentedMake a review only patch. Make one small change, s/TRUE/true, in commerce1_order_item.yml.
Comment #110
quietone commentedretesting
Comment #112
quietone commentedNo code changes. Review the review-only patch in #109 is fine.
Comment #113
heddnI think true (lowercase) in yaml is more accurate?
Do we need to handle CCK or not at this point?
Could we extend CommercePrice and use inheritance instead of constructing a new object? If extended, this could just call parent::transform.
Comment #114
quietone commented1. Oops. That is fixed
2. I don't think so. The change record for the change is dated 2016, back in 8.3.2. https://www.drupal.org/node/2751897
3. Yes, that is better. Fixed.
Comment #115
heddnComment #118
quietone commentedReverted because tests failed with SQLite. Added test to the latest patch.
Comment #119
quietone commentedNeeded a reroll and had to add a new process plugin in commerce1 to combine the discount and line adjustments. I'm not happy with this solution and will look at it again.
Comment #121
quietone commentedAdjust the order item entity count. And change the assertion of the percentage number to use the formatnumber method so the values being compared are the same format irregardless of the database being used.
Comment #122
quietone commentedCleanup the process plugin I didn't like.
Comment #123
quietone commentedAh, migrate_plus has a merge plugin.
Comment #124
quietone commentedFor all my mucking around the changes since RTBC are minimal:
So, this can be committed.
Comment #126
quietone commentedThanks heddn and cameronprince!