Add field value migration to Ubercart 7 products

Sounds like this is a duplicate of this issue I’ve been experiencing: https://www.drupal.org/project/commerce_migrate/issues/3044987

Yes, very similar. For now, lets have this one address the taxonomy problem, the the terms are not selected.

For this issue I've been trying to find out why the field values are not populated. I started out adding some more fields and updating the tests, which of course fail. This failure is strange is unexpected as field values are all done by core and should just work. Obviously that is not the case. I've just tracked it down to the FieldDiscovery. When the migration is for a product node type the field process is not added to the migration where as it works for a 'plain' node type.

There is no proof that this worked before the newish FieldDiscovery service as the tests did not test the field values for additional fields on the product.

Right now looking at FieldDiscovery::getAllFields() which is assigning the product node fields to the commerce product type. That shouldn't be at this stage of the migration.

The patch has an updated ProductTest and new fixture.

This has probably always failed. It is in the PrepareRow event that the source entity type for product nodes is changed from 'node' to 'commerce_product'. This was done to allow later migration_lookups to work correctly. It has the side effect of prevent field values from migrating. This isn't easy to fix.

The first thing to do is to make sure the docs are updated with this information.

The next thing is to sleep on it for a while.

Brief comment added to the docs,

Think I have a working patch now. What I did was change the way the entity type is determined for fields, d7_field.yml. Previously, a hack was made where the source entity type was changed to 'commerce_product' for the fields on product nodes. So, instead of mucking with the source entity_type (which shouldn't be changed) this now just sets a new property on the row to indicate it is a commerce_product type and then a new process plugin sets the correct entity type.

Need to apply similar changes to UC6 and also removed the bulk of the unnecessary fixture changes.

Still working on the fixture

That is looking better. Now fix the failing test. that one changes because the source plugin has changed. And remove uc7_entity_type.

Great, a working patch. This is ready for review.

Anyone able to do some manual testing? From either Ubercart 6 or 7 is helpful.

This was caused because the PrepareRow event was changing the entity_type which then meant the deriver was not able to build the process pipeline correctly.

I was just testing this and was wondering if its only compatible with 8.7 as on 8.6 I'm getting 'You have requested a non-existent service "migrate_drupal.field_discovery".' when running a migration.

If it is only compatible with 8.7 should there be a warning in the module?

If it is only compatible with 8.7 should there be a warning in the module?

Sorry about that, that should have been noted on the project page. It is now.

Cross-referencing issues.

Ran into this myself.

Working on a reroll.

Rerolled.

Streamlining related issues.

This is an Ubercart issue, not a Commerce v1 issue.

Closed #3044987: Ubercart 7 product field data not migrated as a duplicate of this and moving credit here.

Can anyone test this patch and confirm it works and doesn't cause any disruption to existing migrations?

The uc_entity_type process plugin is only used by Ubercart 7 so move it to a new d7 directory.

Fix namespace error.

Closed #3083310: Migrate from UC7 to DC8 fails because of missing fields? as a duplicate of this.

Bumping to major because there is no workaround.

Closed #3156915: Ubercart 3 (D7) items do not migrate as a duplicate, based on 3156915#6

Shoot, wish I saw this patch earlier. I've modified the module pretty heavily at this point and have most everything working that I need for the migration. I think bobburns finished up their migration with the other patch and what sounds like a lot of manual entry/cleanup.

I do have a second test environment with a fresh clone of this module and applied this patch to run a test migration there. I should be able to report back tomorrow if nothing goes terribly wrong with the migration. Then I can at least see if all the field values are migrated with this patch.

Update: this is working! Different product types migrated text fields, image field, entity reference (taxonomy field) for catalog with correct terms selected, list fields.

@Nathaniel, thanks for the confirmation!

I reviewed the patch considering BC and it all looks fine to me except for the source plugin uc7/Field.php. In particular, the getIds() method which adds a third ID, 'commerce_product'. This will be incompatible with any existing migrate_map table that uses this source plugin, which is at least d7_field.yml. When attempting to write a row it will fail.

+++ b/modules/ubercart/src/Plugin/migrate/source/uc7/Field.php
@@ -18,7 +16,7 @@ use Drupal\migrate_drupal\Plugin\migrate\source\DrupalSqlBase;
+class Field extends D7Field {

@@ -77,89 +80,26 @@ class Field extends DrupalSqlBase {
+    return $ids + [
+      // Add a flag to indicate a commerce product node. This third source id
+      // is not required for existing migration_lookups on d7_field to work
+      // correctly.
+      'commerce_product' => [
+        'type' => 'integer',
       ],

If one is in the process of migrating one would have to rollback the field migrations, delete the table migrate_map_d7_field, then do the field migrations again.

Any other solutions?

This implements the suggestion from benjifisher from the migrate meeting, Item #9

Ask the plugin manager for a list of migration plugins that use this source plugin. Then update the map tables. https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Extension...

Has anyone tested/reviewed the update function added in #44? I am setting the status back to NR and updating the remaining tasks in the issue summary.

@benjifisher, thanks. Yes, the update function needs review.

So, to review the update function I reckon something like this will work.

• Start with a minimal install
• Either run a migration with an Ubercart 7 source from the UI or use drush to run d7_field, drsuh mim d7_field and any custom migration that you have that uses the d7_field source plugin
• Check the columns in the migrate_map tables for migrations using the d7_field source plugin. That should be, at least, migrate_map_d7_field', which should have two source id columns.
• Apply this patch
• Run the db update, `drush updb`
• Rerun the migration
• Check the columns in the migrate_map tables for migrations using the d7_field source plugin. Each map table should have one new source id column. So, migrate_map_d7_field', should have 3 source id columns.

Adding another check in the update function so that the field is only added if it does not exist. The function is now is indented with three if statements and I considered changing it to reduce the level of indentation but it is such a small function I reckon it is fine as it is.

I ran into a couple issues testing this one. I tried the steps in #47.

1. It looks like we need the class name here?
$class_name = $source->getPluginDefinition()['class'];

+++ b/modules/ubercart/commerce_migrate_ubercart.module
@@ -54,6 +54,40 @@ function commerce_migrate_ubercart_help($route_name, RouteMatchInterface $route_
   }
 }
 
+
+/**
+ * Implements hook_update_N().
+ */
+function commerce_migrate_ubercart_update_8201(&$sandbox) {
+  $migrations = \Drupal::service('plugin.manager.migration')->createInstances([]);
+  if (empty($migrations)) {
+    return;
+  }
+
+  // Ensure that any migration using the uc7\Field source plugin has a column
+  // for the commerce_product flag.
+  // See https://www.drupal.org/project/commerce_migrate/issues/3052488.
+  $schema = \Drupal::database()->schema();
+  foreach ($migrations as $migration) {
+    $source = $migration->getSourcePlugin();
+    if (Utility::classInArray($source, [\Drupal\commerce_migrate_ubercart\Plugin\migrate\source\uc7\Field::class])) {

2. $table_name = $id_map->getQualifiedMapTableName(); returns prefixed database.table name and that does't seem to work with schema.

After hacking that the sourceid3 field did appear and re-running the migration populated that field.

Then rolling back product nodes and re-running populated the fields.

1. The Utility method works fine when an object is passed.
2. Yes, that way the wrong way to get the table name. That is fixed in this patch.

I also added a test of the update hook but it is failing with "Migration Group: The Migration Group entity type needs to be installed." I've yet to find a way to fix that.

Fix coding standards.

I found a way to install the migrate_plus entities.

The update hook now has a test and is passing.

Anyone want to test that the hook works now?

Yes, this is working as expected now. Ran a migration with the latest dev version. Applied this patch. Update.php installs the new field sourceid3. Running d7_field populates that field. Then product nodes migrate the field data.

@Nathaniel, thank you.