Problem/Motivation

A developer may need to alter migrate queries that obtain field values.

At present the way to alter this queries for Drupal 6 sources is by subclassing Drupal\node\Plugin\migrate\source\d6\Node. In Drupal 7, the option is going a insane path and subclass any entity migration that extends Drupal\migrate_drupal\Plugin\migrate\source\d7\FieldableEntity.

Proposed resolution

Tag the queries.

Remaining tasks

* Review patch.
* (?) Document the new hooks somewhere.

User interface changes

None.

API changes

New hooks available.

Data model changes

None.

CommentFileSizeAuthor
#2 tag-migrate-field-data-queries-2842939-1.patch2.72 KBjonhattan
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jonhattan created an issue. See original summary.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
CreditAttribution: jonhattan as a volunteer commented
Status: Active » Needs review
FileSize
tag-migrate-field-data-queries-2842939-1.patch2.72 KB
phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Assigned: Unassigned » phenaproxima

Self-assigning for review.

phenaproxima’s picture

phenaproxima
he/him
Primary language English
Location Massachusetts
CreditAttribution: phenaproxima at Acquia commented
Status: Needs review » Needs work
Issue tags: +Needs tests 
  1. +++ b/core/modules/migrate_drupal/src/Plugin/migrate/source/d7/FieldableEntity.php
@@ -50,7 +50,8 @@ protected function getFields($entity_type, $bundle = NULL) {
-    $query = $this->select($table, 't')
+    $query = $this->select($table, 't');
+    $query

    Why did this change?

  2. Overall, I feel like the scope of this patch is too small. It seems like it was written specifically to scratch a particular itch, but not to actually be useful across the entire Migrate system. I think it needs to be fleshed out more, and it will also need tests.

    +++ b/core/modules/migrate_drupal/src/Plugin/migrate/source/d7/FieldableEntity.php
@@ -58,6 +59,12 @@ protected function getFieldValues($entity_type, $field, $entity_id, $revision_id
+    $query->addTag('migrate_field_values');
+    $query->addTag('migrate_field_values_' . $this->migration->id());

    I'm not sure how I feel about these tags. They seem oddly specific. I feel like all Migrate queries should get the 'migrate' tag to begin with, and then additional tags on top of that. 

  3. +++ b/core/modules/node/src/Plugin/migrate/source/d6/Node.php
@@ -282,9 +282,14 @@ protected function getCckData(array $field, Row $node) {
+      $query->addTag('migrate_cckdata');
+      $query->addTag('migrate_cckdata_' . $this->migration->id());

    Same here.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
CreditAttribution: jonhattan as a volunteer commented

1. This change is for IDE happiness. Without it, $query object is cast to ConditionInterface instead of SelectInterface.

Is it better to annotate the variable instead?

2a. I don't think it is specific to a particual itch. It allows to alter a concrete query and enables developers to intervene in the migrations significantly. Some use cases I'm already working with:

  • Limit the number of values to migrate (f.e. limit a giant nodereference field to 5 values)
  • Exclude values of some node types in a nodereference field
  • Add extra information from the source in the field values. Useful for custom processing

2b. I can agree it is small. It could be extended to add tags to all of migrate queries. At present the only tagged migrate query is Drupal\migrate\Plugin\migrate\source\SqlBase.

Should we change the scope of this issue to tag all migrate queries, or to identify which ones are eligible to be tagged?

3. Which tags. The ones I've used seems reasonable to me. They identify the query. Indeed I think it is also useful to add at least migrate_field_values_FIELD_NAME.

Should we add migrate and migrate_MIGRATION_ID to all tagged queries? How much specific we want to be?

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

heddn’s picture

heddn
Primary language English
Location Nicaragua
CreditAttribution: heddn at MTech, LLC, Acro Commerce commented
Assigned: phenaproxima » Unassigned

I tend to agree with the feedback in #4. This can be easily fixed by extending the source plugins. I'm tempted to mark this as won't fix.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.