Problem/Motivation
Presently the node source in a migration can only support one node type.
\Drupal\node\Plugin\migrate\source\d6\Node
\Drupal\node\Plugin\migrate\source\d7\Node
\Drupal\node\Plugin\migrate\source\d7\NodeEntityTranslation
It happens that one can need multiple node types in one migration (like a node alias migration).
This is already supported by taxonomy source plugins:
https://git.drupalcode.org/project/drupal/-/blob/9.2.x/core/modules/taxo...
Proposed resolution
Follow the taxonomy term source plugin as an example and change
$query->condition('n.type', $this->configuration['node_type']);
to
$query->condition('n.type', (array) $this->configuration['node_type'], 'IN');
This was discussed by the migrate maintainer and there were some concerns about adding this feature to core (rather than migrate plus) since this doesn't directly help core's built-in migrations. But it does help contrib and definitely custom migrations. It is a really trivial change and will make filtering things from legacy sites just that much easier. See #6
Comment | File | Size | Author |
---|---|---|---|
#34 | 2937989-9.2.x-typo.txt | 1.6 KB | Matroskeen |
#5 | interdiff-2-5.txt | 5.34 KB | quietone |
#5 | 2937989-5.patch | 6.59 KB | quietone |
Issue fork drupal-2937989
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 2937989-node-source-multiple-bundles changes, plain diff MR !77
Comments
Comment #2
jcisio CreditAttribution: jcisio at Axess Open Web Services commentedFor me, $query->condition should default the operator to different value based on the $value argument, but it's another history. Now just fix this in the migration system.
Comment #3
heddnCan we add a test?
Comment #5
quietone CreditAttribution: quietone as a volunteer commentedAdded more tests to the source plugin. Tests getting 1 content type, 2 content types and all content types if the configuration key 'node_type' is not set.
Comment #6
heddnDiscussed this migrate maintainers calls. Code looks fine. We had some concerns about adding this feature to core (rather than migrate plus) since this doesn't directly help core's built-in migrations. But it does help contrib and definitely custom migrations. It is a really trivial change and will make filtering things from legacy sites just that much easier.
Comment #8
alexpottThis can be written with less conditions - which is always a bit simpler.
$query->condition('n.type', (array) $this->configuration['node_type'], 'IN');
Going to commit this anyway this the utility is more important but someone can feel free to file a follow-up issue if they like.Ah.... just spotted something else this can be addressed here.Only adding test coverage to d6 is probably not enough - what happens if we move d6 out of core?
Comment #15
MatroskeenI recently discovered this limitation and will be happy to move this issue forward.
Here is the merge request with 3 commits:
1) Contains test only from #5;
2) Contains a similar test for D7 plugin per suggestion in #8;
3) Contains the actual fix for all node source plugins including
NodeEntityTranslation.php
.I didn't find a way to run tests for specific commit in GitLab, so I'm also attaching patches (tests only + tests with fix). The first one is expected to fail. The second one is equal to the merge request.
I also updated the issue summary according to my report in duplicated issue: #3184835: Allow multiple bundles in node source plugins configuration.
Comment #17
quietone CreditAttribution: quietone as a volunteer commented@Matroskeen, thanks. I may have noticed that MRs are tested. Have you seen, https://www.drupal.org/docs/develop/git/using-git-to-contribute-to-drupa...
I didn't review the tests yet but I did notice that NodeEntityTranslationTest needs to be updated to includes different configurations. Setting to NW and adding tag for this.
Comment #18
MatroskeenAdded one more dataset to NodeEntityTranslationTest with multiple content type configuration.
Moving back to NR.
Comment #19
quietone CreditAttribution: quietone as a volunteer commentedJust some minor tidying up to do.
Comment #20
MatroskeenComment #21
quietone CreditAttribution: quietone as a volunteer commented@Matroskeen, thanks.
Comment #22
MatroskeenThanks for the review. MR was updated.
Comment #23
quietone CreditAttribution: quietone as a volunteer commented@Matroskeen, thanks for the quick reply.
I applied the latest patch and read the NodeEntityTranslationTest. The extra addition provides proof that the configuration value, node_type, works as expected. Nice!
All the points from the review have been addressed and there are no coding standard issues.
So RTBC!
Comment #26
quietone CreditAttribution: quietone as a volunteer commentedTo stop messages about the patch in #15 failing I have made a patch from the changes in Merge request !77 and uploaded it here.
Comment #27
alexpottI think we should document this. Not sure how we document migrate source configuration options.
Comment #28
quietone CreditAttribution: quietone as a volunteer commentedUnfortunately, the source plugin configurations not documented. I have added it here using the style of the migrate process plugins in the migrate module as a guide.
Comment #29
Matroskeen@quitone, thanks!
I applied your patch to open MR and added one more commit with more examples: https://git.drupalcode.org/project/drupal/-/merge_requests/77/diffs?comm...
Comment #30
alexpottThanks for adding the docs - this looks nice. Given it was rtbc in #23 and only docs have been added I'm re-instating that rtbc.
Comment #31
alexpottCommitted 4a30675 and pushed to 9.2.x. Thanks!
Comment #33
MatroskeenPushed one more commit to fix the typo 🙄
Comment #34
Matroskeen@alexpott, thanks for committing the patch to 9.2.x. Can we also fix the typo mention in this note? (please apply the patch 2937989-9.2.x-typo.txt)
Can we also cherry-pick this commit to 9.1.x? Here is a link to the patch if it makes it easier: https://git.drupalcode.org/project/drupal/-/merge_requests/77.patch
Comment #36
alexpott@Matroskeen new features do not get backported to 9.1.x. And please open a new issue to fix the typo. Re-opening fixed issues for typos makes issue / commit history more difficult than necessary.
Comment #38
Ghost of Drupal PastNote this was supported originally and only broke because of #2358079: Require a specific placeholder format in db_query() in order to trigger argument expansion, and require explicit 'IN' parameter for conditions.