Problem/Motivation

In D7, there is an option to auto create a term reference record on the content migration if the term value doesn't exist. This feature parity doesn't exist (yet) in D8.

Proposed resolution

Build it. Do it.

It should have a mechanism to compare on creation if it needs to create the term based on a case-sensitive or insensitive query.

Remaining tasks

Write code.

User interface changes

n/a

API changes

Probably

Data model changes

n/a

Comments

heddn created an issue. See original summary.

heddn’s picture

Possibly we could use something akin to https://www.drupal.org/node/2181783, but a little more specific for term entity reference fields.

heddn’s picture

Issue summary: View changes
heddn’s picture

This is simply a WIP. Marking postponed, since we have two dependencies that need to land first.

heddn’s picture

#2633428: Add a Case Sensitive Condition to Entity Query is no longer a dependency. D7 didn't do the case comparison in a SQL query, probably because it isn't possible there either. We'll have to retrieve results, then do the comparison in PHP. The getter issue is also seeing good progress.

heddn’s picture

Title: Term entity reference fields do not auto create a record » Term entity reference migration do not auto create a record from content migration
jordanpagewhite’s picture

Assigned: Unassigned » jordanpagewhite
jordanpagewhite’s picture

Assigned: jordanpagewhite » Unassigned
ada hernandez’s picture

Project: Drupal core » Migrate Plus
Version: 8.0.x-dev » 8.x-1.x-dev
Component: migration system » Code
Status: Postponed » Needs review
StatusFileSize
new4.1 KB

This patch auto create a term reference record on the content migration if the term value doesn't exist.

heddn’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/migrate/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,139 @@
    +  /** @var \Drupal\Core\Entity\EntityStorageInterface */
    +  protected $migrationStorage;
    ...
    +    $this->migrationStorage = $entity_manager->getStorage('migration');
    

    No longer necessary.

  2. +++ b/core/modules/migrate/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,139 @@
    +    $bundle = $this->getBundle();
    +    $config_target = $this->entityManager->getStorage('field_config')->load($entity_type . '.' . $bundle . '.' . $destination_property);
    

    No longer necessary.

  3. +++ b/core/modules/migrate/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,139 @@
    +    $vid = reset($config_target->getSetting('handler_settings')['target_bundles']);
    

    We should provide the vid from configuration instead. And throw an exception if it doesn't provide one.

  4. +++ b/core/modules/migrate/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,139 @@
    +  protected function getBundle() {
    +    return 'foo';
    +  }
    

    This is unnecessary.

ada hernandez’s picture

Status: Needs work » Needs review
StatusFileSize
new3.94 KB

Sorry, If this it's ok.

heddn’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests
  1. +++ b/modules/contrib/migrate_plus/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,123 @@
    +    $vid = $this->configuration['vid'];
    +
    ...
    +    if (!isset($this->configuration['vid'])) {
    

    This isset check should happen before we grab the value from configuration. In case it isn't set...

  2. +++ b/modules/contrib/migrate_plus/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,123 @@
    +    //Verify if vocabulary id is not empty.
    

    Nit: spaces between // and the sentence.

  3. +++ b/modules/contrib/migrate_plus/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,123 @@
    +    //Verify if vocabulary is exist. .
    

    Nit: Only a single period is necessary.

  4. +++ b/modules/contrib/migrate_plus/src/Plugin/migrate/process/TermReference.php
    @@ -0,0 +1,123 @@
    +
     if (is_null(Vocabulary::load($vid))) {
    

    Nit: this can just be:

    if (!Vocabulary::load($vid)) {

And I guess we should create some unit tests for this.

mikeryan’s picture

Status: Needs work » Postponed (maintainer needs more info)

Doesn't #2642612: Create Entity Lookup & Generate process Base plugins solve this in a more general way?

heddn’s picture

re #13, it does solve part of the problem. But it doesn't create a stub.

mikeryan’s picture

Title: Term entity reference migration do not auto create a record from content migration » Process plugin for importing/creating terms by name
Status: Postponed (maintainer needs more info) » Active

By "stub" you meant implicitly creating a term if none exists, of course (the word "stub" confused me for a minute since the patch does nothing with stubs).

I think the work in that issue could be leveraged here - at the very least derive from that class and let it do the lookup. I wonder to what extent the target entity creation could be generalized beyond terms?

heddn’s picture

Yes, I'm all for offloading the search to the other class. But then we'd want to make as few assumptions there as possible. I noticed you mentioned we'd want bundle, but we really don't need that for terms reference. We just need the destination vid. But making a default_value, "required" seems a little strong-handed. There's already a default value plugin. Use it instead.

mikeryan’s picture

For taxonomy terms, the vid is the bundle.

ada hernandez’s picture

StatusFileSize
new7.38 KB

We've changed to abstract base class that does only query, and just in the plugin term_reference create the term if this don't exist.

A example of process.

field_topic:
    plugin: term_reference
    source: topic
    ignore_case: true
    bundle:
      - vid : topics
    value_type: name
esclapes’s picture

I have tried patch in #10. Applies clean and works as expected.

+++ b/modules/contrib/migrate_plus/src/Plugin/migrate/process/TermReference.php
@@ -0,0 +1,105 @@
+    if (!$this->configuration['entity_type']) {
+      $this->configuration['entity_type'] = 'taxonomy_term';
+    }

Raises a warning when entity_type is not set. Could use empty($this->configuration['entity_type']) instead.

Thanks!

penyaskito’s picture

Shouldn't this be solved by dependencies instead? If the taxonomy migration runs before the reference, this shouldn't be a problem.

heddn’s picture

re: #20, that assumes this is a d2d migration where there is already a separate term migration. But in D7 contrib migrate and in my case on D8, I'm migrating in from CSV. There's 4 taxonomy reference fields on the node. I don't want to create 4 additional migrations. Ideally, I can read over the CSV once and insert all the term stubs at the same time as I'm creating the node. From an I/O perspective, CSV isn't all that performant. So reducing the number of times I have to process the same text file significantly cuts down on processing time.

esclapes’s picture

Shouldn't this be solved by dependencies instead? If the taxonomy migration runs before the reference, this shouldn't be a problem.

Not sure, but I guess for that you would need a normalized source of terms and references.

In my use case I source from a csv where terms are just strings in a column for each article. Being able to do a lookup and create the term when it does not exists is very handy.

ada hernandez’s picture

StatusFileSize
new7.53 KB

Thanks for your comments,
#19. I did your suggestion.

ada hernandez’s picture

StatusFileSize
new4.91 KB
heddn’s picture

Status: Active » Needs review
StatusFileSize
new2.73 KB
mikeryan’s picture

As I commented at https://www.drupal.org/node/2642612#comment-10812080 - why have a model requiring a separate process plugin for every entity type?

heddn’s picture

Status: Needs review » Closed (duplicate)

This won't be necessary after #2642612: Create Entity Lookup & Generate process Base plugins is completed.

brooke_heaton’s picture

Component: Code » API

If you came here looking for a working example but found a myriad of confusing historical comments, you need to do this (assuming that you are using a taxonomy reference field and that the vocabulary is 'category' for example).

process:
  type:
    plugin: default_value
    default_value: node_type
  field_category:
    plugin: entity_generate
    source: category
    entity_type: taxonomy_term
    bundle: category
    property: name

If you are missing any of the following (plugin, source, entity_type, bundle, property), the import will fail.