I can import the "types" field from the JSON file as tags, as defined in this migrate_plus.migration.tags.yml file:
# Migration configuration for article tags.
id: migrate_article_tags
label: Migrate article
migration_group: Migrate articles
source:
plugin: url
data_fetcher_plugin: http
data_parser_plugin: json
urls: http://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson
item_selector: features
fields:
-
name: machine_name
label: 'Unique position identifier'
selector: id
-
name: types
label: 'Types'
selector: 'properties/types'
ids:
machine_name:
type: string
destination:
plugin: 'entity:taxonomy_term'
process:
vid:
plugin: default_value
default_value: tags
name: types
migration_dependencies: {}
They are imported as for example ",nearby-cities,origin,phase-data,scitech-link,", but if I try to separate the items, by adding the explode plugin to the process (see below) the migration fails silently, with just (0 created, 0 updated, 1 failed, 0 ignored) for each import:
process:
vid:
# plugin: default_value
plugin: explode
default_value: tags
limit: 100
delimiter: ,
name: types
What am I doing wrong?
Comments
Comment #2
mikeryanFirst, let me explain what your current .yml is doing:
So, it would be tempting to do
This demonstrates proper usage of the explode plugin. Yet, that isn't going to work in this particular case. It's going to generate an array - a list of term names - but the taxonomy term "name" field is a single-value string, not an array. What you want to do here is create multiple terms, but it's inherent in how migration work that each source row must correspond to a single destination object - i.e., each pass through the processing pipeline will only create one item of the destination type.
There are two things you can do to get your "types" into the tags vocabulary:
entity_generate can be used like this:
The first step of the field_tags processing will explode the types into an array of simple strings - each one will then be passed to entity_generate, which will see if a tag of that name already exists. If it does, it will return a reference to the existing term - if not, it will create the term, then return the reference.
Now, I foresee a little problem - with those leading and trailing commas, explode is going to give you empty strings, and you'll end up with a term with a zero-length name which all articles reference. Off the top of my head I'm not sure of a way to clean that up purely through YAML - it may be simplest to just let the migration create it and manually delete that term after the fact.
One more note - when your migration reports failures, you can see the detailed messages with
To see them while running your migration, add -d to your drush mi command.
Comment #3
ressaThank you @mikeryan for the very detailed write up on how to migrate taxonomy terms, and also a big Thank you! for all your work on the Migrate modules. The information posted here will be very useful to me, as well as to many others, I am certain.
A follow up question: If I use option #2 and generate tags while migrating my articles with the entity_generate plugin, is there a way of including tags when doing a rollback of my article import? Or is it only possible to empty the Tags vocabulary of terms programmatically?
Comment #4
mikeryanAfraid that's the downside of entity_generate - because the referenced entities are generated as a side-effect rather than their own migration, they aren't tracked in a map table and thus can't be rolled back via drush migrate-rollback, you'd need a custom process to clean them out (e.g., something that deletes tags which have no entity references pointing at them).
Comment #5
ressaOkay, I'll see if I can find a way to just wipe all Tags programmatically and post it here if I do.
EDIT: I found this snippet, which will erase all terms from a specific vocabulary, where "tags" is the name of the vocabulary:
From: http://drupal.stackexchange.com/questions/213256/how-do-i-delete-a-vocab...
Comment #6
mikeryanComment #8
leisurman commentedThank you!!!!!!!
I was able to pull in a csv file with multiple terms like this
My csv file is:
Comment #9
leisurman commentedI did not know you can use 2 plugin for one field, is that why you are using the 2 dashes?
Comment #10
leisurman commentedI noticed that I was not using destination: plugin: 'entity:taxonomy_term' I am using destination: plugin: entity:node. And It still added my taxonomy to my nodes and to the drupal vocabulary. Why does this work without using plugin: 'entity:taxonomy_term.
Below is my migration file