I'm trying to migrate nodes from Drupal 7 to Drupal 8 with paragraphs fields, but I can't seem to get the configuration right.
I believe I have a functioning custom source plugin for d7 paragraphs, because when I run it, it correctly (as far as I can tell) migrates paragraphs, but I'm having trouble with migrating the reference relationship when I migrate the nodes.
I see in comments on other issues relating to the migration of paragraphs fields that others are successfully migrating paragraphs fields, but I can't seem to figure out how to structure my node migration to establish the relationship.
This is my migrate_node_page migration. It migrates the nodes, and gets other relationship fields like taxonomy terms, but the paragraphs are just missing:
.
.
.
process:
.
.
.
field_paragraphs:
plugin: iterator
source: field_paragraphs
process:
target_id:
plugin: migration_lookup
migration: migrate_paragraph_items
source: target_id
.
.
.
Does anyone see what I'm doing wrong?
Issue fork paragraphs-2990527
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
safetypinComment #3
safetypinComment #4
havran commentedI think you must use both target_id and target_revision_id like here:
Extract is used because paragraph use compound ID (array of target_id and target_revision_id).
Comment #5
safetypin@havran - thanks for getting back to me.
Unfortunately that configuration also didn't work for me. The paragraphs are migrated, but the relationship between the paragraphs and the nodes is not.
Why do you have "pid" in that migration? Do I need to change that?
Comment #6
safetypinI was looking at the MigrationLookup.php class, and I see that on line 166 there is this block of code:
When I inspect $this->configuration I see an array that appears to be created based on the migration you suggested, however, there is no "source_ids" element to this array, so as far as I can tell, this block of code would never be executed. The $migration_id matches the configuration that I've used (referencing the specific migration I've created to migrate my paragraph entities). Is $this->configuration['source_ids'] supposed to be configured in the migration? I don't see anyone else using anything like that in example migrations for paragraphs.
Comment #7
anu_mathew commented@safetypin : have you got the solution for this ? Am also facing same issue with paragraph and content mapping.
Comment #8
safetypin@anu_matthew I haven't found a migrations based solution. If you are willing to maintain IDs across the migration (make the id numbers the same in Drupal 8 as they are in Drupal 7) then exporting/importing the data isn't very difficult.
The migration will create the paragraphs_item and paragraphs_item_revision data, and you should be able to migrate non-reference field data (numbers, text, emails, links, etc) with drush migrate commands.
Using a migration like this will maintain the ID numbers (note "id:item_id" and "revision_id: revision_id" at the beginning of the process), so you can export/import the entity reference field tables and all you have to do is make some minor changes to the column names (like in D7 the column is "language" and in D8 the column is "langcode"). The data stored in the database is exactly the same:
If you have paragraphs that also have entity reference revisions fields that contain other paragraphs then it gets a little more complicated. In this case, you have an extra step of export/import for each reference field.
I was able to migrate successfully a "columns" type paragraph that contained paragraph reference field, that contained several different other paragraph types using a database GUI that allowed table export and import. I'm often logged into the Drupal IRC channels (my handle there is also safetypin).
Comment #9
mirsoft commentedThe solution that works perfectly for me (based on answer #4, thanks @havran) in latest Drupal 8.6.2 is below.
Please note that I didn't need to migrate past revisions of paragraphs. I just wanted to migrate the last (actual) paragraph revision from the source and attach it properly to the parent node. In reality, I think it's very common use case, because one might not need to fill in the target DB with mostly obsolete past revisions. But if you'd need to migrate *all* the revisions from the source paragraphs as well, you would probably need to extend the code below and migrate also proper revisions references.
These items you will likely need to customize:
I'm setting the same "value" and "index" values for both target_id and target_revision_id fields. That's possible, because without migrating all the past revisions, we are okay to assume that the current revision is the first one, thus with same id as target_id. This would be probably a place to extend if you would need to migrate all the revisions.
Note: This pattern works perfectly for field collections as source as well. Just enter your field collection migration instead of paragraphs into the "migration:" field and follow the guide above for other fields.
Comment #10
mirsoft commentedActually, the previous example can be even simplified like this:
Comment #11
joshua.roberson commented#10 works for me with no issues.
Comment #12
hazong commentedHey Guy, I'a facing the same problem without solution:
here is the structure of my json file:
the firts level i.e. uuid, name, descript... represents the field taht will be migrated to a entity:node with machine name 'produkt'
the content for my paragrap is the array 'productSpecifications'
Here is the yml file configuration for migrating the paragraphs:
Here is the yml configuration for my node (the node migration always works):
I'm really hopeless with this, I'm working on that since one week without sucess, The migrate on import one paragraph.
Don't know what to do.
Help needed!
Comment #13
rviner commentedI can't get #10 to work. It would be useful if the whole yml is posted so I can see the source and destination. It would also be useful to see an example of the yml for one of the previous migrations e.g paragraph_box.
The nearest I can get is #7 but I was hoping to not manually copy the mappings.
Comment #14
amit.drupal commentedMigrate Slideshow Drupal 6 to Drupal 8.
Field by field mapping in migrate_plus.migration..yml file
Create custom_migrate.module file and Implement HOOK_migrate_prepare_row().
Comment #15
amit.drupal commentedComment #16
joshua.roberson commentedHere is my implementation for migrating from D7 field collections to D8 paragraphs.
Migrating data from D7 field collection to D8 paragraph.
migrate_plus.migration.he_fc_education.yml
Migrating field references from D7 field collection to D8 paragraph.
migrate_plus.migration.he_node_person.yml
Note, migration uses revision data, so if you have issues, make sure the data exists in the revisions tables. I had a D7 module installed to delete old revision data to help keep the database small. That is what caused my field collections to paragraphs migration to not work properly. It would migrate without data. To fix it, I simply truncated the field collection revision tables and did an INSERT of the corresponding data tables into them. Make backups first so you don't screw anything up. Another option is to make a custom source plugin that doesn't use the revision data.
Comment #17
zebda commentedAlthough there are some hopeful comments is this thread I can't seem to find out how to migrate my content types with paragraphs from Drupal 7 to Drupal 8. Comment #9 and #10 look promising. But they are only a small part of the full solution.
If i'm right they show a part of the code for migrating the paragraph field and not the paragraph entities. Can someone help me out with a working example or good documentation on this paragraph migrating? Thanks in advance.
Comment #18
mirsoft commented@zebda, yes the #9 and #10 (#10 is recommended as it's simpler) just show migration of the content nodes that have paragraphs attached. The items like paragraph_box, paragraph_grid, paragraph_text_embedded are paragraph entities, and you need to create a migration for each of these. That's an another topic, however a simpler one as these migrations are quite straightforward. An example of that migration (D7 paragraph to D8 paragrah) that worked for me was like following:
Common case however is that in D7 you do not paragraphs as a source. For instance, a popular pattern in D7 was field collections instead of paragraphs. In that case, you need to change source part a bit. For field collections the source would look like following:
Comment #19
mirsoft commentedFor the ones who were missing the full node migration in #10, I'm adding the full yml that worked for me:
Comment #20
zebda commented@mirsoft, thanks a lot that was just what I was looking for. You made my day. I have one follow up question. For some reason only half of the nodes is created the others have an error
Array index missing, extraction failed.. A quick search on the internet points me in the direction askip_on_emptyplugin. To skip when a field is empty.Is this the solution and how can I use this in the code you provided? Thanks again!
Comment #21
agileadamI am going to see if I can eliminate my need for this plugin (because I'd rather use out-of-the-box functionality), but this is what I use to process node and paragraph lookups the same way:
http://agileadam.com/2017/09/drupal-8-migrate-process-plugin-migration_l...
I can just use the
migration_lookup_target_idsafter anymigration_lookupand it just works (on my sites, at least). Please test for yourself.Note that I've had to patch migrate to handle multiple values: https://www.drupal.org/project/drupal/issues/2890844#comment-13158685.
Comment #22
steinmb commentedChanging this to a task. If there is no docs. there should perhaps be one at https://www.drupal.org/docs/contributed-modules/paragraphs ?
Comment #23
steinmb commentedAnd I think it should be in here.
Comment #24
santhosh-kumar-rengasamy commented@mirsoft, I followed the method described in the #19 and it was working fine in my local. Migrating the paragraphs and the nodes having paragraphs. The lookup table in my local was below:
But if i do the same in my development the destid1 and destid2 are different. like below:
Do you have any solution or the cause for the issue identified?
Comment #25
santhosh-kumar-rengasamy commented#24 error occurs when we had existing contents or manually added content in the site. This issue is solved after mapping the target_revision_id with index 1 like below
Comment #26
nevkatz commentedHi there,
Thanks for all the great insight, everyone. The approach below worked for me.
My original approach came from this Softecu post. After replacing the plugins and changing the custom machine names, I was able to migrate the paragraphs but not wire up the paragraphs to the nodes.
https://softescu.com/blog/how-migrate-paragraphs-drupal-7-drupal-8
And my YAML file looks like this:
Best,
Nevin
Comment #27
vacilando commentedThis is I think the clearest article on the topic of migrating paragraphs to Drupal 8: https://understanddrupal.com/articles/introduction-paragraphs-migrations...
It has helped us enormously.
(Check the whole series at https://understanddrupal.com/31-days-of-migrations.)
Comment #28
stevenx commentedHi, I am having 3 paragraphs contents which I need to combine to one paragraph field.
when I add @para_split the migration does not work anymore, when only adding @para_image and @para_text it works. My question is on how to add the split_into_paragraphs to it so it gets imported… when running only the following it works:
Comment #29
stevenx commentedI found how its working, I had to use merge
https://git.drupalcode.org/project/migrate_plus/-/blob/8.x-5.x/src/Plugi...
Comment #30
mkindred commentedI couldn't find an example that worked for my situation (multiple paragraph fields per content type, multiple values per field, and multiple paragraph types to choose from within each field). I was able to write a custom module that handled it. After I finished, I abstracted the module out to make it a bit easier to understand and created its own repository for it.
The repositories and an explanation video are available here: https://www.arsnova.cc/drupal-7-drupal-8-paragraph-migration
Comment #31
franzAs someone suggested using the same value for target_id and target_revision_id I'd like to point out I had weird outcomes. Data was migrated but paragraphs would load incorrectly (a different paragraph showing up). This is hard to replicate because it depends on the data previously stored. In the end I used https://www.drupal.org/project/migrate_process_entity to pull the current revision_id after doing the lookup for the paragraph. This seems to have worked well, the field process section became:
Comment #32
cmah commentedThank you for #25, which finally helped me with my field_collection migration -- all the older examples didn't help at all. I'm posting this in the hopes it will save someone else a bit of time.
Comment #33
ccarnnia commented++ on #25 thank you Santhosh and cmeh.
Comment #34
ressaAs an alternative to Paragraphs, in case a more basic solution might work, migrating nested multivalue data into Custom Field is easy. A structure like this:
... can be mapped with this:
From Migrate multivalue fields into unlimited Custom Fields field.
Comment #35
jepster_I am working on a Drupal 10 to Drupal 10 migration. All the examples here did not help. Looks like they are related to Drupal 7.
My solution was an own migrate process plugin. You can find it here: https://git.drupalcode.org/project/sites_migrator/-/blob/1.0.x/src/Plugi...
The plugin must be implemented into the process like this:
I hope this example helps somebody.
Comment #36
baikho commented#35 works fine, but made a slight adjustment for multiple migrate sources #3531281: ParagraphsSubProcess plugin does not merge paragraph items when using += operator