Problem/Motivation

The Default Content module might benefit from using Migrate to control the import of content:

  • Benefit from available Migrate processor plugins
  • Allows for alternative sources without too much development, such as remote files or different data structures such as XML
  • Removes the development burden for command line integration
  • Allows for 'Highwater marks', so you keep using Default content while is site is actively being used by editors
  • Allows for 'rolling back' content. This is a very useful feature for updating demo content on an active site.

My team is currently using a similar approach for Drupal7: https://www.drupal.org/project/migrate_defaultcontent. My current employer is also willing to sponsor development time internally to get this done.

Proposed resolution

Base content imports on Migrations for great benefit.

Remaining tasks

  • Write a patch
  • Review
  • Commit

User interface changes

None

API changes

Yes, to be determined.

Data model changes

Yes, to be determined.

Comments

idebr created an issue. See original summary.

idebr’s picture

Issue summary: View changes
larowlan’s picture

My first thoughts

  1. default_content's primary goal is to create content in install profiles, at install time
  2. migrate module is experimental - apis are fluid
  3. if we can do 1) with migrate underneath, all good
  4. if we can't but they can live alongside each other, all good
benjy’s picture

It's not a bad idea but I don't think Migrate will make things much better for this module:

  1. Normalizers/Denormalizers in core are maintained together so if your export works so does your import. If we used Migrate for import that could break.
  2. To be frank, the Migrate CLI tools aren't really up-to-scratch so there is little to gain there.
  3. Highwater mark is irrelevant to default content, it's meant to be a 1 time import of content on install, not a continuous content solution. If you want to add more content, export it and then re-install your site.
  4. Rollback, same as above, just re-install your site.
sam152’s picture

This has come up before, I've left my thoughts on it here. They key issue I can see is that migrate isn't at all concerned with exporting content, the current approach can import AND export with very little effort.

fgm’s picture

We have a process similar to this one on multiple sites:

  • a custom dumper exports all content entities in selected entity types (regardless of module) using the serializer, although in a less elaborate way than this module does
  • a series of migrations import the content on demand

We run this after deploying the install profile, as it allows deploying different data sets on different environments, like an anonymized reduced data set for developers outside the EU (privacy laws) or CI platforms, no data at all or only some entities (dev workstations), or the full data set (prod/preprod replication).

The big reasons for migrate are the ability to rollback, perform imports on demand, either in deploy or update mode, and most importantly, handle just about any volume of data thanks to batching, which this module does not use currently.

nedjo’s picture

Status: Active » Closed (duplicate)

For migrate, there is a parallel D8 project, Migrate Default Content. Effort towards a migrate-based default content solution can concentrate there.

ressa’s picture

Just leaving a comment for those who don't need to export content, but looking for a way to create nodes and terms, for example for content during development, in which case Drupal core module Migrate offers an alternative solution.

There is a great ressource at https://understanddrupal.com/migrations, and with Migrate module, a custom module with three files is all that is needed to create nodes and terms--not connected in this example, though that is of course possible (code taken from ud_migrations_first and ud_migrations_multivalue_terms):

ud_migrations_first.info.yml

type: module
name: UD First Migration
description: 'Example of basic Drupal migration.'
core_version_requirement: ^8 || ^9 || ^10
dependencies:
  - drupal:migrate

udm_first.yml

id: udm_first
label: 'UD First migration'
source:
  plugin: embedded_data
  data_rows:
    -
      unique_id: 1
      creative_title: 'The versatility of Drupal fields'
      engaging_content: 'Fields are Drupal''s atomic data storage mechanism...'
    -
      unique_id: 2
      creative_title: 'What is a view in Drupal? How do they work?'
      engaging_content: 'In Drupal, a view is a listing of information. It can a list of nodes, users, comments, taxonomy terms, files, etc...'
  ids:
    unique_id:
      type: integer
process:
  title: creative_title
  body: engaging_content
destination:
  plugin: 'entity:node'
  default_bundle: page

udm_first_terms.yml

id: udm_first_terms
label: 'UD taxonomy term migration'
source:
  plugin: embedded_data
  data_rows:
    - fruit_name: 'Grape'
      fruit_description: 'Eat fresh or prepare some jelly.'
    - fruit_name: 'Red grape'
      fruit_description: 'Sweet grape'
      fruit_parent: 'Grape'
    - fruit_name: 'White grape'
      fruit_description: 'Sour grape'
      fruit_parent: 'Grape'
    - fruit_name: 'Apple'
      fruit_description: 'Eat fresh or prepare some juice.'
    - fruit_name: 'Red apple'
      fruit_description: 'Sweet apple'
      fruit_parent: 'Apple'
    - fruit_name: 'Green apple'
      fruit_description: 'Sour apple'
      fruit_parent: 'Apple'
  ids:
    fruit_name:
      type: string
process:
  name: fruit_name
  description: fruit_description
  parent:
    plugin: migration_lookup
    migration: udm_first_terms
    source: fruit_parent
destination:
  plugin: 'entity:taxonomy_term'
  default_bundle: tags

File structure:

ud_migrations_first/
├── migrations
│   ├── udm_first_terms.yml
│   └── udm_first.yml
└── ud_migrations_first.info.yml