Problem/Motivation

Based on the plans being outlined in #2898704: Content Export Support, the earliest implementation steps should tackle stand-alone commands and services, with unit tests, to export most content-based entities. More complex elements may be handled separately, but the bulk of common use-cases should be accounted for with core export services.

Proposed resolution

Implement a dedicated service for export of content. This service will serve as the gateway to all of the export functionality and will be leveraged by both CLI and web interfaces for consistency and reusability.

Remaining tasks

  1. Research and discuss Serialization API as a solution for export
  2. Note relevance for refactored import process as well
  3. Implement single-entity export operations
  4. Implement batch entity export operations

User interface changes

User-interface changes and exposure of these commands is to be implemented in follow-up tasks.

See #2943912: Content Export Support: CLI Interfaces and #2943914: Content Export Support: Web Interfaces.

API changes

New export services will be implemented to handle export of single entities, and batches of entities.

Data model changes

TBD

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

slucero created an issue. See original summary.

slucero’s picture

Below is a cross-post of some of the discussion points between @johndevman and myself in #2898704-7: Content Export Support and #2898704-8: Content Export Support related to this functionality:

Export Process

I'm cross-posting this section to the new ticket for the core services for this process for reference and follow-up: #2943907-2: Content Export Support: Entity Export Services

I did a small image of how I thought an export could work technically. Based off the idea of using Serialisation and inspiration of other modules.

https://i.imgur.com/mpaW8NQ.png

I think this is a great model to start with. The normalization process was brought up previously and shows a great deal of promise. My concerns from my initial digging into the existing APIs at that point are that the existing core APIs for this may not suit our purposes.

The process means the use of processor plugins (I believe you mentioned earlier as well) which allows developer to opt-in and add additional data, etc.

This and the event system currently in use are both critical features in my mind since to this point these have been the primary methods of not only site-specific customizations, but also adding in support for more difficult or non-standard features like node menu links (#2879468: Nodes Cannot Create Menu Links Automatically), user loading #2876203: Support load user entity by name process callback, and, soon, path aliases (#2883434: Nodes Cannot Create Path Aliases).

Also some questions (perhaps part of the user interface)

  • Do you want to preserve IDs? (Error-prone)
  • Do you want to include entity references children? (Most likely)
  • Do you want to move physical assets? (Probably)
  • Do you want to include all languages? (Maybe)

Since this post is already getting very long, let's follow-up and discuss these decisions in more detail in #2943907: Content Export Support: Entity Export Services.

Things like URL aliases, Menu links etc would be handled by processor plugins. I looked at Facets which has multiple processor stages which is really flexible for developers wanting to opt-in.

I hadn't thought of referencing Facets and their multi-staged system, that's a great idea and could potentially solve some of the problems I've encountered so far with the event-based system.

Since an entity may contain many entities, I figured it would be better to actually split up an export in multiple files (which also would make debugging easier).

I think there are a lot of side-effects that will cascade down from any decisions we make on this front. I tend to agree that the content will need to be split across multiple files, especially to support content reuse and cross-referencing, but the extreme I would like to avoid as much as possible is making the automated exports too difficult to manually interpret and maintain where needed. One example of how this could come about, would be if all of the exported content was keyed by UUID which is notoriously unfriendly.

slucero’s picture

FileSize
79.18 KB

I wanted to attach and expose the previous diagram from @johndevman to make sure it doesn't get lost anywhere. All credit to @johndevman (#2898704-7: Content Export Support).

Content Export Process Diagram

johnwebdev’s picture

When exporting a entity, it is unlikely we are only going to export one entity.

Content entity

Above we have a simplified illustration on what we could potentially expect when exporting Article 1 and we want to include references.
In this example Content is a Paragraphs field with a Link paragraph which references to a internal URI.

slucero’s picture

Component: Code » Content Export
slucero’s picture

Issue summary: View changes

Based on some conversations at DrupalCon, the first steps here should focus on getting a single entity converted into YAML targeting a sequence of scenarios escalating in complexity from simplest to most complex:

  1. A single node with basic data fields, no references, no paragraphs
  2. A single node with single-level paragraph fields, no other entity references
  3. A single node with taxonomy term references

This first pass will focus only on exporting the data relative to a single entity. With this assumed perspective, we can remove the complexity of inter-entity relationships and required processing for content references. This will be a method to produce the content for a single entity to be added to a content file without entirely manual creation, but will assume manual intervention by the developer/content curator to handle more complex elements like content duplication, file creation and reused content references.

Building on this from a centralized export service, an early tool for accessibility to this module through the interface and to lighten the effort of creating content for import, this could be exposed as a local task link for entities, like nodes or taxonomy terms, alongside the existing "edit" and "view" links. I could envision this tab loading the current entity in converted YAML Content format within a text editor field for easy copy and paste into. As an enhancement to this, files and images included in the export could be linked directly in a list below the text field for download and reference in the exported content.

Note that the web UI elements of this are the focus of #2943914: Content Export Support: Web Interfaces and I'll be cross-posting this information into #2943914-4: Content Export Support: Web Interfaces.

Next Steps

  • Discuss blocking discovery items
  • Create entity export service
  • Implement single-entity export methods

Blocking Items

  • Is Serialization API a solution for this?
  • What normalizers would be needed?
  • How do we handle determining properties to include/exclude in exports? (Whitelist/blacklist? Configurable?)