Problem/Motivation

As a follow up of #2831274: Bring Media entity module to core as Media module we will need to provide migrate integration for the media entity.

Migration destinations do not block the media entity API or its file and image formats from being added to core, but will be needed before the full module (and any dependencies) are marked non-hidden and non-experimental.

Proposed resolution

  • Add a media entity destination plugin to be able to migrate stuff into media entities.
  • Provide example migration to migrate files into media entities. This would use the existing file entities, but then wrap them into media entities.
  • Provide migrations/update paths for file/image field configurations. They should become entity reference fields that reference the migrated media entities. #2885002: Provide an optional migration path for sites using files/images to media entities
  • Provide a migration example for an actual file field
  • Provide a migration example for an actual image field
  • Provide a migration example for the file/image field display configuration.
  • ... more?

Remaining tasks

User interface changes

API changes

Data model changes

Files: 

Comments

dawehner created an issue. See original summary.

Gábor Hojtsy’s picture

Component: migration system » media system
Issue tags: +Berlin media sprint

I think this will live with the media system right as with the rest of the integrations. We should define what if any is required for 8.3.0 shipping with media. Need input on that.

Gábor Hojtsy’s picture

I think the release manager review tag is warranted for that.

xjm’s picture

Issue summary: View changes
Status: Active » Postponed
Issue tags: -Needs release manager review

If Media does not have a migration destination and becomes stable, then that moves Migrate Drupal further from being stable. We should be requiring migration destinations for all added storages at this point. They will also help for migrating core image and file data into media entities once they are stable. So, I think we need to have the migration destinations available for Media to be stable (a "must" have on the initiative roadmap).

However, migrations are not a blocker for adding the API to core so long as the module is still in beta or earlier. For media, we agreed that the module itself could considered a stable API addition but experimental otherwise, so long as the module remains marked hidden. The same should apply here. So we can postpone this issue on finalizing the initial API and on finalizing the file and image types (although we may consider incling the migation destinations for files and images in those patches). Marking postponed on the initial entity issue, at least.

I updated the summary with this information.

Thanks @Gábor Hojtsy!

Gábor Hojtsy’s picture

@xjm: thanks! Since this has some farther reaching implications for release management process, continuing this on https://www.drupal.org/node/2825215#comment-11889256

mikeryan’s picture

Just a note - a destination plugin does not appear to be necessary for the media entity, I'm doing a project right now with contrib media_entity and the following works just fine on the destination end:

destination:
  plugin: entity:media
  default_bundle: image

I don't think any process plugins would be necessary either - just source and migration plugins for the upgrade path.

Gábor Hojtsy’s picture

What does the Migrate Drupal UI do for example if there are both migrations for image fields (D6/D7) to image fields (D8) and image fields (D6/D7) to media entities (D8??). I definitely agree update and/or migration paths between D8-D8 would be needed if we abandon (AKA deprecate) image/file fields on entities. That is a big decision to make in itself and needs careful planning as to how it is done.

mikeryan’s picture

What does the Migrate Drupal UI do for example if there are both migrations for image fields (D6/D7) to image fields (D8) and image fields (D6/D7) to media entities (D8??).

It executes all migration plugins in enabled D8 modules with the relevant migration_tag (Drupal 6 and Drupal 7). So, if media were enabled on D8, as it is now it would execute both - we'd want to have something in the media module that would disable the unwanted migrations (hook_migration_plugins_alter()?).

mikeryan’s picture

Actually, I misspoke earlier - there is one desirable process plugin, one to rewrite embedded images (either <img> tags or D7 media JSON data) to <media-entity> tags. I have such a plugin on my current project for the JSON case (although it obviously would need generalization), but the key challenge there for the upgrade path is that this plugin would need to be inserted into the process pipeline for every text field which may contain embedded images. That'll be fun!

mikeryan’s picture

On a moment's further thought... Maybe that processing could be hooked into the text field handling.

xjm’s picture

Issue tags: +migrate

Oh #4 is potentially good news! Thanks @mikeryan. So I guess part of the scope here is figuring out exactly what we need in core for stable Media, vs. optional/followup.

#9 sounds scary. Interesting problem for sure.

Is this correct tag to add for Migrate?

Gábor Hojtsy’s picture

@mikeryan well, media module does not deal with embedding, does it? :) That would be entity_embed or some other module you are using there. Similar functionality is desired for core but definitely not as part of the media entity module itself at first.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

slashrsm’s picture

Issue summary: View changes
jibran’s picture

Issue summary: View changes

I wrote a blog post about D7 file entities migration to D8 media entities.

jibran’s picture

I didn't intend to revert the issue summary. Can someone revert the issue summary to https://www.drupal.org/node/2835825/revisions/view/10404661/10405405, please?

xjm’s picture

I couldn't quite understand from the diff which was the intended revision, but hopefully this is the right one. Thanks @jibran.

PunamShelke’s picture

Hi,

I have done the migration for files as media entity in drupal 8 with the external json resource..

xjm’s picture

Status: Postponed » Active

I think this does not need to be postponed right now, correct?

xjm’s picture

Issue summary: View changes

#2885002: Provide an optional migration path for sites using files/images to media entities covers part of what's listed in the summary here, so I postponed it on this issue, which can add the migration integration itself.

phenaproxima’s picture

Status: Active » Needs review
Issue tags: -D8Media +Media Initiative
FileSize
12.98 KB

Here is an initial patch to generate media items from existing permanent files. There are dedicated tests for the source and destination plugins, plus a single end-to-end test of the migration.

There's a lot more than needs doing here (as per the IS), but setting to NR to see if Drupal CI approves. IMHO we should consider making this issue a meta and opening child issues for all the sub-tasks.