Problem/Motivation

This is an issue with the CckMigration object, which is used several times during the import of a field. It works fine when importing the field types, but when importing field formatters it looks for a field type plugin with the name of the formatter which will not always work.

To reproduce, create a Drupal 7 site with a text field and change the display settings to show the field as "Text Plain" and then try to migrate the site. You will find the following error in the log afterwards:

The "text_plain" plugin does not exist. ([drupal's path]/core/lib/Drupal/Component/Plugin/Discovery/DiscoveryTrait.php:52)

Proposed resolution

My best guess would be that $field_type needs to get populated with the name of the field instead of the formatter's name in CckMigration.php , or be mapped in d7_field_formatter_settings.yml . I'm not sure these are good long term solutions though because a static map can only cover the fields provided by core.

I notice some functions in the MigrateCckField plugins that may be planned for this too, such as getFieldFormatterMap(), called by processFieldFormatter() which is referenced as the cck_plugin_method in the yml's (currently only in d6_field_formatter_settings.yml but may be added to the d7 one as part of #2631736: Cckfield Plugins must distinguish core versions). I'm not sure that's really what the intent of those functions is though, because the MigrateCckField plugin can't even be found to get the correct processFieldFormatter() to run without first knowing the name of the correct MigrateCckField plugin.

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

jmuzz created an issue. See original summary.

jmuzz’s picture

Here is a patch that fixes the errors I had in my use case.

It doesn't handle everything, for example the taxonomy_term_reference_rss_category is not mapped and probably others.

Also nesting the maps under their plugin names wasn't working, for example ['text']['default'] => 'text_default' so the formatters have been moved up a level. In the patch I left the original line there commented out. This probably won't work because of formatters like in this example that have a generic name like 'default' so the nesting will probably need to be fixed.

heddn’s picture

I agree that using a static map is *very* fragile. If we can gather that data, then map it, we are going to be a whole lot better off. I think it is possible to glean the field_formatter data from the d6/d7 site. Or is that a hook_info thing that never get's stuffed in the db?

jmuzz’s picture

@heddn The formatter settings from the d6/d7 site are getting read and processed by the migration system. I'm not sure what you mean by "gather the data, then map it." It seems partially set up for the module providing the field and its formatters to also provide the migration mapping information (getFieldFormatterMap()). My solution is going more in that direction and away from having the mapping information for fields in the field or migrate_drupal modules.

jmuzz’s picture

I might have been wrong about the need for the field formatters to be nested under their field types. Though the current implementation of getFieldFormatterMap for text fields maps plugin names like "default" the actual formatter name in D7 is "text_default". It makes me wonder if the intent is for it to add the "text_" automatically from the field type key. If that's the case it hasn't been working in my tests so far. It may be reasonable to eliminate the nesting and use the full formatter name instead. I do notice that's how the widget maps are structured so it would make it more consistent.

jmuzz’s picture

When formatters have the same name in Drupal 7 and 8 they don't seem to need an entry in getFieldFormatterMap(), so that may explain why the function has been giving the appearance of working.

This is probably outside the scope of this issue but there are setups in Drupal 7 that may not be able to be migrated to Drupal 8 and I'm not sure what to do with them currently. For example D7 text fields get mapped to D8 "Text (Formatted)" however there is no equivalent of the D7 text formatter "text_plain" available to "Text (Formatted)" fields. The D8 "basic_string" formatter it tries to map to is only available to D8 "Text (Plain)" fields and similar. It's not safe to assume they wanted a plain text field just because it's being formatted as plain text in one of the view modes it's using. I wonder what the logic is of preventing "Text (Formatted)" fields from being displayed as plain text in any view mode in Drupal 8. It seems like unnecessary coupling between the content type and how it's displayed. Perhaps it could be made more flexible.

jmuzz’s picture

Created an issue about my remarks in the previous comment.

#2735157: Looser coupling between field type and display options

jmuzz’s picture

hussainweb’s picture

Status: Active » Needs review
FileSize
5.4 KB
4.24 KB

I just faced this issue today when working on #2594631: Migration integration. I had almost the same solution in mind before I found this issue. Since this is three months old, I hope you don't mind me making significant changes. Basically, I removed what I thought were out of scope changes and added a few more things like ability to specify the property name where field_type will be available in a source plugin.

Interestingly, the D6 version of field formatter migration already uses 'type' to indicate the field type and not the widget type. It is only the D7 migration which has this problem and hence I have only changed that YML file. I am wondering if we should make it consistent by setting 'type' to field_type and rename the existing 'type' to 'widget_type'. What do you think?

hussainweb’s picture

I forgot to mention that there is a related issue with fields in general when their plugin ids don't match the field name in source. I have create an issue for that at #2787639: MigrateCckFieldPluginManager mixes up its behaviour for creating and loading definitions and the changes in this patch will clash with that patch.

Status: Needs review » Needs work

The last submitted patch, 9: migrate-field_formatters-2726803-9.patch, failed testing.

quietone’s picture

I am wondering if we should make it consistent by setting 'type' to field_type and rename the existing 'type' to 'widget_type'. What do you think?

Yes, please! This will definitely improve the readability of the code.

phenaproxima’s picture

I am wondering if we should make it consistent by setting 'type' to field_type and rename the existing 'type' to 'widget_type'. What do you think?

I think hell freaking yes. Please do it.

phenaproxima’s picture

+++ b/core/modules/migrate_drupal/src/Plugin/migrate/CckMigration.php
@@ -104,8 +104,12 @@ public function getProcess() {
+      $field_type_property_name = !empty($this->pluginDefinition['cck_plugin_field_type_property_name']) ?
+        $this->pluginDefinition['cck_plugin_field_type_property_name'] :
+        'type';

I see why we're doing this (backward compatibility, right?), but I'm not sure migrations should have THIS level of control over the source plugin. I'm in favor of renaming it to field_type and not making it configurable at all. That will break a lot of tests, but IMHO it'll be worth it.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

The last submitted patch, 9: migrate-field_formatters-2726803-9.patch, failed testing.

hussainweb’s picture

Status: Needs work » Needs review
FileSize
4.11 KB

Rerolling to get started again.

Status: Needs review » Needs work

The last submitted patch, 17: migrate-field_formatters-2726803-17.patch, failed testing.

hussainweb’s picture

Status: Needs work » Needs review
FileSize
5.07 KB
6.16 KB

Fixing at least a couple of tests and then changing the name of field_type to type as per #12 and #13. There might be other failures.

hussainweb’s picture

We don't need this anymore. Removing the ability to specify field type column name from the YML file.

The last submitted patch, 19: migrate-field_formatters-2726803-19.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 20: migrate-field_formatters-2726803-20.patch, failed testing.

hussainweb’s picture

Status: Needs work » Needs review
FileSize
1.36 KB
5 KB

Fixing some more failures (hopefully all).

hussainweb’s picture

mikeryan’s picture

Assigned: Unassigned » mikeryan
mikeryan’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me, thanks!

mikeryan’s picture

Assigned: mikeryan » Unassigned

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 24: migrate-field_formatters-2726803-24.patch, failed testing.

mikeryan’s picture

Issue tags: +Needs reroll

Oh well...

The last submitted patch, 24: migrate-field_formatters-2726803-24.patch, failed testing.

jofitz’s picture

mikeryan’s picture

Status: Needs review » Reviewed & tested by the community

Back to RTBC, thanks!

alexpott’s picture

+++ b/core/modules/field/migration_templates/d7_field_formatter_settings.yml
@@ -51,7 +51,7 @@ process:
-      source: type
+      source: formatter_type

So our one BC issue is this change - right? As anyone who is doing a custom migration might have based things of this template. OTOH right now if this is not working for them they will have to have applied this patch (or something similar) to make it work. So what's the value in a BC layer?

+++ b/core/modules/field/src/Plugin/migrate/source/d7/FieldInstancePerViewMode.php
@@ -24,8 +24,15 @@ protected function initializeIterator() {
+        // Rename type to formatter_type in the info array.
+        $info['formatter_type'] = $info['type'];
+        unset($info['type']);

Hmm... that said - is this is going to break existing working migrations?

Or should the argument be that D7 migration is currently a little rough and this patch helps therefore let's get this in?

catch’s picture

Status: Reviewed & tested by the community » Needs review
mikeryan’s picture

Assigned: Unassigned » mikeryan
mikeryan’s picture

Jo/Hussain - can you review this and make sure I'm understanding and explaining this correctly?

So our one BC issue is this change - right? As anyone who is doing a custom migration might have based things of this template. OTOH right now if this is not working for them they will have to have applied this patch (or something similar) to make it work. So what's the value in a BC layer?

In cases where the field type and formatter type match (which is common), it works - it's only broken when they don't. So, if someone has a migration .yml using 'type', the BC layer not only keeps the old case working, by substituting 'formatter_type' it will now work better for them.

Hmm... that said - is this is going to break existing working migrations?

This is the BC layer - if someone has specified 'type', it gets converted to the new preferred 'formatter_type' and works just ducky.

jofitz’s picture

@mikeryan That is my understanding of the situation - I don't envisage BC issues.

mikeryan’s picture

Status: Needs review » Reviewed & tested by the community

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 31: migrate-field_formatters-2726803-31.patch, failed testing.

mikeryan’s picture

Status: Needs work » Reviewed & tested by the community

Random fail, kicked off retest, presumptively re-RTBCing.

mikeryan’s picture

Assigned: mikeryan » Unassigned

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 31: migrate-field_formatters-2726803-31.patch, failed testing.

The last submitted patch, 31: migrate-field_formatters-2726803-31.patch, failed testing.

catch’s picture

Status: Needs work » Reviewed & tested by the community

Not reviewed yet, just restoring status.

  • catch committed fbafc69 on 8.3.x
    Issue #2726803 by hussainweb, jmuzz, Jo Fitzgerald: Field formatters...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed/pushed to 8.3.x and cherry-picked to 8.2.x. Thanks!

  • catch committed 5213368 on 8.2.x
    Issue #2726803 by hussainweb, jmuzz, Jo Fitzgerald: Field formatters...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.