Problem/Motivation

From #2914974-136: Migrate UI - handle sources that do not need an upgrade, we have the image field plugin in the file module. That doesn't work, because we need the image field type provided by the image module.

It also doesn't show the image module as un-available when the module is disabled.

Proposed resolution

Move the image field plugin into the image module.

Remaining tasks

Move code. I don't think we have to do any BC here, because the plugin name would stay the same. It would just move.

User interface changes

API changes

Data model changes

Comments

heddn created an issue. See original summary.

David Hernández’s picture

Status: Active » Needs review
StatusFileSize
new711 bytes

I think this should deal with the requested changes in the issue. I have not been able to find anywhere in the code base where this file is used.

joelpittet’s picture

Status: Needs review » Reviewed & tested by the community

There would be a slight BC break if anybody extends that class, but not sure if our policy covers that.

Status: Reviewed & tested by the community » Needs work
heddn’s picture

Issue summary: View changes
Status: Needs work » Needs review
StatusFileSize
new51.82 KB

This gets a little tricky here. Because we are also dealing with d6 file fields. I tried changing the destination module and some other things, but it still doesn't show image up on the list as a non available module.

re: BC
No, we'd still want to leave the class in the old location and have it extend from the new location. And we need to do that for d6 and d7.

joelpittet’s picture

Ah, that's a good BC approach @heddn, good call.

heddn’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests

I think I'm making this more complicated (in my mind) than it needs to be. In addition to the d6 change I mentioned, we also need:

  • tests
  • add a module dependency to d6_user_picture_file.yml on the image module

Tests are because we didn't catch it with our current test coverage. We need to add module dependency, because otherwise we don't require the image module to be enabled.

Hmm... I wonder if we want to just add the image dependency onto the d6_user/d7_user migration too? That way we can be sure that the image module is enabled.

gábor hojtsy’s picture

Drupal 6 user pictures are disabled by default and sites actively need to enable it, so should not be a fixed requirement for user migrations.

  $picture_support = variable_get('user_pictures', 0);
  $form['pictures']['user_pictures'] = array(
    '#type' => 'radios',
    '#title' => t('Picture support'),
    '#default_value' => $picture_support,
    '#options' => array(t('Disabled'), t('Enabled')),
    '#prefix' => '<div class="user-admin-picture-radios">',
    '#suffix' => '</div>',
  );
heddn’s picture

Priority: Normal » Major

Bumping priority. Current state provides misleading data on a pretty important module.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new3.26 KB
new1.88 KB

Updated the tests.

quietone’s picture

StatusFileSize
new4.14 KB
new859 bytes
new5.66 KB

With this imagefield will display as missing.

Status: Needs review » Needs work

The last submitted patch, 11: 2934145-11.patch, failed testing. View results

quietone’s picture

Version: 8.4.x-dev » 8.5.x-dev
Status: Needs work » Needs review

I was working on 8.5.x, let's try that again.

jofitz’s picture

StatusFileSize
new4.65 KB
new869 bytes

I'm unable to replicate the test failures locally, but I have identified a minor change that is required.

I am also running the patch in #10 against 8.5.x on the assumption it will suffer very similar failures.

heddn’s picture

Status: Needs review » Needs work

The last submitted patch, 14: 2934145-14.patch, failed testing. View results

jofitz’s picture

Assigned: Unassigned » jofitz
heddn’s picture

Issue tags: +Migrate January 2017 Sprint
heddn’s picture

Issue tags: -Migrate January 2017 Sprint +Migrate January 2018 Sprint
jofitz’s picture

Assigned: jofitz » Unassigned
heddn’s picture

field module is a module dependency of MigrateDrupalTestBase, that's why ImageField is even showing up on random things like Action. Now to figure out why it is erroring...

heddn’s picture

Status: Needs work » Needs review
StatusFileSize
new6.12 KB
new10.3 KB

I'm not sure how much the interdiff is going to help. But I've added one. The reason for the test failures is because when we moved the field plugin to image module, we left the cck plugin. This plugin is no longer masked and gets discovered because file module is enabled in the base migrate test class. Oops.

To solve this, I've moved the cck class from field into image as well. Simple enough. We'll see after tests if it is really that simple.

Also you'll see in the patch links to a CR and deprecation notices added to the file classes that got copied (not moved to retain BC) into image module.

Still need some tests to address the concerns in #7. But let's see how the BC layer is holding up.

marcvangend’s picture

+++ b/core/modules/file/src/Plugin/migrate/cckfield/d7/ImageField.php
@@ -2,42 +2,16 @@
-@trigger_error('ImageField is deprecated in Drupal 8.3.x and will be removed before Drupal 9.0.x. Use \Drupal\file\Plugin\migrate\field\d7\ImageField instead.', E_USER_DEPRECATED);
+@trigger_error('ImageField is deprecated in Drupal 8.3.x and will be removed before Drupal 9.0.x. Use \Drupal\image\Plugin\migrate\field\d7\ImageField instead.', E_USER_DEPRECATED);
...
  * @deprecated in Drupal 8.3.x, to be removed before Drupal 9.0.x. Use

In other places, "8.3.x" was changed to "8.5.x". Shouldn't these be changed too?

heddn’s picture

That's because core/modules/file/src/Plugin/migrate/cckfield/d7/ImageField.php was deprecated in 8.3. We're just updating the reference to point to the new location. The other places are actually deprecated for the first time in 8.5.

quietone’s picture

StatusFileSize
new11.28 KB
new1.22 KB

Correct the destination modules in MigrationProvidersExistTest and renamed a directory from cck to cckfield.

The last submitted patch, 22: 2934145-22.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

jofitz’s picture

Status: Needs review » Needs work

Still need some tests to address the concerns in #7.

quietone’s picture

There are two items in #7

  1. tests
  2. add a module dependency to d6_user_picture_file.yml on the image module

The second was answered in #8, a module dependency can't be added. That leaves the tests.

I'm a bit unclear on the test, it seems to be a test to ensure that this field plugin, image, is in the correct module, that is the module where the image field type is defined. Is that correct? If that is correct, than the test should cover all field plugins and not be limited to any particular one. And then should that be done here or in a followup? Or, do I have it all wrong?

quietone’s picture

Asked on IRC and learned I got that all wrong

heddn> quietone: one thing to consider is that if you have a d6 fixture that has a profile photo. and on d8 the image module is disabled, then we 1) should error the user migration 2) we should show that image module isn't enabled on d8.
<heddn> quietone: and that means that imagefield or whatever they call it should show up in the missing upgrade paths. that currently doesn't happen. our current d6 test enables image so we don't surface this problem.
<phenaproxima> quietone: I think the way to test this is to simply use those deprecated classes and instantiate them, and ensure that they do, indeed, throw deprecation notices. No?
2) we should show that image module isn't enabled on d8.

I take this to mean a test of review page when running from the UI, /upgrade. If so, we could use the new test base class, MigrateUpgradeReviewPageTestBase, from #2914974: Migrate UI - handle sources that do not need an upgrade.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

heddn’s picture

So, writing a failing test here is going to be hard. Since the problem is that we need to enable the image module for things to pass successfully. I guess to show the problem, we add a test of the successful import of images. Right now it looks like we don't have that. Or at least one that doesn't also enable the image module. To show the failing situation, we can upload an almost duplicate patch that doesn't enable the image module.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new13.94 KB
new13.65 KB

This adds a test of image field migration for d6 and a fail test.

The last submitted patch, 32: 2934145-32-fail.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

StatusFileSize
new6.67 KB

Here's an interdiff on the failure and success patch in #32.

heddn’s picture

StatusFileSize
new7.74 KB

And here's an interdiff against #25 for the success patch in #32

heddn’s picture

Status: Needs review » Needs work

Yeah, we've got us valid test coverage now, as seen in the failure test in #32.

Unable to determine class for field type 'image' found in the 'field.storage.node.field_test_imagefield' configuration

The last actionable thing I think we need to do is figure out what to do for d6 user profile photos. See comment in #29. The option to include user profile photos is a config option in D6. And by default it is turned off. If it is turned on, then we want to somehow notify the user upgrading their site that they will loose user profile photos. User profile photos in d6 are unique, because are not managed files. They are just a string location of where the file is located on the disk. However, when that gets sucked into D8, that becomes a managed file. So, the "source" of the data is the user module in D6. The destination is image module in D8. And we want to warn users.

To solve this, we could do a couple things. 1) open a follow-up to address this and get what we have committed. I like that option :)

Or we can press on here and 2) add a kernel test that tests this whole scenario and see what are our options.

Putting to NW for the nits below. If there is consensus on breaking out the user profile stuff into a follow-up, let's also get that opened. Then this is pretty nearly ready to go.

  1. +++ b/core/modules/image/tests/src/Kernel/Migrate/d6/MigrateImageTest.php
    @@ -0,0 +1,48 @@
    + * Node content migration.
    

    This should be updated.

  2. +++ b/core/modules/image/tests/src/Kernel/Migrate/d6/MigrateImageTest.php
    @@ -0,0 +1,48 @@
    +  public static $modules = ['menu_ui'];
    

    I'm not sure this is necessary.

quietone’s picture

Status: Needs work » Needs review
StatusFileSize
new13.65 KB
new144 bytes

36.1 Fixed
36.2 Without menu_ui there are schema errors.

1) Drupal\Tests\image\Kernel\Migrate\d6\MigrateImageTest::testNode
Schema errors for node.type.article with the following errors: node.type.article:third_party_settings.menu_ui.available_menus missing schema, node.type.article:third_party_settings.menu_ui.parent missing schema (/opt/sites/d8/core/lib/Drupal/Core/Config/Development/ConfigSchemaChecker.php:95)

I think it is worth noting that a MigrateImageTest for d7 is not added because the d7 MIgrateNodeTest already includes assertions on a image field.

Yes, lets solve the user profile problem in another issue and keep this about moving the image field plugins.

quietone’s picture

heddn’s picture

Status: Needs review » Needs work
Issue tags: -Needs tests +Needs change record

This is big enough and complex enough that splitting things off to a follow-up make sense. Let's get this merged, then on to follow-ups. I think the last thing is a CR? The code here looks solid. Plus, I cannot RTBC this thing since I worked on it too much.

quietone’s picture

Status: Needs work » Needs review
Issue tags: -Needs change record

There is an existing change record, Migrate Image CCK and Field plugins moved to image module, which looks good to me, so we are all done here.

phenaproxima’s picture

Looks good -- I mean, it's incredibly confusing, as these "move X to another module" issues always are -- but it seems legit to me. The only strange element is MigrateImageTest.

  1. +++ b/core/modules/image/tests/src/Kernel/Migrate/d6/MigrateImageTest.php
    @@ -0,0 +1,48 @@
    +/**
    + * Image migration test.
    + *
    + * @group migrate_drupal_6
    + */
    +class MigrateImageTest extends MigrateNodeTestBase {
    

    Can this doc comment be expanded? It doesn't explain what the purpose of the test is, or why it's extending MigrateNodeTestBase, or...you get the picture. I think we need to explain the context a bit more.

  2. +++ b/core/modules/image/tests/src/Kernel/Migrate/d6/MigrateImageTest.php
    @@ -0,0 +1,48 @@
    +    $node = Node::load(9);
    +    // Test the file field meta.
    

    I find this comment a bit odd.

heddn’s picture

StatusFileSize
new13.7 KB
new1.07 KB
quietone’s picture

@heddn, thanks for making the patch. It looks good, but I don't think I can RTBC since I added the test.

+1 RTBC

phenaproxima’s picture

Status: Needs review » Reviewed & tested by the community

My comments are addressed. Looks legit.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/modules/file/src/Plugin/migrate/cckfield/d7/ImageField.php
    @@ -2,42 +2,16 @@
    -@trigger_error('ImageField is deprecated in Drupal 8.3.x and will be removed before Drupal 9.0.x. Use \Drupal\file\Plugin\migrate\field\d7\ImageField instead.', E_USER_DEPRECATED);
    +@trigger_error('ImageField is deprecated in Drupal 8.3.x and will be removed before Drupal 9.0.x. Use \Drupal\image\Plugin\migrate\field\d7\ImageField instead.', E_USER_DEPRECATED);
    

    Let's add a link to the CR here too... i know this is an existing issue.

  2. +++ b/core/modules/file/src/Plugin/migrate/field/d6/ImageField.php
    @@ -2,12 +2,16 @@
    +@trigger_error('ImageField is deprecated in Drupal 8.5.x and will be removed before Drupal 9.0.x. Use \Drupal\image\Plugin\migrate\field\d6\ImageField instead.', E_USER_DEPRECATED);
    
    +++ b/core/modules/file/src/Plugin/migrate/field/d7/ImageField.php
    @@ -2,35 +2,16 @@
    +@trigger_error('ImageField is deprecated in Drupal 8.5.x and will be removed before Drupal 9.0.x. Use \Drupal\image\Plugin\migrate\field\d7\ImageField instead.', E_USER_DEPRECATED);
    

    This should also link to the CR as per policy.

  3. +++ b/core/modules/file/tests/src/Unit/Plugin/migrate/field/d7/ImageFieldTest.php
    @@ -10,6 +10,7 @@
    + * @group legacy
    

    There should be a corresponding * @expectedDeprecation ImageField is deprecated in Drupal 8.5.x and will be removed before Drupal 9.0.x. Use \Drupal\image\Plugin\migrate\field\d7\ImageField instead. on the test - this message will need the CR added too.

    Also I think we should have a corresponding test for Legacy test for D6 just to show it works

jofitz’s picture

Status: Needs work » Needs review
StatusFileSize
new16.17 KB
new10.62 KB
  • Add links to the CR in "cckfield/d7/ImageField.php", "field/d6/ImageField.php" and "field/d7/ImageField.php".
  • Add @expectedDeprecation to field/d7/ImageFieldTest.php.
  • Create field/d6/ImageFieldTest.php.
heddn’s picture

Status: Needs review » Reviewed & tested by the community

Feedback addressed. Back to RTBC.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 46: 2934145-46.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

heddn’s picture

Status: Needs work » Reviewed & tested by the community

Random test failure

alexpott’s picture

Title: Image field plugin in wrong module » Image field migration plugin in wrong module
alexpott’s picture

Status: Reviewed & tested by the community » Needs work
+++ b/core/modules/file/src/Plugin/migrate/cckfield/d7/ImageField.php
@@ -2,42 +2,16 @@
+use Drupal\image\Plugin\migrate\cckfield\d7\ImageField as LegacyImageField;
...
+class ImageField extends LegacyImageField {}

+++ b/core/modules/file/src/Plugin/migrate/field/d6/ImageField.php
@@ -2,12 +2,16 @@
+use Drupal\image\Plugin\migrate\field\d6\ImageField as NonLegacyImageField;
...
+class ImageField extends NonLegacyImageField{}

+++ b/core/modules/file/src/Plugin/migrate/field/d7/ImageField.php
@@ -2,35 +2,16 @@
+use Drupal\image\Plugin\migrate\field\d7\ImageField as NonLegacyImageField;
...
+class ImageField extends NonLegacyImageField{}

We can't do this. We should leave the legacy code in place and not extend from the non-legacy. This is because image module might not be installed so we can't extend from its classes.

heddn’s picture

Status: Needs work » Reviewed & tested by the community

It's a bit of a catch 22. The field migration is going to fail if image is disabled. Even if we don't extend. Because the field schema didn't exist. That's why there is no annotation on the legacy field any more. Because the annotation gets moved to a place where it can function. The only reason we have a bc layer is for the case where someone directly extended the class.

Yes, this is confusing. Not sure if that explanation is enough to return to rtbc or not. Let's try it.

alexpott’s picture

Adding credit. I confirmed with @heddn in slack that existing migrate plus migrations will pick up the new plugins.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Committed and pushed 544740df31 to 8.6.x and 50f4ceae89 to 8.5.x. Thanks!

Discussed with @catch and we agreed as a migrate_drupal patch which is beta this should be backported to 8.5.x so it's in 8.5.0.

diff --git a/core/modules/file/src/Plugin/migrate/field/d6/ImageField.php b/core/modules/file/src/Plugin/migrate/field/d6/ImageField.php
index 07b0d3fd20..c05d8c3665 100644
--- a/core/modules/file/src/Plugin/migrate/field/d6/ImageField.php
+++ b/core/modules/file/src/Plugin/migrate/field/d6/ImageField.php
@@ -14,4 +14,4 @@
  *
  * @see https://www.drupal.org/node/2936061
  */
-class ImageField extends NonLegacyImageField{}
+class ImageField extends NonLegacyImageField {}
diff --git a/core/modules/file/src/Plugin/migrate/field/d7/ImageField.php b/core/modules/file/src/Plugin/migrate/field/d7/ImageField.php
index d411a9708b..29da4d2b67 100644
--- a/core/modules/file/src/Plugin/migrate/field/d7/ImageField.php
+++ b/core/modules/file/src/Plugin/migrate/field/d7/ImageField.php
@@ -14,4 +14,4 @@
  *
  * @see https://www.drupal.org/node/2936061
  */
-class ImageField extends NonLegacyImageField{}
+class ImageField extends NonLegacyImageField {}

Fixed coding standards on commit.

  • alexpott committed 544740d on 8.6.x
    Issue #2934145 by quietone, heddn, Jo Fitzgerald, David Hernández,...

  • alexpott committed 50f4cea on 8.5.x
    Issue #2934145 by quietone, heddn, Jo Fitzgerald, David Hernández,...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.