Scenario
Migrating a D7 list text field to a D8 list text field. The field is a select field so only one value can be selected. Some of the D7 list fields values were never set (it was not required). In D8, the field is now required so I want to set the default value during the migration for any of those fields that never had a value on the D7 site.
The list text field has three possible values ("top", "bottom" and "center") and we want "center" to be the default if nothing was ever set on the D7 site. Here is the migration config for the process plugin.
field_hero_image_position:
plugin: default_value
default_value: center
source: field_banner_image_position
Expected Behavior
The expected behavior is that the migration will migrate over the content from the D7 field to the D8 field if it is set. However, in the event that it was not set, the default_value plugin should be run in which case it would determine the value to be empty and therefore use center as the default value.
How to Reproduce
Migrate a D7 list text field with unset values (I am not sure if this applies to any other field type) to a D8 list text field and try to apply a default value during the migration.
If you remove the source line from the migration configuration. The default_value plugin works as expected and the default is set on the field.
Why it's happening (see code snippets below)
1. MigrateExecutable.php calls the processRow method for each row. This loops through each field and plugin. In this case we have one field but it sets the field as having two plugins (Get and DefaultValue).
1. The Get process plugin runs first, and tries to grab the value but there is no value set, so it sets $value to an empty array.
2. The process then sets the $multiple variable to TRUE. This is important because it controls what happens in the next iteration of the loop.
3. On the second iteration of the loop, the code falls into the first if statement since $multiple variable is now TRUE.
4. Since $value is an empty array, the process skips over the foreach and the DefaultValue process plugin is never called.
5. This causes the default value of any fields that were not set on the D7 site to remain null/empty on the D8 site.
Here is the relevant code from the processRow method in MigrateExecutable.php. You should be able to follow the steps above and see what is happening.
foreach ($plugins as $plugin) {
$definition = $plugin->getPluginDefinition();
// Many plugins expect a scalar value but the current value of the
// pipeline might be multiple scalars (this is set by the previous
// plugin) and in this case the current value needs to be iterated
// and each scalar separately transformed.
if ($multiple && !$definition['handle_multiples']) {
$new_value = [];
if (!is_array($value)) {
throw new MigrateException(sprintf('Pipeline failed at %s plugin for destination %s: %s received instead of an array,', $plugin->getPluginId(), $destination, $value));
}
$break = FALSE;
foreach ($value as $scalar_value) {
try {
$new_value[] = $plugin->transform($scalar_value, $this, $row, $destination);
}
catch (MigrateSkipProcessException $e) {
$new_value[] = NULL;
$break = TRUE;
}
}
$value = $new_value;
if ($break) {
break;
}
}
else {
try {
$value = $plugin->transform($value, $this, $row, $destination);
}
catch (MigrateSkipProcessException $e) {
$value = NULL;
break;
}
$multiple = $plugin->multiple();
}
}
Here is the relevant code for the Get process plugin. I think pretty much any field will be returned as an array and will therefore set the $multiple variable to TRUE.
public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {
$source = $this->configuration['source'];
$properties = is_string($source) ? [$source] : $source;
$return = [];
foreach ($properties as $property) {
if ($property || (string) $property === '0') {
$is_source = TRUE;
if ($is_source) {
$return[] = $row->getSourceProperty($property);
}
}
else {
$return[] = $value;
}
}
if (is_string($source)) {
$this->multiple = is_array($return[0]); //This is where multiple is set to TRUE
return $return[0];
}
return $return;
}
The Solution
Since I don't completely understand how the Migration process is working, I don't know the best way to fix this without potentially negatively impacting something else.
There may be an alternative solution with a different process plugin. I am also not sure if this is intended behavior but if it is, then I believe this behavior should be documented more clearly.
| Comment | File | Size | Author |
|---|---|---|---|
| #19 | interdiff-3027998-17-19.txt | 3.9 KB | jofitz |
| #19 | 3027998-19.patch | 4.39 KB | jofitz |
| #17 | interdiff-3027998-14-17.txt | 6.38 KB | jofitz |
| #17 | 3027998-17.patch | 3.36 KB | jofitz |
| #14 | interdiff-3027998-11-14.txt | 4.78 KB | robpowell |
Comments
Comment #2
codekarate commentedI was able to get this to work by updating the annotation in the
DefaultValue.phpprocess plugin to add the linehandle_multiples = TRUE.To
This does not seem to effect any of the other migrations I have running. It seems like it shouldn't cause many issues as the Default Value plugin simply checks if a value is empty or isset, both of which should work if an array is passed into the function.
Comment #3
codekarate commentedAttaching a patch of the previous change to set the Default Value process plugin to be able to handle multiples.
Comment #4
codekarate commentedComment #5
codekarate commentedComment #6
heddnNeeds work because needs tests. This should be fairly simple to test though.
Comment #7
robpowellLet's see if this works. I based this off the following plugins that also handle_multiples:
Comment #8
robpowellComment #10
quietone commentedAdd related issue
Comment #11
robpowellLet's try that again, this time we will set the @coversDefaultClass annotation :).
Comment #12
robpowellComment #14
robpowellAfter reading this comment in the doc block,
I decided to make sure we test each of those conditions. Latest update also includes name changes to be more clear on what each test is asserting.
Comment #15
heddnLet's rework those tests into a data provider: https://phpunit.readthedocs.io/en/8.4/writing-tests-for-phpunit.html#dat....
Comment #16
jofitzI'll have a go at this (although I may be a little rusty!)
Comment #17
jofitzReworked tests into a data provider.
I've opted for a simpler test function and a larger data provider array. The data provider could be smaller with the use of a couple of conditions and default values.
Comment #18
heddnNit: can we swap the expected_value and value variables? So they match up with
assertSame's order.This comment seems off given the data provider.
We also seem to be missing a few strict scenarios. I see strict zero and null, but nothing else.
Comment #19
jofitzComment #20
heddnNice work here. This is one of those examples where the tests are more work then the fix. But now we can be sure this won't surface again. Thanks everyone.
Comment #22
heddnComment #23
alexpottCommitted and pushed 11c46fc2ac to 9.0.x and 1f0cd959e3 to 8.9.x and de971ea5f3 to 8.8.x. Thanks!
backported to 8.8.x because the fix is not disruptive and nice for anyone doing migrations.