Problem/Motivation

In the StaticMap process plugin, it's documented that mapping NULL is supported: https://git.drupalcode.org/project/drupal/-/blob/11.x/core/modules/migra...

 * A NULL can be mapped. If the value of the source property 'foo' is NULL then
 * the value of the destination property bar will be 'to'.
 *
 * @code
 * process:
 *   bar:
 *     plugin: static_map
 *     source: foo
 *     map:
 *       NULL: to
 * @endcode

Doing this causes a YAML parse error

Non-string keys are not supported. Quote your evaluable mapping keys instead at line 57 (near "NULL: 0").

If you quote the NULL string then you don't get the parse error, but that then maps the literal string "NULL" not the NULL value so it doesn't have the desired effect.

Mapping NULL is required (in my case) to map a value when a field is empty.

Steps to reproduce

Either remove documentation for the support, or come up with a solution

Proposed resolution

Due to the fact that PHP converts NULLs into empty string in array keys the static map class only currently maps a NULL if an empty string map is present. We should deprecate this and only support NULL values in the static map if bypass or default_value is set.

Remaining tasks

None

User interface changes

None

API changes

If a static map process plugin is configured to map NULLs using an empty string in the map this will trigger a deprecation.

Data model changes

None.

Release notes snippet

N/a - I think the CR in the deprecation covers this if you hit it.

Issue fork drupal-3458923

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

acbramley created an issue. See original summary.

miguelarber’s picture

I confirm this issue exists, as I needed to map a value when one field was empty (NULL), but doing so causes a YAML parse error (as described in the ticket).

alexpott’s picture

Priority: Normal » Critical
Issue tags: +PHP 8.5
Related issues: +#3546535: [meta] Deal with NULL as array key/offset

This is now even more complex because NULL array keys are deprecated in PHP 8.5 - see #3546535: [meta] Deal with NULL as array key/offset. So we need to consider the whole approach here. I guess one thing that this bug's existence means is that the feature is not going to be used so maybe that helps us...

Not sure what to do here.

longwave’s picture

You can use the default_value plugin to handle the case where the source value is NULL, so perhaps we just mention this in the static_map docs? Or just remove this from the docs given that it doesn't work anyway.

alexpott’s picture

@longwave the problem is that we have tests using NULL as an array key though - see \Drupal\Tests\migrate\Unit\process\StaticMapTest::testWithNullSource - so someone one might be using this programatically.

alexpott’s picture

But I think this is the way forward... we deprecate support for NULL in the static map and tell people to use the default value plugin instead. For PHP 8.5 we allow the deprecation to be triggered because it is a legacy code path.

One interesting thing here is that potentially people have worked around this by using an empty string.

 process:
   bar:
     plugin: static_map
     source: foo
     map:
       '': to

Would map to a NULL value because PHP has been converting NULLs to empty strings for us in the background...

Drush Site-Install (Drupal 11.3-dev)
> $a = [NULL => 'test'];
= [ …1]

> $a;
= [
    "" => "test",
  ]

alexpott’s picture

Added test coverage to prove the behaviour in #6

I think what we should do is deprecate NULL maps in StaticMap - you can use the default_value functionality and it is dangerous because NULL and empty string are the same.

alexpott’s picture

The real problem here is that using array keys in a map is problematic because there are quite a few data types they can not be. We also see this with the instructions for booleans...

 * If your source data contains booleans, the boolean is treated as a numeric 0
 * or 1. If the value of the source property 'foo' is TRUE then the value of the
 * destination property bar will be 'bar'. And if the value of the source
 * property 'foo' is FALSE then the value of the destination property bar will
 * be 'bar'.
 *
 * @code
 * process:
 *   bar:
 *     plugin: static_map
 *     source: foo
 *     map:
 *       0: foo
 *       1: bar
 * @endcode
catch’s picture

I think what we should do is deprecate NULL maps in StaticMap - you can use the default_value functionality and it is dangerous because NULL and empty string are the same.

This sounds fine and we can open a follow-up for anything else.

alexpott’s picture

Yeah I think this is the way to go - but we also need to deprecate empty string in static maps ... as we can't differentiate between the two.

catch’s picture

That also seems OK, if the only use case is covered by the default_value case we're just reducing the number of ways to do the same thing.

alexpott’s picture

I've got a new idea here... let's leave the capability to map empty strings alone.

We can deprecate getting NULL values into a static map without a default_value setting and tell people to add a default value process plugin before a static map. That way people can have pipelines that can differentiate between NULLs and empty strings and we have way less complex deprecation work to do.

andypost’s picture

Status: Active » Needs work
alexpott’s picture

Status: Needs work » Needs review
smustgrave’s picture

Status: Needs review » Needs work

awesome to see 8.5 passing! Congratulations.

Only moving to NW as there appears to be failing functional and kernel tests.

Should this be a 11.3 priority to get 8.5 passing going forward?

alexpott’s picture

Status: Needs work » Needs review
alexpott’s picture

Issue summary: View changes
alexpott’s picture

Green and I've updated the issue summary and CR.

smustgrave’s picture

Quick response! I’ll get to it in the morning if no one beats me to it

smustgrave’s picture

Was the gitlab change intended for this issue?

alexpott’s picture

Yes - because we don't want to regress on PHPUnit tests on PHP 8.5 now that they are green.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

okay, then I believe this one is ready to go.

andypost’s picture

+1 RTBC, nice trick with bypass!

quietone’s picture

Issue tags: +Needs followup

A fair bit of work went into adding examples to all the process plugins. This removes the example without adding a replacement. This needs a followup to add the api documentation.

alexpott’s picture

@quietone there is no replacement. The example never worked and there are examples for bypass and default_value already. Which is what you'll need to use to handle NULLs without skipping.

alexpott’s picture

Issue tags: -Needs followup

Removing tag due to #25

catch’s picture

Version: 11.x-dev » 11.3.x-dev
Status: Reviewed & tested by the community » Fixed

Committed/pushed to 11.x and cherry-picked to 11.3.x, thanks!

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

  • catch committed 32f4fde4 on 11.3.x
    fix: #3458923 StaticMap documents that NULL is a supported map value but...

  • catch committed ecec7783 on 11.x
    fix: #3458923 StaticMap documents that NULL is a supported map value but...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.