Problem/Motivation

Right now properties defined in annotations are discarding default values that are set to NULL. In other representations of data (PHP, YAML) we support NULLs. There's no reason not to support NULL while Doctrine handles NULL values.

Proposed resolution

Remove NULL filtering from \Drupal\Component\Annotation\Plugin to allow NULL in annotation default values. Supported in #12

Remaining tasks

Review
Commit

User interface changes

None.

API changes

Developers are able to assign NULL to annotation defaults.

Data model changes

NULL available in annotation default values.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because forbids the using of NULL in annotation defaults even if Doctrine permits that.
Issue priority Not critical because PHP is not strictly typed.
Disruption No disruption

Comments

claudiu.cristea’s picture

Category: feature » bug

Yes, we need to be consistent across PHP <=> YAML <=> Annotation. While NULL is supported by PHP and YAML we need it also in annotations.

BTW, what is the reason for not supporting NULLs?

And this is not a "feature request", it's a bug.

claudiu.cristea’s picture

I just tested NULL values in ImageItem plugin and it work. Looked also in Doctrine and they seems to handle NULLs.

I would close this as "works as designed". What you think @Xano?

xano’s picture

My initial confusion came from the conversion from annotation objects to arrays, which is a different way of accessing data, and where type hinting through code comments gets lost. The fact that any annotation property that ends up being NULL is discarded by the parser and does not end up in the final definitions array was extra confusing, especially because it's something that was explicitly defined beforehand and does end up in the array structure once there is a value.

However, both things are indeed by design, and are a part of Doctrine\Annotations and not Drupal anyway, so this issue may well be a won't fix. However, I'd still like to hear opinions from other Drupal developers.

xano’s picture

Bump.

dawehner’s picture

Issue summary: View changes

However, both things are indeed by design, and are a part of Doctrine\Annotations and not Drupal anyway, so this issue may well be a won't fix. However, I'd still like to hear opinions from other Drupal developers.

I cannot imagine that this is a real problem, if you need something like that use '' or FALSE. Php is not strictly typed anyway.

As a process I would recommend to support it in Doctrine/Annotations and if they don't accept it, just live with it.

xano’s picture

Status: Active » Needs review
StatusFileSize
new840 bytes

This hasn't got anything to do with Doctrine, but with Drupal filtering out NULL values.

Strict typing isn't the issue. We can use FALSE, but that may be misleading. The real question is why NULL values are filtered out in the first place. There is no reason, but it makes using optional definition values more verbose.

Status: Needs review » Needs work

The last submitted patch, 6: drupal_1980556_6.patch, failed testing.

xano’s picture

Status: Needs work » Needs review
StatusFileSize
new820 bytes

So there is one key we can ditch...

Xano queued 8: drupal_1980556_8.patch for re-testing.

xano’s picture

Xano queued 8: drupal_1980556_8.patch for re-testing.

tim.plunkett’s picture

Issue tags: +Needs tests

IIRC, we filtered that out in order to keep the EntityType array cleaner, since some keys were only for each kind of entity type, before we had the ConfigEntity/ContentEntity split.
See #1967294: Add a dedicated @EntityType annotation.

So, I think we can remove that filtering, but we should add test coverage to prove it is possible.

xano’s picture

Issue tags: -Needs tests
StatusFileSize
new2.24 KB
new3.04 KB

Here's a test. Since there was no unit test for the class at all, I took the liberty of also adding coverage for the class's other methods.

tim.plunkett’s picture

Just skimming the test, it doesn't seem to cover the new change. It should be more clear with a NULL value, as well as ensuring 'definition' isn't retained.

xano’s picture

StatusFileSize
new2.87 KB
new11.33 KB

Thanks for pointing out! That is a stupid mistake indeed.

I added a check for NULL values, and random annotation properties, as well as a check for the non-existence of the definition property itself.

xano’s picture

StatusFileSize
new3.67 KB

This is going well...

I forgot to rebase before rolling the patch. Here is a significantly smaller patch made after the rebase.

The last submitted patch, 15: drupal_1980556_15.patch, failed testing.

Xano queued 16: drupal_1980556_16.patch for re-testing.

Xano queued 16: drupal_1980556_16.patch for re-testing.

tim.plunkett’s picture

+++ b/core/tests/Drupal/Tests/Component/Annotation/PluginTest.php
@@ -0,0 +1,107 @@
+  public function testConstruct() {
+    $this->sut = new Plugin($this->annotationValue);
+  }

This is considered a risky test since it has no assertions.

xano’s picture

Do you have an alternative way of testing in mind? The other testing methods provide actual assertions, so there is code coverage for everything. Shall I remove this method and add @covers ::__construct to the other test methods?

claudiu.cristea’s picture

StatusFileSize
new3.54 KB
new571 bytes

Why a test only for the constructor while an object is constructed in ::setUp() meaning is tested in each test. I think adding the annotation to the first test it's enough.

xano’s picture

StatusFileSize
new1.25 KB
new3.71 KB

Let's make sure we have annotations for all methods then.

claudiu.cristea’s picture

Issue summary: View changes

Added IS and Beta Evaluation.

Xano queued 23: drupal_1980556_23.patch for re-testing.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Status: Needs review » Needs work

The last submitted patch, 23: drupal_1980556_23.patch, failed testing.

The last submitted patch, 23: drupal_1980556_23.patch, failed testing.

claudiu.cristea’s picture

Version: 8.1.x-dev » 8.2.x-dev
Status: Needs work » Needs review
StatusFileSize
new3.63 KB

Rerolled, removed @file from test, moved to 8.2.x.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

quietone’s picture

StatusFileSize
new3.43 KB
new1.48 KB
new3.43 KB

Rerolling the patch.

Status: Needs review » Needs work

The last submitted patch, 38: 1980556-38.patch, failed testing. View results

ridhimaabrol24’s picture

Status: Needs work » Needs review
StatusFileSize
new3.08 KB
new1.24 KB

Fixing failed tests.

quietone’s picture

Status: Needs review » Needs work

@ridhimaabrol24, thank you for working on this to fix the failing test. I see that the change is adding the 'minimum_version' property to the migrate source plugins. Is that what we want to do?

I think this needs work to investigate the 'minimum_version' property of migrate source plugins.

Edit: s/field/source

quietone’s picture

Doing some digging.

I'm not familiar with the minimum_version property of migrate source plugins so I grepped:

$ grep -r -e minimum_version core
core/tests/Drupal/Tests/Core/Database/Driver/mysql/install/TasksTest.php:  public function testNameAndMinimumVersion(bool $is_mariadb, string $expected_name, string $expected_minimum_version): void {
core/tests/Drupal/Tests/Core/Database/Driver/mysql/install/TasksTest.php:    $minimum_versio  = $tasks->minimumVersion();
core/tests/Drupal/Tests/Core/Database/Driver/mysql/install/TasksTest.php:    $this->assertSame($expected_minimum_version, $minimum_version);
core/modules/migrate/src/Annotation/MigrateSource.php:  public $minimum_version;
core/lib/Drupal/Core/Database/Driver/mysql/Install/Tasks.php:        $this->fail(t("The MySQLnd driver version %version is less than the minimum required version. Upgrade to MySQLnd version %mysqlnd_minimum_version or up, or alternatively switch mysql drivers to libmysqlclient version %libmysqlclient_minimum_version or up.", ['%version' => $version, '%mysqlnd_minimum_version' => self::MYSQLND_MINIMUM_VERSION, '%libmysqlclient_minimum_version' => self::LIBMYSQLCLIENT_MINIMUM_VERSION]));
core/lib/Drupal/Core/Database/Driver/mysql/Install/Tasks.php:        $this->fail(t("The libmysqlclient driver version %version is less than the minimum required version. Upgrade to libmysqlclient version %libmysqlclient_minimum_version or up, or alternatively switch mysql drivers to MySQLnd version %mysqlnd_minimum_version or up.", ['%version' => $version, '%libmysqlclient_minimum_version' => self::LIBMYSQLCLIENT_MINIMUM_VERSION, '%mysqlnd_minimum_version' => self::MYSQLND_MINIMUM_VERSION]));
core/lib/Drupal/Core/Database/Install/Tasks.php:      $this->fail(t("The database server version %version is less than the minimum required version %minimum_version.", ['%version' => Database::getConnection()->version(), '%minimum_version' => $this->minimumVersion()]));

That's a bit noisy with returns related to the Database system and not a plugin property. The line of note is core/modules/migrate/src/Annotation/MigrateSource.php: public $minimum_version;

I know that checkRequirements does a schema version check so I looked at DrupalSqlBase, grepping for 'minimum_schema_version'.

$ grep -r minimum_schema_version core
core/modules/migrate_drupal/src/Plugin/migrate/source/DrupalSqlBase.php:          if (isset($this->pluginDefinition['minimum_schema_version']) && !$this->getModuleSchemaVersion($this->pluginDefinition['source_module']) < $this->pluginDefinition['minimum_schema_version']) {
core/modules/migrate_drupal/src/Plugin/migrate/source/DrupalSqlBase.php:            throw new RequirementsException('Required minimum schema version ' . $this->pluginDefinition['minimum_schema_version'], ['minimum_schema_version' => $this->pluginDefinition['minimum_schema_version']]);

So MigrateSource defines $minimum_version, as mixed, which is not used anywhere in migrate and DrupalSqlBase uses $minimum_schema_version. The docs for $minimum_version say that migrate drupal source plugins expect minimum_version to be an integer representing the minimum installed database schema for the module. But then DrupalSqlBase tests with $minimum_schema_version, a value that is not defined or set anywhere.

That suggest to me that DrupalSqlBase should be testing with $minimum_version not $minimum_schema_version. For me, that is a bug and DrupalSQlBase should use $minimum_version.

Now, as regarding this issue is it OK to set $minimum_version to NULL on all source plugins? The short answer it yes in migrate. The annotation set the type as mixed. So that is fine. Now, for DRupalSqlBase. It is currently isn't using $minimum_schema so that won't matter. And once DrupalSqlBase is fixed it will be using $minimum_version and an isset is done before doing any comparison with it. Again, that is fine.

quietone’s picture

Status: Needs work » Needs review
Issue tags: +Bug Smash Initiative
StatusFileSize
new1.38 KB
new3.08 KB

Fixing coding standards errors.

quietone’s picture

Created a follow up in migration for the problem described in #42, #3151732: DrupalSqlBase::checkRequirements should test version with $minimum_version.

tanubansal’s picture

Tested via adding patch #43 : Remove NULL filtering from \Drupal\Component\Annotation\Plugin to allow NULL in annotation default values

Changes are visible and this can be moved to RTBC

quietone’s picture

@tanubansal, thank you for interest in moving this patch along and contributing to Drupal. Please read the issue to see if screenshots are needed before adding them. Screenshots are typically only for issues that alter the UI.

As I mentioned in #3159849: Add missing tests of filepath to FileTest Comment #5 there is a wealth of information on Drupal about contributing.

tanubansal’s picture

@quietone : thanks ... i will take care in future

quietone’s picture

@tanubansal, thank you! If I can be of any help I can be in #migration on Drupal Slack.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

quietone’s picture

Issue summary: View changes
StatusFileSize
new993 bytes
new3.09 KB

Needed a reroll

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Needs Review Queue Initiative

This issue is being reviewed by the kind folks in Slack, #needs-review-queue-initiative. We are working to keep the size of Needs Review queue [2700+ issues] to around 400 (1 month or less), following Review a patch or merge request as a guide.

Didn't see a test patch so running locally

d6/FieldDiscoveryTest

Failed asserting that two arrays are identical.

d7/FieldDiscoveryTest

Failed asserting that two arrays are identical.

PluginTest actually passes. (not sure an issue)

Reviewing the fix

-    // Only keep actual default values by ignoring NULL values.
-    $defaults = array_filter($reflection->getDefaultProperties(), function ($value) {
-      return $value !== NULL;
-    });
+    $defaults = $reflection->getDefaultProperties();
+    unset($defaults['definition']);

Appears to be doing what the IS states.

Looks good to me.

tim.plunkett’s picture

Status: Reviewed & tested by the community » Needs work

PluginTest failing without the fix is a blocker IMO. The other two are test tweaks needed to adapt to this change, but the unit test for this code should absolutely cover this.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.