Problem/Motivation

#2296423: Implement layout plugin type in core and #2779647: Add a workflow component, ui module, and implement it in content moderation are both adding experimental code to core/lib. There have been a number of concerns raised about this on the issues. For example, #2296423-150: Implement layout plugin type in core and #2296423-158: Implement layout plugin type in core

One of the main concerns is that code in core/lib is always available and modules that use it might not necessarily have the same level of warnings to the user as an experimental module. Another important concern is that experiments have to be uninstallable and not break sites. Once code is in core/lib it is tricky to guarantee this.

On the other hand, moving code from an experimental module to core/lib once it is mature means all the contrib / other modules that experimented with it have to be updated.

This is a major task because being able to add experimental APIs to core is a key task to ensure that Drupal 8 can experiment and look to provide as continuous an upgrade path to D9 as possible.

Proposed resolution

Whilst discussing this issue with @catch, @xjm, @dries, @effulgentsia, @cilefen and @webchick I realised that we could use the autoloader to fix this for us. The namespaces of the code would still be \Drupal\Core\NewThing but the location of the files would be core/experimental. In order to enable autoloading of the files you would need to install an experimental module that adds the ability for Drupal to autoload this code.

Remaining tasks

User interface changes

None

API changes

An API for experimental modules to add classes in core/experimental to the autoloader.

Data model changes

None

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

alexpott created an issue. See original summary.

alexpott’s picture

Title: Experimental code in core/lib » Experimental core libraries
alexpott’s picture

Issue summary: View changes
xjm’s picture

We also need an issue to discuss @internal vs. @version (should be a separate issue per @alexpott).

alexpott’s picture

Status: Active » Needs review
FileSize
12.16 KB

Here's a first cut at this. Unfortunately we can't get away without storing the experimental namespaces somewhere that is available to the container super super early. I think the nature of the changes makes this 8.3.x only :( - that said I don't think we need an upgrade path here because whether or not core.extension has an empty list of experimental namespaces or just doesn't have the key should not matter.

Status: Needs review » Needs work

The last submitted patch, 5: 2833347-5.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.26 KB
12.7 KB
alexpott’s picture

Status: Needs review » Needs work

The last submitted patch, 7: 2833347-7.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
2.47 KB
12.76 KB

Status: Needs review » Needs work

The last submitted patch, 10: 2833347-10.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
1.17 KB
13.29 KB
alexpott’s picture

Status: Needs review » Closed (won't fix)

Given #2779647-142: Add a workflow component, ui module, and implement it in content moderation I don't think this idea flies. Basically there's quite a few things that depend on directories like plugin discovery so this is not as simple as it seems.

effulgentsia’s picture

Status: Closed (won't fix) » Needs review
FileSize
3.48 KB

Whilst discussing this issue with @catch, @xjm, @dries, @effulgentsia, @cilefen and @webchick I realised that we could use the autoloader to fix this for us. The namespaces of the code would still be \Drupal\Core\NewThing but the location of the files would be core/experimental.

Oh, I understood our call differently. I thought the location would be somewhere within the module's directory. Here's a patch for what I had in mind. No interdiff, since it's a different approach.

Basically there's quite a few things that depend on directories like plugin discovery so this is not as simple as it seems.

Hm, I haven't looked into that complication yet. Can you add a failing test to demonstrate one or several of those problems?

Status: Needs review » Needs work

The last submitted patch, 14: 2833347-14.patch, failed testing.

claudiu.cristea’s picture

  1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
    @@ -1407,9 +1408,16 @@ protected function getModuleFileNames() {
           $namespaces["Drupal\\$module"] = dirname($filename) . '/src';
    ...
    +          $namespaces[$namespace] = dirname($filename) . '/' . $path;
    

    Nit: We can save dirname($filename) into a variable as it can be used twice.

  2. +++ b/core/modules/system/tests/modules/experimental_module_test/lib/Drupal/Core/Test/ExperimentalTest.php
    @@ -0,0 +1,10 @@
    + * This class exists to test experiment class loading for classes that will be
    + * part of core.
    

    Nit: Should be one line, I think.

  3. +++ b/core/tests/Drupal/KernelTests/Core/DrupalKernel/DrupalExperimentalCoreTest.php
    @@ -0,0 +1,28 @@
    +  public static $modules = ['system'];
    

    We need a different patch for 8.3.x because there should be protected static $modules.

claudiu.cristea’s picture

Status: Needs work » Needs review
FileSize
3.56 KB
3.58 KB
1.69 KB

Rerolled, 8.3.x patch and fixed my own comments.

Fabianx’s picture

+1 to #14, this is much easier and a nicer way to do it. (and the other code approach was not really that nice)

FWIW, it is also possible to keep BC later by having the modules stick around and just have the classes in the module extend the ones from the core/lib later.

Or it is also possible to have core/lib as empty classes that extend from the experimental module classes - though that would fail really hard if you tried to use it without the module being enabled.

--

To the discovery:

Yes, plugin discovery works based on the namespaces registered (which several of us disputed back then), however as long as the namespace in core/lib/foo is similar to core/[foo] this should work correctly as there is no special code to handle modules vs. core.

So I imagine #14 could just work (tm).

Given that its also in module/lib/Drupal/Core also should not interfere with the normal autoloader, which only checks and registers src/.

tim.plunkett’s picture

  1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
    @@ -15,6 +15,7 @@
     use Drupal\Core\Installer\InstallerRedirectTrait;
    
    @@ -1407,9 +1408,17 @@ protected function getModuleFileNames() {
    +          $namespaces[$namespace] = "$module_dir/$path";
    
    +++ b/core/modules/system/tests/modules/experimental_module_test/experimental_module_test.info.yml
    @@ -4,3 +4,5 @@ description: 'Module in the experimental package to test experimental functional
    +  'Drupal\Core\Test': 'lib/Drupal/Core/Test'
    

    This should handle trimming \ and / off.
    For example, this should also work:

    '\Drupal\Core\Test': '/lib/Drupal/Core/Test/'

    Okay maybe not the trailing slash, but definitely the leading one on each

  2. +++ b/core/modules/system/tests/modules/experimental_module_test/experimental_module_test.info.yml
    @@ -4,3 +4,5 @@ description: 'Module in the experimental package to test experimental functional
    +namespaces:
    

    I think this needs a more specific name

Otherwise, this is pretty ingenious.

alexpott’s picture

Status: Needs review » Needs work

I don't think discovery works for configuration entities :( - Let's think about...

Using the workflow patch example. We have an new config entity type of "workflow" - which it was proposed that we should add to core. This means thats the config object name should start with core.workflow.... The core comes from the provider. Let's say we add a module called "workflows" to contain the experimental code that we want to add to core/lib. Using the patch in #19 we can make Drupal\Core\Workflow look in core/modules/workflows/src... which will allow us to have an entity directory containing a file Workflow.php with the namespace \Drupal\Core\Workflow\Entity. So far so good. The good thing is that the provider will be set to "core" because we use \Drupal\Core\Plugin\Discovery\AnnotatedClassDiscovery::getProviderFromNamespace() to determine this. Therefore the config entity will have the correct name - core.workflow. The next problem is that the entity will not be removed when the workflows module is uninstalled. This is fixable too we can add code to the entity's calculateDependencies() method to add an enforced dependency on the "workflows" module. This code would need to be removed and the dependencies fixed once the experimental code is moved to core as stable.

So tldr; I think it would work - but we'd just need extra code in calculateDependencies() - not too bad afaics.

+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -1407,9 +1408,17 @@ protected function getModuleFileNames() {
+    $info_parser = new InfoParser();

This will happen on every request that builds the classloader. We don't read the info files on every request so we shouldn't do this here. Hence me putting this in core.extension earlier. I don't think we can make this change like this.

effulgentsia’s picture

Title: Experimental core libraries » Allow experimental modules to come with \Drupal\Core\ namespaced classes that are only loadable when the module is enabled

Trying a more detailed title. I don't want the title to unduly bias implementation details though. While I prefer #17 to #12, #12 might also be an ok approach, so please interpret "come with" loosely. Further improvements to the issue title are welcome.

effulgentsia’s picture

Title: Allow experimental modules to come with \Drupal\Core\ namespaced classes that are only loadable when the module is enabled » Allow experimental modules to rely on \Drupal\Core\ namespaced classes that are only loadable when the module is enabled

I think "rely on" captures it better than "come with" :)

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.6 was released on August 2, 2017 and is the final full bugfix release for the Drupal 8.3.x series. Drupal 8.3.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.4.0 on October 4, 2017. (Drupal 8.4.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.4.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.4 was released on January 3, 2018 and is the final full bugfix release for the Drupal 8.4.x series. Drupal 8.4.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.5.0 on March 7, 2018. (Drupal 8.5.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.5.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.6 was released on August 1, 2018 and is the final bugfix release for the Drupal 8.5.x series. Drupal 8.5.x will not receive any further development aside from security fixes. Sites should prepare to update to 8.6.0 on September 5, 2018. (Drupal 8.6.0-rc1 is available for testing.)

Bug reports should be targeted against the 8.6.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.8.x-dev

Drupal 8.6.x will not receive any further development aside from security fixes. Bug reports should be targeted against the 8.8.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.9.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.7 was released on June 3, 2020 and is the final full bugfix release for the Drupal 8.8.x series. Drupal 8.8.x will not receive any further development aside from security fixes. Sites should prepare to update to Drupal 8.9.0 or Drupal 9.0.0 for ongoing support.

Bug reports should be targeted against the 8.9.x-dev branch from now on, and new development or disruptive changes should be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.2.x-dev

Drupal 8 is end-of-life as of November 17, 2021. There will not be further changes made to Drupal 8. Bugfixes are now made to the 9.3.x and higher branches only. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.15 was released on June 1st, 2022 and is the final full bugfix release for the Drupal 9.3.x series. Drupal 9.3.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.4.x-dev branch from now on, and new development or disruptive changes should be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.9 was released on December 7, 2022 and is the final full bugfix release for the Drupal 9.4.x series. Drupal 9.4.x will not receive any further development aside from security fixes. Drupal 9 bug reports should be targeted for the 9.5.x-dev branch from now on, and new development or disruptive changes should be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.