Allow component namespaces to be autoloaded independently

Here's a POC. It only moves Annotation to a src/ directory, but illustrates that autoloading already works because each component already specifies autoload. :-)

I hadn't remembered that for some reason, so this issue could probably use some re-scoping.

Added #2943856: [meta] Reorganize Components so they are testable in isolation, mostly so I could note that we need to move Transliteration/src/data to tests/data when the time comes, and change the tests to use it.

This is all the components moved to src/ with updated composer.json. Also updating core/composer.json to reflect the components' new-found self-naming agency.

Let's hurry up and get rid of PHP 5 please. :-)

One problem is that the regular runtime autoloader should not be able to autoload tests

Correct. :-)

When we remove the Drupal\Components\ namespace from core/composer.json's PSR-4 section, that lets the components define their own PSR-4 namespace.

Try it: Apply the patch in #6 (or really #10, either way). Remove an autoload section from one of the component's composer.json. Do a composer dumpautoload. Then run the tests: ./vendor/bin/phpunit -c core/ core/tests/Drupal/Tests/Component/ You'll see that the autoloading for that component isn't available.

I actually think we could merge this issue into #2943856: [meta] Reorganize Components so they are testable in isolation

I think we want the tests to remain where they are so we can be consistent with the move here, and then move the tests as a separate step.

+++ b/core/lib/Drupal/Component/Annotation/composer.json
@@ -15,6 +15,9 @@
       "Drupal\\Component\\Annotation\\": ""
...
+    "exclude-from-classmap": [
+      "/Tests/"

Given that each component can define its own PSR-4, we can leave the classes at the top level directory, or put the source in a src/ directory. If we put it in a src/ directory, we don't have to exclude-from-classmap for tests later on.

We are inside core/lib, but core/composer.json says this:

    "autoload": {
        "psr-4": {
            "Drupal\\Core\\": "lib/Drupal/Core",
            "Drupal\\Component\\": "lib/Drupal/Component",
            "Drupal\\Driver\\": "../drivers/lib/Drupal/Driver"
        },

So the only PSR-4 autoloading for core is lib/Drupal/Core/ after we remove the entry for lib/Drupal/Component.

Either way is fine, just less hassle later with src/ if we're going to move tests around, since we'll be able to rely on the tests in this issue.

+++ b/core/composer.json
@@ -192,13 +192,10 @@
-            "lib/Drupal/Component/Utility/Timer.php",
-            "lib/Drupal/Component/Utility/Unicode.php",

+++ b/core/lib/Drupal/Component/Annotation/composer.json
@@ -15,6 +15,9 @@
+    "exclude-from-classmap": [
+      "/Tests/"

If we're not going to use src/ then we don't really need to add an exclude for a non-existent directory. In that case we don't really need to change any of the composer.json files other than core.

The only change we might need to a component compser.json would be to add Timer and Unicode to the classmap in Utility's composer.json if we think that's worth the performance improvement.

I'd give a little pushback on removing the classmap lines from core and putting them in Utility, but it's not a problem until it becomes one at some point. :-)

Testbot 'fail' due to Nightwatch problems: https://dispatcher.drupalci.org/job/drupal_patches/58895/artifact/jenkin...

#2973992: Permission issue in Nightwatch step marks all full testruns as unstable

Re-setting RTBC now that the permission issue is fixed.

How about core/composer.json has components in both require and replace?

I just did this and then did composer run-script drupal-phpunit-upgrade (which rebuilds the autoloader).

Then vendor/composer/autoload_psr4.php contains this:

    'Drupal\\Core\\' => array($baseDir . '/core/lib/Drupal/Core'),
    'Drupal\\Component\\Uuid\\' => array($baseDir . '/core/lib/Drupal/Component/Uuid'),
    'Drupal\\Component\\Utility\\' => array($baseDir . '/core/lib/Drupal/Component/Utility'),
    'Drupal\\Component\\Transliteration\\' => array($baseDir . '/core/lib/Drupal/Component/Transliteration'),
    'Drupal\\Component\\Serialization\\' => array($baseDir . '/core/lib/Drupal/Component/Serialization'),
[..etc..]

And I can run tests:

$ ./vendor/bin/phpunit -c core/ core/tests/Drupal/Tests/Component/

[..]

OK, but incomplete, skipped, or risky tests!
Tests: 9286, Assertions: 11224, Skipped: 2147.

Well it didn't break anything is what I'm saying. And then when someone tries to use it in drupal-project etc, it will require the thing it replaces, without merging.

We were having a conversation about whether to move the stuff around a bunch or leave components in place here: #2943856: [meta] Reorganize Components so they are testable in isolation

Agreed that somehow we should quit doing the replace part, other than maybe for the benefit of dev. In which case we might replace the components in drupal/drupal. But that's getting way ahead on the composer initiative, and not the scope of this issue.

OK.

Re-upping #16, originally from @alexpott, so not mine.

I RTBCd in #23, so I'm re-RTBCing here.

Moving away from the merge plugin is the goal. But we can't even test the validity of the components' composer.json files until we allow them to define their own autoloading and version constraints. The easiest way to make that work without breaking everything else seems to be merging them.

When we have a way to test them, we can then remove the merge plugin.

The patch in #37 basically reverts #2867960: Merge Component composer.json files to account for them during build which was a previous step along this path. It also does not allow for the components to define their own autoloading.

We can test the validity of the version constraints using TravisCI in #2876669: Fix dependency version requirement declarations in components but we need to be able to prove it using DrupalCI, so we can prevent regressions.

Also re: #32: Using replace in drupal/drupal for dev is kind of the plan now: #2958021: Proposal: Composer Support in Core initiative (We had even talked about renaming drupal/drupal to drupal/drupal-dev.) But yah, that's out of scope here.

See #10 and #11.

Also we can rescope #2876669-55: Fix dependency version requirement declarations in components to be about changing DrupalCI once we've done this and #2943856: [meta] Reorganize Components so they are testable in isolation

+++ b/core/tests/bootstrap.php
@@ -134,6 +134,13 @@ function drupal_phpunit_populate_class_loader() {
+  // Add tests from drupal components.
+  $components = glob(__DIR__ . '/../lib/Drupal/Component/*', GLOB_ONLYDIR);
+  foreach ($components as $componentPath) {

We should make the components test a separate testsuite rather than changing bootstrap.php.

But that's out of scope here... We should only set up autoloading via composer.json in this issue, for consistency against the existing test setup.

Getting a bit out of scope here.

Will #27 work for drupal-project?

When we have either #27 or #34 (really #16) working without changing tests, we can then change the tests around and work on DrupalCI in #2943856: [meta] Reorganize Components so they are testable in isolation which is postponed on the narrow scope of this issue.

Rescoping here a tiny bit in order to support #1826054: [Meta] Expose Drupal Components outside of Drupal and #2982674: [meta] Composer Initiative Phase 1: Add composer build support to core and #2912387: Stop using wikimedia/composer-merge-plugin

Let's use path repositories to allow drupal/drupal to install the components locally.

Then we don't merge/replace the components in drupal/core, and require them instead.

This removes this usage of the merge plugin for this purpose, while still allowing us to exercise the component composer.json files.

Here's the composer update command to save you some typing:

composer update drupal/core-annotation drupal/core-assertion drupal/core-bridge drupal/core-class-finder drupal/core-datetime drupal/core-dependency-injection drupal/core-diff drupal/core-discovery drupal/core-event-dispatcher drupal/core-file-cache drupal/core-filesystem drupal/core-gettext drupal/core-graph drupal/core-http-foundation drupal/core-php-storage drupal/core-plugin drupal/core-proxy-builder drupal/core-render drupal/core-serialization drupal/core-transliteration drupal/core-utility drupal/core-uuid drupal/core-version

The error in #56 was this: Source path "core/lib/Drupal/Component/DateTime" is not found for package drupal/core-datetime

That's because the testbot uses a case-sensitive filesystem, and the directory is actually named Datetime.

Interestingly, if you say composer update on a branch other than 8.8.x, Composer will be unable to resolve the components due to weird version inconsistencies with error reports that look like this: - Installation request for drupal/core-utility 2943842.9999999.9999999.9999999-dev -> satisfiable by drupal/core-utility[2943842.x-dev].

Furthermore, if you were to get them to resolve, the lock file would lock to version 2943842.x-dev, which would not install if you're on branch 8.8.x because self.version resolves to version 8.8.x-dev.

I also discovered that there is a dependency between "post-autoload-dump": "Drupal\\Core\\Composer\\Composer::ensureHtaccess", and one of the components. This means that there are plenty of error conditions where Composer dumps the autoloader before running the plugins, such as the merge plugin that we're still using for drupal/core. That leads to a nice dependency trainwreck. This event script will go away in #2982684: Add a composer scaffolding plugin to core (see: #2990257: Determine how to handle composer scripts in drupal/drupal) We'll need to commit that before we cut the Gordian knot here.

This is the same strategy as #58: Path repos for components in drupal/drupal, and direct require in drupal/core.

Handy composer command for anyone who wishes to copy/paste:

COMPOSER_ROOT_VERSION=8.9.x-dev composer update drupal/core-annotation drupal/core-assertion drupal/core-bridge drupal/core-class-finder drupal/core-datetime drupal/core-dependency-injection drupal/core-diff drupal/core-discovery drupal/core-event-dispatcher drupal/core-file-cache drupal/core-file-security drupal/core-filesystem drupal/core-gettext drupal/core-graph drupal/core-http-foundation drupal/core-php-storage drupal/core-plugin drupal/core-proxy-builder drupal/core-render drupal/core-serialization drupal/core-transliteration drupal/core-utility drupal/core-uuid drupal/core-version

#3086644: LegacyProject composer templates wrongly reference 8.x + fix test coverage is RTBC, so it's likely that it will make it in first. That means we'll have to change that test here, to include more path repos in our build tests.

Tried to re-roll this but I ran into dependency issues based on #3272110: Drupal 9 and 10's Drupal\Component composer.json files are totally out of date, so I'm marking this as postponed on that one.

#3272110: Drupal 9 and 10's Drupal\Component composer.json files are totally out of date is in, so here’s this.

This patch changes the drupal/drupal root composer.json to require drupal/core 11.x-dev, but it’s the only way I could get it to work. This might imply that we need to specify the major/minor dev branch for each branch, which could be a drag.