Extensions in Multisite Directories Not Registered When Rebuilding Cache

This issue is somewhat related.

I've encountered this problem with my multisite environments and testing this patch I can successfully `drush cr` with Drupal finding and registering modules and classes in the site-specific code.

The fix looks fine, and not sure this is really testable, but there are many other cases where ExtensionDiscovery is constructed without the sitepath argument. Is it worth reviewing those here too?

core/includes/install.inc:  $listing = new ExtensionDiscovery(\Drupal::root());
core/includes/install.inc:  $module_list = (new ExtensionDiscovery($drupal_root))->scan('module');
core/includes/install.core.inc:  $listing = new ExtensionDiscovery($container->get('app.root'));
core/includes/module.inc:  $listing = new ExtensionDiscovery(\Drupal::root());
core/lib/Drupal/Core/Extension/ThemeHandler.php:      $this->extensionDiscovery = new ExtensionDiscovery($this->root);
core/lib/Drupal/Core/Extension/ExtensionDiscovery.php:   * Constructs a new ExtensionDiscovery object.
core/lib/Drupal/Core/Extension/ExtensionDiscovery.php:   * $listing = new ExtensionDiscovery(\Drupal::root());
core/lib/Drupal/Core/Extension/ExtensionList.php:    return new ExtensionDiscovery($this->root);
core/lib/Drupal/Core/Update/UpdateRegistry.php:    $extension_discovery = new ExtensionDiscovery($this->root, FALSE, [], $this->sitePath);
core/lib/Drupal/Core/Config/InstallStorage.php:      $listing = new ExtensionDiscovery(\Drupal::root());
core/lib/Drupal/Core/Config/ExtensionInstallStorage.php:      $listing = new ExtensionDiscovery(\Drupal::root());
core/lib/Drupal/Core/Command/InstallCommand.php:    $listing = new ExtensionDiscovery(getcwd(), FALSE);
core/lib/Drupal/Core/DrupalKernel.php:      $listing = new ExtensionDiscovery($this->root);

Patch #3 fixed the problem for me. Thank you.

If this patch is considered RTBC for the existing case, is there any chance it could be committed and the other instances of ExtensionDiscovery could be dealt with in a separate issue? "drush cr" has been breaking a lot of sites for months now, it would be really nice to get this dealt with ASAP.

+1 for getting this committed ASAP and opening another issue for the other cases where ExtensionDiscovery is constructed.

+1; patch from #3 works

Okay, I'm not 100% confident I know what the process should be, but in the interest of getting this fixed I'm switching it to 8.7.x-dev (in hopes that it makes it into 8.7) and RTBC. If this isn't the right process, apologies to whoever has to fix my mistake upstream.

Given that ExtensionDiscovery is being constructed with a NULL $site_path parameter I think we need to understand why this code:

    // Find the site-specific directory to search. Since we are using this
    // method to discover extensions including profiles, we might be doing this
    // at install time. Therefore Kernel service is not always available, but is
    // preferred.
    if (\Drupal::hasService('kernel')) {
      $searchdirs[static::ORIGIN_SITE] = \Drupal::service('site.path');
    }
    else {
      $searchdirs[static::ORIGIN_SITE] = $this->sitePath ?: DrupalKernel::findSitePath(Request::createFromGlobals());
    }

is failing under drush. Since as @catch points out we construct ExtensionDiscovery often without a site path argument and in all these cases we fall back to this logic. How is drush cr choosing which site to run against?

@alexpott cr does site selection same as other commands do. that is, we use the value of --uri.

1. See https://github.com/drush-ops/drush/blob/40092d5dcf6f850fdf15d817c33602fe...
2. If you are curious what $request is, see https://github.com/drush-ops/drush/blob/226aa64e93528b3f4c0a1fc49fc6bf59...

Patch #3 worked for me. Thanks.

we just started using patch #3 with our D8 multisite and it seems to solve our immediate issue.

before we could continually reproduce the issue where none of our site specific module code was triggering. The only way to remedy the situation (albeit temporarily) was to uninstall and re-install the module.

so far with the patch and a bunch of drush cache-rebuilds later, we haven't had any noticeable issues with our module's code running.

glad we found this!

I guess that $request, passed to drupal_rebuild(), should be used in all places, called from drupal_rebuild().
Otherwise wrong Request::createFromGlobals() would be called.
Patch attached.

I had the same problem described with cache rebuild. I gave a try to #20 and it appears to be working on Drupal 8.6.10 and fixing the cache rebuild issue. Not tested on 8.7.x-dev though.

Composer is not letting me apply the patch to Drupal 8.6.11.

Usually I would change the status to "Needs reroll", but the patch applies to 8.6.x and to 8.6.11:

$ git checkout 8.6.11
Note: checking out '8.6.11'.
HEAD is now at 8cbc77808e Drupal 8.6.11
drupal8x ((8.6.11))$ curl https://www.drupal.org/files/issues/2019-02-04/2985199-20.drupal.Extensions-in-Multisite-Directories-Not-Registered-When-Rebuilding-Cache.patch | git apply
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  3856  100  3856    0     0  50923      0 --:--:-- --:--:-- --:--:-- 51413
drupal8x ((8.6.11) *)$ git status
HEAD detached at 8.6.11
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   core/includes/utility.inc
	modified:   core/lib/Drupal/Core/DrupalKernel.php
	modified:   core/lib/Drupal/Core/PhpStorage/PhpStorageFactory.php
	modified:   core/lib/Drupal/Core/StreamWrapper/PublicStream.php

no changes added to commit (use "git add" and/or "git commit -a")

Looks like it was an error on my Server. Thank you for the help.

The patch in #20 worked for me in a situation where my module's route was not being recognized (leading to a page not found) after a cache clear, more details #3051123: Clearing cache makes my route disappear in multisite -- how to debug this?.

What a find, this has been causing me problems all day!
Thank for those who dug around this one and come up with a solution.
#20 fixes the problem on 8.7.x

For those struggeling to apply the patch (and those use in composer patching packages):
- Place patch in core root directory
- Run with option -p2

This is the sustainable way to patch with Composer: https://github.com/cweagans/composer-patches

I was running 8.7.0-beta2 when I first encountered this issue, updating to the full 8.7.0 release did not fix it.
After applying the patch attached to the issue, I can confirm that it has now fixed my problem.

I highly recommend getting this into core, if we were to update the D8 sites we run to 8.7, the problem outlined by this issue, would make all of them break.

Of course 8.7.0 did not fix it. This issue remains open. Anyone affected: please provide a review. That is what is preventing this being merged. If anyone could think of a way to add an automated test that would be awesome. But according to #9 it may not be possible to do so.

There is an unsolved discussion in #9 and #12, which could perhaps be handled in a new issue that someone could open.

Hello,
I had installed drupal 8.6.15 multisiting which works good with drush8.
Several days ago I decided to update vendor drush to latest 9 version, but discoverd next issue after 'verndor-drush-path'/drush cr for specific site:

In DiscoveryTrait.php line 52:
  The "some-plugin-name" plugin does not exist

Spent a lot of time to find the reason and today discovered this issue.
I installed patch #20 and issue has been fixed. Now drupal works properly with drush9

Looks like I have some sub sub folders for custom entities which hasn't been loaded with old code & drupal couldn't find this entities.
But custom modules has been loaded properly.

Thanks a lot guys!

Can confirm that #20 works for us. D8.7, drush 9 multisite setup.

I tested the patch in #20 in a 8.6.16 multisite setup and got the expected results on PHP 7.2.

Prior, I had moved a module from /modules/custom/ to /modules/contrib/ and got the PHP error "Cannot find blah blah blah..." even after drush cr, composer dump-autoload, and drush ev "drupal_flush_all_caches();". This patch solves the issue for me. Thanks for the solution.

Since multiple people have marked #20 as working, I'm going to change to RTBC for this patch and issue.

Also tested on 8.7.x and 8.8.x and the patch applies cleanly.

May I ask why are we passing the site path to the PHP Storage factory and on to PublicStream:basePath? The latter, as far as I can see, already retrieves the site path if nothing is passed to it. I might be missing something so I am not resetting this to CNW (and I really don't feel specified to set anything to or from RTBC, anyways).

I'm going to move this back to "Needs Review". After some testing, the fix is not reproducible, meaning that I tried double checking this morning and I could not reproduce the fix.

It should also be mentioned that this seems specific to modules that create entities and the APC cache not clearing when moved back.

Pre-patch, I moved a custom module to contrib, cleared cache, and got this error:
Fatal error: require(): Failed opening required '/var/www/docroot/modules/custom/watson_form_parser/src/Entity/WatsonFormEntity.php' (include_path='.:/usr/local/lib/php') in /var/www/vendor/symfony/class-loader/ApcClassLoader.php on line 112

Applied the patch, cleared cache, and returned the module to /modules/custom and things worked, as expected.

Then moved the module to /modules/custom, cleared cache and ran drush ev "drupal_flush_all_caches();" and got the error again. I'm not sure why it worked for me yesterday, or if this is an unrelated issue that just happened to lead me down this rabbit hole, but I'm not the one to mark this RTBC since it's not solving the problem I had.

If anyone knows of a related issue to the problem I'm describing, please link it. I'm also going to do some digging around to see if I can find an issue as well before creating a new one.

I wasn't able to find a d.o issue with some high level googling, but I did find an issue in the Symfony github repo with some comments by a few good people who have a solution for my problem

https://github.com/symfony/symfony/issues/22975

Posting this here in case anyone's search leads them to this issue first for some reason.

@Chx, I think i found a potential answer to your question. I'm going to assume that $site_path in the patch is for installs with multiple webheads. There is a thread somewhat related to my original issue at https://www.drupal.org/project/drupal/issues/2752961 re: twig caches not being cleared on servers with multiple webheads.

KeyboardCowboy may be able to shed a little more light on the subject though.

Rerolled for 8.8.x.

Patch rerolled

Patch #43 doesn't work because of missing adjustments to /core/lib/Drupal/Core/StreamWrapper/PublicStream.php:

PHP Fatal error:  Uncaught TypeError: Argument 1 passed to Drupal\Core\StreamWrapper\PublicStream::basePath() must be an instance of SplString or null, string given, called in /…/core/lib/Drupal/Core/PhpStorage/PhpStorageFactory.php on line 52 and defined in /…/core/lib/Drupal/Core/StreamWrapper/PublicStream.php:102

Tested #20 which works fine so far.

Please use 8.8.x-dev.

Sorry, it's my fault. We're on 8.7.7.

BTW: thank you for providing a solution.

Then, I beieve, this patch is still needs review although it's possible CNW would be better because it doesn't contain a test.

And I just don't see how this patch does anything at all.

It passes the site path from kernel to ExtensionDiscovery however ExtensionDiscovery::scan only uses the sitePath argument if the kernel service doesn't exist, however the fallback in that case is DrupalKernel::findSitePath(Request::createFromGlobals()) -- but that's the exact same as a normal kernel because DrupalKernel::initializeSettings uses static::findSitePath for the path and the $request variable there is set to Request::createFromGlobals() by index.php. What am I missing?

PublicStream::basePath is the exact same: if there's a kernel, retrieve the path from it, otherwise fall back to DrupalKernel::findSitePath(Request::createFromGlobals()).

Do we perhaps instantiate a kernel somewhere with a request that is not Request::createFromGlobals() ? I do not see where, install_begin_request begins with $request = Request::createFromGlobals(); update.php is very similar to index.php

Please tell me where I am wrong.

As for drush, are you running drush -l www.example.com ...?

Patch #20 re-rolled for D8.8.x.

I'm using this successfully in various environments—it would be great to get this into 8.9.

Sorry this still needs some test coverage.

#48 works for me. thanks

Patch #48 re-rolled for D8.9.x.

Could someone please address #47?

I've re-rolled patch for 9.1

Thanks for the patch,
BTW:
Lots of people getting hit with this here:
 #3056633: It is not possible to uninstall a module that provides a filter plugin via config import

I applied the patch for Drupal 8.9.1 but after trying to add a new field into my custom entity I still see this message:

In DiscoveryTrait.php line 53:
The "decimal" plugin does not exist

$fields['money'] = BaseFieldDefinition::create('decimal')
      ->setLabel(t('Payment amount'))
      ->setDescription('')
      ->setDisplayOptions('view', [
        'label' => 'above',
        'type' => 'decimal',
        'weight' => 3,
      ])
      ->setDisplayOptions('form', [
        'type' => 'decimal',
        'weight' => 3,
      ])
      ->setDisplayConfigurable('form', TRUE)
      ->setDisplayConfigurable('view', TRUE);

The weird thing is that I have the same Base Field Definition('decimal') in another custom entity and I don't see the message, this happens only after adding this field.

> Sorry this still needs some test coverage.

I think that's going to be tricky, given multisite doesn't work in tests anyway! See #3078743: PHPUnit testing within multisite setups.

+++ b/core/lib/Drupal/Core/DrupalKernel.php
@@ -456,7 +456,7 @@ public function boot() {
       // @todo Use extension_loaded('apcu') for non-testbot
-      //  https://www.drupal.org/node/2447753.
+      //   https://www.drupal.org/node/2447753.

@@ -723,9 +723,9 @@ public function handle(Request $request, $type = self::MASTER_REQUEST, $catch =
    * @param \Exception $e
-   *   An exception
+   *   An exception.
    * @param \Symfony\Component\HttpFoundation\Request $request
-   *   A Request instance
+   *   A Request instance.

@@ -992,7 +992,6 @@ public static function bootEnvironment($app_root = NULL) {
     // The .htaccess file contains settings that cannot be changed at runtime.
-
     if (PHP_SAPI !== 'cli') {

@@ -1173,7 +1172,7 @@ public function invalidateContainer() {
-   *   Container object
+   *   Container object.

@@ -1220,7 +1219,7 @@ protected function compileContainer() {
-    // - Plugin
+    // - Plugin.

@@ -1353,7 +1352,7 @@ protected function cacheDrupalContainer(array $container_definition) {
-   * Gets a http kernel from the container
+   * Gets a http kernel from the container.

@@ -1487,7 +1486,7 @@ protected static function validateHostnameLength($host) {
-   *   The request object
+   *   The request object.

+++ b/core/lib/Drupal/Core/PhpStorage/PhpStorageFactory.php
@@ -6,7 +6,7 @@
- * Creates a php storage object
+ * Creates a php storage object.

All these changes are out-of-scope of this fix. Adding them here gives reviewers more to think about.

Can I suggest that this priority be bumped up to critical if it could potentially bring more attention to this issue? This same issue just brought our production site down.

I have seen this before and have been tracking it myself (just found THIS issue today) on our DEV site for a few months now. This was happening to me when our deploy (`$ drush updb`) had failures, which I figured would not happen in production. Then all of a sudden, and I'm sure yet why, my prod sites lost their plugin context. There was no code deploys, no drush clears, just all of a sudden happened and boom our sites were down.
I am going to attempt to try out the patches here and can report back how that goes.

Rerolled for D9.1.

Created a MR for this work on 9.2.x. Cleaning up a few items around the issue. This also addresses the feedback from @alexpott in #61.

I'm bumping up the priority to critical, which I think is warranted. This bug has caused my site to become unusable, with no workaround, which is the threshold for a critical bug ticket.
Looking at some of the related issues, this is actually affecting a lot of users, I think it's just VERY hard to put a finger on what is causing this bug. It also, for me, is VERY unpredictable, and that's something I would like to understand more about the situation that can cause this issue.

I saw it was requested that we include a test for this bug. From the description, this bug is only affecting multisites with site specific modules/code, for which unit testing does not exist in Drupal. Does that unit testing need to be created before this patch can go in? Or are there any suggestions how this can be shown via a test?

In my experience, I have ran into this problem randomly on my production site (potentially a cron job ran around the time my site went down), and I have been able to reproduce these symptoms in a dev environment. I wrote about all of this extensively on these two issues:
https://www.drupal.org/project/drupal/issues/3199185
https://github.com/drush-ops/drush/issues/4666
In my case, I think when my $ drush cron --uri=site1 gets run, and my "container cache" is empty (??), after a deploy, for example, where the $settings['deployment_identifier'] changes, somehow the Drush->Drupal bootstrap is creating my container without knowledge of my site specific modules.

Anyways, I know this will probably kick around without a test... but can we address that, even if we could recreate this scenario reliably, a testing method for this set-up does not appear to exist within Drupal core right now? Or maybe it does, but could anyone with more knowledge on the subject suggest how we could possibly add in testing for this?

Maybe this is more about the Kernel running without the site path explicitly set, and Drupal falls back to Request::createFromGlobals(). How does `::createFromGlobals()` work for a DRUSH command, for instance?

Going to bump to RTBC. I pulled in the last patch and it was fixing my problem. I know I posted another version, but it was just removing documentation changes. I'm a bit skeptical of this patch though, that it's not just working around another problem, so hoping that maybe the RTBC status will get more eyes on it...?

Marking #1792310: Wrong DRUPAL_ROOT with non-standard code structure as related, as I am seeing some removal of code that guesses the app root, and that other issue is about removing guessing of the app root altogether.

I am fairly convinced I can recreate this problem now, but I'm not sure this is an appropriate fix... going back to some previous comments, I have no idea how a test gets built for this.

Set up:

Ideally, you would have a piece of code that lives in a multisite, to show that something is broken. In my case, I have a REST RESOURCE plugin (so a `rest.resource` config, plus a `src/Plugin/rest/resource/MyRestResource.php` class) in the web/sites/example/modules/demo_2985199/demo_2985199.info.yml module. Ensure that your local is set up to run this "example" site.

Step 1.

Change the $settings['deployment_identifier'] for your site.

Step 2.

Run $ drush cron --uri=example (where example is your local URI for this multisite).
Obviously this is bringing a DRUSH run into the mix with Drupal core. I guess this doesn't _have to be_ Drush, but the way Drush calls the DrupalKernel I believe is part of what's going wrong here.
At this point, Drush is going to initialize the Kernel, and there will be no "container definition" that exists in the database cache, because the deployment identifier has changed. This call will build a new container, and store the definition in the database, but at this time, it will not save the container with the multisite modules.

Step 3.

Now visit the site in your browser. You must go to a page that uses multisite code, so in my example above, if I go to the URL for my rest resource, my site will now tell me that the plugin cannot be found.

Step 4.

Sanity check... Run $ drush cr --uri=example and go back to that page, which should now work just fine. It appears to me that Drush is calling the Drupal global methods to clear cache, and that seems to supplement the broken container definition and re-save it correctly.

One thing to note... this scenario I lay out is a little different from the OP. I am wondering if previously the "drush cr" worked differently and caused this error...? Maybe this is a different error altogether.

(Also note I am using Drush 10)

@dpagini I wonder if we should be adding the site path to the container key. I.e. change \Drupal\Core\DrupalKernel::getContainerCacheKey() I also wonder if this is all caused by Drush building a container incorrectly when there is a cache miss and not Drupal's core's fault at all.

> I wonder if we should be adding the site path to the container key. I.e. change \Drupal\Core\DrupalKernel::getContainerCacheKey()

How would that affect using symlinks in the sites folder?

For example, suppose a codebase has web/sites/example.com, but locally I symlink web/sites/localhost.example to it.

OT - @joachim: Would it not be better in your setup to map localhost.example via array in sites.php also to example.com? Then you don't need your symlinks.

@alexpott I had the same thought honestly... asked the same question here.

I wonder if we should be adding the site path to the container key. I.e. change \Drupal\Core\DrupalKernel::getContainerCacheKey()

I sort of think that would work, at least for me. In my testing, the drush command would still build a "dirty container" - but it would save it with the `sites/default` cache key. Then the site loads the page, and it would build a valid container. However, in the example I gave, if `$ drush cron` needed the correct container, you would get a failure there.... so that goes back to, could DRUSH be calling the kernel wrong?

How about thoughts on adding a test for this?

Also, is there any guidance anywhere, documentation, on what the proper way is to instantiate the Drupal Kernel? I almost feel like it's difficult to go to the Drush team and say they're doing it wrong if there is no guidance on how it's supposed to be done. Maybe CORE needs to throw an exception if it's being used in a way that's unintended? Just spit balling thoughts....

@dpagini it might not be possible for drush to build the container correctly first time around. Or it might be very tricky. I think it might need to build a container to discover what commands are available. So whilst the solution of building a container for sites/default and then for the site you want to run the cache rebuild against might not be brilliant it will probably work. Re "the correct way to build the container" - well the correct way is so that when \Drupal\Core\DrupalKernel::findSitePath() is called the correct site path is returned.

All this said you comment in #74 clearly points to this being Drush's problem. If you pass enough info into Drush for the container to be built with the correct site path and then it's not then this is not core's problem tbh.

In Drush cache:rebuild command, the container is built by calling \Drupal\Core\DrupalKernel::findSitePath(). See https://github.com/drush-ops/drush/blob/10.x/src/Commands/core/CacheComm.... I cant spot any meaningful differences between that command and drupal core's rebuild.php.

@mosche-weitzman this actually is not necessarily a problem with $ drush cache:rebuild actually. I know the original IS does say that, but at least since I've jumped on this issue, I've had the same symptoms, but found them to be created in a different way...

The problem comes about when NO CONTAINER exists in cache (for example after a deployment, where deployment_id changes), and then a command such as `drush cron --site=example1` runs. In that case, the drush cron bootstrap is building a container for the default site instead of for the example1 site, and it saves that "dirty container" to cache. Then Drupal goes to serve up a page of your site, gets that "dirty container" out of cache, and your site is broken.

So to me, although cache:rebuild may be building the cache identical to Drupal core, it's really other Drush commands, which are bootstraping the DrupalKernal, that is causing the dirty container to be created.

Does that make sense?

Thanks, that makes sense. Yet drush always respects --uri when provided and should bootstrap to the right Drupal site as identified by Drupal's `\Drupal\Core\DrupalKernel::findSitePath()`. So this 'default' container build should not happen when --uri is provided and is valid. Remember that since Drupal 8 a full URI might be needed to convince drupal to select the right site (http://foo.example.com/bar). You should be able to test different --uri quickly by looking at the site path in the output of drush status.

I wonder if this issue is fixed by #2160091: drupal_rebuild() rebuilds container twice, since drupal_flush_all_caches() also rebuilds it this changes a bit how the container is rebuilt when drush clears the cache.

I've had issues on the command line running scripts with the plugin not found, rebuilding cache on the server then on the command line, doesn't seem to do the trick, maybe because either my command line is cgi nts , or the server php is fpm or cgi or nts , not sure, I was playing around with it late last night because I have multiple php versions installed switching around but was not focused on this , maybe will review more seriously later.

for now my workaround is bizarre, log into Drupal after my migration scripts removed all content, then create a few nodes and delete one or two. then my drush (version 10) seems ok to run my command line scripts. lots of factors to consider

usually occurs for me on the command line after reloading an sql dump file.

I should note that although in my example above my call to drush has just `example` - in practice I am sending the FQDN of the site.

I want to point back to some comments I made on the drush queue. I was trying to compare how Drupal's index.php file creates and calls the kernel versus how Drush does, and there are definitely some differences, and I try to outline it there... specifically:

With regards to Drupal sites....

When I look at the Drupal core index.php file, it creates a new DrupalKernel object, and calls ::handle() and passes the request in.
https://github.com/drupal/drupal/blob/485d2a305f78605de55eef8e870d80f51d...

The ::handle() method calls ::initializeSettings() and the SITE PATH gets set on the Kernel.
https://github.com/drupal/drupal/blob/485d2a305f78605de55eef8e870d80f51d...

With regards to Drush...

It appears to me like Drush creates a new Kernel, and then ultimately calls ::boot() and ::preHandle(). when it bootstraps Drupal. However, neither of those paths ever leads to Kernel's ::setSitePath() being called, as with the Drupal example above.

So maybe drush needs to explicitly call ::setSitePath() on the Kernel instead of letting it be calculated? And I think ultimately Drupal partially does end up recognizing which site a drush command should run against... but I think, when the container cache is empty, at the time that Drush ends up rebuilding it, it is not the container for the correct multisite.

Also, I think the steps in #70 should be pretty straight-forward to demonstrate this problem. I would love to have a test that demonstrates the issue, but I don't know how to build a test since this deals with multisite specific code, which core does not really allow for. But if you manually follow those steps I am fairly confident anyone should be able to recreate this bug and see for yourself.

I've confirmed the steps in #70. FWIW I also can not confirm the steps in the issue summary on Drush 10 / Drupal 9 - it appears that that is no longer occurring.

What is really interesting about #70 is once the site is broken then changing the deployment identifier and making a request via the browser is not enough to fix it. You need to clear the cache.

The problematic interaction is between Drush and this code in \Drupal\Core\Extension\ExtensionDiscovery::scan()

    // Find the site-specific directory to search. Since we are using this
    // method to discover extensions including profiles, we might be doing this
    // at install time. Therefore Kernel service is not always available, but is
    // preferred.
    if (\Drupal::hasService('kernel')) {
      $searchdirs[static::ORIGIN_SITE] = \Drupal::getContainer()->getParameter('site.path');
    }
    else {
      $searchdirs[static::ORIGIN_SITE] = $this->sitePath ?: DrupalKernel::findSitePath(Request::createFromGlobals());
    }

Because there is no kernel - because the container is invalid we fallback to Request::createFromGlobals() and because Drush no longer sets up the globals in order to fake the request - this is broken.

https://github.com/drush-ops/drush/pull/4698 fixes the error reported in #70

Given that #70 has been fixed in Drush HEAD and the steps to reproduce in the issue summary are not reproducible I'm going to change the issue priority to normal.

I think that the work to pass the site path into ExtensionDiscovery is in the long term a good thing to do. We should try to remove any reliance on creating a request using ::createFromGlobals() from the system. Also we should be able to write tests to prove that we're no longer relying on Request::createFromGlobals().

I hit this in Drupal 9.2 when running tests via phpunit where the tests are in multisite dir. Attached is a reroll of the original approach in #3 which fixes this for me. I.e. when running tests, hooks/extensions etc in custom modules are correctly registered.

Just to be clear, normal web traffic picks up the custom modules fine, this just affects phpunit. Cache rebuilding with Drush 10.5.0 (which includes the fix from #86, or a previous version of Drush without that fix, or via the UI - all make no difference - when running tests the custom modules are not picked up.

As @alexpott noted in #87 passing the site path to ExtensionDiscovery is a likely a good thing to do. This still needs work for tests.

I've caught this problem today on Multisite after update of the core from 9.0.4 to 9.3.9. In my case the error was absolutely confusing: "The "xxx" plugin does not exist" so to find this issue I had to spend all day debugging to identify the real reason. The error goes from DrupalKernel on the step of retrieving modules data so in fact any multisite configuration with latest core doesn't work correctly with drush less than 10.6.0 (when @alexpott changes went live). And composer does allow you to use any version older than "drush/drush": "<8.1.10" In my opinion this is critical issue. Also I think current title is not representable so I'm going to change it.

Thanks @fenstrat for providing the patch. This one line saved me from more digging.

This is core issue which affects drush

Patch for new MR!9018 against 11.x (which also applies to 10.3)

thanks for reviving. Moving to NW for test coverage

Rerolled for Drupal 11.3.x

The previous patch no longer applies because the code was refactored in Drupal 11.3. The moduleData() method now delegates to a new private method setExtensionData() (introduced around line 821).

The fix remains the same - passing the site path to ExtensionDiscovery:

$listing = new ExtensionDiscovery($this->root, TRUE, NULL, $this->getSitePath());

Tested with Drupal 11.3.1.

This belongs in extension I think.

Haven't reviewed beyond IS and scanning comments.