Steps to recreate:
Our modules exist here: profiles/[distro-name]/modules
Run drush rr —no-cache-clear
Run drush cc all
I've attached an image of the result.

update.php also triggers this warning - writing an update hook and running the update from the browser will trigger this, as well.

This is where the exception is being caught, and it seems to only occur during the first cache clear: http://cgit.drupalcode.org/drupal/tree/includes/bootstrap.inc?h=7.x#n806

After debugging, it appears that modules from the profiles directory (i.e. profiles/[distro-name]/modules) does not exist in the system db table, so the exception is caught and the warnings appear. After that, the modules exist in the system table and a cache clear runs cleanly.

For example, we have a module that exists called login_disable and this is what happens upon the first cache clear:

I can’t imagine we’re the only ones who place distributions inside of the profiles directory, and according to this it should be an acceptable practice: http://cgit.drupalcode.org/drupal/tree/includes/common.inc?h=7.x#n5506

Hoping for a solution!

Comments

amyvs created an issue. See original summary.

cilefen’s picture

gcb’s picture

I don't believe that the description in that issue matches this issue. The linked documentation of how to fix the problem here says that one of the following is the cause:

1. You removed a module from the file system without disabling and uninstalling it.

This has not happened. Every module that was present on the system is still present, and living in the /profiles/[active-profile-name]/modules/contrib directory.

2. You moved the module inside your Drupal installation

This has not happened. All modules on the system that are not part of core exist inside /profiles/[active-profile-name]/modules

3. There is a bug in a module installed on your site

While this may be true of one or two modules, we are getting the error for every module that is registering as "Not installed" -- which includes lots of submodules that we can't remove from the system, and it does not seem likely that all of those modules have such a bug in them.

Since none of the above situations applies, I do not believe that this is a duplicate issue just because it results in the same error message as the referenced issue.

To be clear, we are using a standard drush make workflow to build sites where we put all our contrib and custom modules inside the active profile's "modules" directory. This is an officially sanctioned place for such modules to exist. However, with Drupal 7.50, we are experiencing a massive flood of inaccurate error messages because something is not checking that (entirely valid) module location.

cilefen’s picture

I did not mark this a duplicate. I referenced a similar issue.

gcb’s picture

Understood, my mistake. I didn't want this issue to get looped in with that implied diagnosis: our use case does not include anything I can identify as improper module configuration.

Sohal Khatwani’s picture

I have the same issue and we use a distro in a similar setup. I posted a comment as well on this problem.

izmeez’s picture

[Edit] Deleted comment. Doesn't fit in this issue.

ericgsmith’s picture

*EDIT*
Although I've attached the patch below, the simplest fix is to add the variable to your settings.php file.

E.g:

$conf['install_profile'] = 'panopoly';

As drush has bootstrapped the configuration this will fix the error.

Original comment

I don't think this is explicitly a bug with Drupal - more so in the use of drush rr.

The command drush rr is set to use the bootstrap phase DRUPAL_BOOTSTRAP_DATABASE - I assume this is by design as if you are needing to rebuild the registry a full bootstrap could cause the command to fail.

This causes issues when registry_rebuild is called, and we eventually get down to the function drupal_system_listing. This is where the module files are scanned. This will call drupal_get_profile to add to the directories to scan - but as we do not have the variables bootstrapped drupal_get_profile will always return 'standard' - hence the profiles directory is not being scanned.

At this point system_update_files_database will delete from your system table any module that is not installed (e.g. has -1 in the schema).

Drush will then bootstrap to DRUSH_BOOTSTRAP_DRUPAL_FULL (after calling registry_rebuild) and clear the cache - so the next call to drupal_get_profile is correctly returning your install profile.

Below might just be relevant to me - but where the error is being thrown for me:

  1. After registry_rebuild has been called (and Drush has fully bootstrapped my site) there is an additional call to system_update_files_database from the features module.
  2. This will collect the correct list of module files from the profile.
  3. Hook_system_info_alter is called.
  4. I have a module using hook_system_info_alter that calls the function module_hook.
  5. module_hook calls module_load_include - which as the modules have not yet been inserted back into the database will raise the error.

This (apart from the ugly errors on rebuild) doesn't affect my functionality as none of these modules are enabled so are not relevant for the hook in place, but I imagine there are various other scenarios where this might be presenting itself.

Unsure if this should be fixed in Drupal.

Anyway, one final thing, kudos to the team for implementing this. As they have said in the main issue - this has alway been happening, its only now just reported. :)

ericgsmith’s picture

Patch attached.

Created against 7.x.

Tested against https://www.drupal.org/project/panopoly latest version with:
- search_api and features enabled (to trigger specific step mentioned above)
- several modules in the profile/modules directly not installed

Again - not sure if this is the best method to tackle this problem, but hoping to get conversation going.

Status: Needs review » Needs work
amyvs’s picture

I like the way you think, @ericgsmith - thank you for taking the time to look at this issue, and drum up possible solutions! I applied your patch, and it did the trick for me. I prefer your patch to setting a $conf['install_profile'] variable to my settings.php file - it feels like a hammer approach to an issue that ought to be managed in Drupal Core. Curious to hear others thoughts on this...

Agreed, kudos to the team for bringing a piece of broken code to light - it's a step in the right direction!

mpotter’s picture

Version: 7.5 » 7.x-dev
Status: Needs work » Needs review

Not sure what the CI error was. Maybe it was because the Version for this issue was set wrong. Let's see if we can get this to pass the testbot again.

Status: Needs review » Needs work
David_Rothstein’s picture

The changes in the patch look pretty reasonable to me.

However, if these steps are required to reproduce:

4. I have a module using hook_system_info_alter that calls the function module_hook.
5. module_hook calls module_load_include - which as the modules have not yet been inserted back into the database will raise the error.

Then it's possible this would also be fixed by some other issues, namely:
#2838075: search_api_system_info_alter() can trigger "module is missing from the file system" in Drush-based module installation - Search API
#2828605: feeds_system_info_alter() can triggers "The following module has moved within the file system" - Feeds
#1948702: module_hook is broken for disabled modules and included hooks - the root issue in Drupal core

fgjohnson’s picture

Patch #9 seems to have resolved my issues re disabled modules in the profiles folder.

https://www.drupal.org/files/issues/drupal-use-db-when-variables-not-boo...

Pol’s picture

Status: Needs work » Reviewed & tested by the community

Hi,

This patch solved the issue for me too.

Thanks.

Pol’s picture

Issue tags: +Drupal 7.60 target

Maybe this patch could be a good candidate for Drupal 7.60. To be checked with Fabianx or Stefanr.

Tagging the issue as such so they will review it and decide what to do with it.

David_Rothstein’s picture

Status: Reviewed & tested by the community » Needs review
Issue tags: -Drupal 7.60 target

I looked at this a little more closely and am not convinced this should be fixed in core (especially given the other fixes mentioned in #14).

If drush rr is running code that needs to access variables, why is it only bootstrapping to DRUPAL_BOOTSTRAP_DATABASE? It seems like it should bootstrap to DRUPAL_BOOTSTRAP_VARIABLES instead, or if it can't do that directly, then put in place a workaround to deal with the problem. It's not clear to me why the workaround should be in core if the problem itself is in drush rr.

That said, if a fix for this does need to go in core, I don't yet see any reason why it would have to wait for a Drupal 7.60 release.

ericgsmith’s picture

http://docs.drush.org/en/master/bootstrap/?highlight=bootstrap

Drush does not map 1:1 to Drupal 7's bootstrap phases.

I'm also of the opinion that the fix is probably more appropriate outside of core in drush rr.