Problem/Motivation
The \Drupal\Core\Extension\ExtensionDiscovery class documentation implies that you can control whether or not extensions underneath a profile directory will be scanned or not. But this is not actually true, unless you fake it.
There are two ways to set the $profileDirectories member variable explicitly. One is in the constructor, and another is by calling the setProfileDirectories() method. The documentation for the member variable, constructor param, and method say:
- Member variable: List of installation profile directories to additionally scan.
- Method: Sets explicit profile directories to scan. // A list of installation profile directory paths relative to the system root directory (without trailing slash) to search for extensions.
- Constructor: The available profile directories
All of these imply (at least to me) that if you set this to an empty directory, no profile directories should be scanned for modules.
However, what actually happens if you set this to an empty directory is that all profile directories are scanned.
What you can actually do in order to forbid profile directories from being scanned for modules is set it to something like ['foo'], so that it will only scan the non-existent "foo" profile. However, if you do that when you are scanning for profiles themselves, nothing will be returned.
Proposed resolution
Either:
a) Fix the documentation so it matches the behavior.
or
b) Fix the code so that if you pass in an empty directory (as opposed to NULL), no profile directories are scanned for modules/themes.
Decided on (a). See comment #7 for why (b) seemed too dangerous.
Remaining tasks
Decide and patch and test.
User interface changes
None.
API changes
None. Patch contains documentation and local/protected variable name changes only.
Data model changes
None.
Release notes snippet
None.
| Comment | File | Size | Author |
|---|---|---|---|
| #24 | reroll_diff_15-24.txt | 7.83 KB | ravi.shankar |
| #24 | 3076098-24.patch | 14.25 KB | ravi.shankar |
| #15 | interdiff.txt | 838 bytes | jhodgdon |
| #15 | 3076098-document-buggy-behavior-15.patch | 14.63 KB | jhodgdon |
Comments
Comment #2
andypostThere's few issues like related that trying to add white/blacklist to discovery but not sure that could help here
Comment #3
jhodgdonI guess you could blacklist "profiles" and that would probably work, if the blacklist was settable.
Comment #4
larowlanI think changing the empty vs NULL behaviour would possibly break things, because of PHPs loose typing.
At least one profile has to be scanned in runtime code, so if this is an issue in testing, I have no issue with using a dummy profile for that purpose
Comment #5
jhodgdonYes, indeed. I didn't mention in my report, but I had already tried on my local machine to change this empty() to isset() in ExtensionDiscovery:
The first test I tried running (the test I was actually working on) failed to even get through installation because it couldn't find the testing profile. It basically broke testing.
So... I looked into this (fairly messy!!!!) code in ExtensionDiscovery some more. I realized that it was tacitly assuming during scan() that if you were looking for profiles, then profileDirectories() must have been set to NULL. This is not necessarily the case -- it is never actually verified. That assumption also means that if you created an ExtensionDiscovery object and scanned for modules, then later scanned for profiles, it would most likely break.
So, I went through and cleaned up the code and documentation, and fixed that assumption. I ran one test locally to verify that this patch doesn't completely break the testing system... let's see what happens in the full test suite.
Comment #7
jhodgdonWell. I looked into the test fails. We have several places in Core where we were doing things like this:
before doing a scan. What this does:
- If you were scanning for profiles, it has no effect, so it would still do what it always did, and return all profiles (see #3076329: ExtensionDiscovery returns testing profiles even when you ask it not to, which is about that buggy behavior).
- If you were scanning for modules, previously this would prevent the automatic filtering to just the active profile, and now it does what I think it should be doing, which is filtering so that no modules in any profile directories are being scanned.
As a side effect, ModuleExtensionList is able to find modules in non-active profiles. Which I think is actually a bug... But some of the test fails are from expecting this behavior. Others are from the TestDiscovery process making that same assumption. So... I think that it is too dangerous at this point to fix the actual bug, and what we should do to "solve" this problem is document the buggy behavior and how to get around it. So, Here's a new patch. I didn't bother with an interdiff because I really had to start over.
This is just a documentation patch, plus changing names of some local and protected variables from "files" to "extensions" where appropriate. And in one case, in a test that was looking into that protected member variable that I changed the name of.
Comment #8
jhodgdonUpdating issue summary.
Comment #9
andypostChanging "files" to "extensions" looks great, because it is what it actually stores, but not sure about scope
Would be great to get more eyes on it... scope creep
I recall inability to find extensions in profiles caused to move all testing ones to
core/profilesMaybe it needs some test added here to illustrate?
PS: tests are often do manual discovery (or do copy #2850875: Dependency on configuration provided by install profile is throwing exception) so makes sense to create follow-up to "normalize" discovery inside profiles
Comment #10
jhodgdonThis issue only ended up doing documentation and local variable name changes. I didn't change the code at all. I decided the scope creep was probably OK because there is no code change, just documentation cleanup... updating title
The code is currently perfectly capable of discovering profiles anywhere it can find modules. If you look at ExtensionDiscovery, it looks everywhere for all extensions, and then categorizes them by type after reading the .info.yml file to find out what type they are. This is necessary, because some test modules (for example) are not under tests/modules but just under tests, so you cannot infer the type unless you look at the .info file. Anyway regarding that see #3076329: ExtensionDiscovery returns testing profiles even when you ask it not to, which moved the profiles to a different location.
Comment #12
daffie commentedThank you @jhodgdon for creating this patch.
I very much appreciate the extra documentation that this patch provides.
Especially where the working of this class is different for profiles.
Also renaming
$filesto$extensionsis an improvement.I have just one remark:
How are the extensions now exactly keyed? This bit is confusing to me.
Comment #13
jhodgdonThat was meant to convey that it is a very deep array. At the top level, the array key is the site root directory. Then at the next level, the search directory. And so on.
I'm not sure how that could be worded better, any suggestions?
Comment #14
daffie commentedCan we change the line: "The array keys are" to "The array is keyed 5 layers deep. The array keys are from top to bottom:". If you can live with that change then it is for me RTBC.
Comment #15
jhodgdonHm... Maybe something like this?
Comment #16
daffie commentedYour changes are also great. A multidimensional array with 5 layers is not very usual. So make it very clear that that is what we have here.
With the latest change it all look good to me.
For me it is RTBC.
Thank you @jhodgdon.
Comment #17
alexpottThis is not correct as far as I can see.
The specific directory listed here
core/profiles/standardis actually scanned when we scan /core. i.e. the first thing on the list. Contributed and custom profiles are scanned as part of the site-wide directory - ie. /Comment #23
quietone commentedJust updating component.
Comment #24
ravi.shankar commentedAdded reroll of patch #15 on Drupal 9.5.x., still needs work for #17.
Comment #25
smustgrave commented@alexpott what is the correct order? Reading #17 seems - the installation profile directory; e.g., /core/profiles/standard needs to be removed but is that it?
Comment #26
smustgrave commented