Problem/Motivation
I marked this critical because this breaks contrib tests that are using the test next minor version flag.
It's due to hook collector pass incorrectly treating symlinked directories as files.
https://www.drupal.org/project/drupal/issues/3442009#comment-15824270
This is a CI ONLY bug as far as we can tell, no one has been able to replicate it locally.
When contrib and many other CI processes install modules using symlinks. If the file iterator in HookCollectorPass::collectModuleHookImplementations hits a symlink pointing to a directory then StaticReflectionParser will try run against a directory which results in an error.
Broken contrib test: https://git.drupalcode.org/project/genpass/-/pipelines/315185
Fixed contrib tests
You can see the patched core works for contrib here: https://git.drupalcode.org/issue/genpass-3482295/-/pipelines/316367
Steps to reproduce
Run a test job in contrib against next minor.
Proposed resolution
diff --git a/core/lib/Drupal/Core/Hook/HookCollectorPass.php b/core/lib/Drupal/Core/Hook/HookCollectorPass.php
index 7765f8e591..273922f69f 100644
--- a/core/lib/Drupal/Core/Hook/HookCollectorPass.php
+++ b/core/lib/Drupal/Core/Hook/HookCollectorPass.php
@@ -138,7 +138,7 @@ public static function collectAllHookImplementations(array $module_filenames): s
* @return void
*/
protected function collectModuleHookImplementations($dir, $module, $module_preg): void {
- $iterator = new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS | \FilesystemIterator::UNIX_PATHS);
+ $iterator = new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS | \FilesystemIterator::UNIX_PATHS | \FilesystemIterator::FOLLOW_SYMLINKS);
$iterator = new \RecursiveCallbackFilterIterator($iterator, static::filterIterator(...));
$iterator = new \RecursiveIteratorIterator($iterator);
/** @var \RecursiveDirectoryIterator | \RecursiveIteratorIterator $iterator*/
Remaining tasks
User interface changes
Introduced terminology
API changes
Data model changes
Release notes snippet
Issue fork drupal-3482283
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3482283-symlinking-a-module
changes, plain diff MR !9901
Comments
Comment #2
nicxvan commentedComment #3
nicxvan commentedComment #4
nicxvan commentedComment #5
ghost of drupal pastComment #7
nicxvan commentedComment #8
nicxvan commentedThis test is not actually working at the moment so I'm pulling it out for a cleaner commit:
Comment #9
nicxvan commentedComment #10
nicxvan commentedComment #11
larowlanAh I see you've added and removed a test, bring on gitlab issues where there's only one place to track an issue 🫠
Comment #12
nicxvan commentedYes we removed the test because the symlink part is dependent on the env, so in order to keep this fix clean for now it's just the fix.
If you can review the test attached in 8 and provide advice we can readd it.
This issue is specifically to how gitlab runs tests, it's treating the symlinks as directories.
Comment #13
cilefen commentedCan someone update the issue summary with the reasons this is a critical bug?
Comment #14
nicxvan commentedComment #15
alexpottThis issue is caused not by gitlab per se. It's caused by https://www.drupal.org/project/gitlab_templates and how https://git.drupalcode.org/project/gitlab_templates/-/blob/main/scripts/... works.
Comment #16
nicxvan commentedI updated the IS but I used this criteria:
Cause tests to fail in HEAD on the automated testing platform for any supported environment (including random failures), since this blocks all other work.
Comment #17
catchGiven contrib tests are broken I think we should commit the hotfix here, it's not as if we're going to silently break this - was noticed within hours.
Comment #18
nicxvan commentedI think I might have a test.
Comment #19
nicxvan commentedOk there is a test.
Comment #20
nicxvan commentedI have a positive test coming soon.
Comment #21
nicxvan commentedOk there is now a test confirming a notice is not thrown when a symlink is encountered and a positive confirmation that hooks inside the symlinks are picked up.
Comment #22
nicxvan commentedI removed the negative test per @alexpott and @elc's comments. There is a positive assertion now too to confirm it read the hooks.
My only concern is the symlink failure was silent so ensuring that notice isn't appearing is probably worth it, but we have the history if we want to add it back.
Comment #23
nicxvan commentedFor credit purposes other than the people chiming in on this issue.
@chi and @claudiu.cristea both alerted me to the issue here: #3442009: OOP hooks using attributes and event dispatcher and helped investigate in slack.
@fjgarlin pointed out how to patch core in a contrib test so we could verify the fix there and @hestenet helped loop in @fjgarlin.
@elc also provided a review, and I used a module he maintains as the test ground.
@ghostofdrupalpast identified the root cause and fix.
Comment #24
alexpottI pushed up a commit to ensure that the test fails if symlinks are not followed and triggered the test only change.
Comment #29
alexpottTest-only change fails as expected... https://git.drupalcode.org/project/drupal/-/jobs/3120262
Just fixing PHPStan baseline.
And adding issue credit as per #23
Comment #30
alexpottI've fixed up the test which has given me a good chance to review the low level code fix here and the surrounding code. This makes sense and is similar to the code in \Drupal\Core\Extension\ExtensionDiscovery::scanDirectory() which is doing something along the same line. I've checked the other options used (\FilesystemIterator::CURRENT_AS_SELF, \RecursiveIteratorIterator::LEAVES_ONLY and \RecursiveIteratorIterator::CATCH_GET_CHILD) there and I do not think they are not needed in this code.
Comment #31
nicxvan commentedWe probably want to get a cleanup phase in the test as well since this is not in vfs:
I removed it locally when I was confirming symlink creation, but didn't add it back.
Comment #32
catchNeeds work for #31, that's a good point.
Comment #33
alexpott@nicxvan @catch let's open a follow-up to do that. We should fix all KernelTests that use proper files and doing that reveals quite a nasty bug in \Drupal\Core\File\FileSystem::deleteRecursive!
Comment #34
catchDiscussed the clean-up with @alexpott and @nicxvan briefly in slack and a follow-up is incoming. Going to go ahead and commit here.
Committed/pushed to 11.x, thanks!
Comment #36
catch#3482449: \Drupal\Core\File\FileSystem::deleteRecursive() will follow symlinks and remove files outside the directory it is deleting is the follow-up.