Problem/Motivation

I marked this critical because this breaks contrib tests that are using the test next minor version flag.
It's due to hook collector pass incorrectly treating symlinked directories as files.

https://www.drupal.org/project/drupal/issues/3442009#comment-15824270

This is a CI ONLY bug as far as we can tell, no one has been able to replicate it locally.
When contrib and many other CI processes install modules using symlinks. If the file iterator in HookCollectorPass::collectModuleHookImplementations hits a symlink pointing to a directory then StaticReflectionParser will try run against a directory which results in an error.

Broken contrib test: https://git.drupalcode.org/project/genpass/-/pipelines/315185

Fixed contrib tests
You can see the patched core works for contrib here: https://git.drupalcode.org/issue/genpass-3482295/-/pipelines/316367

Steps to reproduce

Run a test job in contrib against next minor.

Proposed resolution

diff --git a/core/lib/Drupal/Core/Hook/HookCollectorPass.php b/core/lib/Drupal/Core/Hook/HookCollectorPass.php
index 7765f8e591..273922f69f 100644
--- a/core/lib/Drupal/Core/Hook/HookCollectorPass.php
+++ b/core/lib/Drupal/Core/Hook/HookCollectorPass.php
@@ -138,7 +138,7 @@ public static function collectAllHookImplementations(array $module_filenames): s
    * @return void
    */
   protected function collectModuleHookImplementations($dir, $module, $module_preg): void {
-    $iterator = new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS | \FilesystemIterator::UNIX_PATHS);
+    $iterator = new \RecursiveDirectoryIterator($dir, \FilesystemIterator::SKIP_DOTS | \FilesystemIterator::UNIX_PATHS | \FilesystemIterator::FOLLOW_SYMLINKS);
     $iterator = new \RecursiveCallbackFilterIterator($iterator, static::filterIterator(...));
     $iterator = new \RecursiveIteratorIterator($iterator);
     /** @var \RecursiveDirectoryIterator | \RecursiveIteratorIterator $iterator*/

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

Issue fork drupal-3482283

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

nicxvan created an issue. See original summary.

nicxvan’s picture

Issue summary: View changes
nicxvan’s picture

Issue summary: View changes
nicxvan’s picture

Issue summary: View changes
ghost of drupal past’s picture

Issue summary: View changes

nicxvan’s picture

Issue summary: View changes
nicxvan’s picture

This test is not actually working at the moment so I'm pulling it out for a cleaner commit:

<?php

declare(strict_types=1);

namespace Drupal\KernelTests\Core\Hook;

use Drupal\Core\Hook\HookCollectorPass;
use Drupal\KernelTests\KernelTestBase;
use Symfony\Component\DependencyInjection\ContainerBuilder;

/**
 * @coversDefaultClass \Drupal\Core\Hook\HookCollectorPass
 * @group Hook
 */
class HookCollectorPassTest extends KernelTestBase {

  protected function setUpFilesystem(): void {
    // VFS does not and can not support symlinks.
  }

  public function testSymlink(): void {
    mkdir($this->siteDirectory);
    symlink(realpath("core/modules/user/config"), "$this->siteDirectory/config");
    $container = new ContainerBuilder();
    $module_filenames = [
      'test' => ['pathname' => "$this->siteDirectory/test.info.yml"],
    ];
    $container->setParameter('container.modules', $module_filenames);
    (new HookCollectorPass())->process($container);
    unlink("$this->siteDirectory/config");
    rmdir($this->siteDirectory);
  }

}
nicxvan’s picture

Issue summary: View changes
nicxvan’s picture

Issue summary: View changes
larowlan’s picture

Ah I see you've added and removed a test, bring on gitlab issues where there's only one place to track an issue 🫠

nicxvan’s picture

Yes we removed the test because the symlink part is dependent on the env, so in order to keep this fix clean for now it's just the fix.

If you can review the test attached in 8 and provide advice we can readd it.

This issue is specifically to how gitlab runs tests, it's treating the symlinks as directories.

cilefen’s picture

Can someone update the issue summary with the reasons this is a critical bug?

nicxvan’s picture

Issue summary: View changes
alexpott’s picture

nicxvan’s picture

I updated the IS but I used this criteria:

Cause tests to fail in HEAD on the automated testing platform for any supported environment (including random failures), since this blocks all other work.

catch’s picture

Status: Active » Needs review

Given contrib tests are broken I think we should commit the hotfix here, it's not as if we're going to silently break this - was noticed within hours.

nicxvan’s picture

I think I might have a test.

nicxvan’s picture

Ok there is a test.

nicxvan’s picture

I have a positive test coming soon.

nicxvan’s picture

Ok there is now a test confirming a notice is not thrown when a symlink is encountered and a positive confirmation that hooks inside the symlinks are picked up.

nicxvan’s picture

I removed the negative test per @alexpott and @elc's comments. There is a positive assertion now too to confirm it read the hooks.

My only concern is the symlink failure was silent so ensuring that notice isn't appearing is probably worth it, but we have the history if we want to add it back.

nicxvan’s picture

For credit purposes other than the people chiming in on this issue.

@chi and @claudiu.cristea both alerted me to the issue here: #3442009: OOP hooks using attributes and event dispatcher and helped investigate in slack.

@fjgarlin pointed out how to patch core in a contrib test so we could verify the fix there and @hestenet helped loop in @fjgarlin.

@elc also provided a review, and I used a module he maintains as the test ground.

@ghostofdrupalpast identified the root cause and fix.

alexpott’s picture

I pushed up a commit to ensure that the test fails if symlinks are not followed and triggered the test only change.

alexpott credited chi.

alexpott credited elc.

alexpott credited fjgarlin.

alexpott’s picture

Test-only change fails as expected... https://git.drupalcode.org/project/drupal/-/jobs/3120262

Just fixing PHPStan baseline.

And adding issue credit as per #23

alexpott’s picture

Status: Needs review » Reviewed & tested by the community

I've fixed up the test which has given me a good chance to review the low level code fix here and the surrounding code. This makes sense and is similar to the code in \Drupal\Core\Extension\ExtensionDiscovery::scanDirectory() which is doing something along the same line. I've checked the other options used (\FilesystemIterator::CURRENT_AS_SELF, \RecursiveIteratorIterator::LEAVES_ONLY and \RecursiveIteratorIterator::CATCH_GET_CHILD) there and I do not think they are not needed in this code.

nicxvan’s picture

We probably want to get a cleanup phase in the test as well since this is not in vfs:

foreach (scandir($this->siteDirectory) as $item) {
      $target = "$this->siteDirectory/$item";
      if (is_link($target)) {
        unlink($target);
      }
    }

I removed it locally when I was confirming symlink creation, but didn't add it back.

catch’s picture

Status: Reviewed & tested by the community » Needs work

Needs work for #31, that's a good point.

alexpott’s picture

Status: Needs work » Reviewed & tested by the community

@nicxvan @catch let's open a follow-up to do that. We should fix all KernelTests that use proper files and doing that reveals quite a nasty bug in \Drupal\Core\File\FileSystem::deleteRecursive!

catch’s picture

Discussed the clean-up with @alexpott and @nicxvan briefly in slack and a follow-up is incoming. Going to go ahead and commit here.

Committed/pushed to 11.x, thanks!

  • catch committed c40dd534 on 11.x
    Issue #3482283 by nicxvan, alexpott, ghost of drupal past, elc, larowlan...
catch’s picture

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.