Problem/Motivation

In Drupal 10.5.4, FileSystem::deleteRecursive() silently fails when used with streamWrappers return FALSE from realpath(), this is most common in remote stream wrappers (S3fs, Azure, Flysystem, etc.) however local streams where the raw file is not intended to be directly accessed (compression, encryption, etc) will encounter the same. The method returns TRUE indicating success, but no files are actually deleted from storage.

This breaks critical functionality like image style flushing (drush image-flush) when using remote file systems, causing stale derivatives to remain in storage indefinitely.

This additionally presents a risk of sensitive data being left on disk.

Steps to reproduce

  1. Install Drupal 10.5.4 with S3fs module (or any remote stream wrapper)
  2. Configure S3fs to handle public:// files
  3. Generate some image style derivatives
  4. Run drush image-flush <style_name>
  5. Check the S3 bucket - files still exist
  6. Check s3fs_file table - metadata records still exist

Expected behavior

deleteRecursive() should delete all files and directories from remote storage, just as it does with local filesystems.

Actual behavior

deleteRecursive() returns TRUE but deletes nothing. Files remain in S3 and metadata remains in the database.

Root cause

In Drupal 10.5.x, FileSystem::deleteRecursive() was changed to call realpath() at the beginning:

public function deleteRecursive($path, ?callable $callback = NULL) {
    // Ensure paths are local paths when a recursive delete is started.
    if ($this->streamWrapperManager->isValidUri($path)) {
      $path = $this->realpath($path);  // ← Returns FALSE for remote URIs
    }
    
    if ($callback) {
      call_user_func($callback, $path);
    }
    
    // Allow broken links to be removed.
    if (!file_exists($path) && !is_link($path)) {  // ← file_exists(FALSE) = FALSE
      return TRUE;  // ← Returns here without doing anything!
    }
    // ... rest of method
}

The issue:

  1. Remote stream wrappers correctly return FALSE from realpath() (as documented in FileSystemInterface::realpath(): "does not work for remote URIs")
  2. $path becomes FALSE
  3. file_exists(FALSE) returns FALSE
  4. Method hits the early return and exits without deleting anything
  5. Returns TRUE claiming success

Verification code

// Test realpath with S3
$wrapper = \Drupal::service('stream_wrapper_manager')->getViaUri('public://styles/medium');
$realpath = $wrapper->realpath();
var_dump($realpath); // bool(false)

// Test deleteRecursive
$file_system = \Drupal::service('file_system');
$result = $file_system->deleteRecursive('public://styles/medium');
var_dump($result); // bool(true) - claims success

// But files still exist
var_dump(is_dir('public://styles/medium')); // bool(true) - still there!

Workaround

Delete files individually without using deleteRecursive():

function manualFlush($uri) {
  $database = \Drupal::database();
  
  // Get all URIs from metadata
  $uris = $database->select('s3fs_file', 'f')
    ->fields('f', ['uri'])
    ->condition('uri', $uri . '%', 'LIKE')
    ->execute()
    ->fetchCol();
  
  // Delete each file individually
  foreach ($uris as $file_uri) {
    if (file_exists($file_uri)) {
      unlink($file_uri);  // This works!
    }
  }
  
  // Clean up metadata
  $database->delete('s3fs_file')
    ->condition('uri', $uri . '%', 'LIKE')
    ->execute();
}

Proposed resolution

Remove the realpath() conversion from deleteRecursive(). Stream wrapper URIs should be used directly, just like all other file operations (copy(), move(), delete(), etc.) already do.

The method should work with stream wrapper URIs throughout, not attempt to convert them to local paths.

API changes

None - this is a bug fix restoring previous behavior.

Affected versions

Drupal 10.5.x (possibly 10.4.x - needs verification)

Related issues

  • Similar realpath() issue fixed in EXIF Orientation module: #3284307
  • S3fs image URL generation issues when derivatives don't exist: Various reports in S3fs queue
  • Views Data Export with S3fs: #3325431

Impact

This affects any site using remote stream wrappers for file storage, breaking:

  • Image style flushing
  • Any code using deleteRecursive() on remote URIs
  • Proper cache management for derivative files

This is a regression that breaks a core Drupal feature (image styles) when using remote file systems, which is increasingly common in cloud deployments.

Issue fork drupal-3559132

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

oli4 created an issue. See original summary.

cilefen’s picture

Version: 10.5.x-dev » 11.x-dev
cmlara’s picture

Title: FileSystem::deleteRecursive() fails with remote stream wrappers in 10.5.x due to realpath() returning FALSE » FileSystem::deleteRecursive() leaves files/directories when realpath() returns FALSE
Issue summary: View changes
Priority: Major » Critical
Issue tags: +Security, +Legal, +compliance, +GDPR

I'm increasing the priority out of concern for site security and legal compliance due to this potentially leaving sensitive data at rest. Addtionaly the longer this issue persists the less likely a site owner is to be able to determine all paths that they need to manually remove from storage.

This is a fundamental method that developers expect to function. For any site owner who has contractual or security requirements to delete data this poses a liability risk. While we could argue that for such cases a developer should perform a second validation that the files have been deleted I will note that the API specs that it will inform them of failures with a FALSE return.

While Remote streamWrappers are most likely to encounter this, wrappers using local disk can also be impacted. From what I can see on a cursory review it would appear the Flysystem local storage module may be a real world example of impact involving local disk.

Below are a couple snippets taken from GitLab where sensitive information may be left on disk (these snippets are only intended to be illustrative, no assertion is made that sensitive data was actually stored in these file paths or that they are ideal code usage samples ).

\Drupal::service('file_system')->deleteRecursive('private://oauth_api_keys/');

   // Get the private path for Google credentials.
  $private_path = 'private://google_credentials';

  // Remove the directory and its contents.
  if (\Drupal::service('file_system')->deleteRecursive($private_path)) {
    \Drupal::messenger()->addStatus(t('Deleted directory @path and its contents.', ['@path' => $private_path]));
  }

I will link this issue in #3426047: Bump PHPStan to level 9 and accept a large baseline as an example of a bug PHPStan could have prevented.

  360    Parameter #1 $filename of function file_exists expects string, string|false given.                                                          
  360    Parameter #1 $filename of function is_link expects string, string|false given.                                                              
  364    Parameter #1 $filename of function is_dir expects string, string|false given.                                                               
  364    Parameter #1 $filename of function is_link expects string, string|false given.                                                              
  365    Parameter #1 $directory of function dir expects string, string|false given.      

rory downes made their first commit to this issue’s fork.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

jesseh made their first commit to this issue’s fork.

jesseh’s picture

Status: Active » Needs review
kim.pepper’s picture

smustgrave’s picture

@kim.pepper the tests added from that issue are still passing so assume we are probably good right?

smustgrave’s picture

Status: Needs review » Reviewed & tested by the community

Going on a limb here since it's marked critical.

All the tests added fail without the change https://git.drupalcode.org/issue/drupal-3559132/-/jobs/9171741

The tests added #3482449: \Drupal\Core\File\FileSystem::deleteRecursive() will follow symlinks and remove files outside the directory it is deleting are still passing too so believe there isn't a regression.

alexpott’s picture

Version: main » 10.6.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Committed and pushed 24d12af3c73 to main and 503358465ae to 11.x and 1a191d35cb5 to 11.3.x. Thanks!

Let's backport this 10.6.x too.

  • alexpott committed 1a191d35 on 11.3.x
    fix: #3559132 FileSystem::deleteRecursive() leaves files/directories...

  • alexpott committed 50335846 on 11.x
    fix: #3559132 FileSystem::deleteRecursive() leaves files/directories...

  • alexpott committed 24d12af3 on main
    fix: #3559132 FileSystem::deleteRecursive() leaves files/directories...
alexpott’s picture

Status: Patch (to be ported) » Fixed

Ran the tests locally on 10.6.x and they passed nice. Also ran the commit code checks - all good.

Committed bd95017 and pushed to 10.6.x. Thanks!

Noticed this after commit - but why are we adding @internal to tests - that cannot be necessary.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

godotislate’s picture

Crossposted with @alexpott, but my comments were just nits.

  • alexpott committed bd950175 on 10.6.x
    fix: #3559132 FileSystem::deleteRecursive() leaves files/directories...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.