Warning: mkdir(): File exists in Drupal\Component\PhpStorage\FileStorage->createDirectory

Hello,
As you mentioned scenario, I have checked but I am not getting any errors. Otherwise please explain little bit more.. that will helpful to identify exact things.

It would be really helpful if you could provide a backtrace

I have the same problem after doing "drush8 pm-updatecode" in my test site:

drush8  pm-updatecode


Update information last refreshed: Sat, 03/12/2016 - 10:52
 Name                         Installed      Proposed  Message          
                              Version        version                    
 Drupal                       8.0.5          8.0.6     Update available 
 Migrate Plus (migrate_plus)  8.x-1.0-beta1  8.x-1.0   Update available 
 Migrate Tools                8.x-1.0-beta1  8.x-1.0   Update available 
 (migrate_tools)                                                        


NOTE: A code update for the Drupal core is available.
Drupal core will be updated after all of the non-core projects are updated.

Code updates will be made to the following projects: Migrate Plus [migrate_plus-8.x-1.0], Migrate Tools [migrate_tools-8.x-1.0]

Note: A backup of your project will be stored to backups directory if it is not managed by a supported version control system.
Note: If you have made any modifications to any file that belongs to one of these projects, you will have to migrate those modifications after updating.
Do you really want to continue with the update process? (y/n): y
Project migrate_plus was updated successfully. Installed version is now 8.x-1.0.
Backups were saved into the directory                                [ok]
/home/USER/drush-backups/m-sql-table/20160408172857/modules/migrate_plus.
Project migrate_tools was updated successfully. Installed version is now 8.x-1.0.
Backups were saved into the directory                                [ok]
/home/USER/drush-backups/m-sql-table/20160408172857/modules/migrate_tools.

Code updates will be made to drupal core.
WARNING:  Updating core will discard any modifications made to Drupal core files, most noteworthy among these are .htaccess and robots.txt.  If you have made any modifications to these files, please back them up before updating so that you can re-create your modifications in the updated version of the file.
Note: Updating core can potentially break your site. It is NOT recommended to update production sites without prior testing.

Do you really want to continue? (y/n): y
Project drupal was updated successfully. Installed version is now 8.0.6.
Backups were saved into the directory                                [ok]
/home/USER/drush-backups/m-sql-table/20160408172857/drupal.
Cache rebuild complete.                                              [ok]

Im also experiencing this issue... Haven't nailed down the cause yet as its pretty random but +1 as it is a thing.

We are hitting this error when sites/default/files is a symlink, and not a directory.

We do this on developer machines -- mount an asset directory over the network from a central file server, and then symlink sites/default/files in developer copies to the shared network asset mount.

Because this method is using is_dir() to check for the presence of the file path, this fails the is_dir() test and then throws an exception when it attempts to create a directory in a path where there's already a symlink.

I think this is a common configuration -- we find instructions from Acquia for configuring this way. Seems like a missing configuration option?

P.S. I don't know if this is the OP's original issue or not, but it's the same error, same code.

I just stumpled upon the same error while opening the entity_browser modal for the first time after a clean installation / clearing the caches.
While opening the modal for a second time everything is working fine.

Maybe we should check for the existence of the directory that's going to be created in createDirectory method of FileStorage class.

@freelock hit it on the nose in Comment #9:

We are hitting this error when sites/default/files is a symlink, and not a directory.

I'm in the same situation. Simlink used for the files directory. Same error. Thoughts?

Following up on my comment #11

It looks like I solved this on my end. My simlink was pointing to an invalid location. Once I fixed the simlink to the proper location, I got it to work without that error. My mistake.

Still happening I think is because is executing twice some command but I can't replicate it ... I don't know when happen but is harmless

The error occurs when twig classes are generated:
Drupal\Component\PhpStorage\FileStorage->createDirectory('sites/default/files/php/twig/e2c654aa_status-messages.html.twig_c0aba12839784fc81b2429bd08d64d5d5151def5936159ad900057dd21493350', 511) (Line: 128)
Drupal\Component\PhpStorage\FileStorage->ensureDirectory('sites/default/files/php/twig/e2c654aa_status-messages.html.twig_c0aba12839784fc81b2429bd08d64d5d5151def5936159ad900057dd21493350') (Line: 84)
Drupal\Component\PhpStorage\MTimeProtectedFastFileStorage->save('e2c654aa_status-messages.html.twig_c0aba12839784fc81b2429bd08d64d5d5151def5936159ad900057dd21493350', '<?php

/* core/themes/bartik/templates/status-messages.html.twig */
class __TwigTemplate_a9ae7c2370f62c93405327a1077d9b0d9d55a31195f3bf91e6044f657d64c27a extends Twig_Template
{
...

The symlink directory was owned by the wrong user, properly chmodding it seems to have fixed it:
chown -h www-data:www-data files

I spoke too soon, the issue persists.

I saw this warning today and while I don't know anything about it, I thought I'd report what I know in case it's helpful:

I noticed the warning in the dblog after upgrading a site to 8.2.1. It occurred on the first page I visited after upgrading: /user/login

The site is hosted on Pantheon, and the warning appeared in the dev instance of the site, but not in test or live. Perhaps this indicates that this is some sort of permissions error?

I've also run into this error repeatedly, but not repeatable. Unfortunately :( It does seem somewhat related to Twig templates in debug mode (development.services.yml with debug: true, auto_reload: null, cache: true), and frequently occurs after a cache rebuild (drush cr).

Here's more of the error message, in case that helps:

Warning: mkdir(): File exists in Drupal\Component\PhpStorage\FileStorage->createDirectory() (line 165 of core/lib/Drupal/Component/PhpStorage/FileStorage.php).

Drupal\Component\PhpStorage\FileStorage->createDirectory('sites/default/files/php/twig/74b35ec4_block--system-messages-block.html.twig_18a04e0690e790396e447076c99b64af7163a8846a8b18e49c494b1d08ce9058', 511) (Line: 128)
Drupal\Component\PhpStorage\FileStorage->ensureDirectory('sites/default/files/php/twig/74b35ec4_block--system-messages-block.html.twig_18a04e0690e790396e447076c99b64af7163a8846a8b18e49c494b1d08ce9058') (Line: 84)
Drupal\Component\PhpStorage\MTimeProtectedFastFileStorage->save('74b35ec4_block--system-messages-block.html.twig_18a04e0690e790396e447076c99b64af7163a8846a8b18e49c494b1d08ce9058', '<?php

/* core/themes/stable/templates/block/block--system-messages-block.html.twig */
class __TwigTemplate_b1231619d26a52dbf9c809c0839c633c7ab0ac306eaac08feb4fdf2817d892ef extends Twig_Template
<snip>

I also have this issue. I've had it for longer than just Drupal 8.2.1, but I don't recall exactly when it started.

Occasionally, like 2 or 3 times per day, I see this error on my site. It's usually directly after login, however it'll also happen the first time I visit a page on the site in the morning, even though I'm still logged in from the day before.

This seems to indicate to me it's somehow related to twig cache expiration--as if twig is trying to regenerate a template whose cache has expired and is unable to overwrite the existing file. In my case, I believe the error always traces back to the block template, but I haven't entirely figured out if it's actually related to a particular block or not.

@ #9, I don't think the issue is is_dir, that should work properly on a symlink (See http://php.net/manual/en/function.is-dir.php#118994).

The problem might be in dirname on line 162:

    if (is_dir($parent = dirname($directory))) {

dirname returns the symlink path (See http://stackoverflow.com/questions/12580330/php-dirname-returns-symlink-...). So we should probably change that for symlinks to:

    $parent = dirname($directory);
    // Use readlink() to read the target of symbolic link.
    if (is_link($directory)) {
      $parent = readlink($parent);
    }
    if (is_dir($parent)) {

A quick workaround would be to suppress warnings with @mkdir:

      if ($status = @mkdir($directory)) {

Thanks!

I realize this is an 8.3.x-dev patch, but it applied cleanly on my 8.2.2 installation as the code it's changing seems to be identical, and I haven't seen another of these errors yet (though enough time may not have passed yet for a true test in my case)

I'll report back if I experience any further instances of this error using this method of fixing it.

I'm not sure if it's related but I had a very similar error after installing the Libraries module:

Warning: mkdir(): File exists in Drupal\Core\File\FileSystem->mkdirCall() (line 237 of core/lib/Drupal/Core/File/FileSystem.php).

I've also had the error this issue is about numerous times so I applied the patch mentioned in #23. At first sight I'm not getting the error anymore. Also not when I uninstall/install the Libraries module. Should something change I'll let it know.

Added the @ignore errors for testing purpose.

@mpp that is ok.. but I don't think the answer is suppress the error message..., should be check if exist and if doesn't mkdir

@saphyel, have a look at #23, may need some work though.

I'm still experiencing this error with the patch in #23 applied. It *might* be less often, but it still seems like every time I come back to the site after having been away for a while, I see one of these messages once still even with the patch.

I'm going to try #27 in the hopes it'll at least get this message to stop annoying our client for now, but I'll happily test another potential fix if posted.

I have the same warning on my local dev site, but I don't use symlinks.

I don't know how to reproduce it exactly, but I'll try to explain conditions of how it occurs:

1. The site wasn't visited at least 6 hours.
2. I open front page (w/ user #1 session) in browser. Cron runs - I don't see it, but it takes awhile to load the page.
3. When page is loaded I see couple instances of this PHP warning (I have 'all errors visible' settings in my php.ini and in settings.php).
4. When I go to DBLog list, I see these warnings were logged earlier than all cron messages.

My guess of the problem cause is the following:

1. Obviously, some code that builds twig cache files, it runs not on every page request, but from time to time, like cron.
2. When we visit some page, then multiple PHP requests are made. In each of these requests that same code attempts to build cache files. In all requests is_dir($directory) returns FALSE. One of requests actually creates that dir, so other requests call mkdir($directory) when the dir is already there.

Why do I think there are multiple requests on a single page? In DBLog list I can see "Attempting to re-run cron while it is already running" multiple times between cron messages about "Starting execution of some_cron(), execution of another_cron() took some ms."

Here are some example URLs that are being requested along with the page itself:

• POST http://test.d8/contextual/render
• POST http://test.d8/quickedit/metadata
• POST http://test.d8/quickedit/attachments?_wrapper_format=drupal_ajax

@alexpott, @catch, @cottser, @xjm and I considered this issue and agree this is suitably a major priority bug because it is policy that PHP errors triggered in the UI are major.

I tested the 2685957-27.patch sent by @mpp in the 8.2.7 core version.
The warning was fixed.

I tested the 2685957-27.patch sent by @mpp in the 8.2.7 core version.
The warning was fixed.

@bluccas That's not a fix actually, but a "closing-eyes" workaround. The patch just suppresses the warning, not fixes it.

I'm changing priority back to Major, and setting status to Needs Work as per my comment #31 above.

@bluccas Welcome to the team!

Please do not change issue priority on triaged issues. Read about issue priority and reviewing issues.

@pingwin4eg: agreed that it is a patch and not a solution. However, there may always be two concurrent requests that try to check if there's a directory and try to create it. So this seems to be a race condition: both may detect no directory exists but only one of them will be able to create it.

Solving the race condition would involve a semaphore in Drupal, which imo seems quite heavy for what is happening here.

... there may always be two concurrent requests ...

Yes, that's what I stated earlier. But using @ is a bad practice. It can lead to other "hidden" bugs, that would be hard to find.

In this specific situation the @ seems harmless, but... I don't know, it's kind of a bad feeling.

Ok, let's change the status to Needs Review to attract attention of other people to the issue. Maybe someone can propose a brilliant solution that will satisfy all of us.

P.S.: summary needs an update.

Considering the purpose of this cache backend, I'd actually vote for #27. Race conditions should be tolerable regards anything which is appropriate to be handled by a filesystem storage. AFAIK this problem won't be avoidable without eliminating the benefits of this cache backend.

I don't see a reason why this needs to wait until 8.4.x

#23 is correct re: symlinks with one exception, we don't need any special code because what that's saying is that when 'sites/default/files' is a symlink and is_dir('sites/default/files'); is run, its going to to resolve that to '/srv/myfiles' or where ever and say "oh, yeah that's a directory" and return true.

The problem is as others have suggested the race between requests checking and trying to make the directory which is why the error is on line 170 and not further down on

return $this->createDirectory($parent, $mode, TRUE) && mkdir($directory) && chmod($directory, $mode);

Considering that we care about one failure and not the other the return value from mkdir actually becomes useless and we can simplify this function quite a bit and move all the code to the tail recursion (optimistically avoiding recursion) and only setup the mkdir/chmod logic once.

The patch makes the race tighter(the time between the is_dir and fileperms call) and the only side effect will be an additional chmod which might be a small disk write cost but no user impact.

self review, that documentation is a little wonky. Probably from copy and pasting from previous code.

Tested patch from #46 successfully on core 8.3.6.

+++ b/core/lib/Drupal/Component/PhpStorage/FileStorage.php
@@ -138,34 +138,39 @@ protected function ensureDirectory($directory, $mode = 0777) {
+      // race getting here and the directory could already exists.

nit: could already exist - can be fixed on commit.

Bug fixes normally come with a failing test - is there a way we can reproduce this scenario in a test?

@larowalan, a test would be awesome but do you know of any single race condition test in Drupal, or php for that matter?

I'm asking did we actually find the conditions under which the race occurred and can we simulate it with vfsstream

One of my D8.3.7 sites on Pantheon is getting this error one to eight times per minute. I applied the patch in #46 and so far so good on the errors. Although I'm concerned about why the Twig cache is going so nuts.

I'm not sure how we would change the file structure in the middle of a method call in a test. That seems impossible.

To clarify, the error is that this method has a is_dir check, a parent directory check, then a mkdir. the problem is that in that small snippet of code the directory can come into existence and the mkdir fails loudly. Technically with heavy load just the time between is_dir and mkdir side by side could allow for the bug.

I've skimmed the vfsstream documentation and confirmed that if there is a way to respond to fileystem events and modify it I don't see it which makes this impossible to consistently test.

Thanks @neclimdul - appreciate the effort going into investigation and updating issue summary

The only way to change the file structure in the middle of the method would be by using a Mock object. To Mock you need objects but the code is using php native file functions and unfortunately Php doesn't foresee a native file system object like asp or many other languages and using a new dependency just to make this piece of code testable for race conditions would come at a performance cost.

I guess this is the only thing keeping it from RTBC?

Thanks!

This looks like a good way to reduce the possibility of the race from happening.

However, I think we have a problem. Not everywhere uses the return value of this method - so before we would have had a message now we might not. And we would have stopped execution but now we are going to continue on regardless.

I think we should throw an exception in \Drupal\Component\PhpStorage\FileStorage::ensureDirectory() if we have not ensured the directory. And we should try to add tests around what happens when \Drupal\Component\PhpStorage\FileStorage::createDirectory() fails as we don't have any test coverage of that.

Currently createDirectory does not actually stop execution so throwing an exception could be a pretty big BC break. It actually emits an E_WARNING which manifests in this bug as a DSM of the error or watchdog depending on your error handling levels. I believe I have a screenshot of the DSM from a development environment somewhere I can upload if its relevant but put the message in the summary in red text at the top of the page and you've got it.

But if we consider the warning a byproduct of the method we need we can add a trigger_error before the return false;. That does seem reasonable.

@neclimdul ah how right you are...

>>> mkdir('/Users/bblah');
PHP warning:  mkdir(): Permission denied on line 1
>>> mkdir('bblah/fdgftghrew');
PHP warning:  mkdir(): No such file or directory on line 1

So a warning is not going to stop execution.

Nice idea about triggering a warning if we return FALSE from here. +1

Warning and test of the warning. I tried to test the return of createDirectory but trigger_error still outputs text so even disabling phpunit's exceptions temporarily and that doesn't work. I think we're good though because we know that the warning and the return type are connected.

+++ b/core/lib/Drupal/Component/PhpStorage/FileStorage.php
@@ -170,6 +170,7 @@ protected function createDirectory($directory, $mode = 0777) {
+    trigger_error('mkdir(): Permission Denied', E_USER_WARNING);

Well we don't know why it has failed so saying permission denied seems presumptive - maybe just 'mkdir(): Failed' - I thought about including the directory name but that's not what PHP does and I guess it could result in information disclosure.

@neclimdul, I didn't know about the vfsStream package, nice one!

So technically true but I went off the PHP documentation:

Emits an E_WARNING level error if the directory already exists.
Emits an E_WARNING level error if the relevant permissions prevent creating the directory. 

Since we're handling the existing directory cases, (recursively creating and hiding the existing directory failure). The permission warning seemed the only relevant failure case. I think what that means from PHP's perspective is more of "Either permission error or something that looked like a permission error." I'm open to changing it if that doesn't make sense though.

@mpp Thanks! You might be using it already since its hidden through out core's test suite. It is also what makes the rest of the PhpStorage file storage test work. :-D

Indeed, good to know :-)

In the context of this issue I assume the error could also be that the directory already exists (although chances are very small now the risk of the race condition is a lot smaller).

Noticed some coding standards problems. Must have been working a little too fast.

Alternate version of patch with "mkdir(): Failed" warning text as well.

The if

if (is_dir($directory)) {

will lead to a true statement in the case that the directory already exists so we won't trigger the warning ;)

In theory the if could result false for two concurrent requests in which case the mkdir will fail for one of them.

This was the original problem :-) But I said "could be" and "in theory" as the time between the if and the mkdir has become smaller. But it remains "possible", especially if there's a lot of recursion (subdirectories that need to be created).

I'd consider this issue tackled!

+++ b/core/lib/Drupal/Component/PhpStorage/FileStorage.php
@@ -138,34 +138,40 @@ protected function ensureDirectory($directory, $mode = 0777) {
+    trigger_error('mkdir(): Permission Denied', E_USER_WARNING);

The thing is we don't know if it is the mkdir() that has caused this - I think it could be the chmod when creating a parent dir - so perhaps it is best just to say: \Drupal\Component\PhpStorage\FileStorage::createDirectory(): Failed because that is all we can be sure of.

The if is after the mkdir now so when mkdir fails in a concurrent storm it still looks like it succeeds. The only issues remain around chmod and I think we're safe there because as long as you have write permissions you can chmod repeatedly without error.

I see what you're saying, I was focused on the block doing the work and not the recursion where the chmod could cause a failure. I'll move the warning next to the mkdir so it matches the current behavior better and doesn't report on chmod.

Forgot this needed a reroll to move the error. Fixing the status to reflect that.

Sorry for the delay. For your reviewing pleasure.

This looks great. The patch reduces the possibility of a race and if there is one then the warnings are silenced because there is no real problem.

Agreed with #73.

For those -like me- who need to apply this patch on the 8.3 branch (at least on release 8.3.7), here is the patch.

fails are in HEAD

+++ b/core/lib/Drupal/Component/PhpStorage/FileStorage.php
@@ -138,34 +138,42 @@ protected function ensureDirectory($directory, $mode = 0777) {
+    // If the parent directory doesn't exist, try and create it.

nit: try to.

Fixed that on commit.

Committed/pushed to 8.5.x and cherry-picked to 8.4.x. Thanks!

Thanks!

I have had a recurrence of this problem. Not sure about the etiquette of posting details on this closed issue, so I have posted to the open issue #2845122-7: mkdir(): File exists in BrowserHtmlDebugTrait.php:141 during BrowserTestBase instead.