mkdir() fails in BrowserHtmlDebugTrait.php:141 because of a race condition

Ignore errors whilst making the directory?

Yeah I don't see any other way to prevent this either unless we somehow make run-tests.sh responsible for this since that is the only part running outside the concurrency (and no, I don't think that would be a good idea even if we could get that to work!)

Would this technique help?

I'm having this problem as well. See recent test fails at #3046467: Devel_Generate tests: Port to BrowserTestBase and add Drush command coverage. I've also seen them at #2999914: Update Dependencies Format in .info.yml, https://www.drupal.org/pift-ci-job/1132230, https://www.drupal.org/pift-ci-job/1063933, https://www.drupal.org/pift-ci-job/1068930

Also see the two-years older issue #2845122: mkdir(): File exists in BrowserHtmlDebugTrait.php:141 during BrowserTestBase, which has now been closed as a duplicate of this one.

The link in #3 looks like a good solution, suggesting !is_dir($folder) && !mkdir($folder) && !is_dir($folder).

This is causing 30% of times test failures in devel

Moving to major (even if Priority levels of issues evaluation suggests to mark it as critical because of: Cause race conditions, database deadlocks etc. for which even code with 100% automated test coverage may be affected)

LGTM

Committed/pushed to 8.8.x and cherry-picked to 8.7.x, thanks!

Can we add a comment? It's not trivial to understand why we are adding is_dir two times.

Typo. Sorry.

Yeah, that was a fast commit :)

I'm also wondering if this should be part of \Drupal\Core\File\FileSystem::mkdir(), and this trait should use that instead of calling mkdir() directly, then it would also solve #3036494: Race condition in ImageStyle::createDerivative().

Those are good points. Reverted from both branches for now, let's keep going here.

Fast commits are good. :-)

Yah so basically this is killing Examples HEAD: https://www.drupal.org/pift-ci-job/1266550

It should be added to 8.6.x, so I can at least fix Examples.

I couldn't apply #14 so here it is with what I think is a clearer explanation.

Re: #16 and using FileSystem, see:

+++ b/core/tests/Drupal/Tests/BrowserHtmlDebugTrait.php
@@ -136,9 +136,12 @@ protected function initBrowserOutputFile() {
       // Do not use the file_system service so this method can be called before
-      // it is available.

It would make a good follow-up to add this to the service, though.

The patch doesn't look very simple, but the comment explains it enough so that I can understand why this code was added. It also seems like it would fix random failures I've seen in the facets module over the last couple of months.

I agree with @Mile23, I think #16 is really well suited to being a followup.

+1, we can move the change in my issue referenced in #16 to FileSystem.

'Build Successful' fail seems unrelated:

15:04:00 [Example Test] Test Suite
15:04:00 =============================
15:04:00 
15:04:00 Running:  Test page
15:04:00 
15:04:00 Error retrieving a new session from the selenium server
15:04:00 
15:04:00 Connection refused! Is selenium server started?
15:04:00 { Error: getaddrinfo ENOTFOUND chromedriver-jenkins-drupal-patches-94530 chromedriver-jenkins-drupal-patches-94530:9515
15:04:00     at errnoException (dns.js:50:10)
15:04:00     at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:92:26)
15:04:00   code: 'ENOTFOUND',
15:04:00   errno: 'ENOTFOUND',
15:04:00   syscall: 'getaddrinfo',
15:04:00   hostname: 'chromedriver-jenkins-drupal-patches-94530',
15:04:00   host: 'chromedriver-jenkins-drupal-patches-94530',
15:04:00   port: '9515' }
15:04:00 
15:04:00 error Command failed with exit code 1.
15:04:00 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
15:04:00 ---------------- nightwatchjs Returned 1 ----------------

Re-starting test.

With reference to Berdir's comments in #16 and #22 I've put this it 'needs work' not RTBC.

I think you misunderstood my comment, #20 explains why we can't use FileSystem, and we already have an issue then to do there, so no follow-up needed.

Thanks for all your work on this. Now that the commits are in 8.8.x and 8.7.x would you still expect to see this error happening? I had a daily test fail today with this problem https://www.drupal.org/pift-ci-job/1275084

1) Drupal\Tests\scheduler\Functional\SchedulerDeleteNodeTest::testDeleteNodeWhenSchedulingIsRequired
mkdir(): File exists

/var/www/html/vendor/symfony/phpunit-bridge/DeprecationErrorHandler.php:293
/var/www/html/core/tests/Drupal/Tests/BrowserHtmlDebugTrait.php:143
/var/www/html/core/tests/Drupal/Tests/BrowserTestBase.php:404
/var/www/html/modules/contrib/scheduler/tests/src/Functional/SchedulerBrowserTestBase.php:88

Line 143 is the new line of code in BrowserHtmlDebugTrait.php not the old line, so that shows that the new version is being run.

Got a new instance of this just a few hours ago in facets: https://www.drupal.org/pift-ci-job/1288919

Is the newly-committed code fix working as expected here? If it failed we should see the exception

Unable to create directory: %s

But instead we just get the same old original error

mkdir(): File exists

Do we need to open a new issue, as ordinary users cannot re-open this one.

I think the problem here is that mkdir still emits a warning, so to stop the test from failing we need to suppress that warning on top of the previous additions.

Ah yeah that would make sense. This is really hard to test if this actually fixes it since it’s a random fail. The patch in #34 does look good though, it should be enough to make the errors go away.

The patch by @Lendude is green. Setting this issue back to rtbc.

How to use that on 8.6?
There is no bulk patch with the whole fix :(