Problem/Motivation

We are seeing random fails in the first tests run - see https://www.drupal.org/pift-ci-job/477829.


nstall_begin_request
exception: [Uncaught exception] Line 493 of core/includes/install.core.inc:
Drupal\Core\Installer\Exception\AlreadyInstalledException: 
To start over, you must empty your existing database and copy default.settings.php over settings.php.
To upgrade an existing installation, proceed to the update script.
View your existing site.
 in install_begin_request() (line 493 of /var/www/html/core/includes/install.core.inc). install_begin_request(Object, Array) (Line: 112)
install_drupal(Object, Array) (Line: 485)
Drupal\simpletest\WebTestBase->doInstall() (Line: 457)
Drupal\simpletest\WebTestBase->setUp() (Line: 33)
Drupal\aggregator\Tests\AggregatorTestBase->setUp() (Line: 1049)
Drupal\simpletest\TestBase->run(Array) (Line: 744)
simpletest_script_run_one_test('29', 'Drupal\aggregator\Tests\DeleteFeedTest') (Line: 65)

This potentially could be related to #728702: Visiting index.php should redirect to install.php if settings.php already has database credentials but database is empty.

Proposed resolution

Remaining tasks

User interface changes

API changes

CommentFileSizeAuthor
#19 random-fail-already-install-2806697-19.patch407 bytesarunkumark
#3 2806697-3.patch407 bytesalexpott
#2 2806697-2.patch407 bytesalexpott

Comments

alexpott created an issue. See original summary.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Status: Active » Needs review
StatusFileSize
new 2806697-2.patch407 bytes

Trying to make a patch that exposes the problem quickly.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
StatusFileSize
new 2806697-3.patch407 bytes

Maybe 31 tests is not enough ... bumping 62.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

Discussed a bit of @mixologic - test bots are reused so maybe something is going on here. However if artefacts are left around then maybe that's what is causing this.

However thinking about this a bit more maybe what is occurring is if a bot has a failed test and then the next run might get this bug.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

I think we should call run-tests.sh at the start of test run to ensure that environment is clean. Unfortunately I just broke it.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

See #2807171: Use the correct testing table prefix in simpletest_clean_database()

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

Another case https://dispatcher.drupalci.org/job/php7_sqlite3.8/2041/consoleFull - interesting this looks like another first use of a bot. Maybe we've go an error that only occurs on the first usage of a bot???

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented
Title: Random fail for already installed in first 31 tests » Random fail for AlreadyInstalledException
Issue tags: +Triaged D8 critical

Discussed with @catch, @cottser, @webchick, @xjm, @effulgentsia, and @cilefen. We agreed to keep this critical because random fails are a critical bug. Re-titling so the hypothetical cause is not in the title.

borisson_’s picture

borisson_
Primary language Dutch
Location Mechelen, 🇧🇪
commented

If it helps to have additional data, I had this failure on the facet tests yesterday https://www.drupal.org/pift-ci-job/518158

andypost’s picture

andypost
he/him
Primary language Russian
commented 
+++ b/core/scripts/run-tests.sh
@@ -110,6 +110,8 @@
+// Limit to testing only 62 classes.
+$test_list = array_slice($test_list, 0, 62, TRUE);

I think this comment should tell why 62 and not 42)

xjm’s picture

xjm
she/her
Primary language English
commented
Parent issue: » #2829040: [meta] Known intermittent, random, and environment-specific test failures
mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

Here's one from a sibling random fail (#2823400: Random fail in CopyFileTest): https://www.drupal.org/pift-ci-job/541462

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

Happened twice with the same combo - 8.2.x/PHP 5.5/SQLite: https://www.drupal.org/pift-ci-job/541462

mikeryan’s picture

mikeryan
he/him
Primary language English
Location Pittsfield, MA, USA
commented

Sorry, never mind, second one did pass (the original fail was not cleared).

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

According to @mixologic a new build of DrupalCI has just been put in place that guarantees new database containers for every test. I wonder if that will make this go away.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented

Looks like it didnt. Over in https://www.drupal.org/node/2828559#comment-11857218 there was a battery of tests, and https://www.drupal.org/pift-ci-job/569192 popped up in that battery. So, completely fresh db containers/db directories didnt solve alreadyinstalled exception.

This seems like its some sort of race condition that is somehow getting the same dbname/number for different runs of the children of run-tests.sh.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

This seems like its some sort of race condition that is somehow getting the same dbname/number for different runs of the children of run-tests.sh.

The thing is - the database name is generated using a symlink - see \Drupal\Core\Test\TestDatabase::getTestLock(). A symlink is one of the 2 guaranteed blocking file operations so we are dealing with concurrency here.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

arunkumark’s picture

arunkumark
Location Coimbatore
commented
StatusFileSize
new random-fail-already-install-2806697-19.patch407 bytes

Hi,

I have re-rolled the patch for latest Drupal 8.4.x version.

alexpott’s picture

alexpott
he/they
Primary language English
Location 🇪🇺🌍
commented

Has anyone actually seen this error recently? I've not seen it in quite a while and it used to be quite frequent.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented

This is the most recent example I can find:
https://dispatcher.drupalci.org/job/drupal_patches/89/
And some other somewhat recentish examples:
https://dispatcher.drupalci.org/job/php7_sqlite3.8/2806/
https://dispatcher.drupalci.org/job/default/311093/
https://dispatcher.drupalci.org/job/php5.5_sqlite3.8/1649

Anonymous’s picture

Anonymous (not verified) commented

https://www.drupal.org/pift-ci-job/626952

mondrake’s picture

mondrake
Location 🇮🇹
commented

https://www.drupal.org/pift-ci-job/655711

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

amateescu’s picture

amateescu commented
Priority: Critical » Normal

We haven't seen this in branch tests since the 13th of March: https://www.drupal.org/pift-ci-job/749685 -> https://www.drupal.org/pift-ci-job/620178

I think it's pretty safe to downgrade it from critical.

gambry’s picture

gambry
Primary language Italian
Location Milan
commented

This is killing me for work-related development. I'm assuming it's a weirdness due the combination Jenkins + Docker.
It may be related to docker storage process (improved within last year, which could be the reason for #25) + Jenkins not cleaning the workspace as it's supposed to do ( i.e. https://issues.jenkins-ci.org/browse/JENKINS-41805 ).
But there may be something else as really none of them fully explain the exception.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Mixologic’s picture

Mixologic
He/Him/His
Primary language English
Location Portland, OR
commented
Status: Needs review » Reviewed & tested by the community

Im pretty sure that we solved this over in https://www.drupal.org/project/drupal/issues/2906317, so, hooray!

catch’s picture

catch
he/him
Primary language English
commented
Status: Reviewed & tested by the community » Closed (duplicate)

Marking this as duplicate of that issue.

gambry’s picture

gambry
Primary language Italian
Location Milan
commented

I can't find what has been changed as part of #2906317: Random fail due to problems with database , or any related work.

Last saturday I managed to see an AlreadyInstalledException issue again. It wasn't my machine so I can't give more info, but we manage to make it disappearing by properly set the permissions on folders sites, sites/simpletest and the one in BROWSERTEST_OUTPUT_DIRECTORY.

I promised myself to try to replicate it, thinking it was just a misleading error caused by something else. But now wondering if something has been fixed upstream?

gambry’s picture

gambry
Primary language Italian
Location Milan
commented

I found these potential works/solutions in that issue:
- #2939026: Installer does not fix the database connection after creating the database on MySQL
- #2817833: Delay sql map table creation

Thanks!

xjm’s picture

xjm
she/her
Primary language English
commented
Parent issue: #2829040: [meta] Known intermittent, random, and environment-specific test failures »