Problem/Motivation
Upon (re)installing Drupal 9 or any 8.8.1+ version via the web browser, you’ll run into the The website encountered an unexpected error. Please try again later.
error. The installer works correctly with Drush.
You might also see a variation of these errors in the log files:
NOTICE: PHP message: Uncaught PHP Exception InvalidArgumentException: "Class "\Drupal\system\Controller\Http4xxController" does not exist." at /var/www/html/web/core/lib/Drupal/Core/DependencyInjection/ClassResolver.php line 24"
Or
Uncaught PHP Exception InvalidArgumentException: "No check has been registered for access_check.permission" at /var/www/html/web/core/lib/Drupal/Core/Access/CheckProvider.php line 97
For completeness, this error was also reported at https://drupal.stackexchange.com/questions/289480/class-drupal-system-co...
How to reproduce?
- Blank copy of Drupal with empty DB but with valid DB credentials in
settings.php
- Visit
/index.php
(this will redirect you to/core/install.php
) - Install Drupal via the web browser
- After you’re finished configuring the new site, you’ll get the
The website encountered an unexpected error. Please try again later.
error
The initial bug report also included those steps to reproduce with ddev:
- mkdir d8recommended && cd d8recommended &&. ddev config --project-type=drupal8 --docroot=web --create-docroot
- ddev start
- ddev composer create drupal/recommended-project
- ddev config --project-type=drupal8
- Tail logs with "ddev logs -f"
- Visit the *http* URL (for example, http://d8recommended.ddev.site) and do the web-based installation. You'll get these errors when you finish configuration on the final page.
Proposed resolution
- Find a way to prevent cache pollution when installing Drupal via the web browser.
- Ensure it meets the requirements to solving SA-CORE-2019-009 gracefully.
Remaining tasks
Discuss the best approach and provide a patch with tests.
User interface changes
The web installer works again as expected.
API changes
None.
Data model changes
None.
Release notes snippet
None.
Comment | File | Size | Author |
---|---|---|---|
#116 | 3103529-9.0.x-115.patch | 3.73 KB | alexpott |
#116 | 3103529-8.x-115.patch | 4.85 KB | alexpott |
#116 | 114-115-interdiff.txt | 763 bytes | alexpott |
Comments
Comment #2
rfayComment #3
rfayComment #4
cmah CreditAttribution: cmah commentedI just had this exact same error. The easiest way around it right now is to re-run everything with a
ddev exec drush site:install
instead of doing the web-based installation. Since the web-based installation is assumed in the DDEV Drupal 8 Quickstart, this bug is going to trip up a lot of people.Comment #5
rfay@cmah I assume this failure happened to you with the drupal/recommended-project template, true? As a result of my experience with the new template, I have postponed the ddev docs PR that would have started recommending the "official" template, and am continuing to recommend the drupal-composer/drupal-project template in ddev's docs. AFAICT there are many, many advantages to the traditional, mature drupal-composer/drupal-project template. The official template also really struggles to build on Windows also.
Comment #6
cmah CreditAttribution: cmah commented@rfay, yes, I was using the recommended-project template. Every day some new little thing goes wrong with things I've built using that template. Thank you for the advice, I'm going back to drupal-project for now. I'm sick of cleaning up messes that result from me being overeager to get rid of webflo/drupal-core-strict.
Comment #7
hey_germanoI'm seeing this on Pantheon as well. I ran through an 8.8.1 install in the browser on ~7 new sites, and this error showed up in the PHP error log each time. It yielded a WSOD during the install in a few of those tests, but mostly just surfaced in the logs. Refreshing the page cleared up the WSOD, and I was able to continue to the configuration page as usual.
These installs were over HTTPS, running PHP 7.3, using the drupal/legacy-project template (https://github.com/pantheon-systems/drops-8/blob/default/composer.json). It comes up using both the Standard and Minimal install profiles.
I was not able to reproduce this on a new install on a DigitalOcean droplet over HTTP, with PHP 7.3 and the drupal/recommended-project template.
Comment #8
cilefen CreditAttribution: cilefen commentedIt almost seems an autoloader bug, or something to do with opcache configuration (?)...
Comment #9
chingis CreditAttribution: chingis commentedJust to clarify – this is not only drupal/recommended-project template or ddev issue, as was reported in https://github.com/drupal-composer/drupal-project/issues/541 if you just get plain drupal-composer/drupal-project and replace Drupal core version in composer.json to 8.8.1 you'll get this error
Comment #10
chingis CreditAttribution: chingis commentedDisabling opcache doesn't help
Comment #11
hart0554 CreditAttribution: hart0554 at University of Minnesota commentedI am encountering this issue as well, both in a local DDEV environment as well as on Acquia - PHP 7.2, Drupal 8.8.1 with a subprofile of Lightning.
Comment #12
chingis CreditAttribution: chingis commentedhttps://github.com/drupal-composer/drupal-project/issues/541#issuecommen...
I've installed a clean drupal-composer project without any modifications to composer.json and I still get the WSOD (latest PHP 7.3). No more "class not found" error but the following exception is still here:
Comment #13
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedI have encountered both of the errors reported on this card, and also:
I don't know if all three of these symptoms come from the same root cause, but they all happen at about the same time (right after the configure site page in the Drupal installer).
Comment #14
jaydee1818 CreditAttribution: jaydee1818 commentedI'm also running into this
Comment #15
hmartens CreditAttribution: hmartens commentedHmm, I am also running into this issue. Thank you to whoever manages to fix this.
Comment #16
hmartens CreditAttribution: hmartens commentedThank you cmah, I can confirm that the following command worked:
ddev exec drush site:install
Comment #17
vliefooghe CreditAttribution: vliefooghe commentedI saw a quite similar problem here (I think) : https://github.com/hechoendrupal/drupal-console/issues/1759
I was able to solve the issue when cleaning all cache_* tables (truncate table cache_xxx)
Comment #18
jdleonardI just installed 8.8.1 in the browser using the Lando local development environment with the Pantheon recipe. "lando drush cr" to rebuild the cache seemingly solved the issue for me.
Comment #19
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedFor what it's worth, I have not been able to work around this bug with a
drush cr
; however, truncating the cache tables per #17 worked reliably.Comment #20
catapipperI get about 50% success when running a
drush cr
ordrupal cache:rebuild
. However I have had 100% success when clearing just thecache_container
table. I don't have to clear all thecache_
tables.Comment #21
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedConfirm truncating just
cache_container
per #20 also clears up the problem for me.Comment #22
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedI have not traced the source of the problem (the installer finite-state-machine maze is not an easy place to have a casual stroll), but it does appear that there's something about being in the installer that prevents cache rebuild from building the cache_container correctly.
The attached highly pragmatic patch clears up the problem 100% of the time for me.
Comment #24
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commented#22 is failing multiple times due to a test that is checking for the existence of the cache table I truncated.
Comment #25
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedI took a site that was failing 100% of the time and rolled it back to 8.8.0, and a clean site install worked. That is not necessarily indicative of anything, for an intermittent problem, but maybe if I do the most painful git bisect ever...
Comment #26
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedThere are only a few commits between 8.8.0 and 8.8.1, but I wasn't able to get a reproducible case to hold long enough to bisect them. Might try again later.
Comment #27
uberhacker CreditAttribution: uberhacker as a volunteer and commentedI had this exact same issue a few days ago. It wasn't enough to simply run `drush cr`. I had to truncate all the `cache_%` tables with queries. FWIW, truncating a table does not remove the structure; only the data, so any check for the table existence should still work even after truncating. Is it possible other cache tables need to be truncated as well?
Comment #28
chingis CreditAttribution: chingis commentedJust checked with 8.8.2 – the issue is still here. I'm surprised not that many people responded here, from what I understand just the clean setup via https://github.com/drupal-composer/drupal-project and installation from a browser gives you WSOD
Comment #29
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedHere's a bit of good news. I ran three tests:
To do the third test, I cherry-picked the following commit on top of 8.8.0:
7ae15f9c3d 2019-12-18 Lee Rowlands SA-CORE-2019-009 by mcdruid, larowlan, Heine, alexpott, xjm, DamienMcKenna, dsnopek, catch, greggles
This patch isn't too long. Perhaps we can isolate the problem by inspection, or come up with other experiments to try.
Comment #30
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedSA-CORE-2019-009 is essentially two lines + tests.
Comment #31
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedHm, after #28 I did not try 8.8.2, but I just investigated and found that 7ae15f9c3d was reverted in that version. I also cannot reproduce this problem in 8.8.2.
Fixed?[UPDATE: This observation was in error. 7ae15f9c3d was never reverted.]
Comment #32
kaythay CreditAttribution: kaythay at Aten Design Group commentedWe are still seeing this on a Pantheon upstream install of Drupal 8.8.2. With the patch in #22, the problem is mostly mitigated but still results in a lingering "Plugin (xx) instance class xx does not exist" warning. We have modules that are optionally enabled on the "Configure site" form, which is where the original WSOD occurred. It has not been replicable on a local installation.
Comment #33
fkelly12054@gmail.com CreditAttribution: fkelly12054@gmail.com as a volunteer commentedTrying a new composer recommended-project based install of 8.8.2 (nothing reported in 8.8.2 issue queue yet) I got the error reported in #12
Uncaught PHP Exception InvalidArgumentException: "No check has been registered for access_check.permission" at /var/www/html/web/core/lib/Drupal/Core/Access/CheckProvider.php line 97
after the install site step. Dropping all tables and retrying I wind up with:
and going to the error page I see:
Plus some additional errors. Drush cr says it clears cache but doesn't resolve the problem. Emptying all the cache tables through Phpmyadmin doesn't help either.
Comment #34
fkelly12054@gmail.com CreditAttribution: fkelly12054@gmail.com as a volunteer commentedJust for kicks after reporting #33, I emptied the database and did a drush site:install. Worked like a charm though it does assign it's own admin user and password which I'll have to adjust. Same exact Drupal files I was using with the web install on a local system.
Comment #35
ressa CreditAttribution: ressa at Ardea commentedI just tried installing Drupal 8.8.2 with Lando v3.0.0-rc.23 in Ubuntu 18.04 via
http
, which went well:Installed via GUI at http://d8recommended.lndo.site/ without errors.
Comment #36
rfayFirst, @greg.1.anderson I want to thank you so much for spearheading this issue. You're awesome.
Second, it's weird, but I've done lots of web installs in recent days and haven't been seeing this. It was happening with regularity (not consistent, but regular). Now... I haven't seen it in a week at least. Maybe 2. Drupal 8.8.2? PHP 7.3.13 (23 Jan, made it into ddev with v1.13)?
Comment #37
rfaySo much for that. I just did an 8.8.2 install via drupal-composer/drupal-project and clicked through to the end and got the dreaded "NOTICE: PHP message: Uncaught PHP Exception InvalidArgumentException: "Class "\Drupal\system\Controller\Http4xxController" does not exist." at /var/www/html/web/core/lib/Drupal/Core/DependencyInjection/ClassResolver.php line 24"
It's not fixed, at least with drupal-composer/drupal-project
Comment #38
rfayJust hit this again. drush cr didn't solve it (as far as letting me proceed without the error)
mysql -e 'truncate cache_bootstrap; truncate cache_config; truncate cache_container; truncate cache_data; truncate cache_default; truncate cache_discovery; truncate cache_entity; truncate cache_menu; truncate cache_render;'
also didn't solve it (after the fact). And https://www.drupal.org/project/drupal/issues/3103529#comment-13443470 didn't solve it (after the fact). I bailed and `drush si`
Comment #39
rfayDoes everybody already know (except me) that this can be solved when it goes bad with by just hitting /update.php ? I wish I'd known that all along.
Also, I note that this may not happen often with umami, and that's probably why I didn't hit it for a week or more.
Comment #40
drakythe CreditAttribution: drakythe commentedI had no idea. Does that also hold true for invoking drush updatedb?
Comment #41
alejo-moreno CreditAttribution: alejo-moreno at CodeLar commentedrunning
drush cr
at least 3 times fix it for me.Comment #42
Crell CreditAttribution: Crell at Platform.sh commentedI am seeing a slightly different error message but overall the same symptoms and same apparent cause on Platform.sh. cf: #3113802: Installer fails with cache issue
I also see what rfay reports, that it doesn't happen with the Umami profile. I theorize it's because Umami also downloads translations, which results in an extra cache clear happening which fixes whatever it is the normal install leaves broken.
Comment #43
Crell CreditAttribution: Crell at Platform.sh commented@rfay: Hm. I am not able to clear it by visiting update.php. I get the same error at that page. And, to make life interesting, *sometimes* drush cr fixes it, sometimes not. I have not figured out the difference.
Comment #44
Crell CreditAttribution: Crell at Platform.sh commentedAnd just to make life even more interesting, it looks like the issue results in a non-deterministic error, at least for me.
I ran through the install once, and got this error in the log:
Drupal\Component\Plugin\Exception\PluginNotFoundException: The "user" entity type does not exist. in /app/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php on line 150 #0 /app/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php(269): Drupal\Core\Entity\EntityTypeManager->getDefinition('user')
I wiped and reinstalled, with the exact same code, and got this error:
Uncaught PHP Exception InvalidArgumentException: "No check has been registered for access_check.permission" at /app/web/core/lib/Drupal/Core/Access/CheckProvider.php line 97
(As in #3113802: Installer fails with cache issue.)
In both cases, a drush cr did *not* fix it, nor did update.php. At this point... I'm kind of confused why a cache clear does fix it sometimes.
Comment #45
drakythe CreditAttribution: drakythe commentedI'm glad someone else is getting inconsistent cache rebuild results. I can verify those because the first time I showed this issue to someone else he did a cache rebuild and everything worked. When I tried to replicate his "fix" mine was still broken.
Comment #46
Flo1480 CreditAttribution: Flo1480 commentedHello,
I've exactly the same error, on 8.8.2 on a kubernetes containers drupal:latest.
From the Web interface:
From the pod log:
Florent
Comment #47
Chris Burge CreditAttribution: Chris Burge at University of Nebraska commentedI believe this is being caused by SA-CORE-2019-009. From the advisory:
When I install 8.8.3-dev, I consistently observe the InvalidArgumentException exception documented above (Standard install profile via GUI)
If I revert the code changes made in SA-CORE-2019-009, then installation completes normally. (This basically confirms @greg.1.anderson's work from #29.)
If I apply patch #22 (without reverting SA-CORE-2019-009), then installation completes normally.
In all instances, I started from a fresh drupal/recommended-project project in a new DDEV container.
Because this was a security issue, there's no public issue to which we can cross post. Nine individuals are credited on the security advisory; however, none of them have commented on this issue. I think it's safe to assume that it's not currently on their radar. I just sent Drew Webber (mcdruid), the security issue reporter, a DM.
Comment #48
mcdruidI received the ping, but don't have much bandwidth to look at this at present.
As mentioned previously SA-CORE-2019-009 really only made two small changes, if you exclude tests:
...and:
It might help to isolate what's going on if we could confirm whether reverting just one of these avoids the problem(s) outlined here?
Comment #49
longwave@Crell determined in #3113802-5: Installer fails with cache issue that you need to revert both parts to fix the issue, reverting just one or the other doesn't do it.
As both changes are about dumping the container I guess this makes sense; it looks like whatever we were doing to dump the container during install before was required, now we have disallowed it then this bug occurs.
Comment #50
jaims-dev CreditAttribution: jaims-dev as a volunteer commentedFor what is worth, I have run into this issue when installing from a gzip tarball freshly downloaded from drupal.org. With composer create-project drupal recommended project it worked just fine. Version is 8.8.2.
When creating a subsite for a multisite setup, I ran again into the same issue... which I could circumvent by means of drush cr + visiting /update.php.
Definitely something you want to try because it has worked for me (and for some others). I take it that that is not the case for some other drupalists here, though.
Good luck!
Comment #51
Mile23I saw this same error, using ddev and core 8.8.2. I installed through the web browser.
Based on #47, the workaround is:
This got me to my newly-installed site.
It's fair to say that
drush cr
is the magic incantation for far too many things. :-)Comment #52
Chris Burge CreditAttribution: Chris Burge at University of Nebraska commentedBecause discussion relating to SA-CORE-2019-009 is locked away in security.drupal.org, we don't have access to the discussion that led to the the two code changes contained therein. That makes it challenging to address the resulting regression. If we simply revert the code changes made as a result of SA-CORE-2019-009, then we've fixed the bug and also re-opened the security vulnerability. Is there a way to address SA-CORE-2019-009 without breaking the installer? We can't answer that question. And that's a problem.
At this point, the only way forward (so far as I can see) is to update tests to accommodate patch #22. Hopefully when this issue hits RTBC someone who knows why SA-CORE-2019-009 happened can weigh in and 1) validate the solution from #22 and 2) lead discussion in another direction.
Comment #53
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commented@Chris Burge: The world has advanced since #22. It appears that the fix for SA_CORE-2019-009 was backed out for 8.8.2, but despite that fact, this bug still exists in 8.8.2. My hypothesis is that the security issue was fixed in a different way, perhaps in an attempt to fix this problem. Since this is a security issue, though, I do not have any insight into the motivation or cause. I have not had a chance to try to bisect all of the commits between 8.8.1 and 8.8.2 (or even count how many there are); this is a time-intensive problem, and it took me quite a while to do it for #22.[UPDATE: Confirmed that I was mistaken: SA_CORE-2019-009 was never backed out.]I will also reiterate that
drush cr
only fixes this problem in some instances. It is also possible to install Drupal 8.8.2 from the web UI and occasionally not encounter this problem. I have definitely seen sites get into a state wheredrush cr
does not fix the problem (c.f. #43). Truncatingcache_container
always fixes the problem (c.f. #20), so it seems that bad data in this table can preventdrush cr
from working correctly. This table is very large and complicated, though; I think it would be difficult to try to learn anything by analyzing the difference between a working and a non-workingcache_container
table. I did, however, confirm that whendrush cr
does not fix the problem, that thecache_container
table comes out the same way every time, whereas if you then truncatecache_container
anddrush cr
again, the result has different data in it.Comment #54
Chris Burge CreditAttribution: Chris Burge at University of Nebraska commented@greg.1.anderson, your notes about cache rebuild (via drush) vs truncating the cache_container table are helpful. I've been using a carpet bombing approach when I need to install sites (drush cache-rebuild, hit update.php, truncate cache_ tables, try cache-rebuild again, and so on until the error clears).
I'm not seeing that SA_CORE-2019-009's code was reverted. The code still exists in both 8.8.x, 8.9.x, and 9.0.x.
+1
Comment #55
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedHm I'll have to check this again when I have a chance. A re-roll of #22 for 8.8.2 would be a useful demonstration of applicability.
Comment #56
Chris Burge CreditAttribution: Chris Burge at University of Nebraska commentedUpdated patch attached. It checks if the 'cache_container' table exists before attempting to truncate it, which should address the test failures. The patch also fixes a couple of minor coding standards issue.
Comment #57
Crell CreditAttribution: Crell at Platform.sh commentedBoth #22 and #56 apply cleanly for me and fix the issue on 8.8.2.
I make no claim that it's the right fix, but it does seem to work.
Sadly it doesn't seem to also fix the issue of not being able to install with Redis for the cache backend; that still requires enabling after the install is complete; I was hoping that the sledge hammer approach here would fix it but I guess not. If I have the Redis glue code enabled it still fails with the same type of error we're seeing for this issue. ("No check has been registered for access_check.permission" in my case.)
Comment #58
greg.1.anderson CreditAttribution: greg.1.anderson at Pantheon commentedThanks for #56. I misinterpreted the test fails in #24.
I also do not represent that this is the right fix, but at least it gives us something to work with.
Comment #59
Chris Burge CreditAttribution: Chris Burge at University of Nebraska commentedThe Redis installation issue described by @Crell in #57 is interesting. We're planning on using Memcache; we just haven't deployed it yet (no timeline unfortunately). I would be interested to know if installations using Memcache run into the same issue as Redis.
Here's the real kicker - since this is a core bug, the core folks are going to want a test (a test that fails without the patch but passes with the patch). I have no idea how to reproduce this bug in the automated testing environment. I guess we'll leave it here for now.
Comment #60
mcdruidThere's not a lot more to SA-CORE-2019-009 than what was published in the SA (and what can be gleaned from the tests that were added).
The idea was to prevent the installer from polluting the cache of the container on a site which has already been installed.
It does, therefore, seem quite odd to be having to clear out cache_container at the end of an installation. I've not had a chance to dig into what's actually going on though.
If clearing the container cache makes the installer work again where it's been broken that's great, but it does seem more like a workaround than a long term fix, at least at first glance.
Comment #61
Crell CreditAttribution: Crell at Platform.sh commentedThe Redis install issue I mentioned is here: #2876132: Cannot install Drupal with Redis.
To be clear, that's a far lower priority than this issue so solving just this one for now would be fine. However, since they both revolve around the cache system/installer interaction they may be related, so it may be a hint to how to address one or both of them.
Comment #62
VVVi CreditAttribution: VVVi commentedI also met the error, while installing Drupal 8.8.3 . Then while the new installation, after appearing the first installation page, I have removed the table cache_container, and it helped finish the installation successfully. So I did the same by hands, as in the patch from #56.
Comment #63
Warped CreditAttribution: Warped as a volunteer commentedJust to throw another monkey wrench into the pond, I've run into this several times with web install,
Drupal 8.8.3, drupal/recommended-project, DDEV 1.13.1, PHP 7.3.15-4+0~20200224.55+debian10~1.gbpbea824
This has happen when I installed Drupal Standard profile for the last several versions.
Googled and found an issue from several years old for WSOD that recommended using "drush uli".
Can't find the issue again, but it has worked to log me in after WSOD and site was ok after that.
Comment #64
cilefen CreditAttribution: cilefen commentedThis qualifies as major.
Comment #65
anavarreRemoved 8.8.1 from the issue title as it's happening with 8.8.3 as well.
Comment #66
turbogeek CreditAttribution: turbogeek commented8.8.3 UI installation on Pantheon resulted in: InvalidArgumentException: Class "\Drupal\system\Controller\Http4xxController" does not exist
As per #39, hitting update.php fixed it - thank you. It looks like the installation process completed fine. Being a cache issue I was hoping so.
Comment #67
mcdruidI think what's happening here (at least some of the time) is that we have a blank install of Drupal with an empty database (the configuration is in place so Drupal knows how to connect to the db, but there's no data in there yet), to which we're sending an initial request.
Drupal boots its kernel to handle this request, and as part of that it initializes a new container. As part of this process, it caches (dumps) that container to the db - specifically into the cache_container table. Here's a screenshot showing xdebug catching this process as the container is dumped using an upsert query:
https://git.drupalcode.org/project/drupal/-/blob/8.8.3/core/lib/Drupal/C...
If I understand correctly, that now means we have cached details of a very incomplete service container which is hardly able to actually respond to a http request.
In fact, once Drupal's finished dumping this container, the kernel tries to handle the request and hits an exception because the db is essentially empty and the first tables it tries to access don't exist.
The kernel catches this exception and helpfully redirects the user to the installer:
https://git.drupalcode.org/project/drupal/-/blob/8.8.3/core/lib/Drupal/C...
However, it looks like the changes introduced to the installer to try and stop it from polluting the container cache of an already existing site mean that when the installer's finished and the user is redirected back to index.php, it's the broken incomplete container that Drupal tries to load back out of cache.
IIUC the way it would have worked before, the installer container would have been dumped to cache and then reloaded by index.php, which I don't think was exactly correct... but it was at least not so broken that it couldn't handle the initial http request when the installation finishes.
There may be other paths that people hitting different errors are following, but I think this describes what I just reproduced.
How do we fix this? There are probably several different approaches.
My first thought is that perhaps when we handle the exception because the db is empty/incomplete and decide to redirect to the installer - perhaps we could ensure that cache_container is not full of junk at that point?
Here's a patch which I think does that at an appropriate point in the logic when we're deciding to redirect to the installer. There may well be cleaner ways to implement this, but the idea is to avoid reaching the installer with a broken container in cache as opposed to having to clear the cache once the installer's done.
Does this cause any test fails? Does this resolve the issue people have been experiencing?
Of course, it's a bit of a concern that no tests caught this issue in the first place. If this approach works we should ideally add tests to avoid regressions.
Comment #68
mcdruidOops an interdiff is not a patch (as Apu almost said).
Comment #69
Chris Burge CreditAttribution: Chris Burge at University of Nebraska commented@mcdruid - Thanks for your work on this issue!
When I apply patch #67 to 8.8.x-dev, the installation completes without issue (Standard install with drupal/recommended-project using DDEV). Existing tests passed :-)
Comment #70
neclimdulI haven't looked at _why_ this works but it definitely did and saved me from banging my head against a wall testing something without drush. Thanks everyone.
Comment #71
anavarreIf this happens on D9 indeed, this warrants promoting to critical and back-porting to 8.9.x next
Comment #72
anavarreComment #73
alexpottI think the analysis in #67 is spot on. Reproducing this in a traditional test might be tricky because classloading is fun (especially on DrupalCI with concurrency and APCu around).
The fix looks okay but I wonder if an explicit dump of the container in the installer is a better more robust path.
Comment #74
mcdruidI might be misunderstanding, but I thought we don't want to dump the container from the InstallerKernel because it's (often only subtly) different to proper DrupalKernel... which is what we tried to address in SA-CORE-2019-009.
Comment #75
mcdruidSorry, didn't mean to revert the metadata.
Comment #76
alexpott@mcdruid yeah you're right - but it when we switch back to the other container we want to ignore the old cached container - or for it not to be there. An alternative fix for this would be to include the module list in the container cache key. Imo this is a pretty good idea.
Comment #77
alexpottHmmm.... getting the module list before the container is tricky that's not going to work.
Comment #78
anavarreHere's an attempt at rewriting the IS. Thanks @mcdruid for reviewing.
Comment #79
alexpottAlternatives to consider are:
The conclusion I've come to is that we shouldn't cache the container if core.extension does not exist in config. That means the system is not yet installed.
Comment #80
anavarreComment #81
mcdruidI like the approach in #79; preventing the dumping of the incomplete container's better than deciding when to flush it out of the cache.
Doing it this way may well prevent / fix problems we don't (yet) know about.
I manually verified this works by dropping all tables and visiting index.php which redirects to core/install.php
cache_container
table has been created.I'll leave it for others who have been encountering this problem to RTBC, but definitely a +1 from me.
Comment #82
alexpottHere's a test :) The test only patch is the interdiff.
Comment #84
Gábor HojtsyHm, the fail is
is this what we expected?
Comment #85
alexpott@Gábor Hojtsy yep it is the expected failure. It's coming from the first use of the container where its state is not as expected in the test running class. We could potentially try to get the exact same failure out of the system but it's going to require changes to the InstallerTestBase and the like which seem unnecessary given that the test proves that the process is broken at the moment if you:
and proves that the changes to DrupalKernel fixes it.
Comment #86
Gábor HojtsyAh, ok, makes sense. Looks good then :)
Comment #87
catchThanks for leaving open the possibility of system and user modules being non-required (i.e an empty module list).
Patch looks good. Committed/pushed to 9.0.x/8.9.x/8.8.x, thanks!
Comment #94
Gábor HojtsyBeen reverted so moving back to RTBC for now. Awaiting info on why it was reverted.
Comment #95
catchWe immediately started getting random test failures, https://www.drupal.org/pift-ci-job/1611929 is one example. @alexpott was going to update with info but looks like he didn't get to it yet.
Moving to needs work since we need a new patch here.
Comment #96
Crell CreditAttribution: Crell at Platform.sh commentedTesting update:
The patch in #82 fixes the install issue on Platform.sh using the standard profile.
Curiously, when I enable Redis integration by default it finishes the install now, BUT it still has issues. The home page is a 404 error, there is no site title, and going to the Site Information config page gives a fatal error that "Source path has to start with a slash." A cache through the UI fixes all of that, however.
That issue does not appear when Redis is not enabled by default.
So it fixes *most* of the current installer/cache issues, but not quite all. (Still, as said above the Redis-by-default question is not a critical path so shouldn't block this patch.)
The test fails appear to be all Postgres-related; I've no experience there so I'm not much help.
Comment #97
alexpottUnfortunately this appears to have broken DrupalCI - see https://www.drupal.org/pift-ci-job/1611952 and https://www.drupal.org/pift-ci-job/1611959
It introduced a random fail. It happens on every db driver and appears to be fixed by
Changing the requested url to
$this->drupalGet($GLOBALS['base_url'] . '/admin');
- i.e. getting the installer redirect from a different page.Comment #98
alexpottSo this is being caused by the incorrect caching of config objects used by services prior to installation. Fortunately we have a service provider that can fix this nicely for us. I wonder if this fixes the Redis issue too.
Comment #99
alexpottOops so that fixed the issue of premature creation of cache config but now it is preventing the very first write to config while installing the system module! Fun.
Comment #100
alexpottThis fixes it. We only need to do all this weirdness when \Drupal\Core\Installer\InstallerKernel::installationAttempted() return FALSE. I.e. we've not installed Drupal but we're not in the installer. The installer takes care of itself.
Comment #101
mcdruidNit: s/it's/its
Comment #102
mcdruidD'oh
Comment #103
catchOne more nit: 'to the installer'.
Both are fixable on commit so does not require a new patch necessarily.
The approach looks good, kicking off a couple more tests.
Comment #105
alexpottNice a Simpletest comesback to bite. So the fix is good but
Drupal\simpletest\Tests\KernelTestBaseTest
must be doing something interesting with a container with no core.extensionComment #106
alexpottSo here's a potential fix which says ignore this special case if the Kernel environment is "testing" - which is the environment we use in KernelTestBase tests. I think this is acceptable because we never actually do a real install of the system in KernelTestBase tests. Note that PHPUnit Kernel tests are not affected by this because they set the kernel's module list prior to compiling the container for the first time. Compare \Drupal\simpletest\KernelTestBase::setUp() and \Drupal\KernelTests\KernelTestBase::bootKernel()
An alternative fix is to set the $GLOBALS['install_state'] global and unset it again in \Drupal\simpletest\KernelTestBase::setUp() but that seems even more odd.
Comment #107
catch#106 seems reasonable, presumably we can do the #100 version in Drupal 9 though?
Comment #108
alexpottInteresting it seems not. We've changed something in D9 that has changed how that works. See D9 test run for #100. Will find out what.
Comment #109
alexpottSo here's why PHPUnit kernel tests with no modules (like \Drupal\KernelTests\Core\File\PharWrapperTest) pass on D8 but not on D9. On D8 they are getting the path_alias module :)
Comment #110
alexpottWe can improve the test by doing an earlier assert to ensure we have an empty database. I think this new assert makes the test failure better and also helps us to ensure that nothing unexpected like this happens again.
The changes in #106 to core/tests/Drupal/FunctionalTests/Installer/InstallerTestBase.php whilst technically correct aren't actually necessary.
Comment #112
alexpottI think I've come up with a better way for kernel tests to avoid the pre-installer mode that doesn't rely on a magic string.
Comment #113
Crell CreditAttribution: Crell at Platform.sh commentedTesting the 8.x patch from #112 on Platform.sh against 8.8.3, standard profile: The installer completes successfully, for normal install *and* with Redis pre-configured. Score. :-)
Minor nit:
"Ensure to set" isn't really English. Suggestion: "Set the module list upfront to avoid the kernel switching into pre-installer mode."
(That comment appears twice.)
Comment #114
andypostqueued for sqlite and pgsql because could be affected by #2949229: SQLite findTables Returns Empty Array on External DB.
Comment #115
alexpottSure the "Ensure to set" came from existing code but we can fix that here.
Comment #116
alexpottI meant to @see the code in the installer not kernel test base because that is now handling this its own way.
Comment #117
alexpott@andypost if that doesn't work then none of the called to find tables in any other test would work. It's not an external database - it's querying against the one that is set up for every test. Plus #110 was tested against all the db drivers. Not because of this - but to prove that that approach fixed the random fail that was introduced when we committed #82 - that was failing about 50% (or maybe more of the test runs).
Comment #118
xjmComment #119
andypostAs tests pass +1 rtbc
Comment #120
catchRTBC from me too, good that we were able to avoid the SimpleTest hardcoding.
Comment #121
jungleManually tested on my local, reproduced the bug on the latest 8.8.x, and the patch works as expected. But did not reproduce the bug on the latest 8.9.x and 9.0.x, so RTBC +1 only to 3103529-8.x-115.patch
Comment #122
alexpott@jungle the test proves that this occurs on 8.9.x and 9.0.x - see #110 test only patch. Also the random test fails introduced by committing #82 on all branches show that all branches are affected.
Comment #123
jungleThanks, @alexpott.
It did not mean denying 8.9.0 and 9.0.x patches not working. I Just could not confirm that by my manual tests :)
Comment #127
catchCommitted/pushed to 9.0.x, 8.9.x, and 8.8.x, thanks!
Comment #130
alexpottThe 9.0.x patch got committed on 8.x branches resulting in the Simpletest failing. I reverted and committed the correct patch.
Comment #131
rfayThis still happens in Drupal 8.8.4. Very sad to say.
Comment #132
longwave@rfay: 8.8.4 was a security release, this is only in -dev for now and will be part of 8.8.5.
Comment #134
Lars Skjærlund CreditAttribution: Lars Skjærlund as a volunteer commentedThis is still happening to me with 8.8.5.
Comment #135
drakythe CreditAttribution: drakythe commentedLars - What environment are you installing the new drupal system on? I've used DDEV successfully to install 8.8.5 multiple times in the past week, both from the front end and using drush site-install.
Comment #136
Lars Skjærlund CreditAttribution: Lars Skjærlund as a volunteer commentedThe problem is persistent and reproducible - however, it varies according to the attempted setup.
Basically, I'm trying to build a new site as a Docker image that's going to be deployed in Kubernetes.
My Dockerfile for now looks like this:
This is missing quite a few bits and pieces - but it's only intended for experiments so far.
I'm building the code on an Ubuntu 19.10 machine running PHP 7.3.11 - stock Ubuntu. Composer version 1.10.1.
I create the base code tree with these commands:
and then build a Docker image using the file above. Database is hosted on my MariaDB server, and I run the Docker image locally on the Ubuntu host.
Running install.php, I've tried three options: A recommended install in Danish, a recommended install in English, and a minimal install in English. Surprisingly, the last option creates most problems.
I've tried all options several times and created a bunch of screendumps - the problems appear to be systematic and reproducible.
The least problematic is actually the most involved: The recommended install in Danish only fails with an exception during "Install site". At this point I run
on the DB, reload the page in the browser, and "Install site" starts over - this time with messages in English compared to the earlier Danish messages. It runs to an end - and then I can't remember whether it stops with a WSOD or not. If it does, calling update.php from the browser solves the problems. After this, the site works fine.
Other installation options creates other problems that has to be handled in a similar but much more involved way - like truncating tables more than once and with the minimal install, truncating all cache tables is needed at one point.
Hope this helps - I can provide more info if need be.
Best regards,
Lars
Comment #137
Lars Skjærlund CreditAttribution: Lars Skjærlund as a volunteer commentedArgh, just tried again: Now all of a sudden, I have a lot of problems even installing a Danish standard install. So it isn't so predictable after all.
But I can still manage to install the site by truncating the cache enough times.
Furthermore, I now see errors like
Best regards,
Lars
Comment #138
leevh CreditAttribution: leevh commentedMy site was down today and I found these errors. Site is running 8.9.10 and has been running fine for months, no recent code changes, only users creating content. Cache-rebuild fixed it. I would say this affects more than just new installs :(
a bunch similar to:
a few:
[31-Dec-2020 15:02:13 UTC] Uncaught PHP Exception Drupal\Component\Plugin\Exception\PluginNotFoundException: "The "user" entity type does not exist." at /data/disk/host/static/live/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php line 150
Comment #139
chanderbhushan CreditAttribution: chanderbhushan as a volunteer and at gai Technologies Pvt Ltd for gai Technologies Pvt Ltd commentedwe are also getting the error no recent code changes, only users creating content. . Cache-rebuild fixed it,
Drupal\Component\Plugin\Exception\PluginNotFoundException: The "user_role" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 150 of /app/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php).
Drupal\Component\Plugin\Exception\PluginNotFoundException: The "user" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 150 of /app/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php)
Drupal\Component\Plugin\Exception\PluginNotFoundException: The "node" entity type does not exist. in Drupal\Core\Entity\EntityTypeManager->getDefinition() (line 150 of /app/web/core/lib/Drupal/Core/Entity/EntityTypeManager.php).
Comment #140
myself.csingh CreditAttribution: myself.csingh as a volunteer commentedIt started working for me after following steps below:-
i) Open settings.php (/sites/default/settings.php) in any plain text editor. Add this line to the end of the file and save it:
$settings['rebuild_access'] = TRUE;
ii) Visit http://www.example.com/core/rebuild.php in your browser (where www.example.com is your site’s URL). After a short pause, you should be redirected to the home page of your site, and the cache should be rebuilt.
iii) Open settings.php (/sites/default/settings.php) in a text editor. Find the line you added with $settings[rebuild_access], remove this line, and save the file.
Ref: https://www.drupal.org/docs/user_guide/en/prevent-cache-clear.html
Comment #141
renguer0 CreditAttribution: renguer0 as a volunteer commentedCreating `core/modules/system/src/Controller/Http4xxController.php` after trying to update and fixing some permissions in folders fixes the error here.