In cache-install.inc, there's this code:

  /**
   * Overrides DrupalDatabaseCache::clear().
   */
  function clear($cid = NULL, $wildcard = FALSE) {
    // If there is a database cache, attempt to clear it whenever possible. The
    // reason for doing this is that the database cache can accumulate data
    // during installation due to any full bootstraps that may occur at the
    // same time (for example, Ajax requests triggered by the installer). If we
    // didn't try to clear it whenever this function is called, the data in the
    // cache would become stale; for example, the installer sometimes calls
    // variable_set(), which updates the {variable} table and then clears the
    // cache to make sure that the next page request picks up the new value.
    // Not actually clearing the cache here therefore leads old variables to be
    // loaded on the first page requests after installation, which can cause
    // subtle bugs, some of which would not be fixed unless the site
    // administrator cleared the cache manually.
    try {
      if (class_exists('Database')) {
        parent::clear($cid, $wildcard);
      }
    }
    // If the attempt at clearing the cache causes an error, that means that
    // either the database connection is not set up yet or the relevant cache
    // table in the database has not yet been created, so we can safely do
    // nothing here.
    catch (Exception $e) {
    }
  }

However, the clear() method no longer exists; it was removed in #1272706: Remove backwards compatibility layer for cache API.

Given that this function apparently no longer gets called, I tried to reproduce some of the bugs described in the code comment (one I remember was that during 7.x-dev before this was added, the site name displayed incorrectly on the first page load after a new install) but I couldn't. So maybe if we're lucky something else changed such that we can just delete this, but I doubt it will be that simple...

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

pounard’s picture

There is still a possible silent bugs due to cache stall during install, the install backend remains a partial implementation that doesn't save caches but still delete cruft thanks to delete() deletePrefix() and flush(), we might want to keep it until we get back working on the install issue.

David_Rothstein’s picture

Title: The installer's cache backend implements the clear() method, which no longer exists » The installer's cache backend no longer overrides all cache-clearing methods, which can lead to fatal errors
Status: Active » Needs review
Issue tags: +Needs backport to D7
FileSize
16.74 KB
14.7 KB

OK, now I see why the previous bugs weren't reproducible. Kind of obvious actually: Since the installer cache wasn't overriding the methods it was supposed to be, they just fell back on using the regular database cache backend anyway, so the caches still got cleared.

What that means is that there is actually a different bug here. If the installer happened to call any code early on that tried to clear caches in particular ways, there would be a fatal error because the database isn't available yet. I guess it was just luck that that doesn't happen in the current codebase.

I see that #1323120: Convert cache system to PSR-0 actually fixed some of the bugs here, but there are a few remaining. The attached patch fixes them, and also adds a bunch of tests. We can't test the installer directly, but we can simulate it and thereby at least test that the installer's cache implementation does the correct thing. This is a little wonky and requires a fair amount of code, but it should be worth it.

The basic concept behind the tests is backportable to D7.

David_Rothstein’s picture

The original symptom of this bug has actually reappeared in Drupal 8 (see #1536262: Entering a site name when installing Drupal 8 has no effect until caches are cleared) although strangely the patch here doesn't seem to fix it, so maybe it's actually a different bug.

In any case, here's a reroll against the latest Drupal 8 codebase.

Status: Needs review » Needs work

The last submitted patch, installer-cache-1394648-3.patch, failed testing.

David_Rothstein’s picture

This version should work better.

It now contains fixes discussed in #1536262: Entering a site name when installing Drupal 8 has no effect until caches are cleared, as well as some other required alterations due to recent changes in Drupal core.

David_Rothstein’s picture

Status: Needs work » Needs review
David_Rothstein’s picture

Priority: Normal » Major

Moving to major priority, since the tests in this patch will catch/prevent the bug at #1536262: Entering a site name when installing Drupal 8 has no effect until caches are cleared, and that bug is classified as major so this should be also.

catch’s picture

The patch looks good. With the test itself, would it be simpler to just use the DatabaseBackend and InstallBackend classes directly in the test? Seems like we could avoid both the page requests and the helper module in that case. This is explicitly testing the interaction of those classes with each other, so I think it's fine if it doesn't go through the cache factory.

David_Rothstein’s picture

Status: Needs review » Needs work

Hm, I think you're right - there isn't any overwhelming reason the tests need to be using the cache() function, and removing it would simplify things.

David_Rothstein’s picture

Status: Needs work » Needs review
FileSize
11.99 KB
8.88 KB

Here's a new version that does that, and yes, it makes the tests a lot simpler. We still need the helper module (so we can delete the cache table midway through the test), but the module is now tiny and the tests are more straightforward and don't have to make any page requests.

I also added the new invalidateTags() method to the list of tests performed here.

pounard’s picture

This patch is looking good (it is simple and quite straightforward), I won't do any whitespace review, if it passes tests it's enough for me. I'd say it's RTBC.

catch’s picture

Status: Needs review » Reviewed & tested by the community

This looks fine to me too, marking RTBC.

catch’s picture

Issue tags: -Needs backport to D7

#10: installer-cache-1394648-10.patch queued for re-testing.

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs backport to D7

The last submitted patch, installer-cache-1394648-10.patch, failed testing.

David_Rothstein’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
11.93 KB

The conflict was here:

+use Drupal\Core\Cache\DatabaseBackend;
+use Drupal\Core\Cache\InstallBackend;
 use Drupal\simpletest\WebTestBase;

(The last line wasn't in the codebase before.)

Since there's no actual difference between the attached patch and the previous one, I'm boldly setting this back to RTBC...

Status: Reviewed & tested by the community » Needs work

The last submitted patch, installer-cache-1394648-15.patch, failed testing.

David_Rothstein’s picture

Status: Needs work » Reviewed & tested by the community
FileSize
11.93 KB

Oops. Less boldly this time, but still trying it again :)

The only change is trivial, from this:

+class CacheInstallTestCase extends DrupalWebTestCase {

To this:

+class CacheInstallTestCase extends WebTestBase {
David_Rothstein’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
11.93 KB

Actually, looking at the surrounding code we're probably better off doing this instead:

+class CacheInstallTestCase extends CacheTestCase {

Trying that now.

catch’s picture

Version: 8.x-dev » 7.x-dev
Category: bug » task
Priority: Major » Normal
Status: Needs review » Patch (to be ported)

I'll take the CNR change as an even less bold RTBC since this has come back green. I think the only thing that needs to be backported here is the tests (unless I missed something), so moving to 'normal task' for that. Committed/pushed to 8.x.

boombatower’s picture

Status: Patch (to be ported) » Needs review
FileSize
9.51 KB

Just backported the test only as per catch in #19. Was curious to try out my git backporting method using rebase. It works well (not to say test will pass). :)

Status: Needs review » Needs work

The last submitted patch, installer-cache-1394648-20.patch, failed testing.

xjm’s picture

Issue tags: +Novice

Fatal error: Class 'DatabaseBackend' not found in /var/lib/drupaltestbot/sites/default/files/checkout/modules/simpletest/tests/cache.test on line 449

This class has probably been renamed during a PSR-0 conversion; check the change notices or git blame history (http://www.kernel.org/pub/software/scm/git/docs/git-blame.html, http://drupal.org/node/1462360#comment-5677360) to find the new name. Then, update the patch.

disasm’s picture

attached is a patch that fixes the class names that were changed from 7.x->8.x. It also removes the tests for the methods that don't exist in the old implementation. This test is failing with the error:

PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'drupal8.simpletest320671test' doesn't exist: DELETE FROM {test} WHERE (cid = :db_condition_placeholder_0) ; Array ( [:db_condition_placeholder_0] => cache_one )

naxoc’s picture

Status: Needs work » Needs review
FileSize
970 bytes
4.68 KB

The tests run now. There was an assertion asserting false on something that would always be true

In DrupalFakeCache, isEmpty() is overriden to do this:

  function isEmpty() {
    return TRUE;
  }

There are assertions testing this in the test. That may be fine, but I feel like we need some more documentation on it. It certainly had me scratching my head.

xjm’s picture

Status: Needs review » Needs work

Thanks @naxoc and @disasm!

The issue is filed against 7.x currently, so we do want to test the D7 patches. (So we should remove the -do-not-test from the filename.) Ideally, we should also post a tests-only D7 patch along with the complete patch, in order to verify that the backported tests are providing coverage. Like so:
Only local images are allowed.

Thanks for providing the interdiffs; it's very helpful.

+++ b/modules/simpletest/tests/cache.testundefined
@@ -446,13 +446,13 @@ class CacheInstallTestCase extends CacheTestCase {
     // Store an item in the database cache, and confirm that the installer's
     // cache backend recognizes that the cache is not empty.
     $database_cache->set('cache_one', 'One');
-    $this->assertFalse($install_cache->isEmpty());
+    $this->assertTrue($install_cache->isEmpty());
     $database_cache->clear('cache_one');
     $this->assertTrue($install_cache->isEmpty());

Agreed @naxoc; this change makes me nervous. It seems to directly contradict the comment there? Here's the same hunk in the D8 patch:

+++ b/core/modules/system/tests/cache.testundefined
@@ -457,3 +459,144 @@ class CacheIsEmptyCase extends CacheTestCase {
+    // Store an item in the database cache, and confirm that the installer's
+    // cache backend recognizes that the cache is not empty.
+    $database_cache->set('cache_one', 'One');
+    $this->assertFalse($install_cache->isEmpty());
+    $database_cache->delete('cache_one');
+    $this->assertTrue($install_cache->isEmpty());

Definitely asserting false, which means there's a difference in the functional code. We need to debug this further... Meanwhile, though, let's get those test-only and combined D7 patches for the bot to test. :)

David_Rothstein’s picture

Well, the D8 patch contained a fix for the bug itself:

function isEmpty() {
+    try {
+      if (class_exists('Drupal\Core\Database\Database')) {
+        return parent::isEmpty();
+      }
+    }
+    catch (Exception $e) {}
     return TRUE;

So if we want that test to pass we'd need to make a similar change in Drupal 7.

But I wonder if maybe we shouldn't bother with that (and just leave that test out instead).... I think the current behavior is wrong, but it's not causing any harm either so maybe for Drupal 7 we should just keep it as is.

kid_icarus’s picture

Here are the patches renamed and reuploaded per #25.

The last submitted patch, drupal-installer_cache_test-1394648-27-complete.patch, failed testing.

  • catch committed abb86f9 on 8.3.x
    Issue #1394648 by David_Rothstein: Fixed The installer's cache backend...

  • catch committed abb86f9 on 8.3.x
    Issue #1394648 by David_Rothstein: Fixed The installer's cache backend...

  • catch committed abb86f9 on 8.4.x
    Issue #1394648 by David_Rothstein: Fixed The installer's cache backend...

  • catch committed abb86f9 on 8.4.x
    Issue #1394648 by David_Rothstein: Fixed The installer's cache backend...

  • catch committed abb86f9 on 9.1.x
    Issue #1394648 by David_Rothstein: Fixed The installer's cache backend...