Problem/Motivation

Opcaches such as the one built in to PHP 5.5 are causing problems during install as settings.php is edited by the installer and the database form has to be submitted twice. After submitting database information during the installation, the screen asking for database information ("Database configuration") appears again. One would expect the installation to start using Batch API. Sometimes, after the second submit, get "table batch already exists" error.

Cause: during install.php execution, settings.php is read and CACHED by the opcode cache. After entering the database information, a new settings.php is written, but as the cache entry hasn't expired its TTL, the next page request still works as if the old settings.php without db info was present. This causes the "Database configuration" page to display again.

There are several kind of opcode cache solution, so, need to test for it. Some popular opcode case solution Zend OPcache(php 5.5), Apc(php 5.4), Xcache, Wincache, eaccelerator.
The opcode cache solutions available for both php 5.4 and php 5.5, but perhaps the APC is only available for 5.4.

Proposed resolution

Need to implement opcode cache clearing for all popular opcode cache solution. Create a custom function which checkes, that the specific extension available and if it is enabled clear the opcode cache.
Call this custom function in drupal_rewrite_settings() function.

Sample *temporary* workarounds for settings.php for some extensions

  • Zend OPcache
    fill in sample code
  • apc
    fill in sample code
  • wincache
    fill in sample code
  • XCache
    fill in sample code
  • eaccelerator
    fill in sample code

Remaining tasks

Process comments from #136 to get the D7 backport ready

User interface changes

No

API changes

No

How can reproduce the issue:

  • need to enable one opcode cache solution, see the list above.
  • restart you webserver
  • you can check it easily in phpinfo
  • start to install a D8
  • set your database connetion on install page
  • if the opcode cache is working properly, you need to get back the empty db form

If somebody, can reproduce the issue please, post your configuration to the comment.

CommentFileSizeAuthor
#146 779482-clear-opcache.patch2.23 KBquicksketch
#142 drupal-779482-142-opcode-cc-install-D7.patch2.51 KBmikeytown2
#129 779482-clear-opcode-cache-129.patch1.78 KBBoobaa
#121 interdiff.txt789 bytessun
#121 opcache.121.patch2.19 KBsun
#113 interdiff.txt1.6 KBsun
#113 opcache.113.patch2.52 KBsun
#111 interdiff.txt4.76 KBsun
#111 opcache.110.patch2.5 KBsun
#93 interdiff-779482-90-94.txt2.42 KBsegi
#93 drupal-779482-94-opcode-cache-D8.patch2.65 KBsegi
#90 drupal-779482-90-opcode-cache-D8.patch2.46 KBSweetchuck
#87 drupal-779482-87-opcode-cache-D8.patch2.44 KBsegi
#83 drupal-779482-84-opcode-cache-D8.patch2.44 KBmikeytown2
#81 drupal-779482-82-opcode-cache.patch3.38 KBmikeytown2
#80 drupal-779482-80-opcode-cache.patch3.38 KBmikeytown2
#78 drupal-779482-78-opcode-cache.patch3.37 KBmikeytown2
#70 779482.70.patch3.16 KBalexpott
#70 68-70-interdiff.txt1.15 KBalexpott
#68 drupal-779482-68-opcode-cache.patch3.02 KBmikeytown2
#64 drupal-779482-64-opcode-cache.patch3.12 KBmikeytown2
#62 drupal-779482-62-opcode-cache.patch3.08 KBmikeytown2
#60 779482-opcode-cache-60.patch2.99 KBCameron Tod
#58 779482-opcode-cache-58.patch14.41 KBCameron Tod
#55 779482-opcode-cache-55.patch2.65 KBCameron Tod
#48 flush-settings-opcode-cache.patch2.64 KBjbrown
#45 drupal-779482-44-clear-opcode-on-write.patch1.26 KBmikeytown2
#40 drupal-779482-39-clear-opcode-on-write.patch775 bytesmikeytown2
drupal-779482-39-clear-opcode-on-write.patch775 bytesmikeytown2
#38 drupal-779482-38-clear-opcode-on-write.patch636 bytesmikeytown2
#38 drupal-779482-38-clear-opcode-on-write.patch636 bytesmikeytown2
#11 779482-11-install-opcode.diff2.49 KBdalin
#9 779482-9-install-opcode.diff2.43 KBdalin
#5 779482-5-install-opcode.diff2.27 KBdalin
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

donraman’s picture

The new release of WINCACHE 1.1 Beta2 should address this problem as we have got file change notification now. Details at http://blogs.iis.net/donraman/archive/2010/04/29/file-change-notificatio....

Thanks,
Don.

Heine’s picture

@donramen, thanks a lot for the update!

I'll keep the issue open for now, so we can decide what to do about APC's apc.stat 0 and when file change notification if off (eg on a UNC share).

Heine’s picture

For the record: with the wincache 1.1 beta 2 enabled, I was indeed able to succesfully install Drupal 7.

Heine’s picture

Title: Installation failure when wincache opcode cache is enabled » Installation failure when opcode cache is enabled

This has been confirmed by jbergstroem in #686596: Child before parent class definition order breaks some opcode cachers as an issue with APC where stat off.

dalin’s picture

Status: Active » Needs review
Issue tags: +String freeze
FileSize
2.27 KB

Here's a patch that tests for error-causing settings for APC and Zend Optimizer+. I suppose we'll also need to test for problems in WinCache and eAccelerator, but I have no experience with those products.

Also how do we test if files are on a remote file system (meaning that the opcode cache will not receive notification of file changes regardless of settings)?

webchick’s picture

Hm. I don't really like the idea of us putting in product-specific checks like this into D7. It's a recipe for having to expand it later to cover additional products and having to expand it when new versions of existing products come out, etc.

There's no more general way to check for this condition by chance, is there?

dalin’s picture

I guess another way to do it would be to write a "hello world" file to the filesystem, and immediately execute it. If it prints "hello world" then we know that any opcode cache is setup correctly.

amirtaiar’s picture

I have got this error:

PDOException: SQLSTATE[42S02]: Base table or view not found: 1146 Table 'tikkecoi_tikke.registry' doesn't exist: SELECT filename FROM {registry} WHERE name = :name AND type = :type; Array ( [:name] => DeleteQuery_mysql [:type] => class ) in _registry_check_code() (line 2720 of /home/tikkecoi/public_html/includes/bootstrap.inc).

So I got a guess with a help this is the issue for me. So I have tried to enter -
install.php?q=install.php&profile=standard&locale=en&op=start&id=1

But nothing. Is this the issue for me? Can anybody help me.

dalin’s picture

In this patch we write a 'Hello World' file, execute it, modify it, re-execute it, and see if the output changed when we modified the file.

@amirtaiar I could be wrong, but that error sounds completely unrelated.

Status: Needs review » Needs work

The last submitted patch, 779482-9-install-opcode.diff, failed testing.

dalin’s picture

Status: Needs work » Needs review
FileSize
2.49 KB

Dang url() is sketchy during install.

Status: Needs review » Needs work

The last submitted patch, 779482-11-install-opcode.diff, failed testing.

dalin’s picture

A Friday afternoon haiku:

Glorious Testbot
It helps us code with ease
But when it's borked

Submitting for re-testing.

dalin’s picture

Status: Needs work » Needs review

#11: 779482-11-install-opcode.diff queued for re-testing.

webchick’s picture

LOL.

Dalin's great haiku
Made my otherwise bad
Night into awesome

chx’s picture

Status: Needs review » Needs work

No way! I thought we learned our lesson by now that drupal_http_requesting from the server to the server is not working.

dalin’s picture

@chx I know that it sometimes fails. Which is why the patch checks response codes and only issues a requirement error if the first request is successful, but the second one fails.

Do you think that testing vendor-specific ini strings is preferable?

dalin’s picture

I also imagine that this test might be useful for the update manager.

puregin’s picture

This seems to be a similar situation, not related to settings.php being re-written, since this is for an upgrade, not a clean install. In other words, I doubt that this would be caught by the patch in #11.

[Mon Jan 17 13:49:47 2011] [apc-error] Cannot redeclare class queryconditioninterface in /var/www/drupal-core/drupal-7.0/includes/database/database.inc on line 1648.

Version info:

PHP Version 5.3.2
Ubuntu 10.04
APC 3.1.3p1
Apache 2.2.14

This problem seems to be caused by APC's handling of require_once. I'd set up APC to optimize this handling, and it seems that something in either the new Drupal code, or the updated APC behaviour, is incompatible with the optimization, because the problem cropped up again with the newly installed Drupal 7 after I re-enabled APC.

For now, it appears that the fix is to turn off the include_once optimization: set apc.include_once_override = 0

Is it possible to advise people to temporarily disable opcode caches and accelerators while doing installs/upgrades?

alanpeart’s picture

Took me a long time to discover why I couldn't install Drupal 7. Finally found this thread, turned off APC on my VPS, and the install worked fine. However from reading this thread, a lot of which goes over my head, I'm not 100% clear on the best course of action for me is for the future. Is this regarded as a bug that will be fixed in a future D7 release? If not, do I need to have opcode caching permanently disabled or only during Drupal 7 install/upgrades?

dalin’s picture

@alanpeart the problem should only happen during installation (and maybe updates, though I don't understand why that one could happen). Any large Drupal site will be running an opcode cache.

It could be fixed in a future D7 release, but there needs to be more consensus on possible fixes.

Anonymous’s picture

Just to make things weirder, (and no idea if this information helps)

The problem did not happen for me on installation, but after a combination of dist-upgrading a debian lenny machine to debian squeeze (note: jump to php5.3) and installing php-apc as well. No modification to my existing Drupal 7.2 sites occurred, but the problem immediately began on those sites after that, and puregin's fix solved it for me.

Edit: see also http://pecl.php.net/bugs/bug.php?id=15356

jghyde’s picture

I ran into this problem on Drupal 7 and apc.module too. I set my apc.ini to read like this, and my Drupal 7 fresh install appears to be working.

If anyone has any ideas to better do this, please chime in:

root@campfawcett:/etc/php5/conf.d# cat apc.ini 
extension=apc.so
apc.shm_size=256
apc.include_once_override = 0
apc.enable_cli = 1

Thanks!

Joe
http://www.hydeinteractive.com

uberEllis’s picture

*subscribe*

Narek’s picture

#23 solve the issue, thanks a lot Joe

Perignon’s picture

Just confirmed that setting apc.stat=0 causes Drupal not to finish installing and creates the dreaded "31 tables" then quits. Setting the apc.stat=1 cures the problem.

sridharpandu’s picture

After installing over 100 drupals I had this problem. I had apc and memcached. Never thought that could cause an issue. #26 solved my problem.

A good place to start is to read this thread https://drupal.org/node/332730

Garrett Albright’s picture

It looks like this problem may become more prevalent as PHP 5.5 adoption increases and more folks try its built-in OPcache opcode cache…

Perignon’s picture

Good to know. I just started experimenting with 5.5 of my desktop.

ryoken’s picture

After spending hours on end pulling my hair out trying to install Drupal 7 (with all the usual symptoms of white screen of death, getting stuck on database configuration page, "Call to undefined function field_attach_load()" errors, etc.), I eventually ended up disabling PHP 5.5's built-in Zend OPcache as a last resort... and voilà! The installation magically completed :)

Then I discovered this page. Oh the agony!

eloya’s picture

I have had the same problems for hours on Mamp Mac. I finally found this thread. But, stupid question probably, how do you disable the opcache?
Thanks

eloya’s picture

Never mind. I changed the cache, but the problem is still not solved. I fill in everything and then get stuck at
http://localhost:8888/drupal/install.php?profile=standard&locale=en

Garrett Albright’s picture

eloya, the simplest solution is probably to just go into the MAMP application's preferences and tell it to use a version of PHP other than 5.5.x (preferably 5.4.x, but 5.3.x should work just fine with Drupal 7 as well, if I recall correctly).

mikeytown2’s picture

Is there an equivalent to apc_clear_cache('opcode') and/or apc_delete_file() in php 5.5? Run the equivalent function before including the settings.php file.

Edit: looks like opcache_reset() and/or opcache_invalidate() might do it.

mikeytown2’s picture

Issue summary: View changes

Added info on wincache because the title change lost context.

eloya’s picture

Garrett,
Thanks. Tried that but did not work. I'll dig deeper into it. I have a online D7 test site, but still.

dagomar’s picture

I'm also experiencing this. Note that my installation always works though, I'm doing this:

1. Fill in DB credentials first time
2. Fill in DB credentials second time
3. On blank page that follows, click address bar and press enter. It then continues.

@ryoken, how did you disable the built in OPCode?

[EDIT]
Found it, in my setup (ubuntu 13:10), I found:
/etc/php5/apache2/conf.d/05-opcache.ini

Inside that I did this:

; configuration for php ZendOpcache module
; priority=05
; zend_extension=opcache.so

I hope that doesn't destroy anything, but for my setup it seems to work fine.

ryoken’s picture

@ryoken, how did you disable the built in OPCode?

You beat me to it dagomar! :) I commented out "zend_extension" as per details here: http://www.php.net/manual/en/opcache.installation.php.

mikeytown2’s picture

Issue summary: View changes
Status: Needs work » Needs review
FileSize
636 bytes
636 bytes

Following patch will clear the opcode cache for the settings.php that was just written.

mikeytown2’s picture

mikeytown2’s picture

mikeytown2’s picture

Looking for documentation on xcache_clear_cache(XC_TYPE_PHP) and eaccelerator_clear(). Haven't found any so please post here if you do.

Heine’s picture

Deleting a specific file from cache is a feature request for xcache, see http://xcache.lighttpd.net/ticket/226

Clearing the entire cache seems to need the user to configure and enter a password:
http://www.turnkeylinux.org/forum/support/20110913/configuring-xcache

I image it's easier to ask them to disable the cache during install.

mikeytown2’s picture

I think eaccelerator & xcache will clear the cache only if the php file is configured as an "admin file", so both of these wouldn't be a simple drop in fix like it is for the other 3.

Sounds like the best option is to combine #11 and #40. Clear the cache for the file automatically if possible, if not ask the admin to disable the opcode cache during install.

dalin’s picture

For xCache, Perhaps we just need a try/catch around it. It seems probable that xcache.admin.user and xcache.admin.pass would already be set.

Or better yet, perhaps we need a try/catch around the entire patch so that we catch any problems with any opcode cache.

mikeytown2’s picture

#40 wrapped in a try/catch block and added in xcache_clear_cache() & eaccelerator_clear(). Should be noted that this will require manual testing for each cache backend.

mikeytown2’s picture

Status: Needs review » Reviewed & tested by the community

This seems to fix the issue. RTBC as it's been sitting like this for about a month.

jbrown’s picture

Version: 7.x-dev » 8.x-dev
Assigned: Unassigned » jbrown
Status: Reviewed & tested by the community » Needs work

The opcache in php 5.5 is also causing problems with installation of Drupal 8. With it enabled the database form needs to be submitted twice.

The patch in #45 applies with fuzz to Drupal 8 but does not fix the problem. I will upload an updated Drupal 8 patch shortly.

jbrown’s picture

Assigned: jbrown » Unassigned
Issue summary: View changes
Status: Needs work » Needs review
FileSize
2.64 KB

This patch fixes it for me on Drupal 8.

Status: Needs review » Needs work

The last submitted patch, 48: flush-settings-opcode-cache.patch, failed testing.

mikeytown2’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 48: flush-settings-opcode-cache.patch, failed testing.

rjb’s picture

I'm using Drupal 7 on PHP 5.5, Kubuntu 13.10. The patch in #45 works for me. Thanks mikeytown2!

Cameron Tod’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 48: flush-settings-opcode-cache.patch, failed testing.

Cameron Tod’s picture

Status: Needs work » Needs review
FileSize
2.65 KB

Rerolled.

Status: Needs review » Needs work

The last submitted patch, 55: 779482-opcode-cache-55.patch, failed testing.

The last submitted patch, 55: 779482-opcode-cache-55.patch, failed testing.

Cameron Tod’s picture

Status: Needs work » Needs review
FileSize
14.41 KB

Re-added the try/catch block.

Note that this patch causes issues with xcache on my local MAMP/Mavericks setup. I'm not using xcache, but it is enabled by default in MAMP.

This is the output when running run-tests.sh:



XCache Authentication Failed

XCache Authentication Failed

You're not authorized to access this page due to wrong username and/or password you typed.
The following check points is suggested:

  • Be aware that `Username' and `Password' is case sense. Check capslock status led on your keyboard, and punch left/right Shift keys once for each
  • Make sure the md5 password is generated correctly. You may use mkpassword.php
  • Reload browser cache by pressing F5 and/or Ctrl+F5, or simply clear browser cache after you've updated username/password in php ini.

Check XCache wiki page for more information.


The xcache documentation is terrible. If we want to go ahead with this patch, I'll work out someway to detect the above situation before clearing xcache.

Status: Needs review » Needs work

The last submitted patch, 58: 779482-opcode-cache-58.patch, failed testing.

Cameron Tod’s picture

Status: Needs work » Needs review
FileSize
2.99 KB

I don't know how all that extra junk got into the patch before vOv

Status: Needs review » Needs work

The last submitted patch, 60: 779482-opcode-cache-60.patch, failed testing.

mikeytown2’s picture

Status: Needs work » Needs review
FileSize
3.08 KB

I agree that we need to detect if xcache_clear_cache() will work before calling it. Does this give the correct info? I don't have xcache so I can't test this. http://php.net/ini_get

echo ini_get("xcache.admin.pass");
echo ini_get("xcache.admin.user");

If we do get a value back for xcache.admin.pass we could make sure it is a MD5 value before trying to call the function.

I've attached a patch that will make sure the file exists before trying to clear it from the opcode cache.

Status: Needs review » Needs work

The last submitted patch, 62: drupal-779482-62-opcode-cache.patch, failed testing.

mikeytown2’s picture

Status: Needs work » Needs review
FileSize
3.12 KB

Add in a check to make sure calls to stat work before trying to clear the op code cache.

Status: Needs review » Needs work

The last submitted patch, 64: drupal-779482-64-opcode-cache.patch, failed testing.

Berdir’s picture

This is now also a problem with the new test runner in 8.x.

#2194071: WebTestBase::setUp() fails with PHP 5.5 and enabled opcache is a possible duplicate.

The other issue is currently critical, this should at least be major, not sure why nobody ever tried to increase the priority :)

mikeytown2’s picture

Priority: Normal » Critical
Issue tags: +PHP 5.5
mikeytown2’s picture

Status: Needs work » Needs review
FileSize
3.02 KB

Using @ error suppression as checking if the file exists seems to do nothing.

mikeytown2’s picture

So error suppression makes this pass the testbot. Open to ideas on how to make this work without using error suppression.

alexpott’s picture

FileSize
1.15 KB
3.16 KB

How about this? Confirmed that this patch fixes interactive installer with PHP5.5 + opcache

Status: Needs review » Needs work

The last submitted patch, 70: 779482.70.patch, failed testing.

catch’s picture

Same question from the other issue:

would you mind checking the value of opcache.validate_timestamps?

Berdir’s picture

opcache.blacklist_filename => no value => no value
opcache.consistency_checks => 0 => 0
opcache.dups_fix => Off => Off
opcache.enable => On => On
opcache.enable_cli => Off => Off
opcache.enable_file_override => Off => Off
opcache.error_log => no value => no value
opcache.fast_shutdown => 0 => 0
opcache.force_restart_timeout => 180 => 180
opcache.inherited_hack => On => On
opcache.interned_strings_buffer => 4 => 4
opcache.load_comments => 1 => 1
opcache.log_verbosity_level => 1 => 1
opcache.max_accelerated_files => 2000 => 2000
opcache.max_file_size => 0 => 0
opcache.max_wasted_percentage => 5 => 5
opcache.memory_consumption => 64 => 64
opcache.optimization_level => 0xFFFFFFFF => 0xFFFFFFFF
opcache.preferred_memory_model => no value => no value
opcache.protect_memory => 0 => 0
opcache.restrict_api => no value => no value
opcache.revalidate_freq => 2 => 2
opcache.revalidate_path => Off => Off
opcache.save_comments => 1 => 1
opcache.use_cwd => On => On
opcache.validate_timestamps => On => On

All settings are default, at least there's nothing in opcache.ini other than enabling the extension.

The test problem for me is also not consistent, I usally can run a few tests and suddenly it stops working, without changing anything. I have no idea why.

mikeytown2’s picture

http://php.net/manual/en/opcache.configuration.php#ini.opcache.validate-...

I'll incorporate this into the next patch I roll.

if (function_exists('opcache_invalidate')) {
  $timestamps = ini_get('opcache.validate_timestamps');
  if (!$timestamps || ($timestamps && ini_get('opcache.revalidate_freq') != 0)) {
    opcache_invalidate($filepath);
  }
}
catch’s picture

If it's that setting that's causing the problem for people, then we likely need to step back a bit. settings.php isn't the only PHP file we write, there's also compiled Twig templates, the container etc. and it's likely to affect those too.

This looks like a replacement for apc.stat although no doubt the implementation is different. We've never tried to support apc.stat in any way, just left people to their own devices if they enable it in production.

This is a bit different given it's apparently enabled by default, but I'm wary about introducing an installer-specific workaround for this setting. Feels like we either need to try to fix it everywhere (i.e. on PhpStorage writes too), or just warn in hook_requirements() somewhere and leave it at that.

Berdir’s picture

Priority: Critical » Normal

In my case with tests, I had to force the invalidation, otherwise it didn't help.

I think settings.php is a bit different than PhpStorage, because especially in tests (so far I did not have troubles with the installer), because we're writing and reading it multiple times within a very short timeframe. Cached twig files aren't something that is changing as long as the source isn't changing, and the container is usually not immediately read after being written out, at least not by the same process. AFAIK?

I will try to see what happens when I set revalidate_freq to 0. Would be unfortunate to not support a non-zero value at all IMHO.

mikeytown2’s picture

If I wanted to fix this everywhere, how spread out is the code that writes php files?

mikeytown2’s picture

Status: Needs work » Needs review
FileSize
3.37 KB

Leaving error suppression on for xcache and eaccelerator.

Status: Needs review » Needs work

The last submitted patch, 78: drupal-779482-78-opcode-cache.patch, failed testing.

mikeytown2’s picture

Status: Needs work » Needs review
FileSize
3.38 KB

forgot a )

mikeytown2’s picture

#80 is going to fail. New one

sun’s picture

I did not really agree with the other issue being marked as duplicate of this one, but stayed silent.

If we are sure that the problem primarily affects tests only and we can't reliably reproduce the problem in the regular installer, then it is possible that my patch in #1757536: Move settings.php to /settings directory, fold sites.php into settings.php resolves the problem (again, for tests only). — It removes some unnecessary manual re-inclusions of settings.php by fixing the root cause in the installer instead. That said, it is possible that those changes are only possible with the larger/overall changes performed by that patch.

mikeytown2’s picture

@sun
Help me understand why these are 2 different issues and/or why they can not be merged together :)

Using the coding style from #2194071: WebTestBase::setUp() fails with PHP 5.5 and enabled opcache and merging in the inline comments.

alexpott’s picture

we can't reliably reproduce the problem in the regular installe

Having to submit the database form twice with opcode cache enabled can be reliably be reproduced on my computer

segi’s picture

Assigned: Unassigned » segi
Issue tags: +Needs reroll
segi’s picture

Sorry, I forgot to comment it, I do the reroll.

segi’s picture

Assigned: segi » Unassigned
Issue tags: -Needs reroll
FileSize
2.44 KB

I rerolled and tested the previous patch. The patch solved the issue to me.
I tested on MAMP 2.2, php 5.5.3 and of course with enabled opcache.

effulgentsia’s picture

Priority: Normal » Critical

#76 doesn't explain why this was demoted from critical, so I'm guessing that was an unintentional change.

Berdir’s picture

+++ b/core/includes/install.inc
@@ -165,6 +165,47 @@ function drupal_get_database_types() {
+    throw new Exception(t('Failed to clear the opcode cache for %settings. For the duration of the installation it is recommended to either disable the opcode cache, or reconfigure it to allow code changes to be seen immediately.', array('%settings' => $settings_file)));

This talks about the settings file, but the function otherwise does not. Maybe this try/catch should be in drupal_rewrite_settings() instead?

Sweetchuck’s picture

The patch #87 solve the problem.

PHP 5.5.8
opcache.enable = On
opcache.enable_cli = On
opcache.revalidate_freq = 0
opcache.validate_timestamps = On

In this patch I just fix some coding standards error.

The question of try/catch from Berdir is still exists.

Sweetchuck’s picture

I think the "Clear all caches" button on the "admin/config/development/performance" page should clear the OpCache.

dalin’s picture

Status: Needs review » Needs work

#91, one problem per Drupal issue please. You can open another one for that.

Still needs to deal with Berdir's suggestion in #89

// Use @ error suppression.
@xcache_clear_cache(XC_TYPE_PHP);

This is an unhelpful code comment. It's obvious that the @ is suppressing an error. Code comments should focus on the 'why', not the 'what'.

We need to ensure that exceptions are thrown consistently for failures with any of the different opcode systems. For example apc_delete_file() and wincache_refresh_if_changed() both return FALSE on error. But unfortunately for the others it gets more complicated. opcache_invalidate() returns TRUE on success or if there was nothing to invalidate, FALSE if the opcode cache is not enabled. I can find zero documentation for xcache_clear_cache() or eaccelerator_clear().

segi’s picture

I fixed some problem above requests, alexpott sad that, we should use extension_loaded instead of function_exist to investigate if the module is loaded.
I will update the issue summary, that you can test it.

segi’s picture

Status: Needs work » Needs review
Boobaa’s picture

Using php-5.5.3 with zend_extension=opcache.so loaded I was constantly running into this when running tests, but can't remember running into this during regular installs. Applying the patch in #93 and/or commenting ;zend_extension=opcache.so (so it isn't loaded at all) solves the issue for me.

Regarding berdir's question in #89: drupal_clear_opcode_cache($filepath) is to clear the opcode cache for any given file, not only settings. If you were referring to %settings in the exception message, then the answer is was a bug in (up to) #90 and it's been replaced by %file in #93 (without the bug).

However, if I understand everything correctly, then this should be manually tested with and without 1. PHP-5.5's Zend OPcache, 2. apc, 3. wincache, 4. XCache and 5. eaccelerator. The patch in #93 solves this issue with PHP-5.5's Zend OPcache for me; somebody should test this for each of the others.

iszabi’s picture

Status: Needs review » Reviewed & tested by the community

I've Reviewed the drupal-779482-94-opcode-cache-D8.patch and it worked perfect on PHP 5.4.25 turned on Zend OPcache.
After I given configuration details installer went forward successfully.

dalin’s picture

Status: Reviewed & tested by the community » Needs work

Latest patch has some issues:

- error suppression was removed. Though reading back the comments, I can't tell why it was added to begin with. @mikeytown2 you added it, any thoughts?

- Still need to deal with Berdir's suggestion in #89: Since drupal_clear_opcode_cache() is not specific to settings.php, or to the installation, it can't give an error message that talks about those things. Better to throw an exception with a generic message and let the calling function catch that and give a more specific message to the user.

- Still need to deal with suggestion from #92: We need to ensure that exceptions are thrown consistently for failures with any of the different opcode systems.

- I don't understand the need for $empty_values. I think this should suffice if (!ini_get('foo')) { . There should also be some code comments explaining why we care about those ini values.

Berdir’s picture

The exception part: Do we actually know that those functions throw exceptions? Maybe that code is useless anyway?

I just looked through the documentation of all those functions and none of them are documented to throw an exception. (exception_clear() was hard to find..).

I would suggest to simply remove the exception handling completely. Some are documented to return false, so if anything, we'd need to check that, but I'm not sure if it's worth it?

catch’s picture

Also let's move this to bootstrap.inc, it's got potential use-cases outside the installer (i.e. when we do container rebuilds and similar).

sun’s picture

Speaking of moving a procedural function between legacy include filenames... ;)

Drupal\Component\Utility\OpCacheHelper::invalidate($pathname)

EDIT: Note: The proper PHP5 terminology is $pathname (as opposed to $filepath) → SplFileInfo::getPathname()

segi’s picture

Issue summary: View changes
segi’s picture

@dalin
I talked about error suppression with Jeremy Thorson, because there was a comment above, about that the error suppression was added because the functions cause falling to test bots. He said maybe in the past was some problems with bots and this was necessary, but he think the problem is solved, so it is not necessary furthermore.
I agree with you the $empty_values variable not necessary, I investigate the if(!""){} it it gives back TRUE.

I agree with @Berdir that the try/catch block in useless, because I investigate those functions and I didn't find function which throw exception, it only return with FALSE if something not working.

The idea that, we move the custom function to bootstrap.inc is good because we want to implement a general function.

mikeytown2’s picture

The try catch block is mainly for xcache_clear_cache() & eaccelerator_clear() as the documentation is lacking; see #45.

alexpott’s picture

+++ b/core/includes/install.inc
@@ -165,6 +165,52 @@ function drupal_get_database_types() {
+  catch (Exception $e) {
+    throw new Exception(t('Failed to clear the opcode cache for %file. For the duration of the installation it is recommended to either disable the opcode cache, or reconfigure it to allow code changes to be seen immediately.', array('%file' => $filepath)));
+  }

Doing this was will lose the original exception. If we are just going to rethrow an exception then lets just remove the try catch as the original exception will be more helpful.

YesCT’s picture

Issue summary: View changes

It would be very helpful for people to edit the issue summary and list sample work-arounds for what exactly site builders would set or change in the setting.php to temporarily disable opcode caching (for the various possible cache ).

---

Work in progress:

in patch:
Failed to clear the opcode cache for %file. For the duration of the installation it is recommended to either disable the opcode cache, or reconfigure it to allow code changes to be seen immediately.

concerns:
1. "reconfigure it"
what the heck is "it"? reconfigure the settings.php file? reconfigure the opcode cache? ....?

2. need to make it more generic since this could in the future be used for another file that is not settings.php

--

suggestion:
'Failed to clear the opcode cache for %file. Temporarily disable the opcode cache to allow changes in the file to be seen immediately. See the settings.php file.

--

concerns (cont.)
3. is there a section already in the settings.php file that talks about opcode cache at all? Maybe we should check if there is a doc page on drupal.org we can refer to. (I'm thinking of the similar pattern we have/had for clean url stuff)

segi’s picture

@alexpott I think this message is more detailed as original, but I don't know it.

@YesCT It is a good idea. We can take an uncomment row perhaps: ini_set('apc.enabled', 0) for all kind of opcode solution in the settings.php

Berdir’s picture

There is no exception.

I checked the source code for both the xcache and the eaccelerator function and there's no exception to be seen. Again, if you actually wanted to check something, then you would have to check the return value for those that do something. But at least for eaccelerator(), it seems to always return NULL anyway (https://github.com/eaccelerator/eaccelerator/blob/master/ea_info.c#L270) and for the others that don't have a password I don't see what could go wrong that someone could do anything about?

So again, remove that stuff :)

alexpott’s picture

Yep I'm saying do not catch and rethrow an exception unless you are going to include all the info from the original exception similar to what _drupal_exception_handler() does. But I think this is unnecessary obfuscation. If any of the functions do through an exception then this is going to be more helpful to user than what we replace it with.

Or what @Berdir says :)

sun’s picture

Assigned: Unassigned » sun
Issue tags: -String freeze
hgurol’s picture

Issue tags: +Needs backport to 7.x

I had to disable the opcode cache in php.ini, so I could install D7 on Ubuntu 14.04 (apache 2.4 & php 5.5).

sun’s picture

Status: Needs work » Needs review
Issue tags: -Needs backport to 7.x +Needs backport to D7
FileSize
2.5 KB
4.76 KB

Proper OpCache invalidation utility helper class.

Removed all ini_get()s + conditions + whatnot. The job is to invalidate a given file, just do it.

Also double-checked the functions once more. Only apc_delete_file() is documented to throw a PHP warning in case the specified file has not been compiled yet. Added a corresponding code comment.

catch’s picture

Status: Needs review » Needs work

The utility should be OpcodeCache, OpCache is just the name of the one PHP 5.5 ships with.

sun’s picture

Status: Needs work » Needs review
FileSize
2.52 KB
1.6 KB

Renamed OpCache class to OpCodeCache.

dalin’s picture

Issue summary: View changes
Status: Needs review » Needs work

We're getting close. I just re-read through all the comments and I think by removing the try/catch block we may have removed the fix for comment #58.

Now we just need to test that they all work as advertised:
- Test in PHP 5.5 built-in opcode cache
- Zend OPcache
- APC
- eAccelerator
- WinCache
- xCache

dalin’s picture

Hmmm, apparently I can't use strike tags. The first two are done. The remaining ones to test are:

- APC
- eAccelerator
- WinCache
- xCache

dalin’s picture

Issue summary: View changes
dalin’s picture

Issue summary: View changes

Tested working with APC and apc.stat=0

That leaves
- eAccelerator
- WinCache
- xCache

dalin’s picture

Issue summary: View changes
sun’s picture

Status: Needs work » Needs review

@dalin: Sorry, but I'm not going to add back any further error suppression or exception handling without proof.

None of the comments in this issue that added try/catch and error suppression ever included proof in the form of an PHP fatal error, error, or warning message.

Since most of these functions are poorly documented (or not documented at all), we should stick to the actual + available documentation. The rest can be figured out if people may run into actual errors (but it seems that your tests were all positive).

If this approach does not fly, then the only workable alternative I can see is to remove support for eAccelerator + WinCache + XCache from this patch, only keeping Opcache + APC.

stpaultim’s picture

Not sure if this is helpful, but I was experiencing this problem (having to reload the Database Configuration Page) running MAMP on OSX 10.8.5 with PHP 5.5.3. I experienced the problem using both Chrome (Version 33.0.1750.152) and Firefox (27.0.1).

I applied the patch and it fixed the problem. If you have any more specific questions about my config look me up on IRC.

sun’s picture

FileSize
2.19 KB
789 bytes

Just in case it helps us to move forward, here's a patch that implements #119:

Removed support for XCache, eAccelerator, and WinCache.

If we go with this approach, this does not imply that support could not be added later on (in dedicated issues for each).

(Duly noting, the longer it takes, the more is PHP 5.5's built-in OpCache the standard, and thus the more all of those alternatives become completely obsolete.)

Berdir’s picture

Status: Needs review » Reviewed & tested by the community

Agreed, let's do this. We probably had a dozen people that are now on 5.5 that were affected by this in Szeged, and I've never seen this before we switched the test runner to re-load settings.php, so it seems to be a by far larger problem with PHP 5.5's opcache than ever before.

Let's get this in and then we can think about the other more and more irrelevant opcode cache implementations in separate issues.

  • Commit 2c5978c on 8.x by catch:
    Issue #779482 by mikeytown2, sun, cam8001, dalin, segi, alexpott,...
catch’s picture

Version: 8.x-dev » 7.x-dev
Status: Reviewed & tested by the community » Patch (to be ported)

Completely agreed on just supporting OpCache and APC for now. Committed/pushed to 8.x, thanks!

dalin’s picture

@sun No need to apologize for pragmatism ;-) I'm glad we finally got this in.

So to make this work for D7 I think all we need to do is move drupal_clear_opcode_cache() back to core/includes/install.inc

dalin’s picture

Issue summary: View changes
Berdir’s picture

sun’s picture

Assigned: sun » Unassigned
Boobaa’s picture

Status: Patch (to be ported) » Needs review
Issue tags: -Needs backport to D7
FileSize
1.78 KB

Backported 2c5978c from #121, following advices of @dalin in #125.

ianthomas_uk’s picture

Issue tags: +Needs backport to 7.x

Backport tag should stay until the backport is committed

star-szr’s picture

Issue tags: -Needs backport to 7.x +Needs backport to D7
dalin’s picture

Status: Needs review » Reviewed & tested by the community

Great!

eojthebrave’s picture

Just a note that after #2266859: Move the dependency calculation helper methods from ConfigEntityBase to generic traits was committed older versions of APC (<=3.1.9) cause problems due to an APC bug with aliasing traits. Discovered this trying to install D8 in MAMP2 w. php 5.4.10 which has apc 3.1.9. Upgrading apc to 3.1.11 fixed the problem. I don't think that it's necessarily related to this issue but wanted to point it out. Especially if we're going to do something like only support APC and Opcache we need to document at APC 3.1.11 is necessary.

Possibly related: https://bugs.php.net/bug.php?id=62699

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 129: 779482-clear-opcode-cache-129.patch, failed testing.

Boobaa’s picture

Status: Needs work » Needs review
David_Rothstein’s picture

  1. Note, this is currently broken on 5.5: #2229781: Opcode cache utility breaking cli site install for PHP 5.5

    Doesn't the Drupal 7 patch need to address this too?

  2. +    clearstatcache(TRUE, $pathname);
    

    Will this lead to PHP warnings on 5.2 (where clearstatcache() accepts no function parameters)?

  3. This comment is in #99:

    Also let's move this to bootstrap.inc, it's got potential use-cases outside the installer

    But the patch puts it in includes/install.inc... does it matter?

  4. +    // @see http://php.net/manual/en/function.apc-delete-file.php
    

    (minor, and in Drupal 8 too) I think we try to avoid linking to the English-specific version of PHP manual pages.

Also curious why this is called OpCodeCache::invalidate() in Drupal 8 but drupal_clear_opcode_cache() in Drupal 7... are we trying to introduce even more API changes between the versions wherever we can? :) But it can't be namespaced in Drupal 7 anyway, so whatever, guess drupal_clear_opcode_cache() is fine.

Naturrien’s picture

pilotget’s picture

I can confirm that patch #129 fixed my problem. My environment is as follows:

  • PHP 5.5.9-1ubuntu4.3
  • Ubuntu 14.04
  • nginx/1.4.6

Behaviour prior to installing patch:

  • I would enter database information, and the page would reappear asking for database credentials
  • I would re-enter the credentials and then get a white screen
  • I would hit refresh and re-send the POST data, but it wouldn't make a difference. It was stuck on http://<mysite>/install.php?profile=standard&locale=en&op=start&id=1 with a white screen.
  • I would highlight the address http://<mysite>/install.php?profile=standard&locale=en&op=start&id=1 and hit enter to re-visit the page without re-sending the post data. Now the installation would complete

Behaviour after installing patch:

  • The installation works flawlessly the first time!
rsutaria’s picture

I have a somewhat newbie question, but does patch #129 work for D7?

@pilotget - I'm facing the exact same situation. My environment is:
Drupal 7.30
Ubuntu 14.04
PHP 5.5.9-1ununtu4.3
Apache2

I'm still getting the error after applying patch from comment #129

helmo’s picture

Issue summary: View changes

The patch from #129 still applies and seems OK.

It could still be improved by the comments from #136

Alauddin’s picture

Can confirm patch from #129 works with php 5.5.18, opcache 7.0.4-dev

mikeytown2’s picture

Patch addressing most of the concerns from #136

Fabulous’s picture

Status: Needs review » Patch (to be ported)

Is it possible to include this patch in the core directly ? Because for some people like me who have APC install on the local machine or prod server and do automatic deployment is very painful to think to include it in each drupal release.

andypost’s picture

Status: Patch (to be ported) » Needs review
dstol’s picture

quicksketch’s picture

FileSize
2.23 KB

Here's a cleaned up version building on @mikeytown2's #142:

- Changed $pathname to $filename for consistency with other core files.
- Documentation was overly verbose on backwards compatibility with apcu, trimmed it down.
- Instead of checking extension_loaded(), just use function_exists() in all places.
- Fixed the grammar in the PHP doc.

znerol’s picture

Status: Needs review » Reviewed & tested by the community

So, I've manually reproduced the problem + verified the patch on Debian squeeze (PHP 5.3.3 / apc.stat=off), wheezy (PHP 5.4.36 / apc.stat=off) and jessie (PHP 5.6.4 / OPcache).

Feedback from #136 is addressed, this patch is ready.

znerol’s picture

- Instead of checking extension_loaded(), just use function_exists() in all places.

There is a problem with Debian lenny (PHP 5.2.6, APC 3.0.19, apc.stat=off): apc_delete_file only appeared in 3.1.1. This means that the cache is not cleared on those systems. Given that both, PHP 5.2 as well as Lenny are EOL since a long time, I do not think we need to support that case. Also because this only occurs during installation and only if apc.stat=off is explicitly set (defaults to on).

Berdir’s picture

We just had a very strange bug report against 8.x where someone apparently has the opcache extension enabled but does not have the invalidate function. See #2408533: Installation Fatal error: Call to undefined function opcache_invalidate(). This patch has already been updated to call function_exists() instead, so should be fine, although I still don't understand how that could happen.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 146: 779482-clear-opcache.patch, failed testing.

znerol’s picture

Status: Needs work » Reviewed & tested by the community

Test failure:

Requirements hook failure (HookRequirementsTestCase): exception 'DatabaseSchemaObjectExistsException' with message 'Table variable already exists.'

This looks like a testbot hiccup.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 146: 779482-clear-opcache.patch, failed testing.

Status: Needs work » Needs review
mikeytown2’s picture

Status: Needs review » Reviewed & tested by the community
vbard’s picture

#142 worked for me!
Linux mint 16
php -v
PHP 5.5.3-1ubuntu2.6 (cli) (built: Jul 7 2014 16:54:32)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.5.0, Copyright (c) 1998-2013 Zend Technologies
with Zend OPcache v7.0.3-dev, Copyright (c) 1999-2013, by Zend Technologies

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 146: 779482-clear-opcache.patch, failed testing.

Fabianx’s picture

Status: Needs work » Reviewed & tested by the community

dasrecht’s picture

We just stumbled over this issue too.

PHP 5.5.22 / php-fpm via nginx

how should we continue with this issue?
the patch from #142 worked and i think the patch from #146 can't be rtbc'ed while it's failing at the testbot.

dasrecht’s picture

Small Update :
Patch #146 Passed on Testbot. After a rerun.

MustangGB’s picture

nginx 1.4.6, php-fpm 5.5.9, opcache enabled and D7.34

Went from "Set up database" section reloading, WSOD and "Fatal error: Call to undefined function field_attach_load()" before applying the patch.

To working completely normally after applying the patch.

RTBC

dasrecht’s picture

Thanks MustangGB for the comment.

I tested the Patch #146 on our Servers and I can confirm, it works.

RTBC from my side, since I don't see any further steps to take in this. I also think we should ship this
quite soon since the use PHP 5.4/5.5 starts to ramp up, and depending on the Opcache setup the users would not be able
to install Drupal at all.

Status: Reviewed & tested by the community » Needs work

The last submitted patch, 146: 779482-clear-opcache.patch, failed testing.

Status: Needs work » Needs review
Fabianx’s picture

Status: Needs review » Reviewed & tested by the community
David_Rothstein’s picture

Status: Reviewed & tested by the community » Fixed
Issue tags: +7.36 release notes

Committed to 7.x - thanks!

  • David_Rothstein committed be97f50 on 7.x
    Issue #779482 by mikeytown2, sun, dalin, cam8001, segi, alexpott, Boobaa...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.