Support for Drupal 7 is ending on 5 January 2025—it’s time to migrate to Drupal 10! Learn about the many benefits of Drupal 10 and find migration tools in our resource center.Problem/Motivation
See #2495411-143: Make simpletest fail a test when it detects pages that need more than 64MB and below.
From https://drupal:drupal@mixologic-ci-drupal.redesign.devdrupal.org/opcache... it looks like opcache.max_accelerated_files is set to 2000.
This means a real limit of 3,907 according to the opcache documentation.
The same report says that opcache is 'full' with 3,905 files cached. This is resulting in high memory usage on the bots.
Proposed resolution
Increase the number, probably to 10000 or so to be safe.
Comments
Comment #2
catchComment #3
effulgentsia CreditAttribution: effulgentsia at Acquia commentedIf on https://mixologic-ci-drupal.redesign.devdrupal.org/opcache.html, you click through to the "Scripts" tab, you'll see that a lot of those files are compiled twig files from different test runs (different test ID directories). If the opcache isn't cleared between test runs, then is there any setting of max_accelerated_files that won't fill up? Is there a way to clear the opcache between test runs without restarting the web server?
Comment #4
amateescu CreditAttribution: amateescu for Pfizer, Inc. commentedWe could do a opcache_reset() in tearDown()...
Comment #5
catchI thought PIFR did an apache restart after each full test run.
Isn't this due to concurrency? If you have 8 or 16 tests running at a time, that's 8 or 16 * $templates_in_core plus the container.
Comment #6
Wim LeersThis was deployed accidentally for all newly launched testbots in the last 24 hours and ended up causing a day full of fails for D8: #2552687-16: Test failures in ConfigFormOverrideTest and ContainerRebuildWebTest on newly spun up testbot instances.
Comment #7
cilefen CreditAttribution: cilefen commentedDoesn't increasing max files impact how much RAM you must allocate?
Comment #8
catchThere was plenty of spare RAM in the report above, but if there's a significantly higher number of files that need caching, then that might have got maxed out yes.
Comment #9
MixologicOkay, found a big boo boo with opcode cache settings - the default is to only rescan and recompile a changed php file if it is older than 2 seconds. Which during a test is a lifetime. We ran into a bunch of other issues with the drupalci bots, so we're dropping that setting to 0, which means if you change a php file it should get recompiled.
This is probably what led to all of the breakages last time we set this setting to be higher - because the opcode cache is not an LRU cache (it fills up and just continuously recompiles any further files), we were hitting that 2000 limit and everything else was afterwards was always compiled.
With this set to 0, we should be able to push this back up to 100000 (the max) and not have to worry about the files not working.
Comment #10
basic CreditAttribution: basic at Drupal Association commentedComment #11
MixologicWell... actually ... The 100000 max files got deployed yesterday. Please let us know if things are still broken. They do not appear to be from what we can see.
Comment #12
isntall CreditAttribution: isntall at Drupal Association commentedComment #13
Wim LeersHurray!
Comment #14
cilefen CreditAttribution: cilefen commentedI thought this was a Drupal core issue when I wrote that. Carry on...