Drupal 8 has become a complex, but very powerful system and such over time performance has suffered and memory limits increased from 32 MB to 64 MB.
One answer to the slower performance is caching parts of the system, but this has drawbacks, especially in the installer, where the caches are basically cleared after every module installation.
Another answer is to reduce the complexity of the system by avoiding to load services that are seldom used - even though services might depend on them occasionally.
Last one of the slowest parts in a system is the file system and file_exists() calls can be detrimental to performance.

Last for cold cache performance one needs to distinguish between a pure cold cache performance (which happens the very first time) and re-building performance.

For the first cold cache performance there are two things that can be done:

- Pre-Generate things known not to change in the system (e.g. creation of a static optimized class map, compile parsed YML files to disk and load them as PHP instead, pre-generate a container)
- Avoid loading things that are not necessary.

For the re-building performance, things that are used e.g. during a container rebuild can be cached to persistent memory (e.g. APCu or disk). This includes e.g. YML files and a dynamically generated class map.

Overall I investigated three areas:

- The class loader performance, which without the APC class loader, does make a lot of file_exists() calls.
- The performance when caches are cleared very often (which is similar to when the system is in cold state)
- The installer performance and memory requirements

I have chosen to get things that are already work-in-progress reviewed and moving forward.

Overall I worked on the following issues:

Lazy services:

https://www.drupal.org/node/2407177 (Profile to determine which services should be lazy) == Improved upon the profiling approach, created report with recommendations

Class Loader:

- https://www.drupal.org/node/1818628 (Use Composer's optimized ClassLoader for Core/Component classes) == Created patch, benchmarked
- https://www.drupal.org/node/2296009 (Use APC Classloader by default (when available)) == Created patch, benchmarked (implicitly)
- https://www.drupal.org/node/2253593 (Stop classloader searching filesystem for classes before drupal_classloader() is called) == RTBC, benchmarked

Installer and cold caches (e.g. config entity import):

- https://www.drupal.org/node/2431259 (Optimize FastChainedCacheBackend by introducing heat based shut off) == Created issue and approach
- https://www.drupal.org/node/2395113 (user_modules_installed calls $role->save() and PermissionsHandlerInterface::getPermissions 38x in standard install) == reviewed patch
- https://www.drupal.org/node/2396939 (Make the InstallKernel progressively build the container instead of building, compiling and dumping from scratch on every module install (38x), )
- https://www.drupal.org/node/2427861 (Fast chained backend is incompatible with cli that has apc.enable_cli = 0) == Found bug, opened issue

YAML files caching

- https://www.drupal.org/node/2395143 (YAML parsing is very slow, cache it with APCu in Drupal\Core\Config\FileStorage::read) == Improved upon the patch, benchmarked, added PhpStorage as backend to support PHP 5.4, too

--

I feel this is not yet complete, so I will follow-up on the memory report for the installer.

Thanks,

Fabian

Addendum

# Installer performance and memory requirements analysis

- https://www.drupal.org/node/2294569 -- Profiled a lot of memory usage in installation, found some things to fix, especially Views and ConfigInstaller::loadConfiguration
- https://www.drupal.org/node/2432791 -- Opened issue to investigate skipping config validation during import.

Overall the outcome of this is:

* Container rebuilds are very costly in terms of time and memory needed.
* Router rebuilds are equally costly
* Plugin Discovery is not incremental and a lot of data is cached statically.
* Config installation takes a lot of memory.