[meta] Speed up web tests

Tagging.

Attaching the xhprof file that I was looking at.

Note when you try to repeat this, remove the cpu flag from xhprof_enable(), that increases the overhead a lot, especially of the yaml parsing, as there are *a lot* of method calls involved.

Actually, it's not. It's drupal_static()'ally cached, which we clear 3 times in a test. Seems like we could easily make that a static(), no reason to reset it?

And the function does more than, it also checks version and stuff.

So I prefer to keep them separate static caches.

The attached patch does that, seeing a considerable performance improvement for even a test with a single test method. Should even be bigger for multiple test methods as we save even more calls.

Let's see what the testbot thinks about this.

Maybe, but I'm not happy about the testbot results (not really seeing any improvements there), want to investigate some more and also do another profile run now that this is out of the way.

Maybe keep this is a meta task and create separate issues for specific patches?

Opened #2026367: Use static caches for parsed .services.yml and .info.yml files for the patch, to keep this open as a general profiling issue. Please RTBC the patch there :)

Some new notes, when looking at an xhprof report for running views_ui\DisplayTest, which contains an impressive amount of 12 test methods. Runs 4min without xhprof, 5 m with.

* Parsing config schema still uses 20% of the time, we call FileStorage::readMultiple() 200x times from TypedConfigManager
=> I have no idea what to do about it :( Note that the overhead here is probably quite large as symfony yaml parsing involves a trillion method calls (some methods are called 4M times...)

* 17% of the time is deleting cache tags (6600 calls for 12 test methods)
** 9% of the whole time is entity render cache tags, 98% of those are menu_link'.
** 4.4% is menu_cache_clear(), which is also called for every saved menu link (for the menu)
=> menu_links currently define a view builder and don't use it, we can remove it and then that should save 10% of the time there. @amateescu is already working on an patch, see #2100467: Menu links are not ready to use a view builder so remove it from the annotation for now
=> We can look into doing something similar as we do with menu router rebuild and collect cache tags and write them at the end of the request. The most obvious advantage would be that we only need to update each tag once per request, for example for menu's, for example, on my pretty standard installation, I have 50 links in the admin menu, that means 50 Cache::deleteTags() calls for the same tag when rebuilding the menu links and real sites with have *a lot* more than that.

* Those 12 test methods result in 48 calls to drupal_flush_all_caches(), which takes up ~40% of the whole execution time for me. Those calls are coming from install_finished(), _install_profile_modules_finished() and 2x resetAll(), once in setUp() and once in enableViewsTestModule().
** 80% of that is RouteBuilder::rebuild(),
** 92% of that is spent in the event dispatcher rabbithole,
*** 70% of that is RouterRebuildSubscriber::onRouterRebuild(), this is what triggers menu_router_rebuild() and that menu_link_rebuild_defaults().
**** What's crazy is that we right now spend *83%* of the time it takes to save menu link entities in postSave(), most of that is deleting cache tags as mentioned above and 10% in preSave(), the actual saving of menu links is almost irrelevant ;)
*** 27% is RouteSubscriberBase::onAlterRoutes().
=> We have issues to limit the number of dfac() calls, looks like views ui tests add another call to the 3 common ones.

* DrupalKernel::buildContainer() is called 264 times in total, taking ~13% of the time. Looks like the whole compiler pass stuff takes up a lot of time, especially AnalyzeServiceReferencesPass is a slow one with 5% of the total execution time.

Nope, it's not. That is *only* config *schema*. Unless you want to move that to the database too? And even if we would, we'd still have to read from yml to put it in the database in the first place. Wouldn't change anything.

Clarification: The actual yaml parsing calls are everything, but 75% of the config yaml parsing is config schema. And only meant that with those 20%, all FileStorage::read() is ~25%.

#2171683: Remove all Simpletest overrides and rely on native multi-site functionality instead has apparently slowed down tests, the only thing I could find is that it now uses the Database cache during the installer, which leads to a ton of cache tag delete queries. Opened #2194273: Avoid repeated cache tag deletions to deal with that.

Did some new tests, we did achieve some nice improvements over the last days.

It also looks like a number of other issues like the improved extension parsing and other issues (menu_router removal possibly? improved testbot run times a lot, the fast bots are currently < 30min, which is pretty impressive!

Comparing to the previous results is tricky as quite a few things changed including the total runtime went down quite a bit (3m22, but not sure if I had xdebug enabled then), a few notes:

* Cache::deleteTags() is down to ~1% of the whole runtime
** What's even more interesting is that 35% of that is drupal_get_complete_schema(), I just re-rolled #2185015: Remove SchemaCache and CacheArray which now removes that tag.
** Another 35% is caused by Drupal\Core\Config\CachedStorage::write(), aka the listByPrefix cache, I want to try and switch to a CacheCollector there... That tag is also deleted *a lot* on actual sites. (Should we reset those numbers on a full cache clear? Otherwise they'd climb forever?)

Config reading is actually up in terms of percentage, because that's not much faster, in comparison to improvements that were made elsewhere. But as mentioned before, there is probably quite some overhead hidden in all those method calls.
* Config::save() is 25% of the whole execution time now, 90% of that spent in typed config/type casting.
* FileStorage::read() is 32% of the time, a large part of it through save()/typed config, so those two numbers overlap a lot.

Enabling the yaml extension and trying again, total execution time is 2m17:
* Config::save() is now down to 3%
* FileStorage::read() is <1%.

To figure out how much of that is xhprof overhead, disabling it and running tests again:

With yaml extension: 1 min 51 sec (2m17 with xhprof)
Without: 2 min 48 sec ( 3m22 with xhprof)

Xhprof or not, the yaml extension makes an impressive difference, at least for tests.

Edit: Using the yaml extension needs the patch from #1920902: Add a Drupal Yaml wrapper so we can default to PECL Yaml component if it is available.