On sites with a high number of default views, the views_default_views cache size can grow very large which increases the memory profile for every page that includes a view (see views_get_view()). One strategy to reduce this overhead is to split up the cache from a single array of all default views into individual rows containing each individual view.

This patch is a first attempt at this approach and maintains full API/backwards compatibility. This issue is generalizable and is a general problem to solve when dealing with caching defaults -- probably of interest as well for ctools and other implementers of exportables.

Applies & tested against 2.6 as well as DRUPAL-6--3 HEAD.

Some notes:

  • On some basic tests on Atrium, this patch reduced memory consumption per page by about 2.5MB. The current views defaults loading process costs about 3MB on Atrium, while a load from the segmented cache is about 0.5MB.
  • Why not "sync" default views to the db? We could do this, but we would need to keep track of which views are defaults and which are normal/overrides. Using cache tables for this actually made more sense as it simplifies the code required to protect user-manipulated views data. The cached default views really can be blown away without any concerns.
  • Other things to address: when the cache is initially built, the array passed to drupal_alter() for altering default views is still a single unified array. It may be worth splitting this up as well to keep memory consumption altogether down to little chunks of data so that no matter how many views your sites has the memory profile remains low.
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

merlinofchaos’s picture

Status: Needs review » Fixed

Ok, my first concern was that this would cause problems for get-all-views but it doesn't seem to really be an issue since that skips the cache which is just fine. Committed to all branches.

Note that CTools would probably benefit from this kind of thing too in its handling of defaults.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

yhahn’s picture

Status: Closed (fixed) » Needs review
FileSize
733 bytes

I discovered a bug with my previous patch - the index cache also needs to be localized in order to keep a record of whether the default views have been retrieved and cached in the current site language. Without this patch, sites in multiple languages can have default views that are not retrieved and cached properly.

Patch attached, applies to both DRUPAL-6--2 and DRUPAL-6--3.

bendiy’s picture

@yhahn

Thanks for this patch! I'm developing a site with over 60 different Views, most of them are fairly large and I was experiencing terrible page load times. devel was reporting this query often taking in excess of 3000 ms.
SELECT data, created, headers, expire, serialized FROM cache_views WHERE cid = 'views_default_views:en'

On some page loads, I was seeing results like this:
Executed 76 queries in 296.6 milliseconds. Queries taking longer than 5 ms and queries executed more than once, are highlighted. Page execution time was 6105.71 ms.
They now look like this:
Executed 77 queries in 154.87 milliseconds. Queries taking longer than 5 ms and queries executed more than once, are highlighted. Page execution time was 695.25 ms.

I have applied your patch @ #3 for multiple language sites. It appears to be working fine for me, however, I don't use any multi-language featuers at the moment.

THANK YOU!

YK85’s picture

subscribing

what sort of error would we experience on a multi-language site without the correction in patch#3?
i hope the correction will be committed after review by merlinofchaos

yhahn’s picture

You will experience pretty serious #fail when switching languages. Basically the cache is not properly populated for non-default languages and you will get bad page loads on views in the alternate language. If you're running a multilingual site, definitely apply the patch (and test!) for the time being.

merlinofchaos’s picture

Status: Needs review » Fixed

It looks like this already got committed in a different patch.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

catch’s picture

This introduced a race condition on high traffic sites, see #853864: views_get_default_view() - race conditions and memory usage.