acquia_purge 7.x-1.2-beta1

It is my pleasure to announce version 7.x-1.2-beta1 of the Acquia Purge module, a beta that paves the path towards 7.x-1.2, which is due next week if all goes well. Given the current-day user base of over 2600 installations, broader testing will be required to assure a soft landing of the next stable.

The releases consists of 6 months worth of work involving stabilizing improvements, edge-case features and code reform in the form of a new object oriented core API.

Action requiring changes

• New "Use manual purge blocks" permission
  To better aid real world use cases, the blocks "Manual purge form (current page)" and "Manual purge form (paths)" now have their own permission "Use manual purge blocks". Once you ran update.php or drush updb, all roles that have the purge on-screen permission, will automatically see this new permission assigned. This means everything remains the work as it did, but you still might want to review this.
• Non-breaking API changes
  If you use any functions of the Acquia Purge module in your own modules, please see the section "New object oriented stack" below for more information.

Changes affecting all users

• AJAX client-side processor feels faster
  Targeted at every day users, animation speeds have been increased and UI waiting times reduced. This makes the processor disappear within the blink of an eye for simple content updates, but drastically increases the perception of speed for every day editors that have to work with Acquia Purge.
• Blocking error limit now dynamic
  In previous versions, processing of the queue always jammed after 10 HTTP errors, this limit is now dynamic, higher by default and configurable. The complexity of your configuration (domains, balancers, HTTPS etc) determines the slowdown factor (which you can find with drush ap-diagnosis). This slowdown factor is now used at the basis for the dynamically calculated error limit (limit = slowdown factor^3) and generally becomes much more tolerant for complexer setups. In addition, the dynamically calculated limit can also be overridden by setting $conf['acquia_purge_errorlimit'] = N;, which allows exceptional cases to put it very high (not recommended for most sites).
• Improved failure logging
  When purges used to fail in previous versions, the same lengthy error message was logged with cUrl debugging symbols. Now, different messages are logged depending on the Curl exit codes, to better aid diagnostics after the fact and to prevent users from having to understand every Curl flag.
  Example messages:
  • Cannot connect to 10.0.2.1:80, '/' released to queue!
  • Connection to 10.0.2.1 timed out, '/' released to queue!
  • Connecting to 10.0.2.1 exceeded 2 seconds, '/node' released to queue!
  • Cannot resolve host 10.0.2.1, '/' released to queue!
• Locks now stored as state
  End-users won't and should not notice any difference from this, although it is a behavioral change under the hood. Previously, Drupal's lock API was leveraged together with a 'locked' field within the module's state API, this is now no longer the case and only uses state. This means less queries for the majority of sites, all state information in one placed (memcached/disk) and reduced complexity.
• $conf['acquia_purge_cron'] = TRUE; no longer disables AJAX processor
  Enabling cron processing no longer disables the AJAX processor. Instead of disabling it, cron processing will simply guard regular processing of the queue and do only what its setting suggests.
• AJAX processor cannot be disabled, but visually hidden
  It is no longer possible and in no scenario recommended to technically remove the AJAX client-side queue processor, which forms the last line of defense against voluminous queues. Instead, the opt-in $conf['acquia_purge_cron'] and $conf['acquia_purge_lateruntime'] settings complement processing possibilities which simply reduce the times the client-side processor will kick in. Although revoking the "purge on-screen" permission already accomplishes this per-role, it now also possible to visually hide the processor site-wide by setting $conf['acquia_purge_silentmode'] = TRUE;.

Opt-in changes

• Better sites.php support
  Acquia Purge parses sites.php when it exists and when $conf['acquia_purge_domains'] is not set. However, by default it doesn't detect records ending on paths, for sites running on subdirectories (e.g. /subsite/). By setting $conf['acquia_purge_sphpskippath'] = FALSE;, experimental support for parsing paths is enabled.
• All-risk mode that disables full blockades
  When the queue reaches dangerous levels or when you have a lot of domain names set, Acquia Purge can decide to shut itself down to prevent disasters. In exceptional cases, this proved to be frustrating and distracting where it was also well understood that the site setup has to change on the long term. For these, it is is possible to set $conf['acquia_purge_allriskmode'] = TRUE;, but this also excludes your entitlement to support from Acquia - so be aware!
• Passive mode for development environments
  With $conf['acquia_purge_passivemode'] = TRUE; it is now possible disable the integration with the Expire module, effectively ceasing purges on content changes. This can be handy for development specific settings.php files where the module should remain enabled but not get in the way.
• AcquiaPurgeSmartQueue backend that expires items by TTL
  A new queue backend automatically reduces work load by dropping items that are assumed to have expired already externally, by leveraging the page_cache_maximum_age variable that determines the TTL of cached objects. You should not use this if you are setting the page_cache_maximum_age variable dynamically or when you dynamically influence the Cache-Control header in your own modules or using the cacheexclude module.
• Support for "server-side" runtime processing
  By setting $conf['acquia_purge_lateruntime'] = TRUE;, it is now possible to enable experimental in-request processing of chunks of the queue. Although it only attempts to process as much as runtime resources allow (think memory, time), this likely leads to the AJAX processor no longer showing up for the majority of simple configurations. However, when the queue volume grows bigger at once (bulk content updates, etc), the processing will "flow over" to the AJAX processor. Also note that although this is prevented as much as possible, there is a theoretic risk that pages can crash by Drupal running out of memory or time - so thorough testing is highly recommended!.
• Possibility to set a different X-Acquia-Purge token
  By setting $conf['acquia_purge_token'] = 'mysecret';, it is now possible to alter the value of the outgoing X-Acquia-Purge header sent to your pair of Acquia Cloud load balancers. This helps offset DDOS style attacks but requires you to get in touch with a Acquia support representative in order to set it up beforehand.

Bug fixes

• #2379227: by mattkoltermann: Conflict with Drupal 7.33+ producing "Illegal string offset" warnings on overview page.
• Fixed division-by-zero errors within capacity calculation logic.
• #2442211: by heathdutton: Notice: Use of undefined constant BLOCK_VISIBILITY_NOTLISTED.
• #2451311: by Dave Reid: Errors running updates when module is disabled.
• #2387377: Purge this page button has no effect for second page of view result.
• #2392563: by grasmash, Charles Belov: Using purge button to purge front page doesn't purge front page.
• #2352347: by matthew.lavoie, mcpuddin, lahoosascoots: Cloud Domain Detection in /etc/apache2/conf.d getting too many domains.
• #2375451: Module runs even when $_ENV['AH_SITE_ENVIRONMENT'] isn't available.
• #2392571: by baaj, nmenon: don't blindly use drush_set_error() everywhere.
• #2368513: by emptyvoid: Separate permissions for ajax status and purge buttons.
• #2498329: by nielsvm, Waliur: Acquia_purge update error.
• #2356633: by grasmash, nvmourik: introducing hooks hook_acquia_purge_purge_failure(), hook_acquia_purge_purge_success() and helper method…
• Access logs will no longer get flooded with 403's to /acquia_purge_ajax_processor.

Quality assurance improvements

• #2352417: by Charles Belov, nielsvm: Make manual purge blocks sound less threatening
• #2404983: various grammar and codestyle issues.
• #2387433: by Elijah Lynn: Cleanup docblocks to better meet standards.
• Documentation improvements all over the place.
• Fully processed feedback from coder_review, phpcs and Pareview, code quality up to par again.
• Now deliberately exceeding the :80 columns limit there where breaking did not make sense, in line with newer code style standards (e.g. within t()).

New object oriented stack

This release introduces a object oriented core API to better aid long-term maintenance and align technical terminology with the work being done in the purge module for Drupal 8. The stack resolves around a central AcquiaPurgeService object accessed through _acquia_purge_service() and provides methods similar to their procedural predecessors that have now been deprecated .

• Queue
  The queue API introduces AcquiaPurgeQueueInterface which extends DrupalReliableQueueInterface and adds specific methods to deal with the queue much more efficiently by bundling queries. The backend AcquiaPurgeEfficientQueue is the default with AcquiaPurgeSmartQueue as alternative.
• State
  State data is information that describes the state of the queue and its processing status. This data is kept in sync with the actual data that is within Drupal's queue table and is either stored using the AcquiaPurgeDiskStateStorage backend, or using AcquiaPurgeMemcachedStateStorage when you use memcached. AcquiaPurgeStateCounter objects are numeric AcquiaPurgeStateItem's that keep count of successes, failures and the total size of the queue.
• Processors
  Processors are actors adhering to AcquiaPurgeProcessorInterface, which process chunks of the queue and decide when to do so. The AJAX client-side processor moved entirely into the AcquiaPurgeAjaxProcessor backend, making it easier to maintain. Besides the standard AJAX processor, the opt-in backends AcquiaPurgeCronProcessor and AcquiaPurgeRuntimeProcessor have been introduced.

Without patching the module, it is not possible to register your own queue, state storage or processor backends as this is not a priority with Drupal 8 on the horizon. However, if this is believed to be a necessity, I'm more then happy to hear this through the issue queue to introduce backend registry in the future.

Code example

You can queue items for purging like this:

$service = _acquia_purge_service();
$service->addPath('node/5?parameter');
$service->addPaths(array('news/section1', 'contact'));

If you run the code above as authenticated user during a web request (e.g.
not via Drush), it will trigger the AJAX processor for that users next page
load. However, if you want to process directly yourself, you can work a chunk
from the queue with this:

$service = _acquia_purge_service();
if ($service->lockAcquire()) {
    $service->process();
    $service->lockRelease();
}

The AcquiaPurgeService::process() call will run as long as PHP has resources
for and depends on Acquia Purge's internal capacity calculation. When it runs
from the CLI, it has a higher max_execution_time and will therefore process
much more at once, then it will when triggered through a web server.

Deprecation: third party code still works

With the new stack, 22 public and private helper functions have been deprecated and moved into acquia_purge.deprecated.inc. The old functions now wrap around the new object oriented stack and will log heavily to watchdog to inform you about their deprecation.

These symbols have been deprecated:

• _acquia_purge_get()
• _acquia_purge_ui_path_blacklisted()
• _acquia_purge_ui_trigger_purge()
• _acquia_purge_queue()
• _acquia_purge_queue_add()
• _acquia_purge_queue_clear()
• _acquia_purge_queue_deduplicate()
• _acquia_purge_queue_history()
• _acquia_purge_queue_is_user_purging()
• _acquia_purge_queue_lock()
• _acquia_purge_queue_pop()
• _acquia_purge_queue_processpurge()
• _acquia_purge_queue_processpurge_requests()
• _acquia_purge_queue_stats()
• _acquia_purge_state_initialize()
• _acquia_purge_state_commit()
• _acquia_purge_state_get()
• _acquia_purge_state_set()
• _acquia_purge_state_wipe()
• acquia_purge_purge_path()
• acquia_purge_purge_paths()
• acquia_purge_purge_node()

If you maintain in-house written code making use of any of the above, your code will still work. The deprecated symbols get included in the global scope of acquia_purge.module, which means thta there is no technical difference compared to previous versions. However, this comes at a small overhead cost and it is therefore within the line of expectation that the require_once to acquia_purge.deprecated.inc will disappear in the future. This means you should port over to the new stack to suppress logged messages and protect your code against the announced API change in the future.

Thank you

You read up this far, wow!

I'm very happy to be able to deliver you acquia_purge, but wouldn't be able to do so without the very valuable feedback I have been getting over time. Real world testing and actual human editors provide very valuable feedback and this has enabled me to iterate upon earlier work and continues to improve the module for everybody. Although not always every request can be honored, I'm more then happy to receive more real-world feedback and bug reports when you encounter them, therefore please keep them coming!

Thank you,

Niels