AcquiaCloudPurger should catch AcquiaCloudBalancerException exceptions

Currently, the Acquia Purge module does not catch AcquiaCloudBalancerException exceptions, which kill the PHP process, effectively disabling Varnish cache invalidation. (This exception is thrown for any 4xx response.)

One common place this issue presents is with customers with custom VCL. When Ops provisions a test balancer, that test balancer’s IP is injected into Drupal settings' reverse_proxies array:

/var/www/html/docroot$ drush php:eval "print_r(\Drupal\Core\Site\Settings::get('reverse_proxies'));"
Array
(
    [0] => 13.217.163.195
    [1] => 3.217.80.87
    [2] => 3.238.239.196
)

In this case, 13.217.163.195 is the test balancer.

If the application’s EIPs (ACC) or Web EIPs (ACN) are not listed in an ACL on the VCL configuration, then this can impact production Varnish invalidations. In this case, the test balancer responds with a 421 response, which causes an AcquiaCloudBalancerException to be thrown. As a result, Varnish cache is not invalidated for either of the two production balancers.

The underlying issue here is that a test load balancer should not cause production Varnish cache invalidations to fail.

While this issue can present for customers deploying cVCL for the first time, it can also present when additional non-prod environments are added to an application. E.g. A customer has a uat environment provisioned. Between the time that the environment has been provisioned and the cVCL has been updated with its Web EIP(s), this behavior would be expected.

Proposed resolution

AcquiaCloudPurger should catch and log AcquiaCloudBalancerException exceptions. Error reporting will be retained; however, fatal errors will be avoided.