diff -u b/README.md b/README.md --- b/README.md +++ b/README.md @@ -1,27 +1,85 @@ # drupal-purger -This module allows Section’s global, distributed caching layer to quickly respond to invalidation events from a Drupal instance in exactly the same way that Drupal’s internal cache or a local varnish cache running on the host machine does, ensuring that the content in Section’s global caching layer is always up to date. +This module allows Section’s global, distributed caching layer to quickly +respond to invalidation events from a Drupal instance in exactly the same +way that Drupal’s internal cache or a local varnish cache running on the +host machine does, ensuring that the content in Section’s global caching +layer is always up to date. ## Installation -For installation instructions please [go here](https://www.section.io/docs/how-to/drupal-setup/drupal8/) +For installation instructions please +[go here](https://www.section.io/docs/how-to/drupal-setup/drupal8/) ## Development Notes on Architecture: -- Assuming you have the purge module installed, navigate to the Purger's UI at `/admin/config/development/performance/purge` -- This module depends on the key module to store your password for aperture. Make sure your password is correct because the purger will send hundreds of requests to the API which could potentially lock out your account if the authentication attempts fail. -- Drupal will queue bans with the Core tags queuer, but make sure you have a processor installed (cron, or LateRuntimeProcessor) -- The bundled purger will combine tag purges into one request per every 250 tag invalidations in the queue. Bundled URL purging is not supported but we are open to feedback – after consulting with the back-end team, the opinion is that this would be too resource intensive. -- Tags are hashed to preserve privacy. If varnish caches a page while logged in, you will need to clear the cache. Consider updating your VCL to not cache requests that have either tags specific to authentication, or a session cookie. More info on this in the installation instructions linked above. -- There are many other invalidation types that you can use with other processors and queuers. See the table below -- The actual purger functions are performed by code inside `src/Plugin/Purge/Purger/SectionPurgerBase` as well as the two child classes of it: `SectionPurger` and `SectionBundledPurger`. -- The configuration form is controlled by code in `src/Form`. Data input via this form is stored in `/src/Entity/SectionPurgerSettings.php`. If you want to create a hardcoded variable value that is not customer facing, simply include this variable in this file and make no reference to it in the form. If you'd like a default value that is subsequently adjustable by the user, add that variable with the default value to `SectionPurgerSettings.php` and add its field to the user input form. There are examples of both of these in the existing codebase. Note that in order for a variable to be overwritten by the form input, it needs to be enumerated in the `section_purger.schema.yml` -- This module was developed as a fork of the generic http purger. Any questions on the development history can be answered by comparing the current state to that project — the git history is rather mangled due to moving between repos. -- The generic HTTP purger also contained code for a Bundled Purger (which lived in the Purger directory alongside the Section Purgers). The functionality of this has not been comprehensively understood, but in the abstract it coalesces multiple ban requests into a single API call. Because the choke point for this operation was believed to be Varnish Cache's ability to process bans rather than the API's ability to accept requests, this mode was determined to be unnecessary at the time. If this functionality is desired, the relevant code (it was not adapted for Section in any way before deletion) can be recovered from version control or from the source HTTP Purger. -- The plugin currently supports an optional sitename feature designed to support Drupal multisites. If no sitename is specified, then the plugin clears cache for any pages with the relevant cache tags. If the sitename is filled in, the plugin appends a check for the specified hostname. Each Drupal site within a multisite has its own admin dashboard and its own UI in which to enable the purger. -- The plugin is able to handle alternate varnish proxy names (as written in your section config). Usually the name for the proxy is 'varnish' but some configurations have multiple instances of varnish running for different tasks. Note that if using multiple varnish instances, you will have to enable another purger in order to handle multiple instances of varnish. If using multi-site as well as multiple varnish proxies, you will need a purger instance per-varnish instance per-site. -- If you wish to uninstall the Plugin from a live site for the purposes of testing, the most efficient way to do so is to uninstall the plugin from the admin console. Once it has been uninstalled there, you can delete it from the filesystem of the live box (Drupal may overwrite old files of the same name if you try and upload the same module again — this has not been proven). Deleting the module from the filesystem while it is still installed will cause a total failure of the site. The only known fix for this is a reinstall of Drupal as a whole, although there are likely easier ways to fix it. +- Assuming you have the purge module installed, +navigate to the Purger's UI at `/admin/config/development/performance/purge` +- This module depends on the key module to store your password for aperture. +Make sure your password is correct because the purger will send hundreds of +requests to the API which could potentially lock out your account if the +authentication attempts fail. +- Drupal will queue bans with the Core tags queuer, but make sure you have a +processor installed (cron, or LateRuntimeProcessor) +- The bundled purger will combine tag purges into one request per every +250 tag invalidations in the queue. Bundled URL purging is not supported +but we are open to feedback – after consulting with the back-end team, +the opinion is that this would be too resource intensive. +- Tags are hashed to preserve privacy. If varnish caches a page while +logged in, you will need to clear the cache. Consider updating your VCL +to not cache requests that have either tags specific to authentication, +or a session cookie. More info on this in the installation +instructions linked above. +- There are many other invalidation types that you can use with other +processors and queuers. See the table below +- The actual purger functions are performed by code inside +`src/Plugin/Purge/Purger/SectionPurgerBase` as well as the two child +classes of it: `SectionPurger` and `SectionBundledPurger`. +- The configuration form is controlled by code in `src/Form`. Data input +via this form is stored in `/src/Entity/SectionPurgerSettings.php`. +If you want to create a hardcoded variable value that is not customer +facing, simply include this variable in this file and make no reference +to it in the form. If you'd like a default value that is subsequently +adjustable by the user, add that variable with the default value to +`SectionPurgerSettings.php` and add its field to the user input form. +There are examples of both of these in the existing codebase. +Note that in order for a variable to be overwritten by the form input, +it needs to be enumerated in the `section_purger.schema.yml` +- This module was developed as a fork of the generic http purger. +Any questions on the development history can be answered by comparing +the current state to that project — the git history is rather mangled +due to moving between repos. +- The generic HTTP purger also contained code for a Bundled Purger +(which lived in the Purger directory alongside the Section Purgers). +The functionality of this has not been comprehensively understood, +but in the abstract it coalesces multiple ban requests into a single API call. + Because the choke point for this operation was believed to be Varnish Cache's + ability to process bans rather than the API's ability to accept requests, + this mode was determined to be unnecessary at the time. If this functionality + is desired, the relevant code (it was not adapted for Section in any + way before deletion) can be recovered from version control or from the source + HTTP Purger. +- The plugin currently supports an optional sitename feature designed to support +Drupal multisites. If no sitename is specified, then the plugin clears cache for +any pages with the relevant cache tags. If the sitename is filled in, the plugin +appends a check for the specified hostname. Each Drupal site within a multisite +has its own admin dashboard and its own UI in which to enable the purger. +- The plugin is able to handle alternate varnish proxy names (as written in +your section config). Usually the name for the proxy is 'varnish' but some +configurations have multiple instances of varnish running for different tasks. +Note that if using multiple varnish instances, you will have to enable another +purger in order to handle multiple instances of varnish. If using multi-site +as well as multiple varnish proxies, you will need a purger instance +per-varnish instance per-site. +- If you wish to uninstall the Plugin from a live site for the purposes of +testing, the most efficient way to do so is to uninstall the plugin from the +admin console. Once it has been uninstalled there, you can delete it from the +filesystem of the live box (Drupal may overwrite old files of the same name if +you try and upload the same module again — this has not been proven). Deleting +the module from the filesystem while it is still installed will cause a total +failure of the site. The only known fix for this is a reinstall of Drupal as +a whole, although there are likely easier ways to fix it. | Invalidation Type | Description | diff -u b/Tests/SectionPurgerFormTestBase.php b/Tests/SectionPurgerFormTestBase.php --- b/Tests/SectionPurgerFormTestBase.php +++ b/Tests/SectionPurgerFormTestBase.php @@ -53,7 +53,7 @@ } /** - * Tests \Drupal\section_purger\Form\SectionPurgerFormBase::buildFormTokensHelp(). + * Tests SectionPurgerFormBase::buildFormTokensHelp(). */ public function testTokensHelp() { $this->drupalLogin($this->admin_user); @@ -73,7 +73,7 @@ $form_state->addBuildInfo('args', [$this->formArgs]); $form_state->setValues( [ - 'connect_timeout' => 0.3, + 'connectTimeout' => 0.3, 'timeout' => 0.1, 'name' => 'foobar', ] @@ -85,7 +85,7 @@ $form_state->addBuildInfo('args', [$this->formArgs]); $form_state->setValues( [ - 'connect_timeout' => 2.3, + 'connectTimeout' => 2.3, 'timeout' => 7.7, 'name' => 'foobar', ] @@ -98,7 +98,7 @@ $form_state->addBuildInfo('args', [$this->formArgs]); $form_state->setValues( [ - 'connect_timeout' => 0.0, + 'connectTimeout' => 0.0, 'timeout' => 0.0, 'name' => 'foobar', ] @@ -108,13 +108,13 @@ $errors = $form_state->getErrors(); $this->assertEqual(2, count($errors)); $this->assertTrue(isset($errors['timeout'])); - $this->assertTrue(isset($errors['connect_timeout'])); + $this->assertTrue(isset($errors['connectTimeout'])); // Submit timeout values that are too high and confirm the validation error. $form_state = $this->getFormStateInstance(); $form_state->addBuildInfo('args', [$this->formArgs]); $form_state->setValues( [ - 'connect_timeout' => 2.4, + 'connectTimeout' => 2.4, 'timeout' => 7.7, 'name' => 'foobar', ] @@ -124,7 +124,7 @@ $errors = $form_state->getErrors(); $this->assertEqual(2, count($errors)); $this->assertTrue(isset($errors['timeout'])); - $this->assertTrue(isset($errors['connect_timeout'])); + $this->assertTrue(isset($errors['connectTimeout'])); } /** @@ -143,20 +143,20 @@ 'password' => 'password', 'port' => 8080, 'path' => 'node/1', - 'request_method' => 1, + 'requestMethod' => 1, 'scheme' => 0, 'verify' => TRUE, 'show_body_form' => 1, - 'body_content_type' => 'foo/bar', + 'bodyContentType' => 'foo/bar', 'body' => 'baz', 'timeout' => 6, - 'runtime_measurement' => 1, - 'connect_timeout' => 0.5, - 'cooldown_time' => 0.8, - 'max_requests' => 25, - 'http_errors' => 1, + 'runtimeMeasurement' => 1, + 'connectTimeout' => 0.5, + 'cooldownTime' => 0.8, + 'maxRequests' => 25, + 'httpErrors' => 1, ]; - $this->drupalPostForm($this->route, $edit, t('Save configuration')); + $this->drupalPostForm($this->route, $edit, 'Save configuration'); $this->drupalGet($this->route); foreach ($edit as $field => $value) { $this->assertFieldById('edit-' . str_replace('_', '-', $field), $value); diff -u b/src/Entity/SectionPurgerSettings.php b/src/Entity/SectionPurgerSettings.php --- b/src/Entity/SectionPurgerSettings.php +++ b/src/Entity/SectionPurgerSettings.php @@ -114,7 +114,7 @@ * * @var string */ - public $request_method = 'POST'; + public $requestMethod = 'POST'; /** * The HTTP scheme. @@ -159,7 +159,7 @@ * * @var string */ - public $body_content_type = 'application/json'; + public $bodyContentType = 'application/json'; /** * Performance settings. @@ -169,11 +169,11 @@ * Runtime measurement. * * When FALSE, dynamic capacity calculation will be disabled and based upon - * the connect_timeout and timeout settings. + * the connectTimeout and timeout settings. * * @var bool */ - public $runtime_measurement = TRUE; + public $runtimeMeasurement = TRUE; /** * The timeout of the request in seconds. @@ -187,7 +187,7 @@ * * @var float */ - public $connect_timeout = 1.0; + public $connectTimeout = 1.0; /** * Cooldown time. @@ -197,7 +197,7 @@ * * @var float */ - public $cooldown_time = 0.0; + public $cooldownTime = 0.0; /** * Maximum requests. @@ -208,7 +208,7 @@ * * @var int */ - public $max_requests = 250; + public $maxRequests = 250; /** * Success resolution. @@ -222,5 +222,5 @@ * @see http://docs.guzzlephp.org/en/latest/request-options.html#http-errors */ - public $http_errors = TRUE; + public $httpErrors = TRUE; } diff -u b/src/Form/SectionPurgerFormBase.php b/src/Form/SectionPurgerFormBase.php --- b/src/Form/SectionPurgerFormBase.php +++ b/src/Form/SectionPurgerFormBase.php @@ -26,9 +26,8 @@ * * @var array * - * @todo - * Confirm if all relevant HTTP methods are covered. - * http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html + * @todo Confirm if all relevant HTTP methods are covered. + * http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html */ protected $requestMethods = [ 'BAN', @@ -83,7 +82,7 @@ /** * {@inheritdoc} */ - public function getFormID() { + public function getFormId() { return 'section_purger.configuration_form'; } @@ -209,7 +208,11 @@ for ($i = 0; $i < $form_state->get('headers_items_count'); $i++) { if (!isset($form['headers']['headers'][$i])) { $header = isset($settings->headers[$i]) ? $settings->headers[$i] : - ['field' => 'Section-Cache-Tags', 'value' => '[invalidation:expression]']; + [ + 'field' => 'Section-Cache-Tags', + 'value' => '[ + invalidation:expression]', + ]; $form['headers']['headers'][$i]['field'] = [ '#type' => 'textfield', '#default_value' => $header['field'], @@ -267,36 +270,36 @@ '#group' => 'tabs', '#title' => $this->t('Performance'), ]; - $form['performance']['cooldown_time'] = [ + $form['performance']['cooldownTime'] = [ '#type' => 'number', '#step' => 0.1, '#min' => 0.0, '#max' => 3.0, '#title' => $this->t('Cooldown time'), - '#default_value' => $settings->cooldown_time, + '#default_value' => $settings->cooldownTime, '#required' => TRUE, '#description' => $this->t('Number of seconds to wait after a group of HTTP requests (so that other purgers get fresh content)'), ]; - $form['performance']['max_requests'] = [ + $form['performance']['maxRequests'] = [ '#type' => 'number', '#step' => 1, '#min' => 1, '#max' => 500, '#title' => $this->t('Maximum requests'), - '#default_value' => $settings->max_requests, + '#default_value' => $settings->maxRequests, '#required' => TRUE, '#description' => $this->t("Maximum number of HTTP requests that can be made during Drupal's execution lifetime. Usually PHP resource restraints lower this value dynamically, but can be met at the CLI."), ]; - $form['performance']['runtime_measurement'] = [ + $form['performance']['runtimeMeasurement'] = [ '#title' => $this->t('Runtime measurement'), '#type' => 'checkbox', - '#default_value' => $settings->runtime_measurement, + '#default_value' => $settings->runtimeMeasurement, ]; $form['performance']['runtime_measurement_help'] = [ '#type' => 'item', '#states' => [ 'visible' => [ - ':input[name="runtime_measurement"]' => ['checked' => FALSE], + ':input[name="runtimeMeasurement"]' => ['checked' => FALSE], ], ], '#description' => $this->t('When you uncheck this setting, capacity will be based on the sum of both timeouts. By default, capacity will automatically adjust (up and down) based on measured time data.'), @@ -311,22 +314,22 @@ '#required' => TRUE, '#states' => [ 'visible' => [ - ':input[name="runtime_measurement"]' => ['checked' => FALSE], + ':input[name="runtimeMeasurement"]' => ['checked' => FALSE], ], ], '#description' => $this->t('The timeout of the request in seconds.'), ]; - $form['performance']['connect_timeout'] = [ + $form['performance']['connectTimeout'] = [ '#type' => 'number', '#step' => 0.1, '#min' => 0.1, '#max' => 4.0, '#title' => $this->t('Connection timeout'), - '#default_value' => $settings->connect_timeout, + '#default_value' => $settings->connectTimeout, '#required' => TRUE, '#states' => [ 'visible' => [ - ':input[name="runtime_measurement"]' => ['checked' => FALSE], + ':input[name="runtimeMeasurement"]' => ['checked' => FALSE], ], ], '#description' => $this->t('The number of seconds to wait while trying to connect to a server.'), @@ -339,13 +342,13 @@ public function validateForm(array &$form, FormStateInterface $form_state) { // Validate that our timeouts stay between the boundaries purge demands. - $timeout = $form_state->getValue('connect_timeout') + $form_state->getValue('timeout'); + $timeout = $form_state->getValue('connectTimeout') + $form_state->getValue('timeout'); if ($timeout > 10) { - $form_state->setErrorByName('connect_timeout'); + $form_state->setErrorByName('connectTimeout'); $form_state->setErrorByName('timeout', $this->t('The sum of both timeouts cannot be higher than 10.00 as this would affect performance too negatively.')); } elseif ($timeout < 0.4) { - $form_state->setErrorByName('connect_timeout'); + $form_state->setErrorByName('connectTimeout'); $form_state->setErrorByName('timeout', $this->t('The sum of both timeouts cannot be lower as 0.4 as this can lead to too many failures under real usage conditions.')); } } @@ -372,12 +375,12 @@ $form_state->setValue('headers', $headers); } - // Rewrite 'scheme' and 'request_method' to have the right CMI values. + // Rewrite 'scheme' and 'requestMethod' to have the right CMI values. if (!is_null($scheme = $form_state->getValue('scheme'))) { $form_state->setValue('scheme', $this->schemes[$scheme]); } - if (!is_null($method = $form_state->getValue('request_method'))) { - $form_state->setValue('request_method', $this->requestMethods[$method]); + if (!is_null($method = $form_state->getValue('requestMethod'))) { + $form_state->setValue('requestMethod', $this->requestMethods[$method]); } // Iterate the config object and overwrite values found in the form state. diff -u b/src/Plugin/Purge/DiagnosticCheck/ConfigurationCheck.php b/src/Plugin/Purge/DiagnosticCheck/ConfigurationCheck.php --- b/src/Plugin/Purge/DiagnosticCheck/ConfigurationCheck.php +++ b/src/Plugin/Purge/DiagnosticCheck/ConfigurationCheck.php @@ -29,7 +29,7 @@ protected $purgePurgers; /** - * Constructs a \Drupal\purge\Plugin\Purge\DiagnosticCheck\PurgerAvailableCheck object. + * Constructs a PurgerAvailableCheck object. * * @param array $configuration * A configuration array containing information about the plugin instance. @@ -74,7 +74,9 @@ $labels = $this->purgePurgers->getLabels(); foreach ($plugins as $id => $settings) { $t = ['@purger' => $labels[$id]]; - foreach (['name', 'hostname', 'account', 'application', 'username', 'password', 'environmentname', 'port', 'request_method', 'scheme'] as $f) { + foreach ([ + 'name', 'hostname', 'account', 'application', 'username', 'password', 'environmentname', 'port', 'requestMethod', 'scheme', + ] as $f) { if (empty($settings->$f)) { $this->recommendation = $this->t("@purger not configured.", $t); return self::SEVERITY_ERROR; diff -u b/src/Plugin/Purge/Invalidation/RawInvalidation.php b/src/Plugin/Purge/Invalidation/RawInvalidation.php --- b/src/Plugin/Purge/Invalidation/RawInvalidation.php +++ b/src/Plugin/Purge/Invalidation/RawInvalidation.php @@ -6,7 +6,7 @@ use Drupal\purge\Plugin\Purge\Invalidation\InvalidationBase; /** - * Describes invalidation by raw varnish ban expression, e.g.: 'req.url ~ \.(jpg|jpeg|css|js)$'. + * Describes invalidation, e.g.: 'req.url ~ \.(jpg|jpeg|css|js)$'. * * @PurgeInvalidation( * id = "raw", diff -u b/src/Plugin/Purge/Purger/SectionBundledPurger.php b/src/Plugin/Purge/Purger/SectionBundledPurger.php --- b/src/Plugin/Purge/Purger/SectionBundledPurger.php +++ b/src/Plugin/Purge/Purger/SectionBundledPurger.php @@ -14,10 +14,11 @@ * id = "sectionbundled", * label = @Translation("Section Bundled Purger"), * configform = "\Drupal\section_purger\Form\SectionBundledPurgerForm", - * cooldown_time = 0.0, + * cooldownTime = 0.0, * description = @Translation("Configurable purger that sends a single HTTP request for a set of invalidation instructions."), * multi_instance = TRUE, - * types = {"url", "raw", "wildcardurl", "tag", "everything", "wildcardpath", "regex", "path", "domain"}, + * types = {"url", "raw", "wildcardurl", "tag", "everything", + * "wildcardpath", "regex", "path", "domain"}, * ) */ class SectionBundledPurger extends SectionPurgerBase implements PurgerInterface { @@ -27,19 +28,24 @@ */ public function invalidate(array $invalidations) { /* Since we implemented ::routeTypeToMethod(), this exception should not - ever occur because every invalidation type is routed to a respective function. - And when it does inevitably get called, it will throw an exception easily visible within the drupal logs. + ever occur because every invalidation type is routed + to a respective function. + And when it does inevitably get called, + it will throw an exception easily visible within the drupal logs. */ throw new \Exception("invalidate() called on a multi-type purger which routes each invalidatiaton type to its own method. This error should never be seen."); } /** + * Gets an invalidations array and returns groups of 250 invalidations. + * * Group(array $invalidations) - * This takes an invalidations array and returns groups of 250 invalidations. * * @param array $invalidations + * The array of invalidations. * * @return groups + * Returns a group of invalitations. */ public function group(array $invalidations) { $group = 0; @@ -75,7 +81,8 @@ $opt = $this->getOptions($token_data); $tags = new CacheTagsHeaderValue($group['expression'], Hash::cacheTags($group['expression'])); $exp = 'obj.http.Section-Cache-Tags ~ "(' . str_replace(' ', '|', $tags) . ')+"'; - // Adds this at the end if this instance has a site name in the configuration, for multi-site pages. + // Adds this at the end if this instance has a site name in + // the configuration, for multi-site pages. // the ampersands are url encoded to be %26%26 in sendReq. $this->logger->debug("[Tag] " . count($invalidations) . " tag invalidations were bundled to be: `" . $exp . "`"); $this->sendReq($invalidation, $uri, $opt, $exp); @@ -91,19 +98,23 @@ } /** + * Invalidates everything within the siteName. + * * InvalidateEverything($invalidations) - * This will use obj.status != 0 to ban every page that does not have an empty response + * This will use obj.status != 0 to ban every page that + * does not have an empty response. * * @param array $invalidations - * This takes in an array of Invalidation, and only make one purge because it only needs to ban everything once. - * - * @return void + * This takes in an array of Invalidation, and only make one purge because + * it only needs to ban everything once. */ public function invalidateEverything(array $invalidations) { // Invalidates everything within the siteName;. - $globalExpression = "obj.status != 0"; + // $globalExpression = "obj.status != 0";. $invalidation = $invalidations[0]; - // Only make one request, but if there are multiple everything invalidations queued then it will make sure all of them get marked appropriately. + // Only make one request, but if there are multiple everything + // invalidations queued then it will make sure all of them get marked + // appropriately. foreach ($invalidations as $inv) { $inv->setState(InvalidationInterface::PROCESSING); } @@ -112,7 +123,8 @@ $uri = $this->getUri($token_data); $opt = $this->getOptions($token_data); $exp = "obj.status != 0"; - // Adds this at the end if this instance has a site name in the configuration, for multi-site pages. + // Adds this at the end if this instance has a site name in the + // configuration, for multi-site pages. // the ampersands are url encoded to be %26%26 in sendReq. if ($this->getSiteName()) { $exp .= ' && req.http.host == "' . $this->getSiteName() . '"'; @@ -132,9 +144,9 @@ /** * {@inheritdoc} */ - public function invalidateURLs(array $invalidations) { + public function invalidateUrls(array $invalidations) { $this->logger->debug("[Domain] section does not support bundling URL purges but will pass this queue item on to the non-bundled purger."); - parent::invalidateURLs($invalidations); + parent::invalidateUrls($invalidations); } /** @@ -196,18 +208,26 @@ */ public function routeTypeToMethod($type) { /* - Purge has to be crystal clear about what needs invalidation towards its purgers, - and therefore has the concept of invalidation types. Individual purgers declare - which types they support and can even declare their own types when that makes sense. - Since Drupal invalidates its own caches using cache tags, the tag type is the most - important one to support in your architecture. (and is supported, and required) + Purge has to be crystal clear about what needs invalidation + towards its purgers, and therefore has the concept of invalidation types. + Individual purgers declare which types they support and can even declare + their own types when that makes sense. + Since Drupal invalidates its own caches using cache tags, + the tag type is the most important one to support in your architecture. + (and is supported, and required) domain Invalidates an entire domain name. everything Invalidates everything. - path Invalidates by path, e.g. news/article-1. This should not start with a slash, and should not contain the hostname. - regex This doesn't actually invalidate by regular expression. it allows for varnish ban expressions. e.g. obj.status == 404 && req.url ~ node\/(?).* !!!!!!!!!!!!!!! Invalidates by regular expression, e.g.: \.(jpg|jpeg|css|js)$. + path Invalidates by path, e.g. news/article-1. + This should not start with a slash, and should not contain the hostname. + regex This doesn't actually invalidate by regular expression. + it allows for varnish ban expressions. e.g. obj.status == 404 && + req.url ~ node\/(?).* !!!!!!!!!!!!!!! Invalidates by regular expression, + e.g.: \.(jpg|jpeg|css|js)$. tag Invalidates by Drupal cache tag, e.g.: menu:footer. - url Invalidates by URL, e.g. http://site.com/node/1. The protocol is specific; for example if invalidating an http request, the https equivalent will not be invalidated + url Invalidates by URL, e.g. http://site.com/node/1. + The protocol is specific; for example if invalidating an http request, + the https equivalent will not be invalidated wildcardpath Invalidates by path, e.g. news/*. wildcardurl Invalidates by URL, e.g. http://site.com/node/*. */ diff -u b/src/Plugin/Purge/Purger/SectionPurger.php b/src/Plugin/Purge/Purger/SectionPurger.php --- b/src/Plugin/Purge/Purger/SectionPurger.php +++ b/src/Plugin/Purge/Purger/SectionPurger.php @@ -13,10 +13,11 @@ * id = "section", * label = @Translation("Section Purger"), * configform = "\Drupal\section_purger\Form\SectionPurgerForm", - * cooldown_time = 0.2, + * cooldownTime = 0.2, * description = @Translation("Purger that sends invalidation expressions from your Drupal instance to the Section platform."), * multi_instance = TRUE, - * types = {"url", "wildcardurl", "tag", "everything", "wildcardpath", "regex", "path", "domain", "raw"}, + * types = {"url", "wildcardurl", "tag", "everything", "wildcardpath", + * "regex", "path", "domain", "raw"}, * ) */ class SectionPurger extends SectionPurgerBase implements PurgerInterface { @@ -26,8 +27,10 @@ */ public function invalidate(array $invalidations) { /* Since we implemented ::routeTypeToMethod(), this exception should not - ever occur because every invalidation type is routed to a respective function. - And when it does inevitably get called, it will throw an exception easily visible within the drupal logs. + ever occur because every invalidation type is routed to a + respective function. + And when it does inevitably get called, + it will throw an exception easily visible within the drupal logs. */ throw new \Exception("invalidate() called on a multi-type purger which routes each invalidatiaton type to its own method. This error should never be seen."); } @@ -43,7 +46,8 @@ $opt = $this->getOptions($token_data); $this->logger->debug($invalidation->getExpression()); $exp = 'obj.http.Section-Cache-Tags ~ "' . Hash::cacheTags([$invalidation->getExpression()])[0] . '"'; - // Adds this at the end if this instance has a site name in the configuration, for multi-site pages. + // Adds this at the end if this instance has a site name in the + // configuration, for multi-site pages. // the ampersands are url encoded to be %26%26 in sendReq. if ($this->getSiteName()) { $exp .= ' && req.http.host == "' . $this->getSiteName() . '"'; @@ -54,24 +58,27 @@ } /** + * Invalidates everything within the siteName. + * * InvalidateEverything($invalidations) - * This will use obj.status != 0 to ban every page that does not have an empty response + * This will use obj.status != 0 to ban every page that does not have + * an empty response. * * @param array $invalidations - * This takes in an array of Invalidation, processing them all in a loop, generally from the purge queue. - * - * @return void + * This takes in an array of Invalidation, processing them all in a loop, + * generally from the purge queue. */ public function invalidateEverything(array $invalidations) { // Invalidates everything within the siteName;. - $globalExpression = "obj.status != 0"; + // $globalExpression = "obj.status != 0";. foreach ($invalidations as $invalidation) { $invalidation->setState(InvalidationInterface::PROCESSING); $token_data = ['invalidation' => $invalidation]; $uri = $this->getUri($token_data); $opt = $this->getOptions($token_data); $exp = "obj.status != 0"; - // Adds this at the end if this instance has a site name in the configuration, for multi-site pages. + // Adds this at the end if this instance has a site name in the + // configuration, for multi-site pages. // the ampersands are url encoded to be %26%26 in sendReq. if ($this->getSiteName()) { $exp .= ' && req.http.host == "' . $this->getSiteName() . '"'; @@ -92,18 +99,26 @@ */ public function routeTypeToMethod($type) { /* - Purge has to be crystal clear about what needs invalidation towards its purgers, - and therefore has the concept of invalidation types. Individual purgers declare - which types they support and can even declare their own types when that makes sense. - Since Drupal invalidates its own caches using cache tags, the tag type is the most - important one to support in your architecture. (and is supported, and required) + Purge has to be crystal clear about what needs invalidation towards + its purgers, and therefore has the concept of invalidation types. + Individual purgers declare which types they support and can even declare + their own types when that makes sense. + Since Drupal invalidates its own caches using cache tags, + the tag type is the most important one to support in your architecture. + (and is supported, and required) domain Invalidates an entire domain name. everything Invalidates everything. - path Invalidates by path, e.g. news/article-1. This should not start with a slash, and should not contain the hostname. - regex This doesn't actually invalidate by regular expression. it allows for varnish ban expressions. e.g. obj.status == 404 && req.url ~ node\/(?).* !!!!!!!!!!!!!!! Invalidates by regular expression, e.g.: \.(jpg|jpeg|css|js)$. + path Invalidates by path, e.g. news/article-1. + This should not start with a slash, and should not contain the hostname. + regex This doesn't actually invalidate by regular expression. + It allows for varnish ban expressions. + e.g. obj.status == 404 && req.url ~ node\/(?).* !!!!!!!!!!!!!!! + Invalidates by regular expression, e.g.: \.(jpg|jpeg|css|js)$. tag Invalidates by Drupal cache tag, e.g.: menu:footer. - url Invalidates by URL, e.g. http://site.com/node/1. The protocol is specific; for example if invalidating an http request, the https equivalent will not be invalidated + url Invalidates by URL, e.g. http://site.com/node/1. + The protocol is specific; for example if invalidating an http request, + the https equivalent will not be invalidated wildcardpath Invalidates by path, e.g. news/*. wildcardurl Invalidates by URL, e.g. http://site.com/node/*. */ diff -u b/src/Plugin/Purge/Purger/SectionPurgerBase.php b/src/Plugin/Purge/Purger/SectionPurgerBase.php --- b/src/Plugin/Purge/Purger/SectionPurgerBase.php +++ b/src/Plugin/Purge/Purger/SectionPurgerBase.php @@ -73,26 +73,26 @@ } /** - * SendReq($invalidation,$uri,$opt) - * This does all the HTTP dirty work to avoid code repetition. + * SendReq($invalidation,$uri,$opt) - Utilizes HTTP to avoid code repetition. * * @param Invalidation $invalidation + * The invalidation object. * @param string $uri - * the URL of the API endpoint. + * The URL of the API endpoint. * @param array $opt - * request options (ie headers) + * Request options (ie headers) * @param string $exp - * the ban expression. - * - * @return void + * The ban expression. */ - public function sendReq($invalidation, $uri, $opt, $exp) { - // The banExpression is sent as a parameter in the URL, so things like ampersands, asterisks, question marks, etc will break the parse. + public function sendReq(Invalidation $invalidation, $uri, array $opt, $exp) { + // The banExpression is sent as a parameter in the URL, + // so things like ampersands, asterisks, + // question marks, etc will break the parse. $exp = urlencode($exp); // Append the banExpression to the URL. $uri .= $exp; try { - $response = $this->client->request($this->settings->request_method, $uri, $opt); + $this->client->request($this->settings->requestMethod, $uri, $opt); $invalidation->setState(InvalidationInterface::SUCCEEDED); } catch (ConnectException $e) { @@ -111,7 +111,7 @@ ' ', [ 'uri' => $uri, - 'method' => $this->settings->request_method, + 'method' => $this->settings->requestMethod, 'guzzle_opt' => $opt, 'headers' => $headers, ] @@ -132,14 +132,14 @@ * {@inheritdoc} */ public function getCooldownTime() { - return $this->settings->cooldown_time; + return $this->settings->cooldownTime; } /** * {@inheritdoc} */ public function getIdealConditionsLimit() { - return $this->settings->max_requests; + return $this->settings->maxRequests; } /** @@ -151,13 +151,13 @@ * @return string[] * Associative array with header values and field names in the key. */ - protected function getHeaders($token_data) { + protected function getHeaders(array $token_data) { $headers = []; $headers['Content-Type'] = "application/json"; $headers['Accept'] = "application/json"; $headers['user-agent'] = 'Section Purge module for Drupal 8.'; if (strlen($this->settings->body)) { - $headers['content-type'] = $this->settings->body_content_type; + $headers['content-type'] = $this->settings->bodyContentType; } foreach ($this->settings->headers as $header) { // According to https://tools.ietf.org/html/rfc2616#section-4.2, header @@ -192,15 +192,19 @@ * @return mixed[] * Associative array with option/value pairs. */ - protected function getOptions($token_data) { + protected function getOptions(array $token_data) { $opt = [ - 'auth' => [$this->settings->username, \Drupal::service('key.repository')->getKey($this->settings->password)->getKeyValue()], - 'http_errors' => $this->settings->http_errors, - 'connect_timeout' => $this->settings->connect_timeout, + 'auth' => [$this->settings->username, + \Drupal::service('key.repository')->getKey( + $this->settings->password)->getKeyValue(), + ], + 'httpErrors' => $this->settings->httpErrors, + 'connectTimeout' => $this->settings->connectTimeout, 'timeout' => $this->settings->timeout, 'headers' => $this->getHeaders($token_data), ]; - /* the body is unused as everything is url encoded, so, i see no reason to include this, + /* the body is unused as everything is url encoded, + so, i see no reason to include this, especially since the bundled purger does not combine bodies. if (strlen($this->settings->body)) { $opt['body'] = $this->token->replace($this->settings->body, $token_data); @@ -217,19 +221,22 @@ */ public function getTimeHint() { // When runtime measurement is enabled, we just use the base implementation. - if ($this->settings->runtime_measurement) { + if ($this->settings->runtimeMeasurement) { return parent::getTimeHint(); } // Theoretically connection timeouts and general timeouts can add up, so // we add up our assumption of the worst possible time it takes as well. - return $this->settings->connect_timeout + $this->settings->timeout; + return $this->settings->connectTimeout + $this->settings->timeout; } /** * {@inheritdoc} */ public function getTypes() { - return ["url", "wildcardurl", "tag", "everything", "wildcardpath", "regex", "path", "domain", "raw"]; + return [ + "url", "wildcardurl", "tag", "everything", + "wildcardpath", "regex", "path", "domain", "raw", + ]; } /** @@ -241,7 +248,7 @@ * @return string * URL string representation. */ - protected function getUri($token_data) { + protected function getUri(array $token_data) { return sprintf( '%s://%s:%s%sapi/v1/account/%s/application/%s/environment/%s/proxy/%s/state?banExpression=', $this->settings->scheme, @@ -256,7 +263,7 @@ } /** - * + * {@inheritdoc} */ protected function getSiteName() { return $this->settings->sitename; @@ -266,20 +273,22 @@ * {@inheritdoc} */ public function hasRuntimeMeasurement() { - return (bool) $this->settings->runtime_measurement; + return (bool) $this->settings->runtimeMeasurement; } /** + * This will invalidate urls. + * * InvalidateUrls(array $invalidations) - * This will invalidate urls. The protocol is required and - * this must contain the hostname, the protocol, and path (if any) - * The protocol is specific; for example if invalidating an http request, the https equivalent will not be invalidated. - * e.x.: https://example.com/favicon.ico + * The protocol is required and this must contain the hostname, + * the protocol, and path (if any) + * The protocol is specific; for example if invalidating an http request, + * the https equivalent will not be invalidated. + * e.x.: https://example.com/favicon.ico . * * @param array $invalidations - * This takes in an array of Invalidation, processing them all in a loop, generally from the purge queue. - * - * @return void + * This takes in an array of Invalidation, + * processing them all in a loop, generally from the purge queue. */ public function invalidateUrls(array $invalidations) { foreach ($invalidations as $invalidation) { @@ -313,15 +322,16 @@ } /** + * This will invalidate paths. + * * InvalidatePaths(array $invalidations) - * This will invalidate paths. As per the purger module guidelines, + * As per the purger module guidelines, * this should not start with a slash, and should not contain the hostname. - * e.x.: favicon.ico + * e.x.: favicon.ico . * * @param array $invalidations - * This takes in an array of Invalidation, processing them all in a loop, generally from the purge queue. - * - * @return void + * This takes in an array of Invalidation, + * processing them all in a loop, generally from the purge queue. */ public function invalidatePaths(array $invalidations) { foreach ($invalidations as $invalidation) { @@ -338,16 +348,17 @@ '/([[\]{}()+?.,\\^$|#])/', '/\*/', ]; $replace = [ - '' // Escape regex characters except *. // Replace * with .* (for actual Varnish regex) - , '\\\$1', '.*', + '', '\\\$1', '.*', ]; // Base varnish ban expression for paths. $exp = 'req.url ~ "^/'; $exp .= preg_replace($patterns, $replace, $invalidation->getExpression()) . '$"'; - // Adds this at the end if this instance has a site name in the configuration, for multi-site pages. + // Adds this at the end if this instance + // has a site name in the configuration, + // for multi-site pages. // the ampersands are url encoded to be %26%26 in sendReq. if ($this->getSiteName()) { $exp .= ' && req.http.host == "' . $this->getSiteName() . '"'; @@ -358,15 +369,15 @@ } /** - * InvalidateDomain(array $invalidations) * This will invalidate a hostname. - * This should not contain the protocol, simply the hostname - * e.x.: example.com * - * @param array $invalidations - * This takes in an array of Invalidation, processing them all in a loop, generally from the purge queue. + * InvalidateDomain(array $invalidations) + * This should not contain the protocol, simply the hostname. + * e.x.: example.com. * - * @return void + * @param array $invalidations + * This takes in an array of Invalidation, + * processing them all in a loop, generally from the purge queue. */ public function invalidateDomain(array $invalidations) { foreach ($invalidations as $invalidation) { @@ -382,30 +393,39 @@ } /** - * Since by default invalidateURLs() has the ability to handle wildcard urls, this is just an alias. - * This method is still necessary to exist because purge itself has certain validations for each type. + * This will invalidate urls. + * + * Since by default invalidateUrls() has the ability to handle wildcard urls, + * this is just an alias. + * This method is still necessary to exist because purge itself has certain + * validations for each type. */ public function invalidateWildcardUrls(array $invalidations) { $this->invalidateUrls($invalidations); } /** - * Since by default invalidatePaths() has the ability to handle wildcard urls, this is just an alias. - * This method is still necessary to exist because purge itself has certain validations for each type. + * This will invalidate paths. + * + * Since by default invalidatePaths() has the ability to + * handle wildcard urls, this is just an alias. + * This method is still necessary to exist because purge + * itself has certain validations for each type. */ public function invalidateWildcardPaths(array $invalidations) { $this->invalidatePaths($invalidations); } /** - * InvalidateRawExpression(array $invalidations) * This allows for raw varnish ban expressions. - * e.x.: obj.status == "404" && req.url ~ node\/(?).* - would clear the cache of 404'd nodes. * - * @param array $invalidations - * This takes in an array of Invalidation, processing them all in a loop, generally from the purge queue. + * InvalidateRawExpression(array $invalidations) + * E.x.: obj.status == "404" && req.url ~ node\/(?).* - would clear the + * cache of 404'd nodes. * - * @return void + * @param array $invalidations + * This takes in an array of Invalidation, processing them all in a loop, + * generally from the purge queue. */ public function invalidateRawExpression(array $invalidations) { foreach ($invalidations as $invalidation) { @@ -420,14 +440,15 @@ } /** - * InvalidateRegex(array $invalidations) * This allows for a regular expression match of a path. - * e.x.: obj.status == "404" && req.url ~ "node\/(?).*" - would clear the cache of 404'd nodes. * - * @param array $invalidations - * This takes in an array of Invalidation, processing them all in a loop, generally from the purge queue. + * InvalidateRegex(array $invalidations) + * e.x.: obj.status == "404" && req.url ~ "node\/(?).*" + * - would clear the cache of 404'd nodes. * - * @return void + * @param array $invalidations + * This takes in an array of Invalidation, + * processing them all in a loop, generally from the purge queue. */ public function invalidateRegex(array $invalidations) { foreach ($invalidations as $invalidation) { @@ -436,12 +457,15 @@ $uri = $this->getUri($token_data); $opt = $this->getOptions($token_data); $exp = 'req.url ~ ' . $invalidation->getExpression(); - // Adds this at the end if this instance has a site name in the configuration, for multi-site pages. + // Adds this at the end if this instance has a site name in the + // configuration, for multi-site pages. // the ampersands are url encoded to be %26%26 in sendReq. if ($this->getSiteName()) { $exp .= "&& req.http.host == " . $this->getSiteName(); } - $this->logger->debug("[Regex] ban expression `" . $invalidation->getExpression() . "` was replaced to be: `req.url ~ " . $exp . " `"); + $this->logger->debug( + "[Regex] ban expression `" . $invalidation->getExpression() . "` was replaced to be: `req.url ~ " . $exp . " `" + ); $this->sendReq($invalidation, $uri, $opt, $exp); } } only in patch2: unchanged: --- a/config/schema/section_purger.schema.yml +++ b/config/schema/section_purger.schema.yml @@ -46,7 +46,7 @@ section_purger.settings.*: password: type: key_select translatable: false - request_method: + requestMethod: type: string translatable: false scheme: @@ -74,32 +74,32 @@ section_purger.settings.*: body: type: text translatable: false - body_content_type: + bodyContentType: type: string translatable: false # # Performance settings: # - runtime_measurement: + runtimeMeasurement: type: boolean translatable: false timeout: type: float translatable: false - connect_timeout: + connectTimeout: type: float translatable: false - cooldown_time: + cooldownTime: type: float translatable: false - max_requests: + maxRequests: type: integer translatable: false # # Success resolution: # - http_errors: + httpErrors: type: boolean translatable: false only in patch2: unchanged: --- a/section_purger.info.yml +++ b/section_purger.info.yml @@ -3,8 +3,8 @@ type: module description: 'Purger for clearing cache on the Section Platform' package: "Purge - reverse proxies & CDNs" dependencies: - - purge - - purge_tokens - - key + - purge:purge + - purge:purge_tokens + - key:key -core: 8.x \ No newline at end of file +core: 8.x