Multiple IP's for one varnish purger

Sorry for not giving this attention, currently I have no access to environment with multiple varnish instances and can not test the code. So If anybody could do a review, by my guest :-)

Hi, I can confirm that the patch #7 is working fine on my setup with 2 Varnish instances.

The only drawback is that as mentioned by @zach.bimson, UI has not been updated so the configuration has to be overridden at the settings.php level, which in my situation is exactly what I need/want.

// Varnish config override to handle multiple Varnish hosts
$config['varnish_purger.settings.08101b3405']['hostname'] = ['varnish-01', 'varnish-02'];
$config['varnish_purger.settings.08101b3405']['port'] = '180';

But this might no be the case for most users.

Cheers,
Jérémy

Hello,

Thanks for the patch. I can confirm it works (on at least still one Varnish).

But when going to the config inspector page I got a fatal error.

InvalidArgumentException: The configuration property hostname.0 doesn't exist. in Drupal\Core\Config\Schema\ArrayElement->get() (line 76 of core/lib/Drupal/Core/Config/Schema/ArrayElement.php).

Drupal\config_inspector\ConfigInspectorManager->checkValue('hostname.0', 'vdg.lxc') (Line: 127)
Drupal\config_inspector\ConfigInspectorManager->checkValue('hostname', Array) (Line: 61)
Drupal\config_inspector\ConfigInspectorManager->checkConfigSchema(Object, 'varnish_purger.settings.varnish', Array) (Line: 134)
Drupal\config_inspector\ConfigInspectorManager->checkValues('varnish_purger.settings.varnish') (Line: 147)
Drupal\config_inspector\Controller\ConfigInspectorController->overview()
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 582)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 124)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 151)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 68)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 99)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 78)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 52)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 665)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

I have adapted the config schema. Here is a new patch.

Hello,

Thanks for this patch, I'm also in the situation where I have 2 Varnish servers and the patch in #10 worked well.

I modified the patch to remain compatible when using a single Varnish, or not overriding conf in settings.php.
Multiple hotsnames can be set in the config form by separating names with a single blank space.

We had some changes in the code because of recent minor changes for coding standards etc - so patch fails now - so it would be great if you have the time for a re-roll?

Hello,

Here is the last patch updated for the 8.x-1.x-dev version.
It has been tested successfully with two Varnish servers, both received purge/ban requests.

You can still use multiple hostnames from the UI :

Patch 14 applies cleanly to latest dev and works as per older version of the patches. Thanks for the reroll!

Note that patch #14 is fixing the VarnishPurger but it breaks the VarnishBundledPurger.

Patch #14 worked for me against 8.x-2.1.

Can this get merged soon?

I just queued a retest for the patch in #14 since it had a failure when it was uploaded. Hopefully the tests pass. 🤞

It appears that #16 has an issue with VarnishBundledPurger and this patch doesn't cover that.

At a minimum this should get reviewed and the tests should pass? It sounds like it needs work though.

If I get some availability in the coming months I'd like to take a peek at this unless somebody gets to it first.

Cheers!

This is still the case for 8.x-2.x, but patch does not apply

Merge request use #14, but re-roll for 2.x

Patch #21 worked for me against 8.x-2.2.

Thanks

Could someone please review this?

Just ran into a need for this again. But #21 won't apply.

Could not apply patch! Skipping. The error was: Cannot apply patch https://git.drupalcode.org/project/varnish_purge/-/merge_requests/11.patch
In Patches.php line 331:
                                                                               
  Cannot apply patch Multiple Varnish Server IPs (https://git.drupalcode.org/p  
  roject/varnish_purge/-/merge_requests/11.patch)!

This is with explicitly setting varnish_purge to version ^2.x-dev in composer.json and using cweagans/composer-patches to apply the patch.

I initially tried with 2.1 and 2.2, not dev.

I also pulled up the patch dialog in PHPStorm and it said it can't apply a hunk.

Don't have time right now to dig in deeper. Hope this helps.

if you use the patch derived from the MR 11 of @goz, it should be fine.

use this URL for the patch :

https://git.drupalcode.org/project/varnish_purge/-/merge_requests/11.patch

I have started testing this patch on the last version 2.2 and it seems to work fine.

Er, I'm 99% sure that's the url I used when I was trying to patch.

That said, long story, but I ended up switching to the http purger built into Purge and was able to make it work. Just added a purger per instance of varnish.

For me, using https://git.drupalcode.org/project/varnish_purge/-/merge_requests/11.patch introduces a bug.
It fails to correctly send ban requests, queue is not decreasing, varnish cache is never cleared. I have the following error in the cron logs :

Drupal\purge\Plugin\Purge\Purger\Exception\BadPluginBehaviorException : Only NOT_SUPPORTED, PROCESSING, SUCCEEDED and FAILED are valid outbound states. dans Drupal\purge\Plugin\Purge\Invalidation\InvalidationBase->setStateContext() (ligne 144 de /var/www/ppd/xxxxxxxxx/web/modules/contrib/purge/src/Plugin/Purge/Invalidation/InvalidationBase.php).

I am using a "Varnish Bundled Purger" with only one host set in my purger config.

If your Varnish servers are load balanced, it should be easier to add a specific rules to your load balancer (filtered on drupal IP + PURGE/BAN method) so it send the request to both all Varnish instances, but... traffic mirroring seems to be as complex :( (ex: https://www.haproxy.com/documentation/haproxy-enterprise/enterprise-modu...)

It seems that a another solution exist with the following haproxy config (not tested myself):

https://github.com/PHI67/http-multicaster

acl public_ips src  X.X.X.X/Y

acl varnish_ban_or_purge capture.req.method -m str -i BAN PURGE

use_backend http-multicaster if varnish_ban_or_purge

backend http-multicaster
    mode http
    server http-multicaster 127.0.0.1:8200

Why not take example on the ZeroConfig Purger?
It does multiple call based on the reverseProxies list:
https://git.drupalcode.org/project/varnish_purge/-/blame/8.x-2.x/src/Plu... => (edit) very bad idea

See also issue https://www.drupal.org/project/varnish_purge/issues/3273684

Fresh patch based on 11MR (but it can be applied to latest 2.3 release). And also safe to use with composer.

Exported MR11 for release 2.2 into patch file.