When using the Mailchimp API, we occasionally encounter temporary blocks at the firewall (Akamai).

In some cases, our server’s IP gets blocked for up to 72 hours, which causes the contrib module’s queue processor to repeatedly fail and potentially lose updates.
(Reference: issue #3441822)

To prevent this situation, we implemented a small safeguard in our own integration: we call the Mailchimp /ping endpoint before processing any queue items.

The /ping endpoint is not included in the ThinkShout Mailchimp SDK, so our implementation uses a direct Guzzle request, e.g.:

$client = \Drupal::httpClient();
$response = $client->request('GET', 'https://' . $dc . '.api.mailchimp.com/3.0/ping', [
  'auth' => ['anystring', $api_key],
  'timeout' => 10,
]);

API reference: https://mailchimp.com/developer/marketing/api/ping/

If the ping endpoint is not reachable (timeout, firewall block, 5xx, etc.), we stop the queue processor early so no items are consumed while Mailchimp is unavailable.

Proposed feature
Add an optional “ping before processing” feature inside the contrib module’s queue processor.

Logic

  • Before processing queue items, the processor sends a request to /ping.
  • If the ping returns an error (e.g., blocked IP, DNS failure, timeout): exit the queue processing without consuming items.
  • If the ping succeeds: process queue items normally.

Benefits

  • Prevents data loss when Mailchimp is temporarily unreachable.
  • Avoids repeatedly consuming and re-queuing items during outages.
  • Protects against edge cases such as IP-based rate limiting and CDN firewall blocks.
  • Keeps the queue stable without requiring custom patches or overriding the queue processor.

Additional notes
Our implementation is lightweight and cached (to avoid hitting /ping too often).

Issue fork mailchimp-3558897

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

kiwad created an issue. See original summary.

xenophyle’s picture

Would you mind creating an MR or patch to use as a starting point?

kiwad’s picture

We did most of this in a custom module except for the ClientFactory.php

  $http_client_config = Settings::get('http_client_config', []);
    if (isset($http_client_config['proxy'])) {
      $http_options['proxy'] = $http_client_config['proxy'];
   }

We patched this to be able to have a custom proxy specific for mailchimp without having to make it sitewide.

In a custom module, we did something like this :

// mymodule.module

function mymodule_module_implements_alter(array &$implementations, $hook) {
  if ($hook === 'cron' && isset($implementations['mailchimp'])) {
    unset($implementations['mailchimp']);
  }
}

function mymodule_cron() {
  $config = \Drupal::config('mailchimp.settings');
  if (!$config->get('cron')) {
    return;
  }
  \Drupal::service('mymodule.queue_processor')
    ->process((int) ($config->get('cron_limit') ?: 50));
}

I've tried to retrieve the specific code for this from our custom module and make it generic

// src/Queue/Processor.php

namespace Drupal\mymodule\Queue;

use Drupal\Core\Config\ConfigFactoryInterface;
use Drupal\Core\Queue\QueueFactory;
use Drupal\Core\State\StateInterface;
use Drupal\mailchimp\Queue\Processor as ContribProcessor;
use GuzzleHttp\ClientInterface;
use GuzzleHttp\Exception\TransferException;

class Processor extends ContribProcessor {

  public function __construct(
    ConfigFactoryInterface $config_factory,
    QueueFactory $queue_factory,
    protected ClientInterface $httpClient,
    protected StateInterface $state,
  ) {
    parent::__construct($config_factory, $queue_factory);
  }

  public function process(?int $batch_limit = NULL) {
    if (!$this->ping()) {
      \Drupal::logger('mymodule')->warning(
        'Mailchimp /ping failed. Queue processing skipped to prevent data loss.'
      );
      return 0;
    }
    return parent::process($batch_limit);
  }

  /**
   * Checks Mailchimp reachability via /ping. Result cached in State for 60s.
   */
  protected function ping(): bool {
    $now = \Drupal::time()->getRequestTime();

    if ($now - (int) $this->state->get('mymodule.ping_time', 0) < 60) {
      return (bool) $this->state->get('mymodule.ping_ok', TRUE);
    }

    $api_key  = (string) $this->configFactory->get('mailchimp.settings')->get('api_key');
    $dash_pos = strrpos($api_key, '-');
    if (!$api_key || $dash_pos === FALSE) {
      return FALSE;
    }

    try {
      $response = $this->httpClient->request('GET',
        sprintf('https://%s.api.mailchimp.com/3.0/ping', substr($api_key, $dash_pos + 1)),
        ['auth' => ['anystring', $api_key], 'timeout' => 10]
      );
      $ok = $response->getStatusCode() === 200;
    }
    catch (TransferException $e) {
      $ok = FALSE;
    }

    $this->state->set('mymodule.ping_time', $now);
    $this->state->set('mymodule.ping_ok', $ok);
    return $ok;
  }

}
# mymodule.services.yml
services:
  mymodule.queue_processor:
    class: Drupal\mymodule\Queue\Processor
    arguments: ['@config.factory', '@queue', '@http_client', '@state']

This pattern would be a clean addition directly to the contrib Processor:

  • A ping_before_process boolean in mailchimp.settings (opt-in, default false for backward compatibility)
  • A ping_cache_ttl integer setting (default 60 seconds)
  • The /ping check at the top of Processor::process(), skipping queue processing if unreachable

  • xenophyle committed 808885ed on 3.x
    feat: #3558897 Add ping in queue processor
    
    By: kiwad
    By: xenophyle
    

  • xenophyle committed d5511a51 on 3.2.x
    feat: #3558897 Add ping in queue processor
    
    By: kiwad
    By: xenophyle
    
    #...

  • xenophyle committed b33441d7 on 2.x
    feat: #3558897 Add ping in queue processor
    
    By: kiwad
    By: xenophyle
    
    #...
xenophyle’s picture

Status: Active » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

chrisla’s picture

Just a heads-up that this caused my cron jobs to fail when updating as I didn't grab the updated Mailchimp library when updating the module. It's perhaps worth it to flag this in the release notes