High cURL timeout can lead to Denial of Service

Timeouts on curl are usually related to network issues, not code issues. I am using the module and code right now in development and I do not have any timeout errors.

But do make sure you upgrade Guzzle for the HTTP_PROXY vulnerability that was released yesterday.

duplicated this with Mailchimp VERSION = '1.0.5';
Composer involved, trying to solve now

I find my API matches composer.json

{
  "name": "thinkshout/mailchimp-api-php",
  "version": "1.0.5",
  "type": "library",
  "description": "PHP library for v3 of the MailChimp API",
  "keywords": ["mailchimp", "mail"],
  "homepage": "https://github.com/thinkshout/mailchimp-api-php",
  "require": {
    "php": ">=5.5.0",
    "guzzlehttp/guzzle": "6.2.1"
  }

composer.lock has no mailchimp identified

.. One more thing I am running php 7.0. on Pantheon - Latest versions: Drupal 7.56, Mailchimp 7.x-4.8 using mailchimp subscription form.

(Hi Dan)

bsnodgrass, I am running on the same specs (php 7.0. on Pantheon, Drupal 7.56, Mailchimp 7.x-4.8) and have a similar timeout issue (503 first byte timeout) on the campaigns page. My library version is 1.0.6.

Another note: on Friday, I upgraded to Pantheon's new CDN and the timeout problem seems to have started since then.

I've got the same error a few time since we are using this module, it happens on user registration or when user edit their account

mailchimp
Error
An error occurred requesting list information from MailChimp. "cURL error 28: Operation timed out after 60000 milliseconds with 0 bytes received (see http://curl.haxx.se/libcurl/c/libcurl-errors.html)"

No way to add a catch of the error and put a shorter timeout... like 3 seconds that sounds already a lot, but here it's 60 seconds timeout and before first bytes received, I mean after 3 seconds if first bytes not received, there is most likely an issue with the distant server and our connection is stuck and will not succeed...

+1

+1

The comment in https://cgit.drupalcode.org/mailchimp/tree/mailchimp.module?h=7.x-5.x#n172 is quite ironic. Everyone would consider 60 seconds as being down :)

// Set the timeout to something that won't take down the Drupal site:
  $http_options = [
    'timeout' => 60,
  ];

Is there a specific reason, that we pass such a high value in here? The PHP Library itself has the default of 10 seconds, which would be much more reasonable, if the Mailchimp service is struggling. I would vote for making it configurable via a variable and use a more sensible default.

The Mailchimp Signup forms are usually not critical parts of the website and shouldn't cause such drastic side-effects.

An outage at MailChimp definitely should not cause a Drupal site to break. Need some better error handling.

We have a rudimentary patch here https://www.drupal.org/project/mailchimp/issues/3000365

Not the best solution but it at least it stops the thrown exception from throwing second time.

We had to put our site in maintenance mode in the meantime. Reducing the timeout from 60s to something reasonable sounds right, but preventing the errors from taking down the site sounds like the best approach.

The approach from #12 worked for me, but there are still a few non-critical errors thrown by the mailchimp modules. I've added a few isset checks to the patch attached. Please note, this does not fix the critical issue above, it just supplements it.

An outage at MailChimp definitely should not cause a Drupal site to break. Need some better error handling.

Error handling is one part. A more sensitive timeout is required, too.
Otherwise errors, that are not returned immediately from Mailchimp (like non slow/non responding servers) will still cause huge loading times.

Yeah it would likely make sense to use a local cache of the data from mailchimp and then update that. So Drupal can always fallback to it's own copy of whatever mailchimp last sent. And then the updates to the data happen on cron or something.

That way long timeouts or outages aren't tied to page loads.

I don't know if this will resolve the performance issue, but attached is a separate patch addressing the hardcoded 60 second timeout. This allows for a configuration to override it.

This is less of a priority now that Mailchimp API is working again, can we look to resolve this before it happens again?

We're again experiencing connectivity issues to the Mailchimp service, which drastically slows down the site. Patch #17 solves it with a configurable variable and works fine.

The following patch should do it. Take care, that might need to set the timeout value manually in admin ui, because I didn't wanted to write a update hook, that might mess up the update hook numbers.

$config = \Drupal::configFactory()->getEditable('mailchimp.settings');
$config->set('api_timeout', 60);
$config->save();

Patch for 7.x and 8.x both look good.

Update hook is required for #21 before it can be committed, but the 7.x is RTBC!

Can we look into merging this before there is another Mailchimp API outage?

I'm running into a similar timeout issue but with the Campaigns tab (/admin/config/services/mailchimp/campaigns). Not 100% sure it's related, but I'm guessing it is. So setting back to 'needs work' in case it is.

I manually applied the patch in #17 to Mailchimp 7.x-4.11 (Drupal 7.64), I can now set MailChimp API Timeout. I have 76 campaigns. So I bumped the timeout to 120s and flushed all caches before clicking the Campaigns tab...

Strangely, it timed out after just 60s and the watchdog (which I emptied just before) is filled with 128 pages of Notices (6300!)

Notice : Trying to get property of non-object in mailchimp_campaign_overview_page() (line 234 in /srv/bindings/ad12b2cc.../code/sites/all/modules/mailchimp/modules/mailchimp_campaign/mailchimp_campaign.module).

23 PHP notices,

Notice : Undefined index: 729aa78dfe in MailChimpCampaignController->attachLoad() (line 23 in /srv/bindings/ad12b2cc.../code/sites/all/modules/mailchimp/modules/mailchimp_campaign/includes/mailchimp_campaign.controller.inc).

and 23 mailchimp_campaign errors.

An error occurred while getting campaign 55814xxxxx: 404: Resource Not Found - The requested resource could not be found.

Update - In case it is useful: all this on the dev environment of a Pantheon.io hosted website.

Reporting back after more tests.

I was able to somewhat solve my problem: I'm now able to display the Campaigns tab, although it takes forever every time I load the Campaigns tab. Here are a few interesting notes.

1. The 23 mailchimp_campaign errors (404) seem to indicate that those campaigns no longer exist on Mailchimp (the database is out of sync with the Mailchimp account); so I manually deleted those entries from the mailchimp_campaigns table using MySqlWorkbench.
2. The MailChimp API Timeout settings appear to be overridden by Pantheon. As mentioned above, the timeout is set to 120s but the page goes white after just 60 (The application did not respond in time.).
3. At that point, I simply reloaded the page, and it appears the job continued from where it left off and finally completed after another 40s.
4. But flushing all the caches appears to destroy the work done in the previous steps, and i have to start all over again.

Here's #21 with an update hook.

Re-rolled for latest version of 8.x

Reroll looks as fine as the others.

Patch from #17 didn't completely work for me with version 7.x-5.6. Here's an updated patch.

As we have working patches for Drupal 8 and Drupal 7, I'm changing this issue to target 8.x-1.x-dev to gain attention.

Unplanned or planned outages (https://twitter.com/MailchimpStatus/status/1222489554864787456) can still "take your sites down", when not having the patches applied.

Small suggestion for #29 8.x patch.

+++ b/config/install/mailchimp.settings.yml
@@ -1,4 +1,5 @@
 api_key: ''
+api_timeout: 60
 cron: FALSE

+++ b/mailchimp.install
@@ -50,3 +50,12 @@ function mailchimp_update_8001(&$sandbox) {
+function mailchimp_update_8002(&$sandbox) {
+  $config = \Drupal::configFactory()->getEditable('mailchimp.settings');
+  $config->set('api_timeout', 60);
+  $config->save();
+}

I suggest that the default timeout should be lower from the start.

for any external service like mailchimp 10 seconds should more than enough to answer back.

I suggest that the default timeout should be lower from the start.

I agree with it. You need to see the end user perspective. If the end user needs to wait 60s, because Mailchimp is not responding, he would have already considered the page to be down.

As suggested in #33, I reduced the default timeout to 10 seconds. So we're doing what the Mailchimp PHP library does too.

Also using the 10 seconds in the update hook.

Let's try this.

D8 patch looks good to me, but it looks like @joekrukosky's reroll no longer applies in D7. If we can get that rerolled and reviewed then we can add this to the next release. @aprice42, do you have any time to take a look?

Attached is a re-roll of #31 (still works) against 7.x-5.x