This week there was a problem reaching fastly (Jun 28, 2017).

When fastly is hard to reach, or not reachable at all, this results in an (sometimes extremely) degraded performance hit for sites when trying to purge keys, especially when the site is very busy and a lot of invalidations are happening. Because the request() call is waiting for a response, the webserver isn't unable to handle other requests and everything simply starts piling up very fast until the very moment the site can become completely response for authenticated users.

The proposal would be to use requestAsync() from the http client because the rational is simple:

- if the fastly is up, then we still want the client to have a quick response without it having to know that we are invalidating stuff on the server for anonymous users
- if fastly is down, then the authenticated user should still be able to perform requests, invalidations within drupal will still happen (e.g. render cache etc). Since fastly can't be reached during these times, anonymous pages won't yet be invalidated, but we can do this either manually ourselves later, or the page which couldn't be invalidated yet, will be on a later time.

Using async means we won't be able to detect whether the response will be ok or not, but that's ok if development testing has shown that the initial setup with keys went ok.

Comments

swentel created an issue. See original summary.

swentel’s picture

Title: Use requestAsync for purging » Use requestAsync/postAsync for purging
Status: Active » Needs review
swentel’s picture

Status: Needs review » Active

Woops, no patch yet, sorry

swentel’s picture

So it turns out the requestAsync doesn't fully promise on actually doing a call, unless you call wait .. oh the promise of a name .
Checking if there are other ways.

mfb’s picture

I would have thought guzzle requestAsync was analogous to the Drupal 7 version's use of httprl module. It doesn't guarantee the response went thru per se but does fire off a request.

swentel’s picture

Issue summary: View changes

I though so too, but I did some testing and it seems like it doesn't .. unless I'm doing something completely wrong :)

A different strategy might be playing with 'connect_timeout' because this defaults to 0 (which is indefinitely) ? That way, we can still catch the response and catch actual errors e.g. 'could not connect to fastly' etc ..

(changed some spelling mistakes in the is)

gg4’s picture

Adding a connect_timeout has been on my todo list. We should do this regardless of whether we can find an async solution.

gg4’s picture

@swentel Any preference on your side whether the timeout values comes from a service parameter or class constant?

swentel’s picture

@bonus oh .. no big preference. I guess a service parameter might be a bit more flexible, although I wouldn't know how much I'd play with that setting :)

gg4’s picture

@swentel Also, any preference on what you think would be a sane value?

swentel’s picture

@bonus no preference to be honest, I guess 3 seconds should be more than enough no ?