Purge framework expects that we set the rate limit on the purger level.
attached a patch that lets each zone set its rate limit. The sensible default depends on the outcome of #2878008: KeyCDN can only handle 128 characters of tags in a delete request. At the moment I'm assuming we can send up to 128 tags at a time for purging, and we know KeyCDN limits us to 20 request per minute. That means the rate limit per purger is (128 X 20 = 2560) / (number of keyCDN purgers on the same account). For now I put that calculation in the description, and set the default at 1000. That should be well over what most sites need.
Comment | File | Size | Author |
---|---|---|---|
#6 | keycdn-rate-limit-5.patch | 1.08 KB | ohthehugemanatee |
#4 | keycdn-rate-limit-4.patch | 1.08 KB | ohthehugemanatee |
Comments
Comment #2
ohthehugemanatee CreditAttribution: ohthehugemanatee at Amazee Labs commentedFix stupid use of settings->get
Comment #3
ohthehugemanatee CreditAttribution: ohthehugemanatee at Amazee Labs commentedforgot to include the schema.
Comment #4
ohthehugemanatee CreditAttribution: ohthehugemanatee at Amazee Labs commentedHeard back from support; it's a 128 character limit on the tags value. Including JSON, that means it can only handle 21 tags per request (see the other issue for the calculation). For a single-purger setup, that means a maximum of 420 tags in a single batch.
Attached an updated patch which counts the number of KeyCDN purgers and sets the max count based on that, rather than allow the user to configure it.
Comment #5
ohthehugemanatee CreditAttribution: ohthehugemanatee at Amazee Labs commentedWaiting on commit until I know for sure - I've sent a followup request to KeyCDN support to confirm my understanding.
Comment #6
ohthehugemanatee CreditAttribution: ohthehugemanatee at Amazee Labs commentedTurns out we can get 32 tags per request in. Adjusting the patch and committing.