Interesting module. I've been dealing with some of this as well, mostly implemented via my own cache expiration routines. One of the other things that I need to have happen is to expire content at an upstream CDN, which in my case is Cloudfront. I have been implementing by using a third-party API:
http://www.subchild.com/2010/09/17/amazon-cloudfront-php-invalidator/

Would this be something to consider for inclusion in this module as an optional component? I've been looking for a home for some of this, and right now in my custom module it's not terribly useful to others.

Comments

SqyD’s picture

SqyD CreditAttribution: SqyD commented
Status: Active » Postponed (maintainer needs more info)

Interesting idea. Not sure yet if this project is the right place to do this.
I've taken a quick look at the code you linked to. Not so eager to include a decency on a third party lib that in turn requires a pear module. It should be doable without it but only after some mayor re-factoring I am planning to after an initial 1.0 release. (soon now git is here) Should investigate other cdn's methods too. Didn't the whitehouse project release some similar code for akamai?

rjbrown99’s picture

rjbrown99 CreditAttribution: rjbrown99 commented

Yea I have been struggling with finding the right home for the CDN expiration code. This is mostly for things like images: user changes/updates their content profile with a new imagefield image, and the upstream CDN needs to expire the old one it had cached. Otherwise users could see their old image for up to the normal expire date. My hacked together expiration uses the library linked to above, but it could just as easily adopt the same approach as you are using here - cURL-based communication with Amazon. That part probably wouldn't be too difficult to create and I use PHP+cURL quite extensively elsewhere in my code. Rules integration would also be nice to create a rule "when field has been updated, expire image from the CDN". I'm doing this now with rules, just using custom php code when the rule fires.

There is also a limit to the number of "free" CloudFront invalidation requests in a given month, so it would be worthwhile to track the number you sent to date in an effort to know when you are going to start paying.

Not sure about the Whitehouse and Akamai, that could be.

If you are refactoring for a 1.0 release, it's probably best to hold off until then. The questions would be: should CDN expiration be integrated or perhaps be a sub-module or just a referred project? Perhaps parts of this, such as the cURL interactions, could be tied off into their own shared function called from all of the individual expiration functions? Just thinking out loud.

SqyD’s picture

SqyD CreditAttribution: SqyD commented

We try to maintain a clear separation of functionality between the purge module and the expire module. It would be best to see if we could split this feature between the two

The Cloudfront purge/invalidation method is indeed doable with a more modular request handling. I should be able to add this in the same restructuring that will enable multiple methods, including the nginx method.

The expire image feature should be handled by the expire module. It's currently lacking support fore static files like (profile,filefield) images. Maybe you could post a feature request/patch to the expire module?

The current expire-dev already has rules integration and I will finish up drush integration this week. (Around the same time I finally get to sit down and see what this git hype is all about and release the current purge as a 1.0 ;-)

SqyD’s picture

SqyD CreditAttribution: SqyD commented

I've completed most of the code refactoring needed for this. (now native nginx support in .dev) and might have a go at adding this as a cloudfront module integration feature but I'm still not 100% sure whether to integrate this here or elsewhere...
I've come across the cloudfront, cdn and the akamai modules and have been checking them for overlap or integration possibilities.
All seem to have some logic in them to purge/invalidate objects from the cdn and more and more I am tending towards an expire integration feature in these "native" cdn modules.

I would like to see your code. I am now also co-maintainer of expire and would love to have some of that (profile_)image handling in there. I've created an issue here: #1082860: Support for expiring managed files on update

emmonsaz’s picture

emmonsaz CreditAttribution: emmonsaz commented
Issue summary: View changes

I think purge is a better place to purge/expire/invalidate content than the expire module (which I see as more of a watch-event > trigger-rule service)

Instead of adding a third-party php library (as suggested above), how about looking for a "Proxy URL" Drupal admin setting similar to:

https://cloudfront.amazonaws.com/2014-05-31/distribution/MY_DISTRIBUTION_ID/invalidation?purge_method=cf&auth=MY_AWS_AUTHENTICATION_STRING

which would trigger a curl POST using Amazon's native API:
http://docs.aws.amazon.com/AmazonCloudFront/latest/APIReference/CreateIn...