Support CDN rewriting of ImageCache paths

I ran into this same problem recently. I have attached the code I used to side-step the issue at the theme layer although I'm not sure this is optimal.

<?php
/**
 * Implementation of theme_imagecache()
 * NOTE: change theme_ to your theme's name and put this in template.php
 */
function theme_imagecache($presetname, $path, $alt = '', $title = '', $attributes = NULL, $getsize = TRUE) {
  // Check is_null() so people can intentionally pass an empty array of
  // to override the defaults completely.
  if (is_null($attributes)) {
    $attributes = array('class' => 'imagecache imagecache-'. $presetname);
  }
  if ($getsize && ($image = image_get_info(imagecache_create_path($presetname, $path)))) {
    $attributes['width'] = $image['width'];
    $attributes['height'] = $image['height'];
  }

  $attributes = drupal_attributes($attributes);

  $path = _imagecache_strip_file_directory($path);
  if (module_exists('transliteration')) {
    $path = transliteration_get($path);
  }

  // check to see if the file exists locally.  If not get it so imagecache will
  // build the derivative image
  if (!file_exists(file_directory_path() .'/imagecache/'. $presetname .'/'. $path)) {
    drupal_http_request(url(file_directory_path() .'/imagecache/'. $presetname .'/'. $path, array('absolute' => TRUE)));
  }

  $args = array('absolute' => TRUE, 'query' => empty($bypass_browser_cache) ? NULL : time());
  if (variable_get('file_downloads', FILE_DOWNLOADS_PUBLIC) == FILE_DOWNLOADS_PUBLIC) {
    $imagecache_url = file_create_url(file_directory_path() .'/imagecache/'. $presetname .'/'. $path);
  }
  else {
    $imagecache_url = url('system/files/imagecache/'. $presetname .'/'. $path, $args);
  }

  return '<img src="'. $imagecache_url .'" alt="'. check_plain($alt) .'" title="'. check_plain($title) .'" '. $attributes .' />';
}
?>

Basically I put the imagecache_create_url() code in theme_imagecache and added a line to check to see if the derivative image had been built yet. If not I requested it without the CDN so the derivative would be built. I'm not sure if others had this problem as well but in my case the CDN wasn't causing the derivative images to be built.

Please wait with posting that to the imagecache issue queue.

The CDN integration module uses the custom_file_url_rewrite function. But the Drupal core patch uses a hook: hook_file_url_alter(). Initially, they agreed with the special function approach, as the same approach is used for rewriting URLs. But now they're shifting all of the URL rewriting functions (including file URL rewriting) to using hooks, to prevent confusing developers with this special case, and because then you can have a module implementing it, instead of having to add some special code.

So, I will first have to update the CDN integration module to use the same approach as D7 core. Issue: #610322: Upgrade the Drupal 6 core patch to match the committed Drupal 7 core patch.

It looks like the preceding slash is unnecessary in $icpath. This:

$icpath = ('/' . file_directory_path() .'/imagecache/'. $presetname .'/'. $path);

works better as this:

$icpath = (file_directory_path() .'/imagecache/'. $presetname .'/'. $path);

Otherwise I get double slashes like: http://example.com//files/imagecache/thumb_small/somefile.jpg

Other than that it appears to solve the problem for me. Thank you very much.

Subscribing.

Thanks for posting your approach, rjbrown99!

I did it in a slightly more elegant manner though, which required less code. On the other hand, I did drop support for SecurePages for now, because it won't be necessary when you're using the CDN integration module in advanced mode: then it can be handled automatically already. See my lengthy follow-up #3 in #568954: HTTPS support for details :)

I've attached the included ImageCache patch. I've also added the necessary documentation of course.

Commit: http://drupal.org/cvs?commit=323422.

Is the fix included with the current download or would we need to perform the patch?

It's not included in the ImageCache module, you'll have to apply the patch. After 1-2 months of real-world use, I'll submit it for inclusion to ImageCache. I first want to make sure that it works flawlessly for everybody.

Can this module put images on a remote cdn but still use existing paths? My users need to link extensively to images and work with direct image paths all the time.

Thanks!

Subscribing...

@Wim Leers - so is this patch a go? should we cross post it with imagecache?

subscribing

subscribe..

Cross posting this to imagecache

+ subscribe

I use it and it works great. Url are rewritten properly.

Hi,

I'm trying to run the latest version of this patch for imagecache.module

patch < imagecache_6--2.patch

and am getting this error:

patching file imagecache.module
Hunk #1 FAILED at 320.
1 out of 1 hunk FAILED -- saving rejects to file imagecache.module.rej

Setup:
- Pressflow 6
- ImageCache 6.x-2.0-beta10
- CDN 6.x-2.0-rc5

Any help very much appreciated!

thanks,
Justin

p.s. I the latest from here too, no joy :
http://drupal.org/node/940096

@jphil:

Use patches/imagecache.patch in the CDN module instead.the imagecache_6--2.patch is intended to be used on the latest of the 6.2 branch, not on the 2.0 beta 10 release.

This is documented, too:

2) Note: skip this step if you want to rely on the fallback mechanism!
   Apply the ImageCache patch (patches/imagecache.patch), if you want to serve
   images generated by the ImageCache module from a CDN. This is a separate
   patch because it uses its own custom function to generate file URLs.
   There are multiple patches for ImageCache:
   - version 2.0, beta 10: patches/imagecache.patch
   - version 2.0, head of branch: patches/imagecache_6--2.patch

@module maintainers: here is the updated patch. Feedback is welcome.

my mistake, thank you :)!!

+1

Hi,

I'm trying to run the latest version of this patch like jphil for imagecache.module

and am getting this error:

$ patch < imagecache.patch
patching file imagecache.module
Hunk #1 FAILED at 318.
1 out of 1 hunk FAILED -- saving rejects to file imagecache.module.rej

Setup:
- Pressflow 6
- ImageCache 6.x-2.0-beta10
- CDN 6.x-2.0

I tried both of the patches. The first like jphils throw his error and this is the result of the other.

Any help very much appreciated!

thanks,
Bronco

+1, would love to see this patch committed.

@broncomania: the patch *does* work for beta 10. Please ensure you're actually using beta 10.

Also, we've now been in this issue for >1 year and there hasn't even been a comment from a maintainer. How are we to proceed, then?

Pinged maintainer, drewish, via e-mail listed on his site.

I'm not a maintainer of this module. Have just committed translations. There has been little development on the 6.x-branch, but I guess this patch would have better chance of getting committed if someone reviewed it and set the status to "reviewed & tested by the community".

Thanks! Marked tested.

Is there a reason that this patch specifically checks for the CDN module? It leaves out anyone who implements a custom hook_file_url_alter(). As far as I can tell, there is no need to check for the CDN module, since there is no downside to having ImageCache call file_create_url even if the hook_file_url_alter patch is not applied.

Here is a patch which basically does the same thing, but without relying on the CDN module.

That patch is against 6.x-2.0-beta10

Patch in #34 works great for me.

Finally had a chance to look at this and I don't really like calling file_create_url() from inside imagecache_create_url() but I'd be interested in calling the hook from imagecache_create_url(). Thoughts?

Update: Sorry about the patch prefix... and I really do want to kill off that transliteration stuff but it probably shouldn't have been included here.

I had the same thought about calling the hook directly based on the code comments, but wasn't sure if file_create_url was being used for a reason. That said, I am not sure the patch in #37 does the same thing as the previous ones for the following reasons:

1. With this patch, hook_file_url_alter is called on the path after the files directory path has been stripped, while in file_create_url it is called with the path including the files directory path. This means if an implementation of hook_file_url_alter relies on that part of the path, it will fail for ImageCache paths.

2. If a hook_file_url_alter changes the path, imagecache_create_url immediately returns the result (mirroring the implementation in file_create_url), however in this case that path is for the source image. It does not add the segments for derivative.

The following patch addresses these issues.

Okay, I made a few changes to take into account that a path should be considered final once hook_file_url_alter has run. I also added some comments. I have this running on a test site now.

Latest patch works for me.

I've had the patch in #39 running in production for two weeks.

Agreed, #39 has been in production on a number of our sites using CDN for over a month now, with no issues. Can we get this committed to -dev?

Marked #809184: Add API to modífy imagecache path prefix as a duplicate.

I committed #574204: Wrong integration with Transliteration so that doesn't need to be part of this patch any longer.

Found one small bug in #39 we setting $oldpath but reading $old_path.

Why aren't we calling the alter on private files? Looking at PressFlow's file_create_url() which backports hook_file_url_alter() they run it on both. They also have the code for stripping slashes off Windows paths that I'd copied into #37.

Here's a rewrite of this based on PressFlow's file_create_url(). I tested it locally in the root of a site using public and private file transfers and it worked correctly. I put it up on one of our sites with CDN on it and it seemed to work fine with public transfers.

Update: I ment to mention that it needs testing in subdirectories and without clean URLs.

Tested #35 a bit, seems to work fine, with the caveat that I also don't have non-clean and sub-directory environments to work with.

obrienmd, did you mean #45?

Yes, #45, my apologies.

@drewish: I was just reading this patch. I am wondering now (with calls to the right custom url hook) if this revised function can be used to create and store derivative image files for a theme in a subdirectory under the theme directory (assuming it had right write permissions)? If so, this appears to help pave the way for serving different size theme images to a mobile device vis-a-vis large computer screen.

Thoughts?

Lars, I think you *could* use the alter to do that but I don't think it's the correct way. A better approach would be to have a mobile theme that uses different presets. Otherwise you'll run into a bunch of issues.

Tested w/ subdirectories and w/o clean URLs on 4 different sites, no problems found.

I was still running into issues with derivative images not being created, but CDN taking over and spitting out the alternative URL (e.g. static.example.com/...) even if specified an absolute URL at the imagecache patch level. Since my CDN is just a VHost symlinked to my main directory, unless the main directory (i.e. www.example.com) is accessed from the browser, the derivative image is never built, and a 404 results. Therefore, the solution I went with was to check to see if the file exists at the patched imagecache level, and if not, manually call imagecache_build_derivative(). The file is then still served from the "CDN", and will generate the missing derivative image on any request where the image doesn't exist (like imagecache normally does without CDN).

If others are sharing this problem, and this is the appropriate article to post to, (the focus is a little varied for different users here) then I'll be glad to share the code.

I went ahead and committed the patch from #45 to 6.x-1.x. We've been using it in production for a few weeks and haven't noticed any issues.

I have the same issue.
drupal 6.20 + Win 2003 + apache 2.2.9 + php 5.2.4
public download
no short url

To work I patched imagecache module to get the file like "index.php?q=files/PATH_TO_IMAGE_CACHE_PRESET...".
Whit this trick my my sites works (image get created) but CDN doesn't do the work (the "?" tell to my CDN to get the image every time it is requested).

@emastyle, if you're using a "?q=" in your request (URL) means that Clean URLs are not enabled (possibly because they aren't supported by your server, and/or you have not turned Clean URLs on. Imagecache requires that you have Clean URLs enabled.

Clean URL are disabled ... at the moment I cannot activate clean url ...
The strange think is that on my locally machine (mac + apache) it works well (clean url disabled).
If I call the image without "?q=" for the first time the imagecache creates the image preset correctly.
Not the same on the remote machine ... (on windows + apache).
I check the htaccess files and they seem correct.
Any other suggestions where I can check?
Bye

Clean URL are disabled ... at the moment I cannot activate clean url ...
The strange think is that on my locally machine (mac + apache) it works well (clean url disabled).
If I call the image without "?q=" for the first time the imagecache creates the image preset correctly.
Not the same on the remote machine ... (on windows + apache).
I check the htaccess files and they seem correct.
Any other suggestions where I can check?
Bye

Finally I enabled short url (activate mod_url_rewrite), updated to imagecache 6.x-2.0-beta12 and now all seems working fine. CDN also works well.
Bye
Emanuele

Drewish and I discovered that image cache presets bust for multi-lingual platforms. On my local server, the imagelinks would include a language prefix. Attached is a patch that resolves this error.

subscribe

Committed to 6.x-2.x.

Does this mean a patch is not required anymore for imagecache to work with CDN module?

Yes, but it means you'll need to download the latest -dev version of the module until a new stable release is created.

Does a patch exist for the new version of Image cache Module 6.x-2.0-beta12 ?

Because I tried to patch it with no success.
I use Cdn in Origin Pull mode with nginx

Footeuz, can you provide more detail on what you mean by not working? Are you using Pressflow or the core patch to Drupal?

Even I tried patching. Seems like the original patch is not applicable to beta12. I read somewhere else that beta12 actually integrates the patch, but then CDN reports that my core files have not been patched or updated, which is not true.

Yeah the CDN module issues for that are #1165076: Applying Inagecache Patch for latest version 6.x-2.0 Beta 12 Results in Error or #1151752: Patch for last dev. version of ImageCache

There's a patch to get it to stop complaining on #1064912: Provide internal setting to stop CDN from verifying the ImageCache patch

I've got the same problem. URLs are being re-written to where the derivative SHOULD exist, but the derivatives are never being written. Care to share your solution?

cbrantley, are you using Pressflow or patched Drupal? Are the images served even though they're not written? Or are they just broken links?

Also grab the -dev release since it has the fix for the language code rewriting.

Drewish: Yes, I'm using Pressflow 6.20 and the dev versions of CDN and Imagecache. I'm also using cloudfront.

With CDN disabled everything works fine. When I enable CDN again the images that imagecache has already processed still work (with the rewritten URL pointing to the CDN) but no new images are processed by imagecache and the result are a bunch of broken (rewritten) links.

The simplest test was to edit one of my imagecache profiles where it generates an un-cached preview. With CDN enabled the preview returns a broken link to where the file SHOULD be on the CDN.

There's no errors anywhere, just broken links. It's as if imagecache never gets told it needs to generate the images, yet the URLS get rewritten.

For now I've just added "*/imagecache/*" to my blacklist so they aren't rewritten.

Thanks for your help!

sub

I'm using the latest dev versions of CDN and ImageCache and am having problems similar to cbrantley in #72. I've also had to add */imagecache/* to my CDN blacklist to ensure that images show up appropriately on my site.

I don't know if this is actually fixed or not in the latest dev releases (so I'm leaving it open for now), but I can confirm that my problem seems to have been caused by some malformed rewrite rules in my htaccess file to prevent hotlinking. Correcting those rules appears to have corrected the CDN/imagecache problem for me.

Since your issue was unrelated to imagecache, and turned out to be an .htaccess issue, I'd suggest closing this issue again.

Done.

I'm having the same issue as in #52 and #72. On the same server with just vhosts serving as CDN, derivative images are not created by ImageCache, although the URL points to the CDN'ed version of where the derivative should have been written.

I'd love to have the code update suggested in #52 and an example of what rewrite conditions were causing #72 to fail.

I've added code in http://drupal.org/node/862880#comment-4888404 which effectively applies the patch suggested by illmasterc in comment #52 to fix the problem of ImageCache not generating the images when using CDN (maybe only when combined with Varnish?)

I think if you have access, you can add a rewrite directive on the vhost / CDN such as

  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d
  RewriteCond %{REQUEST_URI} !=/favicon.ico
  RewriteRule ^(.*)$ DRUPAL PATH/index.php?q=$1 [L,QSA]

This is the same code in the standard drupal htaccess file, but you will need to include whatever relevant path to get to your drupal index.php file.

If that does not work, you could try the solution outlines in this issue: #1250156: Enable Imagecache without Clean Urls and/or with alternate image server setup