when i load a page the first time imagechach starts to create the thumbnails one by one.
once the page is done loading some of the thumbnails are not being created (see attached screenshot)
even when i flush and reload the page i will still get the same issue.

any idea?

Files: 
CommentFileSizeAuthor
imagecach_issue.PNG275.4 KBorces

Comments

woodbutcher’s picture

Same problem here with 6.x-2.0-beta9. Some images appear, others don't, a file with a mashup of the original preset filename appears in /tmp, size 0 bytes. Deleting these lock files and reloading the page restores missing images. I found the same issue for 5x-2.0-beta here: http://drupal.org/node/220606 (marked as closed).

woodbutcher’s picture

Very similar problem here too: http://drupal.org/node/459602

orces’s picture

thanks for being on top of this. hope we will get a solution from drewish.

woodbutcher’s picture

It seems to me that this is not a bug in imagecache as such. My problem occurred when generating several images at once -
Set up a gallery with 10 images per page, 8 images work and 2 stick.
Reduce the number of images to 5 per page, all images appear, go to next page all 5 appear.
This is because the GD image toolkit is falling over before the PHP script finishes, leaving the lock files in /tmp. I can't find the crash in the server logs, but imagecache.module relies on a PHP function to clean up the lock files:

touch($lockfile);
// register the shtdown function to clean up lock files. by the time shutdown
// functions are being called the cwd has changed from document root, to
// server root so absolute paths must be used for files in shutdown functions.
register_shutdown_function('file_delete', realpath($lockfile));

register_shutdown_function only runs when the script finishes, and GD toolkit crashes due to memory problems if you ask it to process 10 4Mb images at once.

Installing Imagemagick has solved the problem as it uses memory differently. A bonus is that it reportedly produces better images than GD.

To summarise, either reduce the load on GD by showing fewer images per page, or switch to Imagemagick.

I did have trouble with Imagemagick not being able to find my photos, this was fixed by installing the latest release of ImageApi 6.x-1.x-dev

orces’s picture

thank you for the advise woodbutcher.

I did exactly what you outlined and then flushed my thumbnails. Still the same issue -some thumbs are just not build :(

lsabug’s picture

I'm also using the GD toolkit and I've maybe uploaded a max of 100kb and still no thumbnail from imagecache. In my case, I don't think it is a php memory issue unless imagecache's processing is what's eating it up.

woodbutcher’s picture

If the problem is as described in my post at #4 above, you will find error messages by going to Administer -> Reports -> Recent log entries, something like "ImageCache already generating: sites/default/files/imagecache/NAMESPACE/FILENAME, Lock file: /tmp/NAMESPACEFILENAME".
You will also find "lock files" in your sites /tmp directory with similar names to the thumbnails - deleting them will fix the problem for now, but it will keep coming back.
GD toolkit is crashing PHP - disable it and use Imagemagick instead. This has worked fine for me, if you don't see those error messages in your dblog, it must be some other bug.

lsabug’s picture

I do get these error messages in my log. Strange thing is that on another of my sites using imagecache 5x2.0, imageapi 5x1.1, Drupal 5.7 and, now that I checked again, no transliteration module and thumbnails work as they should.

The site that isn't working is on drupal 5.19 with all the latest modules including transliteration. So after learning this I tried using imagecache without transliteration on my site and lo and behold the thumbs started showing up. And I'm still using the GD toolkit. I will continue to test it since I just installed it about 20 minutes ago.

One thing that was funky about this was that imagecache is still dropping its sample file directly under files rather than in its own directory. In fact, imagecache didn't even bother to create its directory, I did and I had to create the preset folder too. Soooo, not everything is hunky-dory still.

woodbutcher’s picture

Maybe something else is going on here - transliteration doesn't look too memory intensive to me, although if GD is struggling to cope then a small additional load from another module could tip it over the edge? My experience was with Drupal 6.12 and no Transliteration, and I didn't get the directory weirdness. I read somewhere that GD starts a new thread with each image, so if your page asks imagecache for several images at once, PHP can crash, so imagecache's lock files never get deleted. Imagemagick uses resources differently and avoids that problem, so could still be worth a try if you can get it on your server. Good luck.

pharoz’s picture

I'm also getting these lock files error and the thumbnails doesn't show up or only shows up for one image. Here's the details:

Godaddy Hosting
Drupal 6.13
ImageCache 6.x-2.0-beta10
Image-6.x-1.0-beta2

Using GD image kit
PHP 5.2.5
MySQL 5
PHP Memory Limit 96 MB

I tried image magick, but that didn't help either. Also tried deleting lock files, flushing imagecache, and server cache.

pharoz’s picture

Just tried another host and it's the exact same db and setup, but all the image generate pretty quickly without any issues.

I think this issue has to do with the host provider. I was reading somewhere that some host providers limit the execution time of php scripts to like 20s. If you have a lot of images and it eats up a lot of cpu cycle, they will terminate the image manipulation process, thus your image never gets rendered.

The IP I had on Godaddy had like over 3500+ hosts, while the one I currently have from webhostingpad is only 500+. Images generate extremely fast using GD. Good luck!

rcaracaus’s picture

Same Problem! Please Help. My images generate on first time caching after I flush the cache, but then they stop. I can't find any lockfiles, My /tmp is empty. I have tried every solution offered, changing my execution time did nothing. I am not sure what the deal is... such a weird error.

If I delete the preset file and refresh one of my pages then the images (or only a couple if there are alot) get created, but theen none of the other images on any of the other pages get made.

Mathijs Koenraadt’s picture

What we really need, for large sites, is imagecache-queuing / batching.

Don't create all images at once, but process them one by one. This could be done by a cron_hook().

1. Users request a thumbnail not yet generated
2. Show a default image while...
3. ...imagecache stores the image in a queue ...
4. ...until a cron hook creates the images one by one (e.g. every 1-2 minutes)
5. the next user requesting that image will see the generated thumb

On a very large site (a million images), flushing the imagecache is dead-man's joyride.

cancerian7’s picture

Subscribing.

rnjthkmr88’s picture

Subscribing.

jdln’s picture

Subscribing, Ive had to visit every page on my site manually before showing it to a client, and on a few pages refresh to get all the images generated.

unc0nnected’s picture

Yup, all same symptoms here.. I've tried GD and ImageMagick, all perms are correct, tried it with and without custom paths..
Nothing in error logs, flushed cache a few times, disabled image API and re-enabled it, tried putting files into private mode, public mode and back to private, no change throughout.

fizk’s picture

Priority: Critical » Normal
Status: Active » Closed (fixed)

Please reopen if this is still an issue with 6.x-2.0-rc1.

sdsheridan’s picture

Title: fails to create thumbnails sometimes » fails to create presets sometimes
Issue summary: View changes
Status: Closed (fixed) » Active

It's still an issue. I have the latest modules on a client site, and this is still a problem. And it's not just thumbnails, but any preset. What about trying to use the batch API?

Shawn

sdsheridan’s picture

What if in function _imagecache_cache, instead of just testing the existance of the lock file, we also look at its modification time, and if that time plus ini_get('max_execution_time') is earlier than now (i.e., it likely didn't finish generating and thus didn't get cleaned up), we regenerate, like this:

  $lockfile = file_directory_temp() .'/'. $preset['presetname'] . basename($src);
  if (file_exists($lockfile) 
          // PATCH to test if the lockfile was in fact a left-over from a failed generation attempt
          && filemtime($lockfile) + ini_get('max_execution_time') > time()) {
    watchdog('imagecache', 'ImageCache already generating: %dst, Lock file: %tmp.', array('%dst' => $dst, '%tmp' => $lockfile), WATCHDOG_NOTICE);
    // 307 Temporary Redirect, to myself. Lets hope the image is done next time around.
    header('Location: '. request_uri(), TRUE, 307);
    exit;
  }
  touch($lockfile);

Shawn

sdsheridan’s picture

I modified my solution so I could get better messages as to what might be going on, as follows:

  $lockfile = file_directory_temp() .'/'. $preset['presetname'] . basename($src);
  if (file_exists($lockfile)) {
    watchdog('imagecache', 'ImageCache already generating: %dst, Lock file: %tmp.', array('%dst' => $dst, '%tmp' => $lockfile), WATCHDOG_NOTICE);
    // PATCH to test if the lockfile was in fact a left-over from a failed generation attempt
    $max_execution_time = ini_get('max_execution_time');
    $last_modified = filemtime($lockfile);
    if ( $last_modified + $max_execution_time < time() ) {
      watchdog('imagecache', 'Lock file %tmp older (!datetime) than max-execution-time (!mets) ago, likely a left-over - bypassing temporary redirect and regenerating.',
              array(
                '%tmp' => $lockfile,
                '!datetime' => format_date($last_modified),
                '!met' => $max_execution_time,
              ), WATCHDOG_ALERT);
    }
    else {
      // 307 Temporary Redirect, to myself. Lets hope the image is done next time around.
      header('Location: '. request_uri(), TRUE, 307);
      exit;
    }
  }
  touch($lockfile);
vasike’s picture

Version: 6.x-2.0-beta9 » 6.x-2.x-dev
Status: Active » Needs work

i can confirm the solution with code from #21.

@sdsheridan : thank you Sir.
could you provide patch?

vishnu.kumar7’s picture

Hi Shawn,

Will you please more details on how these changes can be done in default imagecache.module file ?. I am unable to find the exact location where i need to append these changes as recommended.

some of files are not getting processed by preset.

Regards
Vishnu Kumar

sdsheridan’s picture

@Vishnu: see post #20 above. I indicated the function there. Just look for the code in that function.