Hello ImageField support crew.

Have a website where it's taken me a while to narrow the problem down but I have it now cloned and simplified where only a few modules are running and ImageField seems to be causing excessive memory/cache and cache_set size files.

If I only have Ubercart, CCK enabled I get page load times of: 30.75MB in 1194ms.
If I have Ubercart, CCK and also ImageField enabled I get page load times of: 241.50MB in 11147ms

I have my php.ini settings set to:

upload_max_filesize = 10M ;
post_max_size = 200M ;
memory_limit = 320M ;

The images I try to load in ImageField are 7kb.

The table for cache_form becomes so large I can't even view it in phpmyadmin (error #2008 - MySQL client ran out of memory).

Every additional 7kb image I try to load to the node increases the length of the page load incrementally until by around the 5th image I get http 0 ahah errors. Before the 5th or so image I can still add images if I am patient enough to wait more time with each additional image.

I uninstalled ImageField, and added it again, no change.

I checked all memory settings and folder permissions.

If I disable ImageField then the cache_form is small and the page load times are "normal"

My questions are:

1. Is this a known issue with a possible solution?
2. How can I investigate further to see what the root cause of the problem is?
3. Surely 7kb files shouldn't be causing memory loads of 241Mb instead of 30Mb for the same page and with page load times jumping a factor of 10x from ~1,000 to ~10,000 ms?

I've also checked the mysql query times and the cache_set call is taking over 700ms when ImageField is enabled.

Note: I also optimized my database tables, checked them for errors and I also deleted the contents of cache_set and other cache_ tables to start clean, this hasn't improved the situation.

Thanks for any guidance or assistance you might be able to provide!




datarazor’s picture

Issue summary: View changes

typo fix

quicksketch’s picture

This may be a matter of ImageField simply being the "tipping point" for your server's APC memory limit. By installing one more module (whether it's ImageField or another similarly sized module), you may have surpassed the amount of memory APC is allocated. You should try checking your APC allocation and see if it's 100% full. If it is, increase apc.shm_size in your php.ini or apc.ini file.

However this doesn't explain the cache_form database table problem. I'm not sure what to suggest there. I would add debugging lines to cache_set() or use a debugger to see why cache_set() is being called so frequently with cache_form entries.

datarazor’s picture

Hi @quickstretch, APC is not currently enabled on this website; is there another way that the memory could be taxed? As mentioned my sites has all modules turned off except Ubercart, Imagefield and CCK and the problem still ocures.

The idea of putting some breaks in the cache_set is a good direction to go, hopefully the code will reveal its secrets to me soon.

Thanks again.

datarazor’s picture

So when I look at xhprof all I see is that cache_set is being called once, but the serialized data it puts into it is 100+MB. When I turn off ImageField is goes back down to much more "normal" levels.

When I look into the serialized data, it has a gigantic amount of data in it right after the "default image" text, even though I have no default image selected. I get this huge amount of data even when I go to the node/add form.

It goes from a 10kb cache_form entry to one of 7mb before any images are added.

If it is useful, here is a chunk of the serialized data, right before it goes into unended binary data:

";s:18:"allowed_values_php";s:0:"";s:6:"widget";a:7:{s:13:"default_value";a:1:{i:0;a:1:{s:5:"value";s:1:"0";}}s:17:"default_value_php";N;s:5:"label";s:10:"Sort Order";s:6:"weight";s:1:"1";s:11:"description";s:0:"";s:4:"type";s:20:"optionwidgets_select";s:6:"module";s:13:"optionwidgets";}}s:17:"field_image_cache";a:16:{s:10:"field_name";s:17:"field_image_cache";s:9:"type_name";s:17:"lighting_products";s:16:"display_settings";a:6:{s:6:"weight";s:1:"2";s:6:"parent";s:0:"";s:5:"label";a:1:{s:6:"format";s:6:"hidden";}s:6:"teaser";a:2:{s:6:"format";s:6:"hidden";s:7:"exclude";i:0;}s:4:"full";a:2:{s:6:"format";s:6:"hidden";s:7:"exclude";i:0;}i:4;a:1:{s:6:"format";s:6:"hidden";}}s:13:"widget_active";s:1:"1";s:4:"type";s:9:"filefield";s:8:"required";s:1:"0";s:8:"multiple";s:1:"1";s:10:"db_storage";s:1:"0";s:6:"module";s:9:"filefield";s:6:"active";s:1:"1";s:6:"locked";s:1:"0";s:7:"columns";a:3:{s:3:"fid";a:3:{s:4:"type";s:3:"int";s:8:"not null";b:0;s:5:"views";b:1;}s:4:"list";a:4:{s:4:"type";s:3:"int";s:4:"size";s:4:"tiny";s:8:"not null";b:0;s:5:"views";b:1;}s:4:"data";a:3:{s:4:"type";s:4:"text";s:9:"serialize";b:1;s:5:"views";b:1;}}s:10:"list_field";s:1:"0";s:12:"list_default";i:1;s:17:"description_field";s:1:"0";s:6:"widget";a:262163:{s:15:"file_extensions";s:11:"gif jpg png";s:9:"file_path";s:0:"";s:18:"progress_indicator";s:3:"bar";s:21:

(and many more lines of binary data {6MBs worth...})

I rolled back to imagefield and filefield 3.2 just to see if the extra large serial data would go away, but it is still present. Can anyone decode the above to see if it is "normal", is it loading a default image somewhere weird? I did set just now my imagefield to only accept files of less than 512kb but it still cahces these very large serialized data in the cache_form of the database.

Any guidance or tips are very appreciated as the problem is certainly elusive.

If it is helpful I did recently do some module updates to make the system secure. But as stated above this problem only happens with very few modules active: Ubercart + CCK + Imagefield/filefield active.


datarazor’s picture

I also tried just now creating a 1x1 px default image, setting it in the imagefield settings of the content to see if it would change things, but it doesn't. It lists the defaultimage in the serialized data, but then it still produces gigantic amounts of data beneath it (no change in percievable size).

If I could just get clarity/clarification that the unending data that reads above like is indeed an "image" then at least I know for sure this is the bug/problem:


Is this data for a default image? Is it "normal" to have this data when no image is added at all, and none is specified? I'll have to try a fully-clean drupal install but maybe someone already knows the answer as now I need to switch gears back to another client/project for a short while.

quicksketch’s picture

Ah, we've had this super-big cache problem reported before. I'll see if I can dig it up.

quicksketch’s picture

Here we are, it's been reported in Insert, FileField, and now ImageField with this issue.

#1091648: Performance issues - unusual huge mysql row size, unusual huge memory and cpu usage
#1095878: Widget array contains 260k+ items causing severe performance issues

Still no idea what causes the issue. Seems like other users have stopped reporting it, so perhaps it was an issue with some version of core/CCK/FileField/PHP that has been remedied by more recent releases. I really just don't know where that issue is coming from as I have dozens of D6 sites that aren't exhibiting the same problem.

If I could just get clarity/clarification that the unending data that reads above like is indeed an "image" then at least I know for sure this is the bug/problem:

That's an interesting suggestion. It'd be strange though, considering FileField/ImageField only store the *file ID* in the database, not the actual file contents itself. As you say it also may be unlikely to be an image contents if it's still 6MB after just uploading a 1x1 pixel image.

datarazor’s picture

Thanks quickstretch.

BTW maybe off topic, but when I am debugging PHP in eclipse I can track variables but is there any way to track/trigger on the *size* of a variable instead of a value? It would make finding the point at which it inflates a lot easier considering how many functions there are to step through and debug.

Do you know if the other cases also had ubercart installed? When I read the referred cases it doesn't seem to be the case.

In the meantime I will try and uninstall all the modules and update them with latest and/or dev versions to see if somehow somewhere a bug was resolved or created -- but at this point I still feel like I'm shooting in the dark.

Good to know that there shouldn't be any binary data in the cache_form, seems like some kind of variable mismatch then where something is being written to the serialized cache form when it shouldn't be.

datarazor’s picture

Also would it be useful for you if I simplified the system down to barebasics (garland theme etc.) and provided the code so that you can actually see a reproducible instance of the issue? I only offer this because you mentioned in one of the tickets that you have never been able to see or re-recreate the issue.



quicksketch’s picture


Also would it be useful for you if I simplified the system down to barebasics (garland theme etc.) and provided the code so that you can actually see a reproducible instance of the issue?

Yes! Absolutely. Though I'm afraid this may also have to do with the version of PHP installed. In any case more information is always useful.

datarazor’s picture

Okay great, I have to deal with some other web-items first and then I will see if I can remove all the sensitive information from this website so that it is as bare-bones as possible. May take me a day or two to make the time for it.

I'll also get you the PHP version, anything else you might want to know?



emena’s picture

Désolée, je n'écris pas l"anglais

J'ai le même problème avec image field.
Tout va bien tant que je ne l'utilise pas, mais si je l'utilise, j'ai des problèmes de mémoire.

Il me faut au moins 196M de cache pour PHP, mes pages prennent 30s à être générées et 120M.

datarazor’s picture

Translation of #11 by emena, for non-french speakers:

Sorry, I do not write in english.

I have the same problem with image field.
Everything is fine when I don't use it, but as soon as I turn it on, I have memory problems.

I need at least 196M of cache for PHP, my pages take 30s to generate and use 120M.

emena’s picture

Juste to not forget and still active

quicksketch’s picture

Is this problem still occurring for people? I've now moved off just about all my D6 sites and never experienced this problem. However now that I'm not regularly working in D6, it seems unlikely I'll ever get it.

Apologies @datarazor for the let-down. It's hard to find time to explicitly set up a sandbox just for this project.

quicksketch’s picture

Issue summary: View changes

additional information on database work.