If you have a freetagging category with thousands of terms the admin/content/node/ page will be very slow or show nothing anymore (WSOD / blank page), even with a memory limit in php.ini over 80MB (!).
This is a result of the taxonomy filter on this admin page.
The (node.module) hack here works: http://drupal.org/node/160090#comment-253244
I think this not only an issue on D5.8 installs.

Comments

ainigma32’s picture

ainigma32 commented
Status: Active » Fixed

Solution provided so setting to fixed.

Feel free to reopen if you think that is wrong.

- Arie

funana’s picture

funana commented

Hi Arie,

is this fixed in node.module? If not, it's still an issue imho.

ainigma32’s picture

ainigma32 commented

I just tried a stock 5.12 install, added the Devel module and generated a category with about 1500 terms. Next I generated about 300 nodes. I also modified the generated category to make it a freetagging category.

I can't seem to reproduce the problem you describe.

Could you check if you are still experiencing this problem using Drupal 5.12?

If the problem still exists I suggest using the Devel module to log the queries and post the slowest queries here. In that case we should set this issue to active and also change it into a bug.

- Arie

funana’s picture

funana commented

Hi Arie,

I'm talking about taxonomies with over 40.000 terms. The server then simply runs out of memory. We have now 8GB RAM and allowed php a usage of 196MB memory, so we don't have the problem anymore.

That does not mean, that this is not a bug.

ainigma32’s picture

ainigma32 commented
Status: Fixed » Postponed (maintainer needs more info)

I never meant to say that this is not a bug.
I'm just trying to reproduce it so it's verifiable and anyone interested can start analyzing without having to bother you for information.

Could you post some more info on your setup? OS, database server, web server, version of Drupal, contributed modules and of course the size of your (other) taxonomies. Are they all around 40.000 terms? And how many taxonomies do you have?
You mentioned having changed the php.ini settings; did you also modify the database server's setup?

FWIW There are more bugs/issues floating around that seem to be related to the scalability of the taxonomy module so you are not the only one.

- Arie

funana’s picture

funana commented

No problem Arie, I didnt mean that you said it would be no bug ;-)

I'll try to give you more details.

Today I checked it once again by using the regular node.module and visited admin/content/node and got a WSOD again.
Error log says:

PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 524288 bytes) in /home/www/doc/11111/site.com/drupal/modules/taxonomy/taxonomy.module on line 1026

I think it always comes up with the "Allowed memory size exhausted error", no matter how much you allow php to use.

No other errors in our log so far btw.

Debian Etch
Apache/1.3.34
MySQL 5.0.37
PHP 5.2.6
Its only one taxonomy with around 40.000 freetagging terms.

What exactly do you mean with modify DB server setup? It's a managed server and they did some optimizations (higher query cache and a few others as far as I remember).

It was Drupal 5.8 but still the same here with the actual 5. release.

I'll send you a module list by mail.

I hope it helps a little bit.

ainigma32’s picture

ainigma32 commented

OK got your list (pretty long one BTW) Next test: setting up a little test environment, doing a stock install of 5.14 and generating or importing the taxonomy of 40.000 terms.
If the node.module doesn't generate the WSOD in that setup, it will probably be one of the modules on your list.

If I find some time I might try it myself but if anyone has a test version ready: feel free to jump in anytime :-)

Also, it would be useful to turn on MySQL's query logging. That way you see what the final query looks like after all the hooks have been invoked.

- Arie

ainigma32’s picture

ainigma32 commented

@funana: I managed to generate a taxonomy of 40.000 terms and although the page is slow it works OK.
Accessing admin/content/node took about 4 seconds and the longest query took 3.5 seconds (!) :

SELECT t.tid, t.*, parent
FROM term_data t INNER JOIN term_hierarchy h ON t.tid = h.tid
WHERE t.vid = 1
ORDER BY weight, name

The amount of memory used - according to the Devel module - is about 50MB.

Can you confirm that this is the last query logged before the out of memory error is logged? Or do you see something else?

- Arie

ainigma32’s picture

ainigma32 commented
Status: Postponed (maintainer needs more info) » Fixed

Looks like funana won't be posting any feedback so I'm setting this to fixed.

Feel free to reopen if you think that is wrong.

- Arie

funana’s picture

funana commented
Status: Fixed » Postponed (maintainer needs more info)

Sorry for the delay...

I can't confirm anything based on logs or Devel, because there is just a WSOD, no logs, no nothing. Maybe I miss something here, would be great if you could tell me where to find this information!

Thank you,

Funana

dpearcefl’s picture

dpearcefl commented
Status: Postponed (maintainer needs more info) » Closed (won't fix)

Considering the time elapsed between now and the last comment plus the fact that D5 is no longer supported, I am closing this ticket.

funana’s picture

funana commented
Version: 5.8 » 6.22
Component: taxonomy.module » node.module
Category: support » feature
Status: Closed (won't fix) » Active

Reopening this old issue, because nothing changed since then. Feel free to close it again, or do whatever with it. Just wanted to document it for who ever may need it.

In Drupal 6.x you will now have to comment out line 152, 153, 154, 155 in node.admin.inc:

  // The taxonomy filter
  // if ($taxonomy = module_invoke('taxonomy', 'form_all', 1)) {
  //  $filters['category'] = array('title' => t('category'), 'options' => $taxonomy);
  // }

to prevent /admin/content/node from loading all of your 3 bazillion taxonomy terms.

Same maybe necessary for advanced search (core) module.

dpearcefl’s picture

dpearcefl commented

Thanks for reopening this issue. if it exists in D6 it certainly should be taken care of.

dddave’s picture

dddave commented
Status: Active » Closed (duplicate)

This seems to be a symptom of: #556842: taxonomy_get_tree() memory issues