Hello,

This is regarding bug in D7.7 installation with the module Internationalization v. 7.x-1.0-rc2

After installing and enabling Internationalization / Taxonomy translation module I started to translate my taxonomy and then I got the error message below.

What is more, forcing the php allowed memory size to 512M or even to 1024M doesn't help too much. I still get this error message even when the higher memory size is enabled.

ERROR MESSAGE:
"Fatal error: Allowed memory size of X bytes exhausted (tried to allocate Y bytes) in /home/***/includes/form.inc on line 3762"

URL THAT TRIGGERS THE ERROR:
http://example.com/#overlay=%3Fq%3Dadmin/structure/taxonomy/mytaxonomy

No other bug-specific messages in the Apache error log.

I don't know if this is D7 Core-specific or module-specific error.
Do you have any idea regarding this one?

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Refineo’s picture

As far as I understand the code this bug happens when executing URL http://****/?q=admin/structure/taxonomy/mytaxonomy
php calls function form_process_weight($element)
in a loop.
This never-ending loop creates an array of thousands of $weights[$n] = $n; entries and finally exchausts php memory.

What function is called when accessing the administrator menu Structure --> Taxonomy --> List Terms ?

Refineo’s picture

It appears I face this memory leakage issue for test taxonomies of > 4000 and > 2000 records.

For a smaller test taxonomy of 1000 records I receive another version of this error message reffering to a file common.inc :
"Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 67701 bytes) in ***/includes/common.inc on line 5670"

Fo a test taxonomy of approx. 750 records I receive another error message reffering to a file theme.inc:
"Fatal error: Allowed memory size of 402653184 bytes exhausted (tried to allocate 309652 bytes) in ****/includes/theme.inc on line 1742"

For smaller taxonomies ( <500 records ) the program (admin/structure/taxonomy/*mytaxonomy*/list) seems to work.

Refineo’s picture

Issue tags: +taxonomy, +terms, +vocabulary, +large, +taxonomy_get_tree
Jose Reyero’s picture

Category: bug » support
Status: Active » Postponed (maintainer needs more info)

> I don't know if this is D7 Core-specific or module-specific error.

Then disable i18n, try again .

If this happens on Structure --> Taxonomy --> List Terms it doesn't look like it is i18n.

Refineo’s picture

Project: Internationalization » Drupal core
Version: 7.x-1.0-rc2 » 7.7
Component: Taxonomy » taxonomy.module
Priority: Major » Normal
Status: Postponed (maintainer needs more info) » Active
Issue tags: -i18n

Jose, you were right, it doesn't seem to be i18n specific error.

I disabled i18n module and the error above on Structure --> Taxonomy --> List Terms is still there for large taxonomies. I reassign this issue to Drupal core, taxonomy.module.

BTW I changed the issue priority to normal as I tested a contrib Taxonomy Manager module and it can handle such large taxonomies but I would still need the standard core list terms functionality.

Do you have any ideas what else to check ? Thank you.

Refineo’s picture

Category: support » bug

It seems to be a bug related to Drupal core, taxonomy.module and handling memory for large taxonomies.
Do you have any simillar experience with D7 ?

sietemonos’s picture

I have the same problem. In may case, I had problems importing the database (allowed-max-packet issue in a shared hosting); the table cache_menu didn't up properly.

Now, I can't edit menu links:
Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 1289048 bytes) in /home/xx/public_html/includes/database/database.inc on line 2108

and i can't access into a page created with de view module:

Fatal error: Allowed memory size of 33554432 bytes exhausted (tried to allocate 317784 bytes) in /home/xx/public_html/includes/bootstrap.inc on line 2680

I used the taxonomy.module on Drupal 7.7

Refineo’s picture

Any hints how to solve this issue ?

orizonmedia’s picture

worked like a charm for us here http://drupal.org/node/1261162

php memory limit set to anything over 128mb for example was not reflected to the server for a reason we actually dont know. We needed to fix the memory limit by another apache setting in .htaccess called «Rlimit». and it worked no more fatal memory error

pfrenssen’s picture

Status: Needs work » Active
Issue tags: -memory +taxonomy, +terms, +vocabulary, +large, +taxonomy_get_tree

This happens on the Taxonomy terms overview page (admin/structure/taxonomy/%taxonomy_vocabulary_machine_name) when showing a vocabulary that has many child terms inside a parent term. I have a vocabulary of 2750 terms spread over 9 parent terms, and over 700MB memory is needed to generate this page:

Page execution time was 59992.86 ms. Memory used at: devel_boot()=6.67 MB, devel_shutdown()=528.43 MB, PHP peak=737.25 MB.

The terms overview page (taxonomy_overview_terms()) will always show at least 2 root elements to make it possible to order them using drag and drop. If they have many children this can result in a very large page. In my case 283 terms were output to the page, and each of these terms have weight select lists ranging from -2750 to 2750. This takes up a large amount of memory.

Here are some profiling results:

Function Calls Wall time (ms) Peak memory usage (bytes)
form_process_weight 283 862,043 335,026,032
form_select_options 283 27,681,435 118,307,128
taxonomy_get_tree 1 53,549 9,322,944
pfrenssen’s picture

Title: Fatal error: Allowed memory size of X bytes exhausted (tried to allocate Y bytes) in /home/***/includes/form.inc on line 3762 » Reduce memory usage of taxonomy terms overview page
Version: 7.7 » 7.x-dev
Issue tags: -taxonomy, -terms, -vocabulary, -large, -taxonomy_get_tree +memory

tags

seanburlington’s picture

Status: Active » Needs work
FileSize
601 bytes

The problem seems to be that complete re-ordering of the vocabulary is allowed on a single page (at least in terms of the weight field)

http://api.drupal.org/api/drupal/modules--taxonomy--taxonomy.admin.inc/f...

Because there might be hundreds or thousands of taxonomy terms that need to be ordered, terms are weighted from 0 to the number of terms in the vocabulary, rather than the standard -10 to 10 scale.

In fact this comment seems wrong - the weight field does not start at zero but -$delta where $delta is something like the size of the vocabulary

I believe that this results in a weight field generated has an options array twice the size of the vocabulary

It seems to be this large array (repeated for each term listed) that is causing memory problems, timeouts - and just very slow pages.

I have made a quick patch which works for me.

I have just hardcoded the maximum delta to 100 - realising that this limits the ability to sort taxonomies - but the current version doesn't work at all for me.

This may not be right for core at the moment - but maybe it will at least be helpful to some people and point others in the right direction ....

seanburlington’s picture

idflood’s picture

subscribing

IcanDivideBy0’s picture

Status: Active » Needs work
Issue tags: -taxonomy, -terms, -vocabulary, -large, -taxonomy_get_tree +memory

subscribing

Inarus’s picture

Hello
Have the same problem. Over 10K taxonomy terms in my vocabulary and get "Allowed memory size of 134217728 bytes exhausted (...) /taxonomy/taxonomy.admin.inc on line 388" :/ .... subscribing

idflood’s picture

#17: you can try the patch in #7 of #941310: FormAPI select/weight fields do not scale. It should help and with some luck it can even be enough.

pfrenssen’s picture

Status: Needs work » Closed (duplicate)

The issue in #941310: FormAPI select/weight fields do not scale is indeed the same as this one, but is more general. Marking this as a duplicate.

pfrenssen’s picture

Issue summary: View changes

Striked out references to i18n, since this is not related to i18n.

sdelbosc’s picture

Issue summary: View changes
Status: Closed (duplicate) » Needs review
FileSize
2.92 KB

I have a taxonomy made of 36184 terms with a single root term. I know this might seem wrong but we plan to add more root terms soon.
My problem is that the terms overview page is already not usable due to its memory usage. It tries to build a page containing everything under the root item.

I wonder if it would be possible to use a configuration variable for the lower level we would like to use to start the pages, cf. suggested patch.

PS: sorry to reopen this ticket but though it is an old one its description matches very well the problem I am facing. Let me know if you would prefer a new ticket for that.

Status: Needs review » Needs work

The last submitted patch, 20: drupal-terms_overview_memory_usage-1241654-20.patch, failed testing.

sdelbosc’s picture

New patch attempt

Jose Reyero’s picture

I think the only option here if we want to keep current features (reset to alphabetical) is getting the terms ordered by the database itself and marking the vocabulary somehow as being 'alphabetical' or not. Also that page cannot rely on taxonomy_get _tree or you'll need an extra parameter for that function.

Anyway it looks pretty complex, api changes etc, so maybe best advice would be to use some contrib module that replaces this page, maybe
https://drupal.org/project/taxonomy_manager

Good luck -me unsubscribing from this one.

sdelbosc’s picture

Thanks for the prompt reply. I understand that there are very few chances that Drupal 7 is updated for this in a near future.
So, as you suggested, I will look for an alternative.

However, this is not the first time that I find the default taxonomy terms management unsuitable to big taxonomies. Anytime we have to deal with more than 3 pages of terms, this screen is not very helpful (when not unusable due to its memory usage).