I installed Features on a large site and got time outs and memory limit errors when going to admin/structure/features/create

My memory limit is set to 256MB and the limit is 30 seconds.

After some hacking around I found that menu_links_features_export_options() was the culprit. If I returned this function before any of it's code ran I could get the page to load immediately.

I found that inside the function

menu_parent_options(menu_get_menus(), array('mlid' => 0));

was returning an array with a count of 2394 items. That's a lot of items to loop through in the following foreach.

I think there are a couple things that could be done here:

  • hook_features_export_options() could have some kind of batching system to handle large results.
  • There could be a UI to determine with components are even available for export.
  • hook_features_api_alter() could be added to allow the removal of menu_links from the components array on a given site.

hook_features_api_alter() seems like the easiest to implement but it's a big API change. Is there another way around this problem?

CommentFileSizeAuthor
#20 1153028-features-component-choice-6.x-1.patch6.77 KBacbramley
#15 1153028-component-choice-13.patch4.54 KBmpotter
#13 1153028-component-choice-12.patch4.54 KBhefox
#3 1153028-features-stop-menu-links.patch924 byteswiifm
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

stevector’s picture

stevector
he/him
CreditAttribution: stevector commented
Version: 7.x-1.0-beta2 » 7.x-1.x-dev

I am still having this problem and it makes the create/recreate page unusable.

wiifm’s picture

wiifm
Primary language English
Location Wellington, NZ
CreditAttribution: wiifm commented
Priority: Normal » Major

Having the same issue on our site, my timeout is set at 120 seconds, and yet it is still taking longer than this. Here is the resulting stacktrace:

Fatal error: Maximum execution time of 120 seconds exceeded in /var/www/mohpub/includes/database/query.inc on line 1950

Call Stack:
    0.0000     644544   1. {main}() /var/www/mohpub/index.php:0
    1.3636   29381216   2. menu_execute_active_handler() /var/www/mohpub/index.php:21
    1.3637   29382728   3. call_user_func_array() /var/www/mohpub/includes/menu.inc:517
    1.3637   29383344   4. drupal_get_form() /var/www/mohpub/includes/menu.inc:517
    1.3637   29384744   5. drupal_build_form() /var/www/mohpub/includes/form.inc:123
    1.3637   29388632   6. drupal_retrieve_form() /var/www/mohpub/includes/form.inc:331
    1.3637   29390864   7. call_user_func_array() /var/www/mohpub/includes/form.inc:787
    1.3637   29391608   8. features_export_form() /var/www/mohpub/includes/form.inc:787
    1.5221   35824368   9. features_invoke() /var/www/mohpub/sites/all/modules/contrib/features/features.admin.inc:93
    1.5221   35825048  10. call_user_func_array() /var/www/mohpub/sites/all/modules/contrib/features/features.module:463
    1.5221   35825536  11. menu_links_features_export_options() /var/www/mohpub/sites/all/modules/contrib/features/features.module:463
  402.1927  233677192  12. menu_link_load() /var/www/mohpub/sites/all/modules/contrib/features/includes/features.menu.inc:143
  402.1943  233707240  13. _menu_link_translate() /var/www/mohpub/includes/menu.inc:2627
  402.1943  233708656  14. _menu_load_objects() /var/www/mohpub/includes/menu.inc:919
  402.1943  233710000  15. node_load() /var/www/mohpub/includes/menu.inc:593
  402.1943  233710904  16. node_load_multiple() /var/www/mohpub/modules/node/node.module:927
  402.1944  233711040  17. entity_load() /var/www/mohpub/modules/node/node.module:908
  402.1944  233711040  18. DrupalDefaultEntityController->load() /var/www/mohpub/includes/common.inc:7585
  402.1971  233750200  19. NodeController->attachLoad() /var/www/mohpub/includes/entity.inc:204
  402.1972  233751304  20. DrupalDefaultEntityController->attachLoad() /var/www/mohpub/modules/node/node.module:3920
  402.1972  233751744  21. field_attach_load() /var/www/mohpub/includes/entity.inc:320
  402.1983  233786672  22. module_invoke() /var/www/mohpub/modules/field/field.attach.inc:704
  402.1984  233793000  23. call_user_func_array() /var/www/mohpub/includes/module.inc:795
  402.1984  233794000  24. field_sql_storage_field_storage_load() /var/www/mohpub/includes/module.inc:795
  402.2533  233872296  25. SelectQuery->execute() /var/www/mohpub/modules/field/modules/field_sql_storage/field_sql_storage.module:346
  402.2533  233872376  26. SelectQuery->getArguments() /var/www/mohpub/includes/database/select.inc:1263
  402.2533  233872376  27. SelectQuery->compile() /var/www/mohpub/includes/database/select.inc:1203
  402.2533  233872376  28. DatabaseCondition->compile() /var/www/mohpub/includes/database/select.inc:1073
  402.2534  233875576  29. DatabaseCondition->mapConditionOperator() /var/www/mohpub/includes/database/query.inc:1834

Is there a cleaner way of disabling the menu_links feature export than simply commenting out parts of this file ?

wiifm’s picture

wiifm
Primary language English
Location Wellington, NZ
CreditAttribution: wiifm commented
FileSize
1153028-features-stop-menu-links.patch924 bytes

Okay, have whipped up a solution that I feels helps everyone here - how about a hidden variable that will remove the menu_links from features?

Attached is a patch that implements this new feature. In order to use it, you can add in your settings.php file:

$conf['features_export_menu_links'] = FALSE;

I can now load the re-create screen for my features now in < 3 seconds (much better than 120 seconds and then timeout)

wiifm’s picture

wiifm
Primary language English
Location Wellington, NZ
CreditAttribution: wiifm commented
Status: Active » Needs review

setting review flag

mpotter’s picture

mpotter CreditAttribution: mpotter commented
Status: Needs review » Needs work

No, that's not the right way to handle this.

Honestly, Menu Links have so many problems that I'm thinking they should be removed from Features entirely and maybe placed into their own module for people who really want them. Menu Links are *content* and Features is not about saving content to code (in my opinion). This is going to be a bigger and bigger issue with D8 with configuration management. I don't want to start an entire flame war about content vs configuration and how Features should handle it.

The Major problem with Menu Links is that Drupal does not use a unique ID for them. So multiple menu links with the same Title conflict and only a single one gets exported with Features. This also causes problems for multilingual menu links. Using UUIDs can help with this, but it's just another case for moving this out of core features.

I'd be interested in other opinions on moving Menu Links into a separate submodule. Then people can just disable the submodule rather than using a kludge config variable as proposed above.

tim.plunkett’s picture

tim.plunkett
he/him
Primary language English
Location Philadelphia
CreditAttribution: tim.plunkett commented

Kill it, kill it with fire.
I'd say separate module, not submodule, if only because it'll give us usage stats.

hefox’s picture

hefox CreditAttribution: hefox commented

+ to fire

stevector’s picture

stevector
he/him
CreditAttribution: stevector commented

++ to taking menu links out of Features core. Use fire as necessary.

drupalninja99’s picture

drupalninja99 CreditAttribution: drupalninja99 commented

++ on taking menu links out - doesn't some other module handle this too? I know I have seen content_menu_links or something like that. So we can probably kill menu links.

drupalninja99’s picture

drupalninja99 CreditAttribution: drupalninja99 commented

I don't know why we need the ajax thing at all. I wish it was server-side bc the ajax thing really bogs everything down

fringedgentian’s picture

fringedgentian CreditAttribution: fringedgentian commented

Thank you, the patch in #3 got the /recreate page appearing again for me when it was all memory timeouts before.

noorLight’s picture

noorLight CreditAttribution: noorLight commented

Thank you, patch #3 has put us back in business of using features again. Awesome work. Looking forward to this patch being implemented into features or some other resolution on how to handle large sites with a gazillion menu links. Just happy to have a fix and actually being able to use features again. Great work!

hefox’s picture

hefox CreditAttribution: hefox commented
Status: Needs work » Needs review
FileSize
1153028-component-choice-12.patch4.54 KB

I was having timeout issues with other components also, so expanded this patch to disable showing components on the feature create form and allow selection what menu to have as options

Needed to add a settings page for features, as currently doesn't have one as far as I can tell

Could move the code that adds the form to a hook that features.menu.inc implements

mpotter’s picture

mpotter CreditAttribution: mpotter commented
Version: 7.x-1.x-dev » 7.x-2.x-dev

See if this requeues the test.

mpotter’s picture

mpotter CreditAttribution: mpotter commented
FileSize
1153028-component-choice-13.patch4.54 KB

Same patch, but adding it again to get the test to re-fire.

Status: Needs review » Needs work

The last submitted patch, 1153028-component-choice-13.patch, failed testing.

hefox’s picture

hefox CreditAttribution: hefox commented
Status: Needs work » Needs review

#15: 1153028-component-choice-13.patch queued for re-testing.

mpotter’s picture

mpotter CreditAttribution: mpotter commented
Status: Needs review » Fixed

Woot, finally passed the testbot. Good patch for handling sad modules that add 100+ variables or permissions. Someday would like to see some code to auto-detect components with more than 100 items and then use Ajax for the component fieldset expansion automatically.

But this is still a useful settings tab, so committed to 8e2a408.

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

acbramley’s picture

acbramley CreditAttribution: acbramley commented
Version: 7.x-2.x-dev » 6.x-1.x-dev
Issue summary: View changes
Status: Closed (fixed) » Needs review
FileSize
1153028-features-component-choice-6.x-1.patch6.77 KB

This needs to be ported back to 6.x-1.x, this significantly sped things up for me when disabling menu links. I'm fairly sure I've got the mapping between the 7.x patch correct, just need clarification as it seems the forms are built quite differently between the 2 versions. This patch includes the menu entry, but if either this or #658772: Provide way to disable rebuild during features_flush_caches() get committed the patch for the other needs altering.

  • mpotter committed 8e2a408 on 8.x-3.x 
    Issue #1153028 by wiifm, hefox, mpotter | stevector: Fixed...
kenorb’s picture

kenorb CreditAttribution: kenorb commented
Version: 6.x-1.x-dev » 7.x-2.x-dev
Status: Needs review » Fixed

Most likely backport to 6.x won't happen, therefore marking as fixed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.