We want to optimize (= speed up) how menu_router_build() writes information into the menu_router table.
This will speed up not only explicitly requested menu rebuilds, but also content type configuration form submits, views configuration submits, and other things.
This issue going into D8 is a requirement for
menu_router vs menu_links
A menu rebuild consists of two parts (in D6 / D7):
- menu_router_build(), dealing with the menu_router table.
- Invoke hook_menu() on all modules. -> not discussed here.
- Update the menu_router table, with the information from hook_menu().
- _menu_navigation_links_rebuild(), dealing with the menu_links table.
-> discussed elsewhere
All of these need optimization, but the second one is dealt with in another issue,
In this issue we only discuss how the data is saved into the menu_router table.
Drupal versions: D6 big speedup, D7/D8 do also benefit
The improvement has the biggest benefit for Drupal 6.
However, also D7 and D8 can benefit.
Diff-based optimization approach
In original D6, menu_router_build will erase the entire menu_router table, and then rewrite it row by row.
In original D7 / D8, menu_router_build will also erase the table, but then rewrite it in a few big queries, each writing a bunch of rows. This is faster than D6, but can still be improved.
The patches (see below) does a big select query, to compare the table contents with the new data and build a diff (rows to insert, rows to update, rows to delete).
In an average case, this diff will be just a few rows.
It will then execute the diff.
The diff data can be made available to other processes, such as _menu_navigation_links_rebuild(). This is to be discussed in another issue.
It was argued that the diff-based approach could have an unacceptable cost in memory, by loading the full menu router with a big SELECT. This would affect both PHP memory and SQL memory.
However: There are already other situations in Drupal where we load the full menu router table, or something as big as it:
- During menu rebuild, we do a module_invoke_all() on hook_menu(). The data generated this way will become the new menu_router, so it has more or less the same size.
- Building the admin menu (contrib) will load something as big as menu_router (both SQL and PHP memory). The same, if we want to show a fully expanded system navigation menu.
- The form for editing the admin menu or system navigation menu. Building this form needs to load a menu with around as many items as the menu_router table.
Benchmarking: Distinguish different parts of menu rebuild.
We need to make sure to benchmark the different parts of menu rebuild separately.
While the above patch does optimize what it's designed for by sth like factor 10, it does have no effects on invocation of hook_menu, or on rebuild of menu_links. So finally the menu rebuild might be around 1/3 faster.
Original text (quite out of date)
As can be seen in various discussions on this site, the function menu_router is a major cause of trouble. See
menu_link_save making multiple (60+) duplicate queries on admin/build/modules
menu_rebuild causes site to lock up
No-op and slow queries during menu rebuild
etc, or try a google search to find more people complain (some of them being duplicates, if I remember right).
Looking at query logs and stack traces, it seems to me that the implementation is far from optimal. For instance, _menu_update_parental_status is called tons of times, and each call causes a SELECT query.
Now, this my naive idea of a decent implementation:
- One big query to load the menu_links and menu_router tables into php arrays.
- Do some computations (without any database queries!!).
- Write the changes to the DB, if any.
- Add some locking mechanics to prevent other requests from doing a menu_rebuild at the same time, or whatever other bad things might happen. But, please don't wait for the locking framework for long operations.
This would save us a lot of queries for this frequent operation.
PASSED: [[SimpleTest]]: [MySQL] 41,753 pass(es). View
PASSED: [[SimpleTest]]: [MySQL] 190 pass(es). View
PASSED: [[SimpleTest]]: [MySQL] 33,774 pass(es). View
PASSED: [[SimpleTest]]: [MySQL] 33,759 pass(es). View
PASSED: [[SimpleTest]]: [MySQL] 32,975 pass(es). View