og_menu scalability

Actually, this is the exact method that organic groups is doing it.. I think it gets called multiple times (even in og_node_access)
The only way to know which groups can be selected is by asking the entity reference selection handler (as there can be completely custom implemented behavior there).
I dont know if there's a better way

> 43.000 is indeed a bit crazy, dont you have a way to limit the entities available for selection, or write a custom selection handler?
If those 43000 entities come with the same number of menus, OG Menu's front end will crash too.

#1477868: OG Scalability says OG is capable of handling +10.000 groups, though.

Right I looked at og_menu_get_group_menus() and that should be fixed as well, maybe in a separate issue, I don't know.

For this issue, I think we would need some upstream changes in OG, but in terms of actual code this should be very easy to do.
If you look into EntityReference_SelectionHandler_Generic::countReferencableEntities() it does:

  public function countReferencableEntities($match = NULL, $match_operator = 'CONTAINS') {
    $query = $this->buildEntityFieldQuery($match, $match_operator);
    return $query
      ->count()
      ->execute();
  }

So there could be a EntityReference_SelectionHandler_Generic::getReferencableEntityIds():

  public function getReferencableEntityIds($match = NULL, $match_operator = 'CONTAINS') {
    $query = $this->buildEntityFieldQuery($match, $match_operator);
    return $query->execute();
  }

I guess we should open an issue titled "Introduce a EntityReference_SelectionHandler_Generic::getReferencableEntityIds()" in OG?!

Edit: Copy-paste failure
Edit2: Again...

I guess we should open an issue titled "Introduce a EntityReference_SelectionHandler_Generic::getReferencableEntityIds()" in OG?!

Actually, this originates in entityreference module., not in OG

In any case, 43000 menus will make the front end crash for sure.. The menu handling would need to be ajaxified

@tstoeckler

JFTR:

• All "EntityReference_SelectionHandler_Generic" issues in the Entity reference issue queue.
• #1673834: EntityReference_SelectionHandler_Generic::getReferencableEntities() falls on it's face with high number of entities.
• #1837650: Allow referencing a specific revision ID

Filed #2059825: Majorly improve the performance of og_menu_get_group_menus() for #2/#4.

Found #2063149: og_menu_node_prepare() does unneccessary heavy-lifting in some cases

I still don't fundamentally grok, why og_menu_node_prepare() does what it does, but this at least avoids the heavy processing in some cases.

Just had hard time debugging an older 2.x site with 1000 menus and a 4 big main menus. node/add page load time was 100 seconds
Eventually I could fix it only because the site doesn't allow manual selection of group audience (prepopulated from url).
So instead of loading all menu trees, I adapted the code to only load it for the groups provided in url(new node) or group audience fields (existing node).

However, this further raises the question if we should replace the menu select with an ajaxified version that only loads what is needed

I think the AJAX version is what the menuperformance module does. Perhaps integrating with that would be a good idea.

I've been looking into this "getReferencableEntities()" part. It causes an entity_load on all items indeed.
But... If I replace it by:

 $result = entityreference_get_selection_handler($field, $instance)->buildEntityFieldQuery()->execute();

to just get the id's, it results in an equal memory footprint but even slower page rendering time. This is because the entities get loaded individually anyway further down the road (e.g.: og_user_access => og_is_group), where those individual loads come from cache in the current codebase.

added note to project page:

Disable menu blocks will completely halt the definition of menu blocks for OG Menu menus. This can improve performance in several places and un-clutter the admin ui.

Thanks - that looks useful

Since this appears to be a meta issue, you all might be interested in #2881709