The usage of the node access grants system was implemented, because the views pager, the search results and the menus were showing nodes, to which users had no access. This brings an non-trivial performance footprint, since node_access_rebuild() is run on every node save via hook_node_presave(), every taxonomy term save via permissions_by_term_submit() and every user creation via hook_user_insert().

Website admins need to be aware, that the process of rebuilding the node access database by node_access_rebuild() can take a relatively long amount of time, when the website has many nodes. Then the max_execution_time in the php.ini file must be set to an appropriate time, to provide enough time to build the node access database.

For performance purpose on large websites (> 10.000 Drupal nodes) it would be good option to have the node access database with PbT optional. Then the direct access to nodes would be prevented by hook_node_access() and menus, search pages and views would be not protected. A configuration entry, that could be set by a PbT config form or the settings.php file should do the job.

Comments

jepSter created an issue. See original summary.

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented

Note: This issue should be an investigation issue in first place. Chances are, that the batch mode in node_access_rebuild() can bring a performance boost. The investigation should happen with ~1.000 nodes to make the effects visible.

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented
Title: Performance improvement: Make the usage of the node access database optional » Performance improvement: Investigate the usage of the node access database as optional and node_access_rebuild() in batch mode
zerolab’s picture

zerolab commented
Priority: Normal » Critical

I think this should be a critical issue.
With 1000 users (U) and 1000 nodes (N), you get 1 million rows to rebuild.

At the very least on user insert, only the grants for that user should be added. That way it is a question of adding N nodes rather than U x N.

Cheers,
Dan

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented

I have improved a few things via the new 8.x-1.15 release. See the release notes:

On module install and an update from a version prior to 8.x-1.13, the node_access_rebuild() function is run in batch mode. There should be now no issue with the max_execution_time php.ini file setting.

node_access_rebuild() is only run on the mentioned module update and install. There is now a lot more granular update process of the database records in drupal's node_access table. The node access records update at node and user update/save are quick now. The node access records update on term creation is also quick. In case of a term update, the process is rather slow. Since all related node access records must be updated. This will change, when the functionality for disabling of the node access records support will implemented.

So I keep this issue opened. At least until the functionality for an optional node access records support will be implemented.

Other performance improvement ideas are welcome.

zerolab’s picture

zerolab commented

Real scenario: running a migration with the module enabled results in out-of-memory errors, even for very large max_memory limits for php. :/

zerolab’s picture

zerolab commented
Category: Feature request » Bug report

As expected, we bumped into this.
With a mere 7000 users, attempting to do anything with a node results in out-of-memory errors even with 1G.

Disabling PbT makes the operation a breeze.

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented

Wow - 7000 users! Then I am optimistic, that you are able to contribute a major performance improvement. ;)

zerolab’s picture

zerolab commented

OK, after a bunch of research, the solution is to simplify the model.

Currently you are creating a separate realm for each user, where is the realm should be per term. So if I create only one PbT term, then there is only 1 realm.
With this, there is no need to touch the node_access table on user insert/update and node insert.

Examples:

Also, we are dealing with 50+K users.

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented
Status: Active » Postponed (maintainer needs more info)

That is a valid suggestion. After some thinking I am quite sure, that we could save a lot of database queries here. We have already the user and role related permissions in the

  • permissions_by_term_role
  • permissions_by_term_user

database tables. From them we could relate to node_access. This would result in a rewrite of the NodeAccess and AccessCheck Service. Probably they could also be merged in this step. I have some new knowledge regarding automated testing, so this process could go with less bugs than in the past of PbT.

However, that's a bit of programming work. Do you have time work with me on this? We could open up two issues and review our patches. To be honest I do not have any project where I am using PbT. No company will pay for my effort. I work on PbT in my spare time. For me PbT is a project for learning Drupal, Symfony and modern PHP programming. My daily job is in Symfony. So it would be really appreciated, if this issue would not wait just for me.

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented
Status: Postponed (maintainer needs more info) » Closed (won't fix)

This issue won't fix. Drupal requires the Node Access records to be by user. Since in the search results hook_node_access() receives only the user account and no node id's. PbT restricts nodes also in search results.

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented
Related issues: +#2901186: Adding/updating new users on site with a lot of nodes gives php timeout

The PHP timeout/memory issue will be fixed via batch processing by work on this issue: https://www.drupal.org/node/2901186

jepster_’s picture

jepster_
Location 🇩🇪Düsseldorf
commented
Status: Closed (won't fix) » Fixed

There has happened a massive performance improvement by adding only 1 node access record per node. See more details at https://www.drupal.org/node/2901186. Improvement has happened in release version 8.x-1.25.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.