Problem/Motivation

Some modules need to act before an entity is loaded, and swap out the default revision with a different one.

For example, the Workspace module in core swaps the default revision with a workspace-specific revision, if one exists.

Proposed resolution

Add a new hook: hook_entity_preload() that is called before the entity loading process.

Remaining tasks

Discuss, review.

User interface changes

Nope.

API changes

API addition, the new hook mentioned above.

Data model changes

Nope.

Comments

amateescu created an issue. See original summary.

amateescu’s picture

Status: Active » Needs review
StatusFileSize
new6.35 KB

This should do it.

Status: Needs review » Needs work

The last submitted patch, 2: 2928888.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
StatusFileSize
new7.15 KB
new815 bytes

Easy fix.

amateescu’s picture

I discussed this issue briefly with @Berdir in IRC and he proposed a pre_load hook instead, which would allow modules for example to load a pending revision of an entity before the default revision is loaded by the storage, similar to how the static/persistent caching logic works.

This would mean there won't be any "swapping" of revisions anymore, and we would simply have the pending revision returned along with default revisions in case of a multiple load.

larowlan’s picture

Hi, can you update the issue summary to indicate why this is preferred over the implementing hook checking $entity->isDefaultRevision()?

I assume there is a reason, e.g. performance or cache or similar. Its not clear why a new hook is preferred.

Couple of minor observations - thanks

  1. +++ b/core/lib/Drupal/Core/Entity/EntityStorageBase.php
    @@ -308,6 +309,31 @@ protected function postLoad(array &$entities) {
    +    foreach ($this->moduleHandler()->getImplementations('entity_default_revision_load') as $module) {
    +      $function = $module . '_entity_default_revision_load';
    +      $function($entities, $this->entityTypeId);
    ...
    +    foreach ($this->moduleHandler()->getImplementations($this->entityTypeId . '_default_revision_load') as $module) {
    +      $function = $module . '_' . $this->entityTypeId . '_default_revision_load';
    +      $function($entities);
    

    Any reason not to call ModuleHandlerInterface->invokeAll()?

    Or failing that ModuleHandlerInterface->invoke()

  2. +++ b/core/lib/Drupal/Core/Entity/entity.api.php
    @@ -903,6 +903,46 @@ function hook_ENTITY_TYPE_storage_load(array $entities) {
    + * The results of this hook will not be cached.
    

    Can you elaborate more on this?

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

amateescu’s picture

Title: Add a entity_default_revision_load hook for modules that need to act only when the default revision of an entity is loaded » Add a entity_pre_load() hook for modules that need to load a different revision than the default one
Issue summary: View changes
Issue tags: -Needs issue summary update
StatusFileSize
new10.38 KB

Thanks @larowlan for the review!

Here's a new patch with the approach described in #5, which also fixes the two review points from #6.

No interdiff because it was written from scratch.

Status: Needs review » Needs work

The last submitted patch, 8: 2928888-8.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
StatusFileSize
new18.61 KB
new8.23 KB

Gotta love unit tests..

amateescu’s picture

Title: Add a entity_pre_load() hook for modules that need to load a different revision than the default one » Add a hook_entity_pre_load() for modules that need to load a different revision than the default one
StatusFileSize
new20.27 KB
new1.66 KB

Testing out this patch in combination with #2924218: Clearing the persistent entity cache every time we switch between workspaces is super wasteful and #1596472: Replace hard coded static cache of entities with cache backends revealed that we need special handling for pre-loading in \Drupal\Core\Entity\ContentEntityStorageBase::loadUnchanged, because that method tries to get entities directly from the persistent cache before going through the regular entity load process.

amateescu’s picture

One small problem that I have with this patch is that "pre_load" is inconsistent with other hook names, like "presave" and "predelete". Should we change it to "preload"?

timmillwood’s picture

Status: Needs review » Needs work

@amateescu yes 😉

amateescu’s picture

Title: Add a hook_entity_pre_load() for modules that need to load a different revision than the default one » Add a hook_entity_preload() for modules that need to load a different revision than the default one
Issue summary: View changes
Status: Needs work » Needs review
StatusFileSize
new20.22 KB
new15.4 KB

Yeah, the more I think about it, the more I dislike 'pre_load'. Done :)

timmillwood’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/EntityStorageBase.php
    @@ -287,6 +299,51 @@ public function loadMultiple(array $ids = NULL) {
    +    // Call hook_entity_preload().
    +    $preload_ids = $ids ?: [];
    +
    +    $preload_entities = $this->moduleHandler()->invokeAll('entity_preload', [$preload_ids, $this->entityTypeId]);
    

    Ubernit: move the comment down to precede the actual line calling hook_entity_preload().

  2. +++ b/core/lib/Drupal/Core/Entity/EntityStorageBase.php
    @@ -287,6 +299,51 @@ public function loadMultiple(array $ids = NULL) {
    +      // If we had to load all the entities ($ids was set to NULL), get an array
    +      // of IDs that still need to be loaded.
    

    Is this tested?

    Also, does this mean $storage->loadMultiple() will load all entities without going through hook_entity_preload()?

berdir’s picture

#15.2 The hook is called when you do loadMultiple(), that's exactly what this code is about. Imagine you have 10 nodes, and 2 of them have a workspace association and you load them, then the storage needs to load the other 8 afterwards. Means an additional query for that but we can't do much about that. Don't know about test coverage.

This looks pretty good to me.

Was wondering about the order of arguments for hook_entity_preload(), seems a bit strange to have $entity_type_if afterwards but that's consistent with hook_entity_load().

Also, we might want to profile this. The new hook is called for all entities, not just content entities, so that's quite a few additional hooks being called I think. What I'm wondering is if there's really a use case for this or is it something we should only support on content entities?

amateescu’s picture

StatusFileSize
new16.5 KB

Here's a 8.5.x version of this patch, without the workspace bits, just for manual performance testing purposes.

amateescu’s picture

StatusFileSize
new20 KB
new3.5 KB

Here's a new test-only patch which also contains a module that counts hook invocations for hook_entity_preload(), hook_entity_load() and hook_entity_storage_load().

Testing steps: enable the entity_test_preload module, visit some pages of the site and then run this drush command to see the results:

drush eval "print_r(\Drupal::state()->get('entity_test_preload_stats', []));"
amateescu’s picture

StatusFileSize
new20.57 KB
new3.71 KB

@timmillwood did a bit of profiling using the test module from #18 on a site with ~9000 entities, and the results show that @Berdir was right to question the performance implications of this new hook being fired for all entities.

Patch from #18:

...
    [1] => Array
        (
            [current_path] => /admin/content
            [hook_entity_preload] => 1022
        )
...
    // frontpage
    [4] => Array
        (
            [current_path] => /node/16
            [hook_entity_preload] => 7864
        )
...
    // 404 page
    [5] => Array
        (
            [current_path] => /sites/default/files/pfizer-logo.svg
            [hook_entity_preload] => 6667
        )
...

Patch modified locally to only run for content entities:

...
    [9] => Array
        (
            [current_path] => /admin/content
            [hook_entity_preload] => 144
        )
...
    // frontpage
    [1] => Array
        (
            [current_path] => /node/16
            [hook_entity_preload] => 2552
        )
...
    // 404 page
    [2] => Array
        (
            [current_path] => /sites/default/files/pfizer-logo.svg
            [hook_entity_preload] => 2063
        )
...

So here's an update to #14 which makes this hook fire only for content entities by default.

Status: Needs review » Needs work

The last submitted patch, 19: 2928888-19.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
StatusFileSize
new12.36 KB
new8.21 KB

Some tests change are no longer necessary :)

hchonov’s picture

For example, the Workspace module in core swaps the default revision with a workspace-specific revision, if one exists.

So using the storage load method, which loads an entity in its default revision leads in that case to $storage->load($id)->isDefaultRevision() === FALSE?

Additionally if you do this then it becomes impossible to load the entity in its default revision.

Moreover can this work together with #2620980-109: Add static and persistent caching to ContentEntityStorageBase::loadRevision() .2 :

Couldn't we instead - whenever we put a revision in the revision static cache check if it is the default revision and then populate the entity static cache directly?

?

amateescu’s picture

So using the storage load method, which loads an entity in its default revision leads in that case to $storage->load($id)->isDefaultRevision() === FALSE?

That's the basic architectural foundation of the workspace module, that it gives you a workspace-specific revision (if it exists) instead of the default one, when you are in a non-default workspace.

Additionally if you do this then it becomes impossible to load the entity in its default revision.

That will be possible after #2968452: Add a way to execute a function in the context of a specific workspace, by executing the storage load in the context of the live workspace.

Moreover can this work together with #2620980-109: ContentEntityStorageBase::loadRevision() should use the static and the persistent entity cache like ContentEntityStorageBase::load() .2 :

Couldn't we instead - whenever we put a revision in the revision static cache check if it is the default revision and then populate the entity static cache directly?

?

When you are in a non-default workspace, for almost all purposes, the default revision should always be replaced with the workspace-specific revision, if one exists.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

hchonov’s picture

Ok, then I understand. I am afraid that this might introduce some bugs when used, so it will be more of a feature - use it with caution.

+++ b/core/lib/Drupal/Core/Entity/ContentEntityStorageBase.php
@@ -961,13 +1000,21 @@ public function loadUnchanged($id) {
+    $preloaded_entities = $this->preLoad($ids);

I think it would be useful to pass the information, that now we are loading the unchanged entity. This will be needed when #2620980: Add static and persistent caching to ContentEntityStorageBase::loadRevision() is ready, as it introduces entity revision cache and a loadRevisionUnchanged method. But knowing this I think it would be best to provide this information to the hook implementations here instead of doing it later, so that the adjustment will be as easy as possible.

amateescu’s picture

I think it would be useful to pass the information, that now we are loading the unchanged entity.

Not sure I understand what's the suggestion here :) Anyway, something that I forgot to mention in #23 is that "revision load" queries are not modified at all when you are in a non-default workspace, so $storage->loadRevision(5) will always give you the revision with ID 5 regardless of the currently active workspace.

hchonov’s picture

I've meant that we have to pass the information about loading the unchanged entity to the hook implementations. So in your case probably you'll have to load the revision through loadRevisionUnchanged instead through loadRevision.

amateescu’s picture

So in your case probably you'll have to load the revision through loadRevisionUnchanged instead through loadRevision.

Sorry, I still don't understand what #25 is about..

hchonov’s picture

When we add a static cache for entity revisions to the storage, then loadRevision will be returning the entities from that static cache and not like now always new unchanged objects.

If there is an implemented hook_entity_preload, which is preloading an entity by loading it through loadRevision then we'll always get the entity revision from the static cache.

Later during the saving process we need the unchanged entity and we use loadUnchanged to retrieve it. Now the implemented hook_entity_preload should not return the entity revision from the static cache, but a new unchanged object. If the hook implementation returns the entity revision from the static cache then you'll be unable of detecting changes because e.g. CEB::hasTranslationChanges will be comparing the same entity object with itself. So inside CEB::hasTranslationChanges $this === $original will evaluate to TRUE, but it shouldn't. In order to solve this problem implementations of hook_entity_preload should have the information whether they are being called from load or from loadUnchanged and in the second case load the unchanged entity revision instead the one from the static cache. For this to happen we need to pass one more parameter $unchanged to the hook implementations.

amateescu’s picture

Thanks, that's much more helpful :)

As far as I see, \Drupal\Core\Entity\ContentEntityStorageBase::loadUnchanged() resets the static cache before doing any other operation, so isn't the job of #2620980: Add static and persistent caching to ContentEntityStorageBase::loadRevision() to also reset the static revision cache for the given $entity_id? That way we won't get into the problem described in #29.

amateescu’s picture

Bump! It would be nice to have this performance improvement for Workspaces in 8.7 :)

@hchonov, does my comment in #30 address your concerns about the usage of the (future) static revision cache?

amateescu’s picture

catch’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/ContentEntityStorageBase.php
    @@ -453,6 +453,44 @@ public function finalizePurge(FieldStorageDefinitionInterface $storage_definitio
    +          ->condition($this->entityType->getKey('id'), array_keys($entities), 'NOT IN')
    

    Shouldn't this have an accessCheck(FALSE)?

  2. +++ b/core/lib/Drupal/Core/Entity/EntityStorageBase.php
    @@ -272,6 +272,14 @@ public function loadMultiple(array $ids = NULL) {
    +    // Gather entities from a 'preload' hook. This hook can be used by modules
    

    If preLoad is only in ContentEntityStorageBase shouldn't we override this method to add this logic instead of adding it here? Or if it needs to be here, add a comment?

amateescu’s picture

StatusFileSize
new12.59 KB
new1.69 KB

Thanks for the review!

Re #33:

1. I really really have to learn at some point when to add accessCheck(FALSE) to an entity query :)

2. It needs to be there because we have to add the preloaded entity objects to the static cache, which is only done in the base entity storage class, and it helps custom storages so they don't have to override and rewrite some code that we already have in the base class. Rewrote that comment to document all this.

Status: Needs review » Needs work

The last submitted patch, 34: 2928888-34.patch, failed testing. View results

amateescu’s picture

Status: Needs work » Needs review
StatusFileSize
new12.6 KB
new632 bytes

I forgot to rename the hook implementation during the reroll :/

berdir’s picture

Status: Needs review » Reviewed & tested by the community

Did some profiling as well on a site that is *not* actually using workspaces and despite 108 additional calls to invokeAll(), the time difference was for that case only 0.3%. It also reported almost 5% difference on the IO wait time, which this shouldn't influence in any way, so clearly the variance is much higher than the difference that is reported here. And that doesn't even include the considerable function call overhead of active profiling. So I think the performance impact is not an issue with the current patch.

The only thing I'd like to put up for discussion is whether we really need the entity type specific hook. Are there really enough use cases to make it worth doubling the additional invokeAll() calls? So far all CRUD hooks are always invoked for both variants, but we do have examples of entity hooks where we don't do that, e.g. hook_entity_view_mode_alter(). I don't have a strong opinion myself, it depends on what we consider to be more important I guess: consistency with other entity hooks vs. performance.

Setting to RTBC as that seems like a question for @catch or a framework manager.

The last submitted patch, 21: 2928888-21.patch, failed testing. View results

amateescu’s picture

StatusFileSize
new11.58 KB
new3.92 KB

I think it's a very good idea to drop the entity type specific hook. IMO, performance is more important than consistency here because this hook has quite a narrow purpose that very few modules need.

catch’s picture

+1 on dropping the entity-type specific hook here. I don't have any further objections, but also the tests haven't finished running yet so not committing right now.

  • catch committed 8293016 on 8.7.x
    Issue #2928888 by amateescu, hchonov, timmillwood, Berdir, larowlan: Add...
catch’s picture

Status: Reviewed & tested by the community » Fixed

Committed 8293016 and pushed to 8.7.x. Thanks!

amateescu’s picture

Yay, #2924218: Clearing the persistent entity cache every time we switch between workspaces is super wasteful is now unblocked. Both this issue and that one are a huge performance improvement for workspaces :)

jibran’s picture

Status: Fixed » Needs work
Issue tags: +Needs change record

We added a new hook so we should add a change notice for this.

amateescu’s picture

Issue summary: View changes
Status: Needs work » Fixed
Issue tags: -Needs change record
hchonov’s picture

We just encountered a small problem while working on the patch for the revision cache in #2620980-122: Add static and persistent caching to ContentEntityStorageBase::loadRevision(). Probably someone could take a look and sign-off the change there or we do it in a follow-up of the current issue so that the other patch keeps the main focus on the revision cache?

The problem there is that we would require that the preLoad occurs before the retrieval from the static cache, otherwise under some circumstances the entity in the default revision might already be present in the entity cache and therefore the preLoad hooks will not be executed and thus the entity will not be exchanged in "preLoad" anymore.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.