Problem/Motivation

Every time user_attach_accounts() is called, it prepares a new user entity for user 0. That means it creates a new content entity which involves to create default values for all fields, which in turn creates all field objects down into the property objects. That's very, very expensive.

It is called every time a group of nodes are viewed. So if you have multiple views/blocks/.. that display a list of nodes, then this is called many times.

Based on new relic reports, on cold caches, I'm seeing up to 50 calls to Drupal\Core\Entity\ContentEntityStorageBase::doCreate(), that takes 250ms.

The same applies to comment_prepare_author(), that is a bit more complicated though, as it creates anonymous accounts with specific values for name and homepage (which isn't actually a user field..)

Proposed resolution

We don't need default values for the anymous user. Creating such an object is not quite easy at the moment, a helper method User::getAnonymousUser() is added to simplify that.

user_attach_accounts() is actually not needed at all, instead, we can make sure that the author formatter uses the standardized entity reference formatter pre-loading by extending the author formatter from the entity reference formatter base class.

comment_prepare_author() is additionally merged into Comment::getOwner(), so that that already returns the correct object.

Remaining tasks

User interface changes

API changes

comment_prepare_author() and user_attach_accounts() are removed.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because this is a performance regresssion
Issue priority Major because we loose 5ms * count($nodes)
Prioritized changes Performance
Disruption Two functions are removed that are very likely not called by anyone.

Comments

berdir’s picture

Status: Active » Needs review
StatusFileSize
new942 bytes

Actually, we might not even need it anymore. It used to be called by comment view builder too, but that's gone since #1548204: Remove user signature and move it to contrib.

The description is completely bogus, as is calling setOwner(), but that just calls ->id() again. The only thing that matters is populating the static cache with a single loadMultiple().

Let's see if this works. It might fail if node uid is NULL, then ->entity won't find anything.

berdir’s picture

Issue summary: View changes

Updated issue summary with some numbers.

dawehner’s picture

Based on new relic reports, on cold caches, I'm seeing up to 50 calls to Drupal\Core\Entity\ContentEntityStorageBase::doCreate(), that takes 250ms.

Does that mean a User::load needs 5ms?

berdir’s picture

Ah, comment_prepare_author() is the other related function, and that one is a bit more complicated, because we create an anonymous user with specific values for name and homepage (which is kind of funny, because authenticated users don't have a homepage ;))

wim leers’s picture

Issue summary: View changes
+++ b/core/modules/node/src/NodeViewBuilder.php
@@ -26,8 +27,15 @@ public function buildComponents(array &$build, array $entities, array $displays,
+       User::loadMultiple($uids);

3 instead of 2 spaces.

#1 doesn't yet remove the function.


I wonder why this is actually necessary at all? We're only doing it for Node entities. Why do other entity types don't need this?

berdir’s picture

And uh, we actually killed the pre-loading for that, so we should add it back.

fabianx’s picture

+++ b/core/modules/node/src/NodeViewBuilder.php
@@ -26,8 +27,15 @@ public function buildComponents(array &$build, array $entities, array $displays,
-    // Attach user account.
-    user_attach_accounts($build, $entities);

Should we remove this dangerous helper then?

dawehner’s picture

StatusFileSize
new746 bytes

So user_attach_accounts() also also doing the preloading, why wan't we rewrite in a way that it doens't create the anonymous account unless needed?

fabianx’s picture

Why don't we create the anonymous user account once and save it as before?

Yes you could delete uid 0 in previous versions, but we could just re-create it if its missing:

User::loadAnonymous() --> lazily creates the account and saves it in a locked state

Edit:

Or maybe not even saved to the database, but just created once per request, which takes 5 ms, which might be okay I think.

berdir’s picture

We have a record for uid 0 in the database anyway, but the anonymous user isn't as locked as you might think. See comment_prepare_author(), that's anonymous user with different name every time.

wim leers’s picture

[…] just created once per request, which takes 5 ms, which might be okay I think.

"Just 5 ms per request" -> *gasp*! Are you sure you're @dawehner and not some impersonator? :P

berdir’s picture

StatusFileSize
new7.71 KB

So, this is more or less what I had in mind. User::getAnonymous() isn't too pretty, but it should be fairly fast.

Note that I removed manual preloading in NodeViewBuilder completely and instead made the AuthorFormatter extend from EntityReferenceFormatterBase, which gives us free preloading. We can do the same for comment once that is using a formatter too.

Total wall time varies quite a bit, but on a node with an actual author and two anonymous comments, this saves ~10ms. See uprofiler screenshots.

Also not that this time is with the profiling overhead, and without that, it's smaller than my other numbers, because this is a plain D8 site, while the other one has a bunch of configurable fields on the user.

berdir’s picture

StatusFileSize
new224.75 KB
new199.59 KB

Forgot the screenshots.

Also note that I disabled the render cache, to simulate a cold cache when profiling.

Status: Needs review » Needs work

The last submitted patch, 12: cleanup-author-preloading-2458817-12.patch, failed testing.

dawehner’s picture

I love the idea to simplify stuff and put the logic onto the user itself. Its a special factory of a user object, so putting that onto the user object is a good place.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new7.71 KB
new1.32 KB

Twig, that was mean. getAnonymous() conflicted with the magic method detection, and it used that method instead of isAnonymous(), so I renamed it to createAnonymous().

wim leers’s picture

100% agreed with #15!

wim leers’s picture

Issue tags: +Entity Field API

This patch looks great to me, but this requires somebody with more Entity/Field/Typed Data API knowledge to RTBC.

tim.plunkett’s picture

+++ b/core/modules/comment/src/Entity/Comment.php
@@ -534,7 +535,11 @@ public static function preCreate(EntityStorageInterface $storage, array &$values
+      $entity = User::createAnonymous(array('uid' => 0, 'name' => $this->getAuthorName(), 'homepage' => $this->getHomepage()));

I think the only weird part is having to specify uid => 0 here. Why not put that in createAnonymous?

berdir’s picture

Created https://www.drupal.org/node/2459807

And yes, adding the uid 0 automatically is a good idea.

fabianx’s picture

Assigned: Unassigned » yched

+1 to RTBC, but I feel yched would be good to review this.

fago’s picture

Status: Needs review » Needs work

Great to see clean-ups and performance improvements at the same time like this. Here a review:

  1. +++ b/core/modules/comment/src/Entity/Comment.php
    @@ -534,7 +535,11 @@ public static function preCreate(EntityStorageInterface $storage, array &$values
    +    if (!$entity || $entity->isAnonymous()) {
    

    If the referenced entity is already anonymous, why would we create another one?

  2. +++ b/core/modules/user/src/Entity/User.php
    @@ -430,6 +430,31 @@ public function getChangedTime() {
    +  public static function createAnonymous(array $values = array()) {
    

    Big +1 for having that method, but I don't like the name as it unfortunately collides with the create semantics of entities. Afair the factory patch would try to differentiate between create() and createInstance(), so maybe createAnonymousInstance(). Anyway, the previous getAnonymous() would be way better. Can we fix the twig problem somehow instead?

  3. +++ b/core/modules/user/src/Entity/User.php
    @@ -430,6 +430,31 @@ public function getChangedTime() {
    +    // Values might not map to fields, so we can't reliable pass them into the
    +    // class. Anonymous users always have uid 0.
    

    Shouldn't we instead document $values to map to fields only - what's the point in passing in something else? This would be faster also as it saves us from instantiating field objects if they are not used later on.

berdir’s picture

Thanks for the review.

1. Because of the additional values that we set. Alternatively, we could also clone it and set them I guess.
2. No, we can't change twig. We could only go with a different name to prevent a conflict, like getAnonymousUser().
3. Because that's exactly what comment is doing (homepage is not a field...). Yes, that stuff is supper weird and I'd honestly rather have two different templates for it, or something, but that seems a rather big change..

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new7.83 KB
new1.52 KB

@amateescu liked getAnonymousUser().

We discussed the performance impact a bit, but considering that it is very likely that those values will be accessed anyway (All three calls immediately run it through the username theme function), trying to optimize for that not happening doesn't seems not really worth adding quite a bit more code. Remember that we would also have to key those values by the langcode and so on, so that they match the internal structure.

amateescu’s picture

Status: Needs review » Reviewed & tested by the community

I think this looks very good now :)

andypost’s picture

+++ b/core/modules/comment/comment.module
@@ -632,31 +632,6 @@ function comment_preprocess_block(&$variables) {
-function comment_prepare_author(CommentInterface $comment) {
...
-    $account->enforceIsNew(FALSE);

+++ b/core/modules/comment/src/Entity/Comment.php
@@ -534,7 +535,11 @@ public static function preCreate(EntityStorageInterface $storage, array &$values
   public function getOwner() {
...
+      $entity = User::createAnonymous(array('name' => $this->getAuthorName(), 'homepage' => $this->getHomepage()));

+++ b/core/modules/user/src/Entity/User.php
@@ -430,6 +430,31 @@ public function getChangedTime() {
+  public static function createAnonymous(array $values = array()) {
...
+    $values = array('uid' => 0) + $values;

+1 to removal, the only question about enforceIsNew(FALSE) - it was here for some reason, maybe just not allowing to save the author entity when comment saved

yched’s picture

  1. +++ b/core/modules/comment/src/Entity/Comment.php
    @@ -534,7 +535,11 @@ public static function preCreate(EntityStorageInterface $storage, array &$values
       public function getOwner() {
    -    return $this->get('uid')->entity;
    +    $entity = $this->get('uid')->entity;
    +    if (!$entity || $entity->isAnonymous()) {
    +      $entity = User::getAnonymousUser(array('name' => $this->getAuthorName(), 'homepage' => $this->getHomepage()));
    +    }
    +    return $entity;
       }
    

    Nitpick for clarity : s/$entity/$user ?

  2. +++ b/core/modules/user/src/Plugin/Field/FieldFormatter/AuthorFormatter.php
    @@ -23,7 +23,7 @@
    +class AuthorFormatter extends EntityReferenceFormatterBase {
    

    Makes sense, but then the viewElements() method should use $this->getEntitiestoView(), like other ER formatters ?

  3. +++ b/core/modules/user/src/Entity/User.php
    @@ -430,6 +430,32 @@ public static function getAnonymousUser() {
    +    // Values might not map to fields, so we can't reliable pass them into the
    +    // class. Anonymous users always have uid 0.
    +    $values = array('uid' => 0) + $values;
    +    $entity = new $class(array(), $entity_type->id());
    +    foreach ($values as $name => $value) {
    +      $entity->$name = $value;
    +    }
    

    Not sure I get why "values might not map to fields", what's special about the anon user regarding values and fields ?
    I'm not sure the generic User::getAnonymousUser($values) method should behave differently than User::__construct($values) regarding what happens with $values ?

    If that's because Comment.php uses that to assign an adhoc, non-field 'homepage' runtime property, then arguably that belongs to Comment.php ? Like so :

        if (!$entity || $entity->isAnonymous()) {
          $entity = User::getAnonymousUser(['name' => $this->getAuthorName()]));
        , $entity->homepage = $this->getHomepage();
        }
    

    Pushing it one step further : "make an anonymous User with tailor-made 'name' and 'homepage' properties that come from 3rd party data (here, data stored in the comment)" feels a bit weird, User shouldn't have to cater for special cases like this. As far as user.module is concerned, there is only *one* version of "the anon user", right ?

    Then shouldn't it be :
    - User::getAnonymousUser() : no param, the anon user is always the same, the method creates the object once and statically caches it for subsequent calls
    - Comment::getOwner() does :

    $user = clone User::getAnonymousUser();
    // Assign values stored in the comment.
    $user->name = $this->getAuthorName();
    $user->homepage = $this->getHomepage();
yched’s picture

Status: Reviewed & tested by the community » Needs review

Back to NR for now ?

berdir’s picture

Thanks, great feedback as always ;)

1. Sure ;)
2. Yeah, it should, not sure why I didn't see that.
3. Yeah, comment is weird, not user. user.module actually cares about it in other places too (username preprocess/template has special logic for hostname, despite that only existing for comments, but that's not a reason to add more special stuff). That means we'd completely remove $values then? Makes sense...

berdir’s picture

Status: Needs review » Needs work

The last submitted patch, 30: cleanup-author-preloading-2458817-30.patch, failed testing.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new7.95 KB
new665 bytes

Status: Needs review » Needs work

The last submitted patch, 32: cleanup-author-preloading-2458817-32.patch, failed testing.

dawehner’s picture

Status: Needs work » Reviewed & tested by the community

Looks like a great solution!

berdir’s picture

Issue summary: View changes

Started updating the issue summary, needs a beta evaluation I think.

fabianx’s picture

Issue tags: +API change

RTBC + 1

Prioritized changes are performance for beta evaluation.

yched’s picture

@Berdir: yay, thanks !

berdir’s picture

Title: user_attach_accounts() is very slow » Creating new user entities for anonymous users is very expensive;
Issue summary: View changes
berdir’s picture

Title: Creating new user entities for anonymous users is very expensive; » Creating new user entities for anonymous users is very slow
alexpott’s picture

Status: Reviewed & tested by the community » Fixed

This issue addresses a major bug and is allowed per https://www.drupal.org/core/beta-changes. Committed 70ea0f4 and pushed to 8.0.x. Thanks!

yched’s picture

That comes a little late :-/, but I was thinking : maybe it's not too safe to trust the caller of getAnonymousUser() to clone the object in case it's going to modify it. If he fails to do that, he then pollutes the static object everyone else uses ?

Maybe getAnonymousUser() should always return a clone of its internal the static object ?

fabianx’s picture

#42: Yes, indeed. We should definitely return a clone instead directly.

Follow-up?

  • alexpott committed fcbb3c0 on 8.0.x
    Issue #2458817 by Berdir, dawehner: Creating new user entities for...
alexpott’s picture

Status: Fixed » Needs work

Let's handle #42 in this issue. Reverted.

  • alexpott committed a4f844d on 8.0.x
    Revert "Issue #2458817 by Berdir, dawehner: Creating new user entities...
berdir’s picture

Not sure. I think comment is a special case. we don't clone entities either when we load them, why should we here?

andypost’s picture

Each anonymous comment author could have different name

PS: profiling of the function (node with 273 comments) - with reverted patch 14.74%

berdir’s picture

@andypost: I'm not sure what you are saying.

We already clone the anonymous user object. The only question is if getAnonymousUser() should always clone or if the caller has to decide if he needs to clone or not.

About those numbers, what exactly is 9% and 14%? The function no longer exists with the patch, so it can't be before/after?

andypost’s picture

@Berdir I wanna show that patch shaves like 7% on rendering of ~300 comments
And +1 to #42

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new7.95 KB
new1.1 KB

Moved the clone.

Not fully convinced that this makes sense. On one side, it does, as two anonymous users aren't the same user on the other side, it kind of moves back knowledge about "weird anonymous users with additional values" back into user.module.

I'm fine with either patch.

fabianx’s picture

Status: Needs review » Reviewed & tested by the community

RTBC, It is safer to return a clone for that common shared object.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

I agree with @Fabianx #51 is safer. Thanks @Berdir.

This issue addresses a major bug and is allowed per https://www.drupal.org/core/beta-changes. Committed 3ab0ef9 and pushed to 8.0.x. Thanks!

  • alexpott committed 3ab0ef9 on 8.0.x
    Issue #2458817 by Berdir, dawehner: Creating new user entities for...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.