Problem/Motivation

The Cookie authentication provider currently executes SQL queries in order to construct an authenticated UserSession.

This is different from the BasicAuth authentication provider which simply loads the user from entity store.

It is quite common to actually have to load the user entity anyway, this could simplify some code (keep AccountInterface, but if authenticated user, then you can assume that you have a user entity).

Steps to reproduce

Proposed resolution

Simplify Cookie:authenticate() and load the user entity directly.

In order to prevent triggering the field translation subsystem prematurely, introduce ContentEntityBase::getFieldValue() which returns an untranslated value and use that for simple user entity properties (User::getLastAccessedTime(), User::isActive(), User::getTimezone(), User::getAccountName()).

Remaining tasks

  • Review
  • Commit

User interface changes

None.

API changes

API addition: ContentEntityBase::getFieldValue()

Data model changes

None.

Release notes snippet

CommentFileSizeAuthor
#170 Screenshot_20260326_224708.png139.35 KBberdir
#152 2345611-152.patch11.49 KB_utsavsharma
#152 interdiff_d11.txt11.49 KB_utsavsharma
#145 2345611-145.patch11.47 KBeugene.brit
#143 interdiff_141_143.txt581 bytesameymudras
#143 2345611-143.patch11.47 KBameymudras
#141 interdiff_140-141.txt949 bytesmrinalini9
#141 2345611-141.patch11.39 KBmrinalini9
#140 2345611-140.patch11.4 KBmrinalini9
#138 reroll_diff_137-138.txt4.96 KBmedha kumari
#138 2345611-138.patch11.46 KBmedha kumari
#137 2345611-137.patch11.72 KBeugene.brit
#132 interdiff_128-131.txt591 bytesravi.shankar
#132 2345611-131.patch11.55 KBravi.shankar
#130 interdiff-2345611-128_129.txt588 bytesanchal_gupta
#130 2345611-129.patch11.54 KBanchal_gupta
#128 interdiff_124-128.txt3.23 KBravi.shankar
#128 2345611-128.patch11.54 KBravi.shankar
#124 reroll_diff_121-124.txt6.7 KBravi.shankar
#124 2345611-124.patch8.31 KBravi.shankar
#121 user-session-load-2345611-121.patch8.56 KBjanlaureys
#120 user-session-load-2345611-120.patch8.23 KBjanlaureys
#119 user-session-load-2345611-119.patch6.8 KBjanlaureys
#117 user-session-load-2345611-117.patch11.8 KBjanlaureys
#112 2345611-112.patch11.8 KBaleevas
#107 2345611-107.patch11.77 KBrichgerdes
#107 2345611-99-107-interdiff.txt1.48 KBrichgerdes
#99 2345611-99-interdiff.txt685 bytesrichgerdes
#99 2345611-99.patch11.83 KBrichgerdes
#98 2345611-98-interdiff.txt684 bytesrichgerdes
#98 2345611-98.patch11.82 KBrichgerdes
#95 2345611-95-interdiff.txt2.79 KBberdir
#95 2345611-95.patch11.72 KBberdir
#92 2345611-92.diff14.45 KBrichgerdes
#88 current-user-load-entity-2345611-88-interdiff.txt962 bytesberdir
#88 current-user-load-entity-2345611-88.patch14.24 KBberdir
#83 current-user-load-entity-2345611-83-review-do-not-test.patch8.62 KBberdir
#83 current-user-load-entity-2345611-83-interdiff.txt5.56 KBberdir
#83 current-user-load-entity-2345611-83.patch13.34 KBberdir
#80 current-user-load-entity-2345611-80.patch13.31 KBberdir
#79 current-user-load-entity-2345611-79-interdiff.txt2.44 KBberdir
#79 current-user-load-entity-2345611-79.patch9.87 KBberdir
#77 current-user-load-entity-2345611-77-interdiff.txt2.2 KBberdir
#77 current-user-load-entity-2345611-77.patch9.94 KBberdir
#70 current-user-load-entity-2345611-70.patch7.98 KBandypost
#70 interdiff.txt1.03 KBandypost
#70 current-user-load-entity-2345611-70-nopart.patch7.15 KBandypost
#68 current-user-load-entity-2345611-68.patch7.98 KBandypost
#67 xhprof-kit-no-regression-in-user-get-timezone.txt4.62 KBznerol
#66 get-user-from-session.png178.69 KBznerol
#66 entity-manager-get-storage.png169.5 KBznerol
#66 on-request-authenticate-no-regression-in-account-proxy.png168.5 KBznerol
#61 ab-head-node-one.txt1.26 KBznerol
#61 ab-patch-node-one.txt1.26 KBznerol
#61 hi.tar_.gz591 bytesznerol
#60 ab-patch-test.txt1.25 KBznerol
#60 ab-head-test.txt1.25 KBznerol
#58 current-user-load-entity-2345611-58.patch7.73 KBandypost
#58 interdiff.txt650 bytesandypost
#55 current-user-load-entity-2345611-55-interdiff.txt2.21 KBberdir
#55 current-user-load-entity-2345611-55.patch7.66 KBberdir
#51 current-user-load-entity-2345611-57-interdiff.txt1.71 KBberdir
#51 current-user-load-entity-2345611-57.patch6.34 KBberdir
#47 current-user-load-entity-2345611-47.patch4.63 KBberdir
#41 user-session-load-2345611-41.patch27.98 KBberdir
#37 user-session-load-2345611-37-interdiff.txt746 bytesberdir
#37 user-session-load-2345611-37.patch28.44 KBberdir
#34 user-session-load-2345611-34.patch27.71 KBberdir
#29 user-session-load-2345611-29-interdiff.txt2.13 KBberdir
#29 user-session-load-2345611-29.patch29.19 KBberdir
#25 user-session-load-2345611-25-interdiff.txt7.48 KBberdir
#25 user-session-load-2345611-25.patch29.12 KBberdir
#22 user-session-load-2345611-22.patch23.84 KBberdir
#13 user-session-load-2345611-13-interdiff.txt2.44 KBberdir
#13 user-session-load-2345611-13.patch23 KBberdir
#11 user-session-load-2345611-11-interdiff.txt3.67 KBberdir
#11 user-session-load-2345611-11.patch20.6 KBberdir
#7 user-session-load-2345611-7-interdiff.txt14.83 KBberdir
#4 user-session-load-2345611-3.patch6.6 KBberdir
#7 user-session-load-2345611-7.patch16.94 KBberdir
#4 user-session-load-2345611-3-interdiff.txt4.5 KBberdir
#1 user-session-load-2345611-1.patch3.84 KBberdir

Issue fork drupal-2345611

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

berdir’s picture

Status: Active » Needs review
StatusFileSize
new3.84 KB

First patch.

Status: Needs review » Needs work

The last submitted patch, 1: user-session-load-2345611-1.patch, failed testing.

andypost’s picture

Does entity api requires user to be usable?
Seems only access checking "prepareUser()" cares about user in entity manager
Anyway EM dependency should be declared

+++ b/core/lib/Drupal/Core/Session/SessionHandler.php
@@ -112,22 +113,25 @@ public function read($sid) {
+      $user = \Drupal::entityManager()->getStorage('user')->load($values['uid']);

should be injected properly

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new6.6 KB
new4.5 KB

Second and better patch.

I have a feeling that all the session related methods on user/account are completely bogus and unused...

@andypost: Don't get the first part, but sure, it should be injected, will care about that when I know that it is actually working..

Status: Needs review » Needs work

The last submitted patch, 4: user-session-load-2345611-3.patch, failed testing.

andypost’s picture

I mean that chicken-egg about session and session account that uses entity api.
and yes, currentUser() is fragile at least with global $user

berdir’s picture

Assigned: Unassigned » berdir
Status: Needs work » Needs review
StatusFileSize
new16.94 KB
new14.83 KB

See #2248297: Ensure routes are rebuilt when install modules.

That and lots of translation issues.

Status: Needs review » Needs work

The last submitted patch, 7: user-session-load-2345611-7.patch, failed testing.

The last submitted patch, 7: user-session-load-2345611-7.patch, failed testing.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new20.6 KB
new3.67 KB

Support TranslationWrapper in config casting (this is a bit weird, as we put translated values there, but we did that before to.

Also merge in the fixes from #2248297: Ensure routes are rebuilt when install modules

Status: Needs review » Needs work

The last submitted patch, 11: user-session-load-2345611-11.patch, failed testing.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new23 KB
new2.44 KB

Fixing those last tests. TranslationWrapper *is* tricky...

catch’s picture

+++ b/core/modules/views/src/Plugin/views/display/Page.php
@@ -336,30 +333,6 @@ public function buildOptionsForm(&$form, FormStateInterface $form_state) {
-        // Only display the menu selector if Menu UI module is enabled.

Is this hunk intentional?

Overall this looks good. The entity system dependency in sessions isn't great, but it's just making an existing dependency explicit.

znerol’s picture

I'm very in favor of this. In the long run we should load and set the current user in the AuthenticationManager and that will move the entity manager dependency out of the session handler again.

Still it looks like this reverses the decoupling which was introduced in #1825332: Introduce an AccountInterface to represent the current user.

znerol’s picture

znerol’s picture

What about introducing a core AccountProviderInterface and implement it in the user module?

berdir’s picture

I had a similar plan yes, firing an event or something after having loaded the session and then user can chime in and provide the user.

I have no idea yet how that would work with basic auth yet, though.

Also, I've planned to remove the methods here, because they are alraedy non-functional now and are obviously unused.

znerol’s picture

Firing an event or something after having loaded the session and then user can chime in and provide the user.

I think this is overkill (and probably out of scope here). I do not think that we should support more than one user-provider (at least not in core) and therefore firing an event is not appropriate here.

Instead the responsibility of identifying the user-id (or perhaps even the user account) associated with a given session should be moved to the cookie authentication provider. However this requires us to pass that information (i.e. the uid) form the SessionHandler::read() to the authentication provider somehow. I planned to simply use the symfony session for that (#2229145: Register symfony session components in the DIC and inject the session service into the request object). This would have the effect that the uid would be duplicated though, once it would be recorded in the uid-column and then also in the serialized session record. I do not consider that a problem. The uid-column on the session table simply is an additional index used to kill all sessions of a given user for example.

Given the above, cookie provider would then work like basic auth instead of the other way around.

Also, I've planned to remove the methods here, because they are alraedy non-functional now and are obviously unused.

Great, in addition you should kill getHostname and move getLastAccessedTime to the User entity. The former is unused and the latter is really only useful to order the list of users in the administrative interface. Also the code in SessionHandler::write() which updates the access-timestamp should be moved to a TERMINATE event handler in the user module.

berdir’s picture

Sure, that works too for me.

We could just put the uid column value inside the session data so that it is available to the cookie provider?

I will pick this up when #2313135: setting page_cache_without_database in settings.php prevents the container from being dumped is in.

znerol’s picture

We could just put the uid column value inside the session data so that it is available to the cookie provider?

That would be wrong, because at this point in time the $_SESSION is not yet present (and also the symfony session bags are not yet associated with it). It will only be populated later from the return value of SessionHandler::read().

berdir’s picture

StatusFileSize
new23.84 KB

Ok, that got in, so here is a re-roll + I also included #2347799: Remove bugged session-related methods from AccountInterface.

znerol’s picture

Status: Needs review » Needs work
  1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
    @@ -633,6 +633,9 @@ public function updateModules(array $module_list, array $module_filenames = arra
    +      if ($this->containerNeedsDumping && !$this->dumpDrupalContainer($this->container, static::CONTAINER_BASE_CLASS)) {
    +        $this->container->get('logger.factory')->get('DrupalKernel')->notice('Container cannot be written to disk');
    +      }
    
  2. This can be removed after #2313135: setting page_cache_without_database in settings.php prevents the container from being dumped

  3. +++ b/core/lib/Drupal/Core/Extension/ModuleHandler.php
    @@ -880,6 +880,8 @@ public function install(array $module_list, $enable_dependencies = TRUE) {
    +    \Drupal::service('router.builder')->setRebuildNeeded();
    +
    

    Not sure about that, but maybe related to #2248297: Ensure routes are rebuilt when install modules or #2230091: Route rebuilding is not guaranteed to finish in time for the next request

  4. +++ b/core/lib/Drupal/Core/Session/SessionHandler.php
    @@ -112,35 +112,34 @@ public function read($sid) {
    +    if (!$user && $values) {
    +      // The user is anonymous or blocked. Only preserve the timestamp from the
           // {sessions} table.
           $user = new UserSession(array(
    -        'session' => $values['session'],
    -        'access' => $values['access'],
    +        'access' => $values['timestamp'],
           ));
         }
    

    Removing getLastAccessedTime() from the AccountInterface would make it possible to kill that entirely. In addition getHostname() is unused and can be removed too.

berdir’s picture

1. 2. Yes, those changes were added here because I needed this. This issue is the reason I started working on all those routing issues :)

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new29.12 KB
new7.48 KB

Rerolled and removed those methods.

berdir’s picture

Status: Needs review » Needs work

The last submitted patch, 25: user-session-load-2345611-25.patch, failed testing.

znerol’s picture

  1. +++ b/core/lib/Drupal/Core/DrupalKernel.php
    @@ -632,6 +632,9 @@ public function updateModules(array $module_list, array $module_filenames = arra
    +      if ($this->containerNeedsDumping && !$this->dumpDrupalContainer($this->container, static::CONTAINER_BASE_CLASS)) {
    +        $this->container->get('logger.factory')->get('DrupalKernel')->notice('Container cannot be written to disk');
    +      }
    

    Looks like a leftover?

  2. +++ b/core/lib/Drupal/Core/Extension/ModuleHandler.php
    @@ -892,6 +892,8 @@ public function install(array $module_list, $enable_dependencies = TRUE) {
    +    \Drupal::service('router.builder')->setRebuildNeeded();
    +
    

    Dito.

berdir’s picture

Yes. Removed that. Also the module handler change seems to fix InstallUninstallTest, some weird caching thing going on. Again.

znerol’s picture

Issue tags: +needs profiling

Status: Needs review » Needs work

The last submitted patch, 29: user-session-load-2345611-29.patch, failed testing.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new27.71 KB

Re-roll.

Status: Needs review » Needs work

The last submitted patch, 34: user-session-load-2345611-34.patch, failed testing.

berdir’s picture

Symfony\Component\DependencyInjection\ContainerBuilder->getDefinition(content_translation.manager)
Symfony\Component\DependencyInjection\ContainerBuilder->get(content_translation.manager, 1)
Drupal\Core\DependencyInjection\ContainerBuilder->get(content_translation.manager)
Drupal::service(content_translation.manager)
content_translation_enabled(user)
content_translation_entity_storage_load([Array], user)
Drupal\Core\Entity\Sql\SqlContentEntityStorage->getFromStorage([Array])
Drupal\Core\Entity\Sql\SqlContentEntityStorage->doLoadMultiple([Array])
Drupal\Core\Entity\EntityStorageBase->loadMultiple([Array])
Drupal\Core\Entity\EntityStorageBase->load(2)
Drupal\Core\Session\SessionHandler->read(lXSu4BT_FwfHwWS_8j2itgNA8Mgrt9AlxzKgK1fZzgw)
Symfony\Component\HttpFoundation\Session\Storage\Handler\WriteCheckSessionHandler->read(lXSu4BT_FwfHwWS_8j2itgNA8Mgrt9AlxzKgK1fZzgw)
Symfony\Component\HttpFoundation\Session\Storage\Proxy\SessionHandlerProxy->read(lXSu4BT_FwfHwWS_8j2itgNA8Mgrt9AlxzKgK1fZzgw)
session_start()
Symfony\Component\HttpFoundation\Session\Storage\NativeSessionStorage->start()
Drupal\Core\Session\SessionManager->startNow()
Drupal\Core\Session\SessionManager->start()
Drupal\Core\DrupalKernel->initializeContainer(TRUE)
Drupal\Core\DrupalKernel->updateModules([Array], [Array])
Drupal\Core\Extension\ModuleInstaller->updateKernel([Array])
Drupal\Core\Extension\ModuleInstaller->install([Array])
Drupal\system\Form\ModulesListConfirmForm->submitForm([Array], Drupal\Core\Form\FormState)
call_user_func_array([Array], [Array])
Drupal\Core\Form\FormSubmitter->executeSubmitHandlers([Array], Drupal\Core\Form\FormState)
Drupal\Core\Form\FormSubmitter->doSubmitForm([Array], Drupal\Core\Form\FormState)
Drupal\Core\Form\FormBuilder->processForm(system_modules_confirm_form, [Array], Drupal\Core\Form\FormState)
Drupal\Core\Form\FormBuilder->buildForm(Drupal\system\Form\ModulesListConfirmForm, Drupal\Core\Form\FormState)
Drupal\Core\Controller\FormController->getContentResult(Symfony\Component\HttpFoundation\Request)
call_user_func_array([Array], [Array])
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Symfony\Component\HttpFoundation\Request, 1)
Symfony\Component\HttpKernel\HttpKernel->handle(Symfony\Component\HttpFoundation\Request, 1, TRUE)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Symfony\Component\HttpFoundation\Request, 1, TRUE)
Drupal\Core\StackMiddleware\PageCache->handle(Symfony\Component\HttpFoundation\Request, 1, TRUE)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Symfony\Component\HttpFoundation\Request, 1, TRUE)
Stack\StackedHttpKernel->handle(Symfony\Component\HttpFoundation\Request, 1, TRUE)
Drupal\Core\DrupalKernel->handle(Symfony\Component\HttpFoundation\Request)

Such fear. much scared.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new28.44 KB
new746 bytes

Reroll, let's see if this change fixes those tests, content_translation tried accessing new services after being installed but we didn't replace the global container yet.

Status: Needs review » Needs work

The last submitted patch, 37: user-session-load-2345611-37.patch, failed testing.

berdir’s picture

Correct, but that wasn't fixed in a way that helps us here. That part was postponed to #2418521: Translating field definition descriptions problematic due to early t(), hard to test.

The problem with moving forward without that is that adding a field to users that does not use TranslationWrapper() could suddenly completely break your site. I'm not sure that's a risk we can take.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new27.98 KB

Reroll.

Status: Needs review » Needs work

The last submitted patch, 41: user-session-load-2345611-41.patch, failed testing.

znerol’s picture

Issue tags: +Needs reroll
berdir’s picture

Yeah, the TranslationWrapper problem stil exists, though, but if someone wants to re-roll this, now would be the time :)

almaudoh’s picture

@Berdir: could you explain the TranslationWrapper problem, it is not reflected in the IS and not very clear from the comments.

I also re-opened #2347799: Remove bugged session-related methods from AccountInterface so we can finish that piece quickly, even though it had been incorporated into this patch.

znerol’s picture

Let's see whether I'm getting that right: The AuthenticationSubscriber needs to run before LanguageRequestSubscriber for some reason. At the time the authenticate() method is executed, the language subsystem is not yet completely initialized.

In the BasicAuth provider we currently load the whole user entity. But in the Cookie provider we execute the queries manually. The question now is whether the former is broken in multilingual sites or whether it would be possible to convert the latter to entities now that we do not authenticated at random points in the request response cycle?

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new4.63 KB

Fresh start, with the discussed getRoles() implementation that avoids fields definitions. Manual testing worked well.

Status: Needs review » Needs work

The last submitted patch, 47: current-user-load-entity-2345611-47.patch, failed testing.

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 47: current-user-load-entity-2345611-47.patch, failed testing.

berdir’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new6.34 KB
new1.71 KB

Fixed the unit tests, will need more.

Test took forever but I really can't explain why that would be related to the patch. Let's see if it's still that slow.

Status: Needs review » Needs work

The last submitted patch, 51: current-user-load-entity-2345611-57.patch, failed testing.

berdir’s picture

Again the test got terminated after 3h. I guess one of them loops forever, but how... will need to diff the test list I guess...

znerol’s picture

Drupal\user\Tests\UserCancelTest seems to be missing.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new7.66 KB
new2.21 KB

Right, that was looping in a weird way because I forgot to check the user status. I added that now and used the entity key to ensure that we can load and check that without having to load the field definitions. And also explicitly unset the uid in the session during the user cancel process.

andypost’s picture

Great! Only user entity changes needs clarification - why status key value used to return field value?

result of \Drupal\Core\Entity\ContentEntityBase::getEntityKey() described as

The value of the entity key, NULL if not defined.

  1. +++ b/core/modules/user/src/Authentication/Provider/Cookie.php
    @@ -73,26 +73,14 @@ public function authenticate(Request $request) {
    +        if ($user->isActive()) {
    +          return $user;
    +        }
    +        else {
    +          $session->remove('uid');
    +        }
    

    else superflowous

  2. +++ b/core/modules/user/src/Entity/User.php
    @@ -48,7 +48,8 @@
    + *     "status" = "status"
    
    @@ -312,14 +326,14 @@ public function setLastLoginTime($timestamp) {
    -    return $this->get('status')->value == 1;
    +    return $this->getEntityKey('status') == 1;
    ...
    -    return $this->get('status')->value == 0;
    +    return $this->getEntityKey('status') == 0;
    

    this looks strange, getEntityKey() should return a value of key (string "status")

berdir’s picture

1. the else is to remove the session key in case the user is no longer active, I think that's useful cleanup so that we don't have to try and load a blocked user on every request.

2. It posts the *value* of the key, which is 0/1. getEntityKey() is an optimization for entity keys that allows to access their values without having to load field definitions and field values without having to go through field objects.

andypost’s picture

StatusFileSize
new650 bytes
new7.73 KB

I mean that

znerol’s picture

I'm trying to come up with a sensible performance test. Trying the following alternative front controller test.php:

use Drupal\Core\DrupalKernel;
use Symfony\Component\HttpFoundation\Request;

$autoloader = require_once 'autoload.php';
$request = Request::createFromGlobals();
$kernel = DrupalKernel::createFromRequest($request, $autoloader, 'prod');
$kernel->prepareLegacyRequest($request);
var_dump($kernel->getContainer()->get('authentication.cookie')->authenticate($request));

ab -c1 -CSESSxxxxx=yyyyyy -n 1000 -k http://localhost:3000/test.php > /tmp/ab-head-test.txt

Requests per second:    101.32 [#/sec] (mean)
Time per request:       9.870 [ms] (mean)
Time per request:       9.870 [ms] (mean, across all concurrent requests)
Transfer rate:          337.21 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:     8   10   2.3      9      29
Waiting:        8   10   2.3      9      29
Total:          8   10   2.3      9      29

ab -c1 -CSESSxxxxx=yyyyyy -n 1000 -k http://localhost:3000/test.php > /tmp/ab-patch-test.txt

Requests per second:    71.43 [#/sec] (mean)
Time per request:       14.000 [ms] (mean)
Time per request:       14.000 [ms] (mean, across all concurrent requests)
Transfer rate:          518.36 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    12   14   3.2     13      39
Waiting:       11   14   3.2     13      39
Total:         12   14   3.2     13      39

I do not think that test is fair, in head the whole entity system is likely not instantiated.

znerol’s picture

StatusFileSize
new1.25 KB
new1.25 KB
znerol’s picture

StatusFileSize
new591 bytes
new1.26 KB
new1.26 KB

Another test with the path admin/get-node-one wired up to the following controller (see attached module):

class HiController {
  public function getNodeOne() {
    return new Response(\Drupal::entityManager()
      ->getStorage('node')->load(1)->getTitle());
  }
}

HEAD:

Requests per second:    38.26 [#/sec] (mean)
Time per request:       26.139 [ms] (mean)
Time per request:       26.139 [ms] (mean, across all concurrent requests)
Transfer rate:          17.48 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    21   26   6.4     23      64
Waiting:       21   26   6.4     23      64
Total:         21   26   6.4     23      64

patch:

Requests per second:    36.31 [#/sec] (mean)
Time per request:       27.542 [ms] (mean)
Time per request:       27.542 [ms] (mean, across all concurrent requests)
Transfer rate:          16.59 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    23   27   6.0     25      64
Waiting:       23   27   6.0     25      64
Total:         23   27   6.0     25      64
berdir’s picture

Yeah...

My results, your test.php (1000 requests) and frontpage with one node (

HEAD test.php:
Requests per second:    179.89 [#/sec] (mean)
Time per request:       5.559 [ms] (mean)
Time per request:       5.559 [ms] (mean, across all concurrent requests)

Patch test.php:
Requests per second:    121.97 [#/sec] (mean)
Time per request:       8.199 [ms] (mean)
Time per request:       8.199 [ms] (mean, across all concurrent requests)

Head front:
Requests per second:    12.76 [#/sec] (mean)
Time per request:       78.370 [ms] (mean)
Time per request:       78.370 [ms] (mean, across all concurrent requests)

Patch: front:
Requests per second:    12.22 [#/sec] (mean)
Time per request:       81.857 [ms] (mean)
Time per request:       81.857 [ms] (mean, across all concurrent requests)

Did some profiling to figure out what makes it slower. Turns out, we do have a few calls to methods that do have to load fields. I noticed getTimezone(), getLastAccessedTime() and getUsername().

znerol’s picture

StatusFileSize
new1.78 KB
new169.44 KB
new162.94 KB
new4.39 KB

Hacked my way around authentication in order to get xhprof-kit working for authenticated users (see attached diff). There is something weird going on: Only 50% of the whooping 4ms increase is contributed by Cookie::authenticate(), the rest comes from AccountProxy::setAccount(). The heavy path there is AccountProxy::setAccount() - drupal_get_user_timezone() - AccountProxy::getTimeZone() - User::getTimeZone() - ContentEntityBase::get() - ContentEntityBase::getTranslatedField() - ...

See also attached logs and screenshots. Not really sure whether there is room for optimization down that rabbit hole.

catch’s picture

For performance test there's two things to look at:

1. Where the current user is not loaded at all on the request with HEAD.

2. Where the current user is loaded in the current request already.

A controller that returns a minimal response, then one that loads a user and does the same would work for testing.

I'd expect #1 to be slower - since we have a user load where there was previously none (although with entity caching this might not be bad either). But on the other hand #2 should be faster, because we only load the user information once rather than doing the direct query also.

znerol’s picture

Regrettably entity caching does not help here. The profiling revealed that most of the regression is due to accessing data through fields on the user entity (username, last accessed time, timezone). @Berdir correct me if I'm wrong.

znerol’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll
StatusFileSize
new168.5 KB
new169.5 KB
new178.69 KB

Needs a reroll after #2347799: Remove bugged session-related methods from AccountInterface.

I've applied the idea of #47 (sidestep field definitions) to a couple of other methods in the user entity (
getPassword(), getCreatedTime(), getLastAccessedTime(), getLastLoginTime(), getTimeZone(), getUsername()), and profiled again. The regression due to the expensive field definition stuff is gone. I.e. the expensive AccountProxy::setAccount() visible in #63 is gone (see the on-request-authenticate screenshots.

From the xhprof results it may look like EntityManager::getStorage() is still a regression. However, that is not the case at all. The method is used a couple of times throughout the request/response cycle and it just happens to be called in Cookie::authenticate() for the first time. The profiler output for EntityManager::getStorage() shows the same inclusive wall-time even though the method is called 28 times (patch) instead of 27 (head).

There is still a minor regression of about 0.5ms(?) due to EntityStorageBase::load which seems to be a little bit slower than manual queries. In my opinion that is acceptable.

Given those results, is there anything in the entity subsystem which we can leverage to access selected properties directly if necessary? I think there is a category of data where translation / typed data should be optional (e.g. timestamps, timezones, boolean flags, password hash) and direct access should be possible.

znerol’s picture

andypost’s picture

Status: Needs work » Needs review
Issue tags: -needs profiling, -Needs reroll
StatusFileSize
new7.98 KB

Re-roll and removal of protected getUserFromSession to inline code

I think there is a category of data where translation / typed data should be optional (e.g. timestamps, timezones, boolean flags, password hash) and direct access should be possible.

+1

dawehner’s picture

  1. +++ b/core/modules/user/src/Authentication/Provider/Cookie.php
    @@ -58,41 +58,14 @@ public function applies(Request $request) {
    +    if ($uid = $request->getSession()->get('uid')) {
    +      /** @var \Drupal\user\UserInterface $user */
    +      if ($user = $this->entityManager->getStorage('user')->load($uid)) {
    

    It is though a little bit sad that, as it is in HEAD, we basically have a dependency to user module here, but yeah this is not a new thing.

  2. +++ b/core/modules/user/src/Entity/User.php
    @@ -154,9 +155,22 @@ public function getRoles($exclude_locked_roles = FALSE) {
    +    // Optimize for the case where the field object has not been initialized
    +    // yet, directly access the values.
    

    Should we point to the code using it, I guess its hasPermission?

andypost’s picture

Just re-roll

#69,2 makes sense, looks we can revert that part because it's no more a case since we use AnonymousUser for that, and we should never get a User object half-initialized

The last submitted patch, 70: current-user-load-entity-2345611-70-nopart.patch, failed testing.

znerol’s picture

Status: Needs review » Needs work
Issue tags: +Needs issue summary update

#70 is not a good idea. @Berdir: Could you please explain the reason for #47. I only recall that the hunk in getRoles() is necessary because the field definitions somehow depend on the language subsystem being initialized (and that one depends on authentication already being performed). It would be fantastic if you also could give some direction on what to do about the slowness of getTimeZone() and related methods.

andypost’s picture

Failed test because UserTest inherited from UserSessionTest
Probably we need decouple them

andypost’s picture

looks we need to deprecate here UserSession or make core depends on user entity as #69.1 said
and how to update #2477213: Move AccountInterface out of Session namespace now

mgifford’s picture

Assigned: berdir » Unassigned

Unassigning stale issue. Hopefully someone else will pursue this.

Version: 8.0.x-dev » 8.1.x-dev

Drupal 8.0.6 was released on April 6 and is the final bugfix release for the Drupal 8.0.x series. Drupal 8.0.x will not receive any further development aside from security fixes. Drupal 8.1.0-rc1 is now available and sites should prepare to update to 8.1.0.

Bug reports should be targeted against the 8.1.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new9.94 KB
new2.2 KB

Still applies with git apply -3.

The reason for #47 is the same as the answer to make getTimezone() faster. Skipping the slow field item list + field item creation.

#2580551: Optimize getCommentedEntity() is an attempt at making that more generic, but I'm not sure if we can or want to support multiple fields.

Status: Needs review » Needs work

The last submitted patch, 77: current-user-load-entity-2345611-77.patch, failed testing.

berdir’s picture

Status: Needs work » Needs review
StatusFileSize
new9.87 KB
new2.44 KB

Lets try this instead.

berdir’s picture

StatusFileSize
new13.31 KB

That doesn't work, here's a combined patch with getCommentedEntity() and using getFieldValue().

The last submitted patch, 79: current-user-load-entity-2345611-79.patch, failed testing.

Status: Needs review » Needs work

The last submitted patch, 80: current-user-load-entity-2345611-80.patch, failed testing.

berdir’s picture

Fixes and cleanup.

Tested this, getFieldDefinitions() for user is no longer called with this on normal pages. Only place it is called is in the comment form. And when viewing the user profile, but only the first time. And both cases are already the case right now.

The last submitted patch, 83: current-user-load-entity-2345611-83.patch, failed testing.

dawehner’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -503,6 +503,51 @@ protected function getTranslatedField($name, $langcode) {
       /**
    +   * Gets the value of a specific property of a field.
    +   *
    

    Should we document atht this is optimized for performance reasons?

  2. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -503,6 +503,51 @@ protected function getTranslatedField($name, $langcode) {
    +  public function getFieldValue($field_name, $property) {
    

    Just curious, could you specify a third $delta parameter with it?

berdir’s picture

1. Was thinking about that, not sure.
2. I thought about that too, actually had it at one point. it's a bit strange with the non-delta specific values, but I think we can simply only check those for delta 0.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.9 was released on September 7 and is the final bugfix release for the Drupal 8.1.x series. Drupal 8.1.x will not receive any further development aside from security fixes. Drupal 8.2.0-rc1 is now available and sites should prepare to upgrade to 8.2.0.

Bug reports should be targeted against the 8.2.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

berdir’s picture

This should fix that test fail.

Status: Needs review » Needs work

The last submitted patch, 88: current-user-load-entity-2345611-88.patch, failed testing.

jibran’s picture

+++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
@@ -503,6 +503,51 @@ protected function getTranslatedField($name, $langcode) {
+  public function getFieldValue($field_name, $property) {

I think we should add delta as an optional argument.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.6 was released on February 1, 2017 and is the final full bugfix release for the Drupal 8.2.x series. Drupal 8.2.x will not receive any further development aside from critical and security fixes. Sites should prepare to update to 8.3.0 on April 5, 2017. (Drupal 8.3.0-alpha1 is available for testing.)

Bug reports should be targeted against the 8.3.x-dev branch from now on, and new development or disruptive changes should be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

richgerdes’s picture

StatusFileSize
new14.45 KB

I came across this issue while working working on a project. The method used to setup the user session make it harder to work with the current user since its not a real user and as said previously you end up loading the user anyway. If this could get merged into core, I am sure it would make many people pleased.

There are a number of modules that rely on \Drupal::currentUser() to get access information but without having a real user this can have unexpected functionality as it is implemented now.

I have rewritten the patch from #88 against the latest version of 8.3. I also added delta as an optional argument as suggest in #90.

berdir’s picture

Status: Needs work » Needs review

Thanks for the reroll, don't forget to set issues to needs review when uploading a patch.

Keep in mind that code still shouldn't rely on the current user being a user entity, but loading the entity now is at least a load from static cache.

Status: Needs review » Needs work

The last submitted patch, 92: 2345611-92.diff, failed testing.

berdir’s picture

Version: 8.3.x-dev » 8.4.x-dev
Status: Needs work » Needs review
StatusFileSize
new11.72 KB
new2.79 KB

This removes the comment specific parts from this, lets see how many test fails are left.

jibran’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -541,6 +541,53 @@ protected function getTranslatedField($name, $langcode) {
    +        if (isset($field_values[$delta][$property]) && is_array($field_values[$delta])) {
    

    Shouldn't this be other way around first check the is_array and then isset?

  2. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -541,6 +541,53 @@ protected function getTranslatedField($name, $langcode) {
    +    return $this->get($field_name)->$property;
    

    We should use delta here.

berdir’s picture

Status: Needs review » Needs work

1. Nope because the $delta key might not be set. This is just there to prevent matching it as string index as it's possible that levels could be a string, depending on how the entity is initialized.
2. yep.

richgerdes’s picture

richgerdes’s picture

StatusFileSize
new11.83 KB
new685 bytes

Uploaded the wrong version of the patch the first time.

The last submitted patch, 98: 2345611-98.patch, failed testing.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

andypost’s picture

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

erik frèrejean’s picture

Status: Needs review » Reviewed & tested by the community

I've been running this patch in production for the past six months without problem, and seeing that all raised issues have been resolved I think this is ready to be RTBC'ed.

hchonov’s picture

Status: Reviewed & tested by the community » Needs work
  1. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -541,6 +541,58 @@ protected function getTranslatedField($name, $langcode) {
    +  public function getFieldValue($field_name, $property, $delta = 0) {
    

    The method should be defined in the interface FieldableEntityInterface.

  2. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -541,6 +541,58 @@ protected function getTranslatedField($name, $langcode) {
    +      if (isset($this->values[$field_name][$this->activeLangcode])) {
    ...
    +      elseif ($this->values[$field_name][LanguageInterface::LANGCODE_DEFAULT]) {
    

    I think, that the elseif needs the same isset check like the if.

  3. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -541,6 +541,58 @@ protected function getTranslatedField($name, $langcode) {
    +        if (isset($field_values[$delta][$property]) && is_array($field_values[$delta])) {
    

    What is the point of the second condition? If the first one is met, then the second one will always be true.

  4. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -541,6 +541,58 @@ protected function getTranslatedField($name, $langcode) {
    +        elseif (isset($field_values[$property]) && is_array($field_values)) {
    

    Same question as the previous one.

richgerdes’s picture

StatusFileSize
new1.48 KB
new11.77 KB

Thanks for the review. 2, 3, and 4 all sound like good suggestions. They have been applied in the attached patch.

For 1 however, I think there is a concern here with moving the function definition. Ultimately, moving it will create a potential backwards incompatibility. If there is a EntityType which implements the FieldableEntityInterface without extending the ContentEntityBase then they won't receive the new function, and code will break. While I hope this doesn't occur, its still possible and as a result has a negative effect since it may prevent sites (and modules) from upgrading to the latest version.

Has this been done with interfaces previously without causing havoc? If not, its probably best to make a new interface to hold this definition. Thoughts?

richgerdes’s picture

Status: Needs work » Needs review

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

avpaderno’s picture

Status: Needs review » Needs work
aleevas’s picture

StatusFileSize
new11.8 KB

Reverted patch from #112 up to 8.9

aleevas’s picture

Status: Needs work » Needs review

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

janlaureys’s picture

StatusFileSize
new11.8 KB

Updated the patch so it applies in 9.2 and up again.

andypost’s picture

Version: 9.3.x-dev » 9.4.x-dev
Status: Needs review » Needs work

Last patch is broken for current core

janlaureys’s picture

StatusFileSize
new6.8 KB

Yeah I was a little quick to upload the patch before verifying it on 9.3.
Updated it again and now verified it applies on 9.3 and 9.4 as well.

Edit: I'm well out of my depth here. Can see all tests failing already. Apologies :(.

janlaureys’s picture

StatusFileSize
new8.23 KB

Fixed the service definition and class constructor to also include the messenger service.

janlaureys’s picture

StatusFileSize
new8.56 KB

Okay I think I'm getting there. Actually ran the commit-code-check script this time and turned out fine.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

casey’s picture

Title: Load user entity in SessionHandler instead of using manual queries » Load user entity in Cookie AuthenticationProvider instead of using manual queries
Issue summary: View changes
ravi.shankar’s picture

StatusFileSize
new8.31 KB
new6.7 KB

Added reroll of patch #121 on Drupal 9.5.x.

avpaderno’s picture

Status: Needs work » Needs review
ravi.shankar’s picture

Status: Needs review » Needs work

Patch #124 didn't pass the test.

berdir’s picture

The patches since #119 are missing the getFieldValue() method that needs to be restored from #117.

ravi.shankar’s picture

Status: Needs work » Needs review
StatusFileSize
new11.54 KB
new3.23 KB

Restored the changes of patch #117 as per comment #127.

avpaderno’s picture

Status: Needs review » Needs work
anchal_gupta’s picture

StatusFileSize
new11.54 KB
new588 bytes
berdir’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -618,6 +618,58 @@ protected function getTranslatedField($name, $langcode) {
    +        // Configurable/Multi-value fields are stored differently, try accessing
    

    make it multiple-value fields then as it doesn't like multi

ravi.shankar’s picture

Status: Needs work » Needs review
StatusFileSize
new11.55 KB
new591 bytes

Addressed typo of patch #128.

ravi.shankar’s picture

Sorry didn't notice, patch #130.

Status: Needs review » Needs work

The last submitted patch, 132: 2345611-131.patch, failed testing. View results

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

andypost’s picture

Issue tags: +Needs reroll
eugene.brit’s picture

StatusFileSize
new11.72 KB
medha kumari’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new11.46 KB
new4.96 KB

Rerolled the patch #137 in Drupal 10.1.x.

avpaderno’s picture

Status: Needs review » Needs work
mrinalini9’s picture

Status: Needs work » Needs review
StatusFileSize
new11.4 KB

Rerolled patch #138, please review it.

mrinalini9’s picture

StatusFileSize
new11.39 KB
new949 bytes

Fixing custom commands failure issues in patch #140, please review it.

avpaderno’s picture

Status: Needs review » Needs work
ameymudras’s picture

Status: Needs work » Needs review
StatusFileSize
new11.47 KB
new581 bytes

Status: Needs review » Needs work

The last submitted patch, 143: 2345611-143.patch, failed testing. View results

eugene.brit’s picture

StatusFileSize
new11.47 KB

Re-roll #137 for D10.0.x

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

znerol’s picture

Issue tags: +ddd2023

Needs a reroll for 11.x. I'll stick around for reviews here. Find me in the sky lounge at DDD vienna if there are any questions.

Also desperately needs an issue summary update.

avpaderno’s picture

Issue tags: +Needs reroll
znerol’s picture

Issue summary: View changes
znerol’s picture

Issue summary: done.

znerol’s picture

  1. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -618,6 +618,58 @@ protected function getTranslatedField($name, $langcode) {
    +   * Only the first delta can be accessed with this method.
    

    This is not true. There is an optional $delta argument.

  2. +++ b/core/lib/Drupal/Core/Entity/ContentEntityBase.php
    @@ -618,6 +618,58 @@ protected function getTranslatedField($name, $langcode) {
    +  public function getFieldValue($field_name, $property, $delta = 0) {
    

    Let's add type hints here.

_utsavsharma’s picture

StatusFileSize
new11.49 KB
new11.49 KB

Rerolled for 11.x as mentioned in #147.
Tried to address pointer 2 in #151.

vsujeetkumar’s picture

This is not true. There is an optional $delta argument.

In #151.1, True $delta argument is optional, As per my understanding, Either we should update the comment like

By default the first delta can be accessed with this method.

or we can remove this line from the comment.

catch’s picture

Issue tags: +Performance

These are two of very few database queries executed on an authenticated page request when dynamic page cache is a cache hit, if we load the user it will usually come from the cache.
See for example https://grafana.prod.cluster.tag1.io/explore?panes=%7B%22taO%22:%7B%22datasource%22:%22tempo%22,%22queries%22:%5B%7B%22query%22:%2269f4dbd9a3acdf0e49e11acc90cbae81%22,%22queryType%22:%22traceql%22,%22refId%22:%22A%22,%22datasource%22:%7B%22type%22:%22tempo%22,%22uid%22:%22tempo%22%7D,%22limit%22:20%7D%5D,%22range%22:%7B%22from%22:%22now-6h%22,%22to%22:%22now%22%7D%7D%7D&schemaVersion=1&orgId=1 this trace from the Umami authenticated performance test. Would only be a small optimization from that perspective but worth doing I think, so tagging with performance for that.

znerol’s picture

Issue tags: -ddd2023

This needs a reroll from #152 , and it should be converted to a merge request.

prem suthar made their first commit to this issue’s fork.

prem suthar’s picture

As per the #155 re-rolled the patch #152 and created the Mr . also Addressed the #151 Points
Please review.

znerol’s picture

Thanks @prem suthar, the MR looks right. Regrettably, there are two test failures in @core/modules/user/tests/src/Unit/Plugin/Core/Entity/UserTest.php@. I can reproduce them locally. The full output:

% ../vendor/bin/phpunit modules/user/tests/src/Unit/Plugin/Core/Entity/UserTest.php
PHPUnit 10.5.38 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.14
Configuration: /home/lo/src/drupal/core/phpunit.xml

.EE                                                                 3 / 3 (100%)

Time: 00:00.199, Memory: 16.00 MB

There were 2 errors:

1) Drupal\Tests\user\Unit\Plugin\Core\Entity\UserTest::testHasRole
InvalidArgumentException: The entity object refers to a removed translation (x-default) and cannot be manipulated.

/home/lo/src/drupal/core/lib/Drupal/Core/Entity/ContentEntityBase.php:609
/home/lo/src/drupal/core/lib/Drupal/Core/Entity/ContentEntityBase.php:597
/home/lo/src/drupal/core/modules/user/src/Entity/User.php:193
/home/lo/src/drupal/core/modules/user/src/Entity/User.php:206
/home/lo/src/drupal/core/tests/Drupal/Tests/Core/Session/UserSessionTest.php:77
/home/lo/src/drupal/vendor/bin/phpunit:122

2) Drupal\Tests\user\Unit\Plugin\Core\Entity\UserTest::testUserGetRoles
InvalidArgumentException: The entity object refers to a removed translation (x-default) and cannot be manipulated.

/home/lo/src/drupal/core/lib/Drupal/Core/Entity/ContentEntityBase.php:609
/home/lo/src/drupal/core/lib/Drupal/Core/Entity/ContentEntityBase.php:597
/home/lo/src/drupal/core/modules/user/src/Entity/User.php:193
/home/lo/src/drupal/core/modules/user/tests/src/Unit/Plugin/Core/Entity/UserTest.php:56
/home/lo/src/drupal/vendor/bin/phpunit:122

--

2 tests triggered 2 PHP warnings:

1) /home/lo/src/drupal/core/lib/Drupal/Core/Entity/ContentEntityBase.php:608
Undefined array key "x-default"

Triggered by:

* Drupal\Tests\user\Unit\Plugin\Core\Entity\UserTest::testHasRole
  /home/lo/src/drupal/core/tests/Drupal/Tests/Core/Session/UserSessionTest.php:71

* Drupal\Tests\user\Unit\Plugin\Core\Entity\UserTest::testUserGetRoles
  /home/lo/src/drupal/core/modules/user/tests/src/Unit/Plugin/Core/Entity/UserTest.php:53

2) /home/lo/src/drupal/core/lib/Drupal/Core/Entity/ContentEntityBase.php:608
Trying to access array offset on null

Triggered by:

* Drupal\Tests\user\Unit\Plugin\Core\Entity\UserTest::testHasRole
  /home/lo/src/drupal/core/tests/Drupal/Tests/Core/Session/UserSessionTest.php:71

* Drupal\Tests\user\Unit\Plugin\Core\Entity\UserTest::testUserGetRoles
  /home/lo/src/drupal/core/modules/user/tests/src/Unit/Plugin/Core/Entity/UserTest.php:53

ERRORS!
Tests: 3, Assertions: 2, Errors: 2, Warnings: 2.

Not sure, maybe it is necessary to modify User::getRoles() and use $this->getFieldValue('roles') instead of $this->get('roles'). Possibly @berdir could help here.

berdir’s picture

The performance aspect of this is a tradeoff, that's also why this issue has been stuck so long.

Yes, we save two queries, but the other side of the coin is that to access the information we need from (status, roles) requires that we load the field definitions and create the field objects for it, which is much slower than the UserSession object.

(FWIW, core has been doing that anyway for years due to #3158130: Many calls to ContextRepository::getAvailableContexts() due to entity upcasting but that's fixed now).

Especially when profiling, that cost is clearly visible and it's hard to compare that with the cost of database queries. On high load sites, possibly with multiple servers and redis/memcache, that cost is easier to balance against extra database queries, but that's not the default setup.

I tried to address that with the getFieldValue() method. But that's complex and I'm not convinced it's entirely reliable and shouldn't be mixed up with this issue iMHO. So either we accept that downside of the change or we postpone it on an issue to improve that performance cost. Which might or might not ever happen.

andypost’s picture

Looking at CI jobs I see no viable perf regression, so except fixing the last test it looks ready to go (good to get it in in early days on 11.2)

berdir’s picture

Well yes, because there's a complex workaround in place to counter the performance problem. But even without that, it is very unlikely to have an impact on CI execution time. But it's been tested and shown a long time ago in comments around #60. I doubt things are fundamentally different now.

richgerdes’s picture

I was able to get the test past the initial failure, however, since we now use an entity and not a session object, there are other dependencies, including the field definitions, which aren't loaded in the unit test scope. We could mock the full User Entity, but this seems overly complicated and costly. Alternatively, we could move the test to a kernel or functional test so we get access to the container and entity schema. Not sure what the core best practice recommendations are here though. I would think we would copy the required test scenarios to the new test and move it to the Kernal Space, if that's enough to bootstrap what we need.

Version: 11.x-dev » main

Drupal core is now using the main branch as the primary development branch. New developments and disruptive changes should now be targeted to the main branch.

Read more in the announcement.

catch’s picture

Title: Load user entity in Cookie AuthenticationProvider instead of using manual queries » [PP-1] Load user entity in Cookie AuthenticationProvider instead of using manual queries
Related issues: +#3572625: Calling $entity->getTranslatedField() results in an entity-sized memory leak

Just ran into a memory leak with ContentEntityBase::get() so I've opened a new issue to add ::getFieldValue() independently of here. If we can decide that's actually worth adding with the now two use cases identified for it, it would then unblock this issue.

catch’s picture

After look at that issue, I think it's theoretically possible to fix the memory leak in a different way, but that still leaves the use case here, so I've opened #3574012: Add a getFieldValue() method to bypass typed data overhead for specific use cases to add the method from here. Leaving the PP-1 on, just points somewhere else now.

@richgerdes moving the test to a kernel test seems reasonable.

berdir’s picture

Title: [PP-1] Load user entity in Cookie AuthenticationProvider instead of using manual queries » Load user entity in Cookie AuthenticationProvider instead of using manual queries
Assigned: Unassigned » berdir
berdir’s picture

Rebased after #3574012: Add a getFieldValue() method to bypass typed data overhead for specific use cases landed.

Testing with xdebug on a dynamic page cache hit, I also had to use getFieldValue() in getRoles(), but with that, I confirmed that we're not loading the field definitions anymore.

amateescu’s picture

Issue tags: -Needs reroll

Left a note on the MR :)

berdir’s picture

StatusFileSize
new139.35 KB

Doing some profiling, this is still a significant regression on a dynamic page cache hit, around 10% or so.

The reason is entity storage, which loads the active entity type and field storage definitions in the constructor and then also initializes stuff. None of that should happen there, although we do need some information from the entity type, but it doesn't need to be the active definition. We just need to know if the entity type supports static/persistent caching to return cached results.

$this->entityType is really awkward to handle if we'd try to do BC for this. because it's an existing property from the parent that we're updating, so we can't really use __get() or so on this.

catch’s picture

$this->entityType is really awkward to handle if we'd try to do BC for this. because it's an existing property from the parent that we're updating, so we can't really use __get() or so on this.

We might be able to handle this with property hooks, maybe.