Problem/Motivation

Function brings additional overhead for comment render by looping each comment listing and initializing

It doesn't do anything by itself, it only sets 'divs/divs_final' properties in comment entities, which are only used in one place: CommentViewBuilder::alterBuild(). (Formerly comment_view($comment), see commit #e9ca778b)

The comment that is in the code now

// The $divs element instructs #prefix whether to add an indent div or
// close existing divs (a negative value).

...doesn't make any sense by itself, and clearly refers to the code that is now in alterBuild(). The whole function should be moved into CommentViewBuilder.

And since it doesn't make any sense by itself, in this case I don't think it even needs marking @deprecated. The function removal can be marked in https://www.drupal.org/node/2299799 (if we don't care about the version number mentioned in there being exactly right).

Proposed resolution

Remove the function by incorporating it into \Drupal\comment\CommentViewBuilder::buildComponents() where it actually used.
Comment entity should not be changed by build/render layer.

Remaining tasks

agree on approach, commit

User interface changes

no

API changes

Removes comment_prepare_thread() as no longer needed and buggy.

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Bug because changes entity properties without need. We should avoid dirtying entities.
Issue priority Major because affects d.o and makes using #pre_render_cache impossible for those entities (needlessly).
Prioritized changes The main goal of this issue is proper cacheability and removal of useless code.
Disruption Not disruptive for core/contributed and custom modules/themes because it used only internally and the functionality incorporated into CommentViewBuilder

Comments

roderik’s picture

Status: Active » Needs work
StatusFileSize
new5.45 KB

Disclaimer: this is the first time I'm looking at this EntityViewBuilder stuff.

Moved into buildComponents(). Seemed to make sense, because

  • needs to be called from a function where we have the full array of comments (not one by one, like alterBuild())
  • unlike EntityViewBuilder::buildMultiple(), this function is part of EntityViewBuilderInterface. Even though buildMultiple() can call buildComponents() multiple times, so we need to implement a process-only-once flag.

drupal_render() -> #pre_render call -> EntityViewBuilder::buildMultiple() -> CommentViewBuilder::buildComponents() is called later than comment_prepare_thread() was called before this patch. So the $comment->divs properties are not set anymore during the viewMultiple() stage. If needed, we can say something like the following in the change notice:

If you want to use / change the $comment->divs properties which Drupal Core 
sets, you need to do it in hook_entity_prepare_view() (or subclass CommentViewBuilder)
If you want to set the $comment->divs properties, instead of calling comment_prepare_thread(),
you should now subclass CommentViewBuilder

About the patch: the logic that is moved from comment_prepare_thread() to buildDepth() is exactly the same. I just renamed a variable and commented things.

roderik’s picture

Status: Needs work » Needs review
StatusFileSize
new7.81 KB
new4.92 KB

OR (my preference):

We just get rid of the depth and divs properties in the comment object and don't care about hook_entity_prepare_view(). Almost noone is using this function anyway. We just make a small note about CommentViewBuilder::buildDepth() in a change notice (e.g. this one).

The properties shouldn't be on the comment entities; they're information about a specific way of rendering -> should properties of $build.

Also deleted: the $comment->depth property. It's not used anywhere. All code (including the 'Depth' field in views) deduces depth from Comment::getThread().

PLEASE NOTE: when testing this manually by creating comments, you may see strange results -- the same strange results you will see withouth this patch, until #2254181: Comment indentation is incorrect for comments following a replying-comment: don't render cache comments for which threading is enabled is fixed.

andypost’s picture

Status: Needs review » Needs work

The last submitted patch, 2: comment-prepare-thread-2318579-2.patch, failed testing.

roderik’s picture

Status: Needs work » Needs review
StatusFileSize
new7.57 KB
new651 bytes

Well, that was a bit silly. *puts missing code back*

@andypost I guess I'll go reroll that patch, and after looking at the code maybe I understand how #1920044 ties into 'skip this for non-threaded forums'.

roderik’s picture

Issue summary: View changes

bump:

As far as I can see, this needs no change. The code that is moved only prepares things by adding 'metadata' to the render array, which we can keep doing in all cases; the code that actually decides whether to use that metadata (and output divs) is in CommentViewBuilder::alterBuild() and is correct.

Patch still applies (with offsets/minimal fuzz).

andypost’s picture

It's really interesting how that could work at all!
CommentViewBuilder does not know anything about comment field that holds setting about threading

+++ b/core/modules/comment/comment.module
@@ -661,7 +627,6 @@ function comment_node_update_index(EntityInterface $node, $langcode) {
-          comment_prepare_thread($comments);

+++ b/core/modules/comment/src/Plugin/Field/FieldFormatter/CommentDefaultFormatter.php
@@ -156,7 +156,6 @@ public function viewElements(FieldItemListInterface $items) {
-          comment_prepare_thread($comments);

nice, but this is a places that "knows" about thread and have to pass this to "builder"

+++ b/core/modules/comment/src/CommentViewBuilder.php
@@ -88,6 +89,8 @@ public function buildComponents(array &$build, array $entities, array $displays,
+    self::buildDepth($build, $entities);

@@ -281,6 +284,48 @@ protected static function buildLinks(CommentInterface $entity, EntityInterface $
+  protected static function buildDepth(array &$build, array $comments) {

@@ -303,8 +349,10 @@ protected function alterBuild(array &$build, EntityInterface $comment, EntityVie
+        $build['#suffix'] = str_repeat('</div>', (int) $build['#comment_divs_final']);

this could be called for comments from different fields/entities so needs to make sure that thread is consistent

roderik’s picture

Disclaimer: maybe you can skip the first part of this comment, it's only introduction.
I don't know much about ViewBuilders or the philosophy behind 'which component should do what when'. I just constructed this story from inspecting code.

It's really interesting how that could work at all!
CommentViewBuilder does not know anything about comment field that holds setting about threading

Right now, comment_node_update_index() does not know anything about threading settings either. It just
- loads comments
- calls comment_prepare_thread() for inserting the $comment->div-stuff. The knowledge for this** is inside the $comments itself, not in the calling function.
- renders them

Why does this work? Because CommentViewBuilder 'knows' about threading settings through injected components (currently still entityManager->getFieldDefinitions).

**Note that the threading check is not done in comment_prepare_thread(); the $comment->div-stuff is always generated. The check is done in CommentViewBuilder::alterBuild() which decides whether or not to use the $comment->div-stuff, based on threading settings.
This behavior (where the check is done) is not changed by the patch.

CommentDefaultFormatter::viewElements()
nice, but this is a places that "knows" about thread and have to pass this to "builder"

Well, as said: CommentDefaultFormatter::viewElements() now calls comment_prepare_thread() but it does not pass the knowledge about threading settings to "builder". CommentViewBuilder::alterBuild() fetches and uses the knowledge, later, by itself.

I can see how you would want to start passing the threading knowledge explicitly into CommentViewBuilder, but I will need you to tell me more about how/when, because I don't know / cannot get enough info from the interfaces.

this could be called for comments from different fields/entities so needs to make sure that thread is consistent

At this moment I do not agree. If you let CommentViewBuilder construct one #sorted render array with comments mixed up from different fields, you will get them rendered in a mixed-up way. That's your responsibility.

I can review my opinion after getting an answer on part #2 of this comment.

larowlan’s picture

Issue tags: +Needs reroll
vedpareek’s picture

Issue tags: -Needs reroll +SprintWeekend2015
StatusFileSize
new7.66 KB

Rerolled

wim leers’s picture

Status: Needs review » Needs work
  1. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -106,6 +107,8 @@ public function buildComponents(array &$build, array $entities, array $displays,
    +    ¶
    

    Tabs that don't belong :)

  2. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -301,6 +304,46 @@ protected static function buildLinks(CommentInterface $entity, EntityInterface $
    +
    

    All of this is indented incorrectly, I'm afraid, with 4-space instead of 2-space indentation.

  3. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -324,8 +368,10 @@ protected function alterBuild(array &$build, EntityInterface $comment, EntityVie
    +      if ($is_threaded
    +	 && !empty($build['#comment_divs_final'])
    +	 && is_numeric($build['#comment_divs_final'])) {
    +	$build['#suffix'] = str_repeat('</div>', (int) $build['#comment_divs_final']);
    

    More tabs, and code style violation.

ashutoshsngh’s picture

Status: Needs work » Needs review
StatusFileSize
new10.39 KB

Fixed identation issues.

wim leers’s picture

Could you please provide an interdiff? See https://www.drupal.org/documentation/git/interdiff.

ashutoshsngh’s picture

StatusFileSize
new8.26 KB

Interdiff attached.

andypost’s picture

Status: Needs review » Needs work

I don't get the reason to make check isset(divs) && is_numeric(divs) they are always a numbers so maybe !empty() is enough

  1. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -44,10 +45,7 @@ class CommentViewBuilder extends EntityViewBuilder {
       public static function createInstance(ContainerInterface $container, EntityTypeInterface $entity_type) {
         return new static(
    -      $entity_type,
    -      $container->get('entity.manager'),
    -      $container->get('language_manager'),
    -      $container->get('csrf_token')
    +        $entity_type, $container->get('entity.manager'), $container->get('language_manager'), $container->get('csrf_token')
    

    unneeded change, please to not change this

  2. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -167,7 +166,7 @@ public function buildComponents(array &$build, array $entities, array $displays,
    -      if ($this->moduleHandler->moduleExists('history') &&  \Drupal::currentUser()->isAuthenticated()) {
    +      if ($this->moduleHandler->moduleExists('history') && \Drupal::currentUser()->isAuthenticated()) {
    

    current_user should be injected

  3. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -302,6 +301,47 @@ protected static function buildLinks(CommentInterface $entity, EntityInterface $
    +   * Calculates the indentation level of each comment in a comment thread.
    +   */
    +  protected static function buildDepth(array &$build, array $comments) {
    

    needs doc-block fix about arguments

  4. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -349,18 +388,18 @@ protected function alterBuild(array &$build, EntityInterface $comment, EntityVie
    -      ->getCountNewComments(entity_load($context['entity_type'], $context['entity_id']));
    +        ->getCountNewComments(entity_load($context['entity_type'], $context['entity_id']));
    ...
    -      ->load($context['entity_id']);
    +        ->getStorage($context['entity_type'])
    ...
    -      ->getNewCommentPageNumber($entity->{$field_name}->comment_count, $new, $entity);
    +        ->getStorage('comment')
    

    indent wrong

  5. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -368,7 +407,7 @@ public static function attachNewCommentsLinkMetadata(array $element, array $cont
    -            'new_comment_count' => (int)$new,
    +            'new_comment_count' => (int) $new,
    

    unneeded change

ajits’s picture

Assigned: roderik » ajits
Issue tags: +Goa2015

Working on it now.

ajits’s picture

Issue tags: +Needs reroll

The patch is no longer applicable. Will reroll first.

piyuesh23’s picture

Issue tags: -Goa2015 +#drupalgoa2015
ajits’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
StatusFileSize
new16.34 KB

Just rerolling for now.

Status: Needs review » Needs work

The last submitted patch, 19: remove-2318579-18.patch, failed testing.

ajits’s picture

Assigned: ajits » Unassigned

Not sure if I did it right. I simply followed the procedure mentioned at documentation for patch reroll.

andypost’s picture

Category: Task » Bug report
Issue summary: View changes
Priority: Normal » Major
Status: Needs work » Needs review
Issue tags: +DX (Developer Experience)
StatusFileSize
new11.37 KB

I think the function is not needed in builder.

Here's a new patch:
1) removes usage and function
2) stores "threading" in #comment_threaded key of the entity build array - DX++ for contrib to not query field settings
3) removes "commented entity" cache hitting because in 99% there's only one commented entity otherwise threading makes no sense (or we should implement threading as separate render function)

rteijeiro’s picture

StatusFileSize
new11.32 KB
new3.61 KB

Removed commented line.

rteijeiro’s picture

StatusFileSize
new768 bytes

Forget that interdiff. This is the good one!

andypost’s picture

+++ b/core/modules/comment/src/CommentViewBuilder.php
@@ -65,7 +65,6 @@ public static function createInstance(ContainerInterface $container, EntityTypeI
-    /** @var \Drupal\comment\CommentInterface $entity */

This is a type-hint not a commented line, this allows to easily use interface defined methods

larowlan’s picture

+1 to #comment_threaded approach, patch at #22 looks good to go to me

wim leers’s picture

This is so much better! This no longer sets random properties on Entity objects just to pass information around; this now sets render array properties, as it should. Great work, thanks!

Only nitpicks:

  1. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -18,14 +24,54 @@
    +   * Constructs a new FeedViewBuilder.
    

    s/Feed/Comment/

  2. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -58,25 +104,31 @@ public function buildComponents(array &$build, array $entities, array $displays,
    +    // A counter that helps track how indented we are.
    

    s/that helps/to/
    s/how indented we are/the indentation level/

  3. +++ b/core/modules/comment/src/CommentViewBuilder.php
    @@ -58,25 +104,31 @@ public function buildComponents(array &$build, array $entities, array $displays,
    +          // or (negative) amount of divs to close to get to this comment's
    +          // indent level.
    

    The first line of this comment makes is clear, these 2 lines aren't.

andypost’s picture

StatusFileSize
new1.4 KB
new11.34 KB

fixed, 3 - is tricky to explain

wim leers’s picture

Status: Needs review » Reviewed & tested by the community

I trust the test coverage in \Drupal\comment\Tests\CommentThreadingTest, we significantly improved that test in #2254181: Comment indentation is incorrect for comments following a replying-comment: don't render cache comments for which threading is enabled.

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: +Needs change record

comment_prepare_thread() is D7 function. Also has anyone done the perf testing to prove that the beta evaluation is correct?

joelpittet’s picture

Issue tags: +Performance, +needs profiling

For instructions on how to profile if someone wants to give it a try, see the instructions here for the setup: https://www.drupal.org/contributor-tasks/profiling

andypost’s picture

Status: Needs work » Needs review

Re-roll, comment generation is broken so use the patch #2422101-7: CommentItem should override the generateSampleValue method and provide sample values to generate comments.

andypost’s picture

StatusFileSize
new11.36 KB
rteijeiro’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me! :)

andypost’s picture

Status: Reviewed & tested by the community » Needs work

It needs CR and performance testing
My measurements shows less IO but more cpu... somehow

fabianx’s picture

Status: Needs work » Needs review

+1 to the change, this depth thingy made render caching difficult as I could not rely on the comment objects being the same as when loaded from the DB.

But yes needs profiling ...

andypost’s picture

fabianx’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: -needs profiling

With render cache off, the performance is exactly the same, which is expected.

Due to the refactoring the function call counts remains funnily enough exactly the same and we have -0.0% wall time (0.0028 ms) - way within margin of error.

With render cache on:

Sum of 100 runs:

=== SUM: 8_0_x-summary..issue-2318579--comment-prepare=thread-summary compared (552492716a776..55249276b36e7):

ct  : 14,013,012|14,013,012|0|0.0%
wt  : 25,450,536|25,505,755|55,219|0.2%
mu  : 2,848,958,816|2,849,139,976|181,160|0.0%
pmu : 3,137,195,320|3,137,456,032|260,712|0.0%

---
ct = function calls, wt = wall time, cpu = cpu time used, mu = memory usage, pmu = peak memory usage

### XHPROF-LIB REPORT

+---------------------------------------+------------+------------+------------+------------+------------+
| namespace                             |        min |        max |       mean |     median |       95th |
+---------------------------------------+------------+------------+------------+------------+------------+
| Calls                                 |            |            |            |            |            |
|                                       |            |            |            |            |            |
| issue-2318579--comment-prepare=thread |    140,100 |    143,112 |    140,130 |    140,100 |    140,100 |
| 8_0_x                                 |    140,100 |    143,112 |    140,130 |    140,100 |    140,100 |
|                                       |            |            |            |            |            |
| Wall time                             |            |            |            |            |            |
|                                       |            |            |            |            |            |
| issue-2318579--comment-prepare=thread |    244,442 |    311,507 |    255,058 |    254,599 |    261,721 |
| 8_0_x                                 |    244,122 |    317,064 |    254,505 |    254,394 |    259,514 |
|                                       |            |            |            |            |            |
| Memory usage                          |            |            |            |            |            |
|                                       |            |            |            |            |            |
| issue-2318579--comment-prepare=thread | 28,483,320 | 28,904,792 | 28,491,400 | 28,483,320 | 28,488,272 |
| 8_0_x                                 | 28,481,640 | 28,902,544 | 28,489,588 | 28,481,864 | 28,482,049 |
|                                       |            |            |            |            |            |
| Peak memory usage                     |            |            |            |            |            |
|                                       |            |            |            |            |            |
| issue-2318579--comment-prepare=thread | 31,366,472 | 31,789,704 | 31,374,560 | 31,366,472 | 31,372,910 |
| 8_0_x                                 | 31,364,152 | 31,786,504 | 31,371,953 | 31,364,152 | 31,364,580 |
|                                       |            |            |            |            |            |
+---------------------------------------+------------+------------+------------+------------+------------+

Again the function calls remain the same, which means this is done before the #pre_render is called, which is fine as such that data is always correct.

--

Patch looks good too and removes setting arbitrary properties on objects.

=> RTBC

alexpott’s picture

Status: Reviewed & tested by the community » Needs work
Issue tags: -Performance +Needs issue summary update

So #39 shows that we're not getting a performance improvement. We need to update the beta evaluation and provide another reason why we should make this change.

larowlan’s picture

Is DX as indicated by @Fabianx at #39 a suitable reason?

fabianx’s picture

Issue summary: View changes
Status: Needs work » Reviewed & tested by the community
Issue tags: -Needs issue summary update

Back to RTBC.

The justification is that we should avoid dirtying entities, else they cannot be re-loaded lazily, which needlessly limits the usage of placeholders (as seen in #2469431-5: BigPipe for auth users: first send+render the cheap parts of the page, then the expensive parts).

While that issue is just in prototype stage, even now dirtying entities will make the job of contrib modules like my render_cache much harder and as the patch shows without any reason.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Okay @Fabianx - I agree that we should be avoiding dirty entities especially in the render pipeline. I think the disruption of removing comment_prepare_thread() is worth the improvement and we've shown that there is no effect on performance. Committed 418da36 and pushed to 8.0.x. Thanks!

  • alexpott committed 418da36 on 8.0.x
    Issue #2318579 by roderik, andypost, rteijeiro, ashutoshsngh, AjitS,...

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.