When sending newsletter mails, if the subscriber is a user, this module switches to the user account for rendering the issue entity. This allows the output to vary, for example some users might see extra fields because they have a related permission.

The disadvantage of this is that it reduces performance, because the entity cannot be cached. In some cases, Drupal knows that the rendered entity could vary per user, so won't cache it, but the site admin knows that in reality there is no noticeable variation. In this case it would be useful to have a setting to disable the user switching.

The situation is made much worse because of a core bug #2940605: Can only intentionally re-render an entity with references 20 times. The bug will be hit if the issue entity contains an entity reference, and there are more than 20 authenticated subscribers. The proposed setting would avoid hitting the bug.

Original summary

Hello There

We create a newsletter content. On this we have a reference field (referenced to the a Simple Page).
We render this page as a teaser.

Wenn we send the newsletter as default by cron, some of the 400 recipient get the newsletter rendered correct. Some of them only get the content of the newsletter node and not of the rendered simple page. We get the Following error:
Recursive rendering detected when rendering entity node: 49, using the field_newsletter_content field on the simplenews_issue bundle. Aborting rendering.

It is not a problem of the mail client. We also have sent the 400 newsletter to one singe account, who get the mail rendert correct.
Also on this account, when we send all 400 news we get some rendered incorrectly.

Any Ideas, why the newsletter is not correct on every recipient?

Thanks for ur help.

Issue fork simplenews-3081703

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

sistocarta created an issue. See original summary.

cola’s picture

We found out the issue.
It appears if a user have a "Drupal User". For testing we comment out this lines and it was working. So the issue must be on rendering the element. Core bug or simplenews bug?

// Switch to the user
// if ($this->uid = $this->getSubscriber()->getUserId()) {
// \Drupal::service('account_switcher')->switchTo(User::load($this->uid));
// }

line 254 bis 257
file -> MailEntity.php

miro_dietiker’s picture

@cola this is a feature. If a subscriber is represented by a user, the system tries to render the newsletter in the context of the recipient (account). You can reproduce this also by test-send to such a recipient or by usinf masquerade and viewing the newsletter node as the recipient user.

It seems your newsletter is rendering wrongly for such recipients. Maybe a bug of customization, not a bug of Simplenews / Core at all?

Do you even get the „Recursive rendering“ error message at all?
You can for instance reduce the mails created per batch run to deal with the accidentally triggered „recursion“ limit.

cola’s picture

@miro_dietiker the problem is, we create only an entity reference on the CT Simple newsletter. This reference field is rendered by Core with the default display view. The idea behind is, that the content manager can select there i.e. some news (with reference field) and they will be rendered into the newsletter mail. We can reproduce it, it is arbitrarily and not always the same users. That happen also if we reduce the sending mails. So is no custom code from us or something else, must be the entity reference field.

miro_dietiker’s picture

So yeah, here's the (likely) explanation of this problem, and again it is a core bug / limitation, not a Simplenews bug:

See /core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/EntityReferenceEntityFormatter.php

  const RECURSIVE_RENDER_LIMIT = 20;
...
      // Protect ourselves from recursive rendering.
      if (static::$recursiveRenderDepth[$recursive_render_id] > static::RECURSIVE_RENDER_LIMIT) {
        $this->loggerFactory->get('entity')->error('Recursive rendering detected when rendering entity %entity_type: %entity_id, using the %field_name field on the %bundle_name bundle. Aborting rendering.', [

Recipients are users, so we render all nodes in context of the user by design. This allows you to vary these things per user, maybe personalize it.

Since it is the very same node in the same hierarchy, the "$recursive_render_id" is identical.
The "recursion" limit is global and not reset after each individual newsletter. There is no API to reset it.
So if a batch contains more than 20 authenticated users, you hit the "recursion" limit.

We have two options here:
a) offer a simplenews setting to disable user switching and render / deliver every recipient as anonymous, in fact rendering only once.
b) provide a core request to ask for a change and offer an api to reset this setting (such as how we reset statics in tests..), and trigger reset after each mail sent.

Maybe if cache contexts are properly set and the teaser is properly rendered (and you didn't accidentally destroy cachability with custom code or special feature usage), then the render cache would avoid hitting this recursion limit.

cola’s picture

ok, thank you for answer. Your implementation makes sense, but the Core implementation is strange. Better is to have core functionality to reset it, but I don't think that this will be done. Or what do you recommend?

adamps’s picture

Category: Bug report » Support request
Priority: Critical » Normal
jrochate’s picture

Hi. IMHO it would be good to just have an option to disable user switching and render or auto-clear the counter.

I'm have the same problem and I was getting crazy until I saw this post :)

This post talks about the subject:

https://www.drupal.org/project/drupal/issues/2940605#comment-13244806

where they and clear static::$recursiveRenderDepth if user changes (happens on every new mail)

thanks.

jrochate’s picture

One more thing:

the hack on #2 has to be complemented with commenting 3 lines on line nr 279 to avoid PHP notices.

here is a patch just to save this, but surely not a bright solution.

miro_dietiker’s picture

Yeah then let's connect the other issue as related.

I do agree that simply offering a global or per-newsletter (?) setting to disable user switching will solve this problem for many "simple" common use cases that are not based on per-user personalization...

Should we convert this issue into a feature request?

jrochate’s picture

That would be awesome, thanks!

adamps’s picture

Assigned: sistocarta » Unassigned
Status: Active » Fixed

We already have a very similar feature #3038794: Setting to enable token replacement in newsletter issue fields. I have updated it with extra info from this issue. Please post any patches in that issue and we can mark this support request as fixed.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

adamps’s picture

Title: Newsletter Mass Mail Rendering Error » Add setting to disable per-user rendering
Version: 8.x-2.x-dev » 3.x-dev
Category: Support request » Feature request
Issue summary: View changes
Status: Closed (fixed) » Active

OK in the end I didn't put this into #3038794: Setting to enable token replacement in newsletter issue fields because they seem to be different. Reopening this one as a feature request.

berdir’s picture

Unsure about this. A setting has the risk of suddenly breaking something when you somehow end up putting something user-dependant in there. But I agree that many use cases have no need for this.

We have our own static cache that by default (\Drupal\simplenews\Mail\MailCacheBuild::isCacheable()) does not cache for authenticated users. there be a version that caches anyway, also as a custom override instead of having to patch simplenews, but that seems even worse than not switching. The use case for that caching service is to optimize it for specific use cases, for example your users might have a setting that can result in 3 variations of the newsletter, then you can cache by that instead of per-user. The default implementation is the safe but slow version.

So yeah, I guess a setting with clear warnings that newsletters must then not contain any per-user variations if that is off is OK.

adamps’s picture

Thanks Berdir.

A setting has the risk of suddenly breaking something when you somehow end up putting something user-dependant in there.

If the setting is enabled, then all entities would be rendered as the anonymous user. So it wouldn't exactly "break" anything - just the user-dependent material would be missing, which is exactly what the setting described would happen. Anyway I'm not going to write this one myself right now, so probably nothing will happen😃.

znerol’s picture

Hit this problem as well. My current workaround: Replace the simplenews.mail_cache with a version which doesn't disable the cache for subscribers who are site users as well.

modules/custom/custom_newsletter/src/Mail/MailCacheJust.php

namespace Drupal\custom_newsletter\Mail;

use Drupal\simplenews\Mail\MailCacheStatic;
use Drupal\simplenews\Mail\MailInterface;

/**
 * Source cache implementation that caches build and data element.
 *
 * This implementation does not discriminate between subscribers with associated
 * user account and subscribers without.
 *
 * Site builders need to ensure that newsletter content is rendered identical
 * for all users before enabling this cache backend.
 *
 * @see \Drupal\simplenews\Mail\MailCacheBuild
 *
 * @ingroup mail
 */
class MailCacheJust extends MailCacheStatic {

  /**
   * {@inheritdoc}
   */
  public function isCacheable(MailInterface $mail, $group, $key) {
    // Only cache data and build information.
    return in_array($group, ['data', 'build']);
  }

}

modules/custom/custom_newsletter/src/CustomNewsletterServiceProvider.php


namespace Drupal\custom_newsletter;

use Drupal\Core\DependencyInjection\ContainerBuilder;
use Drupal\Core\DependencyInjection\ServiceProviderBase;
use Drupal\simplenews\Mail\MailCacheBuild;

/**
 * Replace simplenews.mail_cache with an implementiation which doesn't disable
 * the cache for subscribers who are site users as well.
 *
 * @see \Drupal\custom_newsletter\Mail\MailCacheJust
 */
class CustomNewsletterServiceProvider extends ServiceProviderBase {

  /**
   * {@inheritdoc}
   */
  public function alter(ContainerBuilder $container) {
    if (
      $container->hasDefinition('simplenews.mail_cache') &&
      is_a(
        $container->getDefinition('simplenews.mail_cache')->getClass(),
        MailCacheBuild::class,
        TRUE
      )
    ) {
      $container->getDefinition('simplenews.mail_cache')
        ->setClass('Drupal\custom_newsletter\Mail\MailCacheJust');
    }
  }

}
vlad.dancer’s picture

Thank you @znerol! Works perfectly!

adamps’s picture

In my view #17 is not the complete solution.

The comment says

Site builders need to ensure that newsletter content is rendered identical for all users before enabling this cache backend.

It's rather easy for the site builder to make a mistake and break this rule, which could be a security bug. So the remaining part of the fix is to disable the account switch in MailEntity::setContext().

Aditiup made their first commit to this issue’s fork.

aditiup’s picture

StatusFileSize
new7.02 KB

Here I propose a modified version of the issue solution
Add setting to disable per-user rendering(issue #3081703)

Changes that are proposed to be made:

1) Custom Mail Cache (MailCacheJust.php):
-Implementation of a custom mail cache class (MailCacheJust) that ensures caching of only necessary data and build information.
-This helps in maintaining consistency across newsletter deliveries, irrespective of whether subscribers have associated user accounts or not.

2)Custom Mail Entity (CustomMailEntity.php):

-Extention of the MailEntity class to disable account switching during entity rendering.
-This prevents issues related to recursive rendering errors, especially critical when rendering entities with references.

3)Custom Service Provider (CustomNewsletterServiceProvider.php):

-Creation of a service provider to replace Simplenews services with your custom implementations (MailCacheJust and CustomMailEntity).
-This ensures that Drupal uses your custom classes throughout the Simplenews module, improving performance and consistency.

4)Service Definitions (simplenews.services.yml):

-Definition of the services (MailCacheJust, CustomMailEntity, and CustomNewsletterServiceProvider) correctly in the service definitions.
-This registration ensures that Drupal’s dependency injection container correctly manages and uses your custom services.

adamps’s picture

Status: Active » Needs work

Thanks. Yes, that has the right idea, but it's missing one thing. The purpose of this issue is to add a setting. We could call it mail.per_user_rendering.

  • Add code for the setting to simplenews.settings.yml, simplenews.schema.yml, MailSettingsForm.php. For backward-compatibility the default should be TRUE.
  • Create an update hook in simplenews_install() that sets the value for existing sites.
  • Instead of disabling the switch, we need to switch to the anonymous user - see comments #15, #16.
  • MailCacheBuild and MailEntity can read this setting (so we don't need CustomMailEntity and MailCacheJust).
  • Add a test.
iainh’s picture

At a content level I imagine an "always use cache for all users" setting so that a site builder can say

Structure >> Content Types >> ContentName >> Edit >> Publishing options >> Use as Simplenews newsletter AND IGNORE per-user rendering

I understand one issue here is that there is a need to prevent publishing content that only certain recipents should see, but surely this is already sufficiently under the control of site builders who can specify different Display View Modes where fields can be omitted from display.
Or, if this is not sufficient then the Content's Publishing options could specify one Display View Mode for sensitive information and another for anonymous viewing ... gets complicated to implement I imagine.

Another poossibilty is a switch that could be set per field in the Content's Display View Mode in much the same way as a site builder can specify "Image Loading: eager" for a Media field so that you can say "Structure >> Content Types >> ContentName >> Manage Display >> Email:HTML >> FieldName >> cogwheel >> "IGNORE / MUST DO per-user rendering"

adamps’s picture

Version: 3.x-dev » 4.x-dev

@iainH This setting is only for rendering emails. It doesn't affect normal rendering of the entity as a web page.

iainh’s picture

Yes, I understand. I think this would meet our needs.

bisonbleu’s picture

StatusFileSize
new440.86 KB

@adamps I' working on this patch based on the requirements laid out in #22. I got the first 2 working. But I could use a bit more directions for 3 (disable the account switch: where? any examples?) and 5 (add a test: to test what? any examples?). Thanks!

bisonbleu’s picture

StatusFileSize
new258.47 KB

bisonbleu’s picture

I'm done implementing the new per_user_rendering setting in MailEntity, Mailer, etc. I able to send a test with per_user_rendering enabled and disabled without errors.

About the required test.

Since the new setting (Per user rendering) appears immediately below 'Use cron to send newsletters', I searched for occurrences of 'use_cron' and found 3 of them in tests/src/Functional:

  1. testSendNowNoCron()
  2. testSendMultipleNoCron()
  3. testSendPublishNoCron()

Would adapting testSendNowNoCron() into e.g. testSendPerUserRenderingDisabled() meet the requirements ?

bisonbleu’s picture

Updated and simplified after spending more time analysing simplenews/tests/src/Functional/SimplenewsSourceTest.php.

/**
 * Test newsletter sending with per user rendering disabled.
 */
public function testSendPerUserRenderingDisabled() {
  $this->setUpSubscribers(3);
  
  // Disable per user rendering
  $config = $this->config('simplenews.settings');
  $config->set('mail.per_user_rendering', FALSE);
  $config->save();

  // Create and send newsletter
  $edit = [
    'title[0][value]' => 'Test Newsletter',
    'body[0][value]' => "Hello [current-user:account-name], site: [site:mail]",
    'simplenews_issue[target_id]' => 'default',
  ];
  $this->drupalGet('node/add/simplenews_issue');
  $this->submitForm($edit, 'Save');
  $this->assertEquals(1, preg_match('|node/(\d+)$|', $this->getUrl(), $matches), 'Node created');
  $node = Node::load($matches[1]);

  $this->clickLink(t('Newsletter'));
  $this->submitForm([], 'Send now');

  // Verify mails were sent successfully (main goal: no breakage)
  $mails = $this->getMails();
  $this->assertCount(3, $mails, 'All mails were sent with per user rendering disabled.');

  // Should we check that user-specific tokens are not replaced ?
}
bisonbleu’s picture

The following commits have been pushed to Issue fork simplenews-3081703

  • Add 'Per user rendering' option in settings and hook_update for backward compatibility.
  • Implement switch to anonymous user when 'Per user rendering' is enabled.
  • Update MailEntity instantiation.
  • Add testSendPerUserRenderingDisabled().

Please advise on next steps. Thanks

bisonbleu’s picture

Having a hard time with the GitLab fork workflow : / And then, to make matters worst, and not knowing better, I used the issue’s fork without realizing that it was stale (created July 2024).

Until I'm able to untangled this mess, I'll upload a patch. Standby…

bisonbleu’s picture

Here is the patch…

Update:

  • I added this patch in my composer.json and ran composer install and it applied without issues;
  • At this stage (before Run updates) the Per user rendering option is disabled as expected;
  • After Run updates, the Per user rendering option is enabled, also expected;
  • Created a dummy newsletter and sent a test that was captured in Mailpit;
  • I see no errors in the watchdog;
bisonbleu’s picture

Assigned: Unassigned » bisonbleu
Status: Needs work » Needs review

Setting to Needs review…

bisonbleu’s picture

Just a quick update & some questions.

I was able to rebase the issue’s stale fork. The good news is MR !88 is mergeable.

But in Gitlab, on the MR’s page, I see:

  • Test summary: 144 failed, 242 total tests
  • 78 out of 144 failed tests have failed more than once in the last 14 days

Are all these fails directly related to this MR ? Or is there a backlog of fails that are carried over from other issues ?

If someone can mentor me through this, I'll find more time to fix what needs fixing.

Thanks!

bisonbleu’s picture

UPDATE:

The original patch in #33 changes these files:
/simplenews/config/install/simplenews.settings.yml
/simplenews/config/schema/simplenews.schema.yml
/simplenews/simplenews.install
/simplenews/src/Form/MailSettingsForm.php
/simplenews/src/Mail/MailEntity.php
/simplenews/src/Mail/Mailer.php
/simplenews/src/Spool/SpoolList.php
/simplenews/tests/src/Functional/SimplenewsSendTest.php

So I ran phpstan & phpcs on this list and fixed all the reported issues.

On the issue fork on GitLab, there are still many errors but it looks like most of them now relate to other files that I did not change?

This is starting to feel like a Pandora's box… Maybe I'm wasting my time here?
I'm not judging, but if I cannot get a bit of help - a friendly guide - then I'll have to assume I'm alone in a dead end.

I would be nice to find a way to close this. Turning back and moving on to something else is also an acceptable outcome.

Thanks

bisonbleu’s picture

The latest fix (which also resolved a circular dependency) re-introduces a phpstan \Drupal call issue in SpoolStorage.php. Looks like creating a separate service that handles the immediate sending logic to break the circular dependency is the way to go… I'll come back to this later.

adamps’s picture

Thank you for your contribution. I'm afraid that I'm not going to respond right now. Contrib module maintainers do their work for free in their spare time, and the availability of time comes and it goes. Sometimes I have waited for over a year without any response on other modules. Even when the maintainer doesn't yet answer, the other issue followers might appreciate your work.😃😃

bisonbleu’s picture

Thanks for your comment @adamps. I understand and, like I said, not judging. Synchronicity I guess ; )

adamps changed the visibility of the branch 4.x to hidden.

adamps’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll
generalredneck’s picture

As a heads up, A lot of the changes here are actually DI based changes which is what makes this one hard to keep up. I recognize a lot of the work from the task I was doing over on #3613221: Pipelines warn about coding standards.

Reading back through the comments prior... looks like in #37 @bisonbleu even hit the same circular dependency I hit.

Based on the feedback from @adamps on my issue, I would say breaking out "Fixing the CI build" into it's own task as I did and just implementing the feature request as minimalist as possible here would probably yield more fruit.

I was working through the rebase but I'm hung up on the differences between what you've done and what I've done and I can't "quite" peel out exactly what needs to change. though it look like the majority of the feature is this commit here:
https://git.drupalcode.org/project/simplenews/-/merge_requests/88/diffs?... and the work you did on May 29 2025

adamps’s picture

breaking out "Fixing the CI build" into it's own task as I did and just implementing the feature request as minimalist as possible here

Yes exactly.

Also if you hit a separate bug such as circular dependency then please create a separate issue.

Large MR that do multiple things are not really committable - it's hard to review, hard to follow in the commit history, and hard to understand in a blame.