When sending newsletter mails, if the subscriber is a user, this module switches to the user account for rendering the issue entity. This allows the output to vary, for example some users might see extra fields because they have a related permission.
The disadvantage of this is that it reduces performance, because the entity cannot be cached. In some cases, Drupal knows that the rendered entity could vary per user, so won't cache it, but the site admin knows that in reality there is no noticeable variation. In this case it would be useful to have a setting to disable the user switching.
The situation is made much worse because of a core bug #2940605: Can only intentionally re-render an entity with references 20 times. The bug will be hit if the issue entity contains an entity reference, and there are more than 20 authenticated subscribers. The proposed setting would avoid hitting the bug.
Original summary
Hello There
We create a newsletter content. On this we have a reference field (referenced to the a Simple Page).
We render this page as a teaser.
Wenn we send the newsletter as default by cron, some of the 400 recipient get the newsletter rendered correct. Some of them only get the content of the newsletter node and not of the rendered simple page. We get the Following error:
Recursive rendering detected when rendering entity node: 49, using the field_newsletter_content field on the simplenews_issue bundle. Aborting rendering.
It is not a problem of the mail client. We also have sent the 400 newsletter to one singe account, who get the mail rendert correct.
Also on this account, when we send all 400 news we get some rendered incorrectly.
Any Ideas, why the newsletter is not correct on every recipient?
Thanks for ur help.
| Comment | File | Size | Author |
|---|---|---|---|
| #33 | add-per-user-rendering-option-in-send-mail-3081703-33.patch | 8.17 KB | bisonbleu |
| #27 | simplenews--hook_update_message.jpg | 258.47 KB | bisonbleu |
| #26 | simplenews--per-user-rendering.png | 440.86 KB | bisonbleu |
| #21 | 0001-Changes-in-issue-3081703.patch | 7.02 KB | aditiup |
| #9 | 3081703-newsletter-mass-mail-rendering-error.patch | 986 bytes | jrochate |
Issue fork simplenews-3081703
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
cola commentedWe found out the issue.
It appears if a user have a "Drupal User". For testing we comment out this lines and it was working. So the issue must be on rendering the element. Core bug or simplenews bug?
// Switch to the user
// if ($this->uid = $this->getSubscriber()->getUserId()) {
// \Drupal::service('account_switcher')->switchTo(User::load($this->uid));
// }
line 254 bis 257
file -> MailEntity.php
Comment #3
miro_dietiker@cola this is a feature. If a subscriber is represented by a user, the system tries to render the newsletter in the context of the recipient (account). You can reproduce this also by test-send to such a recipient or by usinf masquerade and viewing the newsletter node as the recipient user.
It seems your newsletter is rendering wrongly for such recipients. Maybe a bug of customization, not a bug of Simplenews / Core at all?
Do you even get the „Recursive rendering“ error message at all?
You can for instance reduce the mails created per batch run to deal with the accidentally triggered „recursion“ limit.
Comment #4
cola commented@miro_dietiker the problem is, we create only an entity reference on the CT Simple newsletter. This reference field is rendered by Core with the default display view. The idea behind is, that the content manager can select there i.e. some news (with reference field) and they will be rendered into the newsletter mail. We can reproduce it, it is arbitrarily and not always the same users. That happen also if we reduce the sending mails. So is no custom code from us or something else, must be the entity reference field.
Comment #5
miro_dietikerSo yeah, here's the (likely) explanation of this problem, and again it is a core bug / limitation, not a Simplenews bug:
See /core/lib/Drupal/Core/Field/Plugin/Field/FieldFormatter/EntityReferenceEntityFormatter.php
Recipients are users, so we render all nodes in context of the user by design. This allows you to vary these things per user, maybe personalize it.
Since it is the very same node in the same hierarchy, the "$recursive_render_id" is identical.
The "recursion" limit is global and not reset after each individual newsletter. There is no API to reset it.
So if a batch contains more than 20 authenticated users, you hit the "recursion" limit.
We have two options here:
a) offer a simplenews setting to disable user switching and render / deliver every recipient as anonymous, in fact rendering only once.
b) provide a core request to ask for a change and offer an api to reset this setting (such as how we reset statics in tests..), and trigger reset after each mail sent.
Maybe if cache contexts are properly set and the teaser is properly rendered (and you didn't accidentally destroy cachability with custom code or special feature usage), then the render cache would avoid hitting this recursion limit.
Comment #6
cola commentedok, thank you for answer. Your implementation makes sense, but the Core implementation is strange. Better is to have core functionality to reset it, but I don't think that this will be done. Or what do you recommend?
Comment #7
adamps commentedComment #8
jrochate commentedHi. IMHO it would be good to just have an option to disable user switching and render or auto-clear the counter.
I'm have the same problem and I was getting crazy until I saw this post :)
This post talks about the subject:
https://www.drupal.org/project/drupal/issues/2940605#comment-13244806
where they and clear static::$recursiveRenderDepth if user changes (happens on every new mail)
thanks.
Comment #9
jrochate commentedOne more thing:
the hack on #2 has to be complemented with commenting 3 lines on line nr 279 to avoid PHP notices.
here is a patch just to save this, but surely not a bright solution.
Comment #10
miro_dietikerYeah then let's connect the other issue as related.
I do agree that simply offering a global or per-newsletter (?) setting to disable user switching will solve this problem for many "simple" common use cases that are not based on per-user personalization...
Should we convert this issue into a feature request?
Comment #11
jrochate commentedThat would be awesome, thanks!
Comment #12
adamps commentedWe already have a very similar feature #3038794: Setting to enable token replacement in newsletter issue fields. I have updated it with extra info from this issue. Please post any patches in that issue and we can mark this support request as fixed.
Comment #14
adamps commentedOK in the end I didn't put this into #3038794: Setting to enable token replacement in newsletter issue fields because they seem to be different. Reopening this one as a feature request.
Comment #15
berdirUnsure about this. A setting has the risk of suddenly breaking something when you somehow end up putting something user-dependant in there. But I agree that many use cases have no need for this.
We have our own static cache that by default (\Drupal\simplenews\Mail\MailCacheBuild::isCacheable()) does not cache for authenticated users. there be a version that caches anyway, also as a custom override instead of having to patch simplenews, but that seems even worse than not switching. The use case for that caching service is to optimize it for specific use cases, for example your users might have a setting that can result in 3 variations of the newsletter, then you can cache by that instead of per-user. The default implementation is the safe but slow version.
So yeah, I guess a setting with clear warnings that newsletters must then not contain any per-user variations if that is off is OK.
Comment #16
adamps commentedThanks Berdir.
If the setting is enabled, then all entities would be rendered as the anonymous user. So it wouldn't exactly "break" anything - just the user-dependent material would be missing, which is exactly what the setting described would happen. Anyway I'm not going to write this one myself right now, so probably nothing will happen😃.
Comment #17
znerol commentedHit this problem as well. My current workaround: Replace the
simplenews.mail_cachewith a version which doesn't disable the cache for subscribers who are site users as well.modules/custom/custom_newsletter/src/Mail/MailCacheJust.phpmodules/custom/custom_newsletter/src/CustomNewsletterServiceProvider.phpComment #18
vlad.dancerThank you @znerol! Works perfectly!
Comment #19
adamps commentedIn my view #17 is not the complete solution.
The comment says
It's rather easy for the site builder to make a mistake and break this rule, which could be a security bug. So the remaining part of the fix is to disable the account switch in
MailEntity::setContext().Comment #21
aditiup commentedHere I propose a modified version of the issue solution
Add setting to disable per-user rendering(issue #3081703)
Changes that are proposed to be made:
1) Custom Mail Cache (MailCacheJust.php):
-Implementation of a custom mail cache class (MailCacheJust) that ensures caching of only necessary data and build information.
-This helps in maintaining consistency across newsletter deliveries, irrespective of whether subscribers have associated user accounts or not.
2)Custom Mail Entity (CustomMailEntity.php):
-Extention of the MailEntity class to disable account switching during entity rendering.
-This prevents issues related to recursive rendering errors, especially critical when rendering entities with references.
3)Custom Service Provider (CustomNewsletterServiceProvider.php):
-Creation of a service provider to replace Simplenews services with your custom implementations (MailCacheJust and CustomMailEntity).
-This ensures that Drupal uses your custom classes throughout the Simplenews module, improving performance and consistency.
4)Service Definitions (simplenews.services.yml):
-Definition of the services (MailCacheJust, CustomMailEntity, and CustomNewsletterServiceProvider) correctly in the service definitions.
-This registration ensures that Drupal’s dependency injection container correctly manages and uses your custom services.
Comment #22
adamps commentedThanks. Yes, that has the right idea, but it's missing one thing. The purpose of this issue is to add a setting. We could call it mail.per_user_rendering.
simplenews_install()that sets the value for existing sites.Comment #23
iainh commentedAt a content level I imagine an "always use cache for all users" setting so that a site builder can say
I understand one issue here is that there is a need to prevent publishing content that only certain recipents should see, but surely this is already sufficiently under the control of site builders who can specify different Display View Modes where fields can be omitted from display.
Or, if this is not sufficient then the Content's Publishing options could specify one Display View Mode for sensitive information and another for anonymous viewing ... gets complicated to implement I imagine.
Another poossibilty is a switch that could be set per field in the Content's Display View Mode in much the same way as a site builder can specify "Image Loading: eager" for a Media field so that you can say "Structure >> Content Types >> ContentName >> Manage Display >> Email:HTML >> FieldName >> cogwheel >> "IGNORE / MUST DO per-user rendering"
Comment #24
adamps commented@iainH This setting is only for rendering emails. It doesn't affect normal rendering of the entity as a web page.
Comment #25
iainh commentedYes, I understand. I think this would meet our needs.
Comment #26
bisonbleu commented@adamps I' working on this patch based on the requirements laid out in #22. I got the first 2 working. But I could use a bit more directions for
3 (disable the account switch: where? any examples?) and5 (add a test: to test what? any examples?). Thanks!Comment #27
bisonbleu commentedComment #28
bisonbleu commentedI'm done implementing the new
per_user_renderingsetting in MailEntity, Mailer, etc. I able to send a test withper_user_renderingenabled and disabled without errors.About the required test.
Since the new setting (Per user rendering) appears immediately below 'Use cron to send newsletters', I searched for occurrences of 'use_cron' and found 3 of them in
tests/src/Functional:Would adapting
testSendNowNoCron()into e.g.testSendPerUserRenderingDisabled()meet the requirements ?Comment #29
bisonbleu commentedUpdated and simplified after spending more time analysing simplenews/tests/src/Functional/SimplenewsSourceTest.php.
Comment #30
bisonbleu commentedThe following commits have been pushed to Issue fork simplenews-3081703
Please advise on next steps. Thanks
Comment #32
bisonbleu commentedHaving a hard time with the GitLab fork workflow : / And then, to make matters worst, and not knowing better, I used the issue’s fork without realizing that it was stale (created July 2024).
Until I'm able to untangled this mess, I'll upload a patch. Standby…
Comment #33
bisonbleu commentedHere is the patch…
Update:
composer installand it applied without issues;Comment #34
bisonbleu commentedSetting to Needs review…
Comment #35
bisonbleu commentedJust a quick update & some questions.
I was able to rebase the issue’s stale fork. The good news is MR !88 is mergeable.
But in Gitlab, on the MR’s page, I see:
Are all these fails directly related to this MR ? Or is there a backlog of fails that are carried over from other issues ?
If someone can mentor me through this, I'll find more time to fix what needs fixing.
Thanks!
Comment #36
bisonbleu commentedUPDATE:
The original patch in #33 changes these files:
/simplenews/config/install/simplenews.settings.yml
/simplenews/config/schema/simplenews.schema.yml
/simplenews/simplenews.install
/simplenews/src/Form/MailSettingsForm.php
/simplenews/src/Mail/MailEntity.php
/simplenews/src/Mail/Mailer.php
/simplenews/src/Spool/SpoolList.php
/simplenews/tests/src/Functional/SimplenewsSendTest.php
So I ran phpstan & phpcs on this list and fixed all the reported issues.
On the issue fork on GitLab, there are still many errors but it looks like most of them now relate to other files that I did not change?
This is starting to feel like a Pandora's box… Maybe I'm wasting my time here?
I'm not judging, but if I cannot get a bit of help - a friendly guide - then I'll have to assume I'm alone in a dead end.
I would be nice to find a way to close this. Turning back and moving on to something else is also an acceptable outcome.
Thanks
Comment #37
bisonbleu commentedThe latest fix (which also resolved a circular dependency) re-introduces a phpstan
\Drupalcall issue in SpoolStorage.php. Looks like creating a separate service that handles the immediate sending logic to break the circular dependency is the way to go… I'll come back to this later.Comment #38
adamps commentedThank you for your contribution. I'm afraid that I'm not going to respond right now. Contrib module maintainers do their work for free in their spare time, and the availability of time comes and it goes. Sometimes I have waited for over a year without any response on other modules. Even when the maintainer doesn't yet answer, the other issue followers might appreciate your work.😃😃
Comment #39
bisonbleu commentedThanks for your comment @adamps. I understand and, like I said, not judging. Synchronicity I guess ; )
Comment #41
adamps commentedComment #42
generalredneckAs a heads up, A lot of the changes here are actually DI based changes which is what makes this one hard to keep up. I recognize a lot of the work from the task I was doing over on #3613221: Pipelines warn about coding standards.
Reading back through the comments prior... looks like in #37 @bisonbleu even hit the same circular dependency I hit.
Based on the feedback from @adamps on my issue, I would say breaking out "Fixing the CI build" into it's own task as I did and just implementing the feature request as minimalist as possible here would probably yield more fruit.
I was working through the rebase but I'm hung up on the differences between what you've done and what I've done and I can't "quite" peel out exactly what needs to change. though it look like the majority of the feature is this commit here:
https://git.drupalcode.org/project/simplenews/-/merge_requests/88/diffs?... and the work you did on May 29 2025
Comment #43
adamps commentedYes exactly.
Also if you hit a separate bug such as circular dependency then please create a separate issue.
Large MR that do multiple things are not really committable - it's hard to review, hard to follow in the commit history, and hard to understand in a blame.