Problem/Motivation

After upgrading to Drupal 10.3.2 we found emails were sometimes being sent with an extra space character in the title.

This is related to the change in #3451611: Fix the format=flowed; delsp=yes encoding of email messages

Steps to reproduce

Send an email with a plain text subject longer than 80 characters

Proposed resolution

Adjust the cleanup logic in the MimeMail plugin that is calling MailFormatHelper::htmlToText($subject)

Should this logic move to MimeMailFormatHelper? That will make it easier to add tests.

Remaining tasks

  1. Make changes
  2. Add test coverage

User interface changes

n/a

API changes

n/a

Data model changes

n/a

Issue fork mimemail-3467705

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

ericgsmith created an issue. See original summary.

tr’s picture

Mmm, I don't think you've diagnosed your problem correctly. Maybe you can give a concrete example of what you're doing and what the resulting raw email (including headers) looks like so I can see what's going on?

I am 99% sure whatever it is you are seeing is because of PHP line endings in email, which vary by OS and which vary by PHP version (and was changed recently - in PHP 8.1?), and which Drupal tries to coerce into being \n all the time for email even though the RFC requires \r\n. This is a long-standing issue with PHP and which Drupal has tried to work-around in a way that is not compliant with the RFC.

I have not tried to follow understand everything in the core issue you linked, but it seems to be the problem is that Drupal should NOT be setting a Content-Type: of "format-flowed" to emails. I guess because Drupal only does non-HTML email, this is a hack to make that non-HTML email look better when displayed in an HTML-capable email client. But my argument would be that Drupal should either do only plain text email or it should be full-in on Mime email - trying to get some of the advantages of full Mime email without fully implementing Mime email is bound to cause problems. In this case, it seems core wants the "line breaks ignored" feature from HTML without having to deal with HTML, and to do so it uses a Content-Type' => 'text/plain; charset=UTF-8; format=flowed; delsp=yes.

Regardless, if the core htmlToText() is replacing \r\n with a space and \n, then I would characterize that as a core bug, and I would describe the way you propose to fix it as a hack at best which has greater potential to do the wrong thing that to fix any bug. Also, be aware the Mime Mail module REPLACES the core Content-Type: for Mime emails, so "format-flowed" is not used at all for messages sent by Mime Mail.

When headers such as the Subject: header in an email message exceed the maximum length allowed by the RFC (which is 998 characters), they are SUPPOSED to wrap the line using "\r\n " (\r\n followed by a space) in order to allow the email client to recognize the wrapped header. And headers less than that length are explicitly allowed to be wrapped as many times as desired.

Here's what I did to test your scenario. I don't see a problem with long subject lines with embedded CRLF. Show me what I did wrong and provide a script or detailed instructions I can use to reproduce your problem.

$ cat aren.php

<?php

use Drupal\Core\Mail\MailFormatHelper;

$crlf_subject = "This is a subject line with CRLF line\r\nbreaks. Additional text to make this subject line very very very long.";
// From src/Plugin/Mail/MimeMail.php
$subject = str_replace(["\n"], '', trim(MailFormatHelper::htmlToText($crlf_subject)));
print_r($subject);

$ drush php-script aren.php | od -A x -t x1z -v

000000 54 68 69 73 20 69 73 20 61 20 73 75 62 6a 65 63  >This is a subjec<
000010 74 20 6c 69 6e 65 20 77 69 74 68 20 43 52 4c 46  >t line with CRLF<
000020 20 6c 69 6e 65 62 72 65 61 6b 73 2e 20 41 64 64  > linebreaks. Add<
000030 69 74 69 6f 6e 61 6c 20 74 65 78 74 20 74 6f 20  >itional text to <
000040 6d 61 6b 65 20 74 68 69 73 20 73 75 62 6a 65 63  >make this subjec<
000050 74 20 6c 69 6e 65 20 76 65 72 79 20 76 65 72 79  >t line very very<
000060 20 76 65 72 79 20 6c 6f 6e 67 2e                 > very long.<
00006b

I have also tried removing the CRLF from the $crlf_subject, with the same result.

ericgsmith’s picture

Apologies for not providing a test case - there are no line breaks in the original subject.

$ cat test.php

<?php

use Drupal\Core\Mail\MailFormatHelper;

$original_subject = "[Website name] Account details for Eric Smith at website name (pending admin approval)";
// From src/Plugin/Mail/MimeMail.php
$subject = str_replace(["\n"], '', trim(MailFormatHelper::htmlToText($original_subject)));
print_r($subject);

drush php:script test.php
[Website name] Account details for Eric Smith at website name (pending admin approval)

Note the two spaces instead of one between admin and approval.

tr’s picture

Thanks I will look at it in the next few days.

tr’s picture

Status: Active » Postponed (maintainer needs more info)

When I run your test case from #4 I get *no* extra spaces or line feeds or carriage returns in the subject line.

I am testing with Core 10.3.2-dev, PHP 8.1.2.

It's quite possible what you're seeing is because of a different version of PHP or Core.

mark_fullmer’s picture

Status: Postponed (maintainer needs more info) » Fixed

Testing the original test in #4, I am able to reproduce the problem with Drupal 11.2.x and PHP 8.3. I also confirmed that the cause is not due to anything special in the text or encoding in #4, by using a different string that I manually entered:

<?php

use Drupal\Core\Mail\MailFormatHelper;

$original_subject = "word word word word word word word word word word word word word word word word word word word word word ";
// From src/Plugin/Mail/MimeMail.php
$subject = str_replace(["\n"], '', trim(MailFormatHelper::htmlToText($original_subject)));
print_r($subject);

Given that this code is used for formatting the subject line, I wonder whether using MailFormatHelper is overkill, which is the very thing that adds the newlines that the custom code is then trying to remove. Is there any reason why we can't instead use PlainTextOutput::renderFromHtml?

<?php

use Drupal\Component\Render\PlainTextOutput;

$original_subject = "[Website name] Account details for Eric Smith at website name (pending admin approval)";
$subject = trim(PlainTextOutput::renderFromHtml($original_subject));
print_r($subject);

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

mark_fullmer’s picture

Status: Fixed » Active