Problem/Motivation

Some of the form fields that are assigned in the email handler can contain special characters (single quote, double quote, less than, greater than).

Before sending the email the special characters get converted to HTML code. The preview however is fine.

Steps to reproduce

  • You can use the "Basic email contact webform" provided with webform.
  • Enter special characters in the Subject field and/or Your name field
  • Optional check the preview (needs to be enabled prior for this contact form)
  • Send/submit the form.
  • Check the site email account or the submissions.
CommentFileSizeAuthor
#23 screenshot inbox.png49.55 KBgeroldk
#22 webform.webform.contact_html_encoded.yml3.76 KBjrockowitz
#21 with #19.png127.72 KBgeroldk
#21 before #13 and #19.png122.75 KBgeroldk
#20 email_handler_field_s-2895140-19.patch18.16 KBjrockowitz
#16 email_handler_field_s-2895140-16.patch15.97 KBjrockowitz
#14 email_handler_field_s-2895140-13.patch4.05 KBjrockowitz
#11 webform.webform.contact_html_encoded.yml3.82 KBgeroldk
#9 4 Submission info.png47.2 KBgeroldk
#9 3 Debug info.png81.39 KBgeroldk
#9 2 Form preview.png64.4 KBgeroldk
#9 1 Form filled in.png58.82 KBgeroldk
#6 email_handler_field_s-2895140-6.patch4.85 KBjrockowitz
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

geroldk created an issue. See original summary.

jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Status: Active » Postponed (maintainer needs more info)

What email module are you using?

@see [#2893655}

geroldk’s picture

geroldk CreditAttribution: geroldk as a volunteer commented

I use Swiftmailer.

However, the message body is not affected by this and the special characters show up fine in the body (i.e. not in HTML code). I am using the latest dev that has the patch from @see #2893655: Swiftmailer stopped working applied.

Note that the HTML encoded special characters also show up under "results -> submissions". I am not sure, but this might be separate to emailing the form, indicating that it is potentially not a Swiftmailer related issue.

jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Priority: Normal » Critical
Status: Postponed (maintainer needs more info) » Active

I am able to replicate the issue by entering special characters in the default contact form's subject.

  • jrockowitz committed c0307e5 on 2895140-email-special-characters 
    Issue #2895140: Email handler field(s) convert special chars to HTML...
jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Status: Active » Needs review
FileSize
email_handler_field_s-2895140-6.patch4.85 KB

  • jrockowitz committed 50efd0a on 8.x-5.x 
    Issue #2895140 by jrockowitz: Email handler field(s) convert special...
jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Status: Needs review » Fixed

I committed the patch. Please download the latest release to review.

geroldk’s picture

geroldk CreditAttribution: geroldk as a volunteer commented
Status: Fixed » Active
FileSize
1 Form filled in.png58.82 KB
2 Form preview.png64.4 KB
3 Debug info.png81.39 KB
4 Submission info.png47.2 KB

That patch/latest dev fixes it for the "Subject" in the email, but that's not all.

I upload a sequence of screenshots showing the issue as a submission is created, previewed, submitted and viewed. Emails blanked from the images by me, used my own webform not the standard contact form. With reference to the screenshots uploaded:

1 Form filled in
Form filled in, all fields OK

2 Form preview
Form previewed, all fields OK

3 Debug info The debug info shown post submission:
First dot point: Subject not ok, From name not ok
Second dot point: Subject ok, From name not ok

4 Submission info The submission info as viewed in the results tab
From name not ok, To name not ok, Subject not ok

Email received (no screenshot)
Subject ok, From name not ok

jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented

Please export and upload an example of the broken form.

geroldk’s picture

geroldk CreditAttribution: geroldk as a volunteer commented
FileSize
webform.webform.contact_html_encoded.yml3.82 KB

Uploaded a webform config. Not exactly the one that I used to create the screenshots, but very similar and it exhibits the same behaviour.

If you want to give it a go, then enter To and From email addresses and send/submit.

  • jrockowitz committed b3619e6 on 2895140-email-special-characters 
    Issue #2895140 by jrockowitz, geroldk: Email handler field(s) convert...

  • jrockowitz committed 50efd0a on 2895140-email-special-characters 
    Issue #2895140 by jrockowitz: Email handler field(s) convert special...
jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Status: Active » Needs review
FileSize
email_handler_field_s-2895140-13.patch4.05 KB

This is getting tricky. The main issue is that most tokens assume the output is HTML which is why the HTML characters are being encoded. An immediate fix is to use [webform_submission:values:subject:raw] in the subject.

  • jrockowitz committed f8223d3 on 2895140-email-special-characters 
    Issue #2895140 by jrockowitz, geroldk: Email handler field(s) convert...
jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
FileSize
email_handler_field_s-2895140-16.patch15.97 KB

Status: Needs review » Needs work

The last submitted patch, 16: email_handler_field_s-2895140-16.patch, failed testing. View results

jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented

Wow Drupal is stripping HTML tags from the subject.

@see http://cgit.drupalcode.org/drupal/tree/core/lib/Drupal/Core/Mail/MailMan...

  • jrockowitz committed fdfd756 on 2895140-email-special-characters 
    Issue #2895140 by jrockowitz, geroldk: Email handler field(s) convert...
jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Status: Needs work » Needs review
FileSize
email_handler_field_s-2895140-19.patch18.16 KB
geroldk’s picture

geroldk CreditAttribution: geroldk as a volunteer commented
Status: Needs review » Active
FileSize
before #13 and #19.png122.75 KB
with #19.png127.72 KB

Installed the latest dev (dev-5.x 8f1114e).
Changed all fields to show raw for submissions. Submission display ok.

Applied email_handler_field_s-2895140-13.patch, which didn't work at all. White screen when trying to view the form. Reversed patch.
Applied email_handler_field_s-2895140-19.patch.

Prior to applying patch -19 : Subject ok, From name not ok, To name not ok (see screenshot before #13 and #19)
After applying patch -19 : Subject not ok, From name not ok, To name not ok (see screenshot with #19)

Debug info shows that "&" have changed to "&amp" for subject which seems to stop this from being sent correctly.
"&amp" is in both cases displayed (before and after applying the -19 patch) for From name.

Unfortunately one step backwards, as subject was ok and isn't anymore.

jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Status: Active » Needs review
FileSize
webform.webform.contact_html_encoded.yml3.76 KB

I think the issue is that you did not execute the database update which change the email tokens from [*:value] to [*:raw].

The attached form works as expected.

geroldk’s picture

geroldk CreditAttribution: geroldk as a volunteer commented
FileSize
screenshot inbox.png49.55 KB

Thanks for sending through the form to test. It shows the same behaviour as the one that I have.

I have uploaded a screenshot of what it looks like when I receive the submission by email into my inbox. The sender name / reply to name are cut off where the special characters are and only the last portion of regular characters is displayed.

I guess I wasn't clear enough in the first place and got sidetracked by what the debug information is displaying, should have uploaded this screenshot a lot earlier.

Not sure if this is webform related or if this is a general thing that email "sender names" can't contain special chars.

geroldk’s picture

geroldk CreditAttribution: geroldk as a volunteer commented
Status: Needs review » Active
jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented

I think the new issue is related to #2896055: From header containing UTF-8 characters appears scrambled in emails.

The patch fixes a major issue and has test coverage. I am going to commit it and we should start working on #2896055: From header containing UTF-8 characters appears scrambled in emails.

  • jrockowitz committed 8bf1eaf on 8.x-5.x 
    Issue #2895140 by jrockowitz, geroldk: Email handler field(s) convert...
jrockowitz’s picture

jrockowitz CreditAttribution: jrockowitz at The Big Blue House commented
Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.