Add an alternative to hook_mail() and deprecate it

Might be out of scope for this issue, but Symfony's Swiftmailer integration is slicktastic.

Since hook_mail() was introduced primarily for multilingual reasons (and to have parity with hook_mail_alter, but we don't have that for moth other alters anyway), marking with D8MI tag to make up for our earlier wrongs in D8 :)

Current ability to translate messages (message templates) is ugly. I think subject abd body should have translation oout of locale probably in own namespace

Oh, yes PLEASE. :)

Here is an initial patch and I need people to take this from here. It was just a quick weekend attack :)

- hook_mail() is gone
- drupal_mail() now gets an array with a structure equivalent to hook_mail_alter() (but obviously drupal_mail() adds lots of data to the array)
- as per @catch's suggestion above, $module and $key are now not separate, there is a single 'id'
- the 'send' argument became part of the message array, can now be altered (not sure of the usefulness, but anyway, it unified the structure)
- update.module is converted
- system.module (action) is converted

TODO (someone should take it from here :):
- Review the new API, I tried to make the array the same as before, so existing hook_mail_alter() implementations would still work; why break there when we don't need to?
- Check the phpdoc updates, unify the array descriptions in drupal_mail() and hook_mail_alter() and/or reference one from the other.
- Move over contact module and user module emails
- Happiness.

- Removed user_mail() hook and moved _user_mail_notify() to the new drupal_mail().
- Fixed mail.test

Moving contact in the next patch, just wanted to use the power of testbots since my machine is very slow :)

+++ b/core/includes/mail.inc
@@ -128,26 +121,19 @@ function drupal_mail($module, $key, $to, $language, $params = array(), $from = N
-  $message['headers'] = $headers;
-
-  // Build the e-mail (get subject and body, allow additional headers) by
-  // invoking hook_mail() on this module. We cannot use module_invoke() as
-  // we need to have $message by reference in hook_mail().
-  if (function_exists($function = $module . '_mail')) {
-    $function($key, $message, $params);
-  }
+  $message['headers'] += $headers;

Why replacing = with += ? this results in fatal error, since $message['header'] is never initiated before.

Powered by Dreditor.

Very nice, the previous patch only failed in Contact's tests which is very expected.

This newer version moved the Contact module to the new drupal_mail(). Hopefully now we get green test results.

This patch needs many cleanups, but first just trying to get it working.

Tested locally, should work.

Why replacing = with += ? this results in fatal error, since $message['header'] is never initiated before.

If you use = instead of +=, people calling this function will not be able to pre-fill any of the headers, all those would just be overwritten. That would be loss of functionality (the hook_mail() function previously could overwrite any headers). Instead, we should initialize the 'headers' key of $message above to an empty array. That initialization itself uses += so existing headers would be passed along.

(Leaving needs review for test bot feedback).

@Gabor: I just retest the += thing, it returned a Notice not Fatal Error. Anyhow, we should check it first (isset()) then use it, something like this:

  $message['headers'] = isset($message['headers']) ? $message['headers'] : array();
  $message['headers'] += $headers;

Updated patch with this check, Any other thoughts on it? Also, some discussion about this new API from Gabor will definitely help.

No, a great side effect of using += is that you don't need to use isset!

+++ b/core/includes/mail.incundefined
@@ -45,74 +42,70 @@ define('MAIL_LINE_ENDINGS', isset($_SERVER['WINDIR']) || strpos($_SERVER['SERVER
+  $message += array(
+    'from'    => $default_from,
+    'subject' => '',
+    'body'    => array(),
+    'params'  => array(),
+    'send'    => TRUE,

Put a 'headers' => array() pieces here.

+++ b/core/includes/mail.incundefined
@@ -125,29 +118,23 @@ function drupal_mail($module, $key, $to, $language, $params = array(), $from = N
+  $message['headers'] = isset($message['headers']) ? $message['headers'] : array();

Get rid of this.

+++ b/core/includes/mail.incundefined
@@ -125,29 +118,23 @@ function drupal_mail($module, $key, $to, $language, $params = array(), $from = N
+  $message['headers'] += $headers;

Now this will work happily :)

9 days to next Drupal core point release.

Included Gabor's review #16.

First of all, even if hook_mail() might have been introduced primarily for multilingual/translation reasons originally, it's primary effect and benefit is re-usability of mail texts today.

This first and foremost affects the mail texts of generic hook_mail() implementations like the one of User module. E.g., Drupal's user registration and related flows are sufficiently abstracted so that you can implement an alternative design, but you can still use the existing mail texts (including administrative interface), because they are not hard-coded inline in some functions.

When removing hook_mail(), we should clarify in the phpDoc of drupal_mail() that generic modules should consider to make their mail texts available in a helper function, and point to User module as an example.

+++ b/core/includes/mail.inc
@@ -45,74 +42,71 @@ define('MAIL_LINE_ENDINGS', isset($_SERVER['WINDIR']) || strpos($_SERVER['SERVER
+ * @param $message
+ *   An array with the following keys:
+ *     - 'id': A unique identifier for the email text being sent, primarily

List of keys is indented one level too much.

+++ b/core/includes/mail.inc
@@ -45,74 +42,71 @@ define('MAIL_LINE_ENDINGS', isset($_SERVER['WINDIR']) || strpos($_SERVER['SERVER
+ *     - 'send' (optional): Boolen value indicating whether the email is sent or
+ *     just built. Defaults to TRUE. If set to FALSE, the email can be later
+ *     sent with drupal_mail_system()->mail().
...
-function drupal_mail($module, $key, $to, $language, $params = array(), $from = NULL, $send = TRUE) {
+function drupal_mail($message) {

The $send argument and 'send' key on the message do not always have the identical use-case.

The key on the message may be set or changed by modules implementing hook_mail_alter() to signify that a mail should not be sent.

However, the $send argument may be set to FALSE ahead of time by the calling code. It means that the calling code invokes drupal_mail() to format the message only, and that the calling code is going to send out the message on its own (possibly using a custom delivery mechanism). In that case, sending is always FALSE, and hook_mail_alter() implementations can set to FALSE as much as they like, but it's going to be sent anyway. Unless there's an indicator the calling code could check.

See also #800434: drupal_mail, allow hook_mail_alter implementation to cancel mail

+++ b/core/includes/mail.inc
@@ -125,29 +119,22 @@ function drupal_mail($module, $key, $to, $language, $params = array(), $from = N
-  $system = drupal_mail_system($module, $key);
+  $system = drupal_mail_system($message['id']);

I want to veto the removal of 'module' and 'key'.

An 'id' is very ambiguous and doesn't clarify or enforce what its value should be.

Furthermore, explicit access to 'module' allows to assign a particular mail system for all mails being sent from a specific module.

> it's primary effect and benefit is re-usability of mail texts today.

> When removing hook_mail(), we should clarify in the phpDoc of drupal_mail() that generic modules should consider to make their mail texts available in a helper function, and point to User module as an example.

I agree to both points.
But then the logical conclusion is that we'd have the same system as we currently do, but without the formality of having it defined as hook_mail is currently.

So contrib modules may or may not follow the pattern or may do their own thing... how is this an improvement?

I agree that hook_mail() is weird and hard to grok.
But if you think about the effect it has, it's actually quite simple:

- define a list of emails that this module knows how to send
- use drupal_mail() to send an email by ID.

I'd rather we rename hook_mail to hook_message_info() or something like that and work on cleaning up the parameters that get passed around rather than rip it out completely and end up with a mess.

Possibly crazy idea, which you'd probably expect from me: Replace the anonymous array of mail info with a proper class. Every mail message is represented as a class (with base class, presumably), and its standard data is properties inside that class. Want to send it? Instantiate it from anywhere. It can still get an alter hook that it passes $this to, much the same way that SelectQuery does. The existence of the class self-documents the mail message.

Alternatively, as #1 suggests there are plenty of good mail systems around. There's no need for us to (re)invent our own. If an existing system is good enough, let's just use it (possibly extend it) and call it a day.

I'm really loving the API of Swiftmailer.

Swiftmailer is really great but also eats a lot of RAM

suppose this change will provide a more flexible mail managment from the side of common message pattern - sending a message to a list of recipints with ability to send same message (probably no alter needed) and send personalized e-mails (each mesage passed token system per recipient with context)

It is great to see there is so much interest in this issue. Note that I was interested in helping Drupal get rid of hook_mail() since I was instrumental to introducing it (yes, I'm very sorry). Its not really my itch to perfect a wholly different solution for sending mail, so I hoped to do a run-by removal of hook_mail() after which people can refine this however they like. We can also just debate the much better solution here without removing the much hated hook_mail() solution first. In that case, anyone feel free to pick this up! I agree with @sun that loosing $module as its own value is not really ideal for the mail system override use case.

Making this clearly an open floor for everyone.

Removing D8MI tag in case anyone thought that would make this be get done. I don't have a personal itch here, so if there is no interest, then we'll have it as it is in Drupal 8.

My take on this is that we should remove the magic and make it clear which module handles which mail.

Thus:

- hook_message_info(): allows modules to declare mail keys that they handle. This would be broadly the same concept as hook_menu(), in that they would define a callback.
- hook_mail_alter(): keep much as it is.
- drupal_mail(): keep much as it is, but now the mysterious key that you use currently has been explicitly defined in hook_mail_info().

So we go from hook_mail() being a magic thing which gets called based on which module you passed to drupal_mail() and which then switches on any number of mail keys to a declarative info hook which tells you exactly who is handling what.

For modules that want to reuse different components among different emails, they can declare the same callback for all their messages -- the callback should thus pass in the message ID as the first parameter.

Relatively small conceptual change (compared to going to an OO architecture) but much clearer IMO :)

Here's a patch of what I mean, just on user module (which AFAIK is the most hairy and unreadable implementation of hook_mail()).

The difference from the patches above is that there's still a central clearing system, to which you say 'hey I want to send mail key 'chickens' and the mail system can go get the message you mean.

My initial reaction:

- we've still got a mostly unreadable mess in _user_mail_text(). Now we use a callback we could break that up into separate functions and even shove them into an inc file.
- you probably shouldn't need to know the $module for the mail you want to send, just the $key. That means we need an info-gathering function, caching of info array, etc.
- we're still holding a ton of raw email text in code, and that feels ugly.

I think there is an advantage to knowing explicitly what mails a module implements. A module may want to know about all mails generated (to provide an admin summary page/ability to override etc) without actually sending the mail or providing $to, $params, etc
So instead of removing hook_mail() I agree with joachim that it would be better to make it perfom like other standard info hooks such as hook_mail() and hook_theme().

I think a solution with the following points would be ideal.
* A module must explicitly define what mail keys it implements and what parameters it requires
* Custom callbacks can be defined for subject and body. Defaults to t().
* Delay param/token replacements in subject and body as late as possible in drupal_mail() to allow overrides/additions in hook_mail_alter().

Trying again.

As a module developer looking at hook_mail() documentation for the first time, I am very happy to find this issue and +1 all the work that has been done so far. Moving to the hook_message_info() design makes sense to me and is the way I was expecting hook_mail() to function when I first started digging into it.

Is there currently a good module or core way to store these "email templates" that are sent with drupal_mail()? Or are developers currently just hardcoding their mail templates into functions? I realize there are complex mailing system modules, but I really just want a way to manage templates to be sent by key using drupal_mail().

Remove unrelated IRC log item.

@catch: not much traction here - Should this be marked as postponed or moved to 9.x directly?

Here's a thought: when a config entity is defined in a config file, does Drupal's config system automatically copy it into the database, or does it only do that when it's overridden?

A mail could be a config entity, stored initially as a config file and then in the database when the user overrides it.

We'd get several neat gains here:

- any module defining a mail automatically gets a UI to let the user customize it
- any module defining a mail automatically gets standardize storage for it -- no need to invent new system variable names to store the subject & body

I've got a very very rough proof of concept which gets the subject & body from a config entity.

The change in user module is that instead of a single config yml file with all the messages, we'd have one per message. That's less compact, but at least they'd be grouped together in alphabetical order.

However, things get a bit messy when it comes to Contact module -- because it already has its own *content* entity for mail messages. It seems a bit daft to need Contact to define config entities which will be empty (or mostly, the 'User foo has sent you a message' bit would maybe be in there), and then to stick in the content from its own message entity.

So I am wondering whether instead of drupal_mail() being passed a message key (which I was going to use as the config entity's ID), it should be passed an entity, which could be either a config or a content entity, which implements a common interface (which would probably become the parent of Drupal\contact\Entity\MessageInterface).

So you'd have:

$message = entity_load('my_entity_type', $id);
drupal_mail($message, $to, $from, yadayada);

// and then drupal_mail does:
$message->getSubject();
$message->getBody();
// where the message entity implements the interface that defines those methods.

For user module, the entity would be a config entity 'mail_message', which would be a type defined by core, intended for system messages. There'd be a UI to let you customize the,, eg the 'user_register' message.

For contact module, the entity would be contact's own contact message entity, which is a content entity created by the user sending the message.

Does that sound reasonable, or is mashing content and config entities together into the same system crazy?

I think that the message notify module could use this idea as well.

#34 was right, moving to 9.x. Drupal 8 is in a stabilisation phase.

I've found a potential problem with the new system and the old system living together concurrently in 8.1.x.

What I'm working towards is this for sending an email:

  // OLD:
  \Drupal::service('plugin.manager.mail')->mail(MODULE, KEY, 'me@example.com', 'und');

  // NEW, for system messages:
  $message_entity = entity_load('mail_message', 'password_reset');
  \Drupal::service('plugin.manager.mail')->mailMessageEntity($message_entity, 'me@example.com', 'und');

Here, mail_message is a config entity type, provided by core. The config entity ID corresponds to KEY. Core and contrib modules can define default entities for system-ish messages. One example would be Commerce defining a system message 'order_complete'.

Equally though, a module like Contact can define a content entity, provided it implements the MailMessageInterface, and mail that too:

  $contact_message_entity = entity_load('contact_message', 42);
  \Drupal::service('plugin.manager.mail')->mailMessageEntity($contact_message_entity, 'me@example.com', 'und');

At least for core's mail_message entity type, an associated plugin is going to be needed, which will take care of performing the work that hook_mail() currently does in some cases. For example, user_mail() needs to replace tokens based on the user account. The code that requests the mail to be sent shouldn't have to do this: that would be a DX regression.

When it comes to compatibility with 8.0.0, the concern is this: an 8.0.x module_foo should be able to use mail(MODULE, KEY) to send a module defined by module_bar message, even if module_bar is now using the 8.1.x system.

My idea for this was to have mail() check whether MODULE implements hook_mail(): if it doesn't, then it's using 8.1.x. mail() should load the entity and proceed as though mailMessageEntity() was called.

The only problem is: given MODULE, how do you know which entity type to load? Modules might use the core mail_message entity, or they could equally define their own type.

Any thoughts?

I was working on a core patch with my ideas, but getting really bogged down with the minutiae of figuring out where to put entity types and plugins and so on.

So I've made a contrib module by way of a demonstration: https://www.drupal.org/project/mail

It's not yet complete, and the matter of how to support the existing hook_mail() system for now has been sidestepped, since I'm not touching core code yet.

there are a handful of modules about mail building/configuring already, commerce plans to use message I think, https://www.drupal.org/project/courier is another example, we worked a bit on that but it didn't fully match our expectations yet.

For the core API, I've been seriously considering to just directly use the swiftmailer API and getting that into core. And using those objects to build mails and then just sending them, with a bit of abstractions/factory something. not sure about getting that into core and the process of moving to it, might need to wait for 9.x?

It looks like swiftmailer would handle the part that's currently handled by mail backend plugins: taking a completed message with subject and body and sending it.

What it doesn't cover AFAICT is replacing hook_mail() so we have emails which are configurable and translatable and have dynamic values inserted into the text.

Now I've actually written code for was I was outlining in #36 onwards, I'm finding it works rather nicely. To implement a message, all modules need to do is provide a default config entity in config/install. They can rely on the admin UI for all messages, or very easily define a route that uses the filtered list builder & a permission to allow editing just the emails that they're providing -- which gives the granularity of permissions that we'd need to separate, say, user account emails from subscription reminder emails from commerce order emails.

I've revisited the module I wrote for this, https://www.drupal.org/project/mail, as another client project needed user-configurable messages, and I am finding it works pretty nicely.

I've found in using this on a new project that creating a config entity and a plugin is rather nicer DX than hook_mai(), if I do say so myself :)

There are a few changes I'd want to make, but I think it has potential to be turned into a patch for this issue.

To recap, here's how the module works:

- there's a config entity, mail_message. This represents an email that can be sent by the system, such as a new account notification message
- The mail_message entity references two plugins: a processor, and a mail backend. Either may be null. These are not editable in the UI.
- The processor plugin does the work that hook_mail() currently does: inserts dynamic values into the email subject and body.
- The mail_message entity has subject and body properties, which are editable.
- The mail_message entity has a group property. This is used by the mail_message entity list builder, which can show only mail_message entities in a particular group. That way, modules can very easily have their own admin UI for just their own messages, by defining the route and using the mail_message's entity list builder with a group parameter.
- the entity mailer service provides a method that lets you mail a mail_message entity, or indeed any entity that implements MailMessageInterface (so for example Contact module could have its message entity implement this interface, and then use the service)

Here's what I'd change in what I've built so far:

- have the mail system automatically provide dynamic permissions for each mail_message group, and add an entity access handler that allows edit access based on entity group
- perform token replacement on the mail subject and body in all cases, as it's by far and large the most common processing that needs to be done
- add a property to the mail_message entity called 'token_types', which lists the token data items that should be passed in the $params array when calling the mailer service. For example, this would ensure that you can't send the account creation mail without passing a user entity as a parameter.

To allow both systems to work side by side until 9.x, I think we need the following criteria. Suppose we convert user module's mail messages to the new system. The following scenarios should not be affected by the conversion:

- my_module sends user module's messages using the existing MailManager
- my_module implements hook_mail_alter() to change or suppress user module's messages
- a site's settings.php overrides the system.mail.interface value for which mail backend plugin to use for a particular hook_mail() message

Here's the changes that I think are needed to allow this:

- add 'module' and 'key' properties to the mail message entity, which should match the old hook_mail() parameters. This is so that hook_mail_alter() implementations aren't affected. These properties can be removed for D9.
- add an invocation of hook_mail_alter() to the new entity mailer service, which uses the module and key properties to recreate the $message array that the hook expects. This means that until D9, both old and new system messages go through hook_mail_alter(). For D9 we can consider how an entity-based alter hook should look.
- change the existing MailManager to first attempt to load a mail_message entity for the given $module and $key. This allows modules that use MailManager::mail() to send another module's mail to not be affected by conversion.
- allow system.mail.interface to override the mail backend plugin ID property of a mail_message entity

Here's what would be marked as deprecated:

- hook_mail()
- MailManager service
- use of system.mail.interface config to override the backend plugin

What next:

- I'd really like some feedback on the module and the plan
- I'm not sure where the various parts of this would go in core. In particular, where would be best to define the entity type and processor plugin type?

I should note that the current system is so counter-intuitive that both mglaman and I completely failed to use it right in Commerce: https://www.drupal.org/project/commerce/issues/3017080#comment-12936027

The "Hook as a place where your main logic actually needs to live" approach doesn't make any sense once you've forgotten D7.

The status of my work on this issue is currently:

- I've finished a proof-of-concept module -- I've scratched my itch to show that the idea I had is feasible
- it works well enough that when I need to use mail on my own projects, I can use that module and get a better DX
- I'm happy to work on getting this ready for core, BUT before I spend more time into it, I'd like some feedback in order to feel more confident I'm on the right track & not wasting my time.

My gut feeling would have expected emails to be sort of similar to help texts and migrations, as in they could be plugins, so they don't require a config import, but for more advanced usecases you might be able to leverage config entities to allow user defined emails.

Should this now be closed as part of #1803948: [META] Adopt the symfony mailer component or simply repurposed and linked into the last step of the "Stage 4: Remove the old mail system" on that issue's summary?

Deprecate then remove the old API hook_mail() and hook_mail_alter()

At the very least, this one is still a long way out because:

The last one could be done one Drupal major version after the others as it provides a big increase to BC with a small amount of code.

My personal opinion:

* The issue title makes this a duplicate of that meta issue.

* The concept of comment #47 and related contrib module, a centralized system for user-configurable, translatable, reusable, ... e-mails that can be used by modules but also things like ECA is a layer/feature on top of the new mail API.

My suggestion would be to close this as a duplicate and optionally open a new issue or just keep discussing it in the meta if and how we'd implement such a system in core.