Problem/Motivation
#3165762: Add symfony/mailer into core introduced the system.mail mailer_dsn config as a way to configure a system wide symfony mailer DSN. The form of this string is an URI (plus some additional syntax sugar for special transports).
The DSN as a single string is convenient for apps which are configured through environment variables and container params (as in case of symfony). However, a more structured form would be more convenient in Drupal. Especially considering that at some point there will be GUIs for transport configuration. Implementing GUI forms for different transport factories will less complicated with structured config instead. Also specifying the values in settings.php is less intimidating when the config is structured vs. a raw URI string.
Steps to reproduce
Proposed resolution
Instead of that (current implementation):
$config['system.mail']['mailer_dsn'] = 'smtp://user%40some-domain.org:%5EC%3E6P2EdL.b%3C%3F9jBXgOu8%2AA%3E@smtp.example.com:25'
Allow people to specify the DSN like this:
$config['system.mail']['mailer_dsn'] = [
'scheme' => 'smtp',
'host' => 'smtp.example.com',
'port' => 25,
'user' => 'user@some-domain.org'
'password' => '^C>6P2EdL.b<?9jBXgOu8*A>'
];
Note that this has one caveat: The mailer DSN string allows for advanced use cases. Namely configuring load balancing over multiple transports. This configuration isn't possible using a structured DSN. Contrib or custom modules could step in and cover those more exotic configuration.
Remaining tasks
User interface changes
API changes
Data model changes
Convert system.mail mailer_dsn from a string to a map.
Release notes snippet
The system.mail mailer_dsn format has changed in 10.2.0-beta2. This feature was new in 10.2.0-beta1; sites that configured this in beta will need to reconfigure their settings.
Issue fork drupal-3399645
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3399645-use-structured-dsn
changes, plain diff MR !5265
Comments
Comment #3
znerol commentedComment #4
znerol commentedComment #5
smustgrave commentedLove the idea, think we will need a change record for it.
Comment #6
znerol commentedI propose to just link and update the change record (if this gets in before the 10.2 release). Basically we'd need to update the examples:
In order to configure the DSN, use the following commands:
drush config:set system.mail mailer_dsn.schema sendmaildrush config:set system.mail mailer_dsn.host defaultdrush config:set system.mail mailer_dsn.schema smtpdrush config:set system.mail mailer_dsn.host localhostdrush config:set system.mail mailer_dsn.port 1025drush config:set system.mail mailer_dsn.schema smtpdrush config:set system.mail mailer_dsn.host smtp.example.comdrush config:set system.mail mailer_dsn.user some-userdrush config:set system.mail mailer_dsn.password some-passComment #7
znerol commentedComment #8
znerol commentedAdded a caveat to the issue summary: The structured DSN doesn't cover a rather exotic use case.
Comment #9
imclean commentedLoad balancing could be supported by providing multiple DSN configs, although it might complicate things a bit.
or
Comment #10
znerol commentedLoad balancing isn't possible with the current mail system in core and it wasn't a goal in the original issue #3165762: Add symfony/mailer into core. I really think core shouldn't try to implement it. It is an advanced use case and for that kind of things we have contrib.
Comment #11
adamps commented> It is an advanced use case and for that kind of things we have contrib.
I agree. It's part of my plan for DSM, but it hasn't been implemented yet😃.
Comment #12
adamps commentedI think it's a very good idea, and it's similar to what we already do in DSM.
I see you have matched the names of each parameter to the DSN class, which does make sense. FYI In DSM we used 'pass' (to match the examples in https://symfony.com/doc/current/mailer.html) and 'query' (because they are query parameters).
Comment #13
smustgrave commentedRebased so I could run the test-only feature
If we add to the existing CR think it should be noted with
$config['system.mail']['mailer_dsn'] = [
'scheme' => 'smtp',
'host' => 'smtp.example.com',
'port' => 25,
'user' => 'user@some-domain.org'
'password' => '^C>6P2EdL.b<?9jBXgOu8*A>'
];
is possible and the problem with load balancing.
rest looks fine.
Comment #14
longwaveI do wonder if we should just move to the structure in #9, allowing multiple DSNs but only supporting 'default' in core, similar to the way we allow multiple database definitions but in practice only the 'default' is used except for special cases. Wouldn't this then make life in contrib easier? Otherwise I think you will have to override this entirely and provide multiple configs elsewhere, and the
system.mailone becomes redundant?Comment #15
longwaveComment #16
znerol commentedAs per @berdir over in #3379794:
.
Mailer transport config is also slightly more involved than the database since transports can be nested. E.g., the Transports transport inspects the
X-Transportheader of a message and dispatches it to other transports accordingly.In order to effectively support multi transport configuration in core we'd have to implement a tree-like config structure. A simple key-value model (as with database definitions) wouldn't be sufficient.
Comment #17
longwaveSending this back to RTBC. I think this is a valid improvement that we should get in before 10.2.0 otherwise we risk having to provide some form of BC.
Comment #18
borisson_I was wondering if this did not mean we lost schema validation but it is the opposite, we are gaining a lot of validation here. I wonder if we should add the same regex we added in #3379102: Add validation constraint to `type: label` + `type: text`: disallow control characters, to disallow control characters?
Comment #19
znerol commented@borisson_ true. Do you think this could go into a follow-up or is that a requirement for an initial iteration?
Comment #20
borisson_I think that can be a follow-up.
Comment #21
znerol commentedAdded #3401255: Tighten config validation schema of system.mail mailer_dsn.
Comment #22
znerol commentedThanks to the fact that the config schema matches exactly the parameter names of the
Dsnconstructor, we actually can unpack the config array directly into the constructor parameters.Comment #23
longwaveDiscussed with @catch. We would like to land this in 10.2.0 to avoid adding backward compatibility later, but because we already released beta1 there is a chance some users will now have the old structure in their config.
As we cannot delete post update hooks, we propose making the old post update hook a no-op and overwriting the config with the new structure in a new post update hook. This will cover users upgrading from beta1 or earlier to the next release; we don't expect anyone to have started using this feature yet so we can safely overwrite the config as long as we do it before 10.2.0.
Comment #24
znerol commentedComment #25
alexpottThe approach to updates from beta1 to 2 make sense.
I guess we could add something to the release notes about needing to reconfigure it.
Comment #28
longwaveCommitted and pushed 0b7c2a60a5 to 11.x and 749e1f030f to 10.2.x. Thanks!
Tagging for change record updates, as the examples there need to be updated now.
Comment #29
longwaveAlso tagging for release notes as we can add a line for beta-to-beta upgraders, just in case anyone started looking at this.
Comment #30
znerol commentedSetting status to NR for the updated change record.
Comment #31
longwaveThe updated change record looks good to me, thank you @znerol. Added a release note snippet as well.