Problem/Motivation

As of #2278353: Update to Symfony 2.5, the core email validation class will be using egulias/EmailValidator #2313669: Bring in egulias/EmailValidator for RFC compliant email address validation. valid_email_address() should use it too. Deprecate valid_email_address().

Proposed resolution

Modify valid_email_address() to use the egulias/EmailValidator. Mark valid_email_address() as deprecated to be removed in 8.x (or 9.x?).

Remaining tasks

Provide fall back to existing functionality for php 5.2

User interface changes

API changes

Creates the email.validator service and \Drupal::emailIsValid().

Beta phase evaluation

Reference: https://www.drupal.org/core/beta-changes
Issue category Task because this is stability-enhancing code.
Issue priority Major because it increases stability of Drupal.
Disruption Not disruptive for core/contributed and custom modules/themes because it maintains the commonly-used valid_email_address(). It will just work consistently now.
CommentFileSizeAuthor
#76 valid_email_address-2343043-76.patch1.06 KBRAWDESK
#75 valid_email_address-2343043-75.patch1.16 KBRAWDESK
#74 valid_email_address-2343043-74.patch96.87 KBsanduhrs
#72 valid_email_address-2343043-72.patch94.98 KBsanduhrs
#45 valid_email_address-2343043-45.patch162.26 KBcilefen
#45 interdiff.txt1.79 KBcilefen
#43 valid_email_address-2343043-43.patch160.4 KBcilefen
#43 interdiff-40-43.txt21.87 KBcilefen
#40 valid_email_address-2343043-40.patch160.69 KBcilefen
#31 valid_email_address-2343043-31.patch1.29 KBcilefen
#30 valid_email_address-2343043-30.patch1.24 KBcilefen
#30 interdiff.txt138 bytescilefen
#29 valid_email_address-2343043-29.patch1.24 KBcilefen
#29 interdiff-27-29.txt957 bytescilefen
#27 valid_email_address-2343043-27.patch1.73 KBcilefen
#27 interdiff-25-27.txt1.28 KBcilefen
#25 valid_email_address-2343043-25.patch1.79 KBcilefen
#25 interdiff-23-25.txt406 bytescilefen
#23 valid_email_address-2343043-23.patch1.68 KBcilefen
#23 interdiff-20-23.txt1.03 KBcilefen
#20 valid_email_address-2343043-20.patch1.61 KBcilefen
#20 interdiff-15-20.txt829 bytescilefen
#15 valid_email_address-2343043-15.patch1.11 KBcilefen
#7 interdiff-6-7.txt998 byteshussainweb
#7 valid_email_address-2343043-7.patch1.05 KBhussainweb
#6 valid_email_address-2343043-6.patch959 bytescilefen
#6 interdiff-3-6.txt372 bytescilefen
#3 valid_email_address-2343043-3.patch985 bytescilefen
#3 interdiff-1-3.txt359 bytescilefen
#1 valid_email_address-2343043-1.patch989 bytescilefen
Support from Acquia helps fund testing for Drupal Acquia logo

Comments

cilefen’s picture

Status: Active » Needs review
FileSize
989 bytes
rbayliss’s picture

Status: Needs review » Needs work
+++ b/core/includes/common.inc
@@ -345,17 +346,15 @@ function drupal_get_destination() {
+  $validator = new EmailValidator(TRUE);

No constructor argument is needed, I think.

cilefen’s picture

Status: Needs work » Needs review
FileSize
359 bytes
985 bytes

@rbayliss You are right. I had been working on another issue and the Symfony class that uses this library has a constructor argument.

droplet’s picture

Noted that it isn't pass Chrome's client-side validation. (didn't test other browsers yet)

https://github.com/egulias/EmailValidator/blob/master/tests/egulias/Test...

andypost’s picture

+++ b/core/includes/common.inc
@@ -345,17 +346,15 @@ function drupal_get_destination() {
+  $validator = new EmailValidator();
+  return $validator->isValid($mail);

return new EmailValidator()->isValid($mail);

cilefen’s picture

@andypost: Are you sure that is possible in PHP 5.4? I think so...

hussainweb’s picture

FileSize
1.05 KB
998 bytes

I am thinking this should be a service. That would allow someone to specify a different validation class. What do you think?

cilefen’s picture

Issue summary: View changes
cilefen’s picture

@hussainweb That is an interesting idea. You'll want to also patch #2278353: Update to Symfony 2.5.

cilefen’s picture

@hussainweb But you can see that on #2278353: Update to Symfony 2.5 there is a Drupal-side class used to configure the validator.

hussainweb’s picture

@cilefen: If my understanding of this page is correct, I guess we can just remove the constraint we have added #2278353-89: Update to Symfony 2.5. I think we will just have to make sure we pass in strict as true and it will use our new email validator class. Is this correct?

cilefen’s picture

@hussainweb Yes, strict must be true.

cilefen’s picture

I would consider as a follow-up to deprecate valid_email_address().

cilefen’s picture

cilefen’s picture

I suggest this issue is a beta priority (see https://www.drupal.org/contribute/core/beta-changes) because it standardizes a library integration and reduces fragility. Without this, valid_email_address and the email validator class do different things, which could lead to unexpected results.

pwolanin’s picture

Why do we need an instance instead of a static method?

If it's a service I guess the motivation is so that people can swap the implementation?

cilefen’s picture

@pwolanin EmailValidator->isValid isn't a static. Is there a way to work around having an instance?

cilefen’s picture

Priority: Normal » Major
Issue summary: View changes
cilefen’s picture

@pwolanin The EmailValidator is not a Drupal class. The idea behind having it as a service is for the same reason we front-end other non-Drupal classes as services—which I think is to make it easier to find and to consider it an integral part of Drupal. If we want this method to be a static we will have to front-end it with a Drupal-side class.

cilefen’s picture

@pwolanin: is this what you mean?

cilefen’s picture

I bumped this to major because core currently has two different email validators that work differently: EmailValidator and valid_email_address().

pwolanin’s picture

So, can we deprecate the function?

Are the other methods in the class generally used? If not maybe the static wrapper could be like

 \Drupal::emailIsValid($mail)

As a direct replacement for the function

cilefen’s picture

cilefen’s picture

Title: valid_email_address() should use egulias/EmailValidator » valid_email_address() should use egulias/EmailValidator and become deprecated
Issue summary: View changes
cilefen’s picture

Added @deprecated.

cilefen’s picture

Issue summary: View changes
cilefen’s picture

Removed the static function and fixed the deprecation line versions.

dawehner’s picture

Are we sure we really want to add a method onto the drupal static class? This is something we should avoid here, because validating a mail is not a often used operation if you ask me.

cilefen’s picture

This is without the \Drupal method.

cilefen’s picture

Removed a double newline.

cilefen’s picture

Reroll.

cilefen’s picture

Issue summary: View changes
dawehner’s picture

I think it totally makes sense to use the same email validation mechanism everywhere.

mpdonadio’s picture

Status: Needs review » Reviewed & tested by the community

Patch looks good, and gave it a quick test drive on a system form where valid_email_address() is used.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

Yep having two different results from validating email address is confusing. Committed 2cdd0e2 and pushed to 8.0.x. Thanks!

Thank you for adding the beta evaluation to the issue summary.

  • alexpott committed 2cdd0e2 on 8.0.x
    Issue #2343043 by cilefen, hussainweb: valid_email_address() should use...
catch’s picture

Crossposted:

Are there any e-mail addresses that would have been allowed by filter_var() that won't be allowed by this library?

This could have implications for the migration path if so. Don't see it discussed in any of the previous issues.

cilefen’s picture

The issue that changed the validator when adding users was #2313669: Bring in egulias/EmailValidator for RFC compliant email address validation. I wonder if anyone has tried, or could try on a large list of valid D7 account addresses.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.

cilefen’s picture

Version: 8.0.x-dev » 7.x-dev
Status: Closed (fixed) » Patch (to be ported)
FileSize
160.69 KB

Here is the Drupal 7 back-port.

cilefen’s picture

Status: Patch (to be ported) » Needs review

Status: Needs review » Needs work

The last submitted patch, 40: valid_email_address-2343043-40.patch, failed testing.

cilefen’s picture

Status: Needs work » Needs review
FileSize
21.87 KB
160.4 KB

I fixed the whitespace errors in the vendor libraries.

Status: Needs review » Needs work

The last submitted patch, 43: valid_email_address-2343043-43.patch, failed testing.

cilefen’s picture

Status: Needs work » Needs review
FileSize
1.79 KB
162.26 KB

The OpenID test was using 'mail@invalid#' as a test of an invalid email address. Evidently, the new email validator would allow it, which could be an issue on its own. For now, I switched it to 'foo', which is used in user.test.

  • alexpott committed 2cdd0e2 on 8.1.x
    Issue #2343043 by cilefen, hussainweb: valid_email_address() should use...
David_Rothstein’s picture

+            "require": {
+                "php": ">=5.3.2"
+            },

I think we'd have to have a fallback for sites on PHP 5.2.

So is there really no hope of this getting fixed/improved in PHP itself?

cilefen’s picture

Anybody’s picture

I agree with #47 that PHP 5.2.5 has to be supported because of the Drupal System requirements.
Somehow we should find a good or at least better solution ASAP because of issues like #1427516: valid_email_address() rejects valid email addresses and can't wait for things to happen in issues sleeping like https://bugs.php.net/bug.php?id=69140.

I think the currently only valuable solution would be to use a custom regex for php versions below 5.3.2 and the solution from the patch in #45 for higher versions. What's your oppinon?

This blog post shows the problems: http://markonphp.com/properly-validate-email-address-php/

TR’s picture

There's no change record for this?

David_Rothstein’s picture

We need a fallback for PHP 5.2, but that doesn't necessarily mean we need to hold up this issue on a fix for PHP 5.2.

In other words, one possible fallback would be to just keep calling filter_var() for older PHP versions, and only load up this library and use it if the PHP version is new enough.

Anybody’s picture

Great! +1 for #51

frob’s picture

I just ran into this issue for a test suite for valid email addresses from a api vendor.

All of that to say +1 for #51. Fall back to existing functionality if the php version is too old.

frob’s picture

Issue summary: View changes
Status: Needs review » Needs work

Updated remaining tasks in issue summary and marking as needs work.

frob’s picture

After testing this with wikipedia's like of strange valid email addresses, only two didn't work.

"very.(),:;<>[]\".VERY.\"very@\\ \"very\".unusual"@strange.example.com
"()<>[]:,;@\\\"!#$%&'*+-/=?^_`{}| ~.a"@example.org

Would this be considered and up stream issue?

cilefen’s picture

Re: #55: yes

frob’s picture

Good, so all that is left to do is put in the 5.2 fallback.

  • alexpott committed 2cdd0e2 on 8.3.x
    Issue #2343043 by cilefen, hussainweb: valid_email_address() should use...

  • alexpott committed 2cdd0e2 on 8.3.x
    Issue #2343043 by cilefen, hussainweb: valid_email_address() should use...

  • alexpott committed 2cdd0e2 on 8.4.x
    Issue #2343043 by cilefen, hussainweb: valid_email_address() should use...

  • alexpott committed 2cdd0e2 on 8.4.x
    Issue #2343043 by cilefen, hussainweb: valid_email_address() should use...
davemckain’s picture

#50: We are working on filling in the missing Change Record for this as part of #2873881: [meta] Add Change Record to @deprecated for common.inc at DrupalCon Vienna. The draft CR can be found at [#2912661]. We are all newbies to this so please be gentle :-)

Mile23’s picture

Published the CR for the 8.x branch. https://www.drupal.org/node/2912661

xjm’s picture

Status: Needs work » Patch (to be ported)
Issue tags: +Needs backport to D7

If this needs backport to D7, it should be Patch (to be ported) until the D7 maintainers create a separate D7 issue for it, and then it can be marked back to Fixed against the 8.x branch it was committed to.

xjm’s picture

For #49 / #51 about the 5.2 fallback I think we also could create a separate followup and link it here, but it's a D7-only issue since D8 requires 5.5.9.

frob’s picture

Haven't we been working on the D7 backport since #40?

David_Rothstein’s picture

Status: Patch (to be ported) » Needs work

Yes, the backport has been in progress here for a couple years (the issue is currently filed against D7), but a separate issue could be created also and then this one closed.

I do recall that this patch was relatively close to being ready though.

David_Rothstein’s picture

diff --git a/composer.json b/composer.json
new file mode 100644
index 0000000..f4240d7
--- /dev/null
+++ b/composer.json
@@ -0,0 +1,5 @@
+{
+    "require": {
+        "egulias/email-validator": "^1.2"
+    }
+}

That said, I am really not sure about including a composer.json in Drupal 7 core - won't that cause problems for people who are already using Composer to manage their sites?

This is mostly just a one-off library, so would it be possible to include the library manually (like we normally would) without having to mess with Composer?

cilefen’s picture

Good point.

frob’s picture

+1 #68

RAWDESK’s picture

In attendance of the official D7 backport release, I just wanted to share my efforts in an alternative solution using xautoload using the Libraries API to autoload the necessary classes from Egulias' email validator, currently at version 2.1.7.

Still struggling with the expected availability of the necessary classes though, and reported here in xautoload doc support
https://www.drupal.org/node/1976234#comment-13021989

Tried to follow the class instantiation approach as applied in patch #45, but without success either :

function valid_email_address($mail) {
  if (module_exists('libraries') && libraries_load('EmailValidator')) {
    $validator = new Egulias\EmailValidator\EmailValidator;
    return $validator->isValid($mail, new Egulias\EmailValidator\Validation\RFCValidation);
 }
  return (bool)filter_var($mail, FILTER_VALIDATE_EMAIL);
}
sanduhrs’s picture

Status: Needs work » Needs review
FileSize
94.98 KB

Attached is a patch that adds the needed libraries from doctrine and egulias to the mics folder.
It loads the classes when valid_email_address() is invoked.
It uses EmailValidator() when php >= 5.5 and falls back filter_var() otherwise.

Please review.

Status: Needs review » Needs work

The last submitted patch, 72: valid_email_address-2343043-72.patch, failed testing. View results

sanduhrs’s picture

RAWDESK’s picture

Hi sanduhrs,

Thanks for providing this working patch.
So from the valid_email_address() point of view my test passed.

There's one side effect I noticed during 1 of my tests where after successful registration of numéros@mailinator.com user, a subsequent registration of numeros@mailinator.com was refused by core user module.

Inside user_account_form_validate() function, the entered email address is also validated against existing users.
For the above case numeros@mailinator.com is seen as an existing user because of the unCOLLATEd query on utf8_general_ci collated mail column.

After applying my attached patch on user module, the validation passed without an error message but for some unclear reason the registration does not complete successful (user_register_submit handler not triggered).

Small side remark with your patch #74:

I tried to replace your class includes with a recursive iterator, but the order in which this happens seems to be relevant for this EmailValidator plugin.

Here's my suggestion for improvement, in case the plugin should receive new php files in the future :

  include_once $directory . '/Exception/InvalidEmail.php';
  include_once $directory . '/Parser/Parser.php';
  include_once $directory . '/Validation/EmailValidation.php';
  include_once $directory . '/Validation/RFCValidation.php';
  include_once $directory . '/Warning/Warning.php';
  $dir_iterator = new RecursiveDirectoryIterator($directory);
  $iterator = new RecursiveIteratorIterator($dir_iterator);
  foreach ($iterator as $file) {
    if ($file->isFile() && substr($file->getPathname(), -3) == 'php') {
      include_once($file->getPathname());
    }
  }

I'll take some time next week to further investigate my new issue.
Regards,

sanduhrs’s picture

Status: Needs review » Reviewed & tested by the community

The issue in #76 is out of scope of this patch IMHO.
The same behavior can be observed in D8/D9.

So the patch in #74 stands, but should be updated with the latest version of egulias/EmailValidator:3.1.2

poker10’s picture

Status: Reviewed & tested by the community » Needs work

Some test are failing on PHP 7.4 and higher with patch #74. The new library needs to be checked/updated.

Trying to access array offset on value of type null

See: https://www.drupal.org/pift-ci-job/2363706 and others.

And I also agree that the change in patch #76 is not relevant to this issue (not mention that this change is not working correctly in DBs other than MySQL/MariaDB).

poker10’s picture

Maybe the new proposed hook in #2966195: valid_email_address() should be easily overridable will better suit the current D7 status and we would not need to include 3rd party library into D7 core?

I think it would be better to close this as Fixed in D8 and for D7 use the flexible approach proposed by the mentioned issue, where sites can potentially add whatever library they want and D7 core does not need to maintain it. What do you think?

cilefen’s picture

We should definitely close this and let contrib take it on after the hook goes in.

cilefen’s picture

Version: 7.x-dev » 8.0.x-dev
Status: Needs work » Fixed
Issue tags: -Needs backport to D7

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.