In https://www.drupal.org/project/drupal/issues/3415582, an unhandled exception when trying to register a duplicate username with different case was fixed. However, we've now noticed that there are some other cases where the same issue occurs (caused by the same change that caused #3415582). The issue does not only affect case-sensitivity (fixed), but also things like accentuation (e.g. "example" vs. "éxample"). I think there might also be some (not yet known) other edge cases where the same issue occurs. Basically, the issue always occurs if the database layer has a different understanding of equality than the php layer.
I think the issue was introduced by https://www.drupal.org/project/drupal/issues/2478663.
1. Set-up a fresh Drupal 10.x/11.x installation (e.g. on simplytest.me)
2. Log-in as admin
3. Create a user with username "éxample"
4. Create another user with username "example"
5. An unhandled exception occurs:
The website encountered an unexpected error. Try again later.
Drupal\Core\Entity\EntityStorageException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry 'example-en' for key 'user__name': INSERT INTO "users_field_data" ("uid", "langcode", "preferred_langcode", "preferred_admin_langcode", "name", "pass", "mail", "timezone", "status", "created", "changed", "access", "login", "init", "default_langcode") VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8, :db_insert_placeholder_9, :db_insert_placeholder_10, :db_insert_placeholder_11, :db_insert_placeholder_12, :db_insert_placeholder_13, :db_insert_placeholder_14); Array ( [:db_insert_placeholder_0] => 3 [:db_insert_placeholder_1] => en [:db_insert_placeholder_2] => en [:db_insert_placeholder_3] => en [:db_insert_placeholder_4] => example [:db_insert_placeholder_5] => $2y$10$h3YBVBuf9CTaL0qk6WpjX.WTmPdNpYLbeEQvpJxVRGAGBOkPNtAqu [:db_insert_placeholder_6] => [:db_insert_placeholder_7] => UTC [:db_insert_placeholder_8] => 1 [:db_insert_placeholder_9] => 1719306986 [:db_insert_placeholder_10] => 1719306986 [:db_insert_placeholder_11] => 0 [:db_insert_placeholder_12] => 0 [:db_insert_placeholder_13] => [:db_insert_placeholder_14] => 1 ) in Drupal\Core\Entity\Sql\SqlContentEntityStorage->save() (line 817 of core/lib/Drupal/Core/Entity/Sql/SqlContentEntityStorage.php).
Handle the case validation for the username in core/lib/Drupal/Core/Validation/Plugin/Validation/Constraint/UniqueFieldValueValidator.php file. The PHP function iconv() with UTF-8 & ASCII//TRANSLIT working fine.
Syntax:
iconv('UTF-8', 'ASCII//TRANSLIT', 'STRING');
As CI builds indicates this issue is database engine and maybe engine configuration specific (collation?) - this has to be double checked and identified whether the new normalization logic should always run or not.
Nil
TRANSLIT word added into the dictionary.
Nil
Nil
Nil
| Comment | File | Size | Author |
|---|---|---|---|
| #11 | drupal11x-ddev-site-admin-people-create-11-21-2024_02_03_PM.png | 616.48 KB | arunkumark |
Show commandsStart within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
changes
Comments
Comment #2
nico.b commentedComment #3
nico.b commentedComment #4
caesius commentedThis was solved for case sensitivity by adding the function
caseInsensitiveArrayIntersect(). Maybe that should be replaced/updated with a function that instead uses something likeiconv()as suggested here https://stackoverflow.com/questions/471021/comparing-strings-in-php-the-...Comment #5
quietone commentedChanges are made on on 11.x (our main development branch) first, and are then back ported as needed according to our policies.
Comment #6
arunkumarkThe issue seems the same as #3419816: Add support for case sensitive validation to UniqueFieldValuesConstraint. On the issue, it was trying to handle the case insensitively with mb_strtoupper(). But am unable to match the words example and éxample.
Below is the observation received on
ISO-8859-1//TRANSLITas per https://stackoverflow.com/questions/471021/comparing-strings-in-php-the-...echo iconv('UTF-8', 'ISO-8859-1//TRANSLIT', 'éxample'); // Output is �xampleI tried this
ASCII//TRANSLITis working as fine. As per, https://stackoverflow.com/questions/24504331/how-to-compare-two-strings-...echo iconv('UTF-8', 'ASCII//TRANSLIT', 'éxample'); // Output is exampleCreating as MR as per the above suggestion. Feel free to update the MR.
Comment #8
arunkumarkComment #9
arunkumarkComment #10
caesius commentedIt sounds like this was probably fixed in the latest Drupal core update, although the vulnerability is only detailed for email addresses, not usernames. https://www.drupal.org/sa-core-2024-004
Comment #11
arunkumark@caesius
Hope the issue with the page break. Still, the issue persists after the https://www.drupal.org/sa-core-2024-004 patch moved. I can replicate it today also on the latest pull.
Hope the issue persists, to handle the exception.
Comment #12
smustgrave commentedIssue summary is missing template sections.
left other comments on MR.
Comment #13
arunkumarkComment #14
arunkumarkComment #15
arunkumarkAs per the #12 comment updated the Issue summary and addressed the changes requested in MR. The pipeline has passed and is moving to NR.
Comment #16
smustgrave commentedAll the open threads still apply so leaving in review.
Comment #17
smustgrave commented1 open thread, tagging for novice as it should be easy for a new user to address.
Comment #18
lavanyatalwar commentedWorking on it.
Comment #19
mxr576@nico.b told me that my #3507777: User names uniqueness is no longer accent-insensitive is actually is a duplicate of this one - I do not know how I did not find this one back at the time, but thanks for intel Nico.
In that issue I had a very similar fix that is being proposed here and because I saw test failures on the MR for other database engines than MariaDB and MYSQL, so I have also triggered test runs on those db engines here and they failed too. So the proposed fix actually needs work:
(source: https://git.drupalcode.org/issue/drupal-3456964/-/pipelines/433653/test_...)
Comment #20
mxr576Also repeating my comment from the other thread, could be useful here, should not be lost.
Comment #22
caesius commentedCurrently working on this with AI help, but noting that #20 suggests a fix that's very similar to issue #3619160 which itself crashed-and-burned because changing collation at the DB level is not the right approach to the problem.
The issue is that different DB drivers have a different handle of near-duplicates due to accents etc., so something that whitescreens on MySQL is perfectly fine on other drivers, even if it's not "correctly" handled from the perspective of needing to prevent impersonation by near-duplication. However, that problem belongs to issue #1518506, not this issue, which is specifically for resolving the whitescreen on MySQL/MariaDB.
The solution is to acknowledge for now the differences in how DB drivers handle these collisions and treat them accordingly. MySQL will prevent registration of a near-duplicate while all the other supported drivers will allow it. Both the code updates and the test suites need to account for this.
Comment #23
caesius commentedComment #24
needs-review-queue-bot commentedThe Needs Review Queue Bot tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".
This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.
Consult the Drupal Contributor Guide to find step-by-step guides for working with issues.