Problem/Motivation
#3456964 and #3507777 Usernames no longer accent-insensitive, and #3419816 Add support for case-sensitive validation all trace back to the same gap: Drupal's PHP-side uniqueness validation (UniqueFieldValueValidator) assumes PHP string equality matches database equality, but MySQL/MariaDB's default collation (utf8mb4_general_ci) is both case- and accent-insensitive while PHP string comparison is neither. A prior fix, #3415582 Error registering a user with different case, patched the PHP-side comparison for case, but the same class of bug recurs for accents and will recur for any other collation-driven equivalence.
#1144644 Specify collation for a database table proposes a general, arbitrary collation-string API for schema columns, but has stalled for over a decade, partly on cross-database portability concerns — MySQL, PostgreSQL, and SQLite each name and support collations differently — and on index/join performance concerns raised in that issue's own discussion.
#1518506 Normalize case sensitivity across database engines explores the related problem of making case-sensitivity behavior uniform across drivers, but its most recent discussion argues against forcing uniform behavior and toward letting call sites opt in explicitly — the direction this issue takes.
Steps to reproduce
N/A — this is a feature proposal, not a bug report. See the attached PDF for a worked mechanism diagram and per-driver DDL comparison.
Proposed resolution
Add an optional, closed-vocabulary schema key — e.g. 'collation' => 'case_sensitive' — on char/varchar/text column definitions, generalizing the pattern already shipped in #1237252 (DB case sensitivity: allow BINARY attribute in MySQL). Each database driver maps the semantic value to its own concrete implementation:
- MySQL/MariaDB:
COLLATE utf8mb4_bin - PostgreSQL:
COLLATE "C" - SQLite:
COLLATE BINARY
PostgreSQL and SQLite already default to case- and accent-sensitive comparison, so the flag is a no-op on those two drivers and only changes generated DDL on MySQL/MariaDB — the actual source of #3456964 (equality mismatch between PHP and database) and #3507777 (accent-insensitivity regression).
Remaining tasks
- Confirm scope and naming of the enum value(s) with maintainers (starting with
case_sensitive; open question whetheraccent_sensitiveneeds to be a distinct value or is implied) - Implement per-driver mapping in each driver's
Schema.php(MySQL, PostgreSQL, SQLite) - Test coverage across all three core-supported drivers
- Documentation for contrib/custom module authors
User interface changes
None.
Introduced terminology
collation(schema key): an optional key onchar/varchar/textcolumn definitions.case_sensitive: the initial enum value, mapped per driver to a byte-sensitive collation.
API changes
New optional schema key, additive and backward-compatible. Omitting the key preserves each driver's current default behavior exactly; no existing schema definition is affected, and altering an existing column's collation still requires an explicit hook_update_N().
Disclosure
This IS was written by AI (surprise!) -- I had intended to rewrite it once a PoC was finished.
| Comment | File | Size | Author |
|---|---|---|---|
| semantic-collation-flag.pdf | 356.33 KB | caesius |
Issue fork drupal-3619160
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
- 3619160-add-caseaccent-sensitivity-option
changes, plain diff MR !16852
Comments
Comment #3
caesius commentedComment #4
caesius commentedThis actually fell through at the proof-of-concept stage; the proposed solution doesn't resolve the case/accent sensitivity issue that it was designed for. It "could still be useful" in other cases where sensitivity is needed at the database layer, but I don't think we should preemptively try to address a use case that might not exist yet.
Closing as "won't fix" in favor of following up on #3456964.
Explanation for why the proposal doesn't work: Applying it to
users.namewould make that column case- and accent-sensitive. The username validator relies on the database's case- and accent-insensitive collation to catch near-duplicates like "Alice" and "alice", or "café" and "cafe". Making the column sensitive removes that protection, allowing the creation of impersonator accounts. Fixing this properly should be done on the PHP validation side.Comment #6
caesius commented