Problem/Motivation

#3456964 and #3507777 Usernames no longer accent-insensitive, and #3419816 Add support for case-sensitive validation all trace back to the same gap: Drupal's PHP-side uniqueness validation (UniqueFieldValueValidator) assumes PHP string equality matches database equality, but MySQL/MariaDB's default collation (utf8mb4_general_ci) is both case- and accent-insensitive while PHP string comparison is neither. A prior fix, #3415582 Error registering a user with different case, patched the PHP-side comparison for case, but the same class of bug recurs for accents and will recur for any other collation-driven equivalence.

#1144644 Specify collation for a database table proposes a general, arbitrary collation-string API for schema columns, but has stalled for over a decade, partly on cross-database portability concerns — MySQL, PostgreSQL, and SQLite each name and support collations differently — and on index/join performance concerns raised in that issue's own discussion.

#1518506 Normalize case sensitivity across database engines explores the related problem of making case-sensitivity behavior uniform across drivers, but its most recent discussion argues against forcing uniform behavior and toward letting call sites opt in explicitly — the direction this issue takes.

Steps to reproduce

N/A — this is a feature proposal, not a bug report. See the attached PDF for a worked mechanism diagram and per-driver DDL comparison.

Proposed resolution

Add an optional, closed-vocabulary schema key — e.g. 'collation' => 'case_sensitive' — on char/varchar/text column definitions, generalizing the pattern already shipped in #1237252 (DB case sensitivity: allow BINARY attribute in MySQL). Each database driver maps the semantic value to its own concrete implementation:

  • MySQL/MariaDB: COLLATE utf8mb4_bin
  • PostgreSQL: COLLATE "C"
  • SQLite: COLLATE BINARY

PostgreSQL and SQLite already default to case- and accent-sensitive comparison, so the flag is a no-op on those two drivers and only changes generated DDL on MySQL/MariaDB — the actual source of #3456964 (equality mismatch between PHP and database) and #3507777 (accent-insensitivity regression).

Remaining tasks

  • Confirm scope and naming of the enum value(s) with maintainers (starting with case_sensitive; open question whether accent_sensitive needs to be a distinct value or is implied)
  • Implement per-driver mapping in each driver's Schema.php (MySQL, PostgreSQL, SQLite)
  • Test coverage across all three core-supported drivers
  • Documentation for contrib/custom module authors

User interface changes

None.

Introduced terminology

  • collation (schema key): an optional key on char/varchar/text column definitions.
  • case_sensitive: the initial enum value, mapped per driver to a byte-sensitive collation.

API changes

New optional schema key, additive and backward-compatible. Omitting the key preserves each driver's current default behavior exactly; no existing schema definition is affected, and altering an existing column's collation still requires an explicit hook_update_N().

Disclosure

This IS was written by AI (surprise!) -- I had intended to rewrite it once a PoC was finished.

CommentFileSizeAuthor
semantic-collation-flag.pdf356.33 KBcaesius

Issue fork drupal-3619160

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

caesius created an issue. See original summary.

caesius’s picture

Issue summary: View changes
caesius’s picture

Status: Active » Closed (won't fix)

This actually fell through at the proof-of-concept stage; the proposed solution doesn't resolve the case/accent sensitivity issue that it was designed for. It "could still be useful" in other cases where sensitivity is needed at the database layer, but I don't think we should preemptively try to address a use case that might not exist yet.

Closing as "won't fix" in favor of following up on #3456964.

Explanation for why the proposal doesn't work: Applying it to users.name would make that column case- and accent-sensitive. The username validator relies on the database's case- and accent-insensitive collation to catch near-duplicates like "Alice" and "alice", or "café" and "cafe". Making the column sensitive removes that protection, allowing the creation of impersonator accounts. Fixing this properly should be done on the PHP validation side.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

caesius’s picture