Problem/Motivation

It lools like utf8mb4_unicode_520_ci implements a newer unicode collation standard, and fixes some shortcomings of current utf8mb4_unicode_ci (see wordpress link below).

Some sources:
* What the folks at wordpress say: #32105 (Database collate should be utf8mb4_unicode_520_ci) – WordPress Trac
* Stackexchange: MySQL Collation utf8_unicode differences - Database Administrators Stack Exchange
* MySQL :: WL#2673: Unicode Collation Algorithm new version (See right in-page tab)
* What the folks at provision module say: Set default collation to utf8mb4_unicode_520_ci

Proposed resolution

Use utf8mb4_unicode_520_ci as default collation if available.

Remaining tasks

TBD

User interface changes

None

API changes

None

Data model changes

None

Comments

axel.rutz created an issue. See original summary.

geek-merlin’s picture

Issue summary: View changes
daffie’s picture

Issue tags: +Needs tests

+1 for me. Drupal is fully multilingual. Drupal uses utf8mb4_general_ci at the moment and that results in: "fails to implement all of the Unicode sorting rules, which will result in undesirable sorting in some situations, such as when using particular languages or characters." See discussion at http://stackoverflow.com/a/766996.

sharique’s picture

Status: Active » Needs review
StatusFileSize
new3.24 KB

Initial patch, in testSchema ASCII string test is failing for me.

sharique’s picture

If I'm correct we need MySql 5.6 for utf8mb4_unicode_520_ci .

Status: Needs review » Needs work

The last submitted patch, 4: 2932820-use-utf8mb4_unicode_520_ci-collation.patch, failed testing. View results

sharique’s picture

Status: Needs work » Needs review
StatusFileSize
new3.15 KB
new1.08 KB

LangcodeToAsciiUpdateTest is failing on permission denied for me, any suggestions. Permission in sites/simpletest are fine.

Status: Needs review » Needs work

The last submitted patch, 7: 2932820-use-utf8mb4_unicode_520_ci-collation-6.patch, failed testing. View results
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

sharique’s picture

Tests are failing because MySql 5.5 does not support 'utf8mb4_unicode_520_ci' collation.

sharique’s picture

I did some experiment using " Database::getConnection()->version();" method, to add checks for version but can't determine appropriate version to use, I'm running Maridb version 5.5.5-10.2.11-MariaDB, which support utf8mb4_unicode_520_ci, but Mysql 5.5 does not support it.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

geek-merlin’s picture

$ drush sqlq "show collation where collation='utf8_unicode_520_ci'"
utf8_unicode_520_ci	utf8	214		Yes	8

;-)

sharique’s picture

Status: Needs work » Needs review
StatusFileSize
new3.39 KB

Here is updated patch.

Status: Needs review » Needs work
sharique’s picture

StatusFileSize
new977 bytes

Thanks, Alex for advice, now we need to updated tests to support for both cases.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

sharique’s picture

Status: Needs work » Fixed

For MySql 8 default collation is set to utf8mb4_0900_ai_ci, so I think this ticket is no more relevant, hence closing.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.