Incorrect transliteration of some Russian Cyrillic characters

\core\lib\Drupal\Component\Transliteration\data\ru.php

<?php

/**
 * @file
 * Russian transliteration data for the PhpTransliteration class.
 */

$overrides['ru'] = [
  0x42F => 'JA',
  0x44F => 'ja',
];

The problem is not only with 'я' symbol, but also for some other characters: ё, ж, й, х, ч, ш, щ, ъ, ы, ь, ю, я.
This already was fixed in 7.x version of Transliteration, but seems lost when porting: #357254: Transliteration of Russian letters

So I extend the list with all needed overrides, patch is attached.

Also we must note, that transliteration rules from ru.php file works only when Drupal interface language, so if we try to transliterate Russian word in non-Russian interface language - this rules will not applied.

So we can apply this rules globally, using /core/lib/Drupal/Component/Transliteration/data/x04.php file - this way will work with any interface language and do correct transliteration of all Russian characters.

Here is another patch, that fix problem with Russian symbols transliteration globally, not only for Russian interface language.

I bet we should fix both!

Here is combined patch that fix the problem globally in both files.

It looks great for me! Much more natural to parse.
Assigning to Maintainer!
Looking at #2926187: Better Greek transliteration probably this require @xjm to commit

We only need the overrides if a character needs to be transliterated differently in a specific language. See #567832-52: Transliteration in core and the next comment for a similar question and answer.

Ok, so here is a patch that fix errors only in default values, without touching overrides.

Nice, the patch looks good to me. Passing over to @xjm :)

Would be great to see this backported to 8.5

Temporary testbot hiccup.

Спасибо.

Вот цветовая разница:

Сейчас я смотрю на юикод (это правильно?). Минуточку...

Okay I read over the diff carefully looking at the order of the actual characters in:
https://en.wikipedia.org/wiki/List_of_Unicode_characters#Cyrillic

Everything in the 0x10 through 0x40 (Russian) looks correct. The other rows appear to be Ukranian which I don't read or speak; does anyone else here on the issue? I can try to read up on it but that will take more time. :)

One small question I had about the Russian transliteration. Ц seems to be transliterated as "c". Is that normal/what Russians use when transliterating? As an anglophone I would phonetically write it as "ts".

Thanks!

Hm, both seem to be used a lot, with "c" about twice as frequent as "ts". "Cvety" does give me pictures of flowers though whereas "tsvety" seems to be about some rock band. :)

So looks like (my phonetic assumption nonwithstanding) it is usually "c". So ignore my final question; just the "help please with Ukranian review". :)

Ah, the only thing changed in the Ukranian rows is ё which is missing from the alphabetical order of the Russian, so I think this is correct.

Back to RTBC. I'll probably commit this later today.

Retitling since I don't think we reviewed the rest of the Ukranian. :)

This fix is for common Russian translit
Ц mostly used as C (traditionally) but sounds like "ts" (Tsar)
The same applies to Ч used as ch but most English speakers pronounce it like "tsh" (probably because they listen it more softer then native "change")

Oops, looks like I forgot to come back to this issue. :)

Thanks @andypost and @Chi, makes sense.

Fixing title capitalization and saving issue credit. I thought about whether this might just be a normal bug, but even the А vs. Я by itself is pretty disorienting, so I've kept it as major.

Committed and pushed to 8.6.x. Thanks! I also backported it to 8.5.x as a major bugfix.

Adding credit for @amateescu as well for the review in #11 (thanks @amateescu)!

(The revert and recommit is to add @amateescu to the commit message.)