Drupal does not allow W3C compliant language codes where you target a numeric region

It looks like this needs a change to LanguageFormBase::validateCommon().

The following regex pattern (taken from the W3C's RDF 1.1 Concepts and Abstract Syntax, and based on BCP47, seems suitable:

@^[a-zA-Z]{1,8}(-[a-zA-Z0-9]{1,8})*$@

I'd welcome suggestions on how to improve the UX on this.

Re-rolled patch against 8.0-alpha6.

Updated assertion in LanguageCustomLanguageConfigurationTest to match new error message text.

Needs a reroll, and probably tests too.

Rerolled for 8.0-beta3, with an updated error message to match the help text for the 'Language code' field.

@djevans would it be possible to add a new test assertion that shows the current bug, but passes with the fix in place?

+++ b/core/modules/language/src/Form/LanguageFormBase.php
@@ -100,8 +100,11 @@ public function commonForm(array &$form) {
-    if (!isset($form['langcode_view']) && preg_match('@[^a-zA-Z_-]@', $form_state->getValue('langcode'))) {
...
+    if (!isset($form['langcode_view']) && !preg_match('@^[a-zA-Z]{1,8}(-[a-zA-Z0-9]{1,8})*$@', $form_state->getValue('langcode'))) {

Eg, a test that breaks without this change. This should be fairly straightforward by copying this code:

  // Test validation of invalid values.
    $edit = array(
      'predefined_langcode' => 'custom',
      'langcode' => 'white space',
      'label' => '<strong>evil markup</strong>',
      'direction' => LanguageInterface::DIRECTION_LTR,
    );
    $this->drupalPostForm('admin/config/regional/language/add', $edit, t('Add custom language'));

and posting the valid langcode.

If you upload just the test patch (which will fail) and then the combined patch including the fix, this shows a) the fix works and b) the current way is broken :)

OK, as requested:
A patch containing only an updated test (which should fail), and a combined patch with the updated code.

Perfect, thanks @djevans!

I've updated the issue summary with a beta phase evaluation.

I'd like @GaborHojtsy to have a quick look at this before commit. Also, removing the ability to have an underscore? How is migrate going to cope with that?

So the UI already refers to W3C language codes (note not language tags, we refrained from referring to language *tags* so far). It is true that according to the W3C definitions, the language code would be part of the language tag, not the whole...

So to align the validation with that definition makes sense. As for removing the possibility of underscores, we certainly have not been using it on localize.drupal.org. Drupal 7 and 6 are both referring to 'RFC 4646 compliant language identifier. Language codes typically use a country code, and optionally, a script or regional variant name. Examples: "en", "en-US" and "zh-Hant".' with a link to http://www.ietf.org/rfc/rfc4646.txt, which does not define _ as a valid separator either. So I don't know when did we use to tell people to use an underscore there.

As for those migrating sites, the validation only happens when they save the language again, it is on the form. So if they used a language code that they were not supposed to in Drupal 6 or 7 they will need to work around that in Drupal 8 with a form alter or something... I think it makes sense to stop that legacy underscore support sometime.

Let's get a small change record to inform developers that support for _ is being removed.

Also, this issue is a normal bug so we need to outline how it fits within the allowable Drupal 8 beta criteria. Can someone add Drupal 8 beta phase evaluation template to the issue summary.

I added the beta phase evaluation already.

@jhedstrrm: looking at which changes are unfrozen (https://www.drupal.org/core/beta-changes) this does not seem to be any of them; however it looks like this would be a prioritized change given that Drupal 7 should allow for numbers the same way (even if underscores will not be removed there), so the summary needs an update.

I've updated the beta phase evaluation to mention that this is unfrozen due to needing a backport. I also took a stab at adding a change record.

Thanks, edited the change record draft slightly for completeness. I think this looks good to go again.

Committed 7d01643 and pushed to 8.0.x. Thanks!

Thanks for adding the beta evaluation.

Thanks all!

Backport for D7

This was committed

Restoring credits.