Part of meta issue #1802750: [Meta] Convert configurable data to ConfigEntity system

Problem/Motivation

The Language module itself uses a custom database table to store languages. This should be converted to CMI Configurable so site maintainers can push new language information to staging/live. Also, we removed native language name specification support earlier in anticipation of the CMI conversion of the language list and translation of language names with CMI.

Proposed resolution

Convert to CMI Configurables.

API changes

Stop using {language} table

#2026227: Separate language_list() usage for maintenance mode

Files: 
CommentFileSizeAuthor
#143 1754246-language-configurables-143.patch232.07 KBwebflo
PASSED: [[SimpleTest]]: [MySQL] 56,392 pass(es). View
#143 1754246-language-configurables-140-143.interdiff.txt2.06 KBwebflo
#140 1754246-language-configurables-140.patch231.65 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] 56,330 pass(es), 0 fail(s), and 27 exception(s). View
#140 1754246-language-configurables-139-140.interdiff.txt8.05 KBwebflo
#139 1754246-language-configurables-139.patch238.97 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] 56,405 pass(es), 0 fail(s), and 27 exception(s). View
#139 1754246-language-configurables-136-138.interdiff.txt471 byteswebflo
#136 1754246-language-configurables-137.patch238.95 KBwebflo
PASSED: [[SimpleTest]]: [MySQL] 57,961 pass(es). View
#135 1754246-language-configurables-135.patch261.34 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View
#135 1754246-language-configurables-132-135.patch2.13 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-language-configurables-132-135.patch. Unable to apply patch. See the log in the details link for more information. View
#132 1754246-language-configurables-132.patch238.08 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] 58,427 pass(es), 10 fail(s), and 0 exception(s). View
#132 1754246-language-configurables-129-132.interdiff.txt7.04 KBwebflo
#129 1754246-language-configurables-129.patch236.07 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] 56,892 pass(es), 48 fail(s), and 65 exception(s). View
#129 1754246-language-configurables-125-129.interdiff.txt3.17 KBwebflo
#127 1754246-language-configurables-119-125.interdiff.txt4.56 KBwebflo
#126 1754246-language-configurables-126.patch234.4 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] 56,717 pass(es), 187 fail(s), and 68 exception(s). View
#125 1754246-language-configurables-125.patch26.74 KBpenyaskito
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View
#123 1754246-language-configurables-119.patch231.38 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] 55,803 pass(es), 420 fail(s), and 405 exception(s). View
#122 interdiff.119-122.txt562 bytespenyaskito
#122 1754246-language-configurables-122.patch230.59 KBpenyaskito
FAILED: [[SimpleTest]]: [MySQL] 54,507 pass(es), 416 fail(s), and 424 exception(s). View
#119 interdiff.117-118.txt3.34 KBpenyaskito
#119 1754246-language-configurables-118.patch230.54 KBpenyaskito
FAILED: [[SimpleTest]]: [MySQL] 51,047 pass(es), 519 fail(s), and 298 exception(s). View
#114 1754246-language-configurables.114.patch230.44 KBpenyaskito
FAILED: [[SimpleTest]]: [MySQL] 55,630 pass(es), 417 fail(s), and 401 exception(s). View
#112 1754246-language-configurables.112.patch224.64 KBpenyaskito
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-language-configurables.112.patch. Unable to apply patch. See the log in the details link for more information. View
#109 1754246-language-configurables.109.patch303.32 KBwebflo
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: failed to login to test site. View
#107 interdiff.txt310.25 KBpenyaskito
#107 1754246-language-configurables.102-107.patch327.19 KBpenyaskito
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-language-configurables.102-107.patch. Unable to apply patch. See the log in the details link for more information. View
#102 language-1754246-102.patch27.37 KBpenyaskito
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch language-1754246-102.patch. Unable to apply patch. See the log in the details link for more information. View
#96 interdiff.txt2.46 KBandypost
#96 language-1754246-96.patch27.39 KBandypost
PASSED: [[SimpleTest]]: [MySQL] 58,195 pass(es). View
#89 interdiff_.txt690 bytesandypost
#89 language-1754246-89.patch27.3 KBandypost
PASSED: [[SimpleTest]]: [MySQL] 58,253 pass(es). View
#87 interdiff.txt4.77 KBandypost
#87 language-1754246-87.patch26.95 KBandypost
FAILED: [[SimpleTest]]: [MySQL] 58,365 pass(es), 0 fail(s), and 1 exception(s). View
#84 interdiff.txt723 bytesandypost
#84 language-1754246-83.patch24.83 KBandypost
FAILED: [[SimpleTest]]: [MySQL] 57,799 pass(es), 8 fail(s), and 60 exception(s). View
#82 interdiff.txt8.71 KBandypost
#82 language-1754246-82.patch24.94 KBandypost
FAILED: [[SimpleTest]]: [MySQL] 57,704 pass(es), 8 fail(s), and 60 exception(s). View
#78 1754246-78.patch23.14 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 54,583 pass(es), 902 fail(s), and 2,612 exception(s). View
#76 1754246-76.patch21.74 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 56,528 pass(es), 48 fail(s), and 72 exception(s). View
#76 interdiff.txt813 bytesswentel
#74 1754246-74.patch20.94 KBswentel
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: PHP Fatal error encountered during run_tests.sh. See review log for details.. View
#69 1754246-69.patch21.08 KBswentel
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-69.patch. Unable to apply patch. See the log in the details link for more information. View
#69 interdiff.txt8.17 KBswentel
#65 1754246-65.patch20.35 KBYesCT
FAILED: [[SimpleTest]]: [MySQL] 55,537 pass(es), 3 fail(s), and 0 exception(s). View
#65 interdiff-64-65.txt1.75 KBYesCT
#64 1754246-64.patch20.66 KBYesCT
FAILED: [[SimpleTest]]: [MySQL] 55,459 pass(es), 3 fail(s), and 0 exception(s). View
#64 interdiff-61-64.txt2.36 KBYesCT
#61 1754246-61.patch20.61 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 55,476 pass(es), 3 fail(s), and 0 exception(s). View
#61 interdiff.txt2.77 KBswentel
#59 1754246-59.patch20.76 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 55,826 pass(es), 9 fail(s), and 9 exception(s). View
#59 interdiff.txt5.74 KBswentel
#57 1754246-57.patch15.57 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 55,881 pass(es), 12 fail(s), and 11 exception(s). View
#57 interdiff.txt4.49 KBswentel
#55 1754246-55.patch13.91 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 55,120 pass(es), 315 fail(s), and 527 exception(s). View
#55 interdiff.txt8.03 KBswentel
#51 1754246-51.patch10.21 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 53,658 pass(es), 914 fail(s), and 279 exception(s). View
#49 1754246-49.patch10.23 KBswentel
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View
#29 26-29-interdiff.txt1.88 KBalexpott
#29 1754246.languages-configurables.29.patch13.65 KBalexpott
FAILED: [[SimpleTest]]: [MySQL] 41,282 pass(es), 3 fail(s), and 0 exception(s). View
#26 24-26-interdiff.txt1.56 KBalexpott
#26 1754246.languages-configurables.26.patch12.3 KBalexpott
FAILED: [[SimpleTest]]: [MySQL] 41,281 pass(es), 5 fail(s), and 2 exception(s). View
#24 1754246.languages-configurables.24.patch11.42 KBalexpott
FAILED: [[SimpleTest]]: [MySQL] 41,284 pass(es), 6 fail(s), and 2 exception(s). View
#19 1754246.languages-configurables.19.patch305.61 KBalexpott
FAILED: [[SimpleTest]]: [MySQL] 35,925 pass(es), 269 fail(s), and 45 exception(s). View
#19 16-19-interdiff.txt283.18 KBalexpott
#16 1754246.languages-configurables.16.patch13.31 KBdagmar
FAILED: [[SimpleTest]]: [MySQL] Invalid PHP syntax in core/modules/language/language.admin.inc. View

Comments

andypost’s picture

Also we have different language-related contexts and language types

Gábor Hojtsy’s picture

locale interface translation - could be converted into "storable entities" and use efq for newly introduced administrative UI

This is not in language module?!

cweagans’s picture

Issue tags: +Configuration system

Re-tagging

sun’s picture

Status: Active » Closed (won't fix)

I'm sorry, but I don't see anything else in Language module or Locale module that could be converted into config aside from the stuff we are currently converting into config already.

Gábor Hojtsy’s picture

Status: Closed (won't fix) » Active

@sun: do you think the configured list of languages themselves should not be in CMI (and therefore should not have translation support, such as the possibility to show a language name it its native form)? Or is that already covered in another issue? That is a key feature we removed temporarily with the explicit intention of getting it back by converting the list to CMI and having CMI language support take care of the rest.

sun’s picture

Title: Separate language Thingies into Storable and Configurabe » Convert Languages into Configurables
Issue tags: -CMI +Configurables

oh, sorry. I thought there was an issue for converting languages already. Yeah, that makes sense.

Clarifying issue title.

mgifford’s picture

Adding related issue #1164682: links with a known language need language identifier to return to after this issue has been fixed.

Gábor Hojtsy’s picture

Priority: Normal » Major
Issue tags: -D8MI-meta +D8MI, +sprint

Bring on sprint. This is important to get back the feature we lost with the removal of native language names (via config translation).

Gábor Hojtsy’s picture

Updated issue summary as well. Who wants to take this on? :)

Gábor Hojtsy’s picture

Issue tags: +language-config
dagmar’s picture

Assigned: Unassigned » dagmar

I'm going to work on this during this week.

Gábor Hojtsy’s picture

@dagmar: how is it going?

dagmar’s picture

I'm on it :)

I have some questions, mostly related to #1216094: We call too many things 'language', clean that up

I'm using the code that @sun wrote here #1588422-84: Convert contact categories to configuration system to convert Languages into Configurables. So basically I have to create a new class called Language that extends ConfigEntityBase.

But it seems that this class already exists in core core/lib/Drupal/Core/Language/Language.php introduced in #1497230: Use Dependency Injection to handle object definitions and it has almost the same properties that the one I'm creating for this issue.

So, the main issue here is, should be provide two different clases for language? One for the configuration handling and other for the language object? If yes, what name should I use for the config class?

Or we just could make the class mentioned above extends the ConfigEntityBase class. What would be the performance implications of doing this?

Also, we have to define a pattern to save the config files. What should I use, the name of the language, for example

language.spanish.yml
language.french.yml

Or the langcode, language.es.yml, language.fr.yml

And finally, again, because we have a lot of things called language, we should define a name to group the languages avialable for the site.

This is because we already have config files that start with 'language', for example, language.mappings.yml, so maybe we could use the name 'site_language' to have something like this:

language.site_language.es.yml <  config for Spanish language
language.site_language.fr.yml  <  config for French language
language.site_language.it.yml  <  config for Italian language

language.mappings.yml  <  general config
Gábor Hojtsy’s picture

I think language.$langcode.yml is fine, mappings, etc. are not valid language codes. I think the substructure proposed looks a bit ugly. We definitely should not use the lowercase English names of the languages, that is derived data (and might change). As for reusing the Language object, if we are not doing that, sounds like when we load or need to save languages, we'd need to convert back and forth, which does not sound good. Loading of languages we do on most if not all page loads already.

dagmar’s picture

Here are some updates. Sorry I cannot provide a patch yet because there are some important decisions to take.

First: We cannot, or better, it wouldn't be nice to have Language as entities, for several reasons:

  1. Entities expect a default language. So...
  2. Entities may have bundles and revisions and a lot of things that I know are optionals but don't really make sense for this kind of configurations.

But, even if we decide to go for the entity path, I still have problems with thinks like $language->default. If this is a property of the entity... Does this means that we will have a columns for that propery in the language table? or maybe there is a way to define properties for entities that are not saved in the db. How can I do that?

Related to the file names, sun suggested via IRC to use language.something.[langcode] to make easier to load all the languages using the prefix language.something.

dagmar’s picture

Status: Active » Needs review
FileSize
13.31 KB
FAILED: [[SimpleTest]]: [MySQL] Invalid PHP syntax in core/modules/language/language.admin.inc. View

Attaching a patch with the work in progress to ilustrate some new issues.

This is not even testeable due ConfigurationEntity is not avilable at install level.

To fix this issue we need the following problem solved.

Alternative 1:

Make ConfigEntityBase as Core component with also implies make Entity a core component.

Alternative 2:

Provide a more generical class to handle objects that are not entities as configurations. And basically replicate a lot of code.

Is there another option to fix this?

Status: Needs review » Needs work

The last submitted patch, 1754246.languages-configurables.16.patch, failed testing.

tim.plunkett’s picture

+++ b/core/lib/Drupal/Core/Language/Language.phpundefined
@@ -16,17 +17,66 @@ namespace Drupal\Core\Language;
+   * Language code, e.g. 'de' or 'en-US'.

e.g. needs a comma after it

+++ b/core/modules/config/config.infoundefined
@@ -3,3 +3,7 @@ description = Allows administrators to manage configuration changes.
 core = 8.x
+; @todo D8: Config module is required for all modules that implement
+;   ConfigEntityBase. Move the entity system and ConfigEntity* into a
+;   Drupal\Core component to remove this required dependency.
+required = TRUE

Is this really needed now? Seems like it's being dealt with in #1760786: Move entity system "back" into a Drupal\Core component

Also, that's a lot of public properties. Couldn't some of them serve better as protected?

alexpott’s picture

Status: Needs work » Needs review
FileSize
283.18 KB
305.61 KB
FAILED: [[SimpleTest]]: [MySQL] 35,925 pass(es), 269 fail(s), and 45 exception(s). View

Gonna see if this patch passes... it does what #1760786: Move entity system "back" into a Drupal\Core component wants to do... but we have the use-case :)

So all the Entity, ContentEntity and ConfigEntity classes have move to Drupal\Core\Entity - cause they are all entities.

re: #18 Language code, e.g. 'de' or 'en-US' is repeated three times in the code so this should be dealt with in a follow up (imho)
re: #18 requirement to enable config has gone!

One interesting question has the requirement to have the entity module enabled also gone?

Status: Needs review » Needs work

The last submitted patch, 1754246.languages-configurables.19.patch, failed testing.

sun’s picture

There are two issues there:

1) Whether the existing Drupal\Core\Language should be the config entity class itself.

2) Every ConfigEntity gets a 'langcode' property injected by default, which clashes with the actual langcode here.

However, 1) cannot work without #1760786: Move entity system "back" into a Drupal\Core component + additionally moving ConfigEntity into Core, AND additionally #1763974: Convert entity type info into plugins... that's a huge dependency chain - let's not block us on that. ;)

For now, I'd recommend to

- Introduce a separate Drupal\language\Language ConfigEntity class that is being used by the admin interface to manage the configured languages.

- Try to use 'langcode' as the id-key of the Language config entity. If that works, good. Otherwise, resort to using 'id' as id-key.

alexpott’s picture

After further investigations with the patch in #19 I would definitely echo the recommendation to introduce a separate Drupal\language\Language ConfigEntity class. There are some really difficult issues with making the current Language object a confgEntity because every entity can create a Language object in the entity->language() method. This leads to a dependency headache and lots of Fatal error: Maximum function nesting level of '100' reached, aborting! messages.

The patch in #19 should not be followed up. I'm not working on this issue anymore unless @dagmar unassigns himself...

dagmar’s picture

Assigned: dagmar » Unassigned

@alexpott go ahead. I prefer that more experienced people in CMI work on this issue due it was more complicated that I expected in first place. I will be helping in other CMI issues in the meanwhile.

alexpott’s picture

Status: Needs work » Needs review
FileSize
11.42 KB
FAILED: [[SimpleTest]]: [MySQL] 41,284 pass(es), 6 fail(s), and 2 exception(s). View

Patch attached is a work-in-progress and will have test failures :( around language negotiation for some reasons that, at the moment, are not 100% apparent to me.

What the patch achieves:

  • Creation of language config entities
  • Upgrade of d7 site with languages to d8
  • Removal of the language table

Things discovered:

  • Using uasort and having an excpetion thrown and caught is a pain (see https://bugs.php.net/bug.php?id=50688). This happens if you use ConfigEntityBase::sort() instead of the new language_entity_sort() function. This is because language_list() gets called at a very low level of bootstrap and for some reason entity->label() calls the language() function which will catch a DependencyInjectionRuntimeException because the language_manager is not available yet!
  • system_update_8000() needs to be convert the existing languages as these have to be available asap.

No interdiff attached as this is a complete re-roll based on the limited scope discussed in #21

Status: Needs review » Needs work

The last submitted patch, 1754246.languages-configurables.24.patch, failed testing.

alexpott’s picture

FileSize
12.3 KB
FAILED: [[SimpleTest]]: [MySQL] 41,281 pass(es), 5 fail(s), and 2 exception(s). View
1.56 KB

Some of the tests fixed :)

alexpott’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 1754246.languages-configurables.26.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
13.65 KB
FAILED: [[SimpleTest]]: [MySQL] 41,282 pass(es), 3 fail(s), and 0 exception(s). View
1.88 KB

the problem seems to be in entity_get_info using the language() function. So we need to have an exception for language entities.

 function entity_get_info($entity_type = NULL) {
-  $language_interface = language(LANGUAGE_TYPE_INTERFACE);
-
+  // We need a special exception for the language config entity as this call to
+  // language will lead to the turtles gettings language negotiaiton wrong.
+  if ($entity_type != 'language') {
+    $language_interface = language(LANGUAGE_TYPE_INTERFACE);
+  }
+  else {
+    $language_interface = new stdClass;
+    $language_interface->langcode = LANGUAGE_NOT_SPECIFIED;
+  }

Additionally since language_list() is statically cached there is no reason to use the entity static cache here either... in fact it might result in unexpected results if we do - although I'm not 100% on this. Since if we do all the language config entity manipulation through the entity sub system - then we should be able to rely on the cache?

Status: Needs review » Needs work

The last submitted patch, 1754246.languages-configurables.29.patch, failed testing.

Jose Reyero’s picture

I think this is not a case of adding php code to handle edge cases but one where the current config system (or at least the config entities) are failing for our purposes.

Since Entities depend on language, we cannot pretend to store languages as an entity. This is a conceptual issue and any code we add to handle it will be ugly. Also the dependencies schema this creates doesn't make sense at all since basically we need to load the full entity system for bootstrapping.

So IMHO either the config system should provide a way to handle storage of simple objects like languages without depending on heavyweight objects/systems like Entity, or we should handle 'manually' language storage, just like image styles are done atm (though I guess that will be converted to config entities too, right?)

catch’s picture

We should not be using entity_get_info() for both ConfigEntity and ContentEntity. As soon as any content Entity bundle is converted to config, then it's going to explode there too (or look at taxonomy_vocabulary_get_names() and taxonomy_entity_info() for the Drupal 7 version of that problem). This is also a good reason to keep them as separate thingies instead of merging them later, unless we somehow find a way to replace the current info hook for entities with something else that doesn't have this infinite recursion issue.

dagmar’s picture

from #31

So IMHO either the config system should provide a way to handle storage of simple objects like languages without depending on heavyweight objects/systems like Entity, or we should handle 'manually' language storage,

Is this possible without have to duplicate a lot of the entity system code?

from #32

We should not be using entity_get_info() for both ConfigEntity and ContentEntity

What do should we use then?

catch’s picture

We can have hook_content_entity_info() and hook_config_entity_info(), something like that?

Jose Reyero’s picture

Well, we can have Entities that are not Entities, and we can add a big warning comment in the code like 'These are not real entities, don't use them as such'....

Just joking... (But really, Config_Entity extends Entity)

1. Instead of making it worse and more complicated why don't we assume we've placed some functionaliaty in the wrong place, and fix that (Entity extends ExportableThing) ?
2. I guess we should be creating another thread to discuss these issues that are CMI specific (for which I don't have the time today, sorry about raising the issue in the wrong place...)

catch’s picture

(But really, Config_Entity extends Entity)

Yes the idea is that the lowest common denominator functionality lives in Entity, and ConfigEntity and ContentEntity then extend that if they need separate stuff, how much of that there is we don't know yet. Regardless, there needs to be different info hook for these - for a start ConfigEntity has hard-coded storage controller to CMI, whereas ContentEntity you can swap it around.

dagmar’s picture

sun’s picture

#1760786: Move entity system "back" into a Drupal\Core component resolved the primary blocker, essentially making the entity system available very early. (Not fully done yet, but the goal is clear.)

#1763974: Convert entity type info into plugins will remove the module/hook system dependency from entity types.

And lastly, but I don't think it's relevant here, #1782460: Allow to reference another entity type to define an entity type's bundles intends to allow to use one entity type as the 'bundles' definition for another entity type.

There is no issue for removing the hard-coded enforcement of language support from entities yet, but given this issue here, I think that should (or has to) happen. Thus, I've created one:
#1782472: Hard-coded and enforced language support for entities conflicts with Language as a ConfigEntity

fago’s picture

#1763974: Convert entity type info into plugins will remove the module/hook system dependency from entity types.

Yep, it would be nice if this would get us out of recursion problems.

andypost’s picture

Also this depends on #1785974: Move ConfigEntity into a Core component
because there's no way to load config module without language init

catch’s picture

Just committed that.

Gábor Hojtsy’s picture

I think this technically could be done after December 1st, its not a new feature. It does help us restore the feature to translate language names (which we need to avoid a regression given we removed native language names in anticipation of this feature). Removing from sprint to focus on real new features and tagging with regression and revisit before release to keep track of the problem (hope these are the right tags).

xjm’s picture

Priority: Major » Normal
xjm’s picture

Issue summary: View changes

Update according to current plans.

xjm’s picture

plach’s picture

andypost’s picture

plach’s picture

Yes, saw that, thanks :)

andypost’s picture

Suppose there should be line when config entity gets "translatability" - when there's more then 1 active language and translation sub-system available\accessible

Pure config entity load (it seems) should not use language entity/system, probably it depends on some config metadata...

swentel’s picture

Status: Needs work » Needs review
FileSize
10.23 KB
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View

New patch, just to see how far we can get currently.

Status: Needs review » Needs work

The last submitted patch, 1754246-49.patch, failed testing.

swentel’s picture

Status: Needs work » Needs review
FileSize
10.21 KB
FAILED: [[SimpleTest]]: [MySQL] 53,658 pass(es), 914 fail(s), and 279 exception(s). View

untested, last minute changes are never a good idea

Status: Needs review » Needs work

The last submitted patch, 1754246-51.patch, failed testing.

Gábor Hojtsy’s picture

Yay, thanks for continuing with this :) My only question is why does it need to implement ArrayAccess? Languages are objects prior to this patch universally, so wherever there is array access attempted, that would be in itself a problem.

swentel’s picture

+++ b/core/includes/bootstrap.incundefined
@@ -2710,13 +2710,14 @@ function language_list($flags = LANGUAGE_CONFIGURABLE) {
       // Initialize default property so callers have an easy reference and can
       // save the same object without data loss.
-      foreach ($languages as $langcode => $info) {
+      foreach ($languages as $info) {
+        $langcode = $info['langcode'];
         $info['default'] = ($langcode == $default->langcode);
-        $languages[$langcode] = new Language($info);
+        $languages[$langcode] = new Language($info->getExportProperties());

@gabor That was for this snippet in bootstrap.inc , $info apparently is an array in HEAD, so when I started I was just adding that as a BC layer because I thought it would become an enormous patch, turns out, it probably isn't, so I'll remove that and fix failtures tonight :)

- edit - removed extra dreditor paste

swentel’s picture

Status: Needs work » Needs review
FileSize
8.03 KB
13.91 KB
FAILED: [[SimpleTest]]: [MySQL] 55,120 pass(es), 315 fail(s), and 527 exception(s). View

Some obvious fixes, let's see now. ArrayAccess layer removed also.

Status: Needs review » Needs work

The last submitted patch, 1754246-55.patch, failed testing.

swentel’s picture

Status: Needs work » Needs review
FileSize
4.49 KB
15.57 KB
FAILED: [[SimpleTest]]: [MySQL] 55,881 pass(es), 12 fail(s), and 11 exception(s). View

Will fix a lot, effectively using 'language.entity' as prefix

Status: Needs review » Needs work

The last submitted patch, 1754246-57.patch, failed testing.

swentel’s picture

Status: Needs work » Needs review
FileSize
5.74 KB
20.76 KB
FAILED: [[SimpleTest]]: [MySQL] 55,826 pass(es), 9 fail(s), and 9 exception(s). View

Should be down to 2 failures - biggest change is the removal of views integration of {language}

Status: Needs review » Needs work

The last submitted patch, 1754246-59.patch, failed testing.

swentel’s picture

FileSize
2.77 KB
20.61 KB
FAILED: [[SimpleTest]]: [MySQL] 55,476 pass(es), 3 fail(s), and 0 exception(s). View

Should be green.

Short version: Stupid langcode property.

Long version: don't use properties on config entities in case it's already used by the config entity system. I was using 'langcode' as the main id, but of course, during upgrade, there's two set() calls doing 'langcode' ...

swentel’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 1754246-61.patch, failed testing.

YesCT’s picture

Status: Needs work » Needs review
FileSize
2.36 KB
20.66 KB
FAILED: [[SimpleTest]]: [MySQL] 55,459 pass(es), 3 fail(s), and 0 exception(s). View

I know there are a few tests failing to fix yet, but I took a look standards wise in the mean time. Patch attached for these things.

+++ b/core/includes/bootstrap.incundefined
@@ -2649,14 +2649,22 @@ function language_list($flags = LANGUAGE_CONFIGURABLE) {
+      // Use language module configuration if available. Always reset the
+      // entity cache as the language objects are statically cached.

this might wrap closer to 80 chars.

+++ b/core/modules/language/language.installundefined
@@ -117,3 +70,32 @@ function language_disable() {
+  foreach ($result as $language) {
+
+    config('language.entity.' . $language->langcode)

extra blank line.

+++ b/core/modules/language/language.moduleundefined
@@ -209,6 +209,18 @@ function language_entity_supported() {
+ * Helper callback for uasort() to sort configuration entities by weight and label.
+ */
+function language_entity_sort($a, $b) {

missing @param and @return
that's probably blocking as it's a core gate minimum for documentation:
http://drupal.org/core-gates#documentation-block-requirements

Also, the one line summary is longer than 80 chars.
http://drupal.org/node/1354#drupal

Isn't callback a menu or router thing? I think this is just a helper.

Actually, figuring out the type (besides just language entity in words) is kind of tricky, and the other sort helper I found (function _system_sort_requirements($a, $b) {
) doesn't have @param or @return either, so I didn't add them.

+++ b/core/modules/language/language.moduleundefined
@@ -469,23 +481,37 @@ function language_get_default_langcode($entity_type, $bundle) {
  * @param $language
- *   Language object with properties corresponding to 'language' table columns.
+ *   Language object with properties corresponding to the
+ *   'language' configuration properties.
  */
 function language_save($language) {

this can get wrapped closer to 80 chars.
(adding type to the @param and type hinting is probably out of scope).

YesCT’s picture

FileSize
1.75 KB
20.35 KB
FAILED: [[SimpleTest]]: [MySQL] 55,537 pass(es), 3 fail(s), and 0 exception(s). View

ok, a bit more thought, and a google for "callback" (http://en.wikipedia.org/wiki/Callback_(computer_programming)
and I think callback is exactly this thing, a function name passed as an arg.

I looked around for more ($a, $b) kind of functions
grep -R "(\$a, \$b)" * | grep " function "
and one of them is almost identical to this:
sort() in core/lib/Drupal/Core/Config/Entity/ConfigEntityBase.php

tim.plunkett had a good idea in irc. This solves the @param dilema also.

Status: Needs review » Needs work

The last submitted patch, 1754246-65.patch, failed testing.

tim.plunkett’s picture

+++ b/core/includes/bootstrap.incundefined
@@ -2649,14 +2649,29 @@ function language_list($flags = LANGUAGE_CONFIGURABLE) {
+      uasort($language_entities, function ($a, $b) {
+        $a_weight = isset($a->weight) ? $a->weight : 0;
+        $b_weight = isset($b->weight) ? $b->weight : 0;
+        if ($a_weight == $b_weight) {
+          return strnatcasecmp($a->name, $b->name);
+        }
+        return ($a_weight < $b_weight) ? -1 : 1;
+      });

Oh, my apologies, I made this recommendation out of context on IRC. This should just use Language::sort($language_entities);

+++ b/core/includes/bootstrap.incundefined
@@ -2649,14 +2649,29 @@ function language_list($flags = LANGUAGE_CONFIGURABLE) {
+        $languages[$langcode] = new Language(array(
+          'default' => ($langcode == $default->langcode),
+          'name' => $info->name,
+          'langcode' => $langcode,
+          'direction' => $info->direction,
+          'locked' => $info->locked,
+          'weight' => $info->weight,

Why not new Language((array) $info); or entity_create('language', (array) $info)?

+++ b/core/modules/language/language.moduleundefined
@@ -469,23 +469,37 @@ function language_get_default_langcode($entity_type, $bundle) {
 function language_save($language) {

This should be divided into LanguageStorageController::preSave() and LanguageStorageController::postSave().

+++ b/core/modules/language/lib/Drupal/language/Plugin/Core/Entity/Language.phpundefined
@@ -0,0 +1,78 @@
+ * Contains \Drupal\field\Plugin\Core\Entity\Language.

s/field/language

+++ b/core/modules/language/lib/Drupal/language/Plugin/Core/Entity/Language.phpundefined
@@ -0,0 +1,78 @@
+ *     "label" = "name",
...
+  /**
+   * The human-readable label for the language.
+   *
+   * @var string
+   */
+  public $name;

This needs an @todo to rename from name to label.

+++ b/core/modules/language/lib/Drupal/language/Plugin/Core/Entity/Language.phpundefined
@@ -0,0 +1,78 @@
+class Language extends ConfigEntityBase {

This needs a Drupal\language\LanguageInterface.

+++ b/core/modules/system/lib/Drupal/system/Tests/Module/DependencyTest.phpundefined
@@ -39,8 +39,9 @@ function testEnableWithoutDependency() {
+    $storage = drupal_container()->get('config.storage');

$this->container

+++ b/core/includes/bootstrap.incundefined
@@ -2652,7 +2652,14 @@ function language_list($flags = LANGUAGE_CONFIGURABLE) {
+      uasort($language_entities, function ($a, $b) {
+        $a_weight = isset($a->weight) ? $a->weight : 0;
+        $b_weight = isset($b->weight) ? $b->weight : 0;
+        if ($a_weight == $b_weight) {
+          return strnatcasecmp($a->name, $b->name);
+        }
+        return ($a_weight < $b_weight) ? -1 : 1;

Oh, my apologies, I made this recommendation out of context on IRC. This should just use Language::sort($language_entities);

swentel’s picture

+++ b/core/includes/bootstrap.incundefined
@@ -2649,14 +2649,29 @@ function language_list($flags = LANGUAGE_CONFIGURABLE) {
+ $languages[$langcode] = new Language(array(
+ 'default' => ($langcode == $default->langcode),
+ 'name' => $info->name,
+ 'langcode' => $langcode,
+ 'direction' => $info->direction,
+ 'locked' => $info->locked,
+ 'weight' => $info->weight,

That's not the language config entity, but Drupal\Core\Language\Language - Confusing as hell for now, this definitely needs to be made clearer :)

swentel’s picture

Status: Needs work » Needs review
FileSize
8.17 KB
21.08 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-69.patch. Unable to apply patch. See the log in the details link for more information. View

Fixed the last remaining tests. Renamed the entity id to 'language_entity', because, although unconfirmed (but most likely), probably raises some conflicts in that sense that it triggers some hooks (like locale_language_insert etc) it shouldn't. This makes sure it doesn't conflict at all.

Most stuff from #67 addressed , besides the sort, preSave() and postSave() and #68, which is not a big issue afaics - note the 'langcode' is important here, so passing on $info would not work at all.

Let's first see if the bot really like this one and then see if we can do more (list, moving calls, tests).

jibran’s picture

Status: Needs review » Needs work
Issue tags: +regression, +revisit before beta, +Configuration system, +D8MI, +language-config, +Configurables

The last submitted patch, 1754246-69.patch, failed testing.

Gábor Hojtsy’s picture

Issue tags: +sprint

This is an amazing patch! What else needs to be done to make this happen. It should happen before the API freeze :) @swentel, can you work on rerolling?

swentel’s picture

Assigned: Unassigned » swentel

Will reroll this weekend and address the remaining points.

We miss an upgrade path too I guess ..

- edit - upgrade path test I mean

swentel’s picture

Status: Needs work » Needs review
FileSize
20.94 KB
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: PHP Fatal error encountered during run_tests.sh. See review log for details.. View

Suprised this was easy to reroll. Nothing address so far, going to bed now. I will look more tomorrow and/or sunday.

Also regarding my comment in #73, there are a lot of language upgrade tests already, so I guess I could just add a couple of lines in testLanguageUpgrade() to test whether the config files contain the content that we expect.

Status: Needs review » Needs work

The last submitted patch, 1754246-74.patch, failed testing.

swentel’s picture

Status: Needs work » Needs review
FileSize
813 bytes
21.74 KB
FAILED: [[SimpleTest]]: [MySQL] 56,528 pass(es), 48 fail(s), and 72 exception(s). View

Bot crashed on disk space + fix in EntityQueryTest

Status: Needs review » Needs work

The last submitted patch, 1754246-76.patch, failed testing.

swentel’s picture

Assigned: swentel » Unassigned
Status: Needs work » Needs review
FileSize
23.14 KB
FAILED: [[SimpleTest]]: [MySQL] 54,583 pass(es), 902 fail(s), and 2,612 exception(s). View

So yeah, calling entity_load_multiple() in language_list() can't happen anymore, because that doesn't work during upgrades.

Added a test for comparing the config after an upgrade. Sorry for the lack of interdiff, I completely messed up my local install.

Gabor: can't take this further this weekend though, so someone else should bring it home, it's close if you tell me.

Missing things:
- move usort to Language
- move bits in language_save() to LanguageStorageController::preSave() and LanguageStorageController::postSave(), see #67 - although there's much to say to todo that in a follow imho.

Status: Needs review » Needs work

The last submitted patch, 1754246-78.patch, failed testing.

Gábor Hojtsy’s picture

So yeah, calling entity_load_multiple() in language_list() can't happen anymore, because that doesn't work during upgrades.

I don't think we should even call API functions like language_list() in update functions? Instead have the update-safe replacement in the update only? :)

andypost’s picture

Assigned: Unassigned » andypost

Mostly the troubles caused by not loading actual config and namespace collisions

+++ b/core/includes/bootstrap.incundefined
@@ -2500,14 +2500,29 @@ function language_list($flags = Language::STATE_CONFIGURABLE) {
+      $language_entities = config_get_storage_names_with_prefix('language.entity');
+      uasort($language_entities, function ($a, $b) {
+        $a_weight = isset($a['weight']) ? $a['weight'] : 0;
+        $b_weight = isset($b['weight']) ? $b['weight'] : 0;
+        if ($a_weight == $b_weight) {
+          return strnatcasecmp($a['label'], $b['label']);
+        }
+        return ($a_weight < $b_weight) ? -1 : 1;

There's no config loaded, just names

andypost’s picture

Status: Needs work » Needs review
FileSize
24.94 KB
FAILED: [[SimpleTest]]: [MySQL] 57,704 pass(es), 8 fail(s), and 60 exception(s). View
8.71 KB

So let's see

Status: Needs review » Needs work

The last submitted patch, language-1754246-82.patch, failed testing.

andypost’s picture

Assigned: andypost » Unassigned
Status: Needs work » Needs review
FileSize
24.83 KB
FAILED: [[SimpleTest]]: [MySQL] 57,799 pass(es), 8 fail(s), and 60 exception(s). View
723 bytes

Clean-up check, have no time to continue (leaving for 2 days)

Status: Needs review » Needs work

The last submitted patch, language-1754246-83.patch, failed testing.

Gábor Hojtsy’s picture

Changes look good to me, keep this up :)

andypost’s picture

Status: Needs work » Needs review
FileSize
26.95 KB
FAILED: [[SimpleTest]]: [MySQL] 58,365 pass(es), 0 fail(s), and 1 exception(s). View
4.77 KB

Another try to fix tests:
1) language_save() could not be used in install time, so added default language config files
2) test LocalePluralFormatTest fixed to use correct language (filter was reseted to "fr")
3) some cleanups

Status: Needs review » Needs work

The last submitted patch, language-1754246-87.patch, failed testing.

andypost’s picture

Status: Needs work » Needs review
FileSize
27.3 KB
PASSED: [[SimpleTest]]: [MySQL] 58,253 pass(es). View
690 bytes

Should be green

Status: Needs review » Needs work
Issue tags: -regression, -revisit before beta, -Configuration system, -D8MI, -sprint, -language-config, -Configurables

The last submitted patch, language-1754246-89.patch, failed testing.

tim.plunkett’s picture

Status: Needs work » Needs review
Issue tags: +regression, +revisit before beta, +Configuration system, +D8MI, +sprint, +language-config, +Configurables

#89: language-1754246-89.patch queued for re-testing.

Gábor Hojtsy’s picture

Status: Needs review » Needs work

Looks great. I don't think I have major things, here are my notes:

+++ b/core/includes/bootstrap.incundefined
@@ -2476,15 +2476,25 @@ function language_list($flags = Language::STATE_CONFIGURABLE) {
+      // Use language module configuration if available. We can not use
+      // entity_load_multiple() because this breaks during updates.
+      $language_entities = config_get_storage_names_with_prefix('language.entity');

Should we just not use language_list() at all in upgrades and use entity_load_multiple() here?

+++ b/core/modules/language/language.moduleundefined
@@ -471,23 +471,35 @@ function language_get_default_langcode($entity_type, $bundle) {
+  // Assign language properties to language entity.
+  $language_entity->label = isset($language->name) ? $language->name : '';

I assume we should have a followup for renaming this in calling code?

+++ /dev/nullundefined
@@ -1,108 +0,0 @@
-<?php
-
-/**
- * @file
- * Provide views data and handlers for language.module.
- *
- * @ingroup views_module_handlers
- */
-
-/**
- * Implements hook_views_data().
- */

Is there generic views config entity support that could replicate this, or this is just a feature removed?

+++ b/core/modules/language/lib/Drupal/language/Plugin/Core/Entity/Language.phpundefined
@@ -0,0 +1,79 @@
+ *   id = "language_entity",
+ *   label = @Translation("Language"),
+ *   module = "language",
+ *   controllers = {
+ *     "storage" = "Drupal\Core\Config\Entity\ConfigStorageController"
+ *   },

Sounds like we also need followups to convert list/form controller as well, right?

+++ b/core/modules/language/lib/Drupal/language/Plugin/Core/Entity/Language.phpundefined
@@ -0,0 +1,79 @@
+  /**
+   * The direction of language (Left-to-Right = 0, Right-to-Left = 1).

This has constants in Language::, let's document those!

+++ b/core/modules/language/lib/Drupal/language/Plugin/Core/Entity/Language.phpundefined
@@ -0,0 +1,79 @@
+   * Flag indicating whether this language is locked or not.
+   *

Add: "Locked languages cannot be edited." to explain what this means IMHO :)

catch’s picture

While it'd be great to get this in before code freeze, I'd consider it a release blocker via #1802750: [Meta] Convert configurable data to ConfigEntity system.

Should we just not use language_list() at all in upgrades and use entity_load_multiple() here?

Yes!

There's a variable_get() in here for the language count, that needs to move to state at some point, doesn't have to be here but I'm not sure which issue is dealing with that if not this one.

Gábor Hojtsy’s picture

I don't think this issue should necessarily cover the count variable. I think I've seen an issue for it already, but cannot find right now. Maybe there is none...

andypost’s picture

Assigned: Unassigned » andypost

If language_list() is api that is not available in update/[un]install so we needs some update_language_list() for consistency
So $language_entity->label = isset($language->name) could be moved to helper without follow-up

hook_views_data() is not needed because there's no language table, but there's some plugins.

About variables #1827038: Remove stale references to language_content_type variable and state #1856976: Convert language_count to the state system.
Conversions for UI: #2005778: Convert language_admin_overview_form to a Controller & #2003592: Convert language_admin_add_form and language_admin_edit_form to a Controllers & #1946426: Convert all of confirm_form() in language.admin.inc to the new form interface

andypost’s picture

Status: Needs work » Needs review
FileSize
27.39 KB
PASSED: [[SimpleTest]]: [MySQL] 58,195 pass(es). View
2.46 KB

Minor changes for #92
Filed follow-up #2026227: Separate language_list() usage for maintenance mode because there's a 93 usages of language_list() in core

andypost’s picture

Issue summary: View changes

Updated issue summary.

andypost’s picture

I'd like to postpone this on #1862202: Objectify the language system - this brings much cleaner approach, and I'll try top swap storage to config on top of the #1862202-102: Objectify the language system

Gábor Hojtsy’s picture

It would be great not to postpone patches anymore on each other. We have 6 days to get patches like this committed in Drupal core.

andypost’s picture

Actually this is a separate task so no matter which patch goes in first

Gábor Hojtsy’s picture

All right, lets get the remaining pieces done then here and get this in! #1862202: Objectify the language system is being worked on heavily still.

Gábor Hojtsy’s picture

Status: Needs review » Needs work
+++ b/core/includes/bootstrap.incundefined
@@ -2478,6 +2478,7 @@ function language_list($flags = Language::STATE_CONFIGURABLE) {
     if (language_multilingual() || module_exists('language')) {
       // Use language module configuration if available. We can not use
       // entity_load_multiple() because this breaks during updates.
+      // @todo Use entity_load_by_properties() https://drupal.org/node/2026227
       $language_entities = config_get_storage_names_with_prefix('language.entity');

I don't think we should defer this todo to a follow, it should be done here. Other followups look good.

penyaskito’s picture

FileSize
27.37 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch language-1754246-102.patch. Unable to apply patch. See the log in the details link for more information. View

Plain reroll of #96.

penyaskito’s picture

Status: Needs work » Needs review

Testbot, please test #102.

tim.plunkett’s picture

Needs follow-ups for ListController and FormController.
The language_admin_add_form conversion blocks removal of MENU_LOCAL_ACTION.

penyaskito’s picture

We hit a big issue here: langcode is the property in language, but also the language that the configuration is in, so we have a collision here. We agree about renaming langcode of language to id before we expected (#1512424: Add a LanguageInterface, and property setters/getters to Language class).

penyaskito’s picture

Assigned: andypost » penyaskito
penyaskito’s picture

FileSize
327.19 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-language-configurables.102-107.patch. Unable to apply patch. See the log in the details link for more information. View
310.25 KB

This is still WIP, but we can see the progress here.

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables.102-107.patch, failed testing.

webflo’s picture

Status: Needs work » Needs review
FileSize
303.32 KB
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: failed to login to test site. View

Fixed the exception in DatabaseStorageControllerNG. Still WIP.

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables.109.patch, failed testing.

andypost’s picture

+++ b/.htaccessundefined
@@ -107,7 +107,7 @@ DirectoryIndex index.php index.html index.htm
-  # RewriteBase /
+  RewriteBase /

this breaks testing

+++ b/core/includes/bootstrap.incundefined
@@ -2396,7 +2396,7 @@ function drupal_installation_attempted() {
-  Drupal::translation()->setDefaultLangcode($language_manager->getLanguage(Language::TYPE_INTERFACE)->langcode);
+  Drupal::translation()->setDefaultLangcode($language_manager->getLanguage(Language::TYPE_INTERFACE)->id);

so now ->id needs to be changed to ->id() this will helps mane this property like we wanna to store

penyaskito’s picture

Status: Needs work » Needs review
FileSize
224.64 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-language-configurables.112.patch. Unable to apply patch. See the log in the details link for more information. View

Two tests failing locally, lets see what the testbot thinks. #111 not taken into account (yet).
Sorry, no interdiff.

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables.112.patch, failed testing.

penyaskito’s picture

FileSize
230.44 KB
FAILED: [[SimpleTest]]: [MySQL] 55,630 pass(es), 417 fail(s), and 401 exception(s). View

Rerolled.

penyaskito’s picture

Status: Needs work » Needs review
penyaskito’s picture

andypost’s picture

Awesome!!! just minor troubles

+++ b/core/modules/language/config/language.entity.en.ymlundefined
@@ -0,0 +1,7 @@
+id: en

+++ b/core/modules/language/config/language.entity.und.ymlundefined
@@ -0,0 +1,7 @@
+id: und

+++ b/core/modules/language/config/language.entity.zxx.ymlundefined
@@ -0,0 +1,7 @@
+id: zxx

now this needs uuids!

+++ b/core/modules/language/language.moduleundefined
@@ -523,7 +535,14 @@ function language_save($language) {
+  $languages = entity_load_multiple('language_entity', NULL, TRUE);
+  foreach ($languages as $language) {

@@ -801,16 +818,23 @@ function language_set_browser_drupal_langcode_mappings($mappings) {
+  $languages = entity_load_multiple('language_entity', NULL, TRUE);
+  foreach ($languages as $language) {

foreach (entity_load_multiple() as $language)

+++ b/core/modules/language/language.moduleundefined
@@ -801,16 +818,23 @@ function language_set_browser_drupal_langcode_mappings($mappings) {
 function language_update_locked_weights() {
...
+  $languages = entity_load_multiple('language_entity', NULL, TRUE);
+  foreach ($languages as $language) {
+    if (!$language->locked && $language->weight > $max_weight) {
...
   // Loop locked languages to maintain the existing order.
   foreach (language_list(Language::STATE_LOCKED) as $language) {
     $max_weight++;
...
+    config('language.entity.' . $language->id)
+      ->set('weight', $max_weight)
+      ->save();

any reason to load all languages twice?

maybe $language_entity->set('weight', $max_weight)->save();

+++ b/core/modules/language/lib/Drupal/language/Plugin/Condition/Language.phpundefined
@@ -39,7 +39,7 @@ public function buildForm(array $form, array &$form_state) {
         // @todo $language->name is not wrapped with t(), it should be replaced
         //   by CMI translation implementation.
-        $langcodes_options[$language->langcode] = $language->name;
+        $langcodes_options[$language->id] = $language->name;

probably $language->label() will help here

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables.114.patch, failed testing.

penyaskito’s picture

Status: Needs work » Needs review
FileSize
230.54 KB
FAILED: [[SimpleTest]]: [MySQL] 51,047 pass(es), 519 fail(s), and 298 exception(s). View
3.34 KB

1. For generating the uuids, I used: drush php-eval '$x = new \Drupal\Component\Uuid\Uuid(); print_r($x->generate() . PHP_EOL)'
2. OK.
3.

+++ b/core/modules/language/language.moduleundefined
@@ -801,16 +818,23 @@ function language_set_browser_drupal_langcode_mappings($mappings) {
function language_update_locked_weights() {
...
+ $languages = entity_load_multiple('language_entity', NULL, TRUE);
+ foreach ($languages as $language) {
+ if (!$language->locked && $language->weight > $max_weight) {
...
// Loop locked languages to maintain the existing order.
foreach (language_list(Language::STATE_LOCKED) as $language) {
$max_weight++;
...
+ config('language.entity.' . $language->id)
+ ->set('weight', $max_weight)
+ ->save();
any reason to load all languages twice?

Not really, I changed both to language_list($flag) because IMHO makes the code more legible. There shouldnt be any performance problem because of drupal_static there, but we can change that in a follow-up if needed.

4. Done.

andypost’s picture

I think this RTBC, only $language->id to $language_entity->id() questionable

Also here's a nice follow-up to clean-up usage of Language as $language and LanguageEntity $language_entity to make code more readable, this out of scope for conversion

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables-118.patch, failed testing.

penyaskito’s picture

Status: Needs work » Needs review
FileSize
230.59 KB
FAILED: [[SimpleTest]]: [MySQL] 54,507 pass(es), 416 fail(s), and 424 exception(s). View
562 bytes

Reverted the way of setting the weight.

webflo’s picture

FileSize
231.38 KB
FAILED: [[SimpleTest]]: [MySQL] 55,803 pass(es), 420 fail(s), and 405 exception(s). View

Found one issue in language_update_locked_weights().

Gábor Hojtsy’s picture

Status: Needs review » Needs work

I think this whole langcode => id change proposal turned out to be way bigger a change than anticipated... So we can just do what the patch always attempted to. Make the config language entity different from the runtime Language object representation. We can at least temporarily make the Drupal\language\Language::langcode be the Drupal\language\Plugin\Core\Entity\Language::id. It looks confusing but its a massive change that we need to not make all at once IMHO.

penyaskito’s picture

FileSize
26.74 KB
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View

Rerolled #102 and starting again from there.

webflo’s picture

Status: Needs work » Needs review
FileSize
234.4 KB
FAILED: [[SimpleTest]]: [MySQL] 56,717 pass(es), 187 fail(s), and 68 exception(s). View

Go!

webflo’s picture

Here is the interdiff for comment 126. Rerolled and removed the langcode property from Language Class and language_default().

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables-126.patch, failed testing.

webflo’s picture

Status: Needs work » Needs review
FileSize
3.17 KB
236.07 KB
FAILED: [[SimpleTest]]: [MySQL] 56,892 pass(es), 48 fail(s), and 65 exception(s). View

Fixed failed tests in locale module by replacing langcode with id.

webflo’s picture

Fixed failed tests in locale module by replacing langcode with id.

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables-129.patch, failed testing.

webflo’s picture

Status: Needs work » Needs review
FileSize
7.04 KB
238.08 KB
FAILED: [[SimpleTest]]: [MySQL] 58,427 pass(es), 10 fail(s), and 0 exception(s). View

Rerolled and move the language update to update_prepare_d8_language().

webflo’s picture

Status: Needs review » Needs work
Issue tags: +regression, +revisit before beta, +Configuration system, +D8MI, +sprint, +language-config, +Configurables

The last submitted patch, 1754246-language-configurables-132.patch, failed testing.

webflo’s picture

Status: Needs work » Needs review
FileSize
2.13 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1754246-language-configurables-132-135.patch. Unable to apply patch. See the log in the details link for more information. View
261.34 KB
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View

Fixed LanguageUpgradePathTest. All tests should pass.

webflo’s picture

FileSize
238.95 KB
PASSED: [[SimpleTest]]: [MySQL] 57,961 pass(es). View

Ooops. The last patch was outdated. This is the latest ...

webflo’s picture

Assigned: penyaskito » Unassigned
Priority: Normal » Critical

Its important for d8mi progress.

Gábor Hojtsy’s picture

Title: Convert Languages into Configurables » Languages should be configuration entities

I think important for D8MI would not be enough for it to be critical :)

1. First of all this is a CMI conversion that we are expected to complete either way.
2. Second, it needs to change the API for language, so it uses the 'id' property as config entity like everybody else does for the machine name (in this case language code).
3. And then it is a regression that we cannot translate the language name, it is only available in English. We will use config translation to get back the native name of language translation feature that was there in Drupal 7 but is currently not available.

webflo’s picture

FileSize
471 bytes
238.97 KB
FAILED: [[SimpleTest]]: [MySQL] 56,405 pass(es), 0 fail(s), and 27 exception(s). View

Rerolled and just another cleanup. We can remove the type cast language_default(). (We added it as an intermediate step ...)

webflo’s picture

FileSize
8.05 KB
231.65 KB
FAILED: [[SimpleTest]]: [MySQL] 56,330 pass(es), 0 fail(s), and 27 exception(s). View

The re-roll in comment 114 was bad and introduced a lot of outdated code. This patch removes it again.

Status: Needs review » Needs work

The last submitted patch, 1754246-language-configurables-140.patch, failed testing.

kfritsche’s picture

Okay I got through the whole patch, line by line. Was a long and hard journey.
Only thing I found is:

+++ b/core/modules/ckeditor/lib/Drupal/ckeditor/Plugin/Editor/CKEditor.php
@@ -208,9 +208,8 @@ public function getJSSettings(EditorEntity $editor) {
-      // @todo: Remove image and link plugins from CKEditor build.

Otherwise fine for me. Great job!

Did a fresh install, added a new language and tested a translation.
One follow-up which should be discussed. My default language was German and so the added language file looked like:

id: da
uuid: 18fc7b3e-99ac-4023-bc00-a04196302714
label: Danish
direction: '0'
weight: '0'
locked: '0'
status: '1'
langcode: de

langcode should be en. But this definitely doesn't concern this patch in any way.

webflo’s picture

Status: Needs work » Needs review
FileSize
2.06 KB
232.07 KB
PASSED: [[SimpleTest]]: [MySQL] 56,392 pass(es). View

Fixed the installer tests by injecting the selected language with drupal_static and restored the missing found by kfritsche. Thanks!

andypost’s picture

langcode should be en

This needs test and preSave() implementation on Language entity to make sure that language label always English only for languages that in standard list.
Suppose this is a separate issue out of scope

Gábor Hojtsy’s picture

Status: Needs review » Reviewed & tested by the community

I believe we worked out the remaining pieces in this conversion. It is not a nice one in terms of the parallels of how the language entity and language runtime classes are doubled and how the installer needs to be tricked into the static language list. However the former is needed to make config language work with language entities (avoid circular dependency) and the later is needed since #1862202: Objectify the language system is not there yet. This also overlaps with several changes in that issue, so they need to get in succession. That will clean up the language system further.

alexpott’s picture

Status: Reviewed & tested by the community » Fixed

I've manually tested this, looked at the config files generated and everything is working as expected.

This is a key step in making Drupal 8's language system work in the same way as the rest of the system and #1862202: Objectify the language system needs this to proceed.

git commit -m "Issue #1754246 by webflo, swentel, penyaskito, andypost, alexpott, YesCT, dagmar: Languages should be configuration entities."

Gábor Hojtsy’s picture

So now we can go write the config schema, yay :) #2031185: Create configuration schema for language config entity

Gábor Hojtsy’s picture

Also opened #2031197: Language configuration entities should be created in English at all times for the followup that @andypost and @kfritsche identified.

Gábor Hojtsy’s picture

Added a change notice for this too since this may make some patches dealing with these lines of code fail as well. https://drupal.org/node/2031267

andypost’s picture

Title: Languages should be configuration entities » Change notice: Languages should be configuration entities
Status: Fixed » Active

Updated change notice https://drupal.org/node/1818734

andypost’s picture

Now thats ready

Berdir’s picture

Title: Change notice: Languages should be configuration entities » Languages should be configuration entities
Status: Active » Fixed

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

Gábor Hojtsy’s picture

Issue tags: -sprint

Yay, removing from sprint.

Gábor Hojtsy’s picture

Issue summary: View changes

up

catch’s picture

Issue summary: View changes
Issue tags: -revisit before beta

Doesn't need revisiting.