Move config translations (language.config.[langcode]) in to new location

I agree the files should be moved into a subdirectory. However, not necessarily into nested subdirectories à la ./translations/$langcode/.

To recap, this is what we originally discussed for storing configuration context overrides:

1. Contextual override config lives in subdirectories of the config directory.
2. The maximum nesting level is 1 → sub-sub-directories are not allowed/supported.
3. For non-file-based configuration storage implementations, each subdirectory denotes a collection name (cf. KeyValueStore).
4. Each subdirectory/collection specifies the context(s) to which it applies in HTTP query string syntax → ./langcode=de/system.site.yml
5. Context namespaces (e.g. langcode) are not owned by any particular module/provider; they use common names → ./domain=example.com/, ./subdomain=support/, etc.
6. Multiple contexts may be combined → ./subdomain=support&langcode=de/
7. Within each subdirectory/collection, configuration objects use the same name as in the main/root directory/collection → ./system.site.yml ↔ ./langcode=de/system.site.yml

Updated to current file naming pattern. I'm not sure I agree with the HTTP naming scheme for the directories, I can see how = and & in directory names could become a nightmare.

WIll take a stab at this tomorrow.

I've been looking at this at various places, and I think this is indeed more complex than what I expected it to be :/ I spent a few hours thinking about this and here is my conclusion.

So all the API in config is centered around the fact that config is accessible by a single string key (and is located in one place). If we introduce / as a possible component in the string (and storage to interpret that as a directory separator), then the string key would still not be applicable to the config schema. So to be directly applicable for the schema, we would need to keep the string key as-is. Then we would need to add additional arguments to look at subdirectories. Especially with @sun's proposal a full on 'context' key support for subdirectories. That sounds like would trickle down all over the API especially on all methods in Drupal\Core\Config\StorageInterface. That complication just so we can apply config schema to language overrides sounds to me like a problem...

So I am thinking if we can extend the config schema system in a way so that it can understand optional prefixes on filenames that if found may be removed from the file to then take the reduced file name for schema lookups. Then we keep the complexity within the schema system and not making the whole config system that much more complex. Also the schema system is designed so that we do actually need to look up the schema and then associate with the data, so we can look up the schema independently of the data, which is to our great advantage here.

Another issue with the all-config-in-one-dir situation as-is now was raised earlier that there may be too many files in the directory, if you have numerous blocks, menus, etc. in 3-10-20 languages. The schema solution would not solve that obviously. I'm not sure if this is a real problem currently given the limitations of number of files in directories are not that bad anymore :) Even FAT32 used to allow 65k files.

Totally untested but theoretically "complete" (functionality-wise) patch applied. I think an extension method for other prefix-based solutions would be good, but otherwise this is what maps into the simple key based solution of config without complicating with alternate buckets of storage for different things on the main config API.

Thanks for kick-starting this, Gábor.

I can see how mixing a subdirectory name into the key name is problematic. Technically, the subdirectory represents a different "collection" of the storage. → I'm fairly sure that the proposal in #2190723: Add a KeyValueStore\FileStorage to replace e.g. ConfigStorage would help us to implement this more cleanly.

I'm not sure if complicating the base API is worth it?

Also: *If* that would block, this that one cannot be less than a critical beta blocker.

@vijaycs85: Yeah that is true, we would need to remove one more part after this until the next dot.

Here is a quick edited version of the above patch. It moves the preg escaping to the array construction and adds provisions for the langcode that I did not have before. This is untested, just hand-edited. @vijaycs85: can you maybe look at testing this? :)

@alexpott: any thinking about extensibility? I think it would maybe a problem to call out to module hooks in config factory, sounds like recipe for circular dependencies.

Should of course not escape the actual pattern part.

So I think my regex was not good :D '\.[^\.+]\.' because the + should be right after ]. Your regex is better but not perfect either :) '\.[^\.]*\.' will match .. (two dots on a row), so as if the langcode is empty. I don't think that would be practically a problem, but strictly it is :)

Also, wow, the tests were already failing thanks to the language file not matching? Is this code also run from the test? Can we do a test with a broke file to catch the exception and prove it works for language files?

Yeah that proves it works, but it would be amazing to have permanent tests, ie. a test with language.config.xx.... that would throw and catch an exception and pass if it got that exception :) Otherwise looks amazing :)

I think this is adequate coverage. The passes show this works:

Schema found for language.config.en.config_test.system and values comply with schema. ...
Schema found for language.config.fr.config_test.system and values comply with schema. ...
language.config.xx... is not valid configuration translation ...

Now we only need to figure out how to make this extensible and probably get other CMI reviewers on board.

+++ b/core/modules/config/lib/Drupal/config/Tests/DefaultConfigTest.php
@@ -91,17 +93,22 @@ public function testDefaultConfig() {
+        // Make sure invalid translation excluded.
+        if ($config_name == 'language.config.xx...') {
+          $this->pass(String::format('!config_name is not valid configuration translation', array('!config_name' => $config_name)));
+          continue;
+        }

+++ b/core/modules/config/tests/config_test/config/language.config.xx....yml
@@ -0,0 +1 @@
+foo: fr bar

This seems a weird test. What are we actually testing here?

@alexpott: that tests that 'language.config.xx...' does not have schema. It indeed does not ensure that 'language.config.xx...' would appear among the ones which don't have schema, that would need a fail() later if this condition was never matched. All the other language.config... files are not specifically exempt in the test and therefore will fail if no schema found for them.

Anyway, based on discussions with @alexpott he is not happy either with the language baked into the factory and that we should also solve the use case here of config files being aggressively overflown by lots of translation files, if there are many languages / lots of config, so the subdirectory solution would be preferred. He has code in the making which supposedly does not affect the config() core API badly. Marking needs work for that, and assigning to him.

Just opened #2201437: [META-1] Config overrides and language to as the solution is far beyond the scope of this patch.

Sounds like to me we need to postpone this on #2201437: [META-1] Config overrides and language or maybe even mark as a duplicate, since that supersedes this (would include a solution for this). Marking postponed for now.

More accurate to say this is a duplicate of #2201437: [META-1] Config overrides and language.