Provide config metadata solely for translation

Isn't this the same as #1648930: Introduce configuration schema and use for translation and #1851498: Polish file format of configuration metadata (for translation) combined, minus some of the TypedData integration?

Sorry, I don't really understand the difference.

On installing metadata like configuration, I'm not sure whether that makes sense. Need to think about it.

I suggested something similar in #1851498-21: Polish file format of configuration metadata (for translation), but keeping the TypedData, but populating entries only for what we know we have a use case for (translatable text elements), and filling it out further as there's reason to. However, I also think providing the full metadata for every element could be cool if we also use it to drive the primary (non-translation) configuration forms, at least in the simple cases. In any case, I see all such explorations to be perfectly doable after initial commit of #1648930: Introduce configuration schema and use for translation, unless there's good arguments here for stripping away the use of TypedData, which could also in theory happen post-commit, though in practice, much less easily than just fiddling with meta format optimizations.

There are two birds that we wanted to kill with one patch so to speak:

1. We need to be able to statically parse .yml config files and indentify all translatable pieces for translation of the software (eg. On localize.drupal.org). This needs less than a schema but need to explain the structure of the items so we can traverse and find relevant elements.
2. build transltaion forms, which actually needs more than a schema (eg labels), but also needs to describe the structure, etc.

@chx: In terms of labels and assumed text type for anything, it can work. A translation UI generation would need some more information, but what it is is hard to tell exactly now. We have plenty of parallel discussions, as @sun pointed out this issue you opened is just a combination of two others, both of which have extensive backchannel discussions in IRC as well, so it is getting pretty unwieldy to follow. Eg. @catch strongly suggested we should expand on the metadata format considerably and generate simple settings forms from the metadata. The more issues we open for this, the harder it is to follow different opinions.

@chx: I absoluly get your point. What I tried to point out is that this separate issue shields awya the rest of the discussions. I've had a productive discussion with @catch on IRC where he vehemently argued the metadata looks like useless duplication, if it repeats form data, like labels and data keys, but cannot be used to generate a simple form. He argued simple forms should be generateable from the metadata. I've also heard webchick in a meeting arguing that the metadata is horrible DX if it repeats info from forms like this, and she specifically wanted to create a hello world module to expose the complexity.

So I can hardly say there is consensus yet as to how broad or limited the metadata should be in trems of overall DX. Just considering the language use case was neither catcth now webchicks point of view.

There are other configuration system issues for which metadata might be the only viable solution.

One of them is #1653026: [META] Use properly typed values in module configuration — which would essentially mean that we'd leverage the metadata for every Config::save() operation; not only to switch from the enforced casting to strings to an enforced type-casting to the expected/defined data type, but potentially also to add a [pre-]storage-level validation of values. (Configuration is rarely written, so this is feasible.)

I additionally want to point out that the combined purpose of #1648930: Introduce configuration schema and use for translation and #1763640: Introduce config context to make original config and different overrides accessible is a base building block for multilingual (!= translatable) configuration; i.e., allowing more than translatable strings to diverge per language.

I do not think the code complexity of #1648930 worths it.

Expanding on #13, also consider that not only does what's already in HEAD plus the improvements in #1763640: Introduce config context to make original config and different overrides accessible allow for per-language overrides of (not just text) configuration values, but the system supports other context overrides too (e.g., Domain Access could allow per-domain customization of number of node teasers to show on the front page, or any other config value). And that's a great, efficient system that doesn't require schema or runtime TypedData overhead. But, what if we want to enable such contrib modules to cleanly provide UIs for editing the various per-context configurations? Being able to access the metadata of any config element via the TypedData API (in situations where you're ok with the performance hit, such as when editing config) is quite awesome in that regard.

Additionally, what if we want to provide the ability to edit configuration via non-form-based RESTful web services. A mobile app to edit config as well as content is not a far fetched use case. #1845546: Implement validation for the TypedData API is working on moving the bulk of validation from forms to TypedData.

So, being able to access config elements via TypedData brings config into closer alignment with entity fields, and I think that's a good thing. I agree that #1648930: Introduce configuration schema and use for translation still feels too complex, but I think those are things that can be improved upon post-commit, rather than indicative of needing to back away from the fundamental architecture choices.

Generating forms, in general, just never gonna work, I said that in the parent issue and I maintain that. You need checkboxes, radios etc etc you need formatting, all the generation can do for you is textboxes one below the other.

I don't yet understand this conclusion. It depends on what we add to the metadata. If we add validation constraints that include enumerations, we're not that far off from having checkboxes and radios. And a metadata element for 'default_form_element_type' (bikeshedding of that name TBD) to be able to specify radios vs. select, for example, doesn't seem that unreasonable. I think it's worth exploring a bit to see whether a few simple metadata additions can get us to 80% of simple settings forms being completely automatable.

Given the variety of subsystems that will store their stuff in CMI files, each with its own "thing" structure, forcing each one to write a proper DTD before it can use CMI sounds like a showstopper

This is one reason why I'm not yet convinced that switching from YAML to XML will buy us much. Sure, XML DTDs are nice in that they're a standard, but they're also clunky. But I also see no reason we can't explore that possibility in a follow up, now that we're much clearer on what our needs are than we were at the time yched wrote that. Also, I want to reiterate that a key feature of #1648930: Introduce configuration schema and use for translation is that it does not require there to be metadata in order for a module to use CMI. The metadata is required if you want to allow for translations, and if you want to allow for smoother integration with per-context configuration editing UIs, and if you want to allow for RESTful editing of your configuration, and if you want to take advantage of an automated form generator if and when we manage to have one, and all these benefits might make a module developer choose to provide the metadata (especially if we make the DX of doing so not totally suck), but I find it a very nice design that getting a quick and dirty module working without all those features is still possible with no metadata/schema/DTD needed.

I do not think the code complexity of #1648930 worths it

The thing is the code complexity of the other one was opening a few more doors, if not for core, for contrib.

Now it seems we are going to have a bit less complex thing, but just for a way more limited (possible) number of uses.

However I'll support this one as long as it fixes the issue (translate configuration strings) so please keep working on it.

Btw, about the TypedData API, it's a good idea not to use it, it just causes more trouble than benefit, I can tell you :-)

Tagging for D8MI board.

Based on Dries' review in the metadata issue at and the discussion after with chx, I wanted to cross-post my feedback from there which has lots of relevance for this one as well. Dries pretty much set the direction: don't attempt to build automated forms, reduce complexity/duplication, etc. So this patch should be down to identifying only the strings extracted for localize.drupal.org AFAIS.

@Dries I'm also not interested in having core automatically generate simple forms from the meta files; I don't want to introduce another form API, at least not in Drupal 8. It's too late for that.

All right, we have been working on this issue since the summer (and had g.d.o discussion pieces and drupalcon/camp meetings of the underlying ideas since 2011 summer), but have indeed had a hard time to get reviewers and people interested to support it. Looks like the form generation task will be fully on contrib then. With the form generation use case discarded though, I don't think D8MI has any reason to push for a metadata of any richness (eg. labels, types), because we will need to do it all over in contrib (either by duplicating Form API structures as-is or introducing some kind of metadata). Then contrib modules will either choose to not be multilingual with their config or support that contrib system. (In short, if this is an ugly system but we don't have better ideas, contribs will need to cope with this either way. If we have better ideas then we'll be able to experiment in contrib. The goal of making Drupal 8 more multilingual for configuration is hardly achieved.)

So the core use case is back down to the "simple" problem of identifying strings for localize.drupal.org, so "exported configuration" as part of the software can be translated and translations can be applied to installations of Drupal and its distributions. A huge chunk of this patch deals with that (importing those strings to the locale system, exporting translations from the locale system to .yml files, etc), so that will need to be carried over to whatever format patch we have for identifying those strings. Also, if we keep the assumption that we need to identify those strings statically, then we also need the inclusion and plugin support in the metadata format. So in fact all we loose if the type and label from the metafile (as thus the TypedData connection altogether), the rest of the complexity for describing nested structures with plugins to identify their pieces to extract is needed. So we will need to salvage that piece of this code as well.

I think the conclusion is that core will be simpler, contrib (wanting to support config translation) will be much more complex but the UX will benefit from it. Contrib wanting to implement multilingual for config was not an easy ride in Drupal 7 either, we'll not have a huge step forward.

I think, given latest developments, it makes sense to raise the priority of this one.

Posted this request for reviews in the Core group (also should be on Drupal Planet): http://groups.drupal.org/node/273268

@chx: Any progress on this one? @Dries postponed #1648930: Introduce configuration schema and use for translation in part on this issue a week ago today :)

No, what I've indicated was that *evaluating the metadata files from different sources* on localize.drupal.org would not work. @heyrocker's working assumption is that .yml based metadata files will exist and get merged for each original config .yml file, and he is looking at that, while @fago advocated we actually run the metadata on the config .yml and only save the resulting strings that are translatable alongside the config data .yml. Either way, we need a way to define the metadata, what turned out in our discussion was that more processing will need to be done on the Drupal instance where the config is created.

I think duplicate is more correct, since we want to solve this at either #1865300: Introduce configuration definition file format and API for reading it or #1866610: Introduce Kwalify-inspired schema format for configuration or #1648930: Introduce configuration schema and use for translation (other options).

Also remove from D8MI sprint.

Updated issue summary.