Instead of doing a really complicated typeddata implementation in #1648930: Introduce configuration schema and use for translation we could just ship with the bare minimum metadata enough for potx to be able to extract the strings from the config files shipping with modules and to be able to create forms based on this metadata. For example, image styles could have the following metadata:

name:
  label: 'Machine name'
label:
  label: 'Label'
  translate: TRUE
effects:
  label: 'Style effects'
  repeats: TRUE
effects.*
  label: 'Image effect'
effects.*.name:
  label: 'Machine name'
effects.*.weight:
  label: 'Weight'
effects.*.ieid:
  label: 'IEID'
effects.*.data:
  label: 'Data'
  include: 'image.effect.[effects.*.name]'

Note that repeats TRUE here is a hint to the UI that a repetition will happen and nothing else.

I would say this could ship in the same directory as the config and we could and should install as we do with config and add a ui: key so that people can mark parts translateable from the UI even if it's not something potx wants to -- the 404 page can be like this. When installing, expand the translate: TRUE into ui: TRUE, translate: TRUE. This is why everything has a label, so that everything can be marked for ui translation as the site demands.

Given we already have a locale override system, the UI just needs to grab the keys from config, compare to metadata, provide the form for UI: true keys, save to the overridden config, end.

Also when installing we can just run the translate: TRUE keys through t() to pull in .po provisioned values and save in the locale override again.

The metadata is never used at any other time, just install and config translate ui. Thus the cost of matching the config keys to foo.*.bar.*.baz is negligible.

As for potx, reads the config file, creates the keys, reads the meta, checks for translate: TRUE, knows what strings to extract and it only needs a YAML reader and a few dozen lines of code.

No typeddata, no nothing, straight as an arrow, small and easy. Some complication might arise from the image.style.meta.%.yml / image.style.meta.large.yml file which we likely want to keep as it was in the parent issue but I think we will extremely rarely need a specific version so that's ok.

Files: 
CommentFileSizeAuthor
#23 1861640_22.patch3.23 KBchx

Comments

chx’s picture

Issue summary: View changes

Updated issue summary.

chx’s picture

Issue summary: View changes

Updated issue summary.

chx’s picture

Issue summary: View changes

Updated issue summary.

chx’s picture

Issue summary: View changes

Updated issue summary.

chx’s picture

Issue summary: View changes

Updated issue summary.

chx’s picture

Issue summary: View changes

Updated issue summary.

chx’s picture

Issue summary: View changes

Updated issue summary.

sun’s picture

Isn't this the same as #1648930: Introduce configuration schema and use for translation and #1851498: Polish file format of configuration metadata (for translation) combined, minus some of the TypedData integration?

Sorry, I don't really understand the difference.

On installing metadata like configuration, I'm not sure whether that makes sense. Need to think about it.

effulgentsia’s picture

I suggested something similar in #1851498-21: Polish file format of configuration metadata (for translation), but keeping the TypedData, but populating entries only for what we know we have a use case for (translatable text elements), and filling it out further as there's reason to. However, I also think providing the full metadata for every element could be cool if we also use it to drive the primary (non-translation) configuration forms, at least in the simple cases. In any case, I see all such explorations to be perfectly doable after initial commit of #1648930: Introduce configuration schema and use for translation, unless there's good arguments here for stripping away the use of TypedData, which could also in theory happen post-commit, though in practice, much less easily than just fiddling with meta format optimizations.

chx’s picture

Well, losing typeddata from that patch means most of the code goes as well. What I want here is very little, very straightforward code. Very likely all the API we want to provide is extremely small.

chx’s picture

effulgentsia, let me get this straight: I do not think the code complexity of #1648930 worths it.

beejeebus’s picture

i agree with #4. but more than that - if we really want for-real schema for our configuration, i think we should step back and reassess yml. if we use XML for config, we can solve the 'we need machine readable schema about our configuration' in a simpler, more powerful and less NIH way.

note - i'm not suggesting that we got to XML. i don't want that. i don't think we should introduce a full blown schema, waaaaay late in the cycle, when heyrocker and others in the CMI team have opposed that for a long time.

the driver seems to be "but what do we do about locale and transation"? that being the case, lets focus on that, and not all the schema things.

let's make the metadata limited and focussed on what values in config are translatable, and we can avoid a large blob of complexity and satisfy the needs of locale.

Gábor Hojtsy’s picture

There are two birds that we wanted to kill with one patch so to speak:

1. We need to be able to statically parse .yml config files and indentify all translatable pieces for translation of the software (eg. On localize.drupal.org). This needs less than a schema but need to explain the structure of the items so we can traverse and find relevant elements.
2. build transltaion forms, which actually needs more than a schema (eg labels), but also needs to describe the structure, etc.

chx’s picture

Elaborate on #6.2. What do you need not covered in #0

Gábor Hojtsy’s picture

@chx: In terms of labels and assumed text type for anything, it can work. A translation UI generation would need some more information, but what it is is hard to tell exactly now. We have plenty of parallel discussions, as @sun pointed out this issue you opened is just a combination of two others, both of which have extensive backchannel discussions in IRC as well, so it is getting pretty unwieldy to follow. Eg. @catch strongly suggested we should expand on the metadata format considerably and generate simple settings forms from the metadata. The more issues we open for this, the harder it is to follow different opinions.

beejeebus’s picture

re #8, i'd be ok with rolling this back into the #1648930: Introduce configuration schema and use for translation, though i'm kinda waiting for Dries to weigh in on that issue. if he thinks the complexity is worth it for all the other non-locale things people want to do with a schema for our config, this issue is moot.

if that issue lands, i'm going to open an issue to move config to XML, so we can do schema properly.

chx’s picture

People don't get me. I want to won't fix #1648930: Introduce configuration schema and use for translation. The scope of the issue is wrong "Introduce configuration metadata" and after that, as an afterthought "and use for translation". No. Don't do that. Provide a minimal, small system so that a) potx can run b) config translation UI can run. The moment you have enough metadata to do that and nothing more, stop immediately.

Gábor Hojtsy’s picture

@chx: I absoluly get your point. What I tried to point out is that this separate issue shields awya the rest of the discussions. I've had a productive discussion with @catch on IRC where he vehemently argued the metadata looks like useless duplication, if it repeats form data, like labels and data keys, but cannot be used to generate a simple form. He argued simple forms should be generateable from the metadata. I've also heard webchick in a meeting arguing that the metadata is horrible DX if it repeats info from forms like this, and she specifically wanted to create a hello world module to expose the complexity.

So I can hardly say there is consensus yet as to how broad or limited the metadata should be in trems of overall DX. Just considering the language use case was neither catcth now webchicks point of view.

chx’s picture

Generating forms, in general, just never gonna work, I said that in the parent issue and I maintain that. You need checkboxes, radios etc etc you need formatting, all the generation can do for you is textboxes one below the other. Unless of course we go XML and schema. Which, obviously, is a bad idea as yched said in http://drupal.org/node/1470824#comment-5757588 : ". "Given the variety of subsystems that will store their stuff in CMI files, each with its own "thing" structure, forcing each one to write a proper DTD before it can use CMI sounds like a showstopper."

chx’s picture

Issue summary: View changes

Updated issue summary.

sun’s picture

There are other configuration system issues for which metadata might be the only viable solution.

One of them is #1653026: [META] Use properly typed values in module configuration — which would essentially mean that we'd leverage the metadata for every Config::save() operation; not only to switch from the enforced casting to strings to an enforced type-casting to the expected/defined data type, but potentially also to add a [pre-]storage-level validation of values. (Configuration is rarely written, so this is feasible.)

I additionally want to point out that the combined purpose of #1648930: Introduce configuration schema and use for translation and #1763640: Introduce config context to make original config and different overrides accessible is a base building block for multilingual (!= translatable) configuration; i.e., allowing more than translatable strings to diverge per language.

effulgentsia’s picture

I do not think the code complexity of #1648930 worths it.

Expanding on #13, also consider that not only does what's already in HEAD plus the improvements in #1763640: Introduce config context to make original config and different overrides accessible allow for per-language overrides of (not just text) configuration values, but the system supports other context overrides too (e.g., Domain Access could allow per-domain customization of number of node teasers to show on the front page, or any other config value). And that's a great, efficient system that doesn't require schema or runtime TypedData overhead. But, what if we want to enable such contrib modules to cleanly provide UIs for editing the various per-context configurations? Being able to access the metadata of any config element via the TypedData API (in situations where you're ok with the performance hit, such as when editing config) is quite awesome in that regard.

Additionally, what if we want to provide the ability to edit configuration via non-form-based RESTful web services. A mobile app to edit config as well as content is not a far fetched use case. #1845546: Implement validation for the TypedData API is working on moving the bulk of validation from forms to TypedData.

So, being able to access config elements via TypedData brings config into closer alignment with entity fields, and I think that's a good thing. I agree that #1648930: Introduce configuration schema and use for translation still feels too complex, but I think those are things that can be improved upon post-commit, rather than indicative of needing to back away from the fundamental architecture choices.

Generating forms, in general, just never gonna work, I said that in the parent issue and I maintain that. You need checkboxes, radios etc etc you need formatting, all the generation can do for you is textboxes one below the other.

I don't yet understand this conclusion. It depends on what we add to the metadata. If we add validation constraints that include enumerations, we're not that far off from having checkboxes and radios. And a metadata element for 'default_form_element_type' (bikeshedding of that name TBD) to be able to specify radios vs. select, for example, doesn't seem that unreasonable. I think it's worth exploring a bit to see whether a few simple metadata additions can get us to 80% of simple settings forms being completely automatable.

Given the variety of subsystems that will store their stuff in CMI files, each with its own "thing" structure, forcing each one to write a proper DTD before it can use CMI sounds like a showstopper

This is one reason why I'm not yet convinced that switching from YAML to XML will buy us much. Sure, XML DTDs are nice in that they're a standard, but they're also clunky. But I also see no reason we can't explore that possibility in a follow up, now that we're much clearer on what our needs are than we were at the time yched wrote that. Also, I want to reiterate that a key feature of #1648930: Introduce configuration schema and use for translation is that it does not require there to be metadata in order for a module to use CMI. The metadata is required if you want to allow for translations, and if you want to allow for smoother integration with per-context configuration editing UIs, and if you want to allow for RESTful editing of your configuration, and if you want to take advantage of an automated form generator if and when we manage to have one, and all these benefits might make a module developer choose to provide the metadata (especially if we make the DX of doing so not totally suck), but I find it a very nice design that getting a quick and dirty module working without all those features is still possible with no metadata/schema/DTD needed.

chx’s picture

I will roll a patch and we can see. Also, the reason I have opened a separate issue and didnt even CNW the other because this way if Dries wants, he can cleanly just go ahead with that issue. I would be disappointed but , there are reasons apparently, so his call.

Jose Reyero’s picture

I do not think the code complexity of #1648930 worths it

The thing is the code complexity of the other one was opening a few more doors, if not for core, for contrib.

Now it seems we are going to have a bit less complex thing, but just for a way more limited (possible) number of uses.

However I'll support this one as long as it fixes the issue (translate configuration strings) so please keep working on it.

Btw, about the TypedData API, it's a good idea not to use it, it just causes more trouble than benefit, I can tell you :-)

Gábor Hojtsy’s picture

Issue tags: +D8MI, +sprint, +language-config

Tagging for D8MI board.

Based on Dries' review in the metadata issue at and the discussion after with chx, I wanted to cross-post my feedback from there which has lots of relevance for this one as well. Dries pretty much set the direction: don't attempt to build automated forms, reduce complexity/duplication, etc. So this patch should be down to identifying only the strings extracted for localize.drupal.org AFAIS.

@Dries I'm also not interested in having core automatically generate simple forms from the meta files; I don't want to introduce another form API, at least not in Drupal 8. It's too late for that.

All right, we have been working on this issue since the summer (and had g.d.o discussion pieces and drupalcon/camp meetings of the underlying ideas since 2011 summer), but have indeed had a hard time to get reviewers and people interested to support it. Looks like the form generation task will be fully on contrib then. With the form generation use case discarded though, I don't think D8MI has any reason to push for a metadata of any richness (eg. labels, types), because we will need to do it all over in contrib (either by duplicating Form API structures as-is or introducing some kind of metadata). Then contrib modules will either choose to not be multilingual with their config or support that contrib system. (In short, if this is an ugly system but we don't have better ideas, contribs will need to cope with this either way. If we have better ideas then we'll be able to experiment in contrib. The goal of making Drupal 8 more multilingual for configuration is hardly achieved.)

So the core use case is back down to the "simple" problem of identifying strings for localize.drupal.org, so "exported configuration" as part of the software can be translated and translations can be applied to installations of Drupal and its distributions. A huge chunk of this patch deals with that (importing those strings to the locale system, exporting translations from the locale system to .yml files, etc), so that will need to be carried over to whatever format patch we have for identifying those strings. Also, if we keep the assumption that we need to identify those strings statically, then we also need the inclusion and plugin support in the metadata format. So in fact all we loose if the type and label from the metafile (as thus the TypedData connection altogether), the rest of the complexity for describing nested structures with plugins to identify their pieces to extract is needed. So we will need to salvage that piece of this code as well.

I think the conclusion is that core will be simpler, contrib (wanting to support config translation) will be much more complex but the UX will benefit from it. Contrib wanting to implement multilingual for config was not an easy ride in Drupal 7 either, we'll not have a huge step forward.

Jose Reyero’s picture

Priority: Normal » Critical

I think, given latest developments, it makes sense to raise the priority of this one.

Gábor Hojtsy’s picture

Posted this request for reviews in the Core group (also should be on Drupal Planet): http://groups.drupal.org/node/273268

Gábor Hojtsy’s picture

@chx: Any progress on this one? @Dries postponed #1648930: Introduce configuration schema and use for translation in part on this issue a week ago today :)

chx’s picture

I actually moved to the language negotiation to plugins based on what you said saying in one of the issues that the declarative metadata won't work? But OK, I will come back.

Gábor Hojtsy’s picture

No, what I've indicated was that *evaluating the metadata files from different sources* on localize.drupal.org would not work. @heyrocker's working assumption is that .yml based metadata files will exist and get merged for each original config .yml file, and he is looking at that, while @fago advocated we actually run the metadata on the config .yml and only save the resulting strings that are translatable alongside the config data .yml. Either way, we need a way to define the metadata, what turned out in our discussion was that more processing will need to be done on the Drupal instance where the config is created.

chx’s picture

FileSize
3.23 KB

This is deliberately not even CNR yet. It's a rough start. It's omfg ugly because of the includes. I will work on cleaning it up / making recursive includes work (yes!) and so on. (If you want to read a highlighted version, https://gist.github.com/4335702)

chx’s picture

Status: Active » Closed (won't fix)

I will support #1865300: Introduce configuration definition file format and API for reading it instead to decrease the forks of this issue. Include will be ugly nonetheless.

Gábor Hojtsy’s picture

Status: Closed (won't fix) » Closed (duplicate)
Gábor Hojtsy’s picture

Issue tags: -sprint

Also remove from D8MI sprint.

Gábor Hojtsy’s picture

Issue summary: View changes

Updated issue summary.