Introduce configuration definition file format and API for reading it

To summarize your proposal, the file naming placement would be: modules/contact/contact.config-definition.yml (where contact is the module name both times). And file content would be including all structures defined by the module, eg from image module:

+image.style.%:
+  name:
+    label: 'Machine name'
+  label:
+    label: 'Label'
+    translatable: true
+  effects:
+    label: 'Style effects'
+  effects.%:
+    label: 'Image effect'
+  effects.%.weight:
+    type: integer
+    label: 'Weight'
+  effects.%.ieid:
+    label: 'IEID'
+  effects.%.name:
+    label: 'Machine name'
+  effects.%.data:
+    label: 'Data'
+    include: 'image.effect.[%parent.name]'
+
+image.effect.image_crop:
+  width:
+    type: integer
+    label: 'Width'
+  height:
+    type: integer
+    label: 'Height'
+  anchor:
+    label: 'Anchor'
+
+image.effect.image_resize:
+  width:
+    type: integer
+    label: 'Width'
+  height:
+    type: integer
+    label: 'Height'
+
+image.effect.image_rotate:
+  degrees:
+    label: 'Rotation angle'
+  bgcolor:
+    label: 'Background color'
+  random:
+    type: boolean
+    label: 'Randomize'
+
+image.effect.scale:
+  width:
+    type: integer
+    label: 'Width'
+  height:
+    type: integer
+    label: 'Height'
+  upscale:
+    type: boolean
+    label: 'Upscale'
+
+image.effect.scale_and_crop:
+  width:
+    type: integer
+    label: 'Width'
+  height:
+    type: integer
+    label: 'Height'

In efffect no "OO-support" to extend existing metadata from base classes, etc. but people would copy those over by hand and maintain it if the base module/class ever updates, right?

Raise to D8MI sprint board (also on http://www.drupal8multilingual.org/issues/focus) as well so we can keep close track of the progress here too, much like #1861640: Provide config metadata solely for translation.

Yeah, the includes were an absolute requirement, not so much for "OO-like inheritence", but for cases where:
- module A (e.g. image) provides the metatdata for the wrapper file (e.g image.style.%),
- but at some point in the tree the metadata for a given entry (e.g the settings of a given wizz_bang_effect within the style), will be provided by a different module (e.g the wizz module that ships the image effect)
Which is typically the 'plugin-based subsystem' case.

To do that routing, the only data we have *in the config file, without some runtime PHP* is the value in a given key (here, the 'name' entry that happens to hold the effect plugin id). That's just a plugin id, not the name of a module.
So we can route to any arbitrary agreed-upon string based on that plugin id, like for instance image.effect.wizz_bang_effect.config-definition.yml (with 'image' here being a pure convention, stemming from the fact that the 'image effect' plugin type is defined by image.module - that module also defines image styles but that's a coincidence)
But we cannot route to [owning_module].effect.wizz_bang_effect.config-definition.yml, we don't have that info in the config file.
(#1862600: Entity type names/IDs are not properly namespaced by owner (e.g., taxonomy.term vs. taxonomy_term), which, despite it's current title, actually discusses forcing all plugin ids to [owner_module.plugin_id], might change that, but there's currently no buy in over there)

Additional constraint being that image effects only know that they are image effects (and thus need provide the code & data that's in the contract), *not* that they are "typically included in a config entity for something called image styles". There are different config entity types that might include configuration for a field formatter, and field formatters cannot need to care.
A plugin cannot make assumptions about where it's used - it's the other way around, the config entity type knows it refers to plugins of a given type at some place in its yaml trees.

@yched, @effulgentsia: in short that means we don't know which module might define that CMI key pattern, so we need to read all of the definitions and look through them (or have a lookup cache) as far as I see?

Posted this request for reviews in the Core group (also should be on Drupal Planet): http://groups.drupal.org/node/273268

image.effect.fancy_effect_1:
_include: image.effect._base
custom_setting_1:

To me, an include seems to be the same as an entity refernce in typed data. Problem is for that to work out on a static basis we need to have a static way of finding/loading this file - what means we have to re-invent functionality of Drupal outside of it (e.g. finding modules, loading the right CMI files, ..).

As explained at https://drupal.org/node/1648930?page=1#comment-6845200, my feeling is that this static parsing makes things unnecessarily complex. Why not do this work in PHP and export the translatable things in a file?

@effulgentsia: Any progress on this one? @Dries postponed #1648930: Introduce configuration schema and use for translation in part on this issue a week ago today :)

@effulgentsia: thanks, I think some discussion of the file format would be useful nonetheless. :) I think interested parties are holding back until they see a working system. However, in any case, a working system will probably be considerably more complex in some way (the merge rules need to be defined somewhere, if not in metadata then disconnected in PHP somewhere).

Raising this to critical in place of #1861640: Provide config metadata solely for translation

A fully working one, which has a few things in common with this one (like a single schema file per module) is here , just in case you can reuse some code (factory, discovery, etc...) #1866610: Introduce Kwalify-inspired schema format for configuration

Tagging as epic.

@efflugentsia I would encourage you to continue work on your side rather than delay for me or anyone else. We never know if they will ever get to a solution, and then time has been wasted for nothing. So please, keep working and we'll see where we end up.

We don't need two critical tasks about this.

Sorry for not taking over and not advancing for almost a week. IMO image.effect.scale this should be image.effect.image_scale just based on name: image_scale in core/modules/image/config/image.style.thumbnail.yml. But this raises an interesting question: how will we know that image_scale is provided by image module? This approach definitely requires this knowledge and if another module provides an image effect, how will we find it?

Also, I am willing to work on this just not sure right now what to do because of #18.

@chx: see #7, which is hopefully a good short summary of #5, where this is better explained with examples.

Hrm, I see , definitely @effulgentsia's code is moving towards a full-blown cache lookup and it constructs the offset name like + include: 'image.effect.[%parent.name]'... tricky. very tricky. So any file can contain the image.effect.wizz-bang definition and we read them all. I will take a quick look but leave this assigned to effulgentsia so I am not holding htis up any more.

@chx,

But this raises an interesting question: how will we know that image_scale is provided by image module?

#1862600: Entity type names/IDs are not properly namespaced by owner (e.g., taxonomy.term vs. taxonomy_term)

Cross-quoting @heyrocker from #1648930-346: Introduce configuration schema and use for translation:

I would like to establish a deadline for deciding how we're going to proceed with this. If we don't, we are going to go back and forth on these decisions in a circle until feature freeze comes and then we're screwed. My feeling is we will probably need about a month to implement any new proposal and get it to RTBC, so I'd like to lay down January 15 as our deadline or deciding what to do. If we haven't agreed on any new proposal, then we reroll the original RTBC patch from #285 and get it in. We have to get this done, and right now I don't see any light at the end of the tunnel which scares the crap out of me.

I very much agree we need to get out of our lack of agreement here, because we are dealing with things we absolutely much do even if only to avoid sizeable regressions in Drupal 8.

In light of this, reviews pro or against the above proposals are more than welcome.

@webchick: re #17, although we do have #1648930: Introduce configuration schema and use for translation, that is marked postponed, and therefore is not visible in active issue lists neither is counted against queue limits, so I'm not seeing why it is a problem that we have at least one active critical task about this. It is definitely not normal overall to solve this problem and #1648930: Introduce configuration schema and use for translation has no visibility while postponed.

Posted this call for feedback on the Core group (also going to Drupal Planet): http://groups.drupal.org/node/275008

Who cares, there is no way we get a good solution. Let's pick the least bad.

Thanks for the feedback!

Also, removing sprint tag.

Added info about file format.