Follow-up to #1866610: Introduce Kwalify-inspired schema format for configuration

Here are some thoughts/questions/suggestions related to the typed data API integration of the config-schema:

- It looks like system.schema.yml re-defines all the simple typed data types. Is that necessary? Looks like a duplication to me.

undefined:
  label: 'Undefined'
  class: '\Drupal\Core\Config\Schema\Property'
mapping:
  label: Mapping
  class: '\Drupal\Core\Config\Schema\Mapping'
sequence:
  label: Sequence
  class: '\Drupal\Core\Config\Schema\Sequence'

We should bring this to regular and proper registered typed data types, e.g. provided by config module. However we need to prefix them with config or make sure they generally make sense. Undefined does, but I'd call it "unknown" - undefined usually means something is null.

The current way of adding typed-data types without registering them is problematic I think. Imho it would be a safe assumption that any type you get from $typed_data->getType() is an actually valid, thus registered type.

Can't we bring the type-registries together, or do we need one in yml also?

$this->definition['mapping']
Are they keys added to defintions by the config system documented somewhere?

- The patch implemented TypedData::validate() methods although that part of the API was not yet finished. It needs to be reworked once #1845546: Implement validation for the TypedData API lands.

    $configuration += $type_definition;
    return parent::createInstance($plugin_id, $configuration, $name, $parent);

This was done by the original manager also, but we did away with it during the performance optimizations as this leads to loads of duplicated array copies in memory.

      if (is_string($value)) {
        $definition['type'] = 'str';
      }
      else {
        $definition['type'] = 'any';

str? I've not seen that elsewhere, maybe wrong?

For performance, I'd suggest using TypedDataManager::getPropertyInstance() wherever possible.

Files: 
CommentFileSizeAuthor
#19 interdiff.txt3.49 KBGábor Hojtsy
#19 typed-data-config-schema-19.patch6.84 KBGábor Hojtsy
PASSED: [[SimpleTest]]: [MySQL] 50,823 pass(es). View
#18 typed-data-config-schema-18.patch4.55 KBGábor Hojtsy
PASSED: [[SimpleTest]]: [MySQL] 50,490 pass(es). View
#8 typed-data-config-schema-8.patch2.45 KBGábor Hojtsy
PASSED: [[SimpleTest]]: [MySQL] 49,127 pass(es). View
#6 typed-data-config-schema-6.patch1.59 KBGábor Hojtsy
PASSED: [[SimpleTest]]: [MySQL] 49,182 pass(es). View

Comments

Gábor Hojtsy’s picture

Issue tags: +D8MI, +language-config

Well, reality is the schema system takes any defined CMI key as a type that can be and is referenced later on. See image effects and views for examples. Not sure typed data would be scalable for registering all CMI keys defined. See http://drupal.org/node/1905070 under Types supported and Dynamic type references.

As for 'str' that looks like a mistake and validate() is not actually used anywhere in the code yet (neither the Parser class I think), so that can be removed AFAIS.

Gábor Hojtsy’s picture

Issue tags: +sprint
fago’s picture

From looking at the schema files it's mostly type "mapping" and sequence + has extra meta information in further array keys. That's perfectly fine for the typed data API as well.

I think having general usable unknown+mapping/map types in the typed data API makes sense anyway, the config system can then make use of them as appropriate. That would solve the problem of having unregistered types.

However, I'm not so sure how to deal with sequence. It's way of adding meta-information should work also, but currently there is "list" or "sequence" type in the typed data API - instead you describe the list item and say "list => TRUE" to denote it's actually multiple.

Dynamic type references make sense to add on top if it, such that the PHP classes can pick this up. But the actual typed data objects exposed should be based upon registered typed data types - else it does not really qualify as typed data.

Gábor Hojtsy’s picture

I think having general usable unknown+mapping/map types in the typed data API makes sense anyway, the config system can then make use of them as appropriate. That would solve the problem of having unregistered types.

Well, maybe not really? The config schema makes extensive use of back-referencing types (such as in image effects and all through views) that were previously defined using the schema for plugin based data types. Everything eventually resolves to basic types of course but the reusability of display controller config or image scale effect logic is implemented by defining types for them in the schema and then using those types. I'm not sure you want to have each CMI key as a typed data type to be registered too? Sounds like too much overhead for types not used in Drupal elsewhere.

As for custom types via classes, when writing the docs I figured out that we don't really have clear guidelines as to what a class implementing a custom type would need to have. That would be great to clean up :)

As for lists, the list = TRUE thing was in the prior metadata patches and *nobody* liked it at all. The unanimous feedback was list is conceptually a type.

fago’s picture

As for lists, the list = TRUE thing was in the prior metadata patches and *nobody* liked it at all. The unanimous feedback was list is conceptually a type.

I see, interesting feedback - thanks. We might want to consider improving this for the typed data API then? Similar concerns have been raised rather early at #1762874: Decouple property item lists from property items.

Gábor Hojtsy’s picture

FileSize
1.59 KB
PASSED: [[SimpleTest]]: [MySQL] 49,182 pass(es). View

Let's start with removing the unused Parser (that you noted above had incorrect type fallback and used typed data validation incorrectly - that was not even in the goals - and the also an unused required property (given that we don't validate). The Parser also attempts to use the nonexistent config_definition(). Fun stuff.

Gábor Hojtsy’s picture

Status: Active » Needs review
Gábor Hojtsy’s picture

FileSize
2.45 KB
PASSED: [[SimpleTest]]: [MySQL] 49,127 pass(es). View

Fix any and str in the create() method of the typed data factory too. "any" got renamed to undefined in the patch but it was not reflected here.

@fago: I'm not well versed at all in typed data, so this is about as far as I can go into cleanup. Carrying in some of the types to Typed Data sound good to me as well as attempting to clean up so that types are properly registered. Although I'm not sure of that as said above in light of config schema reusing schema defined structures as types extensively for plugin handling.

fago’s picture

Patch looks like a good clean-up but it does not really related to the issue title - maybe we should move it in its own issue?

@fago: I'm not well versed at all in typed data, so this is about as far as I can go into cleanup. Carrying in some of the types to Typed Data sound good to me as well as attempting to clean up so that types are properly registered. Although I'm not sure of that as said above in light of config schema reusing schema defined structures as types extensively for plugin handling.

Yep, thanks! I think we should start with adding support/classes for the generic structures in typed data api and base the config-classes on them, then re-iterate on how we deal with type names. We could just make sure we expose/map the config-types to property typed-data types and we should be done.

Gábor Hojtsy’s picture

Patch looks like a good clean-up but it does not really related to the issue title - maybe we should move it in its own issue?

The patch is based on your review in the issue opener, specifically str? I've not seen that elsewhere, maybe wrong?. It is true str and any are not valid types even in the current schema system. Also, you complained about the validation stuff, which is not used either, and the two issues appeared in the same unused code snippet, so it was directly logical to remove the unused code here. Also, I found the same str and any at one more place, and fixed that to use actual schema defined types. Since these were undefined types, I think they are directly related to this issue.

fago’s picture

True - I missed it already fixes validation, great!

To me the patch looks great, however I cannot tell whether the Parser class is anyhow needed or why required can be removed? So leaving for someone with more config-schema knowledge to RTBC.

Gábor Hojtsy’s picture

The Parser is not actually required by any current code neither it is needed for locale integration (#1905152: Integrate config schema with locale, so shipped configuration is translated). The required property can be removed, since we don't do validation and that was the only place it was used. Superfluous.

Gábor Hojtsy’s picture

Issue tags: +Needs tests

Also need to test with a simple yml file in a test module that the defaults are properly applied.

YesCT’s picture

Status: Needs review » Needs work
Issue tags: +Novice

adding the test is doable for someone new, or maybe at one of the sprints coming up. (see http://drupal.org/node/1468170 )
#1866610: Introduce Kwalify-inspired schema format for configuration shows sample yml files, but sounds like what we need here is a much simpler one "to test that the defaults are properly applied"

looks like Gabor answered the question about needing more review. so setting to needs work.

Gábor Hojtsy’s picture

So what we need here is looks like:

- 1 data .yml file (not schema!) in one of the config test modules
- have at least one or more string keys and one mapping or sequence

In sum, something like (name the file eg. "test.noschema.yml"):

testitem: "Should be detected as string"
testlist:
 - "Should be detected"
 - "as undefined"

Now since we don't define a schema for this, the default logic should kick in, and testitem should be detected as string, while testlist itself should be detected as "undefined". The tests in #1866610: Introduce Kwalify-inspired schema format for configuration show how concrete files are tested and how their types are checked in tests. In this case we'd check what happens if there is no schema for a file. (Because all core .yml files should eventually have schemas, we need a dummy config .yml file for this).

Gábor Hojtsy’s picture

@fago: also sorry that this might look like your issue is taken over, but we agreed this is part of the fix needed. Maybe we'll need to open yet more issues for other parts of the fix or return to the rest once we have this covered.

fago’s picture

Gábor Hojtsy’s picture

Status: Needs work » Needs review
Issue tags: -Novice
FileSize
4.55 KB
PASSED: [[SimpleTest]]: [MySQL] 50,490 pass(es). View

I don't think this qualifies as Novice.

Got started with the tests, however it seems like the schema system will not delve into values that it does not know about, so we cannot really get below the main config key level. Using chained get() calls, I can only get Unknown types for any other key (which should individually be detected as a string theoretically). Here is an initial pass at the tests which detects that unknown CMI keys and known CMI keys without schemas will resolve to the same Unknown type on the main level.

Gábor Hojtsy’s picture

FileSize
6.84 KB
PASSED: [[SimpleTest]]: [MySQL] 50,823 pass(es). View
3.49 KB

Here are full tests. A file without schema will not recognize any items so delving into detecting types for them will not work obviously. Once we define some schema for the top level, the second level items are properly detected as string or unknown. Now included test coverage for that too and made the .yml files explain their purpose better. :)

I think this is as complete as the cleanup can go for the type detection. IMHO this should be committed and other cleanups worked on in followups / other issues. Please review!

Jose Reyero’s picture

Status: Needs review » Reviewed & tested by the community

Looks good to me, this is a needed cleanup.

Gábor Hojtsy’s picture

#19: typed-data-config-schema-19.patch queued for re-testing.

Dries’s picture

Status: Reviewed & tested by the community » Fixed

Committed to 8.x. Thanks.

Gábor Hojtsy’s picture

Thanks!

Gábor Hojtsy’s picture

Issue tags: +Configuration system

Add tag.

fago’s picture

Automatically closed -- issue fixed for 2 weeks with no activity.

Anonymous’s picture

Issue summary: View changes

Updated issue summary.