Problem/motivation

In #1866610: Introduce Kwalify-inspired schema format for configuration the schema files were merged into one big file per module/theme on suggestion from @effulgentsia & moved into the regular config directory. Practical application of this (eg. with Views at #1910606: Improve the configurations schemas for Views significantly) proved ineffective and the need to move it out of the regular config directory came up as well as the possibility to split schema files into more logical units as needed.

Proposal

- Modify schema discovery to look for schema files under a subdirectory in config directories called 'schema'.
- Move all existing files there.
- Make schema discovery code find all schema files in that directory not just one.

Files: 
CommentFileSizeAuthor
#36 1914366-move-config-schema-36.patch13.82 KBGábor Hojtsy
PASSED: [[SimpleTest]]: [MySQL] 51,306 pass(es). View
#33 1914366-move-config-schema-33.patch13.82 KBvijaycs85
PASSED: [[SimpleTest]]: [MySQL] 51,304 pass(es). View
#33 1914366-diff-31-33.txt345 bytesvijaycs85
#31 1914366-move-config-schema-31.patch13.49 KBvijaycs85
FAILED: [[SimpleTest]]: [MySQL] 51,246 pass(es), 3 fail(s), and 0 exception(s). View
#23 1914366-move-config-schema-23.patch13.55 KBvijaycs85
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1914366-move-config-schema-23.patch. Unable to apply patch. See the log in the details link for more information. View
#23 1914366-diff-18-23.txt5.13 KBvijaycs85
#18 1914366-move-config-schema-18.patch10.01 KBalexpott
PASSED: [[SimpleTest]]: [MySQL] 50,895 pass(es). View
#16 1914366-move-config-schema-16.patch32.67 KBvijaycs85
FAILED: [[SimpleTest]]: [MySQL] 50,738 pass(es), 0 fail(s), and 2 exception(s). View
#14 1914366-move-config-schema-14.patch35.36 KBvijaycs85
FAILED: [[SimpleTest]]: [MySQL] Drupal installation failed. View
#14 1914366-diff-10-14.txt5.78 KBvijaycs85
#10 1914366-move-config-schema-10.patch34.33 KBvijaycs85
PASSED: [[SimpleTest]]: [MySQL] 50,465 pass(es). View
#10 1914366-diff-5-10.txt3.35 KBvijaycs85
#5 1914366-move-config-schema-5.patch30.99 KBvijaycs85
FAILED: [[SimpleTest]]: [MySQL] 50,411 pass(es), 4 fail(s), and 6 exception(s). View
#5 1914366-diff-2-5.txt5.16 KBvijaycs85
#2 move-config-schema.patch25.83 KBGábor Hojtsy
FAILED: [[SimpleTest]]: [MySQL] 50,354 pass(es), 4 fail(s), and 6 exception(s). View

Comments

heyrocker’s picture

I'm in favor of this. The current way is going to make our import code more complex, which I'd like to avoid.

Gábor Hojtsy’s picture

Status: Active » Needs review
FileSize
25.83 KB
FAILED: [[SimpleTest]]: [MySQL] 50,354 pass(es), 4 fail(s), and 6 exception(s). View

Here is a patch only moving the files. The reading of the files is currently handled with regular config storage (and schema files are indeed copied to the active store, etc, which is not good/intended). So I think we'll need to revert back to the state from #1648930: Introduce configuration schema and use for translation where InstallStorage was used to look up from specific places from the file system. Someone more knowledgeable with the config system would be great to help here.

So far this will only demonstrate it fails if the schema files are moved out from where the system expects them :) So it will bounce itself to needs work.

Status: Needs review » Needs work

The last submitted patch, move-config-schema.patch, failed testing.

Gábor Hojtsy’s picture

Issue tags: +sprint
vijaycs85’s picture

FileSize
5.16 KB
30.99 KB
FAILED: [[SimpleTest]]: [MySQL] 50,411 pass(es), 4 fail(s), and 6 exception(s). View

Thanks for your tips on IRC @Gábor Hojtsy. Here is a patch with my understanding of what need to be done here:

1. Crated SchemaStorage which fetches schema from sub folder "config/schema"
2. Updated changes in InstallStorage (from #1648930-298: Introduce configuration schema and use for translation).

if they are right,
Yet to do:
1. Make SchemaDiscory to use SchemaStorage (i.e.SchemaDiscory->storage = SchemaStorage)
2. Load multiple file in same dir - as per current schema files, it should work without this fix (i.e. single schema file per module).

Gábor Hojtsy’s picture

Interdiff looks pretty good to me, taking over the needed pieces from #1648930: Introduce configuration schema and use for translation. Now the schema discovery need to use this storage and the moving of files would be accomplished. Making it read and merge all the files could be a bit harder, but we don't need to dynamically find keys in files or anything, just merge all files outright from the module AFAIS.

Gábor Hojtsy’s picture

Status: Needs work » Needs review

Status: Needs review » Needs work

The last submitted patch, 1914366-move-config-schema-5.patch, failed testing.

Gábor Hojtsy’s picture

Issue tags: +Configuration schema

All the above fails are expected, since the files are moved, but discovery is not yet updated.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
3.35 KB
34.33 KB
PASSED: [[SimpleTest]]: [MySQL] 50,465 pass(es). View

Updating patch to discover schema configurations. Tests passing locally but need to confirm the way it has been implemented. Sure this patch is not final, but just to confirm the right path.

Status: Needs review » Needs work
Issue tags: -Configuration system, -D8MI, -sprint, -language-config, -Configuration schema

The last submitted patch, 1914366-move-config-schema-10.patch, failed testing.

vijaycs85’s picture

Status: Needs work » Needs review
Issue tags: +Configuration system, +D8MI, +sprint, +language-config, +Configuration schema
Gábor Hojtsy’s picture

Status: Needs review » Needs work

Nonono! It should not copy the schema files to active storage, no. That behaviour is a side effect of the current design that is in fact not desired. The schema should be kept with the module and it cannot / will not be changed. So instead of copying it over, schema discovery should change to not look at the active config store but look at the original place with the component.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
5.78 KB
35.36 KB
FAILED: [[SimpleTest]]: [MySQL] Drupal installation failed. View

I had to register another class to make it check in module's folder and locally some part of test cases still failing because of meta data of configs (i.e. config_typed()->get('system.site')->get() looking at SchemaDiscovery for definition).

Status: Needs review » Needs work

The last submitted patch, 1914366-move-config-schema-14.patch, failed testing.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
32.67 KB
FAILED: [[SimpleTest]]: [MySQL] 50,738 pass(es), 0 fail(s), and 2 exception(s). View

Removing unnecessary class and function added in #14

Status: Needs review » Needs work

The last submitted patch, 1914366-move-config-schema-16.patch, failed testing.

alexpott’s picture

Status: Needs work » Needs review
FileSize
10.01 KB
PASSED: [[SimpleTest]]: [MySQL] 50,895 pass(es). View

The TypedData manager needs access to config data as well as schema so it needs both the config.storage and config.storage.schema services injected.

I've updated the SchemaStorage to issue it's own exceptions as the InstallStorage exceptions do not make sense.

The interdiff is a bit tricky as @vijaycs85 needs to update his git config so the moves of the schema files are detected and not treated as new files see http://drupal.org/documentation/git/configure

Status: Needs review » Needs work

The last submitted patch, 1914366-move-config-schema-18.patch, failed testing.

jthorson’s picture

Status: Needs work » Needs review

Testbot died on APC warnings and a segfault. :(

jthorson’s picture

Gábor Hojtsy’s picture

Status: Needs review » Needs work

@alexpott, @vijaycs85: thanks! Looking at the patch I only found one problem:

+++ b/core/lib/Drupal/Core/Config/TypedConfigManager.phpundefined
@@ -18,19 +18,19 @@ class TypedConfigManager extends TypedDataManager {
-   * @param Drupal\Core\Config\StorageInterface $storage
+   * @param \Drupal\Core\Config\StorageInterface $storage
    *   The storage controller object to use for reading schema data
    */
-  public function __construct($storage) {
-    $this->storage = $storage;
-    $this->discovery = new SchemaDiscovery($storage);
+  public function __construct(StorageInterface $configStorage, StorageInterface $schemaStorage) {

phpdoc not fully updated to current arguments.

Also, do we want to tackle reading more than 1 file from there in this issue or in another one? :)

vijaycs85’s picture

Status: Reviewed & tested by the community » Needs review
FileSize
5.13 KB
13.55 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch 1914366-move-config-schema-23.patch. Unable to apply patch. See the log in the details link for more information. View

Thanks for the review @Gábor Hojtsy. Attached patch address both issues:

1. read all schema files in a module - implemented using storage->listAll().
2. Comments - Fixed.

Gábor Hojtsy’s picture

Status: Needs work » Reviewed & tested by the community

Looks pretty good to me. (Will be committable once testbot comes back green).

Status: Needs review » Needs work
Issue tags: -Configuration system, -D8MI, -sprint, -language-config, -Configuration schema

The last submitted patch, 1914366-move-config-schema-23.patch, failed testing.

vijaycs85’s picture

Status: Needs work » Needs review
Issue tags: +Configuration system, +D8MI, +sprint, +language-config, +Configuration schema
alexpott’s picture

Status: Needs review » Needs work

An additional file has made it into the patch... core/modules/system/config/schema/system.data_types.schema.yml

vijaycs85’s picture

Status: Needs work » Needs review

@alexpott, yes, that is intentional. patch in #23 reads multiple schema files...

vijaycs85’s picture

Status: Needs review » Needs work
Issue tags: +Configuration system, +D8MI, +sprint, +language-config, +Configuration schema

The last submitted patch, 1914366-move-config-schema-23.patch, failed testing.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
13.49 KB
FAILED: [[SimpleTest]]: [MySQL] 51,246 pass(es), 3 fail(s), and 0 exception(s). View

Re-rolling...

Status: Needs review » Needs work

The last submitted patch, 1914366-move-config-schema-31.patch, failed testing.

vijaycs85’s picture

Status: Needs work » Needs review
FileSize
345 bytes
13.82 KB
PASSED: [[SimpleTest]]: [MySQL] 51,304 pass(es). View

Seems config_test module's schema file wasn't at right place. locally ran test cases failing on #31 and got green.

Gábor Hojtsy’s picture

Status: Needs review » Reviewed & tested by the community
Issue tags: +Avoid commit conflicts

Given other changes and introductions of schema, it would be great to prioritise this for commit (there are other RTBC issues for config schemas but those are easier to reroll, this one is harder). So moving back to RTBC and adding Avoid commit conflicts.

YesCT’s picture

Doing a docs review.

+++ b/core/lib/Drupal/Core/Config/InstallStorage.phpundefined
@@ -86,12 +85,79 @@ public function rename($name, $new_name) {
+     * @param array $list
...
+    protected function getComponentNames($type, $list) {

function definition should type hint. array $list

Gábor Hojtsy’s picture

FileSize
13.82 KB
PASSED: [[SimpleTest]]: [MySQL] 51,306 pass(es). View

Added the array typehint to that hunk. No other items found that this would apply to. So no interdiff. Given such minimal change, this should still be RTBC.

catch’s picture

Status: Reviewed & tested by the community » Fixed

The drupal_get_path() and drupal_system_listing() etc. calls aren't much fun, but that's not the fault of this patch. Committed/pushed to 8.x.

Gábor Hojtsy’s picture

Issue tags: -sprint

Thanks! That should help us reroll the 30 or so patches for schemas in http://drupal.org/project/issues/search/drupal?issue_tags=Configuration%... :) Superb!

Added this to the changelog at http://drupal.org/node/1905120 and updating the docs at http://drupal.org/node/1905070 :)

Gábor Hojtsy’s picture

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

chx’s picture

Issue tags: -Avoid commit conflicts