Problem

Configuration schema and data are both versionless. There is nothing to tell if a given yml file is for a given version of a module. Modules however will likely change their configuration structure, such as contrib views plugins would change how they store settings or panels could change how it stores a panel config entity in any release.

The stance so far of the config system on this is "it is up to the module to update the data proper". As with update functions for other things, when you update a module, the update functions would need to take care of updating the config structure from old to new. That in itself is fine.

With the configuration override and schema system however, configuration overrides from the core locale module, from the contrib config_translation module, from the contrib organic groups, from the contrib domain module, etc. might also exist on the system. These contain overrides in the structure of the original config, so when merged they serve as configuration applicable to specific use cases (for languages, domains, groups, etc).

1. When such an update happens, all the modules dealing with configuration overrides need to react to the schema change too. They need to update the structure of the override files, so that they properly map to the new structure. If an override file in an old structure if applied to the new structure, it could lead to negative consequences of different proportions. Therefore when such updates are done, said core and contrib modules would need to have access to the old data and the old schema and the new schema and new data so they can regenerate the overrides in the proper new structure at least to their best effort.

2. Another related problem is how localize.drupal.org works to parse translatable strings from data yml files. When a project is encountered, the parser can look for all the required modules to load their schemas, so any data that is based on required modules can be inspected. Such as a "feature module" that is a couple views and panels and depends on views and panels would need the base schema of views and panels so we can parse their data. Most .info files would not specify concrete version numbers for dependencies (for obvious and good reasons), so localize.drupal.org needs to guess most of the time for the schema version to use. If the module required for the given project has multiple schema variations through its life, the backend will need to pick a version.

Proposal

Discuss!

Comments

Gábor Hojtsy’s picture

Problem (1) and (2) may or may not be the same depending on how we want to approach solving them.

For (1) we can enhance the core runtime system with copying the schema files to active storage too for example, so we can diff old and new schemas (old being in the active store, new being with the updated project in the source code) and use the old vs. new data as well. Core would also need to provide API functions that people would need to invoke when data is updated so the modules providing overrides can react and get the old/new data/schema.

However in (2) there is no runtime knowledge of old and new, there is just one version of data and then multiple versions of schemas. Possible solutions include versioning the schemas and the data, so for example put in a "version: 1.0" key to the data where external types are involved to refer to the version of the type and document in the schema which version of the schema it is. Another possibly uglier option is to include the version number in the type name, so it would be image.plugin.type.v2 or something along those lines. So when referring to a type, the schema would explicitly say which version of the data type is being referred to.

Any great ideas?

catch’s picture

There's an existing issue about versioning/module compatibility - I've looked for it twice since yesterday but can't find it..

Gábor Hojtsy’s picture

Right, I'd love to cross-reference but keep this its own issue, since this has far reaching consequences for both updates AND localize.drupal.org, later which does not have to do anything with updates and a solution is needed for much earlier than release, so we can offer up Drupal 8 stuff for translation there proper to be ready for release.

Gábor Hojtsy’s picture

dcam’s picture

http://drupal.org/node/1427826 contains instructions for updating the issue summary with the summary template.

The summary may need to be updated with information from comments.

jhedstrom’s picture

Is this still an issue?

Gábor Hojtsy’s picture

It is an issue but I think we decided we ignore it for now and just assume that whatever is in releases will need to keep backwards compatibility for configuration structures, so if you used a config key for something before, you'll need to keep backwards compatibility on it going forward. With the same thinking, we marked schema change issues in core critical and D8 upgrade path blocking, because we would need to pick new config keys later on if we need to change the structures.

xjm’s picture

effulgentsia’s picture

catch’s picture

Title: Configuration data vs. schema lacks version tracking » Add version tracking for configuration schema and data
Category: Bug report » Task

This is an API deficiency rather than a functional bug, changing to task.

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.4.x-dev » 8.5.x-dev

Drupal 8.4.0-alpha1 will be released the week of July 31, 2017, which means new developments and disruptive changes should now be targeted against the 8.5.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

xjm’s picture

Status: Postponed » Active

This doesn't need to be marked postponed anymore; that was just before release.

Version: 8.5.x-dev » 8.6.x-dev

Drupal 8.5.0-alpha1 will be released the week of January 17, 2018, which means new developments and disruptive changes should now be targeted against the 8.6.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.6.x-dev » 8.7.x-dev

Drupal 8.6.0-alpha1 will be released the week of July 16, 2018, which means new developments and disruptive changes should now be targeted against the 8.7.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.7.x-dev » 8.8.x-dev

Drupal 8.7.0-alpha1 will be released the week of March 11, 2019, which means new developments and disruptive changes should now be targeted against the 8.8.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.8.x-dev » 8.9.x-dev

Drupal 8.8.0-alpha1 will be released the week of October 14th, 2019, which means new developments and disruptive changes should now be targeted against the 8.9.x-dev branch. (Any changes to 8.9.x will also be committed to 9.0.x in preparation for Drupal 9’s release, but some changes like significant feature additions will be deferred to 9.1.x.). For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 8.9.x-dev » 9.1.x-dev

Drupal 8.9.0-beta1 was released on March 20, 2020. 8.9.x is the final, long-term support (LTS) minor release of Drupal 8, which means new developments and disruptive changes should now be targeted against the 9.1.x-dev branch. For more information see the Drupal 8 and 9 minor version schedule and the Allowed changes during the Drupal 8 and 9 release cycles.

Version: 9.1.x-dev » 9.2.x-dev

Drupal 9.1.0-alpha1 will be released the week of October 19, 2020, which means new developments and disruptive changes should now be targeted for the 9.2.x-dev branch. For more information see the Drupal 9 minor version schedule and the Allowed changes during the Drupal 9 release cycle.

Version: 9.2.x-dev » 9.3.x-dev

Drupal 9.2.0-alpha1 will be released the week of May 3, 2021, which means new developments and disruptive changes should now be targeted for the 9.3.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.3.x-dev » 9.4.x-dev

Drupal 9.3.0-rc1 was released on November 26, 2021, which means new developments and disruptive changes should now be targeted for the 9.4.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.4.x-dev » 9.5.x-dev

Drupal 9.4.0-alpha1 was released on May 6, 2022, which means new developments and disruptive changes should now be targeted for the 9.5.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 9.5.x-dev » 10.1.x-dev

Drupal 9.5.0-beta2 and Drupal 10.0.0-beta2 were released on September 29, 2022, which means new developments and disruptive changes should now be targeted for the 10.1.x-dev branch. For more information see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.

Version: 10.1.x-dev » 11.x-dev

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened, as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule and the Allowed changes during the Drupal core release cycle.