Updated: Comment #0

Problem/Motivation

If a module dev decides to add a config form to their module, they need to write a config file. Without this, they would be forced to reinstall their module to properly get the file in the active store

If your coworker wants to give you a new content type they were working on, without this you need to do a full site config import.

Proposed resolution

Add a form to "import" a single config file for advanced purposes. Clarify that this will NOT resolve any dependencies, and could break their site.

Remaining tasks

User interface changes

Added admin pages

API changes

Addition

#1898794: Allow config entities to be exported. (Resolve regression from views in Drupal 7).
#1831798: Update hook_help() for config manager module

Files: 
CommentFileSizeAuthor
#34 cmi-2099363-34.patch31.65 KBtim.plunkett
PASSED: [[SimpleTest]]: [MySQL] 59,624 pass(es). View
#34 interdiff.txt1.11 KBtim.plunkett
#32 cmi-2099363-32.patch31.52 KBtim.plunkett
PASSED: [[SimpleTest]]: [MySQL] 59,189 pass(es). View
#32 interdiff.txt3.33 KBtim.plunkett
#30 Screen Shot 2013-10-25 at 00.30.24.png43.57 KBswentel
#30 Screen Shot 2013-10-25 at 00.30.44.png44.95 KBswentel
#30 Screen Shot 2013-10-25 at 00.40.24.png24.01 KBswentel
#30 config-2099363-30.patch29.98 KBswentel
FAILED: [[SimpleTest]]: [MySQL] 59,195 pass(es), 3 fail(s), and 0 exception(s). View
#30 interdiff.txt3.46 KBswentel
#29 config-2099363-29.patch27.66 KBtim.plunkett
FAILED: [[SimpleTest]]: [MySQL] 60,053 pass(es), 6 fail(s), and 0 exception(s). View
#25 config-2099363-25.patch29.53 KBtim.plunkett
PASSED: [[SimpleTest]]: [MySQL] 59,507 pass(es). View
#24 config-2099363-24.patch29.66 KBmtift
PASSED: [[SimpleTest]]: [MySQL] 59,481 pass(es). View
#20 config-2099363-20.patch29.64 KBtim.plunkett
PASSED: [[SimpleTest]]: [MySQL] 59,460 pass(es). View
#20 interdiff.txt1.31 KBtim.plunkett
#13 config-import-export-2099363-13.patch28.33 KBtim.plunkett
FAILED: [[SimpleTest]]: [MySQL] 58,976 pass(es), 14 fail(s), and 0 exception(s). View
#13 interdiff.txt1.4 KBtim.plunkett
#11 Screen Shot 2013-10-07 at 9.42.33 PM.png34.98 KBwebchick
#11 Screen Shot 2013-10-07 at 9.45.41 PM.png53.71 KBwebchick
#11 Screen Shot 2013-10-07 at 10.02.24 PM.png39.91 KBwebchick
#11 Screen Shot 2013-10-07 at 10.07.00 PM.png20.75 KBwebchick
#7 config-import-export-2099363-7.patch28.14 KBtim.plunkett
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View
#7 interdiff.txt673 bytestim.plunkett
#3 Screen Shot 2013-09-29 at 3.48.30 PM.png16.45 KBtim.plunkett
#3 config-import-export-2099363-3.patch28.14 KBtim.plunkett
PASSED: [[SimpleTest]]: [MySQL] 58,992 pass(es). View
#3 interdiff.txt20.24 KBtim.plunkett
#1 1export.png71.44 KBtim.plunkett
#1 2export-settings.png58.34 KBtim.plunkett
#1 3export-settings-value.png51.4 KBtim.plunkett
#1 4export-image.png41.22 KBtim.plunkett
#1 5export-image-value.png64.39 KBtim.plunkett
#1 1import-settings.png33.56 KBtim.plunkett
#1 2import-block.png38.2 KBtim.plunkett
#1 3import-success.png35.34 KBtim.plunkett
#1 import-export-2099363-1.patch18.85 KBtim.plunkett
PASSED: [[SimpleTest]]: [MySQL] 59,281 pass(es). View

Comments

tim.plunkett’s picture

Title: Allow single config files to be imported (Resolve regression from Views in Drupal 7) » Allow single config files to be imported and exported (Resolve regression from Views in Drupal 7)
Status: Active » Needs review
FileSize
18.85 KB
PASSED: [[SimpleTest]]: [MySQL] 59,281 pass(es). View
35.34 KB
38.2 KB
33.56 KB
64.39 KB
41.22 KB
51.4 KB
58.34 KB
71.44 KB

Here is my proposed approach.

This is the initial "single export" form, with a dropdown to pick what type of configuration
1export.png

For "simple configuration", generally the MODULE.settings.yml (anything not a config entity), you have a dropdown
2export-settings.png

Selecting one populates the import textarea with the YAML
3export-settings-value.png

The same thing works for all config entity types
4export-image.png
5export-image-value.png

This is the "single import" form for a "simple configuration", where you have to specify the filename
1import-settings.png

For config entities, you choose the type
2import-block.png
3import-success.png

dawehner’s picture

Just did a little bit of manual testing:

  • 4 Tabs for all export/import and single export and import is a bit weird
  • I exported a view and imported the same one ... and no validation got triggered ... this is kind of a regression to d7
    There it was possible to give another name ... so maybe we have to provide such a entity ID import textfield?
  • Most of the time actually the UUID will clash (it did not as glossary did not had a UUID by default)
  • There is no link in link in the actual views UI to export/import but yeah this could be a followup
  1. +++ b/core/modules/config/config.module
    @@ -87,6 +87,20 @@ function config_menu() {
    +  );
    

    Do we really still need to add old style tabs?

  2. +++ b/core/modules/config/lib/Drupal/config/Form/ConfigSingleImportForm.php
    @@ -0,0 +1,167 @@
    +    // For a config entity, create a new entity and save it.
    +    else {
    +      $entity = $this->entityManager
    +        ->getStorageController($form_state['values']['config_type'])
    +        ->create($form_state['values']['import']);
    +      $entity->save();
    +
    +      drupal_set_message($this->t('The @entity_type %label was imported.', array('@entity_type' => $entity->entityType(), '%label' => $entity->label())));
    +    }
    

    As written I guess it would make sense to be able to specify the ID of the new config entity.

  3. +++ b/core/modules/config/lib/Drupal/config/Tests/ConfigSingleImportExportTest.php
    @@ -0,0 +1,96 @@
    +  public function testImport() {
    ...
    +  public function testExport() {
    

    Just in general I think it is odd to never ask the entity manager whether the stuff was really created.

tim.plunkett’s picture

FileSize
20.24 KB
28.14 KB
PASSED: [[SimpleTest]]: [MySQL] 58,992 pass(es). View
16.45 KB

New tab structure:
Screen Shot 2013-09-29 at 3.48.30 PM.png

Also now has a confirmation form because @xjm and @alexpott talked me into it
Major props to @Berdir for the $form_state['rebuild'] = TRUE; suggestion

EDIT: I need to open a follow-up for the ability to specify a custom ID for your config entity, but that can wait until after dinner.

tim.plunkett’s picture

Component: configuration system » config.module
beejeebus’s picture

Status: Needs review » Reviewed & tested by the community

i think this is ready.

discussed validation issues with @timplunkett and @moshe in IRC, but they aren't created by this patch.

we need to fix config entity validation elsewhere.

chx’s picture

Status: Reviewed & tested by the community » Needs work

I don't think so. This very nicely shows that encode / decode is doesnt even belong to the config storage interface. It needs to be moved to a staging (or whatever we call it) interface and then filestorage can implement that and be happy about it. If you switch the active store to a redis / mongodb / heck mysql w/ php serialize store what do you expect from encode ? Nothing normal. Conversely, the staging storage shouldn't be a service but should be hardwired to the YAML FileStorage we have pretty much the same as modules are phptemplate wired in D7 and for the same reasons.

Edit: we can hardwire it into a method in ConfigFactory or something.

tim.plunkett’s picture

Status: Needs work » Needs review
FileSize
673 bytes
28.14 KB
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View

The new interface and hardcoding should be left to a new issue.

This change will at least force the ExportForm to give us something in a format that importing can read.

EDIT: #2105689: Acknowledge that our import/export config storage is not swappable is the new issue

tim.plunkett’s picture

Assigned: tim.plunkett » Unassigned

Nope, that won't work, because config.storage.staging is hardcoded to look in /staging. We can't just use it to read out /active, at least not yet.

I'd still very much like to see #3 go in, since it correctly uses the APIs available to it.

beejeebus’s picture

Status: Needs review » Reviewed & tested by the community

talked to timplunkett and chx, and we agreed to RTBC #3 again. fixing the other issue shouldn't block this.

webchick’s picture

Not looked at the code, but here's a UI review. This patch makes me very happy!!! :D

Screen Shot 2013-10-07 at 10.07.00 PM.png

I'd put export first, then import, personally. That's the logical progression people will do.
I'd also put full ahead of single, since most of the time that's what people should be doing.

Screen Shot 2013-10-07 at 9.42.33 PM.png

Let's alphabetize this list. It's bound to get even longer when contrib modules are added, so finding things will be an issue otherwise.

Screen Shot 2013-10-07 at 9.45.41 PM.png

Can the "what's happening here?" message be moved to the white area, rather than the title, so it doesn't get cut off?
Also, can we kill the import/export tabs from here? That's strange.

Screen Shot 2013-10-07 at 10.02.24 PM.png

Not sure if this is true for other things, but for Blocks anyway, we need some kind of disambiguation between the two "System help" blocks. "System help (Bartik)" vs. "System help (Seven)" for example.

Since "Configuration type" on import dictates which API functions are called, it's important that it's set correctly. How about set it to nothing by default and make it required, so people are forced to make a choice?

webchick’s picture

Status: Reviewed & tested by the community » Needs work
tim.plunkett’s picture

Status: Needs work » Needs review
FileSize
1.4 KB
28.33 KB
FAILED: [[SimpleTest]]: [MySQL] 58,976 pass(es), 14 fail(s), and 0 exception(s). View

This fixes "Configuration type" by making it required, default to nothing, and sorted alphabetically.

No idea how best to handle the "Configuration names" part for blocks... :(

tim.plunkett’s picture

dawehner’s picture

+++ b/core/modules/config/lib/Drupal/config/Form/ConfigSingleImportForm.php
@@ -0,0 +1,243 @@
+  protected $configExists = FALSE;

The variable name is a bit odd. I think $config would be a bit better. A value set to FALSE is pretty clear all the time.

tim.plunkett’s picture

I had $config originally, but while writing the form I twice got it switched up with $this->config(). So I think we should leave it as is.

tim.plunkett’s picture

tim.plunkett’s picture

Issue tags: +DX (Developer Experience), +Configuration system
FileSize
1.31 KB
29.64 KB
PASSED: [[SimpleTest]]: [MySQL] 59,460 pass(es). View

Just a reroll for new tests using the old path

tim.plunkett’s picture

I think that the desire to differentiate the blocks could be solved by either #2111341: Introduce EntityManager::options() to build options lists or #1959806: Provide a generic 'entity_autocomplete' Form API element. But I think its out of scope for this issue.

tim.plunkett’s picture

#20: config-2099363-20.patch queued for re-testing.

mtift’s picture

I'm going to be looking at this one today and testing it thoroughly

mtift’s picture

FileSize
29.66 KB
PASSED: [[SimpleTest]]: [MySQL] 59,481 pass(es). View

Let's start with a reroll

tim.plunkett’s picture

FileSize
29.53 KB
PASSED: [[SimpleTest]]: [MySQL] 59,507 pass(es). View
+++ b/core/modules/config/config.routing.yml
@@ -5,39 +5,60 @@ config.diff:
+    _title: Export
     _form: '\Drupal\config\Form\ConfigExportForm'
     _title: 'Export'
...
+    _title: Import
     _form: '\Drupal\config\Form\ConfigImportForm'
     _title: 'Import'
...
+    _title: Synchronize
     _form: '\Drupal\config\Form\ConfigSync'
     _title: 'Synchronize'

That double added the titles.

mtift’s picture

(I got distracted a lot today.) This looks great to me. I have tested most of the scenarios I can imagine and have not encountered any errors.

I was thinking about adding an export button for the single file export (admin/config/development/configuration/single/export), but I'm wondering if that was left off intentionally because the export file can be altered?

tim.plunkett’s picture

I don't think a download button is needed, its a giant textarea you can select-all and copy-paste from, same as Views/CTools had in D7.

swentel’s picture

Status: Needs review » Needs work
Issue tags: +Needs reroll

This one needs a reroll after the local task conversion - started looking into it but got confused and completely distracted.

Would love to test this with fields and instances .. :)

tim.plunkett’s picture

Status: Needs work » Needs review
Issue tags: -Needs reroll
FileSize
27.66 KB
FAILED: [[SimpleTest]]: [MySQL] 60,053 pass(es), 6 fail(s), and 0 exception(s). View

Rerolled.

swentel’s picture

FileSize
3.46 KB
29.98 KB
FAILED: [[SimpleTest]]: [MySQL] 59,195 pass(es), 3 fail(s), and 0 exception(s). View
24.01 KB
44.95 KB
43.57 KB

Did some tests with fields. Importing gave some problems

The Field(Instance)StorageController::loadByProperties() assume some extra keys - since we were the ones always calling them, those were always ok. (edited)

Screen Shot 2013-10-25 at 00.30.24.png

Importing something illegal gives a nasty error

Screen Shot 2013-10-25 at 00.30.44.png

Made changes to the field storage controllers and added a try/catch to the import of a config entity to catch the exception and show a more friendly message.

It now comes back to import screen. Since we're in the submit, I wasn't sure how to preserve the data; rebuild gave me back the confirm form.

Screen Shot 2013-10-25 at 00.40.24.png

tim.plunkett’s picture

FileSize
3.33 KB
31.52 KB
PASSED: [[SimpleTest]]: [MySQL] 59,189 pass(es). View

Oh yay, unit tests for local tasks!

Fixing the naming insanity.

Also, a couple people asked about hook_help, but there is already #1831798: Update hook_help() for config manager module .

mtift’s picture

Status: Needs review » Reviewed & tested by the community

This is ready to go. Applies cleanly. Code looks good. Tested exporting/importing image styles, text formats, and more.

This will definitely benefit from some scary "Are you sure you want to import this config and risk totally breaking your site?" warnings to be provided in #1831798: Update hook_help() for config manager module .

tim.plunkett’s picture

FileSize
1.11 KB
31.65 KB
PASSED: [[SimpleTest]]: [MySQL] 59,624 pass(es). View

Back in #13 I fixed the sorting of the config types dropdown, but I only did it for import.
This fixes export in the same way (leaving "Simple configuration" at the top)

webchick’s picture

Priority: Normal » Major
Status: Reviewed & tested by the community » Fixed

Sat down with Tim at BADCamp to test this sucker. Overall this both looks and works kick-ass! :) Since this is fixing a regression, it seems to belong as at least a major.

I found a few things while testing:

1) (minor) The options were alphabetized on import, but not on export. Tim fixed that in his latest re-roll.

2) (major/critical) I did something where I copied node.field_tags, and fiddled with it to make it user.field_tags instead, but hit a couple of errors on the way (duplicate UUID, duplicate machine name), but the error I ended up with was that the user_field_tags table already existed. :\ This means that you can currently get into a situation where an entity->save() half-finishes, which needs to be fixed. However, that has nothing to do with this issue, it was merely exposed by it. Will file a follow-up for it.

3) There were some minor UX-ish things, like for example on import you've now lost the name of the file that you had in the export screen, but after talking with Tim, there wasn't really a great way to solve this. Embedding the filename in the YAML output, for example, would make it so that these weren't easily portable. We also discussed the order of the tabs, because this patch makes *single* import/export the second tab after Synchronize. Between Jess, Alex Pott, and Matthew Tift, we couldn't come to consensus on what the order should be, so we decided to just leave it, since it's super easy to change later once we have more user feedback.

So, with that, I'm officially out of things to complain about. :) Sooooo...

Committed and pushed to 8.x. WOOHOO!!

webchick’s picture

Issue summary: View changes

added the hook help

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.