Make it possible for recipes to prompt for input values

I would find this useful for my recipe. I'm setting up a demo module and it needs a unique email address and password that the user must provide or the demo won't be completely set up (some tests will work but those involving emails won't).
https://github.com/Performant-Labs/atk_demo/blob/main/recipe.yml

How about this?

1. Ask for input before work begins so the recipe can be cancelled with no side effects.
2. Running recipe with -y skips asking questions.
3. Control-C cancels.
4. Define questions with:

input:
  email_address:
    - text: Enter email address
    - variable: variable_name
    - config:
      - default: email@null.com
      - sanitize: no

Which provides:
Enter email address ([email@null.com]):

The colon after the input text is provided by Recipes.

5. Input is stored as a variable in the form ${{ input.variable_name }}
6. Blank is allowed in which case the default is used.
7. Run variable_name through checks unless sanitize:no is specified.
8. Straight substitution occurs in actions with ${{ input.variable_name }}.
9. Provide input values on the command line with --input variable_name:.
10. Also provide input values via a file with --input-file .

Plus an additional section that provides tokens that the input above would override:

arguments:
  email_address:
    - variable: variable_name
    - config:
      - default: email@null.com

Maybe they can be merged.

We could eliminate:

- variable: variable_name

And make the variable name the name of the section after input:.

Is there any reason to have the extra flexibility?

The examples here so far are about unique identifiers: e-mail addresses, API keys, URLs, etc.

But there's a different pattern that also requires input: what if there's a recipe that requires some media type/node type/text editor/…, and you may want to apply it to 1 or more such config entity types? For example: adding a field, adding a view mode, adding a filter that references a field, adding a filter that references a view mode, et cetera.

We ran into this limitation in #3417835-13: Convert the Standard install profile into a set of recipes.

In that case, we needed a recipe that creates a contact form to do one of two things:

• Ask the user for an email address
• Copy the system.site:mail config value into another piece of config

Both of these approaches are ways of collecting input, so I think they fall under the scope of this issue. I'm updating the issue summary to also mention this particular use case.

I was thinking about this and I had this idea for how we could define this stuff in recipes.

I propose we add a new section, input, to recipe.yml. In this example, the recipe is creating a contact form. It asks the user for a recipient email address, with a default value coming from config:

name: 'Website feedback contact form'
description: 'Provides a website feedback contact form.'
type: 'Contact form'
install:
  - contact
input:
  default_recipient:
    from: config
    config: 'system.site:mail'
  recipient:
    from: prompt
    question: 'Enter an e-mail address which should receive form submissions.'
    default: '%default_recipient'
config:
  import:
    contact:
      - contact.settings
  actions:
    contact.form.feedback:
      addRecipient: '%recipient'

(Imagine for a sec that contact forms have a addRecipient action.)

Some of my thoughts here:

• Every item in input defines a named variable, which can then be reused in the config:actions section.
• Each named variable can come from either config, or a prompt.
• If a variable can come from a prompt, then it can have a default value. The default value may be a named variable previously defined by the same recipe.
• It's not possible to skip an input prompt conditionally. If you define an input prompt in your recipe, it WILL be presented to the user.
• Input can be provided non-interactively at the command line: --input recipient=sisko@ds9.space

Thoughts on this idea?

Per #6, this blocks #3417835: Convert the Standard install profile into a set of recipes.

#7: I like your proposal! Thoughts:

1. For usability, it's important that all prompts can be collected for a recipe stack (aka a recipe that depends on other recipes, etc.), so that they can all be presented to the user upfront.
2. … which also means that some of these may need to be asked in a particular sequence? Or does that not matter? 🤔
3. For automation, it's important that all prompts can be collected for a recipe stack so that they can be listed, to allow somebody who's automating spinning up a new Drupal site using a recipe to specify all inputs as command line arguments.
4. We must think about collisions on the config key/property path. I think --input-recipient is guaranteed to run into collisions. I think --input--website-feedback-contact-form--recipient=sisko@ds9.space is more verbose but also more reliable. Verbosity is less important in this context, predictability matters more.

This doesn't block #3417835: Convert the Standard install profile into a set of recipes; untagging. Sorry if my comment wasn't clear. We ran into the problem, but can work around it as described in #3417835-14: Convert the Standard install profile into a set of recipes.

That said, I'd guess this does block the recipe system being "finished" and fully usable by authors. So, escalating it.

Responding to the points in #8:

For usability, it's important that all prompts can be collected for a recipe stack (aka a recipe that depends on other recipes, etc.), so that they can all be presented to the user upfront.

Agreed.

… which also means that some of these may need to be asked in a particular sequence? Or does that not matter? 🤔

I think it might make sense for input definitions to have a weight field so that ordering can be tweaked if needed. Otherwise they should probably just be presented in the order that they are defined (inputs from dependencies come first, in their defined order; followed by the inputs from the recipe the user has asked to apply, in their defined order). All inputs in the "stack" would ordered by weight.

For automation, it's important that all prompts can be collected for a recipe stack so that they can be listed, to allow somebody who's automating spinning up a new Drupal site using a recipe to specify all inputs as command line arguments.

That's a good point.

We must think about collisions on the config key/property path. I think --input-recipient is guaranteed to run into collisions. I think --input--website-feedback-contact-form--recipient=sisko@ds9.space is more verbose but also more reliable. Verbosity is less important in this context, predictability matters more.

Yeah, I concur. Collisions are inevitable. Maybe the syntax could be something like --input RECIPE_NAME.VARIABLE=foo.

I notice that Drush GenerateCommands uses the Symfony Console component to get input from the user for use in the commands. Maybe it could be used here?

I foresee some challenges with the current state of recipes that this sort of feature may address. In particular, there may be a need to map site-specific values to the configuration in the recipe. A few examples:

1. Avoid bloat when installing multiple recipes: For example, imagine a small business that is creating a site and wants to include a blog recipe, an event calendar recipe, an image gallery recipe, and a survey recipe. Individually, each one of these recipes may install functionally redundant configuration. A 'blog manager' role, an 'events calendar admin' role, an 'image gallery creator' role, a 'survey creator' role, etc. While it is valid for a user to have multiple roles, this can grow unnecessarily large. I can imagine a command line (or mapping file) that would prompt for things like the name of your content creator role.

2. Overcoming barriers with installing recipes on established sites: For this example, I'm a large, enterprise website that's been using Drupal for many years. I want to use a recipe that turns event content into different calendar feeds (rss, ical, and maybe some JSON standard.) I already have an event content type with named fields, so I don't want to create anything new in my data architecture. I just want to tell the recipe how to map the module's configuration to my content type. Typically we may create a layer of abstraction such as a module settings form or custom plugin. But in this case, I'm just interested in passing a few values such as the name of my content type to the script that is installing the recipe.

Just documenting as food for thought. There are multiple approaches to these issues.

#12

1. That is mostly a problem of making granular enough recipes. You're right though that this implies a recipe should ASK for input rather than creating a role specifically for its purpose. Which means that this issue is IMHO even more important than Major. To avoid "Drupal sites created by Recipes" to become incredibly complicated in its config (because every independent concern installs its own roles, node types, etc.), asking for input is critical.
   In the case of user roles, for example, it's probably necessary to allow the site builder using a recipe to choose to EITHER use an existing role OR create a new one.
2. Agreed. But isn't that what this issue is about already? There is IMHO at least one missing piece to make that feasible though — see below.

There are multiple approaches to these issues.

Interesting … can you elaborate? Because AFAICT we need to support all of these use cases to make Recipes practically usable? 😄

I think the clearest path forward (and I suspect the most reliable) is to use the one piece of information we already have and that is strongly related: recipe dependencies.

Example

Say a recipe BAR depends on a recipe FOO. Recipe FOO has an input for "content creator role". Because BAR depends on FOO, it knows that FOO already needs to know the ID of the content creator role. So FOO should still specify

  content_creator_role_id:
    from: prompt
    question: 'Pick the role that represents content creators'

but then BAR should NOT specify the same; instead it should specify:

  content_creator_role_id:
    from: dependency
    dependency: FOO.content_creator_role_id

#13: I like the idea of being able to explicitly use input values from a recipe you depend on.

Self-assigning to start implementing this.

Got an initial implementation up with some basic test coverage.

This only allows getting input from an existing config value (we can add prompting and such in another issue). It supports reusing inputs from recipes you depend on.

I tested this successfully.

I had previously applied the standard recipe which included the feedback_contact_form recipe.

The Contact form needs to be deleted to re-apply.

I then changed the site email as this is the value that is being used in the recipe to set the contact form.

I then re-apply the feedback_contact_form and see that the changes email address is not the recipient.

I tested this in my local and it works perfectly. However no errors or warnigs were shown for invalid configurations like

input:
  recipient:
    source: config
    config: ['system.invalid_config', 'mail']

Should we add a check to verify if the config exists or not?

Everything looks great and resolved from previous review. Added one comment above about invalid sources. Leaving as NR to let others comment, but I feel this could be an easy fix/win. Current fatal is as such for reference:

In ServiceLocator.php line 137:
                                                                                                                                                    
  Service "json" not found: the container inside "Drupal\Core\Recipe\InputCollector" is a smaller service locator that only knows about the "config" service.                                                                                                                                                        

I like that idea, @b_sharpe. Fixed! Also made some changes to accommodate what's coming in #3466374: Getting user input before applying a recipe.

All looks great, tests pass and expected results tested locally, RTBC

Added some review comments/feedback, happy for all of it to be pushed back on but just some thoughts that came to mind while reviewing.

@alexpott and I agreed that this can probably be committed directly to core, since its scope has been condensed to this one issue. Moving the issue, and will open a new MR.

Gave this a quick manual test and it worked exactly the way you'd expect it to in interactive mode. Screenshot:

CR written: https://www.drupal.org/node/3470507. Tagging for a final issue summary update.

The issue summary is fully up to date!

Marking as rtbc

Would be helpful if someone tried this MR with Drush recipe command - https://www.drush.org/13.x/commands/recipe/. Drush just includes the symfony command from Drupal as is so hopefully it works fine. Testing would be to make sure recipe options appear in command help and can be used successfully.

Adding to highlights

Re:

I propose one hard limitation up front: inputs cannot be used for dynamic IDs (like "enter the name of your content editor role"). Why? Because that makes recipes unpredictable, which would in turn make them less composable, which would violate a foundational design goal. Recipes are automatons, not functions. Inputs can only be used in the parameters passed to config actions.

(Having just written all that, I could see a way to maybe pull off dynamic IDs without breaking composability, but it would need further thought and hashing out, and we could always add it later. It's definitely out of this issue's scope.)

Is there an issue where dynamic IDs as being actively discussed? For all Commerce projects, we'd need to require the currency input from users applying the recipe. Currencies are config entities and the currency code is featured in the config ID. My understanding is that creating a Currency entity programmatically based on the user input would not be possible with the current state of things until support for dynamic IDs is added which is effectively blocking Commerce-based recipes.

It seems like my understanding was wrong :) We are still able to define a custom config action plugin (currencyImporter) that utilizes a currency importer service for dynamically defining currency config entity. The only non-clean part is the signature of the ConfigActionPluginInterface::apply(string $configName, mixed $value): void; that requires $configName which in my case would be something like commerce_price.commerce_currency.XYZ so it doesn't do much and feels hacky. Otherwise, it seems like I can achieve what I am aiming for.