When using the Chat action, I can't seem to figure out what the "Token input" configuration field actually does.

I assumed that any tokens I added to this field would be replaced and added to the prompt, but this does not seem to be the case.

When I need to use a token value in the prompt, I have to add it the usual way - directly in the prompt (with square brackets.)

Looking at the code, it looks like the "Token input" token value is replaced and stored as configuration, but never used (https://git.drupalcode.org/project/ai_integration_eca/-/blob/1.0.x/src/P...)

What am I missing?

thanks,
-mike

Comments

ultimike created an issue. See original summary.

ultimike’s picture

ultimike
Primary language English
Location Florida, USA
commented

It appears that I have the same question for the "Schema" configuration field for this action as well - it doesn't appear to have any affect on the output...

-mike

jurgenhaas’s picture

jurgenhaas
he/him
Primary language German
Location Gottmadingen
commented
Version: 1.0.0-rc3 » 1.0.x-dev
Component: Documentation » Code
Category: Support request » Task
Status: Active » Needs work

This implementation is not ideal and will hopefully be cleaned up soon before we publish the first stable release.

The Token input field optionally takes the name of 1 token, which will be used to replace that token in the prompt. So, if you have a prompt like "Use the model [model] to calculate 1+1." and you use model as token input, then the value of that token will be available to replace {model] in the prompt. However, if no token input is provided, all tokens will be available for replacement in the prompt.

In other words, the token input field seems to be unnecessary and should be removed.

ultimike’s picture

ultimike
Primary language English
Location Florida, USA
commented

@jurgen - understood about the "Token input" field.

Maybe you missed my followup question (comment 2 above) - how about the "Schema" field? Can you give me an example of what this should look like?

thanks,
-mike

jurgenhaas’s picture

jurgenhaas
he/him
Primary language German
Location Gottmadingen
commented

@ultimike when I worked on #3 I did remember that I saw that additional question but couldn't find it. Most likely, I had the issue open in the browser and forgot to reload.

So, here comes the answer to that question: without defining a schema, there is only very little control over the data type and structure that the action receives back from the agent. With some instructions in the prompt, one may be lucky, but it's never a contract.

By providing a schema, the chat will instruct the agent to only ever return the results as a JSON-encoded string following the schema. An example schema may look like this:

{
  "name": "countries_founded",
  "strict": true,
  "schema": {
    "type": "object",
    "properties": {
      "countries": {
        "type": "array",
        "minItems": 1,
        "items": {
          "type": "object",
          "properties": {
            "country_code": {
              "type": "string",
              "pattern": "^[A-Z]{2}$"
            },
            "country_name": {
              "type": "string",
              "minLength": 1
            },
            "year_founded": {
              "type": "integer",
              "minimum": 0,
              "maximum": 2025
            }
          },
          "required": ["country_code", "country_name", "year_founded"],
          "additionalProperties": false
        }
      }
    },
    "required": ["countries"],
    "additionalProperties": false
  }
}

Writing such a schema by hand when configuring the ECA model is a big ask on the user. I think we should find way to make that easier to achieve. As this is something that comes from the AI module, that solution should probably be discussed in that context. On the other hand, if AI usage inside ECA becomes more popular, we may want to get started on this purely in the modeler. I'd be open for that.