Overview
It was discovered in #3529328: Selected component value not getting passed to AI agent in request for editing component after creating it that the AI agent, in some cases or under certain configurations, is not able to make changes to a code component directly after creating it.
I'm consistently able to reproduce the issue using OpenAI Provider as my only provider and using the default settings at /admin/config/ai/settings that are offered after installing it.
⚠️ @utkarsh_33 and @kunal.sachdev haven't been able to reproduce the problem using other providers than OpenAI.
Steps to reproduce
- Install the OpenAI Provider module, configure an API key at
/admin/config/ai/providers/openai. -
Go to AI Settings (
/admin/config/ai/settings) and make sure OpenAI is configured as the default provider for the following operation types with the respective default models:- Chat with Complex JSON: gpt-4o
- Chat with Structured Response: gpt-4.1
- Chat with Tools/Function Calling: gpt-4.1
- Open XB and the AI sidebar, enter a prompt to create a new code component. E.g.: Build a new hero component with a heading and subheading.
- After the component gets created, ask for a change. E.g.: Change the background to a solid blue color.
- Notice that the agent is not able to determine the component for which it is supposed to make the change. This is usually indicated by its response, but sometimes it goes ahead and creates a new component. I believe the latter can be traced back to the same reason: the agent is getting confused about what component to work with.
The same issue doesn't happen when an existing component is opened and changes are asked from the agent.
Proposed resolution
TBD
User interface changes
n/a
| Comment | File | Size | Author |
|---|---|---|---|
| #4 | xb-ai-code-component.mp4 | 1.63 MB | akhil babu |
Issue fork experience_builder-3534900
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #4
akhil babuThe agent uses the ai_agent_get_js_component tool to load existing components. However, it was using the component’s name directly, and if that name contained uppercase letters, the tool returned null because component IDs are always lowercase.
It's working now.
Comment #5
akhil babuComment #6
narendrarThese changes look good and are harmless, but I’m not sure why this issue occurs in the first place for OpenAI. The
selected_component(as machineName) is already passed as context to the agent, so the agent should use the correct machine name from the context.Perhaps we should consider updating the
system_promptforai_agents.ai_agent.experience_builder_component_agent.yml(Case: The tools you have to use for editing components are:) to ensure the provided context is explicitly taken into account.Comment #7
akhil babuYes, we could update the prompt and give explicit instructions to use the selected_component value for loading the component.
Currently, there’s no way for the agent to understand the ID of the code component. For example, I created a 'PieChartComponent' and asked the agent to modify it. In that case, the instruction given by the orchestrator agent to the code component builder agent was:
So the model called the tool like this:
\
which is wrong as the component ID was
piechartcomponentWe could try using the [xb_ai:selected_component] token in the code component generator agent as well.
That said, I’d still recommend keeping the code changes, as they help ensure nothing breaks even if the model hallucinates and uses the wrong ID.
Comment #8
akhil babuComment #11
akhil babuCreated a new MR to directly provide the component ID to the code component creation agent using [xb_ai:selected_component] token. Please review.
Comment #12
marcus_johansson commentedLooks good to me
Comment #13
akhil babuComment #14
narendrarChanges looks good to me. Moving it to RTBC.
Comment #17
tim.plunkettMerged, thanks!
Not backporting to 0.x, will do if someone thinks it needs to be.