Introduce AI Agents and tools to create entire page templates using available component entities

This is now unblocked as #3530733: Introduce AI Agents and tools to Build Pages from components has been merged.

Going to close the current MR. The current approach generates a header and footer for all templates. When generating a template, the agent should generate only the main content of the page. The header and footer will be generated only if the user explicitly requests them. By default, all components will be placed in the content region. Header/footer componetnst would be placed in corresponding header/footer regions if available

I have updated the implementation.
A new form has been added under Configuration >> AI >> XB AI Theme Region Settings, which can be used to provide descriptions for different regions when global regions are enabled. These descriptions help the AI use the regions more effectively.

By default, only the main body will be generated during template generation.
The header and footer will be generated only if explicitly requested.
Header and footer components will be placed in the corresponding global regions if available. otherwise, they will be placed in the content region.

The following types of prompts are expected to work:

• "Create a template for the homepage of a pizza shop website"
• "Create a template for the homepage of a pizza shop, with proper header and footer"
• "Create a header template for a pizza shop website"
• "Create a footer template for a pizza shop website"

#3533881: Generate title and metadata for pages created from components using AI has landed. So this issue needs a rebase

Rebased and added tests. Ready for review

That method was removed in #3537422: XB AI: Validate AI generated Components before adding to XB. We should use the validation added in that issue here.

This is now blocked by #3543519: XB AI: Bring back the 'to' option in addNewComponentToLayout call in AiWizard.tsx

This is now ready for review. It extends on the code changes made in #3543744: Canvas AI: Create a common service for validations. So that issue must be merged first before reviewing this

#3544614: Canvas AI: Agent should pass values to image props correctly is a blocker now.

Linking to current blockers

Have we considered refinements to the header and footer here at all? For example, "I'd like to add a 'Start free trial" button in the header that links to #" or "I'd like to add a 'Contact' link that links to '/contact' to the header".

If the prompt is like Create a homepage template for a pizza shop website with a proper header and footer. Add a ‘Start free trial’ button in the header that links to #. Also add a ‘Contact’ link that links to ‘/contact’ in the header, then the Template Builder agent being developed here would be able to place all those components.

If the template has been generated with a prompt like Create a homepage template for a pizza shop website with a proper header and footer, and the user makes a follow up request such as Add a ‘Contact’ link that links to ‘/contact’ in the header, this will be delegated to the Page Builder agent added in #3530733: Introduce AI Agents and tools to Build Pages from components.

Also, what would happen if user provided a prompt "Make the header sticky so it stays at the top on scroll"? This seems like a common use case which we should think through. Usually you'd probably have some kind of header component which the AI probably should modify in this scenario. Alternatively if the header is an SDC, it could create a new component that wraps the existing component to add the sticky header behavior.

At the moment, this isn’t possible, as both AI agents can only add new components. We have to implement uupdating already placed components in a followup issue

Edit: Created #3549232: Canvas AI: Updating page contents with agents to support updating components, once placed

Ready for review

Now that #3540211: XB AI: User selection should be taken into account while building page is merged, this needs some updates

Ready for review

Thanks for the feedbacks narendrar,

Should create title and metadata if not exist for page as done for page builder earlier

- Done
Create a template for the homepage of an ice cream shop website: It is observed that it did not place button in Empty button slot for hero component. Also content slot is empty in spacer and desired content is placed below spacer.
This can be enforced from the component description settings form.

Create a template for the homepage of an ice cream shop website, with proper header and footer: If content in header/footer region exists, it places duplicate components also.

If user explicitly asks to generte headera and footer, agent will generate contents even if those regions already have components. I think we could open a followup to optimize this behaviour

🤔 The tests appear to run correctly locally.

Edit: The same test failed in the initial commit as well (https://git.drupalcode.org/project/canvas/-/merge_requests/18/diffs?comm...)

Edit 2: Previously the image prop values for the sdc.canvas_test_sdc.card were hardcoded in the test.

content:
  - sdc.canvas_test_sdc.card:
      props:
        title: 'Test Card'
        content: 'Test content'
        loading: 'lazy'
        image:
          src: /modules/contrib/canvas/tests/modules/canvas_test_sdc/components/card/balloons.png
          alt: 'Hot air balloons'
          width: 640
          height: 427
  - sdc.canvas_test_sdc.heading:
      props:
        text: 'Some text'
        element: 'h1'

Though it ran correctly in local, it failed in CI, probably due to some change in the src path, I assume.
I have updated the test to load the default values from the component itself and the test is passing now

Created https://www.drupal.org/project/canvas/issues/3553998 as a follow-up as per #52.

Updated the orchestrator prompt to fix #57. While testing it, I observed another issue

I am using the OpenAI provider to test the prompt:
“Create a header for a restaurant site with a logo, search bar, and main navigation menu.”

The GPT-4.1 model often doesn’t delegate the task to the corresponding sub-agent. It simply says “Your request will be processed.”

I tried the same prompt with GPT-4.1-mini, but this time it refused to create the header, saying that the entity type is not a Canvas page, even though it actually was.

I created #3554026: Canvas AI: Refine the orchestrator prompt to move the entity_type check and title/metadata generation logic to the orchestrator so that the Template Builder and Page Builder agents can focus only on their core tasks. This should at least help fix the latter issue.

The issue with 4.1 is a bit more complicated . I've often observed that it claims it’s going to use a tool, but never actually does.One option is to use the check added in #3548107: Add possibility to check if a tool was run. However, that may not be the optimal solution, because we can't always guarantee that the orchestrator or any sub-agent should use a specific tool.

For example:

• The user might just ask a question like “What can you do?” - no tool is required.
• Or the user might say “Add an accordion component to the page”, but if there is no accordion component available, the agent shouldn’t invoke a tool in that case either.

Thanks for the detailed review. I have pushed the required changes. The template builder prompt is also updated as #3554026: Canvas AI: Refine the orchestrator prompt is now merged. It now focuses only on creating the page. Page tite/metadata generation and Entity type validation is now handled by the orchestrator. So, the frequency of occurrence of the issue mentioned in #58 should also decrease.

This would require refactoring once #3555464: CanvasBuilder::render() logic around specific tool response is unclear lands

Rgarding #65, this might be specific to anthropic claude models. I am using openai and title and descriptions are getting generated correctly.

Placed component are not displayed if text color matches background.

The component description settings form can be used to add instructiins specific to each components (Eg: If there is a theme prop with light and dark options, when to use the 'light' prop and when to use 'dark'). The agent will consider these instructions when generating the template

Checking this today

I have pushed some updates:

1. Previously, the template builder was responsible for generating headers and footers. Now:

• If a user asks to create a page with a header and/or footer, the template builder agent will generate them.
• But when a user creates a page first and then asks to add a header or footer, it will be handled by the page builder agent.

Because, the page builder agent can now add components to multiple regions or positions at the same time, a prompt like “Add a header and footer for a page about comic books” will work. The components will be placed in the appropriate regions.

2. With the above change, the `SetTemplateData` tool used by the template builder agent no longer requires the reference_nodepath argument.This argument was originally needed to place footer components in the correct position. Since the page builder now handles this, the argument can be removed.As a result, the canvasAiPagebuilderHelper::processComponentsBelow() method can also be removed.

3. Updated the orchestrator prompt. Even though it may look like many changes, most of them are spacing and structural improvements, along with new template-builder–specific instructions.

4. Updated the polling messages as follows, since these are more accurate functionally:

'canvas_page_builder_agent' => $this->t('Finding components to place'),
'canvas_template_builder_agent' => $this->t('Designing the page'),

in https://git.drupalcode.org/project/canvas/-/merge_requests/18/diffs?comm..., I have added the following

• A test for the theme settings form
• Updated the existing test of set template data tool to cover complex scenarios
• Updated the schema of canvas_ai.theme_region.settings config as it was incorrect
• Fixed the failing CanvasAiPageBuilderTest

The functional test to validate the form was throwing config validation errors like 'Schema errors for canvas_ai.theme_region.settings with the following errors: canvas_ai.theme_region.settings:region_descriptions.stark missing schema ' upon saving the form. I had to add protected $strictConfigSchema = FALSE; to disable strict config schema validation. This error does not happen on manually saving the form. So not sure why the test was throwing the error

Gonna check this today

Updated the schema of theme region settings config and the test is passing now. Thanks

Looks like hallucination. Is it happening consistently?

Here is how it is supposed to work

If the prompt is in the format "Create a template for the homepage of an ice cream shop" -> Create the page layout without header and footer. Components should be mainly placed in the 'Content' region. It can also place components in other regions like Sidebar if enabled.

If the prompt is in the format "Create a template for the homepage of an ice cream shop with a header and footer" -> Create the page layout with header and footer. Header and footer components will be placed in corresponding regions if available . Else all components will be placed in the content region

Thanks., I have updated the issue summary and attaching a new demo as well. Please review

Thanks. I have pushed the changes. From #91, all suggetsions for the template builder are valid. But for page builder the following are invalid

• Replace the current hero image with a new photo.
• Move the contact form above the location map.

The page builder agent can currently only add new components to a page. It does not support updating existing components, including editing content, moving components, or deleting them. We already have an issue open for this #3549232: Canvas AI: Updating page contents with agents, and I am working on it.

I tried adding the instruction "Only component addition is currently supported. No tools are available to update, move, or delete existing components." to the orchestrator agent. After that change, the orchestrator no longer suggested update-related examples when asked for sample prompts where the page builder would be used. However, if I ask it to change the title of an existing heading component, it still calls the page builder agent.

Because of this, we likely need to update the prompts for both agents to stop this. I would prefer not to make those changes here, since this work is specifically focused on the template builder.

Could you confirm whether you're using an Anthropic model (specifically Claude 3.7 Sonnet)? This is a known issue with that model.

Thanks @jibran, @marcus_johansson
I have now added the post update hook as well.

Created #3571260: Canvas AI: The Page Builder should inform users that it cannot edit a component once it has already been placed on the page as per #95

Thanks. Moved the hook to the canvas ai module