AI can generate invalid code components: props' names may reuse slots' names [#3551659]

Comment	File	Size	Author
	Screenshot 2025-10-12 at 10.26.04.png	200.16 KB	lauriii

Comment #1

12 October 2025 at 07:27

lauriii created an issue. See original summary.

Log in or register to post comments

Comment #2

dinu r. commented 15 January 2026 at 15:51

Assigned:

Unassigned

» dinu r.

Log in or register to post comments

Comment #3

wim leers

Ghent 🇧🇪🇪🇺

commented 15 January 2026 at 16:13

Issue tags:

+Configuration schema, +validation

Yay!

Canvas' config validation reuses SDC's \Drupal\Core\Theme\Component\ComponentValidator::validateDefinition() , and that's where this validation error is coming from 🥳

Log in or register to post comments

Comment #4

wim leers

Ghent 🇧🇪🇪🇺

commented 15 January 2026 at 16:14

Title:

AI can generate invalid prop configuration

» AI can generate invalid prop configuration: props' names may reuse slots' names

Log in or register to post comments

Comment #5

wim leers

Ghent 🇧🇪🇪🇺

commented 15 January 2026 at 16:15

Actually, is this still relevant? This was apparently reported months ago.

Log in or register to post comments

Comment #6

lauriii

he/him

Finnish

Finland

commented 15 January 2026 at 18:34

I don't think this has been addressed. Ideally we could run the validation as part of generating the code and iterate until it passes.

Log in or register to post comments

Comment #7

wim leers

Ghent 🇧🇪🇪🇺

commented 15 January 2026 at 18:46

💯 (That is how I expected this to work in the first place 😅)

Log in or register to post comments

Comment #8

wim leers

Ghent 🇧🇪🇪🇺

commented 15 January 2026 at 18:47

Title:

AI can generate invalid prop configuration: props' names may reuse slots' names

» AI can generate invalid code components: props' names may reuse slots' names

Log in or register to post comments

Comment #9

dinu r. commented 26 January 2026 at 08:29

lauriii, is there a way to reproduce this?

Log in or register to post comments

Comment #10

lauriii

he/him

Finnish

Finland

commented 26 January 2026 at 08:38

There's no way to reproduce this consistently because this is essentially an issue where AI doesn't follow the instructions provided for it. The likelihood of running into this also depends on the model. For example, after some testing, we are seeing that this is happening more often on Claude Sonnet 4.5 than on Claude Opus 4.5.

I don't think it's super relevant for us to be able to reproduce this with an LLM. We know that it's happening occasionally but not very often. The goal of this issue would be to introduce a deterministic validation for the cases where the LLM goes off track with the changes. This would allow us to reliably create several components without having to worry about invalid components it may have generated. Instead of trying to reproduce this with a LLM, we should be able to establish that kind of test case with a mock LLM response.