[Tracker]
Update Summary: [One-line status update for stakeholders]
Short Description: Replace the custom provider/model selectors in the AI Validations FieldValidationRule plugins with the Provider Configuration Form Element, with default models enabled.
Check-in Date: MM/DD/YYYY
[/Tracker]

Problem/Motivation

The AI Validations module currently rolls its own provider/model selection UI in each FieldValidationRule plugin that doesn't allow for default model, making recipes hard. The AI module now ships a reusable Provider Configuration Form Element that handles this consistently, including default model selection. See AI Provider Configuration Form Element.

Proposed resolution

  • Switch AiTextConstraintFieldValidationRule to the Provider Configuration Form Element.
  • Switch AiImageConstraintFieldValidationRule to the Provider Configuration Form Element.
  • Switch AiImageClassificationConstraintFieldValidationRule to the Provider Configuration Form Element.
  • Enable default models on the element so each rule falls back to the site's configured default.

AI usage (if applicable)

[x] AI Assisted Issue
This issue was generated with AI assistance, but was reviewed and refined by the creator.

[ ] AI Assisted Code
[ ] AI Generated Code
[ ] Vibe Coded

- This issue was created with the help of AI

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

marcus_johansson created an issue. See original summary.

ahmad-khalil-imagex’s picture

Assigned: Unassigned » ahmad-khalil-imagex

ahmad-khalil-imagex’s picture

Assigned: ahmad-khalil-imagex » Unassigned
Status: Active » Needs review
marcus_johansson’s picture

Assigned: Unassigned » marcus_johansson

Reviewing

marcus_johansson’s picture

Assigned: marcus_johansson » Unassigned
Status: Needs review » Needs work

Some feedback there. One of them you can skip if you feel like it, the other ones should be done if I didn't missunderstand something.

I also wonder if we should set these rules to final, so we can be pretty lenient with changing them in the future. Less of an issue though.

ahmad-khalil-imagex’s picture

Assigned: Unassigned » ahmad-khalil-imagex
ahmad-khalil-imagex’s picture

Assigned: ahmad-khalil-imagex » Unassigned
Status: Needs work » Needs review

Refactored code as suggested.

marcus_johansson’s picture

Status: Needs review » Needs work
Issue tags: +Needs tests, +AI Initiative Sprint, +AI Product Development

The code looks great, except for the cspell error. Setting back to needs work because of it, see: https://git.drupalcode.org/issue/ai_validations-3586397/-/jobs/9594252

I've set to needs testing before this can be merged - one example of how it can be tested is documented here: https://project.pages.drupalcode.org/ai/1.3.x/contribute/testing/full_pa...

An options is to write functional javascript tests.

ahmad-khalil-imagex’s picture

Status: Needs work » Needs review

Fixed cspell error.

a.dmitriiev’s picture

Status: Needs review » Needs work

As per Marcus' comment, the functional javascript test is needed.

ahmad-khalil-imagex’s picture

Assigned: Unassigned » ahmad-khalil-imagex
ahmad-khalil-imagex’s picture

Assigned: ahmad-khalil-imagex » Unassigned
Status: Needs work » Needs review


Added functional JavaScript test coverage for the Provider Configuration Form Element switch in web/modules/contrib/ai/modules/ai_validations/tests/src/FunctionalJavascript/AiValidationsProviderConfigurationFormElementTest.php.

5 tests, 53 assertions, all passing locally (Selenium 3, headless chromium, ~44s):

  • testTextRulePersistsSelectedProvider — adds an ai_text_prompt_constraint_rule via the UI, asserts the ai_provider_configuration select renders with the test provider/model option, saves, and verifies the rule's data.provider is persisted as the provider__model string.
  • testImageRulePersistsSelectedProvider — same flow against ai_image_constraint_rule (exercises the chat_with_image_vision pseudo-operation path).
  • testImageClassificationRulePersistsSelectedModel — guards the legacy model config key (vs. provider) against accidental renames on ai_image_classification constraint_rule.
  • testDefaultProviderOptionIsAvailableAndPersists — with a default provider configured for chat, asserts the "Default" option appears, selects it, and confirms an empty provider string is persisted (the use_default path).

Test setup notes:

  • Uses WebDriverTestBase with the minimal profile (avoids a transient action-plugin discovery ordering when node is co-installed with ai_test).
  • Uses the ai_test module's echoai provider with gpt-test — same fixture the AI module's own AiProviderConfigurationElementTest uses.
  • Creates a node:article rule set, an image field exposed on the form display, and an admin user with administer field validation rule set + administer ai permissions in setUp().
  • A key gotcha worth flagging: selecting the provider triggers an AJAX rebuild that wipes previously-filled siblings, so the helper primeRuleFormFields() populates title + column BEFORE the provider is selected.
  • PHPCS (Drupal + DrupalPractice) and PHPStan level 1 both pass clean.


On the manual Setup ai_validations walkthrough: I did not automate the end-to-end "Car Image Validation" scenario. The validator's XTRUE/XFALSE protocol is hard-coded as str_contains($response->getText(), 'XTRUE'), and the documented prompt contains the literal string XTRUEai_test's EchoProvider echoes the prompt back, so the validator always passes regardless of the image. Reproducing the fail/pass round-trip would require either a custom test provider or a runtime-built ai_mock_provider_result matching the exact JSON-encoded ChatInput (including base64 image bytes + runtime file id), which felt out of scope for this MR comment. Happy to add it as a follow-up if you'd like.

abhisekmazumdar made their first commit to this issue’s fork.

abhisekmazumdar’s picture

Good work on this MR, the switch to the Provider Configuration Form Element is clean and the functional JavaScript tests added in comment #12 are thorough.

Two small fixes have been added on top of the existing work:

1. AiObjectDetectionConstraintValidator was not calling resolveProviderOption()

The three rule plugins covered in the issue title (ai_text_prompt_constraint_rule, ai_image_constraint_rule, ai_image_classification_constraint_rule) were already updated to call resolveProviderOption() on their validators, which enables the default-provider fallback. The object detection validator (AiObjectDetectionConstraintValidator) was missed, it was still reading $constraint->model directly and passing it to loadProviderFromSimpleOption() without going through the base class method. This means the object detection rule could not fall back to the site's configured default provider, which contradicts the goal of the issue. The fix adds the resolveProviderOption() call to bring it in line with the other three validators.

2. Added an explanatory comment on $strictConfigSchema = FALSE in the test class

The reason for bypassing strict config schema validation was described in comment #12 (field_validation.rule_set.* does not cover the data sub-keys of individual rules), but it was not captured in the code. A doc comment has been added above the property so the next reader does not have to hunt through issue comments to understand why it is there.

Keeping it in Needs Review for another round of eyes before RTBC.

AI disclosure: Code review, test analysis, and comment text were prepared with AI assistance and manually reviewed before posting.

marcus_johansson’s picture

abhisekmazumdar’s picture

abhisekmazumdar’s picture

Tested MR !3 on local Drupal 11 environment with OpenAI and EchoAI providers configured. All paths work as expected.

What I tested

  1. AI Text rule: explicit provider: Provider field renders as ai_provider_configuration select (not a plain text field). Selected OpenAI gpt-4o, saved, re-edited provider was pre-selected on reload. ✓
  2. Default provider option: "Default" appears as the first option. Selecting it stores an empty string in config; on re-edit it correctly reloads as "Default". ✓
  3. AI Image rule: Same behaviour as text rule: explicit provider persists correctly across save/edit. ✓
  4. AI Image Classification rule: Stores to the model key (not provider) as expected for backward compatibility. ✓
  5. AI Object Detection rule: The resolveProviderOption() fix is in place: the validator no longer reads $constraint->model directly. ✓
  6. AJAX workaround (hook_form_alter): Changed provider on an existing rule via AJAX, saved: new selection persisted correctly. ✓

Screenshots

Provider renders as ai_provider_configuration select
Explicit provider selected
Default option selected
Default persists on re-edit
AJAX provider change
AJAX change persisted after save

Plan

I'm satisfied with the state of this MR. If anyone has objections or spots anything I've missed, please comment in the next few days otherwise I'll be merging this shortly.

This also resolves #3538581: Add possibility to add default models to AI Validations.

AI assisted (Claude Code): testing steps and comment drafted with AI assistance, all findings manually verified.

abhisekmazumdar’s picture

And yes, the deprecated error for the field_validation module is fixed here: #3509056: PHP 8.4 implicit nullable deprecation

Next, I will create version 3.0.0-beta7 for field_validation, which will have the fixes and the warnings will be gone.

abhisekmazumdar’s picture

Status: Needs review » Reviewed & tested by the community
abhisekmazumdar’s picture

Status: Reviewed & tested by the community » Fixed

Merged!!
Thanks to ahmad-khalil-imagex for the implementation and marcus_johansson for the thorough review. This also resolves #3538581: Add possibility to add default models to AI Validations.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

ahmad-khalil-imagex’s picture

Created issue to tag a release for this: https://www.drupal.org/project/ai_validations/issues/3600260

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.