Problem/Motivation

ai_validations can validate text fields (text-prompt constraint) and image fields (image-prompt, image-classification, object-detection constraints), but it has no coverage for audio — there is no way to validate the contents of an uploaded audio file.

The AI module provides a SpeechToText operation type (web/modules/custom/ai/src/OperationType/SpeechToText/) whose provider method speechToText() returns a normalized transcript string via getNormalized(). That transcript can then be evaluated with the same techniques the module already uses for text.

Proposed resolution

Add a new Field Validation rule, AI audio constraint, that operates in two steps:

Step 1 — transcribe: build a SpeechToTextInput from the uploaded audio file and call $provider->speechToText($input, $model)->getNormalized() to obtain the transcript.

Step 2 — evaluate: reuse the existing text-evaluation approach against the transcript — either a configurable text prompt returning XTRUE/XFALSE (as in the existing text constraint) or, if the AI moderation constraint lands, run the transcript through moderation().

Implementation

Follow the existing plugin pattern; the image constraints are the closest reference since they already wrap an uploaded file in a GenericType object (use the audio equivalent rather than ImageFile):

1. src/Plugin/FieldValidationRule/AiAudioConstraintFieldValidationRule.php
2. src/Plugin/Validation/Constraint/AiAudioConstraint.php
3. src/Plugin/Validation/Constraint/AiAudioConstraintValidator.php

Provider support

Confirmed in the current code: 5 provider classes implement speechToText().

Notes

This is slightly more involved than the other proposals because it is a two-step operation (transcribe, then evaluate). It pairs naturally with the AI moderation constraint — transcript into moderation gives a clean "screen uploaded audio for disallowed content" flow with no prompt engineering. Consider sharing the transcript-then-evaluate logic so the evaluation half can be swapped between prompt and moderation.

Issue fork ai_validations-3600886

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

marcus_johansson created an issue. See original summary.

abhisekmazumdar made their first commit to this issue’s fork.

abhisekmazumdar opened merge request !5

abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented

How to test

  1. Enable ai_validations. Configure a speech-to-text provider and a chat provider at Administration > AI > Providers.
  2. Create a content type with a File field. Allow audio extensions (wav, mp3, ogg) in the field settings.
  3. Go to Administration > Structure > Field Validation Rule Sets. Create a rule set for that content type and field.
  4. Click Add rule. Choose AI audio constraint.
  5. Configure the rule:
    • Speech-to-text provider: the provider that transcribes the uploaded file.
    • Chat provider: the provider that evaluates the transcript.
    • Prompt: must end with an instruction to return XTRUE (pass) or XFALSE (fail). A default prompt is pre-filled.
    • Message: the error text shown when validation fails.
  6. Save the rule.
  7. Create a node of that content type, upload an audio file, and save.
  8. If the chat provider returns XFALSE, the node does not save and the error message appears. If it returns XTRUE, the node saves normally.
abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Assigned: abhisekmazumdar » Unassigned
Status: Active » Needs review

The 2 step pipeline described in the issue has been implemented: a speech-to-text provider transcribes the audio, then a chat provider evaluates the transcript using the XTRUE/XFALSE protocol the text and image constraints already use.

Two decisions were taken that go beyond the original plan:

Two provider selectors instead of one. The issue described a single provider, but speech-to-text and chat are different operation types and are commonly served by different providers (e.g. Whisper for transcription, a GPT-class model for evaluation). The configuration form exposes two independent ai_provider_configuration elements. To support this in the base class, buildModelDefaultValue() in AiConstraintFieldValidationRuleBase was refactored into buildProviderDefaultValue(string $configKey), which any subclass can call for any config key, not just the primary one.

Transcription helper moved to the shared base. Rather than putting the AudioFile/SpeechToTextInput wiring inside the validator only, transcribeAudio() was added to AiConstraintValidatorBase alongside loadImageFile() and runBooleanChat(). The evaluation step passes the returned transcript string straight to runBooleanChat(), so the prompt format and violation handling match the existing text and image constraints. The helper is also available to future constraints without duplication.

AI disclosure: I used Claude Code to examine the existing code, analyze what needed to be done, summarize everything, and implement the changes. I self-tested everything locally and most importantly reviewed the code before posting.

marcus_johansson’s picture

marcus_johansson commented
Assigned: Unassigned » marcus_johansson
marcus_johansson’s picture

marcus_johansson commented
Assigned: marcus_johansson » Unassigned
Status: Needs review » Needs work

I checked it and it looks good. I've added on comment, you can ignore it if you want and set to "RTBC", but it might be nicer with a generic helper method there.

abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Status: Needs work » Reviewed & tested by the community

Claude was feeling down so I managed to write a line of code, which made him feel great.

Anyway, I'm marking this RTCB.

  • abhisekmazumdar committed 94e5c2d3 on 1.3.x 
    feat: #3600886 Add AI audio constraint using speech-to-text and chat...
abhisekmazumdar’s picture

abhisekmazumdar
he/him
Primary language English
Location India
commented
Status: Reviewed & tested by the community » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

arianraeesi’s picture

arianraeesi commented
Status: Fixed » Closed (fixed)