Offer the chat microphone, reading replies aloud, and a switch for the mode dropdown

Problem/Motivation

The assistant chat can only be typed into, and the mode dropdown cannot be switched off.

deep-chat, which draws both chat surfaces this module already works with, has speech built in: speechToText shows a microphone and dictates into the message box, textToSpeech reads each reply aloud. Neither surface exposes any of it:

• The AI Chatbot module's DeepChat block removes speechToText and microphone from the settings it renders, after its own hook_deepchat_settings has run, so a hook implementation cannot put them back.
• The Drupal Canvas AI panel is mounted by the Canvas editor's own React bundle, so there is no render array to configure.

A site that scopes its assistant by configuration also has no way to stop offering the mode dropdown, short of deleting its modes.

Steps to reproduce

1. Install this module on a site with AI Chatbot, or with Drupal Canvas AI.
2. Open the assistant chat: there is no microphone, and replies are never read aloud.
3. Go to Configuration > AI > AI Agent Modes > Settings: there is no way to stop offering the dropdown.

Proposed resolution

Offer deep-chat's speech API as configuration, in this module alone, with no patch to drupal/ai or drupal/canvas:

• Speech to text. A microphone, with four placements: inside the message box at the start (the default) or at the end, or outside it before or after. Plus dictation language, interim results, stop recording after sending, send after a pause in speaking with its length, interim and final text colours, voice commands, and corrections for words the recogniser mishears.
• Text to speech. Read each reply aloud, with language, voice, pitch, speed and volume. The voice list is filled in by the browser, and says which voices are online.
• A switch for the mode dropdown itself. Off attaches no behaviour, requests no options and renders no selector, on either surface.
• Per AI Assistant. The microphone, its placement, and reading aloud, each falling back to the site setting, stored as third-party settings so they travel with the assistant.

Both directions are reached the way the mode dropdown already reaches those panels: a behaviour rides along with each surface's own library and sets the properties on the mounted <deep-chat> element, so nothing upstream is patched.

Azure is deliberately not offered: it needs a subscription key or token, which does not belong in configuration. hook_ai_agent_modes_speech_alter() is provided for a site that wants it with a short-lived token of its own.

Recognition and synthesis are the browser's own Web Speech support. This module sends nothing to any service and needs no key, but Chromium-based browsers recognise in the cloud, so the audio leaves the machine through the browser. The settings form and the documentation say so rather than promising locality.

Screenshots

The Mode dropdown tab: the switch, and the Drupal Canvas AI placements as cards

The Microphone tab

The Reading replies aloud tab

The three per-assistant choices, on the AI Assistant form

The Drupal Canvas AI input row: attach, microphone, mode dropdown, send

Remaining tasks

• ✅ Offer the full speechToText and textToSpeech options as configuration
• ✅ Four microphone placements, honouring the writing direction
• ✅ Keep the microphone clear of the mode dropdown, the send button and the attach button
• ✅ A switch for the mode dropdown, gating every surface and the options endpoint
• ✅ Per-assistant overrides for the microphone, its placement and reading aloud
• ✅ Unit and kernel test coverage
• ✅ Automated functional (webship-js) scenarios
• ✅ Documentation page
• ❌ Reviewed by a human
• ❌ Code review by maintainers

User interface changes

The settings form is grouped as tabs and opens on Mode dropdown, which carries the new switch and the Drupal Canvas AI placement, whose options are cards with a small diagram each. Two new tabs: Microphone (speech to text) and Reading replies aloud (text to speech). The AI Assistant form gains an AI Agent Modes section with three per-assistant choices.

The AI Chatbot placement question is no longer asked on the settings form: that panel's placement is a per-assistant choice, falling back to the existing site value.

API changes

• New ModeManagerInterface::dropdownEnabled().
• New hook_ai_agent_modes_speech_alter(array &$settings).

Data model changes

New ai_agent_modes.settings keys: show_dropdown, speech_to_text and text_to_speech, the last two mappings. New third-party settings on ai_assistant: speech_to_text, speech_to_text_position, text_to_speech. Two post-update hooks write the defaults, both halves off, so an existing site neither starts listening nor loses its dropdown.

Release notes snippet

The assistant chat can now offer a microphone that dictates into the message box, and can read each reply aloud, using the browser's own speech support. Both are off until switched on, can be placed where they suit the panel, and can be decided per AI Assistant. The mode dropdown itself can also be switched off.

This issue and its changes were prepared with AI assistance (Claude), following the Drupal AI policy. All code and prose were reviewed before posting.