1.1.0-beta3: QA with Drupal CMS AI — stable release readiness

Summary

We performed comprehensive QA testing of ai_provider_mistral 1.1.0-beta3 with a vanilla Drupal CMS installation and the drupal_cms_ai recipe. The goal was to validate all AI features (agents, streaming, image alt text, Canvas AI) work correctly with Mistral as the sole provider — and assess readiness for a stable release.

Overall result: The module works well for core use cases (chat agents, tool calling, image alt text). A streaming bug was found and fixed (see #3572398). Canvas AI had some issues, but those turned out to be upstream Canvas module bugs rather than Mistral provider issues.

Test Environment

- Fresh Drupal CMS installation (via DDEV)
- drupal/ai_provider_mistral ^1.1.0-beta3
- drupal/ai_agents 1.2.x-dev
- Mistral API key configured
- Mistral set as default provider for all AI operations
- Chat model: mistral-large-latest
- Vision model: pixtral-large-latest

Test Results

1. AI Assistant Chatbot (ai_agents / drupal_cms_assistant)

- ✅ Basic chat ("What can you do for me?")
- ✅ Create content type "Event" — added unrequested description field (minor)
- ✅ Create CT "Product" with fields — minor quirks: extra field_description, text_with_summary instead of long text, media reference for image
- ✅ Add email field to CT — first attempt failed (missing field storage), self-corrected on retry
- ✅ Create vocabulary "Event Categories"
- ✅ Add terms to vocabulary
- ✅ Add taxonomy reference field — with retry
- ✅ Image alt text (pixtral-large-latest) — accurate and very fast
- ✅ Multi-turn conversation memory — correctly recalled all created content types and vocabularies
- ✅ Duplicate CT detection — correctly detected existing content type
- ✅ Log review — no unexpected errors

2. AI Image Alt Text Generation

- ✅ Correctly generated relevant alt text using pixtral-large-latest. Very fast response time.

3. Streaming

- ❌ "Cannot rewind a generator" error when using streaming_chat assistant (non-agent).
- Root cause: Partitech SDK returns a raw PHP Generator (single-pass). PromptJsonDecoder and ReplayedChatMessageIterator re-iterate the stream, which fails.
- Fix submitted: #3572398 (MR !30)

4. Canvas AI Agents

- ✅ Create hero section — also auto-generated title and meta description
- ✅ Add three-column features section — self-corrected on missing required props
- ❌ Generate title (standalone) — YAML parsing error with apostrophe in layout context (Canvas bug, not Mistral)
- ✅ Change heading text — with 3 retries, placement strategy issues
- ❌ Generate title (retry) — same YAML parsing error
- ❌ Write meta description — same YAML parsing error
- ✅ Add footer with contact info — with retries, missing required props, self-corrected
- ❌ Remove a component — wrong UUID identified (Canvas limitation, see #3571260)
- ✅ Generate meta description (after removing problematic component)
- ✅ Generate title (after removing problematic component)
- ❌ Remove three-column section — HTTP 500, upstream Canvas bug (#3572864)

Issues Filed / Referenced

Mistral provider:
- #3572398 — Streaming rewind bug (fix submitted as MR !30)

Upstream (not Mistral provider bugs):
- #3565226 (ai_agents) — Constraint named args (fixed in 1.2.x-dev)
- #3571925 (ai) — AiStreamedResponse headers
- #3572865 (canvas) — YAML parsing bug with apostrophes
- #3572864 (canvas) — TypeError on component removal
- #3571260 (canvas) — Page builder can only add, not remove

Conclusion

The Mistral provider works reliably for the core Drupal CMS AI use cases (chat agents with tool calling, image alt text generation). The one provider-specific bug found (streaming) has a fix ready for review. All Canvas AI failures are upstream issues in the Canvas module, not the Mistral provider.

We believe the module is in good shape for a stable release once the streaming fix (#3572398) is merged.