Problem/Motivation
Some AI-initiated work (e.g., image generation via text-to-image models) takes too long to complete within the chat turn's HTTP request. The AI agent needs to commit the entity and respond to the user immediately, while the slow work completes asynchronously and updates the page when finished.
Without a deferred processing system, the options are:
- Block the chat response until generation completes (unacceptable UX — 10-30s wait with no feedback)
- Kick off image generation on the back end and require the user to manually refresh after some unknown delay
Proposed resolution
Introduce a first-class deferred operation system with two client-side execution strategies:
Server-side deferred flow
- AI tool persists entity; field handler emits
DeferredOperationvalue objects for work it cannot resolve synchronously - Event subscriber stores operations in PrivateTempStore and queues a
TriggerDeferredCommand - Client shows a loading indicator (shimmer animation on placeholder), POSTs to
DeferredProcessingController::process - Controller dispatches each operation to the matching tagged processor service
- On success: returns AJAX commands to re-render affected components with highlighting
- On retryable failure (e.g., content moderation rejection): client POSTs to retry endpoint, processor rewrites the prompt via LLM, tries again
Chat-driven followup flow
- Server queues a
TriggerFollowupChatCommandwith an LLM-facing message - Client sends a synthetic chat turn to the DeepChat API (user never sees it)
- Loops on
should_continue(mirrors the agent polling loop) - On completion, POSTs to
RefreshComponentsControllerto re-render with highlights
Image generation processor
The canonical deferred processor implementation:
- Calls the configured
text_to_imageAI provider - Creates a media entity from the generated image
- Updates the element on the entity
- Implements
RetryableProcessorInterface— on content moderation failure, uses an LLM to rewrite the rejected prompt and retries (up to 2x)
Image generation field handler
An entity_blueprint field handler (ImageGenerationHandler) that intercepts entity_reference fields targeting media:
- When the AI sends
{_generate: "description"}, sets a configured placeholder media and emits aDeferredOperation - On update: leaves the field unchanged and emits the deferred operation (element is updated and swaped later)
- Provides guidance text to the AI describing the
_generatesyntax
User interface changes
- Placeholder components show a shimmer animation (dashed outline + gradient sweep) while generation is in progress
- After generation completes, the component re-renders with the real image and receives a highlight animation
Comments
Comment #3
tim bozeman commented