Problem/Motivation
ai_metering_usage records tokens and cost for each AI call but not
how long the call took. Latency is a key signal for comparing providers and
models: a cheaper model that is consistently slow may not suit an editorial
workflow, and a latency spike can indicate a provider issue before it shows up
in error logs.
The AI Initiative issue https://git.drupalcode.org/project/ai_initiative/-/work_items/3586519 lists latency as an explicit requirement alongside
tokens and cost.
Steps to reproduce
- Make AI calls through any configured provider.
- Open the AI Usage Log at
/admin/reports/ai-metering/log. - There is no response time column; latency data is not recorded anywhere.
Environment
- Drupal version: 11.3.11
- ai_metering version: 1.0.0-alpha1
- AI provider module + version: any (e.g. ai_provider_anthropic 1.2.2)
- AI feature module tested: N/A
Proposed resolution
Record a start timestamp in AiPreGenerateSubscriber (the event that
fires immediately before the provider call) and make it available to the
post-generate path. In AiPostGenerateSubscriber, compute the delta
in milliseconds and write it to a new nullable latency_ms integer
column on ai_metering_usage.
Deliver the column via hook_update_N(). Default to NULL for rows
recorded before the update, so the dashboard can distinguish "not yet measured"
from "measured at zero". Surface latency in the AI Usage Log view as a new column
and in the dashboard breakdown as average latency per provider/model.
Remaining tasks
- Add nullable
latency_msinteger column toai_metering_usageviahook_update_N()andhook_schema(). - Capture start time in
AiPreGenerateSubscriberand pass it to the post-generate path. - Write the delta (ms) to
latency_msinAiPostGenerateSubscriber. - Add
latency_msto the AI Usage Log view. - Surface average latency per provider/model in the dashboard breakdown.
- Add a kernel test asserting
latency_msis populated and greater than zero. - Review.
User interface changes
Yes. The AI Usage Log gains a latency column; the dashboard breakdown shows average response time per provider and model.
API changes
N/A
Data model changes
Yes. A nullable latency_ms integer column is added to ai_metering_usage via hook_update_N(). Existing rows receive NULL.
AI assistance
N/A
Issue fork ai_metering-3604343
Show commands
Start within a Git clone of the project using the version control instructions.
Or, if you do not have SSH keys set up on git.drupalcode.org:
Comments
Comment #2
codeitwisely commentedComment #3
marcus_johansson commentedHey, I think the important metric here is tokens/second so you compare apples and apples. When streaming time to first byte could be interesting as well.
Comment #4
codeitwisely commentedComment #5
codeitwisely commentedComment #7
codeitwisely commented@marcus_johansson agreed. Storing raw latency_ms on purpose: per-row tokens/second would average wrong, the right metric is SUM(tokens) / SUM(latency), computed by the rollup in #3608701. Time to first byte is not measurable today: the streaming events only fire after the stream is fully consumed.
Comment #9
codeitwisely commented