Problem/Motivation

ai_metering_usage records tokens and cost for each AI call but not
how long the call took. Latency is a key signal for comparing providers and
models: a cheaper model that is consistently slow may not suit an editorial
workflow, and a latency spike can indicate a provider issue before it shows up
in error logs.

The AI Initiative issue https://git.drupalcode.org/project/ai_initiative/-/work_items/3586519 lists latency as an explicit requirement alongside
tokens and cost.

Steps to reproduce

  1. Make AI calls through any configured provider.
  2. Open the AI Usage Log at /admin/reports/ai-metering/log.
  3. There is no response time column; latency data is not recorded anywhere.

Environment

  • Drupal version: 11.3.11
  • ai_metering version: 1.0.0-alpha1
  • AI provider module + version: any (e.g. ai_provider_anthropic 1.2.2)
  • AI feature module tested: N/A

Proposed resolution

Record a start timestamp in AiPreGenerateSubscriber (the event that
fires immediately before the provider call) and make it available to the
post-generate path. In AiPostGenerateSubscriber, compute the delta
in milliseconds and write it to a new nullable latency_ms integer
column on ai_metering_usage.

Deliver the column via hook_update_N(). Default to NULL for rows
recorded before the update, so the dashboard can distinguish "not yet measured"
from "measured at zero". Surface latency in the AI Usage Log view as a new column
and in the dashboard breakdown as average latency per provider/model.

Remaining tasks

  • Add nullable latency_ms integer column to ai_metering_usage via hook_update_N() and hook_schema().
  • Capture start time in AiPreGenerateSubscriber and pass it to the post-generate path.
  • Write the delta (ms) to latency_ms in AiPostGenerateSubscriber.
  • Add latency_ms to the AI Usage Log view.
  • Surface average latency per provider/model in the dashboard breakdown.
  • Add a kernel test asserting latency_ms is populated and greater than zero.
  • Review.

User interface changes

Yes. The AI Usage Log gains a latency column; the dashboard breakdown shows average response time per provider and model.

API changes

N/A

Data model changes

Yes. A nullable latency_ms integer column is added to ai_metering_usage via hook_update_N(). Existing rows receive NULL.

AI assistance

N/A

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

codeitwisely created an issue. See original summary.

codeitwisely’s picture

Issue summary: View changes
marcus_johansson’s picture

Hey, I think the important metric here is tokens/second so you compare apples and apples. When streaming time to first byte could be interesting as well.

codeitwisely’s picture

Version: 1.0.0-alpha1 » 1.0.0-beta1
codeitwisely’s picture

Assigned: Unassigned » codeitwisely

codeitwisely’s picture

@marcus_johansson agreed. Storing raw latency_ms on purpose: per-row tokens/second would average wrong, the right metric is SUM(tokens) / SUM(latency), computed by the rollup in #3608701. Time to first byte is not measurable today: the streaming events only fire after the stream is fully consumed.

  • fbffa837 committed on 1.0.x
    Issue #3604343 by codeitwisely, marcus_johansson: Record per-call...
codeitwisely’s picture

Status: Active » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.