Problem/Motivation

When a user's cloud quota is exceeded and the fallback provider does not natively
support the requested operation type (for example, Ollama does not expose a
translate_text endpoint), AiPreGenerateSubscriber
bypasses ProviderProxy entirely by reading its protected
$plugin property via ReflectionProperty:

$rawPlugin = (new \ReflectionProperty(ProviderProxy::class, 'plugin'))
    ->getValue($provider);

The bypass itself is intentional: calling the fallback through the proxy would
re-fire PreGenerateResponseEvent and
PostGenerateResponseEvent, confusing other subscribers unaware of
quota rerouting. Reflection is the problem, not the bypass.

The approach is fragile in two ways. First, it couples ai_metering to a private
implementation detail of ProviderProxy that can change between
drupal/ai releases without a deprecation notice. Second, the manual
logUsage() call on this path records a character-count estimate
(mb_strlen / 4) instead of the real token counts, so the dashboard
and exports are wrong for every fallback call.

Investigation result

The original summary assumed drupal/ai 1.4.x ships a provider-selection event
that routes swaps through the normal event pipeline. It does not. The events
available in 1.4.x are PreGenerateResponseEvent,
PostGenerateResponseEvent, PostStreamingResponseEvent,
AiExceptionEvent and ProviderDisabledEvent.
AiExceptionEvent fires after a failed call, so it cannot serve a
preventive reroute that must run before the paid API is contacted.

The current architecture (listen to PreGenerateResponseEvent, then
setForcedOutputObject()) is already the supported mechanism for
preventive rerouting. What 1.4.2 adds is
ProviderProxy::getPlugin(), a public typed accessor for the wrapped
plugin. That is the sanctioned replacement for the reflection hack.

If a pre-call provider-swap event is desirable ecosystem-wide (other
quota/budget modules face the same need), that would be a drupal/ai core
feature request and a follow-up issue, not something this module can solve
alone.

Steps to reproduce

On ai_metering 1.0.0-beta1 or earlier:

  1. Configure ai_metering with Ollama as the fallback provider
    (quota.fallback_provider: ollama,
    quota.fallback_model: llama3.2:3b) and set a per-user monthly
    budget low enough that the next call exceeds it.
  2. As that user, trigger a translate_text operation large enough to push the
    lookahead estimate over budget (a few thousand characters).
  3. The reroute succeeds and one row is logged with provider
    ollama and operation translate_text_fallback.
  4. Compare the logged input_tokens with
    ceil(prompt_length / 4): they match exactly, which shows the
    value is the character estimate, not the count the Ollama tokenizer
    reported. The real count differs by roughly 20% on typical English text.

Environment (bug reproduced)

  • ai_metering 1.0.0-alpha1 through 1.0.0-beta1
  • drupal/ai 1.2 through 1.4.x (the reflection read works everywhere, which is exactly the coupling problem)

Proposed resolution

  • Replace the ReflectionProperty block with $provider->getPlugin().
  • Raise the drupal/ai dependency to ^1.4.2 in composer.json and ai:ai (>=1.4.2) in the info file, since getPlugin() first appears in 1.4.2.
  • Keep the proxy bypass and the manual logUsage() call: the provider populates TokenUsageDto inside chat() itself, so real token counts are available on the returned ChatOutput even without the event pipeline. Read them from getTokenUsage() and only fall back to the character estimate when the provider returns no usage data.
  • Pass the real cached-token count instead of a hardcoded 0.

Verification done

  • Unit tests: real token counts (input 42 / output 17 / cached 5 from a mocked provider) reach logUsage() unchanged; the character estimate only applies when the returned TokenUsageDto is empty.
  • Full module suite green (129 tests, 496 assertions), PHPCS (Drupal, DrupalPractice) and PHPStan clean on the changed files.
  • End to end on a live site (Drupal 11.3.12, drupal/ai 1.4.3, PHP 8.4.17, ai_provider_anthropic as cloud provider, ai_provider_ollama llama3.2:3b as fallback): user at 2,248/3,000 tokens, a 4,640-char translate_text triggers the lookahead reroute. The usage log records provider ollama, operation translate_text_fallback, 944 input / 174 output tokens. Those are the real llama3.2:3b tokenizer counts; the character estimate would have logged roughly 1,209 input tokens, a 22% error on a single call.
  • No duplicate rows: the outer wrapper call returns early on setForcedOutputObject() before any post event, and the inner raw-plugin call fires no events. Exactly one row per fallback call.

Remaining tasks

  • Open MR against 1.0.x
  • Review

User interface changes

None. Reroute behavior is unchanged; token counts in the log become accurate for fallback calls.

API changes

Raises the drupal/ai dependency to ^1.4.2. No changes to ai_metering's own API.

Release impact: this ships in the next release (1.0.0-beta2). Sites on older
drupal/ai are unaffected: Composer keeps them on beta1 until they update
drupal/ai to 1.4.2 or later. The beta2 release notes must state the raised
requirement.

Data model changes

None.

AI assistance

The investigation, patch and tests were produced with AI assistance, then verified end to end on a live site as described above.

Comments

codeitwisely created an issue. See original summary.

codeitwisely’s picture

Issue summary: View changes
codeitwisely’s picture

Title: Replace ReflectionProperty reroute hack with drupal/ai 1.4.x provider-selection event API » Replace ReflectionProperty reroute hack with the public ProviderProxy::getPlugin() API (drupal/ai 1.4.2+)
Version: 1.0.0-alpha1 » 1.0.0-beta1
Issue summary: View changes

  • 96eb28d6 committed on 1.0.x-3604247-replace-reflectionproperty-reroute
    Issue #3604247 by codeitwisely: Replace ReflectionProperty reroute with...

  • 96eb28d6 committed on 1.0.x
    Issue #3604247 by codeitwisely: Replace ReflectionProperty reroute with...
codeitwisely’s picture

Merged in 96eb28d via https://git.drupalcode.org/project/ai_metering/-/merge_requests/20. Ships in beta2, which raises the drupal/ai requirement to 1.4.2.

codeitwisely’s picture

Status: Active » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.