Problem/Motivation
When a user's cloud quota is exceeded and the fallback provider does not natively
support the requested operation type (for example, Ollama does not expose a
translate_text endpoint), AiPreGenerateSubscriber
bypasses ProviderProxy entirely by reading its protected
$plugin property via ReflectionProperty:
$rawPlugin = (new \ReflectionProperty(ProviderProxy::class, 'plugin'))
->getValue($provider);
The bypass itself is intentional: calling the fallback through the proxy would
re-fire PreGenerateResponseEvent and
PostGenerateResponseEvent, confusing other subscribers unaware of
quota rerouting. Reflection is the problem, not the bypass.
The approach is fragile in two ways. First, it couples ai_metering to a private
implementation detail of ProviderProxy that can change between
drupal/ai releases without a deprecation notice. Second, the manual
logUsage() call on this path records a character-count estimate
(mb_strlen / 4) instead of the real token counts, so the dashboard
and exports are wrong for every fallback call.
Investigation result
The original summary assumed drupal/ai 1.4.x ships a provider-selection event
that routes swaps through the normal event pipeline. It does not. The events
available in 1.4.x are PreGenerateResponseEvent,
PostGenerateResponseEvent, PostStreamingResponseEvent,
AiExceptionEvent and ProviderDisabledEvent.
AiExceptionEvent fires after a failed call, so it cannot serve a
preventive reroute that must run before the paid API is contacted.
The current architecture (listen to PreGenerateResponseEvent, then
setForcedOutputObject()) is already the supported mechanism for
preventive rerouting. What 1.4.2 adds is
ProviderProxy::getPlugin(), a public typed accessor for the wrapped
plugin. That is the sanctioned replacement for the reflection hack.
If a pre-call provider-swap event is desirable ecosystem-wide (other
quota/budget modules face the same need), that would be a drupal/ai core
feature request and a follow-up issue, not something this module can solve
alone.
Steps to reproduce
On ai_metering 1.0.0-beta1 or earlier:
- Configure ai_metering with Ollama as the fallback provider
(quota.fallback_provider: ollama,
quota.fallback_model: llama3.2:3b) and set a per-user monthly
budget low enough that the next call exceeds it. - As that user, trigger a translate_text operation large enough to push the
lookahead estimate over budget (a few thousand characters). - The reroute succeeds and one row is logged with provider
ollamaand operationtranslate_text_fallback. - Compare the logged
input_tokenswith
ceil(prompt_length / 4): they match exactly, which shows the
value is the character estimate, not the count the Ollama tokenizer
reported. The real count differs by roughly 20% on typical English text.
Environment (bug reproduced)
- ai_metering 1.0.0-alpha1 through 1.0.0-beta1
- drupal/ai 1.2 through 1.4.x (the reflection read works everywhere, which is exactly the coupling problem)
Proposed resolution
- Replace the ReflectionProperty block with
$provider->getPlugin(). - Raise the drupal/ai dependency to
^1.4.2in composer.json andai:ai (>=1.4.2)in the info file, sincegetPlugin()first appears in 1.4.2. - Keep the proxy bypass and the manual
logUsage()call: the provider populatesTokenUsageDtoinsidechat()itself, so real token counts are available on the returnedChatOutputeven without the event pipeline. Read them fromgetTokenUsage()and only fall back to the character estimate when the provider returns no usage data. - Pass the real cached-token count instead of a hardcoded 0.
Verification done
- Unit tests: real token counts (input 42 / output 17 / cached 5 from a mocked provider) reach
logUsage()unchanged; the character estimate only applies when the returnedTokenUsageDtois empty. - Full module suite green (129 tests, 496 assertions), PHPCS (Drupal, DrupalPractice) and PHPStan clean on the changed files.
- End to end on a live site (Drupal 11.3.12, drupal/ai 1.4.3, PHP 8.4.17, ai_provider_anthropic as cloud provider, ai_provider_ollama llama3.2:3b as fallback): user at 2,248/3,000 tokens, a 4,640-char translate_text triggers the lookahead reroute. The usage log records provider ollama, operation translate_text_fallback, 944 input / 174 output tokens. Those are the real llama3.2:3b tokenizer counts; the character estimate would have logged roughly 1,209 input tokens, a 22% error on a single call.
- No duplicate rows: the outer wrapper call returns early on
setForcedOutputObject()before any post event, and the inner raw-plugin call fires no events. Exactly one row per fallback call.
Remaining tasks
- Open MR against 1.0.x
- Review
User interface changes
None. Reroute behavior is unchanged; token counts in the log become accurate for fallback calls.
API changes
Raises the drupal/ai dependency to ^1.4.2. No changes to ai_metering's own API.
Release impact: this ships in the next release (1.0.0-beta2). Sites on older
drupal/ai are unaffected: Composer keeps them on beta1 until they update
drupal/ai to 1.4.2 or later. The beta2 release notes must state the raised
requirement.
Data model changes
None.
AI assistance
The investigation, patch and tests were produced with AI assistance, then verified end to end on a live site as described above.
Comments
Comment #2
codeitwisely commentedComment #3
codeitwisely commentedComment #6
codeitwisely commentedMerged in 96eb28d via https://git.drupalcode.org/project/ai_metering/-/merge_requests/20. Ships in beta2, which raises the drupal/ai requirement to 1.4.2.
Comment #7
codeitwisely commented