Problem / Motivation

Part of #3572138: [META] Native Anthropic SDK integration for advanced API features. Phase 1 (#3572402: Add effort, adaptive thinking, and native token usage support) delivered the native SDK
integration with effort, thinking, and native token usage; Phase 2
(#3590963: Add prompt caching UI and PDF input support) added prompt caching and PDF input. This issue is
Phase 3 of the plan: the remaining three Anthropic-native
capabilities unused by the provider — token counting (pre-dispatch
cost/limit visibility), citations (source attribution for
document-grounded answers), and context management (server-side
compaction and context editing for long conversations).

Proposed resolution

  • Token counting — public
    AnthropicProvider::countTokens() backed by
    $client->messages->countTokens(). No new
    OperationType: drupal/ai 1.4 has no consumer for one; documented
    provider-native method instead.
  • Citations — opt-in
    CitationsConfigParam on PDF document blocks;
    response citations (all 5 location variants) normalized by a new
    CitationNormalizer into
    ChatOutput::getMetadata()['citations'], on both the
    sync and streaming paths.
  • Context management — routes through
    $client->beta->messages->create() with the
    live-verified beta headers (compact-2026-01-12 /
    context-management-2025-06-27). Hybrid posture for
    the beta-to-GA transition: admin-overridable header, and a
    one-shot retry without context management when the API rejects
    the header. Capability-gated per model via the typed
    ModelCapabilities->contextManagement flags.
  • Admin form gains three opt-in sections following the existing
    prompt-caching pattern; all features default off; config schema
    updated with constraints.

Verification

101 unit tests / 208 assertions green (44 new tests). phpcs
(Drupal + DrupalPractice) clean, drupal-rector clean, composer
valid. Beta header slugs live-verified against
platform.claude.com docs on 2026-07-01. Multi-stage adversarial review (29 findings raised across two review passes, 22 vote-confirmed, all fixed) — details in the MR description. drupalci pipeline green (phpunit, phpstan, phpcs, cspell, composer). All three features additionally live-verified end to end against the Anthropic API; see comment #3.

Remaining tasks

  • ✅ File the issue
  • ✅ Implementation (MR !35)
  • ✅ Automated unit testing coverage (44 new tests; 101 total / 208 assertions)
  • ✅ drupalci pipeline green
  • ✅ Live end-to-end verification against the Anthropic API
  • ➖ Community / maintainer review
  • ➖ Merge + release (1.3.0-beta3)

AI-Generated: Yes

Per the Drupal AI contribution policy. Built via multi-agent AI orchestration (Claude Opus 4.8 workers,
Claude Fable 5 orchestration) against a human-approved architecture:
research, implementation, and tests AI-generated; reviewed through
two adversarial review passes (code audit, 3-persona paper test,
fresh-context review, refutation voting) plus human maintainer
direction throughout. Dependencies, logic, security, and GPL
compatibility verified. Full contributor responsibility assumed.

Issue fork ai_provider_anthropic-3607960

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

camoa created an issue. See original summary.

camoa opened merge request !35

camoa’s picture

camoa commented
Status: Active » Needs review

MR !35 is up against 1.3.x and the full drupalci pipeline is green (phpunit, phpstan, phpcs, cspell, composer): https://git.drupalcode.org/project/ai_provider_anthropic/-/merge_requests/35

Beyond the unit suite (101 tests / 208 assertions, 44 new), all three features were live-verified end to end against the Anthropic API:

  • Token counting — real count returned for a live request; the admin toggle is enforced at runtime (disabled state refuses with an actionable message).
  • Citations — a test PDF produced metadata['citations'] with the correct page_location span and cited_text quoting the source sentence.
  • Context management — the request genuinely routed through the beta messages endpoint (BetaMessage response), the compact-2026-01-12 header was accepted live, and the response carried the contextManagement acknowledgment.
  • Regression — with all toggles off the GA path is unchanged (response class and request shape identical to 1.3.x HEAD).
  • Admin form — browser-verified: sections render, both #states levels behave, save round-trip persists all five new config keys.

Review pointer: the beta-to-GA posture on compaction is the part most worth scrutiny — the header is admin-overridable and a rejected beta header triggers exactly one retry without context management (rationale and evidence in the MR description).

Setting to Needs review.

camoa’s picture

camoa commented
Issue summary: View changes
codeitwisely’s picture

codeitwisely commented

Reviewed from the
ai_metering
perspective. Here is what the usage tracker sees for each feature:

Feature Model Tracked by ai_metering Tokens
Chat (regression baseline) haiku-4-5 Yes 12 in / 4 out
Context management (beta) sonnet-4-6 Yes 84 in / 4 out
Citations with PDF haiku-4-5 Yes 2103 in / 41 out
countTokens() haiku-4-5 No n/a

Citations and context management both route through the shared buildChatOutput() path, so TokenUsageDto is populated correctly in both cases. ai_metering picks them up with no changes needed. Note: context management only routes to the beta endpoint on models that support compaction (the compact_20260112 capability flag returned by the Anthropic capabilities API). haiku-4-5 does not; sonnet-4-6 does.

countTokens() is the one gap. It calls the SDK directly via AnthropicNativeClient::countTokens() without firing any drupal/ai event, so ai_metering never records the request. Quota enforcement, rate limiting, and cost attribution all break silently for sites running pre-flight token checks before agent tasks.

The fix belongs in drupal/ai core: a PostCountTokensEvent fired after a successful countTokens() call, carrying provider ID, model ID, and input token count. ai_metering would subscribe to it and record the usage. Any provider implementing countTokens() benefits automatically.

The MR is mergeable from the ai_metering side. The countTokens() tracking gap is a follow-up concern, not a blocker here.

camoa’s picture

camoa commented

Thank you for testing this from the metering side. The gating confirmation is the part I appreciate most (haiku-4-5 staying on the GA path while sonnet-4-6 routing to beta is exactly what the capability flags should be doing, and now a second module has verified it).

You are right about the gap, and since the fix belongs in core I did some digging to hand you before you file it. Three findings that change the shape a bit:

1. An operation type gets you more than a new event, for less. ProviderProxy auto-detects any interface extending OperationTypeInterface whose method name matches the interface name, and fires the existing ai.pre_generate_response / ai.post_generate_response / exception events for it with operationType as the discriminator. Core has zero per-operation events today (no PostChatEvent, no PostEmbeddingsEvent), so a CountTokensInterface plus a small CountTokensOutput rides the whole pipeline for free: ai_metering filters on getOperationType() === 'count_tokens' using the subscription it already has, and the count inherits requestThreadId correlation and typed exception mapping.

2. The proposal needs to differentiate from the existing ai.tokenizer service. Core already ships countTokens() there, but it is local tiktoken estimation with OpenAI encodings. What is missing is the provider-native exact count (Anthropic's server tokenizer, which is also correct for PDFs, images and tool schemas). Getting ahead of "we already have countTokens" will save the issue a review round.

3. It survives the Symfony AI migration. Symfony AI Platform only carries post-response TokenUsage metadata, there is no pre-flight counting anywhere in it, so this is additive and the 2.0.x branch is still taking new operation types (ExtractiveQuestionAnswering landed there in June). Targeting 2.0.x with a 1.4.x backport offer seems like the right shape. Issue #3541284 is the friendly post-response counterpart worth aligning DTO naming with.

If you file it, link it here and I will implement the Anthropic side (the interface plus CountTokensOutput replacing the current int-returning method) as a follow-up on this project, and happily co-review the core MR.

codeitwisely’s picture

codeitwisely commented

Filed the follow-up on drupal/ai core: https://git.drupalcode.org/project/ai/-/work_items/3586582