I usually need 4096 max tokens for my agents, I see the other modules have that higher limit. This is a very important setting but I don't see it configurable anywhere?

CommentFileSizeAuthor
#2 AI chat generation.PNG134.92 KBvinayakmk47

Issue fork ai_provider_anthropic-3502348

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

drupalninja99 created an issue. See original summary.

vinayakmk47’s picture

vinayakmk47
Location Bangalore
commented
StatusFileSize
new AI chat generation.PNG134.92 KB

When the AI module is installed and Anthropic is configured as the provider, the Max Tokens setting can be adjusted directly in the admin interface. Navigate to the following URL to access this setting:

URL: admin/config/ai/explorers/chat_generator

On this page, under the Provider Configuration section (as shown in the screenshot), you will find the Max Tokens input field. This field allows you to specify the desired maximum token limit for chat completions. For example, you can input 4096 to accommodate your requirements.

This ensures flexibility and control over the token limit for your specific use case without requiring any code-level changes.

vinayakmk47’s picture

vinayakmk47
Location Bangalore
commented
Status: Active » Needs review
abhinesh’s picture

abhinesh
Primary language English
commented
Assigned: Unassigned » abhinesh

Added the possibility to alter the max output tokens in the forked repo.

abhinesh opened merge request !8

abhinesh’s picture

abhinesh
Primary language English
commented
Version: 1.0.0 » 1.0.x-dev

abhinesh changed the visibility of the branch 3502348-max-tokens-too to hidden.

pavnish made their first commit to this issue’s fork.

pavnish opened merge request !10

pavnish’s picture

pavnish commented

Hi Marcus,
Could you please review the MR? We have provided the configuration option to set the token limit and ensured it is used from this configuration.
Please let me know if you need any changes.
Thanks,
Pavnish

pavnish’s picture

pavnish commented
Assigned: abhinesh » Unassigned
majorrobot’s picture

majorrobot commented

I've tested the fork, and it worked as intended (and helped us get past a painful token bottleneck). It would be great to have this feature available. Thanks for your work on this!

(I suspect it may need a rebase, tho.)

robloach’s picture

robloach
he/him
commented
Assigned: Unassigned » robloach

I can review.

robloach’s picture

robloach
he/him
commented
Assigned: robloach » Unassigned
Status: Needs review » Needs work

May need the Config Schema to be updated to include max_tokens.

robloach’s picture

robloach
he/him
commented
Assigned: Unassigned » robloach
robloach’s picture

robloach
he/him
commented
Title: Max tokens too low, no where to configure? » Add ability to change the Max Tokens
Assigned: robloach » Unassigned
Status: Needs work » Reviewed & tested by the community

Added the config schema as nullable. Tested export, and it's working. Thanks!

camoa’s picture

camoa commented
Version: 1.0.x-dev » 1.3.x-dev
Status: Reviewed & tested by the community » Fixed

This feature is now fully implemented in the 1.3.x
branch, which ships as 1.3.0-beta2
(and also as 1.2.0-beta2).

The native-SDK rewrite introduced a proper API definitions layer.
max_tokens is declared in
definitions/api_defaults.yml as a configurable integer
field with a default of 4096 and is exposed automatically through the
standard Drupal AI model-settings UI. The provider reads the stored
value at runtime and auto-adjusts it upward when extended thinking is
enabled (so the budget never exceeds max tokens).

MR !10
targeted the old 1.0.x compatibility-layer codebase; that code path has
been superseded. No further work is needed here.

Thank you to everyone who tested and reviewed MR !10 — your feedback
confirmed the need for this setting and it was prioritised in the
rewrite accordingly.

Closing as fixed.

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

camoa’s picture

camoa commented
Status: Fixed » Closed (fixed)