I usually need 4096 max tokens for my agents, I see the other modules have that higher limit. This is a very important setting but I don't see it configurable anywhere?

CommentFileSizeAuthor
#2 AI chat generation.PNG134.92 KBvinayakmk47

Issue fork ai_provider_anthropic-3502348

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

drupalninja99 created an issue. See original summary.

vinayakmk47’s picture

vinayakmk47
Location Bangalore
commented
StatusFileSize
new AI chat generation.PNG134.92 KB

When the AI module is installed and Anthropic is configured as the provider, the Max Tokens setting can be adjusted directly in the admin interface. Navigate to the following URL to access this setting:

URL: admin/config/ai/explorers/chat_generator

On this page, under the Provider Configuration section (as shown in the screenshot), you will find the Max Tokens input field. This field allows you to specify the desired maximum token limit for chat completions. For example, you can input 4096 to accommodate your requirements.

This ensures flexibility and control over the token limit for your specific use case without requiring any code-level changes.

vinayakmk47’s picture

vinayakmk47
Location Bangalore
commented
Status: Active » Needs review
abhinesh’s picture

abhinesh
Primary language English
commented
Assigned: Unassigned » abhinesh

Added the possibility to alter the max output tokens in the forked repo.

abhinesh opened merge request !8

abhinesh’s picture

abhinesh
Primary language English
commented
Version: 1.0.0 » 1.0.x-dev

abhinesh changed the visibility of the branch 3502348-max-tokens-too to hidden.

pavnish made their first commit to this issue’s fork.

pavnish opened merge request !10

pavnish’s picture

pavnish commented

Hi Marcus,
Could you please review the MR? We have provided the configuration option to set the token limit and ensured it is used from this configuration.
Please let me know if you need any changes.
Thanks,
Pavnish

pavnish’s picture

pavnish commented
Assigned: abhinesh » Unassigned
majorrobot’s picture

majorrobot commented

I've tested the fork, and it worked as intended (and helped us get past a painful token bottleneck). It would be great to have this feature available. Thanks for your work on this!

(I suspect it may need a rebase, tho.)

robloach’s picture

robloach
he/him
commented
Assigned: Unassigned » robloach

I can review.

robloach’s picture

robloach
he/him
commented
Assigned: robloach » Unassigned
Status: Needs review » Needs work

May need the Config Schema to be updated to include max_tokens.

robloach’s picture

robloach
he/him
commented
Assigned: Unassigned » robloach
robloach’s picture

robloach
he/him
commented
Title: Max tokens too low, no where to configure? » Add ability to change the Max Tokens
Assigned: robloach » Unassigned
Status: Needs work » Reviewed & tested by the community

Added the config schema as nullable. Tested export, and it's working. Thanks!