Problem/Motivation

Things have been hunky dory when using Gemini as a provider, but when testing with Anthropic/Claude, we started receiving truncated responses for some calls.

It turns out that the max_tokens for Anthropic is somehow set to 2048 tokens. So, some responses are greater than 2048 and get truncated. See https://www.drupal.org/project/ai_migration/issues/3545198.

Add a way to configure the provider/model in the migration yml. This should include max_tokens, as well as other settings in provider modules web/modules/contrib/ai_provider_anthropic/definitions/api_defaults.yml.

Issue fork ai_migration-3546717

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

majorrobot created an issue. See original summary.

majorrobot’s picture

majorrobot commented
Issue summary: View changes
majorrobot’s picture

majorrobot commented
Related issues: +#3502348: Add ability to change the Max Tokens

majorrobot changed the visibility of the branch 3546717-modify-maxtokens-for to hidden.

majorrobot opened merge request !22

majorrobot’s picture

majorrobot commented
Title: Modify max_tokens for AI calls » Enable Model Config via Migration YML
Issue summary: View changes
majorrobot’s picture

majorrobot commented
Status: Active » Needs review

This is ready for review.

QA Steps
Note: you will need an Anthropic key and a Gemini key to test this branch.

  • Roll back any migrated content: ddev drush migrate:rollback simple_content_migration
  • Clear cache: ddev drush cr
  • Log in as an admin.
  • Go to the default providers page (/admin/config/ai/settings) and under Chat, change the Provider and Model to something other than Gemini or Anthropic. If you only have access to those providers, choose an old model instead.
  • Import with Anthropic (which is pre-set in the example module's yml): ddev drush migrate:import simple_content_migration
  • Confirm 2 nodes import with no issues.
  • If you have access, go to your Anthropic API account logs and confirm that Drupal queried Anthropic: https://console.anthropic.com/workspaces/default/logs
  • Roll back: ddev drush migrate:rollback simple_content_migration
  • Update modules/ai_migration_example/migrations/simple_content_migration.yml, change lines 13-19 to:
          provider_id: gemini
      model_id: gemini-2.5-flash
      config:
         temperature: 0
         topP: 0
         topK: 0
         maxOutputTokens: 32384
  • Clear cache: ddev drush cr
  • Run the import from Gemini: ddev drush migrate:import simple_content_migration
  • Confirm 2 nodes import with no issues.
  • Go to your Gemini API usage page, choose your project, and confirm that you queried Gemini. (Note, the reports may not use your local time.)
dmundra’s picture

dmundra
he/him
Primary language English
commented
Status: Needs review » Needs work

Anthropic migration worked but when I switched to Gemini I got an error,

> ddev drush migrate:import simple_content_migration
 [warning] Undefined array key "parts" Content.php:57
 [error]  TypeError: array_map(): Argument #2 ($array) must be of type array, null given in array_map() (line 55 of /var/www/html/vendor/google-gemini-php/client/src/Data/Content.php) #0 /var/www/html/vendor/google-gemini-php/client/src/Data/Content.php(55): array_map()                                                                                                                                              
#1 /var/www/html/vendor/google-gemini-php/client/src/Data/Candidate.php(55): Gemini\Data\Content::from()  
...

I change the migration as per the qa steps. Maybe I missed something?

majorrobot’s picture

majorrobot commented
Status: Needs work » Needs review

Thanks for finding that, @dmundra. That didn't occur in my testing at first, but now I'm seeing it.

It looks like we need to increase the maxOutputTokens value when using Gemini. It uses reasoning and thus “thinking tokens” which apparently count against your output. I increased the maxOutputTokens by a factor of 4 (8096->32384), and it ran without issue.

Updated the QA steps.

dmundra’s picture

dmundra
he/him
Primary language English
commented
Status: Needs review » Reviewed & tested by the community

Thank you @majorrobot for finding that. It worked for me after that change.

dmundra’s picture

dmundra
he/him
Primary language English
commented
Status: Reviewed & tested by the community » Fixed

Now that this issue is closed, please review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, please credit people who helped resolve this issue.

dmundra’s picture

dmundra
he/him
Primary language English
commented
Status: Fixed » Closed (fixed)