Support batched embeddings

Adding patch from mr to be used in a project.

Adding it with patch extension

start testing

Testing report for #3568659 — Support Batched Embeddings (OpenAI Provider)

Environment:

• Drupal 11.3.9, PHP 8.4.12, DDEV
• ai module (current 1.x-dev), ai_provider_openai (current 1.x-dev)
• OpenAI API key configured, text-embedding-3-small as default embeddings model

Step 1 — Check patch status

Inspected both modules via git log. The ai_provider_openai module already contained the batch embeddings commits:

• e2bfe8e add support for batch embeddings
• 594d8e7 Add error handling to batch embeddings method

However, the dependent patch for the ai base module (#3568648) was not yet applied. The EmbeddingsInput class was missing the isBatch() and getPromptCount() methods that the OpenAI provider was already calling.

Step 2 — Apply the ai module patch manually

Modified web/modules/contrib/ai/src/OperationType/Embeddings/EmbeddingsInput.php based on the MR !1126 diff:

• Changed private string $prompt → private string|array $prompt
• Updated constructor, getPrompt(), setPrompt() to accept string|array
• Added isBatch(): bool — returns TRUE when prompt is an array
• Added getPromptCount(): int — returns count of prompts (1 for single)
• Updated toString() to join batch prompts with newlines
• Added 'is_batch' => $this->isBatch() to toArray()

Step 3 — Unit test the patch via drush

ddev drush php:eval "
use Drupal\ai\OperationType\Embeddings\EmbeddingsInput;

\$single = new EmbeddingsInput('Hello world');
echo 'Single isBatch: ' . (\$single->isBatch() ? 'TRUE' : 'FALSE') . PHP_EOL;
echo 'Single count: ' . \$single->getPromptCount() . PHP_EOL;

\$batch = new EmbeddingsInput(['Hello', 'World', 'Test']);
echo 'Batch isBatch: ' . (\$batch->isBatch() ? 'TRUE' : 'FALSE') . PHP_EOL;
echo 'Batch count: ' . \$batch->getPromptCount() . PHP_EOL;
echo 'Batch toString: ' . \$batch->toString() . PHP_EOL;
echo 'toArray is_batch: ' . (\$batch->toArray()['is_batch'] ? 'TRUE' : 'FALSE') . PHP_EOL;
"

Result: All assertions passed. isBatch(), getPromptCount(), toString(), and toArray() all behave correctly for both single and batch inputs.

Step 4 — Attempt live API call

Attempted an actual embeddings API call:

ddev drush php:eval "
use Drupal\ai\OperationType\Embeddings\EmbeddingsInput;
\$batch = new EmbeddingsInput(['The quick brown fox', 'jumps over the lazy dog', 'Hello world']);
\$result = \Drupal::service('ai.provider')->createInstance('openai')->embeddings(\$batch, 'text-embedding-3-small');
echo count(\$result->getNormalized()) . ' embeddings returned' . PHP_EOL;
"

Step 5 — Bug found: RateLimitException not properly caught

While investigating the rate limit error, discovered a pre-existing bug affecting all operation types in OpenAiProvider.php, including the new embeddings() method.

The OpenAI PHP SDK throws OpenAI\Exceptions\RateLimitException (with the message "Request rate limit has been exceeded.") on HTTP 429 responses. The catch blocks in the provider only check for the string "Too Many Requests" — which never matches — so the exception falls through to the generic \Exception handler and gets re-wrapped as AiRequestErrorException by ProviderProxy. This means callers cannot distinguish a rate limit from a general error.

Fixed in both affected locations:

• embeddings() try/catch block
• moderationEndpoints() try/catch block (runs before every embeddings call when moderation is enabled)

Added a specific catch before the generic one:

catch (OpenAiRateLimitException $e) {
  throw new AiRateLimitException($e->getMessage());
}

After the fix and a cache rebuild, the exception is correctly propagated as AiRateLimitException.

Summary

The batch embeddings logic is correctly implemented. A paid OpenAI API key (Tier 1+) is needed to fully verify the end-to-end batch API response handling.