Problem/Motivation

Streaming chat responses fail with:

Cannot rewind a generator that was already run in MistralChatMessageIterator->doIterate()

Root Cause

MistralChatMessageIterator::doIterate() iterates directly over $this->iterator, which is a raw PHP Generator from the Partitech SDK's SSEClient::getStream(). PHP Generators are
single-pass — they cannot be rewound.

The AI module pipeline re-iterates the stream in two places:

1. PromptJsonDecoder::decodeStreamingMessage() — iterates chunks to test for JSON (line 61), then wraps in ReplayedChatMessageIterator (line 93)
2. ReplayedChatMessageIterator::getIterator() — calls $this->iterator->getIterator() (line 20), which triggers a new doIterate() against the already-consumed Generator

In OpenAI, on the other hand, doIterate() calls $this->iterator->getIterator() — the OpenAI PHP SDK returns an IteratorAggregate that produces fresh iterators. The Partitech Mistral SDK returns a raw Generator.

Steps to reproduce

- Enable ai_provider_mistral and ai_chatbot
- Configure Mistral as default chat provider
- configure a block with a NON-agent assistant, enable streaming, place the block e.g. under content
- Send a chat message
- Observe: Cannot rewind a generator that was already run

Proposed resolution

Replace foreach in doIterate() with manual valid()/current()/next() iteration. foreach implicitly calls rewind() on the iterator before starting, which crashes on an already-started Generator. Manual iteration avoids rewind() entirely, so re-entering doIterate() continues from the current position.

Remaining tasks

User interface changes

API changes

Data model changes

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

petar_basic created an issue. See original summary.

petar_basic’s picture

Issue summary: View changes

petar_basic’s picture

Applied the fix — replaced foreach with manual valid()/current()/next() iteration in doIterate(). foreach calls rewind() implicitly which crashes on the Partitech SDK's Generator.
Tested with both single and double iteration via drush:

drush php-eval '
  $provider = \Drupal::service("ai.provider")->createInstance("mistral");
  $input = new \Drupal\ai\OperationType\Chat\ChatInput([
    new \Drupal\ai\OperationType\Chat\ChatMessage("user", "Say hello in one sentence"),
  ]);
  $input->setStreamedOutput(TRUE);
  $output = $provider->chat($input, "mistral-small-latest", ["stream"]);
  $n = $output->getNormalized();
  echo "Type: " . get_class($n) . "\n";
  $count = 0;
  foreach ($n as $chunk) { echo $chunk->getText(); $count++; }
  echo "\nChunks: $count\n";
  '

Double iteration (the actual bug scenario — PromptJsonDecoder peeks then ReplayedChatMessageIterator re-iterates):

  drush php-eval '
  $provider = \Drupal::service("ai.provider")->createInstance("mistral");
  $input = new \Drupal\ai\OperationType\Chat\ChatInput([
    new \Drupal\ai\OperationType\Chat\ChatMessage("user", "Say hello in one sentence"),
  ]);
  $input->setStreamedOutput(TRUE);
  $output = $provider->chat($input, "mistral-small-latest", ["stream"]);
  $n = $output->getNormalized();
  echo "=== Pass 1: peek at first 3 chunks ===\n";
  $i = 0;
  foreach ($n as $chunk) { echo "  " . ++$i . ": \"" . $chunk->getText() . "\"\n"; if ($i >= 3) break; }
  echo "=== Pass 2: continue from where left off ===\n";
  $i = 0;
  foreach ($n as $chunk) { echo "  " . ++$i . ": \"" . $chunk->getText() . "\"\n"; }
  echo "Pass 2 chunks: $i\n";
  '

Also tested via AI Explorers, and AI chatbot (non-agent assistant needed for streaming option to show up).

Note that I needed to apply fix from https://www.drupal.org/project/ai/issues/3571925 to make the streaming work in chat and in explorers.

petar_basic’s picture

Assigned: petar_basic » Unassigned
Status: Needs work » Needs review

fago made their first commit to this issue’s fork.

  • fago committed 2c76f344 on 1.1.x authored by petar_basic
    fix: #3572398: Prevent streaming rewind error for streamed chat...
fago’s picture

Status: Needs review » Fixed

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.