Problem/Motivation
Currently due to the following code, the system prompt will be set twice. This does not affect quality, however it adds extra input tokens:
if ($this->chatSystemRole) {
$chat_input[] = [
'role' => 'system',
'content' => [
[
'type' => 'text',
'text' => $this->chatSystemRole,
],
],
];
// If its o1 or o3 in it, we add it as a user message.
if (preg_match('/(o1|o3)/i', $model_id)) {
$chat_input[] = [
'role' => 'user',
'content' => $this->chatSystemRole,
];
}
else {
$chat_input[] = [
'role' => 'system',
'content' => $this->chatSystemRole,
];
}
}
Steps to reproduce
Proposed resolution
Make sure its only set once.
Remaining tasks
User interface changes
API changes
Data model changes
Comments
Comment #2
marcus_johansson commentedComment #4
marcus_johansson commentedComment #5
marcus_johansson commentedComment #6
marcus_johansson commentedFor testing, please test before and after the MR, to make sure that it works as advertised.
Comment #7
dan2k3k4 commentedWorking fine for me
Comment #8
marcus_johansson commentedThank you @dan2k3k4, getting merged.