Problem/Motivation

When loading content in bulk to a vector database, certain paragraphs may be flagged by moderation.
The interface does not show exactly what content caused the issue so that it can be addressed.

Proposed resolution

Add code similar to this to OpenAiProvider.php:
(lines 626-628)

if (!empty($response['results'][0]['flagged'])) {
$moderation_alert = 'This prompt was flagged by the moderation model: ' . $prompt;
\Drupal::logger('ai_provider_openai')->notice($moderation_alert);
throw new AiUnsafePromptException('The prompt was flagged by the moderation model.');
}

CommentFileSizeAuthor
#4 ai_provider_openai-log-flagged-prompts-3507407-1.patch634 bytesnico heulsen

Issue fork ai_provider_openai-3507407

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

About issue forks

Comments

mindbet created an issue. See original summary.

nico heulsen changed the visibility of the branch 3507407-log-flagged-prompts to hidden.

nico heulsen changed the visibility of the branch 3507407-log-flagged-prompts to active.

nico heulsen’s picture

nico heulsen commented
StatusFileSize
new ai_provider_openai-log-flagged-prompts-3507407-1.patch634 bytes

I have created a patch implementing the suggested solution. The patch is attached for review.

koustav_mondal’s picture

koustav_mondal commented
Assigned: Unassigned » koustav_mondal

Working on it.

koustav_mondal opened merge request !21

koustav_mondal’s picture

koustav_mondal commented
Assigned: koustav_mondal » Unassigned
Status: Active » Needs review
wouters_f’s picture

wouters_f commented
Status: Needs review » Reviewed & tested by the community

I have seen these logs and its absolutely better than nothing.
Even better would be a dead letter queue for these messages but that would be a search_api or core queue thing.

kenorb’s picture

kenorb commented

Same error here.
Is there a way to disable or bypass the moderation model to flag the content?
Ok, I've found it (see: /admin/config/ai/providers/openai):

Moderation is always on by default for any text based call. You can disable it for each request either via code or by changing manually in ai_provider_openai.settings.yml.

wouters_f’s picture

wouters_f commented

There is a way to skip moderations for your content:
https://www.drupal.org/project/ai/issues/3510599#comment-16028722

koustav_mondal’s picture

koustav_mondal commented

Hello @wouters_f, if the changes are good then please fix the issue.

jonas139’s picture

jonas139 commented

I've created a feature to handle the flagged prompts in a more manageable way. It can be reviewed here https://www.drupal.org/project/ai_provider_openai/issues/3516044.

mrdalesmith’s picture

mrdalesmith commented
Status: Reviewed & tested by the community » Needs work

Tests are failing because of a coding standards breach, so setting back to needs work.

prabha1997 made their first commit to this issue’s fork.

prabha1997’s picture

prabha1997 commented
Status: Needs work » Needs review
divyansh.gupta’s picture

divyansh.gupta commented
Status: Needs review » Reviewed & tested by the community

I have reviewed the issue and now the flagged prompts are also coming in logs.
The changes looks good to me, also the pipeline is green with no merge conflicts.
Thus moving this to RTBC!