Problem/Motivation

The use of AI policy should require disclosing usage at the beginning of any usage. Many times the disclosure is at the end of a comment or after the question is asked, but that is too late.

In the past two weeks there has been a sharp increase in the amount of issues and comments that are essentially copied and pasted from an llm.

After discussions in slack someone suggested marking the issue summary and comment fields as for humans and using attachments to attach llm output.

Steps to reproduce

N/A

Proposed resolution

Update the core issue template to mention that issue summaries and comments should be human generated. And any llm usage should be disclosed up front at the top.

Remaining tasks

Discuss.

User interface changes

New template

Introduced terminology

N/A

API changes

N/A

Data model changes

N/A

Release notes snippet

N/A

Comments

nicxvan created an issue. See original summary.

scott falconer’s picture

scott falconer commented

Here's how I'd approach this:

Mark a set of fields as on Drupal.org as "human_readable" (issue status / comments / MR comments / body / etc.)

For these show the disclaimer

"This field is intended for human readers, and should be written by humans, for humans. "Written by humans" does not require manual typing. Dictation, translation, accessibility tools, and other assistive technologies are welcome as long as the thoughts, conclusions, and understanding are your own.

Machine-generated research, analysis, logs, or other supporting material are still welcome, but should be attached in the designated supplemental area.
"

This should allow well-intentioned contributors to participate in a constructive way that minimizes negative disruptions to humans, while still providing agent generated output and resources that might be useful once the fundamental issue or change is clear.

quietone’s picture

quietone commented
Title: [policy] update the Ai disclosure policy regarding the issue summary and comments » [policy] update issue template to state ssue summaries and comments are to be human generated
Issue summary: View changes

The title refers to the policy but the proposed resolution is about changing the issue template, they are two different things. I have updated the title so this is about the issue template. As for changes to the Drupal policy, issues for that should be files in the Governance project.

And, for reference, the two relevant items for the "Examples of contributions that do not meet this standard" from the current policy are

Using AI to generate issue summaries, comments, or reviews that lack independently verified technical insights (e.g., using an AI tool to summarize a thread simply to gain contribution credits).

Posting issue comments, MR descriptions, or forum posts that are unreviewed AI output, not your own words.

quietone’s picture

quietone commented
Title: [policy] update issue template to state ssue summaries and comments are to be human generated » [policy] update issue template to state issue summaries and comments are to be human generated
cmlara’s picture

cmlara
he/him
commented
Related issues: +#3574093: Ban slop issue summaries and comments

Possibly a duplicate of #3574093: Ban slop issue summaries and comments ?

cilefen’s picture

cilefen commented

Good luck: https://www.drupal.org/node/3156997/discuss#comment-16755820

scott falconer’s picture

scott falconer commented

Tested out this process on a issue that has been migrated to gitlab. It was easy enough to write my comment and attach the deeper agent analysis. https://git.drupalcode.org/project/ai/-/work_items/3586617#note_2378431

It does put the attachment inline, but doesn't display the contents.

Even if we don't move forward with an official human_readable field structure I would encourage people to start working in this way as 1) I presume it will save time for humans upstream but 2) it also saved time for me as a contributor as I was able to quickly write out the important info in the comment, while retaining the background info without needing to modify it into some human/machine hybrid that wouldn't have been ideal for either.

cilefen’s picture

cilefen commented

The <details> tag is good for hiding that stuff from human view on GitLab.

rhovland’s picture

rhovland
he/him
Primary language English
Location Oregon
commented

Supporting this as well. A policy somewhere on a page on the drupal.org site doesn't get in front of the person unknowingly pasting LLM output into an issue/comment like this will. One example of feedback being received about an AI copy paste comment being corrected after they were made aware of policy. They probably wouldn't have copy pasted into a comment if this change was in place.

The spam bots will still spam as they always have. The solution to that would be a monitoring system but that's a whole can of worms for another issue.