Does the policy on AI-generated content need changes?

I think we need a policy for such cases. I'm okay with it being used to for example make wordings of a sentence better. But just plainly copy pasting answers like in the 3 links you posted is a no-go for me.

Here is the policy which Stack Overflow (and all subs) are following: https://meta.stackoverflow.com/questions/421831/temporary-policy-chatgpt...

In a nutshell:

Overall, because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking and looking for correct answers.

EDIT: Just noticed the policy is already in the OP if you click through on the web page of SO.

Yes a policy would be good. I can see no motive other than spamming for posting such stuff. Marking it spam makes sense to me. Can we have an AI bot to test posts for ChatGTP? It usually seems to churn out material (intelligently?) cut and pasted from elsewhere, which should be identifiable.

New examples, some of these have thankfully been deleted or unpublished by gisle already:

https://www.drupal.org/project/drupal/issues/3366843#comment-15158138
https://www.drupal.org/project/drupal/issues/3224941#comment-15155329

These were very long, obviously generated comments, with no relationship to the discussion going on. They could possibly in some cases look more plausible if they were the first comment on a new issue instead of a non-sequitur.

I'm not entirely sure this needs a specific policy yet - these were worthless spam, so we can just ban the spammer; exactly how the text was produced was secondary except for the speed and volume.

The standard definition of "spam" is that it is content posted for the purposes of advertising.

hmm I think that's an incomplete definition. Let's say someone really loves the book module. So they post 3,000 nearly identical messages on random core issues about how much they love the book module. They're not advertising anything except for how much they love the book module, but it's still spam - bulk messages posted indiscriminately and unsolicited. Another example would be someone posting exactly the same support question in 15 different slack channels within the space of a minute.

However, there's a lot of bad content posted on Drupal. Being bad does not make it spam according to the above definition.

This is true, but we've also had cases like someone won't fixing 50 random issues in a day which swiftly resulted in a ban. There is a type of behaviour that is this kind of bulk + low effort posting, which for me is best described by 'spamming'. I think it's fine if there's another term for it, just that's the one I associate with it.

The addition looks basically fine, but I worry that it's going to let through not incorrect but redundant, useless, and lengthy content with a disclaimer that it was generated by AI, which I also don't want to wade through. I guess that can still be covered by 'not relevant' and can always be tightened later.

Yes, spamming is also used for posting the same content multiple times, but we do not treat people doing that like spammers; differently, even the accounts used by people who post the same comment for issues that must be closed (like I do) should be blocked.

We are restrictive about what we call spam because spammers are blocked without warnings, while we warn people who is doing something they should not be do, like repeatedly closing old issues. The account could still be temporary blocked, for example to avoid that person keeps closing issues at the rate of 10 issues per minute, but we also send a message to let the person know what it should not be done on drupal.org.

We need a policy because when people do something they should not do, we contact them and give a link to a page explaining what should not be done. I agree, it can be difficult to write a policy about AI-generated content, but that policy does not need to give too much details about why we do not like AI-generated content in issue queues and other places; it could do like the Drupal.org Terms of Service page, which does not say why shared accounts are not allowed to make commits in drupal.org repositories.

Speaking as a site moderator, I prefer there to be clear and objective policies for deleting content and and banning users.

OK I agree with this, there will be occasions where someone comes up with some new horrible behaviour that's not covered, but then it can at least be added retrospectively. The main question for me was whether there needed to be a specific policy for AI-generated content or whether it could be covered by a less-specific one encompassing the same kinds of content whether human or AI-generated. The 'trash' policy deals with some of that. I haven't had site moderation permissions since I resigned when a spyware module company was re-instated without discussion in about 2010 or similar, so a bit out of touch.

As @gisle noted above, I've made a very basic effort to start developing a policy for AI generated content:

In which I updated both:
https://www.drupal.org/docs/develop/issues/issue-procedures-and-etiquett...
And:
https://www.drupal.org/drupalorg/docs/marketplace/abuse-of-the-contribut...

This may be a bit tricky to write a policy for, because (at least for the moment) I've been leaning towards allowing AI generated content if and only if the use of AI is disclosed, and the user posting has reviewed and edited the material for accuracy.

But maybe that's most of what we need:

Users agree to do the above when using AI, or else the account will be temporarily suspended, and the user contacted with a link to the policy pages above.

Users will be unblocked when they have acknowledged the policy materials and agreed to follow them in future. After an additional violation they may be blocked permanently.

Thoughts on what else we should include?

I think it would be great if the policy would apply to content in the drupal.org planet feed too. As written, the policies wouldn't prevent sites from using generated content if they so choose, but requires a baseline of human review, quality, and attribution that would increase the overall quality of content on the feed.

AI is good at writing. @deviantintegral is there some existing (or predicted) AI related problems within drupal planet feed?

It may be good at the structure of writing, but hallucinations are a challenge still.

There's been some articles over the past 6 months that have felt like AI-generated content in that they were filled with generalizations and so on. But, I'm not confident enough to link to them. I think a policy would get ahead of things so that if content starts publishing problematic content there is a guideline to refer authors and publishers too.

In considering our approach to AI-generated content, the emphasis should be on the quality of the text rather than the means by which it was created.

It's important to recognize the distinction between AI that generates content independently and AI that is employed to articulate and polish a user's pre-existing ideas effectively. While the first can be problematic, the second is merely better grammar correction.

Just to prove my point, AI mostly wrote this based on: `Write a comment: Focus on text quality, not how created; Differentiate AI can be used to make up content, or to formulate users' ideas nicely - the second is good.` Having a first draft and then refining it is much easier for me than writing from scratch.

PS: If someone were actually able to provide correct and helpful comments with an LLM and RAG, that would be wonderful. I wasn't able to do this yet.

You are correct, but I think the issue that started it all here was that some user was posting forum answers by just copy pasting the question and then posting the (very long) answer of ChatGPT or whatever he used.

Like you say, if it’s to help you write what you’re trying to say (grammar/wording) it’s fine to an extent that it isn’t noticeable. Just like your example.

The tools will become better and more difficult to detect anyway over time. I think we need to keep an eye on this and keep evaluating what is happening in the community.

There has been a few veiled attempts at SEO link spam recently, by posting 5-6 sort of correct answers, and also some wrong answers -- seemingly written by an LLM such as ChatGPT -- and then finally a post with a link to a site:

• https://www.drupal.org/u/monapeterson
• https://www.drupal.org/u/albert-s

If you check the visuals and source of the two WP web sites, they look nearly identical. In this post, one even answers the other.

So I agree with the current policy of not allowing LLM generated content in the forums:

Because the average quality of answers generated by ChatGPT and other AI tools seldom makes these answers useful, posting of answers created by AI-tools is considered harmful to community and to users who are coming to Drupal.org seeking support. Therefore, you should not use AI to generate your answer to a support question, feature request or bug report.

https://www.drupal.org/docs/administering-a-drupal-site/troubleshooting-...

Maybe this rule could be used more?

For instance, posting a link to your own website and asking how to convert it to Drupal is too generic (as it cannot be answered beyond "Hire a consultant") and will be routinely unpublished.

https://www.drupal.org/docs/administering-a-drupal-site/troubleshooting-...

Drupal RAG

I agree @rolandschuetz, a functioning Drupal RAG providing precise, non-hallucinatory answers would be useful. Akansha Saxena is closest, I think, see Inside the Codebase: A Deep Dive Into Drupal Rag Integration. She's also on Planet Drupal under Akansha Tech Journal.

There's a much worse problem: people now post AI generated modules which were clearly never even read by a human. The cherry on the cake is they have the audacity to opt such code into security coverage. I just reported a security hole in such and I haven't even started looking seriously. These modules should be removed from security coverage and their publisher needs to lose their security coverage privileges IMO perhaps based on a strike system. I doubt we can stop the proliferation of them because they will just post this garbage to github otherwise but no one should be able to force the security team to deal with slop.

Sure, to err is human, every software has bugs not not like these.

It will be a thankless, contentious job for sure. Probably a vote by X people with Y roles could help?

I tried to review an MR against experience builder recently, and it turned out the entire MR was LLM-generated and had not been reviewed by a human, none of this was disclosed in the issue summary. It wasted more than an hour of my time wondering wtf was going on. See #3515646: Add automated <img srcset> generation.

I think there needs to be a disclosure policy, and when that's repeatedly broken, issues and projects should be treated as spam.

When there's disclosure, and the project code is still un-reviewed slop, which is the situation that chx describes, that is starts to feel like we need a new issue for the Technical Working Group (which is not properly functional)/DA/Security team. Maybe a new issue in https://www.drupal.org/project/issues/securitydrupalorg to start with?

I have seen some truly awful Drupal modules posted in my time, but LLMs allow these to be written and posted much, much faster than they previously could be.

Linking a similar core issue.

First off: I'm aware of the user and modules refereed to in #23, and agree they are poorly written (to the extent I've added them to an internal list of modules to not to utilize and maintainers to avoid).

These modules should be removed from security coverage and their publisher needs to lose their security coverage privileges IMO perhaps based on a strike system.

I am weary of this as an option. Responsible/Ethical/Coordinated disclosure is a key aspect of security. It would be one concept if D.O. did not use wording such as Security issues do not need to be privately reported for the module_name project., with this wording D.O. is actively encouraging public uncoordinated disclosure for those not 'opted in'.

Any action D.O. takes to remove the ability for maintainers to compel private disclosure is an affront to security. D.O. requiring a test and to remain in the 'good blessings' of the Security Team would be conflict with security industry norms.

To put this in another light, how would the community feel if I demanded the Core Team pass an arbitrary test that I personally create, with a drawn out process of months to approve with the ability to revoke based on a condition I decide,, before I would cease disclosing vulnerabilities publicly? I presume the community would be very upset, especially in the case of a Drupelgeddon level vulnerability, and yet that is the standard applied to contrib maintainers today that this suggests extending.

Whatever that occurs for a policy regarding AI generated code there should not be considering the compromising of security privileges in the equation. Consider LLM code spam, consider LLM code unacceptable for commit due to copyright laws, consider it unwanted and target the user on those grounds, whatever is done, do not make a policy that would compromise the ability to request private disclosure.

My two cents: already the security team has the policy to mark unmaintained projects unsupported. Spraying a probabilistic series of PHP tokens into git is not maintainership.

But, it's up to the security team whether they share my concern of being overrun.

@cmlara I actually agree, in this specific case, revoking git access altogether would be better. I really do not like the 'opt in approval' process, although it's better than what it replaced, but we do need to be able to prevent people from bulk-publishing inherently flawed code on Drupal.org. I've opened #3521303: LLM-generated modules have been published.

In the new first-time contributor workshop slides that volkswagenchick developed we have a single slide titled "Use of AI" with the following scripted notes. This could be a good start for an official policy.

There is no doubt that artificial intelligence tools such as ChatGPT can be powerful ways to jumpstart code or content. However, AI systems still have significant flaws. Often times the code they produce is non-functional, and the content they create includes assertions or citations that are untrue.

When using AI in the course of making a contribution to Drupal, we require you to:

Disclose that AI was used in crafting the code or content.
Carefully review and test the output, to ensure it is relevant, and that it works.
Provide human intervention to correct inaccuracies, mistakes, or broken code.
Bulk use of AI when it is not relevant to an issue, provides broken or unusable code, or provides false information will likely result in a ban.

I am not sure why we mention it as required, but it probably should be changed to recommended for now pending policies.

I am not sure why we mention it as required, but it probably should be changed to recommended for now pending policies.

That appears to be a direct pull from the Issue Etiquette page .

Hestnet added basic policy in comment #14 to the Credit Abuse policy and added the Etiquette page text.

Discussion on this issue switched to “does the policy need changes” after that post.

cmlara, thank you. That jarred my memory. I was trying to find where it came from.

Updating title to better reflect the status of this issue after #14's creation of policy by D.A. staff.

I updated the issue summary with links from #14 and added the links to recent issues in the examples item list.

I added an example of a module porting exercise using ChatGPT. The use of which is disclosed as part of the merge request (with prompts included in the merge request).

Adding a related issue for a specific proposal for AI-assisted contribution.

Decide a separate issue was better since this issue has a lot of discussion of AI generated comments and other text, and is a broader focus than just the code contribution stuff.

https://www.drupal.org/project/governance/issues/3565917

Sharing what is not really a policy change, but an update of the language on the page that felt necessary to post as new people come in (especially with the new Google Summer of Code class)

AI-Generated Content

There is no doubt that artificial intelligence tools such as ChatGPT can be powerful ways to jumpstart code or content. However, AI systems still have significant flaws. Often times the code they produce is non-functional, and the content they create includes assertions or citations that are untrue.

AI contribution is currently allowed in the Drupal community, but there are very strict rules. If it's usage becomes too abusive or burdensome, this policy may change. When using AI in the course of making a contribution to Drupal, we require that:

1. You must fully understand the issue and the most recent comments before trying to use AI to solve it. If you don't understand it, you will probably give a bad prompt and wind up posting a bad solution that only annoys the maintainers.
2. You must disclose whenever you have used AI.
3. You must review and understand the output of the AI and test it yourself!
   • Please review this blog: https://dri.es/never-submit-code-you-do-not-understand
4. You must fix any problems with the AI-generated code before posting it to an issue.
5. Most important - you must be a good listener and collaborator with the project maintainer, original issue reporter, and other contributors on the issue. A 'drive-by' contribution where you don't follow up on feedback, or ignore previous discussion on the issue will likely result in an account ban.

Special note for programs like Google Summer of Code - GSOC is meant to be a learning experience. We would strongly recommend not using AI so that you learn the foundations you need. This will serve you  better later even if you do use AI, because you will understand how to prompt and interpret it.

We want to ensure that your contributions follow the best practices we have established as a community, so that you are building good relationships with the maintainers of the projects you are contributing to.

Warning Use of AI when it is not relevant to an issue, provides broken or unusable code, or provides false information—especially if done in bulk across many issues—WILL result in a ban.

I didn't know that we now have this policy.

I would propose that we add a point (phrasing to be improved)
3. You must ask for consent to use AI from a maintainer before posting AI generated content.

I think it would be great to have an opt-in instead of an opt-out of AI generated code and issue contributions. Some modules can declare on their module pages that gen AI is welcome, some may declare that it is not welcome, but I think the default should be to ask first.

A second point (and so that it can be replied to independently) that I think a lawyer should clarify is:
How well AI generated content (to which at least in the US you can not get a copyright for) is compatible with the GPL?
Can you contribute code to a GPL project for which you do not own the copyright?

I think before announcing that AI generated code is ok we should be sure that it actually is, or at least be cautious about it.

Worked with @bredeirt and @scottfalconer in the contrib room here at DrupalCon Chicago.

We have synthesized a lot of the discussion across the several different issues that are currently open, and made a further update to the basic policy that I had written previously.

Why am I just updating this directly?

While there are ongoing and important discussions, I'm also using my role to leverage a 'stop the bleeding' attitude towards these policies, to move quickly on policy adjustments that, while they won't solve everything, will help iteratively improve the situation for maintainers.

There are two or more axes of concerns: Pragmatic policies for usage requirements and restrictions, ethical policies, and perhaps pragmatic policies for improving the quality of output.

Here is the latest change, now moved to a dedicated page, and linked from the Issue Etiquette policy:
Issue etiquette: https://www.drupal.org/docs/develop/issues/issue-procedures-and-etiquett...
New dedicated page: https://www.drupal.org/docs/develop/issues/issue-procedures-and-etiquett...

Why this policy exists

AI tools make it easy to produce a lot of code and text very quickly. This creates pressure on the people who review and maintain Drupal. This policy is not about which specific tools you use. It is about making sure every contribution is something a real person stands behind. 

Most importantly—no matter whether you are using AI or not—you must be a good listener and collaborator with the project maintainer, original issue reporter, and other contributors on the issue. A 'drive-by' contribution where you don't follow up on feedback, or ignore previous discussion on the issue will likely result in an account ban.

The core principle: you are responsible for what you submit

Understanding an issue and collaboratively finding the right solution is a critical part of contributing. Writing the code is simply the execution of that solution, and it will only be successful when built on that solid foundation. AI tools do not change this. If a reviewer asks you to explain a decision or a piece of logic, you must be able to answer. Saying "the AI wrote it" is grounds for immediately closing the contribution.

You are fully responsible for the integrity of your submission. AI tools can hallucinate nonexistent software packages (risking supply chain attacks), introduce subtle security vulnerabilities, or introduces unhelpful refactors and new code that puts an undue burden on others to review. You must thoroughly verify all dependencies, logic, and security implications of AI-generated code before submitting.

Copyright and Licensing

AI models can occasionally output verbatim code from other copyrighted projects. You are solely responsible for ensuring that any AI-generated code you submit does not violate third-party copyrights and is fully compatible with the Drupal project's GPL license. Ignorance of the code's origin is not an excuse for licensing violations.

Examples of contributions that do not meet this standard

These are patterns that create unnecessary work for maintainers and reviewers:

• Dumping code into an issue without reading the thread or acknowledging previous attempts to solve the problem.

• Posting an Merge Request (MR) where automated checks fail, and leaving it for others to fix.

• Adding AI-generated code to someone else's existing MR without their knowledge and without disclosure.

• Using an AI to dump a large patch and then abandoning the issue when human feedback is requested.

• Submitting code that ignores the conclusions of prior architectural discussions.

• Proposing a full rewrite of a module based on an AI review, without first engaging the existing maintainers.

• Using AI to generate issue summaries, comments, or reviews that lack independently verified technical insights (e.g., using an AI to summarize a thread simply to gain contribution credits).

• Posting issue comments, MR descriptions, or forum posts that are unreviewed AI output, not your own words.

Disclosure

Transparency builds trust. If you use an AI tool to generate a significant portion of the code or text you are submitting, you must disclose it. You must disclose this use regardless of how thoroughly you reviewed the output.

What is "significant"? Generating entire functions, classes, architectural scaffolding, or extensive documentation blocks requires disclosure. Minor uses, such as standard single-line autocomplete suggestions or basic syntax corrections, do not require disclosure.

When disclosing, please use existing issue or Merge Request templates if they include a designated AI disclosure section. If no template is available, simply append a clear, human-written statement to the end of your issue summary, comment, or MR description. For example:

AI-Generated: Yes (Used GitHub Copilot to help generate the boilerplate for this feature).

Another example: AI was used in the drafting of this policy, to help review for clarity, clean up language, and check grammar.

Enforcement

Contributors who repeatedly violate this policy by submitting unexplained, untested, or disruptive AI-generated code will face consequences.

Our goal on Drupal.org is to educate first whenever possible. However, in some cases, violations may require a temporary ban so we can provide the necessary guidance and ensure it has been read and understood before restoring the account.

In other situations, when contributors show good intent and respond constructively to maintainers and the community, a temporary ban may not be necessary. In these cases, we will focus on education to help them align with community standards.

Finally, when there is clear disregard for these policies or disrespect toward maintainers or other contributors, a permanent ban may be issued.

We recognize that Drupal Association staff and Drupal.org site moderators have limited capacity to keep up with the volume of these contributions. We appreciate the community’s patience as we continue to scale our education and moderation efforts.

This Policy and the Drupal Terms of Service

This is a contribution policy. It defines expectations for contributor behavior and can be updated through the normal community governance process.

The Terms of Service (TOS), by contrast, is a legal framework that governs all use of Drupal.org and can only be changed by the Drupal Association.

The intent is to establish this policy first as part of our community norms and issue queue etiquette, and then reinforce it through updates to the Drupal.org Terms of Service.

One area the TOS will need to address more explicitly is the use of automated agents acting on behalf of contributors. As AI tools become more capable, clearer rules in this area will be necessary.

Frequently Asked Questions

For Google Summer of Code Contributors

GSOC is meant to be a learning experience. We would strongly recommend not using AI so that you learn the foundations you need. This will serve you  better later even if you do use AI, because you will understand how to prompt and interpret it.

We want to ensure that your contributions follow the best practices we have established as a community, so that you are building good relationships with the maintainers of the projects you are contributing to.

This looks good

It sounds like the latest version is saying this is not part of the Terms of Service for use of the Drupal Site, am I understanding that correctly ?

Thanks for this. I’ve read it and IMO it’s clear and provides the required guidance.

Regarding

“ perhaps pragmatic policies for improving the quality of output”

We’re trying to provide docs and resources for better AI outputs for contributors here:

https://www.drupal.org/project/ai_best_practices

And there is a new #ai-learners channel is Drupal Slack for people to share their best practices and learnings to improve the quality of contributions.