By Drupal Association on

by Drupal Association board member Mike Herchel

Drupal's contribution credit system continues to be unique in open source, in that it provides an attribution and incentive system to encourage greater contribution to the Drupal project, both from individuals and organizations.

Recently, there’s been discussion within the Drupal community on what makes a 'high' vs 'low-value' contribution. There's a perception from some community members that some contributors may be using low effort, low value contributions to gain a more favorable placement within the Drupal Marketplace. Some examples of low-value contributions that people have given include posting unnecessary screenshots to issues, or running automated tooling against many projects to fix minor code quality issues.

As we evaluate these concerns, it's important to remember that contribution recognition is not a zero-sum game. There is no 'winning' contribution.

We very much want many people to be recognized for their contributions, and we want to see new faces in the issue queues. Some contributions which may seem simplistic or low-value may also just be good entry points for someone first beginning a contribution journey, and we should always use these examples as an opportunity to help an individual or organization 'level-up' their contribution skills.

At the same time though, we want recognition to be proportional to the effort put in, and we want our project maintainers, who ultimately control who gets credit for contributions to their projects, to feel encouraged by seeing new faces and not burnt out by policing the system. 

How did we get here

The Drupal Marketplace is intended to showcase organizations that contribute back to Drupal. This includes not only code, but non-code contributions such as testing functionality, event organizing, speaking, volunteering, and more. 

The more work an organization contributes, the higher they will be ranked within the marketplace. This, in turn, leads to more job opportunities and leads.

Inevitably, contributors will try to maximize their contributions to gain a higher ranking. Is that okay? It certainly can be okay - this incentive exists because we want to further encourage contribution, but those contributions need to be authentic, and we hope to see new contributors develop their skills and increase the scope of their contributions over time. 

Today, we are seeing a recent pattern with providing a lot of test screenshots. These can be valuable, but in some cases the users posting them aren't even checking if they have a properly applied patch, so it's difficult to rely on. This is partly because file attachments automatically pre-check the contribution box when users post them to issues, so that's an area where we can use a technical solution to try and correct the situation. 

A quick fix

The Drupal Association Engineering Team is making changes to the issue queues so that contributors that upload images will no longer be automatically assigned credit. By not having this auto-populate, the maintainer won't have to spend time deselecting drive-by contributions of screenshots that were inauthentically posted or otherwise unhelpful. 

Are we running into Goodhart's law?

Goodhart's law is an adage often stated as, "When a measure becomes a target, it ceases to be a good measure". Often invoked when discussing economics, it’s directly related to the issue at hand. Any deliberately designed incentive structure is going to result in people figuring out ways to maximize their results - that's only to be expected. 

But our firm belief is that with careful management, the system can still provide a good measure of contribution. Contribution credit is central to the Drupal ecosystem to help motivate contributors. 

How can we encourage high value contribution?

It's difficult, if not impossible, to evaluate the quality of a contribution algorithmically, although there are some heuristics we can use. It is impossible to algorithmically determine the human intent behind a contribution: was it an authentic first effort, or was it a cynical reach for low-hanging fruit.

We always want to assume good intent, and to use any opportunity we can as a teaching moment. But with collaboration across so many projects and so many people, we do need community standards and guidance to help. 

The first step is to define what is high-value and what is low-value (or perhaps even to decide if these are the right terms!). The Drupal Association is putting together a small group composed of community members, DA board, and staff to create some documentation for exactly this. We have existing documentation that gives guidance on granting credit, but none that is oriented toward the contributor. (Editor's note: The Drupal Core team also has documented standards for how they grant credit). 

The goal of this documentation is to guide contributors on how they can help, and how to avoid being unhelpful through well thought out examples, and processes. This documentation can then be linked to within Drupal issues when a maintainer feels that a contribution was unhelpful. 

Our hope is that reading the documentation will be enough for any new contributors to realize when they are posting unnecessary and unhelpful contributions, and give them a clear pathway to making contributions that have a greater impact. 

Next steps

The Drupal Association is committed to making contributions as fair and equitable as possible. If you’d like to participate in a discussion to create documentation, please reach out by October 1st.