There’s been a lot of positive feedback and a few questions about the recent changes to the marketplace. I want to take a minute and talk about how the marketplace made it to where it is now and mention some plans for the future of the marketplace on Drupal.org.

TL;DR

This is a long post. You may not want to read the whole thing, so here is a condensed version.

We want to highlight individuals and organizations that are actively contributing to Drupal. You can now attribute your work in the Drupal.org issue queues to your employer or a customer. Maintainers can award issues credits to people who help resolve issues with code, comments, design, and more. These credits can roll up to service providers listed in the Drupal Marketplace. (And eventually we want to show customers with credits as well.)

Want to know all the details? Read on.

A little history

Back at DrupalCon Austin (2014), Dries presented a sketch of an idea for highlighting organizational contributions to Drupal to myself and several business owners in the Drupal community. Later that week, those initial ideas turned into A method for giving credit to organizations that contribute code to Open Source.

The idea was to use commit messages to highlight organizational contribution. The first draft looked something like this:

$ git commit -am "Issue #n by INDIVIDUAL@AGENCY*CUSTOMER: message."

It was simple and intended to be parsable from our Git logs. However, contribution is about more than just code—and while code contributions are a good place to start, we also needed to ensure that tools for providing attribution and credit would be extensible to non-code contributions as well.

That initial conversation turned into an issue that spawned lots of child issues and ideas for how best to collect data that allowed a user on Drupal.org to attribute their work to an organization.

The resulting conversations didn’t have a simple solution, but the participation in these issues was lively and full of great ideas. Determining the format for commit credit is still an active issue.

A commit credit format alone was not going to get the information onto Drupal.org. Eventually, we settled on a couple of different solutions for tracking organization involvement and pulled all of this together into the current state of the Drupal.org Marketplace.

Organizations that support modules, themes, and distributions

First, we added a feature to projects on Drupal.org (modules, themes, distributions) that allowed maintainers to reference organizations that were supporting a module—either by providing time to their developers to give back or by directly funding development.

Supporting organizations field

Commit credit user interface

The next part of the work was giving maintainers a better way to create their commit message related to a particular issue. A lot of the complexity of deciding who should get credit for work on Drupal.org was simplified into a UI that showed how many people had participated in an issue, how much each person had participated, and whether that person had uploaded files related to the issue. The UI also generated the Git command that would properly tie the resulting code changes to the issue and people involved. The result for a relatively large issue thread looks like the image below:

Commit credit UI

Attribution

While that effort made it a lot easier to create commit credits, it didn’t solve the problem of attributing issues that didn’t result in code change, nor did it allow people to give credit to the organizations that made it possible for them to work on that issue—by giving them time or by paying for the work directly as a customer.

To do that we had to figure out a way to add organization attribution to the work being done on an issue.

Issues and comments are the primary tools for tracking what needs to be built out in Drupal code. This is where we determine the features, report the bugs, plan out the work, and more. Tracking intent at the point of this interaction was the best way for us to cover the widest range of contribution to Drupal core, modules, themes and distributions.

The resulting UI for attributing a comment to an organization is simple and straightforward.

Attribution UI

While the instructions for attributing a contribution are documented, there’s still some confusion about when to attribute a comment to an organization versus a customer.

Attributions for organizations

These attributions are to allow a person to attribute their participation in a specific issue to a company that allotted time for them to work on the issue. This is typically an employer that pays that person's salary or wage. This organization must be directly tied to the contributor's user profile on Drupal.org as a current or past employer and the organization must have an organization profile on Drupal.org. (Create an organization profile).

When you use this feature, you are highlighting an organization that is contributing code—or other improvements through the issue queues—to the community.

Attributions for customers

When you attribute a customer, you are essentially saying that the work was requested by that customer but you are not employed by them. Typically this means one of three possibilities:

  • The customer organization paid the contributor's employer
  • The customer organization paid the contributor directly
  • The contributor was volunteering for the customer organization

You can select any organization as a customer that has an organization profile on Drupal.org.

When you use the customer attribution, you’re highlighting a different way of contributing to the Drupal ecosystem and often showing relationships between organizations to solve a problem.

Marking a comment with "I'm a volunteer"

Making it possible to attribute organizations who have supported contribution also helps us to gather data on purely altruistic contributions. We went round and round on this attribution feature, but in the end, we decided to add “I’m a volunteer” as an explicit attribution option in the attribution UI. It’s deliberately not the default. This is because we want to collect the explicit intent of the users making attributions. While we are now collecting this data, we haven’t yet figured out a great way to show this intent on a contributor's profile.

Possible intent on a comment could include:

  • I contributed on my own time as a volunteer and community member.
  • On this issue, some of my time was paid for by an employer, but some of my time was my own.
  • I contributed this work as a volunteer for one or more customer organizations.

The intent of volunteerism is complex, which is why it is hard to add up into neat little data to show on a profile.

Regardless whether they mark "I'm a volunteer" or not, contributors can still be awarded issue credit by a maintainer for their work on an issue.

Attribution versus issue credit versus commit credit, what's the difference?

Attribution is the first step of the process to award an issue credit. The contributor attributes their work to one or more organizations and/or as a volunteer. The next step doesn't come until the maintainer of the project (Drupal core, module, theme, distribution) gets involved in the process.

Issue credits

Project maintainers may grant issue credits for issues opened against their projects. At any time during the life of the issue, a maintainer can update the credit UI to credit any of the users who have participated in the issue. Any organizations or customers attributed by those users will be credited as well. When the issue is closed fixed, those assigned credits will appear on the appropriate user and organization profiles.

Issue credits are more comprehensive than just code. They can include reviews, designs, prototypes, or just really helpful comments. The maintainer choses what they want to credit even if the issue does not have code associated with it in the form of a patch. This is a great way to help highlight contributions to policy and work to keep Drupal.org running well.

Commit credits

Maintainers can grant commit credits. A commit credit gets added to the Git commit message that accompanies a commit by a maintainer of a project on Drupal.org (Drupal Core, modules, themes, distributions, etc.). Sites like drupalcores.com track this information in the public changelog and use it to highlight contribution numbers based on the number of times a user is mentioned in a commit message.

Commit credits currently can’t be easily tied to organizations, the parsing would be cumbersome and require a lot of precision when maintainers entered the information into Git, which is a big reason why we needed a UI on Drupal.org to help make all the connections and store that information into a database for easy retrieval.

Showing contributions on Drupal.org

Right now, there are three ways that contributions show up on Drupal.org: the user profile, the organization profile, and the marketplace. (We have some ideas to expand this that I will detail below.)

User profiles

Do you log in to Drupal.org? Great! You have a profile.

On every user profile, we show the total number of issues that a contributor has helped fix in the past 90 days.

You can drill into this information and see the specifics. This is a good way to see a facet of how a person is contributing to Drupal. There are other examples on a user's profile that show other ways they are involved.

Organization profiles

Any confirmed user can create an organization page on Drupal.org. Organization profiles are currently related to a single user, but we have plans to roll out additional permissions to relate multiple users to an organization.

On the organization profile, we show the number of issues that an organization has contributed to fixing in the last 90 days.

Organization profiles also show a list of modules that have been supported by that organization, whether that organization is a Drupal Association member or Supporting Partner, and additional self-reported information about their involvement in the Drupal community.

We are currently working to automate more methods for collecting data about how organizations are contributing to the community through DrupalCon and Drupal Camp sponsorships as well as hosting user groups to help support meetups, and giving to Global Training Day events. The number of case studies associated with an organization is another indicator of how much they use Drupal and how they are helping get others to use Drupal by sharing their success stories.

Marketplace

We recently rolled out some changes to how the Drupal.org Marketplace is ordered in order to better highlight contributing organizations.

Currently, the marketplace is sorted by number of issue credits awarded to an organization in the last 90 days, then whether the organization is a supporter, then by alphabetical order.

Every organization on the marketplace shows counts of its users on Drupal.org, projects (modules, themes, distributions, etc.) supported, issue credits in the last 90 days, and case studies. We also highlight organizations that are funding the ongoing support and development of Drupal.org by being in our supporting partner programs.

Questions we have heard

While most people have been excited about the changes to the Marketplace, there have been several questions.

Why is it only Drupal Service providers?

The marketplace was originally created to highlight Drupal service providers that are active in the Drupal community.

A "Contributing Organizations" list is the next step to expand the types of organizations we highlight. There are community members that are already working to highlight customers using Drupal that contribute back, community volunteers, and volunteers that are working for organizations that rely on volunteerism (many camps fall into this category).

These are great ideas. Contributing to those issues is a great way to help us figure out the best way forward.

What about "x" contribution type?

Yes, there are many ways that organizations contribute. We are definitely hoping to add in contributions that we can objectively verify. There are so many great things that organizations can do to support Drupal, both the local and worldwide community, and Drupal.org, including:

  • Sponsor a DrupalCon
  • Help plan a DrupalCon
  • Sponsor, organize, or host a Drupal Camp
  • Host a Global Training Days event
  • Host a local user group/meetup
  • Post a case study of a successful Drupal project
  • Contribute helpful content to Drupal Planet

There are many other phenomenal ways organizations give back. If you know of one that has objective data that can be added to the algorithm we use to highlight organizations, let us know by creating an issue in theDrupal.org Customizations queue.

Why is it so hard to create an organization profile?

We definitely want to improve this process. There are over 650 organization profiles on Drupal.org, so we know that people have figured it out, but it could be much more intuitive.

We have also had a several requests to add permissions to organization profiles so that multiple people can be editors of those profiles and help administer who is allowed to be associated with the organization. Our current method of tying users to organization by using an exact text match is a bit clunky. (If you want your organization to list you as an employee, you must exactly enter the title of the organization as it appears on the organization profile and list that organization as "current".)

What about more tools for individual contributions?

We have so many thoughts on how to improve the way we highlight individuals on their profile. We already show issue credits, documentation edits, commits, and lots of self-reported information. There is a lot of objective data to pull from including DrupalCon attendance, whether they have been a speaker at a DrupalCon, translation strings submitted to localize.drupal.org, Drupal Association membership, modules maintained, and many more.

Engagement with the community involves a lot of factors and we want to highlight as many as possible while still making the user profile useful and relatively succinct.

Help us make Drupal.org better

In addition to the questions that have come up above, we want your feedback to keep improving the user experience on Drupal.org and make it easier to hightlight the companies that are doing so much to make Drupal the leading platform for the web and the individuals that provide their expertise and commitment to the community. Drop your ideas into the Drupal.org Customizations queue.

Comments

nedjo’s picture

Thanks for the writeup.

In comments on the post last year announcing Marketplace Updates to Highlight Contributing Organizations, which asked for feedback, I and others noted specific ways that the new system is strongly biased towards promoting larger companies. These concerns were not acknowledged or addressed at the time; nor do I see them captured among the "Questions we have heard" here. Hence, I'm raising them again.

joshuami’s picture

Thank you for the direct question. It is an important topic.

We've notice quite a bit of movement on the marketplace pages from day to day and month to month. It's been fun to watch and very informative. The only companies that have consistently stayed at the top are Acquia, MD Systems, Chapter Three, and Tag1. (All of whom have extremely active individual contributors that are funded to do nearly full-time development of core or contrib modules.) Interestingly, the current order reflects shift for all but the top two companies and 7 companies in the top 25 with less than 20 users. (Including Chocolate Lily with only 2 users on Drupal.org—which is impressive @nedjo!.)

I agree with the basic premise that the marketplace has a bias towards companies with at least 20 or more users on Drupal.org that are heavily involved in the issue queues. I'm not sure that's a negative bias—it is certainly better than an alphabetical list in this case. Bias towards highlight organizations that give the most and contribute the most of their employees' time seems like a net positive.

The marketplace also represents the economic base for how Drupal is supported as an open source project—and to some degree how the Drupal Association and Drupal.org is funded. Any choices we make in how we sort the marketplace will result in a benefit to some companies over others. That's not a negative bias if it is aimed at sustaining the community.

nedjo’s picture

Thanks for your responses.

I get that the marketplace ranking changes are designed to be in line with the general advertising and promotion changes on drupal.org that give prominence to a set of larger companies that have a strong position in Drupal's structure and/or pay the most. But for me, those changes are antithetical to what I want to see for Drupal, and I don't feel I should be required to contribute to them in order to contribute to Drupal.

I've personally stopped recording the company I do work for, because I disagree with what's being done with these data. I can opt out in this limited way. But I'm also a maintainer of several modules.

So, very concretely, every day, when I review and apply patches, I'm faced with the choice of either not assigning credit or feeding data into a corporate advertising agenda that conflicts with the values that brought me to free software.

I contributed my first Drupal modules twelve years ago this month and since then have contributed code to over 100 Drupal projects. I've been a strong advocate of using our shared drupal.org infrastructure and over the years have helped contribute to our toolset, like co-writing the first prototypes of what became the Update Manager module and its server-side infrastructure.

I strongly support giving credit. I even support capturing information on what companies are sponsoring development.

But when a corporate advertising agenda starts to permeate into my daily volunteer work? No, that's not working for me.

Jaypan’s picture

And once again the Drupal Organization completely ignores forum contributors as contributing to the project. Not a mention of the forums in this entire post, and no recognition whatsoever to those who would use the forums to spread knowledge of Drupal, and create a community in which users can help each other.

Very disappointing.


Specializing in Drupal 8 upgrades. Checkout my Japan podcasts and my Drupal presentations on YouTube
AlexBorsody’s picture

Jay you are always on the forums providing support and flagging spam, I recognize you!

Jaypan’s picture

Thanks! And people saying thank you is why I keep doing it. But that's all the recognition I and other forum helpers get - the Drupal Organization doesn't consider the forums worthy of investment, nor that forum contributions are contributions to the committee.


Specializing in Drupal 8 upgrades. Checkout my Japan podcasts and my Drupal presentations on YouTube
hargobind’s picture

@Jaypan I too have run across your responses to forums and issues and want to give you some love. Thanks for all that you do! Even though we haven't had a lot of direct contact, I listed you as a mentor.

Jaypan’s picture

Thanks Hargobind! Much appreciated.


Specializing in Drupal 8 upgrades. Checkout my Japan podcasts and my Drupal presentations on YouTube
AlexBorsody’s picture

Wish more people were working on porting modules to 8 there of course it is early but I see many important projects have not even started where will be without contrib modules? This is critical.

gauravmishra7’s picture

I understand the ranking are purely decided on the basis of the issue credits and not the projects supported (modules contributed).

Any particular reason for that?

hestenet’s picture

Issue credits are also granted for contrib issues, so while the mere presence of the projects supported field is not presently factored into the credit algorithm, any issues fixed on those projects will count.

--
Tim L
Drupal Association - Director, Engineering

itapplication’s picture

Several developer invest there time and efforts for creating/developing and contributing lots of useful theme and modules. Drupal.org can't ignore to commits of contributed modules and themes. Please also add credits to module and theme project contributors.

Drupal developer

Drupal Theme developer.

Shivcharan

hestenet’s picture

Issue credits can be granted by the maintainer of ANY project when marking an issue Fixed. It is not limited to core.

--
Tim L
Drupal Association - Director, Engineering

itapplication’s picture

Yes, I got it. But why credit granted for fixing issues only. Why not for other activities? It will be better if user can EARN CREDITS from commits to the new project like New contributed theme or module. Every commits for the git repository of project also need to earn credits.

Drupal developer

Drupal Theme developer.

Shivcharan

hestenet’s picture

Ultimately we do want to update the algiorithm for credit to include many other types of contribution - here is the meta issue: https://www.drupal.org/node/2649100

--
Tim L
Drupal Association - Director, Engineering

gauravmishra7’s picture

Tim- if you review the top10 companies credits, you'd realize how this is getting gamed. I am sure DA would be aware of same.

hestenet’s picture

Thanks for your concern, Gaurav. It is definitely important that we get this right. We are continuing to monitor how the credit system is being used, and have made some small adjustments recently (within the past 90 days, so their effect hasn't been fully reflected in the marketplace results quite yet).

There are a few more adjustments we'd like to make.

Please do feel free to report any specific situations that you see as gaming the system directly to me, via my contact form: https://www.drupal.org/user/54034/contact

You can also open an issue in the drupal.org customizations queue if you are able to describe a generic, technical issue with the credits calculation. Please don't call out any particular organizations in a public forum, of course.
(https://www.drupal.org/project/issues/drupalorg)

Adjustments will continue to be gradual - to avoid too much disruption.

--
Tim L
Drupal Association - Director, Engineering

ChandeepKhosa’s picture

When you say gamed, what do you mean?

nedjo’s picture

By tying issue credits to company rankings, the changes introduced created a market in issue credits.

Predictably, many companies have responded by fishing for issue credits. Common techniques involve combing through projects for opportunities to earn credits with minimal effort, often using automated code scanning. By now many or most project maintainers have had to deal with the result: trivial patches that take maintainers' time while adding little if any value, posted en masse by contributors who have never used the projects in question and are unlikely to go on to contribute substantively to them.

The root problem is not however in these companies. It's in a system that's designed to commercialize contribution.

hestenet’s picture

Thanks for your perspective, nedjo. We certainly are trying to create *incentives* for contribution - and I believe we have seen an uptick in quality contribution - but the potential for poor quality fly-by contributions certainly is a concern. That is partly why the actual credit is only granted if the maintainer chooses to credit those contributions - though you are correct that this creates work for maintainers. 

In addition to incentivizing new contribution, though, we are also recognizing contributions that were already being made but that were previously fairly invisible.

It is a tough problem though - I'd love to hear your ideas on preserving the recognition and incentives that the system provides, while helping to control the extra work created for maintainers reviewing these contributions.

As a sidenote - there are some additional changes to the marketplace ranking upcoming:
https://www.drupal.org/node/2833508
https://www.drupal.org/node/2649100

--
Tim L
Drupal Association - Director, Engineering

nedjo’s picture

It's easy to do a quick scan of a module and find one or two nitpicks but harder to dig in and make substantive contributions. So an indicator of companies making the minor investment needed to accumulate easy issue credits would be many projects listed for the given company but generally few (say, three or fewer) credits in any given project. Going through the "top" companies listed, it's not hard to find listings that match this profile.

So, yes, the sorting algorithm could be tweaked to, for example, assign greater weight when more issues are contributed to per project, suggesting more meaningful contributions. Then again, I previously suggested a very simple algorithm improvement that could address skewing of results to larger companies, and heard that the skewing was by design.

Overall what seems to be missing from the conversation is: who are the users we're trying to serve?

As I and others pointed out when this system was proposed, the solution is heavily weighted towards larger companies. Now, when someone browses the marketplace listings, as well as answering the question "what are the largest companies?", there's apparently a second factor, "what are the companies that invest the most in accumulating issue credits (often through trivial nitpicks)?" And, I suppose, "what are the companies that provide, or claim to provide, the most diverse set of services to the most diverse set of sectors?"

I see how this system serves the interests of a small set of service providing companies, which tend to show up no matter what filters are used. I also see how the Drupal Association has decided to use the rankings as a lever to promote other objectives. But what about organizations looking for service providers? What are they looking for? What are their diverse needs? How could those needs best be served?

Better than tweaking ranking criteria to meet the current aims, energy would better be spent addressing those questions as a starting point.

hestenet’s picture

Thank you for the very thoughtful comments, nedjo. It's greatly appreciated. It certainly gives us food for thought as we think about how to continue to iterate. 

--
Tim L
Drupal Association - Director, Engineering

nedjo’s picture

Thanks for the links to relevant issues. Here is another one that I filed four years ago: Limit terms selected on marketplace nodes.