This issue creates an extensive list of contribution types that we either track or could track. It should be noted that this is not comprehensive. There are forms of contribution that do not leave behind data that cannot be collected. That said, this list does cover lots of types of contribution and not just code.

If we were to apply some math to this data, we could calculate an engagement score for users and organizations. Tracking engagement over time could be a key health metric.

Many of these contribution types would make profiles and organization pages more robust and would be less subjective than self-reported information.

Contribution Type User Org Status Notes
Issue credits X X Displayed on profile Only last 90 days are displayed, but all are tracked. Awarded by project maintainer
Commits X Displayed on profile Git records
Releases created
Projects owned X Not displayed By author field (can change and could be broken down by full versus sandbox)
Projects supported X Displayed on profile Ref on org profile
Change notices (project participation) X Not displayed Technically, we might show this as a post that the user created
Projects maintained X Not displayed We would need to decide how many maintainer types/roles to display
Documentation revisions X Displayed on profile We show this listed as an aggregate. (i.e. "more than 100 document edits") Cannot credit to orgs
Documentation section maintained X Not yet tracked This will be possible with new documentation tools defined in the content strategy.
Forum posts created X Not displayed Technically, you can see these as posts by the user, but it is not a total and not limited to forum posts.
Forum comments X Not displayed Technically, you can see these as comments by the user, but it is not a total and not limited to forum comments.
DrupalCons sponsored X Not displayed We have this for most past DrupalCons.
Supporting partner status X Displayed on profile Supporting partners fund Drupal.org—both infrastructure and staffing costs.
Membership status X X Displayed on profile Individual or organization membership
Donations and support to the Drupal Association X X Not tracked We have totals of money spent with the Drupal Association, but we'd need to aggregate this information and likely abstract it to levels.
Jobs posted X X Not displayed Technically, we could show this for users, but I'm not sure there is value.
Issues created X Displayed Kinda. This is really "posts" of all types on the user profile.
Issue comments X Displayed Kinda. This is really "comments" of all types on the user profile.
Issues followed X Not displayed This seems like an excellent engagement metric.
Projects followed X Not displayed Also an excellent engagement metric.
User/interest groups owned/maintained X Not displayed From Groups.drupal.org data
User/interest group membership X Not displayed From Groups.drupal.org data
User/interest group participation X No displayed By count of nodes created or comments posted
Local user groups sponsored X Not tracked Great organization metric
Camps sponsored X Not tracked Camps would need to help us track this and submit their data to staff for upload.
DrupalCon session submissions X Not displayed We have this data for quite a few past DrupalCons.
DrupalCon speaker X Not displayed We have this data for quite a few past DrupalCons.
DrupalCon Trainer X X Not displayed We have this data for quite a few past DrupalCons.
BoF sessions submitted X Not displayed We have this data for quite a few past DrupalCons.
Translations strings submitted X Not displayed From localize.drupal.org. It would be great to figure out how to attribute this work back to organizations for organization credit.
Translations strings accepted X Not displayed From localize.drupal.org. It would be great to figure out how to attribute this work back to organizations for organization credit.
Translations strings reviewed X Not displayed From localize.drupal.org. It would be great to figure out how to attribute this work back to organizations for organization credit.
Core patches submitted X Not displayed This could potentially be pull requests or merge requests with new tools
Contrib patches submitted X Not displayed This could be pull requests or merge requests with new tools
Case studies submitted X X Displayed Partially displayed. This shows up on the marketplace listings, but not on the profile.
Employees on Drupal.org X Displayed on profile This display could use some work. Acquia's page takes about 10 seconds to load because the list of users is so long.
Drupal Planet Posts X X Not tracked This could be tied back to users with a little customization
Camps organized X Not tracked We might be able to pull this from the groups data with some modifications to the roles/management types.
User/interest groups organized X Not tracked We might be able to pull this from the groups data going forward
# of mentees listing as a mentor X Not displayed This is similar to the concept of "followers", but people community focused.
Is board member X Not tracked We could put this on profiles
Core maintainer X X Not tracked Orgs it could be "hires a core maintain"
Sprint mentor X Not tracked Wouldn't this be awesome? We'd have to decide what sprints qualify—just DrupalCons or all big sprints at camps and Dev Days.
Staff X Not tracked Hey, we know all our staff, but most people don't know if they are talking to a full time Drupal Association employee.
Elevated role on Drupal.org X Not displayed We've given special privileges to these people. It would be nice to highlight that.
Working group member X Not tracked This would help people understand governance roles a bit better.
Security team member X Not tracked We list all of these users, but it would be nice to show it as a badge on their profile pic for interactions in the issue queues.
DrupalCon volunteer X Not tracked We have the data in spreadsheets and thank you notes. It really should show up on profiles.
DrupalCon attendee X X Not displayed Technically, people self-report this information right now, which means it is not accurate. Org would be a little difficult to track on this one, but it could possibly be done
Global Training Days organizer X X Not tracked We'd have to import this data.
IRC support X Not tracked This would require people to give credit to people with IRC handles listed on Drupal.org. Maybe we could use the Karma points from IRC.
Report spam X Not tracked We know when people flag spam. This happens a lot, so figuring out how to give credit to that is going to be a challenge.
Reviews project applications X Not tracked That might speed up this process a bit, but there is other work under way.
Provides support on StackExchange (Drupal Answers) X Not tracked They have an API we could query.

Next steps

Update this issue with related issues as that work is planned.

Files: 

Comments

joshuami created an issue. See original summary.

YesCT’s picture

Thanks for making this meta issue. Is very helpful overview.

Some of these have issues linked from the parent profile meta I think.

"Issues followed" I'm guessing that would really be "Issues following" ? Or do you really mean Issues followed and unfollowed would still count as issues followed?

joshuami’s picture

I meant issues following, but the grammar sounded funny when I wrote it that way :)

As for the related issues, as you see them, please feel free to add. I'll go through and try to match up as many as I can later today.

joshuami’s picture

joshuami’s picture

joshuami’s picture

Issue summary: View changes

It should be noted that this is not comprehensive. There are forms of contribution that do not leave behind data that cannot be collected. That said, this list does cover lots of types of contribution and not just code.

hussainweb’s picture

I think it's a great list. I am especially interested in these:
- Patches submitted (core/contrib) - Undecided, I am leaning towards the idea but it is easy to see this bloat.
- Camp sponsorship - Great!
- Camp organizer - Great!

webchick’s picture

One thing that'd be really cool to see not only tracked but given a big boost in the marketplace scoring for the next ~6-9 months to encourage activity is "port contrib projects to D8."

If we had that deal where we could manually add people who have not commented on an issue to the issue credit, it would be easy-peasy. Just a straight bump in issue credit amount for any contrib_tracker issue marked fixed.

In lieu of that, maybe a boost for commits made to an 8.x branch of a module/theme/distro.

drozas’s picture

This is great news! :-)

I collected a list of contribution activities as part of the first stage of my PhD research from several data sources: documentary analysis from Drupal Planet (http://www.davidrozas.com/lab/drupal_planet_archive.php), semi-structured qualitative interviews and full field notes from participant-observation. You can find the list in the working paper here: http://cress.soc.surrey.ac.uk/web/sites/default/files/publications/worki... Of course, the scope/aim was quite different.

I have been going through the list and I think everything is mostly included, but I thought this might be helpful!

joshuami’s picture

Issue summary: View changes

@drozas, I was hoping you would comment on this one given your keynote at DrupalCon Barcelona.

Thank you for sharing the link to your paper. Great work. I added three more contribution types (that we could track) from your paper.

@webchick, ++ the manual add of issue credits. That enables crediting a lot of different types of work that we don't see as a comment on the issue right now.

I like the port contrib to Drupal 8 bump. I could also see giving a bump to organizations that submit Drupal 8 case studies. (We need those early adopters.)

anish_zyxware’s picture

Regarding issue patches, there could have been priority.

For example, those who port an entire module to drupal 8 and those who fix a single typo gets the same credit since it is a single issue. If we can rate the level of contribution, it would be great.

Also, +1 on localization submissions. My contribution to most free software projects I have contributed is localization. In my local language, I am the one who contributed most to Drupal. There should be way to highlight that in an individual as well as organization level.

A point system does make more sense. For example,

* A translation submission can be one point.
* A trivial issue can be 10 points.
* Porting of module, say 200 points.
* Drupal core can be given more weightage.

Like that. Just a suggestion

drozas’s picture

Thanks @joshuami! Let me know if I can give any other help (I couldn't think of anything else at least for the moment ;-)

Shyamala’s picture

This is an awesome initiative @joshuami. @drozas, great work - enjoyed your observations!

Listed out more contributions that are non code-centric/object-oriented, but more community-oriented:

  1. Being a Project Manager
  2. Being an Initiative Lead
  3. Camp/Sprint Organizers
  4. Drupal Courses in Colleges - as a Community Activity

Would be a great value to rope in more of such contributions. An option could be to have a Contributions project where users can post their contributions that don't fall under categories defined, then have those validated by other Contributors to actually acquire the credit.

Recommend Contributions stats to have 2 scores
- one a recent score that indicates participation around last year
- second an overall participation score
This will ensure continuous participation.

Stimulate participation of Users and Company with a mailer system that alerts them. Some examples could be:
- Congratulating participation - informing where they stands
- Knock on them when they haven't been around with information around activities that would of interest
- Let them know what other's in their organization, region are doing
We can be creative with the scenarios, make the Frequency and content non intrusive.

kattekrab’s picture

There's a huge amount of contribution that goes on in local communities that doesn't touch d.o.

Groups that use meetup.com and not GDO
and DrupalCamps around the world.

People involved in organising those camps the DA supports directly should be easy to credit in some way.

But then there's a challenge for the others.

Acknowledging events logged on G.D.O might help here?

EG - all the D8 launch parties needed to register on G.D.O to get included on the map... perhaps there's something there.

hussainweb’s picture

@kattekrab: That's a fair assumption. We are careful in cross-posting any event (meetup, sprint, camp) to g.d.o at least and then point to whichever page where we are collecting signups (usually meetup, but sometimes our own website for camps). Earlier, there was no incentive other than hoping that someone who is not yet on meetup would notice and signup. The recent promotions on map were further incentives and I think most places would now be used to cross-posting.

Now, if we actually credit events organized on g.d.o, there would be a further push to cross-post. I don't think there is any resistance on using g.d.o for this.

davidhernandez’s picture

Yeah, more rep for organizing events would be nice. Camp/meetup/sprint organizers and volunteers. Also, hosts. We are giving credit to companies that support dev time now. How about those that provide their space for events? That is a big help facilitating sprints and meetups.

joshuami’s picture

Tracking contribution at camps could be done through issue credit attribution—assuming issues were associated with the work being done at the camp. We'd have to figure out whether we wanted camps added using the existing organization type or if we needed something more specific to camps. This could also be extended to local user groups.

There is a lot of new functionality we'd need to figure out. For example, you can attribute your work in issues to customers, but we don't really display this anywhere except on the organization page. The value of attributing something to a camp would be directly impacted by whether or not you could see that camp listed anyplace. We have talked about extending where organizations are listed to a comprehensive list of "organizations that contribute" in addition to the Marketplace. (The Marketplace is currently limited to organizations that provide Drupal services in some form.)

In a couple of the responses above, there was mention of recent versus all time tracking and a scaled point system based on relative value. We've talked about both. Recent versus all time would require the work to backport commit credits to get decent "all time" totals. Those would be biased towards individual users and code commits. A point system based on relative value has also been talked about. I'm in agreement that something like translation strings would get less total points per instance than something like speaking at a DrupalCon. We did some work on this about a year ago when we were working on user role progression. I'm attaching the image. It was all very rough, so please don't judge to harshly. I'm not sure about displaying a user engagement metric like this. (It feels a lot like a certified to rock score—which some people loved and others hated.)

Possible user engagement metrics

These ideas are all great. Keep them coming. We may not be able to track or display credits for them all, but its an important discussion.

joshuami’s picture

Oh... one more thought I wanted to get down on paper.

For a lot of these metrics, we cannot rely on self-reporting. If it is not a self-reported credit, who reports the credit? One of the reasons that issue credits work so well is that maintainers award the credit as part of the issue workflow. For any credit that we track, we probably need to figure out the system for awarding those credits that creates an objective audit-able metric. Some of these will be way easier than others.

That doesn't mean a particular contribution type isn't valuable, but we might not have a way to assign that value in a way that is meaningful and can be displayed.

davidhernandez’s picture

We do have group moderators for the regional groups on gdo, so maybe it is up to them to control who is added as part of an official organizing committee for an event? That isn't as tightly controlled, but no less so than someone in contrib giving extra points to their friends in commit credits.

drozas’s picture

This article by Jimmy Tidey, discussing this issue happening on makers communities as well, might be helpful to reflect collectively on this: http://jimmytidey.co.uk/blog/affective-labour-and-digital-collaboration/

I have found this paragraph on "how to keep the balance" especially relevant:

Perhaps collaborative websites should start measuring affective labour and making it more visible. How many meetups did someone organise, how much mentoring have they done? How often do they resolve a dispute? Could that be presented in the same way that github uses stars? For example, the Drupal community has started to discuss and analyse the ways in which these less visible activities can be tracked and recognised. At the same time, efforts to value affective labour needs to avoid being reductionist – ultimately it’s not about measurement, but the experience of the community members.

Hope this can be useful!

andypost’s picture

I can't find a doc page and way to report issues with statistics calculation.
Would be great to allow users report problems so calculations will get feedback

joshuami’s picture

@andypost, please report the credit calculation error you found to the Drupal.org customizations queue: https://www.drupal.org/node/add/project-issue/drupalorg.

kattekrab’s picture

@joshuami - I just want to clarify something around contributing to camps.

You said

Tracking contribution at camps could be done through issue credit attribution—assuming issues were associated with the work being done at the camp.

But what I meant, was tracking the contribution around ORGANISING camps, not the commits, reviews, comments, etc made on D.O during camps.

Organisers, volunteers, speakers, mentors... this is contribution too - but we currently don't have any visibility into that. And the data simply doesn't exist on D.O.

Having some kind of form on D.O for camp organisers to report back, perhaps with a user reference field (like we do for mentors) to list team members and volunteers... and some other camp metrics around attendance, sponsors, etc? Perhaps using G.D.O's events? Or at least something to brainstorm about.

Note: We SHOULD be able to recognise SPEAKERS at DrupalCons though. Right?

joshuami’s picture

@kattekrab, thank you for asking the clarifying question. In the table provided in the issue summary, we tried to call out camp organization specifically.

Camps organized X Not tracked We might be able to pull this from the groups data with some modifications to the roles/management types.

As we move through the D.o content restructure work, camps could be a type of local interest group and organizers of those groups will have special roles that we can call out. So eventually, I think we'll have the data for that sort of contribution going forward. That might not be the best way to capture it historically, but we might have to draw a line at how much history of contribution we choose to document.

DrupalCon speaker X Not displayed We have this data for quite a few past DrupalCons.

DrupalCon speakers is data that we have historically and would like to pull in to user profiles.

kattekrab’s picture

Thanks @joshuami - that's useful!

And apologies, I didn't go back and review the original list.

MatthewS’s picture

Agreed, tracking contributions around organizing camps would be terrific. For example, I have a core group of people that meet at least once a month and put a ton of time into making the Drupalcamp Colorado event professional and excellent.

itapplication’s picture

We need to move forward.

At least we need to define priority and move forward by applying/updating one by one 'Contribution Type' into the algorithm for CREDIT system.

ChandeepKhosa’s picture

Is there anything I can do to help make these changes a reality? I'm happy to organise surveys & interviews with people if that's useful, also writing blog posts to help spread the word. Happy to help!

ChandeepKhosa’s picture

I'm happy to assist in creating child issues to help break this large task into manageable pieces if anyone thinks that would be useful

xjm’s picture

mlhess’s picture

Issue tags: -d.o crediting contributions +d.o crediting contributions MWDS2016
mlhess’s picture

Issue tags: -d.o crediting contributions MWDS2016 +d.o crediting contributions, +MWDS2016
hestenet’s picture

Issue summary: View changes

Adding the suggestion to track releases created from @colan, per feedback on our recent blog post.

wturrell’s picture

Came to this via Hestenet's blog – last night I was watching the DrupalCon LA 2015 talk How do we encourage repeat contributions to Drupal Core?. Some there, such as YesCT, were saying a major bottleneck was the wait for patches to be reviewed (RTBC) and that a successful metric for sprints should not only be how many patches were created but how many issues reviewed/triaged/moved along - the current core Needs Review queue suggests it's still a major concern.

It doesn't seem to me (albeit as a newcomer) there's much problem with enthusiasm for contributing to core, or getting people to open issues, report bugs, have a go at writing a patch etc., the delay is at the other end because reviewing is (imho) REALLY hard, especially if you're trying to assimilate an unfamiliar issue which has spanned several years. Very satisfying from what little I've been trying of it, from a problem solving perspective, but hard and tiring all the same. Also, many probably don't even know just how small the group of core committers is? Is there a policy of mentioning the bottlenecks / human side of this at sprints?

Anyway, I'd argue RTBC maybe deserves it's own row in the table (bit surprised no-one's specifically mentioned it here or in the blog comments.)

Wim Leers’s picture

@wturrell: interesting perspective, thanks a lot for posting that here :)

Currently, reviewers are treated exactly the same as those who roll the patch. They both get commit & issue credit. It may make sense to split up those two (writing vs reviewing code).

But I don't think it makes sense to call out RTBC specifically. In the vast majority of Drupal core issues that I've worked on, multiple people are reviewing, and it's just that when all feedback has been addressed and there is consensus among everyone that that last patch is the right one, then it's marked RTBC. So, RTBC'ing is not a solitary thing, it's a collaborative thing.

davidhernandez’s picture

Maybe that can be a checkbox in the "Crediting $ committing" section of the issue, to identify those that contributed to reviewing,