In the summer, we organized the very first Drupal Governance Sprint. We sat down and discussed how to evolve Drupal's governance structure to support the Drupal community's continued growth. The result of that meeting was a proposal on how to evolve our governance. After discussing at length and formally chartering the first of these groups, the "Community Working Group" (CWG), we'd like to move on to Drupal.org governance.

The first of these groups is the Drupal.org Infrastructure Working Group (DIWG). Mainly codifying what happens within the existing Drupal.org infrastructure team, the mission of the Drupal.org Infrastructure Working Group is to keep Drupal.org's hardware and underlying software running quickly and stably, by working with the Drupal Association Board of Directors in order to scale up along with our community's growth.

The charter is maintained in Git (http://drupalcode.org/project/governance.git/blob_plain/refs/heads/maste...) but a full copy is provided below.

Drupal.org Infrastructure Working Group Charter

Mission

The mission of the Drupal.org Infrastructure Working Group (DIWG) is to ensure that all of the Drupal.org websites and supporting infrastructure remain stable, performant, and able to scale with the growth and needs of our community.

The DIWG acts as a group to make decisions surrounding the web infrastructure aspects of the Drupal.org websites. They also collaborate and provide support to the other working groups.

Scope / duties / responsibilities

Scope

The primary goal of the DIWG is to keep all the Drupal.org websites running smoothly, and to work closely with the other Drupal.org working groups to make infrastructure-level decisions. They also provide recommendations to the Drupal Association for infrastructure-related budgets and needs.

The DIWG manages and is responsible for all infrastructure-related needs of the Drupal project. This includes servers, Git repositories, mailing lists, DNS management, e-mail management, network, server access, and security. They also handle capacity planning for future growth needs of these resources.

Specific duties

  1. Infrastructure planning: capacity planning, system requirements, timeline, budgets, etc. in collaboration with the Drupal Association Board of Directors and other working groups.
  2. Hardware management: Manage the hardware that underlies the Drupal.org websites (web servers, database servers, mail servers, load balancers, network equipment, development environments, testbots, etc.).
  3. Software management: Manage the software that underlies the Drupal.org websites (e.g. operating system, mailing list software, HTTP server, database software, FTP services, source code management systems, continuous integration tools, backups, etc.).
  4. Providers: Manage the relationship with external infrastructure vendors/providers: e.g. hosting companies, IRC providers, etc.
  5. Processes and documentation: Maintains the documents and processes around the hardware and software they support (e.g. security policy, inventory of all hardware and software), deployment processes.

Exclusions

Items specifically not within the scope of the DIWG's duties:

  • The DIWG cannot make project-wide technical policy decisions (this is the responsibility of the Technical Working Group) nor Drupal.org software decisions (this is done in coordination with the Drupal.org Software Working Group).
  • The DIWG cannot change or extend its own charter.

Process

Budget

The DIWG works within the budget provided by the Drupal Association, and presents a yearly plan with cost breakdown for Board approval.

Transparency and Appeals

The DIWG aims to be as transparent as possible by documenting its decisions publicly.

Individuals who do not agree with a given DIWG decision may escalate to the Drupal Association Board and/or their designate(s), who will review the decision and can choose to either uphold or alter it. In the meantime, the decision of the DIWG stands.

Membership

The current members are (in no particular order):

  • TODO

Members are appointed by the Drupal Association Board and/or their designate(s).

Charter revisions

The charter will be revised as needed. Any proposed charter revisions must be ratified by the Drupal Association Board and/or their designate(s) prior to acceptance into this charter.

Comments

LeeHunter’s picture

I have added this issue which is relevant to all the proposed working group charters #1934018: Formalize stakeholder roles in working group charters.

killes@www.drop.org’s picture

I am unsure if the following two parts are compatible:

Software management: Manage the software that underlies the Drupal.org websites (e.g. operating system, mailing list software, HTTP server, database software, FTP services, source code management systems, continuous integration tools, backups, etc.).

The DIWG cannot make [...] Drupal.org software decisions (this is the responsibility of the Drupal.org Software Working Group).

As we currently handle it, we get approached by people who want to have new software or change existing software, for example introduce a popular view, and then we discuss whether the hardware can actually support this feature, and how the feature should be changed to make it bettter working. In extreme cases this can lead to something not being deployed.

webchick’s picture

Why is the discussion not in the opposite direction? if feature X is deemed strategically important for the contributor community/end users of Drupal, "What extra hardware do we need to support this?"

catch’s picture

Refactoring code to be more performant/scalable is a one-time cost, adding extra hardware means additional cost for servers and time for maintaining them. If the performance issue is against the database it's very rare once you get to Drupal.org's size that adding hardware is going to help much, if at all.

killes@www.drop.org’s picture

Yeah, what catch says is pretty accurate. Also, what is "strategically important"?

IMO, anybody who wants to add a feature, needs to also show up that it is viable to run that feature. That is, they need to provide realistic test data or at least realistic estimates.

webchick’s picture

I'm never going to argue with finding more performant ways to implement something, but I'm talking about something slightly different.

In the future, the plan is that not just "anyone" will be adding features to the site willy-nilly, but instead feature additions will be vetted by the DSWG (in concert with the DIWG + DCWG) and put into a roadmap with both short-term and long-term goals. This ensures that plans are approved up-front ahead of time for a) sanity, b) feasibility, so that volunteers don't sink a year+ into implementing something and have it sitting there, rotting.

That issue, btw, is an example of something I'd call "strategic," as is something like #50605: Add user ratings for projects. Both are addressing a long-standing painpoints around the very thing that makes Drupal so powerful: its extensibility. We have numerous usability studies showing that people can't figure out they can extend Drupal in the first place, let alone how to install modules, because being taken to Drupal.org is an incredibly jarring and overwhelming experience for them. Then once there, they are overwhelmed by the choices and lack tools for making decisions on module selection. These are both fundamentally enormous adoption issues (not to mention the cause of policies that actively discourage contributors) that are holding Drupal back from its full potential.

So I could see both of those issues ending up on a feature roadmap for Drupal.org from the DSWG. I would expect the DIWG to say "Cool, here's how we could do that, but be careful of X and we'll need to do a final check off of the code for performance and security." Not "No." or "*crickets*" or "Sure!" by one person but then "No." by another person. All of which seems to be the current workflow right now, and has killed all momentum we ever built around volunteer contributors on Drupal.org.

webchick’s picture

Also, "That is, they need to provide realistic test data or at least realistic estimates." is a fine thing to put into a policy document explaining what the approval process is for new feature deployments. Ideally, we'd provide enough documentation so that people could just do the baseline performance measurements themselves, and the DIWG meetings involve just looking over the data and say "Cool, go for it" or "Nope, still needs some more work." Even better if there were incremental checks along the way so we don't have months of work and then shooting down at the end, like often has happened in the past.

Crell’s picture

Minor nitpick: We shouldn't mention "LAMP stack" in the scope, but something more generic like "Web server infrastructure". There should be no reason that the DIWG couldn't decide to move to FreeBSD/nginx if they felt that was the right decision to make.

I'm not really involved in the infra side of things at this point, but if I understand the current setup and the proposed setup correctly, it basically boils down to *reducing* the number of people who can say "Go for it" or "Nope" and have it carry weight, and setting it up so that they talk to each other before saying that collectively. That is, make the "Yep"/"Nope" answer more meaningful and easier to achieve one way or the other. True?

Also, this specifically says Drupal.org. It doesn't say *.Drupal.org. I'm assuming the latter is the intention, but that should be stated explicitly.

webchick’s picture

I'm not really involved in the infra side of things at this point, but if I understand the current setup and the proposed setup correctly, it basically boils down to *reducing* the number of people who can say "Go for it" or "Nope" and have it carry weight, and setting it up so that they talk to each other before saying that collectively. That is, make the "Yep"/"Nope" answer more meaningful and easier to achieve one way or the other. True?

Basically, yes. And also making it more explicit to volunteers who among the dozens of *.Drupal.org volunteers are empowered to say yes or no, and making sure that those people say yes or no as a group so an arbitrary member among them can't veto something another person has said yes to (which doesn't happen often, but is heart-wrenching when it happens).

webchick’s picture

Issue summary: View changes

Initial draft

Crell’s picture

That sounds like a good approach to me.

Dries’s picture

Status: Active » Fixed

I've committed a final version of the charter to the Governance Git repository at http://drupalcode.org/project/governance.git/blob_plain/refs/heads/maste.... I've also updated the issue summary with this text so you can view the "diff".

In addition to a variety of minor wording tweaks (including a more generic "web infrastructure" terminology over "LAMP stack" and specifying "the Drupal.org websites" instead of "Drupal.org," per Crell's feedback), I've amended the "Exclusions" section to state that software decisions are made in coordination with the DSWG, per killes's feedback. I do think it's important that the DSWG is empowered to own the feature roadmap for Drupal.org, but it's a very good point that they can't do this without being in close collaboration with the DIWG.

Keep in mind that this will be a living document. Going forward, please propose further refinements by submitting a patch against the charter in Git.

I believe this addresses all comments, so marking fixed. Thanks all!

The next step is to appoint the inaugural members for the Infrastructure Working Group. More about that later.

dww’s picture

Status: Fixed » Needs review

Re: #7: FYI: such a policy document already exists as the Drupal.org development guidelines page, more or less.

Great to see some structure and attention coming to these problems! We definitely need to figure out how to be able to make (and implement!) these kinds of decisions.

However, based on reading the draft charter and this issue, I'm confused about who would be appropriate members of this working group. If it's just the existing team of Infra maintainers, it's not clear what's changed. If it's not those of us (informally) making these decisions today, how do we ensure that the people making the decisions are sufficiently knowledgeable about the problems, resources, etc that the infra team deals with? I.e. if we say this working group is killes, nnewton, damz, drumm, sdboyer, and myself (to over-simplify the current situation) what exactly has been improved? If it's not the 6 of us, what ensures that the 6 of us will actually abide by (and implement) the decisions this WG is making? Let's say the DIWG decides to do something then nnewton says: "sorry, that'll melt the current (and foreseeable future) DB servers." How do we resolve that? Make sure nnewton is on the WG in the first place so he can shoot it down earlier in the process? I totally respect webchick's perspective that the question should be phrased: "since we need this, how do we make it happen?" whenever possible, and the goal isn't to "shoot down" changes at the right point. ;) But, sometimes, that's what has to happen given resources and the trade-offs we often have to make in the real world. Expand this to all the primary pain points the infra team deals with and you've basically said "okay, everyone on the infra team needs to be in this WG", and then we're back to "so, what changed here?" If the answer is: "we'll Get It Right(tm) and appoint Really Knowledgeable People(tm) to sit on this WG" (and I know our community is full of them), I'm still left with the uneasy feeling that there are going to be sticky problems where a group of knowledgeable and smart WG members is still ready to make decisions at odds with what the Infra team believes is actually possible. Then what?

Related problem: just because we have reached a decision doesn't necessarily mean we have the resources to implement it. Maybe this is more often a problem for software problems (and I'll reply at #1929526: Drupal.org Software Working Group Charter (DRAFT)) but this seems to generally be an issue for these governance working groups to grapple with. Perhaps I don't understand the scope of the "budget" and what that implies. Is this WG going to have a budget separately from the Infra team itself? Does this WG's budget replace the current "infra" line-item in the DA budget? I can think of various examples of feature/change X being considered A Good Idea(tm), and generally having agreement from the infra team that it'd be okay to implement, but no one on the infra team had time/energy/resources to actually see it through to completion. It's not obvious from the text of this charter alone what this changes about that fundamental obstacle to progress on d.o. Even when we manage to reach a decision, we can't always drop everything and make it happen.

Certainly, I'm not the only one qualified to deploy changes on d.o. But, after years of doing it, people trust me that I'm not going to break things, that I'm careful, that my excessively high standards and attention to detail (often an obstacle to high velocity for d.o), are basically useful characteristics for the people with root access to our servers. ;) Is this WG empowered to hire other people and give them root access to deploy changes when I'm too busy with other things to do it myself? Should the charter say so?

I hope these comments/questions are seen as the constructive attempts to clarify and avoid problems that they're meant to be, not as a curmudgeon's resistance to change or an attempt to derail progress. As I said, I think it'd be great to have a more formal process instead of the self-selected team informally making decisions now. I just think that in practice the problems are more complicated than this charter addresses, and if we're going to go through all this effort, I hope whatever we come up with can actually handle the real circumstances the WG is going to be operating in.

Thanks!
-Derek

ricardoamaro’s picture

Hey folks,

Sorry to pop in to the middle of this.
I'm volunteering myself for one of these tasks and committing to the success of all the Infrastructure work.
Also:
a) I Agree with Crell that we should avoid the term "LAMP Stack" and go for a more general designation.
b) Derek's comments regarding high standards and attention to detail, in my humble opinion, make all the sense and are probably one of the things that makes d.o. a reference.

A part from that and from what we spoke on #drupal-infrastructure, i'll try and go to the http://drupal.org/project/issues/infrastructure and try to clear stuff there.

dww’s picture

Apparently this was all decided and resolved without really answering my questions and concerns:

https://association.drupal.org/node/18033

This issue should now probably be marked fixed, but I don't want to be the one to do that. I'd rather someone actually tried to answer the questions first. ;)

Cheers,
-Derek

tvn’s picture

Just a note that human-readable copy of the charter is located here: https://drupal.org/node/2001488. And there is a patch in https://drupal.org/node/1929530#comment-7564307 which removes placeholders for member names.

tvn’s picture

Issue summary: View changes

Updated issue summary.

webchick’s picture

Issue summary: View changes
Status: Needs review » Fixed

This group was chartered awhile ago, so marking to fixed.

To answer the questions in #12, the membership is largely the same as the existing infra group, though a bit smaller, and with the addition of some DA staff. I think what's changed is hopefully not much materially, since the infra group does a pretty good job of managing itself, generally speaking. What it does do is provide additional transparency around who's making decisions there, and it also helps to increase the communication between the DA (who own the budget) and the infra team (who hold the plans/do the work). By having that better communication, it puts the DA in a better position to hire staff to execute on things that the existing volunteers either don't have the bandwidth, or don't have the desire, to do. It also gives other groups like e.g. the DSWG and the DCWG a concrete set of individuals to talk to about cross-group decisions, because either they're on the DIWG, or they know who to find the answers from within the larger infrastructure team.

Hope that helps. If you have suggestions to amend the charter itself, please post a patch. If you have questions for the DIWG itself on how it manages things, according to https://drupal.org/governance/drupalorg-working-groups/infrastructure that should be done at https://drupal.org/project/infrastructure with a "Drupal.org Infrastructure Working Group" tag.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.