Drupal - a highly promising and excellently designed open source software - prides itself on being "community-driven". A primary Drupal interest is promoting collaboration and the sharing of views and perspectives.

Yet, paradoxically, the project decision-making structure- a relic from its very early days - is notable for its closed, centralized, and fixed nature. Most decisions on the Drupal 'core' are still made personally by the project owner and founder, Dries. The only exceptions are decisions made by one of two close associates of his. This contrasts sharply with many other open source projects, which are build on the basis of community decision making.

To fully mature as a project, Drupal arguably needs to bring its structure in line with its community spirit.

Hence this set of concrete proposals. Their aim is to set out new, expanded roles for decision making, while ensuring that the current values of rigourous code evaluation and development are maintained.

Note: This proposal picks up on discussion last summer, see http://drupal.org/node/10321, and on ongoing comments and developments. For a summary of Drupal's current structure, see http://drupal.org/node/10257.

The Drupal.org user ids give an idea of just how static the core developer group is. The three project owners have ids of 1, 2, and 10, and were all added in the first week of the drupal.org install. In the nearly four years since then, over 16,000 people have joined drupal.org. Hundreds have made contributions to the core--some small, others increasingly substantial. A handful of these hundreds have been informally designated by Dries as "maintainers" of one or another part of the code. But not one of them has gone on to be accepted into the decision making group.

This centralized, closed structure has unintended but serious implications for the project code.

  • It means that many promising initiatives never see the light of day simply because they fail to take the interest of or fit with the personal priorities or vision of one of the three decision makers. It produces, in other words, a certain conservatism or insularity in the project, almost inevitable in a small, closed group. (Arguably, for example, the fact that several years on images are still marginal to the project reflects this factor.)
  • It means that founding core developers are required to personally review and decide on every proposed change, no matter how trivial. (Or, as often happens, the proposals sit untended due to lack of time or interest in this small group.)
  • It means that the project fails to fully engage the energy, talents, and ideas of new developers.

These issues are not absolute. New initiatives and contributions are indeed approved, and new developers take on work. But Drupal's very success has rendered its earlier models unworkable. Two or three people can't reasonably make every decision any more. Indeed, this structural issue is, arguably, the primary holding back the project's maturation and development.

What's needed to make Drupal truly "community-driven" is simple. The project needs to be opened up, so that there are clearly-defined roles for new developers to grow into and ways for community decision making. It needs to complete the transition from personal, private project owned to one individual (or his friends) to an open structure that welcomes new energy and ideas and is owned by a community.

Fortunately, there are successful, clear, and proven models to work from. In fact, most or all of the most successful community-driven open source software projects have such structures. Examples include PostgreSQL, PEAR, Jakarta, etc. It's a primary source of their success.

Proposed new structure

What changes would complete Drupal's transition from closed, personal to open, community-owned project? Here is a set of concrete suggestions drawing on other successful open source projects, while also considering Drupal's existing structure and culture.

The Drupal Community: Proposed Structure and Decision Making

  • Users.Users are the people who use Drupal. Users aren't contributing code but may be submitting bug reports or feature requests through the issues system and participating in the drupal.org forums.
    Implementation action: none, as this role is fully covered off by current Drupal practice.
  • Contributor.Contributors contribute code patches or documentation for the Drupal core.
    Implementation action: none, as this role is fully covered off by current Drupal practice.
  • Developer. Developers have responsibility for maintaining and patching the Drupal software and have write access
    to the core CVS depository. They do not however make decisions individually; they only apply patches that have been duly reviewed and approved by vote.
    Details:
    • Contributors who have made substantive contributions may apply for Developer status by writing to the Developer list; they may also be nominated by others (Contributors or Developers).
    • Applications for Developer status are decided by majority vote of the current Developers and PMC members.
    • Both PMC and Developer members vote on proposed patches. Net of five votes (positive minus negative) needed.
    • Initial target group size: 10 - 12

    Implementation action: Current "maintainers" form initial Developer group, with possibly a few additional invitations to other proven Drupal contributors.

  • Project Management Committee. The Project Management Committee is responsible for the overall direction of the project. Decisions include: substantial structural changes; introductions of new core modules; developing and issuing plans and statements of direction; managing purchases and financing; managing public representation of project.
    Details:
    • This group makes decisions by votes, with 2/3 majority required for approval.
    • One member of this group is elected annually to be PMC Coordinator (roughly equivalent to role currently played by Dries).
    • New members from among the Developer group are accepted into the PMC by invitation of one current PMC member, with 2/3 majority vote required for approval.
    • Initial target group size: four to six (small enough to make for easy process yet large enough for diversity)

    Implementation action: Convert the current "cvs review team" to a PMC; potentially, invite some current "maintainers" to join.

In sum: These are relatively small changes, ones that would build on Drupal's existing strengths of a strong and dedicated development community. In the long term, though, they are likely essential to ensuring that Drupal becomes and remains a vibrant, innovative project that can solidly claim its status as "community-driven". The project can't afford to let these important matters drift indefinitely. It's time for some simple, proven, effective changes!

Invitation for comments

Please comment on this proposal! Do you agree with the basic point--that a more open structure is critical to Drupal's ongoing success? What do you think of the proposed changes? Do they meet the need? Are they practical? What should be changed in the proposals?

If you have concerns with the proposals, what are they? What potential negative outcomes do you see? Are there changes that could be made to address your concerns and avoid these outcomes?

Please post your thoughts as comments on this forum page. Discussion of these questions can only help!

Additional issues

In addition to the core issues addressed above, here are a couple of priority areas for future consideration.

Review guidelines

A crucial support to structure is process: how discussions and decisions are conducted. To this end, Drupal would benefit from improved protocols. Currently, comments - particularly by the CVS review team - are often abrupt and lacking in detail or alternate suggestions. Many times, it is unclear whether or not a commenter agrees with the need identified. More well developed guidelines on etiquette would be useful.

Drupal extension repository

Parallel with improving the core drupal structure, Drupal would benefit from a more formalized approach to extensions (modules, themes, etc.). This might include a more formal process for membership (based on the current one, where individuals apply for contributions CVS access) and a peer review and voting process for new proposals, similar to the well designed PEAR process. As things stand, the process for reviewing and approving proposed additions is opaque and there is no role for existing repository contributors or maintainers.

Comments

crazybooks’s picture

There is a cost to enlarging the 'top team', especially if there is no clear leader. I have learned this to my cost in another project.

I think a better way to do it might be that all developers get told a decision

1) They all vote 'yea/nay/moreinfo please.
2) The majority vote is then worth a SINGLE vote
3) at the 'project management' level.

Combine that with some non-programmers/documentation writers/helpers as...e.g. Customer Service Managers or whatever, and you might get somewhere. I'm sure I'm not the only experienced community manager who would be willing to contribute time to drupal in a non-programming way.

Drupal has done fairly well under the system it runs under right now. It is cleaner than most other OSCMS. While some queries go unnoticed, and modules go unmaintained, this is no worse than I have seen in places like tikiwiki et al.

More people at the top will make the decision process even longer.

Okay, that was a ramble, and an unformed one. I might try and develop my ideas a bit further later on

----------------
I R Admin Monkey. Damn those users. Damn them with Zim like hyperbole

inteja’s picture

In general I think you're on the right track but I'd like to play "devil's advocate" for a moment. ;-)

Democracy is overrated, give me a benevolent dictatorship any day.

One of the strengths of Drupal (and therefore reasons for its popularity to date) is its consistent vision and the core developers should be applauded for maintaining this in the face of competing interests within the community.

Opening up core development could potentially dilute this strong vision, leading to ad hoc additions and bloatware ...

The main problem with democracy/"majority rules" is that each vote is NOT equal. No 2 voters make their decision based on the same, adequate or "correct" (whatever that may mean) information. Everyone has their own agenda and vision.

If the long-term vision (roadmap?) is spelled out (in a general way) by the Drupal founders and promoted as one of the community's core values, we can hopefully build a more open development community that, on the whole, is still highly respectful of the initial strong vision and implementation that attracted us all to the project in the first place.

Brian.

--
Brian.

carlmcdade’s picture

There are four stages of software development, innovation and expansion are two of them. I forget the what the other two are but Drupal is at the second stage I think. This stage does not seem to be well recieved as of yet.

Anyway I am posting this just to mark it so the I can track the re-actions from the team. It should be interesting.

---------------------------
www.hivemindz.com (running PHP5)
www.fireorb.org (documentation and hacks)
__________________________
Carl McDade
Information Technology Consult
Team Macromedia

clairem’s picture

nedjo, I hope you don't mind me commenting here: I'm a relatively new user of Drupal, and have no intention of becoming a developer. The most I might contribute would be a theme some day, and I'm not promising! I'm just a user who is very grateful that so many folks have worked so hard to make Drupal so good.

My comments are not so much on the details of what is being proposed, but on the rationale. From a lot of experience in voluntary projects (tho not in this field of open-source development), I'd commend some caution. I don't want to sound rude or negative, so please forgive me if any of the following across that way. I'm just trying to suggest a few points which may, I hope, help. If not, please ignore.

'Community-driven' is one of those terms which sounds warm and friendly, like motherhood and apple pie. The problem is, though, that it is vague and hard to argue against ... and I think there's something to be said for being wary of ideas which are quite so warm and fuzzy. They don't often translate clearly into a particular course of action.

The proposed structure seems to me to be more accurately described as 'developer-led'. That's probably a good thing, but it's not the same as community-driven', which implies folks like me having a much stronger say in running things.

The only thing I'd wonder is whether the voting structure might be too cumbersome and unwieldy. I generally reckon that projects wsork best when run with a relatively small core of ultimate decision-makers, but with those folks consulting widely and listening a lot. Too wide a steering group, and it's easy to lose focus.

drumm’s picture

I like how things are going now. Also I feel that evolutionary changes can and will happen as they are needed. I expect this has been happening, but I haven't noticed, or tried to notice, in my year with Drupal.

moshe weitzman’s picture

i too am happy with how the project is running. but can it run better? thats what this proposal is about. nedjo is asking for *proposals* on how to improve the CVS review process. Its OK to react negatively to nedjo's proposal. but please propose something of your own. those are the most beneficial comments.

i will post some of my own deep thoughts on this topic soon.

cel4145’s picture

"To fully mature as a project, Drupal arguably needs to bring its structure in line with its community spirit." I think this point is unprovable and would offer the following counterpoints:

  • If it's not broken, don't fix it. This proposal is a solution to a problem, but I don't hear any serious problems being articulated in this proposal. Of the three "serious implications above," the two that suggest problems I believe would have to be better supported instead of so generalized. For instance, I've been following the development of Drupal for almost 3 years now--even though I can't write code effectively, I now administer close to 60 Drupal sites so I pay attention--and I've seen significant increases in the number of developers working on Drupal. Suggesting a radical change to an organizational structure that is so far running successfully except for some general, arguably unproven assumptions, is a very big risk with potential for failure.
  • Would any question that Linux is community developed? Drupal's project decision-making structure isn't that far removed from Linux. Note the similarity, too, that a computer science student, without strong commercial interests/projects, is the leader of the project. That's sort of a best scenario for a situtation where contributers all have competing interests. Someone with a vision leads the project, rather than someone with their own project to implement for a client.
  • "The Drupal.org user ids give an idea of just how static the core developer group." This sounds more like to me an indicator of the longetivity and devotion to the project.
  • A benevolent dictator does have to listen to the community of developers. So far, there have been no serious forks of Drupal. I would attribute this to good project management, in which Dries and the other core developers have worked diplomatically to follow a path which supports the general good of the community.
  • There is very little flaming and bickering on the development list. That's likely to change if Drupal turns to a democratic community organizational structure. Indeed, Drupal's development mailing list is extremely productive and civil, very focused, much more efficiently than I imagine you might find in other communities. Anyone else ever see a large project of this size which communicates this well?
  • Even in a democracy, you can't please all of the people all of the time. Changes to the organizational structure to a more democratic community does not guarantee that more people will be satisfied with the direction of the project and the way things are run. Indeed, it may introduce additional complications. It may make Drupal development much more political. It may introduce some administrative bureaucracy to slow the development process.

So think smaller, working to make the existing development structure better, rather than drastic reorganization. For instance, if code review is a problem, then consider the Linux model where the leaders allow people to step up to assist them. Articulate the problems more specifically, then work to fine tune the existing structure to deal with them.

sepeck’s picture

So, this is a propose to change CVS access and the development model of Drupal. I don't see any need to do anything as radical and formal as mentioned here. I also disagree with some of the conclusions that formed the basis of this proposal as a result. For the longest time a much larger project has survived with the benevolent dictator model. Here in Drupal there are trusted Colonels (CVS commit and gatekeepers), there are trusted Captains (active developers who are core modules maintainers) and Lieutenants who while not core module maintainers have their contributed modules and are active participants in the development QA process and feature requests. I think that the other project is famous enough that we don't need to mention it here.

The one thing Drupal lacked when I started was a lot of mid level users who could help out on the forums. That has changed a bit in the last year with a lot of new participants who after implementing their site, stick around to help other new people. Documentation also needs to be improved and there are several people that have begun to do just that. Dries has made changes on the permissions to Drupal.org to allow this (much to my surprise when my menu options changed on login to the site).

* It means that many promising initiatives never see the light of day simply because they fail to take the interest of or fit with the personal priorities or vision of one of the three decision makers. It produces, in other words, a certain conservatism or insularity in the project, almost inevitable in a small, closed group. (Arguably, for example, the fact that several years on images are still marginal to the project reflects this factor.)

Or the proposers refuse to operate within the rules and leave in a huff when people won't dance to their tune. Mentioning the marginal image support is fair but fails to mention no clear or interested lead developer for it for a time. Fails to mention that Dries had an expressed desire to see a solution to file management that included image management, but ultimately the proposed solutions by several people fit one scenario well and others very badly (that and the participants working on it last year ran into time issues with the freeze of 4.5 even though Dries delayed the freeze a few weeks in the hope of a solution). So image lived on as a contributed module that is now seeing some nice improvements. Also, the inline module showed up as a way to deal with images and nodes (which I really like)

* It means that founding core developers are required to personally review and decide on every proposed change, no matter how trivial. (Or, as often happens, the proposals sit untended due to lack of time or interest in this small group.)

Anyone can submit a patch, anyone can review a patch and anyone can comment on a patch. If they are involved, then they can debate the comments regarding potential issue's of a patch and advocate it. All to often someone breezes in, tosses in a bunch of stuff without bothering to read the contributors guide. Submits no patches, complains loudly about the patching rules and their inability to follow them or submits patches that do not comply with the coding standards and wont resubmit them, takes all comments personally and instead of cleaning up there code or addressing the comments to refine their contribution they disappear or leave in a huff. If a proposal sits untended, it means the advocate for the change is not addressing the concerns of others or involving the other contributors to get their support. I have seen conversations where Dries committed features to core because support of the active community believed it would be beneficial.

* It means that the project fails to fully engage the energy, talents, and ideas of new developers.

Or new developers do not understand the deliberate process of planning, discussing and advocating a solution. Many of the proposed solutions from new developers seem very narrow and designed to solve a specific need. The most common comment replies I see in the initial reply of a proposed new feature is that it is to narrow of focus and needs to be broadened so that it is more useful to others (or that there are other ways to solve this that already exist). The most common problem I see is that the person proposing it goes off in a huff or complains loudly that no one appreciates their effort and that everything needs to be changed and that others need to be able to commit. The multi-site patch has been a long time in development, it will be part of 4.6 now. The proposer stuck with it and in the end got it in (I know I am looking forward to playing with it).

Drupal's development model is not fast, but it is deliberate. To me, it marks a mature process that functions well. As an end user of the product, I appreciate that (I only have 6 sites). It generally means to me that I will be using a quality project.

I think that the 'unintended' consequences of that tight project leadership has been to keep Drupal core small, code clean and the project stable and the ability to add modules that fill custom niche's and introduce experimental new features that always have the potential to migrate to core. This has resulted in several major sites migrating to Drupal in the recent months and continued to attract contributers. There is always room for improvement, but I don't think this is the way in this particular case.

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

Steven’s picture

I don't think your proposal is necessary, and that you're really looking at the wrong things.

I'm of the three people mentioned with core commit access. However, if I have made a non-trivial/non-bugfix patch, I submit it to the issue tracker like everyone else. If the community thinks my patch is bad, then it will have to be changed or dumped, regardless of my "personal vision".

You seem to put a lot of importance on who can change the actual core: is it really that important? Patching a CVS tree and pressing the "commit" button takes 10 seconds. It is not the bottleneck in the development process.
Making sure the patches are as good as they can be: i.e. usable, valuable, clean, functional and efficient is what takes long. And this is something that anyone can do: download a proposed patch, try it out and improve it. In some cases it's faster to modify a patch than it is to describe what you'd like to change.

In your proposal, applying a patch would still require a vote by several people, so in effect it wouldn't be much different from the current situation, except that we'd lose more time because of bureaucracy.

It means that the project fails to fully engage the energy, talents, and ideas of new developers.

You seem to want to give out merit as a way of rewarding people's commitment to the project. While it is certainly a nice idea, your proposal is not really the way to do it IMO.

Do people really need to be part of some official developer or project management committee to feel accepted and to contribute to the project?

Plus, your outlined scheme focuses mostly on code development. What about people who contribute translations, themes or documentation? We'd need to set up a special merit for them too, otherwise we'd be spreading the message that code is superior over other contributions, when in fact we have too few themers and documentation writers on board.

--
If you have a problem, please search before posting a question.

chx’s picture

"To fully mature as a project, Drupal arguably needs to bring its structure in line with its community spirit." -- arguably: "That can be argued plausibly; defensible in argument."

Show some real arguments for the need of change, 'cos there are a lot of developers who just can not find any problem with the current system. Yesterday evening on the #drupal channel there was a trial vote casting for "Leave Dries alone" and unamiously everyone voted +1 on this...

--
Drupal development: making the world better, one patch at a time. | A bedroom without a teddy is like a face without a smile.

robertDouglass’s picture

Nedjo,

I commend you for posting on such a tricky topic. It is hard to discuss changing the structure without sounding outright critical of the current leadership, which I surely don't think was your intention.

I, like others who have posted, am not in favor of drastic change, but rather would be interested in organic modifications, even if they bring us to the same place in the end.

More research

If you are really interested in introducing structures into the Drupal organisation that are inspired by other open source projects, the burden of research and education is going to fall on you. You've made me curious how PEAR is structured, for instance, but even if I do end up researching it myself, I will be one of the few. You might start by telling us more specifically how that ship is run, why it is more effective, exactly what the benefits are and also what the drawbacks may be. Bring someone from the PEAR team here to testify to the merits of their approach.

Succession

If you asked me what my main concern is with the current structure, I would tell you I worry about what would happen to the project if anything terrible were to happen to Dries (like him getting a full-time job ;-) ) , and if we would be able to either replace him, or if we have a mechanism in place should he, for whatever reason, not be able to show the same level of dedication to the project at some point in the future?

As I see it now, the overall level of development is increasing and is likely to do so for some time. That may mean that the core team will have to grow to keep up, but if it grows from circa three to circa five, that will probably be adequate, I think.

----

One case that I can think of where we would definitely need a more codified structure would be if we start involving money. Should Drupal become a non-profit organisation capable of raising funds (not just the tip-jar), then we clearly will need something along the lines of what you've proposed.

- Robert Douglass

-----
visit me at www.robshouse.net

my Drupal book | Twitter | Director, Product Operations Commerce Guys

Gunnar Langemark@www.langemark.com’s picture

I spend a year looking for a good system to use, before choosing Drupal more than two years ago. I was around the Nukes - and saw the terrible fights going on there.
One reason for favoring Drupal over the others was the excellent core team of developers and a structure with a "benevolent dictator".
Over the two years gone by I have seen Drupal mature and change. It has become a much better system, but it has also seen some less positive developments. I see that understanding the taxonomy system is not something you can expect of all developers and not all newcomers care to spend the time to learn before they attempt to change what is not broken. There need to be a core.
Also don't kill the enthusiasm of the core developers in bureauracy. It need s to be fun to get people to volunteer to develop.

Just my 2c from the sideline.

Dropping in from Langemarks Cafe.

carlmcdade’s picture

Here is a good paper on OpenSource that I read about a month ago when trying to understand just what I was getting into when I started looking at the development of Drupal. Informative. Drupal.org is holding this description but there are somethings missing. Like who is on the core team?

http://sern.ucalgary.ca/students/theses/KimJohnson/chapter4.htm
---------------------------
www.hivemindz.com (running PHP5)
www.fireorb.org (documentation and hacks)
__________________________
Carl McDade
Information Technology Consult
Team Macromedia

sepeck’s picture

In the handbook under a top level section called 'General infromation about Drupal, there is a page called The Drupal Community: structure and roles. http://drupal.org/node/10257

-Steven Peck
---------
Test site, always start with a test site.
Drupal Best Practices Guide

carlmcdade’s picture

Hmm,

I think that a good start would be to use some better typography and content in that book page. It is very hard to read and understand.

When I read the maintainer's text I get no feeling of connection with the actual Drupal code. Like where are the core module names and or groups? For example the taxonomy.module(or any of the default install) is part of the core but it is not listed. Is it grouped with another maintainer? or do all things unsaid fall to the bottom where Dries name resides?
---------------------------
www.hivemindz.com (running PHP5)
www.fireorb.org (documentation and hacks)
__________________________
Carl McDade
Information Technology Consult
Team Macromedia

Gábor Hojtsy’s picture

If you know what 'the rest' means at the bottom, you get the answer to your question.

capmex’s picture

I have seen many cases of projects sinking because of the lack of a good direction and management team.

I think if someone wants to be into the leading group, it first needs to prove its value to the project.

This centralized, closed structure has unintended but serious implications for the project code.

From my point of view, those serious implications are only beneficial to the project.

--
Webmaster Resources for Business Websites

Gábor Hojtsy’s picture

Most of the commenters experssed the resounding feeling that Dries is in a good place, and there is nothing wrong with the Linux kernel like development structure we have. I would also put myself on this side.

Although where I do see a problem (and it is not only me, the three core guys expressed their wishes too about this before) is how reports come in and bugs get fixed. The problem is that we receive quite a few reports without patches, and these quickly become pushed down by those properly engineered ones with fine patches attached. It is very hard to deal with those only describing a problem, or having some fix, but not trying to make a patch out of it. Even if the core developers are committed to fixing these kind of bugs, it should not be their responsibility to code the fixes.

Some infrastructural additions might help a lot to get real patches. Either some educational material on how to create patches on Windows, or a service where you can upload some files, and request a patch against a given Drupal version, so that you can submit it right away to the bugtracker. This might help a lot with those, like Carl McDade, who do a great job fixing stuff, but are not willing/unable to do it in the bugreporting framework.

I think that helping those not able to cope within the framework with documentation and more tools should take off a lot of frustration of those, who see that this structure is not working.

nedjo’s picture

Thanks for the extensive and thoughtful comments on the proposal.

Other open source software projects have benefited from focused work on structure and organizational development. The Plone CMS, for example, last year created the Plone Foundation to support its development, investing the foundation with ownership of Plone while keeping development in the control of project teams, each with a leader.

While it may strike some as bothersome or even lacking in due respect, attention to and discussion of structure and decision-making is an essential part of making a strong, successful project.

I'd like to briefly address some of the main issues raised by commenters, and then present a revised proposal that, I feel, successfully addresses the main concerns while keeping the main benefits of the original proposal.

Some commentators thought my proposal extreme or "drastic". If you take the time to read what I'm actually suggesting - particularly the revised version - I'm pretty sure you'll see that there's nothing drastic about it. Rather, the proposal merely recognizes existing characteristics of Drupal and suggests what are essentially small and fairly obvious next steps.

  • Drupal is, de facto, a community project, rather than a personal, individual one.
  • Drupal has a growing group of skilled, dedicated, respectful, and proven developers--just what's needed for successful collaborative process.
  • Some small changes in the structure of the project can help complete this evolution from personal to community project, removing some of the current barriers in the project while firmly retaining the strengths of leadership.

My aim is to protect and maintain the current strengths of our project--while making some fairly small but important changes to help ensure its long term health and sustainability. I remain hopeful that - like a patch that goes against the established wisdom, but may nonetheless bring considerable benefit if appropriately considered, critiqued, and refined - this proposal can bring lasting benefit to the Drupal project and community. Indeed, if I didn't believe this, I wouldn't be taking the time (and, seemingly, some personal risk) to present it, nor asking the community for space for this dialogue.

Those arguing against any change in the current project structure give reasons that fall (I hope I'm accurate in saying!) into two general positions:

  • If it's not broken, don't fix it. Things are going fine, there are no glaring problems, so changes would be unnecessary and disruptive.
  • A "benevolent dictatorship" is a superior model not prone to the problems of "democracy" (problems that include lack of direction, insolence, etc.). Specifically, Drupal benefits from the current centralized model - particularly since Dries is so capable in his role - and might lose integrity under a model that didn't have a single owner/leader invested with all ultimate power.

I find both these positions to be serious and thoughtful perspectives appropriately based in a desire to maintain - even, to defend - a respected and valuable project and the dedicated individuals who have brought it into being.

But I think they overlook important truths. A corollary to "if it's not broken" is (sticking to old adages) "a stitch in time saves nine." Taking small but significant steps early can have large long-term benefits. It's not always necessary - nor is it desirable - to wait for a crisis before taking ameliorative action. So, I suggest, the fact that there is no crisis doesn't in itself mean that some change is not appropriate--even needed.

The second claim - that of the merits of dictatorship - is not quickly or easily tested (even Mussolini's fame for rail punctuality is not secure!). But likely we can agree on this: there are various ways of producing quality software, and some will be more appropriate to particular cases. If the Linux kernel is one example, something like Postgresql is another, having a broad steering committee and robust group of lead developers and demonstrating the impressive potential of a collective enterprise. That individuals may have had negative experiences with certain group-led projects doesn't prove that the problems they observed were results of the model--much less that such results are inevitable. The one specific example of disfunctional group-led projects cited - the "Nuke" CMSs - can equally be read as reflecting poorly on the "dictatorship" model (since, apparently, splits from the original PHP-Nuke resulted directly from widespread dissatisfaction on the part of PHP-Nuke contributors with the PHP-Nuke owner's approach).

The critical question here, I think, is not "What model is inherently superior," but rather, "What model is the best and most promising fit for the Drupal project?" Or "What model can retain a strong leadership role while opening up new possibilities?"

Looking at "best fit for Drupal" question, here are the core issues I see in a "dictatorship" model:

  • A "dictatorship" model sits uneasily with Drupal's goals and model.
    Rather than being a corporate or individual project, Drupal is defined on the opening page of the Handbook (http://drupal.org/about) as:

    an open source software project maintained and developed by a community.

    A strong emphasis is put on collaborative development, discussion, and involvement on the part of the "community". While it might work and provide efficiencies in other contexts, and provides certain benefits to Drupal, a "dictatorship" just doesn't easily fit with these core characteristics of Drupal. In particular, the "dictatorship" model has a certain overall dampening effect on the free exchange of ideas, and may promote a degree of favouritism and conformity, even given the most "benevolent" of "dictators". Even the tone of some comments on this proposal - with the implied messages that it is inappropriate or even disrespectful to put forward proposals that seem critical of the leadership model - suggest this problem.

  • A group structure, rather than a "dictatorship", will better position Drupal for long-term sustainability.
    Work is already underway to put in place a formal organizational structure, e.g., a Drupal Foundation and discussions with Software in
    the Public Interest
    . As Robert Douglas suggested above:

    One case that I can think of where we would definitely need a more codified structure would be if we start involving money. Should Drupal become a non-profit organisation capable of raising funds (not just the tip-jar), then we clearly will need something along the lines of what you've proposed.

    The case he raises is not a vague future possiblity but a concrete current initiative, a good one for the health of the project, but one that obviously calls for organizational work.

  • The particular type of development characteristic of Drupal make a model where all decisions are made by one individual both unnecessary and inefficient.
    Drupal is developed at a much lesser level of abstraction than, say, the Linux kernel. Even within the core, Drupal is build modularly. Many small issues arise - formatting, errors, minor extensions to existing functionality - that any longtime Drupal developer is competent to help decide. Personally, I have full confidence in every one of the Maintainers (plus in several other proven contributors). Leaving every detail to the founder and his associates is not only unnecessary but also inefficient; it leads to bottlenecks and is not the best use of this top leadership resource.

Commenters rightly pointed out problems with the idea of a "democracy" in which everyone, or a large base, would directly participate in decisions. But that's not at all what I've suggested. Rather, it's what the Jakarta project terms "meritocracy", in which:

the more you do, the more responsibility you will obtain.

This approach is intentionally designed to maintain and protect core developer roles, while at the same time providing for collective rather than individual control.

But rather than getting stuck at these general, rhetorical levels (When is change appropriate? What are the merits of dictatorship?) I think it's much more productive to look to assumptions and values behind these positions. There is, if I'm reading them correctly, a core set of assumptions and values about the current project in the comments made. I'll try to summarize these:

  • There are important strengths to the current structure, which shouldn't be hazarded.
  • Drupal is blessed with a highly skilled and visionary leader whose continued leadership is desirable - even critical - to the project's success.
  • Throwing open the development process to the metaphorical "masses" would likely lead to poor decisions and weak directions.
  • Drastic change is disruptive; it's better to take small steps.
  • Without a clear system for decision-making, discussions can degenerate into unpleasant and unproductive squabbles.

If these are the core assumptions and values being expressed, I can only say that I'm wholly in agreement. In fact, they are among the core considerations I had in mind in devising my proposals.

That said, I now see, thanks to the comments, there are significant refinements that could be made. So I've revised the proposal to make it even closer to existing practice, and even stronger in its protections of the leadership role.

I'm confident that those who were concerned with my initial proposal will find that this revised one directly addresses their concerns. As suggested, I've made the changes even more gradual--just enough to accomplish the aims of moving past a strict "dictatorship" model and to continue putting in place the building blocks of project sustainability. I've provided strong protections and vetos to keep the existing leadership control and avoid tying down the the leadership or sacrificing the positive lead roles. Finally, I've kept the CVS access (which, as Steven correctly says, is a technicality) with the Project Management Committee.

With all of these changes or scaling backs to the proposal, one might ask, "Why then bother?" My answer is my belief that even the minimal - almost cosmetic - changes I'm proposing would make a qualitative difference for the better.

Here, then is the revised proposal (with some deletions and with revised text in bold).

I'd greatly appreciate comments on this revised proposal--particularly ones that actually reference what I'm proposing! As with a software patch, I think it's of most benefit to focus on the specifics. If there is concern that the proposal will lead to mission drift, bickering, etc., why? What part of the proposal would make these outcomes likely? What additional protections could be built in to address these concerns?

Also, I've tried to answer the question "Why is the Drupal project and community well suited to a community-managed structure." From any proponents of the "dictatorship" model, it would help: What particularly is it about the Drupal project and community (as opposed to many other community-based open source projects) that, in their view, makes a "dictatorship" necessary? What characteristics of Drupal make it unsuited to joint decision making?

One line of criticism suggests that my proposal introduces a cumbersome "bureaucracy" of new (and unnecessary) groups, structures, and processes. I totally agree that excessive structure is a drag. But, if you look at what I've actually suggested, it will be clear that I haven't proposed a single new group or position. All I've done is slightly recast existing ones. The existing "Maintainers" become "Developers". The existing "CVS review team" becomes the "Project Management Committee". The existing founder becomes the PMC Coordinator. As for the decision making, I've suggested votes on proposed changes. This is exactly what we have now. The only difference is that, in my proposal, votes of Developers (skilled and proven project contributors) would actually count. As for the potential extra work of tallying votes cast by individuals in a particular group, isn't this just the sort of routine task that Drupal is ideally suited for (e.g., through a minor patch to the project module)?

The Drupal Community: Proposed Structure and Decision Making (revised version)

  • Users.Users are the people who use Drupal. Users aren't
    contributing code but may be submitting bug reports or feature requests
    through the issues system and participating in the drupal.org forums.

    Implementation action: none, as this role is fully covered off by current Drupal practice.
  • Contributor.Contributors contribute code patches or documentation for the Drupal core.

    Implementation action: none, as this role is fully covered off by current Drupal practice.
  • Developer. Developers have responsibility for maintaining code of existing core Drupal modules and components, evaluating and voting on change proposals and patches.
    Details:
    • Contributors who have made substantive contributions may apply for Developer status by writing to the Developer list; they may also be nominated by others (Contributors or Developers).
    • Applications for Developer status are decided by majority vote of the current Developers and PMC members.
    • Both PMC and Developer members vote on proposed patches. A net total of five votes (positive minus negative) is needed for approval. A patch may be approved without a vote from any PMC member, but a PMC member may veto any patch by voting against it. If this happens, the proposed change is considered denied until such a time as that PMC member reverses his or her vote.
    • This group is initially capped at 12 members, with any change to that cap requiring a both a vote of the Developers and PMC ratification.

    Implementation action: Current "maintainers" form initial Developer group.

  • Project Management Committee. The Project Management Committee is responsible for the overall direction of the project. Decisions include: substantial structural changes; introductions of new core modules; developing and issuing plans and statements of direction; managing purchases and financing; managing public representation of project.
    Details:
    • One member of this group serves as PMC Coordinator, responsible for overall coordination and leadership..
    • This group makes decisions by majority vote.
    • New members from among the Developer group are accepted into the PMC by invitation of one current PMC member, with 2/3 majority vote required for approval. The PMC Coordinator may veto any application.
    • This group is initially capped at 6 members, with any change to that cap requiring a PMC decision.

    Implementation action: Convert the current "cvs review team" to a PMC. Dries serves an initial 3-year term as PMC Coordinator, with plans for succession (e.g., he continues, or an election is held) to be made by the PMC at the end of that term.

inteja’s picture

Sounds too bureaucratic to me. We're not forming a new nation state here. Much of the structure you describe is in place already (as you point out) but not quite as formal and just using different terminology.

Your only major structural changes seem to be "at the top". In world politics the danger inherent in the "benevolent dictatorship" model is that "absolute power can corrupt absolutely" leading to despotism. Is this your fear?

But lets not forget we're not talking about world politics and I'm sorry I mentioned the term "benevolent dictator" in the first place. This is a free open-source software project with strong community input. We are all free to do as we please. We don't need to impose safety mechanisms to limit power. After all, how much damage could a "software despot" really do anyway? If such a beast were to arise, we'd all just have a laugh at the absurdity of it ...

In this case there's no financial rewards for Dries. He is doing this for benign reasons and it is a big responsibility with a great deal of work involved I imagine. He should be encouraged to continue in this role as long as he can given what he has accomplished to date.

Brian.

--
Brian.

Steven’s picture

One concern that I haven't seen addressed much is this: if the majority of the current developers think that the current system works fine, will the creation of teams really change anything practically?

For example, the current distribution of work is not just based on history, but also on the time that people have to spend on Drupal. The majority of us work on Drupal in our spare time. If we have elections, won't the same people that now have the time to contribute most be chosen?

I also still think that your criticism of patch acceptance is wrong. If enough people are for a patch, it /will/ be accepted, granted that it meets the standards for core (which are objective, not subjective). Features like image support have been lagging behind because they are hard to implement cleanly, as they depend on external libraries, not because one or two people dislike images.

Whether or not a feature gets neglected has little to do with the personal preferences of the core committers, but the focus of all developers, whether they have core access or not. If no-one wants to work on a particular feature, it doesn't happen. On the other hand, if someone proposes a patch, and someone else joins in and helps out, that patch will have a much higher chance of being accepted: not only because it will have more weight from people, but because it will probably be of higher quality.

In that way, our current model is not really a dictatorship. Everyone's votes already count, we just don't have a metric for it. And I don't think we need one. We reach a consensus through dialogue rather than superficial votes. One well-argued point against a patch made by anyone, core developer or not, can veto, if it is right. With a vote system, it seems to me the voice of the team members would in fact count for more than an outsider, regardless of the message.

Knowing the Drupal community, I know many of us will be reluctant to go through a vote process per patch. Small patches would require the same votes as something huge. That's not right either. So I predict we'd see a lot of bending of the rules, with tension as a result. Rules and regulations are easy to write, but they only work if everyone is prepared to stick to them. Given the current comments here it seems that would not be the case.

--
If you have a problem, please search before posting a question.

nedjo’s picture

Thanks Steven for this highly perceptive summary--the best we've come yet, I believe, to capturing Drupal's decision-making culture. Sometimes it takes these reiterative processes to hit the core of the matter.

Thanks to your clear summary, I'm thinking now that the main issues I'm raising might be addressed simply by making small adjustments to our project issue system.

If I might try to paraphrase your key points:

  • Drupal uses a consensus model where proposed changes are evaluated and refined through open discussion, and are applied only if and when all significant objections have been addressed. This approach ensures that changes are thoroughly evaluated and is a primary factor in the excellent design of the Drupal core.
  • A vote-based system would produce unnecessary work and potential friction. It also would open the door to changes being approved despite remaining issues (e.g., individuals raising issues are simply outvoted), leading to quality erosion.

I wholly agree with this ideal, acknowledge that it's a valid model, and further agree that we have many elements of it in practice.

So, I hope, we can focus on two outstanding issues.

(a) The first is a lack of community-accessible tools for the actual decision point.

It is true that we have a good description of the revision process. We also have an explicit set of criteria that a proposed change must meet.

But what we concretely lack is a clear way for community members - other than the "CVS review team" - to designate a proposal as meeting those criteria.

A proposal may have received considerable support, and all objections may have been resolved. One or more highly experienced Drupal developers may have reviewed it and found it satisfactory. But it will not be actually applied until and unless a CVS review team member - usually, Dries - decides that the criteria have been met.

In practice, a considerable number of proposals sit at the "patch" status indefinately, and often are in the end never applied, despite having apparently met our criteria. This may reflect in part the fact that the single status, "patch", covers everything from "Here's something that I think works" to "This has met all criteria for approval."

(b) The second outstanding issue - probably a lesser one, but still deserving of attention - is that we do have a hierarchy of status within the project, but it is not explicit. As Dries has put it, explaining how he evaluates patches:

the weight of one's vote depends on how much I trust and respect him or her

[based on past contibutions].

That contributors gain responsibility through the merit of their contributions is perfectly appropriate. But it would increase transparency and perhaps also efficiency if this status were explicit and thus visible by the community, rather than relying on the project lead's personal (and therefore not visible) views. In short: Anyone can set a project to "patch", but only certain contributors comments can provide the guarantee "this has been reviewed by a qualified Drupal commuity member, who finds it acceptable". Having an explicit way to make this distinction would be useful.

I'm thinking now that the first issue, and possibly the second as well, could be well addressed simply by making small adjustments to the technicalities of our review status.

  1. Add nuance to the issue status.
    This could be, for example (proposed enhancements to our current system in bold):
    • Issue raised (status = "active").
    • Patch in place (status = "patch").
    • Patch has been reviewed (by qualified reviewer) and found sound and meeting acceptance criteria (status = "reviewed" or "tentatively approved").
    • (Possibly) Sufficient time (e.g., one week) has passed to allow other reviewers to identify overlooked problems (status = "pending application")
    • CVS review team member approves and applies patch (status="applied")
  2. (Possibly,) Create new user role on Drupal.org for "developers", make changes to or from these new status categories dependent on membership in this role.

Or, alternately, if we wanted less change, I guess we could introduce a single new status, "approved", intermediate between "patch" and "applied", with either everyone or only people in a "developer" role being able to set this status, which translates as "This patch meets all criteria for application."

In either case, the final decision remains with the CVS review team, as per our current practice. This simply refines the process, providing for a community role in the decision-making point and thus an explicit prioritizing of proposals in the queue for CVS review team evaluation/approval.

fx’s picture

In a democracy, people vote and the vote is recorded and the results determine the outcome. Unless you live in Florida or Ohio.

What you have here is more like a very simple form of technocracy; rule by an elite group of technical experts. That is a vastly simplified definition of the term, and there is a case to be made for the argument that what Nedjo is proposing is actually much more akin to a formal technocracy. But that argument stretches my academic knowledge of the subject matter; I'm not going there.

Strictly speaking what you might have is a pure autocracy. That is rule by one individual. It may happen that it is benevolent. One of the concerns people may then have -- a longer-term concern -- for something like Drupal is: what happens when the torch is passed. Does it remain an autocracy, and are people going to be pleased with whomever assumes the role of autocrat? More importantly; what assurance do the contributors have that this is going to result in the continued health and well-being of the thing they have helped create?

What ever your opinion on the matter, I would be surprised if anyone who spent any meaningful amount of time thinking through these issues could honestly ignore the fact that the current situation is less than ideal.

It more closely resembles what I identify with as a roman senate (which has historically been misconstrued by some as a democracy), than a modern democratic senate. Sure, anyone can participate. But the people who "win" are the ones who have enough time to spend a) making good stuff and b) articulating why it is good. This leaves room for good stuff to fall by the wayside, just because someone a) didn't want to or b) didn't have an opportunity to engage in the necessary discourse to convince an elite few that it was a good idea.

When you put it to a vote -- even a limited vote, and to your point re: votes on big and little patches, that [limited voting] needs to be considered -- then you have, in that, both

A) a mechanism to compel people to go on the record

and

B) an END-POINT for a process which someone who has limited time/incentive can focus on which may very well motivate them to overcome any obstacles to articulating themselves and presenting the case for a given thing, such as a patch.

If you don't have a recorded vote then how can you know who is, as you put it "for a patch"?

As I said elsewhere, the problem with not having a vote is people can waffle, and quibble. They can equivocate and delay. A vote is about making a decision, not having a conversation. Delaying can result in patches going stale. Patches that go stale, if they are good (and even if they are not) can result in the alienation of contributors. If you think this is a good thing (and I'm not saying it definitely ISN'T) or that it is otherwise acceptable and/or serves a purpose, please explain why. When people understand why something is allowed to happen they are more likely to accept it.

There's a lot of contradiction inherent in the statements that "everyone's vote counts", and "we just don't have a metric for it". if you don't have a metric for them, how do the votes "count"? And what does "counts" mean, then?

Eschewing a vote in favor of consensus through dialogue benefits the people who have the time and the capability to engage in that process. It leaves out those who don't. Consider that there are those who don't, but whom are still likely vectors for good stuff.

I agree with your contention that "whether or not a feature gets neglected has little to do with the personal preferences of the core committers". However, it should have nothing to do with them. Whether or not that is even possible -- dunno.

I'm equally wary of bureaucratic "improvements" to the current state of affairs however it seems that in the least the conversation engendered by proposals for such could result in some discourse that leads to positive change.

Cheers.

Jose Reyero’s picture

Though I find the topic worth discussing, committees are worst thing in the world for decission taking.

I think a voting system -if someone else sends the patch I guess ;-) could be ok, provided that there's only one person that sees the votes and eventually takes the decission. And also funny, imagine.... oh, let's see how many votes gets my patch ;-)

But when it comes to decission taking, please, give me one person with a clear vission, good committment and reasonably open to other people's ideas and suggestions.

Ok, maybe it is good if there are more people, each one responsible for an area/core module and also is good to know 'who decides what'. But it should be more a top-down process, I mean, when the person in charge cannot manage all the stuff in time, he can delegate a given area/api/piece of code to somebody else.

Do you really think we would have something if this had been run by a committee from the beginning?

From my short experience as a Drupal occasional contributor, I think the system just works, and goes ahead.

Change perspective and think of a module contributor/maintainer. Let's say I contribute a module and I am maintaining it and -could be an illussion- but feel like I have some kind of control of what happens with the module.

If every patch that gets applied should have to go through such a complicated democratic system -let's not forget that democracy takes time and is expensive-, I think:
a) Some patches I need wouldn't be applied and some that would be applied could make the module unusable for me.
b) I would gradually lose interest in maintaining the module
c) I would end up.... forking 'my' own module!!! or just keeping it for myself.
.... then somebody else takes ownership of the module.... and back to step a)
z) We have a module that really doesn't fit anybody's needs and nobody will find worth maintaining, ...at least for free

No, please, let's hope not to see some patch in because 'I vote yours, you vote mine', or 'we against them, our patches, their patches, fuck them...'. It is called 'politics' I think ;-)

So I think the problems should be addressed as they come, and I really don't see the problem for now.

killes@www.drop.org’s picture

The problem with voting on patches is that you never know why people liked a particular patch. All too often people only care about the added functionality than about the elegance etc of the code. Also, people often cannot be bothered to really test a patch before voicing an opinion on it. So we would not need to have a simple voting process but a more complicated one. We would need to ask the voters if they a) understand what the patch is about b) have read the patch c) have tested the patch. Only if the answer to c) is "yes" we should record the vote...

Oh, and the list of voters should be made available to the CVS maintainers. Not all votes should be counted equal.
--
If you have troubles with a particular contrib project, please consider filing a support request. Thanks. And, by the way, Drupal 4.5 does not work with PHP 5.

fx’s picture

Conversely, the problem with not voting is that everyone can waffle and quibble endlessly, and all that waffling and quibbling is subject to interpretation, which goes on, and on, while patches grow stale, people see their work go to waste, become disenfranchised, and move on to some other project.

Which is ultimately probably fine (up to a point, anyway), but in some cases, maybe not.