Choose your Drupal

I don't think I could ever support this as enthusiastically as I wish I could due to time contraints, but I thought I'd mention that, ironically , this is the same conclusion that I came to about 4.5 years ago: http://groups.drupal.org/node/2164#comment-6111.

.

I think that we'd want the starter and plain core to be fixed entries on the page, and the rest to be the top 3 most downloaded. (Or top one in each category?) That way we avoid the trap of sponsoring any particular company's product.

.

So, sorry can't help to start photoshop around and delete thing everywhere. See attached.
I really agreed with the concept, there should be one place which is very simple stupid for anyone to download them all.

I just realized, you are into something. Our website is too big and bloat, and documentation team also look tired and burn out. we should clean our home and reorganize, standardize each category to one page.

If all of the basic math of mankind can fit in one page, why can't we? See it here:
http://www.khanacademy.org/

To my eye, the essence of everything of Drupal website boiled down to only 3 things:
Code. Contribute. Content.

This vision would be so nice. I can't stand behind it as things stand right now. There are two problems. One, who is going to build the quality products when core couldnt deliver one for so long? You say, hey, people build websites based contrib, contrib is so much more agile. Yes, but Drupal Core, even with all of its flaws stands head and shoulder above basically any contrib. What's the test coverage of Views, without which, arguably, noone will build a website based on contrib?

Let me post my action plan.

This looks really great.
Yes, there are still issues to take care of, but huge cred for getting it up in so short time.

Chx think don't call this a framework -- maybe later, maybe never , I'm so guilty have to do photoshop again. But Kernel sound nice too :)

On other aspect I don't know too, because I don't understand. Someday we will know anyway.

Subscribing.

Just posted #1255674-43: [meta] Make core maintainable. IMO we need categories from 1-4 represented in any Drupal platform (it would be nice to make things from #1 easy for people to pull out to use in other systems, but #1 should not have a separate development workflow and that wouldn't be possible for years even if we wanted one). So for me there is no real difference between Drupal starter and Drupal platform (well the starter profile might have some modules actually enabled after install).

Ideally snowman or similar will be 99.9% configuration, based on solid components in the platform. That needs to be developed as part of core during Drupal 8, otherwise we risk having to little to show people when Drupal 8 is actually released.

I have thought about something like this page being integrated into a project browser and downloader in core, so you can just install 'Drupal' then select a profile as part of that process - anything available on Drupal.org, not just the ones in core (which might still be there at the top of the list though and integrated in core as well as for people without internet connection etc.).

I have thought about something like this page being integrated into a project browser and downloader in core, so you can just install 'Drupal' then select a profile as part of that process - anything available on Drupal.org, not just the ones in core (which might still be there at the top of the list though and integrated in core as well as for people without internet connection etc.).

This is an interesting idea as well, maybe in the longer term in particular. That could also help address #1164760: [meta] New users universally did NOT understand that you can extend Drupal. In the installer, we give them a list of install profiles, and also tell them where other install profiles and additional modules are available.

The biggest issue I have with products being distributions, is that many sites require multiple products.

Could an e-commerce site have an active blog on it? Sure. Could a community site need e-commerce functionality? Sure. The combinations are endless.

Distributions are great starting points, but you essentially get to pick just one starting point.

I would rather see products being equated with something like features, where multiple modules are bundled together to add a specific set of functionality to a site. The challenge I see in this is that multiple module bundles could contain the same module, with different default configurations. I would think resolving those configuration differences would be non-trivial.

Still, I see something along those lines being more beneficial, because while some sites may have a singular focus that exactly matches the pattern established in a distribution, many more do not. Adding extra bundles of functionality needs to be easy for those sites too.

@catch This is similar to the lines I was thinking along the other day, where I was discussing how difficult it might be to build a "profile builder" for d.o. Sure, have pre-built profiles for easy quick selection, but allow a "Customize your selection" link which slides out a module/theme selection option. Where I get caught up is trying to figure out if we would offer top 100 downloaded modules, etc, in that list, a search box, both, perhaps categories.

Categories would certainly be non-trivial, as it would require contrib co-operation, which could perhaps be a good incentive for contrib maintainers to organize their modules? Tag your module, be included in the configurator, etc.

Just some ideas, but I like where this could be headed.

The biggest issue I have with products being distributions, is that many sites require multiple products.

Could an e-commerce site have an active blog on it? Sure. Could a community site need e-commerce functionality? Sure. The combinations are endless.

Distributions are great starting points, but you essentially get to pick just one starting point.

I think that concern is an important one, but it's not fundamentally about profiles or distros: It's about the level of granularity we're talking about at any given time. e-commerce and multi-user blogging are kinds of features, classes of functionality that sites can have. In Drupal we practice re-use on a number of levels: on the code level, at the "common functionality" level, and also at the Use Case level. the individual features you're describing are 'Functionality reuse,' but Install profiles live at a higher level up: use case capture.

It may be a bit blurrier since a number of the sample sites in the screenshots above are focused on one particular kind of functionality, but I think it's important to realize that Install Profiles are about kinds of sites, and kinds of use cases, not just single kinds of functionality. Because I was bored on the flight home from Drupalcon, I even whipped up an illustration:

As far back as DrupalCon Barcelona, Boris Mann was talking about the need for some "unit of functionality reuse" below the install profile and above the API/Building Block module. Perhaps one of the reasons our discussions can be so fuzzy is that 'modules' can exist at any of those layers, or just one of them, depending on how the author implemented them or intended them to be reused.

In any case, I think it's important to remember that properly isolating smaller units of functional reuse doesn't eliminate the need for a high level mechanism for capturing cross-functional use-cases and preconfiguring a site to satisfy them. WIthout the "Functionality Unit," Profiles can still preconfigure sites for use cases. But without install profiles, the "Functionality units" are basically just modules. Both are beneficial, but only the Installation Profile mechanism is explicitly necessary to provide the distro-driven approach being discussed.

sub

@eaton I like the way you've delineated the difference between a use case and a functionality reuse.

I think in theory a functionality reuse is probably most equivalent to a Feature in the current Drupal toolbox, while as you point out, a distribution.

My concern is that right now, both the functionality reuse scenario and the use case directly package up a set of modules and configurations, and it seems to me that the functionality reuse (Features) is used more internally by organizations that create many sites, rather than being put out to users as a coherent product. Most of the energy seems to be in creating use case distributions.

That works fine if every site fits a single use case: however, it’s a lot messier than that.

What I would rather see is for more of a focus on functionality reuse with sets of modules and configurations bundled together as a Feature or product or whatever this ends up being called. Use case distributions would bundle up these functionality reuse chunks, rather than directly bundling up the modules themselves.

If that were the case, then a conference use case could bundle together several different chunks of functionality, such as a schedule complete with RSVPS, and maybe a registration chunk, and maybe a few other useful chunks.

There could also be a membership site use case. Maybe it has a chunk related to membership registration and renewals, and another chunk for internal membership communication, like a listserv. And maybe more.

The important point is that if my site doesn’t fit just one use case, if I have a membership site that also has a conference, I can pull the schedule chunk and the conference registration chunk from the conference use case, and I can pull the membership registration and renewal chunk as well as the internal membership communication chunk into my site.

I don’t want to set up separate sites with a different distribution for each of my use cases. And while I could set up a dummy site with a distribution I am interested in, and then pull out the chunks I want by setting up Features, that is not intuitive or easy for most site builders. The only other alternative is to try to cobble together a number of modules on my own directly, maybe trying to scour a distribution’s documentation and issues and module files to try to figure out how to do so. Again, not very easy at all.

The key to dealing with this scenario is to beef up that middle layer, the functionality reuse layer. Use cases can bundle together bits of functionality then... and so can site builders.

I do not think it is uncommon for sites to have more than one use case. I have run into this issue on a number of sites I have worked on, and I can’t be alone in that.

I would like to see Localized Drupal Distribution in top 5 most downloaded Drupal products.

Just a little comment on this: the current Drupal 7 installation screen proposes a "standard" and a "minimal" profile. But even as someone with Drupal experience I have absolutely no idea what either of the options means.

It should follow best-practice of many desktop software installers and show the modules that go along with each package.

That works fine if every site fits a single use case: however, it’s a lot messier than that.

I agree wholeheartedly: this is why there will always be a need for site builders and developers in the Drupal world. There will always be people whose sites outgrow their original use case, who don't understand their use case well when they originally start, or those whose needs haven't previously been identified by a team that was willing to put together an installation profile.

That's where we're at today, for the most part: people install modules and assemble functionality using Drupal's internal site-building tools, or its APIs if they are developers. People who know they'll be working that way often start out with the "Minimal" install profile in Drupal 7; it gives them a bare foundation on which they can add Feature Modules, API modules, site-building configuration, and so on. That ability to use the building blocks to make whatever you like is one of the things that makes Drupal incredibly powerful!

However -- and this is the big however -- we absolutely positively do not have the current set of tools needed to give people an 'ala carte' profile builder on Drupal.org. We do technically -- in the sense that they can pick and choose the modules they want and receive a pre-built distro that activates the modules. Building that is the relatively easy part. But the critical mass of cross-compatible feature modules isn't there yet, and the standards for ensuring that feature modules don't collide with each other have yet to be fully adopted (see BuildKit). I'd strongly encourage anyone thinking about this space to at least take a quick look at The Story Of OpenDoc, an insightful retrospective about Apple's attempts to launch a very similar "build your own everything" project in the mid 1990s.

Using installation profiles as they exist today does not prevent you from building or adding more functionality and expanding your site's focus. I'm concerned that unless we remain very clear on the related but separate goals of "Features-style" configuration capture and "Profile-style" use case capture, we'll let the perfect become the enemy of the good.

If we treat profiles on d.o. as a way to provide people with a sound foundation for their desired use case, and provide a 'bare metal' profile for those who know they want something that doesn't fit the available profiles, we get big gains and the people whose use cases are defined and understood benefit greatly. (There are a lot of them out there! It's not all crazy edge cases, after all...) We can then can treat solving the 'Feature Reuse' problem as a next step: a way to improve life for the people not well served by a single profile, and a way to make assembling useful profiles faster and simpler.

Just a little comment on this: the current Drupal 7 installation screen proposes a "standard" and a "minimal" profile. But even as someone with Drupal experience I have absolutely no idea what either of the options means.

That's a serious design and messaging issue. If the question is, "What kind of site am I making?" then neither of those options are terribly enlightening. "Minimal" would probably make sense as "Create your site from scratch with Drupal's site-building tools," and the focus problems with "Standard" are why we started working on the Snowman Project to design a profile specifically geared to small clubs and groups who want to promote their organization.

"Profiles as collections of modules" will always be confusing to anyone who hasn't already broken down their desired outcome into a list of known Drupal features and modules. Emma Jane Hobgin, author of Front End Drupal and a frequent DC speaker, talks a lot in her training about presenting people with desired outcomes. In other words, for people who are trying to make sense of the system, asking what their desired outcome is makes more sense than asking them what set of modules they want. The module list may be interesting -- in the same way that the ingredients on packaged food are important -- but they are rarely the primary decision metric.

Using installation profiles as they exist today does not prevent you from building or adding more functionality and expanding your site's focus. I'm concerned that unless we remain very clear on the related but separate goals of "Features-style" configuration capture and "Profile-style" use case capture, we'll let the perfect become the enemy of the good.

I just want to reiterate this because I've seen a lot of people blurring this in the discussion and it's led to misunderstandings of sun's proposal. And eaton explained it much better than I have. :P Here, for this, we should focus on the Profiles-type part of it.

We can then can treat solving the 'Feature Reuse' problem as a next step: a way to improve life for the people not well served by a single profile, and a way to make assembling useful profiles faster and simpler.

I’m just hoping that this is something that can be worked on for Drupal 8, rather than having it wait until Drupal 9 or beyond. Beefing up Feature Reuse would make it significantly easier for site builders to build out a site that does not neatly fit into the existing use cases in available distributions. Of course you can always add contrib or custom modules to a distribution. However, when a distribution creates a feature through a complex collection of features and possibly custom code, that is not easy for the average site builder to duplicate.

I’m just hoping that this is something that can be worked on for Drupal 8, rather than having it wait until Drupal 9 or beyond. Beefing up Feature Reuse would make it significantly easier for site builders to build out a site that does not neatly fit into the existing use cases in available distributions.

Agreed -- I would argue that hard work in this area has been going on during the whole D6 cycle, with the rise of Features module and other tools like Patterns that preceded it. heyrocker's work on config management has the potential to improve it, as well. I don't mean to imply that it's not important, just that it must be seen as a parallel, complimentary set of work to the stuff going on with installation profiles.

Of course you can always add contrib or custom modules to a distribution. However, when a distribution creates a feature through a complex collection of features and possibly custom code, that is not easy for the average site builder to duplicate.

Yes, this is one of the reasons that the Snowman work has emphasized that core install profiles should only set up things that could also be built by hand using the site building tools. No sneaky special hook implementations snuck into the install profile, etc. The challenge of building genuinely reusable and remixable features is nontrivial: the django world grapples with it a lot in their 'Apps' model, which is much closer to the Features model you're talking about than traditional Drupal modules.

To reiterate: is it safe to say that this discussion about 'pick your drupal' style profile selection on Drupal.org should focus on the install profile/high level site story level of reuse rather than feature-granular reuse? I think yes.

I recalled this week from my (pre-Drupal) CMS evaluation days that I tried out EZPublish once, for a good while, as it seemed to offer a good level of flexibility (notably e-commerce amongst other things).

This doc is from 2005, notable is the section marked "Site type", which offers a selection of default profiles out of the box. This is 6 years ago. It might even have done that in a previous incarnation.

http://doc.ez.no/eZ-Publish/Technical-manual/3.7/Installation/The-setup-...

It strikes me that if people aren't aware that Drupal can be extended, we at least have a duty to defuse that situation. A simple way is as per above, but of course that is easily said, not easily done...

Further to the above, there is also a "Site functionality" section, where additional "features" can be enabled, like contact forms, forum etc. This is a nice introductory way to present module enabling/config, right at the very start of the site building process. It is effectively demonstrating the configurable aspects of the site. This is also potentially a point where we could pause the core "packaged" download install and get other bits (ex-core modules) from d.o during the install process to extend the site. These are small looking features. I think the value of them being small (single modules, sorta) is decent, as you're not getting involved in excessive amounts of inter-module dependency.

This is probably already clear, but the above is a nice reference point of what could be achieved.

Additionally, IMO Chx' point in #7 above about Views is well made.

This vision would be so nice. I can't stand behind it as things stand right now. There are two problems. One, who is going to build the quality products when core couldnt deliver one for so long? You say, hey, people build websites based contrib, contrib is so much more agile. Yes, but Drupal Core, even with all of its flaws stands head and shoulder above basically any contrib. What's the test coverage of Views, without which, arguably, noone will build a website based on contrib?

Test coverage is not the measure of quality, just a tool used to make refactoring less error-prone. The issue is not one of pitting core against contrib in a contest of awesomeness, but a question of focus. Drupal core has never had the luxury of committing to a use case, because it has to apply to all possible use cases. Thus, it is unfocused and undifferentiated in its "out of box" experience for new evaluators. You may feel that COD, for example, is not up to the standards of core. but it is without question a better event site management system than Drupal core.

The success of Using Drupal was due in large part to providing simple recipes for specific kinds of sites, and a lot of that "running start" can be replicated in installation profiles.

I wholeheartedly agree that we're in dangerous territory if we gut Drupal core and assume that good "on-ramp" experiences will simply spring up in contrib. Fortunately, we don't have to take that all or nothing approach. I talked about it for an hour in my Drupalcon Session, and we've been talking about it a lot since then. The discussion in this thread, IMO, is about how we want to present and promote the concept of pre-packaged installation profiles and "light distros". That's something good to figure out even if we do not immediately rip everything out of core, and planning this stuff before we need it will set us up for success in the future.

Actually not - the idea of this issue is to demonstrate how the drupal.org download page for Drupal would look like, if Drupal, the product, was a separately packaged installation profile.

Doh. My apologies for the misunderstanding. Thanks for the clarification, sun!

However, my personal stance is: at the point Snowman/Onramp needs or wants a product-level module to better serve its specific use-case, it would have to be a separate installation profile (next to standard/starter, and next to plain core). That is, because the very same feature could be interesting for other use-cases as well - but still, not necessarily for all.

I agree wholeheartedly. We've been talking about this a lot over in the snowman group, and we're much more interested in small, iterative improvements in the general site building tools that ship with core (making Book a bit more flexible as a general purpose manual-content-grouping tool, and the per-bundle-list-page stuff). Those things can be used to build out a large number of different specific features, and are easy to extend or replace in contrib without running into an odd, half-formed "Wiki Module" in core, or something like that.

This is absolutely amazing. Having the project that people download have stuff from contrib allows us to scale and grow much faster than how Drupal core is growing currently.

Thanks everyone I now learned a lot, especially @sun for this thread. I have a better overview of Drupal project now.

IMO This easy download stuffs will become one of the key in Drupal marketing. Drupal (both distro/modules) distribution problems are one of main causes of marketing, many marketing problem will naturally be solved too.

I hope what we are discussing are on the plan. Do this related to any going on intiative? WSCCI? If not, I hope snowman and related stuff e.g. features, kits will become one or equivalent.

It is getting interesting, seem everything now a lot related to each other. And now we also have Open App Standard.

I've been following this on the various blogs and discussions and am a big proponent of this direction. Another win that I could see coming from this is giving some people a more focused way to contribute. I imagine there are a lot of folks, project managers for example, out there who will be more energized to contribute to a full fledged "product" like a blog or commerce profile than contributing to modules or core offers. Yes there are profiles now but they don't get a lot of attention.

Interesting direction. Subscribing.

Subscribing, because I need to be reading more meta-issues.

Also subscribing: maybe we can create a meta-meta issue to track all of the meta issues?

really happy to see this discussion going on.

have we talked about whether these 'products' would be 'contrib' that anyone can add to or whether we're going to work out a manageable set of products that will serve as a good starting point for a big chunk of the types of projects that would start out this way, or perhaps a combo of these two approaches?

have we talked about whether these 'products' would be 'contrib' that anyone can add to or whether we're going to work out a manageable set of products that will serve as a good starting point for a big chunk of the types of projects that would start out this way, or perhaps a combo of these two approaches?

After a lot of consideration, I'm in favor of the Canonical Drupal Download that new users and evaluators are pointed to -- drupal.tar.gz, if you will -- containing several installation profiles. I agree with Sun's proposal that we need a top-level browseable archive of tailored profile-based Drupal products, but I still think there's a strong need for several profiles that utilize the "baseline site building tools" that are always part of Drupal.

I don't think it's either-or.

Until there is a healthy and thriving ecosystem around distributions that are maintained here on drupal.org, I don't see how we can possibly even entertain this vision. http://drupal.org/project/installation+profiles is a wasteland of abandoned product attempts, a graveyard of "Foo language Drupal" and "Foo module dev" profiles, and a gaggle of pointers to off-site downloads and separate communities for the handful of successful distros. I cannot understand the logic of kicking the very first thing that all Drupal users will see to a contrib mechanism that has yet to actually work for any substantial distribution. Maybe in Drupal 10.

Though if you're interested in attacking some of the blocking issues, I've started a tag drupal.org distribution blockers, and summarized the outstanding issues as best I can understand them in the issue summary at #779452: Whitelist for external dependencies.

Until there is a healthy and thriving ecosystem around distributions that are maintained here on drupal.org, I don't see how we can possibly even entertain this vision.

We can certainly entertain it, plan for it, and prioritize for it -- we just can't safely implement it until there is a solution to the "I downloaded Drupal, what now?" problem. I suspect it's just semantics -- there are really a handful of possibilities that have been floated:

1. Split the current Drupal rep, with framework, platform, and site-building-components in one repo while Standard Install and the list of feature-oriented modules live in another repo. First time users would receive the same download they do today, but it would be packaged differently, but it would be the first step towards a distro-driven future.
2. Build one or more product-style install profiles that only utilize Drupal Core's site-building-components and tools. Bundle them with Drupal core, then work on splitting them out to use the packaging-driven solution described in option #1.
3. Tear out some/many/all/two modules, giving us a super-tiny Drupal Core, and allow third-party distros or hosted services to fill site builder and end user needs. This is the option I think would be cataclysmic, because at best there would be a post-release slump of at least a year or so in which evaluators and users would have to dig up an appropriate 'distro' before being able to really use Drupal.

#1 has the advantage of being transparent to end users as long as the 'Main Drupal Download' is still driven by Standard Install Profile and its requirements list. #2 has the advantage (at least in my opinion) of testing our tools and infrastructure against a richer set of use cases than the currently unfocused Standard Profile, as well as potentially improving things for evaluators and new builders. #3 has the advantage of quickly and dramatically reducing the amount of code that the core dev volunteers have to carry around, but the downside of stranding folks who aren't already reasonably experienced contrib-wranglers.

There are a lot of paths towards this model that don't force the kind of traumatic experience on new evaluators that I suspect you're concerned about. An excellent goal would be the eventual shift of our "Download Drupal" page to a gallery with the "Standard Drupal Download" always featured prominently, and beneath it a small rotating gallery of Drupal install profiles that have actual supported release versions. Even if there's only a small handful of those Install Profiles, we don't need 11,000 of them to provide people with genuine and interesting choices.

Though if you're interested in attacking some of the blocking issues, I've started a tag drupal.org distribution blockers, and summarized the outstanding issues as best I can understand them in the issue summary at #779452: Whitelist for external dependencies.

There's also a fairly long list of things that makes building out long-term-sustainable installation profiles challenging: #1242448: [meta] More flexibility for core installation profiles. The packaging infrastructure is a big blocker for Acquia Commons, Managing News, and OpenPublish level distros but some of the smaller tangles in that meta-issue are even tougher. Reducing some of those hassles, IMO, has the potential to considerably reduce the implementation cost/time of many simple Drupal based product install profiles.

We can do #2 and #3 in eaton's list at the same time. Add one or more install profiles to core, while also ripping out some old crufty features that are unmaintained or no longer make sense.

If some old crufty features get polished up to be included in a new install profile, that's great. If they're still not enabled by any core install profile, time to go.

"Tear out some/many/all/two modules, giving us a super-tiny Drupal Core"

the problem here is -- to tear out real problematic, challenging modules, you would end up with something ... useless? 'Cos you would need to throw comment, node, taxonomy and user on a pyre and light it with block.

giving us a super-tiny Drupal Core

Or maybe three Drupal cores, and many Drupal distributions. For example:

1. Drupal Developer Edition (small core). Probably also including drush, and a few developer modules: coder, devel, module builder, schema...
2. Drupal Builder Edition (medium core). Similar to Drupal 7.
3. Drupal User Edition (extended core). Including some remarkable contributed modules such as Views, etc., that would keep their own maintainers. Possibly something similar to the Acquia Drupal distribution. Snowman is planning to use just Drupal core, but if there is an extended core with Views, perhaps could use it?
4. Drupal Distributions. In addition to http://drupal.org/project/installation+profiles , there is another list with the main Drupal distributions, most of them downloadable from external websites because of some drupal.org issues (external libraries, etc.). That distribution list is: Additional distribution documentation (including categories: generic, academic, business, community, conservation, event, government, hosting, intranet, magazine, media, nonprofit, performance, social, video, etc.).

subscribe

Also look at http://symfony.com/download

Nice to see some progress!

Cross-reference:
#908022: Small Core / Drupal Distro

Sub

Subscribe.

Sub

Sub

Subscribe

Seems to me the best way to present this would be to leave core as the only download and present these package options during installation as has been mentioned above. Having multiple download options obfuscates the process and may alienate users right from the start. In fact, people who are new to Drupal that are presented with these platforms might not realize that Drupal can serve as your blog AND your store.

Seems to me the best way to present this would be to leave core as the only download and present these package options during installation as has been mentioned above.

That's one possibility, but there are still some concerns I have about it.

- It makes the drupal.org infrastructure a choke point for actually installing a copy of Drupal, not just obtaining it
- It requires much more "from scratch" development work on the drupal.org servers compared to a package-and-download approach, which is already supported
- If we 'bake' the profile selection process into the install workflow, we make it much more difficult for people to build turnkey solutions that don't let people download arbitrary stuff from Drupal.org during the setup process.

Having multiple download options obfuscates the process and may alienate users right from the start. In fact, people who are new to Drupal that are presented with these platforms might not realize that Drupal can serve as your blog AND your store.

We'll need to document the differences well. If we have the "Drupal Starter" package first, and document it correctly, the majority of people will grab that, while experienced site builders will read further down the list and find a better package suiting their needs.

Jeff, your chart and explanation in #14 notwithstanding, I don't quite understand why we need a multitude of installation profiles rather than pre-packaged sets of features that could be checked off for inclusion in an initial download/installation (along with language to the effect that there may be alternatives, even numerous alternatives in contrib for how to implement some of these features).

I'm not saying, don't have a Snowman. I think Snowman (or Snowman + Views) would be a great improvement on the current standard installation.

I'm saying, I don't see the point of prominently listing links to profile downloads for Community Site, E-commerce. News Site, Wiki, etc. I agree with #16 that easily installing FEATURES (which can be mixed and matched) is of much greater utility. If I want to make a community site that sells products and/or has a blog, then I really can't tell which profile to start with and add the other features to. But if I see a list of features/desired outcomes I can check off and install, then I am a lot further down the road toward completing my site, and if I don't quite like how some of the features were implemented, then I can go exploring contrib for other solutions.

Another possibility might be a wizard that asks a series of questions about what features/outcomes are desired and then offers a choice (if a choice exists) of solutions, describing the differences between the choices, using components found in core and/or contrib. Then the wizard would generate a profile on the fly and download and install all the necessary components.

There's no reason we can't have that, other than the fact that it hasn't been built, and that building it is actually a very difficult and complex task. The value in an installation profile isn't just the combination of modules that it comes with, but the pre-configuration that it does in activating and setting up the modules for a particular task.

Installation profiles and distros based on them are in place and well-tested here on Drupal.org. There are a number of core bugs and packaging issues that need to get ironed out before they can be used for a "Choose Your Drupal" style system that is sustainable, but that list of issues is considerably smaller than the list of tasks between us and an online wizard-style interface for building a web site out of compatible modules.

I think there is a lot of value in that long-range plan, but I believe we can walk with profiles before we run with more-granular build-your-own-Drupal-Download functionality.

Sub

Sub

@cpelham:

Some Drupal distributions (NodeStream, Acquia Commons, Open Atrium...) have multiple features (using the Features module) that can be individually enabled/disabled... after installing the distribution. This is a level of detail similar to enabling/disabling modules. For now, we are talking more about downloadable packages.

However, maybe you are proposing something like a selection of "featured features" rather than an exhaustive, very long list. That could be an interesting idea, in addition to the different packages discussed here. But, as eaton suggests, interactive combinations seem more long term than the simpler pre-packaged downloads.

Anyhow, feature servers are a related idea (see the fserver project).

There was a Google Summer of Code project this summer that worked on an integrated module/theme browser in-app. Here's a demo of it working:

http://www.youtube.com/watch?v=mFDBQqTfG-8

So offering "mix and match" capabilities isn't out of the realm of possibilities. I agree that well-done distros though are much more than just "this and that and this and that module" so I think it's slightly different.

I like it visually. Not sure if we should call them "Product"

sub

/me listening

Apologies as I recently started reading about this debate/issue, it is a lot to digest.

I've been working on a Drupal music distribution (Pushtape) and noticed building a distribution takes more time and rigor and has different design considerations than building a normal site. To me a distribution is less of a "product" and more of an "application" (App?) and requires more of an application mindset. I want to limit and lock down functionality to make things stable. I need clear, well defined, bounded design goals. In this application mindset, that application framework IS Drupal to me. It doesn't matter if it's a combination of core, contrib, Features, etc the goal is still the same. It's all inside of a blackbox.

The term "product" also has some connotations of consumerism/commercialism, but not all of these install profiles will necessarily be made by or for commercial entities. Application seems like a better and more fluid term, with appropriate connotations in the software world, and will encourage developers to start thinking in terms of building apps instead of websites.

sub

@FreddieK:

Edit: Except that sun's fancy abuse of inline CSS is hiding the follow button. Whoops. :)

In general, though: http://drupal.org/node/1306444

.

OMG! I thought I'd never have to "subscribe" again.

removing link to /project/standard per http://drupal.org/node/1301806. According to Bojhan it never existed.

Hi everyone

I very much like the idea in #41 juan_g, "Drupal Developer Edition" (small core), "Drupal Builder Edition" (medium core), "Drupal User Edition" (extended core), "Drupal Distributions" (like the current installation profiles).

Going forward into the future I think Drupal Core should be easilly maintainable and easilly upgradable with little updates to the core it's self required, e.g. just adding features and functions to core, that would allow for little or no maintenance to modules to keep them up to date with the latest version of core.

Updating a Drupal site to a majour release should be able to be done in the update manager built into core along with updating the database for that magour release.

I also think that Toolbar and Overlay should stay in core as they are very nice to help navigate around the administrative interface, manage a site, and keep the site and administration seperate.

Drupal 8 should set the base and standard core for the long term future releases of Drupal.

AaronMcHale

About the possibility in comment #41, Drupal Developer Edition would be the real core or framework, maintained by core developers, and smaller than the current one.

The other two, Drupal Builder Edition and Drupal User Edition would be the basic, generic distributions or bundles, including officially endorsed modules with their own maintainers, like for example Views, etc.

Naturally, in addition to these possible, generic, Drupal editions, there are already many specialized Drupal distributions.

I don't think we need to complicate things: the debate for the past few years has been 'Is Drupal a product or a framework?". Why should we have a bunch of different "editions" of Drupal when we just need to have a separation of Product and Framework?

I've said this before, but I think we should split the framework off into http://drupal.org/project/core and keep the product at http://drupal.org/project/drupal.

This way, products are simply implementations of a framework. We can do cool architecture stuff in the framework project, and worry about UX and accessibility in the product.

Sounds fine, however remember what I said above about upgrading:

Updating a Drupal site to a majour release should be able to be done in the update manager built into core along with updating the database for that magour release.

Drupal 8 should set the base and standard core for the long term future releases of Drupal.

I also think that core modules are better maintained and managed when they have their own projects, with thier own issue tracing, and ther own release schedules.

AaronMcHale

Why should we have a bunch of different "editions" of Drupal when we just need to have a separation of Product and Framework?

That possibility of three official editions/downloads for the three main types of Drupal users -in addition to the many unofficial distributions- comes from the fact that many developers consider the current core too big, and many users consider it too small. Core developers, and those who do a lot of custom module coding, would prefer a small core/framework as a basis. On the other hand, builders and users are sistematically adding many essential modules for site building missing in the current core.

Of course, the other possibility is to have two main downloads (core and product) instead of three (developer, builder, user), given that the holes are covered by distributions and SaaS platforms.

Since the focus of this issue is "Choose your Drupal", it includes possibilities for both official and unofficial editions or distributions. A possible classification according to use case and type of user could be for example the following:

1. Generic Drupal: The main downloads or editions, for all kinds of sites.
   
   • Drupal for developers: A future smallcore/framework, ideally the one maintained by core developers.
   • Drupal for builders: Currently, this is Drupal 6 and 7. In the future, a possible main Drupal product for site builders could contain the core framework and a selection of contributed modules that would keep their own maintainers.
   • Drupal for users: New users who are not site builders yet often prefer something near to out-of-the-box solutions. There are now SaaS platforms in this line like Drupal Gardens, Buzzr, etc. but they are hosted, not downloads. A middle point between builder and user are generic distributions like Acquia Drupal and Cocomore Drupal, which -at least in the D6 versions- include core and contributed modules like Views, etc. The Snowman project is related to this idea of Drupal for users, however it is planning to try to do it using just core.
2. Specialized Drupal: That is, Drupal distributions.
   
   • Drupal for communities: Acquia Commons, Open Atrium, VoiceBox...
   • Drupal for nonprofits: Donor Rally, Open Outreach, Watershed Now...
   • Drupal for magazines: NodeStream, OpenPublish, ProsePoint...
   • Etc...

Just to bring some consensus here, I think we can all agree that there needs to be at least Drupal Core, just Drupal, with very little modules or themes, and the Drupal "Product" which contains Drupal Core along with lots modules and themes.

I think we can all agree on that.

AaronMcHale

Yes, I think we agree that core+product is a good start, at least.

Parts of the "Drupal product" could live in contrib, and be packaged up when people download Drupal. This means that the core module maintainers could make commits to those modules, and not have to rely on catch/webchick/Dries to make those commits. This speeds up the Drupal release cycle and allows the core module maintainers to actually maintain their respective modules. This is the same modal that Symfony runs under, as they package a number of different external systems with their "product" (Twig, SwiftMailer, etc).

What we have to do is follow the Drupal 8 Unofficial Framework Initiative and decouple some of the core sub-systems, cleaning up the whole system. Once a particular sub-system is decoupled, it could move over to the Platform Initiative. The Drupal.org Install Profile system supports packaging Drupal distributions via drupal-org.make, we really should use it.

Just asking, but has anyone in this thread advocating for this model actually maintained a distribution of any size before? I don't mean this snarkily, I'm genuinely curious.

By far, the hardest part of building and maintaining a distribution is not selecting a use case. Nor is it picking and choosing which modules to use. It's not even putting a ton of elbow grease into the UI to hide unwanted options and create a seamless experience for users, although that certainly takes a great deal of time as well.

It's the fact that contrib modules can get committed to at any time, including UI changes, string changes, blowing away features you depended on, etc. Security issues come up, and updating for those often breaks one of your alter hooks somewhere, requiring a burst of development in order to make sure your users get a timely release that doesn't also break their site, and/or the UI you worked so hard on.

Most of the work of distributions isn't creating cool things and product design. It's un-breaking things. It's very boring, tedious, frustrating work. Drupal core doesn't currently have this problem because the same people who maintain the product also maintain all of the code that's part of that product. So if we need a change made to module X, we can make sure that change is included in our product.

When control of the modules the product depends on is federated across multiple maintainers and multiple issue queues, managing a product becomes exponentially more difficult. You're holding the core product maintainers and developers responsible for not only being familiar with 3MB of source code and 9500 issues in the Drupal core queue today, but hundreds of MB and thousands of issues in dozens of issue queues. Since we're already burning people out with the current mental burden of core, I'm not sure how it's remotely sustainable to increase that 100-fold.

Then there's the matter of hosting distributions on Drupal.org itself. drupalorg.make files are severely limited compared to Drush make files. You can only reference stable releases of contrib. You can't reference patches. You can't reference dev releases. You can't include external libraries such as WYSIWYG editors and JS libraries. There is not a single example of a distribution that the average person has heard about that's distributed via Drupal.org's tools. So the idea of moving the thing people think about as Drupal to this model is absolutely crazy to me.

We also currently have a test suite that can tell us when we change a line in include X we just broke product module Y (and this is good not only for Drupal core, but other contributed modules using those includes). Where's the replacement for that in this model?

Basically, there has been all kinds of discussion here about academic benefits of separation of framework and product, but almost no discussion on how that would actually work in practice. It's frustrating.

Just asking, but has anyone in this thread advocating for this model actually maintained a distribution of any size before? I don't mean this snarkily, I'm genuinely curious.

As a matter of fact, I have. I maintain an intranet system for a fairly sizable company here in Idaho.

By far, the hardest part of building and maintaining a distribution is not selecting a use case. Nor is it picking and choosing which modules to use. It's not even putting a ton of elbow grease into the UI to hide unwanted options and create a seamless experience for users, although that certainly takes a great deal of time as well.

It's the fact that contrib modules can get committed to at any time, including UI changes, string changes, blowing away features you depended on, etc. Security issues come up, and updating for those often breaks one of your alter hooks somewhere, requiring a burst of development in order to make sure your users get a timely release that doesn't also break their site, and/or the UI you worked so hard on.

And this is why you don't depend on -dev releases. You use stable releases. They don't change unless you explicitly tell them to by updating your Drush Makefile.

When control of the modules the product depends on is federated across multiple maintainers and multiple issue queues, managing a product becomes exponentially more difficult. You're holding the core product maintainers and developers responsible for not only being familiar with 3MB of source code and 9500 issues in the Drupal core queue today, but hundreds of MB and thousands of issues in dozens of issue queues. Since we're already burning people out with the current mental burden of core, I'm not sure how it's remotely sustainable to increase that 100-fold.

I'm not sure what you mean. We already have maintainers for Node module. Why can't we split Node module out into it's own repo and let the Maintainers actually maintain the module? They don't need to be familiar with all the other modules. Core modules are core modules...doesn't matter where they're located. I could see this kind of model making maintenance easier because the bottleneck is no longer you (webchick), or catch, Dries, or Gabor. What we're talking about (I think) is having a bunch of different "core" repositories. One for the main "framework" part of Drupal (whenever that gets clearly defined), one for node module, one for user module, etc. http://drupal.org/project/drupal would then just become an assembly of all of these different modules. Absolutely *nothing* would change for end users...it would only be a change in our development workflow.

Then there's the matter of hosting distributions on Drupal.org itself. drupalorg.make files are severely limited compared to Drush make files. You can only reference stable releases of contrib. You can't reference patches. You can't reference dev releases. You can't include external libraries such as WYSIWYG editors and JS libraries. There is not a single example of a distribution that the average person has heard about that's distributed via Drupal.org's tools. So the idea of moving the thing people think about as Drupal to this model is absolutely crazy to me.

Core modules would still be core modules. They'd just have separate repositories and for-reals maintainers with commit access to their respective repos. The fact that there's not a big-name distro that's using Drupal.org's tools tells me that the tools have some inadequacies that need to be addressed. Moving core to a more decentralized development model would certainly light a fire to get those inadequacies fixed.

We also currently have a test suite that can tell us when we change a line in include X we just broke product module Y (and this is good not only for Drupal core, but other contributed modules using those includes). Where's the replacement for that in this model?

This seems like it should be a tiny fix. Instead of git clone /path/to/repo, run drush make /path/to/core/makefile, then run the tests as normal.

I'd like to point out that this is not an unusual development model: it's how Android is developed. Check this out: https://github.com/android All those different repositories get assembled into the final Android binaries. CyanogenMod (one of the popular 3rd party Android ROMs) also follows that development model: https://github.com/cyanogenmod

It's a big change, but I think it will be worth it.

And this is why you don't depend on -dev releases. You use stable releases. They don't change unless you explicitly tell them to by updating your Drush Makefile.

What? You've never hit a bug that hasn't fixed upstream at the time you want to release? You've never needed to add a feature to a module that the committer hasn't gotten around to looking at yet? I find that absolutely stunning. I have never built a Drupal site without running into this situation, let alone a distribution.

I'm not sure what you mean. We already have maintainers for Node module. Why can't we split Node module out into it's own repo and let the Maintainers actually maintain the module? They don't need to be familiar with all the other modules. Core modules are core modules...doesn't matter where they're located.

Individual project maintainers do not. However, the core product maintainers (e.g. Dries and catch) and integrators ("code gardeners" in the current core team) do need to be familiar with Node module and the other 50+ issue queues that would make up the core team. They have to be, because the thing that gets packaged as Drupal is the thing that 99% of Drupal users will see, and it needs to be extremely carefully maintained. If there's a critical data loss bug in the Node module queue, the Drupal core maintainers need to know about it, as does the Drupal core development team who's working on fixing bugs.

I could see this kind of model making maintenance easier because the bottleneck is no longer you (webchick), or catch, Dries, or Gabor.

Sorry, but I call BS on this point. The D7 & D8 RTBC queue has nothing in it older than a week. The core maintainers aren't bottlenecking anyone.

What is bottlenecking is the community review process. We desperately need more reviewers, and always have. But that's a good thing to need, because that's what leads to Drupal core having a substantially high code quality, which goes away when committers are able to just commit stuff without review, as they can in contrib.

Absolutely *nothing* would change for end users...it would only be a change in our development workflow.

Exactly. It's those problems with our development workflow changes that I'm raising and you're poo-pooing. :) I want to see us actually discuss them. They're vitally important.

This seems like it should be a tiny fix. Instead of git clone /path/to/repo, run drush make /path/to/core/makefile, then run the tests as normal.

Great. Please get involved in the PIFT queue then and make it happen.

I'd like to point out that this is not an unusual development model: it's how Android is developed.

All of those repos seem to be owned by the same user. So what I see there is Android essentially has forks of all of the sub-projects it cares about, and they selectively merge in things that they want and things they don't. That's exactly my point about "You're holding the core product maintainers and developers responsible for not only being familiar with 3MB of source code and 9500 issues in the Drupal core queue today, but hundreds of MB and thousands of issues in dozens of issue queues."

Also, I realize I sound overly critical about this general direction in this post. I'm not, really. It makes all kinds of logical sense to de-couple how fast our user-facing product can move from how fast our APIs they rely on can move. And in fact, as part of my day job, I'm working on efforts to unblock progress on this, like a creating listing of distributions on Drupal.org, and acquiring funding for fixing the Drupal.org tools problems.

But it drives me absolutely nuts when people look at this enticing mockup as a done and dusted solution, and then take this smug tone that anyone who's not jumping for joy about it right now must be some old crusty dinosaur.

I'm not an old crusty dinosaur (though I do definitely have more grey hair than I did when I started with Drupal ;)). I'm someone who has over 6 years of experience on the code dev team, who has managed a core release with 1,000 contributors, and who has fairly extensive cat herding experience both within and outside the Drupal community. I'm raising issues that we know are challenges in the current, centralized model and I'm asking how they will be handled in a distributed model. And we need to discuss those questions. If this is where we truly want to go as a community, then we need to understand how to get there.

What? You've never hit a bug that hasn't fixed upstream at the time you want to release? You've never needed to add a feature to a module that the committer hasn't gotten around to looking at yet?

I'm more focusing on the Core development workflow. I think we might be mixing conversations here... In any case, if there's a bugfix, I create a patch, post it, and include that patch in my Drush Make file until it gets committed upstream. Same with features. Pretty standard, except that we can't do that on Drupal.org. For core, we don't need to do that. Core get's new point releases on the last Wednesday of a month. Core modules should roll stable releases the day before so that everything is ready to be pulled together for the core release.

However, the core product maintainers (e.g. Dries and catch) and integrators ("code gardeners" in the current core team) do need to be familiar with Node module and the other 50+ issue queues that would make up the core team.

They already do. The only difference is that all of the components are mashed together in one queue.

If there's a critical data loss bug in the Node module queue, the Drupal core maintainers need to know about it, as does the Drupal core development team who's working on fixing bugs.

And why can't the Node module maintainers simply make you or Dries or catch aware of said bug? Again, I don't see how moving node module to a separate project will affect the release process: currently, if there's a critical bugfix for Node module, it delays the main core release. If Node module is in a separate repository and there's a critical bugfix, it still delays the core release.

Sorry, but I call BS on this point. The D7 & D8 RTBC queue has nothing in it older than a week. The core maintainers aren't bottlenecking anyone.

What is bottlenecking is the community review process. We desperately need more reviewers, and always have. But that's a good thing to need, because that's what leads to Drupal core having a substantially high code quality, which goes away when committers are able to just commit stuff without review, as they can in contrib.

That's not what I'm saying. The most daunting thing for me when I first started contributing to Drupal was the waiting. It was a constant wait for code reviews, buy-in, and committing. It sucked. Core module maintainers don't have any incentive to do super-detailed reviews of the modules they maintain because other than saying "Yep, looks good", they can't actually do anything about a given patch. If they were able to commit patches to the modules that they maintain, I could see that changing. I really don't think it'd be that big of a problem: most of the people in MAINTAINERS.txt are people who's judgement I would trust in deciding whether or not to commit something. And I don't think it's unreasonable to expect continued code quality if maintainers could directly commit to their modules: merlinofchaos maintains views and panels just fine, the code is very nice, and it works as expected in 95% of the sites it's installed on (There's still weird little edge cases that I've hit before, but I can say the same thing about core).

Exactly. It's those problems with our development workflow changes that I'm raising and you're poo-pooing. :) I want to see us actually discuss them. They're vitally important.

I'm not poo-pooing anything: I think we need to rethink our development workflow, and want to have a discussion about it, hence my continued participation in this thread.

Great. Please get involved in the PIFT queue then and make it happen.

I'm not going to bother if we're not going to use it for core.

All of those repos seem to be owned by the same user. So what I see there is Android essentially has forks of all of the sub-projects it cares about, and they selectively merge in things that they want and things they don't. That's exactly my point about "You're holding the core product maintainers and developers responsible for not only being familiar with 3MB of source code and 9500 issues in the Drupal core queue today, but hundreds of MB and thousands of issues in dozens of issue queues."

They are owned by the same organization (collection of users), but look at the commits. https://github.com/android/platform_frameworks_base/contributors vs https://github.com/android/tools_repo/contributors. There's some overlap, but it's a pretty distinct set of users working on each repository.

And I still don't understand what you mean by "not only being familiar with 3MB of source code and 9500 issues in the Drupal core queue today, but hundreds of MB and thousands of issues in dozens of issue queues.". It's the exact same amount of code. The only difference is that it's housed in it's own repo and has it's own issue queue, as opposed to being housed in the same repo as the rest of core and still having it's own issue queue (sort of).

But it drives me absolutely nuts when people look at this enticing mockup as a done and dusted solution, and then take this smug tone that anyone who's not jumping for joy about it right now must be some old crusty dinosaur.

The mockup is irrelevant to me, and I'm certainly not trying to be smug. I simply want to modernize our development workflow to take advantage of the infrastructure and tools that are available to us.

I'm raising issues that we know are challenges in the current, centralized model and I'm asking how they will be handled in a distributed model. And we need to discuss those questions. If this is where we truly want to go as a community, then we need to understand how to get there.

I believe that's what we're doing *right now*: discussing how this would or wouldn't help. I'm presenting my opinions about how it could help. Anyone else is welcome to jump in at any time. Even if that mockup is *not* where we want to go, I think we still need to look at reevaluating our core development workflow.

Edit: I am just going to delete my post here so as not to encourage further digression. Sorry for noise. Let's discuss frustrations with the core development workflow elsewhere, and keep this issue about actionable steps toward the goal in the mockup.

Wait, so you're advocating only splitting everything except for /modules and /themes into one repo, and then the /modules and /themes individually into N individual repos? Not trying to build a "Drupal" product that's actually useful for people? How does that accomplish anything, other than make core development incredibly difficult to follow? :\

I don't think that's what everyone else in this thread is advocating. They're advocating Drupal core == low-level libraries and APIs that only expert-level developers would download and use as a basis to hack on, "Drupal" (or in the mockup, Drupal Starter) == the thing most users will download when they download Drupal. This would include not only the "product" modules that are in Drupal core today, but also potentially much more: Views, Panels, Rules, Flag, and other modules useful to general site building, all bundled together with some really nice UI polish in a distribution, just like other Drupal distributions. That means things like handling patches in drush.make are a reality for "Drupal", just as they are for Commons or Open Atrium.

But sure, let's suppose that we just did what you're talking about, and split Drupal core into some 45 repositories: /drupal, one for each thing in /modules, and one for each thing in /themes. And we'll ignore for the moment the fact that most of the maintainers in MAINTAINERS.txt are for components (which would be part of /drupal and would continue to be "held up" by core committers, unless we really break Drupal core into more like 100 repositories) and not for modules, and we'll also ignore for the moment the fact that a non-trivial percentage of those modules are currently maintained by '?'. When you go through that list of maintainers, you'll see that there's very little overlap between one core module and another. Let's be generous about overlap, and call it only 30 maintainers across all of those modules/themes.

Agreed so far?

So now it comes time to release a new point release of "Drupal" the product, because we found a security hole in some place or another. Now, instead of one person (me) simply tagging the release that consists 100% of code committed and signed-off on by either me or Dries (two people) and that's been frozen for a few days to allow for any critical, release-blocking issues to be identified and fixed, I instead say a few choice prayers, close my eyes, and press the "compile" button which sucks in the latest stable release of those 45-100 repositories.

Surprise, surprise. There are bugs. There are string changes. There are new features poking out that our documentation isn't updated for. Now I need to track down 4-20 of those module maintainers and ask them to fix things so that core can make a point release. See http://drupalcontribstatus.com/ for a glowing example of why that simply won't work. Those are the most-used, most-depended on modules that we have as a project, and many are still waiting on stable releases almost a year out from D7's release. Those maintainers have client gigs. They have families. They have lives outside of Drupal. These folks cannot be called at 8pm the night before release and asked to commit patches they haven't had a chance to review, or roll releases that they haven't vetted yet.

So the "Drupal" product maintainers and development team—who also aren't paid, BTW—are left to deal with the fall-out. That means emergency testing and patching. That means "hacking core". That means forking projects.

So let's say we get a point release out, and by some miracle it doesn't contain any critical issues that result in a second release the very next day. Yay. Awesome. Except NOW, you need to spend the next N weeks making sure that those things you had to hack and change get integrated upstream. That means coordinating with 4-20 maintainers, making sure that your patch re-rolls get reflected in the "Drupal" project. And guess what? Before you're finished with that process, it's time for a new point release again. GOTO Step 1, but now with an incredible pile of technical debt that's ever-increasing.

And THAT is what it's like to maintain a distribution. :\

Can you understand my reluctance to thrust this workflow on the one thing that gives Drupal its very name? And this is without even touching questions like how on earth a core-wide initiative like HTML 5 could happen when it involves pinging 35 people to commit patches instead of 2.

You say:

The most daunting thing for me when I first started contributing to Drupal was the waiting. It was a constant wait for code reviews, buy-in, and committing.

And I'm saying, you go this route, and you're only compounding the waiting game. Now instead of having 2-3 bottlenecks, you have 35-60. This is not a recipe for increasing our velocity.

Wait, so you're advocating only splitting everything except for /modules and /themes into one repo, and then the /modules and /themes individually into N individual repos? Not trying to build a "Drupal" product that's actually useful for people? How does that accomplish anything, other than make core development incredibly difficult to follow?

Yes, that's precisely what I'm advocating for. Aside from the fact that Core development is already hard to follow if you only have ~1-2 hours/wk to spend on it, I think doing that will force us to disentangle our code. Technically, Node module shouldn't be any different from any of the thousands of modules already on Drupal.org. Right now, it's in core, so we've taken certain...."liberties"....with it because we can and that doesn't seem right to me.

Aside from that, not everything needs to stay in core. IMO, core should be something like CodeIgniter, which does absolutely nothing out of the box, but provides lots of useful APIs. If you want user management, turn on user module. If you want content management, turn on Node module. Etc. If we're going to build a solid product, it's not necessary to limit that product to just "core" modules (for some definition of core). In fact, I don't think it would be possible to build anything substantial with just core modules, and even then, end users would go download WYSIWYG, Pathauto, etc.

We can build a product without having a ton of stuff in core. Our product then becomes an implementation of the core APIs (the difference being that currently, our APIs and our product are currently sort of intertwined).

There's also the benefit of being able to build non-content-centric applications. At DIWD a couple years ago, somebody talked about a Twitter clone built with Drupal. At the time, they said it's probably not a good idea to do this because of the amount of stuff you have to undo first. I don't see why we can't provide a tiny framework that something like Twitter could be built on top of without having to undo everything that core does currently.

The Android developers (along with lots of other 3rd party ROM developers) follow this process. Can we see how they manage their releases and deal with the problems you mentioned? How hard could it be?

And I'm saying, you go this route, and you're only compounding the waiting game. Now instead of having 2-3 bottlenecks, you have 35-60. This is not a recipe for increasing our velocity.

First off, your analogy doesn't make any sense. 60 bottlenecks should pass 20x more liquid than 3 bottlenecks under ideal conditions. Secondly, I don't see how you can make such an assertion when there are other software projects on the internet that are doing *exactly* what I proposed with hundreds of times more code and users than we have and they're doing just fine.

IMO, core should be something like CodeIgniter, which does absolutely nothing out of the box, but provides lots of useful APIs.

Sure, that's fine, and what a lot of people are advocating for. No one here is aruging with the architectural purity of this vision.

But that's not what "Drupal" currently does, nor is it what Drupal has done for 11 years or more. So even if we split off a "Drupal Core" or "Drupal Framework" project that does that, you still need to package something called "Drupal" (or "Drupal Starter" or whatever) that will meet the expectations of users who've been using it for 11 years as a site building tool. And it's that product that my questions are based around, because we will need it, even if "Drupal core" is just an index.php and an includes directory and nothing more.

What I'm seeking here is an honest and frank discussion for us to talk about how this would actually work in practice if the vision was fulfilled. What is the workflow for the developers and maintainers of the "Drupal" product? How do we avoid it being the train wreck of a catastrophe that maintaining a distribution on Drupal.org today is? What actionable steps need to happen in order to help this vision see the light? Instead, my questions are getting completely side-stepped and I'm just getting re-iterated the architectural purity of the vision. It isn't helpful.

The Android developers (along with lots of other 3rd party ROM developers) follow this process. Can we see how they manage their releases and deal with the problems you mentioned?

This is not my itch to scratch. But I encourage others advocating for this model to follow your recommendations and find out more about other projects' processes so we can discuss what aspects of it we could bring to Drupal core development. It'd be great to see if there are some best practices we can adopt, or improvements we can make to our development process.

My first question though would be to ask how many of the people who package up Android are volunteers vs. paid full-time by Google to be release managers and deal with integration issues? But assuming it's most or all of them, as I suspect, that'd be approximately an apples to buffalos comparison. :\

How hard can it be?

Once again, the canary in the coal mine here is http://drupalcontribstatus.com/. Pretty much every single site built on Drupal depends on most of those modules. Every single Drupal person that earns money off building Drupal websites depends on most of those modules. They are as "core" as our contributed module system gets. And yet, these maintainers are actively struggling with solving tricky bugs outside of their expertise, not having enough (or in most cases, any) reviewers, not having time or energy to triage their issue queues, and so on.

Also, having written individually to each of the maintainers of those modules in order to create that "hit list" blog post, I can tell you definitively we are not operating under "ideal conditions" here, in terms of bottlenecks. Getting something done across multiple contributed projects is literally conversing with 40 separate people on 40 separate schedules. I literally can't imagine shipping Drupal 8 with something like HTML 5 and accessibility support if we moved to this model.

So help me understand how it could be made to work.

It's worth getting Dries's two cents on this vision here, as well.

I like the idea of Core v Product. I do understand that this might not be an attainable goal.

I put a lot of time into Drupal Core. I can only see the separation of modules / themes into separate projects as taking more of my time and energy, and therefore making it less likely that I contribute as much as I do.

So, where is the "What's in it for me" for Drupal Core developers who currently work across Core components? Working on both accessibility and HTML5 I can't imagine enjoying the new infrastructure much at all.

x-post, cannot re-assign to Dries

Oops. Re-assigning.

@webchick: I'm not sure this is an argument against your thoughts at all, but I'd like to point out that the way it is now, we already have to say a prayer when we install a new official release of a contrib module, especially modules that touch others like i18n or CTools or Views. They often require updates to other modules (Date, Calendar, Node Reference, so many). But now this is hashed out by the site owners in the contrib issue queues rathe then by the core maintainers. And often one has no choice but to run with multiple dev versions of modules because it can be a year or more between official releases. So a real world install of Drupal7 (with a full compliment of contrib modules) is always in a precarious state.

My point is that we already don't really supply a stable, trustable real-world Drupal7. You have to triage from day 1 and every time you venture to update anything substantial.

So maybe we just stop promising that elusive golden egg (until such time as Drupal Assoc or Acquia or some such entity can provide paid maintainers of these install packages including key contrib modules)?

Or when Core gets its monthly point release you/we are MUCH more conservative about what new releases get pulled in from contrib? Then you don't have that problem. Yes, the official packaged releases would still proceed more slowly, but at least the contrib modules that are now in core could still develop faster on their own and site owners that want to risk the instability could update to newer releases of those modules on their own, which would be easier to do than it is now since you wouldn't have to learn about git or patching etc to simply install a newer point release or dev version of a module.

If I am describing this all wrong, I apologize.

For anyone else that can't find a way to unfollow this issue since the following mechanism seems broken:

Go to your user profile and click on the Notifications tab, and then at the bottom of the page, enter "Drupal Platform" as the project name and select "None" for the send e-mail field, and hit save.

This wouldn't be necessary to post here, except this node is apparently actually an issue attached to a sandbox project, that is completely excluded from search on d.o, so it took me awhile to figure out what was even going on in spite of the "clever" hack to style this issue.

I think we might be mixing conversations here...

Indeed, a small problem with this issue is that, in recent comments, there have been two main topics discussed: One that possibly would need its own issue about different opinions on core modules development workflow. And the original topic of this issue "Choose your Drupal" proposing several Drupal packages/products downloadable by drupal.org users.

About the last points discussed on core + product and the difficulties of including a framework-type core and a selection of contributed modules in a future official Drupal download/edition/product for site builders, those maybe 30 or 40 featured/endorsed contributed modules should probably have a special status different from the rest of thousands of contributed modules.

This would include for example exposure of a collecting issue queue for those product/edition modules in the contributor links of the drupal.org dashboad. And perhaps something like the Views Bug Squad.

@webchick So help me understand how it could be made to work.

Completely with you here, Angie. There's so much going on in this discussion, making it difficult to understand the scope and put together an action plan. This is why I'm unsure why Dries has to get involved at this point. He already stated his thoughts at #1224666: [meta] Unofficial Drupal 8 Framework initiative:

@Dries Generally speaking, I'm fully in support of "decoupling core" (better separations), but not necessarily of "unbundling core" (moving things to contributed modules). If a specific question needs answering, I'd be happy to try and answer them and to help us make progress. Just re-assign the issue to me along with your specific question(s).

Given that, it seems like we already have an action plan before any of the above discussed can take place:

1. Framework Initiative

"Decouple core" by cleaning up things in core/includes/randomfiles.inc.

2. Snowman Project

Expose what is and what isn't possible with Drupal core alone, allowing us to #1273344: Establish heuristics for core feature evaluation.

3. Platform Initiative

Once those heuristics are established, evaluate "unbundling core" and help make managing a Drupal distribution easier.

webchick wrote:

They're advocating Drupal core == low-level libraries and APIs that only expert-level developers would download and use as a basis to hack on, "Drupal" (or in the mockup, Drupal Starter) == the thing most users will download when they download Drupal. This would include not only the "product" modules that are in Drupal core today, but also potentially much more: Views, Panels, Rules, Flag, and other modules useful to general site building, all bundled together with some really nice UI polish in a distribution, just like other Drupal distributions. That means things like handling patches in drush.make are a reality for "Drupal", just as they are for Commons or Open Atrium. (...)

Once again, the canary in the coal mine here is http://drupalcontribstatus.com/. Pretty much every single site built on Drupal depends on most of those modules. Every single Drupal person that earns money off building Drupal websites depends on most of those modules. They are as "core" as our contributed module system gets. And yet, these maintainers are actively struggling with solving tricky bugs outside of their expertise, not having enough (or in most cases, any) reviewers, not having time or energy to triage their issue queues, and so on. (...)

So help me understand how it could be made to work.

Yes, that's a real problem for the goal of core+product to fulfill the needs of both developers and site builders. But it seems that it's a manageable problem, because the moderate number of contributed modules featured in a future main Drupal product should of course have a special treatment, with much more exposure to patch contributors, etc.

So, if perhaps in 2014 there is a "Drupal Core 8" (framework) and a "Drupal Standard 8" (for site builders), or any other name like editions, etc., in that case surely there will be a flag for the "standard" contributed modules or "DS modules" (Views, etc.), to facilitate for example a collecting issue queue like "12 Critical bugs (DS8)", in addition to the current core "13 Critical bugs (DC8)" in the contributor links.

Also, since it would be a package for site builders -like Drupal 6 and 7- and not for final users, it's in fact a simpler case than large distributions like for instance Acquia Commons. In the case of Commons (with Drupal core, and about 20 features and 70 contributed modules) there is a list "PATCHES.txt" in the contributed modules folder of the profile. Currently, it references 23 fixes from the issue queues, which they apply for bugs in 13 contributed modules (D6) that have not yet been fixed by the module maintainers.

In the case of a Drupal Standard edition with a selection of contributed modules, it would be a really happier case, because of course there should be a small team of remarkable developers with power to commit patches to contributed modules needed for Drupal releases, in addition to the module maintainers, given the critical importance of those selected modules. That is, bugs would be fixed directly in the essential Drupal modules instead of patched in some distributions. And much more quickly than now.

So, these are just a few possible ideas, as a rough draft of how it could be.

For example, like this (with some optimistic numbers):

In relation to this issue, there are some interesting views on "content management platform = content management framework + tools" in NodeOne's article Separation of Drupal the platform from Drupal products.

Just restoring the assignment, since Dries wanted to take a look at this issue.

See you in http://drupal.org/project/issues/search/?issue_tags=drupal.org+distribut...

webchick wrote:

And yet, these maintainers are actively struggling with solving tricky bugs outside of their expertise, not having enough (or in most cases, any) reviewers, not having time or energy to triage their issue queues, and so on.

About this point on contributed modules, something that could help is Michelle's issue #1244152: Create a place for contributor support

Here's one to add to the list: Drupal Forum.

I'm a bit late to the party, I see, but I have a different class of suggestion for the initial download page. I wonder if people should be encouraged to download something more like Drush than Drupal itself, to build a local site and have the tools to upload it to a live site. Those who want to play online (who doesn't?) without commitment might do better to start on Drupal Gardens.
Drush is getting more and more powerful and making the devel - staging - live business easier for advanced users. Of course in its present form it is much too scary for beginners. If you could run on your local dev site in Drush something that behaved like Gardens and allowed you to choose your "Drupal Product" at that level we might make life simpler for inexperienced web devs in the long run.

I'd only call myself an intermediate user at best and I have certainly done some things previously that now cost me more time as I struggle to get a slicker approach.

Afowle, our intention here is to welcome begginers, and show them that Drupal is sweet.

We don'y want to scare them do death, and say that the best way to do stuff is on the shell, that's the way pros do it so they better learn!

I know this is issue is set to 'postponed' and I believe that is the right thing to do. However, I've finally have found the time to comment on this and I still wanted to share my thoughts.

I believe in the vision of having distributions and have promote that since 2006 (see http://buytaert.net/drupal-distributions). Drupal distributions have great potential -- turnkey solutions help us compete in new and different markets, something that could help Drupal become a significant player.

I believe in core not getting in the way of people building distributions, and being fully pluggable and extensible.

I believe in a cleaner separation between the “product” and “framework” sides of Drupal: looser coupling, better and more consistent APIs.

Where our visions collide, is making Drupal core smaller.

I believe that core needs to provide more, and not less, infrastructure for people to build websites and distributions, including WYSIWYG editors, workflow modules, etc.

I want to add more to core so that it provides more broadly-useful features out of the box, and so that there is a more consistent user experience across multiple distributions without them each having to reinvent the wheel.

There is a risk involved with distributions as well, which means that we need to approach them the right way. The risk is fragmentation, and it is why I feel it is important that distributions build on the usability patterns set by Drupal core.

If by enabling more distributions all we do is create a lot of inconsistent niche products, some of which will be competing with one another, the Drupal project will lose momentum. If we can build competitive distributions that strengthen and accelerate the shared Drupal brand by using shared design patterns and user experience, we'll win.

Dries: Agreed. Core needs to provide more *plumbing*, and more *pieces of porcelain*, but less *pre-made sinks* (to abuse the Git terminology). It should be up to distributions to build the kitchen sink, and there's no problem with core including a sink as long as that sink is built using that plumbing and porcelain.

Unassigning as Dries commented in #100

Added "stalking sun" into my interest list. This guy is great :-)

Build Custom Download

I've noticed many new Drupal users complain that certain features are not part of core.

I believe there's a better way of connecting new Drupal users to the features/modules they seem to be expecting are in the default distribution.

On the Drupal download page, we could have a mix-and-match "Build Your Download" feature similar to http://jqueryui.com/download that lets you package your download with contrib modules that offer certain features.

As oppose to Drupal distributions, a "Build Your Download" page is highly dynamic, flexible, and more precisely connects new users with the features they're looking for.

We can have both "Build Your Download" and "Choose Your Drupal" (i.e. Drupal distributions) - these two improvements do not conflict.

Updated issue summary.