The problem

Drupal core is not maintainable anymore. Too much cruft. Too many half-baked features that no one actually maintains. This causes the following issues:

  • Some core features compete directly with better solutions in contrib, so people don't use them or they do and their expectations aren't met.
  • Having a large number of features in core means that any refactoring often has to touch a lot of code (including ones trying to decouple things).
  • Several core contributors are getting burned out fixing things that 1. they don't use themselves 2. They don't believe many other people use either), especially in the context of what is currently a very, very high bug fixing workload.

The solution

  • Define heuristics/criteria for deciding which features actually belong in core, this is happening in the #1273344: Establish heuristics for core feature evaluation issue.
  • Start applying those heuristics to specific features in core to see how they apply - this is being done in separate issues and in 'lists' posted to this issue. Please do that only in the context of developing the heuristics, kill vs. keep is not that helpful.
  • Find maintainers for features in core that are unmaintained or poorly unmaintained, but which meet the criteria for "things that belong in core". Make sure there's a plan somewhere to fix bugs in those features or bring them up to date. Set deadlines for some of this and actively look for people.

Original report

Drupal core is not maintainable anymore. Too much cruft. Too many half-baked features that no one actually maintains.

here's only one way to get it back under control: Rip out everything that does not have to be in core.

Attached patch removes the most obvious cruft, to be maintained in contrib by folks who care, and can be pulled back into a Drupal product:

However, several core component maintainers already mentioned that the following other components are equally not maintained in core, and thus should also be moved into contrib:

Removal of these is not included in the patch yet.

A separate discussion is going to be required regarding Node, Comment, and Taxonomy.

Update: while issues are not a democracy, someone is trying to maintain a tally of opinions expressed in this issue at https://docs.google.com/spreadsheet/ccc?key=0AjtNrWLTvACJdEJicFIwVXVoT1F... which give a summary of the actual consensus on what should or shouldn't be removed.

Files: 
CommentFileSizeAuthor
#109 1- Welcome.jpg108.18 KBClean
#109 2- SIte Name.jpg140.85 KBClean
#109 3 - Tagline.jpg144.86 KBClean
#109 4 - Theme.jpg177.61 KBClean
#109 5 - Add Block.jpg191.67 KBClean
#109 6 - Options.jpg136.98 KBClean
drupal-framework.remove-cruft.0.patch943.48 KBsun
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch drupal-framework.remove-cruft.0.patch. This may be a -p0 (old style) patch, which is no longer supported by the testbots.
[ View ]

Comments

sun’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Updated issue summary.

sun’s picture

Issue summary:View changes

Updated issue summary.

berenddeboer’s picture

Project:Drupal core» Drupal Platform

+1

let's take the idea of distributions seriously, and have a "standard" distribution with this stuff, but remove the above (except search) from core.

Dave Reid’s picture

Creating one giant patch to remove everything is counter-productive at this point.

sun’s picture

@Dave Reid: It is productive, because it had to be done. At this point.

berenddeboer’s picture

What if the patch contains a "Drupal CMS" profile?

mansspams’s picture

+1 here too. better now (relatively early) than later. I think "opensourcing" every single of these modules would just make them better and would speed up core development.

jide’s picture

Yes, let's make Drupal core something compact, clean and robust !

effulgentsia’s picture

Are you suggesting we support no upgrade path from D7 to D8 for D7 users using any of these modules? Or are you suggesting that these modules will somehow get maintained in contrib, even though they're not being adequately maintained in core? Do we hold up an 8.0 release until all these new contrib modules have stable D8 releases, in order to keep our traditional promise of supporting an upgrade from the last prior release of core to the new one?

How about we split this issue up into manageable chunks that can be done while preserving an upgrade path? Any chance of getting enough momentum to get Views into core, or somehow else ensure that a stable release of it is available when 8.0 is released? Perhaps then we can remove Blog, Forum, and Tracker modules entirely, replacing them with views.

Any other ideas on how we can get to where we want to be, but in a way that makes sense?

IceCreamYou2’s picture

What if during the upgrade path to Drupal 8 we automatically detected if modules that were moved out of core used to exist, and if they do, automatically download and install them from contrib?

catch’s picture

Are you suggesting we support no upgrade path from D7 to D8 for D7 users using any of these modules? Or are you suggesting that these modules will somehow get maintained in contrib, even though they're not being adequately maintained in core? Do we hold up an 8.0 release until all these new contrib modules have stable D8 releases, in order to keep our traditional promise of supporting an upgrade from the last prior release of core to the new one?

I'm not sure this argument holds for all these modules, only a few - we provide an upgrade path for data (or alternatively leave it unchanged in the database to be dealt with later), but we have never supported a complete backwards compatible upgrade path for code, features or configuration.

I'd also point out that we still don't support a working upgrade path from Drupal 6 to Drupal 7 even for critical user data, and that is partly down to the loss of focus in core development and the addition of features on a massive scale.

We are still finding serious issues even now, and there has not been more than a week without a critical upgrade path in the queue for about two years, see #1239370: Module updates of new modules in core are not executed for just a recent example. Drupal 7.0 haemorrhaged user data , it took six months to get a fix committed, and the process of doing so nearly drove me off core development completely. This discussion (or at least the wider discussion this issue stems from) is about never allowing a core release to happen like that again.

In terms of supporting an upgrade path, we should look at actual impact rather than where things are located. Over 200,000 Drupal 6 sites use CCK module, but we did not provide an upgrade path from CCK to Field API in core nor wait for one to be ready before releasing.

Compare this to the usage statistics for themes that were removed from core on the condition that someone volunteered to maintain them in contrib.
http://drupal.org/project/bluemarine
http://drupal.org/project/pushbutton
http://drupal.org/project/chameleon

(it's ~150 in total). Some things should just be allowed to retire gracefully.

Some features were removed from Drupal 7 but don't have 1-1 replacements in contrib yet. I can think of 'access rules', taxonomy synonyms (both removed on purpose), RSS enclosures (removed by mistake/as a side effect) etc. there have not been massive outcries about any of these. These were mostly also admin-facing features rather than site visitor things.

So in terms of prioritizing support I would focus on forum, profile, aggregator that actually hold irreplaceable user data in terms of keeping promises, this isn't the case for toolbar, overlay, PHP module, trigger (as examples, there are more modules in both categories).

How about we split this issue up into manageable chunks that can be done while preserving an upgrade path?

This has already been done, with some issues open for years. I gave up trying to clear out entire modules about three years ago, because it is usually a fruitless task, but I still think in principle we need to stop holding onto some of these these historical accidents in core as if our life depended on them and be prepared to chuck them out of the hot air balloon before it crashes.

Any chance of getting enough momentum to get Views into core, or somehow else ensure that a stable release of it is available when 8.0 is released? Perhaps then we can remove Blog, Forum, and Tracker modules entirely, replacing them with views.

Blog module has special breadcrumb handling, a content type and it's own permissions. Forum has similar (mainly the wonky Taxonomy dependency and {forum_index}). Tracker module now includes the denormalization from tracker2 module in core now.

So there is no way these can just be replaced with default views if we want to maintain feature parity. There are ways the underlying APIs could be improved to 'hollow out' these modules, and that may include moving some things to Views, but there's lots of other stuff to do there as well. I'd rather focus on hollowing out node, user, comment, taxonomy further with a proper Entity API and improving Field API integration than deal with the extra complexity Views would bring to core at - least until the entity and field APIs are a lot more mature.

geerlingguy’s picture

Subscribe, and yes to #14, but no to removing Search, Statistics, and Tracker (maybe Tracker, though...). These three modules are quite helpful on smaller community sites.

oskar_calvo’s picture

@geerlingguy

but there are not good for blogs, enterprises websites, or other kind of websites.

the core should be small, with the required elements to build whatever think.

You are thinking about you product "small comunities".

Oskar

wojtha’s picture

Subscribing and agree.

I agree even with the removal of the Openid despite I'm its core maintainer... but the core committing process is driving me nuts. I spent more than 100 hours in the Openid issue queue during last several months mostly with my 4 major/critical patches which all are on the edge of the security issues. Just one was commited during these 7 months of effort (!!!) to get all these fixes in... Especially Drupal 7 OpenID implementation is broken for all providers, it works only "by accident", using completely wrong or stripped identifiers, which are still in most cases unique, but not safe in the longer time interval...

chx’s picture

We want features in core that give you a huge bang for the buck -- biggest impact vs burden. Let's see.

  1. Aggregator. Aggregator allows pulling in data from various sources. That's something many sites need. Keep it. I suggested (years) earlier to replace parts of aggregator and comment with a very small, micro entity. Let's drop our own XML parser and use something better, likely SimplePie. After these two, little remains of aggregator.
  2. Blog. Was there a time, ever, when someone wanted to host a multiuser blog based on this module? That's a good joke. This module buys us extremely little. You can move it contrib, it will die but noone will cry over it.
  3. Book. This actually has a chance of surviving in contrib. If pwolanin is willing to step up to be the owner of it, I have little doubt it won't die. All the UX work never touched it so it doesnt seem to be a focus for those either.
  4. Color. On the "bang for the buck" scale -- while it certainly could be better, there are not many actual bugs with it but the bang is huge. From what I hear, people love the easy cheap quick customizing of their site this gives em.
  5. Contact. This has 16 bugs and tasks open. I do not even remember having headaches with contact module ever. There is some bang but we spend practically nothing on it so I vote on stay.
  6. Dashboard. Oh dear. I can haz something working? Do not move to contrib just delete. The real issue mending the morale of those poor suckers who tried to fix it.
  7. Forum. This needs to be poster child #1 for trying to find and nurture maintainers around it and then throw it out.
  8. Overlay. 52 issues. Everyone loves to hate this but combined with contextual links i am really wondering how big the bang for the buck is. Needs a separate discussion with a level headed calm demeanour :).
  9. PHP. If you use PHP eval, you do something utterly wrong. Seriously. If you can write PHP, implement a hook. RIP. We can move to contrib as a token gesture but I wonder who will pick this up.
  10. Profile. /me goes behind the barn and comes back without profile.
  11. Shortcut. 29 issues -- that's more than trigger. I am not too convinced about the bang either. Is this intuititive? used much? need UX folks to report on it. Separate issue but I think a trip to contrib will be the result.
  12. Statistics. RIP. I have nothing to say. Even small sites can use google analytics or whatnot.
  13. Toolbar. Unless we replace with something like admin menu we need it. With 20 issues we can survive with it.
  14. Tracker. For a module which has 4 issues this gives us more than enough bang to compensate.
  15. Trigger. Is there anyone who still thinks this deserves to live? Don't bother with contrib even. OK, it triggerred the era of core testing, thanks but it did because it was broken when it got in and was abandoned.
bojanz’s picture

Keep overlay, toolbar, shortcut. They provide the D7 look and feel. Keep contact. Shoot everything else.

The "product" modules in core are, have been and will be dead. The contrib world is much healthier.
And if they die in contrib, that's just indicative of their quality. It's survival of the fittest there.

mlncn’s picture

+1 To this plan and chx's list.

Note: A search API plus a decent functioning implementation is necessary for Drupal. Everything else listed here should be kept only if there is a commitment to maintaining it in core, which as wojtha notes is harder than in contrib.

Bojhan’s picture

This issue obviously started before we discussed what our framework and product separation should mean. It's hard to give feedback on this in a heartbeat, because we need to make more informed decisions on what people actually use. I am quite uncomfortable making this evaluation with actual data or strategy what is needed for a core product. But given that much of that is a different discussion, I am giving specific feedback on the question asked.

I am looking at the list from a product perspective, where I prefer to leave "nice-to-have" functionality out when the UX is horrible or superior in contrib.

  1. Aggregator. It seems to be an often used feature, but I do wonder how often its used - because its use is limited.
  2. Blog. I feel even for the core "use-cases" this not a justifiable feature, the UX is worse than perceived because its hard to figure out what it actually does for you.
  3. Book. Creating content with a structure (not terms), is actually a very core CMS feature. We should spend more UX resources, because its features are hard to discover.
  4. Color. People simply love this, from all of the testing we have done in the past years anything that touched this page got people excited. There are a few UX issues, but they are easy to overcome
  5. Contact. From a product perspective, this is quite essential - the ability to contact whoever is making the site. I do wonder certain functionality in there
  6. Dashboard. Let's be clear here, I never liked it going in. I always loved that Drupal didn't have a Dashboard, because its often the most useless page in a CMS. Workbench is showing us a better UX in contrib, the thought that contrib creates beautiful dashboard blocks is a sad illusion. I have chased a couple contribs and even supplied mockups, I think its clear that we simply don't have the contrib community that wishes to pursue this.
  7. Forum. I am on the fence about this one, there are a few UX issues with it - but Drupal always positioned itself to be "community/social" orientated.
  8. Overlay. I think there are few in this queue who use this module, or have client experiences of usage. From the UX testing we have done so far, it seems to be a very succesfull pattern. Additionally from community members who have clients using it, they are also presenting us with positive feedback. It is important to remember that the Overlay, just like the other items are not just "features" they define our brand. It is what creates the first impression, and makes people recognize Drupal.
  9. PHP. No idea why this is in, I am probably not a good one to answer this question.
  10. Profile. Already vouched for removal.
  11. Shortcut. We need more feedback on how useful this is, obviously we got good initial feedback on it. But other than the discover ability of the create content link this serves little purpose in core. I do see in the future a possibility that this is removed in favor of improvements to toolbar. Additionally, the UX of this module is really screwed.
  12. Statistics. We have attempted to make this more useful, but it didn't even create useful statistics in 2007. The Dashboard wanted to use this but couldn't because its so useless.
  13. Toolbar. I believe this is more critical than any other module, it is what enables navigation. Given that its a first iteration there are many improvements to be made, I believe the linear approach it takes to the IA requires more experimentation.
  14. Tracker. This is in the same realm as the Forum module, there are really only a few UX issues and is part of our "community/social" product mantra of a while ago.
  15. Trigger. Remove, it serves no workflow purpose because people cannot figure out the UX.

I do still wonder how useful this feedback is in isolation to the "define our standard profile product" discussion, which I feel people make a far more scary discussion than it needs to be. Let me know if you need additional feedback.

Damien Tournoud’s picture

I am largely in agreement with chx list. I would like to see Blog, Dashboard, Forum, PHP, Profile, Statistics and Trigger go away.

That said, I would like keep Book, and refator it, Taxonomy and Menu based on a single "Parent entity" hierarchical field.

cweagans’s picture

I'll chime in here too:

  1. Aggregator: It's a nice feature, but could really use some love. In my opinion, it could be removed from core in favor of using contrib Feeds module. If this stays in core, the fixes that chx proposed would be critical.
  2. Blog: Why anybody would use this module is beyond me.
  3. Book: This is a good feature to keep. Structured data is a fairly common need.
  4. Color: Personally, I don't see the need for this. That said, I also am able to create subthemes very quickly, whereas new Drupal users may not have that luxury. I'd say keep this one, especially in light of the improvements from D6->D7
  5. Contact: Great feature. Easy to maintain. No reason to get rid of it.
  6. Dashboard: FML. Why this got into core is a mystery to me. Please leave this functionality to contrib.
  7. Forum: While it does facilitate a community website, the forum module is the red-headed stepchild of core developers. If this stays in core, it needs a really aggressive rewrite. In my opinion, it should be removed from core and integrated with advanced forum or Artesian.
  8. Overlay: Initially, I didn't much care for this module. However, as I used it and got it deployed on client sites, I grew to enjoy it quite a lot. I'd say leave it.
  9. PHP: If you can write PHP, you should be writing it in a module. One thing that is useful about this module is the "Use PHP for block visibility" feature, but there's no reason that cannot be moved to contrib
  10. Profile: It's already deprecated, let's remove this.
  11. Shortcut: A couple of my clients have mentioned that the shortcuts bar is very convenient. My vote is for keeping it.
  12. Statistics: There's no reason to ever use this module. It's slow and useless. Google Analytics FTW. Lose this one.
  13. Toolbar: Toolbar is a good addition to core. I think that it might be better if it were more similar to the tried-and-true Admin Menu, but the current implementation is a good start.
  14. Tracker: I'm not sure that this belongs in core - while it's function is useful, it doesn't seem like something every site will need.
  15. Trigger: With things like Rules in contrib, and the plethora of hooks available in core, Trigger is no longer needed.

While we have previously said that core is "Community Plumbing", I think that we should consider viewing it as a platform upon which communities can be built using contrib modules + custom code. Including a boatload of modules in core that may be needed for building a community does not seem like a good solution.

Clean’s picture

Could 13.Toolbar & 11. Shortcut be replaced with limited function/permission of Admin Menu?

This is Toolbar with shortcut: http://drupal.org/files/issues/d7-admin-toolbar.png
This is Admin Menu: http://drupal.org/files/issues/d7-better-toolbar.png

It seem Toolbar with shortcut look like fixed position, cutted down version of admin menu with some more customization from the front end. Not sure about the benefit though.

sun’s picture

To clarify:

As already mentioned on http://www.unleashedmind.com/en/blog/sun/crisis-conclusions

The intention is NOT to just drop. The intention is to free core developers from product constraints and responsibilities.

The intention is to focus on core architecture. And to turn Drupal into a platform. For many products.

» Choose your Drupal

berenddeboer’s picture

chx, I think you didn't really answer the question of Drupal core, but you tried to come up with a distribution.

On the modules:

  1. Aggregator: have used it only once. I'm sure hardly anyone uses it.
  2. Color: again, this really is a distribution thing, it doesn't need to be in core.
  3. PHP eval: I'm sure this will be maintained, it's the only way to get forms/tables/created stuff on a page with normal user content. Very handy if you need it.
  4. Statistics: how are we going to implement most popular content without it?? Sorry, needs to be in core.
renat’s picture

>I would like keep Book, and refator it, Taxonomy and Menu based on a single "Parent entity" hierarchical field.

I think, it would be even better to make this modules (will they be in core or in contrib) use more generic approach to hierarchy creation, something like it is in Relation project. Hope this kind of functionality could be counted as "core", because so much modules could standardise on it and not to create it's own incompatible ways to make relations.

catch’s picture

Here's mine. It'd be good to move some of this feedback into a table with keep / drop / needs maintainer columns or something.

  1. Aggregator: I agree with chx and cweagans. I don't care where it lives, but if it's in core it needs a maintainer (liked sun's idea of setting deadlines). Right now we have directly competing solutions between contrib and core for something very self-contained and relatively simple, that is not healthy and has been the case for years. That needs to be resolved either way. Also this has active major bugs in the queue so definitely has a maintenance cost.
  2. Blog: time to let this go and I've been pointing out how confusing it is for about three years now.
  3. Book: Agree with Damien we should refactor this along with taxonomy hierarchy and menu (and I'd add comment threading to the list). There is a question whether we need both menu module and book but that's not a question asked by this thread, and currently book module wins in some areas over menu.
  4. Color: core themes use this so even if people wanted to, it's impossible to git rm it without breaking stuff. That is a definite line that says it needs a lot more thought and discussion. Also it's a user interface element rather than a feature in itself.
  5. Contact: not much maintenance but I'd love to separate out user contact forms to contrib and just keep the site-wide ones.
  6. Dashboard: yeah let's kill it and fix the blocks system instead.
  7. Forum: advanced forum is likely not going to exist at all in D8 and that is the only thing that has made core forum bearable since about 2007. Instead we'll have a complete fork in Artesian. Same as aggregator we should not have a limping module in core and a fork in contrib - so if people rally around Artesian we should deprecate forum for that, or move Artesian into core - but has to be one of these.
  8. Overlay: I don't like it and it has been a pain for maintenance (not sure how bad it is now), if other people are going to maintain it I don't care though.
  9. PHP: the filter 100% needs to go. The php-eval for admin stuff I'm about 90/10 on but happy to kill that if everyone else is too.
  10. Profile: we have the DIE! tag for a reason
  11. Shortcut: someone showed me last night that the toolbar actually works and looks great without shortcut, was much nicer.
  12. Statistics:the view counts in statistics module are only a part of what it does. If we really want to keep view counts in core (for most popular) we could isolate that functionality and remove the rest (or most of it). I think these are hard to replace with analytics or awstats - but anything directly competing with those should go at this point.
  13. Toolbar: didn't like it much when it went in but it's grown on me and people use it. Core should provide some kind of administration interface and toolbar makes the current one bearable.
  14. Tracker: it's low maintenance and there are sites that use it, but it also doesn't do much. abstain
  15. Trigger: time to go.
mansspams’s picture

Please do not post "your lists", I think this issue is not about what to remove and what to keep. Everything will be kept (in contrib). There are just 2 things to discuss - how fast can we remove bloat and how to make contrib accessible for new users (better module discoverability, better info on what works with what, reviews, ratings, bookmarks, distributions, install profiles etc.), but that's another issue

catch’s picture

this issue is not about what to remove and what to keep

That's exactly what it's about. Let me quote the opening post:

Rip out everything that does not have to be in core.

Attached patch removes the most obvious cruft

There are at least the following categories of stuff in core, at least this is how I think about it roughly. While any specific API or feature in these categories may or may not be essential, unless core drastically changes its offering and focus, it's essential to have at least some functionality at each of these levels.

1. Essential for anything else to work (database layer, cache system, sessions, locking, queue, string sanitization, logging, other things that mainly live in /includes). These generally have some kind of corollary in PHP frameworks although they are not organized as such in core.

2. Essential to call Drupal something beyond a strict framework - the entity API and field system are good examples, also token, contextual links. These are the APIs that site building tools are built from, and can also be used directly by developers. Do not provide either user or admin-facing features by themselves, may provide scaffolding like UI elements (vertical tabs or similar, or the token browser).

3. Essential to allow site builders to do stuff: Default themes, default /admin provided by system module, Field UI.

4. Essential for having 'a website' - specific entity types like nodes, users or taxonomy, certain field types like text, image, organisation tools like blocks and menu modules. At this point it includes things that site visitors can see (like node/comment forms and listings).

---
5. Essential to serve particular kinds of site or use cases (poll, tracker, forum, profile).

Within those 5 categories there are some sub-layers, and this is not at all a comprehensive breakdown. I'm sure people would disagree with some of the distinctions I've drawn, there are some very blurry lines and edge cases between #1 and #2 at the moment, as well as #3 and #4 and #4 and #5.

However there is definitely stuff in #5 is quite clear cut - many of these modules depend on things in the #4 category for a start, and this makes them by definition more peripheral when there's a strict dependency. A lot of them are also in really bad shape.

This issue is about clearing out as much stuff as possible from the 5th category to make room for cleaning up and defining everything in 1-4 a lot better.

That means two things need to happen.

1. We need to make sure stuff we consider 5th category actually belongs there and not #4 (i.e. people have made cases for aggregator and book module belonging in #4 or at least being half way). The edge cases are important.

2. We need to come to an agreement on exactly how much in the 5th category we can get rid of (as soon as possible), bearing in mind that:

- there is nothing to replace these yet in terms of 'stuff you can do with Drupal core', just ripping them all out will make core look a bit sad at least at first.
- snowman is like to be that replacement, and may need some functionality in this category, but we don't know what that is yet (or if it will map to any of the existing functionality).
- some modules have perfectly viable alternatives in contrib (in some cases better than the core offering) so we may not even want to move them to contrib (or only do so nominally).
- Some have no viable contrib alternatives and there is a general community interest in ensuring they are maintained somewhere (for example Drupal.org and many sites use forum module).

One-off attempts in the past to remove specific modules from core have met with a great deal of resistance, not least from Dries and webchick themselves. However recent discussions point out that the sheer effort in maintaining all this is killing us - both in terms of workload and morale, so a co-ordinated effort like this and the general conversation is extremely important to have. But it requires defining where these boundaries are, what is contributing to the maintenance burden the most, what things people actually like and use - which is what the individual lists are trying to do.

aspilicious’s picture

There is a problem with shortcut as it is tightly coupled within overlay and some themes probably. If you're going to remove shortcut from core, it can't be installed as the theming will be gone. Something to think about.

kika’s picture

AFAIR the story of aggregation in Drupal:

D6 aggregator (in core) -> FeedsAPI (better, modular agggregator in contrib) -> D7 aggregator (modular, loads of ideas from FeedsAPI) -> Feeds (contrib, from ppl realizing D7 core aggregator is not good / expandable enough).

It's time to stop those parallel tracks and declare Feeds the winner of evolution.

franz’s picture

+1 do chx.

As already said by chx, shortcut + toolbar + overlay (and others of the kind) shouldn't be hard to maintain, so it'll be easier to attract new people to do so.

techczech’s picture

"The intention is NOT to just drop. The intention is to free core developers from product constraints and responsibilities. The intention is to focus on core architecture. And to turn Drupal into a platform. For many products."

As a Drupal site builder and evangelist but NOT developer, I actually love the idea of freeing core developers from maintaining these modules and also freeing these modules from core developers. I've been using Drupal since 4.6 and haven't used Forum, Blog, Aggregator, Statistics or Profile since 5.0. Mostly because they're just not very good. It took me a while to realise this so getting rid of them will only help new Drupal users with this. I only build small websites and love Search and Contact (even though here I would use Webform now). Book should become much more prominent because it offers hierarchies which are something people coming from other CMSs expect. I use it all the time but it needs lots of work.

But I'd be fine even if all of those modules were cut out of core. However, I don't think that Core should equal Drupal any more. Which it actually hasn't since D6 - how many sites are there without CCK/Views? I think shipping a version of Drupal that doesn't offer features like search, forum, blogging, etc. out of the box would be quite detrimental to its reputation and eat particularly into the hobbyist userbase. Also, there should be some upgrade path for many of these features. If Drupal 8 can ship with several product-oriented profiles, it should no longer matter which modules are in core.

ParisLiakos’s picture

However, I don't think that Core should equal Drupal any more. Which it actually hasn't since D6 - how many sites are there without CCK/Views? I think shipping a version of Drupal that doesn't offer features like search, forum, blogging, etc. out of the box would be quite detrimental to its reputation and eat particularly into the hobbyist userbase

Exactly.Separate the final product and the framework.the framework would be the core that core developers maintain (finally relief from all that burden) and the product will be sth kinda an installation profile.
Everything should move to contrib.

tsvenson’s picture

+1 to most. My comments here are from a site builders perspective. Only listing the modules I have objections about, the rest can go.

  • Book - While being able to structure content is good, having the Book module in core will not make it more modern. Relation and other modules are going to provide much better ways to implement this, as well as making it more adaptable for mobile devices.
  • Color - As pointed out, it doesn't have many issues, greatly improved for D7, gives a positive response from users. Also, Bartik core theme is using it. Its worth the little pain to keep it.
  • Contact - While webform easily can replace it, many small sites really only needs a simple and lightweight contact form. Only thing it really miss is the possibility of having custom text above/under the form. A workaround is using blocks, but that is ugly!
  • Overlay - Really like it and find it very useful as well as increasing productivity. This should definitely be in core as it then will be a common UX standard.
  • Shortcuts/Toolbar - The toolbar must stay, its a UX thing and something that is expected to be in a platform. Shortcuts are good to have to save a lot of clicks in for example Configurations. Should be made much more dynamic and configurable though so it can be better customized, with drop down menus and theming, for each sites/users individual needs as well as become a better API so Admin Menu doesn't have to be a drop-in replacement.
  • Statistics - As long as the needs for popular content and view counts are left, the rest can go.
  • Tracker - Its useful, but can also easily be replaced with Views.

Looking forward to follow the progress with this and how it will get Drupal Core back on track.

Rene Bakx’s picture

Strangely enough I miss 'comments' on this list ;) And yes.. before you'll put me on a steel-bean and carry me out of here. Let me elaborate what i mean. Comments are needed, but I found i rather strange that the way comments are handled in D6 was almost the same as in D7. There is the excellent field-ui, entities and yet comments still remain something that is smacked onto a node without real control, or being able to have more fields.

Anyway, it is really good to see from a 'end-user/developer/noname entity in drupal land' point of view that the Drupal community has the balls to tell the world that they created a mess, and is taking steps to clean it. Thumbs up for everybody that helped to finally get this clutter of wooly code back into a ball that's going to roll smoothly again! :D

catch’s picture

Comments were made fieldable in Drupal 7 (i.e. you can add extra fields to them), and there were some major API changes in terms of CRUD and rendering, but yeah the relationship to nodes is unchanged, as is comment threading (apart from some bug fixes).

However there is an issue open to fix this #731724: Convert comment settings into a field to make them work with CMI and non-node entities.

Going from #43 I would say comment is a 'category 4' module. It needs to be made a lot smaller via an improved field and entity API, but it has applications for a very large number of sites and use cases (especially if we make it entity-agnostic in terms of what can be commented on), and as such belongs as part of a Drupal Platform, not the same category as most of what's under discussion here.

nickvidal’s picture

We should keep in core what's essential AND *irreplaceable*. If we have 2 modules that offer similar functionality, then they should live in contrib. If they are *essential* in the sense that most sites will need them, they should be part of distributions.

Example: chx's argument above for keeping aggregator:

"Aggregator allows pulling in data from various sources. That's something many sites need. Keep it."

But we have Feeds as a strong alternative. So drop it from core and have distributions addressing this need.

xmacinfo’s picture

My first reaction to this type of mega patch. Do a Drupal fork! :-)

xmacinfo’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Updated issue summary.

sun’s picture

Issue summary:View changes

Updated issue summary.

xmacinfo’s picture

Project:Drupal core» Drupal Platform

By catch: Statistics:the view counts in statistics module are only a part of what it does. If we really want to keep view counts in core (for most popular) we could isolate that functionality and remove the rest (or most of it). I think these are hard to replace with analytics or awstats - but anything directly competing with those should go at this point.

By tsvenson: Statistics - As long as the needs for popular content and view counts are left, the rest can go.

Like tsvenson said, I'd like to see a simplified version of Statistics, not it's removal.

Please also remove these themes:
Garland: Time to die for Garland.
Stark: Who uses Stark really, or learn from it?

xmacinfo’s picture

xmacinfo’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Profile is gone \o/

timmillwood’s picture

xmacinfo, what is it a duplicate of?

I understand that there are parts of the statistics module that are useful, but they are not built in a good way, and does not work on all environments I vote for a complete rewrite.

Therefore, remove statistics, then a new one can be written from scratch.

RobLoach’s picture

Project:Drupal core» Drupal Platform

[EDIT] Tim wins...

Here's a couple more things we should add to the node:

RobLoach’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Updated issue summary.

colan’s picture

Project:Drupal core» Drupal Platform

Actually it might be better to keep each of these as a separate issues so that things are kept manageable, and individual discussions can happen over there. This can be the master issue that tracks all of those.

I just created #1261130: Remove the Book module from core which I added to the list above.

effulgentsia’s picture

@catch: re #43: I like your first draft of the 5 categories. I'd like to make sure it covers the UX and design side of things too. How should we, for example, think about Color and Overlay? I think some people might consider this #2, and others #5, and that's kind of a big jump to make on purely subjective grounds, which could make it hard to reach any sort of consensus.

Rene Bakx’s picture

@catch in #52, Ah i failed to find the UI for actually fielding comments, wrong assumption from my part. But I somehow do agree that comments should remain in core, however the little devil on my other shoulder shouts stuff about making it a true citizen of entity town and decouple it completely and use something like the relations module of perhaps a more block suitable block definition.

But let's move that discussion to where it really belongs, so anybody who's got any ideas on how/what feel free to spill your beans in #731724 :)

And I will see what i can do to make comment suck less :)

Rene Bakx’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Added a link to the book-specific issue.

ParisLiakos’s picture

Issue summary:View changes

added tracker issue link

catch’s picture

Project:Drupal core» Drupal Platform

@effulgentsia: I would put overlay and toolbar in #3 as 'admin interface', and color in #2 or #4. We might differ on whether overlay/toolbar etc. make a useful admin interface or are sufficiently generic but we need an admin interface and usability testing showed the D6 one to be lacking.

Currently we have optional and non-optional admin interface elements but eventually we should be able to disable all of /admin for live sites without a hook_menu_alter() so this wouldn't the case any more.

The two modules from d7ux that are considered 'at risk' in this issue as I see it from posts so far are dashboard (should never have gone in with the blocks system as it is and more a feature than an UX element), and shortcut (also more a feature).

catch’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

added OpenID issue link

sun’s picture

@xmacinfo: Stark is essential to Drupal, it actually exposes Drupal. All other themes belong to certain use-cases, products, and different understandings of Drupal. See also #1181776: Change theme_default variable to Stark

jrowny’s picture

I definitely agree with where this conversation is going.

When I tried to explain to colleagues who primarily use other CMS they say things like "Drupal doesn't even come with a WYSIWYG!" and I tell them "well yea, because you want to be able to choose the WYSIWYG." Or "Drupal doesn't even come with _______ out of the box!" It's inconsistent for me to make the argument that core should be "slim" and therefore infinitely customizable... when core is not really slim.

And the slope towards a full featured core has been proven to be slippery with things like dashboard. One person's dashboard is another person's worthless screen that will be replaced with HomeBox.

RobLoach’s picture

#911054: Remove Garland from Core as it can now live in contrib with http://dgo.to/garland .

nickvidal’s picture

@jrowny

Friend: "Drupal doesn't even come with a WYSIWYG!"

You: "For what you want, try this distribution that comes with this WYSIWYG + Blog + Aggregator + etc."

~ Make Drupal more like Linux, and distributions more like Ubuntu.

nickvidal’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Added shortcut issue link

arianek’s picture

Project:Drupal core» Drupal Platform

+1 to chx's list

Book: looks like book in contrib has been veto'ed already but i hope that either way it will get some love. it could really be updated with some nice features that wouldn't take too long - things we discussed a lot around the docs infra. i thought maybe it would thrive more in contrib, but if there is someone willing to take it on, then great! the rest i'm totally on board with, and

Shortcut: people don't realize that this exists, but when they do, they like it. The couple site admins I've trained up on D7 so far loved it.

Color: I realize this is totally extraneous, but when you are in your first few months of learning Drupal or installing your first sites, it can be really great to play with. I don't underestimate these kinds of features in core as far as getting new users interested enough to keep building their sites.

Contact: no question, must stay.

Not much to say on the rest!

arianek’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Updated issue summary.

sun’s picture

Project:Drupal core» Drupal Platform

Here's my list and stance, which mostly did not change:

  • Book: Undecided, as it's more or less the only API consumer for advanced menu trees and links in core. However, I think it needs as massive revamp to modernize it, be more generally useful, actually match user expectations, and start to solve actual hierarchical content trees and real world use-cases we also have on drupal.org. I somewhat doubt that is going to happen if it's in core.
  • PHP: Remove the input filter. Undecided on the remaining parts, which are mainly the user permission and the safe php_eval() wrapper around eval(). Rather keep the rest for now and revisit later.
  • Poll: Remove it if no maintainer steps up. It has to become a field in D8. But ideally, we'd entirely replace it with @fubhy's Vote, which attempts to merge VotingAPI, Fivestar, Flag, and others into a generic, flexible, and robust API. In light of that, I'd rather remove the stone-age burden of Poll module now, so rather -1.
  • Statistics: Agreed with others - rip it off, but retain the Most viewed functionality; I can't believe there are no contrib modules for that already we could pull in instead. Also agreed that dblog should provide better filtering and aggregates for 404/403s instead.
  • Search: As far as I understood, the component maintainers want to strip it down to a extremely simple implementation, and move everything else into contrib. If that is going to work out, then I'm with it. At least this also means there's some sign of maintenance and people actually care for Search module's future (regardless of whether the future is more fancy or less).
  • OpenID: As much as I can get behind @wojtha's pain, I believe this entire initiative will allow us to focus more on stuff like this, and we need an initiative for it instead. So I'm going to quote myself:

    To stay cutting-edge and on top of the game, we not only need to improve our OpenID client implementation and add an OpenID server, but also add OAuth or SAML implementations. All of these are fairly complex open standards, which won't succeed in contrib.

    The Drupal project has a unique opportunity to lead the Internet to what has been coined as Web 3.0 some years ago: a semantic, transparent web of services and identities.

    In other words: I'm considering these technologies as a crucial part of the web already today. They pretty much apply to every kind of use-case and product you can build with the Drupal platform, regardless of whether you do a simple single-user blog or huge enterprise site. Also, these technologies are enablers for additional contrib functionality, and least to say, closely related to the web services initiative. To stay on top of the game and competition, we need to enforce them, and I'm personally willing to maintain this in core and put some muscles behind them.

  • Everything else, remove or move into contrib. I'm not willing to maintain features of a product that doesn't fulfill my needs and use-case.
xmacinfo’s picture

@sun: I agree 99% with your list. I'd just remove the last sentence that reads: “I'm not willing to maintain features of a product that doesn't fulfill my needs and use-case.” :-)

For statistics, in #1018716: Remove Statistics module from D8 core, @bojanz prefers using http://drupal.org/project/radioactivity but I feel that this is not a simple counter and something most users are not looking for.

Garland may still die, after all: #911054: Remove Garland from Core.

Cheers!

Owen Barton’s picture

I don't think anyone could disagree with making core less interdependent, and I think the vision of having a simpler, framework focused core, with separate product downloads is a wonderful one, whatever the actual route to that is. I think we do need to ensure we find high quality owners for "product feature" modules that will really take care of them properly, both from a strategic and code point of view. As the de-facto owner of most of these, I think the direction for that process is in Dries court. I would suggest having a public process, whereby individuals lay out their vision for taking on a specific "product feature" module and Dries can select one or more initial owners, with input from the community. To allow time for the likely code untangling, perhaps prioritize a list and try and split off one every few weeks. [posted this initially on http://www.unleashedmind.com/en/blog/sun/crisis-conclusions#comment-2804]

cweagans’s picture

We have packaged installation profiles on Drupal.org. Why not change http://drupal.org/project/drupal to an install profile that pulls in various modules from Drupal.org to create the "product", and create http://drupal.org/project/core for "framework" stuff?

drunken monkey’s picture

I'm definitely in favor of kicking out almost all of the modules from the first post, especially if a good way of still (?) providing a good first experience for new users can be found.

However, a good administration interface is really needed on all sites, so some of these should be kept, I guess. I especially find Shortcut very useful (with the nesting level of admin menus, it saves tons of time), and since I just discovered it is useless without Toolbar, that would have to be kept, too. Also makes life easier.
It can also really be argued that this represents a lot of the Drupal "Look and Feel", and therefore strengthens Drupal as a brand. This would also mean that Seven and Overlay should be kept—the latter annoys the shit out of me, but if UX folks say it's good, keep it. And if Color module really is such an overwhelming UCX success, I guess it won't hurt that much, either (unless from a technical framework point-of-view, from which it's really, really wrong).

Finally, things like Contact and Search are probably needed by such a vast majority of sites (even though larger ones will need advanced versions of those) that there is also an argument for keeping them, depending on how extreme a "clear separation" view you take. If it's really about losing everything "that does not have to be in core", as the first post states, 80% of the other modules would have to go, too.

chx’s picture

I can repeat this, too: the future is snowman/onramp as a product but until then we must have a plan B and you can not rely on contrib to deliver it because contrib doesnt have the documentation and testing dedication core has. Once again, if you rely on contrib to deliver a product and that inevitable leads to Views. Adding adequate test coverage to Views would require, probably thousands of man-hours. It took core years and about a hundred people to get there test-wise where is is now. Are you going to do that to Views, custom breadcrumb and whatever else? My plan is much simpler: don't maintain the product much. Cut out the dead pieces (underway), fix the bugs in the remnants and put them aside. sun doesnt need to maintain any of this. The current mismash of a product is the plan B until onramp comes. And it can't be contrib because contrib simply doesnt have the manpower to deliver something you can bet Drupal's future on.

Michelle’s picture

I'm coming to this late and my opinion is the same as most on most of the contrib. I just wanted to say a few things about forum:

* Data is nodes and comments plus a bit to tie it to a forum. As long as nodes and comments don't go away, there will be an upgrade path of sorts for the actual thread data even if core forum was dropped. (And, yes, I realize dropping isn't necessarily the plan; I'm saying _if_ :P )

* While I'm not the fastest coder and have limited time for volunteer work, we're talking D8 here. Artesian will be ready for D7 for sure by then and then it's just a matter of how hard it is to port to D8.

* I am not interested in putting Artesian in core and have no patience for working on core.

* At this point, I'm on the fence as to whether forum needs to stay in core. I used to be against it knowing that it was unlikely to be cared for in contrib and I'd probably get stuck doing it because I needed it as the base for Advanced Forum. But Artesian is going to be doing its own thing and I'm not going to care about core forum anymore once it's usable. I also think that forums are more and more being looked at as "old school". They definitely still get used but I'm not so sure the expectation is there anymore that a CMS _must_ have one "in the box". As long as there is a viable contrib solution, that may just be good enough.

* A counter to my previous comment... Artesian will not be going the taxonomy/node/comment route. There will be an upgrade path but some people may not want to move from that way of doing things. If Artesian ends up being the only choice, that could piss off people who liked forum just fine the way it was.

Michelle

larowlan’s picture

I hinted at this in irc, but now I'm doing it officially - I'll put my hand up to maintain forum.
Will add a new issue when i get to work - typing on a mobile is too slow.

jennifer.chang’s picture

Sub

Jelle_S’s picture

sub

tsvenson’s picture

@larowlan: Just an idea, but what about joining force with Michelle on Artesian and build a much better forum?

emmajane’s picture

I disagree with the frame of this issue. I think it asks the wrong questions in attempt to solve a very real problem. Without data we don't actually know which modules aren't used by the thousands of site builders who do not participate in the issue queue. It makes sense to start collecting data #1036780: Drupal.org should collect stats on enabled sub-modules and core modules before defining cruft.

nyl_auster’s picture

sub

chx’s picture

@emmajane sorry but I cant agree. We are asking those thousands to come forward and maintain. It's happening, deekayen blog (in contrib), swentel shortcut (in core), dixon comment (in core), larowlan forum (in core). You seriously cant ask us to maintain that we dont want to. The trigger issue didnt have anyone saying "i love trigger".

bojanz’s picture

While I don't doubt larowlan's enthusiasm, I wonder how much a single man can do to take the core forum to the next level.
Perhaps him joining efforts with Michelle on Artesian in contrib would be a much better idea for Drupal in general.
Having the whole D7 cycle to iterate, communicate with users, do the right thing, and actually make D7 forums not be a one (wo)man effort would probably make more sense then waiting months for superficial patches to land in core. But I might be too harsh on this, so take it with a grain of salt.

larowlan’s picture

@bojanz, my intent was not to take forum to the next level (at least in the first instance) but rather ease the load on sun, chx, catch et al so they can focus on what is important, see this post: http://www.drupal4hu.com/node/303 I'm putting my hand up for step 2 of the plan.

[Edit: fixed link]

ParisLiakos’s picture

I dont understand it.This is posted in Drupal Framework's issue queue.Maybe it should move to core?
I thought it was not debatable on whether those modules should be removed from the 'framework' but i guess i am wrong.
I see people make lists on what they think about each module.It is not a poll people!

Or just dont call it framework.Whats the point?

If you have stuff like book,or overlay in the framework,then trust me its not a framework anymore.
The Drupal Framework should only have the APIs in it(Field,Form,Batch etc).

Now whether those modules should move to contrib or not,i actually dont care since i will have the ability to download and work on the framework only.But thats a different issue and doesnt belong in the Drupal Framework's issue queue.

chx’s picture

Right. I cheer larowlan for offering his help to fix the 100+ bugs with forum and to find / lead others in doing so. No need to take it to the next level for now.

C_Logemann’s picture

Think API

Hello everybody,
I understand both sides of lean core and protecting the look and feel of drupal.
But even sitebuilders love drupal for the amazing flexibility and the big web applications the drupal specialists are doing with drupal as a framework. Additionally drupal products like open atrium or drupal commons should make drupal easier to use. So there is probably one "product" only build on current core modules and this is the drupal distribution itself. Everything else is a growing variety of drupal products depending on several contrib modules and often just containing not activated modules like book, blog, statistics because they are part of the current core.

Here is my idea:
We can create a new category of special maintained modules, maintained like the core. Based on the new lean core and this special modules we can provide a new main distribution or installation profile.
Maybe the main distribution can look like the current main release. So a normal site builder may not feel a big difference to the current situation. This is like the new entity layer in drupal 7 "under" the node infrastructure. But the entity layer is a very important change to more flexibility and lean data structures for further developments.

I think every module which is not providing an API should not be in the lean core. Maybe even the user interface for administration which somebody would replace or don't need in some use cases can be moved into the new special module.
As I understand even joomla since version 1.7 is now following a strategy with maintaining some core functionality as a "framework" without the standard CMS.

If there is any need of more flexibility in the core to extend drupals flexibility without security problems and maybe only with only a little costs of performance this should be in core to reduce the need of core patches like for pressflow and forblockcache_alter. But I don't mean to get in the direct functionality of this solutions, only the possibility to hook in. It's maybe not useful to make everything hackable or changeable if no module is using this. But if there is a need and a patch for this and if a new hook doesn't cause any compatibility problems why don't implement it fast, even in older versions like drupal 6? Very important concepts could come into core in next versions if they are very useful like the field concept in drupal 7. A lean core is maybe the next big thing for improving drupal at all.

If something is only an implementation of one kind of technology like OpenID there should only be an API in core to hook on. Even API extenders like wysiwyg API don't need to be in core if they can hook on core like it is currently working. Especially wysiwyg API is a good example for a module which probably should also be a special maintained module, too. I think modules like rules and views etc. – where a lot of other modules hook on – are very important and maybe also need a special maintaining situation.

I think the lean core should not be called "framework" because there are so many contrib modules extending the API like organic groups, rules, panels, flag, commerce and so on. Drupal is so flexible that even the framework is extensible itself.

I am complete against any kind of reducing the flexibility of drupal! This mean even the PHP filter should exist because it's a kind of API. But if some solution can exist in contrib I have no problem with moving it there. I think the most drupal sites are already depending on one ore more contrib modules. So if somebody is thinking to move a functionality into contrib is a bad thing, (s)he should help on improving the contrib situation at all.

kind regards,
Carsten Logemann

giorgio79’s picture

@larowlan
The current Forum can already be replaced easily with a View with links to nodes. I put together a support forum in 2 minutes with Views and CCK here http://www.sharemacro.com/support-forum
Never ever turned the core forum on :)

Solutions like forum and blog are more of layout modules for nodes, but these are easily replicated with Views and CCK. Maybe put a VBO in there for maintenance tasks and some Rules, so nodes that have not had a comment for 3 weeks are autoclosed etc... Sayonara forum.

larowlan not sure how much effort you plan on putting in this but the easiest solution maybe is to replace the entire forum module with a view and a custom forum content type and distribute it as a Feature :P http://drupal.org/project/features

Jelle_S’s picture

Small update:
Profile and blog module have been removed from core in 8.x see #301071: Remove profile module from core (Bye bye Profile) and #233301: Remove blog module from core (Bye bye Blog)

Jelle_S’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Updated issue summary.

Michelle’s picture

Project:Drupal core» Drupal Platform

@giorgio79: You can't replicate the forum list with Views and that is the part of Forum that has the most problems with it. As someone who has been working with Drupal forums for years, I can guarantee you that this isn't something you can just slap together in a couple hours.

@all: While having larowlan come work on Artesian sounds sensible on the surface, the fact is that I am no where near ready to take on a co-maintainer. I'm also not convinced that Artesian should be the _only_ Drupal forum choice. It's going to be a pretty big, complex, module and, in some ways, less flexible than the existing forum. Some people just want small and simple and some are not going to like the move away from nodes. I think it's good to have the core forum live on, whether in core or contrib, and am glad larowlan stepped up to care for it.

Michelle

giorgio79’s picture

@Michelle

Thanks for the comment, but I think I did replicate it. :) Not in hours but in minutes. :P

Michelle’s picture

@giorgio79: You can think what you like. The rest of us can see the obvious. I'm not going to debate you here and take this issue further off topic.

Michelle

eaton’s picture

Over in the Trigger thread, Crell raises a very important point: we have not settled the heuristic we're using to determine what modules are eliminated in The Great Culling and what ones stay. To quote:

Honestly, I'm not comfortable removing anything from core until we decide what the heuristic for what is in core is. Once we have that, we can apply it logically. Until then, though, we're just flailing and smacking modules we don't like.

I don't think it's quite as random as that, but the danger of not being clear about our evaluation heuristic is not to be underestimated. After reading this thread, going over the google Docs that have been used to gather some of the consensus, and chatting with Sun, chx, and catch, I think I can articulate what I think is being used to evaluate code for elimination:

  1. Does the module provide 'Product features' for people who want to blog/discuss/etc (as opposed to 'Framework features' for people who want to construct a custom CMS?)
  2. Do the most active core developers feel that it is essentially a "waste of energy" to maintain in core?
  3. Can equivalent functionality be implemented using other tools available in Drupal core?
  4. Can equivalent functionality be implemented using popular and well-established contrib modules?
  5. Is the module's functionality 'uncoupled' from the essential parts of the Drupal framework? (aka, 'We can't dump System module')
  6. Has the module's feature set stayed relatively static over its time in core?

The modules that have been on the cutting block so far (Profile, Blog, Trigger, Forum, Poll, etc.) all match quite a few of these criteria; it's no surprise that they are in the crosshairs, though debates about their merits are important: heuristics aren't hard and fast rules, just helpful filter criteria we can use to speed up our discussions as long as everyone agrees on the heuristics.

While my core conversation in London wasn't as focused as I'd hoped, I ended up mentioning a number of those as criteria for deciding what could go into core, and it makes sense to use them to determine what should stay in as well. However, there are a couple of other questions that I think are also important to consider. Namely:

  1. Does this functionality improve the "first four hours" of a Drupal evaluator's experience with Drupal core?
  2. Can the functionality be implemented simply, both from a UX and a development standpoint?

Because so much of Drupal's real power lies in the tools that exist in the contrib modules repository, we must assume that even a moderately experienced Drupal site builder will know of, and make use of, contrib tools like Views. However, during the "first four hours" of evaluating Drupal, many newcomers don't even realize that "third-party modules" exist. Using the general 'site building framework' tools like Taxonomy, Content Types, and Fieldable Entities require complex mental models that take time to build; by stripping out the "click and go" modules from core we run a very real risk of making Drupal fundamentally incomprehensible to someone who isn't already used to decomposing their desired outcome into our uniquely Drupally mental models.

Those evaluators are not simply "dumb users" who would simply be a drag on the developers, firehosing feature requests: they're the crowd that many of our third party module developers and eventual core contributors come from once they've passed the initial hurdle and decided that Drupal is worth sticking with. The size and complexity of core, which is exacerbated by carrying around lots of "feature modules," is also a barrier for entry to core development, but we also have to remember that a growing pool of "users" generally grows our pool of developers as well, even if it's by a much smaller number.

Which returns us to Larry's point: removing functionality isn't bad, and actually serves many of the primary goals I've been advocating for years. Drupal core can't continue to be a "Clown car" of features. So, hooray! However, we need to make sure that we have a vision of what the end result looks like and ensure that Dries buys into that vision for D8. He seems to be onboard with the elimination of some of these candidate modules from core, but I'm very, very concerned that there isn't yet any official public buy-in on the packaging profile driven solution to the "First four hour" evaluator experience that Sun has advocated in IRC and a couple of threads.

If we don't have buy-in around both elements of the solution, we could solve one problem only to exacerbate another.

karschsp’s picture

As I mentioned in IRC last night, the common answer to the "first four hour problem" has been distributions and install profiles. However, once I install the Open Atrium distro I cannot then install Open Public on top of it and merge their features. Or, more simply, once I install the single_user_blog profile, I can't then borrow stuff from the community_forum profile. I'm all for ripping stuff out of core, but that stuff was there for a reason, and we need to make it easy for new users to add it back in.

eaton’s picture

As I mentioned in IRC last night, the common answer to the "first four hour problem" has been distributions and install profiles.

Well, as I said in my post, that is a proposed solution with its own challenges and significant list of tasks between 'idea' and 'reality'. In the past, Dries has expressed great skepticism and disapproval of that approach; if he doesn't agree with the proposed solution to the first-four-hour problem, then our efforts to pare down the core install will only exacerbate our already daunting user experience issues.

drifter’s picture

Isn't this something the plugin manager, once there's a decent implementation, could solve? Core Drupal could be just core, definetly not the default choice for downloads on drupal.org. The default download could be Snowman or a choice of distros.

But even with the core download, a decent plugin manager would help, if enabling an existing module wouldn't be much different than downloading and installing one from drupal.org.

catch’s picture

I completely agree with 2-6 on Eaton's list.

With the first criteria I think this is where we need to be really, really careful. It's also why I really like Eaton's 'platform' terminology as a way of getting rid of the false framework vs. product dichotomy.

The standard profile provides (just) enough for people who want to blog - you can post articles to the front page of your site, tag them, and you get an RSS feed and the ability for people to comment on them. This is enough for my (albeit crappy) personal blog I use for spamming Planet twice per year, and likely some other core developer blogs as well. It's very basic, but you can make an actual website out of it.

All of this is done via node, taxonomy, comment, field API, text, user - via saving configuration in standard_install().

For me this kind of functionality in core is what brought me to Drupal in the first place (node and taxonomy modules to be exact), and bugs in comment module are what got me into core development.

If core really wanted to be a 'framework', then it would ditch node, comment, taxonomy and just have an entity API - then you build up from that.

That's not what I want core to look like - if we want to allow people to use 'core' without some of these things, I would rather allow every individual component to be separately packaged - so you can get the database layer or cache API as standalone components then put them back together yourself, this doesn't mean we want separate projects or development processes for them though.

If you look at forum and blog modules. These are not holding content (like node or comment), nor are they ways of organising content (like taxonomy, book, or menu). Blogs and forums are a particular social way of organising particular kinds of content, which makes up a distinct section of a website (or the entire site potentially), in a way that is generally recognised as a pattern by site visitors (i.e. people know what blogs and forums are). This automatically limits their usefulness as generic site building tools compared to the modules they depend on (node, taxonomy, comment). It also raises expectations that they will provide a particular set of features - that you might expect to find on a blog or a forum site, in a way that node and comment module don't as much.

If we want core to provide examples of use-case specific features like that, then this ought to be possible to do entirely from configuration in an install profile (with maybe a few lines of custom code in the profile itself). If it's not, then that just means we have work to do to make that possible (or we need to find an equivalent feature that can be done in this way maybe).

I would also add that we shouldn't really be talking about ditching modules, it is more about ditching features - sometimes these comprise only part of a module - like access rules which was removed in D7. There is tendency to look at this as razing large areas of core to the ground, I think we should look at it more like an overdue hair cut or clear out of the garage.

catch’s picture

However, once I install the Open Atrium distro I cannot then install Open Public on top of it and merge their features. Or, more simply, once I install the single_user_blog profile, I can't then borrow stuff from the community_forum profile.

This is more for #1242448: [meta] More flexibility for core installation profiles. I agree on this and will respond on there.

bojanz’s picture

Basically, what catch said. +10.

eaton’s picture

Blogs and forums are a particular social way of organising particular kinds of content, which makes up a distinct section of a website (or the entire site potentially), in a way that is generally recognised as a pattern by site visitors (i.e. people know what blogs and forums are). This automatically limits their usefulness as generic site building tools compared to the modules they depend on (node, taxonomy, comment). It also raises expectations that they will provide a particular set of features - that you might expect to find on a blog or a forum site, in a way that node and comment module don't as much.

I'm about to check out of my hotel and don't have a lot of time to expand on it, but this is a very interesting take on the problem with some of our core experience: why new users are disappointed with the features AND experienced builders use other things as a starting point to replace them.

This 'Too generic to deliver on its promise, but too specific to be a reusable tool' issue is a heuristic I'm going to be thinking hard about over the next day or so. Thanks.

tsvenson’s picture

@eaton: Great comment, lots of very important questions that needs more analysis and good answers.

Looking in the mirror about how new major versions of Drupal Core have been launched as well as reviewed by both media and first time users, you are very correct that if this is taken to far it might very well hurt the future growth.

Especially at the .0 launch of a new major, media will basically only have core to test. History tells us contrib is not up to scratch for many months. Thus, Drupal 8 risk looking like a much bleaker release than what D7 was compared to D6. On the surface it risk having fewer features. Looking at the initiatives for D8, even my limited Drupal dev experience tells me it will be an upgrade nightmare, not only core to core, but porting contrib project to it.

So, while we within the community understand why this is needed, the market does not. Especially if we launch D8 as previous version, that is - "Here it is, play with it...".

To be able to pull this off and not only make core developers life easier, but also make sure we get the market to understand it, will require a whole new level of marketing and education of what kind of beast it will be.

Even more important though is that we don't repeat the D7CX. D8CX must work, we need to identify a number of contrib projects that simply must be available from day one. The "clean" Drupal 8 Core must be complemented with a few basic distributions/install profiles for typical sites, including blog, mobile and so on.

I believe that will be the only way to make sure that media not only understand that core is a Framework, but also the benefits, including how distributions/install profiles means flexibility bar none. For first time users we need to make sure they opt for the distributions when evaluating.

We will also need to better educate contrib developers when it comes to UX and the UI standards they should follow. This is to make sure their modules can be enabled and have a look and feel as if they actually was part of core. D7 meant an incredible improvement for this, D8 needs to continue improving that, not only with the visual look but also for the terminology used.

catch’s picture

One more thing, if a module absolutely meets all the criteria of 2-6, I don't think it needs to meet the first criteria - for example we removed 'throttle' module in D7 as well because that was adding a lot of low level complexity, and trigger is up for discussion at the moment (rules in contrib, it's unmaintained etc.).

Aggregator is also borderline as a core feature, but directly competes with contributed modules, and the contributed modules are winning. Regardless of the feature, the contrib offering are important to take into account - a lot more than they were even 3-4 years ago.

xmacinfo’s picture

[edited] How is it that my name is associated to some revisions for this issue? A bug? See catch explanation below.

@eaton: Looks like Dries already buys in to some extent to the overall issue. Profile and Blog are now gone (both of which is understandable).

We may looks forward to have a product oriented platform for Drupal. But like Sun says in the first line of this issue, core developers don't want to maintain most of what is now Drupal 7.

So we need to find a balance between the product we want to put in the hand of users and what core maintainers want to bring to the next level.

catch’s picture

@xmacinfo, it's a bug #1217286: Posting a comment changes too many issue node revision properties.

One more thing to mention here, it might be obvious but it just occurred to me when I looked through the profile module issue queue for issues to close...

Any module we remove from core, we will be maintaining in Drupal 7 for security and bug fixes for the next 3-5 years (which is a good reason to try to find maintainers for the unmaintained ones). In terms of reducing workload, it only enables us to stop updating those modules with new features and dealing with new regressions that might get added. That's going to help some, but it's not like they disappear off the face of the earth if they're removed from Drupal 8.

This also means there's a really long time from now to when sites that are using those modules are going to absolutely have to upgrade their sites and figure out how to replace them- this is very different to trying to remove things a few months before a major release.

That means if we definitely want to shed some functionality, it's better to remove it now than later - there's a longer run up to set expectations, and we're actually saving ourselves work updating tests etc. over the next couple of years when working on Drupal 8, whereas if we have a pretty good idea we're going to remove modules (as we did with profile), keeping them ticking over is going to take work which then goes to waste (since they need to be maintained across three versions, one of which is a moving target - instead of just two stable ones).

chx’s picture

re forum, @Michelle, @larowlan 's role is not with Artesian but with the current forum module. There are bugs and we dont want to deal with them, so we let larowlan do it. We are not asking him to make it the Great Drupal Forum. Just make sure the current one is OK, keeping this forum alive as a plan B more or less.

Michelle’s picture

@chx: I was responding to the people suggesting he come help with Artesian _instead_ of working on core forum. There's been more than one who's mentioned it, here and elsewhere, so I wanted to explain why that wasn't a good idea.

Michelle

Clean’s picture

StatusFileSize
new136.98 KB
new191.67 KB
new177.61 KB
new144.86 KB
new140.85 KB
new108.18 KB

1. Does this functionality improve the "first four hours" of a Drupal evaluator's experience with Drupal core?

1.1 For Drupal product side, seem product experts not here. So, learn from researched Drupal products.
1.2 Don't make user think. Do Onboarding essentials and cool new features, see attached for how subhub step-by-step work (we aren't that cool, but in-place block editing and back end should do).
1.3 To keep evaluator WOW 4 hrs. Pick onboarding features :
- Essential functions e.g. Site name / create content / attached image / add menu / select theme, color / add block / add modules
- Ask webchick, best part of Drupal from our researches
1.4 Then, keep these modules for marketing.
1.6 If only take out modules and upcoming one not going to do this, adopted one if necessary. Cool enough to find maintainer.
1.7 Onboarding user with Project browser.
1.8 User can see how cool is Drupal, problem solved.
1.9 WOW <4 hrs. if developer resources limited. Shorter onboarding, but really WOW.

And Drupal look 2012. Appealing ;)

xmacinfo’s picture

@Virtual: You ideas are good, but you are not in the correct issue. You should start discussions on IRC, groups.drupal.org, start a UX site for Drupal 8, convince people and then start creating issues on drupal.org.

Crell’s picture

Not surprisingly, I agree with eaton and catch above. Modules aren't the issue; functionality is, and the lack of clarity on what functionality "should" be in core.

What I'd suggest is that we first agree on a heuristic for what functionality belongs in core, and document it as part of the Developer Documentation. Then, we can go over core and see what that heuristic would say doesn't belong (which may or may not map 1:1 to existing modules) and exterminate those bits that we agree don't make the cut.

(I would also strongly urge Dries to *not* remove anything else from core until that heuristic is determined. A machete is not the right solution here.)

As a first attempt at such a standard, based on discussion with eaton and the "first four hours" concept, I'll throw out this description so that we can start bikeshedding somewhere...

"Drupal core contains functionality that meets one or more of the following criteria:

1) Framework functionality that is used to build application-level functionality that cannot be easily replicated in contrib.

2) Application-level functionality that is difficult or impossible to implement in contrib.

3) Application-level functionality that supports a pleasant initial experience with Drupal for new users, and encourages them to further explore Drupal's offerings including contributed extensions."

That would, for instance, put xmlsitemap and IP-blocking squarely in contrib (it's irrelevant to the first four hours and is no easier to do in core), would put path in core (it needs deep integration to be useful, and non-sucky URLs are legitimately a "I look for this when first installing" feature), and leaves pathauto open to debate on #3 (which is fine; it's only a heuristic and guideline).

*dons flame-retardant suit*

sun’s picture

Please shoot me, I totally failed to clarify on-issue here:

I totally agree with @catch's and @eaton's considerations on this issue. They are spot-on.

The current calm-down and slow-down is an absolutely required part of the process. While Profile was a no-brainer, I actually wasn't happy to see the removal of Blog this soon already, as we didn't even discuss the related technical upgrade path issues or anything else about it.

We've identified very tough problems, but I don't think that anyone wants to quickly do changes that are caused by frustration, not (yet) well thought through, but which very potentially have a very deep impact on Drupal's future.

Also, I've the impression that many read my blog posts, but mayhaps didn't click/follow the links. We're possibly leaving the majority of the community behind and discussing/deciding on things that not everyone is aware of. At least I don't want to make and stress on decisions about crucial things without having heard the stances of the other core maintainers that didn't chime in with a single word yet.

We've essentially started to discuss only a single topic of the topics I outlined in my follow-up post (namely, what to do with stuff that no one wants to maintain [whereas we didn't set a deadline upfront]). Having the other discussions and coming to an agreement is an absolutely required prerequisite. The only benefit of nuking Blog this early is that it's putting a lot more pressure on us to have these discussions.

I'm undecided where to share the opinions on the other things we need to discuss. Originally, I thought of everyone just simply copypasting those discussion topics, and publishing their input, opinions, and arguments on each item, on their very own blogs. That is, because these perspectives are not going to be directly actionable, but instead, they are independent answers to pretty tough questions. I'd still be in favor of doing that (and perhaps revising/completing the list of questions/discussion topics first, not sure whether it's complete). The only downside I'm able to see is that those opinions can't be "tracked" easily, and also can't be "merged" back into a d.o issue easily due to the fragmentation/distribution. But even with those constraints, the independent answers (not +1s) would be far more interesting and appealing to me. Lastly, I think everyone knows - publishing something on your own blog instead of some random issue on a community site makes you think twice about what exactly you write and propose ;)

karschsp’s picture

Just trying to wrap my head around this...

So, given those criteria:

1) Framework functionality that is used to build application-level functionality that cannot be easily replicated in contrib.

  • Block, Comment, Content translation, Database logging, Field, Field SQL storage, Field UI, File, Filter, Help, Image, List, Locale, Menu, Node, Number, Options, Path, RDF, Search, Syslog, System, Taxonomy, Testing, Text, Update Manager, User

2) Application-level functionality that is difficult or impossible to implement in contrib.

  • Book, Color, Contact, Contextual Links, Dashboard, Overlay, Shortcut, Toolbar

3) Application-level functionality that supports a pleasant initial experience with Drupal for new users, and encourages them to further explore Drupal's offerings including contributed extensions.

  • Aggregator/Feeds, Book, Blog, Color, Forum, OpenID, PHP Filter, Poll, Statistics, Tracker, Trigger
  • Everything else in contrib...
Bojhan’s picture

@sun I think it would be good to ask webchick this question (core g.do group?). I also feel we are rushing things and not able to have good discussions with more contributors. A good thing to note here, we never had good platforms for this - we might just be dreaming it is different now.

carlos8f’s picture

I think Drupal has moved past the point of Ready-packaged-forum/blog/thing and toward Amazing-hook-based-modular-PHP-platform. Drupal core should be exactly that -- core. User module should consist of basic session-oriented operations to get things started. Other modules should be in contrib. System should not be a module, that is stupid. Then we need a package-based installer. Modules should not assume flavor modules like "poll" or "book" are installed. That's the only way to make core maintainable and have everything else be interest-based.

Clean’s picture

FYI. Yes, that's why we have to take load out of core kernel developer. Let's them do what they are amazing at.
But. Drupal core product still needed as marketing if you still want to sell Drupal to mass

The purposed is to free loads from amazing core devs, so they can continue to do cool things happily.
Then, give that product function to other people who expert in product happily.
So Drupal can become both real framework and product. Not half baked as is being.

Drupal devs is happy devs. I really want that.

larowlan’s picture

larowlan’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Bye, blog

Clean’s picture

Project:Drupal core» Drupal Platform

@chx @catch @Eaton and everyone.

I see this everywhere in community now:
- People DO understand the causes and agreed that our core developer are burn out with workloads.
- People DON'T understand the causes and not agreed, don't know why & what we are talking about here.
Just another thought. Who is the appropriate and give loud voice & informed the situation to our community? So people can be buy-in, or even we can get those unforeseen helps, like what chx did experienced. I think there're many people who like to see Drupal become real product, get their help in core.

IMO Dries is not the appropriate person, no one don't want him to say anything that is not happy words to public. But that's the 80% who do judgement. Help Dries solved, and he will agreed anything ;)

Then we can get back to the right way to do it:

“People come to me and ask how they can contribute. I always tell them to do what they want to do. If I told them what to do, then it wouldn't always be fun and it wouldn't necessarily translate into the passion that I'd like to see happen.”
—Dries Buytaert, Drupal Project Lead

“The principles that come to mind are those of self-organization and scratching your own itch. Getting out of people's way as much as you can, and enabling people to accomplish what they are passionate about. When people can come together and take action collaboratively, impressive things can happen.”
—Dries Buytaert

Clean’s picture

Project:Drupal Platform» Drupal core
Issue summary:View changes

Added forum issue for marking larowlan as maintainer

nbz’s picture

blog, forum can be replaced by a flag type module.

Tracker is a blog type module.

So branching out tracker to allow content flagging may be a way forward allowing them two modules to be removed (or changed into activateable features) without losing functionality.

EclipseGc’s picture

I'm going to weigh in on this topic at this point because I think we're missing a critical distinction. Understand, I essentially agree with eaton, sun, catch, et al... that being said, as someone who's been discussing this since at least DCDC, the understanding that I've been trying to convey from day one is that "my core is not your core". This is actually sort of critical, and actually beneficial to everyone involved.

My core is not your core:

What does this mean? It means that what gets distributed as "drupal" is not the problem. The problem lies is what gets maintained as core. That is to say that the drupal/standard "distribution" can be whatever we (the community) wants it to be. If that has a forum and a blog and overlay, etc, etc... I don't really care. I think all of those modules have their place. The critical point here is that those modules are important to the standard install profile... NOT to the kernel of the system we call Drupal... and really that's the entirety of the distinction.

What does that mean?

What it means is this: When I say "move blog to contrib" what I mean is "remove it from the kernel that is drupal core". What I do NOT mean is "remove it from the distribution that is drupal/standard". I think we all recognize that blog (for example) will be a bit of a PITA to forward port into D8 sans an actual blog module in the distribution. Moving blog to contrib is all sorts of ++ because this essentially means that it can improve separately from core. But that notion does NOT mean that the distribution we consider to be drupal/standard shouldn't have a blog anymore. (All of this goes for forum and most of the D7ux components as well)

We WANT that stuff in the distribution that is standard... we just need to recognize that standard is not our kernel... Menu, and Form API and entity and fields and dbtng and and and... those are our kernel... and they're a pretty awesome kernel.

What is my perfect world scenario?

In a perfect world we (don't recoil) break drupal core in half. We keep the half that we define as being a "kernel" to what we do, and we contribute the rest to contrib repos (individually preferably). This will require a lot of discussion, I think anything in Crell's 2/3 groups are a good candidate for this depending on how the bikeshed turns out. We then Package drupal the distribution up for the public in such a way as that they continue to get our standard install profile, JUST LIKE THEY ALWAYS HAVE. The methodology for doing this will also give us the ability to package other distributions with core if we desire (a single user blog for example) and other "typical" use cases on the web would probably be HUGELY smart of us. By and large this buys us the logical separation of "what we build with" and "what we're building". I am NOT trying to say that our "framework" should EVER be publicly distributed as a framework... it would be nice if I could drush_make it (instead of the current drupal core) but that is sort of a different topic.

The final code base that new users download should be essentially unchanged from today, it will just be assembled and maintained differently.

This was kind of a long post, my views may not reflect those I agree with on various topics, I do not represent anyone's views but my own. I HAVE however put in a LOT of thought on this topic, and despite how various portions of my opinion may come across, just keep that last bolded sentence in mind as the final goal for me.

ParisLiakos’s picture

@EclipseGc exactly!
Thats what i wanted to say,but couldnt.
eg look what Expression Engine did with CodeIgniter.
we dont have to give the end user the framework.
it will make no difference to end user but it will help the development of the drupal,
since it will be easier for newcomers developers to understand wth is going on in this beast called drupal and actually contribute to it!

andypost’s picture

Shortcut is really useful module and should be joined with admin_menu because it make no sense without overlay or toolbar

kika’s picture

Could somebody create a separate issue to deal with all those shortcut+toolbar+overlay intertwinings, both from UX and API side? It does not matter if those modules will be touched with axe, they need to be properly separated first.

bojanz’s picture

Exactly. I don't see anything that needs to be discussed around shortcut+toolbar+overlay in this issue. We pretty much already agreed we don't want them removed.
All technical problems can be addressed separately.

Michelle’s picture

@EclipseGc: Those have been my thoughts as well for a long time. How I see it:

Core - Only the essential things without which you can't build a site at all. Mostly internal / admin stuff, very little end user facing.

"Golden" contrib - Add on modules that need to be kept to high standards and kept up to date with core. New core versions not released until these are ready to go. These need to be more tightly controlled by the community to ensure they meet the same standards as core.

General contrib - What is contrib now, stuff maintained by individuals / groups as they will.

If we can then make official distros from Core + Golden as well as contributed distros from all 3, then we can be all things to all people. :)

Michelle

effulgentsia’s picture

#75 by @Michelle

I am not interested in putting Artesian in core and have no patience for working on core.

#123 by @EclipseGc

Moving blog to contrib is all sorts of ++ because this essentially means that it can improve separately from core.

#128 by @Michelle

"Golden" contrib - Add on modules that need to be kept to high standards and kept up to date with core. New core versions not released until these are ready to go. These need to be more tightly controlled by the community to ensure they meet the same standards as core.

Part of me really likes the idea of a "golden" contrib. As a Drupal user, I'd love to see modules like Views and Rules make their way into there, in addition to some of the functionality that's currently in core.

However, as a Drupal contributor, I see some contradictions in the above quotes. How will these golden contrib modules be tightly controlled to high community standards, without resulting in the kind of burdensome process that drives contrib developers like Michelle (and merlinofchaos) to lose patience to work on them? Will these modules really be able to improve separately from core if we hold up core releases until they're ready? These questions are not criticism. Just a start at trying to get more clarity about what this approach would look like in reality.

mdrummond’s picture

As a site builder, the key for me is that if we are removing the product features from core there needs to be a corresponding way for people starting a site to easily select those features when setting up a site.

If removing those product module features from core helps the core developers to better focus on the key APIs and framework that make Drupal great, awesome.

That trimmed-down core would not and should not be the public face of Drupal. It must be couple with a more full-featured distribution that makes it easy for site builders to actually create a functional site.

Removing modules from core is one-half of the equation. Efforts like Project Snowman are the other half of the equation. Both are equally critical to make Drupal shine.

tsvenson’s picture

@EclipseGc @Michelle @effulgentsia: This is something I have been advocating for as well. We need something between core and contrib. Something that has the same principals as core when it comes to maintaining and code standard, but with the freedom of contrib to be able to evolve and be improved upon all the time.

These projects should not have a person as an owner, instead something similar to the initiatives owners for D8 core. That is a lead maintainer that is chosen by the community and/or the leaders. It would probably be a good idea to look into the team merlinofchaos have built around his modules as inspiration for best practice.

I would actually go as far as to say that both the Update Manager and Upgrade support should be moved to this group. They are both products that is not essential for a site to work, but they both greatly need to be improved upon constantly. Placing them in the Golden/Core Contrib would both give them that freedom and at the same time remove the risk that they will delay the release of D8 like they did for D7.

Another big advantage I see with creating this special contrib is that each project in there must only use public API's. That will ensure that the core developers will be able to focus on core an nothing else.

Dries’s picture

My list:

  1. Aggregator: remove
  2. Blog: remove
  3. Book: refactor -- we need a powerful and easy way to organize content in hierarchies. I'm comfortable removing book if we can make this easier through the menu system.
  4. Color: keep
  5. Contact: keep partially -- site-wide contact forms are important to keep but I'd be happy to remove the per user contact forms.
  6. Dashboard: keep for now -- many competing CMSes have powerful dashboards. I think we need to figure out how to make a much more effective dashboard.
  7. Forum: keep for now -- let's find someone to help improve it.
  8. Overlay: keep
  9. PHP: keep
  10. Profile: remove
  11. Shortcut: keep
  12. Statistics: keep partially -- I use view counts, page generation times and a few other Drupal specific features of statistics module. I'd be OK removing functionality that competes with Google Analytics though.
  13. Toolbar: keep
  14. Tracker: replace with an e-mail notification module for comments
  15. Trigger: remove
  16. Poll: remove

Per my keynote at DrupalCon London and the survey results, we should consider adding new things to Drupal 8 core as well:

  1. Better media/asset management tools -- media module?
  2. WYSIWYG
  3. Mobile support -- responsive design, HTML5, web services, mobile friendly themes
  4. Configuration management -- UUIDs, etc
  5. More UX improvements
  6. Content staging
rafamd’s picture

It's very nice to see this issue is starting to get shape, specially people agreeing that "taking something out of core" or (why not) "putting something into core" shouldn't directly obey to the product needs but to a strategy that better architects our whole ecosystem (at all levels: tech, community, marketing, etc)... all things that are being talked about.

I don't see why Larry's "2) Application-level functionality that is difficult or impossible to implement in contrib" shouldn't live in "core".

Both should care for each other :) Core should make contrib life easier and contrib should make products out of core. The line is always going to be a bit (just a bit) fuzzy but synergy is essential. And we've got rid of 3) :)

fubhy’s picture

I am not that sure on the Book module as I agree that hierarchical content is a needed feature but It would be great to see a more generic approach for that.

Regarding the toolbar module I see great potential for improvement (@see admin_menu).

I fully agree on the rest.

oskar_calvo’s picture

I think that color don't should be in the core, it don't give nothing and not all the themes suport color and the same with forum.

Módules like forum, blog are more a product that a part of a framework. In fact with views and taxonomy can be build also a forum like pages.

I think that core shoulb be minimal.
menu
block : we need to improve it and also add some funcionalitys like http://drupal.org/project/block_instances or context.
content type/node/entitys. If I have cotent type i don't need neither a blog or a forum because i can build them.
taxonomys.
field
filters
content translation.

Also a miss between simple views and views should be great to build landingpages, and subhomes.

Oskar

nevergone’s picture

Poll module?

valthebald’s picture

I have the same feelings on "Golden contrib" proposal as in #129. It sounds really great for me as a user, but I am afraid this can hurt contributor community.

oseldman’s picture

sub

chx’s picture

Our best comment is sun's caution comment. Dries' list, as was mine, is too much too soon. Let's ponder on where we want to be before we begin to run amok, OK?

Crell’s picture

Dries: Please do not post a hitlist of modules you want to kill. We've all been making such lists forever, and they are useless. On what grounds do you say that poll should go but forum stay? Without a clear definition of why your list is what it is, your list is just additional bikeshed fodder like everyone else's.

Instead, please comment on the proposed heuristic in #111. We need a standard by which to judge what functionality (*not* modules) belongs in core, and then we can refactor as appropriate. Please do not remove anything else until we have that standard agreed upon and documented.

Michelle’s picture

@effulgentsia: My initial thought for the "golden contrib" are the modules that are now part of core but not critical functionality. Forum, poll, etc. These are already "community owned" modules so the challenge is to find a caretaker to lead the development on them while still having the community help out with them. Instead of Dries/#2 having to sign off on _everything_, then, he and his co can focus on the kernel core and the caretakers of each "golden contrib" can commit patches to those. It's sort of like initiatives, but with discrete modules that are able to be decoupled from core.

These modules would be "nearly core" but have their own space so a bug in one wouldn't break all of Drupal (during the dev stage). They would be expected to be ready to go at the same time as core so that the official release distributions could make use of them. Having the separation would mean that a development branch could be opened up on just the module that works on new features at any time, with the only requirement being that there is a stable release available that matches the supported core versions at all times. I think this would allow these modules more freedom to improve while still being tied enough to core that we can have a "Drupal product" with them.

Modules that are now in contrib moving to "golden contrib" would still be tricky. The current maintainers would have to give up a lot of control when moving from owning the module to being caretakers. Not everyone will want to do that. While having Views, CTools, Rules, and other highly used modules be part of it would be wonderful, it's really up to the people who wrote/maintain them now whether they would be happy with being tied so closely to core releases.

Michelle

Richard Archer’s picture

+1 for "The final code base that new users download should be essentially unchanged from today, it will just be assembled and maintained differently". This point is absolutely critical.

+1 for Golden Contrib. This would really help to deliver a quality user experience on major version updates. It might also bring more developers to these key projects. Increase in pain level for devs should be minimised!

+1 for "9. PHP: keep". This functionality has saved my bacon on numerous occasions.

noisetteprod’s picture

sub

sun’s picture

  1. In light of #112, please see #1262752: Discussion items.
  2. To stop derailing the original and technically unrelated unofficial framework initiative that @catch and me kick-started and are actively working on, I've just coined a new Platform Initiative tag. The "Remove xyz" issues need to be re-tagged.

I think that @Michelle is spot-on in #146. While the OP suggested to death-murder-kill out of frustration, discussion has moved on to a point where nothing would be removed, but instead, merely our packaging process for the default Drupal product would change. The example shows a lot more of many future possibilities - whereas our focus should be on simple first steps. In fact, I could imagine a very first incarnation of this that just simply moves product features elsewhere, changes the Drupal download link, but keeps these projects maintained by the very same people that are maintaining them now. -- Essential difference to now, for each and everyone involved: None. But a very important step under the hood for Drupal's future.

C_Logemann’s picture

Thinking of modules can only be a "change the current way a little bit" or we can talk about a new way to handle a lean core.
I agree with Crell (#111, #144) that we need a plan/heuristic how to decide what should be in core.

In #87 I wrote that I think a lean core should only provide an API and not product features. Additionally I already proposed a new kind of special maintained modules which we can name "golden" like Michelle (#128). The idea of Michelle to combine the release of golden modules with the core is a good plan to make the situation for normal users like it's always been. Maybe the golden modules needs some maintaining strategies for fast security updates?
I like to read some comments about this "lean core and golden distribution" strategy.

And: I agree that we need a better forum, easy hierarchical content and so on. But we need this in the "golden" distribution where we provide standard CMS functionality! This special/golden contribution should be improved especially in usability like we already do this with the standard drupal distribution.

sun’s picture

effulgentsia’s picture

Regarding hueristics, I generally agree with catch (#43), eaton (#94), and Crell (#111), but I have the same reaction to #111 that catch had in #98: that I agree with it only for a definition of "framework" that goes beyond just APIs, but also includes a "site builder's framework". In other words, even if we implement the "golden contrib" idea, I would still like what is left in Drupal core to be something that a site builder could install, and build a website with it, without writing a line of code. I consider the "Field UI" module as essential to this as the "Field" module, and the "Menu" module as essential as the menu.inc file. In this respect, I think I'm echoing catch's #43.1 - #43.4.

Even the heuristic of "can you build a website with it?" has lots of grey area. For example, is Comments functionality needed to satisfy this? What about Views? It remains up to us as a community to make the call on what we consider essential, vs. what makes sense as a contrib add-on, and even if we agree on something being essential (like most of us would with respect to Views), whether there's other heuristics to consider (like its ability to be maintained in core).

Where I like the "golden contrib" idea is for stuff that maps to #43.5 (use-case specific) for very common use-cases. Prior to the initiative owner approach for scaling Dries, Dries was considering appointing separate Framework and Product maintainers. If Dries and eaton both thought it was a good idea, I would be fully in favor of eaton serving the Product maintainer role via a Snowman distribution that contained modules like Forum. Perhaps we could even name this distribution simply "Drupal", and call core "Drupal core" or "Drupal kernel" or whatever.

effulgentsia’s picture

One more comment about "lean core". I think it's important for us to be focused, but not necessarily minimalist. For example, core would be leaner without vertical tabs, contextual links, overlay, and other d7ux features, and you'd still be able to make a website with it. But, whichever features have proven to be successful in improving the general administration process for a large segment of users, and can be successfully maintained in core, should stay in core, in my opinion.

tsvenson’s picture

@effulgentsia: I would say vertical tabs, contextual link, overlay, toolbar and other UX/UI stuff is part of the framework as well. They are defined standards to help developers build primarily administrative interfaces for site builders and site administrators. They have already proven to help making contrib modules easier to learn since users doesn't have to learn different modules different ideas about how to administrate and work with them.

As a bonus, they most likely make the total site footprint smaller as well.

Yes, they are not perfect, but a good start that can be improved upon for D8.

Dries’s picture

Re @Crell in 144: the list is based on my experience with people using Drupal. The lack of heuristics does not make such lists useless in my opinion.

Some things can't be expressed in rules. That said, my goal (heuristic) for core is to give both newbies (primarily) and experts (secondary) a good initial experience with Drupal. Thus, we add what they want or expect, and remove some of the clutter (i.e. blog module) and unexpected features (i.e. aggregator module). It's one of the reasons I did the State of Drupal survey.

catch’s picture

Even the heuristic of "can you build a website with it?" has lots of grey area. For example, is Comments functionality needed to satisfy this? What about Views? It remains up to us as a community to make the call on what we consider essential, vs. what makes sense as a contrib add-on, and even if we agree on something being essential (like most of us would with respect to Views), whether there's other heuristics to consider (like its ability to be maintained in core).

This is important, and it's where this discussion really needs to develop more.

For me the best product Drupal has is the 'platform' functionality that lives in the middle - people at least at the moment come for 'CCK and Views' (or taxonomy and nodes), neither Open Atrium nor system module. When I say "people" and "come" I mainly mean potential contributors on Drupal.org which is all I care about in terms of what core offers - I'm not interested in maintaining features in core for Drupal consumers, but I am interested in having features in core which allow first time users to get started quickly (i.e. whether it is as simple as articles with tagging or comments or larger infrastructure like snowman or project browser) even if I might not personally work on those as my main focus any more.

However while I think we need Drupal core to be framework + platform (and decide what 'product' is or should be with Snowman and ship something like this in core at least), we are never going to have the entire framework or the entire platform in core, and this is where it gets interesting. There are really two parallel discussion that need to happen here, part of why we should move this away from lists of modules.

There is "what is the highest level of functionality that we think should be in core", what I tried to get at pointing out that blog and forum modules currently fail in that they solve specific feature use cases (but not well enough to meet expectations). Then there is what specific functionality, at all levels, that we think is maintainable and desirable to have in core.

Current 'framework' functionality that is not in core:

- any pluggable cache, lock, queue, backend (memcache, redis, mongodb).

- contrib database drivers (Oracle, MS server).

- the queue API in Drupal 6 (backported as a contrib module)

It's not impossible we might move something like file-based caching or an extra database driver into core, but there is always going to be some stuff in core and some not. There is always going to be alternative implementations we do and don't ship with, and there is always going to be new stuff that appears in contrib first. We need to refactor core to make this easier (the database layer is a good example from both back ends, backportability, and the way we managed applying it to the rest of core), and to allow really low level components to be included at lower levels than a 'module' - this is what the framework/CMI/WSCCI iniatives are tackling and it is not strictly related to this discussion (thanks for the new tag sun).

When we get to platform this is even more obvious, there are several issues there.

We added Field API to Drupal 7, this took a long time and the job is not finished - we never got to profile or poll module, we realised while working on taxonomy and other conversions that core was completely missing a generic Entity API to match the Field API, and this was patched together after the fact and far too close to code freeze to get it finished. When converting taxonomy to be a field type, forum had to be changed as well simultaneously, and this was really messy. There are other dark corners like search module where bits got 'missed' from field API conversion (like search result theming) and these are still being discovered and worked on now.

If we want to make core a better platform, we really do need to give it a haircut and remove some things so it can move a bit faster and areas don't simply get overlooked for years - and that mainly applies to functionality at higher levels rather than low level APIs (I'd consider vertical tabs a low level API, at least on a spectrum that includes Poll module).

On the other hand, working on Field API or entities without node, comment, taxonomy, files, images in core would be a complete vacuum and risks making it completely unusable (the amount of bugs, performance issues, API limitations etc. found via converting to those was not small). Trying to separate a really, really low level framework out in terms of repository or development process is a non-starter at the moment until many, many other things are dealt with if we ever wanted to do it anyway.

Then looking at the more site-building platform functionality, there is always going to be some things of this class of functionality that live in contrib. Views is the obvious example, CCK and token were in D6. Whether we agree on the specifics of views in core or not (if, when, how), there is always going to be /something/ really useful at this level that is in contrib, and there may be things in core that outlive their usefulness 5 or 10 years after they were added.

Views itself, with the bug squad and multiple maintainers is becoming an exemplary example of contrib maintenance (and core is learning from that by stealing the bug squad idea for office hours) - I don't think this requires an official golden contrib, but we might need an initiative to identify modules like views we really care about and see if we can move them to a more core-like maintenance process (in terms of peer review, testing etc. and maybe keeping bleeding edge versions that chase head). That requires manpower or shifting existing people's energies around so it is non-trivial but it can happen within existing tools.

Rather than the contrib repository itself, blurring the lines between core and contrib at this level I would like to see happen via a project browser so contrib can actually be pulled into core as part of the initial experience, and core itself made more genuinely modular so we have less shortcuts taken there (i.e. breaking system module up).

I think it is worth having infrastructure in core specifically to give that window to contrib although I don't personally plan to work on update manager/project browser so it depends on having several people who do there as well - however that feels like a central way to resolve some of these issues in terms of initial user experience. This doesn't mean we ship core without any install profiles - people can install Drupal without an outgoing internet connection and Drupal.org can go down, as well as us having no way to guarantee that things get done in time for X.0 releases.

Then we also need to look at the really bad examples in core, actions and triggers are a popular one. Here's what I think went wrong there:

- committed only one month or so before code freeze, no time to convert anything in core to use it.

- the functionality is not used anywhere in core at all, not even the standard profile.

- trigger was a brand new module, it was added almost entirely because we didn't have automated testing to prove that actions worked. This is very different to Field UI which already had a massive user base in contrib via CCK.

- became unmaintained/frozen almost immediately after commit, I think this is partly related to timing and inability to embed itself as something that either fundamental core functionality or the standard profile was using.

- now has a viable competitor in contrib via Rules, which as far as we can tell is what people mainly use instead of trigger - so whereas core comment module has 'won' over nodecomment in contrib (via field API conversion), trigger module has 'lost'. This compounds the previous issue since attentions in this area are completely divided.

Trigger is not a 'product' module, it's definitely platform/site building tool, but it has serious problems in terms of competition from contrib, lack of maintenance, and lots of outstanding bugs. Dashboard is similarly in people's sights for the same reasons.

So when looking at platform functionality, both for inclusion or exclusion, we need to look at the following. I'm using trigger and comment as the two examples.

1. Is it high quality (or at least of comparable quality to the rest of core)? (trigger module is about an D, comment module is about a C or B- although there is a viable plan to improve comment module via entities).

2. Does it have viable, popular competition in contrib (trigger module has rules, comment module doesn't - in D6 there was http://drupal.org/project/nodecomment - but this is largely superceeded by fieldable comments, so in this case core won the competition via refactoring).

3. Does the standard profile (or another core module) make use of it (again, no for trigger, yes for comment).

4. Does it have outside dependencies (OpenID), or multiple internal dependencies (forum) which make it hard to maintain or can interfere with lower level refactoring.

5. Is it built on APIs in core that are already in extremely poor shape an need refactoring themselves (like Dashboard was on block module).

These need to be criteria for moving things in, as well as moving them out, and they are as much social as technical issues.

catch’s picture

If you are posting a +1 on this issue, please really think about what that means:

- is it +1 to core being 'maintainable' - do you really know in which ways it is and isn't maintainable at the moment? Have you tried to contribute to core issues before? Did the issues discussed here make that harder?

- is it +1 to the arguments on here? there is some level of consensus but loads of disagreement, so are you agreeing with everything that everyone said on this issue simultaneously.

This issue is already long, it is going to get longer, and flurries of +1/subscribe do not help to advance the conversation at all. There is nothing to vote on yet either.

Chi’s picture

+1 for Golden Contrib.
+1 standard profile is not our kernel
+1 lean core

Installation profiles have the potential to help in making core maintainable.

catch’s picture

Another thing in terms of 'product features' and what that means, IMO at least in the short term for removing features, we need to switch from discussing modules, to 'features that are at the top of the dependency stack'.

For example, nothing in core depends on the postgres driver (or the MySQL driver, or the sqlite driver) - we could remove any two of those if we wanted to, but we can never remove all three or the database layer itself. This is extremely low level functionality, but support for specific databases is actually a 'product feature' if we look at it this way (and SQLite and postgres have been broken lots of times since they are not used by most core developers so the maintenance issues are true there too, although they do have active maintainers).

Also, nothing in core depends on profile, trigger, OpenID, poll, forum or tracker (including either of the install profiles we ship with). And there are features in node or user module that nothing in core depends on, but which are there anyway, that could potentially be removed if we wanted to.

Anything lower in the dependency stack (i.e. depended on by either the install profiles core currently ships with, or other core modules) is moot without an agreement on those outer/upper layers.

David_Rothstein’s picture

I was surprised to come back from a week's vacation and see Blog module randomly removed from core. I agree with Crell (and others) that we should have general guidelines for what core is supposed to do before we randomly remove modules from it.

Those guidelines won't be easy to develop, but if we succeed, we will actually have something concrete to point to. Otherwise, this issue will be no different than the other ones like it that seem to pop up every 6-12 months and then disappear with nothing productive getting done.

Looking at the proposal in #111, it seems like a really good start to me. Quibbles:

  • For #1 and #2, "cannot be easily replicated in contrib": this is defining core based on what it isn't, which is not ideal. But I get what you're going for and don't have a better idea at the moment.
  • For #3, "supports a pleasant initial experience with Drupal for new users": I think this should be more specific. For example, if a feature exists that is only good for the "first four hours" (i.e., product evaluation) I don't see why it belongs in core. I think the minimum bar should at least require that the feature helps new Drupal users build a functional (but simple) website. Given that the Snowman project is aiming to push the limits of what can be built with core out of the box, would it make sense to align any statement here with the goals of that project? It might be something like:

    Drupal core allows new users to easily build a simple community website with a number of common/popular features and teaches them how to easily extend their website (e.g. with contrib modules) as its needs grow.

    Obviously, "common/popular features" can never be defined exactly, but things like the State of Drupal survey (and looking at other web trends) can help make that more concrete.

***

Also, regarding "Golden Contrib" and in particular Michelle's comment in #146:

Having the separation would mean that a development branch could be opened up on just the module that works on new features at any time, with the only requirement being that there is a stable release available that matches the supported core versions at all times.

I don't see why we need to move a module out of core and into "Golden Contrib" for that to happen. If you want to create a contrib project that effectively serves as the "development version" (a.k.a. 7.x-2.x) of a stable core module, what stops you? As far as I know, people already do this in some cases (e.g. backporting features that were committed to Drupal 7 core to a Drupal 6 contrib project).

There may be some technical advantages to having each Drupal core module be a separate project (given the way Git repositories work) so it's worth talking about in general, but I wonder if it should be a separate issue. Drawing a line between "core + potential future golden contrib" and "plain old contrib" and figuring out what should be maintained where is complicated enough on its own :)

David_Rothstein’s picture

Following up on some other side comments above:

  1. rootatwc (#85):

    I dont understand it.This is posted in Drupal Framework's issue queue.Maybe it should move to core?

    I don't understand this either. Shouldn't this issue be moved to the Drupal core issue queue, where more people will notice it (and given that it's actually about Drupal core)?

    I also discovered #908022: Small Core / Drupal Distro, which is an older issue in the core queue which seems related to the discussion here.

  2. kika (#126):

    Could somebody create a separate issue to deal with all those shortcut+toolbar+overlay intertwinings, both from UX and API side? It does not matter if those modules will be touched with axe, they need to be properly separated first.

    For shortcut + overlay, there is already #782652: Shortcut-specific code shouldn't be in the Overlay module.

    For shortcut + toolbar, they aren't particularly intertwined as far as I know, but see #1262966: Make the Shortcut module depend on the Toolbar module (closed - won't fix) for further discussion.

catch’s picture

Project:Drupal Platform» Drupal core
Version:» 8.x-dev
Component:Code» other

I'm going to go ahead and move it to the core issue queue.

scroogie’s picture

Project:Drupal core» Drupal Platform
Version:8.x-dev»
Component:other» Code
Status:Needs work» Needs review

I think #157 makes some very very good points. So just to recap a bit:

- Drupal will always be framework & product, but "we are never going to have the entire framework or the entire platform in core"
- It's actually desirable to have both, as e.g. the framework features would be developed in a vaccum if we didn't, and many problems (performance, API, etc.) wouldn't have been discovered without.
- There are other possibilities to help in both directions (project browser, more core-like development process for contrib modules, etc.)

So the whole issue is one of compromise and trade-off. The post contains many criteria as well:

- "maintainability" in contrib: it says that very low-level stuff is not maintainable in contrib if I understood that correctly. I'm no good judge on this, but who is if not catch.
- being so useful or somehow "central" so it won't get overlooked for years. In my eyes this is both usefulness for end-users as well as for core (is it used by other parts of core?)
- maintenance and code quality status
- does it have well-maintained competitor in contrib
- does it have dependencies that make it harder to maintain
- does it hold up important refactoring of other components

I hope I didn't miss stuff, I just didn't want it to drown. So everyone go read #157. :P

scroogie’s picture

Project:Drupal Platform» Drupal core
Version:» 8.x-dev
Component:Code» other
Status:Needs review» Needs work

Argh, sorry for cross-posting.

scroogie’s picture

Issue summary:View changes

consistent formatting

DjebbZ’s picture

So the issue is now talking about defining criteria to split core functionality, whether it's useful or not, whether it's maintainable or not. Here's a recap of proposed heuristics :

eaton in #94 :
1) Does the module provide 'Product features' for people who want to blog/discuss/etc (as opposed to 'Framework features' for people who want to construct a custom CMS?)
2) Do the most active core developers feel that it is essentially a "waste of energy" to maintain in core?
3) Can equivalent functionality be implemented using other tools available in Drupal core?
4) Can equivalent functionality be implemented using popular and well-established contrib modules?
5) Is the module's functionality 'uncoupled' from the essential parts of the Drupal framework? (aka, 'We can't dump System module')
6) Has the module's feature set stayed relatively static over its time in core?
+
7) Does this functionality improve the "first four hours" of a Drupal evaluator's experience with Drupal core?
8) Can the functionality be implemented simply, both from a UX and a development standpoint?

Crell in #111 :
1) Framework functionality that is used to build application-level functionality that cannot be easily replicated in contrib.
2) Application-level functionality that is difficult or impossible to implement in contrib.
3) Application-level functionality that supports a pleasant initial experience with Drupal for new users, and encourages them to further explore Drupal's offerings including contributed extensions.

catch in #157 :
1. Is it high quality (or at least of comparable quality to the rest of core)? (trigger module is about an D, comment module is about a C or B- although there is a viable plan to improve comment module via entities).
2. Does it have viable, popular competition in contrib (trigger module has rules, comment module doesn't - in D6 there was http://drupal.org/project/nodecomment - but this is largely superceeded by fieldable comments, so in this case core won the competition via refactoring).
3. Does the standard profile (or another core module) make use of it (again, no for trigger, yes for comment).
4. Does it have outside dependencies (OpenID), or multiple internal dependencies (forum) which make it hard to maintain or can interfere with lower level refactoring.
5. Is it built on APIs in core that are already in extremely poor shape an need refactoring themselves (like Dashboard was on block module).

I'm going to sum this up in a big matrix. I hope that's gonna help the issue move forward and help making decisions. I know there has been other proposals as well. Feel free to add them in the matrix. For now I won't put any data in it, the point being first defining the global matrix, then fill it, then make decisions (seems like a reasonable process).

Module/Functionality Framework feature Product feature Waste of energy Equivalent in core  Viable competition in contrib Uncoupled Stayed static First 4 hours Simple to implement Defines core (aka cannot be easily replicated in contrib) High quality ? Standard profile uses it Outside / multiple internal dependencies Built on poor API Needs refactoring
Aggregator
Block
Blog
Book
Comment
Content Translation
Contextual links
Dashboard
Database logging
Field
Field SQL Storage
Field UI
File
Filter
Forum
Help
Image
List
Locale
Menu
Node
Number
OpenID
Options
Overlay
Path
PHP filter
Poll
RDF
Search
Shortcut
Statistics
Syslog
System
Taxonomy
Testing
Text
Toolbar
Tracker
Trigger
Update manager
User
Form API
DBTNG
Cache system
Entity
Installer
Installation profiles
catch’s picture

Could we put the matrix into either a groups wiki or a google doc (latter is easier to edit but harder to add people)? It's already too big to fit into d.o theming and it'll likely get bigger rather than smaller.

We should try to make all the criteria the same way positive or negative - so we can do 1, 0, -1 and tally them.

dixon_’s picture

As an experienced Drupal developer, but as a very new core contributor, I can only agree saying that it's incredibly hard to wrap your head around many issues in the core queue, because of core's growing complexity. So I think that this discussion is very important and healthy to have. Maybe one of the most important discussions we've had regarding Drupal's future...

So, I'm very glad to see how this discussion has evolved. We should definitely push installation profiles further in core and think harder about what cool product features we want to provide to attract new users and contributors. This needs to be discussed, defined and documented for this and future discussions.

How the download is bundled matters less to me, as long as we have clear leaders and maintainers for all parts that is downloadable as a Drupal thing. But naturally, product features needs to change and adopt to markets quicker than platform features does. And good platform supports that. So to me, it makes sense to have product features and framework features living separately with separate release cycles. That also separates our concerns in a healthy way, because we clearly have different interests, but a common goal to make what people see as Drupal better.

So, we need to more clearly define all this, and what our purpose is. This discussion is on a good way there. So let's continue this very healthy discussion. I'm so glad to see that we have so many smart and engaged people in this community.

Owen Barton’s picture

I have put up a rough matrix at https://docs.google.com/a/owenbarton.com/spreadsheet/ccc?key=0AujolEbu-Z... - it is world editable, feel free to go at it there [edit: it has lots of detail in already, so I changed edit access for now to just active core developers to we can refine gradually/carefully - feel free to propose changes in the comments here].

I have added the heuristic from eaton as "negative indicators" and Crell's as "positive indicators", and a simple total column on the right. There may be others from the matrix in #173 we want to add also.

Along the left are features/components - I avoided saying "modules" here, as I we may find some modules have features that break down differently, and as catch describes we need to apply the same criteria to our lowest level components also, so we can see if the heuristic actually work's how we want it to.

DjebbZ’s picture

Thank you Owen Barton. I'm going to add to this matrix, and respect catch's recommandation : 1, 0 and -1.

catch’s picture

I've been using 1, 0 and 0.5 in the spreadsheet in case it helps, same thing..

Shadlington’s picture

The the heuristic seems to be working in the main, judging solely on how the negative totalled modules align with many of the already hit lists that have been shared.
(This is a very superficial judgement I know, but I wanted an excuse to subscribe)

nicl’s picture

+ 1 to sub.

But also, just want to say I'm very much for splitting the framework from the product. The list of modules most of us don't use (forum, blog, etc. etc.), for the majority of our sites anyway, in my mind shows that aiming for an ideal core product is impossible and for most people results in unecessary bloat/complexity (and of course more difficult to maintain).

fgm’s picture

The XML-RPC subsystem evaluation does not seem correct, and I would suggest the following changes for its components, but the file is locked

  • xmlrpcs (server side): on the positive, it would have to be 1/1/0 or 1/0/0 if one takes performance into account, as it has specific bootstrap optimizations (uses its own dispatcher instead of the menu system), or could be 0/1/0 or 0/0/0 if performance is not considered, as its functionality, hooks included, can be entirely implemented in user space (Services offers a similar service with more features already, just less simple in terms of DX). On the negative, "Product features" exist: the module supports remote posting, allowing direct integration with some services like Flickr with just some setup. "Core developers": this is the part of core I work the most on, and do not feel it to be a waste of energy, as 100% of my sites rely on it. "Equivalent": yes, except performance-wise. "Uncoupled": yes, and this is even a problem to some extent. "Static". Significant changes of API happened for D7 (syntax changes, actual multicall support, new hook)
  • xmlrpc (client side): on the positive, it could be 0/0/0 or 1/1/0 since this is an application building brick with no user interaction surface. On the negative, however, there are no user "Features" not implying code. Were the implementation to be moved to contrib, it would basically be raw PHP with little core support.

The main problem with core xmlrpcs, as I see it, is code duplication. It basically reimplements its own routing system with its own dual-syntax "menu-like" system (hook_xmlrpc) and access control (via hook_xmlrpc_alter), all of which could - and probably should - be part of a single unified routing system, as is envisioned in the WSCCI initiative. Also, as a mature technology, predating SOAP, it has every reason to be quite static, and the changes in D7 were the most significant since the library change in 4.7.4. The client part is essentially a bonus using most of the same code as the server part, and because of this more efficient to maintain in the same package than separately.

All in all, this functionality ought to remain in core, and I'm willing to continue maintaining it and ensuring its smooth integration with WSCCI, but there is no reason why it should remain in its present form, and every reason why it should be integrating in the more general core communications services. Given that context, it should probably be turned into a core plugin.

nbz’s picture

Another thing to consider when talking about core maintainability and attracting new users is the release cycle.

Are there many people who have an itch and are willing to work on core to contribute their first patch who are willing to wait 3 years for that patch to be useable in production?

fgm’s picture

@nbz that would be another reason to consider D8 as a target for an actually "finished" D7 with the needed API changes that could not make it into core for 7.0, like some form of Entity API integration, with a shorter timeframe for release, like summer 2012, and all of the current work being done on CMI, WSCCI, and the other longer-term initiatives being squarely targeted at a later release, with the same "3 years after 7.0" time frame as what seems to be currently envisioned for D8: this time scale was mentioned several times on the questions during Dries' last core conversation at DC London, and never contradicted.

This would make a lot of sense both business-wise and technically, and has even been already touched upon (although not in these terms), since Dries and Webchick both reminded us (DC Chicago and DC London) that some (compatible) API changes are allowed during the D7 lifecycle, slightly opening the door for such a nor-really minor/not-really major release for D8.

But is another can of worms entirely.

eaton’s picture

I have put up a rough matrix at https://docs.google.com/a/owenbarton.com/spreadsheet/ccc?key=0AujolEbu-Z... - it is world editable, feel free to go at it there [edit: it has lots of detail in already, so I changed edit access for now to just active core developers to we can refine gradually/carefully - feel free to propose changes in the comments here].

Wow, that's a great mix!

Just a quick point, I think that the "provides product features" and "is felt to be a waste of time" metrics might be adding a bit of extra skew to the spreadsheet. Crell's positive metrics, and the more specific negative metrics, capture a lot of what I was thinking of. At the time I wrote up that list, I was attempting to capture what I felt were the unspoken metrics in use by various parties, even if they had some overlap. If we turn all of those metrics into our 'master list' I think we'll end up heavily penalizing modules, since most of the negative metrics capture why core devs think the modules are a waste.

nbz’s picture

@184 - the idea of having an interim release is interesting to me. Maybe with some API changes too (to allow for comments for ann entity types, and the movement of certain modules from core into contrib), so something more than a point release, but less than what "drupal 8" is meant to be.

David_Rothstein’s picture

Some comments on the spreadsheet numbers (I don't seem to have permission to edit but it's better to discuss changes anyway):

  1. Has the module's feature set stayed static

    Why do contextual and toolbar get a 0, but overlay, shortcut and dashboard get 1? All these modules were added late in the D7 cycle, so it's almost meaningless to rate them this way anyway :) If rating them instead on whether or not there were a lot of diverse contributors from the Drupal community volunteering to help implement the initial set of features (probably a more reasonable metric for newly-added core features), I'd say something like:

    contextual: 0, overlay: 0 (lots of contributors helped implement features)
    toolbar: 0.5, dashboard: 0.5
    shortcut: 0.75 or 1 (fewest contributors helped implement features)

  2. Supports a pleasant initial experience

    Not sure how Aggregator gets a 0 but Open ID gets a 1? They are both in the category of integrating Drupal with other websites, and between those Aggregator is likely more useful for a new user building a simple website. Maybe at least:

    aggregator: 0.5, openid: 0.5

    In this category, Blog and Poll are also arguably higher than 0, but I'm not sure they make it all the way up to 0.5 either :)

  3. Is there equivalent functionality (core) or equivalent functionality (contrib)

    Not sure how Blog gets a 1 for both of these? You can't really build a meaningful multi-user blog with the rest of core. And you can with established contrib modules, but it's not exactly point-and-click. I'd go with:

    blog (core equivalent): 0
    blog (contrib equivalent): 0.5.

  4. Dashboard

    This should really get a +1 or so somewhere (it's debatable which category) for the idea that you need a centralized dashboard system in core in order to convince contrib module authors to actually rally around writing blocks that would be appropriate to put on an administrative dashboard. Not sure whether that will happen or not in D7, but the idea makes sense in theory. For that reason, I'd give it:

    0.5 or so each in the "provides framework features" and "difficult to implement in contrib" categories

  5. Difficult to implement in contrib (framework) and difficult to implement in contrib (application):

    I can't figure out how dblog and syslog got a 1 for each of these? They are basically just hook_watchdog() implementations and could easily be implemented in contrib. Dblog does have some menu callbacks that other modules can link to, so I'd go with

    framework: dblog: 0, syslog: 0
    application: dblog: 0.5, syslog: 0

The above changes would not shake up the list drastically, but would definitely alter it noticeably.

David_Rothstein’s picture

Just a quick point, I think that the "provides product features" and "is felt to be a waste of time" metrics might be adding a bit of extra skew to the spreadsheet. Crell's positive metrics, and the more specific negative metrics, capture a lot of what I was thinking of.

I noticed that too, especially for the first one. For example, the site-wide contact form seems to get a +1 for being a common product feature that new users look for/expect, and then gets a -1 for almost the exact same reason :)

However, is it possible this category could be kept, but rephrased more like "is the feature a one-off solution that is not reusable/extensible by people who want to construct a custom CMS"? In that case, the Contact module would still get a +1 and -1, but if (for example) Webform module were in core and the site-wide contact form were implemented as a default webform, then it would probably be +1 and 0. So there is actually a meaningful difference.

catch’s picture

With implementations of pluggable APIs (dblog, syslog, db drivers) they are not hard to implement in contrib, but arguably we need to keep at least one implementation in core (shipping core with no working logging, cache, lock or database back end would be strange).

To me one implementation is a requirement, more than one implementation is a feature. This is not something that fits well in the spreadsheet, and it somewhat applies to menu module vs. book (or even menu vs. toolbar) as well in terms of application stuff.

Maybe we could add "API + implementation" as rows for those sections?

I agree that aggregator and OpenID are either both 0, or both 0.5 or both 1.

I think we should add a column for "does anything in core depend on it at all (excluding tests)) - this would for example highlight all the modules that are enabled by the standard profile, vs. the ones that are not enabled by any core install profile, which I think is a useful measure - i.e. d7ux modules are enabled, we added an image field to the standard profile. Poll, trigger, statistics, these are only accessible if you go to the modules page and enable them.

scroogie’s picture

I don't understand what the "Has the module's feature set stayed relatively static over its time in core?" metric is expressing here. Could you perhaps add a sentence or two justifying the metrics?

I also feel that two points that have been mentioned in this topic are not really represented in the matrix:

1. the "maintenance burden": How much relief would it actually be to drop the module? How much work is it to maintain it with core? Or is this what is meant with the "Do the most active core developers feel that it is essentially a "waste of energy" to maintain in core?" metric?

2. the architectural / design question, also regarding future plans: Does the module / component fit into the state of the art or is it based on deprecated API? For example the profile API earns a big negative here, as its a quite a replication of functionality. Also I wouldn't be sure with the comment module, as its still based on nodes, while in my opinion it should be based on entities.

kika’s picture

Would "rough amout of bugs against the component" be helpful metric as well (not contributing to overall score, just giving more background)?

Owen Barton’s picture

I have added an additional sheet on the doc that displays all items in a sorted list (it is a formula so will update), which seems like an interesting way to look at things.

arianek’s picture

Still on board with most of this except Contact - why is this on the chopping block? It seems like an extremely low-maintenance module that provides simple but great functionality upon installation. Even the per-user contact forms are a great feature for eg. to be able to pitch to clients as a feature that it ships with. If it's posing virtually no maintenance burden, why remove it (or part of it)? Having to add a contrib module just for per-user functionality seems kind of silly.

scroogie’s picture

Yes, this is what I meant in #192 as well. It's an important point that is currently neglected.

catch’s picture

The issue with modules that have static feature sets and are simple like contact is that they'd be really easy to maintain in contrib. This isn't true for say forum module where it has tonnes of dependencies in core. This counts both for and against them, but one argument against moving modules to contrib is that they go there to die, if they're frozen they'll last a long time anyway.

Contact isn't only replacable by webform, per user contact forms also competes a bit with privatemsg module - which is a top 120 module (although a lot more to it than contact has).

I also vaguely remember running into issues years ago when there was both contact forms (which iirc can only be disabled on a per user basis via a permission, there's no actual setting) and privatemsg on the same site - and someone not familiar with Drupal had access to both and got confused. Unless something's changed you can't have one feature without the other at the moment.

bleen18’s picture

subscribing ...

re Catch (#192): We all know how annoying the subscribe/+1 problem is but its unfair to expect people not to follow such an important discussion because we (still) dont have the ability to flag an issue on d.o..

catch’s picture

@bleen18: I'm fine with 'subscribe'.

However +1 indicates some kind of approval/vote, not just "I want to see what happens", which doesn't make any sense in the context of this issue since there's multiple versions of ways to go with it.

catch’s picture

Issue summary:View changes

Updated issue summary.

Michelle’s picture

... Or how about just weighing in on the issue? If it's important enough to follow, it's important enough to share your opinion. :)

Michelle

David_Rothstein’s picture

I have edit access to the spreadsheet now, so if there are no objections I will go ahead and make the changes outlined in #189.

Re syslog/dblog/etc (@catch's comment in #191): yeah, I assume we should be viewing the numbers that come out of this as a guide, but not anything like an exact formula. (Regardless of what the numbers say in the end, I definitely don't foresee both dblog and syslog being removed from core as a result of this issue, because that wouldn't really make any sense.)

oskar_calvo’s picture

Although the contact module it's a great módulo, I think also that it should be an entity, and also should be out because it's a product, that give a ui to configure.

In d7 can be build a entity that make the same thing.

In fact, in d8 contact should be a new entity and have fields options.

Please take out of the core contact and leave alone as a contrib module.

thanks

Oskar

TonyK’s picture

There is no real need to keep most of modules in the core. I think only three of them really should be there:
- System (by obvious reasons);
- User (permissions, roles, etc);
- Menu (for maintaining pages, etc).

lelizondo’s picture

That would be minicore and I think we should have a reasonable balance, for instance modules like search, comments, taxonomy I understand that would be very difficult to implement in contrib. There's also some modules that were added in D7 that should remain in D8 like RDF and Fields in core.

arianek’s picture

@catch thanks for explaining further.

i think i am holding onto some of what my first experience building a drupal site was 5+ yrs ago was, where i was able to set up a basic site so easily just with core. it was the ease of this initial experience that let me to want to learn more and add to the site. i guess i will have to trust that some good install profiles will provide this same initial experience for users, despite ripping a lot of these "easy setup" modules out.

effulgentsia’s picture

@arianek: you're not the only one holding on to that nostalgia.

Dries from #154:

That said, my goal (heuristic) for core is to give both newbies (primarily) and experts (secondary) a good initial experience with Drupal.

I think we all want it to be easier from within a Drupal core installation to be able to easily find and install additional contrib modules (#1164760: [meta] New users universally did NOT understand that you can extend Drupal), and we all want to see Snowman and other products flourish. Until we have confidence that one or both of these will actually happen, however, it seems like there's this pressure to hedge our bets a bit, and keep some product-oriented features in core, for now.

hgurol’s picture

@effulgentsia
...keep some product-oriented features in core, for now

Thats not we all want. You think wrong.

effulgentsia’s picture

@hgurol: please reread my last paragraph in #209. "I think we all want ..." applies to the first sentence only, not to this part:

Until we have confidence that one or both of these will actually happen, however, it seems like there's this pressure to hedge our bets a bit, and keep some product-oriented features in core, for now.

My own opinion is that I'd like to eventually see the things in #43.5 separated from the rest of core, but only after we have a solid plan for at least one robust community maintained distribution (which I hope Snowman becomes) to incorporate the ones that we believe are necessary for newbies to have a good initial experience with Drupal. Sorry if I implied that to be a unanimous opinion.

TRB143’s picture

Reading this thread with interest from a users perspective the only module proposed to be removed that I used was Aggregator.

As I am in the process of building a new D7 site and using Aggregator I have taken the opportunity to migrate over to Feeds.

From my perspective Feeds in much better (but could be a bit clearer but that is a different thread) when combined with Views and custom CSS.

Feeds is more flexible in it's handling of field mappings and combined with Content Types and Fields is very easy to follow.

ressa’s picture

I agree that the very specialized modules which few people use, and the modules which require a lot of effort, but return very little value (for example the "pretty" Overlay module) should be removed from Core.

eaton’s picture

@effulgentsia - you make a very, very important point about the future dream vs. the present reality. For better or worse, "feature modules" like Blog, Forum, and so on are the first indication of Drupal's flexibility that new evaluators have. Other solutions -- like building out similar functionality manually using content types etc., diving into contrib and finding good alternatives -- require a level of comfort with "drupalisms" and module evaluation/configuration.

One of the challenges is that traditionally, "specialized" modules like Blog and Forum could add a couple of special tricks (say, hierarchical listing pages and breadcrumb control). Those tricks are what's difficult with core-building-tools alone. Which is where the Snowman inspired patches like #1210366: Per-bundle node listing pages, blocks, feeds. come in. I'm definitely in favor of moving some of those 'toy car' modules out of core as long as they're matched by some improvements to the 'building blocks' that it comes with to balance things out. If too much feature-functionality is removed, projects like Snowman will have a much harder time doing their work.

oskar_calvo’s picture

@TonyK

add entity and fields (in fact fields are entities) and we will have the perfect smallcore.

Oskar

cpelham’s picture

Why do we need some toy product functionality on first download if there is already much better user-friendly product functionality out-of-the-box with Acquia Gardens and Lullabot's product, etc.? Can't we just point to each of those? and then say if you'd rather roll your own, download Drupal core and explore contrib?

David_Rothstein’s picture

Product functionality has been part of Drupal from the very beginning and is an integral part of what Drupal (a.k.a. the software on drupal.org) is all about, so it wouldn't make sense to outsource that to hosted solutions run by particular companies.

This issue is only tangentially related to "first download" experience anyway; we're mainly talking here about what should be maintained as core, which is not really the same thing as what you do and don't get in a particular download. For those other issues, see, e.g. #1164760: [meta] New users universally did NOT understand that you can extend Drupal (for improving the experience of exploring contrib from within Drupal core) and #1260214: Choose your Drupal (for improving the experience of finding other distributions/install profiles as opposed to just downloading Drupal core).

Also, I'm not sure what (if anything) currently in Drupal core constitutes "toy product functionality"... if there were a lot of that, it would be a whole lot easier to figure out what to remove :)

eaton’s picture

Product functionality has been part of Drupal from the very beginning and is an integral part of what Drupal (a.k.a. the software on drupal.org) is all about, so it wouldn't make sense to outsource that to hosted solutions run by particular companies.

Seconding that. Drupal Gardens, Buzzr, SubHub, and other Drupal-based hosted site building products are certainly cool, but they aren't a substitute for actually building useful software. Also, as has been noted in other threads, building at least one "product-style" installation profile that ship with core would allow us to exercise the APIs and underlying building blocks that ship with Drupal core, verifying that we haven't cut too deeply or painted the people doing Buzzr, Gardens, and SubHub style work into a box.

My ideas about how to best approach the core/contrib/product split have been shaped in large part by my work in building Buzzr (and the pain points I discovered along the way), my conversations with people who've worked on building Gardens, and extensive work with UX and IA folks on other Drupal projects. I'm a huge believer in reducing complexity in core, but we can't simply punt the problem of user affordances and the evaluator experience to hosted commercial services.

catch’s picture

we can't simply punt the problem of user affordances and the evaluator experience to hosted commercial services.

Couldn't agree more with this.

giorgio79’s picture

Lot of arguments in this thread, but lets admit many of them are guessing on the need for and popularity of Forum, Blog and many other modules. Without cold hard statistics these are just guesses whether they contribute to initial user experience or Drupal success in general.
We should strongly push for measurable module installs, so we know what we are talking about, and it wont end up on "who is a more eloquent speaker" but on simple cold hard facts.

Also, Blog and Forum are not enabled in the default installs, so all the more for stats, as that would express a clear need or interest for it...

I made the argument on Sun's post that the inclusion of Toolbar instead of Admin Menu (180 000 installs!!!!! ) was influenced by what a select few think, instead of what actually is demanded by Drupal webmasters. Someone in the toolbar dev thread told me install counts dont matter... Well I dont know where that person works at but at multinational companies if you cannot measure it, it does not count...
It comes down to numbers vs how persuasive you are. I would go for install count... (Oh and I personally dont think that Forum and Blog should be in core as mentioned, but that's just what I think :D, it could be they are enabled on all the 400 000 Drupal installs)

catch’s picture

Lot of arguments in this thread, but lets admit many of them are guessing on the need for and popularity of Forum, Blog and many other modules. Without cold hard statistics these are just guesses whether they contribute to initial user experience or Drupal success in general.
We should strongly push for measurable module installs, so we know what we are talking about, and it wont end up on "who is a more eloquent speaker" but on simple cold hard facts.

That's a nice idea except:

- popularity is one metric for modules, however this issue has identified many other things to look at when figuring out whether something should be in core or not.

- blocking the removal of any feature from core on being able to measure it's usage restricts this conversation only to modules, and ensures that it will not progress for another 3-4 years while being blocked on per-module usage stats being implemented then collected.

- it is impossible for optional core modules to be enabled on more sites than required core modules, or for modules that specify dependencies to be enabled on more sites than those that don't. We don't need per-module usage stats to figure out for example that node is enabled on more sites than comment, or comment and taxonomy are enabled on more sites than forum.

eaton’s picture

Lot of arguments in this thread, but lets admit many of them are guessing on the need for and popularity of Forum, Blog and many other modules.

Well, I can't speak for others but I'm arguing in favor of providing pre-configured functionality that can be easily enabled and used by Drupal evaluators. That's not about popularity, though I suspect that there are more than a few Drupal sites that use the Forum module. It's about providing affordances for people who are checking out Drupal, kicking the tires, and determining whether it is worth spending time climbing its Learning Cliff.

Long term, I'm 100% in favor of improving our underlying 'raw building tools' like entity management, field management, hierarchy/outline creation and management, menu systems, and so on -- and demonstrating Drupal to people by building out useful functionality from those elements in install profiles. Right now, however, the "turnkey" modules like Blog, Forum, Poll, and so on are the only real demonstrations that don't require reading FieldAPI tutorials and investing considerable time in learning somewhat advanced/complex tools.

I agree that gathering stats on the number of Drupal installs that use specific modules would be worthwhile, but it's also possible to advocate for certain approaches based on something other than raw module popularity.

catch’s picture

Right now, however, the "turnkey" modules like Blog, Forum, Poll, and so on are the only real demonstrations that don't require reading FieldAPI tutorials and investing considerable time in learning somewhat advanced/complex tools.

I'd say the default article type in the core install profile provides about as much functionality with zero configuration as blog or poll. You get a tags field and image upload with that which gets you further towards a very basic single user blog (and something that could be used as part of a more complicated site) than either blog module or poll do for their general use cases. I've been considering a patch to break this out into a separate module rather than the standard profile .install (so it could be added as a dependency instead) but not sure if that issue would make this one easier or harder to discuss.

Book module is pretty close to this (for hierarchically structured set of pages) as well, without being as specific as blog or poll.

catch’s picture

eaton’s picture

I'd say the default article type in the core install profile provides about as much functionality with zero configuration as blog or poll. You get a tags field and image upload with that which gets you further towards a very basic single user blog...

But single-user blogging isn't what Blog module supports -- it's for multi-user blogging, which for better or worse is actually a pretty useful feature on certain kinds of sites, like small group web sites where the staff/core members/etc blog publicly and other members participate in forums, etc. The things that make forum, blog, and poll unique are their listing pages, interaction flow, and their unique chrome. For example, blog module provides the author of a blog with a 'Post a new entry' link at the top of their own blog, manages breadcrumb trails, and so on. Poll provides a custom 'latest poll results' block, and Forum firehoses table cells and rows because boy, howdy.

I think we've got three possible choices ahead of us in areas like that:

  1. Keep them in core, because they can't be duplicated with current core "LEGO Bricks"
  2. Improve our core "LEGO Bricks" with things like simple listing pages, better breadcrumbs, etc. until reasonable alternatives to these modules can be constructed with them.
  3. Remove them from core, and don't try to replace their functionality in any way (other than contrib).

I tend to favor #2 whenever it can be done very simply and efficiently. Again, I'm biased in favor of the Per Bundle Listing Pages patch, and a couple of other general purpose improvements like more flexible book pages... I think that "core" should focus exclusively on providing a good starter library of LEGO brick style site building components, as well at least one pre-assembled "Example model" that's built using just those pieces. (Thus, the Snowman concept). I understand that others may disagree, tha'ts just where I'm coming from. At the very least, I think it's important to remember that the "little extra bits" on modules like Poll, Blog, etc. are the reasons why they're extremely useful and well-suited to particular kinds of tasks.

lelizondo’s picture

I've been using Drupal for 3 years now. The first year was really hard. Since then, I can say that I haven't built a single site that I haven't tweaked a lot. I always change most of the "default" functionality and for the last two years I've been using it more like a framework than a product.

For me, Drupal is almost useless out of the box. I never managed to do anything that I really wanted to do without touching code, at least at the theme level. I always end up using hook_form_alter, custom modules, and a bunch of modules from contrib.

I get the intention of including more newbies so Drupal can grow, but honestly, if a newbie downloads D7 for the first time, it will be difficult, confusing and it probably won't do much with it. A newbie needs a distribution, like a blog distribution or an intranet distribution, not core, even as it is right now.

For me, to make core usable out of the box, we need to add like 20 modules including Views and like 5 different themes, and that's not an option.

oskar_calvo’s picture

Well, I can't speak for others but I'm arguing in favor of providing pre-configured functionality that can be easily enabled and used by Drupal evaluators.

The goog things are that Drupal can build profiles or product easily,and there are a lot of install profile in drupal.org. We don't need pre-configured functionalities in the core, it's easy to install a install profile.

The better that is our small core the better that will be our install profile or our product.

Oskar

catch’s picture

Sorry I meant in terms of blog that the article content type gets you a better single user blog, than blog module gets you a multi user blog (since blog module doesn't add tags or image uploads which you'd probably want on either).

I definitely didn't mean that blog is for single user blogs, that's part of why I originally posted the issue suggesting we remove it back in 2007 since at that point I don't think it even told you what it was used for.

I completely agree with doing #2, and per-bundle listing pages are a step towards ditching some stuff that's copied around core in tonnes of places (while also not punting that clean-up on a 'views in core' effort and in such a way that it should never compete with such an effort if and when that happens).

However I don't think this is mutually exclusive with #3. We removed throttle module from core in 7.x, despite it not even having a contrib alternative - because it added API and UI complexity quite deep in core (like module_list()) and there are other ways to solve performance issues and traffic that have emerged since it was originally added.

In other words, I don't think we should be committed to keeping feature parity in core with previous releases, just because it's there now. I'm personally committed to keeping certain classes of features, but not specific ones (and there's some I'd very much like us not to any more - like IP blocking and the PHP text filter).

eigentor’s picture

What I understand from the entire initiative is the point of decoupling the layers of drupal.
First thing is to identify "product-like" features at all.
Discussion if things can be removed is somehow leading in the wrong direction as I read people read from it it is gone then.

As Snowman or any other name for that is meant to be kept in the default download, it is somehow still core. And it has to be maintained with similar scrutiny that any part of drupal core has to. I can tell from the enormous issue plethora that had to be solved to get Bartik in. Nobody would ever go through that in contrib.

But there is also a practical difference as to who works on the stuff. A big problem I found in the entire crooked D7UX thing was frontend people constantly got into arguments with say "Platform people" and understanding was hardly possible because two different sides of the medal were seen by either. One talking about the taste of the apple and the other one about the logistics.

If this initiative could come up with a way that the more frontend-oriented and invested people can work more freely on Snowman without messing up the APIs, this would be a huge step forward.

A thinking of basically a KDE or Gnome for Drupal (I know this comparison holds no real technical validation) that seperates the interface from the core has been long in my mind.

How can we get the frontend people to their side and the API people to their side? As a frontend guy I can say it would be a huge motivation if we could work on the interface more freely. And I am pretty sure the more API oriented people would be happy to to have us off their backs :)

Maybe this can give another viewing angle for this discussion and help to get seperation points clear.

catch’s picture

@eigenator, a big problem in core is it's hard to work on anything, without that work touching lots of other pieces of core. That's what #1224666: [meta] Unofficial Drupal 8 Framework initiative tries help solve, but we need to actually do that work first to make this remotely possible.

A big issue with d7ux for me was not front end vs. API, but that the functionality was built on some really out of date APIs in core that aren't up to the task (i.e. dashboard and block). Now if anyone wants to update block module, they will have to simultaneously keep dashboard working too since it is directly using things like the block UI.

cpelham’s picture

Somewhere, amongst Mini Panels and Boxes (if that is the right project name) and Block issues and what have you, there is an entirely new and better conception of these reusable pieces I believe. It's unfortunate that the new dashboard is tied to the old block, but I imagine that most sites use only one if any dashboard and could quickly rebuild it if they had to do so. That would probably be trivial compared to rebuilding all the blocks themselves.

But yes we should try to avoid adding any new dependencies on API that we know will get retired in the next few years! It seems like that problem would be a lot easier to avoid if we had a roadmap to consult, even an unofficial one, that lists all of these initiatives, including unofficial ones, in one place. It is so time consuming to check issue queues all over the place for so many different core and contrib issues simply to get a sense of what is changing when and how. It's not efficient or practical at all, so I can understand how these mistakes sometimes get made.

This surge of macro view future of core threads seles seem like a great step in that direction (making it easier to get everyone on the same page).

Ranko’s picture

I couldn't agree more with giorgio79 in #221. One way to see at least some data would be to enable usage statistics for core modules.

Another option would be to implement a feature outside the Update Status module. An option (perhaps set during the install) that would phone in module statistics in a more relevant manner: once the module is turned on and once the module is turned off. That way we could take a look at what is on, where, when and for how long. That way it would be more feasible to judge if a module is enabled to poke around or for real.

Additional options could be added for people to enter in an organized status like manner:
Is the site a tes/dev/production/other
Predefined list of why the module was disabled, optionally add name of the replacement module.
Etc.

There are a lot of good examples on how to implement something like this: MS Office 2007 redesign video; or the Android Market uninstall process that includes a list of reasons why you are doing it...

j.somers’s picture

My comments might be a bit late, and I might not have much experience with core as others do, but I just want to give some of my insights.

I develop roughly two-three websites per year. That's not a lot and results in very simple websites with an average of 1-5 users. So no big community websites but big enough to require a CMS. My users are computer dummies so everything has to be very simple and easy to understand. Years ago I choose Drupal because a) I did not have the time to write my own CMS systems anymore; b) I did not like the chaos the WordPress code-base was at that time; c) I saw an awesome presentation at FOSDEM 2005 by Dries which convinced me of the power of Drupal. And after the success in developing that first site I kept on using it.

I recently developed my first website with Drupal 7 for a dummy user and I want to mention some of the things I came across and give my opinion on the different modules. I will base my list on the one from chx in comment #31.

  • aggregator: I have never ever used this module. I once made a Delicious module but because none of the things I needed to do with this module were possible, I ended up using SimplePIE as an XML processor and went from there. Does not belong in core.
  • blog: I tried it once, ended up using a simple story content type instead and I still stick with that approach. Even for other websites which more or less have a concept of multiple blogs it's just easier to create my own content type and discard the entire blog module. Does not belong in core.
  • Book: I have used it and I can see the use of it in some websites. The question is whether enough websites use it. Might stay in core.
  • Color: Nice little feature, should stay.
  • Contact: I use it on all my websites to create contact forms. Should stay.
  • Dashboard: Works, but I can live without it.
  • Forum: To be honest, if I want forum software I'd rather choose a real forum solution because the current implementation in Drupal can't compete with things like vBulletin, phpBB and others. Does not belong in core.
  • Overlay: I disabled it per request. Administrators and website editors found it confusing and somewhat annoying.
  • PHP: Never used it, therefore I don't know whether I can give a decent opinion on it.
  • Profile: Used it heavily in D6, did not upgrade any of those websites to D7 yet, can't give a good opinion.
  • Shortcut: Did not have a need for it.
  • Statistics: I always use Google Analytics and therefore this module is disabled. Might be useful in a simplistic way.
  • Toolbar: Users found it easier to select a top-level item and move from there on rather than searching it somewhere on the bottom of the page in the big overview.
  • Tracker: I always disable it. Can be removed from core.
  • Trigger: Disabled. Have no use for it.

Most of the websites I develop are very basic and therefore a lot of modules that come with the default Drupal install are of no use for me and I can live without them. I disabled most modules which are enabled by default and have absolutely no use for them.
I also want to note that I got a comment from some administrator once who started uploading modules of his own to extend the website, on why there were so many modules uploaded by default when they were never going to be used on that website. I had to explain that I did not upload a forum module or a blog module but it was part of the default installation profile. He also thought that the forum module was quite large and used a lot of expensive space on the server. I know that the forum module does not take much space, but I can understand his concern.

I believe the right direction is to minimize the default Drupal install to a bare minimum. Enough to create a simple website for the local bakery which doesn't want anything more than provide some contact information, history page and a list of some special cakes they once baked.

Alternative Drupal installation bundles can be created which ship with extra modules. Another more advanced level of website is the local soccer team. They want rankings, results, squads... A few modules which should be removed from the minimal install mentioned in the paragraph above are required, and possible some contributed modules as well.

It might be a good idea to start from a minimal profile, and provide a wizard like installer which holds configurations for let's say 5 different typical website setups. Select a profile and let it download the required modules.

Owen Barton’s picture

Regarding much of the discussion from #210 onwards, and supporting what I think @effulgentsia and @eaton were getting at in #211 and #214 is that most of the "disagreement" here seems to simply between people talking about the ultimate vision of "Drupal as a platform/kernel/framework/etc" verses what it is sensible to do now. For example, there is a big difference between saying "lets throw this list of modules in contrib today" and "if we had a great entry level Drupal product already available, I would only need this list of modules available in a platform/kernel/framework/etc, and here is why".

I think it would really help this conversation if people could be clear to what level they are speaking on - i.e. vision, strategy or implementation - see my blog post to get a summary of this (albeit a few days old now). More specifically:

  • Vision - if your vision is that the Drupal platform/kernel/framework/etc should only have A, B and C modules, please share why you think this (i.e. what is the heuristic behind your thinking). In the same way (and just as importantly), please try and describe what your vision is for Drupal as a product - clearly we still need to offer a product - what could this product look like, how might people find and use it?
  • Strategy - there are many ways of getting from A to B - moving modules from core to contrib is one, others include the Snowman project, the framework initiative and so forth (again, see see blog post for a rundown). What strategies do you feel are most effective to reach (y)our vision and why?
  • Implementation - what practical things could we do right now to support these strategies? Bear in mind that whilst some things can and should start with incremental changes can start right away, for bigger items the "implementation" would quite likely start out as "more detailed discussion to try and build consensus around this strategy".
nicl’s picture

Re #235 I agree, but surely we can jettison everything now and have a good downloadable product ready for the time D8 is actually released (18 months in the future ? - at least, we know it will be a while).

I guess it depends on whether we can agree on this strategy.

ps. my strategy/ vision etc. is for a separate core 'framework' which is targeted at developers and is very stripped down and easy to customise. And then various products (including perhaps one 'approved product') which are targeted at new users / site-builders and generally non-experts.

Product downloads would be mirrored on installation profiles and could draw in elements of contrib. - I'm not sure I buy into the idea of Snowman if (as I understand it) it seeks to be purely an implementation of core. As I'm not sure a stripped down framework is compatible with a decent 'out-of-the-box' product.

Risks :

- confusion for new people to d.o (what do I download)
- division of effort / lack of clarity over how to present installation profiles (whether to have an 'approved' one etc.).
- creating a framework without the supporting infrastructure required (ie. decent installation profiles / people to maintain these / lack of d.o infrastructure and revamp to suit this model).

webchick’s picture

"but surely we can jettison everything now and have a good downloadable product ready for the time D8 is actually released "

I'm extremely uncomfortable with this way of thinking. This is how we ended up shipping Drupal 7 with Profile module: "Surely 'someone' will have created a fields-based Profile module for core and an upgrade path by the time D7 is released." :( And there is still is no upgrade path from Profile module to user fields in D7 nor D8, nor Profile2 module in contrib, 9 months later.

Putting the very thing that people think of as "Drupal" on the plates of some phantom mystery "product" maintainers that have yet to actually materialize, using the installation profile/distribution tools on Drupal.org which no successful Drupal distribution has ever managed to do, while at the same time splitting our core dev team into "elite, hardcore devs" and "all the other rabble" and raising the barrier to getting involved in core even more astronomically high, and then siccing those elite, hardcore devs on the task of transforming a 'framlication' with 10 years of legacy baggage into something that can directly compete with the likes of CakePHP and Rails in < 18 months is... well let's just say it's not a really vision I can get behind. Maybe in Drupal 10.

Things I can get behind are stuff like actively recruiting more help for the core dev team to assist with the obvious burnout issues, working on various issues linked from #1224666: [meta] Unofficial Drupal 8 Framework initiative and #1242448: [meta] More flexibility for core installation profiles that just make good sense for improving/bringing consistency to Drupal's underlying APIs, fixing drupal.org distribution blockers, and building a team around snowman to try and articulate the vision for Drupal's "product" before indiscriminately chucking out all of the tools they might've made use of to do so. Let's make sure we can walk, before we start running, or we're almost guaranteed to head straight into a wall.

eigentor’s picture

Thanks for the reality check, webchick.
Yes - throwing out things from one level of the needs a solid strategy where they go in the Framework Layer Model http://ca.tchpole.net/node/4
So maybe this should be the topic of another issue, while this is more about moving things out from the lower levels?

nicl’s picture

Ok, I agree with what you are saying - and indeed I do identify these risks at the bottom of my post. Perhaps, I can restate my position as 'this is where I'd like drupal to go' while appreciating the risks in getting there and that there are a lot of things that need to happen before it gets done.

But having said that, while I was being stupid in saying 'jettison everything' I still think we should dump quite a few of the modules listed. Why? Because I have never (seriously !) used any of them. And nor am I aware of my colleagues using them either.

The functionality exists (and in better form) in contrib - google analytics, etc. etc.

This is of course my personal experience only (and I am not even a very experienced developer) but it is the only metric I can go by from my limited experience.

I think if we are really being honest, the idea of drupal being usable 'as is' for nearly any realistic use cases is kind of a false one. So it is already serving as a framework (albeit with some product elements) for most people.

ps. re-reading Webchick's post again, she does make a strong case against a pure 'framework' (so when I used the word 'framework' above, I perhaps would have preferred to use 'a lighter version of core').

pps. also, I guess I may have got caught up in the excitement of Drupalcon (my first) and the recent discussions so it is useful to have a more experienced perspective on how difficult some of these things will be to achieve.

Michelle’s picture

"Because I have never (seriously !) used any of them. And nor am I aware of my colleagues using them either."

There are hundreds of thousands of Drupal users. Going by what any person or even a group of people think isn't useful because they don't use them isn't a useful metric. For every module you don't use, there will be someone out there that Must Have that module and will fight its removal.

Michelle

nicl’s picture

Hi Michelle - of course this is only my opinion (and I say as much later in the post). And I guess one could say the plural of opinions is not data (to coin a phrase). Nevertheless, how as a community can we get a sense of which modules are useful if people don't express their opinions on what they use? Having said that, maybe this particular thread is not the avenue to do so.

Michelle’s picture

My point is that all of them are useful and all of them are not useful if you go by who uses them depending on who you ask. If we had numbers of how many sites have them enabled, that would tell us something but we don't have that. And just having a bunch of people say, "I don't use forum so it should be removed" doesn't really get us where we need to be.

Better to look at other issues like: is someone willing to maintain it in core? is someone willing to maintain it in contrib? If it is moved to contrib, can we still have it as part of the initial Drupal experience? does it need to be? etc...

Michelle

David_Rothstein’s picture

Nevertheless, how as a community can we get a sense of which modules are useful if people don't express their opinions on what they use?

One way we could partially do that is via #1036780: Drupal.org should collect stats on enabled sub-modules and core modules which is the issue @emmajane linked to in one of the comments above.

catch’s picture

We can also look at contrib modules with dependencies and large sites.

advanced_forum is used on over 20,000 sites. I'd be very opposed to removing forum until artesian is up and running (but fine deprecating for artesian if that works out as we all hope it does - similar to profile module and profile2).

Book and forum are both used on Drupal.org extensively.

Rules is used on 60,000 sites. those sites probably aren't using trigger.

Better to look at other issues like: is someone willing to maintain it in core? is someone willing to maintain it in contrib? If it is moved to contrib, can we still have it as part of the initial Drupal experience? does it need to be? etc...

That's really what a lot of this is about though. Eaton was going to try to work up some positive (only positive) heuristics for core features that we could then evaluate existing (and proposed new) functionality against - will ping him to see if he got far with that.

oskar_calvo’s picture

@Michelle @nicl

I think that the idea is not if a module it's used or not, the point is if it's a product or not.

Forum, blog, etc... are products ready to be used, not a framework.

The idea is to have a small-core, and everyone should be able to build it's how site.

Oskar

bleen18’s picture

@catch: I'm curious about this statement:

Book and forum are both used on Drupal.org extensively.

What do you propose this indicates?

catch’s picture

@bleen18: for one that we'd need an upgrade path before we could upgrade Drupal.org to Drupal 8. There was a time when d.o ran on release candidates for the new core version, and that used to iron out bugs in core as well as getting some contrib modules upgraded. Also we know that forum is both high use, and in use on high traffic sites.

It's easy to find lots of high profile sites using comment module or taxonomy module as well.

I can't remember the last time I saw a Drupal site using poll though. So I'm mainly just trying to dispel the idea that we have zero data on how much any of the optional modules in core are used - there is a middle ground of actual data between collecting per-module stats and personal anecdotes.

tsvenson’s picture

@nicl:

The functionality exists (and in better form) in contrib - google analytics, etc. etc.

Google Analytics does not solve all the needs, there are also many alternatives that are preferred by many site owners.

For example, the statistics module provide read count as well as the needed data for the popular articles. As pointed out by others, using Varnish doesn't (currently) work with this. On the other hand, Google Analytics does not provide full data and all modern browsers now has an opt out setting that many sites respect, meaning that the data for many users will not show up in GA.

Thus, for many thing to work properly, such as popular content, Drupal must have features that make sure that data is as relevant as possible, its the only way to ensure that!

tsvenson’s picture

@catch: Was actually toying with the idea of proposing that a new major release can't be made until *d.o is running on it. Could be seen as the last litmus test ;)

Lars Toomre’s picture

#249 proposal makes sense for the release of Drupal as a product.

philbar’s picture

Nice to see some progress!

Cross-reference:
#908022: Small Core / Drupal Distro

Michelle’s picture

@catch re" "That's really what a lot of this is about though." I know. :) I was trying to steer it back to that rather than going down the "I don't know anyone who uses this so let's rip it out" tangent. :)

Michelle

webchick’s picture

Assigned:sun» Dries

Going to add this to Dries's list (sorry sun!). Though he already chimed in above, the discussion's wandered quite a bit since then.

chx’s picture

#250 or so completely shows that this issue have reached its viable lifetime many moons ago and lost its touch with reality. There are a ton of contrib modules running on drupal.org and holding up the release on those is just asinine and as a litmus test doesnt buy us much. It takes a good six months at least past release for contrib to begin to shape up.

I am completely inclined to close this -- I can't fathom why should we waste Dries' time with reading all the going back and forth for 250 comments in here. I can sum it succintly:

  1. We want to remove core modules or parts of them. However, separate issues with solid technical arguments are underway for these.
  2. Product. We need one. People want it in contrib. Can't do that yet for lacking the tools, the process and the culture. Not to mention first we need to make a much cleaner Drupal kernel to be able to better ship products on top.
cweagans’s picture

Exactly what chx said is correct, but there is a third component:

We need to establish a baseline for what should and shouldn't be in core and use that baseline for deciding what we should rip out of core. Without that concrete baseline, we're still shooting in the dark wrt module addition and removal.

catch’s picture

OK here's one easy, positive metric:

"Things that should be in core are things that are required by at least one core install profile."

In terms of modules this is currently http://drupalcode.org/project/drupal.git/blob_plain/HEAD:/profiles/stand...

The minimal profile does not use any modules that aren't in here.

So we can take sun's patch, remove anything in this list, then also remove anything that Snowman uses.

I'm sure there are some modules which are discussed in the other issues that will be left over, those are the ones to think seriously about removal of. This begs the question what Snowman will need of course.

cweagans’s picture

That works for modules and themes, but what about other subsystems? For instance, what about the XML-RPC server code? Actions?

catch’s picture

For those I'd look at:

- does anything in core depend on them? (if it does then they can't be removed without removing that first anyway)

- ways they could be used to clean up core code - i.e. do we re-invent them in some place that we could instead add a dependency (actions was supposed to help with this when it was committed but that work was never done).

- do lots of contrib modules depend on them (this is true for tokens, which is not used much in core (although more than actions is)).

- are they actively maintained?

cweagans’s picture

Okay, so the revised issue summary that I'd like to post is as follows. Please proofread and post if you like it:

The Problem

Drupal core is not maintainable anymore. Too much cruft. Too many half-baked features that no one actually maintains. This causes the following issues:

  • Some core features compete directly with better solutions in contrib, so people don't use them or they do and their expectations aren't met.
  • Having a large number of features in core means that any refactoring often has to touch a lot of code (including ones trying to decouple things).
  • Several core contributors are getting burned out fixing things that 1. they don't use themselves 2. They don't believe many other people use either), especially in the context of what is currently a very, very high bug fixing workload.
  • There is no definitive direction for the Drupal Product in core, nor is there a way to turn off the core product and allow contrib to build out an alternative product on top of the Core Kernel.

The Solution

  • Use a new, concrete heuristic for deciding which features actually belong in core. The proposed heuristics are as follows: Modules and themes which are not used by a core installation profile should not be included in core. Subsystems such as actions or the XML-RPC server code should only be included if
    1. something in core depends on them,
    2. they clean up core code, reduce duplication, or make core more understandable,
    3. lots of contrib modules depend on them, and/or
    4. they are actively maintained
  • Start applying those heuristics to specific features in core to see how they apply - this is being done in separate issues and in 'lists' posted to this issue. Please do that only in the context of developing the heuristics, kill vs. keep is not that helpful.
  • Find maintainers for features in core that are unmaintained or poorly unmaintained, but which meet the criteria for "things that belong in core". Make sure there's a plan somewhere to fix bugs in those features or bring them up to date. Set deadlines for some of this and actively look for people.

I think we need to consider the needs of Snowman before we start tearing things out of core, though. Either that or get a specific roadmap from Snowman and make sure the dependencies are in place through use of existing code or completely rewritten code/code taken from other projects (Symfony, $your_favorite_framework_here).

Personally, I'm in favor of taking code from other frameworks as much as possible. It's add lines of code for us, but we don't necessarily have to maintain it. We just have to keep it updated.

David_Rothstein’s picture

... nor is there a way to turn off the core product and allow contrib to build out an alternative product on top of the Core Kernel.

I'm not sure what this means. At least for large parts of the "product", it's very easy for a contrib install profile or distribution to hide it completely and replace it with something else.

(Not to say it couldn't be made easier, but the bigger issue preventing that is e.g. breaking up system.module into smaller feature modules, not removing existing stuff from core.)

The proposed heuristics are as follows: Modules and themes which are not used by a core installation profile should not be included in core.

Why shouldn't they be? (Also I don't think that's what @catch was saying above either - it sounded to me like his suggestion was more like this is the list of candidates to remove from core, but they still need to be evaluated on other metrics.)

We also have a nice spreadsheet attached to this issue with a good start at heuristics in it already; if the proposal here is to scrap that and start with new ones, I don't understand why.

Michelle’s picture

"Some core features compete directly with better solutions in contrib, so people don't use them or they do and their expectations aren't met."

Be careful with this metric. I've been thinking a lot about this lately because of my forum work. I'm going to be diving into Artesian in less than a week and my goal is to make a total replacement for core forum rather than one that goes over the top like Advanced Forum. When I first started planning this last May, my thought was that Artesian would deprecate Advanced Forum and possibly even core forum as well.

But I've changed my mind on that. Artesian will be big and feature rich but what if you really only need small and simple? What if your site isn't a true community site but you'd like to have a forum just as a small side discussion area? I'm not saying we must keep forum in core but I think we shouldn't be quick to chuck it out, even once Artesian is ready for use. Having something small and simple that you can just enable right out of the box has a lot going for it.

Poll is the same way. Sure, it's pretty primitive and not good if you need fancy polls or do polls on a regular basis. But what if you are thinking of making a change to the site and want to throw up a quick poll to see what the users think? Again, being able to just turn something on that you already have is quick and easy.

I know blog is already out, but a similar argument can be made for that. The main issue there is confusion because you don't want to use it for a single person blog. But make its use clearer and I think it makes more sense.

Of course, this all changes if we get a way to easily make distros that pull in contrib and that becomes the norm. Until that time, though, I advise not using "there's something similar in contrib" as a metric for pulling something out of core but rather work to educate users that what they get in core is a base version and contrib has "upgrades" available.

Michelle

lelizondo’s picture

@Michelle I think that if forum or poll are removed from core they should be created as a contrib module. That way, there could be options for site builders. If they need something simple they could just download some simple module, if they want something fancy and with more functionality, they can download your module.

The point is that if we keep modules in core that are similar to another option in contrib, then we'll end up with what we have now, with modules like trigger, tracker, poll, forum, etc.

Also, the original problem will remain, those modules are not maintained, they can't grow, they have a ton of bugs to be fixed, and have to adjust to the core releases agenda.

Michelle’s picture

"The point is that if we keep modules in core that are similar to another option in contrib, then we'll end up with what we have now, with modules like trigger, tracker, poll, forum, etc."

And my point is that is not necessarily a bad thing.

"Also, the original problem will remain, those modules are not maintained, they can't grow, they have a ton of bugs to be fixed, and have to adjust to the core releases agenda."

Well, yes. I was only addressing one criteria. The other problems still need to be address.

Michelle

cweagans’s picture

"The point is that if we keep modules in core that are similar to another option in contrib, then we'll end up with what we have now, with modules like trigger, tracker, poll, forum, etc."

And my point is that is not necessarily a bad thing.

I think there will be a severe disagreement around this. The way I see it, if we have a bunch of stuff in core that's completely superseded by things in contrib, isn't it a waste of time and energy for core developers to keep dragging those modules along?

The proposed heuristics are as follows: Modules and themes which are not used by a core installation profile should not be included in core.

Why shouldn't they be? (Also I don't think that's what @catch was saying above either - it sounded to me like his suggestion was more like this is the list of candidates to remove from core, but they still need to be evaluated on other metrics.)

In most cases, if we aren't using them in core, I don't think they should be included in core. Exceptions to this are things like Token, where a freaking assload of sites use it. Things like Trigger....tear it out. I reread and re-reread catch's post, but I couldn't see any other meaning than "If we aren't using them and there's not a really good reason to keep them around, get rid of them".

We also have a nice spreadsheet attached to this issue with a good start at heuristics in it already; if the proposal here is to scrap that and start with new ones, I don't understand why.

This was accidental at first - I forgot that the spreadsheet existed. When I read your comment, I went back and looked over the spreadsheet (which, BTW, is located here: https://docs.google.com/spreadsheet/ccc?key=0AujolEbu-ZDQdGxkWEt1RGkxWUl...).

I think we need a more simplified way of codifying those heuristics. A lot of what's in that spreadsheet seems like opinion. I'd tend to lean towards cold, hard facts in this particular case.

Also, if a newbie were to come into #drupal-contribute and ask "Why was this module removed from core?" or "Why was this module added to core?", I don't want to have to explain: "Well, it got a score higher than the minimum threshold required on this spreadsheet based on some arbitrary formula in somebody's Google Docs (that's not linked to from anywhere on Drupal.org) and a lot of people liked it/hated it so we took action on it".

It's much easier to simply say "It helps core not duplicate code (token)" or "It hasn't been maintained for a while and there was a good reason to remove it (blog)".

Michelle’s picture

"I think there will be a severe disagreement around this. The way I see it, if we have a bunch of stuff in core that's completely superseded by things in contrib, isn't it a waste of time and energy for core developers to keep dragging those modules along?"

Depends on how you define superseded. More features doesn't always equal better. Even with the existence of Advanced Forum, some people still use core forum. Why? Because they want simple. They don't need all the bells and whistles. If they can flip a switch in core and have their needs met, that's great.

Anyway, I'm not necessarily arguing that these must stay in core. There are other factors to consider. I'm saying don't throw them out based solely on the metric of "is there a contrib module that does this" because sometimes built in and basic meets the use case better than add on and fancy.

Michelle

hgurol’s picture

@Michelle

And my point is that is not necessarily a bad thing.

Thats definitely a bad thing, no questions about that.
Thats what it is today, and its gonna get even worse.

Thats what all these talk is about.
I believe you are missing the major base point of this discussion !!

hgurol’s picture

I believe the metric is already defined as...

"If a module is a product, not part of the real framework, should be removed."

webchick’s picture

The base point of this discussion is that "Core needs to be more maintainable".

Currently, the focus point on that discussion is jettisoning modules from core that are not currently actively maintained. It's perfectly on-topic to say "Actually, even if those modules aren't currently maintained, they're still valuable because..." It might even cause someone to step up to maintain them.

rafamd’s picture

In response to #265, I understand and share your point, but having 2 separate modules isn't the only nor the best solution (imho). Reading your post, I remembered about this comment I read some days ago.

cheers...

hgurol’s picture

@web-the_resistant-chick
Someone stepping up to maintain them doesnt solve the problem at all, to my understanding.
That someone can still maintain them in the contrib, while moving them to the contrib will provide a solution.

nbz’s picture

@comment 270 - removing core of all functionality so that new users are put off by the lack of features will help no one.

I am sure that most people who use drupal were first enticed by existing core "product features" that gave them ideas and then later on they looked further.

Look at other framework only systems (Typo3? django etc), and only power users will consider them because they are too "frameworky" for ease of use, especially for beginners who will want to play with the product before deciding if it is right for them.

ParisLiakos’s picture

doh...why cant none seems to get that core!==product ?
@273 you dont have to give the end user the framework,,give him an installation profile!
just like what codeigniter and Expression Engine does.

This is not about ripping features out of drupal just getting them out of core so it will be maintanable and mainly help new devs contribute!

Having that beast that it is now,we will eventually lose the few contributors left when in d8 it will be much more complex

hgurol’s picture

@273
Thats where the snowman will become handy, no?

Michelle’s picture

I was going to let this go but it keeps getting marked as "new" over and over again and has overcome me biting my tongue...

@hgurol: The point of this issue is "Make core maintainable". Not everyone agrees on what that means, which is why we are having a discussion. If you think the only right way is the way you think it should be done and no one else's opinion is allowed, then I think it is you who has missed the point.

Michelle

philbar’s picture

What is our goal here:

  1. Make core maintainable
  2. Switch core from Product to Framework (Distributions as Product)

I'd love to see #2 happen, but think it's beyond the scope of this issue.

Crell’s picture

I'm frankly inclined to agree with chx and mark this entire thread off topic. Throwing modules out of core may feel good, but doesn't solve a damned thing. That we are still after 270 comments debating which modules to remove shows that we are not actually paying attention to anything useful. I hate to be so negative, but I calls 'em as I sees 'em.

1) We do not need metrics for what to remove from core. Any such attempts are short-sighted counter-productive. Period. Without exception.

What we need are heuristics for what should be put INTO core. There have been a couple such suggestions above, which have been mostly ignored unfortunately. Why is that preferable? Because once we have that heuristic for what SHOULD be in core, anything that doesn't meet that heuristic is what should !(be in core) (which is not the same as "should be removed from core").

We want guidelines going forward that can help guide the project, not a game plan for how to vent our frustrations on poor modules that no one in this thread likes.

2) I think we have also lost sight of a very important point. There is a very real problem that you cannot sneeze near Form API without poll.module breaking. That makes core much harder to maintain.

That is not poll.module's fault. It is Form API's fault. Blaming a user of an API for it being fragile is bass-ackwards.

The root problem is that Drupal the framework is a tangled mess of spaghetti, so anything we build on top of it, whether it's in core or not, cannot aspire to be anything more than cheap pasta. You cannot untangle spaghetti by eliminating the meatballs. That just means you're less likely to notice when you screw up your spaghetti.

"Making core maintainable" and "what should be in core" are completely orthogonal issues. Anywhere they are not is a sign not that too much is in core, but that we have a design flaw in our code or in our process, or potentially both. That is the bug to be fixed.

Let's develop a heuristic for what should be in core. Then build snowman/onramp/whatever eaton's calling it these days and let that guide revision to the heuristic. Meanwhile, clean up our APIs to not be such a pile of spaghetti. Then, and only then, is the question of removing things from core even useful.

hgurol’s picture

@276
No, I don't think it that way and I also don't delete posts that I don't like, and you shouldn't bite your tongue neither.

I also dont fix/replace my faucets when I have a problem with the main valve at home.

Michelle’s picture

@Crell: +1 billion :)

@hgurol: I don't delete posts I don't like, either. Might not want to go around assuming. There's an Odd Couple episode that explains why.

Michelle

sun’s picture

Status:Needs work» Closed (won't fix)
cweagans’s picture

@Crell, developing a heuristic for what should be in core invariably means that you're also creating a heuristic for what should NOT be in core. Call it what you will, but the end result is that there is a filter for what is in core, which prevents us from just adding random things as we please with no direction or end-goal.

That said, I guess the next step for getting things out of core (when deemed appropriate) is to get #1224666: [meta] Unofficial Drupal 8 Framework initiative and all of it's children moving.

lelizondo’s picture

"In most cases, if we aren't using them in core, I don't think they should be included in core. Exceptions to this are things like Token, where a freaking assload of sites use it."

Well, let me be sarcastic.

Yes, let's add such an amazing module just to remove it in D9 because now nobody wants to maintain it and the core developers are too busy fixing other critical bugs. In the meantime, let's wait 6 months more for D9 because Token it's creating a few critical bugs that make Drupal unstable. There is a few examples of this happening in D7.

Do you see my point?

If it works great in contrib, leave it there. Even if 99% of the sites use it, let's leave it there. Seriously, show me 3 great sites that uses 0 modules from contrib. I can't do it. So if I'm building a site and I'm gonna download 10+ modules, why not download another one like Token or whatever?

This is not about Token, but about any other module. Poll, Forum, you name it. There could be a simple forum in contrib an advanced forum module. I don't see what's the problem with that.

Richard Archer’s picture

Maintainability is inherent in the code itself. Moving core modules to contrib isn't going to reduce the maintenance burden, only move it elsewhere. The only thing that can reduce the maintenance load is to avoid breaking modules in the first place.

To achieve this, the core's APIs need to be robust and stable. If the APIs are stable, the underlying code can be restructured, refactored and perfected while still presenting a stable interface to the world.

Drupal's "code as art" paradigm has attracted to Drupal some of the best programmers I've seen in open source projects. But the cost is that the best software engineers are driven away by this same approach to development.

Einstein famously said "Problems cannot be solved by the same level of thinking that created them".

To solve the "unmaintainable core" problem requires a change of thinking.

David_Rothstein’s picture

Status:Closed (won't fix)» Active

Fixing things when they break is only a small part of maintaining a module. And this issue is specifically about making core more maintainable, not Drupal as a whole.

This issue was going along well for a while; let's not lose focus. To recap:

  1. There's a spreadsheet containing some heuristics describing what should be in core.
  2. Per #244, there is some work to improve those heuristics, maybe phrase them so they are a bit less subjective, maybe make them all positive rather than a mix of positive/negative so we can say this is a good description of "what should be in core".
  3. There are some numbers in the spreadsheet for a bunch of current core features, which can be revised and added to as time goes on.
  4. The numbers aren't a magic mathematical formula (i.e. we won't be killing all core features that score below a -4 or something bizarre like that), but rather just a good way to see at a glance where/how different features (old and new) stack up and get an idea of how to prioritize things.
  5. The heuristics, once "finalized" (which hopefully won't take another 200 comments :) can live in a doc page on drupal.org also.

I think this is good and useful work.

exaboy’s picture

Very interested to see how this all turns out!

rafamd’s picture

+1 #278

eaton’s picture

Based on the need for a dedicated discussion, I've split off #1273344: Establish heuristics for core feature evaluation and proposed a summary of "positive questions" that we can use to gut-check proposed core changes. Hooray for heuristics!

Owen Barton’s picture

Given that the more targeted "determine and document heuristics" conversation is going on in #285 I think we should close this.

arianek’s picture

+1 on closing or postponing this monster, and picking discussions on specific modules on other issues and/or after the heuristics question has been addressed further

cweagans’s picture

Status:Active» Postponed

I agree.

webchick’s picture

Assigned:Dries» Unassigned

Since there's nothing actionable here atm, removing from Dries's hit list.

js’s picture

2 cents and sub. So many comments here have me smiling. Thanks for that and improving the direction Drupal.

Thanks @chx for #31 (and Sun for recent posts on fixing Drupal that led me here).

Core should always be stable and brilliant.

Looking forward performance is key and will be overwhelmingly so--I believe.

It will be difficult for Drupal to perform acceptably a few years without core working with alternate technologies, like nodejs, Redis, and other noSQL. To me core should set the pace for coding, api, and technology and everything else should be in contrib. Some system of testing a site to see if the modules are ready for upgrade should not be too difficult.

The guaranteed upgrade path should be core to core versions and contrib be driven by demand--not obligation.

js’s picture

Issue summary:View changes

Added a link to an issue that deals with removing the aggregator module

AaronMcH’s picture

Hi everyone

First of all, there is no way I'm reading this huge thread, I have read some of it so forgive me if I get some information wrong.

I think this is a good idea, however it is vital that new user coming to Drupal know that there are these basic modules such as Blog and Forum, so on the modules page of Drupal I think it is important that modues such as these are mentioned to the modules page of a Drupal site, so that new users coming to Drupal know that there are these pre-built tools already availalbe to install.

I also think that the Overlay, Toolbar, PHP, and Search should not be removed from core. I also think that the Views, and possibly Pannels and CTools (Chaos Tools Suite).

AaronMcHale

AaronMcH’s picture

Hi again

I have an idea, if we remove at lot of the core modules it will make Drupal core more light weight, and it will make these modules more maintainable. However the problem is that it removes modules from Drupal that serve as basic starting points for new users and new website, such as being able to easily create a blog. New users to Drupal (and upgrading users) will not know about these functionalities and it may drive people away from Drupal a little.

A solution to this that I see is to still list these modules where they are right now, however instead of an enable/disable button beside each of these modules, there should be a Download and Install button, which would just take the user to the FTP login screen built into Drupal core to allow them to download and install that modules as if it was any other module.

This would still make Drupal core just as maintainable, and make these modules just as maintainable, however it would mean that new users and upgrading users would easily be able to easily install these modules.

Thanks
AaronMcHale

Crell’s picture

AaronMcHale: The conversation has long since moved on, and throwing modules out of core is no longer on the table. Instead, we're discussing this: #1273344: Establish heuristics for core feature evaluation

webchick’s picture

Should we just close this issue? It seems like we're discussing the fall out of this discussion in many other places now, and I'm not sure how this ever becomes "fixed".

colan’s picture

Better than closing it would be locking it (or both), but if so, we should provide links to the proper discussions that are going on elsewhere (assuming we haven't listed them all yet).

RobLoach’s picture

Title:Make core maintainable» [Meta] Make core maintainable

There we go :-) .

sun’s picture

Title:[Meta] Make core maintainable» [meta] Make core maintainable
Priority:Major» Normal

Demoting to normal, so to not block any other issues.

lpalgarvio’s picture

tim.plunkett’s picture

Issue summary:View changes

Just added link to the #1273344 issue.