[policy, no patch] By default deprecate non-experimental modules that are used by less 5% of sites before the next major version

I would suggest that this list is greatly skewed by what is installed in standard. And what reports back - update 100% :D

80% of sites using RDF :P

@tedbow yeah I'm more concerned that this work will used to support maintain rdf in core - and prevent things like #2152459: [Policy] Deprecate RDF module and move it to contrib

This issue is also very related to #1273344: Establish heuristics for core feature evaluation

I would suggest that this list is greatly skewed by what is installed in standard.

That points to another problem. The list of enabled by default modules in Standard profile is not sensible.

color,264768,75.03%

That's another curious thing here. I guess the usage of Color module should less than 1%. I think the statistics would be accurate if we only account modules on sites using minimal profile. Not sure if core update module supports this. We could probably ask some Drupal hosting providers (Acquia Cloud, Pantheon, etc) to collect such data for us.

color module has it's own dedicated issue at #2808151: [policy] Move the Color module to a contributed project when Bartik is deprecated.

@catch just updated #1136482: [policy] Deprecate aggregator.module in D9 core and remove it in D10 pointing out that an Aggregator dependency is now holding core back in fully supporting PHP8.

Adding #2152459: [Policy] Deprecate RDF module and move it to contrib as related issue.

From #3:

I would suggest that this list is greatly skewed by what is installed in standard. And what reports back - update 100% :D

+100

Personally, there are several that I usually turn off or forget to turn off but don't use (when I'm thinking about it, I start with the minimal profile though)... color, comment (used very occasionally), contact (replaced by webform), shortcut, rdf, tour.

Is there any discussion/issue about reducing the number of modules in the standard profile? I searched the issue queue but didn't find anything. Seems like something that would be good to revisit every major release.

I don't think there's an issue with an overview of modules to add/remove from standard (as opposed to removing from core of which there are many), so opened one here: #3196014: Revisit the modules enabled by the Standard profile

We could do something about the update.module problem by splitting out functionality that would report back to drupal.org but I am not sure that is something we want to do.

There is lot of room for improvement, I've summarized it in #3199472: Improved data sharing with d.o

It is good to see this could happen in Drupal 10. I suggest a similar idea 3 years ago.

https://www.drupal.org/project/ideas/issues/2961289

For Drupal 8, we have made +++++++++++ for it. Big Pipe is great, add it; Rest is great, Add it; migrate is great, add it; Composer is great,add it... For Drupal 9, I suggest, we need do more ---------- work for it. Instead of add more modules to it, I suggest consider remove some modules from core.For examples:

aggregator
book
color
contact
forum
rest
tracker
big_pipe
field layout
migrate
migrate_drupal
migrate_drupal_ui
breakpoint
responsive_image
rdf
basic_auth

These are modules that I think it less important.
big_pipe is great, but I never use it.
migrate is a dedicated module, it is better move it to contributed.

Saving current data from @mixologic for posterity, received yesterday.

Here's a script that parses #19 and generates the following output:

system: 448399, 100.0%
user: 448396, 100.0%
update: 448330, 100.0%
field: 425152, 94.8%
file: 419732, 93.6%
dynamic_page_cache: 419469, 93.5%
block: 416825, 93.0%
filter: 414338, 92.4%
text: 414307, 92.4%
node: 413878, 92.3%
page_cache: 413824, 92.3%
views: 408124, 91.0%
link: 407842, 91.0%
breakpoint: 407732, 90.9%
image: 407456, 90.9%
editor: 407304, 90.8%
path: 407107, 90.8%
menu_link_content: 407087, 90.8%
toolbar: 406962, 90.8%
menu_ui: 406596, 90.7%
options: 406562, 90.7%
ckeditor: 406384, 90.6%
datetime: 404180, 90.1%
taxonomy: 404081, 90.1%
config: 403421, 90.0%
field_ui: 402004, 89.7%
path_alias: 400173, 89.2%
block_content: 398072, 88.8%
views_ui: 392794, 87.6%
contextual: 390714, 87.1%
dblog: 377252, 84.1%
automated_cron: 370255, 82.6%
rdf: 369659, 82.4%
help: 364303, 81.2%
shortcut: 363948, 81.2%
history: 347093, 77.4%
color: 341802, 76.2%
tour: 318571, 71.0%
contact: 311209, 69.4%
comment: 304285, 67.9%
quickedit: 295570, 65.9%
search: 293040, 65.4%
big_pipe: 266711, 59.5%
language: 253645, 56.6%
locale: 219099, 48.9%
media: 210289, 46.9%
serialization: 157702, 35.2%
responsive_image: 151800, 33.9%
layout_discovery: 146453, 32.7%
media_library: 132644, 29.6%
content_translation: 128436, 28.6%
telephone: 125529, 28.0%
seven: 120832, 26.9%
config_translation: 119935, 26.7%
rest: 115684, 25.8%
standard: 110447, 24.6%
bartik: 99449, 22.2%
stable: 92816, 20.7%
datetime_range: 91550, 20.4%
classy: 88572, 19.8%
workflows: 81902, 18.3%
migrate: 78285, 17.5%
content_moderation: 75585, 16.9%
layout_builder: 70920, 15.8%
inline_form_errors: 61664, 13.8%
syslog: 56869, 12.7%
hal: 55885, 12.5%
ban: 44977, 10.0%
migrate_drupal: 44708, 10.0%
jsonapi: 42265, 9.4%
action: 42030, 9.4%
statistics: 40516, 9.0%
basic_auth: 39048, 8.7%
mysql: 27209, 6.1%
migrate_drupal_ui: 21323, 4.8%
aggregator: 20619, 4.6%
book: 20253, 4.5%
entity_reference: 19624, 4.4%
forum: 19158, 4.3%
settings_tray: 18284, 4.1%
field_layout: 16146, 3.6%
tracker: 13645, 3.0%
sqlite: 5492, 1.2%
claro: 3893, 0.9%
stark: 3778, 0.8%
update_test: 3608, 0.8%
olivero: 2897, 0.6%
ckeditor5: 2522, 0.6%
pgsql: 2427, 0.5%
help_topics: 2296, 0.5%
migrate_drupal_multilingual: 2203, 0.5%
minimal: 2043, 0.5%
locale_test_translate: 2004, 0.4%
workspaces: 1591, 0.4%

color: 341802, 76.2%

Once again, statistics for modules installed by default is useless. See my comment #10.

Here's a script that breaks it out by Drupal 8 and Drupal 9, and the results (first number is 8, second number is 9, sorted by total):

system: 174050, 264766
user: 100.0%, 100.0%
update: 100.0%, 100.0%
field: 99.4%, 93.4%
file: 99.4%, 91.8%
block: 99.3%, 91.0%
dynamic_page_cache: 96.6%, 92.4%
filter: 99.3%, 90.2%
text: 99.3%, 90.2%
node: 99.1%, 90.2%
page_cache: 95.7%, 90.8%
views: 99.2%, 88.4%
link: 99.2%, 88.3%
breakpoint: 99.1%, 88.3%
image: 99.1%, 88.2%
editor: 99.0%, 88.2%
path: 99.0%, 88.1%
menu_link_content: 99.0%, 88.1%
toolbar: 98.9%, 88.2%
menu_ui: 98.9%, 88.0%
options: 99.0%, 88.0%
ckeditor: 98.9%, 88.0%
datetime: 98.6%, 87.3%
taxonomy: 98.4%, 87.4%
config: 98.4%, 87.1%
field_ui: 98.1%, 86.8%
block_content: 97.8%, 85.5%
views_ui: 97.0%, 84.0%
path_alias: 72.3%, 100.0%
contextual: 96.1%, 83.8%
dblog: 93.7%, 79.9%
automated_cron: 93.9%, 77.6%
rdf: 92.1%, 78.5%
help: 91.4%, 77.0%
shortcut: 91.9%, 76.5%
history: 89.3%, 71.9%
color: 88.1%, 70.7%
tour: 83.6%, 64.6%
contact: 82.4%, 62.9%
comment: 81.5%, 60.9%
quickedit: 83.1%, 56.9%
search: 77.7%, 59.1%
big_pipe: 57.7%, 62.3%
language: 60.4%, 54.9%
locale: 53.7%, 46.5%
media: 40.5%, 52.7%
serialization: 35.7%, 36.0%
responsive_image: 31.9%, 36.2%
layout_discovery: 30.8%, 35.0%
media_library: 20.1%, 36.8%
content_translation: 30.9%, 28.0%
telephone: 27.7%, 29.1%
seven: 44.5%, 16.4%
config_translation: 29.4%, 25.8%
rest: 27.9%, 25.2%
standard: 39.1%, 16.0%
bartik: 36.0%, 13.9%
stable: 45.0%, 5.5%
datetime_range: 17.6%, 23.0%
classy: 45.0%, 3.9%
workflows: 14.8%, 21.1%
migrate: 16.9%, 18.3%
content_moderation: 13.2%, 19.8%
layout_builder: 10.2%, 20.0%
inline_form_errors: 10.8%, 16.2%
syslog: 12.0%, 13.5%
hal: 14.3%, 11.7%
ban: 10.5%, 10.0%
migrate_drupal: 10.7%, 9.8%
jsonapi: 9.6%, 9.6%
action: 10.0%, 9.2%
statistics: 11.5%, 7.7%
basic_auth: 10.9%, 7.5%
migrate_drupal_ui: 4.6%, 5.1%
mysql: 0.4%, 7.5%
aggregator: 3.6%, 5.3%
book: 4.5%, 4.6%
entity_reference: 9.6%, 1.1%
forum: 4.2%, 4.4%
settings_tray: 3.5%, 4.6%
field_layout: 4.6%, 3.1%
tracker: 3.1%, 3.0%
claro: 0.8%, 1.0%
stark: 1.5%, 0.5%
sqlite: 0.0%, 1.4%
update_test: 0.6%, 0.8%
olivero: 0.0%, 1.1%
ckeditor5: 0.0%, 0.9%
help_topics: 0.3%, 0.7%
migrate_drupal_multilingual: 0.7%, 0.4%
minimal: 0.6%, 0.4%
workspaces: 0.3%, 0.4%
locale_test_translate: 0.0%, 0.6%

Cool script! Here is the data in table format, for easier readability:

The 8 vs. 9 stats are very interesting. It's good to see layout builder go from 10% to 20% and media_library go from 20% to 36%. Shows new modules added later in 8.x picking up speed.

For some of the downward trajectories, I wonder how much of it is reflecting this:

is driven by this:
standard	39.1%	16.0%
bartik	36.0%	13.9%

The sites currently running Drupal 8 are the ones that are already four months out of support, so more likely to be smaller sites - installed with standard instead of a custom profile or distro (or minimal + install from config), either using bartik, or enabled another theme without uninstalling bartik etc.

So to some extent this may reflect a diverging install base of more frequently vs. less maintained sites as opposed to what the new sites are installing with. If all of the 8.x sites updated to Drupal 9 tomorrow it'd even out some of those stats. But then we still have the issue that we can't tell whether people running standard are using the modules it enabled for them or just haven't uninstalled them.

One caveat to that data that I always like to repeat: Usage stat data is *not* ground truth, it gives us a good idea of the relative usage, but many sites do not enable updates, so we do not get their data. And many sites that *do* enable updates are default sites that are not necessarily seeing usage.

The stats also get some pollution/weighting by multisite features. One multisite with tons of subsites replicates their requests for each subsite, inflating their numbers.

But its all we have, so if we can imagine any reason that a particular module might be used by sites that dont report, more than others, we might want to consider that.

One way we can look at this is how core has changed over time, and how we want it to continue to evolve.

If you compare core of Drupal 6/7 to core of Drupal 8/9, one of the key differences when it comes to modules is that many more of the core modules we have now provide "building block" functionality: Views, JSON:API, Layout Builder, Media, Workflows, etc. Compared to some of the modules that are still there from the days of Drupal 6/7: Book, Color, Tracker, Book, Ban, etc.

There's a trend here, that being those modules which core has introduced in recent years are much more like "building block" modules, where they give you many ways to solve a problem. Whereas some of those modules of earlier years are much more prescriptive in nature, they solve a very specific use-case in a very specific way.

What we can draw from that is that we are moving in that direction where the modules shipped with core are mostly designed to address a particular problem space, but do it in a way that enables many possible different solutions, and those modules complement the existing set of modules. You can use combinations like Field, Views and JSON:API to do some really interesting things. In other words, you can use those modules like building blocks, to build a solution.

Maybe that is what we should be using as a metric, if a module which is designed to solve a particular problem doesn't do it in a way that compliments other modules and doesn't promote that idea of building blocks, then maybe it doesn't belong in core, or maybe it needs to be rewritten (Search is one example that comes to mind which could probably do with being rearchitected). Sure that's more open to interpretation, but in any research you need not just quantitative data, but also qualitative data. To me, what I've outlined here is probably a pretty good metric when it comes to that qualitative side of the coin, which we can use to compliment the quantitative data that we already collect.

@AaronMcHale: I think we used a variant of that as a deciding factor for the modules we are removing from Drupal 10. We called it "does it have a contrib ecosystem", which does not only indicate whether the module is a building block, but whether people actually used it to build something. Since we are talking about modules that have been in Drupal core for a while, I think this way to look at it works well and is even better than your "is this a building block".

@Gábor Yes that's a good way to look at it. The "does it have a contrib ecosystem" question could cover "is this a building block", because a module being a building block means it's easier for a contrib ecosystem to evolve. However, it's still possible for a contrib ecosystem to evolve around a module that's not designed to be a building block, it's just likely that such an ecosystem is smaller and more niche, in which case the module probably doesn't need to be in core.

Thinking about it from a slightly different angle though, when we're evaluating a core module it's probably still worth considering the question of whether it's a building block, because we might answer that question as "yes, but it could be better", which can then lead us down a different path of "okay well we're keeping it in core but we now recognise that some resource is needed to improving or rearchitecting it".

We might also find ourselves saying "this module does not currently have a contrib ecosystem but it's strategically important enough to improve the module so that one could evolve around it", I don't have any examples of that off the top of my head, but it's a scenario that we could run into at some point.

So if you think of it like a flow chart, I think we can use both the "building block" and "ecosystem" questions to determine the best course of action.

I want to +1 this idea.

We all agree several modules and themes do not need to be in core. They could and certainly should live as contrib. However it might be dangerous to deprecate projects automatically with strict predefined rules.

I think what we need in the first place are tools to:

1. consolidate better usage statistics
2. make this open data availabe on d.o
3. gather real feedback from users/developers/site owners

IMHO future of core must be defined by needs form the real world users, not only by us, developers.

I wonder if there are some additional caveats to the data that we are not aware of yet. I don't know if I'm alone with this but it seems strange that only 93.4% have Field enabled, and that only 91.0% have Block enabled. But then Menu UI is enabled by 88.0% of the sites 🤔.

Even more glaring to me is that Claro + Seven have total of 17.4% usage which seems really low. The two of the most popular contrib admin themes have total of around 74500 usages reported which is around 17%. They both depend on either Seven or Claro. Do we think it's possible that there are only 0.4% of sites using Seven or Claro as their admin theme?

@lauriii at one point tests with update status enabled were reporting back to Drupal.org, if this is still the case then that'd be a lot of sites every month with 'stark' enabled.

If that's true, then shouldn't we be ignoring all update status checks to d.o for sites with stark enabled? 😇 I think only an incredibly small number of Drupal sites will eve have gone into production on stark 🤓

Looks like Stark usage is between 0.5% and 1.5%. Based on that, I'm not sure that #31 helps explain #30.

@lauriii that's a good point and also reassuring :)

The table in #23 is from March 2022, I think we could expect usage of Claro to have gone up significantly since then since it was marked stable and added to the standard profile in April 2022 (and then 9.4 was released in June). For me, that alone explains low Claro usage - too new to appear in the stats. Even with updated stats, it will probably still take some time to pick up - i.e. for existing sites to switch to Seven to Claro as part of a Drupal 10 upgrade, still a few months of that to go.

This doesn't explain the combined low usage though. Seven was on 44.5% in Drupal 8 according to #23 which seems more what I'd expect although still quite low.

The Drupal 8/9 comparison also shows the drop off with the other modules you mentioned like field - it's at 99+% in Drupal 8, more like 91% in Drupal 9, so that could be indicating collection issues (unless there's a real trend).

Something else that seems off about the data is that between Drupal 8 and 9 Stable usage went from 45.0% to 5.5%, and Classy usage went from: 45.0% to 3.9%. Did this many people really remove their dependency on Classy? If we look at the contrib Classy usage and compare it to Drupal 10.0.x sites, around 13.3% of Drupal 10 sites have Classy installed.

What is the latest data on this? Where can I find this?

Also are there any plans to do the opposite? i.e. move contrib modules into core that are installed on more than 5% of sites

https://www.drupal.org/project/project_module?f%5B0%5D=&f%5B1%5D=&f%5B2%...

ckeditor5 usage has probably gone up as well?

@2dareis2do the data is only available via querying the d.o database, we'd need to ask the DA for updated stats.

The current list of modules that are actively being prepared for a move to contrib is in #3265680: Deprecate dependencies, libraries, modules, and themes that will be removed from Drupal 11 core.

Also are there any plans to do the opposite? i.e. move contrib modules into core that are installed on more than 5% of sites

There are individual issues like #229568: [policy, no patch] Pathauto in core although e.g. pathauto is installed on more like 75% of sites. The main focus on making it easier to find and install modules is on project browser.

Thanks @catch

Can I create an issue to make this data publicly available?

Current data is nearly 2 years old. Drupal 10 has been released since then.

Feedback loop is too long.

@2dareis2do there is probably already a public issue, I would look in the 'drupalorg' issue queue first, and open an issue there if there's not one already.

Ok I have added an issue here

https://www.drupal.org/project/infrastructure/issues/3417187

Also are there any plans to do the opposite? i.e. move contrib modules into core that are installed on more than 5% of sites

With Project Browser and Recipes soon coming into Core, we might find that we actually don't need to bring more module into core, because the barrier to entry for getting those modules becomes so much lower.

What I mean is that I can imagine a day where, when you setup Drupal, you don't install the standard profile, you pick from a curated set of recipes right in the installer, the recipe takes care of getting all of the modules it needs. Some of those recipes might not even be included with Core, they might life in contrib and the recipe you choose might then be downloaded during the installation. Maybe then core can become even smaller, and thus even more maintainable.

Now, I'm not saying this is definitely where we should go, but what I am saying is that I can see this being a place that we are gradually moving towards.

Project Browser reminds me a little of Drupal Gardens.

Our goal is to make it easier to find and install modules for people that are (1) new to Drupal and that are (2) site builders*

Moving popular contributed modules into core also acheives this aim.

Here is updated data.

To clean the data, this is how I got the core components:

SET group_concat_max_len = 10000000000; SELECT group_concat(DISTINCT pcc.name SEPARATOR '\\|') FROM project_composer_component pcc INNER JOIN field_data_field_release_project fdf_rp ON fdf_rp.entity_id = pcc.release_nid AND fdf_rp.field_release_project_target_id = 3060\G

tried to reformat it, double checking is needed.

The Ideas project is being deprecated. This issue is moved to the Drupal project. Check that the selected component is correct. Also, add the relevant tags, especially any 'needs manager review' tags.

In the last table, pgsql module is 4%. Reading the proposed resolution, seems like that the module would qualify for removal, even if it is a DB driver. Can we exclude DB drivers modules explicitly, as I suppose we do not want to remove PostgreSQL support even if the module usage will be under the treshold?

There was a request for updated data in Slack, it is attached.

More notes for myself for next time:

Get the current core components:
SET group_concat_max_len = 10000000000; SELECT group_concat(DISTINCT pcc.name SEPARATOR '\\|') FROM project_composer_component pcc INNER JOIN field_data_field_release_project fdf_rp ON fdf_rp.entity_id = pcc.release_nid AND fdf_rp.field_release_project_target_id = 3060\G

Filter the usage data:
grep '\sdrupal|.*\([the core components]\)$' < /var/log/updatestats/counts/[most recent].submodule_release_counts > drupal-component-usage-by-release-[today].txt

Thanks @drumm,

Formatting data:

Thanks @grimreaper! Here are the percentage values sorted descending for Drupal 10 and Drupal 11 values. I finally figured out how to sort numbers in Libre Office Calc: "Data" > "Sort", and "Natural Sort" (number sort) under "Options".

Drupal 10 sorted descending

Drupal 11 sorted descending

I was looking at #3466088: [meta] Deprecate dependencies, libraries, modules, and themes that will be removed from Drupal 12 core and remembered this issue.

Drupal 12 should be released with 65 core modules. These 11 core modules will be removed, where some are already moved to contrib, others are in the process:

• ban
• contact
• field_layout
• history
• migrate_drupal
• migrate_drupal_ui
• search
• settings_tray
• shortcut
• telephone
• toolbar

13 core modules have been added since July 2020:

• announcements_feed
• ckeditor5
• mailer
• mysql
• mysqli
• navigation
• package_manager
• pgsql
• phpass
• sdc
• sqlite
• workspaces
• workspaces_ui

... and 28 core modules have been removed:

• action
• aggregator
• ban
• book
• ckeditor
• color
• contact
• entity_reference
• field_layout
• forum
• hal
• help_topics
• history
• migrate_drupal
• migrate_drupal_multilingual
• migrate_drupal_ui
• quickedit
• rdf
• search
• settings_tray
• shortcut
• simpletest
• statistics
• telephone
• toolbar
• tour
• tracker
• workspaces

Perhaps it's possible to get a list with updated data in September, like in 2024 and 2025? (comment #43 and #47)