Introduce tags[] in module.info.yml file to categorize modules by provided functionality.

Updated issue summary.

Proof of concept patch and screenshot attached.

oh, yay - fixing PIFR to be able to manually install Drupal with this patch will be extreme fun! ;) I'll try to make sure to roll further patches with the do-not-test suffix.

Regardless of that, this PoC patch works pretty nicely already. I'd recommend to try it out.

Some interesting and quite important observations:

1) Technically we could automatically tag modules with "api" if they ship with a $module.api.php file.

2) Every module that exposes an API additionally forms an own tag name.

3) Logically, every module that implements an API of another module should be tagged as "functionally related" to that API module.

Please bear in mind that this information cannot be determined or retrieved automatically, since that operation would require to execute and parse code at runtime. We are still talking about meta information that needs to be available before code is loaded and executed.

I only recorded the above observations while working on this patch. Technically, 3) should be contained in this patch (but not necessarily complete). 1) and 2) are food for thought.

Changes:

1. Added tags for all core modules.
2. Added missing empty option to tag filter select dropdown.
3. Sort tag filter select options alphabetically.

Dare I say... holy shit? Try this out. — A decade ends and a new era begins.

It wasn't my intention to ignore comments, but much rather, first try to whip up a proof of concept change proposal and additionally validate it against all modules in core. That essentially happened with #6 - which is still "alpha" quality and not meant to be complete, but the moment you try it out, I can almost guarantee that you do not want to go back. ;-)

re: #2: @Bojhan:

A "select list" to filter by tag, seems like the wrong UI for something like this.

Hm. Can you explain why?

Coming from a task perspective, I see the following:

1) I seek to enable certain functionality.

2) I go to the Modules ("Extend") page.

3) The available functionality is unfiltered by default. In case I know what I want, I can execute my task directly [and my task ends here]. [There's no inane list of undiscoverable packages/groups, just a plain list of modules sorted alphabetically.]

4) I have a vague idea of what functionality I want to enable, so I consult the "Filter by tags" dropdown in order to strip down the massive amount of options to the ones of my interest.

5) As mentioned in the OP, yes, the internal/machine tag names might be an issue. We might need to translate the internal tags into human-readable names (on the assumption that the tags are too technical, which very likely requires formal usability testing to be sure whether that is actually the case).

6) After filtering by a functional tag, I can see and operate on the results. [Any knowledge of "packages" no longer exists in the first place, so I get a plain list of modules filtered by tags.]

7) If this result does not satisfy my needs, I rewind and go back to 4) and try a different tag.

Potentially extended/alternated by:

7b) If I'm able to phrase my need as a textual keyword, I can also try to enter a search phrase instead. This functionality is not within the scope of this issue though, but bound to #396478: Searchable modules page instead. [Note: That issue becomes a piece of cake in combination with the changes here, since this patch here removes the package grouping altogether.]

The point where HEAD utterly fails is 4), 5), 6), and 7): With Drupal core only, you are not able to reproduce it. That's why I've added the screenshot of a D7 site in the OP: The list of modules is essentially "pre-filtered" (grouped) by a very technical "package" thingie that doesn't make sense at all to you:

It is entirely impossible to discover a particular module of interest in the pre-filtered/grouped list of modules, since "packages" do not present the filter that matches your task of finding "functionality being related to X." The steps 4), 5), and 6) do not exist at all, since it is impossible to remove the pre-filtered grouping. You're stuck with 7) - manually trying every single group by manually expanding it and manually parsing all available options within, in order to check whether the functionality you seek for is contained - or not. If it is not, rewind and continue the same procedure for almost all groups that are exposed on the page: The functionality you're looking for could very well be buried into one of them.

re: #7: @pcambra:

For some reason I think that the filter by tag would look more natural as an autocomplete.

That's again closely related to #396478: Searchable modules page — however, please bear in mind that "searching" is a completely different task than "filtering":

1) A user that searches commonly knows already what he wants to find - at minimum to a certain textual extent, which allows to phrase the search term in the first place.

2) Searching requires to manually type in a search term. Thankfully, the New World Order™ of mobile/touch devices sufficiently clarifies why the Average Joe does not want to use a keyboard.

If a user does not have a very concrete idea of a textual term to search for, then the user will not use the search. And if it's easier and quicker to use a filter, the user won't use the keyboard to type. A filter is not only used to discover, it is also used to save manual work.

EDIT: Sorry, forgot:

3) A search yields results on textual comparison only. Even if you searched for the right term, that does not inherently mean that you will get all available/possible/matching results.

Anyway, bottom line: search != filter. Probably for a dozen of essential reasons. :)

@klonos: Views is a module that may not be enabled, so the Modules page cannot use it, because it is the very page that allows you to enable it.

@dawehner: #states is an interesting idea. However, I'm not sure whether it is up for the task - #states works well for triggers, conditions, and states applied to individually targeted elements - but for this task here we want to apply a dynamic value as a trigger and state condition for a collection of elements. I don't think that #states support dynamic values currently. Investigating that would be a nice (separate) issue though; should not hold up this issue - we can always improve/simplify later.

However, I think we need some more feedback on the actual change proposal first. I haven't seen anyone screaming "NO!!!" ;) but I also haven't seen anyone saying "I tested this; Awesome! +1000!!!" :)

@Bojhan:

are you expecting this to work like tags or like categories? Because for categories, a select list is fine but if you expect people to apply two/three tags to a module - you can't have a single select input method.

It sounds like you're giving the terms "tags" and "categories" a lot of underlying meaning, which I've troubles to decipher right now :)

Modules would definitely be tagged with multiple terms. Each term/tag represents a certain, related functionality or topic. A "facet" user interface that allows to refine and drill down the results further is an interesting idea. However, I'm not sure whether it really applies here, since it is mainly used for filtering interfaces for the task:

"Show me all items that are related to X, Y, and also Z."

I might be totally wrong, but I think that the task and use-case of the Modules page does not involve such a multi-filter task. Instead, as a user I typically go there with the task:

"I need functionality related to X. Show me available options."

The fact that items are tagged with multiple terms doesn't really matter here. What matters is that (I believe) the user has only one topic/term in mind, because he/she is coming to this page with a concrete goal/task in mind; e.g.:

1. "I want to enhance my site to do or have X."
2. On the Extend/Modules page, I see all possible options. Unless I know exactly what I want already, that's not helpful at all.

So let's presume the X I wanted is "A form on my site so people can mail me."

3. Given this (prominent) filter option, I'd probably consult it to search for the terms "message", "form", "e-mail", or "mail", and filter by them.
4. If I find a related term, I'd try it, and otherwise, I'd look through the available terms to see whether there's one that comes close to what I want.

Now, adding another term on top of an already existing term doesn't really make sense to me here, since 1) the amount of filtered results will be small, and 2) the combination of more than one term will most likely yield a single result only in many cases.

That is, because the total set of available options is large, but it is not infinitely large — i.e., a facet/refine filtering makes sense if you have a store with 14,000 items that can be filtered, which inherently means that there will still be dozen or possibly even hundreds of results after adding and applying 2-4 facet filters. In this case here, we're dealing with a maximum amount of ~200 items. Applying a single tag filter will drill it down to <~20 items already; whereas 1/10 is sorta the maximum; the reality for most terms will probably be <=8. In turn, applying an additional filter to 2-20 items is rather pointless.

I'm not sure where exactly that falls into your separation between "tags" and "categories" (and why). ;)

@YesCT:

Let's bear in mind that this tag facility is only about modules that are available on your local system already. Your thoughts on a potential correlation to the project filtering on drupal.org are interesting, but in the end we're talking about two vastly different things here: d.o is about projects, while the Modules page is about modules. It is probably true that d.o could parse and aggregate the tags from module .info files within a project's repository, in order to potentially replace the current manual categorization of projects. However, in that case, we'd want to design the system from the bottom up; i.e., generate data, and only afterwards, we can use the data.

I'd recommend to keep a potential re-usage of the data by drupal.org out of this issue for now.

Would a sort by tags give essentially the same thing that we have now, with modules grouped by project? But with duplicates, as some modules will be under more than one category.

Essentially yes (including the part about duplicate results).

However, to clarify, the current modules are not grouped by projects, they are grouped by (a single) "package", and the meaning of packages is very abstract and really hard to communicate and translate into human speak, and thus also very hard to educate developers about — therefore, the UX problem we're facing starts way before the user interface layer, it starts with a technical DX/understanding problem already - and despite various attempts to clarify and improve the situation in the past ~6 years, the problem still exists.

Core would be one of the tags? so we could still get a list of just the core modules.
It would probably make sense to have a custom of using a "contrib" tag. (the opposite of core)

Possible. Though from a user perspective, I'm afraid I don't see the use-case of filtering by "core" or "contrib". I'm searching for functionality related to X, and that's not really functionality I'm searching for?

In the expanded description [...], I suggest listing the tags. There the clicking on the tag link would filter the extend page by that filter.

That's a nice idea! :) I'd probably defer that possible enhancement to a follow-up issue though.

@Grayside:

The connotations you've brought up are related, but I think they're not the same. The tags as proposed here are still defined freely, so contrib can invent and introduce any kind of tag at any time.

There is, however, indeed a desire for curating the final/resulting list on a particular site - that is, in the sense of not wanting to end up with hundreds of tags, since, at that point, you'd potentially have more tags than you have modules. ;)

Next to the bare volume/amount, there is also a risk and danger of "senseless" tags. For instance, to provide a stupid but perhaps even not so unreal example, I could go on and tag all of my modules with "unleashed mind", and some other folks in here could as well do that by tagging theirs with "acquia". ;)

Now, you will probably agree that this would be "Ugh, no way." — but if you come to realize it, OTOH, the tagging mechanism would inherently allow us to automatically add a tag to all modules that are part of and shipped with the installation profile with which you might have installed Drupal. "Show me everything that's part of my Open Atrium distro." is a use-case that would make sense to me (contrary to the core/contrib scenario mentioned above).

So, yeah:

Ultimately the challenge is to invent a categorization system within a multi-dimensional system, which is neither limited top-down, nor too lax/senseless, and in the end, still accomplishes its original goal to actually help the user in performing his/her task.

The proposed approach is to open the doors wide open, but to implant a good sense into the DNA by shipping and starting with a sophisticated set of tags, and leaving/combating the rest through education (of developers). A concise handbook page about "Do's and Don'ts" will totally be required.

Upon re-reading #17, what I described in points 3. and 4. in reply to @Bojhan is actually a facet filter, but NOT one that stacks filter conditions as AND conditions, but instead, as OR conditions. ;) I essentially described it:

3. I'd probably consult it to search for the terms "message" OR "form" OR "e-mail" OR "mail", and filter by them.

[4. If I find any other related term, I'd simply add it.]

I'm still not sure whether a multi-facet filter interface will make sense, but at least the idea of multiple OR filter conditions makes a lot more sense than multiple AND filter conditions.

1. Added handling for untagged modules.
2. Added output of tags in module descriptions + clickability.
3. Added tags to new .info files.

Fixed bogus HTML markup in module description tag list.

I'll re-roll this shortly. That said:

Since #1848064: Allow to filter modules by arbitrary search strings on the Modules page landed recently — did anyone use the Modules page more extensively in the past days, to gain some more conclusions?

As mentioned over there, the search definitely works for users, who already know what they want to find. New/novice users don't know that. That's where tags could kick in.

I'll perfectly admit though, these lowercase, untranslated, and rather technical tags might not make ultimate sense to novice users either. In the grand scheme of things, I'd probably classify them as "intermediate" (vs. beginner/advanced), as they presume that you know the tag/category under which you could find a module that relates to your interest.

That shouldn't block us from introducing them though (and finally getting rid of exposing packages) :)

1. Merged HEAD.
2. Added tags for new modules in core.

Did not update the UI/JS to work in combination with the new search facility yet, so please note: Right now, only one of both filters works at a time.

I'd highly recommend to test this live by applying the patch and provide your feedback.

Sorry, I should have used a do-not-test suffix for previous patches. Due to the form structure changes of the system_modules form, PIFR is not able to enable Simpletest module. I've added a temporary BC layer for now.

@klonos: I believe it was me who suggested custom/user-defined tags for modules some years ago. It once sounded like a potential solution, but by now, that doesn't make any sense at all to me anymore. Users are using Drupal to manage and classify plenty of things. But modules/extensions do not belong to that. Doing that would be a violation of separation of concerns. This data is not generated by me, so the fundamental idea of asking me to classify data that I did not author is pointless and wrong.

Meanwhile however, I spent quite some time to think some more about what I stated in #6 already:

1) Technically we could automatically tag modules with "api" if they ship with a $module.api.php file.
2) Every module that exposes an API additionally forms an own tag name.
3) Logically, every module that implements an API of another module should be tagged as "functionally related" to that API module.

1. Plugins

   Part of the New World Order™ and reality we're facing in Drupal 8 is that a big part of the concept of "functionally related" happens to be primarily expressed by plugins.

   A pretty dumb and trivial plugin type + implementation discovery algorithm would be able to deliver all of these module classifications already:

   • Entity types
   • Blocks
   • Field types
   • Field widgets
   • Field formatters
   • Filters
   • Text editors
   • Views fields
   • Views styles
   • ...

   That is, in a fully automated way — without the need for manually specified tags. Additionally, aforementioned human-readable labels can be determined literally in the way I stated them, because they are defined as translatable strings in code already.

   And due to the way the plugin system works, this classification would inherently support "parent namespaces" already; i.e., we're easily able to list all "Field"-related modules.

   Furthermore: A plugin category would only appear if the module that provides the plugin type is available. I.e., if you have a dozen of modules that provide plugin implementations for Pathauto, but Pathauto does not exist in your filesystem, then there is no point in exposing this category as a filter. A plugin category only makes sense when you actually have the plugin type providing module available (not necessarily installed) on your site/in your filesystem.

2. Dependencies

   Another major chunk of "functionally related" can be easily derived from the existing module dependencies[] declarations.

   This primarily covers cases that are not covered by plugins. For example:

   • Views UI depends on Views. Views UI does not provide views plugins. But the dependency information allows us to list it when I'm filtering the complete list by all (generic) "Views"-related modules.
   • Field UI ... ditto.

3. Other file-based discovery mechanisms

   E.g., as originally stated, the mere existence of a $module.api.php in a module folder allows us to classify it as an "API"-providing module.

4. Categories?

   After eliminating and automating that entire baseline, there may still need a need for custom tags — for all higher-level usage concepts that cannot be determined through automated code discovery.

   Those tags would typically include concepts like these:

   • Developer functionality
   • Layout / Structure functionality — That said, once all routes are converted into $module.routing.yml files, it would be piece of cake to scan those files and simply look for /admin/structure/. Positive match? Classify accordingly.
   • User interface functionality
   • Web services functionality
   • Administration functionality
   • Security related functionality
   • etc.

   Those are the major "conceptual" tags that I've applied to various modules in the patch in #27.

   In combination with the other automations, we might be able to cut down this list to a list that is actually limited. In turn, we'd talk about predefined categories, not tags.

   Our existing project categories on drupal.org might be able to serve as a starting point (although I believe that list could use some serious love, reviews, and reduction).

   What's important to understand is that this static list of categories would always be enhanced by the dynamic list being derived from plugins and other automated discovery methods. That's the only reason for why predefined categories would work for me.

What do you think?

@klonos:

1) "Modules as entities" makes no architectural sense at all, sorry. Entities represent typed data. Entities are of a certain type, and for each type, multiple entities may be created, updated, or deleted. Modules/extensions are not entities, in absolutely no way.

2) There is no need to customize the meta-data of modules, and it is guaranteed that no one wants to do that, since there is no point in doing so. See @bleen18's reply in #29. As I've mentioned before, this entire train of thought of "allow users to customize" is solely based on the poor state of our current meta-data and user interface. It is the wrong way to approach the problem space.

3) Instead, we need a more clever and smart way to classify modules/extensions. Once that is in place, the idea of "customizing" that meta-data inherently becomes obsolete.

re: #31: @Wim Leers:

I didn't fully think through that yet. One potential way would be to split the plugin-derived categories into a "Functionality" filter (select/dropdown) and the predefined categories into a "Topics" filter (select/dropdown).

But they could also appear altogether in a single filter dropdown; i.e., similar to earlier screenshots (and the latest tags-based patch).

Both might work. One might work better than the other. That first requires data, and then, testing. :-)

In general, I've privately drawn the conclusion that we're facing two semi-independent tasks here:

1) Machine-based and automatic/manual classification of extensions.

2) Exposing a suitable user interface to filter/drill-down extensions by (abstract) "type".

Please note the revised thinking and proposal in #30.

That said, core vs. contrib can be determined very easily in an automated way, so it would be compatible with that.

Whether "Core" and "Contrib" filter categories would be helpful for users of this page in any way is a separate question though (as those categories would not denote functional topics). :) However, one would be able to make the point that you have to disable all contrib modules for e.g. a Drupal core upgrade, and in that case, you'd want to filter by "non-core functionality".