Problem

Goal

  • Fix the discoverability of core and contrib and custom modules for users that are searching for particular functionality.
  • Design for the user's task, not technical aspects.
  • Re-invent to scale and solve the problem for the next 10 years, instead of a stop-gap fix that's tacked onto a design of the past 10 years.
  • Clean up and simplify the categorization on the Modules page.

Details

  • Instead of "package", we introduce tags.
    • A module can be classified into multiple tags/keywords/categories, if it provides related functionality.
    • Instead of a single package, there will be multiple tags/categories, meaning that modules like Taxonomy can easily appear in both "entity" and "field". — In which of both would you search it? Doesn't matter, it's there!
    • At the same time, the total set of all categories makes more sense, because we stop the "packagitis" problem that is being caused by tons of modules creating their own, new package, because they think they are special (they are not).
    • Lastly, to avoid tagitis/categoritis, we automatically strip away all tags that only appear once. Thus, if only one module tags itself as "beavis", and only one other module tags itself as "butthead", then neither of both categories will appear. Each of both modules will have tagged itself with some other terms and can be found within those (hopefully more reasonable) categories.

Proposed solution

  1. Introduce tags[] for module .info files.

  2. Remove the details elements that are based on package.

  3. Add a select dropdown to filter by a tag, so admin/modules?tag=foo only shows a filtered list.

  4. Additionally enhance with JS: Expose the tags to the front-end via data-tags=",foo,bar," HTML attributes, and dynamically filter in the front-end.

Todo/TBD

  1. Tag values:

    What exactly constitutes "related functionality"? Let's think ahead of time:

    A tag "field" could mean a "field storage", "field type", "field widget", "field formatter" providing module. There will be a lot of field related modules. Just Drupal core alone provides 8+ field type modules out of the box. But what if the user was looking for field storage modules? OTOH, what if the user actually did want to find out about all available field-related modules?

    What if tags would additionally be able to be namespaced? E.g., just field, or field/storage, field/type, field/widget, field/formatter (and any combination thereof).

    Note that tags do not necessarily refer to module functionality. "mail" would be a tag that refers to E-mail related functionality (for which there is no module in core, only a core subsystem/component).

    Likewise, issues like #623544: How to categorize a module as developer module are seeking for a way to classify certain modules as "pure UI" modules (such as Field UI, Views UI, etc) as well as developer modules, which can be safely disabled on production sites. Such modules could potentially be tagged with just "api/ui" and "dev".

  2. Human-readable categories/keywords vs. internal/machine tags/identifiers

    History and experience with human-readable package names essentially eliminates that option for tag values:
    #730374: Module packages using different letter-casing appear multiple times
    Also closely related: #591682: Config pages are effectively hardcoded to some subcategory of 'admin/config'

    Internal tags/identifiers are more robust, because they refer to identifiers that are usually related to code instead of arbitrary strings that might have a particular meaning today, but not necessarily tomorrow.

    Human-readable filter option values will likely become an issue though. We should probably consider to introduce an additional tag-to-label mapping, whereas Drupal core would be expected to provide labels for many tags, not only for the tags it uses itself. This mapping would also have to live in .info files, since the information is needed before a module is enabled.

  3. Tags in a multidimensional, modular system.

    The usual drama of 2D vs. 3D. Some contributed modules might use a "thing" tag, while some other contributed modules are using a "things" tag. For example:
    #227687: Properly capitalize "User interface" package name

    To some extent, this problem space could be combated by a simple/naive Porter-Stemmer algorithm when tags are collected and processed.
    To some further extent, Drupal core can help to combat it by providing a good, consistent, and extensive set of tag names; i.e., if there is an existing tag in core, then contrib will use it, too.

Files: 
CommentFileSizeAuthor
#27 module.tags_.27.patch29.46 KBsun
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch module.tags_.27.patch. Unable to apply patch. See the log in the details link for more information. View

Comments

sun’s picture

Issue summary: View changes

Updated issue summary.

Gábor Hojtsy’s picture

This might be useful *if* we end up in #1833184: Find a consistent naming scheme for translation-related modules not creating a new core project group.

Bojhan’s picture

Could you expand a bit on how you imagine this works? A "select list" to filter by tag, seems like the wrong UI for something like this.

I would like to have some examples of classification, what the impact will be on the module page in terms of displaying the tags and what rules we need to make for contrib. I am perfectly fine with this idea, if it turns out similar to the "Configuration" page where we provide very good defaults. However I am often told the "Configuration" page, is not fitting - although I have seen only some evidence to this, it seems the majority of contribs are correctly categorized.

I am for this change, but it needs to be expanded upon. Because the current categorization is visually quite simple, and tags will complicate this matter.

sun’s picture

Title: Introduce module .info file tags[]/keywords[] to categorize modules by provided functionality » Introduce module .info file tags[] to categorize modules by provided functionality
Status: Active » Needs review
FileSize
15.4 KB
10.88 KB
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: failed to enable simpletest module. View

Proof of concept patch and screenshot attached.

module-tags.3.png

Status: Needs review » Needs work

The last submitted patch, module.tags_.3.patch, failed testing.

sun’s picture

Status: Needs work » Needs review

oh, yay - fixing PIFR to be able to manually install Drupal with this patch will be extreme fun! ;) I'll try to make sure to roll further patches with the do-not-test suffix.

Regardless of that, this PoC patch works pretty nicely already. I'd recommend to try it out.

sun’s picture

Some interesting and quite important observations:

1) Technically we could automatically tag modules with "api" if they ship with a $module.api.php file.

2) Every module that exposes an API additionally forms an own tag name.

3) Logically, every module that implements an API of another module should be tagged as "functionally related" to that API module.

Please bear in mind that this information cannot be determined or retrieved automatically, since that operation would require to execute and parse code at runtime. We are still talking about meta information that needs to be available before code is loaded and executed.

I only recorded the above observations while working on this patch. Technically, 3) should be contained in this patch (but not necessarily complete). 1) and 2) are food for thought.

Changes:

  1. Added tags for all core modules.
  2. Added missing empty option to tag filter select dropdown.
  3. Sort tag filter select options alphabetically.

Dare I say... holy shit? Try this out. — A decade ends and a new era begins.

module.tags_.6.png

pcambra’s picture

Probably this would help here too #1323826: Make it easier to identify and discover ecosystem modules
For some reason I think that the filter by tag would look more natural as an autocomplete.

sun’s picture

Issue tags: +Usability

It wasn't my intention to ignore comments, but much rather, first try to whip up a proof of concept change proposal and additionally validate it against all modules in core. That essentially happened with #6 - which is still "alpha" quality and not meant to be complete, but the moment you try it out, I can almost guarantee that you do not want to go back. ;-)

re: #2: @Bojhan:

A "select list" to filter by tag, seems like the wrong UI for something like this.

Hm. Can you explain why?

Coming from a task perspective, I see the following:

1) I seek to enable certain functionality.

2) I go to the Modules ("Extend") page.

3) The available functionality is unfiltered by default. In case I know what I want, I can execute my task directly [and my task ends here]. [There's no inane list of undiscoverable packages/groups, just a plain list of modules sorted alphabetically.]

4) I have a vague idea of what functionality I want to enable, so I consult the "Filter by tags" dropdown in order to strip down the massive amount of options to the ones of my interest.

5) As mentioned in the OP, yes, the internal/machine tag names might be an issue. We might need to translate the internal tags into human-readable names (on the assumption that the tags are too technical, which very likely requires formal usability testing to be sure whether that is actually the case).

6) After filtering by a functional tag, I can see and operate on the results. [Any knowledge of "packages" no longer exists in the first place, so I get a plain list of modules filtered by tags.]

7) If this result does not satisfy my needs, I rewind and go back to 4) and try a different tag.

Potentially extended/alternated by:

7b) If I'm able to phrase my need as a textual keyword, I can also try to enter a search phrase instead. This functionality is not within the scope of this issue though, but bound to #396478: Searchable modules page instead. [Note: That issue becomes a piece of cake in combination with the changes here, since this patch here removes the package grouping altogether.]


The point where HEAD utterly fails is 4), 5), 6), and 7): With Drupal core only, you are not able to reproduce it. That's why I've added the screenshot of a D7 site in the OP: The list of modules is essentially "pre-filtered" (grouped) by a very technical "package" thingie that doesn't make sense at all to you:

It is entirely impossible to discover a particular module of interest in the pre-filtered/grouped list of modules, since "packages" do not present the filter that matches your task of finding "functionality being related to X." The steps 4), 5), and 6) do not exist at all, since it is impossible to remove the pre-filtered grouping. You're stuck with 7) - manually trying every single group by manually expanding it and manually parsing all available options within, in order to check whether the functionality you seek for is contained - or not. If it is not, rewind and continue the same procedure for almost all groups that are exposed on the page: The functionality you're looking for could very well be buried into one of them.


re: #7: @pcambra:

For some reason I think that the filter by tag would look more natural as an autocomplete.

That's again closely related to #396478: Searchable modules page — however, please bear in mind that "searching" is a completely different task than "filtering":

1) A user that searches commonly knows already what he wants to find - at minimum to a certain textual extent, which allows to phrase the search term in the first place.

2) Searching requires to manually type in a search term. Thankfully, the New World Order™ of mobile/touch devices sufficiently clarifies why the Average Joe does not want to use a keyboard.

If a user does not have a very concrete idea of a textual term to search for, then the user will not use the search. And if it's easier and quicker to use a filter, the user won't use the keyboard to type. A filter is not only used to discover, it is also used to save manual work.

EDIT: Sorry, forgot:

3) A search yields results on textual comparison only. Even if you searched for the right term, that does not inherently mean that you will get all available/possible/matching results.

Anyway, bottom line: search != filter. Probably for a dozen of essential reasons. :)

klonos’s picture

...since you mention #396478: Searchable modules page, I just had a crazy idea over at #1864980-38: [meta] Figure out how to integrate Views into core that we converted the modules page to a view.

If each module entry would actually be an entity, then it could be tagged and people would be able to customize the vocabulary they used for this tagging + perhaps export their settings for future uses in other installations.

I'm just saying. I don't know if this can actually be implemented.

dawehner’s picture

Is there a specific reason why you don't want to use the #states system for the filter?
In views we basically have a similar UI (adding new fields) but we couldn't managed to use #states when converting to a table,
so maybe you have a good reason to use just JS.

sun’s picture

@klonos: Views is a module that may not be enabled, so the Modules page cannot use it, because it is the very page that allows you to enable it.

@dawehner: #states is an interesting idea. However, I'm not sure whether it is up for the task - #states works well for triggers, conditions, and states applied to individually targeted elements - but for this task here we want to apply a dynamic value as a trigger and state condition for a collection of elements. I don't think that #states support dynamic values currently. Investigating that would be a nice (separate) issue though; should not hold up this issue - we can always improve/simplify later.

However, I think we need some more feedback on the actual change proposal first. I haven't seen anyone screaming "NO!!!" ;) but I also haven't seen anyone saying "I tested this; Awesome! +1000!!!" :)

Bojhan’s picture

FileSize
13.83 KB

I am not really sure I follow, are you expecting this to work like tags or like categories? Because for categories, a select list is fine but if you expect people to apply two/three tags to a module - you can't have a single select input method.

More common interface for tags, is something like;

The usability of this really depends on how we expect modules to be tagged, inside-outside core - I haven't really seen your ideas around that. Other than simply doing it in the patch, which doesn't provide much guidance

damiankloip’s picture

Yeah, I'm not sure #states can deal with this at the moment, we have a similar views issue with this problem (#1832862: Make views add field scannable - I think).

I think the concept of this idea is great, and is something we have regressed from currently on the views listing page. So it would be good if we could have some reusable code that could help with this?! Although, I understand that the module listing page is a different animal to entity listings.

The patch removes the group too? I really like how this works, but I think we should have this as well as the grouping, as this is better for browsing.

YesCT’s picture

#11 about views... other pages in core are being converted to views. Will views be enabled by default? Will just views ui be disabled, so default views, that are included will still work?

Also from #11

However, I think we need some more feedback on the actual change proposal first. I haven't seen anyone screaming "NO!!!" ;) but I also haven't seen anyone saying "I tested this; Awesome! +1000!!!" :)

I think the page can use some help so exploring this seems like a good idea.

I'm concerned about the idea of tags.

To clarify: Who does the tagging? Module mantainers? Answering my own question: yes, since it's done in the .info file.

Would this be dual purpose and reused on d.o? Lets say a site has 100 projects. That is on the order of 100 times more, but maybe we can look to that for ideas. [edit: here is the page I was looking at on d.o http://drupal.org/download ]

d.o has module categories. are these like the tags?
it also has "new"
and "index" which is I think alphabetical

maybe this tells us we could use sorts for:
categories (tags)
newly enabled
newly installed/downloaded
index/alphabetical

sorts are different than filters, and the proposal here is to filter by tags

I think that an autocomplete for tags would not be useful, because it will not be known what tags there are. So a drop down shows all the choices and then the best first guess can be made.

I guess as a start to think about it, project maintainers would tag each module in their project with the project name. And then any projects that closely relate to it.

Will we have a list of tags used so far... some process of scraping that info out of d.o and showing it to module maintainers so they can see ones already in use and re-use them?

Would a sort by tags give essentially the same thing that we have now, with modules grouped by project? But with duplicates, as some modules will be under more than one category.

Core would be one of the tags? so we could still get a list of just the core modules.
It would probably make sense to have a custom of using a "contrib" tag. (the opposite of core)

In the expanded description, in addition to version, required by, configuration, permissions and help, I suggest listing the tags. There the clicking on the tag link would filter the extend page by that filter.

module-tags-s01-2012-12-18_1332.png

tim.plunkett’s picture

Since the {system} table went away, Views cannot list modules anymore, until #1846454: Add Entity query to Config entities is fixed and someone writes a ConfigEntity query backend for views.
So let's not waste any time discussing Views here.

Grayside’s picture

Strikes me as the right direction and a positive step.

This sounds like stepping through the life cycle and governance issues of 1990s internet directories. It makes me wonder if the curated structure doesn't go far enough.

Brainstorming even further: wouldn't a full-on search index be more effective than agonizing over keywords, stemming, synonyms, etc?

Edited for clarity and reduced brusqueness.

sun’s picture

@Bojhan:

are you expecting this to work like tags or like categories? Because for categories, a select list is fine but if you expect people to apply two/three tags to a module - you can't have a single select input method.

It sounds like you're giving the terms "tags" and "categories" a lot of underlying meaning, which I've troubles to decipher right now :)

Modules would definitely be tagged with multiple terms. Each term/tag represents a certain, related functionality or topic. A "facet" user interface that allows to refine and drill down the results further is an interesting idea. However, I'm not sure whether it really applies here, since it is mainly used for filtering interfaces for the task:

"Show me all items that are related to X, Y, and also Z."

I might be totally wrong, but I think that the task and use-case of the Modules page does not involve such a multi-filter task. Instead, as a user I typically go there with the task:

"I need functionality related to X. Show me available options."

The fact that items are tagged with multiple terms doesn't really matter here. What matters is that (I believe) the user has only one topic/term in mind, because he/she is coming to this page with a concrete goal/task in mind; e.g.:

  1. "I want to enhance my site to do or have X."
  2. On the Extend/Modules page, I see all possible options. Unless I know exactly what I want already, that's not helpful at all.

So let's presume the X I wanted is "A form on my site so people can mail me."

  1. Given this (prominent) filter option, I'd probably consult it to search for the terms "message", "form", "e-mail", or "mail", and filter by them.
  2. If I find a related term, I'd try it, and otherwise, I'd look through the available terms to see whether there's one that comes close to what I want.

Now, adding another term on top of an already existing term doesn't really make sense to me here, since 1) the amount of filtered results will be small, and 2) the combination of more than one term will most likely yield a single result only in many cases.

That is, because the total set of available options is large, but it is not infinitely large — i.e., a facet/refine filtering makes sense if you have a store with 14,000 items that can be filtered, which inherently means that there will still be dozen or possibly even hundreds of results after adding and applying 2-4 facet filters. In this case here, we're dealing with a maximum amount of ~200 items. Applying a single tag filter will drill it down to <~20 items already; whereas 1/10 is sorta the maximum; the reality for most terms will probably be <=8. In turn, applying an additional filter to 2-20 items is rather pointless.

I'm not sure where exactly that falls into your separation between "tags" and "categories" (and why). ;)


@YesCT:

Let's bear in mind that this tag facility is only about modules that are available on your local system already. Your thoughts on a potential correlation to the project filtering on drupal.org are interesting, but in the end we're talking about two vastly different things here: d.o is about projects, while the Modules page is about modules. It is probably true that d.o could parse and aggregate the tags from module .info files within a project's repository, in order to potentially replace the current manual categorization of projects. However, in that case, we'd want to design the system from the bottom up; i.e., generate data, and only afterwards, we can use the data.

I'd recommend to keep a potential re-usage of the data by drupal.org out of this issue for now.

Would a sort by tags give essentially the same thing that we have now, with modules grouped by project? But with duplicates, as some modules will be under more than one category.

Essentially yes (including the part about duplicate results).

However, to clarify, the current modules are not grouped by projects, they are grouped by (a single) "package", and the meaning of packages is very abstract and really hard to communicate and translate into human speak, and thus also very hard to educate developers about — therefore, the UX problem we're facing starts way before the user interface layer, it starts with a technical DX/understanding problem already - and despite various attempts to clarify and improve the situation in the past ~6 years, the problem still exists.

Core would be one of the tags? so we could still get a list of just the core modules.
It would probably make sense to have a custom of using a "contrib" tag. (the opposite of core)

Possible. Though from a user perspective, I'm afraid I don't see the use-case of filtering by "core" or "contrib". I'm searching for functionality related to X, and that's not really functionality I'm searching for?

In the expanded description [...], I suggest listing the tags. There the clicking on the tag link would filter the extend page by that filter.

That's a nice idea! :) I'd probably defer that possible enhancement to a follow-up issue though.


@Grayside:

The connotations you've brought up are related, but I think they're not the same. The tags as proposed here are still defined freely, so contrib can invent and introduce any kind of tag at any time.

There is, however, indeed a desire for curating the final/resulting list on a particular site - that is, in the sense of not wanting to end up with hundreds of tags, since, at that point, you'd potentially have more tags than you have modules. ;)

Next to the bare volume/amount, there is also a risk and danger of "senseless" tags. For instance, to provide a stupid but perhaps even not so unreal example, I could go on and tag all of my modules with "unleashed mind", and some other folks in here could as well do that by tagging theirs with "acquia". ;)

Now, you will probably agree that this would be "Ugh, no way." — but if you come to realize it, OTOH, the tagging mechanism would inherently allow us to automatically add a tag to all modules that are part of and shipped with the installation profile with which you might have installed Drupal. "Show me everything that's part of my Open Atrium distro." is a use-case that would make sense to me (contrary to the core/contrib scenario mentioned above).

So, yeah:

Ultimately the challenge is to invent a categorization system within a multi-dimensional system, which is neither limited top-down, nor too lax/senseless, and in the end, still accomplishes its original goal to actually help the user in performing his/her task.

The proposed approach is to open the doors wide open, but to implant a good sense into the DNA by shipping and starting with a sophisticated set of tags, and leaving/combating the rest through education (of developers). A concise handbook page about "Do's and Don'ts" will totally be required.

sun’s picture

Upon re-reading #17, what I described in points 3. and 4. in reply to @Bojhan is actually a facet filter, but NOT one that stacks filter conditions as AND conditions, but instead, as OR conditions. ;) I essentially described it:

3. I'd probably consult it to search for the terms "message" OR "form" OR "e-mail" OR "mail", and filter by them.

[4. If I find any other related term, I'd simply add it.]

I'm still not sure whether a multi-facet filter interface will make sense, but at least the idea of multiple OR filter conditions makes a lot more sense than multiple AND filter conditions.

sun’s picture

FileSize
5.85 KB
26.92 KB
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: failed to enable simpletest module. View
  1. Added handling for untagged modules.
  2. Added output of tags in module descriptions + clickability.
  3. Added tags to new .info files.
sun’s picture

FileSize
734 bytes
26.91 KB
FAILED: [[SimpleTest]]: [MySQL] Failed to run tests: failed to enable simpletest module. View

Fixed bogus HTML markup in module description tag list.

Status: Needs review » Needs work

The last submitted patch, module.tags_.20.patch, failed testing.

sun’s picture

I'll re-roll this shortly. That said:

Since #1848064: Allow to filter modules by arbitrary search strings on the Modules page landed recently — did anyone use the Modules page more extensively in the past days, to gain some more conclusions?

As mentioned over there, the search definitely works for users, who already know what they want to find. New/novice users don't know that. That's where tags could kick in.

I'll perfectly admit though, these lowercase, untranslated, and rather technical tags might not make ultimate sense to novice users either. In the grand scheme of things, I'd probably classify them as "intermediate" (vs. beginner/advanced), as they presume that you know the tag/category under which you could find a module that relates to your interest.

That shouldn't block us from introducing them though (and finally getting rid of exposing packages) :)

sun’s picture

Status: Needs work » Needs review
FileSize
2.87 KB
29.29 KB
FAILED: [[SimpleTest]]: [MySQL] Setup environment: Test cancelled by admin prior to completion. View
  1. Merged HEAD.
  2. Added tags for new modules in core.

Did not update the UI/JS to work in combination with the new search facility yet, so please note: Right now, only one of both filters works at a time.

I'd highly recommend to test this live by applying the patch and provide your feedback.

The last submitted patch, module.tags_.23.patch, failed testing.

Wim Leers’s picture

Status: Needs work » Needs review

#23: module.tags_.23.patch queued for re-testing.

Status: Needs review » Needs work
Issue tags: +Usability

The last submitted patch, module.tags_.23.patch, failed testing.

sun’s picture

Status: Needs work » Needs review
FileSize
571 bytes
29.46 KB
FAILED: [[SimpleTest]]: [MySQL] Unable to apply patch module.tags_.27.patch. Unable to apply patch. See the log in the details link for more information. View

Sorry, I should have used a do-not-test suffix for previous patches. Due to the form structure changes of the system_modules form, PIFR is not able to enable Simpletest module. I've added a temporary BC layer for now.

klonos’s picture

This idea is a great step forward, but leaves us with the situation where tags[] maintenance is left to module maintainers and end users cannot customize them without loosing their customization after the next module update.

We need a way to allow users to tag their modules themselves and I vaguely recall someone suggesting that modules (or their entries in the modules page to be more precise) should also be entities. Crazy, I know, but if it could be done, then it would allow admins to customize their local "Modules categories" vocabulary as they saw fit and simply tag/untag their modules. I guess converting the modules admin page to a view would also help in that direction (#1823450: [Meta] Convert core listings to Views / #1823608: Admin views in core).

bleen’s picture

We need a way to allow users to tag their modules themselves...

Why?

I dont see a good use case for this unless I was able to tag modules across sites so that every site I work on has a consistant categorizations of modules. If I (for example) tag pathauto as "seo" that's only helpful if every site I work on always has pathauto tagged "seo" or if I am coming to the module page a lot looking for the pathauto module.

The former is not practical; the latter seems like a big edge case... why would anyone bother?

sun’s picture

@klonos: I believe it was me who suggested custom/user-defined tags for modules some years ago. It once sounded like a potential solution, but by now, that doesn't make any sense at all to me anymore. Users are using Drupal to manage and classify plenty of things. But modules/extensions do not belong to that. Doing that would be a violation of separation of concerns. This data is not generated by me, so the fundamental idea of asking me to classify data that I did not author is pointless and wrong.


Meanwhile however, I spent quite some time to think some more about what I stated in #6 already:

1) Technically we could automatically tag modules with "api" if they ship with a $module.api.php file.
2) Every module that exposes an API additionally forms an own tag name.
3) Logically, every module that implements an API of another module should be tagged as "functionally related" to that API module.

  1. Plugins

    Part of the New World Order™ and reality we're facing in Drupal 8 is that a big part of the concept of "functionally related" happens to be primarily expressed by plugins.

    A pretty dumb and trivial plugin type + implementation discovery algorithm would be able to deliver all of these module classifications already:

    • Entity types
    • Blocks
    • Field types
    • Field widgets
    • Field formatters
    • Filters
    • Text editors
    • Views fields
    • Views styles
    • ...

    That is, in a fully automated way — without the need for manually specified tags. Additionally, aforementioned human-readable labels can be determined literally in the way I stated them, because they are defined as translatable strings in code already.

    And due to the way the plugin system works, this classification would inherently support "parent namespaces" already; i.e., we're easily able to list all "Field"-related modules.

    Furthermore: A plugin category would only appear if the module that provides the plugin type is available. I.e., if you have a dozen of modules that provide plugin implementations for Pathauto, but Pathauto does not exist in your filesystem, then there is no point in exposing this category as a filter. A plugin category only makes sense when you actually have the plugin type providing module available (not necessarily installed) on your site/in your filesystem.

  2. Dependencies

    Another major chunk of "functionally related" can be easily derived from the existing module dependencies[] declarations.

    This primarily covers cases that are not covered by plugins. For example:

    • Views UI depends on Views. Views UI does not provide views plugins. But the dependency information allows us to list it when I'm filtering the complete list by all (generic) "Views"-related modules.
    • Field UI ... ditto.
  3. Other file-based discovery mechanisms

    E.g., as originally stated, the mere existence of a $module.api.php in a module folder allows us to classify it as an "API"-providing module.

  4. Categories?

    After eliminating and automating that entire baseline, there may still need a need for custom tags — for all higher-level usage concepts that cannot be determined through automated code discovery.

    Those tags would typically include concepts like these:

    • Developer functionality
    • Layout / Structure functionality — That said, once all routes are converted into $module.routing.yml files, it would be piece of cake to scan those files and simply look for /admin/structure/. Positive match? Classify accordingly.
    • User interface functionality
    • Web services functionality
    • Administration functionality
    • Security related functionality
    • etc.

    Those are the major "conceptual" tags that I've applied to various modules in the patch in #27.

    In combination with the other automations, we might be able to cut down this list to a list that is actually limited. In turn, we'd talk about predefined categories, not tags.

    Our existing project categories on drupal.org might be able to serve as a starting point (although I believe that list could use some serious love, reviews, and reduction).

    What's important to understand is that this static list of categories would always be enhanced by the dynamic list being derived from plugins and other automated discovery methods. That's the only reason for why predefined categories would work for me.

What do you think?

Wim Leers’s picture

I think all of that makes sense, but: how are you going to present this in the UI? There could potentially be many categories?

klonos’s picture

... Users are using Drupal to manage and classify plenty of things. But modules/extensions do not belong to that. Doing that would be a violation of separation of concerns. This data is not generated by me, so the fundamental idea of asking me to classify data that I did not author is pointless and wrong.

To clarify things, this is not about Drupal site users but about Drupal site admins (though one could argue that they are users as well). And although site admins do not generate modules themselves, they are the ones that add them to the site and the ones that manage them. So the same way a site author adds an article to the site and wants to be able to classify it, site admins add modules to the site and they should be able to classify them.

The idea to have module (entries) as entities would bring a lot of "goodies" along and still sounds worth pursuing to me. They would become fieldable and the modules page could be converted to a view that could in turn be tweaked as each site admin saw fit. A lot of the disagreements and endless discussions we had in #538904: D8UX: Redesign Modules Page would have been avoided this way. All we'd have to do is provide a basic view with most common features to be used out-of-the-box and then add "recipes" as documentation (how to add/remove module descriptions for example). Dependencies could be references to other module entries entities and by adding the update status as a field, admins could combine the modules page with the available updates page. Well, all these and more...

So -to bring this back to the topic at hand- if we had module entities, they could also be (free)tagged in order to be categorized as desired and filters exposed so that they could be filtered. We could have an "official" vocabulary (synced with d.o perhaps) to provide a default filtering, but that could be overridden by (or combined with) any custom one.

There's definitely not enough time to bring this functionality in D8, I know, but even so, the tags[] entries within .info files solution suggested here does not provide a way to allow site admins to make customizations that stay persistent across core/contrib module updates. They'd need to coordinate with module maintainers in order to add/remove tags and even then it would take a considerable amount of time before changes were available to them (because these kind of minor changes simply do not justify new releases of modules).

Now, if we came up with a way to allow for "local" changes in order to make these hard-coded tags override-able, I would then stand down.

sun’s picture

@klonos:

1) "Modules as entities" makes no architectural sense at all, sorry. Entities represent typed data. Entities are of a certain type, and for each type, multiple entities may be created, updated, or deleted. Modules/extensions are not entities, in absolutely no way.

2) There is no need to customize the meta-data of modules, and it is guaranteed that no one wants to do that, since there is no point in doing so. See @bleen18's reply in #29. As I've mentioned before, this entire train of thought of "allow users to customize" is solely based on the poor state of our current meta-data and user interface. It is the wrong way to approach the problem space.

3) Instead, we need a more clever and smart way to classify modules/extensions. Once that is in place, the idea of "customizing" that meta-data inherently becomes obsolete.


re: #31: @Wim Leers:

I didn't fully think through that yet. One potential way would be to split the plugin-derived categories into a "Functionality" filter (select/dropdown) and the predefined categories into a "Topics" filter (select/dropdown).

But they could also appear altogether in a single filter dropdown; i.e., similar to earlier screenshots (and the latest tags-based patch).

Both might work. One might work better than the other. That first requires data, and then, testing. :-)

In general, I've privately drawn the conclusion that we're facing two semi-independent tasks here:

1) Machine-based and automatic/manual classification of extensions.

2) Exposing a suitable user interface to filter/drill-down extensions by (abstract) "type".

Wim Leers’s picture

#33: your two semi-independent tasks seem sensible, if everybody's convinced this is a must-have. I'm afraid that's not yet the case. That's why I think for this to get in, it needs to be sold based on a convincing UI mock/prototype. Maybe ask Bojhan to chime in with some ideas in that area?

YesCT’s picture

During user testing I noticed that when the search box was used on the modules page for say: language, that the other related modules for translation were hidden. So maybe something like tags might help in that case.. to limit the giant list of modules there, but also show *groups* of modules that have related functionality and show those together.

See point 4 and 20 for user 1 http://groups.drupal.org/node/285978#comment-894708

mitchell’s picture

jhodgdon’s picture

I'm coming a bit late to this discussion (I was the one who filed #1910278: Modules are disorganized, which has now been closed as Won't Fix in favor of this issue)...

So, reading the issue summary for this issue, I do not agree with the Goals section entirely -- it doesn't cover my use cases and for me it would be a regression from the D7 and previous modules pages.

Background: I build and maintain a lot of sites for clients -- that is what I do for a living -- and way too often take on projects that someone else has started. For me, there is something important missing from the Goals and I think from this design, which is that it's often very important to know which modules are part of Core and which are ones I've added to Core. This is not just some "technical aspect" to dismiss out of hand as unimportant. It's important to me in these situations:
- When I update Core, I want to know which modules will potentially be changed.
- For major version updates, I want to know which modules will require more attention.
- When debugging a problem, I need to know where to look to find the module code and also it's quite a bit more likely the problem is from a contrib or custom module than core in general, so it gives me a clue where to look.
- When taking on someone else's project, I want to be able to see at a glance which contrib modules are in use.

So... what I'm trying to say is that the core vs. contrib distinction is meaningful. The proposed design in the issue summary loses the ability we have had in all previous versions of Drupal to make that distinction (and it's already been lost with the Multilingual category (not with the word Core) that is in D8 core now). We also used to have a distinction between "core required" and "core optional" which I think is also lost by this design. So, I would strongly request that some method be present to distinguish modules at a glance as core required vs. core optional vs. not core -- perhaps a filter? Or maybe "core - required" and "core - optional" are tags?

Gábor Hojtsy’s picture

@jhodgdon: I think core has a module built for this, its called the update status module. It not only tells you about your version numbers for projects (which are displayed nowhere on the modules page), but also which module belongs under which project. Eg. if there is a project shipping modules for different categories (eg. 3rd party integration and a submodule for Development), you can still see them together as coming from one project on the update status screen. The existing modules screen is not indicative of where modules came from already for contrib (which is also likely where people will wonder more, since they'll see unfamiliar module names vs. core where the same names repeat on all sites all over). In short I think the update status module serves this use case perfectly and the modules page is not targeted to also do this.

Gábor Hojtsy’s picture

Correction: version numbers are displayed on the modules page, but you need to manually open down each details element for each module to find them. Again underlines this page was not designed for this interaction.

jhodgdon’s picture

RE #38 - that is a good point. But could we not also have tags for "core-required" and "core-optional" on the main Modules page, along with the functional tags?

sun’s picture

Please note the revised thinking and proposal in #30.

That said, core vs. contrib can be determined very easily in an automated way, so it would be compatible with that.

Whether "Core" and "Contrib" filter categories would be helpful for users of this page in any way is a separate question though (as those categories would not denote functional topics). :) However, one would be able to make the point that you have to disable all contrib modules for e.g. a Drupal core upgrade, and in that case, you'd want to filter by "non-core functionality".

jhodgdon’s picture

How about putting the actual proposal into the issue summary? :)

Bojhan’s picture

@sun I think this can be much easier to follow with some images. I still don't really get what you are proposing.

YesCT’s picture

@jhodgdon I had a similar thought (#14) wanting to have tags also for core and contrib.

yoroy’s picture

Assigned: sun » Unassigned

Tags for categorizing modules so that a module may show up in multiple categories (when filtered) sounds great. Other than that, I have a difficult time visualising what's being proposed.

klonos’s picture

Title: Introduce module .info file tags[] to categorize modules by provided functionality » Introduce tags[] in module.yml file to categorize modules by provided functionality.

No more .info in D8 ;)

...unless of course we want to "backport" this (as a best/recommended practice) so that contrib can leverage this. AFAIK extra entries in .info files are simply ignored if they are not properties directly used by core (the recommends[] property for example).

jibran’s picture

Issue tags: -Usability

#27: module.tags_.27.patch queued for re-testing.

Status: Needs review » Needs work

The last submitted patch, module.tags_.27.patch, failed testing.

Anonymous’s picture

Issue summary: View changes

Scaled down image.

almaudoh’s picture

Title: Introduce tags[] in module.yml file to categorize modules by provided functionality. » Introduce tags[] in module.info.yml file to categorize modules by provided functionality.
Version: 8.0.x-dev » 9.x-dev
andypost’s picture

Version: 9.x-dev » 8.1.x-dev
Status: Needs work » Postponed

This could be introduced in minor version

andypost’s picture

Also having tags for themes make sense too

Version: 8.1.x-dev » 8.2.x-dev

Drupal 8.1.0-beta1 was released on March 2, 2016, which means new developments and disruptive changes should now be targeted against the 8.2.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.