[meta] Help system overhaul

If the User Guide (https://www.drupal.org/project/user_guide) proves to be successful then it could replace the module help texts (2nd function of the hook_help as mentioned in description). We could then move all that text out of drupal core into asciidoc files in the User Guide contrib module. This idea is kind of related to the idea of replacing hook_help with advanced help (#402404: Use approach like D7 Advanced Help for the Help module).

Has there been much usability testing on the existing system? It would be good to do some tests, even if we know the results will be poor. This seems like a good set of feedback from developers who implement help, but I think it's important to address the feedback from users accessing help too.

Getting feedback from the people who actually want to use this module and read help documentation would be a good way to establish objectives for the re-architected system so you don't end up building something that has a whole other set of usability issues.

@meeli this is from the last usability test: https://www.drupal.org/node/2497361#help with this corresponding issue: #2516902: Introduce a visual hierarchy to the help page.

[must enable] Would it be possible to have the help texts in a yml file instead of in the module? Then it would be possible to already read the help page without the module being enabled. (Similar to the module description displayed on the modules/extend page that also comes from a yml file.) Then it would also sense to say more about dependencies on libraries or so in the help.

[module and/or topic based help]
I think it is good to have a module based help that explains the functionality of that module, because these texts can also explain words used or the specific action of a specific tick box somewhere. It also means leaves the responsibility to explain that module - but only that module - to the module maintainers.

A topic based goes crosses several modules, and maintaining it especially for contrib modules will be more complex. Just as an example: Geofield, Leaflet and Openlayers modules can all provide their individual help/explanation for the module, but in a topic based help they would all need to go together somehow - and each additional geo-related module would need to be include there somehow as well.

Maybe it would be useful to group the different module-help pages by topic/functionality (similar to the Configuration page).

And we could make use of the User Guide by adding a section on the module help page with links to the relevant pages in the User Guide. That way we would have the best of both without overburdening module maintainers.

Do we want help to be contextual help, or do we want better in-app docs?

If its help, I would prefer something akin to tour module, which shows help in the context of the page.

If its docs, something like https://github.com/rtfd/readthedocs.org feels viable to me - a collection of structured articles in markdown/sphinx format that can be written or read in any text editor, but which can be easily parsed into useful, searchable, structured HTML. If you needed the per-path functionality of hook_help, you could have a module.help.yml file in the root of your module which mapped help doc files to paths, eg

doc: path/to/docs/file.md
  paths:
     route: routename
     doc: path/to/docs/thatroutefile.md

Could we include a more systematic glossary into the help system?
Because we seem to write more and more documentation - and systems to provide help - without an actual basis of what which word means.

There are lots of attempts to give longer explanations of how to do things with Drupal, and the D8 User Guide will hopefully bring us a long way. But often we miss the basic explanation of what term actually means. This is certainly the case for site builders who are working with core and contrib modules. It seems to be easier to find videos and blog posts, then a straight forward explanation of what the difference is between "entity" and "entity bundle", the three different uses of "context" provided by three different modules, and why one groups module refers to "non-members" and the other to "outsiders"?
Different glossaries exist for example on d.o., in the User Guide, or in several books, but there is no systematic or sustainable approach to keep them updated. So we end up with users who have to search through longer documents to find a basic explanation, and with contrib developers who have no place for checking whether terms are already used.

I know this probably goes beyond this issue, but could we provide a more robust glossary through the modules?
Could each module contain a list of terms and their definitions, that can be displayed as a glossary on a site, similar to the help page?

Then site builders and administrators would have the terms they need at hand, making Drupal easier to understand.
It would make module maintainers responsible for defining the words they use, instead of leaving it as an afterthought.

Here it is: #2592487: [meta] Help system overhaul

After some talking with @alexpott this morning, here is an alternative approach, which could be both flexible but also nice to work with: #2820166: Flexible plugin based help system
Unlike before we are combining the power of plugins with the niceness of twig templates, while still be able to keep the existing hooks in a BC layer.
Any feedback would be great!

This sounds like a good candidate for the Drupal core ideas queue.

There's already so much detail in the issue summary that I can't really see the general goal. From the "reasons to make changes" list, which are the main goals, improvements we want to aim for?

(For me it would be moving from per module to a goal/task related topical grouping)

Hi!

Thanks for all your work on this.

I just created: https://www.drupal.org/node/2873475 with a small fix.

Who do we need to ask to know if there Is interest to promote this module as replacement to Hook_help?

I think Hook_help is something that we really need to improve or replace with something better.

Did a quick test drive of the sandbox module. It's really nice:

Help topics overview

The module adds a section to the help page that lists the help topics.

Help topics admin list

In the admin there's a list of topics. You can add/edit new topics or unlock the ones that are provided by modules:

Edit a help topic

Here I added the existing "Getting started" instructions as a help topic. Looks like the content creation interface brings its own "help HTML" text formats and it has a "simple paragraphs" (?) approach to create structured content. For which there very likely are very good reasons (maybe even spelled out in this issue!) but first impression was that it felt awkward.

---

Thanks @jhodgdon, I hadn't taken the time to actually look at the thing before, but I really like it and think it would be a welcome improvement for Drupal 8. This probably needs some more fleshing out in terms of where to start and what the MVP should be. Moving this from "idea" to "proposed plan". Most importantly it would need a team of people willing to work on this.

@gnuget so yes, you're very welcome to help move this forward :)

Here I added the existing "Getting started" instructions as a help topic. Looks like the content creation interface brings its own "help HTML" text formats and it has a "simple paragraphs" (?) approach to create structured content. For which there very likely are very good reasons (maybe even spelled out in this issue!) but first impression was that it felt awkward.

The main reason to set this as "paragraphs" is (see https://www.drupal.org/node/1918856#comment-8445883) to keep the translations separated if one typo is fixed only that paragraph should be marked as "outdated" and not the whole page.

I'm willing to work on this to help make this happen.

- David.

I did a quick demo of this during yesterday's UX meeting: https://youtu.be/tKm4-MCUzSg?t=42m10s

@jhodgdon I would love to get this part of core. not sure how much time I can spend getting it in though.

Hi @jhodgdon, I've downloaded the sandbox module and am reading through the code. This is the first time I'm seeing this issue, but I'm very interested in helping out. I'd like to throw my hat in the ring to be the core maintainer, something I think I could do with some guidance/mentoring. I'm willing to learn the process and set aside time/energy for it.

Hi jhodgdon,

Happy to contribute my time for this.

Thanks.

@jhodgdon I'm in) Going to dig the code but can maintain the module in core

@jhodgdon Yes, let's proceed.

Regarding who should be the Initiative Coordinator, certainly you have carried the torch with regard to vision and getting the project to this point, so this role is yours if you want it, but you mentioned availability constraints, so I am willing to be put down as this, too. I'm confident we can communicate here or in backchannels to ensure that I'm up-to-date on the project and understand the scope and vision.

Issue summary, team, and plan all look great. As you mentioned, we can add more team members as we go along. I daresay between us, we will be diligent about adding documentation for the module, so I'm not concerned about the lack of docs lead right now.

Do you have a core patch ready that adds the existing Sandbox module to Core as an Experimental module? I think that's the next step.

Ha, looks like I took too long writing my comment. Thanks @jhodgdon for moving forward with creating the core patch!

Sounds good. Thank you.

Exciting to see this initiative picking up steam again!

To provide product manager sign off, we need an opportunity to see what you have in action. As mentioned over in #2920309-39: Add experimental module for Help Topics, the most straight-forward way to do that is probably to attend a UX meeting and demo the patch. Let us know when you'd like to attend!

I'm in favour of this as an idea.

I reviewed the patch on November 17th but did not post my review, as I didn't want to give any false hope before this issue was approved.

If this were to be an approved plan, I'd be happy to work to help shape the patch.

My only concerns with the current state of the patch were in relation to the section concept, where we were creating a new concept. This is similar to the concerns @xjm posted on the patch issue.

I don't think those concerns are insurmountable.

Normally what we look for in this queue, like in any other queue, is an issue to get to RTBC, which means that there's been a peer review process + community members have looked it over and deemed the idea solid and worthy of sign-off by a maintainer. So I didn't realize you were blocked on us; I thought this was still in discussion.

I am in support of the Quick summary of desired features from the issue summary. They are all admirable goals and certainly highlight the current short comings. One thing I would add is a desire that if all of this were to get in, ideally we could deprecate hook_help for Drupal 9 in favour of this approach.

So I think that counts as sign-off for this plan.

Correct me if I'm not doing this right..still finding my feet.

I signed off on the idea as well. I think signoff from framework and release managers is mostly needed for the actual implementation plan, but @larowlan wants to see this feature happen too, all the better :)

Onwards!

Thanks! We've tentatively queued this for review on the product management meeting next Thursday.

Yay, I'm so glad to see this moving forward!

Gábor, @yoroy, and I looked over the plan on today's product management meeting.

A couple of things:

1) Searching help is currently listed as "Lower-priority but still desirable features." Yet, if you look at any modern help system, normally it leads with search, even moreso than a topic listing (see for example, Chrome). It feels like that ought to move to a "must-have" list rather than a "nice-to-have" list (for stable release, not initial commit).

2) During the UX review, @yoroy flagged the authoring UI as a no-go since it was popping up a new WYSIWYG editor per block-level element. We discovered during the call that you have been hard at work on this at #2926651: Better UI needed for entering body text, but haven't been able to see that in action yet to give sign-off. Overall, though, looks very promising! Thank you SO much for tackling this head-on!

3) As part of that issue, it probably makes sense for a more frameworky/subsystem maintainery person to take a look at the "chunking" approach. If it passes the sniff test, this could potentially be a really good way to solve what will end up being the same problem in the OOTB initiative with sample content.

Marking "needs review" in lieu of that sign-off, but keep up the great work!

Related issue #1885192: Field locales_source.source is not suitable for long texts and huge config objects explains why help texts should be "chunked", I faced with it on #2836206: [Translations] Translation of big webforms broken

Some work in this area happened in ctools #2844054: Inline Revisionable Content Blocks but it will hit the same limit of string translation

It seems rather odd that there really isn't much discussion around using the existing theme system for these.

Of all the output I've seen from help, it always contains HTML.

HTML does not belong in PHP.

HTML does not belong in YAML.

HTML does, however, belong in Twig; an HTML templating engine (which supports our translation system).

Even if this were to move into some abstract help directory and using a closed twig templating system.... how is a theme like Drupal Bootstrap supposed to alter the markup to display topics appropriate using the existing framework supported markup/classes?

I'd really rather avoid circumventing the theme system on this one.

As it stands now, there is absolutely no [proper] way to hook into/style help output.

I really think this should just create plugins (automatically via YAML) and let the template name be extracted automatically based on the various criteria of the plugin definition.

Using a similar example from #2820166: Flexible plugin based help system:

foo:
  type: topic
  title: "Topic title"
  template: ... # Automatically determined, but can be explicitly overridden.

There should be a global template (theme hook) registered: help.

Then, when it comes time to render each help plugin, it's invoked using theme hook suggestions:
'#theme' => 'help__' . $plugin->getType() . '__' . $plugin->id(),

This would allow individual topics to be customized for their given hook:
templates/help--topic--foo.twig.html

By leveraging the existing theme system, help would finally give themes the ability to properly preprocess or override any help output. Not to mention the added benefits of using Twig itself (including the ability to use blocks and extend templates as needed).

Yes, I'm advocating for #2820166: Flexible plugin based help system, but with the added bit of using the existing theme system + theme hook suggestions for automatical template discovery (no need to specify it manually in the YAML file).

Actually "plugin-way" is in core, we have some default plugins - why there's no twig help plugin?

Configurable storage for help topics is blocker on way of getting rid of hook_help

I'm digging into #2937483: Defining a new type of section storage relies on magic strings and hidden assumptions

Core already have a storable "sections" so topical help module could use it as wrapper for topics

I guess another question I have about the plugins-plus-twig approach is whether we think that content should be edited/stored in Twig templates.

The "content" you're referring to is and has always been static, stored somewhere in a file.

Semantically, the help topics should be content entities, because they are content. But it's problematic to make them content entities because we don't have a good way to manage, distribute, and update content with modules, themes, and distributions.

Semantically, content entities are intended to be mutable. In the sense that it is done so by some external source. They are also intended to live in the DB, not provided by a file somewhere.

I don't understand having an argument about whether HTML markup should be or shouldn't be in the body of the topics, unless you think we shouldn't really have markup in the body of nodes or content blocks or any of the other content entities.

Again, this isn't mutable content, nor is it like the "body" field in nodes that have CKEditor. No one is going to be "editing" help topics from core/contrib from the site's UI... nor should they be able to.

Thus, the only place that makes sense to house said static HTML is inside a Twig template. Not PHP, not YAML. Besides, having it in Twig automtatically gives the ability for future expansion, i.e. "Oh crap, we forgot X"... "That's OK, we'll just send another variable that the Twig templates can use."

Usability folks nixed the idea of editing it that way -- too much of a paradigm shift.

Probably because you were still mixing where the "content" (YML) and where the "HTML" (PHP) lived, no?

People writing prose just want to write it in an HTML editor.

And they could do that all in a Twig template. Where the content + HTML is combined.

It may not be purely themeable, but as a practical matter, I think the theme has to give up the ability to get down to the inner parts of prose body areas.

I think this is a lot like saying "themes don't matter" and why major advancements to the theme system in core is still in shambles. Just because themes "haven't done X before" doesn't mean they shouldn't. It often means that they "can't" due to the limitations of core's theme system.

I mean seriously, what if I wanted to wrap parts of the help system with collapsible panels, or vertical tabs or modals? You're telling me that as a themer I have to just "suck it up"? Usability is all and well, but when it impacts the core value of the system, extensibility, we have a problem.

One of the points of the Twig initiative was to remove all HTML markup from PHP files. Yes, there are a few outlying rouge elements (e.g. #2638250: The label "Member for" on user profiles is hardcoded markup that is different from other user fields), but as you can see from that example, something even seemingly as innocuous as <h4 class="label"> can cause major issues when dealing with an external front-end framework like Bootstrap.

The point is that core, contrib, nor prose editors know exactly how or what will ultimately be displayed to the end user. That is the sole existance and primary purpose of themes.

If a prose (contrib) editor decides to stray from said "simple" markup and uses classes and markup from "core themes" like Bartik/Seven, etc. How is an external framework like Bootstrap supposed to counter that?

What is "practical" is to reuse existing systems (plugins/twig) instead of trying to reinvent the wheel here and circumvent the rendering process.

I think #77 is/should be an entirely separate issue/goal.

It conflates site-specific help with core/contrib help.

I personally do not want to get issues in my contrib projects saying that their site documentation says X, only to find out it's because some "marketing person" (who may not understand how something actually works) decided to edit the documentation provided by my contrib project. This is a pandora's box if I ever saw one.

Site-specific help should only be supplemental in nature, not authoritative over components it doesn't author.

Thus, I think that there are two types of "help" here:

1. core/contrib - Not "content" per se because it lives in files
2. site-specific - Supplemental content entities that are stored in the DB

The reality though is that core does not currently ship with mechanisms that this endeavor would seem to need:

https://www.drupal.org/project/default_content (converts file-based "default content" into DB content)
https://www.drupal.org/project/paragraphs (content structure)

I have reviewed the recent discussion (Comment #76 on). I think that the underlying problem is that @jhodgdon and @markcarver disagree on the goals of the proposed system. There is not much hope of agreeing on an implementation without first agreeing on goals.

Here are a few points that @markcarver has made:

1. I'd really rather avoid circumventing the theme system on this one. (#79)
2. I really think this should just create plugins (automatically via YAML) and let the template name be extracted automatically based on the various criteria of the plugin definition. (#79)
3. No one is going to be "editing" help topics from core/contrib from the site's UI... nor should they be able to. (#86)
4. Just because themes "haven't done X before" doesn't mean they shouldn't. (#86)
5. I think #77 is/should be an entirely separate issue/goal. It conflates site-specific help with core/contrib help. (#88)

In my opinion,

1. This is a new requirement. If we can find a solution that meets this without giving anything up, let's do it!
2. This points to a possible compromise. More on this idea in my next comment.
3. This opinion fundamentally conflicts with the goals stated in the issue summary. But also see (5) below.
4. This is in response to @jhodgdon's comment that "as a practical matter, ... the theme has to give up". If there is a practical solution to give the theme more control, then I do not think that @jhodgdon or anyone else will object. Patches are welcome!
5. Yes, the config-based solution conflates two types of help. I think we need some more discussion on whether this is a good or bad thing.

I guess that (1), (3), and (5) relate to the goals. Perhaps @jhodgdon or @Amber Himes Matz should comment on (5). That is, what is the argument for having one system to be used by writers of modules, themes, and profiles (MTP) as well as by site builders and site administrators (SB/SA)? The alternative is to have one system for MTP writers and another for SB/SA. The second system could be a contrib module, not part of core.

I will now ignore my own advice: I will discuss implementation without first agreeing on goals.

In #79, @markcarver talked about creating plugins from YAML files.

In #2920309-79: Add experimental module for Help Topics (and quoted in #76 on this issue), @dawehner wrote, "migrate in contrib allows you to add additional migrations (i think by using config entities)".

I think these comments point to a possible compromise. We already have a model: the core Migrate API defined a migration as a plugin, and the contrib Migrate Plus module defines the config type migrate_plus.migration.*, which is converted to a plugin. The Help Topics module could define (as it already does) the config type help_topics.topic.*, and we could convert those to plugins.

Any module that needs to implement complex logic, or that cannot allow site builders to edit its help topics, can implement the plugin using PHP code and/or Twig files. Any module (or theme or profile) maintainer who wants an easy way to write help topics can use the WYSIWYG editor and save the result as the YAML file help_topics.topic.*.yml. All of these methods are translatable.

The plugin definition could require that the build() method (or whatever it is named) return a render array. When generating plugins from config entities, we would have to make that happen. Once we have render arrays, the theme system should be able to do its part.

I am now going to be crazy optimistic. We can

1. Release the Help Topics module in its current form (as an experimental module).
2. Start writing help topics.
3. Develop the #2820166: Flexible plugin based help system.
4. Rewrite the Help Topics module to use the plugin system.

in that order.

BTW, do we really want to call it the Help Topics (plural) module and not Help Topic? I am reminded of "Paragraphs types" in early versions of the Paragraphs module.

> I will now ignore my own advice: I will discuss implementation without first agreeing on goals.

;)

I think we do really need to get the goals nailed down. We are getting really bogged down with details. Part of the problem is that there is no existing Drupal construction which quite fits (entities, plugins, templates, etc). So there's a lot of effort going into thinking about how to bend things a bit or combine things to make it work. But we're not all on the same page as to what we mean by 'work'.

> Rewrite the Help Topics module to use the plugin system.

It's true that getting something out sooner rather than later is good.
However, ending up with two systems, one deprecated and one new, is confusing.

> BTW, do we really want to call it the Help Topics (plural) module and not Help Topic? I am reminded of "Paragraphs types" in early versions of the Paragraphs module.

The module title can be plural, while the entity type label should be singular.

It's true that getting something out sooner rather than later is good.
However, ending up with two systems, one deprecated and one new, is confusing.

My idea is that it would be completely backwards-compatible. Any module that creates help_topics.topic.* config entities will work with the current version of the (experimental) HT module. Once the HT module is rewritten, those same entities will be converted to plugins, following the Migrate Plus model, but that will be transparent to the implementing module.

I did call this idea "crazy optimistic".

It won't be transparent to contrib module maintainers and developers in general though. We'll have contrib modules that adopted the 1st system early, but now are sticking with it because the maintainer hasn't had time to convert, and contrib modules that use the 2nd system. It'll make it more confusing to work on patches to modules because developers will have to work with both systems.

The analogy of this being similar to a "Body" field is flawed.

What is being proposed here are not fields. They are items inside a YAML file that contain static HTML.

This goes against almost every single design and architectural paradigm that currently exist in core and contrib alike.

At least with fields, it goes through a text formatter which can be intercepted and altered as needed; albeit rather expensively and only as a last resort. Its why modules like Paragraphs exist: to separate the pieces of content into separate objects (i.e. independently themeable).

Definitely there is none in Core, and the "paragraphs lite" system proposed in early iterations on #2920309: Add experimental module for Help Topics was rejected soundly by the Usability team as being unfamiliar and thus unusable.

This was only one individual that provided feedback on this topic. That being said, I actually agree. Despite the obvious theming advantages with highly structured content like this, it's ultimately a very poor and frustrating UI to actually work with. It's not the solution.

content editors definitely need to be able to indicate semantic markup in their content

Then use HTML... in a Twig template, as I originally suggested:

• Single place that a content creator can create semantic markup
• Don't actually have to know Twig syntax as its just a (loose) superset of HTML and any HTML works
• Part of the existing theme system
• Translatable
• Extendable (blocks, filters, functions, macros, and various other helpers, etc.)

It's a win/win scenario for everyone involved and actually gives content creators more control/abilities to provide semantic markup.

The primary purpose of YAML (in Drupal) is to define configuration, not provide content. Which is sadly often misunderstood and frequently abused.

Twig, while usually not in the business of providing "content" per se (at least in core), often does in actual production sites. Especially when it is relatively static content.

As a practical matter, HTML is a very convenient way to indicate semantic markup, complete with convenient visual editors that make it easy for non-technical people to put in that markup.

I would actually argue that Markdown is easier for "non-technical" people as its only supplemental in nature simply granting the ability to provide "syntax sugar" as desired. HTML is a language with specific syntax and elements that one has to remember. Which would be easier for a "non-technical" individual to write?

- Item 1
- Item 2
- Item 3

vs.

<ul>
<li>Item 1</li>
<li>Item 2</li>
<li>Item 3</li>
</ul>

Nevermind that you're also asking these "non-technical" individuals to also understand HTML syntax within YAML syntax. How exactly is that "better" for "non-technical" people? This makes no sense.

Fun little idea I just had: if we truly want to just arbitrarily create a whole new system, then maybe we should try doing so in a way that makes more sense. Like combining Twig + Markdown since they're both really just (loose) supersets of HTML.

We could then have a unique extension: .md.twig. Then the "non-technical" individual can then choose how they want to write it. Markdown, HTML, Twig... all the above. I think this would give us the most "bang for the buck".

Regardless of that last bit...

We already have a theme system.

We already have a plugin system.

We should use them, not avoid them.

@Mark your proposal is totally fine for coders but it missing 2 main features
- ability to edit topics in UI for site builders
- translatability - delivery of new template will break relation to existing transtlation

The compromise is already in topic entity that allows to take over theming for each topic in a common to Drupal way, also each plugin that renders parts of topic could be replaced with contrib/custom one that can inplement theming

Drupal is all about to store structured data & templates, md files and so on are not it

No, it's not.

- ability to edit topics in UI for site builders

We can (technically) do this with Twig templates now (previously done in 7.x, but as contrib and considered insecure since it was storing PHP in the DB)... we just haven't yet (aside from Views). We also know that two-way communication with Twig templates is possible (from conversations at DCs), but again... hasn't happened yet.

- translatability - delivery of new template will break relation to existing transtlation

Not if you leverage the existing Translation API and use the appropriate filters/tags in Twig.

Drupal is all about to store structured data & templates, md files and so on are not it

It was a crazy idea that popped into my head. That's all. The main "point" was to show how ludicrous the arguments for "non-technical" individuals really are. If we were truly worried about "non-technical" individuals, we'd get away from any syntax-based language (HTML, YAML, and Twig) and opt for a more human-readable format: Markdown.

Example: your last comment starts with "your proposal is totally fine for coders" which implies that one of the primary goals is for the ease of use with "non-technical" individuals. However, your last statement completely contradicts this: "Drupal is all about to store structured data & templates, md files and so on are not it".

So which is it?

@Mark I mean that we already have "coder-focused" hook_help() which is mostly unused in contrib/custom code, so moving it to templates will make it a bit "clear" from readability POV but passing additional data is harder (tokens, routes, etc)

The main point of my comment is that we should try make help system more friendly to *site builders* by allowing "traditional" UI to edit help and get better feedback from community about where to expand this experimental module

Ok, fine.

There are inherent flaws and utter disregard for existing APIs with this approach, but OK... experiment away.

I'll just stop trying to help.

Sorry for lending my expertise and comprehension on these topics.

"community"--

> The goals that I think have come out of all of those discussions over at least the last 8 years are in the issue summary here

Is that the 'Quick summary of desired features' section in the summary? it could do to be clearer I think.

Also, I think there are some of the more technical features that are missing. E.g: Changes to help topics in a new release of a module should automatically update the site's help topics.

> The advantages of help topics as plugins on #2820166: Flexible plugin based help system would be (a) presumably smaller code base (since there is no UI to maintain), and (b) config entities have disadvantages as far as migration goes.

But isn't a UI one of the desired features?

> And whether the name of the template file is chosen via theme suggestions or is in the plugin YAML, it would mean that a theme wanting to override the markup for help topics in general would need to override every single one of these template files, because each one contains the content and markup for an individual topic.

This is the bit I also don't understand about the Twig proposal.

If each help topic is a separate twig template, then the content is inside the template. That's surely going completely against the whole way the theme system works: templates are containers, and the theme engine flows the content into them.

Surely, if we want help to be themeable, we need only ONE template: help_topic.

> Here is the current proposed plan: Keep hook_help() in place, for "Module Overview" help topics. Add a new Help system that would co-exist, using config entities. Each module, theme, or profile can provide topics in config/install

This is confusing.

hook_help() is being kept for overview topics. But config entities provide topics.... what sort of topics are those?

And where is the route-specific help going?

Thanks!

So the system proposed here doesn't cover route-based help at all. Which is probably a good thing, as that needs to be able to respond to routes dynamically.

The issue summary should be updated with your improved description.

I'd also like to rewrite some of the criteria to be clearer. For example, this one starts off by expressing the current way of doing things which we want to change. If you skim-read, you could think that this is the way the new system should work!

> [non topical] Each module can only provide one main help page for the admin/help listing, and you'd need to know what each module does to find help for its functionality, or read all the topics (and they're just listed in alphabetical order). A more ideal technical writing system would let modules, themes, and profiles (instead of just modules) each provide multiple topics [...]

> Sure, if you want to rewrite the issue summary, please do! If you're asking me to rewrite it, I can do that too. :)

Well whoever gets to it first :) Me, on this occasion.

I've updated the proposed resolution - split into bullet points & incorporated some of your latest comment.

Another question:

Presumably, the magic hook_help() route "help.page.$modulename" for providing a module topic would become deprecated, rather than cease to work?

Upping the level of the headings in the summary & changing the subheadings of the 'desired features' list to be heading tags.

(Keeping changes to the summary separate for easier reversion if needed.)

> That would still be the way to provide a module overview (which is a help topic that appears, with title being the module name, in the Module Overviews section of admin/help)

So what happens if a module provides the magic route for the module overview, AND also a config help topic that is marked as top-level?

I had assumed that the list of top-level topics would be replacing the content at admin/help.

If not, where do the top-level topics get output?

I've been thinking over some of the comments and discussion and I have some thoughts that I hope might clarify the over-arching objective of this proposal. In any case, I wanted to get these thoughts out of my head and hopefully contribute to the discussion.

One of the mantras in Drupal's configuration management system is the "sites own configuration." It is my opinion that this mantra should be extended toward help: "sites own their help documentation".

I interpret "sites own configuration" to mean: While a module or theme can provide default configuration, configuration can then be updated at will by the site owner, exported to a site-specific sync directory, and deployed to another instance of the site.

With help, currently a module can provide (or alter) a module overview, route-specific help, or field text help. This can provide a range of guidance. This type of help is definitely useful but has limits depending on how the site is used or customized. (This proposal does not change or remove this existing functionality.)

However, once a site is built to site owner specifications, administering or updating content for the site is dependent on a unique blend of installed modules, form alterations, workflows, and processes. And so accomplishing a task such as updating a hero image on the front page slideshow is totally dependent on how the site was built. Just as the configuration of how a hero image is updated on the homepage slideshow is owned by the site, so should the documentation explaining how to accomplish this task should be owned by the site.

And this is where this proposal fits in. Not only to allow core and contributed modules and themes to provide help topics to perform certain common tasks, but also for site owners to provide help to their internal team on how to accomplish their unique site-specific tasks -- and to integrate this type of documentation with Drupal's existing help system.

Because of this, the creation and maintenance of "Help Topics" in particular should have an administrative UI so that site owners and editors can maintain their site-specific help topics without needing access to the filesystem/server. This (an admin UI) is how Drupal enables true ownership of many/most administrative tasks and this ability should be extended so that site owners can have true ownership of their site-specific help documentation.

Thank you @Amber Himes Matz, great overview and rationale for the proposed direction. I'm completely aligned with especially those last 2 paragraphs. I don't have the expertise (nor desire) to understand the detailed pros and cons of the different options for technical implementation, but from what I gather "help content as configuration" seems to be more directly aligned with these goals than "help content inside theme files".

Yup, thank you for the very comprehensive explanation of the motivations behind the proposed technical approach.

It might be an idea to add a bit of that to the summary. In particular, this:

> It is my opinion that this mantra should be extended toward help: "sites own their help documentation".

-- because when I first saw the plan to use config entities, one of my thoughts was 'site builders are not going to like having loads of help files dumped into their config!'

With that goal explained though, it makes sense.

I do have some concerns about the proposed plan though:

1. not deprecating hook_help() module overviews:

b) each module can provide 1 "module overview" help page, describing the module. These are (as of Drupal 8.1.x or so? don't remember exactly when we did that) listed on admin/help in a section called "Module overviews".

This proposed/experimental module adds a new section, which is called "Help topics", also with a descriptive overview.

That means that a module that provides help with the new system would be providing 2 overviews: the hook_help() one and the config help top-level topic.

That's going to confuse developers:
- should they drop the hook_help() overview?
- should they duplicate the content, or do the two things serve slightly different purposes?

It's going to be confusing for site builders and users too: do they read 'Module overview' first, and then move on to 'Help topics'? Or the other way round? How are the two sections different?

I would suggest:

- deprecate hook_help() overview topics
- for modules that have not yet provided config help, if a hook_help() overview topic exists, import that as a single top-level config help topic for that module
- don't allow a module to provide a hook_help() overview topic if it also provides config help -- throw an exception during a rebuild.

2. More generally, I'm a bit iffy about the various bits of config functionality that aren't there yet -- such as config update, or search.

Updated issue summary (under Background) and reformatted to hopefully clarify the proposal's primary purpose.

This needs to be done in Core. In past versions of Drupal, there were contrib modules (such as Advanced Help) that attempted to fix some of the problems with the Core help system, but they were not widely adopted. Getting the improved help system into Core would provide:

- an officially-sanctioned way for Contrib maintainers to provide better help with their modules, themes, and distributions
- for Core to provide better help for itself
- for site owners to provide help to their internal team on how to accomplish their unique site-specific tasks

Integrating better -- that is, single task/concept-based documentation -- with Drupal's existing help system rather than hoping that people adopt a system provided by a contrib module.

@joachim - does that issue summary update I just provided help clarify things? Do you see a need for further clarification in the issue summary? (Asking because you added the "needs issue summary update" tag.) Thanks!

Edited the issue summary (under Background) to also include a reason for considering site owners in the proposed solution (since this was requested in comments and was much discussed).

Why should site owners be included here? Because site's own their configuration, they should also own the help that describes how to use that custom configuration. (See also comment #114.)

Added a link to my DrupalCon presentation about this issue to the issue summary.

Also, removed tag "Needs issue summary update" as I did update it previously and (I think) addressed the concerns.

I am not sure if this is the right issue for that but I just like to ask a question about that. What's about the idea to open this module to all the companies out there solving problems with drupal by producing a video or a video series about a problem with drupal? Webforms help page demonstrates this with the old help system as well but wouldn't it be great to tell the community that they can produce a video in their native language for a beloved module that get's into drupal's help system if it met some sort of criteria? You already trying this with the user guide right now. But i'am not sure if you've tried to find some people to internationalize the drupalize.me videos for the user guide as well.

The tour module has a lot of accessibility problems. I've documented these, with some ideas to improve it, in #2961001: Improve accessibility of tour module. A brief summary:

• Currently, tours are only helping sighted pointer users.
• I have some ideas about how they can be improved for sighted keyboard-only users.
• Screen readers are more challenging to address with tour. The tours assume you start from the beginning, and point out stuff along the way. The tour tips are spatially associated with the thing that they describe, but this doesn't help a blind user.
• I describe an idea for "hop-on/hop-off" tours. Basically, tour balloons could be invoked from an inline help icon-button near the thing (e.g. fieldset...) they describe. This would make the tour tips more discoverable for everyone.