Add Clickable Anchors and Table of Contents to Documentation Pages

BTW those past discussions were spread over multiple channels. Including but not limited to IRC, documentation comments, forum comments, and so on. And spreading years. I would love to go dig each one of those, but that's not realistic with my present workload.

I think drumm's and my reaction comes from the fact that we expected "past discussions" to be a discussion in the drupal.org issue queue - with a conclusion. Anyway, let's focus on the task at :-)

I'll add a 5 option to the summary:

1. Don't add visible anchor links to short pages without a TOC
2. If a page has a TOC, the items in the TOC are visible anchor links.

I wonder if the first point, "Don't add visible anchor links to short pages without a TOC" could be a general rule.

Have to run!

if you want to have visual anchors, I like the way http://guide.bash.academy/inception/ is doing it. otherwise, the making jist the title clickable is the norm.

I'm adding the "Documentation Initiative" tag as this is a related UX improvement related.

I'd like to cast a vote for 2 things:

• Use a clickable # prefix. Laravel documentation uses it, and it works well. Making a prefix allows the hash tag to be consistently placed. If it were a suffix, its position would change based on title length.
• Create a table of contents listing links to anchors

To make maintenance easier, how about adding anchors as well as TOC automatically, based on h2, h3 and h4 headers on a page?

@bertboerland's example of visualization in #7 at http://guide.bash.academy/inception/ is interesting, not only because it uses automatic anchors and TOC, generated from h2, h3 or h4 headers, from this script: https://guide.bash.academy/js/main.js

Scripts disabled

<h2>
What's going on here?  Text, terminals, bash, programs, input, output!
</h2>

Scripts enabled

<h2 id="h2.2">
<a class="bookmark" href="?=What's_going_on_here?_Text,_terminals,_bash,_programs,_input,_output!#h2.2">#</a>
What's going on here?  Text, terminals, bash, programs, input, output!
</h2>

From: http://guide.bash.academy/inception/

Just a comment about automatic TOC and clickable anchors: Great, really great, but numeric anchor IDs aren't stable. We should use anchors IDs based on heading text which are slightly more stable since they only change if someone changes the text. Numeric anchors change every time someone adds or remove a heading (at the same level).

Maybe the golden middle way could be to insert anchors manually for the headings that we know will be linked to from Drupal core itself or other documentation, and use autmatic numeric anchors for the rest? (The script should use any existing anchors, only generate when missing.)

numeric anchor IDs aren't stable. We should use anchors IDs based on heading text which are slightly more stable since they only change if someone changes the text.

I agree, numerical anchors are inferior to the more stable header, as seen at Wikipedia:
https://en.wikipedia.org/wiki/Protein#Cell_signaling_and_ligand_binding

Trying to consolidate this conversation into a list of requirements:

The requirements appear to be:

• Drupal.org documentation should automatically generate anchors for h2, h3, and h4 headings.
• Anchors should use non-numerical names that are generated based on heading text.
• The anchors styled appear as clickable hash tag prefixes for headings.
• A table of contents should be automatically generated displaying all heading anchor links.

Agree or disagree?

Thanks for summarizing @grasmash, that looks very good to me. Perhaps it could be added, that the table of content should be indented, helping visualize the hierarchy?

• Drupal.org documentation should automatically generate anchors for h2, h3, and h4 headings.
• Anchors should use non-numerical names that are generated based on heading text.
• The anchors styled appear as clickable hash tag prefixes for headings.
• A table of contents should be automatically generated displaying all heading anchor links.
• The TOC should be indented to properly represent the hierarchy of headings. E.g., h5 would appear indented under h4.

Nice, thanks!

The summary in comment 14 looks good. (A detail regarding the TOC: I think we should ignore h5 and below.)

When we "all" agree on this, we should move the issue to the drupal.org queue. Hopefully the needed JavaScript can be added quite quickly.

Updated reqs as per #16:

• Drupal.org documentation pages should automatically generate anchors for h2, h3, and h4 headings.
• Anchors should use non-numerical names that are generated based on heading text.
• The anchors will be styled to appear as clickable hash tag prefixes for headings. See Laravel documentation for example.
• A table of contents should be automatically generated displaying all h2, h3, and h4 anchor links.
• The TOC should be indented to properly represent the hierarchy of headings. E.g., h4 would appear indented under h2. See Laravel documentation for example (top TOC).

All respondents in the last month seem to agree on this list, that would seem to constitute a quorum? I'm moving this issue to Drupal.org Enhancement queue for further work.

Note that the implementation method is not part of the requirements. I'd like someone from the Drupal.org team, with knowledge of its architecture, to provide guidance for how we should implement these requirements. E.g., using javascript vs. text formatters, etc. I'd personally lean toward using a text formatter so that the markup is modified server-side, cached, and delivered to the client in its final form.

It looks like this issue started as a proposal to manually and optionally add anchor links as a matter of docs standards. It has since evolved into a feature request to automatically generate anchors. Changing title and description to match new state.

I had a look at the available Drupal 7 TOC modules, and if using a text formatter is preferred, as opposed to using javascript, then the TOC Node module might serve as a starting point.

It does not use the title for anchor, but generates a numerical anchor, and also doesn't insert visible anchors at the destination, so these two features have to be added.

I'd personally lean toward using a text formatter so that the markup is modified server-side, cached, and delivered to the client in its final form.

While not completely disagreeing, I would like to point to the aspect that this TOCs are actually only small footprints on the clients side but a huge footprint in summary on the servers side while pages are in edit wars. Since Drupal.org has already a huge amount of server side processing in its pockets.

@drumm Do you have any pointers for implementation? I'd like to make this issue actionable.

I had a look at the available Drupal 7 TOC modules, and if using a text formatter is preferred, as opposed to using javascript, then the TOC Node module might serve as a starting point.

It does not use the title for anchor, but generates a numerical anchor, and also doesn't insert visible anchors at the destination, so these two features have to be added.

This probably won’t use the text/filter format API, that’s not meant to be context-aware, it doesn’t know what field or content type it is processing.

toc_node looks like it acts on every page load, toc_node_node_post_render(), which isn’t ideal, since that does HTML parsing and manipulation.

I think a field formatter might be best, especially if the results land in the field cache.

An advantage of server-side processing is that it can attempt to keep anchors going to the same place as revisions are made and headings are edited. toc_node looks like it does this; I haven't dug into the algorithm to see how effective it might be.

I've created a fork of the Table of Contents module and posted it in #2983144: Refactor module to provide and use field formatter.

The fork provides all of the TOC functionality required.

Provides a field formatter for text fields which will:

• Add anchors to configured selectors (h2, h3, h4). Anchors are named using text contents of selector.
• Create a block with a table of contents based on those selectors.

The rest of the functionality can be achieved with custom styling.

@drumm can you review the implementation in the fork and assert that it is acceptable for Drupal.org? Also, what is the preferred way to use a patched or forked module in the Drupal.org codebase?

I'll follow up with a patch to the drupalorg project once the implementation is approved.

Need a quick review to determine next steps.

If implementation is approved, this issue will require patches that:

• Add fork of table_of_contents to Drupal.org
• Add simplehtmldom library to Drupal.org sites/all/libraries
• Update the default content for new documentation nodes:
  • The field formatter must use the new Table of Contents format
  • The table of contents block must be added to the right hand sidebar

Instead of the new library, can we use filter.module’s filter_dom_load() and related functions?

I was not aware of filter_dom_load(), I'll look into it.

I think that using DOMDocument, which used by filter_dom_load(), would limit the functionality of the module.

From what I can tell, it's not possible to use CSS selectors like "h2.toc" with DOMDocument. We'd need to limit selector specification to _only_ class or _only_ tag, and the parsing would be much clunkier.

In short, we _could_ use it, but it would require a degradation in functionality and would add more complexity to the module itself.

This issue has been in "needs review" status for 27 days. How can this be moved forward?

Adding patches. Steps to merge:

• Apply attached patches
• Add table_of_contents 7.x-2.x to Drupal.org
• Add simplehtmldom library to Drupal.org sites/all/libraries

@drumm asked for a performance review before merging this.

I confirmed that the DOM was processed twice per page load and not cached.

Looking at table_of_contents’s 8.x branch, I also found a way to use selectors with PHP’s DOM library - symfony/css-selector.

Both of these are addressed by #3072354: Add caching, use PHP’s DOMDocument and symfony/css-selector.

We’ll also need #3072743: Allow configuring block when field is only displayed via Panelizer so the table of contents block pane doesn’t disappear.

This is now deployed!

I’ve updated the 3 example pages linked from the issue summary to remove the bespoke anchors and let the automated table of contents take over:

• https://www.drupal.org/docs/7/update/introduction revision https://www.drupal.org/node/2917897/revisions/view/10980268/11518943
• https://www.drupal.org/docs/develop/standards/coding-standards revision https://www.drupal.org/node/318/revisions/view/11276285/11518940
• https://www.drupal.org/drupalorg/docs/content/documentation revision https://www.drupal.org/node/2778369/revisions/view/11480022/11518955

IDs are automatically assigned based on the heading content, if there is not already an id attribute on the heading tag. When updating pages, be sure to move over any existing IDs.

Followup for a mobile styling oversight: #3072799: Fix styles for documentation heading anchors on smaller-than-desktop screens

Thank you, drumm. I'm so happy to see this live!

Thank you @grasmash @drumm and everybody else involved with getting this done! It looks really great, and will be super useful for getting an overview of documentation pages, as well as allow linking to a certain documentation page section.

@neil Thanks so much for the merge!

At present, it looks like the new capability is not present on the Official Documentation pages, like:
https://www.drupal.org/docs/official_docs/en/_local_development_guide.html

Can it be added there?

I opened a followup for that: #3075522: Add clickable anchors to official documentation pages