Adopt CommonMark spec for Markdown files

I am sorry for my stupid question; what is the current status on the issue? Where can we help you regarding this issue? Any changes on remaining tasks or still the same?

Is this the proper issue queue to involve the TWG?

As I read on https://www.drupal.org/governance/technical-working-group
There should be an issue on their issue tracker to contact them. Do they also check this queue regularly?

I ended up here through the rabbit hole of having the README.md on a repo as the project description.

+1 to the need for this. Clarification per the request of #7 would be helpful for moving forward with #3012906: Add Markdown to list of module documentation file types, as I have encountered a lot of issues of the permutation of #3082868: Change README.txt to README.md.

Having this resolved would allow consensus and policy to be built to cascade down to revise, standardize the onslaught of contrib README.MD in-contrib documentation.

Running across this issue while looking at coding standard to see if we had any for Markdown. This appears to have stalled during the period where there was no active Coding Standards committee. The newly reformed committee is doing a good job of working through active issues now.

I've updated the issue to use the new standard template to re-start this through the process.

Under the new rules this will need 3 "supporters" to put their name on the change. Setting to needs-work based on this requirement.

I will note I have a few concerns.

1. CommonMark's specifications are confusing to read and would likely not be sufficient for average users requiring more documentation on D.O. to be written.
2. The standard makes no specific requirement in many cases, allowing a range of options.

Here are some questions I was asking when reviewing CommonMark that I still do not have an 'always correct' answer for:

• How many spaces should I use to indent an order list?
• Should I left align or right align the ordered list?
• How man spaces should I indent a sublist?

The specification essentially gave me a range for all of the above leaving it to me to decide what option I want to use. This isn't bad from a parser specification standpoint, however from a coding standard it allows for deviations even within the same project as each file may use different specifications.

To me CommonMark appears to be more of a parsing engine specification rather than a Coding standard. It obviously has its place to define the basis of what is acceptable within a limit, however because of its wide leeway it also doesn't necessarily provide a consistent policy across the ecosystem.

To make it more complicated, I was researching this as part of using mkdocs, which uses python-markdown, which advises that the original Markdown spec specifies 4 spaces indent https://python-markdown.github.io/#differences while CommonMark itself will accept 2-4 spaces.

Perhaps this wide specification is sufficient as a limit, and maybe even necessary given the wide range of Parsers available to render Markdown that we can't define a specific limit for some of these.

On a related note, it appears GitLab is using a CommonMark parser: https://about.gitlab.com/blog/2019/06/13/how-we-migrated-our-markdown-pr...

Thanks for reviving this issue, the README.md template is mostly aligned with GitLab Flavored Markdown (GLFM), and the consensus reached in the Discussion section.

I agree with @cmlara in #10 and @ressa in #11. GitLab Flavored Markdown (GLFM) represents a decent and for practical purposes complete implementation of CommonMark. As GitLab users the Drupal community is best served by converging on GitLab Flavored Markdown, instead of trying to match the somewhat opaque CommonMark specification.

I will also note that an effort to document contrib projects in situ has been started: https://drupal.slack.com/archives/CGKLP028K/p1703059851634719
And there is some other related internal discussion as well.
All this to say that converging on a known MarkDown specification is going to be vital.

Thanks for sharing the link @ChristophWeber, but not everyone is on Slack, and it requires log in to read the content ... Would it be possible to copy-paste the relevant bits here? Or is there a page or issue on drupal.org?

https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-dr... is the relevant section of user guides on D.O.

Some of us are using this to move our docs into the repo (as oppose to the Documentation section of D.O.) for easier management of changes (change a feature you change the docs in the same commit) and wider distribution (the documentation ships with code making it available offline).

As noted in #10 mkdocs requires complying with Python Markdown specifications that are stricter than CommonMark. Mkdocs isn’t the only system (any static site generator can be used) however it is the one that is currently a part of the template. I’ve seen other non Drupal projects use mkdocs.

The gitlab_templates project is just started using this to document itself (WIP site) https://project.pages.drupalcode.org/gitlab_templates/.

#13 @ressa, let me Slack Dump it for you :)

> fjgarlin [U6K08UK96] @ 20/12/2023 08:10:51 Z:
New feature for contrib: *GitLab pages support*

You can read about it here: <https://www.drupal.org/docs/develop/git/using-gitlab-to-contribute-to-drupal/gitlab-ci#s-publishing-a-documentation-site>
You just need to place a couple of files in the repo and you’ll get a documentation page for your module!!

In case you’re interested in the issue where this was added have a look <https://www.drupal.org/project/gitlab_templates/issues/3384688#comment-15367208|here>. Big thanks to <@U1Z8HKMAR> for it.

> saschaeggi [U3A2ZDSE4] @ 20/12/2023 08:32:44 Z:
<@U6K08UK96> awesome! Thank you – now just Issues and we’re good to go basically :stuck_out_tongue:

> fjgarlin [U6K08UK96] @ 20/12/2023 08:33:27 Z:
I’m actually writing documentation about them this week :slightly_smiling_face: 2024 is going to be exciting

> saschaeggi [U3A2ZDSE4] @ 20/12/2023 08:34:06 Z:
<https://media2.giphy.com/media/yBwgX64KAPrHW2ltZ2/giphy-downsized.gif?cid=6104955e3gwc9pfpqq9hryxgiqabcjuuko0ohhwy3v6ox392&ep=v1_gifs_translate&rid=giphy-downsized.gif&ct=g|https://media2.giphy.com/media/yBwgX64KAPrHW2ltZ2/giphy-downsized.gif?cid=6104955e3gw[…]wy3v6ox392&ep=v1_gifs_translate&rid=giphy-downsized.gif&ct=g>

> wimleers (he/him) [U5HCX3HB5] @ 20/12/2023 11:21:05 Z:
DWIGHT!

> Gábor Hojtsy (he/him) [U1A7P6GTZ] @ 21/12/2023 08:48:53 Z:
Is the goal that module docs move from <http://drupal.org|drupal.org> handbook to gitlab pages?

> fjgarlin [U6K08UK96] @ 21/12/2023 08:50:51 Z:
right now, I don’t think we thought about that as a goal, but it’ll defo be a nice consequence if people start adopting this format.

it has great flexibility and you’ll have the documentation within the repo itself as well, which is nice too.
…and it’ll take some load off from d.o

> saschaeggi [U3A2ZDSE4] @ 21/12/2023 08:52:50 Z:
come to me
Move all the things :tada: 

> moshe [U1Z8HKMAR] @ 21/12/2023 12:42:31 Z:
>  Is the goal that module docs move from <http://drupal.org|drupal.org> handbook to gitlab pages?
IMO yes. One friction point is that our issue forks do not play super nicely with the WebIDE.

> saschaeggi [U3A2ZDSE4] @ 21/12/2023 15:16:18 Z:
<@U1Z8HKMAR> once migrated, can we get rid of the forks? Or what would keep us doing it?

> moshe [U1Z8HKMAR] @ 21/12/2023 15:21:25 Z:
The DA is planning to keep issue forks indefinately. The rest of the world uses personal forks.

> saschaeggi [U3A2ZDSE4] @ 21/12/2023 15:22:58 Z:
This limits some features, too I believe

> wimleers (he/him) [U5HCX3HB5] @ 22/12/2023 09:36:17 Z:
Personal forks prevent collaboration from others, right? That’s why d.o went with issue forks?

> fjgarlin [U6K08UK96] @ 22/12/2023 09:57:37 Z:
Yeah, i think that’s the main reason. People can ask push access on a shared fork and get instant access, as opposed to waiting for a person to grant it on a personal one. 
I’m sure there is more to it that just that tho. 

cc <@U3U1SD32R> 

> moshe [U1Z8HKMAR] @ 22/12/2023 09:58:59 Z:
The proposal was for personal forks with a bot that would auto-grant access. So there would be collaboration with personal fork as well.

> wimleers (he/him) [U5HCX3HB5] @ 22/12/2023 10:40:35 Z:
I see :thinking_face:
Even better would be no issue forks and just allowing everybody to create branches/MRs on every project. Yes, that requires write access to all projects. But the official branches would only allow pushes from the maintainers. That would remove _all_ of the `git remote` shenanigans :pleading-eyes:  A few months worth of Drupal core `git remote`s yields this ridiculous unsustainable and unmaintainable result:
But I’m sure there were good reasons to _not_ do that :shrug: :neutral_face:
(No I do not use that font size, it was just to fit it all in one screen)

> moshe [U1Z8HKMAR] @ 22/12/2023 10:49:03 Z:
Thats a great model used within most companies but its not commonly used among untrusted collaborators. When you git clone, you are getting all the potentially malicious and pornographic code pushed by others. VS Code and others now have a trust model that allows for this, but I'm still uneasy. In all other ways, your proposal is superior.

> wimleers (he/him) [U5HCX3HB5] @ 22/12/2023 13:13:02 Z:
Well … I’ve yet to see the first case of me getting pornography from a Drupal issue? :sweat_smile: We’d quickly ban those users anyway? :thinking_face:

> moshe [U1Z8HKMAR] @ 22/12/2023 13:14:10 Z:
For sure. But their commits are in the canonical repo permanently.

> wimleers (he/him) [U5HCX3HB5] @ 22/12/2023 14:41:17 Z:
Right. We would need that “MR/branch is obsolete, clean it up now please” logic.

@cmlara:

Some of us are using this to move our docs into the repo (as oppose to the Documentation section of D.O.) for easier management of changes (change a feature you change the docs in the same commit) and wider distribution (the documentation ships with code making it available offline).

This makes sense, and definitely makes it easier to keep code and docs in sync. Though, it could also be a blocker for updating the documentation, since an administrator needs to get involved when adding to, or updating it. But I guess, one platform (D.O docs vs. MkDocs) doesn't exclude the other, and for some projects one of them makes more sense, than the other. For example a niche, highly technical module should probably use MkDocs, whereas a module with lots of GUI configuration, D.O docs could be a better fit, allowing many users to create doc pages freely.

And by the way, about your questions about lists (#10), GitLab Flavored Markdown (GLFM) > Lists defines how to format them. This is also where "Bulleted lists denoted by dashes (-)" and "Ordered lists use 1" in README.md template come from.

Awesome @mxr576, thanks!

It's so sad to me that all these great conversations on Slack are locked in a black box, and will over time disappear ... There are great nuggets of information and insight in there.

Somebody should create a Slack-scraper and paste the Drupal conversations into a "The Drupal Slack Daily" issue :)

Somebody should create a Slack-scraper and paste the Drupal conversations into a "The Drupal Slack Daily" issue :)

You are not the only one with this idea, and probably AI could also help with summarizing these conversations ;)

That sounds wonderful, I hope it will happen! Though I don't think AI is needed here -- the raw messages would work well, both to have them accessible for reading, but also to get them indexed in the search engines, to make them findable.

It is confusing that the current text section shows other issues. Skimming through I don't see that there is consensus on CommonMark. Since I think this needs more discussion I am changing the status to 'active'.

I agree that there's not been consensus yet.

I don't think CommonMark allows for having tables, and I think that is an important extension to have. It is in Github and Gitlab flavored markdown, and given our integration with Gitlab, it might be a good idea to go for that?

I agree, given that drupal.org issues will move to Gitlab, GitLab Flavored Markdown (GLFM) would be a better choice.

Perhaps we need to create a new issue "Adopt GitLab Flavored Markdown (GLFM) as standard for Markdown files"?

Or can we change the title in this issue, if that's where where the consensus seems to be? Then the discussion and arguments will not be spread over two Markdown issues ...

@balu_ertl: What do you think about that?

Is GLFM a coding standard or a parser standard?

GitLab Flavored Markdown (GLFM) would be a better choice

Is GLFM fully compatible with Python Markdown (goes back to the mkdocs discussion before)?

I’ve spent more time dealing with markdown since #10. My experience now is that it may be very hard to specify a coding standard and unlikely that we will find own pre-made.

There is a big difference between a coding standard and a parsing standard.

A coding standard dictates how detail must be laid out even though the language supports a diffrent layout, while a parser standard defines how it must be laid out to parse and any deviation will result in an error state:

To put the above in the form of an example:
PHP defines a parsing standard.
drupal/coder defines a coding standard.

Is it within the Coding Standards charter to define the use of a parser standard?

To answer #23, the Coding Standards committee itself does not have a charter. It is considered part of The Technical Working Group (TWG). The Job of the TWG is given in the TWG Charter, which states, "to ensure that the necessary policies, procedures, and standards exist to address technical concerns affecting the Drupal community". That would cover making a decision about which style of Markdown to use.

And my personal view is that using the GitLab style makes the most sense.

The comments here from 2024 onward accept the fact the community has chosen GitLab and then it follows that Drupal will be using GLFM. And that is what is happening. Various projects, including the Drupal governance and this project are moving documentation to GitLab pages.

Perhaps we need to create a new issue "Adopt GitLab Flavored Markdown (GLFM) as standard for Markdown files"?

That seems unnecessary because we have adopted GitLab. I think the other way, that is, if the community chose a markdown different than the default GitLab one, then it should be specified.

If I am not missing something to discuss, shall we close this at WAD?

If I am not missing something to discuss, shall we close this at WAD?

The original request was to formally adopt a standard. That would mean this would still need a coding standards vote to adopt and update https://project.pages.drupalcode.org/coding_standards/ with new text. That would not be WAD, if nothing is to be formally adopted it would be "won't fix" which would be saying "each project may write markdown files however they want"

Various projects, including the Drupal governance and this project are moving documentation to GitLab pages.

I will note the coding standards and Drupal governance project pages are not using GLFM, since the pages are generated by mkdocs they must align to python markdown, which is is different from and in some cases incompatible with, GLFM. #3521645: Formatting of PUWG charter mkdocs output is wrong is a real world example where parsing differences between GLFM and Python Markdown.

In #10 and #23 i mentioned that
Adopting GLFM is equivalent to saying "The coding standard for PHP files is the PHP engine specification", nothing about unified writing style has been said other than "the parser must not error'.

Here are a few questions that are related to standardized writing of markdown from the first couple pages of the GLFM specifications, each is a decision to make, each choice is valid, and each option can be mixed in the same file (and in some cases even the same line) and still be valid GLFM:

• Underline or Hash style heading indicators
• Max line lengths (do we keep the old PHP 80 char limit or do we embrace unlimited line lengths which better align to markdown usage)
• Double underscores or double asterisk for bold
• Hyphens, asterisks, or underscores for horizontal rules
• Should ordered lists use the 'repeating same digit' method of numbering or should they manually number each item (so that it is readable in plaintext too)

This is just a fraction of the choices that would need to be made to unify the writing of markdown files under a common style.

Edit: several typo’s of GLFM as GFLM

@cmlara, You're right that 'won't fix' is a better choice. That was just a typing mistake on my part. To clarify a few points. per the approved process only issues that are RTBC and met the criteria are formally discussed by the committee. And the coding standard committee itself uses consensus style decision making, not voting.

Since the title here is to adopt a particular standard, the items at the end of the list in #26 should be in a separate issue.

We should al be aware that GLFM does use the CommonMark spec. According to Differences with standard Markdown

GitLab Flavored Markdown consists of the following:

Core Markdown features, based on the CommonMark specification.
Extensions from GitHub Flavored Markdown.
Extensions made specifically for GitLab.

Is it not enough then to simply use GLFM?

Setting this to RTBC so it will be brought up at a committee meeting.

This was brought up in this meeting #3549265: Coding Standards Meeting Wednesday 2025-10-29 0900 UTC. The members agreed to close this as won't fix for the reasons summarized in #27.

Thanks to everyone to help resolve this.