Allow markdown on drupal.org comments (with a non-default markdown-tuned input format)

rfay++ - I'd also love to see this.

+1 from me as well.

Technically I suggest we introduce it as a new format which wouldn't be the default. That would be the least disruptive and guarantee everything continues working. In the long run we could consider making it the default depending on feedback to it.

In the worst case scenario (that the filter needs to be removed) the content still looks decent anyway.

The Markdown module has a history of strong maintainers that I think we can trust ;)

I *hate* markdown, personally, because:

1. It tends to completely butcher things like nested lists and sticking headings where you don't want them, etc. I struggle with this on g.d.o a fair amount.
2. Therefore we'd probably need to do what g.d.o does and provide both a Markup version and "what I actually freaking typed" version of all input formats. That'll be a slightly worse UX experience on d.o because we also have "Documentation" in addition to Filtered/Full HTML.
3. This would also make editing handbook pages and issue summaries an inconsistent experience; sometimes lists will be in <li> tags; other times they'll be in * format.
4. That same inconsistency could well cause issues with tools that want to suck out information from the handbook and turn them into PDFs or hook_help() documentation, for example. I'm pretty sure something like this is on the docs team's roadmap somewhere.

So I would vastly prefer to go the route of getting BUEditor on d.o and stick with HTML everywhere. But, I won't stand in the way if there's a clear consensus that we should do this from others, despite the problems listed above.

I <3 markdown.

I wrote a markdown live preview module for Drupal 7, but I could backport it to Drupal 6 if there's interest: http://drupal.org/project/showdown (Also, that code is sort of ugly. But it works.)

Something like how the github editor works would be interesting:

@webchick, perhaps the live preview would help with your markdown troubles? Here's a demo using the same JS that I used in my module: http://github.github.com/github-flavored-markdown/preview.html

fwiw, nested lists are just this:

* List item 1
* list item 2
* list item 3
    * sublist item 4
    * sublist item 5
* list item 4

Nested items have to be indented by four spaces.

Also, +1.

cweagans: I know how to write markdown, thanks. :) The problem on g.d.o comes in that I'm trying to write HTML but markdown seeps its way into what I'm writing, because it's smooshed in with the default format. Making it optional would help on that front, but note that 3 and 4 in comment #3 does affect everyone, even those who don't want to use it. g.d.o has wiki pages, but they are not a fundamental way of working within the community. Book pages and issue summaries are.

In response to webchick's points comment #3:

1. and 2. seem like problem and solution in a bundle.
3. That is a possible problem. I think we could solve that by simply form_altering the markdown filter out of the places where it would cause that problem (books, issues). The place I really want markdown is for comments and this problem isn't present for comments.
4. I don't fully get this problem because anything that exports the handbook should run it through check_markup before doing anything to be sure they get it in right format. Doing so would put everything (markdown'd or otherwise) into everyone's favorite consistent output of HTML. If that's not good enough, as I said about #3, there are solutions.

Title fix.

In the BUEditor thread killes said.

If I don't hear serious complaints, we'll simply try this live since we can actualy just revert it.

Unlike markdown, I think.

It's actually not super painful to remove markdown from a site as long as you don't pair it with (and want to retain the feature of) other complex filters. It would be easy to create a drush script that finds markdown enabled content, runs check_markup on it, sets the body field to the results, sets the input format to a similar one, and saves a new revision.

+1 for markdown.

@webchick: in #3 you state that you hate markdown. In #6, you seem to say that markdown in itself is not the problem, but the combination of markdown with other input filters. What if it would be just markdown, like greggles suggests in #8? :)

+1 for Markdown.

Bumping this to see if finally something will be done about it.

+1 for Markdown, but webchick's concerns are quite valid. I've gone both ways on personal sites. Having a live-updating preview below the textarea (a-la Stack Exchange) helps me a bit... but that adds a bit more space on comment forms, node forms, etc. :(

@geerlingguy, can you clarify which of webhick's concerns seem valid? I think I addressed them in #7/#8.

The main thing, really, is this issue:

This would also make editing handbook pages and issue summaries an inconsistent experience; sometimes lists will be in <li> tags; other times they'll be in * format.

Right now, I simply know to use HTML everywhere on Drupal.org—with or without the simple assistance of BUEditor—but with a markdown filter some places, and not others, I'll likely run into many situations where I have to go back and edit things again because I forgot md wasn't available for a certain type of content.

On my own sites, I usually make this difference obvious by using a Wysiwyg on fields that require straight HTML (disabled by default, but with the toggle link), and markdown for other fields. It's disappointing, and annoying, when I enter something in markdown and have to manually switch it to HTML (even with a keyboard shortcut or script, it's still an annoyance).

Wouldn't be the end of the world, of course.

What's the status of this issue, now that we have the BUEditor? It looks abandoned and should probably be set to "works as designed".

I still love markdown though, especially the combination rich editor and markdown with preview on Stack Overflow sites.

I don't see this as conflicting with bueditor:
* markdown would be a different, non-default input format
* if someone uses bueditor buttons after selecting markdown then the html would pass through just fine

Just checking, since the original issue mentions that this could "be a stepping stone" between plain text and implementing BUE.

This issue has been quiet for almost 2 years now. Do we want to close it?

Should more worth while to implement now after the D7 upgrade.

I would still love it as an option, but BUEditor seems the best compromise for not-markdown-fanatic-people who just want to make lists and bold text more quickly (probably the majority of d.o users), so no skin off my back if this issue is closed. On the flip side, it seems more and more sites (e.g. GitHub, Stack Exchange) are bringing markdown into the fore... so maybe it's time to consider it again.

I would really love markdown. Please don't close this :)

Supporting not closing this issue.

Except for comments, all of Drupal.org's content is likely to have multiple editors. There will be some amount of people bumping into Markdown, when they prefer HTML, and the other way around. That won't go away, unless we really decide that Markdown should be used by everyone; and the old HTML content gets stale or manually converted.

In the long run, I'd like to be able to switch on the WYSIWYG that comes with Drupal 8, after we upgrade to it. We should prefer to do things as Drupal does them, when they make sense for the site. I'd rather offer a (good) WYSIWYG than require people to learn a new markup language. Since a WYSIWYG edits HTML, the content isn't forked into two formats; we have flexibility for who has plain HTML or WYSIWYG as it is rolled out.

That's actually a nugget of a really decent solution. What if we allowed markdown for comments and not nodes?

I don't know of a core/contrib that offers that feature, but we could form_alter it very quickly.

I would be 90% satisfied with that solution (since I make 9 times as many comments as nodes).

I found this issue after apparently hallucinating the option of the markdown filter on a d.o site... and then being disappointed when i went to use it. I was excited, and i think many would be, because Markdown is very common for README files (and is indeed recommended here, and copying and pasting between documentation files included in projects and release notes and d.o project documentation is convenient and, in my opinion, should be encouraged.

This would require Markdown on nodes, not just comments, and the use case isn't addressed by good WYSIWYG alone.

Of course, i would probably be even happier with being able to link to the version-controlled README.md file in a place which converts its markdown automatically. I suppose i can just dual-post al modules to d.o and github, and link to the github documentation... i'm glad we had this talk.

I would like to have this, too.
The most important being bullet lists and numbered lists. I hate to write those in html. BUE helps, but only for initial writeup, not for later editing.

But I can see the concerns raised by webchick and others about inconsistent experience for wiki-like content.
I'd say we should not rush into it, and consider the alternatives - e.g. based on wysiwyg.

IF we go down this path, some things to decide:
- Is it compatible with PHP code filter? (I think it is, but remember some quirks. The order of filters is important)
- Should we use github-flavored markdown? For my taste, absolutely yes! I much prefer how github handles line breaks. I much prefer authoring on github (github-flavored) than e.g. on stackoverflow (regular markdown).

Something really awesome would be seamless client-side conversion of input formats :) but that's a pipe dream..

Actually, I now believe we should just won't fix this issue, and instead turn on CKEditor like D8 will have out of the box, and be done.

/me RUNS ;)

In seriousness, hand-crafting HTML tags around Google Docs headings/bullets is about the most painfully time consuming part of my week, each and every week. And markdown would just mean I'd need to look up special syntax now to do the same hand-crafting.

Hi,

And markdown would just mean I'd need to look up special syntax now to do the same hand-crafting.

No :)

Markdown would be the non-default option.

Even if it were the default option, Markdown-enabled filters don't require you to use markdown. You can still just paste html.

I agree. A well-working WYSIWYG will be better than remembering which flavor of Markdown works.

(I agree with webchick.)

Even if it were the default option, Markdown-enabled filters don't require you to use markdown. You can still just paste html.

Unless I'm editing something that someone else wrote in Markdown.

instead turn on CKEditor like D8 will have out of the box, and be done.

Actually, yes. And we can begin by getting that into dreditor. CKEditor Widgets for inserting URLS, that automatically map to issue references (like #1224506: Allow markdown on drupal.org comments (with a non-default markdown-tuned input format)) but show up like the rendered end result that d.o spits out, for referencing comments nicely, for referencing issue participants nicely, etc.. And of course, we get the regular HTML stuff (lists, including nested ones) for free, with sane keyboard support.

And… gasp… finally be able to select the part of a comment you're replying to, and have just that part, and including its formatting be put into a <blockquote> automatically.

Markdown is not a complete solution here. It doesn't help with tables. It doesn't help with issue references. It doesn't help with quoting parts of previous comments. Yes, it makes manually writing HTML comments easier. But what if we can just remove that problem altogether?

Yes, yes, WYSIWYGs create crappy markup, yadda yadda yadda, except… not anymore with CKEditor >=4.1 and the way Drupal 8 uses it. Drupal.org could also use it that way: http://wimleers.com/article/drupal-8-structured-content-authoring-experi....

OK then.

Is anyone opposed to a markdown-optimized, non-default text format that is *only enabled for comments*?

It still means that we "pollute" the database, and also people's mental model of drupal.org, with a separate input format.

I think I would like to get some experience with the wysiwyg, before pushing for markdown again.

I checked with Greenpeace and markdown is not a hazardous material. We are safe to store it in any database (whether sql or nosql - they were clear on that point).

In the past 2 years the commenting box on d.o has grown dramatically. Please put a critical eye towards this image:

I really struggle to see how a single additional item hidden inside of a dropdown will dramatically increase the cognitive load on content contributors.

As @mlncn suggested in #27 I also like to see the markdown filter in project nodes. For me this can help to write documentation for readme files and project pages together.

I really understand the problems with a mix of markdown and pure HTML in collaborative nodes like documentation and issues. Also project files can also be edited by all enabled maintainers but there I see more pros than cons. On other nodes like forum posts it would be an helpful option. I like this on g.d.o-posts when I organize my local user group.

I love Markdown and would like to see this happen. I will add more when I've had a chance to read all the way through the thread, but really, I'm not that picky. Anything but manually writing HTML in comments.

Adding related issue. Personally, I think that issue should really be discussed first, but I'll leave setting this issue as postponed to others.

Just an FYI: That issue has inadvertently lead to my developing https://www.drupal.org/project/commonmark.
An example of this in action: http://drupal-bootstrap.org/filter/tips/commonmark

There's also this related issue. Clearly we need some sort of consensus on how we treat/implement "Markdown" on *.d.o sites.

markcarver wrote in #41:

There's also this related issue. Clearly we need some sort of consensus on how we treat/implement "Markdown" on *.d.o sites.

IMHO, there already exists a concensus. The Markdown filter module is the de-facto implementation of MarkDown and is already used on some Drupal.org sites such as groups.drupal.org as well as for README.md-files in the repo (these are rendered by the Advanced Help module and we really want to be able to use these existing README.md-files on project pages).

To me it looks like the related issue injected in #40 (#2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN)) is oblivious to this existing practice and concensus. That's OK if the proposed syntax (CommonMarc) is restricted to the use stated: "Component documentation for core/lib" - but it is bound to be disruptive if CommonMarc extended to other uses such as comments and project pages.

I, too, have learned to like MD a lot. This is, for one, thanks to its (as has already been pointed out) quite acceptable degradation, and also for its IMO maximal typing convenience (I do not even need to turn on Javascript and still do not have to manually code HTML). For another, I cannot agree with BUeditor being so much of the very best at all, not only because JS, but also since it has some weird issues under certain browser conditions (no need to go in depth here though).
More importantly, I do manage some projects in multiple environments (primarily Gitlab and Jira) which both provide quite fair MD support, and I find myself too often re-coding descriptions, feature lists and whatsoever just for posting them on d.o.

This brings me to the question (pardon me if I overlook something), what could possibly be the downside of allowing it as another option (no need to make it a default IMO)? I would also like to ask this to those who also do their primary maintenance on e.g. Github.

Thanks for some enlightenment!

IMHO for developers coming from other parts of the web, d.org feels extremely predated - in part due to lack of MD support in comments. A working WYSIWYG editor is OK, but the current implementation is pure hell. How are we as a community supposed to attract new developers if we can't stay up to date in our own backyard? Whether MD is a good format or not really doesn't matter that much in my opinion - the fact that it's standard everywhere else is more important, because that means people (i.e. developers) are used to it.

I didn't see this until just now, but re: #43

Just because something is first, doesn't make it the best or even the most appropriate solution. I'm very well aware of that existing module. However, that module was created/exists solely to fill a gap for the lack of any official stance by the community, nothing else.

It uses a very outdated and severely non-performant PHP implementation (the original PHP port). It also lacks features and extensibility. Please stop perpetuating that I'm some noob and didn't do my research, it's insulting.

In #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN) (if you read the whole thing), I was actually originally strongly in favor of using the Parsedown [Extra] PHP library.

It was actually @Crell who mentioned using CommonMark if we're going to use anything at all. I wasn't sold right away (as evidence by my research into the various PHP libraries).

However, I decided that @Crell was correct. If d.o/community standards is going to "officially support" markdown, in any capacity, then it should adopt something that actually has specs and is consistent (which is what CommonMark is all about).

See, I can listen to my fellow community members. I got onboard with a better solution. I didn't belittle someone because they decided to make a module out of it and push forward for a newer/better standardization of something that has never been "officially supported/adopted".

I'm just trying to help consolidate and standardize how we (as a community) treat markdown (in core, code, d.o... everywhere). So please, @gisle, get off my back about this and stop trying to make me out to be the "bad guy".

@markcarver wrote:

So please, @gisle, get off my back about this and stop trying to make me out to be the "bad guy".

For the record: I am not.

However, I do disagree with you about the universal need to adopt CommonMark (or anything else) as "how we (as a community) treat markdown". I hope it is possible to disagree with you about this without being accused of disparaging you?

For the record: I am not.

https://www.drupal.org/node/1674976#comment-10939291
https://www.drupal.org/node/1674976#comment-10943733
https://www.drupal.org/node/2191525#comment-10944579
https://www.drupal.org/node/2191525#comment-10944579
https://www.drupal.org/node/2191525#comment-10944935
https://www.drupal.org/node/2191525#comment-10948959
https://www.drupal.org/node/2191525#comment-10958079

And even @anavarre could see how I was being treated: https://www.drupal.org/node/2191525#comment-10944777

Also, @gisle, 'this person' has a first name. His first name is Mark.

You have been extremely antagonistic towards me and all these regarding "markdown", simply because I postponed an issue you "thought was RTBC". That's what all this has been about. I postponed it because I saw the need for doing all of this the "right way", not just implement an existing module that was outdated and severely non-performant. Oh and BTW, 10k installs isn't "mass adoption" when it has simply been the only option available (until now).

I hope it is possible to disagree with you about this without being accused of disparaging you?

You can certainly disagree. That isn't the issue. It's how you have gone out of your way to make this about "me pushing my agenda [CommonMark]" when it's been clear (in other issues) I'm not the only one who thinks there should be "universal" something in place to consolidate and standardize how we [officially] treat "markdown" in the community, its sites and in code.

As I stated above, I originally wanted to use a different library other than CommonMark. Just because I decided to "create a module" (more as a POC than anything) is of no real consequence. The module didn't exist. I created one. Simple as that.

And for the record, I'm not accusing you, I'm tell you that you have (evidence from the above links) and that you need to stop please. This doesn't move the issue(s) along.

I've experienced small formatting problems on g.d.o, e.g. when greggles and I drafted https://groups.drupal.org/node/516645, but also mostly experienced it working fine in that case. Not sure why this issue is a big deal; how often do you edit someone else's comments so what does it matter if they use a different input filter? The output is still output. Why do we need to control how other people write? It wouldn't be the default option. We would still use the plain old Filtered HTML by default.

With github and other things using Markdown by default for READMEs etc., I think it's becoming more and more an expected input format that people are fluent in.

The point of Markdown is that it is readable both in its raw and rendered format. What I'm interested in is using Markdown for collaboratively drafting release notes for core, as we spend a lot of time hand-writing HTML in google docs for minors that duplicate a special-flower formatted almost-Markdown text file. Not only do we spend hours and hours writing a redundant Google doc of HTML, but every single time we have to spend like an hour fixing the gosh darn HTML when we do a release, and we spend a ton of time duplicating content of the CHANGELOG. The part of the CHANGELOG that works well is that it's a plain old patch that we can work on in normal Drupal issue process. The dumb thing is that we then take that content and mark it up by hand, differently, for the release notes. I've written scripts to generate HTML for it. But that is all a dumb waste of release manager time. It would be way more straightforward to just let the Internet do its thing and use a format that is legible in both raw text and browser output.

I've also experienced the horrible pain of trying to maintain HTML in a git repository, since we wrote the core governance doc in HTML and I am like the only person who patches it. I patch an unruly text file that's hard to read or edit. Then once the changes are committed I have to copy the changes over to a handbook page to make it readable. That's also a dumb waste of my time.

So, @drumm, how much work would it be to just add this as a non-default option?

It doesn't need to work with the grrrr WYSIWYG (that eats frontpage post HTML and causes Gábor and I massive headaches and wastes our time when publishing information about minors); every Drupal site I've ever built has formats that support the WYSIWYG and others that just don't use it. It doesn't even really need to work with BUeditor or the new code filter. Markdown module has like 10K usages and is actively maintained. Tuning it slightly to work with other filters for an additional text format is a small and solved problem. If people experience formatting problems with it, that's their problem and they can edit their comments or use the default Filtered HTML.

Also, reading some of the other comments on this issue. I think it's a mistake to conflate node and content with coding standards. We don't have coding standards for how people write release notes or issue comments.

We don't have coding standards for how people write release notes or issue comments.

Well, that isn't entirely true: https://www.drupal.org/issue-summaries

Regardless, my whole concern has been primarily around the syntax of markdown (or rather the lack there of) and what "features" will be supported, e.g. GitHub Flavored Markdown (GFM) (which, coincidentally, is now just a strict superset of CommonMark).

It's along the same lines of the "short vs. long array syntax" argument for PHP. This is a syntax specification, which is part of its "coding standards".

So yes... if we officially adopt markdown, anywhere, it should be consistent; regardless if it's used on nodes or in repos.

And that should happen first before any implementation is done.

---

Also, as I have stated many times, the libraries that the Markdown module uses are antiquated, slow, not easily extensible and do not always parse the "syntax" properly.

There have been several "syntaxes" over the years, but CommonMark has attempted to standardize it.

If the Markdown module were to adopt/replace current library implementations with CommonMark, then this really wouldn't be an issue.

The only reason I created the separate CommonMark module was as a Proof of Concept more than anything. I would be fine if it were merged into the current Markdown module or perhaps even replace it. I don't want or need to maintain another module.

My argument has only ever been about what we will officially adopt moving forward so we're all in sync with what syntax will be expected/supported for everything (comments, issues, repo, etc.).

I'd rather go with something that has the backing of the entire PHP/dev community rather than just slapping something on d.o because the contrib module in question "has some numbers" (which are inherently skewed because it's been the only option for years).

@markcarver, my post was not directed at you. Sorry for any misunderstanding.

I know and I didn't take it personally.

I think it's a mistake to conflate node and content with coding standards.

I was simply trying to show that, while coding standards doesn't exactly affect node content per se, it would be very odd and confusing to have some syntax that works or is parsed differently on d.o rather than what is done in repos.

This is why I was attempting to reiterate and refocus the issue back to #2191525: [PP-1][policy, no patch] Extend Markdown coding standards to support API module (DOXYGEN).

We need an official community stance towards markdown that provides consistency and stability, not convenience.

The only reason that "everyone just wants to slap on the existing markdown module" is because it's been the only "significant contrib module" around this topic; despite its serious shortcomings and ignorance of the various advancements to the concept/syntax/libraries of "Markdown" in recent years.

I just think it would behove all of us to know the syntax we're actually "supporting", on *.d.o or inside repos, before any actual implementation is done.

I also think that we should support something that is easily extensible for potential future needs (e.g. allowing the API module to parse markdown files for verbiage based topics in our documentation).

I, for one, would support markdown on d.o comments. It's mostly a progressive enhancement on plain HTML and it's what I use everywhere but here (and Jira).

FWIW, both @gisle and I are now maintainers of https://www.drupal.org/project/markdown and will be merging https://www.drupal.org/project/commonmark into it.

Also, the actual adoption of the CommonMark spec was separated out into its own issue in an effort to move it along more quickly.

#2952435: Merge in the CommonMark project is fixed. Time to make this come true soon!

Updating IS. Added 2 modules to the Proposed resolution section

• https://www.drupal.org/project/simplemde
• https://www.drupal.org/project/markdown

What do we have to do to get this moved forward?

Now we have Gitlab, we can use markdown there in comments; it feels both inconsistent and a step backward when writing comments on d.o and having to go back to using manual HTML tags instead of backticks for code blocks, etc.

Since the overall mood towards this suggestion is positive, and the integration with Gitlab only growing, like @longwave asked more than one and a half year ago, how can we move this forward?

With issues moving to GitLab, adding a second type of content for migration would double our work in that area. And the flavor of Markdown that’s achievable with Drupal might have incompatibilities with GitLab’s flavor of markdown. So this will not be enabled for issues comments or summaries.

For the rest of Drupal.org, we’ve been enabling WYSIWYG #2741227: Enable CKEditor for more content types editing. I’m not sure that it makes sense for Markdown and WYSIWYG editing to coexist.