Comments containing HTML are broken (only leading tag is stripped)

Unfortunately, the state of comments in HTML is vague:

• The XML specs (and thus XHTML 1.0 by extension) state that a comment must begin in <!-- and end in -->.
• The HTML4 specs say that a comment begins with <!-- and end in --, optional whitespace and >.
• The SGML specs (and HTML4 by extension?) says that a comment begins with <!, ends with > and contains an arbitrary amount of segments which each begin and end in --.

I know that correct application of the SGML rule is one of the checks in the ACID2 browser compliancy test. So it would seem that the HTML specs only serve as simple examples of the SGML comment rules. The phrasing in the HTML specs is a bit vague anyway.

Part of ACID2:

<div class="parser"><!-- ->ERROR<!- --></div></div> <!-- two dashes is what delimits a comment, so the text "->ERROR<!-" earlier on this line is actually part of a comment -->

Which one do we follow? Given that unremoved comments are still subject to the proper filtering later, it does not matter that much. What's important is that we catch the most common usage (which would be the HTML form, IMO).

Ungreedy matching should be avoided though.

For the 'stupid' HTML syntax, we should use /<!--(-[^-]|[^-])*--\s*>/s.
If we want to follow true SGML syntax, it needs to be /<!(--(-[^-]|[^-])*)--\s*)*>/s.

The code comment is misleading though, about "potentially nasty contents". HTML comments do not pose a security risk, for their 'potentially nasty' content is removed with or without this patch anyway. It is merely ugly and undesirable to print out any part of comments on screen.

Pushing this up!

@Steven: I think we should follow XHTML specification.

This patch needs only opinionated opinions by someone of the core team.
I'd go for the XHTML one since it's the only one that I ever heard about.
I guess it's possible to setup an ugly regex that handles all three standards.
Cheers,
flevour

rerolled for HEAD

The ungreedy matching in the patch's regexp will cause a lot of unnecessary backtracking.

Is this an issue in the current 6.X Drupal?