Unprivileged users should only get one text format by default

Not sure I like this.

See also #735616: Do not use the phrase "text format" for end users

1 available text format for anonymous is a good thing.

Removing "filtered HTML" format because there is no wysiwyg editor by default totally make sense.

The latest patch is working for me, I'm just wondering how to make it pass all tests.

The problem in most tests seems to be that we are trying to edit a node which has a format that the current user is not allowed to use so that gives us the disabled body field with the "you are not allowed to change this message".

I guess we can either fix this by giving the users the necessary permission when testing this or creating the node with a format which all can use (change the default filter format to plain text?). The filter tests fail because they test that you have access to that format, so they need to be updated.

hello D8

I objected to the proposed solution since the beginning. Today I know why:

In almost all cases, this UI makes absolutely no sense.

We should simply remove the fallback format from the options, if 1) there is another option available, and 2) the fallback format is not assigned already.

However. Since there can be legitimate use-cases for always exposing the fallback format, we should conditionally execute this fallback-format-removal behavior based on a variable only. The setting could be exposed as (the first) global Filter module setting. Alternatively, it could be an internal configuration value that cannot be changed in the core UI (i.e., leaving that to a contrib module, if necessary).

We should simply remove the fallback format from the options, if 1) there is another option available, and 2) the fallback format is not assigned already.

I absolutely agree with this statement. The appearance of the fallback format in D7 is a major point of confusion for users. If possible, users should *never* be asked for a format. Exceptions may be made for choosing between actual different formats, such as Markdown or BBCode, but choosing between Filtered HTML and Plain Text is a completely unnecessary decision for end-users to understand.

This patch implements the suggestion of only showing the fallback format if it's already in use, or if it's the only format available. It's hidden in all other situations from all user roles. There's a new variable with no UI to change this behavior so that the fallback always shows, which defaults to FALSE.

Doh, I wish Eclipse would add new lines at the end of patches.

I'm relatively sure that this #options processing needs to happen at the very end of filter_process_format(), since various security and access checks are performed at the end of the function already.

The fallback format should only be unset in the options, not in the $formats array.

Also, we should consider to "hide" the option by setting #access to FALSE on it; i.e.:

$element['format']['format'][$fallback_format]['#access'] = (bool) $config->get('show_fallback_format');

Sorry, #14 was a bogus suggestion; I wrongly assumed that #type select #options would already be render elements like the #options of checkboxes and radios. I think there's an issue for that somewhere. Would be nice to get that finally in sync for D8... anyway, off-topic.

Here's what I'd suggest:

1. If there are multiple options, remove access to the fallback format.
2. If only one option remains, turn the format selector into an item.

That said, I think we need to add a settings page for the fallback format settings in D8, since IMHO this involves too much magic now, and the entire fallback format concept is not really explained anywhere (in the UI). Even as a developer, I wouldn't be aware that these hidden settings can be changed.

Thanks for reviewing this @sun.

+    if ($element['#format'] !== $fallback_format && count($formats) > 1) {
+      unset($element['format']['format']['#options'][$fallback_format]);
+      $element['format']['guidelines'][$fallback_format]['#access'] = FALSE;
+    }

This seems weird that the $fallback_format help text is still there even though the option has been removed. Had the suggestion in #14 worked, I could see setting #access on the filter tips as being possibly okay, but now this just seems uneven that the option is removed entirely while the help is just hidden. If a contrib-module wanted to restore the fallback format after it was removed, wouldn't it be easy enough to just populate $element['format']['guidelines'][$fallback_format]? I don't like that we'll be generating a part of the form that won't be displayed on 99% of Drupal sites.

+  // If only one format can be chosen, turn the selector into an item.
+  if (count($element['format']['format']['#options']) == 1) {
+    $element['format']['format']['#type'] = 'item';
+    $element['format']['format']['#markup'] = check_plain(current($element['format']['format']['#options']));
+  }

This seems like unnecessary information to present to a user. I'd argue that an end-user doesn't need to know the internal name of a text format (or what a text format is) if they're unable to choose a different one. I think it would be preferable to show nothing at all and spare the end-user the task of understanding a concept they don't have any control over. They *do* need to know what tags are allowed though, which is still displayed in the filter tips. For developers and site-builders, they also can deduce which format is active for a user based on these filter tips.

That said, I think we need to add a settings page for the fallback format settings in D8

I don't know about this. The fallback format is indeed an obscure concept that few sites actually employ. If it's not something most sites need to worry about, I don't think we need to populate the UI with additional options for it.

Here's a reroll that should get through most of the tests. Based on my feedback in #17, I don't think there is justification for the changes in #15, so this patch uses the same approach from my initial patch in #12.

+1 for the patch in #18.

I tried to move this forward by fixing the test failures:

• SelectComplexTest failure seems to be a random failure, works fine locally and is a DB test, so it's impossible for this patch to break that.
• JavaScriptTest failure also seems to be a random failure.
• CommentPreviewTest seems a correct failure, but… locally it crashes with "General error in cache_invalidate_tags() (line 80 of core/includes/cache.inc).". If I just retain the testCommentPreview() method and have nothing in there but a "return" statement, even then it crashes. Impossible to debug.
• SearchCommentCountToggleTest fails due to a fatal PHP error locally.

So all I can do, is try to come up with untested modifications to the tests, and have testbot test them. Besides those changes (which turned out to be 3 removals), I did only very minor clean-up to the patch in #18.

Since this patch is a requirement for #1911884: Enable CKEditor in the Standard install profile, tagging so that the CKEditor folks can track this more easily.

#20 looks like it adds a bit more robustness to the tests. Obviously they're passing now. This general idea has support from everyone in the queue so far, so RTBC.

Anyone interested in the default text formats available in Drupal should take a look at #1911884: Enable CKEditor in the Standard install profile also, which splits the "Filtered HTML" format into two (one for anonymous and one for authenticated).

I also think this is going to be hard to explain. Administrators will be confused if they see this format appearing and disappearing somewhat randomly as an option on textareas (the fact that you can edit content and switch it from "Plain text" to something else but then not be able to switch it back again, for example).

I don't think it's likely that administrators will regularly run into plain text content after the option to use plain text has disappeared throughout the site. It could cause strange behavior for sites that don't grant anonymous users access to other formats, but I think that's uncommon considering how restrictive plain text is (and the fact that you can't rename it to something else).

I think this could be explained easily enough in the admin UI by changing "All roles may use this format" to "This format is shown when no other formats are available". And that's pretty much it isn't it?

For the purposes of solving this issue alone, it would obviously be simpler just to have the Standard install profile configure the fallback format to behave like Filtered HTML (and not use plain text at all), but I see how that conflicts with the goals of #1911884: Enable CKEditor in the Standard install profile to split the format in two.

If we split the text format in two in that issue and make it so that the fallback format is reconfigured to be the less-privileged format, we'll solve the problem exposing the concept of text formats to anonymous users, but all authenticated users would continue always having at least two text formats. Since you can't remove access to the fallback format, authenticated users would be stuck forever having a fallback format and showing the text format dropdown everywhere. If we get into trying to hide the fallback from just authenticated users, the problem has just been shifted to a different audience.

EDIT: Fixed typo.

#24++

We can still do #735616: Do not use the phrase "text format" for end users, but the most pressing part of the problem is addressed in this issue, for the majority of sites.

The main issue we're trying to solve is depicted and outlined in #9. I think the patch does that.

The only part that looks problematic to me is that this baked-in manipulation is irreversible. I.e., a module cannot easily undo this for a particular form. It would have to re-implement most of filter_process_format() to get it right.

Minor: One of the last test file hunks uses a format ID with _format suffix, which looks like a copy/paste mistake.

@David_Rothstein: This may be putting you on the spot, but do you have any ideas for showing only one format for authenticated users? Most of my sites these days don't have authenticated users other than editors/admins, but that's probably not the case for many, many Drupal sites out there. I notice d.o even doesn't display the "Plain text" format, maybe we're using Better Formats? I would prefer not to pass such a trivial problem as "show fewer options" to contrib for this particular problem space.

I think you may also be overstating the desire for plain text. Browsers are smart enough to figure out problematic, unencoded < and > brackets. i.e:

The equation to solve is: x + 2 < 3
The equation to solve is: x+2<3
The equation to solve is: x + 2 < 3 > y + 1
The equation to solve is: x+2<3>y+1

Correct HTML, no. Problem for browsers? Also no.

@David_Rothstein: In #17 I voiced the opinion:

I don't know about this. The fallback format is indeed an obscure concept that few sites actually employ. If it's not something most sites need to worry about, I don't think we need to populate the UI with additional options for it.

Although I still think this is a silly option to add, would it make the situation acceptable if we modified our conditional form_alter()ing of the fallback format (admin/config/content/formats/plain_text) to add checkbox for "Always show fallback format"? We already are form_alter()ing specifically on the fallback format to disable the role checkboxes. It would trivial to publicly expose this option to administrators. It would also give us an opportunity to explain the behavior of the fallback format.

Sorry, but the given example looks invalid to me. I can < type any amount of > HTML alike < chars here. >

As you can see, my raw user input appears like it should. HTML filter + filter_xss() are pretty smart + bad-ass string parsing constructs. Actually, they're so complex that almost no one touched them since their inception.

As I've clarified in #1911884-49: Enable CKEditor in the Standard install profile already, the fallback format was never meant to be exposed in the way it currently is. There seems to be a pretty big misunderstanding with regard to its originally intended architecture vs. the current, unfortunate impact on all sites.

re: #29:
drupal.org does not show a Plain text format, because it runs on D6. The Plain text / fallback format was introduced with D7.

Actually, given the further analysis of the situation in the other issue, my concern in #26 is a bit bogus. Instead of doctoring around the edge-case dynamics of the fallback format, we should move forward here and also remove it entirely from the administrative UI (in a separate issue).

(Gmail being one such example, but I'm pretty sure I've seen that in a lot of other applications too).

Gmail (and other mail clients) aren't quite the same situation. In the world of e-mail, there's a significant difference between HTML e-mail and text e-mail. They're actually two different formats. In the web-publishing world, there's only one format (HTML), and "plain text" doesn't really exist. Users aren't expecting to be able to post in plain text, but they do expect to be able to manually enter HTML instead of using a WYSIWYG (such as in WordPress). In our immediate situation, that's already solved by having CKEditor's "Source code" button enabled, though we may elaborate on that to actually disable the editor entirely.

Anyway, I'm fine with adding the option if it's desired (and we could use it in the upgrade path). I don't have a strong opinion on it, but my preference would be towards both not having the option and changing the behavior of the fallback during the D7->D8 upgrade by not adjusting the setting (whether its represented in the UI or not).

Weird, no idea how I removed the tags.

(#34: d.o sometimes does that. You can't really do anything about that.)

I think this could be explained easily enough in the admin UI by changing "All roles may use this format" to "This format is shown when no other formats are available".

Because this seems like significant improvement towards describing the new behavior, I rerolled the patch to show this help text if the "always_show_fallback_formatl" option is set to FALSE (the default).

The help text on the format page itself (beneath the disabled radio buttons) is still accurate in both situations, though honestly not particularly helpful in either situation.

As mentioned previously, we should rip the entire fallback format from the administration UI instead. The concept was never intended to be exposed in that way.

Therefore, the tweak in #36 is nice, but those entire lines of code should be sent to /dev/null instead.

#1932544: Remove all traces of fallback format concept from the (admin) UI

So, just a question, having reviewed both this and #1911884: Enable CKEditor in the Standard install profile... Not marking down from RTBC because I might be missing a clue-bat here (sorry, I'm getting back to the queues after ~2 weeks away).

Why are we choosing the course of action here of splitting Filtered HTML into two ("Basic" and "Filtered", where it is not immediately clear what the difference is) and then hiding the "Plain Text" fallback in various magical ways, although not in the admin panel where formats themselves are configured ("Wait, I see it, right there, WHY WON'T IT SHOW UP?!?")... and then only putting a WYSIWYG on one of the * HTML formats, but not the other, requiring what is probably bound to be the *least* technical audience of a site (basic auth/anon users) to continue to hand-type HTML like it's 1993...

...rather than just doing what's in the OP, which is:

"I think we should fix this by removing permissions for filtered HTML from at least anonymous users, and perhaps authenticated too. This would leave them with plain text (which is not totally plain, since it does contain e.g. the line break and URL filters)."

Then:

a) Anon / Auth only see one text format, therefore don't get the selector (achievement for this issue: unlocked!)
b) Admins don't have two similarly-named formats to be confused by when assigning permissions
c) We can ship D8 with a default WYSIWYG editor on the default text format with no special tricks.

Everyone wins..?

1) From my perspective, this patch is just a stop-gap fix and a precursor to correct a larger architectural misconception.

2) To fix the larger thing, see #1932544: Remove all traces of fallback format concept from the (admin) UI

3) You do not want to force-load 0.2+ MegaBytes of traffic on every single innocent visitor of your site. Not by default, not if we take mobile serious, not in any recommended way, no thanks.

4) The two text formats actually vary in some minor details. That's the part in which I agree with you and inherently still disagree with all others: There's not really a need for these differences. We should look into allowing both user groups to use the same format, but anonymous without an editor.

Hm. Did what I say come through? I don't understand #40.3.

If we followed the OP's recommendation, all innocent visitors would get just access to "Plain text" (which we should rename to "Basic text" probably, since it's not "Plain" at all). They would not get a 0.2+ MB worth of WYSIWYG editor, since they couldn't enter HTML. Links would automatically be linkified thanks to the autourl filter, and <p> tags auto-inserted, but that's it. Out of the box, this audience can't create content on the site at all, only comments, so this shouldn't hurt anyone's feelings.

For the life of me, I don't know why we're discussing to remove all HTML capabilities for comments by anonymous users.

The first and foremost thing that someone who comments on something wants to do is to <blockquote> the snippet of the context/parent post he's commenting on. Secondly:

Bold, italic, bullet and ordered lists, links, anyone?

I can't make sense of the idea of restricting comments on a (HTML) web page to plain text.

In any case, this is getting a little off-topic here and the discussion rather belongs to #1911884: Enable CKEditor in the Standard install profile. This issue is solely about removing the fallback format from the user interface. The fallback format is not supposed to appear at any time, during regular operations and runtime. (The "fallback" in fallback format is completely meaningless currently.)

Whether the text format that anonymous users are allowed to use permits HTML, or not, or Markdown, or BBCode, and whether it has an editor, or not, is completely irrelevant here.

What's relevant here is that the fallback format is unconditionally exposed in all cases, which clutters the UI, doesn't make any technical and architectural sense, and which wasn't intended to cause that kind of havoc when it got introduced.

Every user should get the default format that he's allowed to use. Anonymous users should only get one and no choice in a common setup. No one should ever see the fallback format, unless the user isn't allowed to access any other formats due to some weird configuration problems.

In short: The current treatment and exposure of the fallback format is the anti-definition of "fallback." This patch here takes preliminary steps to fix that.

The reason for the current approach is just what @sun stated: Anonymous users should be able to use basic HTML, but don't get a WYSIWYG.

The advantages of this approach:

• Authenticated users get images (finally) out of the box. Anonymous users do not get images because it requires either that you allow images from anywhere (dangerous) or you need to enable the same-domain images filter and allow anonymous users to upload to your server (even more dangerous).
• Anonymous users get rel="nofollow" on their links. The lack of which has been driving SEO people batty for years, because Drupal with anonymous comments is a perfect target for making spam machines.
• If you did want a WYSIWYG for anonymous users, having a separate format makes enabling it trivial, as it's simply one-option enable on the Restricted HTML format. If we only allowed anonymous users plain text, you couldn't enable a WYSIWYG on it. The only "easy" approach would be to grant anonymous users the same permission format as authenticated users, which would nullify the first two improvements.

I wouldn't mind enabling a WYSIWYG for anonymous users, but I'm guessing that would stretch the limits of acceptability for the Drupal community. In either case though, we'd need two formats based on the differences in configuration between anonymous and authenticated formats. I like that this makes it so easy to enable a WYSIWYG for anonymous for the sites that want it.

The disadvantages:

• We add another text format out of the box. (I'm not sure this is really a disadvantage, IMO it seems more advantageous that we're assigning the 3 different roles 3 different levels of access. It makes the purpose of text formats more clear in my mind).
• The fallback format (Plain text) will show up when editing content from upgraded sites that have used the fallback intentionally. But if the format is changed to something else, the format may not be changed back.

The second disadvantage is the bigger problem IMO, but it's an edge case. It only affects upgraded sites; only when editing existing content by a user that has access to additional formats (likely administrators); and only when the upgraded site has intentionally disallowed a roles access to all other formats or when a user intentionally changed the text format to plain text. That's a very small number of affected sites; an even smaller number of users that will actually encounter the behavior; and a fraction of those users that would be confused by it. The advantages greatly outweigh the small downsides.

("Wait, I see it, right there, WHY WON'T IT SHOW UP?!?")

I think it's pretty clear in the latest patch why it won't show up. It says it right in the "Roles" column of the format: "This format is shown when no other formats are available". The fallback format is an exception no matter how we decide to treat it in the out-of-box configuration. I agree with @sun that ideally we'd get rid of 99% of its presence entirely in #1932544: Remove all traces of fallback format concept from the (admin) UI, though I'm not sure how realistic that is.

Ok, cool. Thanks for the additional explanation.

Committed and pushed to 8.x. Thanks!

Thanks!

+++ b/core/modules/filter/config/filter.settings.yml

+always_show_fallback_choice: false

Added the removal of this new configuration setting to the task list of #1932544: Remove all traces of fallback format concept from the (admin) UI

With that issue, in the end, we should be able to restore sanity to the fallback format concept in the text format system.

Btw, AFAIK, there are some other feature requests related to the fallback format in the core queue to make it even more configurable and even more flexible and even more confusion than it is today already (but I wasn't able to find them with keywords that you'd think they'd match...) — those can effectively be marked as won't fix.

I'd much rather recommend to move this back to fixed and focus on #1932544: Remove all traces of fallback format concept from the (admin) UI instead.

The entire administrative UI exposure just simply needs to go away. As clarified over there, the fundamental architectural bug is that we tried to make this as visible, configurable, and flexible as everything else in the first place.

Consequently, what we're doing here is to doctor around help texts and descriptions that are cluttering the UI, which are not read nor understood in any case, and all of that complexity for a concept that exists for edge-case scenarios only and should not be visible to begin with.

However, at a minimum we need the attached followup. If we're going to have a bunch of disabled checkboxes which imply that everyone can use this text format, we at least need to try to explain why none of those users are likely to ever see the text format.

I agree that fixing up the checkboxes on the fallback is a good idea. I'd prefer that the display of the checkboxes was removed entirely since they only clutter the interface with options you can't change. If we do decide to just fix the description, it needs to adjust its text based on the always_show_fallback_choice option. Either approach we take, it's only loosely related to this particular issue. The help text was bad before this patch and this patch didn't make it any worse. I think that makes it material for a separate issue.

.