Note: This is issue is part of #2721129: Workflow Initiative and is only meant for planning and governance sign-offs. Work will happen in child issues of this plan.

Target version: Drupal 8.2

The proposal here is to just rename and drop-in Workbench Moderation an experimental module into core.

The purpose of this is to:

  1. Embrace what has already become a popular workflow module for D8 (aGov, Lightning, Deploy module etc are already using it)
  2. Uncovering bugs in core with forward revisions, while in experimental state
  3. Get early feedback on the functionality by end-users

Proposed module name: Content Moderation (content_moderation.module)

Prototypes

For the very first iteration the Workbench Moderation module itself serves as the initial prototype.

Implementation

Required sign-off before commit

Owner Status Description
Product manager Needs work Pending some resolution to new UX debt introduced by numerous state transitions (either #2757275: Decide on default moderation states or #2757349: WI: Deal with scalability issues in the UI), and a default way to view content in different moderation states (e.g. adjustments to the existing admin/content view or separate moderation tab, etc.)
Framework manager Signed-off https://www.drupal.org/node/2755073#comment-11346437
Release manager Needs work Pending data integrity issues introduced by uninstallation: #2757119: Data integrity and uninstall issues with moderation state field and the security impact review
Sub-system maintainers Pending -

Required before stable release

Must-have

Should-have

  • Usability test of entire feature since it’s a 90% feature. This will likely uncover all kinds of other things (e.g. does the tabs pattern work, vs. the yellow box thing in D7?).
  • Move menu item to admin/config/ instead of admin/structure/
  • Make the names of permissions the descriptions of permissions instead (which are a lot more clear), and remove the descriptions to help cut down on visual clutter.

Could-have

Won’t-have (for this feature)

  • “My workbench” dashboard from D7.
  • Workbench Access-like gargantuan permission matrix.
  • A full-blown translation workflow.

Other related discussions

Sign-offs needed

Product manager

A product manager needs to sign-off on this plan as the above phases are required as part of a planned initiative.

Framework manager

A framework manager needs to sign-off on this plan as the above phases introduces major API additions (archive and purge).

Release manager

A release manager needs to sign-off on this plan as the above phases have impact on shippability.

Sub-system maintainers

The sub-system maintainers for the Entity API needs to sign-off on this plan as it significantly impacts the Entity API.

Comments

dixon_ created an issue. See original summary.

dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes
dixon_’s picture

Issue summary: View changes

Updated the issue to make clear that we're targeting 8.2.x for this.

catch’s picture

For me of all the user-facing features we could add related to workflow, a UI for forward revisions makes the most sense, since it opens up a whole series of bugs that are currently hidden in core. Stuff like #2708735: Creating a draft removes published node from taxonomy view and #2684213: Preview on forward node revisions throws an error are obvious known issues but there'll be more. Also a 'workflow initiative' doesn't really make sense to me without forward revisions.

On the other hand I haven't reviewed workbench_moderation in 8.x in any real depth yet.

dixon_’s picture

Dumping here the raw notes from today's Workflow Initiative meeting at Dev Days in Milan:
https://docs.google.com/document/d/1-_AgTnZu9e2iAuEpnp262mguFInNaiIcSxsM...

webchick credited Bojhan.

webchick’s picture

Assigned: Unassigned » webchick

Just spent like 2 hours reviewing this with @Bojhan. Assigning to myself while I write up a big-ass comment. :)

webchick’s picture

I want to preface everything I'm going to say with holy crap I'm so excited OMG OMG YAYYYYY!!, because I definitely am! But you did ask for a product manager review, so that always comes with a certain amount of soul destroying. :D

Here's a walkthrough of the module, for those who are unfamiliar.

Set up

First, you turn the module on and then get your standard couple of warnings about experimental modules:

Content Moderation shows up in Experimental modules.

Next, you go to content types admin and go to "Manage moderation." Here you can check a box to "Enable moderation states" for this content type, and are presented with a list of options:

Select from published vs. unpublished states, as well as a default moderation state

Creating a draft

When you go to create a piece of content of that type, there'll now be more options available under the drop-button. (It defaults to the default state specified in the settings page, in this case "Create New Draft".)

In addition to Save and publish, also see Save and Create Draft and Save and Request Review

When the node is posted, it will appear unpublished, because draft is an "unpublished" state type. Note the new tab "Edit draft."

Article created with unpublished styling

I also tested this in draft mode with in-place editing, and it seems to work well:

Quick-edit on body field
New revision with log message of 'Updated Body field through in-place editing'

Sending to review

Next, send to review:

Choose 'Save and request review' drop button option
Node still in unpublished state

One thing I immediately notice is that there's no visual way to differentiate a "draft" from a "needs review" piece of content, even from the status message. This is a pretty big departure from the D7 Workbench Moderation module, which had a display like this:

Yellow box at top of node explains what state a piece of content is in.

Fixing this feels like must-have, but probably does not need to block commit.

And here... I got stuck. I went to look for things that were in review, and the natural place to look was admin/content, but that page remained unchanged:

Same old boring admin Content page.

I first looked for a "Moderation" tab here, but no such tab existed. Then I tried expanding the options under "Published status" but those are the same old boring "published/not published."

Adding some means of listing in-draft/needs review content feels like must-have, and I lean toward it blocking commit, but open to arguments on that, especially if we do the later proposal of simplifying the options that are shipped. (Spoiler alert! ;))

Segue: Moderation views

For giggles and grins, I went into Views to try and make a moderation view myself:

Make a content view called moderation
Add a filter criteria, both options say 'Moderation state'
Absolutely no idea which option to pick there. Another must-have (?) (but not commit blocking) is somehow differentiating the descriptions of those two options so people know which to choose when.

Then you have a fun thing where your "needs review" content doesn't actually show up because of the default filter for published = yes. Once again, a good argument for blocking commit until there's a pre-made listing page, because that took awhile to figure this out.

Then once you get it working, you have this fun thing where the exposed filter is an empty text field rather than a drop-down:

Moderation state gives no hint as to existing state

I tried "Review", "review" (best guess at a machine name) and several other variations before I gave up and navigated all the way to the Workflow States admin screen and figured out the machine name for it is "needs_review," and filled that in, and voila. :P So it seems like another must-have (but not commit-blocking) is to get that widget to be a drop-down instead, since I don't think we want this to be the content author experience. :)

Publishing content

Any who. Back to what we were trying to do. Back on the node edit form, we have a new option "Save and Send Back to Draft" or "Save and Publish."

New drop button option

Choosing Save and Publish publishes things in the usual way. Note that the "Edit draft" tab has changed to a "New draft" tab.

Published content

You can then go forth and create a new draft while the existing version stays published, and you'll get a third tab, "Latest version" to get back to the that one. There's this interesting new UI there with a drop-down log message field right on the node which wasn't present before on the "proper" node while it was unpublished. That seems a bit odd, especially since we could certainly use a visual indicator there as to what state the node's in. Probably a must-have (?) (but not commit blocker) to make the experience, whatever it is, consistent regardless if it's a new new draft or a new draft on an existing revision.

Tried adding a tag on the not-yet-published revision, and it correctly didn't show on the existing published revision, so this functionality seems to be working well.

Unpublished draft has drop-down to change states + a log message field

Administration

Now looking into admin options and here's where things get a slight bit horrifying. :D

The module defines two new admin pages under admin/structure/content-moderation: "Moderation states" and "Moderation state transitions" (side-note: There's actually a bug in the patch where "Content Moderation" doesn't show up under admin/structure. Presumably something isn't quite renamed right somewhere yet.)

Sub-landing page in admin panel for moderation.

"Moderation states" is a pretty straight-forward, vanilla Drupal admin page.

List of moderation states with machine names and edit/delete drop-downs

"Moderation states transitions" is... ermmm... less so. ;)

Lots of options, including from/to state

For reference, here's the Drupal 7 version of the same screen:

Fewer options, from/to state

Then anything you configure here is also represented on the permissions page like so:

Lots and lots and lots and lots of permissions

So I find the D8 version of these two things very, very overwhelming. :( I'm wondering why the decision was taken to introduce 4 workflow states in the default install of D8 vs. only the 1 that D7 had? I'm assuming to show off the power of what's possible, but we have publishing-focused distributions such as OpenPublish for that. Core should be catering to the 80% case, and it seems like just the various to/from "Draft" states would be fine for that. 25K installs of Workbench Moderation seems to back that up.

So I'd like to propose a must-have that blocks commit of reducing those other sets of options, possibly to reintroduce them at a later date that also attempts to address some of the scalability challenges in the UI. (number of drop-button options, number of permissions, number of state transitions, etc.)

I'm also confused why we prominently display machine names for state transitions and why we got rid of the arrows from the D7 UI. While that's definitely a bit cheesy, it really helps to visually show what's going on. Doing some small iteration on this UI for better understandability seems like another must-have (but does not need to block commit).

When editing a transition you get this screen:

Edit page for moderation state transition

I can think of some "should-have"/"could-have" stuff here; for example the colons on the "Transition to/from" field labels should be removed, and we might also want to explore repeating the relevant segment of permissions UI on the Edit page here, since they're so connected.

Whew! So that was a lot. Going to attempt to summarize my "product manager" feedback in the next comment.

webchick’s picture

Issue summary: View changes

So I read through the notes from the Dev Days discussion and was super dismayed about the consensus being to drop the UI from this patch. :( :( This patch will be a frigging absolute game-changer for Drupal, and give us something completely exciting in 8.2 to whet peoples' appetites as to what's possible through experimental modules and our new agile core process. Additionally, breaking off the UI is sort of the worst of both worlds... you don't get the user-facing functionality indicating you've actually shipped a true MVP, and you don't unburden contrib maintainers. If anything, you give them more work to do, because they now have 2 queues to watch. :(

From the UI POV, there's very little that needs changing in the UI before this patch can be committed. Basically two things:

  • Reduce options to just “Needs draft”; this reduces the UX impact and gives time to discuss scalability issues many options introduce in other places of the UI, while delivering an incredibly useful feature to 100% of Drupal 8 users.
  • Add some way to view a list of content that’s in $moderation_state mode (e.g. override the “Status” column of the existing Content admin page, a separate “Moderation” view, etc.)

That's it. Everything else might not be perfect, but they're also not show-stoppers. And we'd be able to fix those bugs and improve those UIs faster if we got this in front of more people, and an experimental module in 8.2 is the perfect way to do this. I know @Bojhan in particular has concerns about introducing UX debt that we don't ever clean up, but to me that's addressed by a) the requirement of a must/have/should/won't list prior to commit (which is needed regardless if this module ships with a UI or not), and b) we also put a timeline on the must-haves to get resolved, which is typically a year (I might argue for that to be extended in this case, since it's a strategic initiative, but we could always start there and see how we go), per https://www.drupal.org/core/experimental.

So my personal opinion is, "Hell yes! Let's do it! UI and all!" :D

Added a list of the pre-commit / must/have/should things in the issue summary that came out of @Bojhan and my's call. I'm sure it needs expanding, but it's a start.

Crell’s picture

webchick: Thank you for the review! Some background information that may prove helpful, as there's a surprising number of edge cases we chewed on while writing WBM for D8:

1) Note that the in-place quick-editor works only on the latest revision. Once you publish a draft and make a forward revision, quick edit is disabled on the view page; this is unavoidable, as trying to start a new revision from the not-latest revision is pronounced "data loss" without a crapton more work to reimplement Drupal in Git. :-) (Or put workspaces in core, which is a totally different beast.)

2) Note that when there is no published revision, the latest revision will always be made default. You only get forward revisions once there is at least one published revision. That seems weird at first, but is the least weird option available and we believe fits with user expectations.

3) The "status form" shows on, I believe, any forward revision. That's the form you're missing. Given point 2... yeah, you're not going to see it until you have at least one saved revision. That's an interesting problem. :-) We spent a long time sorting through when to do what, because there's lots of edge cases. And that's before you start talking about block content, which doesn't have a published flag. That's another fly in the ointment.

4) We omitted a stock view from the module largely because making it dovetail nicely with other views from other modules is, um, hard. The D7 version of the module used hook_views_alter on the various other workbench suite views, which is an approach we really didn't want to take for D8. Adding another view at /admin/content/forward-revisions or something should be pretty straightforward.

5) For the revision/non-revision version of the "Moderation state" field, I don't believe that's an issue of this module. That's simply an entity reference field that is revisionable, and that's how Views handles fields that are revisionable. I don't know that anything can be done in this module about that, specifically.

6) The published=true filter is a default added by Views. It's annoying in this case, yes. :-) Including a view out of the box is probably the best we can do here. Alternatively, we could modify Views to NOT include that filter by default when the base table is a revision table. (Filtering by moderation state makes little sense unless you're filtering revisions, and showing only latest revisions.) We'll probably also want #2702041: Add views relationship from content to latest revision, which is already RTBC.

7) The text-based filter instead of drop-down filter is a core bug. See: #2654908: Add Views support for entity reference fields that point to configuration entities. Fix that issue and it gets fixed here, too.

8) Workbench Moderation D7 also included a Needs Review state by default, I believe. At Palantir, we found that Draft/NR/Published/Archived covers something like 90% of all use cases, even if clients didn't admit it at first. :-) That said, I am also entirely fine with dropping the NR state from core by default. It's trivial enough to re-add.

9) One reason we showed the machine names is that the transitions are, in some cases, named the same. The transition labels are used in the UI for the labels in the save button and the moderation form; "Publish" is the correct label whether you're coming from Draft or from Needs Review, and there's nothing we can do to prevent users from creating multiple revisions with the same label. Only the machine name need be unique. I certainly agree that there's room for improvement in the admin UI, though. It was mostly built for ease of implementation, with intent to iterate later.

I also agree entirely that splitting off the UI "temporarily" is a self-defeating waste of time.

Another note: The permission-based transition control was the easiest to implement, but not a final solution. There is, currently, no way to hook into the transitions and make them per-OG, for instance. Rewriting that part of the module to be more flexible is, IMO, a high-importance not-commit-blocking follow-up.

Bojhan’s picture

Assigned: webchick » catch

Given that I've been involved in most of the discussions, I'd like to add that getting this into 8.2 is crucial. This is the first time we would be providing our users with key functionality they expect from a CMS through our new experiment process.

The state transition UI, although unnecessarily complex - is key to enabling users to do more with Drupal than today and matches a key user need. The UX debt that is created from this, seems manageable to me and by reducing the initial set of states we limit the impact sufficiently to avoid creating the perception that experimental modules are unusable. I see no reason this cannot be resolved within a single release, the challenges are debatable but not complex from a UX POV.

@catch/alex Can you provide feedback on the framework viability of this approach? Are the steps outlined sufficient and/or do you see any foreseeable technical debt that impacts our ability to release?

To be clear, we want to get approval from product/framework and relevant sub-system maintainers - to avoid wasting a lot of time on making patches that won't pass review.

Bojhan’s picture

Assigned: catch » alexpott
larowlan’s picture

Re Draft/NR/Published/Archived you have to have archived, else there is no way to unpublish.

So that means Draft/Published/Archived are must haves.

Agree needs review is the 90% of sites I've used this on.

webchick’s picture

https://www.drupal.org/files/issues/Screen%20Shot%202016-06-27%20at%2012... is the screenshot from D7; it does come with Draft and Needs review, but not Archived. So I can see that argument. But why do we need Archived and Published as additional statuses, cluttering up the various UIs, when D7 didn't?

larowlan’s picture

If you don't have archived, then you cannot remove a node from public display once it has been published.

Moving it to any other state will keep the previously published version visible.

Archived is like published in that it becomes the current revision, but the difference is it marks the node as unpublished too.

larowlan’s picture

If the main driver here is removing UI clutter, then we should be targetting transitions rather than states.

E.g. You could have this set of transitions:

draft -> draft (keep in draft)
draft -> needs review (request review)
needs review -> published (approve)
published -> archived (archive)
published ->draft (create new draft)
published -> published (hotfix edit)
archived -> draft (bring back to life)

and probably have something reasonably functional

The default wbm config has almost every combination of to and from state.

so it also has

needs review -> needs review (save some changes during review but don't publish)
needs review -> draft (reject approval)
published -> needs review (create new draft and submit in one go)
archived -> needs review (same)

webchick’s picture

Yeah, sorry if that wasn't clear; simplifying the UI is the goal here, and removing transitions allows us to side-step the open-ended "fix the state transitions admin screen so it's less daunting" must-have task until post-commit.

Those simplified transitions sound like a start, but can we just literally copy what's in D7 workbench moderation for the purposes of initial commit and not try and get super fancy? I believe D7 WBM just uses the core published/unpublished flags to signify published/unpublished nodes/revisions. Why do we need separate states (and all the various to/from transitions/permissions/etc. that accompany them) to do that? That creates two different concepts for the exact same thing in core, which seems sub-optimal?

chx’s picture

I hate to but I need to chime in: while I have not read the issue through I did search for i18n and multilingual and translation and found none of those words and that concerns me quite a bit (see that this comment is attributed to two companies, that's how much it concerns me).

dawehner’s picture

@chx
The module certainly includes some support for translations, backed by entity API. It includes even test coverage for that, see ModerationLocaleTest

chx’s picture

I really have two questions here: is it possible to restrict publishing a piece of content until all translation drafts are approved? Once they are, can the transition to go live be done in a single step? I do not know how far an experimental module needs to go but these two are fundamental in a real multilingual environment, imagine releasing an EU press release in the 24 official languages.

But, of course, at the same time, it should be possible to relax this on a case-by-case basis. To quote http://ec.europa.eu/languages/policy/linguistic-diversity/official-langu...

the European Commission aims to provide visitors with web content either in their own language or in one they can understand, depending on their real needs. This language policy will be applied as consistently as possible across the new web presence. An evidence-based, user-focused approach will be used to decide whether many language versions are required or not.

For example http://www.consilium.europa.eu/en/press/press-releases/2016/06/20-fac-co... is not available in all languages (by far) but http://www.consilium.europa.eu/en/press/press-releases/2016/06/24-joint-... probably is (although I haven't checked all 24 by hand but the presence of a 25th suggests the importance of this document).

webchick’s picture

A full-blown translation moderation workflow is definitely out of scope for this patch, and we should probably debate in a separate issue whether or not core needs to provide that detailed translation moderation workflow at all, or if something like https://www.drupal.org/project/tmgmt in contrib is fine.

That said, it would be interesting to know the answer. But as long as there's no data integrity issues w/ translations (which #21 would seem to suggest) it's fine with me for initial commit. Possible we need to extend the list of must/should haves though if we run into gnarlies.

tkoleary’s picture

Thanks Aurelian.

@chx, @dawehner, @larowlan, @webchick #2753717: [PP-1] Add select field to choose moderation state on entity forms specifically addresses the problem you are discussing.

alexpott’s picture

I've reviewed the module and discussed with @catch in IRC. The biggest issue @catch raised was the potential for an entity's status getting out of sync with its moderation status - #2757119: Data integrity and uninstall issues with moderation state field

I've already addressed architectural concerns around the config entities (names and dependency calculation) and the use of services in hooks. The current patch on #2725533-60: Add experimental content_moderation module (#60) does not present any blocking architectural concerns and is compliant with core's automated coding standards checks.

I consider the framework manager review done.

tkoleary’s picture

Just re-read this issue in it's entirety.

It seems to me that the multiplication of transitions as workflow states are added promises to be a big UX problem.

Would it be possible to simply make all states transition to all other states by default? The design in #2753717: [PP-1] Add select field to choose moderation state on entity forms would be the author facing expression of this.

I understand that administrators would like to have the ability to create custom transitions, but why not make that an explicit choice and not cruft-up the UI?

For example, if each state has all possible transitions enabled by default but contains an "add custom transition" link in it's configuration then if the user adds a custom transition to another state (for the sake of argument I'm assuming all transitions are "from -> to"" and are children of the initial state), the user would see only the custom transition in the UI and all others would be assumed.

This would be a far simpler UI to administer as the only transitions displayed would be those explicitly created to serve some workflow purpose, eg. restricting permission to use that transition by role.

If we implement: #2753717: [PP-1] Add select field to choose moderation state on entity forms this would be shown to the author in that UI as either a disabled state button, or a validation message, eg. when the state is selected and the node is saved the user would see "you do not have permission to change the state to x"

Crell’s picture

At present, the module treats each language independently I believe. So you can have a published English and French, and a forward revision just in English, for instance.

Angie: The underlying architecture of D7 vs D8 is quite different, so an exact replica of D7 WBM is not possible. D7 tracked a lot of information independently of the entity being moderated, while D8 tracks (nearly) everything just in the entity itself as an ER field. The core published/unpublished checkbox becomes a purely derived value off of the state. We cannot go back and modify the revision after the fact without hacking into the database directly, which is off the table (and would be a massive break in this module's assumptions anyway).

That said, IMO the lack of an Archived state in WBM D7 by default was a design flaw, not something to emulate. As larwolan notes, without that once an entity has a published revision it can NEVER be taken down without being outright deleted unless you have the Archived state, that is, the "unpublish but don't delete" functionality of core goes away. The archived state is the only way to solve that. I disagree that it's a huge UX burden.

We included by default every possible transition between the four default states, mainly because it meant that during testing uid 1 could transition anything to anything and you could then control who could do what exclusively through permissions; honestly, we expected most users to never even bother with the transitions page, just the permissions page, and never look at the admin at all. That was the goal.

Trimming the default transition list is fine, but I don't believe we can realistically go lower than Draft/Published/Archived states Draft->Pub, Pub->Draft, Pub->Archived, Archived->Draft, and the self transitions for Draft and Published. Anything less than those 3 states and 6 transitions renders the default config non-functional, which is a non-starter.

Kevin: If all possible transitions are defined implicitly, then what "custom transition" would even exist? There are n^2 possible transitions, and if we auto-create all of them, there's nothing else to create...

Currently, the "can I edit/save this entity" logic asks "is there a valid transition from $current_state to $possible_target, is it enabled for this bundle, and does the user have permission to do so?" As noted above, we need to make the latter more flexible but that's not commit blocking. Making all transitions legal automatically doesn't resolve the "enabled for this bundle" question; you can pick and choose which transitions you want to allow on news nodes or event nodes, for instance, even disallowing some self-transitions. (Eg, the published->published transition lets you quickly edit a typo without needing to go back to Draft; it's up to the admin if that's something that should be allowed or not, and can be set per-node type.)

We also need to have a transition object that is user-editable because that's where the label of the transition is defined, which is what's shown in the UI for the drop down save button and for the select box on the moderation form.

Unfortunately, state transition management is a problem space that is intrinsically complex, and the number of possible options is *at least* n^2. Once you add in permissions, or more advanced access control, it grows even faster. That is unavoidable, and trying to remove some flexibility to reduce the number of options in the UI does just that: Removes flexibility that we've seen is needed in practice. Rather, we should be focusing on sufficiently-sane defaults that most users don't need to think about the more complex cases, and then for those users that do need them we should be trying to better explain and express the options available, not removing them.

tkoleary’s picture

Kevin: If all possible transitions are defined implicitly, then what "custom transition" would even exist?

What the user would see as "creating" a custom transition would in reality be "exposing" an existing transition (automatically generated when a state was created) so that it can be configured to differ from the default (presumably the most permissive eg. "All users can use all transitions for all bundles").

Currently, the "can I edit/save this entity" logic asks "is there a valid transition... it's up to the admin if that's something that should be allowed or not, and can be set per-node type.

Well then that's even more complex than I thought. Another variable further compounds the number of possible transitions, and with it the potential for confusion and error. This really strengthens the argument to hide all but "explicitly configured" transitions.

We also need to have a transition object that is user-editable because that's where the label of the transition is defined, which is what's shown in the UI for the drop down save button and for the select box on the moderation form.

Of course we need such an object, but we don't need to show all of them in the UI, nor do we need to label them if we implement #2753717: [PP-1] Add select field to choose moderation state on entity forms . This suggests to me that #2753717 should be a commit blocking must-have for this issue because with it there is no explicit need for author-facing transition labels, only state labels.

Unfortunately, state transition management is a problem space that is intrinsically complex, and the number of possible options is *at least* n^2.

All the more reason not to expose all of them.

Once you add in permissions, or more advanced access control, it grows even faster. That is unavoidable,

Precisely.

and trying to remove some flexibility to reduce the number of options in the UI does just that: Removes flexibility that we've seen is needed in practice.

Maybe I haven't explained what I'm proposing well enough. The user can choose to configure any transition between any two states for any bundle or any user with any role or permission. No functionality or flexibility that exists now is in any way removed.

The only thing different would be that no transition would appear in the UI until the user has explicitly chosen to configure one.

Rather, we should be focusing on sufficiently-sane defaults that most users don't need to think about the more complex cases,

That's all I'm suggesting.

for those users that do need them we should be trying to better explain and express the options available, not removing them.

Of course. When the user explicitly chooses to create a custom transition the options should be explained clearly.

honestly, we expected most users to never even bother with the transitions page, just the permissions page, and never look at the admin at all. That was the goal.

So let's just make that expectation a reality. :)

webchick’s picture

Angie: The underlying architecture of D7 vs D8 is quite different, so an exact replica of D7 WBM is not possible. D7 tracked a lot of information independently of the entity being moderated, while D8 tracks (nearly) everything just in the entity itself as an ER field. The core published/unpublished checkbox becomes a purely derived value off of the state. We cannot go back and modify the revision after the fact without hacking into the database directly, which is off the table (and would be a massive break in this module's assumptions anyway).

I think my concerns here are basically covered by #2757119: Data integrity and uninstall issues with moderation state field so I'll keep my eye on that one.

We included by default every possible transition between the four default states, mainly because it meant that during testing uid 1 could transition anything to anything and you could then control who could do what exclusively through permissions; honestly, we expected most users to never even bother with the transitions page, just the permissions page, and never look at the admin at all. That was the goal.

Right, but UX bloat caused by the n^2 problem extends to the permissions page, as well, which is a "90%" admin page. I had to bump my font size down twice to even get all of those options to show up on one screen to take a screen shot of it. :( Hence, the desire to simplify, at least for initial commit.

Trimming the default transition list is fine, but I don't believe we can realistically go lower than Draft/Published/Archived states Draft->Pub, Pub->Draft, Pub->Archived, Archived->Draft, and the self transitions for Draft and Published. Anything less than those 3 states and 6 transitions renders the default config non-functional, which is a non-starter.

Sure, that seems like a decent place to start. Once again, if we find out that this isn't sufficient after 8.2.0 ships, and in reality 90% of sites are adding a "Needs review" state, we can always add it back (ideally, after making some cosmetic tweaks to the transitions admin screen and/or permissions to reduce the UX clutter).

Rather, we should be focusing on sufficiently-sane defaults that most users don't need to think about the more complex cases, and then for those users that do need them we should be trying to better explain and express the options available, not removing them.

I agree as well that's where we ultimately want to end up, but I didn't want to block commit on figuring out those larger design questions. Hence the workaround of simplifying the available options for now, to possibly add back in later once we have resolved those design questions.

webchick’s picture

Btw, if we want to go the other way, and fix the UX problems first, thereby allowing us to keep all the 279 different options, I can try raising this in UX meeting later on today and see what suggestions come out of that.

webchick’s picture

Issue summary: View changes

Updating the issue summary a bit.

tkoleary’s picture

@crell @webchick

I agree as well that's where we ultimately want to end up, but I didn't want to block commit on figuring out those larger design questions. Hence the workaround of simplifying the available options for now, to possibly add back in later once we have resolved those design questions.

Because #2753717: [PP-1] Add select field to choose moderation state on entity forms means the user action is to choose the next state and not the transition to the next state. By implementing #2753717 we would remove the need to expose any default transitions or their labels to the user, and with it the need to expose all the cruft in the permissions UI as well.

I would suggest that we could actually reduce the scope of this issue (assuming #2753717) by making the management of custom transitions a follow-up.

webchick’s picture

webchick’s picture

Issue summary: View changes
webchick’s picture

Issue summary: View changes

Spun off #2757349: WI: Deal with scalability issues in the UI to discuss the UI scalability challenges more.

timmillwood’s picture

Crell’s picture

During development of WBM D8, we went back and forth a couple of times on whether to expose state names or transition names. We settled on transition names as it allowed for a more action-oriented user experience. That is "Do X" rather than "set this value to Y". The state names don't necessarily mean anything to people, and may even be more internal-ish. The transitions allow for a same/different description with context of both the from and to state. Eg, you fire the "please review this thing" action rather than the "set moderation state to needs_review" behavior. When shown in the save button, it makes even more sense to do it this way; "Save and $action" rather than "Save and transition to $state", which it was originally and sucked. :-)

becw, I think, can speak more to that; her memory may be better than mine. :-)

In other words, exposing state names is "just dumb crud, mapping the db table to the UI", whereas the transition name is a user action.

webchick: Do we need a workaround for initial commit beyond just dropping Needs Review? The rest just takes care of itself, at least good enough for now. I 100% agree that the the access handling needs to change, but that's not a in-this-issue sized change. :-)

To clarify:

* n^2 is the bare minimum number of possible transitions
* n is the maximum number of possible destination states FROM a given entity.
* Currently we allow per-bundle configuration of what states can be moved to, period, so it could be lower.
* Permissions may reduce the number of legal destination states even further.
* If we add support for other access control mechanisms (per-OG, per-bundle-per-permission, etc.), then the legal destination states reduces even further.

That is, for the content editor the total number of states/transitions to think about will almost always be quite small, even in complex configurations, because there's so many possible restrictions. The absolute upper bound is the number of states, which is unlikely to ever get particularly large. (The Lightning Team said they had a case with 7 states, which is likely the highest we'll see.) Most situations will see far fewer; if you have 7 states that probably means a tightly controlled set of transitions and some sort of multi-step approval process, so you won't see more than 2-3 transitions in the UI.

The scale issue only affects administrators setting up those conditions in the first place. That is a problem that we should solve, but I don't think any part of it is a blocking issue.

I'd actually be against #2753717: [PP-1] Add select field to choose moderation state on entity forms, as I don't think it solves any problems that actually exist. It also implies a linear-ness that is not at all inherent in the system, which we've discussed before and is why that was never implemented for WBM in contrib.

webchick’s picture

During development of WBM D8, we went back and forth a couple of times on whether to expose state names or transition names. We settled on transition names as it allowed for a more action-oriented user experience. That is "Do X" rather than "set this value to Y". The state names don't necessarily mean anything to people, and may even be more internal-ish. The transitions allow for a same/different description with context of both the from and to state. Eg, you fire the "please review this thing" action rather than the "set moderation state to needs_review" behavior. When shown in the save button, it makes even more sense to do it this way; "Save and $action" rather than "Save and transition to $state", which it was originally and sucked. :-)

Right, but that's only so long as the save button is done that way, and there are known scalability issues with that approach. An active issue in the post-commit "must-have" list to change it to just a plain old "Save" button and instead use checkboxes and/or radios and/or drop-downs or something to "Save and" something. #2068063: Change "Save and keep un-/published" buttons to a "Published" checkbox and an included "Save" button If/When we go that route, a plain old state name in a drop-down labeled "Transition to" is probably just fine.

webchick: Do we need a workaround for initial commit beyond just dropping Needs Review? The rest just takes care of itself, at least good enough for now. I 100% agree that the the access handling needs to change, but that's not a in-this-issue sized change. :-)

I don't think so, but let's discuss over in #2757275: Decide on default moderation states. I was suggesting UI tweaks because I was getting so much pushback on what seemed like a very simple workaround as a compromise, just to remove items from the various lists.

tkoleary’s picture

@crell

I'd actually be against #2753717: Add button bar for workflow states to /node-edit, as I don't think it solves any problems that actually exist.

Talk to Thomas Howell, our Docs and Library owner about problems that actually exist, he has a whole team of authors who hate Workbench Moderation's UI with a consuming passion that approaches the spiritual, and one of our (very large) customers hated it so much that our PS team was forced to completely rip it out and replace it with a custom solution.

The specific problems they expressed to me are the ones I addressed in this design.

It also implies a linear-ness that is not at all inherent in the system, which we've discussed before and is why that was never implemented for WBM in contrib.

The linearity that it implies primarily helps the author comprehend the flow as a flow. But the linearity is only implied. There is nothing to stop the user from going from Archived to Draft if that is permitted by the workflow.

webchick’s picture

Let's keep comments about #2753717: [PP-1] Add select field to choose moderation state on entity forms to that issue, please, so this one can stay laser-focused on defining the overall spec for the MVP. We only have about 5 weeks left to get this into 8.2.0, and we're so close!

larowlan’s picture

Note that we cannot do away with transitions, things like Workbench email react to the transition and need to know both the from and to state.

And either way, the drop button is just the widget. As with any other field in Drupal, we can add more widgets and let the admin choose their preference.

larowlan’s picture

Also for some real-world insight here - the current project I'm working on has the following requirements

Content can be in one of five states:
- Draft
- Needs approval
- Published
- Archived/Expired
- Needs review

The additional needs review state is applied via scheduled updates.

Workflow is as follows:
- editors create drafts and move content to 'needs approval' when they're ready for it to be published
- approvers review the content and add a scheduled review date - many Government departments have regulations/policies around regular review of content for continued relevance.
- approver publishes
- when review date passes, content remains published and a new 'needs review' forward revision is created
- this shows in the editors workflow list (they are also emailed)
- editor checks content is still relevant and sends for approval again (transition from needs review to needs approval)
- approver repeats process above

Things to note:
- there are 5 states
- whilst there could be 5^2 transitions, there is not. It makes no sense to have draft -> needs review or needs approval -> needs review. The only transition with a to state of 'needs review' is published to needs review, as this aligns to the requirements.
- editor has access to subset of transition permissions (they cannot publish)
- approver has access to all transitions, they can hotfix (published to published) or skip the whole process (draft to published)

There are various email rules configured (workbench email module) such as
- email to approver when things need approval
- email to editor when their content is published
All of this is tied to the transitions.

Further complicating the situation, anonymous users can create event content. This goes via a submission process too.
Because the default 'draft to needs approval' transition has label 'Submit for approval', this isn't appropriate for the requirements of anonymous submitted events. Instead the client wanted an alternate title. This was possible using the fine grained permission controls by adding another 'draft to needs approval' transition with a different label.
Editors didn't have access to this transition, so continued to see the 'Submit for approval' label. Anonymous users did, so saw 'Submit to event calendar'. This was all achievable without any custom code thanks to the existing transition/permission architecture.

So whilst we may be striving for UX purity, we should be careful not to water down the great flexibility and power the existing feature set provides.

webchick’s picture

I don't think Kevin or anyone else was suggesting doing away with transitions, that's obviously core to this module's functionality. He was suggesting all permutations of state A -> state Z simply being auto-generated, thus negating the requirement to manually specify them, and thus negating the need for the problematic UI. Visibility of options could be controlled via permissions, and "transition bloat" could be reduced by grouping specific sets of states into "Workflows" which could then be assigned to content types (so blog posts use the "Simple" workflow and go through New -> Draft -> Published -> Archived but press releases go through the "Overlord" workflow with New -> Draft -> Draft #2 -> Super-Draft -> Drafty McDrafterson -> Pending Legal Review -> Published -> Archived -> Really Archived -> etc.). This still provides the requisite flexibility while not needing to enter n^2 UI scalability problems. (None of this is required for the patch to get committed though, it's just brainstorming.)

The alternate label for editors vs. anon is a nice touch, though I once again wonder if when #2068063: Change "Save and keep un-/published" buttons to a "Published" checkbox and an included "Save" button comes to pass, that same transition re-labeling functionality will be necessary at all.

And can we please not belittle the UX/design efforts being done here as "striving for UX purity"? To be clear, we are all extremely excited about this functionality making its way into core, and are bending over backwards trying to find workarounds to get this to a minimally committable state without throwing a bunch of hurdles in the way. But just as we need to be cognizant of what technical debt new features are adding, we also need to be cognizant of what UX debt we're adding. Especially since the latter is much harder to find random passerbys to work on.

larowlan’s picture

Apologies if I was belittling, didn't mean to be. I support the UX/design efforts.

Wim Leers’s picture

A bit of cautionary feedback from a passerby…

I 100% agree that the the access handling needs to change, but that's not a in-this-issue sized change. :-)

… that is exactly what people said about REST's access handling. #1816354: Add a REST module, starting with DELETE added REST to D8 in 2012, much like this will add Content Moderation. Improved (less complex — also exactly the same!) access handling was postponed until later.

Making access handling better (less complex) never happened.

We're still dealing with the consequences. And not breaking BC is non-trivial.

So, I'd urge the committers to:

  1. require a follow-up issue to improve Content Moderation's access handling
  2. create a Plan issue that will revert all of Content Moderation before 8.2 ships if critical problems (like access handling) are not addressed in time

That would ensure those very important things do actually happen in time, so we will not be faced with thorny BC problems after 8.2.0.

dawehner’s picture

Isn't the point of experimental problems that BC can be broken at any point in time?

Wim Leers’s picture

Eh, right, I forgot for a moment that this would be experimental.

So fixing access control should be something that happens before making it a stable module.

Bojhan’s picture

@Wim Leers Just to be clear, you think fixing acces control is a must-have. This is the plan issue, so this is exactly the kind of feedback we are looking for.

In order to add it as a must have I would like to ask you to elaborate on "why its so important" and ideally some ideas on "how to solve it". Otherwise we might be throwing ourselves into must-haves that have no clear guidance on how to solve them and how they can be distinguished from "should haves" before stable release.

catch’s picture

Alex Pott suggested using a 'WI critical' tag for issues like that, similar to 'migrate critical'. That will ensure we at least triage all those issues before anything becomes stable.

Dries’s picture

I'm still catching up on this issue but one thing that confused me was some of the labels on the Save-button. For example, what exactly does "Save and create new draft" mean? It sounds as if it would save the current node leaving its current workflow status unchanged (e.g. published) but clone the node and set the clone's state to draft. In other words, it feels like I'll end up with a clone. For the default, it would be better to have "Save as draft" Not sure the "Save and $action" pattern is the best.

Dries’s picture

FileSize
54.89 KB

Ideally, you'd create the state transitions in a graphical environment that could look like this (courtesy Phase2). In other words, you create transitions by creating arrows between two states. I think it would help a lot to bring it all together in one screen that provides an instant overview. Obviously not a must-have, but a could-have. Nothing some JavaScript can't do. ;-)

Mautic, an Open Source marketing automation tool, built something comparable and the libraries should be available as open source. See https://youtu.be/J5hmI6ae2gY?t=1067.

Bojhan’s picture

@Dries Yup. I mentioned this to @webchick as well - you quickly get into this realm. Definitely something we should look into after the initial module is committed. It's worth taking a look at tools such as https://github.com/the-grid/the-graph, which do very nifty things to also make this mobile friendly but also https://developer.mulesoft.com/.

For the permission workflow (Concrete 5 provides inline adding of roles; http://documentation.concrete5.org/editors/dashboard/workflow/basic-work...).

catch’s picture

In terms of the UI I'm also wondering what the most basic approval case is, for example a single author site where I want to make edits to an already published node, save them as draft, then publish them. In that case I need the ability to create forward revisions, and set a forward revision as the default, but I don't need any workflow beyond that.

webchick’s picture

Issue summary: View changes

Sounds like release managers are pushing back on #2757119: Data integrity and uninstall issues with moderation state field being a commit-blocker as well. Adding to the summary.

Also adding exploration of fancier user interfaces to "could have." That looks hot. :)

I think #54 is an accurate description of the simple case, but I wouldn't recommend removing the ability to do fancier use cases as part of implementing it. Some wording tweaks aside, it seems like the current content author workflow is fine as a starting point.

@larowlan: No worries, thanks. Sorry if some of this feedback comes off as frustrating.

tkoleary’s picture

webchick’s picture

Issue summary: View changes

Clarifying the current status of sign-offs.

xjm’s picture

Issue summary: View changes

Thanks @webchick!

@dixon_ and I discussed this with the group at DDD and the summary does indeed reflect what I discussed would be needed for this. (Adding one clarification.)

webchick’s picture

Issue summary: View changes
agentrickard’s picture

Late to the party and noting that there is a UI recommendation in #2753673: Add status and workflow state to page title block..

Bojhan’s picture

Issue summary: View changes

Version: 8.2.x-dev » 8.3.x-dev

Drupal 8.2.0-beta1 was released on August 3, 2016, which means new developments and disruptive changes should now be targeted against the 8.3.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

xjm’s picture

Title: WI: Content Moderation module » WI: Content Moderation module roadmap
Issue tags: +Needs issue summary update

I think #2725533: Add experimental content_moderation module has some followups that aren't reflected on the roadmap here; can we add them to the issue summary to clarify what is Must/Should/Could/etc.?

xjm’s picture

It would still be helpful to get that issue summary update.

@catch, @cilefen, @Cottser, @alexpott, @effulgentsia, and I discussed the status of this module yesterday and agreed it's still in alpha based on the outstanding work remaining. @alexpott clarified that since this module now depends on the Workflows module, that module should reach beta before this one would.

However, I don't see anything about Workflows or any link here to that module's roadmap. Can we please update this issue with the current status?

As a reminder, Content Moderation needs to become stable by 8.4.0-beta1, six months from now, or it may be removed from core.

Thanks everyone for your intense ongoing work on this module and on the Workflow Initiative overall!

alexpott’s picture

Issue summary: View changes
gambry’s picture

Issue summary: View changes
alexpott’s picture

Issue summary: View changes
Related issues: +#2843494: WI: Workflows module roadmap

Version: 8.3.x-dev » 8.4.x-dev

Drupal 8.3.0-alpha1 will be released the week of January 30, 2017, which means new developments and disruptive changes should now be targeted against the 8.4.x-dev branch. For more information see the Drupal 8 minor version schedule and the Allowed changes during the Drupal 8 release cycle.

dkre’s picture

It's great that this is being worked on as an addition to core, it's a must have for any site with more than two content maintainers/creators.

Just a couple of questions:

I used Workbench Moderation for my first project (a sister site to the one I'm currently developing) and it works fine. Is there any cross compatibility between this and the original project? Can I upgrade/migrate from Workbench at a later date or is this being planned at all?

Also I've noticed the beta requirement message a couple of times:

"As a reminder, Content Moderation needs to become stable by 8.4.0-beta1, six months from now, or it may be removed from core."

While I understand and appreciate the reasons for this from a product ownership point of view it makes it a little hard to make good long term decisions in development. My concern with this initiative is that Workbench will become unmaintained if this reaches beta (or possibly earlier) and I may not have an upgrade path. Or I start my content migration process now with this core version and possibly risk it being removed and mothballed.

I know both possibilties are very unlikely but this is likely to be a pattern with core development in the future so it'd be really good to get an idea of how I should approach these instances.

timmillwood’s picture

Experimental modules have 1 year to become stable, Content Moderation was added in 8.2.0 and therefore needs to be stable by 8.4.0. However Content Moderation now depends on Workflows, which was added in 8.3.0 and therefore needs to be stable by 8.5.0. Content Moderation can't be stable until Workflows is. So, what do we do? Give Content Moderation until 8.5.0 to get stable, or rush Workflows into being stable within 6 months of release?

catch’s picture

In terms of stability, the module itself seems like it's on track to me in general.

However as I mentioned on the original roadmap issue #2721129-35: Workflow Initiative there are still large architectural issues with parts of core and forward revisions. I remembered the path alias one earlier this week and opened #2856363: Path alias changes for draft revisions immediately leak into live site as a critical bug. larowlan also mentioned menu links likely have the same problem - going to test that now and open an issue if so. These to me are if anything more of a barrier to stability than issues in the modules themselves, since they break people's published data.

xjm’s picture

Thanks @dkre for raising the above points. Regarding:

While I understand and appreciate the reasons for this from a product ownership point of view it makes it a little hard to make good long term decisions in development. My concern with this initiative is that Workbench will become unmaintained if this reaches beta (or possibly earlier) and I may not have an upgrade path. Or I start my content migration process now with this core version and possibly risk it being removed and mothballed.

In general, we do not recommend using experimental modules in production, as they also don't provide upgrade paths and can break at any time. For most sites, if there is a stable contrib equivalent, that stable equivalent should be used. I would not invest resources in migrating content to the core experimental module until it at least reaches beta. You can read about alpha vs. beta vs. RC vs. stable here: https://www.drupal.org/core/experimental#alpha

Edit: If an experimental module is removed from core, it will be moved into contrib as-is and can still be reconsidered for core inclusion in the future, but without any support for data migration guaranteed.

Part of the reason that experimental modules are given one year to stabilize is also the problem you raise about them becoming a competing solution against stable contrib. By limiting the uncertainty to one year, we hope constrain the time that contrib development might be stalled if the core module "experiment" doesn't work out to be a stable core solution in the end.

I hope that helps!

For @timmillwood:

Experimental modules have 1 year to become stable, Content Moderation was added in 8.2.0 and therefore needs to be stable by 8.4.0. However Content Moderation now depends on Workflows, which was added in 8.3.0 and therefore needs to be stable by 8.5.0. Content Moderation can't be stable until Workflows is. So, what do we do? Give Content Moderation until 8.5.0 to get stable, or rush Workflows into being stable within 6 months of release?

Thanks for bringing this up too. I agree it's both important to stabilize and important to not rush things to "stable" and risking incomplete solutions that might cause more problems later. Workflow/Content Moderation definitely have more requirements than a pure UI feature like (e.g.) Settings Tray and so we should take that into account when setting expectations.

I agree with catch that we need to at least have any data integrity issues affecting Workflow, Content Moderation, or future improvements resolved by 8.4.0-alpha1 (August). I think given the scope of work needed, it's appropriate to consider more than 1 year for Content Moderation as we finalize Workflow. However, I do think we need to get to a beta by 8.4.0 to know that the modules will be successful and mitigate the risks that @dkre and others might have to cope with.

So @catch and I agree that:

  1. Critical/data integrity problems affecting Workflow, Content Moderation, etc. should be resolved by 8.4.0-alpha1.
  2. Other major issues affecting forward revisioning and therefore Workflow, Content Moderation, etc. should be resolved by 8.4.0-alpha1.
  3. Workflow and Content Moderation should both reach beta stability by 8.4.0-beta1.
  4. Workflow and Content Moderation should reach RC and then stable prior to 8.5.0-alpha1 and ideally as early in 8.5.x development as feasible.
  5. Workflow and Content Moderation also need to be stable before we add the next features in the Workflow featureset to core as experimental.

We haven't run this by the product managers yet, but we wanted to propose it for consideration.

xjm’s picture

Assigned: alexpott » Unassigned

I don't think this was supposed to still be assigned to alexpott.

timmillwood’s picture

@xjm, I think Workflow and Content Moderation Beta by 8.4.0, then stable by 8.5.0 makes sense, and once decided we need to update https://www.drupal.org/core/experimental to change the deadline to 8.5.0

My only concern with #72 is the comment:

Workflow and Content Moderation also need to be stable before we add the next features in the Workflow featureset to core as experimental.

I assume "next features in the Workflow featureset" means, next experimental module from the Workflow Initiative. We were hoping to get an Alpha version of the Workspace module in to 8.4.0. To prepare for this we have been working in the 8.x-2.x branch of the Workspace contrib module, aiming for this to be a straight drop in to core. It is being rewritten to try and pass all the core gates. I guess we can move this discussion to #2784921: Introduce the concept of workspaces.

alexpott’s picture

One thing that is becoming apparent is that we need to update our entity hook documentation so that people who implement them are aware about some of the issues with forward revisions. #2856363: Path alias changes for draft revisions immediately leak into live site and other issues are occurring because of the assumptions the code is making about the latest revision always being the default revision.

xjm’s picture

Issue summary: View changes

To stabilize Content Moderation, we also need to address bugs with revisions (especially forward revisions) that become worse when they are exposed by Content Moderation. #2856363: Path alias changes for draft revisions immediately leak into live site is one thing that was identified during initial Content Moderation reviews but lost somewhere along the way. #2766957: [PP-1] Forward revisions + translation UI can result in forked draft revisions is another.

Furthermore, #1239558: Deleting a node with revisions does not release file usage was supposed to be a must-have for Phase A, which was originally supposed to block this one I think. I actually think it was a good choice to start work on this without all of Phase A being done, but #1239558: Deleting a node with revisions does not release file usage is a bug that also gets worse when exposed to more users by use of Content Moderation. So I think that has to go on the must-haves here too. For that issue, current thinking is that the best way to address it and other file deletion issues is to explore a completely new solution for file usage management, because attempts to hotfix it and the related bugs have not been successful. If so, that means it also encompasses bugs that affect other initiatives, like Media and Migrate. Therefore, I'm adding it to the "Must" have issues here, but with a note that we should work with stakeholders from other initiatives to come up with a solution that addresses all initiatives' needs.

Adding these to the summary. Thanks!

ifrik’s picture

The help text of the Workflow and the Content Moderation modules still need to have the explanations on how to actually use the modules. I've made two issues for this: #2830832: Edit hook_help text for Content Moderation module and #2861849: Edit hook_help text and module description for Workflows module assuming that this is the right place to link them.