This is a research spike to identify possible backend implementations needed to support auto-save, drafts, and publishing features since there is no "Save" button in the UI. It belongs to #3455753: Milestone 0.2.0: Early preview.
Overview
This is a research spike to identify possible backend implementations needed to support Auto-save, Semi-collaborative editing, Saving a draft, and Publishing Experience Builder work. since there is no "Save" button in the UI. It belongs to #3455753: Milestone 0.2.0: Early preview.
Proposed resolution
Leading with the proposal on account of the length of the rest of the issue summary... Per #28:
- Use Autosave Form to handle automatically saving XB data into tempstore, modifying it as necessary to account for the fact that most XB operations don't involve actual Drupal forms.
- Add the ability to display details of the auto-saved state elsewhere in the CMS.
- Optionally modify the module to use shared tempstore as opposed to its current per-user implementation.
- Possibly extend it to attempt to validate an entity on auto-save and indicate to the content creator whether the entity can be published in its current state or would be prevented by an error.
- Possibly use Content Lock to handle the non-concurrent editing problem when two users are trying to use the XB interface at the same time for the same entity.
- Use Core Workspaces to handle reviewing sets of changes across multiple entities before publishing them.
- Possibly implement #3095414: Add the concept of workspace providers to completely hide unnecessary UI elements without taking away the ability of advanced users to use full Workspaces functionality.
Requirements
The specific requirements described below come from @lauriii. Some necessary assumptions have been made pending clarification from him.
Legend
🔴 = High priority question / 🟡 = Unconfirmed / 🔵 = Confirmed but unclear or ambiguous / 🟢 = Confirmed and clear / ⚪ = n/a (just for formatting)
Features
Feature: Auto-save
"Auto-save" refers to the behavior where the state of a content creator's work in Experience Builder is periodically saved (to the server) so that it can be resumed later, on the same computer or a different one, by the original content creator or someone else. This requirement does not seem to have a corresponding user story in #3455753: Milestone 0.2.0: Early preview.
🟢 Rule 1: Auto-saving takes place automatically
🟢 Scenario 1.1: Event-based
|
🟢 Rule 2: Auto-save states are automatically resumed
⚪️ Background:
🟢 Scenario 2.1: Close and re-open the same browser
🟢 Scenario 2.2: Refresh the browser
🟢 Scenario 2.3: Open a new browser elsewhere
|
🟢 Rule 3: Auto-save states can be transferred
⚪️ Background:
🟢 Scenario 3.1: From one user to another
🟢 Scenario 3.2: From one computer or session to another
|
🟢 Rule 4: Auto-save states can be simultaneously maintained for multiple entities
🟢 Scenario 4.1: Working in multiple entities concurrently
|
🟢 Rule 5: Unpublished XP progress can be abandoned
🟢 Scenario 5.1: Deleting created entities when abandoning progress
|
🔵 Rule 6: Related entities created in Experience Builder are cleaned up if progress is abandoned
🔵 Scenario 6.1: Deleting related entities when abandoning progress
|
Non-functional requirements
- 🟢 It must be fast enough that it doesn't degrade the Experience Builder user experience.
- 🟢 It must not degrade overall site performance during use.
- 🟢 It must not create long-term performance and scalability or operational costs problems, such as database inflation.
Assumptions
- 🟢 The undo/redo stack will not be captured in auto-saves and will therefore be lost when resuming a saved state.
- 🟢 Revisions-handling behavior will be unaffected by auto-savings--it will behave just like normal.
- 🟢 Only one user at a time can be working on a given entity in Experierce Builder. (Others can observe.)
- 🟢 Workspaces can be a hard dependency of Experience Builder.
- 🟡 Experience builder will be installed via Recipe.
- 🟢 Only one user at a time can work on an entity in Experience Builder, and the entity will be locked and all changes to it forbidden as long as there's an active session.
Constraints
- 🟢 Content Moderation can not be a part of the solution due to the long term desire to phase it out in favor of Workspaces.
⁉️ Open questions
- 🟡 What happens if the same user tries to edit the same entity in Experience Builder on two different computers (or browser tabs)? For example, what if a user goes back and forth between home and the office?
Feature: Semi-collaborative editing
Semi-collaborative editing refers to the ability for one or more users to view the work-in-progress of another user in Experience Builder without being able to edit it themselves. This is not true collaborative editing, which would entail multiple users actually making concurrent changes to the same content and is out of scope for this issue.
🟢 Rule 1: Users can view others' work "live"
⚪️ Background:
🟢 Scenario 1.1: Viewing a list of others' in-progress work states
🟢 Scenario 1.2: Viewing someone else's in-progress work
🟢 Scenario 1.3: Switch to and from watching and editingThis is just Rule 3: Auto-save states can be transferred considered from another perspective.. |
🟢 Rule 2: Access is controlled by permissions
⚪️ Background:
🔵 Scenario 2.1: Listings
🟢 Scenario 2.2: Direct access
|
🟢 Rule 3: Only one user can edit an XB-enabled entity at a time
⚪️ Background:
🟢 Scenario 3.1: Listings
🟢 Scenario 3.2: Direct accessSee Rule 3: Auto-save states can be transferred. 🟢 Scenario 3.3: Unlocking
|
⁉️ Open questions
- 🟡 Obviously, this raises UI/UX questions. Does it have any architectural implications germane to this issue?
Non-functional requirements
- 🟡 Viewers must see updates "fast". (How fast, exactly?)
Assumptions
- 🟢 True collaborative editing is out of scope--only one user will be able to actually make changes to the sae piece of content at once.
- 🟢 Viewers will not see the content editor's live interactions with the UI as if watching over their shoulder, e.g., drag-and-drop animations--only frequently updated snapshots in time.
- 🟢 UI affordances and exact UX details are out of scope.
- 🟢 Viewers will only be able to see live updates within the Experience Builder UI. (The behavior of revisions and published content will behave as usual.)
⁉️ Open questions
- None presently.
Feature: Saving a draft
The user should be able to mark their work in the Experience Builder for a piece of content as a draft that is visible to all users with a permission.
Assumptions
- 🟡 Saving drafts in Experience Builder is basically synonymous with creating revisions and will work essentially the same as the existing functionality in core today in terms of architecture. (Is there more to it than that?)
⁉️ Open questions
- 🟡 Obviously, this raises UI/UX questions. Does it have any architectural implications germane to this issue?
Feature: Publishing Experience Builder work
The user should be able to publish work in Experience Builder that then will be viewable by all users with the correct permission.
- 🟡 Publishing Experience Builder work in Experience Builder is basically synonymous with publishing any other content entity, e.g., nodes, and will work essentially the same as the existing functionality in core in today terms of architecture. (Is there more to it than that?)
⁉️ Open questions
- 🟡 Obviously, this raises UI/UX questions. Does it have any architectural implications germane to this issue?
User interface changes
Doubtless a lot. Details are out of scope for this issue except insofar as they bear on the architectural questions at hand.
Comments
Comment #2
tedbowLeaving assigned to myself because I still working on the summary to describe the issue.
Unfortunately we don't have the auto-save feature I am describing for issues on drupal.org😜
Comment #3
tedbowComment #4
tedbowComment #5
tedbowComment #6
tedbowComment #7
tedbowComment #8
tedbowComment #9
tedbowComment #11
effulgentsia commentedAdding @hchonov (maintainer of https://www.drupal.org/project/autosave_form) to this issue per #3462360-20: [PP-1] Partial preview updates: update preview of modified component only, not entire component tree.
Comment #12
catchContent moderation has severe limitations - for example if media, taxonomy terms (via tags autocomplete) or other similar entities are created within experience builder while editing a node, it is not able to track their publishing state along with the node, so the media entity or taxonomy term will 'leak' into the live site.
These problems are fully solved by workspaces, so I'd strongly recommend looking at that first especially since it's stable in core now. Workspaces also supports full-site preview, so once a draft revision of an entity is saved, it can be properly viewed in context in the rest of the site - including in views listings and any other contexts it might appear.
The disadvantage of workspaces at the moment is that the 'workspace' concept is always exposed, but #3095414: Add the concept of workspace providers is already open to address this, and would eventually allow for a lite/transparent interface for workspaces that could fully replace content moderation.
Another reason that auto-saving can't do a full entity save is validation. For example when first creating a node in a form, the entity is not a valid node (no title for example), so entity validation would fail. Saving without validating would create invalid revisions which if they were ever published could break custom logic, lead to information disclosure, or might not even be allowed with the database schema. Even once an entity is valid, it could be temporarily invalid again in the form state - e.g. if I delete the current content of the title field to replace it, the title field will be empty but required and autosave would be blocked (or risky) until there's something in there again.
Comment #13
tedbow@catch thanks for the feedback.
It seems like the drawbacks Content Moderation that Experience Builder would face such as entities "leaking" into the live site would be similar to just creating a node saving it as unpublished. Any media entities or terms would still "leak" into the live site.
I am not up-to-date as far as how much of hurdle it is use workspaces on a site. Is there friction as far as site building, contrib compatibility, or custom code? I won't want it to be a hurdle to people using XB if it was a dependency
My understanding from the reason Workspaces was originally developed was or organizations that had pretty strict requirements for this type of entity "leaking" into a live. I am not sure if everybody who wants XB would have the same requirements
Maybe Workspaces could be the suggested method for sites that have this concern. We could ship a recipe or a XB/Workspaces integration module
yep once we expose entity field editing within XB this will start to be a problem
I am not sure if the XB UI will actually have to be an entity form. We could do the auto-save and even a real save via a JS call like are we doing now with the
experience_builder.api.previewroute which calls\Drupal\experience_builder\Controller\SdcController::previewand sends it thelayout, which maps to theComponentTreeStructureon the back-end, and `model`, which maps to `ComponentPropsValues`. The client could send other data related to the entity fields that are updated.Right now the front-end react UI doesn't really have a concept of XB Canvas is a field on an entity. It mostly just knows about component, props and slots.
Comment #14
catchDisclaimer that I haven't actually used the Drupal 10 version of workspaces in production, but it's completely transparent - the only case where it wouldn't be is hard-coded entity-like or entity-related storage that does not use the entity revisions/views/entity query system properly. The only friction is that you have the workspaces concept exposed, vs. 'only' workflow in content moderation. Recipes could probably smooth out any initial set up now.
It's only partly about the 'leaking', but as much about being able to publish sets of changes at the same time (which IMO is potentially useful for any site bigger than a single-person-blog), and having whole-site-preview so that changes can be reviewed together and in context (relevant for read-only collaborative preview if not quite collaborative editing as such).
Comment #15
lauriiiIt seems like you are only listing the drawbacks of the alternative solution. Have you considered what are the limitations and drawbacks of the current solution? For example, are we able to show these auto-save states in the content listing for when the content was last updated?
At minimum, we could have a soft dependencies like this. If we can provide better experience by having a hard dependency, we could consider that too.
I agree with @catch that Workspaces from UX perspective would likely to be closer to the ideal experience (in the long run). With Workspaces, it is possible to edit multiple things and preview them together. For example, when I make changes to a design token as well as components, it's likely that those changes would be better reviewed together. Same as when we make code changes; if we were limited to making changes just in a single file per MR, it would be really hard to actually review changes. I think we should at least consider this as an option since we believe it could provide the best experience.
FYI, there's an existing implementation of this in the Blokkli module which may be worth reviewing: https://git.drupalcode.org/project/paragraphs_blokkli/-/blob/1.0.x/src/E....
Comment #16
tedbow@lauriii re #15
It is not clear that was actually a requirement. It feels like we don't have information of what the purpose of auto-save is.
Is it so that someone doesn't lose their work in XB if they forget to save or is so that content is save that acts like all other content, i.e, appears in lists.
If the put the Semi-collaborative editing section under Auto-save but maybe that was mistake. Maybe these are really 2 different requirements? Do they both belong in this issue? Is one more important in the near term the other?
From technical some pro/cons of 2 ways we could Auto-save
Saving the state of the front-end application to a tempstore
Pros
Cons
Saving the entity in real revisions using a new Auto-save state with Content Moderation
Pros
Cons
Workspaces also needs pro/cons but I don't have time right now
I could see us using a combo of both these approaches.
If we need just an auto-save feature right now we could implement with Content Moderation and an auto-save workflow state. If we don't need the features I listed under Semi-collaborative editing we could push this off until later. We could plan to implement this with something other than real entity saves but if we are able to somehow making this as responsive as people expect and solve the validation issue with real entity saves then we may be able to use the same method.
Comment #17
effulgentsia commentedI haven't caught up on the comments in this issue, so this is just a drive-by idea, but I wanted to post it here in case it leads to any good discussion or helps spark other ideas.
The key problems with an auto-save saving an entity "for real" (whether as a draft revision or in a non-live workspace) as opposed to "just" in the temp store are:
So here's a possible idea for mitigating the above by combining Workspaces and Temp store:
This would mean that even without the Workspaces module enabled, we still get auto-saving to the temp store. But with Workspaces enabled, and when in a workspace configured to allow auto-saves, we also get real entity saves happening for valid states and protected from stampedes without having to slow down the rate that we're able to save to the temp store.
Comment #19
tedbow@effulgentsia interesting idea.
I do wonder if Workspaces is really a small thing for existing site to turn on. I think #17 kind of assume it is because otherwise you don't get "real" entity saves and the pros that come with it.
From @catch's comments it also makes Workspaces seem like such a low barrier that requiring for basic functionality seems like "no big deal". But when I try to find people that Workspaces on actual sites I haven't really found that many. Maybe it is just recently stable so that is why? I have found people that thought about using but then decided not to though.
It just seems like we already have the need "Saving a draft" that is mentioned in the summary. This seems to be a very good fit for Content Moderation and this seems to be a module many more have used in real site. It seems like it would also work well for auto-save.
Why we would want to require Workspaces for very basic functionality such as auto-saving content. I know #17 proposes an idea where you get some form of auto-save without Workspaces and then an additional feature if you have Workspaces on. To me it just introduces yet another decision a site builder has to make when deciding if they want to use XB. Or maybe if we don't require Workspaces, people will just get an auto-save feature that is suboptimal but they would never think to have to install Workspaces or another module to get the optimal auto-save functionality .Auto-save just seems like such a basic piece of functionality that we should make the default experience be what we, or the product/ux XB folks, think is optimal. You should have to go out of your way to get the non-optimal functionality not the other way around. I know there might be different "optimal" functionality for different sites but I feel XB should have a very strong opinion on this.
What do we expect the default way to get XB to be? Is it just installing the XB module? Is it a recipe that has XB and other modules? Are we willing to make XB dependent on Workspaces, would that stop some people from using XB?
If default way to get XB is a recipe with Workspaces(or Content Moderation if deemed better for auto-save) but it is not an XB module dependency, how do we stop a site builder from uninstalling Workspaces?(assuming they didn't know they were changing the way auto-save in XB worked) We could make an XB auto-save module(credit @travis.carden for idea) that depends on Workspaces that would be install with the default XB recipe
Or maybe just saving to the tempstore is the optimal auto-save functionality and other add-ons are for special cases?
Comment #20
johnwebdev commentedI’ve rarely seen this as an actual problem on the websites I’ve worked on. Neither on different CMS that have same limitation, like Contentful.
Comment #21
tedbow@johnwebdev thanks for the input. My suspicion is that your experience is probably that same as many people but I have not done site builds for a long time so my experience is not current. But I think if we require Workspaces for the optimal Auto-Save experience, whether is actual module requirement or not, we would be adding a lot of complication to solve a problem that is a very real for some site but not for a majority of sites.
Working with Workspace does seems like it changes a lot things about working with Drupal site. I know this is for good reasons but not sure it would the problem all or most XB users have
Comment #22
wim leers#limit_validation_errorsor Field API's\Drupal\Core\Entity\EntityConstraintViolationList::filterByFields(). Although you're right that some field/entity DB schemas may simply not allow certain invalid values. But doesn't that mean that Content Moderation and Workspaces automatically have to be discarded as possible implementation methods? 🤔 Because autosaving should work from the very start, not only once the entity is in a valid state…/admin/content? This could be solved using a new computed field on all entity types, or even just an additional computed property on an existing field type.Comment #23
tedbowre #22
I think that would @catch's argument(just based on my reading of comments, no other info) and maybe that will be the way we will go. But I am not convinced yet, so could hold off making those changes to the summary.
I will try to post comment soon with more of the reasons not to do this
Comment #24
traviscarden commentedI'm going to try to process all the details here and update the issue summary.
Comment #25
catchYes, this is why I brought up autosave_form in #3462360-20: [PP-1] Partial preview updates: update preview of modified component only, not entire component tree after going through much of the same thought processes that are happening in this issue in June - you just cannot do autosave as-such with entity revisions. There are other things you can do with entity revisions (like full site preview) but it is not the same thing, and the two can work together (i.e. autosave can preserve your edits, before you then save a new draft revision).
Comment #26
tedbowI would like to suggest that we do something like @effulgentsia's proposal in #17 but replacing Workspaces with Content Moderation.
Copy/paste from #17 with replacements
So here's a possible idea for mitigating the above by combining Content Moderation and Temp store:
If these are true then unless we want to make Content Moderation a hard dependency of XB then
In both cases we would need to decide whether we always want to create a new revision or want to be more careful about keeping many revisions. Beside DB size, one reason not to keep all auto-save revisions is the user might want to revert back to a previous state later, if we have an auto-save revision for every 30 seconds it could be overwhelming to find a point to revert to.
If want to minimize the number of revisions we could locally save the revision id of the last auto-save and not create a new revision if most recent revision matches.
My reasoning is:
I agree if implemented very well, the integration could provide the best experience and that XB should be very opinionated about the best UX.
My concern is another goal of XB is to unify the page building ecosystem in Drupal. I know creating UX that is better than anything currently in Drupal will be part of how we unify, but I think if we make it too difficult for existing users to use XB or XB requires a too specific way of building sites then we risk the page building ecosystem remaining very fractured.
@johnwebdev in #20 is an example of someone who hasn't seen the problem Workspaces solves (over Content Moderation) as an actual problem that needs to be solved for their sites. On the other end if someone does have the problem of entities leaking into the live site or wanting to publish a bunch of content entities in one batch will they may not want to solve it in the way that Workspaces provides. There are numerous contrib modules that handle content staging between actual Drupal environments. Do we want to try to lure current users who are using these solutions, which they may be heavily invested in, to use XB on their current or next site? Will it be harder if they have to also use Workspaces for XB to have a decent UX(including auto-save)?
I know @effulgentsia proposal in #17 would not require Workspaces but I think it effectively requires it because very basically functionality, auto-save, that users would expect to just work requires it.
This would allow sites to not use the recipe if
But again we would recommend the recipe that Drupal CMS should also be used as the default way to use XB even if you didn't start with Drupal CMS.
Comment #27
traviscarden commentedOkay, I've fleshed out the requirements for the Auto-save feature, and this is on track to become the longest issue summary in history. 😬 (I blame d.o's CSS for the odd markup choices I made to make it more readable.) I plan to do the same for each of the other two features and then distill the discussion in the comments. Also, it's past 1:00am my time and too late for proofreading. Here's hoping for the best. 😛
Comment #28
longwave@longwave, @lauriii, @catch and @hchonov discussed this issue at Drupalcon Barcelona. We agreed that we need a combination of solutions for the product requirements for autosave functionality - but the good news is that much of what we need to implement this already exists.
We should be able to use Autosave Form to handle automatically saving XB data into tempstore. It provides an endpoint that XB should be able to talk to with little or no modification. It does not yet support displaying details of the autosaved state elsewhere in the CMS but this should not be difficult to add. It also currently uses per-user tempstore but converting it to (optionally) use shared tempstore should not be a problem.
This will allow us to save valid or invalid entities. We may want to extend it to attempt to validate the entity on autosave in order to indicate to the content creator whether the entity is capable of being saved or if there is an error that would prevent publishing.
We may be able to use Content Lock to handle the non-concurrent editing problem when two users are using the XB interface for the same content item.
As XB may involve editing multiple entities (content or config) at once, we can use Workspaces from core to handle reviewing sets of changes across multiple entities before publishing them. We may want to implement #3095414: Add the concept of workspace providers to move Workspaces entirely behind the scenes so the content creator does not need any knowledge of workspaces - but advanced users will still be able to use full workspaces to e.g stage multiple sets of changes or go through longer review workflows.
We have ruled out Content Moderation as there is a longer term desire in core to phase out CM as it currently exists and replace it with a simple implementation based on Workspaces.
Comment #29
traviscarden commentedUpdating the issue summary to reflect the elimination of Content Moderation as an option per #28.
Comment #30
traviscarden commented...and again to updated the proposed resolution. I'll leave it assigned to me to continue fleshing out the distinct user stories for drafts and publishing.
Comment #31
catch@lauriii gave a very quick idea of how he envisaged XB editing to look like, and it looked very, very similar to how workspaces works already, which was both encouraging and surprising.
I had been assuming we'd want to hide the workspaces concept almost entirely for a default UX (which is the main missing piece from what it can already do compared to content moderation), but @lauriii said things like 'but what if I want to edit both the site menu and tweak a node from one page in XB and then publish both changes at once'. XB can't do that yet, but workspaces can already do this including config entity changes (in the wse_extras contrib module), so it is just a case of getting those things to meet.
The main UX difference was that the list of affected entities was shown right on the XB edit form, and it was possible to publish the changes directly from there without needing to go to the workspaces UI as such. We still might still want to hide the 'workspace' as a thing separate to the current set of edits itself (so still need #3095414: Add the concept of workspace providers), but not the concept of reviewing/pubishing/staged changes and the fact that multiple entities are affected - just localise it to the XB form when you don't have a separate review workflow. This all seems like less work/changes than I was thinking it would be - not that much of a jump at all.
Comment #32
tedbow@longwave thanks for the write up!
Right now the XB ui is not a Drupal Form so we would have to make it Drupal Form to take advantage of AutoSave.
Right now the front is doesn't use the Drupal form except for the side bar but not the complete tree and model values. In fact when in the UI and the sidebar is not open there is no
<form>on the page at all.From the discussions with @wim.leers it sounds like we pretty much have all we need in data currently passed from the client for the
previewroute to perform a tempstore and/or entity save, at least for the XB field.I was just wondering if it would make more sense for the JS client to save/auto-save via an API call.
We may get some functionality for free if we use a drupal form but also there would probably be some work to get there instead of js api call from the react app
Just initial thoughts, but my worry with the front-end becoming too “drupal aware” is that we start limiting the people who can work on it. Or you start to need a combination of React Skills and Drupal knowledge(probably not novice either)
You will already need modern JS skills we can’t go back on that(and probably a good thing) but my guess is right now you don’t need a lot of drupal skills to work on the Front-end app.
Also have thoughts on Workspaces but will put that in another comment...
Comment #33
tedbowI talked with @lauriii, @wim leers, @traviscarden and @effulgentsia about the Workspaces requirement.
I brought up my idea from #19 again of making a recipe that would require Workspaces to provide UX we want with full site preview but not making Workspaces a depdency of XB itself.
Pretty much everyone in that list of folks was concerned about ever complexity it would take to support both that case where Workspaces was enable and when it was, which is a valid concern.
@lauriii suggested that we could make Workspaces a dependency of XB now and if it is determined later making Workspace a dependency of XB is big barrier for many sites using it we could change this decision before it is stable.
This seems like a reasonable idea to me. I still have my concerns about making Workspaces a dependency but
If we do decide to depend on Workspaces I think it would great make it a dependency as soon as possible to make sure it the direction is as clear as possible. If we only make it dependency after we have spent months to make integration really nice that could shorten that time people to raise concerns before XB becomes stable. I assume the people how are following this issue or would be somehow "in the loop" to know are plans will be a very limited number of people.
If we intend that you should only use XB inside a workspaces then could simply:
xb/node/1to see if they are in a Workspace.It doesn't have to be exactly like this but it would good to have follow-up like "Ensure XB can only be used in Workspace" ASAP if we make this decision that will force us to start using it. My guess is providing the optimal UX for Workspaces + XB will take a lot of work in XB, Workspace, and possibly Workspaces Extras. It would great if we could be practical with this part of module like we have with other parts, like the Canvas UI demo that doesn't allow saving, to get working with Workspaces in even rough way first. Though we shouldn't stop saving if you are in workspace.
Another concern I want to raise about relying on Workspaces is that if want to also be able to edit and stage Config changes as @catch mentioned in #31 we need to rely on Workspaces Extras. While amateescu, the Workspaces core maintainer, is also maintainer of this and it is under active development which are really good signs, it doesn't actually have a published release and as far as I can tell only has 2 tests(functional only). If config support is must have we will probably need to factor in work on WSE or at least get a confirmation on the modules state.
Comment #34
catchWorkspaces is slightly different to a lot of experimental modules in core. It's been used in production on at least a handful of large production sites including at least one fortune 500 company for at least a couple of years or more. The last handful of patches that were required to get it to stable were also run on production for several years. The reason it took so long to get to stable was because the company that originally pushed towards getting it into core withdrew support for it more or less at the last minute, and then other companies started using it after that, but the core momentum was lost for some time in between. There were also issues with the initiative that extended the timeline, which I'm still bitter about, but too late to fix those now. Me and Fabianx were also running workspace's predecessor on production on Drupal 7 as early as 2015 (while the name is different, CPS, the architecture is almost identical).
All of this to say the data model and functionality is very stable, it just took a long time to be able to flip the core flag.
However this very long timeline has definitely affected wider adoption and I'm pretty sure there will be usability issues that will have to be fixed both for Drupal CMS and XB. But... both Drupal CMS and XB picking it early means there is time to identify and solve those issues starting from now, rather than in a year's time having run out of options with content_moderation and having to start later, so it should result in a much quicker timeline overall.
This is true, but the config staging is holistic (as is trash module support), so while there will be work involved to get config workspace support into core and stable at some point, it should be a lot less work than trying to implement config staging within XB without the 10+ years of work that has already gone into it. Trying to implement config moderation with content_moderation would be a complete non-starter.
100% agreed with this. I think the easiest way to approach things would be to just make workspaces a requirement, and then add some basic configuration so that people installing XB don't need to set up any workflows etc. and live with exposing the workspaces concept for a while. Then hopefully some of the workspaces UX work can happen in parallel, and not necessarily directly in XB (although the final touches might need to be if it's in react).
edit: to put the above differently. The risk here is from the requirement of draft, workflow, autosave, potentially multi-user editing as first-class functionality in experience builder. I agree these are good problems to tackle, but if they are going to be tackled,then the lowest risk way to do this is to use workspaces (and probably autosave or at least the same conceptual distinction between form and revision data that it has). Any other approach risks repeating all the moderation and workflow mistakes and complexity of the past 15 years all over again - the problems of which CPS/workspaces was created to solve.
Comment #35
traviscarden commentedComment #36
traviscarden commentedOkay, my updates to the issue summary are complete. I think I've captured more-or-less everything. Assigning to @lauriii for input on the unconfirmed requirement details.
@lauriii, we've documented our current understanding of the requirements in the issue summary. We would like your confirmation so we can confidently finalize our architectural decisions. Every detail we've identified is marked with a colored circle indicating its status (confirmed or otherwise). (See the legend.) Basically, anything that isn't green needs confirmation--mostly from you.
Incidentally, the issue summary is now rather massive, and I could see it being extracted to its own issue for reference and discussion, if desired. I'll happily do that if asked.
Comment #37
catchThe workspaces DrupalCon session is now online and even for me who helped build the initial workspaces implementation and reviewed much of the core work, I had no idea what was implemented in workspaces extras so it was useful to see it all pulled together into a demo https://www.youtube.com/watch?v=bgJ-oubGjbc
Comment #38
catchCouple of comments/questions on the issue summary:
The wording of these user stories assumes that the entity has already been explciitly saved (i.e. exists as an entity). autosave_form also handles the case where nothing has been explicitly saved - in this case when you go back to the node/add/* form it will tell you there's an unsaved draft to resume. I don't know if that is an XB requirement but it would be good to explicitly rule it in or out, currently it's not mentioned.
This could potentially be done with trash module which doesn't depend on workspaces but does integrate with it. It does add another layer of things but it implements soft delete + restore + purge. It can also be tackled by moving entities selectively into a different workspace so some can be deleted and some can be preserved.
A lot of the semi-collaborative editing section can be achieved in workspaces (e.g. you can send a link with a token to an anonymous user and they can see what you're working on in the workspace, or an auth user with permissions can browse workspaces)... but this does not include being able to see the editing UI live, which I think is what it's point at (?) - and I agree with the questions about discoverability - would they just be able to browser to the node/n/edit form, but then it would need to be a read-only version of that form.
I think this is now just workspaces - except there's a question whether XB (or Drupal CMS) provides default workflows for that - i.e. a separate review stage doesn't make sense for a single-user site, even though they might want to save in-progress drafts and preview them sometimes.
Comment #39
larowlanDid we give any thought to CRDTs here?
Comment #40
traviscarden commentedCRDTs = Conflict-free replicated data types. Some interesting discussion on Drupal Slack.
Comment #41
cosmicdreams commentedSame Page Preview reuses NodePreviewController's method of using TempStore to persist the FormState of an unsaved node.
What's being discussed here appears to be a decision to use workspaces and revisions instead of tempstore when building something with Experience Builder. I presume the "something" here will be stored as an entity somehow, but maybe that something isn't a node.
My question is, when we consider nodes, are we talking about modifying the underlying mechanism for storing unsaved nodes, so that they are previewed?
Also, when previewing nodes, is there the possibility of reusing Experience Builder's preview capabilities? In the XB future, does thinking about content as nodes even matter anymore?
Comment #42
catchNo not quite..
At the moment, the current state of discussions leads to something like:
When an item is explicitly saved -> new revision.
When an item is autosaved -> autosave_form (similar but not identical to temp store).
This is because an item that is autosaved may not be a valid entity - it could have required fields missing, so it can't be an entity revision in a useful way. Similarly on the node/add form, there is no entity yet to save a revision for at all.
Comment #43
cosmicdreams commentedAh yes, incomplete data.
When we think about auto save, data that is incomplete and would fail validation, are solutions that persist data on the frontend able to be a candidate solution?
I'm not familiar with yjs or other solutions of that class. But even something as simple as JavaScript's local storage might provide a means to store complex data temporarily
Storage API
Indexed DB
And React is in the mix so, I guess Redux
All provide various advantages / use cases.
Do we have to auto save to the backend? Is there no way we could solve this on the frontend?
Comment #44
catch@cosmicdreams if we never autosave to the backend, then there is no ability to start something on desktop and resume it on phones. I use that feature on slack/gmail at least and even if there might be less of a use case for a website than an email draft, people will know it exists and potentially miss it if it's not there. Similarly if your laptop runs out of battery or browser updates etc. Collaborative editing will also have to save to some kind of backend even if it's different/evolves from the autosave one too.
Comment #45
longwave@cosmicdreams there is a product requirement:
While full collaborative editing is some way off, autosaving changes on the server side means that other creators could still see a preview of something that a creator is working on without them having to explicitly publish it, in near-realtime.
I built an initial proof of concept using tempstore that allows saving and reloading of the current layout: #3478299: Implement basic auto-save
I think CRDTs will probably become useful and perhaps a requirement in order to build full collaborative editing, but to start with I think we can introduce single-user editing and locking which would still allow collaborative preview.
Comment #46
traviscarden commentedGood thinking, @catch. I assume it is not a requirement since, as far as I know, there's no way to use Experience Builder except on a piece of content that already exists.
I've updated the issue summary to include this assumption.
Comment #47
johnwebdev commented#46 Maybe laurii can confirm? Personally I think starting in XB when creating a new page would be better UX than first having to create and save a node before getting started.
Comment #48
traviscarden commentedA lot of the details in this issue (that assumption included) are pending confirmation from @lauriii. See the legend for interpreting the status markers.
Granting our assumptions, it seems like we're more or less in agreement on the proposed resolution at this point. (Tell me if I'm wrong.) I'll see if I can get @lauriii's eyes on the issue, and in the meantime, I'll find a good place in the codebase to document our architectural direction. Since the scope of this issue is research, that should resolve it, and we can create follow-ups to start on implementation.
Comment #49
lauriiiLooked at this with @bostonjillian and @Renee Lund and got through the auto-save section.
We want users to not have to think about this action, which means that it needs to be close to immediate (~few seconds). I guess based on that, it should probably be based on when changes are being made?
There should be a way to delete the draft by using a delete action but once the user has entered the content form and inserted any content there, we should retain the saved draft.
From UX perspective, there might be different expectation between the two examples given in the user story. Media library is often something that is considered to be separate from the content (i.e. page) that is being created. This is highlighted by the fact that it opens up in a modal dialog and has explicit steps for uploading and saving. This is different from the use case of Taxonomy Term because in that case you're just adding a tag inline using an autocomplete.
I don't think this is the right assumption. Content types that are using Experience Builder should open in XB even when creating content.
It should be possible to revert back to previous revisions and auto-save states. It would be okay if auto-save states get truncated into a single revision once content gets published.
I think we need to work with the assumption that we depend on Workspaces for now. It doesn't seem that the UX of going with content moderation would meet the needs. I don't think we need to implement support for using XB without Workspaces now, but it would be great if we could build this in a way that we could reconsider this decision closer to the release. This way we could add a way to use XB without Workspaces later if needed.
We could do something similar to: https://www.youtube.com/watch?v=CsRBypaT_hM&t=27s. We could apply this both for the scenarios where the user is the same and where it's different.
Is this relevant if we do above ^?
Comment #50
catchOne extra question here - is this only when the content is actually published, or is it also when it gets sent for a workflow step like review? I think autosave_form would currently truncate drafts whenever the entity is explicitly saved. In the Barcelona meeting, we briefly discussed saving a new forward revision as part of the autosave when the entity is valid (or every so often when it's valid), this would provide a partial interim-history of autosaves which could more easily be preserved for longer (but still with some metadata noting they're implicit rather than explicit revision saves).
fwiw I think that aspect is something that could be experimented on once the initial implementation is up and running, feels like there are a couple of options and it will need to be built on a working foundation with some trial and error both in terms of UX and implementation. But also good to think ahead to it.
Comment #51
traviscarden commentedRe: #29
I don't understand. I assume "saved draft" means an auto-save state. Is that right? What do you mean by "content form"?
But you can also configure Taxonomy term fields to "create referenced entities if they don't already exist". The point is that it’s possible to create separate entities from a node edit screen. Does the rest of the scenario look right?
Okay, so when you create an XB-enabled node, for example, there is no standard
node/addform at all? You go straight to Experience Builder and that's the only way to interact with the entity? If so, that should be added to the assumptions. And we probably need a new scenario for editing the overall entity information--such as title, URL alias, and menu settings--from within Entity Builder. Would you agree?Wouldn't that mean that the "truncated" (meaning, squashed or merged, I assume) auto-save states are always identical to the list of revisions--making them effectively useless? Wouldn't that mean that every revision would have an identical auto-save state?
Given the workflow depicted in the video, I suppose that would mean a user on computer two would have to request control from computer one. And ignoring the logical challenges with that, it would mean that our auto-save states would no longer be associated with a user but rather with a browser session, right? Otherwise the concept of locking doesn't make any sense and we haven't solved the problem in question at all.
Now that I understand that only one user can be working on a single entity at a time, problem can't actually materialize, then as long as starting an editing session in Experience Builder locks the whole entity, this is irrelevant.
We're making great progress here!
Note: I'm updating the issue summary to capture what we've learned. I may have made some small mistakes, but it's the end of the day and I don't have time to give it a thorough proofread. I'll keep updating it as the conversation proceeds.
Comment #52
catchIf autosaves are always newer than the latest revision, it's much easier to think about because you have revision 1, 2, 3, 4, 5 then autosave 1, 2, 3, 4, 5, not revision 1, autosave 1, revision 2, autosaves 2 etc..
Then to have interim checkpoints between explicit saves to potentially revert back to, the idea discussed of implicitly saving a revision in the background when the entity is valid comes into play (with metadata so we know that's what happened) - while this adds complexity, it would still result in two linear histories of the entity/form.
If all the above happens, then autosaves can be garbage collected every time an entity revision is saved (whether this is explicit from a button or implicit when the autosave state is a valid entity).
Comment #53
catchWhile the end user might not need to know the difference between an auto-save state and a forward revision, I think it would be very good to use the terms exclusively in this discussion and not mix and match 'draft' to mean both things.
Comment #54
tedbow@lauriii re
to be clear, since there is a lot going in that video, the only part you are talking about is requesting hand off control, of the XB edit, correct?
Comment #55
effulgentsia commentedI agree with this sentiment, but I think picking the right words is tricky. I think we need to have terminology for three states. Taking an implementation-perspective, we can call these states:
However, as the UX for this hasn't been fully designed yet, and the product requirements not even fully flushed out yet (though this issue is making good progress towards that), I wonder if it would help the requirements/design process to also have product/UX-oriented terminology for these states rather than jumping straight to the implementation-oriented terminology.
Perhaps something like:
?
Comment #56
catchTo me it's not more accurate, because a 'non-live revision' includes past, previously-default, revisions too, and technically 'published' revision also includes non-live, previously default revisions, so we can't just use non-live and published. So we probably need to explicitly talk about four states/things :/
That would mean:
If we add 'old stuff' or 'published history' here too that probably covers it.
Comment #57
effulgentsia commentedHow about this?
Comment #58
effulgentsia commentedI strongly request that we descope "revert back to previous auto-save states" from the initial implementation of Save functionality and from the 0.2.0 milestone. I opened #3481006: Implement time travel (undo/redo/revert) of auto-saved states across devices and users to discuss and work on just that feature in its own issue.
Comment #59
effulgentsia commentedThe issue in #58 proposes to use a CRDT library for implementing that issue, both to eventually support collaborative editing, and also because those are the libraries/algorithms that also implement the most mature, robust, and efficient storage of large histories.
Comment #60
tedbowI agree with #58
I think to make safer to not implement the reverting to auto-saves now but make it possible later we should be very clear that our auto-save implementation is entirely internal. We should be free to change it later no have to worry about BC. Especially before 1.0 and if we still haven't done #3481006: Implement time travel (undo/redo/revert) of auto-saved states across devices and users by 1.0 we should continue to keep it internal so are free to support reverting later.
I agree that #3481006 would be a lot of work and it doesn't seem realistic to do it with all the product requirements we are trying to get done
Comment #61
tedbowI don't think I fully understand Rule 1: Users can view others' work "live" -> Scenario 1.2: Viewing someone else's in-progress work
Given 🟢 Rule 3: Only one user can edit an XB-enabled entity at a time and discussion we have had
I would assume that if
Then there would be no choice if I wanted to see the other user's progress in XB for node1 or not, I would automatically see their current work and when their auto-save happened my version would be updated.
To me it would make no sense for me start a "a new, distinct editing session" otherwise we have to figure out to merge the 2 sessions.
We have to work out "🟢 Rule 3: Auto-save states can be transferred" but think that would be where I would able take over the XB session for node 1 from the other user. I think after the auto-save state was transferred to me and I wanted a ""a new, distinct editing session"" I could either 1) clear the canvas or 2) revert to the last revision. But after the auto-save state is transferred to me then it would be no different that if I was the sole user editing XB node 1 and want to "start over".
Comment #62
effulgentsia commentedI opened #3481986: Lazily create draft revisions from valid auto-saved states on demand rather than on save or only on schedule as a child issue, with an idea for how we can make valid auto-saved states essentially equivalent (from a UX standpoint) to real draft revisions, but without the performance cost of doing a full entity save for every auto-save.
Comment #63
luke.leberChiming in as an early-ish adopter of content moderation. When we upgraded to Drupal 8, workspaces wasn't really a thing. We now have years of muscle memory built up around content moderation, having explained in depth to our users why it works the way it works as opposed to something like, say, the Oasis Workflow plugin for wordpress (which has been lauded above the Drupal tooling btw...).
In my experience, teaching the technical mechanics of a solution has been secondary to the "why doesn't this work like other workflow solutions that I am used to" barrier.
For us, it would be a very hard sell a switch to something completely different again with different justifications involved. If a motion to strong-arm use of a specific workflow tool is inevitable, I would urge looking at the most popular (by number of installs globally) workflow tools and to derive the end-user facing terminology from those tools versus Drupal implementation detail nomenclature.
Not sure if this is helpful at this stage of the game or not, but I wanted to get this point of view out there.
Comment #64
effulgentsia commentedRe #63, it's very likely that for XB we'll build an XB-specific UI for listing, previewing, approving, and publishing draft content, even if we use Workspaces as the storage model. Your comment is great input for the design work being done on that. If the designs implemented for XB are well received, they can potentially make their way upstream to the core Workspaces UI eventually.
By using Workspaces as the storage model, if people don't like XB's listing/previewing/approving/publishing UI, they could potentially replace it with core's Workspaces UI, or any other UI built on top of Workspaces, such as even Content Moderation if/when that UI is rebuilt on top of Workspaces (see #28).
For XB, Workspaces is only viable if it can also handle config entities. I opened #3493461: Identify roadblocks to staging config in Workspaces (e.g., via wse_config), so if anyone has knowledge to share there about
wse_config, that knowledge would help us enormously in planning out our roadmap.Comment #65
tedbowIt seems like this should be a stable blocker, at least to decide if we are going to use Workspaces or Workspaces Extra for 1.0.0. The other option would be to continue to use our current auto-save -> Publish all functionality.
It may be more practical to use the current functionality if we are willing accept some of the limitations.
Comment #66
lauriiiI think we defined the path for 1.0.0 mostly at this point. I would appreciate if we could complete #3512616: Document the reasons for not using Workspaces for saving in XB 1.0.0 which documents the current direction. That said, I don't think we need more research on this topic until later when we actually implement Workspaces support.