Stable release of Wysiwyg module, v7.x-2.4 please

I want to get there too, belive me. I'm currently at an improvised post DrupaCon sprint trying to accomplish exactly that.
Basically, I want TinyMCE 4 support in, which there is a pretty good and working patch for, but it looks really bad without some further modifications to the Wysiwyg core (for example, the toolbar runs off screen when all buttons are enabled because it puts them all in one group/row). There are also a couple of issues with upgrading existing editor profiles from TinyMCE 3 that really need some work or a large number of site administrators will get very frustrated at if they suddenly see a release which claims "support for TinyMCE 4", but isn't able to work with existing TinyMCE 3 profiles when they upgrade the library at the same time.

This is really important. Running dev sites on production is a security problem. We've got to be moving towards stable releases, particularly for very popular modules like this one.

How is running a -dev snapshot a security issue? A release would just mean tagging the code which is currently in -dev and give it a prettier name.

There are "no security advisories for development releases" https://www.drupal.org/security-advisory-policy

Therefore we can't leverage the security infrastructure in d.o when security problems are established.

It's not a best practice to update every time there's a new dev release. Formal releases make it way better for site owners to know when, how to upgrade their modules.

@mgifford, I 100% agree with you that this is important, but want to note that the whole sentence reads "That means no security advisories for development releases (-dev), ALPHAs, BETAs or RCs."

So while a 7.x-2.3-beta would help site administrators to know that there are better options than a two-year old Wysiwyg release, it won't directly help the security release situation. Indirectly it will help push towards a proper release.

TwoD - what do you need most from contributors to get there? (Obvious things are on the project page - review patches, send cash, help in the queues.)

Can we make this a (beta?) release co-ordination issue then?

@xurizaemon - true, true.. Thanks!

Help in the queues would be great. There are lots of issues where I've not been able to reproduce a bug and finding the exact steps (preferably from a clean install) to do that will make it much easier to find a way to fix it. If you can also make a patch (or reroll/update/improve/review an existing one), that would be even better.

THE task to get done before a new release is proper TinyMCE 4 (and CKEditor 4) support. The current patch works with new profiles, but working with settings for existing [TinyMCE 3] profiles is a bit unreliable.
I have a local code branch which basically introduces a "migration" callback, allowing editor implementations to alter profiles used with older editor versions before the settings form is generated or the editor is about to be displayed. This allows the settings form to always match the features of the currently installed editor version, while not throwing errors or losing settings because the names have changed or plugins have been renamed. To make this reliable each profile now stores the editor version it was created for.
I've got most of this weekend off so I'll post the code in a new issue for you to test, along with the existing TinyMCE 4 patch rebased on that. (Downgrading to an older editor version won't be supported, but it would be easy to add later if someone really needs it.)

TinyMCE 4 will still look pretty bad if a lot of buttons/plugins are enabled, but those could be rearranged with the alter hook if we can't get that in for this release. (I'd really like that though, spent much of the sprint time in Amsterdam making something similar to the D8 toolbar config area.)

I'm at work now so I can't spend too much time here, but I promise to upload that patch for TinyMCE 4 tomorrow this week.

Patch is finally up: #2375679: Improve profile management.
Didn't reroll TinyMCE 4 to use it yet, work came in between, but I think it'll be needed. Either way, I'm much happier with how the UI works with that patch.

Thank you for your patience.

Is this not a duplicate of #1953106: Roll 7.x-2.3 release of Wysiwyg?

Looks like it to me. Thanks Damien.

I'll move over some of the issues in the summary.

Sorry, I am just reconsidering this.. Think this should be the active issue as it's already defined as the parent.

Basically, I want TinyMCE 4 support in, which there is a pretty good and working patch for, but it looks really bad without some further modifications to the Wysiwyg core (for example, the toolbar runs off screen when all buttons are enabled because it puts them all in one group/row).

If it's of any use I fixed the toolbar issue with the following "hack" in the adminimal theme style.css:

/* Tiny MCE button fix -------------*/
.mce-menubar {
    clear: both;
}

.mce-toolbar-grp .mce-btn {
    float:left !important;
}

+1 to this. This is annoying to see the WYSIWYG module as the only module to be not up to date on the module update list....

This is a monumental task. Please understand that I'm not complaining and have the deepest respect for the WYSIWYG maintainers. I'm probably stating the obvious - the usage of this module is declining and perspective is that it is becoming obsolete. I'm a system builder, like many, struggling with the selection of a D7 WYSIWYG config with an eye to the future. CKeditor seems to be dominant for drupal and is highly customization/feature rich. Other modules interface with this and it also provides wonderful flexability. The major concern is that

1. This module usage is declining https://www.drupal.org/project/usage/wysiwyg (especially relative to https://www.drupal.org/project/usage/ckeditor)
2. Ckeditor library support is quite old (4.4.1 vs 4.4.7 according to: https://www.drupal.org/node/596966) and 4.5 is in late beta.
3. It's very difficult to keep up with patches and support a dev version in production

IMO this is the elephant in the room. This is one of the most popular D7 modules being used so I'd imaging others share this concern! Please consider sharing your thoughts on these issues on the summary page and help mortals like me understand how to avoid a path to obsolescence.

It seems that the CKEditor module is winning the Drupal editor fight (and drupal 8 will continue in this direction). Should we consider WYSIWYG as abandonned for future projects ?

@zmove Yes, I 100% agree with you. I just only use CKeditor module. I just can't understand why this module recommended release is from 2012...

@zmove and @rcodina - I don't think we can just abandon over 365,000 active installs without a plan. It could be a death blow for Drupal. Furthermore, there are many many modules that depend on WYSIWYG.

This is a question for @TwoD and other major contributors - perhaps the core team. My hope is that we can build some consensus, unity and a rational migration path forward.

Wysiwyg module is not abandoned, nor will I do so. It's still needed for D6 and D7 as there is be no backport of the new Editor module from D8.

The latest release is old, yes. It has its flaws but is not critically broken, as it works well with still supported versions of the editors.
The 7.x-2.x (and 6.x-2.x) branch is not yet in a release ready state becase I've not had time to do that on my own.
The -dev snapshots have support for the latest major release of CKEditor 4, and TinyMCE 4 support is not far behind. However, there were quite extensive changes to the packaging procedure of those two major editors, and their redesigns have shown a great need to improve the handling of plugins and toolbars, as well cleaning up the API (readability and AJAX-related stability).
I've spent most of the spare time I did have on trying to come up with ways to deal with that, while maintaining backwards compatibility and trying to not get stuck in similar situations in the future. It has resulted in a very large amount of local git branches trying various approaches, few of which were good enough to even ask for reviews.

A couple of people have recently stepped up and begun helping me both in pruning the issue queue and rolling patches, which is awesome. Thank you so much! I will get to reviewing your contributions ASAP!

This has allowed me to take time to refocus my efforts, break the huge branches I have down into more and smaller branches that can become good incremental patches towards a single goal.

That single goal, as indicated by many of the new meta-issues, is all about decent toolbar management. I don't want to go into too much details here as there are separate issues for various sub-tasks already, and it's already hard enough to keep track of where things are discussed.

Buttons are currently indistinguishable from plugins, which makes both the admin UI and internals very confusing to work with. We'll split those up into different concepts (which also allows us to improve the cross-editor plugin API, but that's really a secondary concern).

Once that is done, we can allow buttons to be sorted, grouped and split into rows. This lays the groundwork for making TinyMCE 4 look good with more than a few buttons enabled. Technically, CKEditor 4 needs this too, try using it without the 'use default toolbar groups' hack and you'll see what I mean. Heck, almost every editor needs this to look great.

There is one major task left though. Upgrading editor libraries - or I should say "upgrading the Wysiwyg profiles" - became more complicated with the 4.x releases of CKEditor and TinyMCE because of all the button/plugin/setting renaming. I can't guarantee trying to use a profile created for a 3.x library version with a 4.x library will work at all. We basically need 'hook_upgrade_N' for editor profiles.

Then we rebase the TinyMCE 4 patch on top of that, add in the profile upgrade path (both TinyMCE 3->4 and CKEditor 3->4), smooth the rough edges, tag a release and call it a day.

We'll of course fix as many other bugs we can while doing this, prioritizing by time and difficulty needed to fix.

Just to mention it, there's a slightly of-of-date backport of D8's CKEditor module available: http://drupal.org/project/wysiwyg_ckeditor

@TwoD thanks so much for this prompt and clarifying update! Perhaps you could link this comment to the module summary page (might I suggest under Further Documentation "June 1st 2015 stable release update" or somewhere more appropriate. Also, consider asking for specific help related to these issues.

You have no idea how valuable this kind of communication can be to mortal system builders :).

@DamienMcKenna, Thanks for the reminder about that module. It's not a full backport of the D8 Editor API though, but an alternative [experimental] implementation of CKEditor which still relies on Wysiwyg module's API for D7.

@sonicthoughts, Good point, added the link.

#20 - NOTE: That module does NOT provide image upload integration or support. Hope this saves someone the time I spent to figure that out.

@joshuautley, neither does this module. There are many other modules dedicated to that task so no need to cram that into Wysiwyg module as well. Several of them work well with Wysiwyg though.

#24 - I'm not being clear I suppose.

It would be nice if wysiwyg_ckeditor integrated with wysiwyg_ckfinder or even just the CKFinder lib. Make sense?

It has been over 3 years sience the last stable version of this module has been released. I have seem many sites running the dev release of this module. I am willing to help with any roadblocks in getting out a stable release. What issues should need the most attention to get out a stable release? Are there any features that we can hold off on implementing till 7. x-2.4 to help get a release out?

@chrissnyder, thank you for the offer, it's much appreciated.

I just got the opportunity to get back to working on Wysiwyg again, much thanks to getting settled at our new home and my new job.
I'll look over things this weekend and compile a proper list of what needs to be done for the release.

The main task for this release is TinyMCE 4 support, and that needs the plugin/toolbar management refactoring before it's ready for practical use. The integration for CKEditor 4 (and the other editors) will greatly benefit from these changes as well.

The largest hurdles are maintaining backwards compatibility while being flexible about which editor versions are usable. Support for really old versions will be dropped if I deem them too difficult or impractical to maintain. TinyMCE 2 comes to mind as the first one to go.
I won't completely drop support for an editor library in the next release (as in make them unusable), but some may become deprecated (show warnings somewhere) and destined to be dropped in the release after that. OpenWYSIWYG would likely be one of them as it no longer works with modern browsers.

@TwoD Appreciate the update!

Might it be worth starting a new branch, without support for the older editors?

To be clear, the last thing I want to do is further delay a new release. But if it would be faster to release a new 3.x branch, without that backwards compatibility, and then later update the 2.x branch -- either with that support or with some kind of upgrade path (not sure if that's even feasible in the context of third-party editors) -- that might be worth considering.

Then you'd get 80% of users on a new, stable, full release -- instead of grinding away at trying to provide that old support and delaying a release for everyone. What do you think?

Also, don't be shy about asking for help. There are multiple people in this thread who have offered time and funding to help get this done.

I've considered that, and we tried it long ago when taking the first steps towards Wysiwyg 3.x.
That branch got so far behind it wasn't worth to revive. Making a leap and then stepping back to fill in the gaps between the new features and the older branches was harder than just building on top of them.

We have a lot of modules interacting with Wysiwyg and the editors in one way or another. Too many features broke when we started dropping the old APIs in favor of new ones, which is why I've opted to expand rather than cut out parts after that.

Help to follow-up support requests and triage bug reports would be great! Testing, reviewing and writing patches to squash bugs would be even more awesome!
I'm currently looking over and re-rolling some of the larger patches that are in progress for the changes I mentioned earlier, and some things I think are related.

Even if I haven't committed anything after the weekend is over, I should at least be able to update the list in the original post here during the week to better show what I'm trying to do and the latest code I have.

Funding would of course be appreciated, but I'm afraid I wouldn't know what to do with it. I already have a full-time job and money won't buy me time away from it or family.

@TwoD - THANKS! Please keep in mind that there are several modules that depend WYSIWYG rather than the native ckeditor module (and you have quite a few active users.) so just getting current ckeditor support would be a big benefit for CKeditor - even if there is a native module as a solution.

removed a duplicate and sorted the list

@xurizaemon Thanks!

I'm still going through all issues to work out which ones must go in before the release. Many of them overlap either as problems or solutions...
I've done this for smaller "scopes" before but now I'm really trying to cross-reference everything and lay out the current state of the module and issue queue visually.

One specific thing already stands out a lot, and that is the need to version control Wysiwyg module's API, and that functionality will be in the first issue to go in (I'm testing the code now). Like @sonicthoughts said; several modules depend on Wysiwyg module (it even used to be called "Wysiwyg API"), so we need to take care of them as well as the users!

Making any changes to improve the API is very very difficult if we can't break anything and have to put if-this-version-else-this-version special cases to handle version differences everywhere they could pop up over time. The best thing would be if modules using the API could help and tell Wysiwyg when it should not need to apply conversion routines or other special treatment to maintain backwards compatibility when it's not needed. If a module using the API does not declare it's using a specific version, we could handle those that separately and apply conversion from what the current API expects module hooks to return to whatever the new version requires, then we need a lot fewer special cases later.

Several Drupal modules have already figured out how to do this well, like Views and Panels to name a few. If we could depend on CTools we would get a lot of this through its API. But... Introducing a new dependency just like that isn't very user friendly, and it would still require special treatment for the current API version. I will however borrow the pattern used there, to almost make it appear as if we had completely dropped backwards compatibility like @rootwork suggested. If a module declares it does support a new API version, we'll outright trust them on conforming to that version number instead of having Wysiwyg guess whether it fully (or worst case, partially) supports a feature.
If Wysiwyg has moved on from the version a module declares (if any), it's hook implementations will be ignored. If Wysiwyg could still support that older version, we'll invoke the hooks in the old way (or expect an older return value format) and then apply a conversion before moving on.

I'm still doing this in my spare time (today being an exception where I took some time between tasks at work to write this) and I'm realizing fully committing to a steady schedule of X hours per day or week at home is currently not possible. It is however much easier to take the time to do stuff than it was before. (Some of it thanks to it being absolutely freezing outside and thus having to postpone other things...;)

So, to summarize the direction it's currently going in:

• Wysiwyg will invoke hook_wysiwyg_api() before any of the other hook_wysiwyg_*() implementations in the same module.
• If a module does not implement hook_wysiwyg_api() the module will be treated as if it's implementing the current API version ("current" as in what's described in wysiwyg.api.php before the patch lands). If that means the current version is "0" or "1" is TBD and a implementation detail more about practicality at this point.
• Modules will be able to implement hook_wysiwyg_api() to tell Wysiwyg the [latest] API version it supports.
• Modules will be able to call wysiwyg_api_version() to get the current API version.
• Modules will be able to call wysiwyg_mimimum_api_version() to find out if their hooks/expectations are too old and will be ignored.
• Modules will be trusted to not attempt to alter Wysiwyg's behavior (or UI) if the versions returned from the two calls above indicate the currently installed Wysiwyg version is not compatible with the module's expectations.
• API differences will be noted in wysiwyg.api.php, wysiwyg.api.js and wherever else necessary.
• How version numbers will be incremented, and what calls for it, is TBD. Looking at how to apply semantic versioning.

There were a couple of other things I'd like to mention now but I'm running out of time, mostly about how this versioning enables refactoring of the cross-editor plugin code, which makes it easier to implement the new cool toolbar managment on top of it, which in turn lets us drop in TinyMCE 4.

I am very grateful to @TwoD and so many others for providing this module, and indeed Drupal as a whole as free software. As I read the earlier comments, especially about trading off family time versus Drupal I sympathise hugely. I can see that threads like this one can put pressure on individuals to work on Drupal, and I don't want to be part of that.

However, even having read all the above, I have to admit I'm still confused.

• I am one of probably many who reluctantly tested and then installed a specific dev version to get better new ckeditor (so losing out on the protection from security announcements). Personally speaking it seems to be working well.
• I revisit the issue queue here every 6 months or so, and each time the list of "essential tasks for stable release" seems to be longer. But surely a release missing some features is better than no release at all? E.g. even if the new release didn't have TinyMCE 4 support, it would still be a huge improvement for all ckEditor users - and no worse for those who stick with TinyMCE 3.

Is there any possibility to remove items from the essential tasks list here that are "improve X", and then focus strictly on those that are bugs/regressions. This would allow a release that was a big improvement for many of us, and at least not going backwards for the others. And it would potentially take a lot of pressure of you, allowing you to spend some family time!

The remaining "prime-time" life of D7 is maybe a couple of years before D8 starts to attract more and more of the active sites. Given that as I understand it this module is obsolete in D8, then there is little benefit in having a perfected version just as D7 starts to die!

So in essence - my proposal is to be ruthless in pruning out the essential tasks for next release list. Similar to the last phases of D8 or any other large project, think of a ship-stopper as something that used to work in the previous release but now doesn't. Any feature request that is not yet ready could wait until the release after.

Thanks again for all the hard work!

PS You have indicated here that financial support won't help you, but the project page still has text asking for contributions.

Dev branch works well on sites I've deployed for a long time. Just sayin'. :)

@Chris Charlton Thanks for the reply.

I think it is helpful in one sense because it emphasises that we are likely not very far from having a stable release.

On the other hand it potentially creates some confusion regarding the importance of a stable release.

• Firstly, "dev" is not a specific release. Currently we are on iteration 75.
• Is every single dev release stable and good to use on a live site for all scenarios? - surely not. There are likely to be bad check-ins that are later corrected, and possibly also check-ins that are partially working to allow others to improve or test them. (E.g. I vaguely that earlier releases had ckEditor defaulting to pruning any tags it didn't agree with from the master copy of your nodes.) If you make a policy of instantly installing ever new dev release on your live site, sooner or later there is likely to be a problem.
• So I'd argue a better approach is to pick a specific dev release, run some personal testing, then deploy to live. But possibly I don't pick the best one, and everyone is testing separately rather than pooling our resources.

The stable release mechanism was created for a good reasons, and is used on almost all Drupal modules to provide the following benefits:

• It protects live sites from changes that haven't been fully tested.
• It allows us to all test and run the same level of code. If a significant bug is found, there can be a new stable release, and people are automatically notified they need to upgrade.
• It provides the security bug/announcement system. Any security fix in stable code is not discussed in public. Once the fix is ready, then there is an announcement with high visibility through various channels.

I propose we pool our resources to try and figure out how stable the current dev is, and what if anything needs doing before it can be stable. Please everyone join the new issue at #2688223: Tag a 2.3 release of what's currently in 7.x-2.x where you can either

• confirm you believe dev is stable
• let everyone know any issues that mean it is not

Thanks!!

"If we could depend on CTools we would get a lot of this through its API. But... Introducing a new dependency just like that isn't very user friendly, and it would still require special treatment for the current API version."

(Generally, it is a good thing to avoid unnecessary dependencies.)

FWIW, I do not think I have any sites that does not use any (mostly several) contributed modules that already have cTools dependency. As the core features of cTools are also part of Drupal 8 core, I think this particular dependency would be fine.

FYI, summary of the discussion in #2688223: Tag a 2.3 release of what's currently in 7.x-2.x is so far unanimous in favour of a new stable release based on the exact code in dev now.

@TwoD how about it???? Many people would be very grateful and hopefully it's a pretty quick thing for you to do. I'm sure people would volunteer to help creating a release note or anything like that if it would help.

Yes of course the are more issues that would be great to have, but a release now wouldn't delay those - there can always be another release when needed.

[Also I would second @Leeteq that a CTools dependency is likely to be fine if that would help you make progress, as so many sites have it installed already.]

I doubt CTools being a dependency for future releases would be an issue. It's popular enough, as pointed out above.

One option is to consider a 3.x branch, if it's a real big change to this module's core setup.

Hi,
+1 for creating a new release (even alpha). it's nearly 4 years without a new release !
i also maintain modules and i know that it's frustrating to have a release without putting into it all the features we wanted to have or fixing all the issues we wanted to fix.

We can maybe at least have an alpha or beta release, that would push people to test it and use it, and this will make the stable release well tested :)

Yes please, an alpha/beta release would be fine.

Over in #2688223: Tag a 2.3 release of what's currently in 7.x-2.x I proposed that we tag a 2.3 release of what's currently in 7.x-2.x. Following that, I propose that the blockers outlined in this issue would be the target for a 2.4 release.

I'm taking the freedom to update the title of this issue to reflect this.

Please contribute to #2788879: Offer to co-maintain WYSIWYG; restructuring project engineering and maintenance to help get the next release published.

Tagged and released.

See my comment in #2688223-30: Tag a 2.3 release of what's currently in 7.x-2.x.

The issues mentioned here will be considered for future releases instead.

I created a new issue to plan 2.4: #2840252: Plan for WYSIWYG 7.x-2.5 release