Plugin to convert p- and br-tags to newlines

That - as a Drupal plugin - would most probably make sense to add to Wysiwyg module. It would be the companion plugin for the Auto-paragraph filter in Drupal core, which is usually enabled for the "Filtered HTML" input format.

@fajerstarter

I'd like to test this, but can you give a little more detail for how to apply the extra code? It is not clear how to load this new file.

I got it and will post below.

Here's what I added:

-- Saved the above as linebreaks.js
-- Add the following to wysiwyg_load_editor():

    drupal_add_js($path . '/wysiwyg.init.js', 'core'); // Existing line
    drupal_add_js($path . '/linebreaks.js');

-- Add to wysiwyg_add_editor_settings():

    // to use a string instead of the format id.
    $config['setup'] = 'Drupal.wysiwyg.plugins.linebreaks.setup';

From this, we can roll a proper patch as a first step.

For more information on Drupal plugins, see

http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/wysiwyg/plu...

http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/wysiwyg/wys...
http://cvs.drupal.org/viewvc.py/drupal/contributions/modules/wysiwyg/wys...

I'm not should why this would be a plugin and not a setting as part of the API module.

Here's a quick example of how a Drupal plugin implementing that code would work. The framework to do this kind of editing is already there in the form of these plugins, and using it requires less changes to the module itself. A Drupal plugin won't work with an editor unless the editor implementation has the necessary features, but that's usually not too hard to implement. Wysiwyg module currently assumes a Drupal plugin is supposed to have a button, but that button doesn't need to do anything.

The files should be placed at wysiwyg/plugins/linebreaks.inc and wysiwyg/plugins/linebreaks/linebreaks.js, then you'll have a 'Force linebreaks' button on the profiles page for editors supporting Drupal plugins. Currently, that is TinyMCE, FCKEditor, CKeditor (apply last patch from #610132) and jWYSIWYG (apply last patch from #483452).

It should not be hard to turn this into a proper patch if anyone's interested.
Btw, I noticed that it inserts &nbsp when there are multiple spaces at the end of the contents.

Interestingly, this plugin implementation makes clear that we missed "Support for extensions" (as already existing for native editor plugins) in the 3.x roadmap. ;)

Yeah, that's why I brought it up. Inserting whitespaces using a Drupal plugin and comparing the output to what's expected might also be a good testcase to see things are done in the correct order...

Another good thing about a plugin like this is that there would no longer be a need to convert old content to using paragraph and break tags when assigning an editor to a format where the linebreak converter was enabled.

Yes, given the current situation. I don't think we should introduce more settings to "Cleanup and output" or the other fieldsets which are the same for all editors before we have fixed #313497: Allow configuration of advanced editor settings. There is already too many settings on the profile page which only works for a few editors. Because the editor implementations are incomplete, there's no reliable way to get and set the editor contents for editors not supporting Drupal plugins.

Here's a patch of TwoD's version

It works for Fck editor but not TinyMCE, I'll try to fix that right now.

Patch fails on FCKEditor 2.6.x, giving a JS error that the button is undefined.

On a larger note, this should not be a button plugin. It needs to be a global setting, like "Remove linebreaks".

Here's the working patch. Move linebreaks.js to its own folder called linebreaks once you apply the patch.

We have this patch working properly with FCKEditor, if that helps. Need to test on TinyMCE and others.

Subscribe.

Need the same thing for TinyMCE. Any working code coming out soon?

That apart does any one where to put

tinyMCE.init({
   forced_root_block : false,
   force_br_newlines : true,
   force_p_newlines : false
});

so that the same end can be achieved ?

@kaakuu, you can try the method described in #313497-30: Allow configuration of advanced editor settings (comment #30) to inject those settings (that's probably a better place for discussing this). You could also modify tinymce.inc to include those options in the settings callback, but then you'd have to do those modifications each time you update Wysiwyg.

Thanks TwoD - I saw that but was/am confused about "formatN should match the numerical id of the Input format for which the editor is enabled" - what is the value of N ? I am using "Wysiwyg" and from where do I get the value of N ?

I posted here for actually I wanted to do a +1 to have the feature in "Wysiwyg" as a configurable option like FCK.

You can see the numerical id of your input formats if you go to admin/settings/wysiwyg and click or hover over the Edit link for the corresponding format/profile.
The numerical format id is seen in the last part of the path (admin/settings/wysiwyg/profile/N/edit). The editor profile id currently corresponds to the format id. For technical reasons we can't use just the number in the editor settings arrays/objects when they are sent to the browser, so we prepend 'format' and turn it to a string. Note the last post of that issue, it seems we can't override already set settings, only add new ones using this method.

Most configurable settings which are part of the editor library are to be given a GUI as part of that other issue. The implementation should also allow this GUI to be extended by Drupal plugins and by the hook for native plugins. Essentially, we're trying to remove as much as possible from the 'default' profile page and keep only that which is common to as many editors as possible (enabling/sorting buttons f.ex.).

So what is the final word on this issue? Are there any recent final patches or plugins to install, with instructions, and where can I get it?

Also, can someone please confirm me if this is the same issue that I am having: In trying to provide an editor for a client to input columns, they are cutting and pasting either from Word or from email and in so doing, trying to preserve linebreaks, and what they see is not what they are getting. I don't want them to have to wrestle so much with the formatting as thy don't have the patience or skills.

The pasting from Word is not a problem as I have set filtered html in FCK editor to properly get rid of the MSo junk, and I am encouraging them to work with raw text instead. The problem is what happens raw text newlines. There is a setting in the Input Format Profiles that I thought would replace them with < br /> tags, but it is instead inserting
tags, which are not rendering properly of course. It looks OK in the editor window, which is still showing the \ n's, but not on the site in a normal view (cause the < br />'s are wrong). This seems like a bug.

I just want them to be able to cut and paste into the editor and be able to then either preserve the formatting that came in by converting the newlines to html < br / > tags, or put it in paragraph tags. So far it is wrapping it all in a paragraph and trying to convert the newlines, but failing.

Will this plugin address the problem?

General comment: Why is this such a problem in Drupal. Other CSM systems I use don't have quite so many issues using a WYSIWYG in editors. I should be able to input text, have it appear the way it would look on the web, and be able to edit it and format it the way it will look.

I am now looking at an example of where I have something saved with paragraph tags in it, but when it is displayed on the Web, they are gone. How in the world does that happen? Does Drupal pre-process text after retrieving it for display? Whatever goes into the database should be what it outputs, no?

Please reformat the above message using code tags around the special characters and tags so they are not removed by Drupal's filter.
I can't tell much about your problem without knowing exactly which characters you are referring to where.

The input format settings (which filters are enabled) only affect what happens when the node is viewed and has no effect in the editor (as it runs on the client and the filters run on the server).

Assuming default settings and no special plugins here.
If the pasted content is plain text (if you paste it in Notepad you see HTML markup), the editor will treat it as plain text, meaning it will escape </code> to <code>&lt; and to   etc. This content needs to be pasted in the editor's 'source mode' and should have <br /> and/or <p> tags at the appropriate places. Any other whitespaces like tabs or \n between tags will be treated as source formatting and the editor will do as it pleases with these (sourcecode formatting options apply).

If the pasted content is in HTML format (pasting it in Notepad only shows the non-markup portions of it), the editor will treat it as such and you'll see the expected result in WYSIWYG mode and the full markup in the 'source mode'. This content needs to be pasted into the editor's WYSIWYG mode (and already contains the proper <br /> and <p>...</p> tags. Pasting it in the 'source mode' or the regular textarea before activating an editor will have the same effect as if it was pasted to Notepad.

The code/plugin in the patch above would be useful in cases where you have plain text contents without any <br /> or <p>...</p> tags, which needs to be pasted into 'source mode' because it does have other markup like spans or tables. Normally, the editor would eat the whitespaces and turn all your paragraphs (delimited by double newlines, \n\n) and newlines into a single paragraph without newlines. This extra code does the opposite of what Drupal's 'line break converter' filter does and makes sure your paragraphs and newlines use actual tags as delimiters before the contents are sent to the editor (and the opposite when retrieved from it).

Essentially, this means it gets a lot easier for users which do not want an editor or do not have JavaScript enabled. They can now write content using the same paragraph delimiting we use for these comments, which requires Drupal's 'line break converter' filter to render correctly. That filter would normally add lots of awkward newlines to the rendered content created by users which do use an editor, especially if the editor is set to format the source. The above code should help in those situations by reducing this code/newlines mix to newlines, \n, only.

I'm still not quite sure what you mean.
You say

There is a setting in the Input Format Profiles that I thought would replace them with <br /> tags, but it is instead inserting [what should be here, \n?] tags, which are not rendering properly of course.

The <code> </code> tags will preserve the characters for you so you don't have to insert spaces.
It would be great to see an example of how the pasted content is formatted, and what you would like the final HTML to be. A 'picture' says more... ;)

WYSIWYG in general is a very tricky subject. Even the slightest differences between the styles used in your editor and those applied to your nodes may cause the content to appear in a very different way, and there is of course the matter of the editor even being able to represent the final node.
Drupal does pre-proccess each piece of markup before it is viewed (plain text excluded), to keep the content stored in the database as harmless as possible. Each input filter enabled on your input format is a piece of code which your content is put through before the node is viewed. They can do anything from splitting the content at the <!--break--> marker into two separate "fields", to replacing \n\n with <p>...</p> and \n with <br /> (as done in these comments) or closing open tabs so the markup stays valid. The editor has no way to tell what these filters do, or how to represent them in WYSIWYG mode, which is why we (site admins) need to make sure the enabled plugins and settings in the editor match the filters run by Drupal.

TWO D, thanks for the in-depth answer and attention to my question. Admittedly I was quite tired when I wrote it, having spent many hours in the editor profiles and input format options turning things on and off and seeing what happens. Admittedly I'm fairly new to Drupal, coming from a mainly Joomla background, and there were some things I just didn't get at first. I have been digging into quite a few Drupal books, sites, and the online manual, and I'm by no means a Web design noob -- but sometimes in this business what you already know is the problem....

In any case one of the things I did not quite get at first was how the filters set in Drupal's Input Formats do actually filter the out html tags before sending to the browser 9as you pointed out). This means you can have html tags in your article which will be stripped out when displayed if it is saved with the input format filtered html. This was making me go around circles. While I realized that Drupal does mention that some filters will affect the data when sending to the browser -- in other words, on output, not on input -- it doesn't really tell you which ones do it at what point. As as I just pointed out, they're called input format filters, not output filters. It seems to me an input filter works as the text is being input and saved--for example, the paste from Word, or the line break converter, etc.. Filters that strip tags out of the data after pulling from the database and before sending to the browser are more aptly named output filters. In Joomla, what is in the database is what is sent to the browser (unless another plugin grabs it along the way --but generally they do not handle html and raw text conversion issues, which are just dealt with when you enter the text to begin with).

I realize all this so far has nothing to do with the plugin in question. I just think Drupal should clarify this. So far in all the materials I have referenced that I've found, nothing explains or shows clearly what these different filters do and at especially at what point in the process.

My issue here was simply being able to have a truly WYSIWYG process. Since the input format filters affect the text afterwards, she had to sit there with two tabs open -- one with the view, one with the edit, to see where the line breaks (as newlines) were showing up in the view, then backspace them out in the editor to put in paragraphs or breaks. Anything that changes the text after the fact does not help the average user at all, but rather just confuses them. The average user is not going to want to or be able to open the source editor and all that. They should just be able to work in a true WYSiWYG environment.

The answer was to turn off the filters that stripped the html and set it up so that as she entered text, it would strip MSO with a good paste from Word filter, or convert the newlines to paragraphs and breaks from raw text. Then, the formatting and line breaks she enters in the editor would be html, not newlines, and she would see the effect immediately in the editor. I installed FCK editor instead of TinyMCE, which offered features like rutn rich text on/off (more for me), a more effective paste from Word plugin (with a better pop=-up paste box interface), and seemed to convert raw text newlines better as well. It does this with a minimum of injected html.

The Plugin discussed here won't do what I need since I've elected to go with using html and an editor.

Thanks!

I have tested with FCKEditor 2.6.5, and CKEditor 3.1.0.4891, and it work swimmingly. (with latest -dev release of WYSIWYG).

Would love to see this get rolled into the module at some point.

[Edit: FYI, having the plugin as a button is a major wtf for me. Is there another way to implement the filter?]

No, there is not. Not without an API change, AFAIK.

Yeah I agree it's a wtf. Drupal plugins need to support "extensions" for this to not show up in the toolbar in 2.x-versions. See sun's note in #9. When that is added to 3.x it should be backported to 2.x as well, even if the whole [advances] settings redesign doesn't make it.
Edit: Quick question: why is there a pee variable in there? :s

Re: The pee variable; I have no idea. That seemed odd to me too... maybe someone didn't have access to the bathroom when he was coding this script?

maybe someone didn't have access to the bathroom

LOL

If you can't get line breaks in tinyMCE and you are using safari browser, then instead of trying to patch the tinymce module, simply turn on safari compatibility in the settings and that will fix the problem.

Converting p- and br-tags to newlines when saving!
Converting newslines to br-tags and/or p-tags when loading nodes from database!

These two things everybody needs you uses TinyMCE or (F)CKEditor.

The advantages is that plain text and wysiwyg can co-exist much easier.
This advantage must be preserved.

My problem is:
I still do not know how to solve this problem in TinyMCE or CKEditor. I did not found any way to solve this, but everybody needs this!

If there is a patch or solution please tell what exactly has to be done to get this work.
[again: problem is, that nodes loaded from drupal database, which has been created with plain textfield, loose all the plain lines, loose all the line breaks. And when saving from TinyMCE/FCKEditor, then all textfields should be saved in a way that the contents can be used in plain textfields, that meens br-tags should be convertet to empty lines]

Solution is needed very soon, thank you very much.

@DrupalFan2 - Please apply the patch in #16 to your copy of wysiwyg, and then enable the button in WYSIWYG.

@sun - I have found a few little issues with this plugin that I might fix in a quick patch later; can't roll it right now (from work). Basically, there's no linebreaks.css file, nor a /images/linebreaks.png file (for the toolbar icon). I made a little icon that I think is good enough (based on teaser break button).

Also, there's a major problem (besides XHTML validation) when the teaser break plugin is enabled, and someone edits a node in Internet Explorer. This module adds in a <br /> tag every time there's an all-caps HTML tag (like <TR> <TBODY> etc.) on a new line. This causes a ton of empty space to appear in the node body :-(

See issue: #510552: Invalid XHTML: missing trailing slashes, absolute urls and uppercase tags

I was having some trouble with extra  's added before <hr /> tags... I was able to fix most of these extra spaces by adding 'hr' to the blocklists in linebreaks.js, but it's still not perfect (and I'm not sure why).

Attached is a zip file with all the files you should copy into the plugins folder (so you'd have plugins > linebreaks.inc) to get this patch tested/working. I can confirm it works pretty well with both CKEditor and TinyMCE.

If anyone can help improve the oddball cases of <hr /> tags on their own lines, and inside blocks of content, that would be great (they should probably be treated somewhat like <br /> tags, no?

I'll try to roll this into a patch later... don't have time right now.

Patch #16 seems to work with TinyMCE. Thank you very much.

Patch to account for horizontal rules (see #37 above). Please review... this is my first patch for a contributed module, and I'm not sure if I have the file adding stuff correct... The linebreaks.png image is also included.

Apply the patch from wysiwyg root (to latest dev checkout), and put the attached image inside wysiwyg/plugins/linebreaks/images/

Bump... I've been testing the patch above successfully on five different websites now, and it works marvelously! WYSIWYG is finally my preferred choice over CKEditor now.

The only issue I can see with this patch is the extra   added after a block of text inside which a <hr /> is located. If someone could bugfix the linebreaks.js file, that would be perfect. But I don't see why this can't go in without that fix...

+++ plugins/linebreaks.inc	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,22 @@
+function wysiwyg_linebreaks_plugin() {
+  $plugins['linebreaks'] = array(

If I understand this issue + patch correctly, then we now need to find a way to

a) either limit certain plugins to text formats having a certain input filter enabled only

b) or ideally auto-enable + hide certain plugins for text formats having a certain input filter enabled.

Heh. That's quite a challenge ;)

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,125 @@
+// Adopted from Wordpress 2.8.1-beta editor.js.

Is there any original author we could or should credit?

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,125 @@
+  invoke: function(data, settings, instanceId) {
+    alert('This button does nothing, it belongs to the linebreaks plugin.');

Not sure whether we want to keep this alert().

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,125 @@
+  pre_wpautop : function(content) {
...
+    wpautop : function(pee) {

Those method names

1) should be private or at least be prefixed with the plugin name.

2) absolutely need better/proper names, something along the lines of htmlToText() and textToHtml() or similar (these suggestions are still not good).

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,125 @@
+              return a.replace(/[\r\n]+/g, '');
...
+      pee = pee.replace(new RegExp('<br />\\s*<br />', 'gi'), "\n\n");

Only a few replacements use the shorthand regular expression syntax. I don't really see the need for the longer syntax. Changing all the replacements will also make the expressions shorter, because backslashes do not need to be escaped there.

Powered by Dreditor.

I'm not sure about the author...

I think the method names should be changed as well; don't know why I didn't do that earlier. I'll work on cleanup sometime this week and hopefully repost a patch.

I have some problems with this patch/plugin using TinyMCE:

Within tables (td) line breaks often get lost.

GREAT GREAT PROBLEM:
When I edit an existing article and insert some text and then click on the undo-button there are inserted a lot of/hundred new lines, the whole article is broken!

Undo-botton destroys everything. This is a matter of linebreak patch.

Some idea? Please help!

For now, please don't use the Undo button :-)

I'll look into why that may be happening. Are you using Internet Explorer, by chance?

[Also, resetting category/priority - this is for the WYSIWYG module in general, not for specific problems with the patch.]

Undo botton problem is a problem in firefox.

Internet explorer has another problem: When I edit an article and switching between FCKEditor and TinyMCE a lot of newlines get lost, article gets broken. But using Google Chrome or Firefox switching between FCK and Tiny is no problem.

All this problems seem to concern linebreak plugin.

Next problem:

When I click on preview and then after previewing click on save linebreaks within table cells get lost.

I need a working solution.

@DrupalFan2 - I'm not having any trouble with tables or switching editors on either of the sites on which I'm using this patch... there might be a problem with the interaction of linebreaks.js and your editor settings in your WYSIWYG profiles; try toggling the settings for 'clean up source formatting' and other filters that apply to different profiles.

Next problem:

In Internet Explorer force linebreaks plugin does NOT work. When I save an article alle linebreaks get lost.

Try this:
New Artikel --> Deactivate richtext --> Copy article text in blank drupal textfield including a lot of linebreaks and double linebreaks --> save .. OR active richtext and save .. OR active richtext and preview and save. ---> linebreaks get lost in internet explorer (works in firefox/chrome).

@ geerlingguy: May I test this on your site (send private message)?

Because of #47, #48, #50 I would prefer a solution without JavaScript. A solution in PHP could be very stable.

This JavaScript version will never(!) work stable in all browsers (see #47, #48, #50).

Because of #47, #48, #50 I would prefer a solution without JavaScript. A solution in PHP could be very stable.

This JavaScript version will never(!) work stable in all browsers (see #47, #48, #50).

Maybe we can realize this using the line break converter function in filter.module

function _filter_autop($text)

running this function before(!) node body content is loaded into the body textfield of the edit page (node/xxx/edit).

But how we can start to realize this (please tell me how I can hack this for my site).

subscribing

@DrupalFan2

You have to have it in JS, because people toggle the editors on and off with JS. Doing it in PHP is done now, and is not sufficient. I think the other problems you report are not actually related to this issue.

@DrupalFan2 - it seems that a lot of the issues you've been encountering are tangential; also, please know that you can edit your previous response, rather than continuing to add extra responses. This preserves the flow of the thread and can keep the issue more on-topic.

[Edit: Also, unfortunately, I am not able to let you test on my site right now... someday I'll have a nice test site set up that I can co-troubleshoot on, but that may just be a pipe dream.]

subscribe

I'm almost finished fixing up all the little issues I've encountered with this patch, namely:

• If the WYSIWYG-enabled field/textarea contains an <hr />, extra newlines will be added below the next element (sometimes).
• The .js file has a lot of pee in it. I need to change that and make sure everything is golden.

I've been using the patch on four different sites now, and have not had any troubles with either CKEditor of TinyMCE on any of the sites. I'm not sure what was causing DrupalFan2's problems, but I think they are an edge case, and once the other bugs are worked out of this patch, I hope it can be committed—it is a GREAT help for using legacy content!

Great job @geerlingguy, @TwoD, @agentrickard and all!

I've tested the patch from #39 and it works great with TinyMCE 3.2.4.1.

But I do have a problem. Pasting from Word, even with the "Paste from Word" plug-in turned on, seems to include "\n" where lines break within a wrapped paragraph. In other words, pasted text from Word needs to have those removed before pasting into Drupal in order for the paragraph to wrap properly. Note that the problem is the same with rich text turned on OR off.

I want to use the "Force line-break" functionality of the Wysiwyg module to transition away from what I do now on most sites: have Drupal's auto line breaking turned off. TinyMCE will then just ignore the "\n" and so the paragraphs display as wrapping properly.

But I much prefer to use the "Force line-break" functionality to keep the paragraph and break markup out of the database but mostly to make toggling back and forth between rich and plain text MUCH more stable.

Any ideas about how to get those "\n"s out of word pastes?

Thanks,

Shai

@Shai - sounds like it could be more a problem with the Paste from Word plugin in TinyMCE; you might want to file another issue, or file an issue over on the TinyMCE forums (unless I'm reading your response wrong...).

Maybe this can help:
http://tinymce.moxiecode.com/punbb/viewtopic.php?pid=3723

Status update on this patch: It's working perfectly for me on 12 different sites now (I hate all the ugly no-linebreak markup generated by TinyMCE, so I always use this patch—and it works fine with CKEditor too), and the only issue I have encountered (which is a marginal use case, imo) is that horizontal rules (<hr />) sometimes have an extra newline added after them when switching between the editor and plain text.

This exact bug is still present in the current 3.x version of WordPress, and I don't hear too many people complaining about it there :-/

I think we could apply the above patch, and mark it as 'needs work' again, or simply open a new issue titled "Linebreaks plugin adds extra space after horizontal rules."

(Of course, I 'pee' is still used as a variable in this patch... if changing that can push it through to commit status, I will do that asap).

Is there anything holding up a commit on this? Great work @geerlingguy.

Shai

subscribing...

Attaching an updated linebreaks.js file - don't have time right now to do a patch, but this fixes the 'pee' issue, and includes code cleanup that changes wp-specific variable names to generic 'lb' tags.

The only remaining issue (I'm using this on four different production sites, so I'm pretty confident that the plugin works great across IE, Safari, Firefox, Chrome and Opera, all versions, on Mac & PC) is the horizontal rule problem - and it's not too bad - an extra empty line is added after a paragraph, but ONLY if the <hr /> tag is at the beginning of a paragraph... if it's at the end of a paragraph, the rule works great. If it's on its own line, extra spaces are added before and after :(

Hey everyone, geerlinguy pulled me into this one. I believe I have solved the <hr /> problem. Certain block-level elements look nicer when surrounded by two newlines (e.g., blockquote, hr, ul, table), while others look nicer when surrounded by a single newline (e.g., li, tr, td). Somehow setting the spacing properly around these elements, so that they look nicer when in plain-text, fixed the issues when passed back to WYSIWYG.

Attached is an updated linebreaks.js file with these changes.

I am using this on archstl.org and stlouisreview.com, as well as a few smaller sites, and it works great!

If this would speed up implementation of this patch, I'll try creating the patch on my own again—but it will take some time, as I have forgotten how to add a folder and a file in a patch using CVS. :-(

Using the latest JS file from #66, here is a patch.

Shai

Works perfectly - I'm using it on four different sites now, with absolutely zero problems, on CKEditor and TinyMCE.

Patch applies cleanly to 6.x-2.x HEAD, then the linebreaks.png file needs to be moved into the plugins/linebreaks/images/linebreaks.png directory manually... I don't know how to do that in a patch! (See comment #39 for the .png file).

Please, please please get this committed — on two of my sites, all legacy content (over 12,000 nodes' worth) requires this plugin, or the editor will strip all whitespace! Plus, it makes the db tables look so pretty :D

Just wondering when / if this is going to be committed?

"Just wondering when / if this is going to be committed?"

I want to know too.

@sun, can you please share you latest thoughts on this. I can't use wysiwyg without this patch. I would really like to see this as part of the module.

Thanks much,

Shai

I'm not Sun but I do have some say about this module too. ;)

I only have time for a super-fast code review now though. This needs some major testing if it's to get in. Considering it's quite a big patch, all already implemented as a plugin, it might be better to put this in its own project for now. There's been some talk about collecting useful things like this in a wysiwyg_contrib/"add-on collection" module, maybe this could be the basis of such a module?

Having a button in the toolbar which doesn't do anything isn't an optimal user experience, but I guess it'll have to do until cross-editor plugins can implement pure extensions. That or rewrite this as a utility function called from each editor implementation, but I think it makes more sense as a plugin since everyone isn't going to need it.

The amount of potential content manipulation done here is quite staggering. Ideally we should have some automated testing here to compare content before and after attaching and detaching the plugin so the right substitutions have been made. Another test should also confirm that content looks the same after doing both attaching and detaching as it did before. Doing all this manually would be very hard. take a lot of time, and it'd be easy to make mistakes. Anyone who knows about a testing framework that would be suitable for doing this? Just coming up with and writing the tests will take time so I'd hope we wouldn't need to create the testing framework too.

So, some coding style nitpicks:

Why isn't the literal RegExp notation used? (As in lbvar.replace(/<br \/>\\s*<br \/>/gi, '\n\n'); etc)

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,149 @@
+    ¶
+//  isNode: function(node) {
+//    console.log("Got here");
+//    return ($(node).is('wysiwyg-linebreaks'));
+//  },

Can we remove this?

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,149 @@
+  },
+  ¶
+  detach: function(content, settings, instanceId) {

Non-empty line.

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,149 @@
+  // This function runs when the content is saved (or when WYSIWYG is disabled manually)

80 char comments ended with a period, please. See other comments as well.

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,149 @@
+      if(lbvar == "") return lbvar;
+      ¶
+      if ( lbvar.indexOf('<object') != -1 ) {

Non-empty line.

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,149 @@
+      lbvar = lbvar + "\n\n";
+      lbvar = lbvar.replace(new RegExp('<br />\\s*<br />', 'gi'), "\n\n");
+      lbvar = lbvar.replace(new RegExp('(<(?:'+blocklist+')[^>]*>)', 'gi'), "\n$1");
+      lbvar = lbvar.replace(new RegExp('(</(?:'+blocklist+')>)', 'gi'), "$1\n\n");
+      lbvar = lbvar.replace(new RegExp("\\r\\n|\\r", 'g'), "\n");
+      lbvar = lbvar.replace(new RegExp("\\n\\s*\\n+", 'g'), "\n\n");
+      lbvar = lbvar.replace(new RegExp('([\\s\\S]+?)\\n\\n', 'mg'), "<p>$1</p>\n");
+      lbvar = lbvar.replace(new RegExp('<p>\\s*?</p>', 'gi'), '');
+      lbvar = lbvar.replace(new RegExp('<p>\\s*(</?(?:'+blocklist+')[^>]*>)\\s*</p>', 'gi'), "$1");
+      lbvar = lbvar.replace(new RegExp("<p>(<li.+?)</p>", 'gi'), "$1");
+      lbvar = lbvar.replace(new RegExp('<p>\\s*<blockquote([^>]*)>', 'gi'), "<blockquote$1><p>");
+      lbvar = lbvar.replace(new RegExp('</blockquote>\\s*</p>', 'gi'), '</p></blockquote>');
+      lbvar = lbvar.replace(new RegExp('<p>\\s*(</?(?:'+blocklist+')[^>]*>)', 'gi'), "$1");
+      lbvar = lbvar.replace(new RegExp('(</?(?:'+blocklist+')[^>]*>)\\s*</p>', 'gi'), "$1");
+      lbvar = lbvar.replace(new RegExp('\\s*\\n', 'gi'), "<br />\n");
+      lbvar = lbvar.replace(new RegExp('(</?(?:'+blocklist+')[^>]*>)\\s*<br />', 'gi'), "$1");
+      lbvar = lbvar.replace(new RegExp('<br />(\\s*</?(?:p|li|div|dl|dd|dt|th|pre|td|ul|ol)>)', 'gi'), '$1');
+      lbvar = lbvar.replace(new RegExp('(?:<p>|<br ?/?>)*\\s*\\[caption([^\\[]+)\\[/caption\\]\\s*(?:</p>|<br ?/?>)*', 'gi'), '[caption$1[/caption]');

Lots of complex RegExps here. Comments on what they do and why?

+++ plugins/linebreaks/linebreaks.js	26 Feb 2010 00:00:30 -0000
@@ -0,0 +1,149 @@
+      ¶
+      // Fix the pre|script tags

Non-empty line.

Powered by Dreditor.

I like the wysiwyg_contrib module idea.

What kind of changes would need to be made to wysiwyg, if any, for wysiwyg_contrib to work?

@geerlingguy, do you have cvs.drupal.org privileges?

I would also support an even more focused module like, "wysiwyg_ptweak". Small modules are good!

As long as wysiwyg maintainers were willing to make reference to such sub-modules on the wysiwyg project page, I don't think it would be at all confusing for site builders.

This might actually get some energy going for developing cross-editor plug-in capability as well!

In short, I'm +1 for pulling this patch out and making it "contrib", whether in a general wysiwyg contrib module or a single purpose one.

Shai

Forgot to change status before...

Wysiwyg shouldn't have to change at all for a wysiwyg_contrib module to work, we've already got hooks etc in place.

We'll have to wait for Sun's input on project page references, I can't change that. I know the wysiwyg_imageupload module and maybe a few more would like to be mentioned there as well. There's also the Wysiwyg handbook pages here on d.o, don't remember if the ones I'm thinking about are open for editing by all but it would be good to mention plugins like this there too.

I will look into making this into a contrib module... don't have much time before the end of the year, though, so I'm going to keep it as the plugin for now.

For future reference/code snippet borrowing purposes: WYSIWYG Image Upload module.

Hi,

I'm kind of newbie on drupal (1 month of practice) but with a big website coded on drupal 6.
I use the Module Ckeditor + plugin ckeditor
And I want to add this plugin.

So I put the content of the zip file #37 on the directory /modules/contrib/ckeditor/plugins/linebreaks/
And I've changed the js by the #66 js file.

But I don't understand how to say to ckeditor module to use this plugin.
Does I have to add something in php files ?

You cannot use this plugin with the ckeditor.module. It's an unrelated project which only focuses on integrating the CKEditor library with Drupal, while this module (wysiwyg.module) works with several editor libraries.
The ckeditor.module does not have the same plugin architecture wysiwyg.module does, so you'd have to ask those maintainers to adapt it for you, or do it yourself.

sub.

Patch #68 works for me with WYSIWYG 6.x-2.1.

Working on a new module to implement this functionality now... will post back with more info.

@geerlingguy

Excellent news!

Beginning next week I can help some as well with documentation.

Shai

Marking as closed (won't fix), and redirecting discussion of the plugin to the new WYSIWYG Force Linebreaks module, which adds in the functionality separate from the WYSIWYG API module.

I'm using this new module on one production site, and it seems to be working okay for me across IE6-9, Safari, Chrome and FireFox.

[Edit: I also incorporated TwoD's excellent suggestions in #72. Thanks for the help!]