Webform currently hardcodes an LF in delimited exports.
A lot of people using Windows want to open these exported CSV/TSV files in software that expects CRLF line-endings.
We recently had this request from a client. Instead of extending the exporter class I though it might be a better idea to provide this as a configuration option, as I guess more people would want this functionality.
Patch will follow.
| Comment | File | Size | Author |
|---|---|---|---|
| #6 | webform-csv_export_crlf-2533010-6.patch | 2.51 KB | danchadwick |
| #3 | alternative_line_endings-2533010-3.patch | 5.94 KB | rv0 |
| #1 | alternative_line_endings-2533010-1.patch | 6.09 KB | rv0 |
Comments
Comment #1
rv0 commentedComment #2
danchadwick commentedI'm a bit plus/minus on this. Most windows applications can open text files with unix LF line endings. For example WordPad will (but Notepad won't).
Assuming for the moment that we want to add this feature, I don't think it warrants a global default. We have a number of settings that one might want a global default for, and the list keeps expanding. I'm not sure what the right way to handle that is, so I hesitate to add a little-used option to the admin options.
I also question the need for CR. This was the format used by the Macintosh before OS-X. Who would use this options? This would allow the option to be a simple question: Line endings: (x) Windows (CRLF) ( ) Unix/universal (LF) -- for example.
Last, the options as described in the drush command should match.
Comment #3
rv0 commentedHi DanChadwick, thanks for your review!
The need for CRLF: (sadly) a lot of companies and organizations rely on software and protocols that aren't that flexible when it comes to how data is handled. In my specific usecase, the whole backend administration of a town (±60000 inhabitants) runs on these CSV files with CRLF. They can't just change the software to use LF because that would break all other programs that feed csv to it. In general, most applications that do csv exports have 2 options: delimiter and line endings, so it does make sense to me (hence me writing this as a patch instead of just another contrib add-on module.)
The global default: I guess I can remove that if you want.. It just made sense as the delimiter setting is also global and both are kinda related. Perhaps I just hardcode it?
The need for CR: none really.. I removed it.
Not sure what the issue is with the drush command?
In any case, here's a new patch
Comment #4
danchadwick commentedI installed your patch. Thank you for the effort. I just can't get beyond the amount of extra UI complexity it adds that hardly anyone needs. Webform has 400,000+ installations, and as far as I know this is the first issue to request CRLF export files.
Am I correct that for your client, 100% of the webform downloads need to be CRLF (or at least can be CRLF)? I'm thinking that we just introduce a blind (i.e. no UI) config variable for the terminator. Its value would either be
"\n"or"\r\n"-- i.e. the actual terminator.Further, I might be inclined to change the default to CRLF on the theory that just about anyone can accept a Windows file because they are so ubiquitous. I'm not sure about this though.
So if someone wants to override this, they can either put a variable into the database or edit settings.php with
$conf['webform_csv_line_ending'] = "...";.Thoughts?
Comment #5
rv0 commentedI'm ok with a blind config variable!
As for changing the default to CRLF: NOOoooo!! I bet that would break 1000s of workflows, as you say yourself: 400,000+ installations!
Shall re-roll a patch soon, got some other stuff to do first!
How/where should I document it?
Comment #6
danchadwick commentedI took care of the patch, in part because I wanted to tweak the variable name. Thanks for your work.
Re documentation: if you could find a place (or make a place) on the community documentation pages for Webform and document this, that would be great. There are other "blind" configuration variables to, such as using legacy tables rather than views for the results tables.
Committed to 7.x-4.x.
Comment #7
danchadwick commentedNeeds D8 port. Because CMI.
Comment #9
rv0 commentedSadly, this change doesn't work the way it should.
if you variable_get the line_ending, you get a literial string (ie.: you get the equivalent of '\n' instead of "\n")
The resulting CSV file wil just contain the string \n at the end of every line, except when using the default setting.
This is why in my original patch I hardcoded the line endings.
This is also why the webform_exporter_delimited class contains code like this:
I did a little research and it seems like there isn't a good fix for this, it cannot just be converted: https://bugs.php.net/bug.php?id=43901
I also read:
So basically, I think the variable method of storage should be changed back to CR/LF/CRLF/.. strings like I did in my original patch, and do the conversion in a switch case.
I'm willing to provide a new patch for this, what do you think @DanChadwick?
Comment #10
danchadwick commentedWebform provides no UI to save this variable. You get back from variable_get exactly what you set using variable_set or $conf in settings.php. The issue is that *you* set a literal backslash rather than the desired newline character.
So you want something like
$conf['webform_csv_line_ending'] = "\r\n", rather than with single quotes.Comment #11
rv0 commented@DanChadwick
Sorry about that, you are right. I had set the variable using drush vset, it appears there's no way to do it like that (or at least, not a way I know of).
See:
Funny! See how the output is different the 2nd time I run vget.. I'll see what I can find about this, at least it is documented here already ;)
Comment #12
fenstratCommitted and pushed to 8.x-4.x. Thanks.