I recently had to install Drupal on an old SuSE server that someone else built. It turns out that this server is using rsync 2.6.8, protocol version 29. Every other rsync install that I have encountered has been rsync 3.x, which is protocol version 30, and this is apparently the case for all other drush users, because sql-sync does not work with rsync protocol 29.

The problem is that the flag --remove-source-files that drush uses was --remove-sent-files in protocol 29. The attached patch special-cases this flag, so drush sql-sync @live @dev --protocol=29 will work.

n.b. this patch requires an exact match for protocol 29, so protocol 28 and earlier will still break. I don't have access to any servers running a version of rsync that old, so I simply consider anything older than protocol 29 as out-of-scope. rsync 2.6.8 has a copyright notice that runs through 2006.

CommentFileSizeAuthor
#18 drush-legacy-rsync-2.patch2.88 KBgreg.1.anderson
#16 drush-legacy-rsync.patch2 KBgreg.1.anderson
drush-rsync-p29.patch908 bytesgreg.1.anderson

Comments

greg.1.anderson’s picture

greg.1.anderson commented
Assigned: greg.1.anderson » Unassigned

Shouldn't have assigned this to myself; someone else should review.

greg.1.anderson’s picture

greg.1.anderson commented

This can also affect pm-download; see #961134: When i download code via drush i get rsync error.

luchochs’s picture

luchochs commented

Note the --remove-source-files option was added in version 2.6.9 (6 Nov 2006), protocol version 29.

greg.1.anderson’s picture

greg.1.anderson commented

Hm, thought that the rsync on my server that's using protocol version 29 was 2.6.8, but no matter; seems we don't need to support protocol version 28 or earlier, since rsyncs that old do not support a key feature needed by sql-sync and pm-download. Thanks.

luchochs’s picture

luchochs commented

Greg, yes, rsync version 2.6.8 also is under protocol version 29.
That is, the name of the option should not be 'protocol' since the feature (--remove-source-files) does not depend on this one.

greg.1.anderson’s picture

greg.1.anderson commented

The option --remove-source-files in protocol 30 is called --remove-sent-files in protocol 29. With this patch, if you set --protocol=29, then it causes drush to automatically rename the protocol-30 option to its protocol-29 name, so that your code can invariantly use --remove-source-files (as sql-sync and pm-download do).

luchochs’s picture

luchochs commented

The option --remove-source-files isn't only a feature of the protocol version 30. See #3.

luchochs’s picture

luchochs commented
Status: Needs review » Needs work

Perhaps --old-rsync-version ( or --rsync-version-lower-that-2.6.9 or something) instead of --protocol=29?
Then, if the option is set, use --remove-sent-files instead of --remove-source-files.

N.B.: Both rsync options don't work the same way, --remove-sent-files doesn't delete the files in source if they exist in destination.

greg.1.anderson’s picture

greg.1.anderson commented
Status: Needs work » Needs review

This is what man rsync says in 2.6.8:

       --remove-sent-files
              This  tells  rsync  to  remove  from  the sending side the files
              and/or symlinks that are  newly  created  or  whose  content  is
              updated  on the receiving side.  Directories and devices are not
              removed, nor are  files/symlinks  whose  attributes  are  merely
              changed.

This is what man rsync says in 3.0.6:

       --remove-source-files
              This tells rsync to remove  from  the  sending  side  the  files
              (meaning  non-directories)  that  are a part of the transfer and
              have been successfully duplicated on the receiving side.

You are correct that remove sent files will occasionally leave something behind, but only if it exists and has the same content as the source file. This should never be the case for pm-download, which now deletes the target first. For sql-sync, you could leave a temporary dump file behind if there were absolutely no changes to the database, and you are using postgres. Under mysql, timestamps are included in the dump, so they are always different every time, even if the db has not changed.

Since the behavior is close, and there is no available alternative, I think that reverting to --remove-sent-files is reasonable.

Regarding --protocol=29, I choose that because that option is passed on to rsync; however, this reasoning is spurious, as this option only affects batch mode. Direct transfers will automatically adjust the protocol versions to match. I still prefer --protocol=29 to something vague like "old" version or something long like lower-than-2.6.9.

I see your point, though; I misread what you said in #3 the first time, that --protocol=29 is not accurate. Perhaps "old" is better? Opinions?

luchochs’s picture

luchochs commented

They are only trivial (and efective if you can see my point) examples .
"old" is vague, we agree. "lower-than-2.6.9" is long, we agree again. But these two extracts become options more appropriate than 'protocol'; why? Because both are next to the word 'version'.

Other example:
--rsync-version=x.y.z, if lower than 2.9.6 then ... .

My opinion is my point, i.e., that the new option should have the word 'version', this issue depends on it after all.

greg.1.anderson’s picture

greg.1.anderson commented

I guess the thing is (for me) that regardless of length, 2.6.9 is a bit of a mouthful. How about --use-legacy-rsync, or just --legacy-rsync?

luchochs’s picture

luchochs commented
Title: drush support for rsync protocol 29 » drush support for rsync version lower than 2.6.9

If I would have to use this option I would set his value in an alias or config file. Then, for example:
$options['rsync-version'] = 2.6.8;
$options['legacy-rsync'] = TRUE;
The principal difference that I see between both is that one is more specific than the other.

greg.1.anderson’s picture

greg.1.anderson commented
Status: Needs review » Needs work

$options['rsync-version'] = 2.6.8; would require more code, though, as you would need to make sure that 2.12.8 is considered to be a later version than 2.2.8. Maybe convert them into 21208 and 20208 before comparison?

luchochs’s picture

luchochs commented
Status: Needs work » Needs review

The version 2.12.2 does not exist (at most 2.9.6), but anyhow the following thing should work:

-  $protocol29 = (drush_get_option('protocol', '30') == '29');
+  $rsync-version = version_compare(drush_get_option('rsync-version'), '2.6.9', '<');
   foreach ($rsync_available_options as $test_option) {
     $value = drush_get_option_override($additional_options, $test_option);
-    if (($test_option == 'remove-source-files') && $protocol29) {
+    if (($test_option == 'remove-source-files') && $rsync-version) {
luchochs’s picture

luchochs commented
Status: Needs review » Needs work

Unnecessary change of status, still there is no new patch here.

greg.1.anderson’s picture

greg.1.anderson commented
Status: Needs work » Needs review
StatusFileSize
new drush-legacy-rsync.patch2 KB

Here is an updated patch per #14 with docs in example.drushrc.php.

luchochs’s picture

luchochs commented

Greg, is also necessary to warn that the option works only for versions higher than 2.6.3?
If so, the user might enter his specific version, i.e., '2.6.X'.

greg.1.anderson’s picture

greg.1.anderson commented
StatusFileSize
new drush-legacy-rsync-2.patch2.88 KB

Good point. Here is an improved patch that gracefully degrades for rsync versions earlier than 2.6.3, and improves the explanatory text in examples.drush.rc to match.

luchochs’s picture

luchochs commented

(Detail: there is a trailing whitespace in the inter-paragraph line.)

Looks good to me.

greg.1.anderson’s picture

greg.1.anderson commented

Any objections to committing this?

luchochs’s picture

luchochs commented

I'm not sure if we need to add some extra code to drush_move_dir() related to this new option.

greg.1.anderson’s picture

greg.1.anderson commented

Yes, that was my plan, but I wanted to commit this first rather than tangle the two. But I'll tangle here if that's preferable, esp. since drush_move_dir has been committed.

moshe weitzman’s picture

moshe weitzman commented

No objections from me.

moshe weitzman’s picture

moshe weitzman commented
Assigned: Unassigned » greg.1.anderson
greg.1.anderson’s picture

greg.1.anderson commented
Status: Needs review » Fixed

Committed, but forgot to set status.

http://drupal.org/cvs?commit=453986

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.

hanoii’s picture

hanoii
Primary language Spanish
Location 🇦🇷UTC-3
commented
Version: » 8.x-6.x-dev
Issue summary: View changes

I struggled with this for quite a bit, I can't seem to put this option into the alias file, instead I fallback to drushrc.php and it worked, but it says in the example drusrc that the alias file is the default location.

I have an $alias['live'] and $alias['dev']. My logical step was to put it in the alias live, both within the array and as $options['rsync'], and nothing, version is always the default 2.6.9. I also tried to put it in dev. I also tried putting it in command-specific for sql-sync, but it's not being picked up.

Am I missing something?

helmo’s picture

helmo commented

[Stock response from Dreditor templates and macros.]

Drush has moved to Github.
If this feature is still desired, you may copy it to our Github project.

Please ask support questions on Drupal Answers.