Description

"[overwrite] = TRUE" for zip file with Drush 5 overwrites instead of merging as in Drush make 2.3

It's not merging with Drush 5.10.0.
It was merging fine with Drush 4.5 and Drush make 2.3.

Example of the objective

libraries[kplatforms_tinymce][directory_name] = tinymce
libraries[kplatforms_tinymce][download][type] = get
libraries[kplatforms_tinymce][download][url] = https://github.com/downloads/tinymce/tinymce/tinymce_3.4.9_jquery.zip
libraries[kplatforms_tinymce_fr][directory_name] = tinymce
libraries[kplatforms_tinymce_fr][download][type] = get
libraries[kplatforms_tinymce_fr][download][url] = http://www.tinymce.com/i18n3x/index.php%3Fctrl%3Dexport%26act%3Dzip%26la%5B%5D%3Dfr%26pr_id%3D7%26la_export%3Djs
libraries[kplatforms_tinymce_fr][overwrite] = TRUE

Impacted

includes/7/modules.make:libraries[kplatforms_tinymce_fr][overwrite] = TRUE

includes/6/modules.make:libraries[kplatforms_tinymce_fr][overwrite] = TRUE

Workaround

To manually download what is missing:

cd sites/all/libraries/tinymce
wget http://www.tinymce.com/i18n3x/index.php%3Fctrl%3Dexport%26act%3Dzip%26la%5B%5D%3Dfr%26pr_id%3D7%26la_export%3Djs
unzip index.php\?ctrl\=export\&act\=zip\&la\[\]\=fr\&pr_id\=7\&la_export\=js
mv tinymce_language_pack/* ./
rm -fr tinymce_language_pack index.php\?ctrl\=export\&act\=zip\&la\[\]\=fr\&pr_id\=7\&la_export\=js

References

Issue upstream for drush:

May be related to:

Internal reference numbers:

  • Redmine 12386
  • RT 286497

What to do when is resolved

Uncomment the libraries using "[overwrite] = TRUE". See what is impacted above.

Comments

j0nathan’s picture

Some code from /usr/share/php/drush/commands/make/make.project.inc

 34   /**
 35    * Default to overwrite to allow recursive builds to process properly.
 36    *
 37    * TODO refactor this to be more selective. Ideally a merge would take place
 38    * instead of an overwrite.
 39    */
 40   protected $overwrite = TRUE;
177     // This directory shouldn't exist yet -- if it does, stop,
178     // unless overwrite has been set to TRUE.
179     if (is_dir($this->download_location) && !$this->overwrite) {
180       drush_set_error('MAKE_DIRECTORY_EXISTS', dt('Directory not empty: !directory', array('!directory' => $this->download_location)));
181       return FALSE;
182     }
j0nathan’s picture

Issue summary: View changes
j0nathan’s picture

Issue summary: View changes
j0nathan’s picture

Issue summary: View changes
j0nathan’s picture

Based on https://drupal.org/files/drush-copy-overwrite-1539076-14.patch from https://drupal.org/comment/5982698#comment-5982698 with

+ *   Action to take if destination already exists.
+ *     - FILE_EXISTS_OVERWRITE - completely removes existing directory.
+ *     - FILE_EXISTS_ABORT - aborts the operation.
+ *     - FILE_EXISTS_MERGE - Leaves existing files and directories in place.

I tried some syntax without success

libraries[kplatforms_tinymce_fr][overwrite] = TRUE

libraries[kplatforms_tinymce_fr][merge] = TRUE

libraries[kplatforms_tinymce_fr][overwrite] = FILE_EXISTS_MERGE

libraries[kplatforms_tinymce_fr][overwrite] = merge

libraries[kplatforms_tinymce_fr][FILE_EXISTS_MERGE] = TRUE
j0nathan’s picture

Title: "[overwrite] = TRUE" does not work » "[overwrite] = TRUE" in makefile with Drush 5 overwrites instead of merging as in Drush make 2.3
Issue summary: View changes
j0nathan’s picture

Impacted:

$ grep -r -i overwrite *
includes/7/modules.make:libraries[kplatforms_geshi_puppet][overwrite] = TRUE
includes/7/modules.make:libraries[kplatforms_tinymce_fr][overwrite] = TRUE
includes/7/civicrm-42lts.make:libraries[civicrm_l10n][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_CA_gtn][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_CA][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_FR_gtn][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_FR][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_CA_gtn][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_CA][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_GB_gtn][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_GB][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_US_gtn][overwrite] = TRUE
includes/7/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_US][overwrite] = TRUE
includes/7/civicrm-43.make:libraries[civicrm_l10n][overwrite] = TRUE
includes/6/civicrm3.make:libraries[civicrm_l10n][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_geshi_puppet][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_clusterer2_js][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_passwordhash_php][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_tinymce_fr][overwrite] = TRUE
includes/6/civicrm-42lts.make:libraries[civicrm_l10n][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_CA_gtn][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_CA][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_FR_gtn][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_fr_FR][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_CA_gtn][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_CA][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_GB_gtn][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_GB][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_US_gtn][overwrite] = TRUE
includes/6/civicrm-l10n.make:libraries[civicrm_l10n_latest_en_US][overwrite] = TRUE
includes/6/civicrm-43.make:libraries[civicrm_l10n][overwrite] = TRUE
includes/6/unsupported.make:libraries[kplatforms_swfupload][overwrite] = TRUE
includes/6/unsupported.make:libraries[kplatforms_swfupload_swf][overwrite] = TRUE
includes/6/unsupported.make:libraries[kplatforms_swfupload_queue_js][overwrite] = TRUE
$ grep -r -i overwrite * | egrep -v "civicrm|unsupported"
includes/7/modules.make:libraries[kplatforms_geshi_puppet][overwrite] = TRUE
includes/7/modules.make:libraries[kplatforms_tinymce_fr][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_geshi_puppet][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_clusterer2_js][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_passwordhash_php][overwrite] = TRUE
includes/6/modules.make:libraries[kplatforms_tinymce_fr][overwrite] = TRUE
j0nathan’s picture

j0nathan’s picture

To manually download what is missing:

Drupal 7 and 6 :

cd sites/all/libraries/geshi/geshi
wget https://raw.github.com/jasonhancock/geshi-language-files/7fd7a709d857f74b78d42990a2381a45eeb93429/puppet.php
cd sites/all/libraries/tinymce
wget http://www.tinymce.com/i18n3x/index.php%3Fctrl%3Dexport%26act%3Dzip%26la%5B%5D%3Dfr%26pr_id%3D7%26la_export%3Djs
unzip index.php\?ctrl\=export\&act\=zip\&la\[\]\=fr\&pr_id\=7\&la_export\=js
mv tinymce_language_pack/* ./
rm -fr tinymce_language_pack index.php\?ctrl\=export\&act\=zip\&la\[\]\=fr\&pr_id\=7\&la_export\=js

Drupal 6 only :

cd sites/all/modules/contrib/gmap/thirdparty
wget http://www.acme.com/javascript/Clusterer2.js
cd sites/all/modules/contrib/phpass
wget http://cvsweb.openwall.com/cgi/cvsweb.cgi/~checkout~/projects/phpass/PasswordHash.php
j0nathan’s picture

I opened an issue upstream for drush:
https://github.com/drush-ops/drush/issues/295

j0nathan’s picture

Issue summary: View changes
j0nathan’s picture

I tried with drush 5.10 from debian package and with drush 6.3-dev from git.
I test with Drupal 7 makefiles (modules.make).

It seems "[overwrite] = TRUE" for only one file (geshi_puppet) works with both drush version.

It seems "[overwrite] = TRUE" to uncompress a zip file does not work with both drush version.

Here is the part of the makefile I used for geshi_puppet without overwrite:

libraries[kplatforms_geshi][directory_name] = geshi
libraries[kplatforms_geshi][download][type] = svn
libraries[kplatforms_geshi][download][url] = https://svn.code.sf.net/p/geshi/code/branches/RELEASE_1_0_X_STABLE/geshi-1.0.X/src/

Here is the part of the makefile I used for geshi_puppet with overwrite:

libraries[kplatforms_geshi][directory_name] = geshi
libraries[kplatforms_geshi][download][type] = svn
libraries[kplatforms_geshi][download][url] = https://svn.code.sf.net/p/geshi/code/branches/RELEASE_1_0_X_STABLE/geshi-1.0.X/src/
; GeSHi Puppet language definition
libraries[kplatforms_geshi_puppet][destination] = libraries/geshi
libraries[kplatforms_geshi_puppet][directory_name] = geshi
libraries[kplatforms_geshi_puppet][download][type] = get
libraries[kplatforms_geshi_puppet][download][url] = https://raw.github.com/jasonhancock/geshi-language-files/7fd7a709d857f74b78d42990a2381a45eeb93429/puppet.php
libraries[kplatforms_geshi_puppet][overwrite] = TRUE

Here is the part of the makefile I used for tinymce_fr without overwrite:

libraries[kplatforms_tinymce][directory_name] = tinymce
libraries[kplatforms_tinymce][download][type] = get
libraries[kplatforms_tinymce][download][url] = https://github.com/downloads/tinymce/tinymce/tinymce_3.4.9_jquery.zip

Here is the part of the makefile I used for tinymce_fr with overwrite:

libraries[kplatforms_tinymce][directory_name] = tinymce
libraries[kplatforms_tinymce][download][type] = get
libraries[kplatforms_tinymce][download][url] = https://github.com/downloads/tinymce/tinymce/tinymce_3.4.9_jquery.zip
libraries[kplatforms_tinymce_fr][directory_name] = tinymce
libraries[kplatforms_tinymce_fr][download][type] = get
libraries[kplatforms_tinymce_fr][download][url] = http://www.tinymce.com/i18n3x/index.php%3Fctrl%3Dexport%26act%3Dzip%26la%5B%5D%3Dfr%26pr_id%3D7%26la_export%3Djs
libraries[kplatforms_tinymce_fr][overwrite] = TRUE

Drush 5.10 without overwrite for tinymce has the directory jscripts:

~/platforms$ ls -l drupal-7-2014.01.06-drush510-sans-overwrite/sites/all/libraries/tinymce
-rw-r--r--  1 aegir aegir 142072 Feb 23  2012 changelog.txt
drwxr-xr-x  3 aegir aegir   4096 Jan  6 11:00 examples
drwxr-xr-x  3 aegir aegir   4096 Jan  6 11:00 jscripts

Drush 5.10 with overwrite for tinymce does not have the directory jscripts but has the directory langs:

~/platforms$ ls -l drupal-7-2014.01.06-drush510-avec-overwrite/sites/all/libraries/tinymce
drwxr-xr-x  2 aegir aegir 4096 Jan  6 10:46 langs
drwxr-xr-x 14 aegir aegir 4096 Jan  6 10:46 plugins
drwxr-xr-x  4 aegir aegir 4096 Jan  6 10:46 themes

Drush 6.3-dev without overwrite for tinymce has the directory jscripts:

~/platforms$ ls -l drupal-7-2014.01.06-drush63-sans-overwrite/sites/all/libraries/tinymce
-rw-r--r--  1 aegir aegir 142072 Feb 23  2012 changelog.txt
drwxr-xr-x  3 aegir aegir   4096 Jan  6 11:32 examples
drwxr-xr-x  3 aegir aegir   4096 Jan  6 11:32 jscripts

Drush 6.3-dev with overwrite for tinymce does not have the directory jscripts but has the directory langs:

~/platforms$ ls -l drupal-7-2014.01.06-drush63-avec-overwrite/sites/all/libraries/tinymce
drwxr-xr-x  2 aegir aegir 4096 Jan  6 10:29 langs
drwxr-xr-x 14 aegir aegir 4096 Jan  6 10:29 plugins
drwxr-xr-x  4 aegir aegir 4096 Jan  6 10:29 themes

The file puppet.php for geshi is available on the two platforms with overwrite and this is the only difference, which is the expected behaviour:

~/platforms$ diff -r -q drupal-7-2014.01.06-drush510-sans-overwrite/sites/all/libraries/geshi drupal-7-2014.01.06-drush510-avec-overwrite/sites/all/libraries/geshi
Only in drupal-7-2014.01.06-drush510-avec-overwrite/sites/all/libraries/geshi/geshi: puppet.php

~/platforms$ diff -r -q drupal-7-2014.01.06-drush63-sans-overwrite/sites/all/libraries/geshi drupal-7-2014.01.06-drush63-avec-overwrite/sites/all/libraries/geshi
Only in drupal-7-2014.01.06-drush63-avec-overwrite/sites/all/libraries/geshi/geshi: puppet.php
j0nathan’s picture

I tried with drush 5.10 from debian package.
I test with Drupal 6 modules.make.

It seems "[overwrite] = TRUE" for only one file works.
It seems "[overwrite] = TRUE" to uncompress a zip file does not work.

$ diff -r -q /var/aegir/platforms/drupal-6-2014.01.08-drush510-sans-overwrite /var/aegir/platforms/drupal-6-2014.01.08-drush510-avec-overwrite
===
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-sans-overwrite/sites/all/libraries/tinymce: changelog.txt
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-sans-overwrite/sites/all/libraries/tinymce: examples
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-sans-overwrite/sites/all/libraries/tinymce: jscripts
===
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-avec-overwrite/sites/all/libraries/tinymce: langs
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-avec-overwrite/sites/all/libraries/tinymce: plugins
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-avec-overwrite/sites/all/libraries/tinymce: themes
===
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-avec-overwrite/sites/all/libraries/geshi/geshi: puppet.php
===
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-avec-overwrite/sites/all/modules/contrib/gmap/thirdparty: Clusterer2.js
===
Only in /var/aegir/platforms/drupal-6-2014.01.08-drush510-avec-overwrite/sites/all/modules/contrib/phpass: PasswordHash.php

NB:
sans (fr) = without (en)
avec (fr) = with (en)

j0nathan’s picture

Title: "[overwrite] = TRUE" in makefile with Drush 5 overwrites instead of merging as in Drush make 2.3 » "[overwrite] = TRUE" for zip file with Drush 5 overwrites instead of merging as in Drush make 2.3
Issue summary: View changes
sluc23’s picture

Hi,
I confirm the issue using drush version 5.9
[overwrite ] option seems pretty useless after drush4 :(

  • Commit 455508b on 8.x-2.x, lock by j0nathan:
    Commented libraries overwrite in Drupal 7 (geshi_puppet, tinymce_fr) for...
  • Commit 5b00e72 on 8.x-2.x, lock by j0nathan:
    Commented libraries overwrite in Drupal 6 (geshi_puppet, clusterer2_js,...
  • Commit ea26fb2 on 8.x-2.x, lock by j0nathan:
    Issue #2131945 : re-enable "[overwrite] = TRUE" for single files in...
  • Commit f4d9e79 on 8.x-2.x, lock by j0nathan:
    Issue #2131945 : re-enable "[overwrite] = TRUE" for single files in...

  • j0nathan committed 455508b on 8.x-3.x
    Commented libraries overwrite in Drupal 7 (geshi_puppet, tinymce_fr) for...
  • j0nathan committed 5b00e72 on 8.x-3.x
    Commented libraries overwrite in Drupal 6 (geshi_puppet, clusterer2_js,...
  • j0nathan committed ea26fb2 on 8.x-3.x
    Issue #2131945 : re-enable "[overwrite] = TRUE" for single files in...
  • j0nathan committed f4d9e79 on 8.x-3.x
    Issue #2131945 : re-enable "[overwrite] = TRUE" for single files in...