I started getting this all of a sudden. Drush is at latest HEAD. Possibly PEBCAK. I tried searching and found things like #726970: Drush bootstrap problem on OS X 10.6 but didn't immediately seem like a duplicate.

webchicks-MacBook-Pro:Sites webchick$ drush dl drupal
rsync: link_stat "/var/folders/X4/X4SWL9zbEOO5EnKsj9viy++++TI/-Tmp-/drush_tmp_1287878009/drupal/." failed: No such file or directory (2)
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-40/rsync/main.c(992) [sender=2.6.9]
Project drupal (6.19) downloaded to /Users/webchick/Sites/drupal.    [success]
Project drupal contains 0 modules: .

Even though it says "success", the drupal folder is completely empty.

That weird-ass tmp directory seems to be coming from sys_get_temp_dir(). We removed that function call in D7 core due to problems it was causing with Mac users. (See http://drupal.org/node/551658#comment-2919160 and down) Not sure if that's also my problem here or something else.

CommentFileSizeAuthor
#16 drush-dl-no-tests.patch2.19 KBjonhattan
#12 drush-950928.patch3.97 KBjonhattan
#6 debug.txt25.55 KBwebchick
#3 drush-950928.patch2.6 KBjonhattan

Comments

moshe weitzman’s picture

moshe weitzman commented
Assigned: Unassigned » jonhattan
Category: support » bug
Priority: Normal » Major

We made some changes to this code recently and broke something. I can reproduce the error. Hoping jonhattan can take a look.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented

It is sys_get_temp_dir() and the rsync thing is because rename() fails in drush_move_dir().

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented
Status: Active » Needs review
StatusFileSize
new drush-950928.patch2.6 KB

So the only solution I think of is having our own file_directory_temp(). Patch attached.

I'm not happy with the highly unprobable dead end someone can reach with this patch. Is fine to just exit() if no tmp directory available?

OTOH, does http://es.php.net/manual/en/function.tmpfile.php raise a similar error? This function is used by core-cli and sql-query.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

This gives me:

webchicks-MacBook-Pro:Sites webchick$ drush dl drupal
rename(/tmp/drush_tmp_1288107124/x drupal-6.19,/tmp/drush_tmp_1288107124/drupal-6.19): No such file or [warning]
directory drush.inc:654
Project drupal (6.19) downloaded to /Users/webchick/Sites/drupal-6.19.                                 [success]
Project drupal contains:                                                                               [success]
 - 1 profile: default
 - 6 themes: pushbutton, minnelli, garland, marvin, chameleon, bluemarine
 - 33 modules: user, upload, update, trigger, translation, tracker, throttle, taxonomy, system, syslog,
statistics, search, profile, poll, ping, php, path, openid, node, menu, locale, help, forum, filter,
dblog, contact, comment, color, book, blogapi, blog, block, aggregator

So other than that odd warning, seems to be working great now. Yay!

And I had no idea about drush core-cli. OMG! How cool! Do you have suggestions on how I might test to see if the error occurs there? I did "dl cck" from within drush shell, and it did seem to work ok.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented

I think entering core-cli is enough but to be sure, test if you have cdd available (it is not a command but a custom bash function).

I'm confused about that spurious warning, and I think it is a bug if you use --drupal-project-rename. I think the problem is around tar -xvf in commands/pm/package_handler/wget.inc:50:63. The first line in the tar output is expected to be drupal-6.19/.

Can you provide the output of drush --debug dl drupal?. Note it is a big output so pipe it to a file.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
StatusFileSize
new debug.txt25.55 KB

Hm.

drush> help cdd

Fatal error: Call to undefined function conf_path() in /Users/webchick/drush/includes/environment.inc on line 692

Call Stack:
    0.0012     104292   1. {main}() /Users/webchick/drush/drush.php:0
    0.0206    1481816   2. drush_main() /Users/webchick/drush/drush.php:40
    0.1134    3339812   3. drush_dispatch() /Users/webchick/drush/drush.php:95
    0.1138    3342096   4. call_user_func_array() /Users/webchick/drush/includes/drush.inc:42
    0.1138    3342380   5. drush_command() /Users/webchick/drush/includes/drush.inc:0
    0.1150    3345796   6. call_user_func_array() /Users/webchick/drush/includes/command.inc:385
    0.1150    3345924   7. drush_invoke() /Users/webchick/drush/includes/command.inc:0
    0.1176    3354184   8. call_user_func_array() /Users/webchick/drush/includes/command.inc:318
    0.1176    3354468   9. drush_core_help() /Users/webchick/drush/includes/command.inc:0
    0.1177    3357068  10. drush_show_help() /Users/webchick/drush/commands/core/core.drush.inc:350
    0.1420    3369612  11. drush_bootstrap_validate() /Users/webchick/drush/includes/drush.inc:720
    0.1422    3370388  12. _drush_bootstrap_drupal_site_validate() /Users/webchick/drush/includes/environment.inc:246

Drush command could not be completed.                                                                  [error]

Here's the debug output.

moshe weitzman’s picture

moshe weitzman commented

just type cdd at the command prompt, not help cdd.

you can learn more about cdd with help core-cli

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

Is "x drupal-6.19" a valid filename? I thought those Xs next to drupal-6.19/themes/pushbutton/tabs-option-off-rtl.png et al were merely decorative.

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

Oops. :P

drush> cd drupal7/sites/all/modules/
drush> cdd
cd /Users/webchick/Sites/drupal7
drush> pwd
/Users/webchick/Sites/drupal7

Yes, it works fine. :)

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented

@webchick this is a Mac rarity. Expected output is:

Executing: tar -xvf drupal-6.19.tar [1.61 sec, 3.28 MB]                                                   [notice]
drupal-6.19/
drupal-6.19/includes/
drupal-6.19/includes/actions.inc
drupal-6.19/includes/batch.inc

I'd love to see also the output of tar -tf drupal-6.19.tar.gz (this is what drush used before a big change in pm-download). If it does not prepend x to each line we have to revert this part. If it's also x-prepended, a workaround will be needed, in despite of kittens.

It could be also need to test this on windows...

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented

tar -tf does not prepend X:

webchicks-MacBook-Pro:core webchick$ curl -LO http://ftp.drupal.org/files/projects/drupal-7.0-beta2.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 2577k  100 2577k    0     0   150k      0  0:00:17  0:00:17 --:--:--  159k
webchicks-MacBook-Pro:core webchick$ tar -tf drupal-7.0-beta2.tar.gz 
drupal-7.0-beta2/
drupal-7.0-beta2/includes/
drupal-7.0-beta2/includes/database/
drupal-7.0-beta2/includes/database/mysql/
drupal-7.0-beta2/includes/database/mysql/database.inc
drupal-7.0-beta2/includes/database/mysql/install.inc
jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented
StatusFileSize
new drush-950928.patch3.97 KB

So the fix is to revert to the old way: tar -xf & tar -tf

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
Status: Needs review » Reviewed & tested by the community

Yay! :D

webchicks-MacBook-Pro:Sites webchick$ drush dl drupal-7.x-dev
Project drupal (7.x-dev) downloaded to /Users/webchick/Sites/drupal-7.x-dev.                           [success]
Project drupal contains:                                                                               [success]
 - 3 profiles: testing, standard, minimal
 - 7 themes: update_test_subtheme, update_test_basetheme, test_theme, stark, seven, garland, bartik
 - 94 modules: user, aaa_update_test, bbb_update_test, ccc_update_test, update_test, update,
trigger_test, trigger, translation_test, translation, tracker, toolbar, taxonomy, system, syslog,
statistics, actions_loop_test, ajax_forms_test, ajax_test, batch_test, common_test, database_test,
entity_cache_test, entity_cache_test_dependency, entity_crud_hook_test, error_test, file_test,
filter_test, form_test, image_test, menu_test, module_test, requirements1_test, requirements2_test,
session_test, system_dependencies_test, system_test, taxonomy_test, theme_test, update_test_1,
update_test_2, update_test_3, url_alter_test, xmlrpc_test, simpletest, shortcut, search_embedded_form,
search_extra_type, search, rdf_test, rdf, profile, poll, php, path, overlay, openid_test, openid,
node_access_test, node_presave_test, node_test, node_test_exception, node, menu, locale_test, locale,
image_module_test, image, help, forum, filter, file_module_test, file, field_ui, field_test, text,
options, number, list_test, list, field_sql_storage, field, dblog, dashboard, contextual, contact,
comment, color, book, blog, block_test, block, aggregator_test, aggregator

(my, that's a lot of modules. :))

Marking RTBC. The only other thing I'd maybe suggest (if this was the Drupal core queue) is commenting the -tf / -xf lines if there's a high probability someone comes in later and tries to "clean up" this section of the code again.

Thanks so much for your quick work, jonhattan! Much appreciated.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented
Status: Reviewed & tested by the community » Fixed

Commited, with doc for the need of -tf.

moshe weitzman’s picture

moshe weitzman commented

Ideally, we should not be listing hidden modules after success, even though drush is quite capable of enabling them.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented
StatusFileSize
new drush-dl-no-tests.patch2.19 KB

Quick shot. not all the hidden ones but all in tests/.

The full solution is to grep .info files, but it will only catch testing profile.

$ rgrep "hidden = TRUE" drupal-7.x-dev/* | grep -v tests
drupal-7.x-dev/includes/common.inc:function drupal_add_tabledrag($table_id, $action, $relationship, $group, $subgroup = NULL, $source = NULL, $hidden = TRUE, $limit = 0) {
drupal-7.x-dev/profiles/testing/testing.info:hidden = TRUE

don't know if there're hidden but non-test modules in contrib/. If not, I like the approach in this patch. We can just add a special case for profile == testing.

tomorrow more/

webchick’s picture

webchick
she/they
Primary language English
Location Vancouver 🇨🇦
commented
Status: Fixed » Needs review

Seems like we should probably break this into another issue, but...

Here's the output this patch gives me (had to use cvs because ftp was timing out):

webchicks-MacBook-Pro:Sites webchick$ drush dl drupal-7.x-dev --package-handler=cvs
Project drupal (7.x-dev) downloaded to /Users/webchick/Sites/drupal-7.x-dev.                           [success]
Project drupal contains:                                                                               [success]
 - 3 profiles: testing, standard, minimal
 - 4 themes: stark, seven, garland, bartik
 - 45 modules: user, update, trigger, translation, tracker, toolbar, taxonomy, system, syslog,
statistics, simpletest, shortcut, search, rdf, profile, poll, php, path, overlay, openid, node, menu,
locale, image, help, forum, filter, file, field_ui, text, options, number, list, field_sql_storage,
field, dblog, dashboard, contextual, contact, comment, color, book, blog, block, aggregator

Much more betterer. :)

While there might be a use case for a non-test hidden = TRUE module (I'm thinking something like http://drupal.org/project/paranoia or http://drupal.org/project/killswitch), I would imagine that's by far the exception rather than the norm, so this approach should be fine. I guess it's possible that contrib also doesn't follow core's best practice of putting these modules into a sub-folder called 'tests', but then we could file task issues on those case-by-case.

I don't understand the code here enough to RTBC.

jonhattan’s picture

jonhattan
Primary language Spanish
Location Plasencia
commented
Status: Needs review » Fixed

Moved to a new issue #954702: Do not list test/hidden projects after drush dl

Status: Fixed » Closed (fixed)

Automatically closed -- issue fixed for 2 weeks with no activity.