Undefined index: protocol drush_make.download.inc:377 [#1114028]

When running drush_make in verbose mode, the error above message showed up.

Looking at the code, I noticed that $url_array['protocol'] never gets set. See the code below, the last line is line 377.

  $url_array = array();
  
  // Get the protocol, site and resource parts of the URL
  // original url = http://example.com/blog/index?name=foo
  // protocol = http://
  // site = example.com/
  // resource = blog/index?name=foo
  $regex = '#^(.*?//)*(.*@)*([\w\.\d]*)(:(\d+))*(/*)(.*)$#';
  $matches = array();
  preg_match($regex, $download['url'], $matches);
  // Assign the matched parts of url to the result array
  $url_array['user']     = $matches[2];
  $url_array['port']     = $matches[5];
  $url_array['host']     = $matches[3];
  $url_array['resource'] = $matches[7];
  
  // clean up the site portion by removing the trailing /
  $url_array['host'] = preg_replace('#/$#', '', $url_array['host']);
  
  // clean up the protocol portion by removing the trailing ://
  $url_array['protocol'] = preg_replace('#://$#', '', $url_array['protocol']);

I fixed this, by adding the following line (368):

  $url_array['protocol'] = $matches[1];

Btw. Is there any reason for not using the php parse_url function here?

After fixing this, I ran in to the following message (please note I was using --working-copy):

The authenticity of host 'git.drupal.org (140.211.10.6)' can't be established.
RSA key fingerprint is 69:35:9f:1b:a8:26:ad:6d:cc:93:29:25:6e:d7:25:2b.
Are you sure you want to continue connecting (yes/no)?

From the coment in the following code

        // github uses two different urls, working copy urls using scp format
        // git@domain:repo export url format are git://domain/repo
        if ($wc) {
          // working copy is set
          $url = 'git@'. $url_array['host'] .':'. $url_array['resource'];
          break;
        }

I concluded this was github specific, so I decided to add a check for github:

 if ($wc && $url_array['host'] == 'github.com') {

However, it might be a good idea to add an option for this, for extra security.

After fixing that, another typo show up:

        // @TODO: implement port & user options
        $url_array['protocol'] .'://'. $url_array['user'] . $url_array['host'] .'/'. $url_array['resource'];

Should be:

        // @TODO: implement port & user options
        $url = $url_array['protocol'] .'://'. $url_array['user'] . $url_array['host'] .'/'. $url_array['resource'];

After fixing this, everything works great again.

I'll attach a patch in a comment so I can include the issue number in the patch name.

Comment	File	Size	Author
#1	issue1114028-1.patch	1.41 KB	Anonymous (not verified)

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

Anonymous (not verified) CreditAttribution: Anonymous commented 2 April 2011 at 13:20

File	Size
issue1114028-1.patch	1.41 KB

So here's the patch

Comment #2

helmo CreditAttribution: helmo commented 14 April 2011 at 09:14

I noticed the same problem.
Your patch looks good although I would suggest to move the preg_replace on protocol into the else block. See my patch.

Rewriting this using parse_url() seems like an improvement, and even is there is a good reason against it refactoring this code into a separate function would be cleaner.

Comment #3

marafa CreditAttribution: marafa commented 12 June 2011 at 03:50

i had the same error
i then discovered http://drupal.org/node/1133048 specifically comment #3 http://drupal.org/node/1133048#comment-4376908.

so after install git (on rhel based) or git-core (i presume debian) the hostmaster install worked.

Comment #5

dmitrig01 CreditAttribution: dmitrig01 commented 28 July 2011 at 04:09

Status:

Needs review

» Fixed

fixed, thanks.

Comment #6

ao2 CreditAttribution: ao2 commented 1 August 2011 at 09:33

Status:

Fixed

» Needs work

Hi, sorry to reopen this one, but the patch in #1 breaks a couple of things:

Anonymous cloning using the git:// scheme does not work anymore on github, try with something like:

projects[geocode][type] = module
projects[geocode][subdir] = custom
projects[geocode][download][type] = git
projects[geocode][download][url] = git://github.com/phayes/geocode.git

after #1 the URL gets rewritten, but there is no reason to: even when cloning anonymously we still get a valid git repository, and I think we can very well consider this a "working copy" (BTW I couldn't find a definition of what a working copy is exactly in the README).

Maybe I don't understand why we need to mangle the git URL, shouldn't it be the makefile writer to decide if he wants to use the ssh transport (the user@host:path/to/repo.git syntax which can also be written as ssh://[user@]host.xz[:port]/path/to/repo.git/ see git clone manual) or the other transports?

Couldn't we ditch the check for $wc altogether?

@@ -391,23 +391,18 @@ function drush_make_download_git($name, $download, $download_location) {
     // currently all protocols seems to use the same url schema
     switch ($url_array['protocol']) {
       case 'git':
-        // github uses two different urls, working copy urls using scp format
-        // git@domain:repo export url format are git://domain/repo
-        if ($wc && $url_array['host'] == 'github.com') {
-          // working copy is set
-          $url = 'git@'. $url_array['host'] .':'. $url_array['resource'];
-          break;
-        }

The git file:// transport does not work anymore, the format for the components (user, host, port, resource) depend on the URL type (see again git clone manual), and maybe the parsing could even be done outside the git fetcher, or not done at all, letting git clone parse the URL...

This workaround makes cloning from the filesystem work again:

       case 'ssh':
       case 'http':
       case 'https':
       case 'ftp':
       case 'ftps':
       case 'rsync':
-      case 'file':
         // @TODO: implement port & user options
         $url = $url_array['protocol'] .'://'. $url_array['user'] . $url_array['host'] .'/'. $url_array['resource'];
         break;
+      case 'file':
+        $url = $download['url'];
+        break;

       default:
         drush_make_error('DOWNLOAD_ERROR', dt('unknown protocol @protocol in %project', array('@protocol' => $url_array['protocol'], '%project' => $name)));

If you agree I can send proper patches to address those issues.

Thanks,
Antonio

P.S. I also notice that the patch in #1 addresses _different_ issues in the _same_ patch, well let's try to avoid this, having a cleaner history helps finding out where problems are.

Comment #7

helmo CreditAttribution: helmo commented 16 December 2011 at 22:04

Status:

Needs work

» Closed (won't fix)

[ Powered by #1115636: Issue Macros and Templates - Drush Make]

Drush make is being merged into drush core (discussed in issue:#1310130: Put drush make in drush core)
This means that the issue queue is also moving. The Drush project has a component called 'Make' for this purpose.

We would like to take this opportunity to leave behind old/obsolete issues, allowing us to focus on a stable make command in core. E.g. one of the major tasks ahead is making more use of the Drush core code for handling downloads and decompression.

If you feel that this issue is still relevant, feel free to re-open and move it to the Drush queue.

More information will be posted on the drush_make and drush project pages.