Debian upgrade error 3.4 to 3.5 (hosting_git) [#2729891]

While upgrading a server from 3.4 to 3.5 debian package I noticed the error below.
The new platform was build OK, and the site was left running fine on the old platform.

PHP Fatal error: Cannot redeclare drush_provision_pre_provision_git_pull()

Workarounds

Disable the hosting_git feature before upgrading ... and make sure the hostmaster site is re-verified.
OR: Apply this patch to provision, after debian unpacked the code in a first attempt
OR: use our unstable debian repo deb http://debian.aegirproject.org unstable main, where the patch from this issue is build in.

More log:

Do you wish to run all pending updates? (y/n): y
Performed update: hosting_update_7000                                                                                                                                                                                                                                           [ok]
'all' cache was cleared.                                                                                                                                                                                                                                                        [success]
Finished performing updates.                                                                                                                                                                                                                                                    [ok]
PHP Fatal error:  Cannot redeclare drush_provision_pre_provision_git_pull() (previously declared in /data/aegir/hostmaster-7.x-3.4/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131) in /data/aegir/hostmaster-7.x-3.5/profiles/hostmaster/modules/aegir/hosting_git/pull/drush/provision_git_pull.drush.inc on line 27

Fatal error: Cannot redeclare drush_provision_pre_provision_git_pull() (previously declared in /data/aegir/hostmaster-7.x-3.4/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131) in /data/aegir/hostmaster-7.x-3.5/profiles/hostmaster/modules/aegir/hosting_git/pull/drush/provision_git_pull.drush.inc on line 27
Drush command terminated abnormally due to an unrecoverable error.                                                                                                                                                                                                              [error]
Error: Cannot redeclare drush_provision_pre_provision_git_pull() (previously declared in /data/aegir/hostmaster-7.x-3.4/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131) in
/data/aegir/hostmaster-7.x-3.5/profiles/hostmaster/modules/aegir/hosting_git/pull/drush/provision_git_pull.drush.inc, line 27
The external command could not be executed due to an application error.                                                                                                                                                                                                         [error]

Comment	File	Size	Author
#5	debian_upgrade_error-2729891-5.patch	1.08 KB	helmo
#4	debian_upgrade_error-2729891-4.patch	2.02 KB	helmo

Support from Acquia helps fund testing for Drupal Acquia logo

Comments

Comment #1

20 May 2016 at 08:41

helmo created an issue. See original summary.

Comment #2

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 08:43

Running apt-get install again did not help.

One workaround can be to manually edit ~aegir/.drush/drushrc.php and remove the line about 'git_pull' from the $options['include'] array.

Comment #3

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 09:24

Another workaround could be to disable the hosting_git feature before upgrading ... and make sure the hostmaster site is re-verified.

Comment #4

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 10:22

Status:

Active

» Needs review

File	Size
debian_upgrade_error-2729891-4.patch	2.02 KB

My first workaround to manually edit ~aegir/.drush/drushrc.php seems to fail .. disabling the hosting_git module worked.

This untested patch might help in updating the ~aegir/.drush/drushrc.php earlier ...

Comment #5

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 12:25

Status:

Needs review

» Needs work

File	Size
debian_upgrade_error-2729891-5.patch	1.08 KB

:(
PHP Fatal error: Call to undefined function hosting_get_features() in /usr/share/drush/commands/provision/Provision/Config/Drushrc/Aegir.php on line 22

hosting_get_features() will only be available after bootstrapping the hostmaster site, which is what I was trying to avoid here.

provision-deploy has DRUSH_BOOTSTRAP_DRUPAL_ROOT

When I add require_once DRUPAL_ROOT . '/profiles/hostmaster/modules/aegir/hosting/hosting.features.inc'; it's missing the cache_get function ... then raising the level to DRUSH_BOOTSTRAP_DRUPAL_DATABASE gives:
* PHP Fatal error: Call to undefined function drupal_system_listing() in /var/aegir/hostmaster-7.x-3.x-2016-05-20-1406/profiles/hostmaster/modules/aegir/hosting/hosting.features.inc on line 275

Strange as it seemed ... bootstrapping to DRUSH_BOOTSTRAP_DRUPAL_FULL worked, see new patch.

Comment #6

omega8cc CreditAttribution: omega8cc commented 20 May 2016 at 13:01

Ouch, forgot to report this! We have seen this on BOA already.

It is a direct result of these two commits:

http://drupalcode.org/project/hosting_git.git/commit/4d88d2a
http://drupalcode.org/project/hosting_git.git/commit/be99788

This issue should be moved to https://www.drupal.org/project/hosting_git perhaps?

Comment #7

omega8cc CreditAttribution: omega8cc commented 20 May 2016 at 13:04

Plus, https://www.drupal.org/project/hosting_git after these commits no longer works! The Git tasks are greyed out as unavailable.

We had to revert to previous version to make it work.

Comment #8

ergonlogic

he/him

English

Montréal, Québec 🇨🇦

CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented 20 May 2016 at 13:05

Status:

Needs work

» Reviewed & tested by the community

tested, and confirmed that the patch in #5 works nicely.

Comment #9

ergonlogic

he/him

English

Montréal, Québec 🇨🇦

CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented 20 May 2016 at 13:11

Re. #6, this is really an issue about stability upgrading the hostmaster site in general. So this is a good change either way. As a follow-up, we should probably split out the common code into a separate function.

Re. #7, hosting_git works for me with those commits. But if you're seeing issues, then that would be a separate issue for hosting_git :)

Comment #10

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 18:26

Issue summary:

View changes

Comment #11

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 18:41

Issue summary:

View changes

Comment #12

ShaneOnABike CreditAttribution: ShaneOnABike commented 20 May 2016 at 19:08

For me I was receiving the following error when I do the apt-get upgrade because of this issue.

PHP Fatal error:  Cannot redeclare drush_provision_pre_provision_git_pull() (previously declared in /var/aegir/hostmaster-7.x-3.4/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131) in /var/aegir/hostmaster-7.x-3.5/profiles/hostmaster/modules/aegir/hosting_git/pull/drush/provision_git_pull.drush.inc on line 27
Drush command terminated abnormally due to an unrecoverable error.   [error]
Error: Cannot redeclare drush_provision_pre_provision_git_pull()
(previously declared in
/var/aegir/hostmaster-7.x-3.4/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131)
in
/var/aegir/hostmaster-7.x-3.5/profiles/hostmaster/modules/aegir/hosting_git/pull/drush/provision_git_pull.drush.inc,
line 27
The external command could not be executed due to an application     [error]
error.

I can report that the patch in #2729891-5 to Drush works beautifully!

Comment #13

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 19:19

Status:

Reviewed & tested by the community

» Needs work

Grumbles ... on the manual install it has failed a bit earlier ... the just in the provision_drupal_push_site() call, called from drush_provision_drupal_pre_provision_deploy

Changed permissions of /var/aegir/hostmaster-7.x-3.5/sites/hostmaster.example.net/drushrc.php to 440                                                                                                                                                                           [success]
Platforms path /var/aegir/platforms exists.                                                                                                                                                                                                                                     [success]
Platforms ownership of /var/aegir/platforms has been changed to aegir.                                                                                                                                                                                                          [success]
Platforms permissions of /var/aegir/platforms have been changed to 755.                                                                                                                                                                                                         [success]
Platforms path /var/aegir/platforms is writable.                                                                                                                                                                                                                                [success]
PHP Fatal error:  Cannot redeclare drush_provision_pre_provision_git_pull() (previously declared in /var/aegir/hostmaster-7.x-3.x-2016-03-27-1141/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131) in /var/aegir/hostmaster-7.x-3.5/profiles/hostmaster/mod
ules/aegir/hosting_git/pull/drush/provision_git_pull.drush.inc on line 27

Fatal error: Cannot redeclare drush_provision_pre_provision_git_pull() (previously declared in /var/aegir/hostmaster-7.x-3.x-2016-03-27-1141/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131) in /var/aegir/hostmaster-7.x-3.5/profiles/hostmaster/modules/
aegir/hosting_git/pull/drush/provision_git_pull.drush.inc on line 27
Drush command terminated abnormally due to an unrecoverable error.                                                                                                                                                                                                              [error]
Error: Cannot redeclare drush_provision_pre_provision_git_pull() (previously declared in /var/aegir/hostmaster-7.x-3.x-2016-03-27-1141/profiles/hostmaster/modules/aegir/hosting_git/drush/provision_git.drush.inc:131) in
/var/aegir/hostmaster-7.x-3.5/profiles/hostmaster/modules/aegir/hosting_git/pull/drush/provision_git_pull.drush.inc, line 27
The external command could not be executed due to an application error.                                                                                                                                                                                                         [error]
Changed permissions of /var/aegir/.drush/hostmaster.alias.drushrc.php to 640

Comment #14

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 20 May 2016 at 19:55

I've published a 3.5.1 Debian package, fixing atleast part of the problem pending further investigation on the manual installs

Comment #15

gboudrias CreditAttribution: gboudrias at Praxis Labs Coop commented 23 May 2016 at 17:53

I don't know if this warrants a new issue but the stable repo is upgrading things to unstable versions**:

aegir@vps59087:~$ cat /etc/apt/sources.list.d/debian_aegirproject_org.list 
deb http://debian.aegirproject.org stable main

aegir@vps59087:~$ ls -alh /var/aegir/
[...]
drwxr-xr-x  9 aegir aegir 4.0K Nov 10  2015 hostmaster-7.x-3.2
drwxr-xr-x  9 aegir aegir 4.0K Feb 24 14:51 hostmaster-7.x-3.x-2016-05-23-1339
[...]

Comment #16

memtkmcc CreditAttribution: memtkmcc commented 27 May 2016 at 02:59

This hotfix (in Provision) breaks upgrade from Aegir 2.x

http://cgit.drupalcode.org/provision/commit/?id=c4d8621
http://cgit.drupalcode.org/provision/commit/?id=424298a

It results with:

  views_view
  vocabulary
  vocabulary_node_types
Drush bootstrap phase : bootstrap_drupal_full() [8.4 sec, 8.67 MB]                                                                   [bootstrap]
exception 'PDOException' with message 'SQLSTATE[42S02]: Base table or view not found: 1146 Table 'o5tquadtto8io_0.blocked_ips'           [error]
doesn't exist' in /data/disk/o5t/aegir/distro/002/includes/database/database.inc:2171
Stack trace:
#0 /data/disk/o5t/aegir/distro/002/includes/database/database.inc(2171): PDOStatement->execute(Array)
#1 /data/disk/o5t/aegir/distro/002/includes/database/database.inc(683): DatabaseStatementBase->execute(Array, Array)
#2 /data/disk/o5t/aegir/distro/002/includes/database/database.inc(2350): DatabaseConnection->query('SELECT 1 FROM {...', Array,
Array)
#3 /data/disk/o5t/aegir/distro/002/includes/bootstrap.inc(1935): db_query('SELECT 1 FROM {...', Array)
#4 /data/disk/o5t/aegir/distro/002/includes/bootstrap.inc(1948): drupal_is_denied('127.0.0.1')
#5 /data/disk/o5t/aegir/distro/002/includes/bootstrap.inc(2399): drupal_block_denied('127.0.0.1')
#6 /data/disk/o5t/aegir/distro/002/includes/bootstrap.inc(2254): _drupal_bootstrap_page_cache()
#7 /data/disk/o5t/tools/drush/lib/Drush/Boot/DrupalBoot7.php(85): drupal_bootstrap(7)
#8 /data/disk/o5t/tools/drush/includes/bootstrap.inc(354): Drush\Boot\DrupalBoot7->bootstrap_drupal_full()
#9 /data/disk/o5t/.drush/sys/provision/platform/deploy.provision.inc(169): drush_bootstrap(5)
#10 [internal function]: drush_provision_drupal_provision_deploy('/data/disk/o5t/...')
#11 /data/disk/o5t/tools/drush/includes/command.inc(366): call_user_func_array('drush_provision...', Array)
#12 /data/disk/o5t/tools/drush/includes/command.inc(217): _drush_invoke_hooks(Array, Array)
#13 [internal function]: drush_command('/data/disk/o5t/...')
#14 /data/disk/o5t/tools/drush/includes/command.inc(185): call_user_func_array('drush_command', Array)
#15 /data/disk/o5t/tools/drush/lib/Drush/Boot/BaseBoot.php(67): drush_dispatch(Array)
#16 /data/disk/o5t/tools/drush/includes/preflight.inc(66): Drush\Boot\BaseBoot->bootstrap_and_dispatch()
#17 /data/disk/o5t/tools/drush/drush.php(12): drush_main()
#18 {main} [8.46 sec, 10.33 MB]
Changes made in drush_provision_drupal_pre_provision_deploy have been rolled back. [8.46 sec, 11.41 MB]                                  [debug]
Dropping database o5tquadtto8io_0 [8.46 sec, 11.42 MB]                                                                                  [notice]

Comment #17

ergonlogic

he/him

English

Montréal, Québec 🇨🇦

CreditAttribution: ergonlogic at Ergon Logic Enterprises for Aegir Cooperative commented 27 May 2016 at 15:22

I think we may want to revert this patch, and just update the release notes to highlight the need to disable hosting_git (if it's in use) prior to the upgrade.

Comment #18

kenfordesign CreditAttribution: kenfordesign as a volunteer commented 15 June 2016 at 19:21

Re #15, Has this issue been addressed?

I had the same problem when using apt-get upgrade from 7.x-3.4 to 7.x-3.5.

The script created a hostmaster using the 7.x-3.x dev branch similar to your output. Automated backups stopped working also. This happened on 2 different server instances. Both had identical behavior.

Comment #19

gboudrias CreditAttribution: gboudrias at Praxis Labs Coop commented 15 June 2016 at 20:03

Seems like the upgrade removed the cron entry... Not sure if this is related to this issue though.

Comment #20

kenfordesign CreditAttribution: kenfordesign as a volunteer commented 16 June 2016 at 13:03

Has the issue been addressed with aegir3 upgrade to unstable versions?

Comment #21

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 16 June 2016 at 15:01

Has the issue been addressed with aegir3 upgrade to unstable versions

No, but I don't think we have a proper issue to track that.

We should probably do a 3.6 soonish to get 7.44 in anyway.

Comment #22

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 16 June 2016 at 20:04

@ #16 (upgrade from 2.x)

The only alternative I could think of is to add a version check ... could we in that code determine it's an upgrade from 2.x?

Comment #23

memtkmcc CreditAttribution: memtkmcc commented 17 June 2016 at 11:41

Maybe there are more reliable checks to use, but we could just check (early enough) if some D6 specific table exists, which doesn't exist in D7. Or if some D7 specific table already exists. Something like:

if (db_table_exists('node_revisions')) {
  ### this is Aegir 2.x, because in D7 it was renamed to node_revision
}

if (db_table_exists('field_data_body')) {
  ### this is Aegir 3.x
}

Comment #24

memtkmcc CreditAttribution: memtkmcc commented 17 June 2016 at 11:53

Hmm.. but in function drush_provision_drupal_provision_deploy() it is apparently too late for this type of check.

Comment #25

ergonlogic

he/him

English

Montréal, Québec 🇨🇦

CreditAttribution: ergonlogic commented 17 June 2016 at 12:43

There's a VERSION constant: https://api.drupal.org/api/drupal/modules%21system%21system.module/const......

Comment #26

memtkmcc CreditAttribution: memtkmcc commented 17 June 2016 at 13:15

Yes, checking VERSION is even simpler, just not sure which check will work reliably when we are already in function drush_provision_drupal_provision_deploy().

Comment #27

helmo CreditAttribution: helmo at Initfour websolutions for Aegir Cooperative commented 22 June 2016 at 19:57

I've been thinking about this a bit more

The drush_provision_drupal_provision_deploy hook where the offending code is will be executing on the new platform in a 2.x to 3.x migration scenario ... so that would make VERSION == 7.44 having no distinction from a 3.4 to 3.5 upgrade.

I'll revert my hotfix tomorrow.