Call to undefined function hosting_get_most_recent_task [#2755693]

After enabling the hosting backup queue module in aegir hostmaster 3.6, aegir fails to perform any queue task.

I've found the cause of this by manually calling the dispatch from the command line:

drush @hostmaster hosting-dispatch
PHP Fatal error: Call to undefined function hosting_get_most_recent_task() in var/aegir/hostmaster7.x-3.6/profiles/hostmaster/modules/aegir/hosting_site_backup_manager/hosting_backup_queue/hosting_backup_queuemodule on line 71
Drush command terminated abnormally due to an unrecoverable error.

So this failing command prevents all other cron process from taking place.

Comments

Comment #1

25 June 2016 at 17:44

Neograph734 created an issue. See original summary.

Comment #2

gboudrias CreditAttribution: gboudrias at Praxis Labs Coop commented 25 June 2016 at 18:46

Priority:

Major

» Critical

If someone can confirm this bug it is indeed important as it would break all queues.

Comment #3

gboudrias CreditAttribution: gboudrias at Praxis Labs Coop commented 25 June 2016 at 18:59

I can't reproduce this on my staging server. Can you list your versions (PHP, etc)?

Since this may be important I would like others to check if they can reproduce this bug.

Comment #4

Neograph734

he/him

Netherlands

CreditAttribution: Neograph734 as a volunteer commented 25 June 2016 at 19:11

I am currently away from the computer. Will provide a list of versions and enabled modules tomorrow.

Comment #5

Neograph734

he/him

Netherlands

CreditAttribution: Neograph734 as a volunteer commented 26 June 2016 at 10:39

As of now I can no longer reproduce the failing of scheduled tasks. I just updated several system packages and maybe that solved it. The php error is still there when running drush @hostmaster hosting-dispatch, but it no longer appears to be blocking the processing of tasks. Below you'll find the post I was initially writing.

Let me start by explaining my installation was far from ideal because I managed to install exactly between the releases of provisioning and hostmaster. (Provisioning failed to download hostmaster-7.x-3.6 as it was not yet released and perhaps both packages should be published in another order? On the other hand, I was probably the only one unlucky enough to experience this.) I resumed the installation a couple of hours later and it worked. I don't expect this to be related, but it might.

My development machine is a vmware virtual machine running in Windows. It has 1GB of ram, 1 processor and 20 GB of disk space.

The versions are
Centos: 7.2.1511
Apache: 2.4.6
PHP: 5.6.22

Enabled Drupal modules:

[aegir@aegir ~]$ drush @hostmaster pm-list --type=module --status=enabled
 Package         Name                                               Version
 Administration  Actions permissions (VBO) (actions_permissions)    7.x-3.3
 Administration  Administration menu (admin_menu)                   7.x-3.0-rc5
 Aegir Backup    Backup queue (hosting_backup_queue)                7.x-3.5
 Aegir Backup    Hosting Site Backup Manager                        7.x-3.5
                 (hosting_site_backup_manager)
 Aegir Extras    Hosting Drush Alias (hosting_drush_alias)          7.x-3.5
 Aegir Extras    Hosting Sync (hosting_sync)                        7.x-3.5
 Aegir Extras    Hosting tasks extra (hosting_tasks_extra)          7.x-3.5
 Chaos tool      Chaos tools (ctools)                               7.x-1.9
 suite
 Core            Block (block)                                      7.44
 Core            Color (color)                                      7.44
 Core            Database logging (dblog)                           7.44
 Core            Field (field)                                      7.44
 Core            Field SQL storage (field_sql_storage)              7.44
 Core            Filter (filter)                                    7.44
 Core            Help (help)                                        7.44
 Core            Menu (menu)                                        7.44
 Core            Node (node)                                        7.44
 Core            Overlay (overlay)                                  7.44
 Core            System (system)                                    7.44
 Core            Text (text)                                        7.44
 Core            Update manager (update)                            7.44
 Core            User (user)                                        7.44
 Hosting         Client (hosting_client)                            7.x-3.6
 Hosting         Client Quota (hosting_quota)                       7.x-3.6
 Hosting         Database Server (hosting_db_server)                7.x-3.6
 Hosting         DNS support (hosting_dns)                          7.x-3.6
 Hosting         Hosting (hosting)                                  7.x-3.6
 Hosting         Hosting Cron (hosting_cron)                        7.x-3.6
 Hosting         Hosting queue daemon (hosting_queued)              7.x-3.6
 Hosting         Hosting task garbage collection (hosting_task_gc)  7.x-3.6
 Hosting         Hosting tasks (hosting_task)                       7.x-3.6
 Hosting         Package management (hosting_package)               7.x-3.6
 Hosting         Platforms (hosting_platform)                       7.x-3.6
 Hosting         Server (hosting_server)                            7.x-3.6
 Hosting         Site migration (hosting_migrate)                   7.x-3.6
 Hosting         Sites (hosting_site)                               7.x-3.6
 Hosting         SSL support (hosting_ssl)                          7.x-3.6
 Hosting         Web Server (hosting_web_server)                    7.x-3.6
 Other           Entity API (entity)                                7.x-1.7
 Other           Overlay paths (overlay_paths)                      7.x-1.3
 Other           Redirect 403 to User Login (r4032login)            7.x-1.8
 User interface  Better Login (betterlogin)                         7.x-1.4
 Views           Views (views)                                      7.x-3.14
 Views           Views Bulk Operations (views_bulk_operations)      7.x-3.3

Comment #6

gboudrias CreditAttribution: gboudrias at Praxis Labs Coop commented 26 June 2016 at 15:12

Priority:

Critical

» Major

It may be hard to reproduce since a lot of us are on Debian.

(Set to "major" as it doesn't seem to break everything anymore.)

Comment #7

helmo CreditAttribution: helmo at Initfour websolutions commented 26 June 2016 at 19:48

Status:

Active

» Fixed

I see that the missing function was removed in #2474801: Task cancellation broken

I've now restored it. You can apply
this patch of upgrade to the dev version.

PS: About the packaging delay during the 3.6 release ... it took 110 minutes before the hostmaster package was build ... normally that's just a few. :( Luckily someone from the D.o team fixed the package build.

Comment #8

Neograph734

he/him

Netherlands

CreditAttribution: Neograph734 as a volunteer commented 26 June 2016 at 19:56

Thanks a lot for the fast response guys.

Regarding the ps, it explains a lot.

Comment #9

sozkara CreditAttribution: sozkara commented 27 June 2016 at 11:13

I just upgraded my aegir and the error is still there. There are no backups created still.
Error: Call to undefined function hosting_get_most_recent_task() in .../aegir/distro/007/profiles/hostmaster/modules/aegir/hosting_site_backup_manager/hosting_backup_queue/hosting_backup_queue.module

Comment #10

helmo CreditAttribution: helmo at Initfour websolutions commented 27 June 2016 at 11:42

@sozkara did you upgrade to the dev version?

Check that hosting/task/hosting_task.module has the function as shown in http://cgit.drupalcode.org/hosting/commit/task/hosting_task.module?id=31...

Comment #11

Neograph734

he/him

Netherlands

CreditAttribution: Neograph734 as a volunteer commented 27 June 2016 at 11:43

sozkara the hostmaster distribution is not yet updated. You need to download the dev version of the hosting module manually and place it in your installation profile (/var/aegir/hostmaster-7.x-3.6/profiles/hostmaster/modules/hosting). That solved it for me.

You might be able to download it with drush @hostmaster dl hosting, but that results in the same module being available to Drupal twice and potentially leads to stange behavior.

It worked for me and the error is gone.

Comment #12

sozkara CreditAttribution: sozkara commented 27 June 2016 at 12:31

@helmo i applied the patch as per http://cgit.drupalcode.org/hosting/patch/task/hosting_task.module?id=313...
error is gone from the hosting-dispatch, but still no backups created in task queue.

edit: same with dev version. no errors, but no backup tasks either.

Comment #13

sozkara CreditAttribution: sozkara commented 27 June 2016 at 14:15

Can this be related?

Acquired lock on backup_queue queue. [1.4 sec, 27.98 MB]                                                                 [notice]
Too early for queue backup_queue. [1.4 sec, 27.98 MB]                                                                    [notice]
Releasing backup_queue lock. [1.4 sec, 27.99 MB]                                                                         [notice]
Acquired lock on backup_gc queue. [1.51 sec, 28 MB]                                                                      [notice]
Too early for queue backup_gc. [1.51 sec, 28 MB]                                                                         [notice]
Releasing backup_gc lock. [1.51 sec, 28 MB]                                                                              [notice]

Comment #14

Neograph734

he/him

Netherlands

CreditAttribution: Neograph734 as a volunteer commented 27 June 2016 at 14:16

@sozkara since it is not related to the error anymore (thus not related to this issue), I've created a separate issue for this #2756411: Backup tasks are not being generated.

UPDATE:

Yes that could be related. Could you place that in the other issue as well please?