We are running Hosting version 3.13.0. When a user queued up a migration of about 40 sites on a single platform, many of the migrations failed with errors in their task logs indicating that the maximum CPU load had been exceeded. This same operation worked fine before we upgraded Hosting from version 3.9 to 3.13.0 and we have never seen this error before. Testing showed that queued tasks were being released to run even when the number of running tasks exceeded the queue maximum set via the "Hosting-->Queues" management page. This resulted in such a large number of concurrently-running tasks that the maximum for CPU load set by Hosting was exceeded.

The root cause of this is that the 'hosting_task_count_running()' function in 'hosting_task.module' always returns 0 for the number of running tasks. The SQL query that determines the number does a subtraction of an integer from the value returned by the 'NOW()' SQL function. That function returns a string, so the result is invalid and causes the query to return 0.

The attached patch fixes this by using the 'UNIX_TIMESTAMP()' SQL function instead of 'NOW()'.

CommentFileSizeAuthor
#2 hosting_task_count_running_fix-#2945421-comment_#2.patch785 byteskfolsom

Comments

kfolsom created an issue. See original summary.

kfolsom’s picture

kfolsom commented
StatusFileSize
new hosting_task_count_running_fix-#2945421-comment_#2.patch785 bytes

Patch to fix the bug.

jon pugh’s picture

jon pugh
he/him
Primary language English
Location Newburgh, NY
commented
Priority: Normal » Critical
Status: Active » Needs review

Wow, so this means that this function never worked as intended?

Looks great to me. A simple query reveals NOW() returns a human readable date time!

mysql> select NOW();
+---------------------+
| NOW()               |
+---------------------+
| 2018-02-20 16:14:04 |
+---------------------+
1 row in set (0.00 sec)

mysql> select UNIX_TIMESTAMP();
+------------------+
| UNIX_TIMESTAMP() |
+------------------+
|       1519143253 |
+------------------+
1 row in set (0.00 sec)
kfolsom’s picture

kfolsom commented

Thanks for confirming. I stared at the query for a while before trying the NOW() on its own and realizing what was going on. I'm not a frequent-enough user of raw SQL to recall the differences in the function returns. I'm glad it was a simple fix for our problem of blowing through the CPU load limit on running tasks.

  • helmo committed ce29543 on 7.x-3.x authored by kfolsom 
    Issue #2945421 by kfolsom: Fix for hosting_task_count_running() Query
helmo’s picture

helmo commented
Status: Needs review » Fixed

Thanks, committed

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.