I've made several commits recently where the commit messages have not been automatically added to the issues.

Examples:

Commit: http://cgit.drupalcode.org/drupal/commit/?id=f1b3b88
Issue: https://drupal.org/node/2091459

Commit: http://cgit.drupalcode.org/drupal/commit/?id=1dee6b7
Issue: https://drupal.org/node/2277623

Commit: http://cgit.drupalcode.org/drupal/commit/?id=122c9b4
Issue: https://drupal.org/node/2263047

Any ideas what is going on? I know the issue number in the commit message is OK, because when I look at those commits on
https://drupal.org/node/3060/commits?page=4
they link to the right issues.

Killed queue items:

a:2:{s:7:"repo_id";s:5:"65569";s:4:"elid";s:6:"959343";}
a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"961137";}
a:2:{s:7:"repo_id";s:5:"52530";s:4:"elid";s:6:"962929";}
a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"963329";}
a:2:{s:7:"repo_id";s:5:"35852";s:4:"elid";s:6:"963457";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"964221";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"965465";}
a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"968389";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972767";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972775";}
a:2:{s:7:"repo_id";s:5:"65569";s:4:"elid";s:6:"972993";}
a:2:{s:7:"repo_id";s:5:"35852";s:4:"elid";s:6:"973511";}
a:2:{s:7:"repo_id";s:5:"42576";s:4:"elid";s:6:"978983";}
a:2:{s:7:"repo_id";s:5:"68201";s:4:"elid";s:6:"981221";}
Files: 
CommentFileSizeAuthor
#5 killed_events_data.txt1.86 KBmarvil07

Comments

drumm’s picture

Assigned: Unassigned » drumm
Issue summary: View changes

During DrupalCon, the cron-versioncontrol_repository_event_processor-drupal.org job did get stuck for ~4 days. Certain commits cause it to OOM and currently have to be killed manually. I've added my log of killed queue items to the issue summary.

drumm’s picture

Assigned: drumm » Unassigned

I was able to re-run the queue worker on util for the three commits with code like:

versioncontrol_repository_event_processor_run_worker(array(
  'repo_id' => 2, // Core
  'elid' => db_query('SELECT elid FROM {versioncontrol_git_event_data} WHERE new_sha1 = :hash', array(':hash' => '122c9b4696f2e07b7afd6959ee34e57effed6557'))->fetchField(),
));

The comments now show up normally, although late. The code completed normally and quickly, so they were indeed not the cause of any problem, but collateral damage from last week's trouble.

Doing the same type of debugging on git7site with the killed queue items should run into the memory leak and let us get to the root cause.

jhodgdon’s picture

Thanks! Good luck with the debugging... I wasn't so concerned about having those particular commit messages added, as making sure someone was aware that it had happened. Looks like you already were.

marvil07’s picture

I have not yet tried to replicated this, but I took a look at the related event data for killed items and I found a pattern(update: see file in next comment, it just looks bad inline here).
All of those events are adding a new branch to a drupal core repository clone in a sandbox.
So by best guess is that it is trying to parse the full history of the branch, which start at drupal core repository first commit, which takes time and probably can cause a OOM.

marvil07’s picture

FileSize
1.86 KB
drumm’s picture

Issue summary: View changes

Adding a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"968389";} to the killed list.

drumm’s picture

Issue summary: View changes

Adding

a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972767";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972775";}
drumm’s picture

Issue summary: View changes

Adding

a:2:{s:7:"repo_id";s:5:"65569";s:4:"elid";s:6:"972993";}
a:2:{s:7:"repo_id";s:5:"35852";s:4:"elid";s:6:"973511";}
drumm’s picture

Title: Commit messages not being added to issues » Commit messages comment cron OOM
Issue summary: View changes

a:2:{s:7:"repo_id";s:5:"42576";s:4:"elid";s:6:"978983";}

drumm’s picture

Issue summary: View changes

a:2:{s:7:"repo_id";s:5:"68201";s:4:"elid";s:6:"981221";}

DanChadwick’s picture

A couple of days ago, commit messages for the webfor 7.x-4.x branch stopped being automatically posted. I generally push to both the 7.x-4.x branch and 8.x-4.x branch at the same time. I am getting commit messages for the 8.x-4.x branch.

List of recent commit messages:
https://www.drupal.org/node/7404/commits

Here's an example issue:
https://www.drupal.org/node/2467877

Here the 7.x-4.x branch commit that doesn't have a comment in the issue:
http://cgit.drupalcode.org/webform/commit/?id=755af13a6964bc621ad8795f0e...

drumm’s picture