I've made several commits recently where the commit messages have not been automatically added to the issues.

Examples:

Commit: http://cgit.drupalcode.org/drupal/commit/?id=f1b3b88
Issue: https://drupal.org/node/2091459

Commit: http://cgit.drupalcode.org/drupal/commit/?id=1dee6b7
Issue: https://drupal.org/node/2277623

Commit: http://cgit.drupalcode.org/drupal/commit/?id=122c9b4
Issue: https://drupal.org/node/2263047

Any ideas what is going on? I know the issue number in the commit message is OK, because when I look at those commits on
https://drupal.org/node/3060/commits?page=4
they link to the right issues.

Killed queue items:

a:2:{s:7:"repo_id";s:5:"65569";s:4:"elid";s:6:"959343";}
a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"961137";}
a:2:{s:7:"repo_id";s:5:"52530";s:4:"elid";s:6:"962929";}
a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"963329";}
a:2:{s:7:"repo_id";s:5:"35852";s:4:"elid";s:6:"963457";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"964221";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"965465";}
a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"968389";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972767";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972775";}
a:2:{s:7:"repo_id";s:5:"65569";s:4:"elid";s:6:"972993";}
a:2:{s:7:"repo_id";s:5:"35852";s:4:"elid";s:6:"973511";}
a:2:{s:7:"repo_id";s:5:"42576";s:4:"elid";s:6:"978983";}
a:2:{s:7:"repo_id";s:5:"68201";s:4:"elid";s:6:"981221";}
CommentFileSizeAuthor
#5 killed_events_data.txt1.86 KBmarvil07
Members fund testing for the Drupal project. Drupal Association Learn more

Comments

drumm’s picture

Assigned: Unassigned » drumm
Issue summary: View changes

During DrupalCon, the cron-versioncontrol_repository_event_processor-drupal.org job did get stuck for ~4 days. Certain commits cause it to OOM and currently have to be killed manually. I've added my log of killed queue items to the issue summary.

drumm’s picture

Assigned: drumm » Unassigned

I was able to re-run the queue worker on util for the three commits with code like:

versioncontrol_repository_event_processor_run_worker(array(
  'repo_id' => 2, // Core
  'elid' => db_query('SELECT elid FROM {versioncontrol_git_event_data} WHERE new_sha1 = :hash', array(':hash' => '122c9b4696f2e07b7afd6959ee34e57effed6557'))->fetchField(),
));

The comments now show up normally, although late. The code completed normally and quickly, so they were indeed not the cause of any problem, but collateral damage from last week's trouble.

Doing the same type of debugging on git7site with the killed queue items should run into the memory leak and let us get to the root cause.

jhodgdon’s picture

Thanks! Good luck with the debugging... I wasn't so concerned about having those particular commit messages added, as making sure someone was aware that it had happened. Looks like you already were.

marvil07’s picture

I have not yet tried to replicated this, but I took a look at the related event data for killed items and I found a pattern(update: see file in next comment, it just looks bad inline here).
All of those events are adding a new branch to a drupal core repository clone in a sandbox.
So by best guess is that it is trying to parse the full history of the branch, which start at drupal core repository first commit, which takes time and probably can cause a OOM.

marvil07’s picture

FileSize
1.86 KB
drumm’s picture

Issue summary: View changes

Adding a:2:{s:7:"repo_id";s:5:"18942";s:4:"elid";s:6:"968389";} to the killed list.

drumm’s picture

Issue summary: View changes

Adding

a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972767";}
a:2:{s:7:"repo_id";s:5:"47158";s:4:"elid";s:6:"972775";}
drumm’s picture

Issue summary: View changes

Adding

a:2:{s:7:"repo_id";s:5:"65569";s:4:"elid";s:6:"972993";}
a:2:{s:7:"repo_id";s:5:"35852";s:4:"elid";s:6:"973511";}
drumm’s picture

Title: Commit messages not being added to issues » Commit messages comment cron OOM
Issue summary: View changes

a:2:{s:7:"repo_id";s:5:"42576";s:4:"elid";s:6:"978983";}

drumm’s picture

Issue summary: View changes

a:2:{s:7:"repo_id";s:5:"68201";s:4:"elid";s:6:"981221";}

DanChadwick’s picture

A couple of days ago, commit messages for the webfor 7.x-4.x branch stopped being automatically posted. I generally push to both the 7.x-4.x branch and 8.x-4.x branch at the same time. I am getting commit messages for the 8.x-4.x branch.

List of recent commit messages:
https://www.drupal.org/node/7404/commits

Here's an example issue:
https://www.drupal.org/node/2467877

Here the 7.x-4.x branch commit that doesn't have a comment in the issue:
http://cgit.drupalcode.org/webform/commit/?id=755af13a6964bc621ad8795f0e...

drumm’s picture

jhodgdon’s picture

I'm having this problem the past several days with my Sandbox module:
https://www.drupal.org/sandbox/jhodgdon/2369943

This module used to get commit messages. For example, here is a closed issue from April with one:
#2873475: WSOD in the help-topic Page.
But there have been numerous commits in the past few days and none of them has shown up. Here's one example:
#2920479: Get rid of rendering in hook_help
http://cgit.drupalcode.org/sandbox-jhodgdon-2369943/commit/?id=ee7dd80

Should I make a new issue?

jhodgdon’s picture

About 22 hours ago, all the commit messages for the past week suddenly showed up in the module mentioned in #13. So, whatever was broken suddenly got fixed... thanks whoever took care of that!

drumm’s picture

That was me. The versioncontrol event processor queue which posts these comments, among other things, was jammed up for quite awhile until I tracked down the root cause.

markcarver’s picture

This seems to be happening again :-/ Last 6 commits haven't posted to the related issue: https://www.drupal.org/node/259843/commits

drumm’s picture

This is catching up now. A job needed to be deleted to unblock the queue.