The destination= query parameter for the delete links is set wrongfully to /heartbeat/js/poll when the heartbeat is loaded via AHAH (in includes/heartbeatactivity.inc:172).

The bug can be worked around with the following JS:

$(document).bind('heartbeatBeforePoll', function(context, post) {
        post.destination = Drupal.settings.activePath;
});

This could also be used to fix it in the heartbeat module itself. I'll provide a patch when I find time for that.