This is a bug in Drupal 7 and Drupal 8.
$variables['created'] = format_date($comment->created);
$variables['changed'] = format_date($comment->changed);
The template provided by comment module renders neither of these, but format_date() is very expensive when called up to 600 times per page.
One obvious fix which should be backportable to Drupal 7 would be to only format the date once if $comment->change and $comment->created are the same timestamp.
For Drupal 8 we should look at whether Twig will let us take these out of generic preprocess and lazy-generate the variables based on the compiled template (i.e. it's supposed to know whether specific variables are used or not, so there shouldn't be a need to front-load things that are never printed). If not, let's just stop preparing those variables and let people use their own preprocess in custom themes if they want to.