I own a big blow, where can be more than 100 comments at the post. And it's totally bad idea to send notifications to all of this people.
The comment_subscribe module provides such functionality, but when I saw it's code it was full of garbage.
So, I decided to make a feature for this module that allows admin to disable notifications to whole list of commenters, but only for people who actually got response. I saw several issues with such requests too, I hope that guys will be happy :)