Consider the following situation:
Bob creates a node. Jill posts a comment, with the "Notify me when new comments are posted" box checked. Bob then posts a comment. Jill receives an email notification. So far so good.
Jill decides to post another comment, but decides that she no longer wants notifications, so she UN-checks the "Notify me when new comments are posted" box. However, she still continues to receive email notifications for that thread!
It appears that if a single comment by a user in a thread was ever submitted with the box checked, the user can't actually unsubscribe from that thread by unchecking the box. In fact, even if the user edits that comment that had the box checked, unchecks the box, and then clicks save, it has no effect. That comment still has a 1 in the notify column of the comment_notify table, and upon editing the comment again, you'll see that the box is still checked (despite having unchecked it and clicked save).
In case it makes a difference, I don't allow replies to specific comments, so all comments are treated as replies to the original post.