Annotate box appears in places where I don't want it to

Annotate box appears in places where I don't want it to.

For instance, I have set the Annotate box to appear only on the following node types: Blog, Event, Story. Yet it appears in many other content types: Forum topic, Group, Page, Usernode and many other places. This is very annoying to the end users, especially since it occupies so much screen-space and is not usually useful for most node types.