Confirmation of core taxonomy access behaviour requested

I've raised a bug issue on the Forum Access module about forum taxonomy term URLs returning a 404 (Page Not Found) rather than a 403 (Access Denied) when the user does not have access to see them.

The module developer is saying that this is the behaviour of the core taxonomy module and how Drupal handles entity access in general. This isn't my understanding of how taxonomy access works in Drupal, but it's possible that I'm misinformed here. I was hoping one of the Taxonomy maintainers could confirm whether or not taxonomy terms on a clean install are expected to return 404 if the user has no access?