Emails sent, in spite of UNchecked "Send email upon password expiring"

Problem/Motivation

I've just added password policies for all my developers on several of our sites. I want to avoid developers getting emails from all the sites at once about changing their passwords, so I UNchecked the box to send emails.

Developers are still getting emails.

Steps to reproduce

1) Install password policy, and any of the other required modules to add a policy.
2) Create a policy, UNcheck the box for "Send email upon password expiring", apply to a role
3) save the policy
4) export and commit your config
5) deploy the code + config to a new environment,
6) do a config sync
7) get the email you asked not to get