Multi-value field with default value keep adding new addresses

If you have a multivalue address field, with a default country, it keeps adding new addresses every time you save the node. Because it thinks the "empty with default country" is a new address.