If a user has uploaded a profile picture and wishes to delete it, there's an checkbox marked "Delete picture" on the edit profile page, which the user can check and press "Save". This will remove the profile picture from the user's profile, but the picture will not be removed from the server. The next time the user uploads a profile picture, this will added to the server instead of overwriting the existing one.

If, however, the user chooses a new picture to upload before pressing "Save", the initial picture will be overwritten - whether or not the "Deleted picture" checkbox has been checked. I'm assuming this is the correct behavior and that checking the "Delete picture" checkbox should actually remove the picture from the server.