The original functionality of the 'answer' button is to provide an anchor link down to the answer form. But what if you are not logged in, or if you don't have access to post an answer? We should not be displaying the answer button.
Also, if you are logged in, but no one has answered the question, the answer form will be at the top. Its not intuitive to someone not familiar to commons to know that the answer button means (show me this answer form)