D7 Edit is only ever going to work on the front-end

Right now, Edit's JS is being loaded on all pages, as long as the current user has the right permissions. That includes pages like admin or node/1/edit.