dman has written some excellent documentation on the site alias feature.
@dman: I have some specific feedback on a few points.
You made a comment that site aliases in a drushrc.php file in a site folder was not supported, but this limitation was in fact fixed. One thing to note, though, is that drush only loads the drushrc.php file in a site folder during the drupal site bootstrap, so only one of these will ever be loaded, and it doesn't really do any good to put an alias to a site in its own site folder, because drush will see it too late. So, your comment is correct, but the actual reasoning is a little different.
Regarding authentication keys, drush requires these to be set up. The commend in backend.inc says that password access is okay, but the code says "ssh -o PasswordAuthentication=no", so drush will refuse to contact a remote machine unless you have set up the authentication keypair. I use the script "pushkey" to set these up; I have attached it to this incident for your general amusement. (Please always review scripts such as these carefully before using them, to insure that no one has inserted code to push your private keys out to some nefarious location...)
You recommend setting up an alias to the site folder on the target location. This is a fine idea, but if you go this route, you will need to set up symmetrical symlinks -- that is, each drupal instance should have a symlink to the other's site folder. This is important because drupal sometimes writes the path to the sites folder into certain fields of the database; if you do some operation on the target site (e.g. configure the color module) and then push the database back to the original site, then your database may contain references to both of the site folders, so you will need aliases in both locations to make sure things always work. I hear that this will not be necessary with D7, though. In the meantime, another solution is to insure that the sites folder is has the same name on both sites. If you originally set up a site "drupal.org" and push it to a remote site "dev.drupal.org", then you can configure your web server to serve "dev.drupal.org" from a drupal root folder dedicated to that site, and drupal will still serve up content from the "sites/drupal.org" folder due to the way drupal searches for the settings.php file. This explanation is a little brief; perhaps I should draw a diagram. Creating aliases is perhaps easier to understand.
You mentioned that '!drush-script' used to be '!drush', but this was changed before the site alias feature was committed, so few people will have seen this older version of the code. I'd recommend documenting it as it is now: !drush-script is the path to the script, and !drush is the path to the drush installation folder.
In the same general area you mention that drush must be installed on the target machine. This is the case only if drush needs to look up the database specifications for you. If you define 'database' or 'db-url' in your site alias, then the remote call is not done, and drush only needs to be installed on the local machine. If you pass '--database' or '--db-url' to the site alias command, it will include the database specification in the site alias record to make this more convenient for you. If you do this, you may have troubles if someone changes the database specifications on the remote machine, though.
Regarding the !dump specification, just fyi, there is a pending issue that will address this. sql sync will someday allow you to specify a folder instead of a file (e.g. in $options['dump']), then the filename will be composed from elements of the site alias (e.g. remote-host, root and uri). Perhaps there will be a separate option, 'dump-dir', or perhaps it will key off of whether or not the final character of !dump is a '/'. Please comment in if interested.
Finally, I always like to do a
drush sitename cc all after pushing a site, just to insure that theme and block changes are picked up on the target. I also do a
watchdog delete all, but I can't quite remember if this is superstition, or if it actually clears up a problem. Perhaps the former, since I include
--skip-tables-key=common when I sync the database so as to avoid transmitting the watchdog table.
Thanks again for the documentation -- it's great!