A well tuned reverse proxy cache, sometimes also referred to as a "web accelerator", can dramatically increase a websites performance. Squid and Varnish are popular open source software packages with reverse proxy cache functionality
In general a proxy cache server acts as an HTTP client/server between your clients' browsers and your web server (Apache, IIS etc). Traffic flowing trough a proxy server can be monitored, filtered, cached, altered and secured when needed.
Note: proxy servers are widely known as secure outbound internet sharing systems in larger environments like schools or offices. This use case is called a forward proxy server and is not in scope here.
Reverse proxy caches
Reverse proxy cache servers are setup between the web server and the incoming internet connection. This has several advantages:
Reverse proxy servers are generally much faster then full fledged web servers because of their dedicated function. You can also use some reverse proxy server software for load balancing between multiple web servers.
Traffic shaping and modification
Many reverse proxy software packages allow modification of the HTTP requests flowing though them. This can be very useful for things like redirects, rewrites, improving cache-ability or dirty hacking of unwilling web applications
The reverse proxy can cache requests, "files" once, and serves it out to many clients without bothering the web server. When used just for static files like images or CSS this will already help. The real benefits come when you can cache html page requests so the PHP and database layers are also released of a lot of work. The proxy server will register the expire times of each request and clear it from the cache so a new version will be fetched from the web server the next time a client requests it.
Purge/Ban/Flush/Expire/Delete/Invalidate objects from the cache
When caching content in a reverse proxy cache an inherent side effect is a delay in content delivery. Decreasing expire times will minimize the effect but increases the load on the server. One solution is to make the underlying CMS send commands to the reverse proxy servers to remove objects from the cache the moment the content is changed. This mechanism travels under many names (see the heading) and I will call "Purge" here. Many reverse proxy cache, CMS software and CDN services have some form of purging. Currently all of these methods differ in implementation and are out of any formal HTTP spec. In 2011 a standard called Link Cache Invalidation was proposed but has not been approved or implemented yet.
Drupal Core and reverse proxy caches
Drupal 6 doesn't work well with reverse proxy caches by itself. This can be fixed by applying a patch found here. You can also minimize the kitten slaying and download Pressflow, a friendly fork of Drupal 6.x that is optimized for high traffic use.
Drupal 7 itself will work out of the box with a reverse proxy cache because it incorporates all of the Pressflow 6.x optimizations.