TJ Stein gave a really good presentation at WordCamp Chicago 2011 about scaling and performance issues with WordPress installations. He focuses a lot on using ngnix instead of Apache, but also covers PHP object caching, Varnish, CDNs, benchmarking, etc. It’s a good overview of current practices, but there’s also a lot of specific tips throughout.