Robert, we had a default OpenACS 4.5 installation. Of course we are not using a global cache on the two webservers, but if you wanted to have one you would have to look at some of the work that was done by aD where you could specify a cluster in the configuration and when you flushed the cache on one webserver, it would request a special URL that would flush the cache on the other(s) machine(s). We haven't implemented that in these machines.
Mohan, the two webservers and the database server are on a 100Mbps switch, so to answer your question, there is nothing special between all the machines, of course the latency between the machines should be reduced to the strict minimum. You should look at 1Gbps ethernet these days even though the network is not really the bottleneck between the DB and the webservers.
One of the problems we are experiencing right now, and I think it will be better with AOLServer 4 on the horizon, is that the webservers are on 100% of CPU utilization at peak times, meaning that the CPU is our bottleneck on those machines, but there are problems in the configuration of the load balancer as well. What I mean is that if your DB is properly configured and you are using caching of database queries or pages then you should be careful to have enough RAM and enough CPU on the webservers... But adding a single processor webserver is much cheaper than looking into upgrading our Sun 250 to another model!.