In this case I do have control over the firewall, but in many cases the application administrator will not (like in multi-hosted environments) so I think both are beneficial.
Jade - what part of G-Mail's interface do you mean? I haven't seen it yet.
On my personal site, which has had 1386 hits inside /register/ since last October, 924 are from msnbot, 154 from FAST-WebCrawler, 126 from "http://www.almaden.ibm.com/cs/crawler", and another 21 from miscelaneous robot-like UAs... and /register/ is in the robots.txt... at least google seems to be doing the right thing...
jeez... you'd think at least MSN would be capable of respecting standards on the web... :)