Forum OpenACS Q&A: Re: opencs.org stops working at night?

Collapse
Posted by Mike Sisk on
Our network seems to have an affinity to attract googlebot...

Yep, googlebot started going through the site last night and at one time there were over 20 googlebot connections stacked up.

There is a load problem with the site that Don and Jeff know about but haven't gotten around to fixing yet.  Until we get this issue resolved I added a robots.txt file to block googlebot from crawling the site.

Also, the site isn't running with keepalive, which we run on all our other sites and something we'll be adding here soon, too.