1
This weekend, I had two clients with XOOPS websites go over their allotted bandwidth. And it wasn't even the middle of the month!
Upon further investigation, I found out that GoggleBot was the culprit. Apprarently, it really liked the piCal module.
After researching the XOOPS site, I found this:
In your XOOPS installation the default robots.txt file is fine the way it is for *most* cases. The only time you should ever have to chage that file is if you install piCal or if you just don't want search engines to index certain areas of your site.
The reason you want to disallow access to piCal is because the spiders will each up all your bandwidth searching through the calendars internal links month by month. To disallow piCal, just add the following to robots.txt:
Disallow: /modules/piCal/
(Taken from
https://xoops.org/modules/smartfaq/faq.php?faqid=306)
Well, before I found this, I disabled the piCal module on one of the sites because it turned out that they weren't using it. Thought that would solve the problem - but it appears that Googlebot is STILL trying to access it. So I have added the above "disallow" to see if that works. Ugh!
Hopefully this will take care of it. Just wanted to post it here in case someone else has the same problem.
Kelly Ling
Shamrock's Web Design
http://www.shamrocksweb.comIf you're coming to see my house - give me two weeks. If you're coming to see ME - come any time!