SmartFAQ is developed by The SmartFactory (http://www.smartfactory.ca), a division of InBox Solutions (http://www.inboxsolutions.net)
HOW TO USE A XOOPS WEBSITE
How do you fine tune robots txt?
and Answered by
on 2006/3/29 16:48:58 (6600 reads)
There is an excellent
Wikipedia entry here
, with links to several articles and websites that specifically discuss robots.txt and how to fine tune its configuration.
The comments are owned by the author. We aren't responsible for their content.