XOOPS 
Forum
Forum Index General Forums Community Support Modules Support Themes Support Development International Support Module Hack Reviews Theme Designer Talk
  1. Board index / 
  2. XOOPS Community Support forums / 
  3. Beginner's Corner 
  4.  / msnbot Meta Crawler locks up site
Register
1
JamesSAEP
msnbot Meta Crawler locks up site

  • 2005/10/6 20:33

  • JamesSAEP

  • Just can't stay away

  • Posts: 732

  • Since: 2005/2/28


Every so often my sites would not come up:
"Error [Xoops]: Unable to connect to database in file class/database/databasefactory.php line 34"
and I couldn't figure out why. I've installed the module myReferer and it shows a Meta Crawler called msnbot (+http//search.msn.com/msnbot.htm) that seems to be crawling the site at the times it's not responding.

Is there anything that can be done about this?

Thanks.
2
JMorris
Re: msnbot Meta Crawler locks up site

  • 2005/10/6 21:37

  • JMorris

  • XOOPS is my life!

  • Posts: 2722

  • Since: 2004/4/11


Yes, disallow the robot to sections of your site that require large numbers of database queries. Here's a great tutorial on the robots.txt file where you will need to disallow the search bots.

Link: robots.txt tutorial

Best Regards,

James
Insanity can be defined as "doing the same thing over and over and expecting different results."

Stupidity is not a crime. Therefore, you are free to go.
3
JamesSAEP
Re: msnbot Meta Crawler locks up site

  • 2005/10/6 21:43

  • JamesSAEP

  • Just can't stay away

  • Posts: 732

  • Since: 2005/2/28


Thanks for the reply. The only thing that has a large database is the news and that is what I do want to be indexed.

So I guess there really isn't anything that can be done.

Login

Register now!  |  Lost Password?

Search

Advanced Search

Recent Posts

Who's Online

422 user(s) are online (60 user(s) are browsing Support Forums)

Members: 0

Guests: 422

more...

Donat-O-Meter

Stats
Goal: $100.00
Due Date: Nov 30
Gross Amount: $0.00
Net Balance: $0.00
Left to go: $100.00
Make donations with PayPal!

Latest GitHub Commits