| Re: New Google Bot or Someone Spoofing Google IPs? |
| by Dalton on 2004/9/28 14:21:49 Yes i´ve some visits of the crawl in my web. But Google does not index it. |
| Re: New Google Bot or Someone Spoofing Google IPs? |
| by FlySwatter on 2004/9/27 1:32:06 Sorry for the mis-understanding. Bot Trap doesn't ban Google; or any other Bot for that matter. It just adds an entry to the Robots.txt file disallowing a certain directory. If a Bot ignores the directive it is logged. It can be very informative. Hope this clears things up. 
|
| Re: New Google Bot or Someone Spoofing Google IPs? |
| by tl on 2004/9/26 23:47:19 I am not looking to ban or for ways to ban Google, just information or confirmation on the new log entries. I don’t want spambots pretending to be Google to crawl my sites. It is not difficult to ban Google or any other bots. But in general for most people, it would be foolish or suicidal to ban Google, unless you have an Intranet or a top-secret site to protect. |
| Re: New Google Bot or Someone Spoofing Google IPs? |
| by FlySwatter on 2004/9/26 23:08:31 Quote: BTW, unless you don't want Google to crawl your site, you can not stop people with (spoof) Google IPs from crawling your site. But if Google DOES obey robots.txt, won't Bot Trap ID them for you? Then one can at least know. Anyway, I had an occassion to question Google about a simialr thing and did receive a reply from a real person. Not just an auto-respond. |
| Re: New Google Bot or Someone Spoofing Google IPs? |
| by tl on 2004/9/26 22:04:36 Quote:
Thank you for your suggestion. Sorry I am just looking for confirmation if anyone has seen similar bots and has info on them. BTW, unless you don't want Google to crawl your site, you can not stop people with (spoof) Google IPs from crawling your site. |
Search
Recent Posts
Who's Online
Donat-O-Meter
| Stats | |
| Goal: | $15.00 |
| Due Date: | Jul 31 |
| Gross Amount: | $0.00 |
| Net Balance: | $0.00 |
| Left to go: | $15.00 |
|
|