1
carnuke
Is this how to make google crawl your site immediately?
  • 2004/8/22 13:48

  • carnuke

  • Home away from home

  • Posts: 1955

  • Since: 2003/11/5


I recently posted an article on my personal site that was about a national news item. It was packed with keywords relevant to the story and I submitted it to gogole in the usual way.

Within 24 hours the article had been crawled and the rest of my site. Ok, maybe it was due for a google re-visit anyway, but it's just occured to me that google may frequently scan all it's new page submissions and build a priority visiting list based on title keyword content. Thus if a page title has keywords relevant to major news items the URL becomes proritised for immeadiate crawling. This seems a logical startegy for the googlebot as it wants to match search term demand with maximum feature content from its index cache. Plus it runs it's own google news section that is heavily used by the press.

If this is true, it would be possible to create news or article items as 'bait' to entise the googlebot to your site simply by using the keywords in the title and article body.

Is this a working method? If so, is it ethical? Assuming your bait article is bogus and not really intended for public consumption.

Does anyone have any thoughyts or experience about this...?

Richard


2
JMorris
Re: Is this how to make google crawl your site immediately?
  • 2004/8/22 15:01

  • JMorris

  • XOOPS is my life!

  • Posts: 2722

  • Since: 2004/4/11


I'm no SEO expert...yet, but the subject of SEO and ethics intrigues me.

Just from an ethical standpoint, I would not practice "bait and switch tactics". Such practices can only harm a website's reputation in the long run.

Personally, I would do the best I could to word my content so that it has good keyword saturation, I would word my site title with relavent keywords [alternating regularly if need be], I would design my theme so that it is Googlebot friendly, and I would hard code GoogleAds into my theme, which entices the Googlebot to crawl every page of your site. I'm sure the short URLs hack wouldn't hurt either.

Actually, hard coding your GoogleAds is extremely effective. In fact, the Googlebot chewed up over 5GB of bandwidth on one of my sites one month because I forgot to put disallow /modules/piCal in my robots.txt file.

Additionally, I would try to get my site listed on as many other sites as possible. If you're selective about who you approve, I've found Starlinker to be very effective. Just simply create a single page module with links.php as the start page and insert the code. The rest is taken care of for you.

I realize that there is a large debate over link exchange programs, but from my experience, if you use good judgment and you don't go crazy with it, you can significantly increase your traffic with them. My referer stats will attest to that.

Just my 2c.

3
FlySwatter
Re: Is this how to make google crawl your site immediately?
  • 2004/8/22 18:38

  • FlySwatter

  • Just popping in

  • Posts: 71

  • Since: 2004/6/2 1


Quote:
hard coding your GoogleAds is extremely effective


How do you do that?

4
tedsmith
Re: Is this how to make google crawl your site immediately?
  • 2004/8/22 18:56

  • tedsmith

  • Home away from home

  • Posts: 1151

  • Since: 2004/6/2 1


I'd like to know that too? How did you know when Google had been to your site? Is there a setting or notification option to find this out?

Also, what is the optimum setting to choose for search engine spidering with regard to the options in System Admin\Preferences\Meta Tags & Footers. I have it set as 'Index, Follow' at the moment. Is this the best one? How does it work?

Lastly, does having the XooGle module installed help your Google ratings? I have it installed and have a registered account? I assume this will help, yes?

5
FlySwatter
Re: Is this how to make google crawl your site immediately?
  • 2004/8/22 19:03

  • FlySwatter

  • Just popping in

  • Posts: 71

  • Since: 2004/6/2 1


Quote:
? How did you know when Google had been to your site?


Do you have a stas program like webalizer, etc.? Usually found in you site manager. I have Cpanel; others are ensim,pleask...

6
CeBepuH
Re: Is this how to make google crawl your site immediately?
  • 2004/8/23 7:26

  • CeBepuH

  • Not too shy to talk

  • Posts: 128

  • Since: 2002/6/10



Hmmm. From my experience Google crawls a site in 24 hours, but sometimes up to a week after submission. And I have never filled the Comments field.

Lately, Google visits my site every day and indexes any new content. And every day I have something new so googlebot keeps returning.
I've read that Google makes schedules for revisiting sites. So for example if it visits you every day but you don't have anything new for several visits, your site would be rescheduled for a weekly visit etc.
On the other hand Google hasn't recrawled some of my older pages. I did a redesign several months ago and in Google cache when I view articles that were published months ago, the old design shows up. I understand that is due to the ? character in my URLs :)

I don't think you are asking the right question about the ethical side of bogus content. It should be "Are there any rules against it?" And actually Google has some rules. If they catch you it can result from lower PR to being removed from Google index. Besides, nobody would visit a site with bogus content a second time. I certainly don't.


7
carnuke
Re: Is this how to make google crawl your site immediately?
  • 2004/8/23 8:46

  • carnuke

  • Home away from home

  • Posts: 1955

  • Since: 2003/11/5


From what I'm learning about google, its a complex machine. There are several factors that influence indexing and site rankings. My personal feeling is this. Google prioritises what is popular and relevant. It's a case of supply and demand. If everyone is searching for material on 'Xsmith' google is bound to want to supply that demand. This is simple marketing policy. Therefore creating regular articles that discuss 'popular subjects' is sure to attract revisits and good ranking. Well, that's part of the story, anyway. The issue about 'keyword stuffing' ie baiting the bot without real content, will certainly reduce rankings. Google can also time how long viewers spend on pages it serves out and registers 'bounce visits' ie when a page is left within a few seconds. This is its way of assessing the quality of a page from a 'human' perspective. Frequent bounces from a page it served= poor quality or irrelevant.

Query strings in urls are not an issue with google (unlike many other S/Es) I ran the short URL hack foe several months and I saw absolutely no change in my indexing patterns AND no increase in other S/Es visiting my site. I have now removed the hack without any ill effect. (except now my modules work properly)

8
YourHelp
Re: Is this how to make google crawl your site immediately?
  • 2004/8/25 9:15

  • YourHelp

  • Friend of XOOPS

  • Posts: 479

  • Since: 2003/6/9 6


When i post a new artical on my site in the news it comes up in google very quickly?

9
simon132
Re: Is this how to make google crawl your site immediately?
  • 2004/8/25 11:30

  • simon132

  • Just popping in

  • Posts: 48

  • Since: 2003/11/19


Some of you guys are saying that google is indexing and page ranking your articles or other pages that inlcude a ?

You must be using some great content because I just can't get it to happen on any of my 5 XOOPS sites.

I admit that not all have great content but some sure do. If there something else to the story? Can someone direct me some of these pages so I can see where I may be going wrong.

I have been waiting for shortURL to become viable for 2.0.7.1 as none of my pages put together with WF-Channel or FreeContent are getting indexed; save for the module entry page which doesn't have any ? in it.

cheers,

Simon

10
carnuke
Re: Is this how to make google crawl your site immediately?
  • 2004/8/25 13:47

  • carnuke

  • Home away from home

  • Posts: 1955

  • Since: 2003/11/5


I don't think content is an issue with Google crawling a page, however content will effect ranking to some degree. To state the obvious ... make sure your meta-tags are set for 'index, follow' in this setting here
http://youyrsite.com/modules/system/admin.php?fct=preferences&op=show&confcat_id=3

Sorry if that was obvious

Richard

Login

Who's Online

374 user(s) are online (310 user(s) are browsing Support Forums)


Members: 0


Guests: 374


more...

Donat-O-Meter

Stats
Goal: $100.00
Due Date: Nov 30
Gross Amount: $0.00
Net Balance: $0.00
Left to go: $100.00
Make donations with PayPal!

Latest GitHub Commits