I understand how a robots.txt file works, what I am not clear about is whether excluding folders of 'system' files in a CMS like XOOPS actually affects SEO?.

I guess a robot should finish indexing the site quicker than trawling through 'templates_c' and finding nothing of value, but are there any real benefits to this. E.g., does the spider dig deeper into relevent content, does it affect ranking, etc.?

I have no need to change the file that comes with Xoops, I just wondered if any SEO experts had an opinion.

Look this article in french but translate by google.

Oh no... more work! Thx for that link christian.

I confess I had heard of, but never got around to researching Google/XML sitemaps. I just looked at the Google FAQ and (if I got it right) submitting a map does appear to promote the re-indexing of changed pages, which would be nice.

-- Edit --

OK, so I found the 'Sitemap' module in the repository, which I gather creates an appropriate sitemap from your XOOPS site (although some folks appear to be having a little trouble getting Google to accept it). I also found a few online sitemap generators.

My question now is, if the sitemap has to be submitted to Google manually, how often are you supposed to do that? On a dynamic site it's going to be out of date pretty quickly, or is it just to intended to give the spider a map of the site structure to follow?