XOOPS 
Forum
Forum Index General Forums Community Support Modules Support Themes Support Development International Support Module Hack Reviews Theme Designer Talk
  1. Board index / 
  2. XOOPS Modules Support forums / 
  3. Module requests 
  4.  / Search API and XOOPSBot / Xpider
Register
1
ajaxbr
Search API and XOOPSBot / Xpider

  • 2004/6/26 23:25

  • ajaxbr

  • Quite a regular

  • Posts: 276

  • Since: 2003/10/25


Hi all,
Just an idea that I had the other night and someday shall try to implement, but since it could be useful for most XOOPS users (and I'm not quite good at code), here it is:
It would be pretty neat to have the ability to search external related sites. We can do that using Google, but filtering which sites it returns isn't that easy, and sometimes the info you want would be buried at low page ranks. Since each XOOPS site has its own search engine, it could also provide an API for remote searching.

And for interesting related sites that aren't searchable but by Google, "XOOPSBot/Xpider" could index the site just as any search bot. PhpDig, ht://Check, mnoGoSearch, SWISH++ and many other spidering programs are good candidates for such a module. This could be something really revolutionary, allowing XOOPS users to have a real search engine besides links/downloads directories. However, this is exactly the kind of app that can bring a server to its knees, so if you feel like trying one of those spiders, I'd recommend running it locally first, if possible.
2
tl
Re: Search API and XOOPSBot / Xpider

  • 2004/6/26 23:35

  • tl

  • Friend of XOOPS

  • Posts: 999

  • Since: 2002/6/23


Quote:

Hi all,
Just an idea that I had the other night and someday shall try to implement, but since it could be useful for most XOOPS users (and I'm not quite good at code), here it is:
It would be pretty neat to have the ability to search external related sites. We can do that using Google, but filtering which sites it returns isn't that easy, and sometimes the info you want would be buried at low page ranks. Since each XOOPS site has its own search engine, it could also provide an API for remote searching.

Have not tried it, but is this doing the same as you suggested?
https://www.google.com/adsense/ws-overview

Not sure about your second point - if contents have not been indexed by google and you want to index those sites? How? I would think you need to create another Google or Xoogle to index those sites that currently are not indexed by Google. We are talking about a huge infrastructure to support these spiders. I don't think XOOPS is set for that.
3
ajaxbr
Re: Search API and XOOPSBot / Xpider

  • 2004/6/26 23:58

  • ajaxbr

  • Quite a regular

  • Posts: 276

  • Since: 2003/10/25


Quote:

tl wrote:
Have not tried it, but is this doing the same as you suggested?
https://www.google.com/adsense/ws-overview

Not sure about your second point - if contents have not been indexed by google and you want to index those sites? How? I would think you need to create another Google or Xoogle to index those sites that currently are not indexed by Google. We are talking about a huge infrastructure to support these spiders. I don't think XOOPS is set for that.

Hey tl,
Thanks for answering my post, I can see now how unclear I was. What I meant was along the lines of:

Search API: allow remote searches in other XOOPS sites. Example: you're in docs.xoops.org and want to search for a keyword, and under the "Advanced Search" there could be checkboxes to include dev.xoops.org and www.xoops.org in the search.

Indexing/Spidering: that's basically it, you run a program much like GoogleBot and in the end you can provide a Google-like search. Those links point to implementations of this, and I agree that it's a heavy task that might be totally out of XOOPS scope.
4
tl
Re: Search API and XOOPSBot / Xpider

  • 2004/6/27 0:16

  • tl

  • Friend of XOOPS

  • Posts: 999

  • Since: 2002/6/23


Quote:
Search API: allow remote searches in other XOOPS sites. Example: you're in docs.xoops.org and want to search for a keyword, and under the "Advanced Search" there could be checkboxes to include dev.xoops.org and www.xoops.org in the search.


It is a good idea. I think it would be very difficult to implement.

First, you would have to compile a list (a central directory might do);
secondly, you are searching something that might not be available or refused by the external site because you are draining its resources; and
third, connecting to the external site may take forever due to many factors that are totally out of your control. The API might slow your site down dramactically, or in worse cases, crash your site.

All in all, I think it is totally out of scope as XOOPS is set to do.

Login

Register now!  |  Lost Password?

Search

Advanced Search

Recent Posts

Who's Online

106 user(s) are online (68 user(s) are browsing Support Forums)

Members: 0

Guests: 106

more...

Donat-O-Meter

Stats
Goal: $100.00
Due Date: May 31
Gross Amount: $0.00
Net Balance: $0.00
Left to go: $100.00
Make donations with PayPal!

Latest GitHub Commits