PDA

View Full Version : Yahoo Slurp Spider?

Kev_Hectic
03-25-2006, 08:26 PM
Does anybody know what's up with this?. There's about 100 or so of them listed in the who's online section... :confused:
Yatta
03-25-2006, 08:27 PM
It's a search engine spider. These bots feed data to search engines so you get accurate results when you search for stuff.
Kev_Hectic
03-25-2006, 08:36 PM
^Oh, I see. It just seemed odd to me. :dopefish:
ADM
03-25-2006, 11:14 PM
Yahoo's Slurp Spider is a disgrace. Seriously, they really have to learn to control their spiders. There's only supposed to be 4 at a time on a site but they go overboard.
Yatta
03-26-2006, 12:23 AM
Yahoo's Slurp Spider is a disgrace. Seriously, they really have to learn to control their spiders. There's only supposed to be 4 at a time on a site but they go overboard.
Competition, man. :p
ADM
03-26-2006, 12:58 AM
They nearly took down my site due to the amount of database connections they used up constantly. I had to limit them via a robots.txt file.
Joe Siegler
03-27-2006, 11:19 AM
BTW, all these spiders searching the forums is not new. They used to do it on the old forum, too. The only difference is that vB has an option to allow you to see who they are, instead of them just being "Anonymous".
avatar_58
04-26-2006, 04:56 PM
BTW, all these spiders searching the forums is not new. They used to do it on the old forum, too. The only difference is that vB has an option to allow you to see who they are, instead of them just being "Anonymous".

Could you perhaps block them if they can be identified? Or at least restrict the amount of them?
Joe Siegler
04-26-2006, 05:37 PM
Could you perhaps block them if they can be identified? Or at least restrict the amount of them?

Why would I block them?
avatar_58
04-26-2006, 09:07 PM
Why would I block them?

Something to do? ;)
Joe Siegler
04-27-2006, 12:18 AM
Since you gave me a silly answer. I repeat. Why would I block them? You made the question originally - why?
The Baskinator
04-27-2006, 01:04 AM
I'm not sure he has a serious answer to satisfy you. :)
Joe Siegler
04-27-2006, 01:44 AM
Then why ask this in a non joking fashion? There had to be a reason to say this in the first place.

Could you perhaps block them if they can be identified? Or at least restrict the amount of them?
avatar_58
04-27-2006, 02:41 AM
I was only asking, out of curiosity. ADM made it sound like they add a little excess strain on the server, so I wondered if there was a way to cut them off.
ADM
04-27-2006, 03:35 AM
To MY server. The server I was on before was a shared server and to that it added strain (and even then the shared server could still cope with it ok, not I said it could nearly take down my site), it's like a very minor thing to the equipment hosting the 3DR forums.
ShadeEX
04-27-2006, 03:48 AM
Ahhh nice i didn't know this ...

i just hope they don't strain any other servers..
Vivi
04-27-2006, 04:00 AM
Hissssssss.

Never did like spiders. They slow my site down as well.
Qbix
05-01-2006, 04:27 AM
Hissssssss.

Never did like spiders. They slow my site down as well.

Well if you want to get visitors from a search engine......
Those spiders are quite important. Maybe they don't behave that nice all the time, but the internet wouldn't be as transparant as it is today without them.
Rellik66
05-01-2006, 01:05 PM
I've noticed many vbb3 sites will usually show up in search engines using the archived light version (http://forums.3drealms.com/vb/archive/index.php/) or the printable versions of the thread before the normal version.