Swampy
August 1st, 2007, 08:18 AM
Some of you have noticed over the past couple of months that the board has been getting a little slow. This was not because of the upgrade here, but Yahoo adding more and more "spiders" that come in and index the website for their search engine.
Google and others handle things much differently and more efficiently by popping on with one or two clients to do their indexing.
Yahoo on the other hand has taken the stance of get it all and get it all NOW by having up to 100's of clients pop on and just hammer the website. I've seen reports of in the 1000's of clients hitting websites to index.
We've noticed the impact here on the forum as well as our personal web browsing experience where even Trillian would have trouble staying connected on my personal computers. Finally had a chance to really dive into it early this morning and the results weren't good.
Yahoo has also taken the stance that they don't care what problems their spiders were causing and pretty much publicly wrote in their support forums that we, as end users, need to figure out a way to control their spiders.
So, in short and after reading on multiple websites that yahoo was taking over everything without any assistance, BJ and I basically banned Yahoo spiders from this website a little while ago. People using yahoo for searching on camera stuff will not find any new material on there moving forward.
My own personal view of Yahoo is horrible. I quite using them as a search engine many years ago when Google came out and that's what I use for my search engine today.
If you use Yahoo today, just know that many websites are also doing this same ban on Yahoo and you'll find yourself in the dark unless Yahoo fixes this problem and we let them back in to index.
There is another problem with bandwidth and my ISP that I am still working on, but hopefully, the forums will be a little faster now all the time.
Google and others handle things much differently and more efficiently by popping on with one or two clients to do their indexing.
Yahoo on the other hand has taken the stance of get it all and get it all NOW by having up to 100's of clients pop on and just hammer the website. I've seen reports of in the 1000's of clients hitting websites to index.
We've noticed the impact here on the forum as well as our personal web browsing experience where even Trillian would have trouble staying connected on my personal computers. Finally had a chance to really dive into it early this morning and the results weren't good.
Yahoo has also taken the stance that they don't care what problems their spiders were causing and pretty much publicly wrote in their support forums that we, as end users, need to figure out a way to control their spiders.
So, in short and after reading on multiple websites that yahoo was taking over everything without any assistance, BJ and I basically banned Yahoo spiders from this website a little while ago. People using yahoo for searching on camera stuff will not find any new material on there moving forward.
My own personal view of Yahoo is horrible. I quite using them as a search engine many years ago when Google came out and that's what I use for my search engine today.
If you use Yahoo today, just know that many websites are also doing this same ban on Yahoo and you'll find yourself in the dark unless Yahoo fixes this problem and we let them back in to index.
There is another problem with bandwidth and my ISP that I am still working on, but hopefully, the forums will be a little faster now all the time.