Sunday, February 17, 2008

Web Search Aggregators?

Back in the day, before Google had pretty much conquered the web search market, there were many search engines out there for finding web content. Some, like AltaVista, HotBot and Excite made use of web spiders for locating and indexing new content. Others such as Yahoo maintained a staff of humans that would maintain a directory of sites in various categories and allow users to search it.

And there were others, such as (now just, that made use of advanced natural language processing algorithms to attempt to answer your question intelligently rather than simply pattern matching the fulltext of the web for your search term. Ask Jeeves was especially nice, because it felt more like you were able to intelligently ask a question and get a response, such as "Where can I find the exchange rate from dollars to GBP?" None of the other search engines really were able to do that.

One other site that stuck out was It didn't actually maintain an web index of its own, but instead, it would send out your search query to all the major search engines, rank the results, and then show an aggregated list of potential hits. Before I started using Google, MetaCrawler was my default search engine for pretty much any query (with being a close second for things that could be formed into a question easily).

It appears that MetaCrawler is still around, still searching, as are most of the other big players. But does anyone actually use them anymore, with Google as the golden standard of web search? Will some new player enter the field with some amazing new technology that could challenge Google's dominance? I'm looking forward to that day, even if it simply means that it will force Google to innovate once again.

No comments: