The future of online searches may involve faster and greater computing power. Or maybe it will include clever new algorithms that would let people type in specific questions, asking for what they want. Or maybe it will add people-power to the equation, letting the masses express their human intelligence to augment computer programs.
Actually, all of these approaches are being tried, from search leader Google to a slew of little-known contenders with names such as quintura, clusty, hakia, cRANKy, and kartOO.
That's because while the Internet's content grows all the time, it's only useful if you can find what you're looking for. Google made its reputation by performing searches better than anyone else, and it now has the deep pockets, impressive talent pool, and market domination to defend its position. People used Google for nearly half (47.5 percent) of their Internet searches in January, according to comScore Networks, which measures Web activity. The company's closest competitors – Yahoo (28.1 percent) and Microsoft (msn.com, with 10.6 percent) – trail by a wide margin.
But some say Google's position isn't unassailable. "I don't think Google is on any safe ground," says Philipp Lenssen, who founded and edits Google Blogoscoped (blog.outer-court.com), a Google-watching website, from his home base in Stuttgart, Germany. Don't forget what happened to AltaVista, Mr. Lenssen says. A leading search engine in the mid-1990s, it was surpassed by Google's new algorithms, which were based in part on the value of links between sites to produce better search results.
If a competitor could develop a system that truly understood ordinary human language – responding accurately to a request such as "I want to find the best pizza place in my area that's open on Sunday evening" – that would be a huge improvement over keyword searches, he says.
But to understand "natural language," computers must be equipped with artificial intelligence (AI). One of Google's top priorities is to be the leader in AI, according to an internal Google document leaked to Lenssen last year. Speaking to a group of scientists in San Francisco earlier this month, Google cofounder Larry Page said, "My prediction is that when AI happens, it's going to be a lot of computation and not so much ... clever algorithms [as] brute computing power." And AI may be coming soon, he adds. "I don't think it's [as] far off as people think."
Powerset Inc., a San Francisco-based start-up, is going the "clever algorithm" route, using natural-language technology developed at the Palo Alto (Calif.) Research Center. "In technology, there are incremental changes, and there are breakthroughs," said Powerset investor Charles Moldow when the collaboration between Powerset and PARC was announced early this month. "We expect Powerset to fundamentally alter how people interact with their computing devices."
But maybe a simpler solution is at hand. Why not let users themselves refine the search process? That's the idea behind Search Wikia (search.wikia.com), a new search engine that Jimmy Wales, cofounder of Wikipedia, hopes to have up and running later this year. Although details of how it will work are still being ironed out, it will be based on the idea that a "virtuous cycle of feedback" from users will bring the best search results to the top and eliminate spammers or other vandals from skewing the results. User-generated features will build on already available open-source search engines like YaCy and Lucene.
Lenssen is withholding judgment, but he is eager to see Mr. Wales's new project go live. "So many people have announced Google-killer products," he says. "But Jimmy Wales is a bigger figure ... he might actually have something."
So what effect will newcomers Search Wikia and Powerset have on the search-engine field? "Probably nothing," says skeptic Danny Sullivan, editor in chief of Search Engine Land (SearchEngineLand.com). "I'm sure we'll see more use of humans" in guiding searches, he says, but don't forget that has already been tried, he says. Remember the old AskJeeves search engine? Hiring human editors to improve results is expensive, he says, so the twist that Search Wikia may provide is that such work will be done for free by volunteers.
As for Powerset, natural-language search concepts have been "a well-used pitch" for years, Mr. Sullivan says. "I'm sure we'll see that come into play, but I don't see it as a Google killer." For one thing, many queries aren't that complicated. Users just want to be directed to a site like "Yahoo" or "weather," he says. A single word will do. Natural- language applications should be able to figure out that if you query "liberal Democrat healthcare," for example, one person you'll want information about is Hillary Clinton, he says. "But any site with [the words] liberal Democrat healthcare is likely to be already mentioning Hillary Clinton anyway."
You need more than just a smart algorithm, Lenssen adds. "What are you going to do with it?" You need 100,000 or however many computers that Google has to be able to serve results to users as fast as Google. "That's a huge barrier to cross," he says. "It might be easier to just sell your algorithm to Google or get hired by Google."
But in 1998, nobody saw Google coming, either. "Google is the leading technology company in the world, or [at least] the most exciting to [work] at," Lenssen says. "But maybe in five years, it will be some other company."