Internet search giant Yahoo is adopting a new system for indexing Web pages that will charge business to include more material currently unlisted in its online search engine, marking the first move in what may be a heated battle with its top search competitor, Google.
Yahoo is touting the approach as a practical way to ensure that its search engine captures more of the so called ”Deep Web”, the billions of pages that aren’t found during periodic crawls of the Internet.
The method, often called “paid inclusion”, will also help Yahoo’s search engine keep better tabs on the most current material on a Web page, company officials said.
More than 99 per cent of Yahoo’s search index will consist of Web links that don’t pay fees, said Tim Cadogan, the company’s vice president of search.
Search engine analysts generally applauded Yahoo’s move, saying it could open a rich new vein of content that’s lacking from all Internet search engines.
But the fees required to participate in the programme are likely to raise worries about Yahoo creating an online caste system dividing the haves and have nots of the Internet.
To ease those concerns, Yahoo isn’t charging non profit Web sites to add unlisted links to its search engine. The non profit sites initially participating in the new indexing system include National Public Radio and the Library of Congress.
While Yahoo’s index will continue to include Web sites that don’t pay the fees, there’s no guarantee on how frequently those destinations will be visited, Cadogan said.
