Yahoo Intros New Search Robot – Yahoo! Slurp

Yahoo just got a step closer to dropping the Google search results from its search function and replacing them with Yahoo’s own Inktomi search engine- which will be a bit of a blow to Google, and a sign of potential dominance by Yahoo.

Yahoo has just unleashed a new site indexing robot to crawl the web with – Yahoo! Slurp.

Yahoo’s new robot keeps a similar name to the Inktomi Slurp crawler and some features listed on Yahoo include:

* Yahoo! Slurp has the ability to crawl dynamic links or dynamically generated documents.

* The Yahoo! Slurp crawler collects documents from the web to build a searchable index for search services using the Yahoo! search engine (this helps verify a soon addition of Inktomi to the Yahoo search results). These documents are crawled since other documents on the web contain links to these documents.

* Yahoo! Slurp crawls from your site in the Yahoo! search engines immediately. The documents will be indexed and included into the search database in the near future.

* Yahoo! Slurp will offer cache indexing (similar to Google) and obeys the noarchive meta-tag. If you place: META NAME=”robots” CONTENT=”noarchive” in the head of your web document, Yahoo! Slurp will retrieve the document, but it will not cache or archive the document for use in the PageCache system.

* Yahoo! Slurp also obeys the noindex meta-tag. If you place: META NAME=”robots” CONTENT=”noindex” in the head of your website, Yahoo! Slurp will retrieve the document, but it will not index the document or place it in the search engine’s database.

Last year, Yahoo announced that they plan to make the change over to Inktomi results in the first quarter of 2004, which gives them about 40 days to meet that goal. In addition, its nice to see that the Slurp robot and search functions are all branded Yahoo.

It gives it more of a unified feel to have all of its main functions together, indivisble, under the united brand umbrella of Yahooooo!

Seen Yahoo Slurp in your server logs? What do you think? Leave a comment.

Written By:
PG

| Search Engine Journal | @lorenbaker

Loren Baker is the founding editor/creator of Search Engine Journal and remains an advisor and Editor In Chief to this publication.

More Posts By

Comments

  1. Scott Lafans says:

    This new Yahoo Search is really bad..When you type in rental cars” you get GINZ.com and stuff I’ve never heard of..

  2. John Foster says:

    Competition to google would be a godsend. Competition will increase the quality of search, and decrease the cost of advertising. Now, let MSN come on baby! Who uses Kanoodle? ahhahahahahhaahahhaha who do they think they are?

  3. Loren says:

    Kanoodle? The do have CBS MarketWatch as clients and some other decent partners. A lot of the Sprinks people went over to Kanoodle after Google bought it from PRIMEmedia.

    If you click the link below, Kanoodle will gift you a $5 account – no strings attached.

    Enhance has also popped up lately, anyone use them?

    As for Yahoo Slurp, its exciting to see that after all of this SEO News speculation, Yahoo is really doing something BIG!

    Anyone see Yahoo Slurp in your logs?

  4. WebRankInfo says:

    I saw some visits of Slurp these last days. I wonder when the pages crawled will be on Inktomi or Yahoo! ?

  5. John Smith says:

    If only Yahoo or Google can create the same search and ranking technology as http://www.StockRing.com did that would be even better.

  6. Amadhea says:

    When will AllTheWeb and other search engines join this competition?

  7. salmanrobot says:

    ewewe

  8. salmanrobot says:

    Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed:

  9. For some reason, Yahoo’s Slurp seems uninterested in my site. Google and MSN constantly crawl as well as send a decent amount of traffic my way. Slurp on the othher hand only crawls a few pages here and there(the same ones every time) and does not have my site in its index.

  10. wwa says: