Ask Jeeves and MSN Search Engine Technology Comparison
With so much talk about relevance these days, I thought I’d introduce you to some of the technology behind the search engine, and what the potential differences between them could be. There are some interesting takes on search technology from pre-ranking results on the fly to neural networks to community based searching. In the last article I introduced you to Google and Yahoo! and how their search technology works. In this article I look at MSN and Ask Jeeves.
MSN – MSN Search is the newest player in the search market. And despite that they are considered the #3 search engine on the web. Not bad considering they just launched their technology this year.
MSN takes advantage of a technology that no other search engine has tackled – neural networks. A neural network is a series of computers which are supposed to learn based on input provided.
Think about that for a second – a learning computer. One that just doesn’t follow rules assigned to it (which is what the more traditional algorithmic search engines like Google and Yahoo! do) but actually can learn from its results.
Essentially MSN search learns from input given to it. For example, if the search engine is told that Ebay is considered an authoritative site on online auctions, then when a person performs such a search they should see Ebay.com at the top of the search results.
Upon analyzing Ebay.com the search engine can then learn why it is considered an authority and apply that learning to other sites to see if they are also authoritative.
The biggest advantage of such a platform is the engineers at MSN can “train” the system to understand what is considered relevant and important and what isn’t. As time goes on we would expect to see MSN search become one of the most relevant of all the search engines simply because the system is designed to improve itself over time.
Of course like any search engine, MSN could be tricked. If we knew what those factors were, we could create a page which could be considered highly relevant, based on the MSN search criteria but would in fact be a garbage page. However because of its ability to learn, the system could quickly adapt to such spam content and readjust rankings “on the fly” to filter out these bogus results.
Another advantage to MSN is that the system should be infinitely scalable. Which means as the use of the search grows, it should only be a matter of introducing new hardware, or requirements into the system, having it adapt to the additions and begin using them as if they’ve existed all along.
Therefore, as new spam techniques are developed, its simply a matter of training the system to watch out for the new technique, flag it as potential spam and even potentially react to it by filtering all sites using the new technique.
By now you are probably saying “holy cow that type of technology must use a ton of resources” and you’d be correct.
The amount of computational power required by such a system would be immense. Just the storage capacity needed to store what the system has “learned” would have to continue to grow. In addition, the system also has a great crawler out indexing more and more content all the time.
It’s not your typical algorithmic based engine. With most algorithmic systems, the ranking algorithms are finite in size. With this system, one would expect the Neural Net to continue to grow as new pathways are created.
Consider this structure as similar to a human brain – as we develop new thoughts and ideas, new synaptic pathways are developed linking areas of the brain to other areas where links previously didn’t exist. Essentially this is what a Neural Network does. While its pathways may not be physical, it does nonetheless develop relationships between previously unrelated sections.
Therefore, the engineers at MSN have developed ways to “shortcut” the requirements for ranking. Essentially they have said “sure there are over 500 factors determining the page quality, but in this category only 150 are used, therefore you can use the same 150 associated with this category.”
Overall, as long as Microsoft can continue to support such a system, I would think that it could win out in the “search engine wars.” The system appears (at least on paper) to be superior to algorithmic based systems, and appears to be able to adapt more quickly to changes on the web because it doesn’t have to wait for an algorithm change to adapt, it only has to learn of the change and apply itself.
Ask Jeeves
Ask Jeeves also takes an interesting approach to the web. One that may also be superior to the more traditional algorithmic based engines. Ask Jeeves uses the idea that the web is a series of communities. Therefore they have structured their ranking systems around this idea of communities to determine relevance and authority on the web.
Because they consider the web to be comprised of multiple communities, pages or sites within a community should only related to each other, yet there could be some relation between communities.
A page or site with high inbound community links is considered “worth” more than a site with high inbound links that are not necessarily all community links.
In other words, a pet site will be considered more of an authority in the pet community if most of its links are from other pet related sites, and if its links are primarily from other non-pet sites (such as plumbers, computers stores, electronics manufacturers) then it won’t be considered as much of an authority.
This structure takes the Bowtie theory of the web a step further. The Bowtie theory says there are a hub of largely concentrated authority sites with links pointing in and out of this hub to the bows.
Ask Jeeves says there are multiple bowties – each created of related sites and that between these bowties there are connecting tendrils – sites that may be considered relevant or authoritative that can provide linkage between the communities.
Using this structure, a site must become a member of a community to be considered relevant. In order to become a member, one must have an adequate number of links within the community. Therefore linking to, and receiving links from, members of the community are what helps you build your authoritative status in Ask Jeeves.
This is a great way for Ask Jeeves to combat link spam. One issue in the past with Google and Yahoo! has been link spam – where a site gets thousands of non-relevant inbound links, inflating its link popularity and pushing it to the top of the search results.
By using such a link structure, where the links that mean the most are related links, Ask is limiting this potential.
Search technology aside, Ask also has some cool features. One of the best I think is that you can ask it a question and almost always get the correct answer.
I’ve found myself turning to Ask many times in the past when I couldn’t form a query to return the right results in Google. Many times the only right query is a question, yet there doesn’t seem to be a good way to form that in non-question format.
For example, you can go to Ask and type in: “What is the population of Mozambique?” And get the correct answer. However you can’t ask this question in any of the other engines. And even if you try something like: “The population of Mozambique is” – you won’t always get the right answer.
Granted the other engines have gotten better at this, but for a long time you couldn’t simply ask such a question.
One thing I’ve learned as I’ve researched the technology behind these engines is that maybe Google and Yahoo! don’t have it entirely right. Perhaps MSN And Ask Jeeves are further down the path to more relevant search results.
My feeling is that both MSN And Ask Jeeves will continue to erode the market share of Google and Yahoo! as people realize that they are great alternatives to the bigger players.
Sure MSN has got some relationship building to do to win back users. This is because for years MSN results sucked (to put it bluntly). It will be harder for them to turn their negative image around than anything.
However, the proof will be in the results – if MSN can demonstrate that they are better then they will begin to win users back.
Similarly, Ask Jeeves while still tiny in comparison to the rest can do great damage in the years to come. Especially now that there is money behind them (in the form of the company’s acquisition by IAC/InterActive Corp.) Now that Ask has the funds to truly compete we should see them begin to make great strides in search.
Unless Google and Yahoo! can continue to innovate and provide new solutions we could see the balance of power shift in the coming years. Because as I’ve illustrated in this article, regular algorithmic search seems to lack in some key areas, such as adaptability and scalability, and the ability to quickly adapt to the changing search landscape.
–
Rob Sullivan of Text Link Brokers is an SEO Specialist and Internet Marketing Consultant.
