Why Mahalo will Fail and the Problems with General Search

SMS Text

Jason Calacanis talked at Gnomdex 7 in September 2007 about the days on the internet when only a handful new site appeared on the web every day and could be announced via an email newsletter and checked out by everybody every day.

The sites were generally good and rich on content. Junk was the exception, so was email spam, comment spam and other worthless content.

Yes, everything is nice and clean, if it is small, non-mainstream and not commercially exploited. It is also easier to manage, organize and to know each other. If you can list everything there is on one page or even a few pages, you don’t get problems. The page evolved to a directory with more and more categories. The directory was called Yahoo! and it worked fine for a long time until the growth of the internet was faster than it was possible to keep up with it and list everything in a directory. It also became harder and harder to find anything that you are looking for as a user in this giant directory with hundred thousand and more listings. The Dmoz directory was able to keep up for a while due to its open nature and vast number of volunteer editors, but even that was not enough at some point in time anymore.

Full text and crawler based search engines became more important while the internet grew and grew. Those search engines were not edited by humans anymore. Algorithms replaced editors and enabled at the same time manipulation and cheating.

A full text search engine is like a web directory with millions of top level categories. Each search term is like its own category in a web directory. While it “has” incredible many categories, which are easily accessible, if you know about them, are they also very limiting at the same time. You cannot browse them and they are not structured like the hierarchical categories of a classic web directory. Unlike a classic web directory, a typical search engine result page does only have 10 entries. Web directories might have 1-50 or more entries for a category and show all of them on one page. People who had sites listed in very popular categories knew that a higher listing towards the top is better than a listing towards the end. Web directories often sort the entries for a category simply alphabetically, site names that started with “A” and later even with a number became very popular and the favorite practice to “skew” directory listings.

Directories started to experiment with ranking entries based on quality rather than just alphabetically but full text search engines really made this aspect part of the core of their business. Early means of qualify sites were some forms of rating mechanisms. It became clear quickly that it is very hard to implement a rating mechanism that is hard to manipulate by people who try to trick the system to get their stuff ranked higher artificially. Financial incentives of doing so were and are to this day a strong motivator to attempt tricking any of such systems.

Modern crawler based search engines continue to try to rank results based on quality, even though the mechanisms used to “rate” the quality of listings became much more complicated over the years. It became more complicated to make it harder for people to manipulate the rating results. The incentive for attempting to manipulate rankings increases over time as more and more people go online and use the Internet and search engines as the means to find stuff.

It does also get harder for legit and honest content producers to rank in those results where they should at the same time. Also more quality content becomes available that fit each of those millions of “categories” or search result pages. Short, easy to remember and widely used “categories” are getting more and more crowded. The number of listings for each of those categories remained only 10 though.

There are no “sub categories” in modern search engines that allow users to drill down further into the details the current category, although search engines started to experiment with “suggestion” features that serve a similar purpose.

Add to that the fact that modern day search engine ranking algorithms are using a large number of factors for their ranking score, which are for the most part still dumb, technical and artificial (even though their purpose is to emulate intelligent human behavior). The more factors a webmaster ignores, the lower his ranking score will be, factors that do not have anything to do with the quality of the actual content itself, but how it was published on the internet.

White hat and legit search engine optimization is about making sure that those technical factors are being met properly to allow the content itself rank well where it deserves to rank. Of course can this be taken further to manipulate factors artificially to increase ranking of content that does not deserve it. The more artificial manipulation is done, the more questionable it gets and the pure and legitimate optimization turns from white hat over gray hat to what is called black hat SEO.

The epitome of black hat SEO is to get worthless content that is garbage and of no use to anybody rank high in categories that are already crowded enough with good content. It is also called spam.

Declaring SEO as a whole to spam and say that it only is an unethical method used to skew search engine results is not only wrong, but also irresponsible, because it causes people to believe that they will rank in search engines appropriately where they should with their content, regardless how they publish the content on the web and what else they are doing with it. This is bulls* as it is to say that SEO is bulls* as well.

The problem with SEO is that there are no defined lines between what is good and ethical, questionable and flat out bad and selfish. The search marketing industry failed so far to establish rules that define good practices and outlaw bad practices to allow players who play by the rules to distance themselves from unethical business practices. This lack of self regulation hurts the industry as a whole and might even more hurts if the government steps in to define the rules for everybody based on their limited (and often wrong) understanding of the realities in this business. Direct marketers for example were so full of themselves and unable to do something on their own that the government stepped in to force the CAN-SPAN act upon them. The idea was to reduce email spam, which is to a large degree caused by unethical direct marketers. It failed miserably, because the government did not know better. The spam was not reduced, but the regulation caused a lot of issues for legitimate direct marketers who uphold ethical business practices.

Affiliate marketing has the same issue, probably even worse than it is for search engine marketers. This does not make affiliate marketing BS, nor useless nor every affiliate marketer a selfish, spamming and Internet polluting piece of useless faeces how Jason Calacanis was basically summarizing the industry during his keynote at Affiliate Summit a few days ago.

Jason, your Mahalo project will not solve the problems. You cannot be comprehensive, up to date and free of manipulation and skewing at the same time. You might at the beginning while everything is small and limited in size, but it will change more and more while you grow and diversify. You cannot remain clean and complete at the same time. I made recommendations for some niche subjects at Mahalo where no content exists today, which was not reviewed yet. Some of it was recommended over 50 days ago, but is fortunately timeless content and not related to current events. What do you think will happen if your project multiplied in size over the coming months and years? It will only get worse.


Nobody did figure out a solution for the problems that come with general search and I don’t even know if there is a solution for it at all. I personally believe that the future will be specialized search and segmentation. General search will change and become the means to find those specialized search engines and segments. Those specialized search engines will be embedded in highly specialized social communities and be maintained by members of that community to ensure the highest possible relevance of results within that particular niche. Some niches will do a better job than others and multiple competing communities will thrive to make their results better than their competitors and improve the quality of the results by doing so, naturally.

The amount of fragmentation will be different from subject to subject and be determined by the size of community that identifies itself with that specific subject. It will be a very dynamic space, which makes it important that general search engines will do a good job in identifying those groups early and accurately. Since the volume of content to review and rank in each of those individual engines (including the general ones) will be limited to a manageable size (or breaks up into multiple more, if it grows too large), quality can be controlled and kept high, while skewing it will be virtually impossible (unless the whole community is skewed as a whole).

This is my personal prediction of the future of search. What do you think will happen? Is there a solution for general search to be comprehensive, up to date and free of manipulated results? Please use the comments feature below to share your thoughts about this subject.


Carsten Cumbrowski

Carsten is an entrepreneur, internet marketer, affiliate and search engine marketing advocat and a “free spirit”. Find out more about Carsten.
“Complaints without suggestions for making it better are useless.”

Carsten Cumbrowski
Carsten Cumbrowski has years of experience in Affiliate Marketing and knows both sides of the business as the Affiliate and Affiliate Manager. Carsten has over... Read Full Bio
Get the latest news from Search Engine Journal!
We value your privacy! See our policy here.
  • Interesting piece. you should do a followup piece where you show all the videos sites that failed and then explain why YouTube will fail.

    … or you could do a blog post about all the reasons why SixDegrees, Frienster, and Ryze failed and explain why MySpace and Facebook will fail.

    … or you could profile Napster, Scour, and the first wave of music service and their failure and explain why Steve Jobs and iPod would fail.

    I think you get my point… in our industry a lot of failures in a space many times means a lot of lessons learned and a better mouse trap being built.

    If you’re confident in what you’re saying perhaps we should make a long bet? 🙂

  • Hi Jason, I am not one of those guys who just rant and see the end for everything that is being launched. I wrote the article in a way to call you out, which obviously worked out nicely.

    I think Google will actually be able to adjust and not “fail”, unless they become more full of themselves as they already are and miss to see what happens around them to be able to react before they miss the boat.

    You are a smart guy too and Mahalo is not put in stone. I would assume that you will do course corrections down the road if and when they become necessary. That said, I believe that the odds of winning a bet are in your favor. However…

    I am in for a fun bet that will be beneficial for the community regardless, who wins the bet.

    For how long do you have funding available? Add to that 1 year as the end date for the bet. If you loose, I want you to apologize to the search and affiliate marketers who you consider worthless and the scum of the earth (not to the black hat folks and spammers, but to the folks who are honest and righteous).

    I actually hope that you will do that one day regardless of loosing a bet, but just in case that you don’t hehe.

    Let me know if that sounds fair to you and what you want me to do in the case that you win the bet.

    And don’t forget to post the date when your funding for Mahalo stops.


  • Why apologize? He did exactly what was expected.

  • Mahalo will fail, simply because Jason Calacanis is an Ass****.

  • Another reason why Mahalo will fail is because Jason is not Hawaiian & his website has nothing to do with Hawaii or the Hawaiian culture. Jason is simply exploiting one of the most sacred words in the Hawaiian language which is one of the worst curses a person can bring upon themselves. Taking one of Pele’s lava rocks home with you as a Hawaii vacation souvenir is nothing compared to the bad mojo Jason has brought upon himself.

  • very interesting, He did what was expected.

  • “Taking one of Pele’s lava rocks home with you as a Hawaii vacation souvenir is nothing compared to the bad mojo Jason has brought upon himself.”

    Is that from the Brady Bunch episode where Peter took the statue from the cave and Marsha had the giant spider crawling on her?

  • Noooo… You got it all wrong! Hawaii doesn’t even have giant spiders anyway. Greg must have brought it with him. ;^)
    But seriously… It’s bad luck to exploit the Hawaiian culture to promote things that are not Hawaiian.

  • Hmmm….why logo DMOZ on the picture in grey? “He is alive” ? ® Dr. Frankenstein 🙂

  • Rainman,

    hehe.. the tombstones are not in chronological order and the time-line begins in the past.. good eyes for the detail on your part though 🙂

  • btw. Dave, .. “funny”, but it does not help us in the discussion… and your choice of language causes PSA ads to appear on this page all the time now.. oh wait, that’s a Verisign ad.. my bad.. never mind 😉

    you did not respond to my offer for the challenge (that you called me out for). If you don’t like the stakes, make a counter offer.


  • Bob the English teacher

    Good article, good writing, but one glaring spelling error. “Ipidimy” isn’t a word. I think you meant “epitome”.

  • Thanks Bob, I fixed the error in the post. 🙂

  • Mahalo will fail because:
    -It’s far too slow and cumbersome to use (why would I use a search engine or directory that takes an age to load?)
    -It only lists the “top 10” for the “most popular” search phrases (how can you say they are the best for sure? and what if there are 11 that are equal?)
    -Is human edited (dmoz had the problem of editors that were, shall we say a little corrupt? at the last point I guess some are more equal than others :P)
    -Doesn’t cover all search phrases (so while it may be the most relevant it’s going to suck at the long tail – which simply does convert better)

    I could go on… but yeah… machines may be stupid, but at least they can do so much more stupid and so much quicker.