May 07 2009

Twitter Search to Start Indexing Pages and Launch Reputation Algorithm

Some rather interesting changes are headed for Twitter search according to Cnet’s Rafe Needleman and Twitter’s new VP of Operations, Santosh Jayaram.

Jayaram, formerly the VP of Search Quality for Google, broke the news that Twitter Search will begin to start crawling the links which are posted in tweets and index the content of those pages. Instead of simply indexing the tweets of its users, Twitter Search will now offer search technology which will transform it into more of an open window of the live conversations and real time indexing which is happening within the Twitter community.

Twitter Search will also be getting a reputation ranking system, which Needleman reports will take into account the reputation of the user who actually wrote the tweet, and rank its search results based upon that reputation. There is no news yet however how this reputation will be calculated.

Perhaps such a ranking system could be built upon trust factors and influence. In a similar fashion that Google has always counted links as a system of relevance and popularity, maybe retweets and follower relationships and interaction will influence user reputation.

The end result, Twitter harnesses its search capabilities and almost instant ability to not only index a brand new web page or document, but also determine a relevancy and reputation ranking will give the company outstanding leverage in further valuing itself AND even licensing out its technology with search partners in the future.

Written By:

Loren Baker | Search Engine Journal | @lorenbaker

Loren Baker is the Editor in Chief of Search Engine Journal and a Managing Partner at Search & Social. Involved in SEO for more than a decade, Loren launched SEJ in 2003 with the goal to share search marketing information and reporting on happenings in search.

More Posts By Loren Baker

Minneapolis Design Guy

I have seen quite an impact recently with G following the links from profiles but nothing beyond that.
sohbet

thankks you

sohbet chat
Jerry Okorie

I can foresee a crash here, I encountered five bad urls and 3 orphanpages throuigh twitter in 5 mins last week. Imagine if all 11million twitter users encounter a similar problem - and all URL's are indexed - WarZone!!
Tobias Stapleton

I agreed with Gerald Weber that Twitter search is too young but its application is truly effective and exclusive.
Pete Foder

It will be very interesting to see how Twitter handles the enormous amount of urls it gets daily and indexes them.
Gerald Weber

Twitter search is still very young, but it's definitely going to be a part of the future of search and the internet.
Henry Alzamora

Let's remember, algorithms, the one and many nonrecursive functions and programs method, recurrence relations, and the analysis of recursive functions, r t point here AND they aren't finalized yet. Meanwhile the reputation f those u follow, spammers u follow, how old is r Twitter ID, numbers f Tweets per day, RT's average and who reply t you, profile updating, spammy phrases in your url or spammy sites, numbers f time a user is blocked, Twitters ID from same IP, etc, are some of the mathematical functions so far, so take it into account :)
Rick - Consulting

It will be very interesting to see how all of this turns out with the numerous urls and how Twitter will handle all the urls and indexing them.
Colorado Springs Real Estate b

Loren: Great article -- I'm a huge fan of Twitter, and I'll be watching to see what happens with the changes.
Eric Ward

Twitter posts can be signals for Google, like any other online medium can. It will still circle right back around to trust, in this case, of the Twitterer him/herself. We know the spammers have already arrived at twitter, and this will only make that worse, thus it will be vital for those who hope to have any signal influence at Google via Twitter to be cautious with who they follow and what they tweet about. And this is a GOOD thing folks. It will make it more important to be followed-and to follow- the right people. Sheer numbers mean nothing. There are fee based services already that will get you 8 billion followers by sundown. Tweet a URL to a follower base created that way and it's is pointless, and useless as a signal for Google. As with any URL (and twitter pages and tweets all have addresses/URLs) , a backlink profile will emerge, and backlink analysis will show who can and can't be trusted.
Nathan, Rocket 55

I could see this changing a lot of things, but I don't know if this will effect Google too much, at least not right now. Twitter is still in the 'trendy' stage of its life. It needs to stick around for at least a few more years before we can really tell the impact of its service.
Prashant Kumar Pracheta

Hello Loren..Really a nice post.

I also want to publish my articles on SEJ. So please let me know how can i publish.
Mike

Google has to be watching. They are in danger of becoming merely a research platform like Wikipedia.
I am already finding myself referring to Twitter Search for certain terms over Google.
Loren Baker, Editor

@wbw_Jeff : Thanks Jeff, yes, I thnk it would be quite easy for Twitter search, and most search engines, to use a similar technology to identify the specific source of the shortened URL.

@Nicole : I really think the point here is to look beyond link popularity and link passed PageRank and dive into the core meaning of Google counting links, which is to identify the sites which people are "talking about".

It used to be that "talking about" sites or businesses was done on other websites or blogs via links, and the anchor text used to talk about those sites is/was viewed as the relevant external meta data used to classify the meaning of those sites, along with the internal data determined via titles, internal navigation and contextual relevancy.

Now, what we have here in Twitter is the masses talking about specific businesses, sites, ideas and what not ... but no longer via traditional websites or even blogs anymore, but via their tweets.

So the question of NoFollowing Twitter links therefore becomes irrelevant. Because we're not talking linking for ranking, but we're talking identifying the authority or most timely and relevant sites that targeted niche demographics and groups of people are discussing .. in real time.
wbw_Jeff

re: Andrew's comment on URL shorteners, there is already a [great] service called backtweets.com that automatically reinflates the URL's before producing search results. Does a good but not perfect job so apparently the technology is there.
Nicole

How will this affect search results, if at all? Google does not recognize no-followed links, and all links on Twitter are just that.
Loren Baker, Editor

@Andrew The handling of URL shorteners does sound interesting, as traditionally they are regarded as not being spider driven links, but the Twitter handling of these, which will probably lead to a different direction from the other engines, could very well change this.
Andrew

It will be curious how they handle all those URL shorteners directing to the same page.
CAP Digisoft Solutions Inc.

It's great to hear from Twitter for the new search implementation and indexing tweet links and content in those pages as well.

But regarding the reputation it's most concerned how they use in their's website for all the twitter posts.

Looking forward for the new functionality to seen in action.