Tools

Yahoo Pipes Social Bookmarks Search

Yahoo Pipes, to date, seems one of those unsung tools that could really be useful, especially to SEOs. However, when I scour the sixty plus SEO blogs whose feeds I subscribe to, I don’t see Yahoo Pipes mentioned much. (I just checked.) But it can be of use:

  1. Aggregate multiple related web feeds.
  2. Filter for desired topics.
  3. Sort multiple feeds by date, to aid in detecting temporal trends (topics).
  4. Sort by keywords in feed item titles.
  5. Sort by member who submitted the story, in the case of social bookmarking sites.

I’ve been posting a number of simple Yahoo Pipes video how-to examples over at Tubetorial, but here’s a fresh one: social bookmark search. (Splashcast video at bottom.) I’ve taken the several feeds and merged them:

  1. del.icio.us.
  2. digg.
  3. reddit.
  4. slashdot.

Okay, Slashdot isn’t really a social bookmarking site per se, but it’s kind of the predecessor – no voting. And Digg and Reddit are more social news than social bookmarks. But it’s simply an example. You can change the feed list to your liking. [Yahoo Pipes could really do with an OPML import feature.]

After merging the feeds, an user-supplied search term is applied as a topical filter on the feed item titles or descriptions. The result is a new RSS feed that you can subscribe to and check on social news/ bookmarking trends more easily. A simple video how-to on building the Social Bookmarking Search pipe (which you can clone and tweak) is below.

Notes
A couple of things to take note of:

  • Digg had once blocked Yahoo Pipes, but I haven’t had any problem lately. In fact, I have another Pipe that organizes a day’s home page stories sorted by Digg submitter. (I’ll find it and post it here.)
  • There may be some problems with the audio of the SplashCast. If so, try again later. However, you may not need my narration to follow along.)

Comments are closed.

17 thoughts on “Yahoo Pipes Social Bookmarks Search

  1. Apologies: There’s a problem with this theme, I believe, in displaying SplashCast videos. Loren will have to take care of it when he’s back.

  2. Great post — I just feel as though the ways of crowdhacking should be taken into consideration for the content of this post. To an SEO, the SEO only wants to build links and increase traffic, Yahoo pipes gives an excellent research section but doesn’t give as much as to the whole effect as link-building like sites such as PageFlakes and the “My”s of other sites (MyYahoo, iGoogle, etc.) —

    Considering Google studies user data in order to give greater rankings to their sites, I feel as though the best way to game something/build traffic from a resource would be to both utilize the organizational efficiencies of the site as well as the user data sent through the site (build MyYahoo accounts and get the URL of your site stated in as many feeds and “My” sections of sites as possible — the more mentions of your URL, the higher your levels of user data)

    Just something to build up on the convo =o). Great post

  3. iMarketingGuru: Thanks. Good points you’ve mentioned. More sites will do this, though Pipes is still in beta. I know that SplashCast, the video player I’ve used above, will list the SplashCast video/ show above in their media directory. So anyone who watches that directory will see the video and may come visit SEJ.

    Obviously, a similar feature would be nice for Yahoo Pipes. They list user pipes but not any user details. That may of course be due to the fact that you can’t exactly embed a Pipe anywhere. You can grab the resulting RSS feed and SOMETIMES use it in other mashups. (E.g., SplashCast can’t use Yahoo Pipes feeds but Tumblr can.)

  4. UPDATE: I’ve replaced the original hard-to-follow video with something easier on the eyes. And voice is working, so you can hear my narration. There is a slight error in the Filter rule, but it’ll still work. I didn’t, however, show you how the Pipe is run because that’s pretty simple. The link to the Pipe is in the post above. You can clone it and tweak to your heart’s content.

    I’m also working on a Pipe that checks the Digg home page and sorts the day’s stories by the story submitter. So you can see who, if any, is getting more than one story to the home page on the same day.

    These are simple examples, but should hopefully illustrate the powerful applications you could use Yahoo Pipes for, including as a research and analysis tool for SEO.

  5. I’ve been experimenting with Yahoo! Pipes over the last few weeks to merge several of my RSS feeds, and from my experience, the service is still pretty buggy.

    Earlier this week, the service randomly stopped fetching one feed and I got a parsing error for another feed for no apparent reason. A few hours later, it was magically working again.

    Earlier this month, I had problems because Pipes reformated my HTML code in WordPress feeds so it displayed code rather than showing links, images, and text formatting.

    Thankfully, a few members of the Pipes development team frequent the Developers’ forum, and have been pretty helpful thus far. I would have never thought of the WordPress workaround they pointed me to (using the “rename” operator in the Pipe).

    I think this tool will be extremely powerful once they work out the kinks.

  6. Krista, agreed. It’s still extremely buggy, sometimes annoying, but will it ever be powerful once it’s out of beta. I think part of the problem is that not only are their servers overloaded, but they rely on other services, whose servers may be overloaded or decrease access to mashup services such as Pipes.

  7. Thinking beyond simply aggregating content with Yahoo! Pipes, this is the kind of thing that’s got the potential cut through ‘middleware’ providers.

    Might even save you the odd monthly subscription fee.

    With access to enough feed-based content, you could pretty much keep an eye on any type of data — be that housing availability in your locale, up-to-the-minute flight times from a selection of cities, right down to .. err, news!

  8. Wayne, absolutely. Pipes is a powerful tool that seems to have gone mostly ignored. If not, bloggers seemed to have stopped writing about it. And it really isn’t that hard to use.

    The only disappointment is that users don’t have more control over the actual XML format that gets generated in some cases. For example, I can’t create a YouTube search Pipe then feed it into SplashCast. The latter cannot find the necessary video enclosures. Pipes has quirks that way

    But I’m absolutely certain that Pipes should be in the toolbox of all SEOs in some form or another, and I hope to have a hardcore Yahoo SEO Pipes how-to series over at http://rsscases.marketingstudies.net in the near future. The simpler tutorials are at Tubetorial.com

    I’ll be providing a few more examples here at SEJ as well.

  9. Hi Raj!

    I wasn’t totally won over by Yahoo! Pipes originally, but I warmed to it over time.

    It may be that it’s because it’s so versatile that it’s not clear what it’s one purpose in life is.

    Most people are more familiar with the idea of the one-trick-pony web service, and Yahoo! Pipes isn’t that!

    I’d like to see what happens when the likes of an Adobe Apollo web application integrates with Pipes, or maybe Google Gears…

  10. I decided not to sign up for Apollo (can’t remember why) but I’m looking into gears.

    If I were to summarize Yahoo Pipes’ purpose, it’s a tool to mashup RSS feeds and manipulate the data. Sort of but not quite a replacement to the XSLT spec for XML manipulation.

    So if you want to filter, query, collate, sort, augment or otherwise manipulate RSS/Atom web feeds and have a visual interface to rapidly prototype, that’s Yahoo Pipes.

  11. I tried Yahoo! Pipes ages ago and was quite disappointed.

    Agreed, it was back then even more buggy, less stable and had less features, but I had several things in mind which I wanted to do and could not get it done in a reasonable amount of time or not at all.

    I was going to use other tools and services to accomplish what I wanted. I use Google Reader since about one year to aggregate RSS feeds and get them sorted by date automatically. I simply use one tag for all the feeds I want to consolidate, make it public and get the feed for the tag. Then pass it through feedburner to make it “pretty”. See the aggregated SEO news and affiliate/internet marketing news.

    Adding or removing a feed is done in no-time and was a pain with Y! Pipes if you wanted to consolidate 20-30 feeds. You mentioned OPML support already. I guess that would make it easier (maybe).

    I also tried to use Y! Pipes to convert a Google Calendar XML feed to RSS, convert that the event date will be the published date instead of the date when I added an event to the calendar. Furthermore did I want to merge the event start and end date/time to a string and add it at the beginning of the description, because RSS does not support an “End Date”.

    I was unable to get it done with Y! pipes and do it now via XSLT. Check out the RSS version of my industry events calendar to see what I mean.

    I was at first very excited about Yahoo! Pipes until I realized that the things I wanted to do either can’t be done with it or are so cumbersome and time consuming that it does not make sense to use the service.

    The filter option and alternative sorting is a plus and not as easy or not possible to do with existing tools. If they would just go the full nine yard and provide the needed flexibility it would be great, but from that are they still far far away IMO.

  12. Hi Carsten!

    Reading your experiences with Yahoo! Pipes is reminiscent of my first experiences with Apple Automator — seemingly full of promise, but a huge let-down.

    Have you been in contact with the guys at Yahoo! to let them know?

  13. I am usually always providing feedback, if I check out something that is new so I believe I did (Its over a year ago :). I did that with Google tools en-mass and I am sure that I did it with Yahoo!, if they did have a feedback system back then.

    I got last month twice a problem with Yahoo! as a result of well intended feedback and reporting bugs (in non-beta applications). One was only making me mad, the other cost me money, which makes me more that mad.

    Different story. I am currently giving Y! a bit more time to get their act straight before I blog about it in a major rant, that Loren might has to go over it to remove the “f” word a couple times hehe. So currently, no help for Y! from me.

  14. I’ve been bouncin’ feedback at Google for long enough, so I feel your pain.

    It’s interesting that you should mention Google, because they recently did a developer day on Yahoo! Pipes and how to get the most out of it…

  15. Hey Carsten, okay, Pipes was a let down for me, too, because as a long time XML evangelist, there are tons of manipulations I’d like to do. Yahoo gave Pipes some advance features and left others out. Even their Regex is one-sided, only usuable for changing content, not split feeds.

    However, they are very good at taking input about it:

    Yaho o Pipes discussions.

    I’m really hoping that they can truly match XSLT’s (theoretical) capabilities and beyond. But there’s still a lot to be said for Pipes and its usefulness to SEOs. I haven’t done so yet, but I figure that anything I cannot do in Pipes, I can send to a Perl or PHP script running XML modules and finish there. If I come up with anything useful, I’ll release it to everyone.