7 Things You Can Do With Xenu Link Sleuth

SMS Text

Xenu is a highly valuable tool (that’s another great exception from the rule: it is both free and awesome). Let’s see how we can use it for SEO purposes:

1. Find broken links (to make your site easier to crawl and thus increase your crawl rate):

Xenu: find broken links

2. Identify your site internal duplicate content issues (duplicate titles):

Xenu: duplicate content issues

3. Find largest pages/images (to optimize your page load time):

Xenu: optimize your page load time

4. Find least linked-in pages (quick tip: to learn which exactly pages link to the current page, right-click / ALT+ENTER on the URL and choose “Properties”):

Xenu: Inlinks

5. Find pages with most outlinks (increase your site crawl rate by minimizing the number of each page outbound links):

Xenu: outlinks

6. Find your deepest-buried pages and learn how easily they can be found (by “status” and “level” properties):

Xenu: deep pages

7. Find images that lack alternative text:

Xenu: missing alt text

More tips:

  • Right-click on an URL to get instant access to Google Cache and Internet Archive;
  • CTRL+R will retry all failed URLs;
  • Set less parallel threads in “Options”=> “Properties” to have fewer failed connections.
Ann Smarty

Ann Smarty

Brand amd Community Manager at Internet Marketing Ninjas
Ann Smarty is the blogger and community manager at Internet Marketing Ninjas. Ann's expertise in blogging and tools serve as a base for her writing,... Read Full Bio
Ann Smarty
Subscribe to SEJ!
Get our weekly newsletter from SEJ's Founder Loren Baker about the latest news in the industry!
  • g1smd

    I have been using Xenu for many years. It is invaluable. There’s yet more stuff you can do:

    — Run a list of test URLs (saved in a text file) covering every possible URL *format* that might ever be requested (Note “every URL format” *not* “every URL”), to make sure that everything non-canonical gets bounced with either a 301 or 404 response. It is amazing how many ways a CMS might have to still generate a templated page full of random content, or even generate no content on a page with full outgoing navigation.

    — Scan the site as normal, and look at the final report (generated to an HTML page) and then identify navigation links within the site that are hitting a redirect because they point to a non-canonical URL which already has a redirect in place to correct it.

    — Look at the HTML report page, at the list of outgoing links to other sites, and identify broken 404 links to fix or delete, and outgoing links that hit a “stupid” redirect on the other site; one that can be easily corrected by changing the outgoing link URL here.

    — Find errors where relative internal navigation points to the wrong place and can cause crawl errors.

    — Find links within your site that you didn’t know existed. These occur in some design programs, where when you delete a link, only the anchor text is deleted. In the HTML code the link still remains like [“a href=”somepage.html”][/a] and is still pointing at some URL (which may or may not exist) but which search engines may treat as being a “hidden” link.

    — Find bot traps, with infinite content, such as an unbounded calendaring system. I hit one of those recently; managed to spider the calendar back to the 1700s and forward to the 2300s with every page (except for those in 2007 and 2008, and early 2009 ) being devoid of any unique content.

    — Find images that are waaay too big. I have caught someone with 1 MB files in the “thumbs” folder before now.

    — Check the operation of bot traps, where those limit speed of spidering, or number of pages accessed (there are better tools for that, but this can be useful sometimes).

    — Generate a basic HTML sitemap (look at the HTML report). If the visual structure of the sitemap isn’t close to that you had envisioned for your site, you have problems with your internal linking. Especially look for items in the Sitemap that say “301 Moved” or “302 Moved” too.

    Yes. A very useful tool and one that I use nearly ever day for something or other.

  • SEO Mumbai

    Thanks Ann and g1smd for your tips. Xenu is really wonderful tool which can help us for both on-page as well as off-page SEO

  • Jack

    I’m always looking for good tools to help me with my “best practices” game. Half of SEO is making sure the onpage factors that we can control are all squared away. If this tool helps me make better pages I’ll use it a lot. Thanks for the tip

  • Amit Nyamtabad

    Xenu is really invaluable for SEO research. I really liked point #4 and #6 a lot πŸ™‚

    Thanks a ton for sharing.

  • WebSite Design Orange County

    Hmmmmmm, no Mac version. WTF! Looks like we’ll stick with ALM.

  • SEO company

    Very interesting post.

  • Mher

    Has anyone used this tool for joomla based sites. I have had nothing but issues with this tool.. it always seems to crash.. is it because of the dynamic urls?


  • g1smd

    I have never had Xenu LinkSleuth crash outright on me.

    However, if there is a situation where a site has very bad duplicate content issues often caused by internal linking that results in an infinite loop of crawling, then the process that generates the HTML report can take many *days* to do its stuff – and may well run out of memory before that.

  • g1smd

    For Joomla-based sites, is the link checker spidering a part of your site that happens to be an infinite, unbounded, calendar? That can cause issues to any bot, but Googlebot et. al will abort after some while.

  • Sebastian

    Is it possible to teach it going on subdomains?

  • Arnie K

    I would be curious to know about search subdomains too. Great tool.

  • Bfri

    I have come across the tool several several times but never had the time to really check it out. This is definitely a very useful post and provides me with another reason to give the tool a try. πŸ™‚ Thanks!
    Also good sources to compare 404 reports with the reports from Google Webmaster Tools.

  • Software Testing

    I recently used the same tool to find the number of pages in my website πŸ™‚

  • Ben McKay

    Hi Ann,

    A great post. I use Xenu like many SEO’s loads but didn’t know it could do so much.

    I love how the uglier the tools the better they perform too…it makes an interesting graph.

    Remember how ugly Firefox used to be?!

    Ben M

  • Jeff

    Thanks for letting us know about this tool. I installed it and within a few minutes I was able to fix some problems with my web site that I was unaware of. Thanks again SMARTY! πŸ™‚

  • SeekGeek

    Great Post Ann

    What would be really awesome is if it could show only “follow” outlinks (point 5) i.e. outlinks excluding “no follows”

  • SeekGeek

    The other thing that would improve it plenty is the ability to sort by more than 1 column (like excel’s sort by…and then by.. function) So you could sort by eg. type then title.

    Oh well, its a freebie, can’t be too demanding…

  • Matt Inertia

    Been using xenu religiously on sites for the last year. Great article.

  • Tamil Movie News Online

    Interesting post and valuable information about Xenu tool. Thanks Ann

  • Rathis

    Great tips! Very helpful for me. Thank you very much.

  • cw360

    Xenu in invaluable to me. I have been using it for about a year now. I would not be able to maintain the sites I maintain without it.

  • Imathon

    Totally agree with SeekGeek. Excluding “nofollow” is a must, hopefully the next feature!

  • g1smd

    *** What would be really awesome is if it could show only “follow” outlinks (point 5) i.e. outlinks excluding “no follows” ***

    *** Totally agree with SeekGeek. Excluding “nofollow” is a must, hopefully the next feature! ***

    You are totally missing the point. This tool is for checking that the links actually *work*, and that your aren’t linking to a URL that returns 404 or a redirect. So, every link should be followed so that it can report which ones are useless to the people who click on them.

  • SeekGeek

    @g1smd the title of ann’s article was “7 things you can do with xenu…” not ” 1 thing…” Yes, locating 404s is the main thrust of the prog (as covered in point 1) but when counting outbound links (point 5) the “nofollow” variable is critical info.

  • Kevin Kirkpatrick

    I’m trying to figure out how to have Xenu limit the pages it crawls. For example, if I just want it to crawl the homepage and then stop. Then from there I could say ok, now go crawl these pages. Instead of doing it backwards by limiting every page.

  • Neil

    Great post. Thanks AS.

  • nuevojefe

    If crawling very large sites I would recommend going to Options >> Preferences >> Advanced Tab and unchecking “Fail all URLs with Failed Host” so you can leave it running overnight (for example) and not come back to find that it finished prematurely because of a temporary response problem.

    I have never had Xenu LinkSleuth crash outright on me.

    It can depend on the machine obviously. I had a few year old cpu running xenu for me and when the URL count got around 800,000 it would tend to crash. I just recommend pausing and saving the file every now and then for large projects.

  • Ahmed

    I just finished exploring XENU 1.3 and found it really helpful in testing web applications. We also tried Xenu for Site menus (such as drop downs ,site navigation menus etc) but were unable to get any result. Can you please confirm if we could infact use it for site navigation menu testing as well?(without manually clicking on each menu sub level individually)The site menus are written in JavaScript while each sub level menu points to some live urls (which are obviously hard coded)o I wanted to inquire is there a way or any Xenu feature , through which Fiddler can check all urls/link specified in Menu drop down without manually clicking them

  • Mark

    Thank you for this article. I discovered Xenu a couple of days ago and it has taken me that long to correct my broken links to external sites which I didn’t know I had – hopefully my site will improve as a result. I read your article to learn all the features so I get the most out of it.

  • Andrew

    I’ve had Xenu installed for a while now but I hadn’t actually used it much until today. As usual Ann, your tips are extremely helpful (and g1smd) and have made my task of analysing our link profile a whole lot easier. Thanks!


    I have got some crazy problems where two of my posts are causing huge headaches. They are creating hundreds of links with the same category name over and over e.g. ../../featured/featured/featured/../../../post-name

    The worst thing is that every single combination of the links actually opens up the article.
    I’ve attached a screenshot.

    Please help. I’m desperate.

  • Yaroslav Vorozhko

    Hi guys,
    there are a new online service this the same features http://linkaider.com.
    Try it and share what you think.
    And it is free.

  • Carlos Machado

    Hi Ann,

    Congratulations for your blog.
    The Xenu is a very useful application.
    I recomend it, like you, for all.

  • Ed

    It’s quite a nice tool thanks Ann for showing us how many other functions it has just found it yesterday and i am still learning how to use it.

  • John Turner

    Great article, and I just wanted to add that for the Mac folks theres a free tool that seems to work well (maybe not as powerful as Xenu) but can work in a pinch: Integrity


  • David

    Great post ann, and i advise people who are having issues with larger sites to trial the 64 bit version of the application, it seems to run much better if you have a 64 bit system.

  • BM Models

    Amazing tool, I've been using it for a while but I don't know it is capable of so many different yet interesting things. Many thanks!

  • rexwilliam

    thanks buddy,really xenu is very useful ,i hav been using xenu from last couple of month,but today i got 2 know all the fields abt xenu thanks buddy

  • rexwilliam

    thanks buddy,really xenu is very useful ,i hav been using xenu from last couple of month,but today i got 2 know all the fields abt xenu thanks buddy

  • Robin Moore

    Thanks. Great tips and a great tool – will use it more often for technical SEO. Thanks to G1SMD for the further detailed uses – really helpful, thanks.

  • Jijo

    Thanks for the wonderful article. Could you please mention how can we manage the Xenu, if account authentication is required.