SEO · Tools · WebMaster Resources

No Fear of Remapping URLs of your Site

Search Engine Watch published an interesting case study today about a site that remapped all dynamic URLs via a tools such as mod-rewrite/.htaccess file (Apache Webserver) or Helicon ISAPI Rewrite (Windows IIS Webserver) to static URLs and 301 redirected (Not 302!) all dynamic URLs to the new static looking ones.

For example  “site.com/category.aspx?Cat=991″ , which is a .NET script that dynamically renders different product category pages depending on the value of the “Cat” parameter value, probably pulled from a Database, was changed to  “site.com/Cat/Bathroom-Faucets/991″.

The rewrite tool translates in the background the seemingly static URL to the old dynamic one based on specified rules, so no code needs to rewritten or even changed. You should of course change your website internal links to the static looking ones, which probably takes a bit programming work, depending on the amount of internal links and if those links are also generated dynamically by a script or if they are “hard coded”.

 The case study showed that this simple change alone had a significant impact on the sites search engine visibility. Nothing else was changed or optimized during the study. The number of indexed pages in Google almost doubled, tripled in Yahoo! and almost quadrupled in MSN.

The traffic from organic search results increased by 50%. Not bad for a simple change like that. This result can not be seen as an average, nor as specifically dramatic or low. Results depend on a lot of other factors, such as the number of pages (percentage) of all available (unique) pages already indexed by the search engines.

That a change of dynamic URLs to static ones will do little miracles like this was nothing new to me. My eyes looked in disbelief at the numbers when I did it for the first time for a site of mine in 2002. Search engine spiders had back then much bigger problems with dynamic URLs than they have today. Google even changed their Guidelines not to long ago regarding dynamic URLs. However, this study shows that it still makes a difference for the search engines today and that you look into this for your own website, if you have cryptic looking dynamic URLs.

This Webconfs.com article seems also to confirm the results of SEW’s case study. If you want hear what Google’s Matt Cutts has to say to this, check his video at Google Video.  Do you need more proof? How about an additional reason for the change. Rand Fishkin at SEOMoz pointed out once that static URLs also tend to improve their usability. 

To the best of my knowledge is it not proven yet that static URLs shown for a page in the SERPs (Search Engine Result pages) instead of a dynamic URL with ? and & characters will increase click-through rates, but I do not need a statistic to prove to me that static URLs look “friendlier” to non-Geeks than URLs with one or multiple weird and cryptic characters. Add to the static URL some descriptive element such as the category name or product name and I bet my retirement that the new URL will outperform the old in regards to CTR (Click-through Rate).

Another thing I got out of this case study was something else. The merchant changed virtually ALL URLs of his site over night. The simple thought of the possible consequences of such change alone can cause nightmares for Webmasters and SEO alike. Will the search engines pickup the new URLs and drop the old ones without risking that may be all pages will disappear because of the feared “duplicate content penalty” (not filter!).

While those fears seem to be warranted to some extent* (see the SEOMoz post “Expectations and Best Practices for Moving to or Launching a New Domain”) does it seem to be misplaced for a change and 301 redirection of all URLs on the same domain. There was virtually no negative impact of the change for the site in SEW’s case study.  The number of pages indexed and traffic  simply was going up only. In record time I might add.

A behaviour as seen for whole site moves to  a new domain would be reason enough for a business to think more serious about doing this. if it is worth the risk and if the potential loss of traffic for a limited period of time is financially feasible.

Here are some additional resources to the subject of redirection, URL and URL rewrites you might find helpful.

Helpful Resources

Search Engine Watch Thread: mod rewrite tips and tricks

Rand Fishkin at SEOMoz: 11 Best Practices for URLs 

Cumbrowski.com: SEO Webmaster Tools and Resources

Do you know other good resources to this subject? Share them with and post a comment below.

Carsten Cumbrowski
Internet Marketer and Entrepreneur and owner of the free internet marketing resources portal at Cumbrowski.com

* If you want to move your whole site to a new domain, I strongly recommend to contact an expert SEO, who is specialized in this and can show already completed and successful moves of sites to his credit. Things are constantly changing, to the better or to the worse. Discuss options, possible consequences and create together with the SEO an action plan to make the transition as smooth as possible and with the least possible negative impact on your site’s traffic in the short, mid and long term.

 No Fear of Remapping URLs of your Site
Carsten Cumbrowski has years of experience in Affiliate Marketing and knows both sides of the business as the Affiliate and Affiliate Manager. Carsten has over 10 years experience in Web Development and 20 years in programming and computers in general. He has a personal Internet Marketing Resources site at Cumbrowski.com.To learn more about Carsten, check out the "About Page" at his web site. For additional contact options see this page.

You Might Also Like

Comments are closed.

47 thoughts on “No Fear of Remapping URLs of your Site

  1. Gamermk: make sure that all old URLs will be redirected properly. Changing internal linking to the new links is also important.

    As ongoing task: try to get as much inbound links that have the old format changed to the new, especially the really important ones.

    The case study does not say anything about the Merchant actively contacting other webmasters to change links, but it surely does not hurt, especially in the long run.

  2. @Whiteboard
    mod_rewrite is independent of PHP but can certainly be used with a site built around PHP. You should google “mod rewrite basics” and “mod rewrite php” and check out some of the documentation & examples. I am fairly certain there is no all-knowing magic tool that can rewrite your links for you.

  3. This is a great point, but everyone who plans to do this should note that the regular expression engine in mod_rewrite is notoriously difficult to work with. Even the best of the best sometimes claim to have no clue why/how one regexp works and another doesn’t.

  4. I am no PHP developer. I can tweak PHP code, but not write it. I am more the DB guy and throw a couple gigabyte of data around hehe.

    I have classic .ASP code that might helps.
    I took the time to write some generic functions that take care of things like www. and non .www urls, multiple domains for the same site that need to be redirected to the primary one and some code to to strip out tracking parameters after you tracked your stuff to redirect to the same page with all parameters except the tracking ones. I might put them on my site. This stuff seems to be in demand at the moment.

    ASP has a disadvantage over PHP. ASP can not determine the URL shown in the Browser so for the one part, the resolution of dynamic URL parameters, what this article is about are you not getting around on a windows server to setup an ISAPI filter (equivalent to mod_rewrite). I still prefer to do as much with the code, just to keep the rewrite rules limited in number and complexity.

    I think I have some PHP code also flying around for the dynamic URLs, but I need to ask for permission to make it public now since it is part of an upcoming book (Professional Search Engine Optimization with PHP) by Jaimie Sirovich from seoegghead.com and Cristian Darie.

  5. Important is to know that no matter what you use as script language, If your URL is changed a lot, especially if it looks like that your site has a bunch of sub folders (“/” in the url after the host name), but the sub folder do not really exists, mod_rewrite or an ISAPI filter will be necessary to make it work.

    As I said earlier, it is possible to move a lot of the logic to the script code, but not all of it.

    Also a reminder, you increase the number of 301 redirects from one to potentially 2 if the mod_rewrite rule must be used (sub directories) and the rules in your script code.

    That is by itself not a big problem, but keep that in mind if you have multiple sites and a lot of redirection and moves going on that you don’t shoot yourself in the foot by redirecting one request xx times and upset the crawlers. I don’t know how much redirects (301 and 302 combined) does a crawler follow until it stops.

    They have to stop at one point, because if you don’t be careful and make a mistake, you might developed an endless loop where you end up redirecting to yourself over and over again. That would trap the SE spider, so they need to stop following the redirect to protect their crawler from getting trapped.

  6. We’ve been using this very same ISAPI control on our site to clean up the URLs. It works perfectly and combined with keyword-rich URLs has catapulted our search engine ranking. Of course, all code is semantic XHTML with CSS for the best ingestion into the engines.

    Anyone else on .ASP or .NET this is the only way to go. Those of you on Apache already have this with mod_rewrite.

  7. Interesting. Of course ideally “site.com/Cat/Bathroom-Faucets/991″ should just be “site.com/Bathroom-Faucets/”. The 991 still has an air of table-id-ness to it and it makes customers wonder what 991 is all about. I personally would expect to be looking at faucet number 991.

    Also, “site.com/Cat/Bathroom-Faucets/991″ makes me think that if i crop off directories, I could get to other pages, so I’d try and make sure that “site.com/Cat/Bathroom-Faucets/” and “site.com/Cat/” don’t throw 404s.

    This actually reminds me of something Amazon started doing — putting text in their URLs for SEO — text that doesn’t really do anything. Note how http://www.amazon.com/Visual-Display-Quantitative-Information-2nd/dp/0961392142/ goes to the same page as http://www.amazon.com/dp/0961392142/

  8. Codex: I agree with some of your points. However, you need the ID somewhere, because at the end of the day must be a dynamic script be called that needs parameters that tells it what data to pull from the DB.

    Alternative rewrites to “site.com/Cat/Bathroom-Faucets/991″ are: “site.com/Cat/991/Bathroom-Faucets.html” or “site.com/Cat/991_Bathroom-Faucets” (the ID is part of the Name, separated by underscore)

    There is way to get rid of the ID, but that would require additional programming and also take away from the Websites performance.

    It requires unique Category and Product Names, because you would use the names for the look-up in the Database instead of the ID. The URL would look like this: “site.com/Cat/Bathroom-Faucets” or “site.com/Cat/Bathroom-Faucets.html” and would translate to: “site.com/category.aspx?Cat=Bathroom Faucets” (The fact that the URL is not encoded at this point does not matter).

    Why does that take away from the Website Performance, if there is only one Category with the name “Bathroom Faucets”? That has to do with how Databases work.

    Databases do an awesome job when the Keys for Look-ups and Joins are Numbers, Integers without digital point. The smaller the better. Text/Strings are fine if they are in the results of a query, hey that are DB’s for.

    That is also the reason why most “IDs” in everyday life are Numbers, in case you never noticed that.

    Strings/Text can be used as Keys, but it takes the DB a heck of a lot more resources to pull the wanted data. I would not do that for sites that either have a high volume of traffic and also not for sites that have a large number of SKUs (100,000+).

  9. Hmm … I’m not so sure about the case study title: “Remapping Dynamic URLs to Static URLs can Increase Traffic.”

    Granted, they did remove the query string, but I’m hesitant to conclude that such an action automagically increases traffic. (To be fair, the article states as much. The study title, of course, suggests otherwise!)

    Read my weigh-in entitled “Query String Removal Helpful?” here:

    http://www.joesapt.net/2007/03/25/07.54.55/

  10. i have been doing that for a while now, first with my wordpress blog. it really does help and some categories will rank quite highly.

  11. Off topic -A few quick edits for you:
    ..My eyes looked in disbelieve at the numbers.. -> disbelief
    ..I do not need a statistic to proof to me.. -> prove
    …fears seem to be warranted to some extend…->extent

    Thanks for the excellent article.

  12. Joe: If all pages of your site are already indexed and in the search engines, then it will not help you. The rewrite of the URL improves the crawl-ability thus search engine spiders will pick up more of your content.

    More pages in the search engine’s index = more traffic. It is as simple as that. Also, think about going even further and get descriptive elements, such as the product name or category into the URL.

    Also, wouldn’t you agree that a URL with that looks much more reassuring to a user, especially if the url has now a matching keyword, who sees the URL in the search results? That increases your chance that the user clicks on your link and not any of the other 9+ (Ads etc.) he can choose from.

    cc: (that are not your initials :) ), anyway. Thanks, I appreciate it. I will fix the spelling errors right away. English is my second language, but I am getting better at it :)

  13. Carsten – point well taken! I’m all for readable/reassuring links, and I understand they were going from “not all pages indexed” to “most (if not all) pages indexed.”

    Still, I’d be interested to know if these spellings would have been just as effective:

    “site.com/Cat/Bathroom-Faucets?id=991″

    “site.com/category.aspx?name=Bathroom-Faucets&id=991″

    Not that I would use them! I’m just wondering if the additional context works equally well in the query string.

    When content or IA changes (often times out of necessity), previously bookmarked links with keywords or other textual info can go out-of-context.

    So long as they’re mapped to keep in sync, all’s well. Having worked on some high viz sites though (Media Metrix top 50), I don’t see that happen as often as I’d like/want. :(

    Still, that doesn’t mean it’s not worth doing – you’re right on!

    More POVs on link spelling:

    http://mar.anomy.net/entry/2003/06/22/17.15.00/

    http://diveintomark.org/archives/2003/08/15/slugs

    http://www.w3.org/Provider/Style/URI

  14. Joe: The best is of course to implement something like discussed in the article right from the start, but when new sites are being developed, search engines are usually not on the minds of the people involved. Search Engines becomes often at a later point important when the site is already more or less indexed and has inbound links from numerous other websites, not to mention user bookmarks.

    For that reason is it critical to make sure to know your existing site structure, which does hopefully not other problems as well (which would go beyond the scope of this discussion), know the new structure and do a 301 redirect from all old URLs to the corresponding new one.

    The 301 redirect is key here to

    a) transfer power of all inbound links from the old to the new URLs
    b) redirect users that follow an old link to the new page instead of showing a 404 page and upset the user
    c) redirect users that have an old URL in their bookmarks. Just as b)

    After the rewrite are they all new pages for search engines, if just looked at them by itself, the 301 tells the SE that it’s the old page at a new address. You can compare it with the mail forwarding service by the USPS. Anybody who moved probably used that service.

    The difference is, that it’s hard to get all references from the old addresses changed to the new, which makes it kind of a permanent thing and a reason why you should not go through that exercise very often (see my previous comments regarding number of redirects).

    Dynamic URLs with ? and & in the URL have always been a problem for search engines. Yahoo’s old spider was notorious for getting dynamic URLs wrong. I remember how a colleague of mine wrote a routine in the custom 404 error-page just for that spider. He “fixed” the URL and redirected to the correct URL. We used that routine for more than one site.
    Google got better over the years, but they are still not perfect.

    Google might be the most important, but is not the only one. Most other search engines have more problems with those URLs than other. You saw the case study results and the increase in number of pages indexed by the next two biggest search engines.

    Why do they have a problem? I can only speculate, but I believe that it must have to do with all the other routines they have to figure things out like “Session IDs”, “Search Queries”, “Printer Friendly” versions of a page. Parameters that only control the display of the data on a page, but not the content itself (sorting etc.). Those are just a few, I can think of a lot more. Its how I explain how the Yahoo spider got the URLs wrong all the time. They try to figure out what the URL of the best version of the page is and discard the rest, because its the same stuff. This is only speculation on my part, but it makes sense to me.

  15. Here are some functions written in classic ASP – Active Server Pages (Microsoft IIS web server) for 301 redirection. as I promised in an earlier comment.

    They don’t cover (yet) the part to make the URL “pretty”, but they are the foundation. The current example code covers the first issue with URLs, which are canonical URLs and multiple domains pointing to one site.

    I explained everything in great detail, which took some time. I already ,mentioned things to come soon. Lets do this one step at a time hehe.

    I did not proof read the content yet so beware of the typos :)

  16. This is a bit funny, because I’ve done the whole ‘redirect’ thing several times without any hitches, and I have no clue why people get so worried about it.

    If you know what to do and if you keep things simple, you’ll do fine!

  17. I did a similar thing to my site a while back and noticed a huge increase in traffic. mod_rewrite is an amazing tool and should be used by all site admins. your not 2.0 without ses urls!

  18. Sanjeev, you have much much bigger issues because of the frames. Have a look at what Google has in the index for your site here. Product Details. Okay, but those details are usually within a frame set. If a user comes from a search engine and wants to buy it, problem. I can add it to the cart, but where is the shopping cart?

    To solve that issue, check out the solution here. That piece of code forces the rendering of the frame set, if a page that runs usually within a frame set is accessed directly.

    The second part of the solution makes sure that the user ends up at the same page again after the frame set was enforced and not just on the sites homepage.

    And what is up with the search to the left as replacement of product categories. Why did you do that?

  19. I have classic .ASP code that might helps.
    I took the time to write some generic functions that take care of things like www. and non .www urls, multiple domains for the same site that need to be redirected to the primary one and some code to to strip out tracking parameters after you tracked your stuff to redirect to the same page with all parameters except the tracking ones. I might put them on my site. This stuff seems to be in demand at the moment.

  20. ??? Hi Rohit,

    I doubt that two people do exactly the same thing at the same time. See my code and documentation here at Cumbrowski.com. I pointed to in an earlier comment already., but who knows. Weird things happen from time to time.

    You did not provide a URL to your code in your comment. Would you be so kind and post it here?

  21. Looking at this issue from the point of both a user and a spider, it is always good to rewrite dynamic urls to static ones. It will be easier to remember a static url as a user.

  22. I think the whole point of the url is so that the site can be easily recognised thats why domains such as pizza.com have sold for so much money.

  23. Hmm. Once your websites are recieving some traffic, it is a little scary to change things because of fear of loss of the little amount of traffic you are already geting. But nothing ventured, nothing gained
    I think the key is changing one thing at a time.
    Your tips on remapping URLs is gratefully welcome

  24. Thanks for the tips, we are in the process of moving one of our sites into a new platform and I will use the tips provided.

  25. To the best of my knowledge is it not proven yet that static URLs shown for a page in the SERPs (Search Engine Result pages) instead of a dynamic URL with ? and & characters will increase click-through rates, but I do not need a statistic to prove to me that static URLs look “friendlier” to non-Geeks than URLs with one or multiple weird and cryptic characters. Add to the static URL some descriptive element such as the category name or product name and I bet my retirement that the new

  26. I recently did a remap of an e-commerce website and just did a 301 redirect from the old dynamic URL to the new ‘static’ URL. I learned some things along the way, and the end result was better easier to understand URL’s. SE results seemed to have improved as well.

  27. Indian career club is a bridge to link the education seekers and education providers in India and globally, Study, study India, study abroad, studying India, studying abroad, study abroad, study in abroad, study in India, study India, studies abroad, higher study, higher study India, higher study in India, higher studies in India, higher study abroad, higher study in abroad, higher studies in abroad, abroad universities, abroad universities, higher study in abroad, higher study in India, indian higher education, higher education in India, higher education in abroad, higher education India, higher education abroad, study medicine India, study medicine India, study medicine, study engg, study engineering, study engineering India, study engineering in India, study engineering in abroad, study engineering in abroad, study medicine in abroad, study management India, study management in India, study management, study management in abroad, study management abroad, study management abroad, study online, study with scholarships, study with scholarship, study abroad programs, study abroad programs, study India programs, study programs, study India, India

  28. Indiancareerclub is a bridge to link the education seekers and education providers in India and globally, Study, study India, study abroad, studying India, studying abroad, study abroad, study in abroad, study in India, study India, studies abroad, higher study, higher study India, higher study in India, higher studies in India, higher study abroad, higher study in abroad, higher studies in abroad, abroad universities, abroad universities, higher study in abroad, higher study in India, indian higher education, higher education in India, higher education in abroad, higher education India, higher education abroad, study medicine India, study medicine India, study medicine, study engg, study engineering, study engineering India, study engineering in India, study engineering in abroad, study engineering in abroad, study medicine in abroad, study management India, study management in India, study management, study management in abroad, study management abroad, study management abroad, study online, study with scholarships, study with scholarship, study abroad programs, study abroad programs, study India programs, study programs, study India, India