Checking Page HTTP Headers

SMS Text

To ensure that search engines understand your website structure correctly you should control each page HTTP headers. Your server should handle all requests properly – by this you will both achieve better crawl rate and higher rankings.

Understanding HTTP header status codes:

I don’t want to go into too much details here and describe only what is really important for SEOs. Basically there are 5 main status code classes:

  • informational (1xx);
  • successful (2xx) request;
  • redirection (3xx) = further action needs to be taken by the user agent in order to fulfill the request;
  • client error (4xx);
  • internal server error (5xx).


Out of the above status codes SEOs most often have to deal with the following ones:

  • 200 = the request has succeeded;
  • 301 = moved permanently;
  • 404 = not found.

It is necessary to make sure that in each particular case the correct status code is returned: e.g. don’t use redirect for non-existent pages – return an optimized page with 404 header code.

Cases SEOs Should Look out for:

Most often the following cases may result in 200 status code when handled improperly:

  • index.html vs. directory root;
  • www vs non-www version;
  • double slashes in the file path;
  • mixed up letter cases;
  • non-existent pages;
  • https protocol requests; etc.

HTTP Header Checking Tools:

There are numerous web-based tools that will show you HTTP header for any URL, a few examples include:

http header tools

  1. Web-Sniffer;
  2. HTTP Header Checker;
  3. Header Checker; etc

There is also one handy FireFox extension that allows to view all http requests issued by the Web page loaded in my browser: it adds information “Headers” tab under “Page Info” of a web page.

Ann Smarty
Ann Smarty is the blogger and community manager at Internet Marketing Ninjas. Ann's expertise in blogging and tools serve as a base for her writing,... Read Full Bio
Ann Smarty
Get the latest news from Search Engine Journal!
We value your privacy! See our policy here.
  • Ann,

    Simple but effective write up! The Firefox extension comes in handy. Thanks for sharing.

  • @Ann,

    would suggest 302 to the list that SEOs must often deal it shouldn’t be there. Also how abt customized 404 page?

  • g1smd

    I think 307 and 410 also deserve a mention these days, and I’ll second the call for mentioning 302 and why it can be a bad thing to use.

  • @Raghavan, @g1smd : agreed, I also gave the link to the full list of status codes in the post and here it is also.

  • Other good tools to check headers:

    Charles (OS X): http://www.charlesproxy.com/
    does some sweet stuff, acts as a proxy between the web and client capturing http requests

    Wireshark: http://www.wireshark.org/
    goes way deeper and acts as a complete network diagnostics tool

    And of course my favorite of all, good old wget -S

  • nice fireox add-on, it is often useful to know more about http header when debugging websites

  • Good job, Ann. The information about title and keywords was most useful. I was surprised that 3- and 4 -word keyphrases were considered and didn’t like the results my page produced. Back to the drawing board!

    Thanks for your insight.

    Frederick

  • Thanks Ann, this article was very useful