Barry points to a Webmaster World thread which asks how long does Google send their crawlers to a page which begins serving a 404 error not found message?
According to Search Engine Roundtable & the thread, the answer is forever.
Google will continue to try to see if the 404ed page has ever been reinstated as a live page.
Google even provides these errors in Google Webmaster Central, where you can see web crawl errors including 404 not found errors.
Instead of serving a 404 error message which makes it look like your site is full of broken pages and links, it’s much better to customize your 404 error page with:
- A site map, or link to the site map and link to the homepage which will direct the crawler to the main content rich sections of your site.
- If you run a blog try aggregating your RSS or archive feed to serve links to the 5-10 most recent posts.
- A search box which will help your users find what they are looking for on the site.
Comments
6 responses so far ↓
Barry Schwartz on Mar 9, 2007 at 1:01 pm
Yea, I did not advocate not to use blank 404s. I know you didn’t say I said that, but I just want to clarify.
Loren Baker, Editor on Mar 9, 2007 at 1:33 pm
Yeah, Barry, thanks for clearing that up.
Wouldn’t want users thinking you advocate blank 404’s :)
For those of your reading, Search Engine Roundtable uses a splendid 404 Page with archives, calendar, a search box and site map.
Loren Baker, Editor on Mar 9, 2007 at 1:33 pm
Which reminds me, it’s time for SEJ to implement a custom 404 Error page :p
Nico on Mar 9, 2007 at 2:17 pm
Of course he try to check 404 pages, why ? beacause there aren’t really deleted from the serveur : use 410 status code => http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.11
Carsten Cumbrowski on Mar 11, 2007 at 1:52 pm
I get 404s again and again and are not 100% where they come from. They have no referrer Information and I can’t the wrong link anywhere, not on my site and not anywhere else.
I did not only made recommendations regarding maintenance of sites and major site launches to the Google Webmaster Central team (and other SE), but also sent Google recommendations about those Error Pages stats you get at the WMC interface.
It should tell you where the heck they picked that link up and when. You should also be able to tell them that you know about that wrong link but can’t do jack about it and Google can ignore it (and maybe look at the source a bit closer if more and more Webmasters report to ignore wrong urls from the same source) .
Rhea Drysdale on Mar 12, 2007 at 3:59 pm
Great points… I might add try to do something fun or creative with the page as well. It doesn’t help with the SEs, but users appreciate warmth after getting a nasty error. I think Stuntdubl had a great post about that awhile back… http://www.stuntdubl.com/2006/05/08/custom-404s-dont-make-your-users-feel-stupid/
Don’t you just hate do as I say not as I do situations?
Leave a Comment