Search Engine Journal

In trying to help webmaster’s life easier, at least the three top search engines could agree on something. As announced in their respective webmasters’ blogs, Google, Yahoo and Microsoft are unison in adhering to the cross-submission of sitemaps to the three search engines.

With the simultaneous announcement of the three search engines, webmasters who are managing several websites, each with its own sitemap hosted on the sites URL, can now make these sitemaps hosted in just one site. Previously, the search engines requires that sitemaps have the same host and path as the URLs they contain. But with this addition to the sitemap protocol, all that webmasters need to do is to reference the sitemaps URLs and path in the robot.txt where the URLs it contain are hosted.Yahoo Search blog explains this by an example:

For example, say you have a Sitemap (sitemap-www.xml) for the URLs on http://www.example.com but you want to put that Sitemap on http://sitemaps.example.com. That is now possible. To make the Sitemap valid and preserve data security you need to refer to it from the robots.txt file on the site where the URLs it contains are located. For example, add the following line to http://www.example.com/robots.txt:

Sitemap: http://sitemaps.example.com/sitemap-www.xml

The Google Webmaster blog explains this as:

Say for example you want to submit a Sitemap for each of the two hosts you own, www.example.com and host2.google.com. For simplicity’s sake, you may want to host the Sitemaps on one of the hosts, www.example.com. For example, if you have a Content Management System (CMS), it might be easier for you to change your robots.txt files than to change content in a directory.

You can now exercise the cross-submission support via robots.txt (by letting us know the location of the Sitemaps):

a) The robots.txt for www.example.com would include: Sitemap: http://www.example.com/sitemap-www-example.xml

b) And similarly, the robots.txt for host2.google.com would include: Sitemap: http://www.example.com/sitemap-host2-google.xml

and the Microsoft Live Search Webmaster Center blog explains:

Say you run a web site like MSN.com, which has a bunch of sub domains like health.msn.com, travel.msn.com and moneycentral.msn.com. And, due to a technical requirement, you would like to host all of your sitemaps in one location like sitemaps.msn.com. Until now the protocol did not support this scenario, each sitemap would have needed to be hosted directly under the domain it described. This update now introduces support for this scenario, with the requirement that you simply include a reference to the sitemap in your Robots.txt file. For example, moneycentral.msn.com/robots.txt would need to include this line:

Sitemap: http://sitemaps.msn.com/index_moneycentral.msn.com.xml

This is definitely a welcome development and a very useful one at that. Creating and managing sitemaps for individual sites is definitely a time killer for webmasters managing multiple sites.Imagine how tedious it is to create sitemaps for 10 sites, each requiring that you access your control panel only to update a small file containing some statements. Hats off to the three search engines for adapting this measure.