Webmasters Can Now Auto-Discover With Sitemaps
- Posted April 11th, 2007 at 5:00 am by Yahoo! Search
- Categories: Site Explorer
Since working with Google and Microsoft to support a single format for submission with Sitemaps, we have continued to discuss further enhancements to make it easy for webmasters to get their content to all search engines quickly.
All search crawlers recognize robots.txt, so it seemed like a good idea to use that mechanism to allow webmasters to share their Sitemaps. You agreed and encouraged us to allow robots.txt discovery of Sitemaps on our suggestion board. We took the idea to Google and Microsoft and are happy to announce today that you can now find your sitemaps in a uniform way across all participating engines. To do this, simply add the following line to your robots.txt file:
Please provide the complete URL for your Sitemap on this line. We will pick it up wherever you put it in your robots.txt file. This directive is not specific to user-agent. If you have multiple Sitemaps, you can point to your Sitemap index file on this line. Details about the Sitemaps protocol including this addition are available on the protocol website — http://www.sitemaps.org.
If you prefer, you can continue to issue Sitemaps to Yahoo! Search by simply inputting the URL for your Sitemap and submitting. Or add feeds to a site you are already managing under ‘My Sites’ in Site Explorer. This also allows us to provide more feedback to you about what we are doing with the sitemap.
We’re also happy to have some east coasters, Ask and IBM, announce their support for Sitemaps. The more the merrier!
We’ll also be sharing more this week at SES NY.
If you have other thoughts about how we can collaborate with other search engines on standards such as robots.txt, we’d love to hear from you — visit our suggestion board.
Product Manager, Yahoo! Search
- 56 Comments