Be Dynamic, Be Confident — Yahoo! Search Supports You
- Posted August 21st, 2007 at 7:05 am by Yahoo! Search
- Categories: Site Explorer
Please excuse the dramatic start to this post. Between the anticipation of rolling this out and my incessant Harry Potter reading, I couldn’t resist.
Once upon a time, on the World Wide Web, all URLs were fixed strings — static in form. The idea of URL parameters then came along, allowing for database driven sites and session ids in URLs to create personalized experiences for users. At that time, the Web was alive with rich data and experiences. Then came the crawlers, which made it easier for users to navigate through the Web; however, they inevitably battled with dynamic URL parameters and every webmaster had to choose between a dynamic site and search traffic.
Today comes a new wave for search engines with the first-ever Beta launch of ‘Dynamic URL Rewriting’ in Site Explorer. The new feature provides the ability for site owners to alert Yahoo! of the dynamic parameters in URLs that they’d like Yahoo! to ignore, which we’ll then automatically rewrite accordingly. Try this out for all the cases where you’d want to use parameters in your URLs that don’t affect the content of your page, but that have other important uses.
How to get there?
- Login to Site Explorer from Yahoo! Search.
- Add to My Sites and then authenticate any sites that you own or manage.
- For any sites that you have authenticated, you’ll see a ‘Dynamic URLs’ tab.
- On this tab you can enter parameters you want us to either remove from URLs or always crawl with a specific value.
- Once you enter the parameter, we’ll show you the # of URLs we estimate will be affected.
- After you confirm the action, we’ll modify our crawler such that every time we see a URL from your site with that parameter, we’ll automatically rewrite it within our system as per your instruction.
So you might wonder what the feature really gives you. Utilizing the ‘Dynamic URL Rewriting’ feature enables:
- A more efficient crawl of your site, with fewer duplicate URLs being crawled.
- Better and deeper site coverage, as we’ll be able to use our crawler capacity to find and index more new content on your site.
- More unique content discovered, as we’ll handle more dynamic parameters in your URLs (if you remove the content-neutral dynamic parameters).
- Fewer chances of crawler traps, or web page sets that can cause an infinite number of requests or a poorly constructed crawler to crash.
- Cleaner and easier-to-read URLs displayed in the search results.
- Better site ranking due to reduced fragmentation of links and anchor text to your site’s pages.
Looking for more details on when to use URL parameters? Visit the Site Explorer Help page for additional background on the Beta feature to help define and omit what dynamic URLs Yahoo! should ignore.
We’re here to address any questions/ needs that you have, so let us know how it works for you.
for Lakis, Amit B., Amit S., Jay, Judy, Srikanth, Zheng
- 46 Comments