« Desktop Search Goes Live! | Main | Announcing the Open Content Alliance »

September 29, 2005

Webmasters, tell us what we don�t know

Having been to a couple of Search Engine Strategies conferences, I realized how much you look to search engines for information on how your content is indexed by them. I�ve heard stories of elaborate scripts that scrape search engines, using �site:�, �link:� and �linkdomain:� queries to understand your content�s relationship to other pages on the web. Through these queries, Yahoo! provides unique information, but often there is more that you are looking for.

Today we are launching Site Explorer from Yahoo! Search, a webmaster tool we talked about at SES San Jose. WebmasterWorld and Search Engine Roundtable have been tracking it, and so have some folks on My Web. Currently, you can use Site Explorer to:

Site Explorer is geared towards your needs, providing 50 results by default, web services APIs, the ability to export the data to a TSV file for further analysis, as well as free submission for missing URLs.

Tell us what we don�t know. If you don�t find a URL that you expect to be in the index, use free submit. In case you hadn�t heard, we are also accepting lists of URLs, so you don�t have to provide us one URL at a time.

This is a starting set of features of what we hope becomes a truly valuable tool for you to interact with us. So please send us feedback, tell us how the product works for you, and let us know what else you�d like to see. Enjoy exploring, and tell us what you don�t find!

Priyank Garg
Product Manager, Yahoo! Search

Comments

This is a helpful and efficient asset for both Webmasters and SEOs.

Thank You :-)

Anticipate using it very often

Really looking forward to more innovations from Yahoo!! (That's Yahoo! - plus exclamation)

An asset to SEOs? I certainly hope not! I like to see relevant search results, not "optimized" ones.

This site is an option for SEO site research, as opposed to their using the Search.yahoo.com for their analyses


Yahoo's Search SERPs are still as relevant as before - it is just that siteexplorer ONLY accepts - "URLs" -
as queries, and adds a more related format.

The 5000 rate limiting will be a big problem for Onlinetools. So the most will probably prefer to scrape your sites in the future as they do are doing it now. Furthermore I hoped to discover with the siteexplorer more than 1000 links. At the moment I am a little disappointed.

Simple. For vbulletin or similar sites, let us submit the url like this:

http://www.domain.com/forums/showthread.php?t=1
through
http://www.domain.com/forums/showthread.php?t=1001

where you increment the numbers. And you can infer that this number grows over time so we shouldn't have to revisit all the time to submit.

Maybe give us an interface for telling you through a file on our site updated daily saying the high and low number for the various url schemes.

Thanks! That's a neat toy, very much appreciated. As requested, here is a greedy geek's wish list.

Oups, tags stripped out ... here is the wish list:
http://www.smart-it-consulting.com/article.htm?node=148&page=104

Very cool. I saw this as SES and was anxiously anticipating it. My only feedback would be allow downloading a TSV of more than 50 items. I tried to browse to the second and third SERPs hoping this would allow me to download a tsv of the next and subsequent 50 links, but each page only download the original 50.

I am ALL FOR getting info from tools from the SE's. This stuff about looking at your incoming links is cool and all, but what about learning more about who's clicking through?

With personalized search results becoming all the rave, and every person receiving a different set of search results for the same search, how will we know where we ranked? How will the API's work with that?

With personalized search, the SE's will have lots of info about the people clicking through to our site. But how will that info be passed on to us?

I'm hoping information provided to search markers about their target markets & the places they advertise (free or not (PPC or organic)) will be much more than just a site where we can look up some link & page relationships, and instead maybe advancing to more the level of information we get about the people we advertise to in traditional marketing.

IM glad your paying attention to webmasters a little more.

For those who are interested I put together a script for IPB and vBulletin forums users to create a list that can be submitted to Site Explorer. The info for this app can be found here:

http://www.bleepingcomputer.com/yahoo_site_explorer.php

The inlink API does not seem to work correctly when specifying a start param.

Try this one:
http://api.search.yahoo.com/SiteExplorerService/V1/inlinkData?appid=YahooDemo&query=http://search.yahoo.com&results=5&start=3

Notice that in the resultant firstResultPosition is 1 though it should be 3, and totalResultsReturned is 7 though it should be 5.

HELP!

This is an asset for SEO's. We must start using this to figure out all the back links and status of web pages submission at Yahoo!

Hey Priyank... neat tool, but it doesn't really change what was available. Long time webmasters already know how to check links to a page and links to a domain - and they already know how to check indexing relating to the whole site or only folders or subdomains.

I see a few advnatages - one there is no description and two there is an API - but, it is missing some features to make it truly useful (thus, beta).

Give the ability to condense IBL's from the same domain into one listing (maybe with a number beside it totalling how many are from that domain) so that we can weed scraper sites. Allow us to download all the results, or at least a bit more than the first page, especially when you can't condense URL's.

Good move though in a show of making an effort at webmaster relations - more than some of the other engines have done.

How many URLs I can add to urllist.txt? For exaple: I have a website http://www.shoppingwallet.com with more than 10 000 000 Pages and I think it's too much for urllist.txt or urllist.txt.gz In one!!!

If In the present time the internet craze has swept the nation and these days everything and everyone is online. Everyone having fun on internet and get knowledge from the web site.
you want to know how to make a free web site, the good news is that you've set an easy goal for yourself. Learning how to make a free web site is about more than just getting your web domain for free - you need to know how to put information on that site as well. Many domains offering free web sites also feature free online tutorials that will help you write your web pages. Some sites are so user-friendly; you don't have to write any of your own web code at all! You can select colors and font sizes from the domain's own page editing service. When the domain is willing to write your pages for you, it's very easy to learn how to make a free web site.
Once you have decided on the topic of your website, it is time to start writing. The text should flow naturally and be divided into easily read paragraphs. Your first objective is to make sure your visitors can easily navigate your website and quickly access your content. According to the web designer point of view If you try hard enough, you can probably come up with some funky style for a page with for example, a new wacky navigational menu. The problem is that everybody expects to see a web page layout follow certain basic rules; navigation sections have to be arranged in a certain way, links have to look a certain way ... being consistent makes the web site easier for the visitor to use. Your site should have content that changes frequently, encouraging return visitors.
And once you know how to make a free web site, you'll have your own space on the Internet that others can visit. It's free, it's easy, and it's your own piece of the World Wide Web. After all, everyone else is on the Internet - why should you be left behind?

SeanMcGill

The author is Business Head for The Web Design Company http://www.webdesigningcompany.net, and is into web-design, development and Search Engine optimization.

How many URLs I can add to urllist.txt?

Try this one:
http://www.peles-group.com

It's indeed a very helpful presentation by Sean Mcgill, describing every aspect of designing a user friendly website and what are the basic precautions must to keep in. I always go through his articles and blogs. They are certainly very helpful for beginners. May be others shall like my suggestion that its good if you should visit their site : Top Web Design Firm ,they might be a lesser known to you ,but it�s much easy to grasp a concept from their plateform.Even I am planning to redesign my flash website with attractive modifications soon.

Right, I often use 'link' to use my backlinks. The different result showing in differnt engines suprise me. So, far, I think the 'link' tool in Yahoo is the best.