Site Explorer Update
- Posted August 8th, 2006 at 3:00 pm by Yahoo! Search
- Categories: Site Explorer
We opened a little window into Yahoo! Search last year, when we
Explorer. We hoped it would be useful to webmasters–providing you
with information about the links to and from your site, neatly
categorized and displayed in an easy-to-use interface. We’ve listened
to your feedback, and are now ready with the next version of Site
Explorer–our biggest update since
December.
We’re now organized around sites you’d like to track. You can explore these,
and add feeds to each site. Once you authenticate your site, you can see much
more information about your URLs as you explore your site, and monitor feeds
you’ve submitted.
So what’s new?
- More
information about sites you own, including: - Last Crawled Date and Language for your Site URLs
- Subdomains of your site
- Feed
submissions are much smoother. You can submit RSS, Atom and URL lists,
and manage all of them from one place. For authenticated sites, you can
also track when they were submitted and processed.
Web Service to notify us of feed or site updates, part of the suite of
Site Explorer
APIs you already know and love. Since these return the same data as
the tool, we recommend using them for automated applications.
We hope you’ll like our new interface, with a lot of little
details sprinkled all over, such as the expandable results to reduce clutter, the
ability to download more URLs from sites you own, and robust authentication.
Share your comments through our
form or see what others are saying on the new Site Explorer
We welcome you through the doors, and hope you’ll forgive our tacky metaphors! :-)
Amit Kumar, Priyank Garg
and the entire Yahoo! Site Explorer Team
- 25 Comments
- Subscribe
Will we ever be able to authenticate a Yahoo! store?
Can we use sitemap.xml for a feed?
Nice work guys! I really like the interface. This will certainly come in handy. Hopefully more tools to come!?
This is fantastic, i’ve signed up a site, looking forward to seeing what it can do
Just wondering. Submiting a txt file with a url list it sounds as if we have to re-submit whenever there are changes. Or will this be checked up on every now and then? I would use a php to construct the txt file doing so I can call on the api to notify of changes to that file:
http://api.search.yahoo.com/SiteExplorerService/V1/updateNotification?appid=YahooDemo&url=http://www.yahoo.com
Of course with the proper paramters.
Is this a suggested way to go and would it work?
This way I can use a cronjob to check and update the file automatically when needed while automatically notifying Yahoo! of updates to the file (such as new urls).
Also is there a size limit or URL limit to the txt file?
Arubicus -
Submitting a urllist file, and periodically using the UpdateNotification call to let us know of updates to that file, is certainly a recommended way to go!
Amit
Site Explorer Team
It would be nice to include some ajax too on the interface
Just to clarify on what Amit said…
Any URLs or feeds we have in the crawler, we will refresh periodically based on our algorithms, but you are welcome to notify us of updates using the updateNotification API and we will use that as additional input to direct our crawl.
Thanks for the quick replies no more questions yet. For me everything went real quick and smooth to verify as well as get the urllist.txt submitted and processed.
Hey got another question. Recently we have seen many pages fall out of the search index. Never has our site experienced this. Also slurp seems stuck on a few select pages hitting them over and over again as well as the robots.txt file. I noticed that using the site explorer it shows that we have a sub-domain. We don’t use any sub-domains. When I click on to view the sub-domain it shows our home page. Is there a problem here?
This may be asking a little too much, however, it would be nice to have an ‘add this page’ button to Yahoo Toolbar if it’s not in the index or a checkmark stating it is already in it.
Can you add an alternate mechanism for authentication like “inserting a meta tag”..
I am suggesting this because bloggers on Blogspot or MSN Spaces or yahoo 360 who have no access to their servers may not be able to upload the file and authenticate the site.
Hi, I just saw this latest release, regsitered, activated Site Explorer and tried feeding the page /gallery_2nd_XI.html (located in my root folder) but it wasn’t accepted – any ideas why ? Have I formatted something incorrectly ? Thanks, AL. (novice webmaster :-))
Why does site authentication take so long? Shouldn’t it be instantanious?
I registered a website (waiting for authentication). I have an original RSS-Feed within a subdirectory of my website that validates with feedvalidator.org. Site Explorer did not accept the feed although it is wellformed, reachable and not already added.
So I tried to add my feed I have at feedburner. But Site Explorer seems to require the feed to be on the same domain. What is reasonable because it is difficult to authenticate a feed on another domain.
Is there any other way to register my feed?
Maybe Site Explorer could look for the appropriate tag in the source code of the index page:
Then Site Explorer could list all entries (there might be two or even more) after authentication and I could select one in this list as my favorite feed?
I’m confused on URLLIST.txt. If I create a list of my RSS feeds and name it URLLIST.txt, then what?
Do I put that file at http://www.mydomain.com/urllist.txt and submit that as a “feed”.
If not, what do I do with it?
EVO
Are there any advantanges to using RSS or atom over urllist.txt?
Atom has a last modified element. Would Yahoo use this or any other element in RSS or atom?
Point being that urllist is easy to generate but if rss or atom gives Yahoo something, I will be please to create it.
I like the new feature. It would be great if you could allow us to strip out other domains when we are examining back links. The sites I personally deal with have millions of links, and it would be so much easier to not have to see 100s from the same domain if I wanted.
RSS submission still with problems.
When you select an RSS feed, then delete it and THEN try to add a NEW RSS feed, you will get an error message…
Further to my “August 10, 2006 05:37 AM” querie above … It seems that feeds which use capitals are rejected !? Might be worth looking into. With help I have overcome this problem (others may not be able to though) and am now happily set up – many thanks for the facility ! :-)
I have a Yahoo Store MS account using the “store” verson as opposed to using the “hosting” version (where I create my own site and put in the root directory).
How can I authenticate my site?
Yahoo site maps show my site as having a subdomain where none exsits. This appears to be resulting in some type of penalty – how can this be fixed.
Why does the yahoo site explorer does not help to better index our site like the google sitemap when we submit our feed. Yahoo crawlers have been unable to crawl our complete site. Most of the important pages are always missing from yahoo index.
Your listing includes our test subdomain, which is hosted on Yahoo Small Business.
How do we exclude this from any listings? IT’s not supposed to be public.
According to your listing there are no links in to our subdomain. I do not know of any, either.
So did you taken this from your server data? How are we supposed to keep our testing pages private if you make them public?
pls upload the domain
In the help section you state that after authenticating the site, webmasters can “See when the data was last refreshed by our crawlers.”
Where can I view this information other than the cached page which is available even before authenticating the site?