We’ve just announced an all-new Yahoo! Search experience, with many new features powered by SearchMonkey data.  Since launching our open developer platform in May 2008, Yahoo! Search has enabled thousands of developers to shape the search experience for millions of Yahoo! users. If you are interested in building semantic applications similar to what we’ve come up with at Yahoo! Search, here are some details to get you started.

What structured objects are available?

All of the objects listed on the SearchMonkey homepage are available to you. With the new feature “object refiners,” users can now restrict the search results to specific object types. Site owners contribute data of these objects by marking up their pages with RDF or microformats, or by providing dataRSS feeds. If you’re interested in the actual data of these objects, use the Yahoo! Search BOSS API to request the SearchMonkey data as part of the search request.

How can I access these structured objects?

The SearchMonkey team has been encouraging developers to use our structured data to build semantic Web applications ever since we partnered with BOSS.  Using the BOSS API, you can access SearchMonkey structured objects.

To restrict the result set to pages with SearchMonkey objects, just add “searchmonkey:<objectType>” to your query. The result set from BOSS will only contain URLs that have objects of that type.

For example, the following query returns all of the documents in the Yahoo! Web index that has the words “Sunnyvale” and “pizza” – about 3 million pages.

http://boss.yahooapis.com/ysearch/web/v1/sunnyvale+pizza?appid=wX7OZ3zV34Fy2Y4W4in_vsjFmRhruQNgCxdxn6RUke2c2JVDZdw6bfc1rcEjVnw-&format=xml

But if you only want pages with local business objects on them, you can add “searchmonkey:local” to the query:

http://boss.yahooapis.com/ysearch/web/v1/sunnyvale+pizza+searchmonkey:local?appid=wX7OZ3zV34Fy2Y4W4in_vsjFmRhruQNgCxdxn6RUke2c2JVDZdw6bfc1rcEjVnw-&format=xml

This query returns about 25,000 pages.

Yes, we’ve just thrown out over 90 percent of the result set – but we are after the most relevant results, not simply the greatest number of results. Our new object refiners use SearchMonkey’s structured data to narrow your query from “pizza+Sunnyvale” to actual local business listings within those results. You can use BOSS to retrieve the same structured data and construct any presentation you like.

You can take it a step further and add any of these terms to the query:

• searchmonkey:video – restricts the result set to videos.
• searchmonkey:product – restricts the result set to products.
• searchmonkey:local – restricts the result set to local businesses.
• searchmonkey:event – restricts the result set to events.
• searchmonkey:document – restricts the result set to presentations, spreadsheets, and similar document formats.
• searchmonkey:discussion – restricts the result set to blogs and forums.
• searchmonkey:game – restricts the result set to Flash games.

What don’t I get?

Not all structured data we’ve collected is part of the BOSS API.  For example, some third parties who provide us with feeds have elected to keep that data outside of BOSS. Structured data annotations from technologies built by Yahoo! Research are also not available to third party developers via BOSS. However, we aim to include all data we find embedded in web pages that deploy microformats or RDFa.

Our goal is a successful semantic Web where we extract the semantics as we process Web content. Every page marked up with semantic data makes that much easier for us to extract meaning from that page. And it’s not just us! Google Video Search has recently adopted the same video markup (RDFa and Facebook Share) that SearchMonkey supports.

What’s next?

We will make many more object types available to you soon. In the mean time, you can learn more about SearchMonkey and how we acquire structured data annotations from this new from this post on the YDN Blog.

Kevin Haas

Senior engineering manager, Yahoo! SearchMonkey

First, we’d like to thank everyone who deployed microformats, RDFa, and feeds in response to our blog post in May. Thanks to your efforts, we’ve finished user testing for the new enhanced results templates and have deployed these templates in production. This means that in addition to Video, Documents, and Games, you can now add Products, Local Businesses, Event, Discussions, or News items to your pages. Anyone who provides structured data according to the specified format will automatically gain SearchMonkey default-on status, as long as it adheres to our terms of use.

For example, here are some results you’ll see when you search for products:

Here are some results you’ll see when you search for local businesses:

You can see the ratings, number of reviews, phone number, and address for local businesses right on the search results page. You can also see the ratings, price, and number of reviews for products, helping you decide which page fits your interest most, and find what you’re looking for faster.

In addition to these SearchMonkey templates, we’re also releasing a number of custom SearchMonkey default-on applications. Entertainment buffs will be particularly excited about a few of the Entertainment apps, including RottenTomatoes, Netflix, IMDB, and Yahoo! Movies.

The following are just a couple of examples you’ll see when you search for entertainment-related info:

Beyond entertainment, we’re also automatically turning on results for high performing sites in as the social networking, reference, and download categories. The list includes Friendster, Britannica, and FileHippo.

We laud the efforts that developers everywhere have put into developing the SearchMonkey ecosystem and their contribution toward improving the search experience for users.

Great apps are built every week so look forward to even more SearchMonkey in your search results in the future – we’re looking forward to it, too.

Yi-An Lin and Nick Cox

Senior Product Manager

Yahoo! Search

New Enhanced Results – Products, Events, News and More

Back in March 2009, we announced a simple way for site owners to embed video, games, and documents in Yahoo! Search results. Starting today, we are expanding this capability by giving site owners the power to display enhanced results for product pages, local information, events, news, and discussions.

If your site’s data falls into one of these categories, add a few lines of markup to your pages, and SearchMonkey will do the rest of the work. After we recrawl your page, we’ll extract the structured data and use it to display your data as an enhanced result.

For example, a retail website could add a few lines of code so that its product pages display as an enhanced result that includes the overall rating, price, reviews, and product photo directly on the search results page. Let’s say we have a fictional store called Sytore.com and the site owners have added the following code to their product pages:

<div typeof="product:Product"
xmlns:product="http://search.yahoo.com/searchmonkey/product/"
xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#">

<span property="product:listPrice">49.99</span>
<span property="product:salePrice">39.99</span>
<span property="product:currency" content="USD" />

<span property="rdfs:label">Pinball Maven : Video Games : Electronics</span>

<span rel="rdfs:seeAlso media:image">
<img src="http://www.sytore.com/product.jpg"/>
</span>

<div rel="review:hasReview">
<span typeof="review:Review">
<span property="review:rating">4</span>
<span property="review:totalRatings">17</span>
</span>
</div>
</div>

Sytore.com’s product pages (such as its product page for “Pinball Maven”) would then display as an enhanced result:

(*Example only)

Enhanced results bring users the information they need while helping site owners stand out on the search results page. You can add code to display local information as enhanced results with phone numbers and addresses. You can also display location and date for festivals, concerts, and other events.

A news website can take advantage of the SearchMonkey news object type and add code to enhance how their pages display in search. For example, a news website such as the (fictional) Thenewsy.com could add a few lines of code to its news article pages to display a photo and publication date. A query on “Obama Iraq” could display an enhanced result from Thenewsy.com:

(*Example only)

If your site contains a forum, blog, or other types of online discussion, you can add some markup to display the number of comments and thread date.  You can learn about how to get started with each of these object types on our overview page.

Enhanced results for these new data types will appear in Yahoo! Search results a few weeks after you add the markup, and after we’ve crawled your pages to extract the necessary structured data. There is no sign-up process, so we encourage you to begin adding markup to your sites now so that your results can be visible to users.

From the beginning, SearchMonkey has been powered by open formats, which is why we are continuing to support the use of RDFa, microformats, and now NewsML for these additional object types.  With the help of site owners and developers, we are moving more rapidly towards structuring the Web and enabling new search experiences.  As we mentioned a few weeks ago, RDFa structured data collected by SearchMonkey has increased by 413% since October 2008. With the release of these new object types, we look forward to seeing that figure continue to climb.

Google Base

Since its launch in late 2005, there has been a growing community of tools and partners for Google Base, Google’s online repository for user-contributed structured data. Today, Yahoo! Search will accept five popular Google Base feed item types: Event, Product, Review, Job, and Personals.

Why is this important?  First, site owners who have Google Base feeds containing Event and Product information can now automatically have their enhanced results displayed in Yahoo! Search by submitting their existing feed through Yahoo! Site Explorer.

In addition, for all five item types, it’s now easier to use your Google Base feed within Yahoo! Search.  Site Explorer will convert your existing feed to DataRSS XML, allowing your data to be stored within Yahoo! and accessible to developers through BOSS for building third party search engines and the SearchMonkey Developer Tool for building applications.

For detailed instructions, refer to the full documentation within the Yahoo! Developer Network site.  For information about how to build your Google Base feed, refer to the Google Base feed documentation.

Please let us know if you have any questions or comments.  We welcome your feedback.

Yahoo! Search

Along the way, we’ve made great headway in opening up Yahoo! Search by accelerating the adoption of structured data across the Web and empowering developers to innovate in search. Here’s a look at some of the important initiatives we’ve accomplished with SearchMonkey and BOSS.

With BOSS, we’ve made key updates including allowing developers to monetize through third-party platforms and enabling access to SearchMonkey structured data. The BOSS API is on track to hit 1 billion monthly queries in May, without even including the volume from traditional Yahoo Search syndication. This is more than three times the queries served just six months ago, and ranks ahead of the combined searches on Ask and Facebook, and just behind Microsoft¹.

With SearchMonkey, we’ve launched numerous valuable initiatives over the last year. We turned applications default-on for many sites to improve local , reference, and social searches. We also enabled publishers to more easily display enhanced results for video, games, and documents by adding a few lines of code. Additionally, as a part of our effort to promote a more meaningful understanding of the Web, we have continued to support semantic tech gatherings such as VoCamp.

SearchMonkey is currently live in 23 markets around the world, reaching some significant global milestones:
• 70 million enhanced SearchMonkey results are viewed by users every day.
• Site owners have seen a more than 15% increase in the click-through rates of their SearchMonkey search results when tested against non-SearchMonkey results.
• 200 people enter the developer tool and start creating an application each day.
• Over 15,000 developers have registered to build applications, with over 400 applications available for use in galleries.
• RDFa structured data driven by SearchMonkey has increased by 413% since October, 2008.

In the coming months with SearchMonkey, we will be driving efforts toward increasing structured data on the Web, more uses for existing structured data, and easier ways to display enhanced results for some data types. We’ll throw in a little fun, too, with some open customization of the Yahoo! Search results page. With BOSS, you can look forward to more specialized searches that will help consumers reach their content more easily than ever.

We want to extend our thanks to the developer communities within both SearchMonkey and BOSS and to partners including Facebook, Yelp, and LinkedIn, for joining us in our efforts to make search richer and more open, and for helping us reach these milestones. We’re pleased with what we’ve done in this past year. More importantly, we’re excited about and focused on where we’re taking you next.

Larry Cornett
Vice President, Consumer Products, Yahoo! Search

¹comScore qSearch April 2009. Queries delivered via the BOSS API and served by Yahoo! partners are not counted as Yahoo! Search queries by comScore or other metrics providers.

The premise is simple: if you have a flash video, game, or document (like a presentation or spreadsheet) embedded on a page, adding a few lines of code will make it appear as an enhanced result after we recrawl your page. Very little semantic markup knowledge is required, and you don’t have to build your own application. SearchMonkey does the heavy lifting, taking your markup and extracting the necessary structured data to display it as an enhanced result.

For example, check out the code that Hulu added to create an enhanced result for a clip from The Simpsons. Just the first two lines of code are required to generate an enhanced result. The other four lines are optional and assist with the display.

<link rel="image_src" href="http://thumbnails.hulu.com/9/967/32912_145×80_generated__VfWxSNY3rESG1ntOzexNRQ.jpg" />
<link rel="video_src" href="http://www.hulu.com/embed/GREW9Qw0P7KjIyjJydQYRw" />
<meta name="video_height" content="296"/>
<meta name="video_width" content="512"/>
<meta name="description" content="Video description: Homer gets upset at a vending machine filled with apples."/>
<meta name="video_type" content="application/x-shockwave-flash"/>

By adding this markup, Hulu enabled enhanced results for their page in the Yahoo! search results for the query “The Simpsons Apples.”

When users click on the thumbnail image, an expanded video drops down so they can watch the video clip directly on the search results page.

As Yahoo! Search continues to promote the use of open and de facto formats, we are supporting the use of RDFa and Facebook Share markup (an existing format currently deployed on many sites) to power such enhanced results. Code for site owners is currently available for flash video, games, and documents. We encourage you to add markup to your sites now so our crawlers can start indexing them right away. Users can already see the enhanced result for video on their search results, and we’ll begin to support documents and games in the next month or so.

In order to provide the best search experience for Yahoo! users, we’ll be reviewing sites that want to display rich media content before they can appear in search results. We’re currently supporting a number of popular video players including Hulu, YouTube, and MetaCafe. Learn how to participate and submit other players for consideration from our overview page.

We think enabling users to preview rich content right on the search results page will help them find what they’re looking for faster than ever. By allowing participation using simple markup, site owners can make their content easier to find while taking another step toward a more structured Web.

We welcome feedback as we continue to roll out improvements and new features for SearchMonkey. Please let us know what you think in the comments section below or on the SearchMonkey forum.

Yahoo! SearchMonkey Team

P.S. We want to wish a Happy Birthday to Hulu. Congratulations!

Here’s an example of the Facebook enhanced result with Alex Moskalyuk, a key Facebook engineer on this project.

See the SearchMonkey app for Facebook in action yourself and try a search for your friends. Here at the Yahoo! Search Blog, we had fun checking out the Facebook profiles of marketing VP Raj Gossain and senior product marketing manager Graham Mudd.

We care about privacy as much as you do, so you’ll only see results for Facebook users who have enabled their profiles to be publicly searched and viewed. If you’re interested in “social-izing” your search results page further, check out other SearchMonkey apps for social networking sites such as StumbleUpon, Delicious, and MyBlogLog.

We hope the SearchMonkey app for Facebook and our other social apps make finding and connecting with friends on the Web easier than ever. Let us know what you think.

SearchMonkey Team

This new Wikipedia app includes a photo (if there’s one in the corresponding article), deep links to the first four sections of the article, and a snippet pulled from the article summary. For example, say you’re a mid-century furniture fan and you’re looking to learn more about Eames chairs. A search for Eames chairs will bring up two Wikipedia results for two of their famous chairs. You can use the photo to help you decide which result is best for your needs (this is called result disambiguation by us search geeks) and then click a deep link to jump directly to your answer.

While Wikipedia may be the most well-known reference site on the web, it’s certainly not the only one. In fact, there are SearchMonkey apps for many other top sites, including HowStuffWorks, eHow, CIA World Factbook, Merriam-Webster, and a lot more.

We hope these apps make searching for reference information on the web easier and faster. Let us know if you have any feedback.

SearchMonkey Team

p.s. Today appears to be the 8th anniversary of the launch of Wikipedia. Congrats to Wikipedia and its many contributors!

The Yahoo! SearchMonkey team would like to congratulate our colleagues at Intel MashMaker, who have just received Argentina’s Sodosky Award for Technology Innovation. As we’ve reported before, MashMaker is not only a powerful tool that enables non-programmers to extract semantic data from websites to create mashups, but you can also use MashMaker to rapidly create extractors for SearchMonkey applications. If you haven’t tried using MashMaker and SearchMonkey together yet, you can download MashMaker, click on the little monkey icon and give it a try.

In related award news, we are pleased to announce that ReadWriteWeb has recognized SearchMonkey as one of its Top 10 Semantic Web Products of 2008 for “the best use of Semantic Web by an Internet bigco this year.” We’d like to thank ReadWriteWeb for the recognition, and congratulate all of our co-winners: Powerset, Calais, Dapper, Hakia (powered by Yahoo! Search BOSS), TripIt, BooRah, AdaptiveBlue, Zemanta and UpTake. We know we’re in good company here and we look forward to making greater progress in 2009.

Evan Goer
Yahoo! SearchMonkey Team

When developers submit their app to the gallery in the SearchMonkey Developer Tool, it will be reviewed by our team. The parameters for inclusion in the gallery are relatively straightforward, but often minor details such as a missing logo or a non-conditional display can keep innovative apps from reaching the gallery. Starting today, even if an app doesn’t meet all of the gallery guidelines, it will still be quickly approved as an experimental app provided the basic functionality operates as designed and we don’t initially see any other major issues with it.

To view experimental applications, simply click on the ‘Show Experimental Enhancements’ link at the bottom of the application directory pages in the gallery.

Once you’ve opted in, you can access all the experimental SearchMonkey applications and provide direct feedback as well as feature requests to the developers. You can also get automatic upgrades to the latest versions when changes are made to the applications.

Let us know what interesting applications you’ve discovered!

Nick Cox
Yahoo! Search

For example, Citysearch’s app provides ratings, hours of operation, addresses and phone numbers for millions of restaurants directly on the search results page. It also includes deep links to each restaurant’s profile, user-generated reviews and a map, so users can get directly to the answer they’re seeking. To see the app in action, try a search for slanted door san francisco or cones new york.

We’ve also seen the number and quality of applications in the Yahoo! Search Gallery continue to climb and improve. Here are a couple recent additions we particularly like:

Take a look at the recently developed Infobar for TV.com. The app displays the next air date of a television show, user rating, summary of the show, and links to videos, photos and reviews, as well as information about the cast. Try a search for Dancing with the Stars.

Chris Lindsey recently built a great Infobar for Food Network results. It displays some essential info about recipes, like cook time and level of difficulty, and includes a deep link taking users directly to a printer-friendly page. Nice work, Chris!

We’re pleased with all of the new apps that have been developed and we’re looking forward to what new ideas and innovations the development community will come up with. If you have any ideas for apps that would improve the search experience or you’ve built something you want to let us know about, add a comment below.

The SearchMonkey Team