« April 2005 | Main | June 2005 »

May 27, 2005

Yahoo! Research Labs Releases Yahoo! Mindset

Sometimes at Yahoo! Research Labs, an idea is just too exciting to keep to ourselves. Mindset is just such an idea. Today we've posted it on Yahoo! Next for you to try out and let us know what you think.

What is Mindset? A new twist on search that uses machine learning technology to give you a choice: View Yahoo! Search results sorted according to whether they are more commercial or more informational (i.e., from academic, non-commercial, or research-oriented sources).

Sometimes you want to buy stuff and sometimes you just want to do research. In a typical search page, results point to commercial pages that are mixed together with non-commercial pages, so it's harder to find the type of information you're looking for. Mindset is our attempt to help solve that problem.

Through the use of machine learning for text classification, we try to classify each web page in the top 100 search results for a query. Then we sort those results according to the preference you set. Mindset uses an intuitive slider in the interface, so you can set the bias for commercial vs. non-commercial results. Often, we come across a web page that hasn't been classified yet. In those cases, Mindset tries to classify that web page in the background, so it'll be classified along with the rest of the results next time you do the same query.

A full description of Mindset, what it is, and how it works, can be found on the Yahoo! Research Labs Mindset pages.

When using Mindset, please bear in mind it's not a finished product. We think of it more as a technology demo and an ongoing research project. We want to determine whether: a) the underlying technology is effective and b) you like it. That's why we're very keen to know what you think and invite you to discuss it on our Mindset forums.

Try Mindset and then share a piece of your mind with us.

Bernard Mangold
Sr. Director, Yahoo! Research Labs


[ Yahoo! ] options

May 20, 2005

We're on the hunt for great uses of Y!Q...

Every few days I come across some cool new site, application, or hack made possible by Yahoo's RSS feeds and Web Service APIs. My current favorite is Yahoo! News Tag Soup which Havi recently wrote about. It's more evidence of the kind of innovation and creativity that emerges when you open things up a bit and let the world remix your data and services.

And the most amazing part is that the majority of hacks are built by people doing it just because they can. They're not selling anything--just hoping for a bit of recognition or trying to impress friends. (Oh, and a few might get job offers!)

Well, that gave us an idea... What if there was a concrete incentive to play with some of the stuff we've been building? Like maybe a cold, hard $5,000. That'd buy you a nice new Powerbook. Or a dozen iPods. Or a year's worth of unlimited music for 100 of your closest friends.

That's what the Y!Q Challenge is all about. Your job is to impress us with your use of Y!Q on your web site. Our job is to be impressed and fork over the cash.

What's Y!Q? It's an on-page contextual search DHTML widget that I wrote about at the time of the beta launch. In essence, it lets people search without leaving your web site.

The Y!Q Challenge FAQ and the legal blah blah are on the site, but all you really need to get started is your own web site and the ability to dig in to the HTML a bit. You do some pretty interesting things by hacking the CSS and using some of the advanced features list in the documentation.

Go ahead. Impress us.

Jeremy Zawodny
Yahoo! Search


[ Yahoo! ] options

May 17, 2005

Marc Canter on Ourmedia's Support of Media RSS

From time to time we've invited guest bloggers to write on the Yahoo! Search blog. Today we have a post from Marc Canter, one of the main forces behind Ourmedia.org.


Since its launch in late March, Ourmedia.org has quickly gained a reputation as the place to upload and store works of personal media for a global audience. We're a nonprofit out to make it easy for the masses to publish their videos, audio files, podcasts, photos - any digital media.

And today we're announcing our support for Media RSS output.

This not only gets us into sync with state-of-the-art media publishing, but it also allows us to get all of the Ourmedia.org content indexed into the Yahoo media search engine - so that LOTS of people can find our material, whether it's under traditional copyright or a Creative Commons license.

As the open media world grows, having support from folks like Yahoo really helps. Yahoo brings cred, resources and a professional approach to a world rampant with hobbyists, amateurs and lovers of media - but devoid of code repositories, test rigs and good QA control.

We've been working hard to bring the vision or free storage and bandwidth to digital creators and having Media RSS support now enables us to get that stuff to a wider world out there.

Marc Canter


[ Yahoo! ] options

Announcing Media RSS 1.0

Since we first announced Media RSS back in December, we've been hard at work with the rss-media community (now over 250 members strong!) to hammer out a release version of the standard. Thanks to a lot of overall participation, we now have Media RSS 1.0 completed and published. A very sincere thanks to everyone in the rss-media group who made this milestone possible!

Media RSS support outside of Yahoo! has also begun to take root, with support for Media RSS already present in online RSS tools like FeedBurner's SmartCast feature, Blogdigger, and the Windows version of the FireANT video aggregator, in addition to content sites publishing Media RSS feeds such as Fotothing and blogs like solitude.dk. Of course, we're already supporting Media RSS in our Web Services XML API for Video Search.

As I mentioned in my last blog post on Video Search, one of the big drivers for online video hosting are sites like Ourmedia. I'm glad to have a chance today to introduce Marc Canter from Ourmedia to the Yahoo! Search Blog, and to talk more about about the future of online video.

I'll let Marc take it from here. I'm looking forward to see what applications people come up with for Media RSS!

Andy Volk
Product Manager, Yahoo! Video Search


[ Yahoo! ] options

May 16, 2005

Tag Soup du Jour From YSDN

Ever since the Yahoo! Search Developer Network (YSDN) launched a few months ago, we've been keeping an eye out for trinkets, toys, and hacks that folks are building with our publicly accessible application programming interfaces (APIs). Open APIs make it possible for developers to create new applications based on Yahoo! Search content (types of data) and services (ways of sharing data).

Developers are busy writing plug-ins, smart utilities, Flash-based widgets, and a host of cool apps in their favorite web programming languages, using Yahoo!'s open web services, which include several flavors of web search as well as local search, image search, news search, and more. You'll find many of these interesting applications listed on the YSDN Wiki.

John Herren's Yahoo! News Tag Soup was definitely this past week's hot app on campus. Tag Soup is inspired by the growing popularity of tags for describing information. Herren's application riffs on rising interest in folksonomy or social tagging: a collaborative, non-hierarchical way to organize and display information by assigning freely chosen keywords to web sites, photos and digital images, blog posts, URLs, you name it. A self-described PHP fan, Herren spent a few hours over a recent weekend coding Tag Soup because he "thought it would be fun to see what happens when you automate the [tagging] process."

One notable by-product of tagging is a new mode of displaying weighted lists of keywords. This visual representation, referred to as a tag map or tag cloud is turning up in many unexpected places. A tag cloud shows importance or frequency of word occurrences by font size and/or bolding. Usually tags are displayed in alphabetical order, sprawling fluidly across a squarish chunk of the page. Perhaps you've seen this horizontal list metaphor on Flickr, Technorati, or 43 Things.

For his Tag Soup recipe, Herren uses Yahoo!'s content analysis web service to extract significant words and phrases. (This type of term extraction also powers Y!Q, our contextual "embedded" search. Y!Q analyzes the content of the web page you're on or the text you select to provide results "at the point of inspiration.")

Next, Herren grabs a collection of Yahoo! News RSS feeds, massages them into a database to eliminate duplicate stories, and extracts the key ingredient (his tags) from the article headlines and summaries. Finally, he uses a simple scaling function to display the most popular and frequently occurring terms. Last time we looked, President Bush was far and away the biggest and boldest tag, followed by: United States, In Iraq, United Nations, and Microsoft. Go figure.

When tagging is practiced by a community of users in a social context--not just as a personal system for labeling information--network effects begin to take place. This is part of the enchantment of Flickr, where inventive tags like squaredcircle, longline, and lenstagged acquire a life and momentum of their own. Tags become a medium of ideas, connecting people, starting conversations, and transmitting ideas. This is already happening with Herren's Tag Soup.

While writing this post, we spotted Justin Flavin's tag hacks, inspired by Herren's work and the launch of BBC Backstage. (The Backstage motto is: Use our stuff to build your stuff.) Flavin, an Irish developer based in the UK, is fetching business and tech news RSS feeds from multiple news sources, and running them through Yahoo!'s content analysis (term extraction) service to generate his keywords. He's working on a similar hack for entertainment news.

John Herren describes his project as "a proof of concept to show the kind of cool stuff that can happen when folks, in this case Yahoo! opens up content and services for a developer to use." Justin Flavin's pages prove that cool stuff is catchy.

Meantime, we're keeping the Yahoo! News Tag Soup concept on a low simmer and giving it an occasional stir, because we believe its flavors can only improve over time and nourish other promising innovations. What do you think?

Havi Hoffman
Yahoo! Editorial


[ Yahoo! ] options

May 12, 2005

Yahoo! Shortcuts: Find It Fast

One of the items I work on is Yahoo! Shortcuts. A Yahoo! Shortcut is a quick way to use web search to get to the information you want, ideally faster than combing through the web results. Shortcuts results appear automatically when they're relevant to your search and link to content on Yahoo! or across the web. That's a bit of an awkward description, so let's get a better definition and see a shortcut in action.

A web query for "define shortcut" gives The American Heritage Dictionary's definition of "shortcut" as "A more direct route than the customary one" in the Yahoo! Shortcut section (look for the red "Y!" next to it near the top of the web search results page. The pattern of "define x" can be used to get definitions, as is pretty obvious, aka apparent, clear, evident - you can get a whole list of synonyms by using "synonym obvious".

Back to the definition of "a more direct route", there are a number of patterns or key words you can use to get results from other areas of Yahoo!. Looking at the tabs across the top of the search box:

  • Images: "photos" or "pictures" in a web query will give 4 images and a link to all Yahoo! Image Search results. Example: "puppy pictures" (a favorite of my wife)
  • Video: "video" or "videos" in the query gives a link to all Yahoo! Video Search results. Example: "brad pitt videos" (another favorite of my wife)
  • Directory: some terms will trigger links to the Yahoo! Directory's pages of categorized sites. Example: "History"
  • Local: a pattern of [city] [state] [business name / type] will give 3 results from Yahoo! Local plus a link to all results. Example: "San Francisco ca plumbers"
  • News: "news" in the query will gives three news headlines via Yahoo! News and a link to all results. Example: "Iraq news"
  • Products: various products or categories of products queries will give links to Yahoo! Shopping. Example: "digital cameras", "canon powershot"

Many people also end up using the Shortcuts for navigation. For example, "Pool" is used by many to quickly get to Pool on Yahoo! Games. Some savvy users just put an "!" on the end web queries that are Yahoo! properties to be taken straight there. "Mail!" will take you to Yahoo! Mail, "music!" to Yahoo! Music, "maps!" goes to Yahoo! Maps, and so on. (Ok, you smart people will point out that for maps you can just do a web search for an address and get a map at the top of the page, such as "550 Geary St, San Francisco, CA").

Lastly, with over 30 shortcuts on web search, there's plenty of ways to string them together to get things done faster. My wife will be headed to Las Vegas with some friends next month (yes, it is a bachelorette party and no, I'm not invited). She can use shortcuts to book her flight, look at prices and reviews for hotels in the area or a specific one, and line up a limo. When it's time to go, she can check the weather, get airport info, see if her flight is on time, check traffic, and see where to get cheap gas on the way to the airport.

Take a look at the web search shortcuts and see how they can help you find things faster. You can always click on the Shortcuts link on search.yahoo.com, our dedicated search home page, or on the top right of the web search results page, or by clicking the About link that's at the bottom of every shortcut. For access while on the go, take a look at the shortcuts for mobile devices.

Adam Durfee
Product Manager, Yahoo! Search


[ Yahoo! ] options

May 09, 2005

Yahoo! Local Traffic Dashboard Widget and RSS

Speaking of Mac OS X "Tiger", blogs have been buzzing with talk of Safari's new RSS capabilities, performance improvements, Spotlight (Apple's own "desktop search"), and Dashboard.

Dashboard is a framework for building lightweight "widgets" which pull in data from a variety of sources and display them in interesting ways on your desktop. We worked with Apple to build a Yahoo! Local Traffic Widget that's currently featured on their dashboard site. That motivated me to finally upgrade my Powerbook to Tiger this weekend.

It didn't take long for folks to dig behind the scenes and realize that it's powered by an RSS feed. It was on the O'Reilly Radar before we knew it.

It's great to see Apple putting more Web 2.0 tools on all those desktops and laptops.

Remix away!

Jeremy Zawodny
Yahoo! Search


[ Yahoo! ] options

May 06, 2005

Tiger Buzz

With Apple's recent release of OS X Tiger, I wanted to check out the buzz on the latest iteration of OS X. I'll spare you any lame Tiger roaring references, but if you want the dirt on hackneyed headlines, I highly recommend Jason Kottke's compilation of where the pack mentality went wrong.

While searches on Apple are up 11% for the week, Tiger hasn't truly dented the buzz to the extent I expected. The latest OS just can't compare to the glamour of the iPod. Witness the top Apple searches over the last week:

  1. Apple iPod
  2. Apple Store
  3. Apple iPod Mini
  4. Apple Quicktime
  5. Apple Laptops

Perhaps the searches on the Apple Store are from those looking to upgrade to Tiger, but given the buzz, we'd guess most shoppers are ending up with those oh-so-distinctive earbuds.

In looking at searches on the word "tiger," we had to keep in mind that while some of the interest was Apple-related, the bulk of those looking for tigers were probably looking for the striped variety. In any event, queries on "tiger" were up 19% for the week, a nice boost either way. Pressing on, I delved deeper into tiger-related terms to come up with these top OS X related searches from the last week:

  1. Apple Tiger
  2. Mac Tiger
  3. Tiger OS
  4. OSX Tiger
  5. Tiger Widgets

Buzz on operating systems doesn't exactly measure up to the Britney and Idol buzz I'm used to researching, but I'm always open to checking in with columns on other tech-related search trends. Please feel free to let me know what sorts of buzz you're curious about.

Erik Gunther
Yahoo! Buzz Index Editor


[ Yahoo! ] options

May 04, 2005

Video Search Goes Primetime

It's been a few months since I blogged about the appearance of the Video Search tab on the front page of yahoo.com, and it's time to give you the latest update on our product. Tonight we've shed our "beta" tag and released Yahoo! Video Search 1.0.

One of the reasons we launch a beta product is to get your feedback. We've been listening to what you've told us about video search, and the biggest request wasn't for functionality changes in the product, but for more content in video search. Addressing this was our #1 goal in this video search release.

So what's changed in 1.0? We've partnered up with some major content publishers to fortify our content offering, including MTV, Buena Vista (including the latest clips and trailers for The Hitchhiker's Guide To The Galaxy), CBS News, Bloomberg (check out the latest news on the Federal Reserve), Reuters, The Discovery Channel, Scripps Networks (the good people who produce Home & Garden TV and The Food Network), VH1, and more.

However, great video search isn't just about content from large publishers, it's also about the long tail content from smaller publishers and individuals as well. To that end, we're indexing the Internet Archive's Moving Image Archive. One of the great things about the Moving Image Archive is that it encompasses a wide range of content -- everything from the Prelinger Archives (a collection of over 48,000 "ephemeral films" from 1927 through 1987), to user-created Open Source Movies hosted by the Internet Archive, and other collections of video. (One of my favorites is the animated legos from Brick Films. Don't miss seeing their version of Grand Theft Auto done entirely in Lego).

Those of you who are budding independent filmmakers, or think you've created the next Numa Numa Dance, we're keeping you in the fold too. Besides making enhancements to our web crawl which indexes video files published online, you can also submit a Media RSS feed directly to us of the video on your website.

If you want someone to host your video content for you as well, you can have your video hosted by sites such as OurMedia, the Internet Archive's Open Source Movies, or any other hosting service that publishes video files directly to the web or with Media RSS. We applaud these hosting efforts -- it's our goal that once you publish your video content on any hosting service, we'll index the content in Video Search and help our audience find your content. Supporting an open model for video hosting gives you the most choices, so you can find the publishing solution that works best for you.

So whether you're a video consumer or a publisher, we've got you covered. Stop on by the new Video Search and take a look.

Andy Volk
Product Manager - Yahoo! Video Search


[ Yahoo! ] options

Hosting by Yahoo!