« August 2004 | Main | October 2004 »

September 30, 2004

Search Geeks Gone Wild

We’re making the trek north today to Lake Tahoe for the 4th annual Gnomedex conference. In fact, Yahoo!'s the title sponsor, so if you’re attending, swing by our table and introduce yourself.

[UPDATE: Jeremy and friends created a really cool page where you can easily add to My Yahoo! the blogs of many of the people attending Gnomedex.]

Nancy Evars
Yahoo! Search

[ Yahoo! ] options

September 28, 2004

Live Testing at Yahoo! Search

My job reminds me of an online BBS game called Planet Busters I used to play about 15 years ago. In that game, every day I would look at the landscape of the universe, the condition of each of my planets, and the results of the previous turn, and give commands for the next turn: send 5 ships to colonize planet X, send 1 ship to scout out planet Y, send 100 ships to conquer planet Z. After committing my commands, I'd have to wait until the next day to see the results. Every afternoon logging onto the BBS was like a treat. I excitedly fired up the game -- "Did my strategy work? Shoot, all 100 ships were destroyed?? My scout ship arrived and took over an undefended planet Y! Yes!"

At Yahoo! Search, I coordinate live testing for the Web search user interface (UI). Similar to Planet Busters, I plan out moves in the form of tests to try out on Web search. Even more exciting than the online game I used to play, however, is that if a test goes well, it results in a change to the Web search UI that helps people find what they are looking for even more accurately or more quickly than before.

Making changes on Yahoo! Search is analogous to changing the tires on a car going 100 miles per hour. We need to be able to make positive changes to the UI without stopping the service and without users ever missing a beat. Live testing allows us to do so. In a live test, we give a slightly modified UI to a small number of our users for a few days. We collect metrics on how these users interact with the test UI and then make inferences on whether the changes are good or not-so-good for users. Good changes are released for everyone. The not-so-good ones are brought back to the drawing board for refinement or modification.

Who knew that playing strategy games in my childhood would be training for my eventual job at Yahoo! Search?

Fred Delse
Product Manager, Yahoo! Search

[ Yahoo! ] options

September 27, 2004

New Beta Version of My Yahoo!

Earlier today we launched a beta version of the all-new My Yahoo!. If you already have a My Yahoo! page, just go to the Beta Update Page and click the "Update My Page" button to get it.

As you may remember, earlier this year we started testing the ability to add RSS feeds to My Yahoo!. People told us that they wanted to use My Yahoo! to keep up with content & services from across the Web and customize their personalized start page even more.

And thanks to content syndication standards like RSS and Atom, we now have a great way to offer more choice for our users.

Well, it took off like wildfire. People sent us a ton of feedback about how they loved this feature and what we could do to make it even better. Soon thereafter, "Add to My Yahoo" buttons started popping up all over the Web.

Then we decided to go a step further and completely remake My Yahoo! around the simple idea of giving users the content they want, when they want it - all in one place. The hard part was this: now that you can add literally millions of things to your page, how do we make it easy to find what you are looking for?

We tried to address that issue in a few ways:

Search - Thanks to people adding feeds to our beta module, we have built up a searchable database of over a quarter million feeds (and growing). My favorite example: search for "cub scouts" (Pack 34 has an RSS feed? Love that!)

Directory - Since there's so much out there, we asked our editorial team of Web surfers to pull together some of the best content into a browse-able directory. Wanna see all the best parenting blogs & info? We've got a category for that.

Popular from around the Web - I'll be keeping an eye on this as we roll out the new My Yahoo!. Right now it shows what's the most popular from our RSS module. I'm looking forward to watching this grow as users find out all the content they can add to their page.

Editors Picks - This is what's so cool about having a team of editors helping us out: they scour the Web for interesting sites that publish content and then highlight some of the stuff you might not find otherwise like New Urban Legends from scopes.com.

Can't decide? - The most "hidden" feature is our News Search tracker. Do a search over on Yahoo! News Search and then click the "Add to My Yahoo" link and you can track the latest news on anything you can come up with: like My Yahoo!. ;)

I've probably run out of room for this posting, so go ahead and get the beta from the Beta Update Page. Then be sure to tell us what you think (you'll see a link for feedback after you update). The way we look at it, this release is only the beginning.

Scott Gatz
My Yahoo! Team

P.S. If you publish an RSS/Atom feed from your site or blog, add it to your own My Yahoo! page. It's automatically added to our database for anyone to discover, and it's pretty cool to see your blog on a page next to Reuters, USA Today, and your Stock Quotes. Then put an "Add to My Yahoo" button on your site. (Tons more info is in our FAQ for Publishers. Check it out.)

[ Yahoo! ] options

September 24, 2004

May The Buzz Be With You

In looking at popular searches on movies for the last five years, distinctive trends have emerged. For big-budget blockbusters with A-List stars, buzz builds months in advance -- usually when the initial trailer hits the Net. Lesser-hyped movies may do a slower and smaller build, but the result is almost always the same.

The result involves a large spike in search on the Friday the movie opens, followed by sustained searches throughout the first weekend and a eventual leveling off. Some stick around longer than others, but almost all movies meet the same fate and flatline two to three months following their debut.

There is one movie that is the exception, the outlier that defies common wisdom. I assume you know where I'm headed. To a galaxy far, far away.

Star Wars is more than a movie -- it's become a universe of dedicated fans who flock to the Internet to meet, discuss, and gather information. This sci-fi touchstone was in our top 20 movie searches when we began tracking buzz in 2000 and has never departed. It's faced some recent challenges from powerhouse trilogies Harry Potter, Lord of the Rings, and The Matrix. These three mighty movies have also built strong cult followings on the Web, but will they still be popular in search in 2030?

That's the true testament to the popularity of Star Wars. A film released in 1977, it's in our top 100 searches today. No other movie is quite like it. Below is a graph of the movie titans in search over the last 30 days. Luke and Leia eclipse Harry with the recent DVD release of the first three Star Wars films, while Neo and Frodo keep chugging along at a consistent pace years after their film debut.


What films do you think we'll be searching on 25 years from now? Will The Force still be with us? Are there any concepts out there in the current world of pop culture that will still resonate a quarter-century from now? I'd love to hear your thoughts.

Erik Gunther
Yahoo! Buzz Index Editor

[ Yahoo! ] options

September 23, 2004

Help Wanted! - Great Engineers and Scientists

Web search really is unique. It requires attacking many simultaneous world class technical problems. It has had a huge positive impact on how people work, play, and interact with computers while being provided as a free service. And it is a vastly successful business, generating the revenue needed to drive further investment and innovation. Our combined team has been working on search continuously since 1995 and we have many internet firsts to our names.

Right now we are focused on taking the state of the art in search engines to the next level. To do that we need to add folks to our team who are as excited about the search challenge as we are. If you have wrestled with really large challenging software problems, have a record of deployed commercial successes, work well in teams and are intrigued by the idea of working in a domain where we will need to invent the next generation of tools and techniques in our field simply to start our work, send us your resume!

We have dozens of openings for Engineers and Scientists with significant applicable experience. Listed below are many of the openings in Yahoo!'s web search team. We are always posting new openings as well, both linked directly to the search engine: http://tools.search.yahoo.com/jobs/ and on our careers.yahoo.com site. Try searching for "search" or "web search" if you are using the careers site.

Interested? You can apply to the jobs listed directly by following the links below or contact Yahoo! Search Technology recruiting directly at: websearch-jobs@yahoo-inc.com.

The Yahoo Search Technology Team

Crawling and Indexing

The Crawling and indexing team designs, implements and deploys the clustered applications that retrieve, analyze and index the billions of documents in our web search engine. This requires large scale clustered applications. It also requires a focus on efficient document processing and graph tools.

Senior Software Engineer - Web Search Crawling

  • advanced understanding of filesystems/networks
  • Linux
  • system design
  • c++ and other programming languages

Senior Software Engineer - Web Search Classification

  • C++
  • design of APIs
  • performance

Senior Software Engineer - Web Map

  • C++
  • performance
  • algorithms
  • storage
  • bonus: graph algorithms, statistics

Senior Software Engineer - Web Search Indexing (3 jobs)

  • C/C++
  • API algorithms/data structure design
  • understanding of space-time tradeoffs
  • multi-threaded app. design

Job Links:

Search Engine Core

The search engine core team executes thousands of queries/sec considering billions of indexed full text documents per query. This is a large distributed application! Good query execution requires relevance ranking, spelling, automatic summarization and a focus on high performance.

Search Solutions Engineer - Search Core

  • C/C++
  • Systems background
  • Unix development
  • Large project experience
  • good communication skills

Senior Software/Algorithms Engineer - Results Presentation (Search Core)

  • C++/C
  • algorithms
  • data structures
  • Unix server development
  • multi-threaded app. design

Senior Search Core Engineer - Spelling Team

  • C/C+
  • Unix server development
  • project lead experience
  • algorithms
  • multi-threaded app. design

Senior Software Engineer - Search Core

  • C/C++
  • Unix server development
  • project lead experience
  • algorithms
  • multi-threaded app. design

Senior Engineering Manager - Cluster Automation

  • C/C++
  • management experience
  • clustered application experience
  • operational service experience

International Team

Our international team is focused on bring our world class search technology to the world. This includes work on region and language specific technologies and also on building our global search platform. The platform initiative involves building presentation and document processing technologies that work well for all languages.

Senior Search I18n - Japanese Search Team (2 jobs)

  • C/C++
  • Unix server development
  • Perl/Python
  • i18n preferred
  • fluency in Japanese

Job Links:

  • First
  • Second
  • Relevance and Metrics Teams

    The web search problem is full of open research challenges. To address this we need researchers, scientists and engineers to attack some of the worlds largest problems in statistics, information retrieval, datamining, and machine learning. The relevance team focuses on applied research and development in these areas. The Metrics and Analysis team builds systems to collect and analyze the data needed to assess and improve our service.

    Senior data analyst (senior engineer)

    • data analysis
    • data mining
    • statistics
    • Unix
    • Perl or other programming languages

    Senior Search Relevance Engineer - Web Search International Relevance Team

    • C/C++
    • Perl
    • Python
    • modeling
    • machine learning
    • computational linguistics
    • data mining

    Senior Software Engineer - Automated Metrics and Process Tools Team

    • Perl
    • PHP
    • MySQL
    • web-based tools
    • database applications
    • automation

    Engineering Manager - Relevance Tools

    • Team leadership
    • web-based tool development
    • DB applications
    • Unix
    • Perl or other languages

    Search Marketing Solutions

    Our search marketing solutions team works with web masters, businesses and Overture to make sure that relevant commercial content is included in our search results.

    Software Engineer - Paid Inclusion Content

    • C++
    • Systems
    • UNIX
    • algorithms
    • data structures
    • APIs

    [ Yahoo! ] options

    September 20, 2004

    Meet the Yahoo!'s

    I started working at Yahoo! in July and as with any new job, getting to know everyone can take a while. So when I was approached to start the "Y! People Profile" posts for this blog, I jumped at the chance to learn the organization while getting to know all the interesting and quirky people I see in the halls.

    In the coming weeks, I'll introduce you to some of the folks here on the search team as well as others from across the company. You'll meet everyone from designers to researchers, to engineers and product managers. I'll post my interviews here so you can get to know them the same time as I do.

    I am planning to focus on how they're thinking about search, but I'll ask them everything from the serious to the silly so you can see who they are on and off the Yahoo! campus (yes, some of us actually have lives outside of Yahoo!).

    My first profile will be Paulien Strijland, Head of User Experience and Design for Yahoo! Search & Marketplace. Paulien hails from the Netherlands and has spent the better part of 15 years at Apple, PayPal and NetFlip.

    I'm sitting down with Paulien shortly so if there is anything you'd like me to ask, post a comment and I'll try to throw it her way.

    'til then....

    Yvette Irvin
    Y! Profiler (and In-Product Marketer)

    [ Yahoo! ] options

    September 17, 2004

    Blogging Buzz

    Everyday, I go through my bookmarked blogs. I know. Before you even say it, I know RSS is the way to go. I guess you can call me old-fashioned because I don't mind actually visiting a site to read the latest posts. Some may find it unbearably slow or even anachronistic, but I enjoy my daily slog through my bookmarks. Some were sites I used for fodder when I worked on Yahoo! Picks, others wound up on my list through research for buzz topics, and still others ended up in my bookmarks through pure serendipity.

    When you think about blogs, search may not be the first thing that comes to mind. I find most blogs end up staring back at me from the browser window thanks to links from other blogs. But people search for blogs on a variety of subjects -- in the past week, we've seen over 3,000 different search terms with the word "blog" in them -- everything from "Dream Blog" to the more philosophical "What does blog mean?"

    Tools that help a person establish a blog presence on the Internet are insanely popular in search. Xanga and LiveJournal are the top two spots for those looking to start a blog. Both of these services are firmly ensconced in our top 500 search terms. Blogging seems to have replaced the old "Dear Diary..." among teenagers and these two destinations are the hottest places for teens looking to work through adolescent angst.

    Here's how they measure up in searches over the past 60 days:


    Xanga's on top, but LiveJournal runs a close second. What's interesting to note is the spikes that occur every weekend as people jump on the blog bandwagon in their spare time.

    Although I'm loathe to give up my trusty bookmarks, someday I'll see the light. I swear. Beyond me switching to an RSS reader, I'd love to know what trends you see taking place in the blogosphere over the next few months.

    Erik Gunther
    Yahoo! Buzz Index Editor

    [ Yahoo! ] options

    September 15, 2004

    Quest For a Car

    We live up in the Santa Cruz Mountains (Northern California), and as a result we were stuck with dial up access until a few months ago. My daughter, who is 16 ½, has been on the "Quest for a Car" for some time now, but prior to our getting broadband, her search pretty much consisted of calling me from her cell phone whenever she came across a car with a For Sale sign. Also, not wanting to waste any time, she would have already called the owner and would have full details to relay to me when she called. Now, of course, this always took place while I was at work, and in the middle of trying to get something done.

    Then, about 3 months ago, DSL made its way into our neck of the woods, and it wasn't just our Internet access that went into high speed, so did her car search. Suddenly I'm getting emails about cars listed on craigslist (often followed by a phone call to make sure I got the email) -- things like convertibles, 4x4s, old trucks, and other vehicles designed to make a parent groan inwardly.

    Somewhere along the line, I pointed out to her that there were other websites that listed used cars. Over the next few weeks she taught herself how to use the search refinement tools on the used auto sites. It was quite entertaining from my end to watch as she slowly came to terms with the kinds of cars that would actually be in her price range (yes, evil parents that we are, we made her come up with a fair percentage of the cost). Eventually she had it narrowed down to a handful of makes from a certain range of years that were most likely to come up with a car that she could afford.

    One of the most interesting parts of the experience for me as a parent, was seeing how different the process of finding a car was for her compared to me when I was her age. I either looked at the classifieds in the newspaper or kept an eye out when passing by the unofficial used car lots in the small town I grew up in. For my daughter, her first instinct was to go online. No used car dealers -- that would take too much time. No newspapers -- what good is three lines of text? She wanted details -- does it have a CD player, is it an automatic, etc., etc. Most of all, she wanted pictures. Was it a good color, did it look like it was in good shape, did the insides look okay? All of those things were checked out before ever giving the owner a call to set up a time to actually look at the car.

    In the end, she found a car by searching on Yahoo! Autos (tell me you didn't see *that* coming!). It is now sitting in our driveway, and next we get to go through "The Driving Test."

    Jerry Welch
    Lead Surfer

    [ Yahoo! ] options

    September 13, 2004

    Guest Blogger: Danny Sullivan, Editor, Search Engine Watch

    From time to time, you'll see some non-Yahoo! folks talkin' it up on this blog about various topics related to search. We're excited to have Danny Sullivan, creator and editor of the popular Search Engine Watch newsletter, as our first guest blogger. As many of you know, Danny knows a thing or two about the search space. He's been covering search engines since '95 and is considered one of the top experts in the space. [Update: Just to put any doubt to rest, guest bloggers are not compensated in any way by Yahoo!. We'll be sure to make that clear with each guest blogger post.]

    In 1989, drama in The OC wasn't on television but between two newspapers, the Orange County Register and the Orange County edition of the Los Angeles Times.

    The LA Times wanted to win in Orange Country, where it had half the circulation of the OC Register. It started pouring resources and people into its coverage of OC. That's how I got my start in journalism, as part of a new wave of hires helping the Times in its newspaper war.

    The Register didn't sit still, of course. It ramped up its efforts and people as well. As a journalist, being in the middle of this battle was great fun. Everyone was charged and excited about beating the competition. As a result, OC had news coverage like it had never seen before.

    Why this trip down my memory lane? Many readers of this blog no doubt know there's something of a search war going on right now. Like the newspaper war I experienced, the result is great for those the search engines serve. The resources, efforts and people being tossed into trumping the competition should ultimately benefit the searcher.

    I got to see the energy being unleashed first hand a few weeks ago, when I made a regular trip out to California to meet in person with various search companies. I've come to Yahoo! each year since 1997, but this last visit was special. I saw more new and innovative things in the works than ever before. But more important, the people themselves were pumped up and energized to be first in search.

    I can't talk about the new stuff yet, so I wanted to instead highlight the most important thing Yahoo!'s done this year. It's not super cool search commands (though my colleague Tara Calishain just did an excellent job of pointing out some of those, so check them out).

    It's not Yahoo!'s search shortcuts that have been blogged about already (but those are great -- keep them coming). RSS feed searching? Travel search? Local search? All great. But above it all, I like that in February, Yahoo! returned to having its own unique "search voice."

    Search voice? How can a search engine have its own voice? I need some search engine 101 to explain, and apologies to those who know this stuff already!

    When we search, the search engine checks what's called its index. The index is a collection of documents the search engine has gathered from crawling the web. Think of it as a big book of answers it consults, every time you do a search. No search engine has exactly the same set of documents in its index as another.

    Each search engine also uses is own unique method of flipping through the index to decide which documents should be top ranked. That method is called the search engine's algorithm.

    The combination of having a unique index and a unique algorithm is what gives a search engine its own voice, as expressed in the results it presents. (Want to easily see this? Check out this new comparison tool).

    You can think of it like movie reviewers. Is the movie Cellular that just opened good? Checking Yahoo! Movies, I find that of 11 different reviews, grades ranged from C- to A. Each reviewer saw the movie differently, judging it against their own unique tastes and knowledge.

    The same is true with search engines. We want a variety of opinions, alternatives we can consider. When Yahoo! unveiled its own search technology earlier this year, the company regained its unique search voice, and the web as a whole benefited from the greater diversity.

    I fully expect Yahoo! to keep building on the progress its made so far -- and its rivals are just as full of energy and ideas as well. I hope the competitiveness of the search wars keeps going strong. We all benefit from the search companies being kept on their toes.

    Danny Sullivan
    Editor, Search Engine Watch

    [ Yahoo! ] options

    September 10, 2004

    A Little Movie That Could...

    If you don't mind me dropping in every week to discuss subjects from the frivolous to the enlightening, I'll be happy to kick a little knowledge your way. Let me say I appreciate the depths of your interest in certain topics that are always hot (can anyone say Britney?), but even more, I appreciate the breadth of what you're seeking. The ways in which you turn to the search box on such a variety of subjects offers a reflection of what drives your life.

    I'm on the search beat every day, so I tend to get a little jaded about hype. Everyone loves an underdog, and I'm no exception. So when a little buzz ripple blossoms into a full-fledged trend, I tend to identify. Such is the case with the movie Napoleon Dynamite. While many big-budget summer blockbusters have come and gone, the little engine of Napoleon Dynamite is chugging along in our top 500 searches. Reaching out to the coveted teen audience, the indie flick did a super-slow-mo build to prominence. But unlike the shooting stars who performed mightily in their opening weekend only to slip quietly toward video, Dynamite has held steady in search for the better part of three months. On its way to cult fandom, the film inspired searches on "Napoleon Dynamite quotes" (+18%) and "Napoleon Dynamite soundtrack" (+62%) over the last week -- adding further evidence of the film's geeky yet oh so-powerful legs.

    To celebrate a little movie that could, we present this indisputable visual evidence for your enjoyment:




    Do you have a favorite movie, TV show, or band that built slowly toward mainstream acceptance? Comments? I'd love to know what you think.

    Erik Gunther
    Yahoo! Buzz Index Editor

    [ Yahoo! ] options

    September 09, 2004

    A Vacation From Search? Hardly!

    Last Monday I was just back from a long-delayed one-week vacation in Mexico, and got two emails from two VPs at Yahoo! about some posts on a search forum. A thread complained about quality/relevance of Yahoo! Search results for certain queries, and another was on error 999 that a few users encountered while using Yahoo! Search.

    Man, talk about back to reality: from basking in the sunshine on the sandy beaches of Cancun, to having to dig through all the software changes and log entries to investigate these two seemingly random posts. Hmmm...Now I know why search developers have no social life: having to do no evils at the cost of no vacations (or vacations immediately purged from one's short/long-term memory). But I digress. The investigation actually turned out to be fruitful: one inadvertent bug (is there any other kind of bug?) and one intended behavior (error 999 would be served to machines sending "excessive, anomalous, or abusive traffic" - and on the heel of myDoom, a few users might get this, not knowing that their machines might be infected with myDoom. This should be a topic of another blog post.)

    But I digress again. The point here is that we get feedback in nearly real-time now. Blogs, forums, message boards, you name it. I am impressed that Yahoo! Search really cares about its product quality to monitor the posts from its users, especially when the monitoring was actually done by the "higher ups" (hey, we have QA and surfers here at Yahoo! Search too you know). Before having a search engine of our own, we relied on others and had to wait for escalation, feedback, service level agreements, etc ... before an issue was resolved. Now it seems like everyone here helps keep an eye on things. For that, I am thrilled.

    Whether that thrill qualifies to erase my annual vacation so fast from my memory, the jury is still out.

    Nam Nguyen
    Technical Yahoo!

    [ Yahoo! ] options

    September 08, 2004

    A Search Toolbar for Non-IE Users

    Like sizable percentage of my engineering minded friends and coworkers who don't use Windows, I often find myself using Mozilla FireFox rather than Internet Explorer.

    But being on a non-Windows platform means that none of the Search Toolbars work for us. In fact, I'm often asked when we'll see a toolbar for non-IE browsers.

    Well, Pat McDonald e-mailed me to mention the FireFox Toolbar (beta, of course). And Ross Mayfield noticed it too. So I had to check it out.

    True to its claims, it works just fine on Mac OS X. (It probably works just fine on Linux/BSD too.) It supports a ton of search services, including Yahoo Search and Yahoo Finance. I suspect we could add Yahoo News without much effort too.

    Anyway, if you're a FireFox user, check it out.

    Jeremy Zawodny
    Technical Yahoo!

    [ Yahoo! ] options

    September 07, 2004

    Travel Search Beta

    As you may have heard, earlier this summer Yahoo! acquired FareChase, a travel search company. FareChase's technology enables people to perform real-time searches against more than fifty travel sites.

    Tonight we began testing this new travel search engine. It's still in beta but we're hoping you all can bang around on it and let us know what you think.

    Jim Yang
    Director of Product Management

    [ Yahoo! ] options

    September 01, 2004

    Buzz on Political Daughters

    Tracking trends on the Internet through search is actually a really cool job. I get to put my finger right on what people are looking for every day...and I know more about Britney Spears, J. Lo, and Paris Hilton than I ever truly wanted to.

    Right now, one of the hottest topics on Yahoo! Search is the offspring of presidential candidates. Between appearances at the MTV Video Music Awards and the RNC, political daughters are red-hot right now. Searches on the Bush twins and the Kerry sisters spiked dramatically over the last week as this graph illustrates:


    Jenna Bush leads the pack followed closely by Alexandra Kerry. Queries on "Kerry Daughters" and "Bush Twins" are also up over 500% in the past week.

    Shameless plug here -- if you're interested in trends and the latest in pop culture, top search terms from the Buzz Index are also available via RSS feeds.

    Erik Gunther
    Yahoo! Buzz Index Editor

    [ Yahoo! ] options

    Hosting by Yahoo!