People Blog Posts

Earlier this week, we spent some time catching up with Adrienne Bassett, an interaction designer on the Yahoo! Search team. Adrienne was one of five Yahoos that recently took a leave of absence to redesign the ONE.org website, the online arm of the ONE campaign, an organization founded by U2's Bono that's fighting global poverty and AIDS.

This project was the latest focus of the "Yahoo! for Good Scrum" initiative, an internal program that allows Yahoo! employees to take time off from their typical day jobs to apply their technical talents to projects with a social mission.

We asked Adrienne to share her experience working behind the scenes on this project.

Enjoy!

Adrienne, what exactly is a Scrum?

A scrum is basically a small team of people working on a project that's accomplished in short, concentrated bursts of activity with very specific goals. They can be pretty intense, although the ONE.org project was technically more like a charrette or a hack day, our team was working on a combination of design, usability and functionality problems all at once.

What was the team trying to accomplish with the redesign?

The ONE campaign is all about how people can incite change, one by one, to fight AIDS and poverty. The campaign has a huge global community of supporters, but it wasn't very visible with the previous website. Our goal was to change that, to capture and infuse community back into ONE.org. Also, to use the site for creating and growing awareness of the ONE campaign.

What were some of the ways that the team "infused community" into ONE.org?

Something we learned fairly quickly was that ONE campaign communities were already forming and thriving online, so part of our challenge was simply aggregating, organizing and supporting these communities via the ONE.org site. I'll give you a few examples:

Several ad hoc Yahoo! Groups have formed around the campaign in the last two years, with the new site, we're now showcasing these groups for supporters that might not have otherwise known about them, we're also providing easy-to-use tools and resources to encourage new group forming at a local level. ONE Groups are now surfacing in cities across the U.S. In fact we're using the Yahoo! Maps API to capture and track this growth via the "Where is One" page.

Another good example is the "Who is One" module on the front page. Often you see lists of names of people who have pledged their support for a cause, ONE.org has this too, but we wanted to take things a step further and enable people to share their faces as well. The Who is One module is a living and breathing photo mosaic of the people behind the ONE campaign. I think it adds an interesting dimension to the site. People are no longer just names on a list. You can see them. They can see you. It visually humanizes the campaign in a powerful new way.

There are several other examples I could point to, ranging from ways we've incorporated community education and learning via Yahoo! Answers, to a customized ONE toolbar, we've even created virtual ONE tees for people's Yahoo! avatars.

Tell us more about those avatar tees...

I think for the same reason people wear the white ONE wristbands in the real world as a sign of support, the avatar t-shirts are a way for people to share their support on the web. It's also simply a unique way to get people talking and connecting with each based on common interests.

Now you took three months off from your day job to work on this project. How tough was that?

At first it was difficult, leaving my team wasn't easy, but they were all very supportive which helped. As luck would have it, I was also between projects when this opportunity surfaced, so it was good timing for me. I've been with Yahoo! for a little over five years now, and it was a good chance for me to detach from my typical assignments, to wear a different hat and to work with a different purpose.

How did you get this entire project done in three months!?

We had an amazing team of people working on this project -- all day, everyday -- each of us with a unique skill set. It wasn't a big team, I was only one of five, but we shared a collective interest and passion for this project that was clear from the get-go. I also have to thank folks like Meg Garlinghouse and Geoff Ralston who were incredibly supportive and gave us very valuable feedback and guidance along the way.

Were there any significant challenges you had to overcome?

You mean other than getting this project from start to finish in three months!? Yeah, we hit a few bumps, nothing too significant, I think our biggest challenge had to do with ways we could balance user-created content, like comments, photos, etc., with some reasonable backend controls for moderation. There's a degree of risk the ONE.org website had to accept by enabling communities to connect and express their opinions and feelings freely via the site, our team tried to mitigate this risk by building and baking in some simple controls.

What would consider your big personal takeaway, now that it's complete?

I certainly feel invested (emotionally) in the ONE campaign, I feel pride with what we've accomplished, I'll continue to do as much as I can to support it. I also walk away with gratitude toward Yahoo! and my team for giving me the freedom and flexibility to work on such a cool assignment, I'm looking forward to returning and digging back into things.

I don't think anyone on our team will forget this experience. It was good for the mind and soul.

Yahoo! incorporated in 1995, the year I discovered the World Wide Web. That year, I made a decision that changed my life: I dared myself to use the Web to find a job on the Internet. I was a natural-born information junkie who could read, write, edit, and catalog--and fearlessly follow hyperlinks wherever they might lead.

I bought a fast Pentium running shiny new Windows 95. I got ISDN. I downloaded each new beta browser. In early 1996, I was hired to build a directory of web sites for one of Yahoo!'s now vanished competitors. I stepped into the fast-moving current, riding wave after wave of discovery, gathering a daily catch of tools and trinkets: image maps, javascripts, dancing widgets, canonical lists of nearly everything. I was getting paid to websurf!

In those days, we studied Yahoo! to see how directory was done. I walked the tree, and pondered colon classification and what it meant that Ranganathan was a Yahoo!. Web search scaled and evolved quickly to colonize the new info landscape, but the algorithms were young, and results were erratic and sometimes surprisingly irrelevant.

Yahoo! hired me on my third try, in 1998. The Web seemed vast, but finite. We still believed there was an end of the Internet. Then, as now, the Yahoo! Directory exemplified the value of informed human intervention, aggregating and organizing the best of the Web, creating choice out of chaos. And Yahoo! was fast, free, and fun, with invisible, reliable, leading-edge technology.

Over the past seven years, it's been a privilege to participate as Yahoo! and the Web grew up together. Through the tumultuous boom and bust years, search technology thrived. Yahoo! enjoyed a succession of relationships with great search providers. Then, more recently, we reinvented ourselves and launched Yahoo! Search Technology.

These days, search engine is a household word. The power of search has captured the public imagination and become essential in the lives of millions. And though we're continually innovating, we've just begun to explore the multi-faceted, multimedia knowledge exchange that becomes possible when search technologies mature and get smarter. Stay tuned.

And now it's time to celebrate. You're invited to Yahoo!'s 10th birthday party. There's even a present waiting for you there. Feeling nostalgic? Don't miss our amazing, entertaining web installation, Netrospective: 10 years, 100 moments of the Web. We'd love to hear from you.

Havi Hoffman
Yahoo! Editorial

You could never say that Reiner Kraft lacks vision or inspiration. This unassuming guy with the soft voice and thick German accent comes up with ideas--and incredibly viable ones--the way Snoop Dog flows lyrics.

Reiner's recent brainchild, Y!Q was launched in beta last week. Based on his concept of "disruptive distribution" technology, he believes it will significantly change the face of search.

Here's what Reiner had to say about his passion for search innovation and what it means to provide information "at the point of inspiration."

Q: You've coined this phrase "disruptive distribution" technology and you use it a lot when talking about Y!Q. What exactly is it?

A: It's a mechanism for distributing search boxes all over the Internet. As it relates to Y!Q, it's an API for webmasters that lets them insert icons within their content so that their readers can access related information about that content without having to leave their page.

Q: So the distributive part makes sense. Why "disruptive?"

A: Because it changes, or potentially changes, the way people search. Rather than having to go to a special page to perform a search, a search box is always a click away. You don't even have to type in a query. You can, if you want to refine your search further, but really it's optional.

Q: How does all this roll into Y!Q?

A: The key to Y!Q is the idea of contextual search or relevant "information at the point of inspiration." People liked to use that phrase before but with Y!Q it's becoming a reality. The idea is that there is always a context to what a user is reading or working on. So if they want to do a search, that search will be related to it somehow.

With Y!Q, we're able to identify what that context is and provide search boxes right where you need them. Then a user can dig deeper and ask more questions without interrupting their workflow.

Then of course, there's the API that the content owners or webmasters can use to integrate Y!Q into their pages. Now their readers can click on the Y!Q icons and automatically find more information about a subject. So in this case, the user isn't specifying the context, the content provider tells us, "this is the piece" that the user is interested in. It works as fine as when the user selected the context themselves.

Q: I like that it's the user can specify what they want. That's probably appealing to a lot of people.

A: Right. The other thing is that if we tried to automatically identify the context, we'd never get it 100% right. We'd just be guessing. But because the user says, "this is the piece of information I'm interested in," Y!Q can get the context right on the first try.

What's happening is the information they've highlighted gets transmitted to our search where our algorithms extract the key concepts and give them relevant results back.

Q: This question was posted by a blogger who thinks content publishers could use the Y!Q icons to help generate ad revenue. He asks, "Are there any plans to add contextual advertising to Y!Q?"

A: That's an interesting proposition. Y!Q is a new beta product and we're planning a lot of enhancements; but first and foremost we're focusing on giving publishers more control over the display and content in Y!Q. As we develop new features, we'll make sure to post them on the blog.

Q: Another blogger asks, "do you think Y!Q will phase out once the novelty factor wears off?" and "do you think it'll be used as a serious search solution by working professionals, [not] just cool kid teens?"

A: Y!Q was designed to address two key issues: First, we want to provide convenient access to search functionality at the point of inspiration. Second, we want to push relevant and enhanced results related to the context and provide superior relevancy for search results. If we're doing a good job for one and two, I think Y!Q has a very good chance of being adapted and used widely. Users generally use the search tool that is easiest to use and produces the best results. So I believe that Y!Q will be gradually accepted as the next generation search tool of choice.

For the second question: I already use Y!Q as my default search engine in Firefox, and it produces more relevant results compared to other plug-ins. Therefore anybody can use it as a default search tool. I don't think there is a preferred audience.

Q: Tell me a bit about your patents. You actually have one hundred?

A: I don't know the exact number. I filed probably over 100, and so far on the order of 40 have been issued. It typically takes about 2-4 years for patents to issue, so they're coming all in gradually.

Q: Wow.

A: That was mostly between the time of '98 and around 2001 maybe.

Q: Are they all related to search technology?

A: No. A lot of them are, but there are many others that are related to different type of Web technologies, for example e-commerce or location awareness technologies. Especially the latter ones may become more important soon once GPS devices [e.g., cell phones] appear on the market and become more broadly used.

Q: Aren't you also finishing up your thesis?

A: Yes, it's about domain specific search and is based on what I call iterative filtering meta search. The idea is to leverage the search engine infrastructures to create a filtering mechanism that automatically helps you get documents for a specialized information need. For instance, we built a buying guide finder that helps you to find just buying guides.

Q: If I hadn't checked out your website , I'd think that everything you do revolves around relevancy and search! A lot of people at Yahoo! don't know that you were part of a German band and that you've composed over 30 rock songs. How do define yourself first; composer or inventor?

A: (laughs) I just like to think about new ideas. So to me, it's all the same thing. You create some music piece or you create some ideas or some algorithms to do something. It doesn't have to be specific to search but ideas related to web technologies in a broad sense.

Q: What's the biggest satisfaction for you in working in Yahoo! Search?

A: I think the satisfaction at the end of the day is that you've invented something that you think is cool and useful and people are able to use it and it helps them simplify things. That's particularly true with the Y!Q project. I think it could be a new paradigm for how user's search. Hopefully if people like it and use it a lot, it'll become the default method for how we search. If that could be achieved, then of course that's kind of a nice thing. You've had some impact essentially--you've developed something people will use now and years to come.

Yvette Irvin
Y! Profiler

And speaking of Reiner Kraft, I�ll be sitting down with him shortly to talk about his take on everything from Y!Q to German rock bands. If you have anything you�d like me to ask him, just post it below.

Yvette Irvin
Y! Profiler

This is the second half of the Interview with Ali Diab

Q: Going back to the products you work on at Yahoo!. I know that Yahoo! Maps is about to introduce some new features. Can you talk about them yet?
A: Actually, we recently launched our new real-time traffic feature for Yahoo! Maps. It provides driving conditions, incident reports, the speed of traffic, the severity of each incident and a bunch of other things that help people get where they�re going with less of a headache. In the future, I think you can expect Yahoo! Maps to become much more context-aware, user-friendly and inter-connected with other services and device interfaces, be they mobile phones or PDAs.

Q: Earlier you talked about the reviews feature of Yahoo! Local. This question came in from a blogger who questioned how Yahoo! would sell advertising to businesses whose customers might write negative reviews. He wanted to know why he�d pay to advertise and risk getting a bad review from an unhappy customer.
A: Well, the key concept behind Local is the idea of community. To build any community you have to allow people to hear and be heard. Ratings and reviews allow customers to voice their opinions about the businesses and services they�ve used. Of course, there will be a broad range of opinions from negative to positive and all the ones in between, but to add real value to the Local user who�s trying to decide whether to use a certain business, you have to have an open forum.

In some ways, I also see the reviews as a kind of checks and balances for the business. It holds them accountable to their customers who now have an additional platform to voice their likes and dislikes. People are going to talk about what was right or wrong with a business anyway. At least this way, the business can monitor what�s being said and maybe learn from it.

But ultimately, the businesses will have to weigh the pros and cons. When all is said and done, Local is still one of the best and easiest ways to reach a massive audience that would cost a bundle to target in the traditional marketing world.

Q: So the reviews aren�t censored by Yahoo! to create a kind of biased, business-partial directory?
A: Again, it�s about providing honest feedback from real customers. If we slanted them in anyway, they wouldn�t be reliable. We do provide guidelines for what to write and not write in the reviews, but for the most part, we trust that customers will use the platform responsibly.

Q: I heard you just got married a few months ago. Is there any similarity between planning a wedding and launching a product?
A: I did learn some very interesting lessons planning the wedding that I�ve applied to my job and vice versa. First you need to go with the people you trust will do a good job and who have a good track record. Then you have to let them do their job. You can�t try to micromanage them because you can�t control everything. If they enjoy what they�re doing, they�ll put their heart into it and go beyond the call of duty.

In terms of what I learned from the wedding, obviously Nora has great taste and is very strong willed -- like me. (laughs) But our tastes differed on a lot of things so we came up with a rule that I think is very important in the work place as well: if it matters more to that person than it does to you, then let them have the final say--obviously within reason. That�s something I�m increasingly applying in my work and I find that the results are great because people who care passionately about something typically have thought about it a lot more than you have and may have more insight into it than you.

Q: I understand that you have a Math degree from Stanford and you�ve talked about always having a passion for computers and technology. When did you first realize that this was something you truly loved?
A: Well my parents always say that when I was a toddler, my dad had a really old micro-computer and I was fascinated watching him work on it. In addition I�ve always been interested in taking things apart and putting them back together again. I�ve also always been product oriented. I used to make furniture when I was in high school and build my own bikes (I've been racing motorcross since I was a kid). I must have torn down and built up probably a half dozen cars and motorcycles.

I like building things that have a purpose or that lead to a certain outcome. I find it interesting to research the best materials and resources to figure out the most effective way to build something. It�s really fun.

Q: So what are you working on right now? What are you building?
A: I�m an avid skier and I�m trying to get my hands on an old ski-press so I can create my own pair of skis. I bought these really phat powder skis for the season and they�re nice but I don�t think they�re going to have the edge-holding characteristics I need. Skis that are very good for holding an edge on ice tend to have a lot of metal or a lot of wood in them. They also tend be heavy and narrow so they can hold an edge against a mountain. Whereas powder skis tend to be wider and lighter to float and keep you above the powder so you don�t sink.

I�d like to build these kind of hybrid skis that have the best of both worlds. I want to use a certain type of wood that is very porous and lightweight so that it�ll float well on powder but at the same time have very good vibration dampening characteristics so that it will hold well on ice and chattery snow.

Q: As we wrap this interview up, what do you look for when you�re recruiting for the Local team?
A: We recruit all the time and we�re always looking for smart, self motivated people. I especially like recent grads. I feel like they�re hungry and I like watching people evolve and teach themselves how to do things. Its fun to see people just getting out of school. They make mistakes but it�s seeing that energy and honesty and that effort that makes it really really satisfying.

I also look for people who have a passion outside of work�be it sporting, musical, philanthropic, whatever, because I think that kind of balance is important.

But balance doesn�t always come easily. At first you go from one extreme to the other�it�s like a pendulum, and then eventually you find that happy medium�that harmonic frequency that works.

Yvette Irvin
Y! Profiler

Next up in my "quest to meet the most dynamic people in search" is Ali Diab. Ali doesn't sit on the sidelines. He's the kind of guy who likes to roll up his sleeves, figure out a problem, and dive in a make it happen. You see it in his passion for developing exceptional Search products as well as in his general zest for life.

In this two-part series we'll find out what Ali has to say about competition in the Local Search market, his love of technology, and his passion for helping those in need.

Q: Describe to me what you do for the Local team here at Yahoo!.
A: I oversee product management for Yahoo!'s local products, which includes City Guides, Local, Maps, Yellow Pages. I also work with other groups who want to leverage or add some type of a local aspect to their products. That obviously includes heavy integration with Yahoo! Search, as well as teams like Personals, HotJobs and all sorts of other areas of the network.

Q: What made you decide to join the Yahoo! Local team?
A: After meeting the team, I was really impressed. They are not only some of the top technologists and engineers I've ever worked with but they also have a very strong understanding of their consumer needs. They are genuinely a good bunch of people to work with which is always a huge plus. I'd actually say, that's one of the most important things.

Q: What would you say is the biggest challenge working on a product that has such strategic importance to the organization?
A: I think the biggest challenge for us as a team is continuing to innovate and build products that our consumers really want. The level of competition in this area is immense and comes from many different directions. So staying ahead of the competition, so-to-speak, can be challenging.

Q: Speaking of competition; what's unique about Yahoo! Local compared to similar local offerings?
A: I think there're a lot of things that differentiate Yahoo! Local. Of course we have the ratings and reviews which gives people an idea of what others think of the business or service before they try it themselves.

But I think one of our biggest differentiators is the depth of structured content we offer. We provide the basics like business address, phone number, website and then we take it a step further: we let businesses provide information that's unique to them like hours of operation, payment methods, specialty, and ambiance.

Q: What if the business doesn't have a website? Can I still find them in Yahoo! Local?
A: Yes, you'll still find them. Most people assume that if the business doesn't have a website then it can't be listed, but that's not true. If they have a physical place of business, we'll include them in Yahoo! Local. This is important because more and more people are turning to the Internet for local information and they need to find more than just those businesses that have their own website. It also helps smaller businesses connect to a much larger audience then they could traditionally.

Related to that, we have a substantial number of businesses and services on Local and we're continuing to grow that content. We have over 15 million listings and we've made it easier for people to add or update businesses. This is something that our users said they wanted when we were in beta and now we've added it. It essentially lets business owners either add their business or edit their existing one and it also encourages non-business owners to suggest a business listing or alert us to business changes in their community. [See Search Engine Watch article]

We also provide things like our refine and sort, which are unique to Local. These features let you define the specific type of business you're looking for. For example, instead of just looking for any and all restaurants in San Francisco, I can specify that I want an elegant restaurant with entertainment, a great bar and within a certain price range.

Q: What do you think you uniquely bring to what you do?
A: I think I'm good at building teams and recruiting people and motivating people to perform. I believe you have to find the right people to do the job and then give them breathing room. You have to let them demonstrate that they can succeed.

Zod [Farzad Nazem] our CTO, is a really good role model in terms of how to run an organization. He's told me many times that you need to just give people the benefit of the doubt and you need to let them do their job even if it isn't always in the way that you think it should be. Even if you're right, it's kind of the nature of democracy; people have free will. You can probe and you can question, but at the end of the day, if people are in a role and you want them to be successful and the company to be successful, you have to let them do their job.

Q: At thirty, you've accomplished quite a lot. What's the biggest lesson you've learned from your experiences so far?
A: I guess I've learned through both my academic and professional experiences that you need to pace yourself. I feel like if I want to be happy long term, I really have to enjoy everything I do and that may require me to focus on doing fewer things.

I can't say that it [pacing] has been an easy thing for me. In some ways I've had to rework my wiring from being always driven, always pushing, to sometimes kind of laying back a little bit and letting things happen at their own pace. And being in a type A driven industry and a type A driven company in particular, it's sometimes hard to pace yourself because you often feel like you're foregoing opportunities or you're not rising a fast as your peer group or whatever, which may sometimes be the case. But I do believe ultimately-long term-if you're going at your own pace and you're doing the things that you really enjoy, you'll achieve the things that you want to achieve when it's right to achieve them. And you'll enjoy yourself along the way which is more important.

Stay tuned for part II next week.

Yvette Irvin
Y! Profiler

It was announced today that Usama Fayyad and Bassel Ojjeh have joined Yahoo! as Chief Data Officer and VP of Technology-Data, respectively. You can read all about their pretty impressive experience and accolades here, but suffice to say, it's great to see Yahoo!'s continuing commitment to recruit the best and brightest technical and scientific people!

Usama puts it nicely in the press release:
"The opportunity to bring a disciplined, scientific approach to data technologies at Yahoo! was irresistible on both a person and technical level," said Fayyad. "I look forward to creating a scientific environment at Yahoo! that will continue to attract some of the best technical minds in the industry, while working to apply that knowledge in ways that will benefit all of our businesses."

Hm...Usama and Bassel would be great folks for Yvette to profile here! We'll have to bug them after they get settled. Stay tuned...

Nancy Evars
Yahoo! Search

Ali Diab studied Economics, French Literature, and Math at Stanford and graduated business school at age 23 as part of Oxford's first MBA class. From there he joined Goldman Sachs where he worked on some famous mergers and IPOs (including the IPO of NTT DoCoMo, Japan's largest mobile phone company). He started his own Internet company, BuildPoint, worked for Microsoft and was offered the job of Chief Product Officer at T-Mobile Online before joining Yahoo!--all well before his 30th birthday.

As head of production for Yahoo! Local Products, Ali keeps that same momentum going. He provides strategic direction for the Local product line which includes City Guides, Yahoo! Local, Maps, and Yellow Pages.

I'll be sitting down with Ali soon to talk about all this and more. If you have questions you'd like me to ask him, just post them below.

Yvette Irvin
Y! Profiler

No -- I don't show up here only to post about the Yahoo! parties, but we did have one doozy of a year end party (YEP) this past Saturday night in SF. This was my fifth YEP and as with all the previous years, I thoroughly enjoy seeing friends and colleagues dressed in something other than t-shirts and jeans!

In addition to the ice sculptures, amazing food and great music, one of the reasons I love attending the YEP, is the charity auction held every year. The auction starts a week in advance online through Yahoo! Auctions and culminates with in-person bidding at the YEP for the big ticket items. We all have the chance to bid on cool donated items and experiences ranging from a Stanford hoops game with Jerry & David or an LA movie premiere from Terry, to coveted parking spots or a weekend in Tahoe. Proceeds go to the Yahoo! Employee Foundation(YEF), our grassroots philanthropic organization that helps out non-profits in the community. Proceeds from both the online and live auction brought more than $120,000 to YEF, far more than any previous YEF auction.

A great way to start out the holiday season!

Nancy Evars
Yahoo! Search

As promised, this is the second half of the Interview with Tim Converse.

JQ: There's been this sort of continuum evolving since the Alta Vista days, where we started with primarily static content and then the next generation was catalogue type shopping like Amazon. Now it's this whole micro-publishing independent thing. What do you consider the biggest challenge right now in terms of classifying content?

A: One of the big challenges for us is just understanding what's out there. It's almost like astronomy where you're just trying to catalogue all the different things out there and track how they're growing, to some extent. So it's important for us to know just how many sites and kinds of documents there are so that we can catch trends.

JQ: You talked about comprehensiveness. There's this perception that there's the web that most of us see and then this dark web: the stuff that the crawlers don't reach. How do we try to get that data into the index? Are there barriers that webmasters put up that they should avoid to help us better index the content?

A: At it's simplest, webmasters aren't aware of robots.txt and it's uses. Redirection can also be problematic if people create content by creating lots of domains or hosts so we encourage people to organize their sites in many documents before they get a new host.

And of course, there's also the issue of crawler traps which some people do intentionally but much more often, they've unintentionally created crawler traps....

JQ: ...and a crawler trap is...

A: A crawler trap is something where you crawl a page and it has a link, usually in the same site that's dynamically created and then you follow that link and it has another analogous link that's dynamically created and often, just because people make mistakes, you're attaching on another directory every time which doesn't exist and takes you back to an automatically generated error page which has the same link. So you can fall into traps where there are an infinite number of pages that don't have any content.

Another thing people can do to help us is, this is sort of geeky but, don't make page not found pages that return a status 200.

JQ: I was just about to ask that. 404 pages back in the day, were these ugly grey things with block text that all looked the same and now they're done up to look like regular pages to be more appealing to users.

A: We do actually have ways of detecting that but it's a lot easier for us if a web server just says, "this page doesn't exist" as opposed to creating a nice page for the user that to a crawler looks like any other page. In general, if the server tells us 404, then we discard it.

YQ: I worked for a company that used CIDs instead of cookies to follow users through the site and it turned out to be a disaster. We went from having pretty much every page indexed to hardly any. So what about CIDs and how they affect the crawlers?

A: If you have differences in the URL that don't actually make a difference in the site, that can be hard for us to untangle. We're getting better at it. One of the scenarios you're talking about there would just create a lot of duplicates for us. So it's nicer for us if we have one URL per actual content but we understand that you're not designing this just for us. And we obviously do a lot of duplicate detection--actually, we do duplicate detection in a couple of different ways. Finding out if documents are the same; finding out if sites are mirrors of each other.

JQ: This question came up today on a mailing list that I'm on. The concern for this particular company is that they want to move their site to a new domain but they don't want to become invisible for the next six months or year or however long it'll take for people to point to their new website. What can we tell people like that?

A: We can tell them that in the future, if you actually want to move your site, you want to use a 301 redirect which will do as much of the right thing as we can.

YQ: What actually happens there? I've heard of companies who have used 301 redirects and yet their old pages continued to show up in the search engines anyway. Why is that?

A: The underlying problem is that people out there haven't changed their links and search engines do pay attention to links.

I can't give you a date, but we're changing how we deal with redirects. The thing about redirects is that everyone thinks it's obvious how a search engine should treat them and the obvious answer is not really that helpful. Any policy you develop with redirects is going to make someone unhappy but what we're about to roll out we will pay better attention to 301 redirects and the exact problem you're talking about should be less.

[In the time since we met with Tim, the team has rolled out a fix for 301/302 redirects. Documents will be handled by the new redirect policy as they are re-crawled and re-indexed and webmasters will start to see many of the sites change in the next couple of weeks. The index should be fully propagated within a month. See Tim Mayer's Webmaster World presentation for details.]

YQ: You mentioned earlier that you'd just bought a piano. I read on your website that when you were eight, you ran away from home to escape piano lessons. Is that true and did you just hate piano back then?

A: Yeah, that's true but I never hated piano, just lessons. Now that I've bought the piano, I'm practicing again. Right now I'm learning a classical piece by Bach. I'm a slower learner now though--I should have stuck with it when I was eight. But when it comes to the kind of music I actually listen to, I like rock, hip hop and classical. I'm not too into jazz.

JQ: Which do you listen to when you're programming?

A: (laughs) I don't actually. I don't deal with headsets well.

JQ: In terms of freshness, there's a lot of talk about how quickly an RSS-watching engine will pick up new content as opposed to getting stuff into Yahoo! Search. The question they ask is why can't we just ping Yahoo! and get the crawler over here?

A: Well, that's not the only source of latency or possible delay. We build very large databases and it's kind of a large industrial process involving lots and lots of machines. There's some delay between the last document we heard about and the time we actually put something live. In some cases that delay matters more than any delay in finding out if something has changed. We also pay a lot of attention to whether something has changed. But I think you'll see us getting fresher and fresher.

YQ: When you're looking for things that are changing on the page, what are you specifically talking about? I'm sure it's not enough to just change a hidden date stamp in a footer.

A: Yes, It's more than that. Most of the web is just static even without there being date stamps. We do have a more nuanced notion of what it means to change so we can detect a trivial change from a significant one. We can tell a major change from a trivial one.

YQ: You mentioned that you were between the scientist and the programmer. What do you think you uniquely bring to what you're doing?

A: Well I'm hoping that one side thinks I'm the other side and that the other side thinks I'm the one side. (laughs) I'm hoping I've got them all fooled but I have this feeling that I don't. But no, I think that what I uniquely bring is that I can talk to both sides, I've been a programmer; I've trained to some extent in the direction that the scientists have trained and went to grad school for a long time in related topics so increasingly, though I never thought I would play this role and it's not what I envisioned, what I'm bringing to it is I can talk to a lot of different sides and I can prioritize the stuff and my grasp of the technology's not so bad either.

Yvette Irvin
Y! Profiler

Tim Converse isn't your average search geek or for that matter, your average guy.

Though he's ostensibly shy and unassuming, he has a definite adventurous streak. Tim has tried everything from skydiving to African safaris, he likes rap (but only old school) and he once played keyboard in a punk rock band. But what really sets Tim apart is his knowledge of the inner workings of search. Unlike most of us who may have a decent understanding of the search world (or us novices who know just enough to type in a query and hit the "search" button), Tim understands the mysteries behind what makes it all work and how to make it better.

As an engineering manager in Yahoo!'s Content group, Tim and his group help make search results more relevant.

Jeremy and I spoke with Tim last week. Here's what we learned about content classification, what Tim likes to do for fun, and some little-known facts about Yahoo!'s obsession with foosball.

JQ: You've said that your group is charged with content classification. What exactly is content classification and why is it important to search?

A: Well, the more we know about documents the better. So part of what the Classification group does is label web pages and sites, or put them into categories. And while I can't get into specifics about the categories we use, a big part of this is trying to detect who's spamming us--or trying to trick us into ranking their sites higher in our search results.

Our classification code gets deployed in the Content system, which does the crawling and indexing to build search indexes that we end up serving queries from. That's mainly for our own group YST [Yahoo! Search Technology], which handles the back end of web search, but we also provide data to other groups, including Image Search.

My group also writes tools to interact with the Content system. We can query it in all sorts of ways to find out what's happening with particular sites or URLs. This is a challenge because the Content system is very distributed and heterogeneous.

YQ: It seems to me that if you're writing code for something, at some point, you've written it and it's done...

A: Well, we're never really "done." A few years ago my cousin asked me what I was working on and I told him "Excite's web search engine." He said, "so that would imply it's not done? Or it needs work or what?" (laughs) And so yeah, especially with how competitive the market is, these things are always under development. There's far more ways we can think of to make it better than we have engineers to do it. So even just with our list of ideas right now, we could be going for five years and there are always new ideas.

I should point out that although we're focused on deploying code for YST, there's a lot of expertise in the company and several different groups of scientists focused on classification. A lot of the challenge for me is just managing to benefit from that expertise for YST...

JQ: And connecting those dots in the company?

A: Yeah. It's kind of a cool job because I'm sort of in between scientists and programmers and there's such a spectrum of roles and responsibilities. We have people all the way from kernel hackers to linguists and needless to say, a kernel hacker can't really talk intelligently to a linguist or vice versa but you have to have this long chain of people who can really talk to each other so I've kind of got scientists on one side and programmers on the other.

YQ: What has been the biggest change in the way you approach writing the code and how you approach content classification?

A: I don't think the way we think about writing the code has changed. The way we're approaching search itself has changed a lot.

For instance, comprehensiveness is a much bigger deal these days. In the Inktomi days we wanted just one copy of anything that was good because serving documents costs so much. Now we'd really like to have everything.

So then the challenge is ranking everything appropriately. You really want to put everything out there but then...

JQ: ...that assumes there'll be a lot of junk?

A: Right and so then the challenge is identifying and appropriately ranking it all.

The big things for us are "relevance," "comprehensiveness," "freshness," and "presentation." That's "RCFP" and it's kind of our mantra. I'm much more focused on the "R" part of the relevance, although we have a whole group of scientists and modelers who are totally devoted to relevance too. My buddies in my group who work on crawling and indexing are focused on comprehensiveness and freshness as well.

YQ: Switching gears for a moment, Tim. What do you do for fun? What kinds of things are you into?

A: I like games. All kinds. Strategy games, pool, 8-ball, 9-ball, billiards. I'm a little worried about Yahoo!'s growth plans, because I think our pool table may not be scaling with the hiring we're doing. Is anyone looking into that? We had a nice one at Inktomi, but I think it's in storage somewhere.

I guess foosball is a Yahoo! game of choice, so I'm trying to catch up on it. It's a little known fact that one of the game's experts, Phu Hoang, is here at Yahoo!, and the game was named after him. I'm also interested in music and I just recently bought a piano. I hadn't played piano in a long long time.

JQ: When you're hiring someone for your team, what are you looking for?

A: We look for pretty senior engineers and like I said before, it takes a lot of types of expertise to make a web search engine. In terms of skills, we're looking for C++ coders, strong problem solvers, and people who understand CS algorithms. Obviously there are particular roles that require some particular expertise like experience in classification and textual analysis.

Right now, we're hiring pretty aggressively.

JQ: When it comes to fighting spam, there's all kinds of software and many people trying to stop spam attempts. With all of us trying to detect this, is there a way to tell the search engines about it?

A: We get a lot of that data on our own. We have a pretty large view and we're approaching the spam problem from a lot of different directions. But nobody should expect to see any sudden change in spam just yet.

Take weblog comment spam, for example. Two things will have to happen for comment spam attempts to decrease; one is that spamming will have to not work for search engines and the second is that comment spammers will have to realize it. (laughs). There could be a long lag there where, even if every search engine totally nailed them, spammers could still operate under the belief that it worked. What we can do from the search engine point of view is make spam not help.

Next week Tim talks about redirects, index-able pages and why he doesn't listen to music while he's programming.

Yvette Irvin
Y! Profiler

Yvette and I are planning to sit down to chat with Tim Converse. Much like she did with Paulien, we'll ask some questions about where Tim came from as well as what he and his group are up to these days.

I asked Tim for a description of what his group is all about so that we could solicit questions from those of you outside of Yahoo. He said:

I manage the Content Classification group within YST (that's the backend of Yahoo search). The Content group does all the crawling, indexing, and webmapping of documents for web search, and my group is responsible for categorizing those docs. We write software to algorithmically classify web pages with a special focus on catching search engine spam. We also write software to help us understand what the Content system is doing.

So if there's something you'd like us to ask Tim, leave a comment below.

Jeremy Zawodny
Technical Yahoo!

With Yahoo! approaching its 10th anniversary, the question I'm hearing a lot lately is "what's next in the world of search?"

Ten years ago, we were focused on a simple yet vast problem: finding better ways to aggregate and organize information so people can find it. Today, the challenge is different. On the one hand, there's a lot more information to aggregate and it's not just more in terms of quantity; there's a larger variety of content as well -- from products and images to news and business information. In addition, we're pulling content from more sources than ever before.

On the other hand, our user's expectations have also changed. It's no longer enough to simply provide a structure for users to find what they want on the Web. Today, people expect to find precisely what they're looking for exactly as it relates to them. It's the old example of the "Java" search query. Are you looking for coffee or for the programming language? People want to define what's relevant to them in their own personal way. They also want to tap into the source of their information at will and they want to manage it all to personally suit their needs.

That's what is exciting about where we are today. Search as a problem is still far from being solved. The user is in the driver's seat: they want an experience that is increasingly personal, more relevant, and ties into their task more integrally. Search is just a way to get that integrated experience, but it's all about what the users want - when they want it, how they want it, and who they want it from.

Jeremy hit on it in a recent blog entry; we have to "make search more relevant and personal." Those two things are the natural progression for search and they are tightly connected to our concept of seamless integration. Search has to reach a higher bar: it has to enhance the user's life on a daily basis. Integration of search, community, personalization and content builds the foundation for relevancy in people's lives.

Because the Net is obviously a bigger part of people's lives than 10 years ago, we at Yahoo! also have an opportunity to integrate into people's lives more deeply than before. Yahoo! Local and the beta version of My Yahoo! Search are just two of the examples of how we're enabling people to manage their search content, search within locations of their choice, and build personal communities online. Users can connect to people with similar interests and they can gather and share search information at will.

Fortunately, we're also at a time when the technology is helping us plug into people's lives even more richly. For instance, at this year's Web 2.0 Conference there was a lot of talk about RSS and wireless technology. This is stuff we only dreamed about ten years ago and its helping redefine what we do with search today. RSS is allowing people to access exactly what they want and wireless is letting us deliver the information wherever you are. People aren't chained to their PCs anymore and neither is search. Yesterday's introduction of Yahoo! Search for Mobile is just one example of how technology is propelling search forward. Search is literally in your pocket and at your bus stop. It doesn't get more integrated than that.

The question to ask now isn't if or when; it's "what else." What else can we do to take search to the next level? What else can we do to make search even more useful and accessible to you?

These are the challenges that will keep us busy for at least another 10 years and we're getting closer everyday. At Yahoo! it's our job to stay ahead of consumer needs and expectations and, based on the responses of our users, I believe we're doing a really good job so far -- but it's still very early. It's one of the reasons I remain really excited about how we can continue to provide real solutions to people's problems, and make a difference. While I'm not nearly as technical as I was 10 years ago (I got my hint when David Filo changed the password on me so I can't touch code anymore), I firmly believe that the technology we are building today makes the future of the Web even more useful, informative, and entertaining. As long as there's a way to help people find more precise and more relevant information on the Web, you'll find me in the thick of things searching for it.

Jerry Yang
Chief Yahoo

Paulien Strijland is Yahoo!�s director of User Experience Design (UED) for Search and Marketplace and when you first meet her, you can tell that she�s creative. She is a striking figure at 6�1� and wears expressive, flowing outfits and chunky, eclectic jewelry. She speaks enthusiastically about UED and she always seems to be in the middle of something interesting.

But what Paulien brings to Yahoo! is a lot more than creative energy. She is a business-savvy pragmatist who values collaboration tempered with practicality. But it may be her penchant for diplomacy, more than her pragmatism, that helps her provide unique direction for Yahoo! UED.

Here�s what I know about UED: you can build the best engineered product around but if no one understands how to use it, then who cares? It�s like the new cell phone Paulien was fiddling with when we spoke, �this phone�s got at least 100 features,� she said. �But all I care about is getting to the two or three that I want. They�re randomly buried in with all the others so it�s hard to find them and get to them fast. That�s not good user design.�

I sat down with Paulien over coffee last week as she shared her thoughts on user design and the world beyond Yahoo!.

Q: You�ve been involved in user interface design for over ten years now. What changes have you seen in the direction of UED and how it�s perceived?

A: Years ago there was no formal training for UI (User Interface) design and it was a discipline that wasn�t really recognized or viewed as important. Most companies didn�t even have UI designers. These days, even the smallest organizations have an appreciation for the field. So you spend less time trying to explain how UED affects the bottom line and more time getting to the design.

On top of that, the numbers of people using computers has significantly increased. This means we�re now designing for new types of users with different perspectives and different levels of computer savvy. Our designs have to be easy enough for the novice to use but compelling enough for the power user.

Q: What�s the toughest aspect of your job?

A: Everyone has an opinion! Yahoo! is very collaborative and everyone is a user on some level or another. The toughest thing is understanding the value of hearing everyone�s feedback but knowing that everyone�s opinion can�t go into the product. If it did, we�d have a hodge-podge design that really served no ones purpose. You have to be very diplomatic.

You also have to remember that we�re not the typical user. Our teams do a lot more searches in more ways and with more comparing than regular users. So we may not see things the same way they do. When you understand that distinction, you�re able to really hear what users are telling you about the product and about the design. It comes down to striking a balance between what your original product design goals might have been and what you�ve learned from the people who are going to use it.

For example, before Yahoo! Local was in beta, we'd received lots of very positive user response about the "view results on map" feature. The problem was that once we'd made the beta public, users weren't even aware that a "view results on map" feature existed. We had designed a button for it and we thought it as very visible and very intuitive. In our minds, it was "right there." But users still weren't using it. They just weren't registering it visually.

We ended up sitting back down and seriously rethinking how we'd treat that feature and it was very different from our original design concepts. When Yahoo! Local came out of beta, we'd found a much more effective way to call it out.

Q: But how do you really know that it's effective? Maybe users still aren't using it.

A: Well, we can tell that it's a very used feature now. Our reports show that people are clicking on the button so they must be finding it. On top that, they're telling us themselves that they really like the feature in the feedback they send to us. So we know they're using it. I'd say the redesign worked.

Continue reading "An Interview with Paulien Strijland of Yahoo! User Experience Design" »

We�re making the trek north today to Lake Tahoe for the 4th annual Gnomedex conference. In fact, Yahoo!'s the title sponsor, so if you�re attending, swing by our table and introduce yourself.

[UPDATE: Jeremy and friends created a really cool page where you can easily add to My Yahoo! the blogs of many of the people attending Gnomedex.]

Nancy Evars
Yahoo! Search

I started working at Yahoo! in July and as with any new job, getting to know everyone can take a while. So when I was approached to start the "Y! People Profile" posts for this blog, I jumped at the chance to learn the organization while getting to know all the interesting and quirky people I see in the halls.

In the coming weeks, I'll introduce you to some of the folks here on the search team as well as others from across the company. You'll meet everyone from designers to researchers, to engineers and product managers. I'll post my interviews here so you can get to know them the same time as I do.

I am planning to focus on how they're thinking about search, but I'll ask them everything from the serious to the silly so you can see who they are on and off the Yahoo! campus (yes, some of us actually have lives outside of Yahoo!).

My first profile will be Paulien Strijland, Head of User Experience and Design for Yahoo! Search & Marketplace. Paulien hails from the Netherlands and has spent the better part of 15 years at Apple, PayPal and NetFlip.

I'm sitting down with Paulien shortly so if there is anything you'd like me to ask, post a comment and I'll try to throw it her way.

'til then....

Yvette Irvin
Y! Profiler (and In-Product Marketer)

Last Monday I was just back from a long-delayed one-week vacation in Mexico, and got two emails from two VPs at Yahoo! about some posts on a search forum. A thread complained about quality/relevance of Yahoo! Search results for certain queries, and another was on error 999 that a few users encountered while using Yahoo! Search.

Man, talk about back to reality: from basking in the sunshine on the sandy beaches of Cancun, to having to dig through all the software changes and log entries to investigate these two seemingly random posts. Hmmm...Now I know why search developers have no social life: having to do no evils at the cost of no vacations (or vacations immediately purged from one's short/long-term memory). But I digress. The investigation actually turned out to be fruitful: one inadvertent bug (is there any other kind of bug?) and one intended behavior (error 999 would be served to machines sending "excessive, anomalous, or abusive traffic" - and on the heel of myDoom, a few users might get this, not knowing that their machines might be infected with myDoom. This should be a topic of another blog post.)

But I digress again. The point here is that we get feedback in nearly real-time now. Blogs, forums, message boards, you name it. I am impressed that Yahoo! Search really cares about its product quality to monitor the posts from its users, especially when the monitoring was actually done by the "higher ups" (hey, we have QA and surfers here at Yahoo! Search too you know). Before having a search engine of our own, we relied on others and had to wait for escalation, feedback, service level agreements, etc ... before an issue was resolved. Now it seems like everyone here helps keep an eye on things. For that, I am thrilled.

Whether that thrill qualifies to erase my annual vacation so fast from my memory, the jury is still out.

Nam Nguyen
Technical Yahoo!