A chat with Andrei Broder (Part II)
- Posted March 9th, 2006 at 9:00 am by Yahoo! Search
- Categories: Interviews
Last week, we published the first of a three-part interview with Andrei Broder, Yahoo! Research Fellow and VP of emerging search technology for Yahoo! In today’s segment, we spend some time chatting with Andrei about what he means by ‘search without a box’ and moving from information retrieval to information supply.
Where do you see web search being right now?
Some things still haven’t been solved. If you look back at papers in the WWW conferences from the mid-90s, about duplication, crawling strategies, web graph analysis, and so on, they are still relevant now. All the problems are still with us and plenty of improvements are possible. In the same vein if you look at cars, you still have technology improvements in steel, engines, structural framing, but the focus of research is on hybrid cars and so on. For web search I believe that the next stage for research is on the side of Information Supply and the integration of multiple sources.
Would you say we (as an industry) have made good steps since the beginning days of search?
Yes, absolutely. When AltaVista first came out, we needed 3 months to build a 30 million corpus of documents, and it had lots of duplicates and other problems. But, a 50,000 corpus was big in the early 90s. Then ‘big’ meant millions, and now ‘big’ is tens of billions. It wasn’t just quantitative, but qualitative, improvements that happened and made web search much better.
So, Andrei, where do we go from here?
My paper on the taxonomy of Web search talks about three generations of web search. I believe that we are now entering an entirely new phase. I call this next phase ‘search without a box’. Search today is confined to putting in something and getting something back, a pull model. The next stage is for information to come in a context without actively searching, a push model. My favorite example is GPS. Instead of looking up your way on a paper map, you are in your car, and your GPS navigator gives you directions, shows gas stations near you, and so on. A year or two from now perhaps it will show you where those gas stations are, but only when you are low on gas. So you get information on an ‘as needed, when needed’ basis without explicitly asking for it. In the same vein, we will move from information retrieval to information supply.
Is RSS like that?
Alerts are an information supply that answers recurrent needs. What I’m talking about is more contextual. For example, advertising is a form of contextual information supply. The key is for the supply to be appropriate to the context. For instance in a skiing magazine ‘ ads for skis are a perfectly desirable form of content. Information supply as a science will continue to grow because of advertising.
And those are some of the things you are working on?
Yes, I am trying to understand how the information supply will take shape– there is a fine line between annoying and useful. We also want the user to help define their role in this. You have to understand the context, the user, and the social effects. If we understand what other people like you are doing, we can sometime move from information retrieval to information supply by understanding the class of equivalent users. But we still do not have a theory of information supply, or a definitive model. It’s completely open area. it is not necessarily something we’ll see next year, but it’s the next stage.
In fact, we’re already pretty good in some contexts, commerce sites for example. You go to a travel site, you do a search, you find that the temperature is nice or stormy or whatever, and here are some hotels where you might like to stay, and here are some things you might want to do, etc. That’s already a case of information supply. But we have to come up with how to do it in less constrained contexts.
Essentially we’re going from 2.7 words per query to 0! How do we do that? There’s a funny Dilbert cartoon about buying things online, instead of 1-click shopping, you have 0-click shopping. If you don’t say no fast enough, Dogbert ships you something! (He laughs). It’s tricky, you need a lot of magic behind the curtains and good UI to hide it; it’s a good research direction.
Know where this cartoon is? Drop a comment below. Next week, in our third and final segment, Andrei fields several reader questions. Stay tuned!
- 4 Comments