Q&A with Vik Singh on Yahoo! Search BOSS and Open Web Search
Vik Singh, architect of the Yahoo! Search BOSS team, is the brains behind the BOSS Mashup Framework. Since last summer, he has built many sample apps using the framework. Most recently Vik released an informal mashup (not an official Yahoo! product) called TweetNews, which fuses Yahoo! News with the Twitter API to provide a new ranking model for breaking news queries. Vik took some time to answer our questions about his new application, creative uses of BOSS, and future innovations in search.
Yahoo! (Y!): What’s TweetNews and how does it work?
Vik Singh (VS): TweetNews is a mashup that reorders Yahoo!’s latest news search results based on how popular they are in Twitter. For example, if you search ‘iPhone’ on TweetNews, the service queries BOSS for the latest news results on the ‘iPhone’ and fetches the latest Twitter comments on ‘iPhone’ via Twitter’s API. The service computes how many of these Twitter messages relate to each Yahoo! news result by looking at how much textual overlap there is, then displays the news results re-ranked based on their number of related Twitter messages. Basically this service uses Twitter to determine authority for content that is so fresh it doesn’t have links yet.
Y!: Why did you build it? And what does it demonstrate about what’s possible with BOSS?
VS: One emerging area of search that I think no one has really solved is real-time authoritative search. When breaking news happens (like the Mumbai bombing, Hudson River plane crash, or wildfires), it’s difficult for traditional news sources to discover and prioritize all the information in a timely fashion. It can take several minutes or hours for traditional media to converge on the important stories. However, new social media outlets like Twitter are breaking these important stories faster than traditional media. By looking at the number of users chatting about these topics, one can measure the future newsworthiness of a very fresh story despite its potentially minimal traditional news coverage at that moment.
Although Twitter can be used to discover and rank content, it’s not necessarily the best place to get in-depth, factual information since Twitter messages are very short and unverified. This service ranks actual news stories which provide integrity and in-depth coverage while using Twitter as a signal for ranking. A couple of days ago I searched ‘tweetnews’ on TweetNews and underneath the top result, one of the related Twitter messages described the service as: “The reliability of news with the speed of Twitter.” That pretty much sums up the value proposition of this idea.
I think overall the service was well-received. The traffic filled up my allowed server quotas in minutes. It was quite an honor to read that Wired thought it was the best mashup they’ve ever seen. This idea shows that there are still inefficiencies in search, and that with BOSS, anyone can go out and solve them. All the code and tools I used are open source. It took a little less than 100 lines of code to represent all the search logic, thanks to BOSS. This application couldn’t exist pre-BOSS. Going from nothing to a 100-line search engine is a pretty big advancement!
Y!: What are your thoughts on publicly releasing mashups versus fully “productizing” new ideas before releasing them?
VS: It depends on the idea, but I find that sometimes releasing “quick and dirty” to the world is a great way to test a proof of concept with a real audience. Many ideas that are easy to prototype and test with real users instead go straight through the expensive and slow productization process only to be a dud when they go out to market. In this particular case, releasing TweetNews as a more open third-party mashup provided us invaluable feedback on the idea and empowered our BOSS developer community with more source code and search design patterns.
Y!: Any examples of mashups or search products built using BOSS that you think show potential?
VS: I really like Trogdor, OneRiot, PostRank, InsiderFood, and BuildaSearch, to name just a few. Trogdor saves the users time because they don’t have to press enter or refresh the page; it automatically updates your search results while you type. OneRiot and PostRank aim to solve the social freshness issues that TweetNews highlighted but employ other sophisticated ranking techniques. InsiderFood provides a new user experience for discovering and searching international cuisine.
I like BuildaSearch because it lets anyone build a search engine using BOSS without having to write a single line of code. A user just goes to their site, picks some colors and favorite URLs, and voila, BuildaSearch generates a search engine. I also like many others that have been linked to on this blog.
Y!: Where do you see the most potential for innovation in search?
VS: Blending and personalization of vertical and web search results – I believe vertical search (shopping, local, auctions, news, social) provides invaluable, unique results from what we typically find in Web search. However, users need to learn to pick and choose from multiple search engines based on their query and intent. Wouldn’t it be nice to skip this step and do this automatically for the user? Can we be smart enough to know which search engine is best for that query, and if we’re not exactly sure, can we blend results together from the best verticals and personalize the experience and ranking for the user? I believe every search engine a user encounters should be totally comprehensive, relevant, and personalized based on which site the user is searching from. I strongly believe BOSS can help sites build this new wave of search experiences.
- 0 Comment