What do you believe is true even though you can't prove it?

At The World Question Center, a virtual watering hole for intellectual discovery hosted by the Edge Foundation, a collection of scientists and cognoscenti have gathered to respond to this year's big question, "What do you believe is true even though you can't prove it?"

Edge is the brainchild of John Brockman, "cultural impresario," thought catalyst, and flamboyant literary agent. Brockman is a man who's made a career out of thinking big. He represents rock-star physicists, mathematicians, biologists, cognitive scientists, and authors who straddle many intellectual domains. Brockman believes that asking big questions in a roomful of big minds can yield rich and stimulating discourse, and feed the pursuit of intelligent hunches that benefit all of us.

In his introduction to the 2005 annual question, Brockman refers to the age of "searchculture" and ponders whether, as search technology gets better and better at answering our queries, we can continue to frame the right bright questions. The annual Edge science question smackdown is his contribution to this human quest.

Computer scientist Marti Hearst, one of 120 contributors to this year's Edge exercise, looks at how we use language to pose questions and find answers when we search on the Internet. She believes, but can't prove, that "the Search Problem is solvable"; that elegant, innovative advances in technology will allow people to find the answer to any question that's already been documented and answered in text. For Hearst, understanding queries is key to making Internet search tools more effective.

The Web has made human knowledge publicly accessible via vast electronic repositories of data. Search engines have an endless appetite for this bounty of information (and misinformation) as they ceaselessly crawl and consume the expanding online universe. Computational linguistics, natural language processing, and related technologies uncover rules that help search engines communicate better with us humans.

Hearst's work focuses on algorithms and interfaces that help users locate information without drowning in data. Researchers can discover new, unanticipated answers to unsolved problems through a process known as text mining. Users can find their way through complex information spaces like archives or image galleries, if the interface is designed for a flexible and
flowing experience.

As computer scientists discover new ways to "teach" the search engine to respond "intelligently" to patterns in our search behavior, search technology helps us hone in on that elusive needle in the haystack. Occasionally, it even spins us wonderful golden threads we couldn't have imagined in the days before the Web.

Personally, I'm a big believer in serendipitous discovery (in life and in search), and I believe, but can't prove, that serendipity is closely related to the mind's ability to weave truth out of hunches and accidental discoveries.

Meaning out of chaos. Isn't that what science, and for that matter Yahoo! Search, is all about?

Let us know what you believe is true about Search, even if you can't prove it. We welcome your comments and ideas.

(Note: Marti Hearst is currently serving as a consultant and science advisor to Yahoo! Search.)

Havi Hoffman
Yahoo! Editorial