Lately my mom looks at me like she has vastly overestimated my
intelligence for all these years. “You work on spelling?” she asks.
“It’s your generation. Back when I was in school, they taught us how
to spell correctly.” Stammering to my own defense, I argue that she
uses her word processor’s spellchecker. She responds smugly, “So why
can’t you just use something like that and work on something that
makes a difference instead?” Sigh.

Most people feel that spelling should be pretty easy–most of the
time, people are pretty good at it. Or at least they think they are.
The truth is, between 10 and 15% of the queries Yahoo Search receives
are misspelled. That means if we didn’t do any spell correction, many
people wouldn’t find what they’re looking for. So why can’t we just
use something like a word processor’s spellchecker? Try typing the
following into your word processor and seeing if you get the right
spelling corrections:

• evanesescence (should be evanescence, but I get no suggestion from my word processor)
• tofurkey (should not be corrected, but I get turkey as a spelling correction)
• sponge bob square pants games (should be spongebob squarepants games, again no suggestion)
• jules holland (should be jools holland, but my word processor fails me again)
• casini (should be cassini, but I get casino from my word processor)
• rims for hyundai sonta (should be rims for hyundai sonata, but I get rims for hyundai santa)

Topics of interest are changing constantly. New company names pop
up, new celebrities come into public awareness, and new products are
launched. We’ve got one shot to correct every word in your query and
present just one suggestion to you. On top of that, all the words in
the query interact with each other. “Brittany Spears” should be
corrected to “Britney Spears” but “Brittany Murphy” should stay the
same. So we have exactly one chance to correct all the words in the
query using an ever-changing vocabulary and keeping in mind the entire
query’s context to determine which corrections to make. Piece of
cake, right?

People find it to be a useful feature because correcting accidental
misspellings helps them find results for a query their fingers got in
the way of expressing, and let’s be honest–unless you’re a former
national spelling bee champion, there are some words that you simply
don’t know how to spell in the first place. Here’s a quick exercise.
Close your eyes and spell the governor of California’s name. It’s
S-c-h-w-a-r-z-e-n-e-g-g-e-r–how’d you do? I know I got it wrong on the first try.
Don’t tell my mother.

Luckily our spellchecker does a better job than I do on my own. We’ve
taken a great deal of editorial and linguistic input to define what it
means to do spelling correction well. Then we did a massive amount of
data processing to build a system that embodies those policies. The
end result is a system that is better than its creators at spelling.
There are still cases it gets wrong, but these cases are getting fewer
and fewer with every release. With our latest release, we’re giving
many more suggestions with higher accuracy than ever before.

So next time your fingers turn to mush and you get a “Did you mean…”
on Yahoo! Search, don’t feel bad. Spelling is hard and my team and
our spellchecker are there to help you. Now I’ve just got to convince
my mom.

Michael Mathieson

Project Lead, Query Spelling Service