Developed at Berkeley, the AI system edged out its top human competitor by 15 points.
AT THE ANNUAL AMERICAN CROSSWORD PUZZLE TOURNAMENT in April, some 1,300 contestants raced to see who could complete the eight puzzles the fastest. The winner was Dr. Fill, an AI system developed by Matthew Ginsberg and helped to victory by Berkeley’s Natural Language Processing Group (NLP), headed by Professor Dan Klein.
The first computer to win the event, Dr. Fill completed most puzzles in well under a minute and only made three mistakes, edging out its top human competitor by 15 points.
This was not Fill’s first rodeo: In 2012, the Doctor placed a respectable 141st out of 660.
In March 2021, the NLP reached out to Ginsberg to see if they could borrow Dr. Fill’s code to compare it with their own Berkeley Crossword Solver. Instead, Ginsberg suggested they join forces and enter the competition again.
What does it take to make a crossword champion? Nicholas Tomlin, a graduate student with the NLP, explained that while computers are generally “really good at language tasks,” they face many of the same challenges humans do when completing crosswords. Obscure knowledge, current events, and riddles tend to trip up Dr. Fill just as they do people. Like humans, computers do better on the easy Monday New York Times crossword puzzle than the tougher Saturday one.
But, despite what you might think, 100 percent accuracy is not a must for crossword champs. Unlike Siri, which we expect to give us the right answer every time, Tomlin says Fill only needs to achieve 90 percent accuracy to be king of the crossword domain.
Tomlin wants to improve Dr. Fill by better equipping it to handle recent facts: One model allows Fill to search Wikipedia for clues.
While Fill’s success may discourage some flesh-and-blood crossword solvers, know that the tournament’s $3,000 cash prize was awarded to the top human performer this year, not the algorithmic Dr. Fill. In the meantime, at least one puzzle constructor has vowed to make a crossword for next year’s competition that will thwart a computer. Game on.
From the Fall 2021 issue of California.
