There’s a scene in Raiders of the Lost Ark, the first Indiana Jones movie, where the villain swallows a fly. It happens right after Indy catches up with a Nazi convoy transporting the Ark of the Covenant through the Egyptian desert, on its way to deliver the Ark to Hitler.
“I’m gonna blow up the Ark, Rene!” our hero shouts down from his vantage point on the cliffs.
The Nazis down in the canyon hustle toward Indy’s girlfriend, whom they’re holding hostage, and then the camera zooms in on the face of Rene, the evil French archaeologist. He stares up, steely eyes flashing pure hatred. “Your persistence amazes even me!” he shouts back. And then a fly walks up his chin and into his mouth.
Whoa. It’s easy to see, with this video pointing it out, but I’ve watched that movie a dozen times and never noticed anything funny about the scene. It turns out that’s probably just the way I’m wired. According to a recent study done at UC Berkeley, when we look at an object or person, we’re not just seeing them moment by moment, as you might expect, but rather their average over a short period of time.
According to Jason Fischer, who conducted the study published in the journal Nature Neuroscience while he was a doctoral candidate at UC Berkeley, your brain expects whatever it is you’re looking at to remain pretty much the same from one second to the next, and so it mashes together the image of the object as you’re seeing it now with the images of the object as you’ve just seen it. That’s why I missed Rene eating the fly. Even though it’s something I would’ve liked to have seen, my brain just wasn’t expecting it.
“It’s a trade-off the visual system makes,” Fischer, now a post-doctorate fellow at MIT, says of the averaging phenomenon, called serial dependence. While in some ways it makes our vision less accurate, he says that the trade-off benefits us by allowing us to filter out the noise. Imagine, he says, trying to read a street sign in pouring rain. Each drop of rain passing through your field of vision slightly changes the picture. “If you were sensitive to every little fluctuation in vision you’d have a very difficult time reading the sign,” he says. “Serial dependence smoothes over those little fluctuations.”
Fischer and co-author David Whitney, an associate professor of psychology at Cal, tested their hypothesis using experiments that, if you’ve ever had an eye exam, would probably feel vaguely familiar. For each of their four experiments, subjects sat with their chins in chinrests, staring at a white dot in the center of a gray screen positioned close to their faces.
In the initial experiment, subjects were first shown a half second of a stripy squiggle called a “gabor” sitting to the right of the white dot. Then the squiggle was replaced by a smudgey shape to clear the visual palate, followed by a flash of just the gray background. Then a short white line appeared, which the subjects moved with arrow keys to try and match the direction of the stripes on the squiggle they’d seen a couple seconds before. Then, after two more seconds of gray background, a new trial started with another squiggle, its stripes pointing a different way.
Each test subject sat through hundreds of these five-step trials, trying over and over to match the line to the stripes. It seems like a simple task—just remember which way the stripe was pointing a second or so earlier—and indeed, Fischer says the test subjects he talked with were confident they’d done it accurately. “Nobody had any perception that they were biased,” he says.
To his surprise, though, subjects’ guesses at the orientation of the stripes on the current squiggle were skewed on average about 10 degrees toward the orientation of the previous squiggle. Without their awareness or consent, Fischer says, their brains were telling them that the familiar object they were seeing in the present was similar to the way it was as they’d seen it five seconds before.
They slightly varied the trials in subsequent experiments, and found that the averaging effect can last even longer, up to 15 seconds or more, and that it can cause two similarly oriented squiggles to look different from each other, or two differently oriented squiggles to look the same. The effect also works even with eight squiggles on the screen at once, all changing, so long as the viewer only focuses on one. In the last experiment, where the position of the squiggles jumped randomly around the screen from trial to trial, the effect was broken, which makes sense, Fischer says. “That should be a cue—objects in the real world don’t spontaneously jump in position.”
Even for someone who knows what’s coming, the averaging phenomenon is hard to overcome, as Fischer found when he took his own test. “I didn’t think it was going to work on me,” he says, but when he looked at the results he found that his brain had suckered him like all the rest.
He acknowledges, though, that it’s for his own good. While he and Whitney were initially surprised at their results, their findings are actually pretty intuitive, he says. “The world changes smoothly over time,” he says, “It makes sense for the visual system to capitalize on that fact.”
When he puts it like that, I guess I am glad my brain is fooling me, just a little bit. But I sure do enjoy seeing Rene eat that fly.
