A Berkeley neurobiologist looks at the science behind film.
Berkeley psychology professor Arthur Shimamura is unabashedly obsessed with cinema, but his day job is studying the brain. So, as both cineaste and scientist, it made sense to merge his livelihood with his passion.
Shimamura has coined the term “psychocinematics” to describe the cognitive aspects of the movie-going experience. He has a blog and a new book, Psychocinematics: Exploring Cognition at the Movies (Oxford University Press, 2013). For anyone who revels in the smell of popcorn and the roar of the Dolby sound system, it’s fascinating stuff.
Shimamura says there’s a reason you feel transported by cinema: Your cortex—the portion of the brain that controls memory, emotion, language, vision, and decision-making—is jacked up. As he put it in his blog, “movies are extraordinarily potent stimuli in directing our attention.”
Using functional magnetic resonance imaging (fMRI), researchers can monitor brain activity in real time while subjects view movies. Shimamura credits the work of neuroscientist Uri Hasson with some particularly interesting findings regarding the “neural underpinnings of our movie experience.”
While there may not be real magic at the movies, there are real magicians—illusionists, anyway. They’re called film editors, and their work is the focal point of Shimamura’s current research. It’s his view that they, not the directors, are the true filmmakers, the people responsible, he said, “for both moving us through time and space—and simply moving us.”
Film editors rely on the brain’s quirks to work their sleight of hand, said Shimamura. For instance, they rely on the brain’s tendency to interpret still images flashing along at the rate of 24 to 48 frames per second as movement. Our brains can’t detect the gaps.
Editors also manipulate our perceptions through cutting techniques, said Shimamura. “They were doing this almost a hundred years ago. Through trial and error, they discovered editing techniques that trick the brain. It is the thousands of seamless cuts in a film that create these rich, powerful illusions.”
One technique is the match-action edit. As an example, Shimamura described a scene of a person drinking from a mug. If the film jumps to a close-up of the person drinking, our minds don’t register the actual cut. “Because the editor cut at the precise instant of movement,” he said, “and most people don’t see the cut [during an action], even when they’re told it’s coming.”
Shimamura cited an experiment by a psychologist in which participants watched a movie and marked each cut. “On average, they missed 15 percent of the time,” he said. “But during an action (such as a cough or bodily movement), they missed one-third of the time. Some editors will cut when the actor blinks, because they know that tends to make the viewer blink—so they’ll miss the cut.”
Film editors also exploit the natural tendency of the eye to rest on a certain spot or object, jump around, then rest on another point. Such “saccadic eye movements” are extremely rapid—about 3/1,000th of a second—and can be manipulated in film to focus attention on a particular character or object crucial to the narrative, intensifying the viewer’s experience.
A prime example of this technique occurs in the opening sequences of Stanley Kubrick’s film 2001: A Space Odyssey, wrote Shimamura by email. “That’s where an ape throws a bone in the air and as it’s spinning there’s a cut to a space satellite with the same shape and orientation of the just-seen bone.”
Oddly, the way our brains interpret real-life action is similar to stop-action movies. Due to saccadic eye movements, our focus fixates on a point, then moves around to fixate elsewhere (about three times a second). “Each time we do so, it is as if we capture a single-shot frame of the world,” Shimamura writes in his blog. He explains that psychocinematics explores how movies work from a psychological and biological perspective—the way they seize our senses, tell a story, and ultimately elicit emotional responses.
“[The science allows us to] use motion pictures … to understand how the mind/brain interprets the world around us—that is, how moving pictures can simulate the way we sense our environment much better than the typical research stimuli—which are more often words than pictures.”
By using moving images as test stimuli, Shimamura writes, scientists “get a better handle on how we perceive and remember the real world. In this way, movies can help us understand how the brain works.”
