The Science of Comic Strips

Language is more than just a series of words strung together. A sentence must have some essential structure, some system of rules governing words and clauses--a grammar. You don't have to be Strunk or White to recognize this system at work; it's automatic in the brain. In Noam Chomsky's famous example, people know that the meaningless sentence "colorless green ideas sleep furiously" is grammatically correct the very first time they see it.

Psychologist Neil Cohn of University of California at San Diego believes comic strips operate the same way. Far from just a series of panels strung together, comics represent to Cohn a coherent, complex "visual language." His research over the years--culminating in a new book called The Visual Language of Comics (Bloomsbury)--suggests that our brains register the presence of a grammatical system in the funny pages just as they would in a book's pages.

"We would expect that the brain would treat, say, grammars of different systems in similar ways," Cohn tells Co.Design. "That's kind of the motivating assumption."

Cohn says any language has a "holy triumvirate" of elements: expressive form, grammar, and meaning. Comics, he argues, meet all three requirements. Their expressive form is the visual strip. Their grammatical structure consists of a basic vocabulary (such as stink lines or speech bubbles) and a syntax (a hierarchical panel structure). And, when done right, the images have a semantic relationship--a clear message.

“AT A STRICTLY COGNITIVE LEVEL, PEOPLE TREAT COMIC STRIPS AS A NORMAL SENTENCE.”

The panel structure that forms the basis of comic grammar deserves a bit more explanation. Take the following six-panel Peanuts strip from one of Cohn's research papers. At the top of the hierarchy is a narrative arc, more or less equivalent to a sentence. From there it's broken down into "initial" and "peak" sequences that behave a little like clauses. These then break down into various components somewhat analogous to nouns, verbs, and other parts of speech.

"Comics end up being visual languages because they obey these properties," says Cohn. "They have a sequence, and they also have a systematic ways of drawing things."

Some of the best evidence that comics function as a visual language was published in a 2012 issue of Cognitive Psychology. For the study, Cohn and his collaborators showed test participants four types of comics strips. One was a "normal" strip that represented a meaningful, grammatical sentence. A second strip had panels with a semantic relationship but no grammatical structure. A third strip was grammatical but meaningless, in the spirit of Chomsky's "colorless green ideas sleep furiously." The fourth was a scrambled mess.

Sitting at a computer screen, the study participants were shown a "target" panel to identify in a strip. They were then shown the whole strip and had to push a button when the target appeared. The study was designed to echo a famous sentence structure experimentconducted in the 1970s, which found that people process "normal" sentences more quickly than ones that violate linguistic rules. Cohn wanted to see whether his participants reacted to comics the same way.

Sure enough, they did. Cohn and his fellow researchers found that reaction times to the target panel were fastest in the "normal" sequence. Target reactions in the semantic- and syntax-only sequences, shown second and third above, were slower than normal (though similar to one another). Reactions in the scrambled strips were slowest of all. At a strictly cognitive level, people did seem to treat normal comic strips as a normal sentence.

“A LOT OF THE COMPLEXITY OF COMICS IS NOT RECOGNIZED.”

In other tests, Cohn dug deeper into the brain. Using electrode caps, he measured the neural responses of test participants as they read various types of comics. He found that participants reading a "normal" comic strip showed the same brain activity that people show when processing semantics in standard sentences. He also found that participants reading an abnormal comic--with a blank panel inserted mid-strip, to interrupt the underlying structure--showed the same brain activity that people show when registering grammatical violations.

"I'd say a lot of the complexity [of comics] is not recognized--certainly not the idea of grammar," says Cohn. "That is controversial, and people often are very surprised by the results that I show."

Cohn's research probably says more about the brain than it does about comics themselves, but it does have creative applications. Understanding the "visual language" of comics should help artists draw strips with more engaging narratives. The work also suggests that young artists who want to learn this language would be better off buying a book on how to draw comics (the equivalent of a Rosetta Stone program) than an actual comic book (the equivalent of a novel).

“UNDERSTANDING THE VISUAL LANGUAGE OF COMICS COULD HELP ARTISTS DRAW STRIPS WITH BETTER NARRATIVES.”

Before becoming a research scientist, Cohn was a professional comics artist. As a kid he spent lots of time at Comic-Cons, and he's collaborated on full-length graphic novels. This background as a native comics speaker, if you will, gives Cohn a leg up in the laboratory. He can create panel sequences that reflect proper semantics and syntax, and therefore create studies that measure true linguistic understanding, whereas other researchers might just be stringing panels together. "The fact that I was a comic creator, and have been doing this for so long, means that I'm fluent in the visual language," he says.

Source:
http://www.fastcodesign.com/3023074/evidence/the-science-of-comic-strips