One of the most difficult things about doing animal cognition research is that it’s impossible to get reflective feedback from animals. We can’t ask them, “What strategy did you use to figure out this problem?” or “Why did you come up with this particular answer?” This kind of feedback could be useful because, in addition to investigating whether an animal possesses a particular cognitive ability, researchers must also verify that the ability that an animal appears to demonstrate actually is the ability in question.
Perhaps the most famous example of the importance of this is the case of Clever Hans. In the early 1900s, a teacher named Wilhelm von Osten claimed to have taught his horse, Clever Hans, to do arithmetic and tell time, among other cognitive tasks. von Osten would pose a problem to Clever Hans, who would then tap out the answer with his hoof. The pair traveled around their native Germany, exhibiting Clever Hans’s “abilities”.
Eventually a psychologist, Oskar Pfungst, decided to study Clever Hans. Pfungst found that Clever Hans didn’t possess any of his purported abilities; rather, he relied solely on the body language of von Osten and the audiences he performed for. As Clever Hans’s taps approached the correct answer, the postures of the people around him got tenser and tenser. When he reached the correct answer, the tension was released, visibly changing the peoples’ postures. Clever Hans had learned to stop tapping when he saw this change (which really is clever, just not in the way von Osten thought). Pfungst found that Clever Hans was nowhere near accurate when he couldn’t see anyone as he answered, or when the people around him didn’t know the answer to the question.
Pfungst took his study a step further by investigating whether humans could respond to body language in the same way as Clever Hans. First, he asked subjects to think of a number, which he would then guess by tapping out the answer. Pfungst found that he was much more accurate than chance, even though he only relied on the body language of the subjects. Moreover, when Pfungst and the subjects switched roles, the subjects were also much more accurate than chance when guessing Pfungst’s number. Knowing what he did about body language, Pfungst attempted to remain as still as possible, but he still made involuntary movements that cued the subjects.
This phenomenon, where an experimenter’s involuntary cues influence a subject’s performance, became known as the Clever Hans Effect (or, more generally, the Observer-Expectancy Effect), and has greatly influenced how we design experiments involving both animals and humans. Some ways to prevent the Clever Hans Effect are for the experimenter to be unaware of the correct answers (a “double-blind” study), or to have the experimenter hidden out of sight of the subject. Either way, the experimenter is unable to inadvertently cue the subject to the correct answer.
While the Clever Hans case warns us specifically of the possibility of experimenters cueing subjects, it also shows the general importance of confirming that subjects are actually demonstrating the ability you’re studying, rather than relying on other cues or strategies. This is especially important when working with animals, since we can’t just ask them to describe their cognitive strategies.
Thanks to Project Gutenberg, Pfungst’s original (translated) paper is available for free online. It looks long, but it’s very interesting and readable.
I’ll continue this short series on experimental design next week with a look at how different training procedures can lead to different results. Plus, our first paper about seal cognition!
Pfungst, Oskar. Clever Hans:(the horse of Mr. Von Osten.) a contribution to experimental animal and human psychology. Holt, Rinehart and Winston, 1911.