August 10, 2011

The Wonderlic makes no sense. So what does?

Via Dan Peterson

There is a very interesting article in the upcoming ESPN magazine by Jonah Lehrer about the Wonderlic, and why on earth NFL teams insist on continuing to use it. The article is interesting, and addresses the fundamental silliness of using the wonderlic as a measure of a quarterback’s fitness for pro ball. This is, in essence, because the wonderlic tests a type of intelligence and decision making that is so different from what happens on the field that it is basically useless. (Check out some sample Wonderlic questions here.)

Ryan Leaf's Wonderlic: 27; Dan Marino's: 16

Ryan Leaf's Wonderlic: 27; Dan Marino's: 16

From the ESPN Article:

Consider a recent study by economists David Berri and Rob Simmons. While they found that Wonderlic scores play a large role in determining when QBs are selected in the draft — the only equally important variables are height and the 40-yard dash — the metric proved all but useless in predicting performance. The only correlation the researchers could find suggested that higher Wonderlic scores actually led to slightly worse QB performance, at least during rookie years. In other words, intelligence (or, rather, measured intelligence), which has long been viewed as a prerequisite for playing QB, would seem to be a disadvantage for some guys. Although it’s true that signal-callers must grapple with staggering amounts of complexity, they don’t make sense of questions on an intelligence test the same way they make sense of the football field. The Wonderlic measures a specific kind of thought process, but the best QBs can’t think like that in the pocket. There isn’t time.

Lehrer posits that, rather than analytical decision making, quarterbacks use a kind of quick, emotional decision making. This is an interesting idea, if unproven. Lehrer is making a pretty big leap in the research he cites to support the point, as it is based on people picking stocks, which is obviously a much different, and slower process than what happens on a football field.

So how, then, do they make their decisions? Turns out, every pass play is a pure demonstration of human feeling. Scientists have in recent years discovered that emotions, which are often dismissed as primitive and unreliable, can in fact reflect a vast amount of information processing. In many instances, our feelings are capable of responding to things we’re not even aware of, noticing details we don’t register on a conscious level. Let’s say you’re given information about how 20 different stocks have performed over a period of time. (Their share prices are displayed on a ticker at the bottom of a TV screen.) If somebody asks you which stocks performed best, you’ll probably be unable to give a good answer; there’s just way too much financial data to keep track of. But if you’re asked which stocks trigger the best feelings — now it’s your emotional brain that’s being quizzed — you’ll suddenly be able to identify the top stocks. According to Tilmann Betsch, the psychologist who performed this experiment, your emotions will “reveal a remarkable degree of sensitivity” to the actual performance of the shares. The investments that rose in value will be associated with the most positive emotions, while those that fell will trigger a vague sense of unease.

This exercise captures why it’s so important for quarterbacks to rely on their feelings and not their analytical intelligence.

Lehrer also summarizes some very interesting research that has been performed around “grit” as a predictor of future success, arguing that time and experience form the foundation for the high-speed decision making that quarterbacks engage in, and so it is grit that teams ought to be looking for in their potential quarterbacks.

This notion of practice led Ericsson to collaborate with Angela Duckworth, a psychologist at the University of Pennsylvania. Duckworth is best known for her work on grit, a character trait that allows people to persist in the face of difficulty. A few years ago, she was commissioned by the Army to measure the grittiness of cadets at West Point. Although the academy is highly selective, about 5 percent of cadets drop out after the first summer of training, known as Beast Barracks. The Army has long searched for the variables that predict which cadets will graduate, but it wasn’t until Duckworth tested them using a short questionnaire — consisting of statements such as “Setbacks don’t discourage me” or “I am diligent” — that the Army found a measurement that actually worked. Duckworth has since repeated the survey with subsequent West Point classes, and the results are always the same: The cadets who graduate are the ones with grit.

It’s a good article and worth a complete read. Lehrer exhibits a kind Malcolm Gladwell-style populism–one that argues against the idea of talent and innate ability in favor of diligence and hard work. There is obviously a lot about this that is true. It does maybe discount genius a little bit too much. Two athletes who put in the exact same amount of deliberate practice are probably going to end up differing in their ability to make the high-speed decisions that their game requires. If this were true, the correlation between years of experience and quarterback performance ought to be better, and we should rarely see quarterbacks do well in their first few years in the league (in fact quarterbacks tend to peak in the mid-late 20’s, younger than you would expect if the position was only about experience). The article’s line of thinking possibly underestimates the role of a specific kind of sports intelligence, maybe because it is harder to study than things like grit and practice time. Just because we haven’t pinpointed exactly how to measure something, doesn’t mean that it doesn’t exist.

Tags:, , ,

2 Responses to The Wonderlic makes no sense. So what does?

  1. Pingback: The subtle indicators of NFL success | Axon Potential

  2. Pingback: The 10,000 hour rule and expert athletic performance | Axon Potential

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Planets Twitter Rss Facebook