Cook’s Theory of Performance Evaluation

The ideas presented here evolved from a post titled “Evaluate people at their best or their worst?” on John Cook’s blog. In order to make this post a little tighter, I’ll refer to John’s ideas as “Cook’s Theory of Performance Evaluation” and describe it as follows.

John identifies three ways a person’s performance can be evaluated:

  1. How good are they at their worst?
  2. How good are they on average?
  3. How good are they at their best?

Cook observes that schools evaluate performance using the first two benchmarks and markets use the third benchmark. To illustrate, consider the following assignment grades for a student in a hypothetical course:

Assignment Score
1 100
2 90
3 92
4 87
5 100
6 45
7 90
8 100
9 95
10 100
Average: 89.9
Result: B+

Graphically, this looks like:

Academic Performance EvaluationThe student’s worst performance pulls the grade average down and results in a B+ for the course. Performance evaluation in markets, however, is only interested in how well you do, that is, your best. Consider the following sales volumes for a fictional author for each of ten books:

Book Copies Sold
1 1,000
2 2,500
3 900
4 1,100
5 3,400
6 1,000,000
7 42,000
8 6,500
9 2,750
10 3,100
Result: Bestseller!

Graphically, this looks like:

Market Performance EvaluationNumber 6 must have been some story. But as they say, you can’t argue with success and this author will forever be known as a bestseller. Subsequent flops won’t change that.

So there you have Cook’s Theory of Performance Evaluation. The consequences of this theory when played out in real life are noted by Cook:

We all want others to see the best in us. There are ethical and economic reasons to look for the best in others. But years of education can incline us to look for the worst in others and in ourselves.

Another point that can be made is that in school, everyone starts out with a perfect score that for most students erodes as the class progresses. In markets, everyone starts out with essentially a zero score that for most entrepreneurs improves over time, commensurate with the individual’s effort. Money, of course, is another way to keep score in market-based performance evaluations.

If education has conditioned us to look for the worst in others and ourselves, it has also conditioned us to become demoralized when encountering even the slightest failure that diminishes our chances at succeeding. Once lost, the perfect score can never be regained, so we settle for less. The greater the failure, the less we must settle for.

Moreover, we are conditioned that we can never exceed the highest possible achievement as defined by academia. The best we can do is match it. Most come up short. This conditioning is difficult to shake and in my own experience took several years after obtaining my undergraduate degrees. Nothing like 100+ job rejection letters to cause one to reevaluate the nature and size of the door opened by a couple of college degrees.

There are other ways to evaluate an individual’s performance.

  1. How good are they compared to others (past and present)?
  2. How good are they compared to themselves in the past?
  3. How good are they compared to their personal criteria and expectations?
  4. How good are they compared to the criteria and expectations of others?

The answer to each of these questions can be radically different depending on the referential index of the questioner. “How good am I when compared to others?” is significantly different from “How good is he/she when compared to others?”

The answers to each of these questions can also be significantly influenced by various biases and prejudices. Confirmation bias, hindsight bias, self-serving bias, the Dunning–Kruger effect, the misinformation effect, self-handicapping, self-fulfilling prophecies, introspection illusion, groupthink, the affect heuristic – numerous ways an individual can skew the evaluation of their own performance.

When the performance evaluation comes from a third party, for example a university professor evaluating a student’s performance, there are a different combination of biases in play which can have an independent impact on the performance score. The fundamental attribution error, confirmation bias, the illusion of transparency, credentialism or argument from authority – more ways the individual’s eventual performance score can can be unconsciously influenced. The combination of unconscious incompetence and the Dunning–Kruger effect can have a particularly adverse effect on the student who asks questions that expose a professor’s incompetence.

Here again, the level playing field appears to be with market-based performance evaluations. An individual’s ability to understand and mitigate biases and prejudices affecting their success will have a direct impact on their performance in the market. Students, however, have less influence over these drivers when they are manifest in individuals working from a position of authority.