Illusion of Validity
Type: Cognitive Bias
Local HTML: illusion_of_validity.html
Definition
Confidence in judgments exceeds their actual accuracy. We feel certain even when we’re wrong — especially with more information.
Kahneman’s insight: More information doesn’t improve accuracy, just confidence. A simple algorithm with 3-4 variables often beats expert judgment.
Why It Matters
Expert overconfidence: Doctors, judges, investors — all more confident than accurate. Information overload: More data ≠ better decisions, just more certainty. Algorithms vs humans: Simple models consistently outperform expert intuition. Planning: Detailed plans feel more valid, fail just as often.
The Paradox
| More Information | Result |
|---|---|
| Accuracy | Flat or slightly worse |
| Confidence | Skyrockets |
We build elaborate stories that feel compelling but aren’t predictive.
Examples
- Clinical judgment — Doctors confident in diagnoses; algorithms more accurate
- Hiring — Interviews create confidence, don’t predict performance
- Investing — Detailed analysis feels valid; index funds win
- Political forecasting — Experts no better than dart-throwing chimps
Fighting It
- Use algorithms — Simple models beat expert judgment
- Track accuracy — Compare predictions to outcomes
- Limit information — Too much data hurts
- Calibrate confidence — “How often am I this confident and right?”
Related Biases
- [[Overconfidence Effect** — Broader overconfidence
- [[Dunning-Kruger Effect** — Incompetence + confidence
- [[Hindsight Bias** — “I knew it all along”
Audio
Podcast episode: Illusion of Validity
Part of the Cognitive Bias Reference