Information Hazard
Type: Existential & Civilization
Local HTML: information-hazard.html
Definition
Some knowledge is dangerous to know or spread. Information itself can cause harm.
Nick Bostrom’s framework: Ideas can be weapons, risks, or pollutants. Knowing how to build a virus, hack infrastructure, or manipulate masses.
Types
| Type | Description | Example |
|---|---|---|
| Data hazard | Specific data causes harm | Terrorist targets, home addresses |
| Idea hazard | Concept is dangerous | How to build bioweapons |
| Attention hazard | Focusing attention causes harm | Suicide contagion, copycat crimes |
| Knowledge hazard | Knowing changes behavior | Prediction of disaster causing panic |
Why It Matters
Biosecurity: Publishing virus research — help science or enable bioterror? AI: Open-sourcing powerful models — democratize or weaponize? Privacy: Data aggregation enables surveillance. Social: Viral misinformation, manipulation techniques.
The Dilemma
Openness vs Security:
- Science advances through sharing
- But some knowledge shouldn’t be shared
- Who decides? Who guards the guardians?
Dual-use: Most dangerous knowledge is also useful knowledge.
Responses
- Responsible disclosure — Tell authorities, not public
- Differential technological development — Delay dangerous tech, accelerate protective tech
- Informational security — Treat dangerous info like dangerous materials
- Education — Teach recognition, not methods
Related Biases
- [[Availability Heuristic** — Dangerous ideas spread because they’re vivid
- [[Optimism Bias** — “No one would misuse this”
- [[Illusion of Control** — Thinking we can contain information
Audio
Podcast episode: Information Hazard
Part of the Cognitive Bias Reference