About the Author xi
Preface xiii
Acknowledgments xvii
Part One An Introduction To The Crisis 1
Chapter 1 Healthy Skepticism for Risk Management 3
A "Common Mode Failure" 5
Key Definitions: Risk Management and Some Related Terms 8
What Failure Means 14
Scope and Objectives of This Book 17
Chapter 2 A Summary of the Current State of Risk Management 21
A Short and Entirely-Too-Superficial History of Risk 21
Current State of Risk Management in the Organization 25
Current Risks and How They are Assessed 26
Chapter 3 How Do We Know What Works? 35
Anecdote: The Risk of Outsourcing Drug Manufacturing 36
Why It's Hard to Know What Works 40
An Assessment of Self-Assessments 44
Potential Objective Evaluations of Risk Management 48
What We May Find 57
Chapter 4 Getting Started: A Simple Straw Man Quantitative Model 61
A Simple One-for-One Substitution 63
The Expert as the Instrument 64
A Quick Overview of "Uncertainty Math" 67
Establishing Risk Tolerance 72
Supporting the Decision: A Return on Mitigation 73
Making the Straw Man Better 75
Part Two Why It's Broken 79
Chapter 5 The "Four Horsemen" of Risk Management: Some (Mostly) Sincere Attempts to Prevent an Apocalypse 81
Actuaries 83
War Quants: How World War II Changed Risk Analysis Forever 86
Economists 90
Management Consulting: How a Power Tie and a Good Pitch Changed Risk Management 96
Comparing the Horsemen 103
Major Risk Management Problems to Be Addressed 105
Chapter 6 An Ivory Tower of Babel: Fixing the Confusion about Risk 109
The Frank Knight Definition 111
Knight's Influence in Finance and Project Management 114
A Construction Engineering Definition 118
Risk as Expected Loss 119
Defining Risk Tolerance 121
Defining Probability 128
Enriching the Lexicon 131
Chapter 7 The Limits of Expert Knowledge: Why We Don't Know What We Think We Know about Uncertainty 135
The Right Stuff: How a Group of Psychologists Might Save Risk Analysis 137
Mental Math: Why We Shouldn't Trust the Numbers in Our Heads 139
"Catastrophic" Overconfidence 142
The Mind of "Aces": Possible Causes and Consequences of Overconfidence 150
Inconsistencies and Artifacts: What Shouldn't Matter Does 155
Answers to Calibration Tests 160
Chapter 8 Worse Than Useless: The Most Popular Risk Assessment Method and Why It Doesn't Work 163
A Few Examples of Scores and Matrices 164
Does That Come in "Medium"?: Why Ambiguity Does Not Offset Uncertainty 170
Unintended Effects of Scales: What You Don't Know Can Hurt You 173
Different but Similar-Sounding Methods and Similar but Different-Sounding Methods 183
Chapter 9 Bears, Swans and Other Obstacles to Improved Risk Management 193
Algorithm Aversion and a Key Fallacy 194
Algorithms versus Experts: Generalizing the Findings 198
A Note about Black Swans 203
Major Mathematical Misconceptions 209
We're Special: The Belief That Risk Analysis Might Work, but Not Here 217
Chapter 10 Where Even the Quants Go Wrong: Common and Fundamental Errors in Quantitative Models 223
A Survey of Analysts Using Monte Carlos 224
The Risk Paradox 228
Financial Models and the Shape of Disaster: Why Normal Isn't So Normal 236
Following Your Inner Cow: The Problem with Correlations 243
The Measurement Inversion 248
Is Monte Carlo Too Complicated? 250
Part Three How to Fix It 255
Chapter 11 Starting with What Works 257
Speak the Language 259
Getting Your Probabilities Calibrated 266
Using Data for Initial Benchmarks 272
Checking the Substitution 280
Simple Risk Management 285
Chapter 12 Improving the Model 293
Empirical Inputs 294
Adding Detail to the Model 305
Advanced Methods for Improving Expert's Subjective Estimates 312
Other Monte Carlo Tools 315
Self-Examinations for Modelers 317
Chapter 13 The Risk Community: Intra- and Extra-organizational Issues of Risk Management 323
Getting Organized 324
Managing the Model 327
Incentives for a Calibrated Culture 331
Extraorganizational Issues: Solutions beyond Your Office Building 337
Practical Observations from Trustmark 339
Final Thoughts on Quantitative Models and Better Decisions 341
Additional Calibration Tests and Answers 345
Index 357
