THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS - By Nassim Nicholas Taleb

Statistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of risk-taking; it is the applied tools of epistemology; you can't be a modern intellectual and not think probabilistically—but... let's not be suckers. The problem is much more complicated than it seems to the casual, mechanistic user who picked it up in graduate school. Statistics can fool you. In fact it is fooling your government right now. It can even bankrupt the system (let's face it: use of probabilistic methods for the estimation of risks did just blow up the banking system).
-

The current subprime crisis has been doing wonders for the reception of any ideas about probability-driven claims in science, particularly in social science, economics, and "econometrics" (quantitative economics). Clearly, with current International Monetary Fund estimates of the costs of the 2007-2008 subprime crisis, the banking system seems to have lost more on risk taking (from the failures of quantitative risk management) than every penny banks ever earned taking risks. But it was easy to see from the past that the pilot did not have the qualifications to fly the plane and was using the wrong navigation tools: The same happened in 1983 with money center banks losing cumulatively every penny ever made, and in 1991-1992 when the Savings and Loans industry became history.
-

It appears that financial institutions earn money on transactions (say fees on your mother-in-law's checking account) and lose everything taking risks they don't understand. I want this to stop, and stop now—the current patching by the banking establishment worldwide is akin to using the same doctor to cure the patient when the doctor has a track record of systematically killing them. And this is not limited to banking—I generalize to an entire class of random variables that do not have the structure we think they have, in which we can be suckers.
-

And we are beyond suckers: not only, for socio-economic and other nonlinear, complicated variables, are we are riding in a bus driven by a blindfolded driver, but we refuse to acknowledge it in spite of the evidence, which to me is a pathological problem with academia. After 1998, when a "Nobel-crowned" collection of people (and the crème de la crème of the financial economics establishment) blew up Long Term Capital Management, a hedge fund, because the "scientific" methods they used misestimated the role of the rare event, such methodologies and such claims on understanding risks of rare events should have been discredited. Yet the Fed helped their bailout and exposure to rare events (and model error) patently increased exponentially (as we can see from banks' swelling portfolios of derivatives that we do not understand).

...

What Is Fundamentally Different About Real Life

-

My anger with "empirical" claims in risk management does not come from research. It comes from spending twenty tense (but entertaining) years taking risky decisions in the real world managing portfolios of complex derivatives, with payoffs that depend on higher order statistical properties —and you quickly realize that a certain class of relationships that "look good" in research papers almost never replicate in real life (in spite of the papers making some claims with a "p" close to infallible). But that is not the main problem with research.

-

For us the world is vastly simpler in some sense than the academy, vastly more complicated in another. So the central lesson from decision-making (as opposed to working with data on a computer or bickering about logical constructions) is the following: it is the exposure (or payoff) that creates the complexity —and the opportunities and dangers— not so much the knowledge ( i.e., statistical distribution, model representation, etc.). In some situations, you can be extremely wrong and be fine, in others you can be slightly wrong and explode. If you are leveraged, errors blow you up; if you are not, you can enjoy life.
-

So knowledge (i.e., if some statement is "true" or "false") matters little, very little in many situations. In the real world, there are very few situations where what you do and your belief if some statement is true or false naively map into each other. Some decisions require vastly more caution than others—or highly more drastic confidence intervals. For instance you do not "need evidence" that the water is poisonous to not drink from it. You do not need "evidence" that a gun is loaded to avoid playing Russian roulette, or evidence that a thief [is] on the lookout to lock your door. You need evidence of safety—not evidence of lack of safety— a central asymmetry that affects us with rare events. This asymmetry in skepticism makes it easy to draw a map of danger spots.

...

Bottom Line: The Map

-
Things are made simple by the following. There are two distinct types of decisions, and two distinct classes of randomness.

-
Decisions: The first type of decisions is simple, "binary", i.e. you just care if something is true or false. Very true or very false does not matter. Someone is either pregnant or not pregnant. A statement is "true" or "false" with some confidence interval. (I call these M0 as, more technically, they depend on the zeroth moment, namely just on probability of events, and not their magnitude —you just care about "raw" probability). A biological experiment in the laboratory or a bet with a friend about the outcome of a soccer game belong to this category.

-

The second type of decisions is more complex. You do not just care of the frequency—but of the impact as well, or, even more complex, some function of the impact. So there is another layer of uncertainty of impact. (I call these M1+, as they depend on higher moments of the distribution). When you invest you do not care how many times you make or lose, you care about the expectation: how many times you make or lose times the amount made or lost.
-

Probability structures: There are two classes of probability domains—very distinct qualitatively and quantitatively. The first, thin-tailed: Mediocristan", the second, thick tailed Extremistan. Before I get into the details, take the literary distinction as follows:

-

In Mediocristan, exceptions occur but don't carry large consequences. Add the heaviest person on the planet to a sample of 1000. The total weight would barely change. In Extremistan, exceptions can be everything (they will eventually, in time, represent everything). Add Bill Gates to your sample: the wealth will jump by a factor of >100,000. So, in Mediocristan, large deviations occur but they are not consequential—unlike Extremistan.
-

Mediocristan corresponds to "random walk" style randomness that you tend to find in regular textbooks (and in popular books on randomness). Extremistan corresponds to a "random jump" one. The first kind I can call "Gaussian-Poisson", the second "fractal" or Mandelbrotian (after the works of the great Benoit Mandelbrot linking it to the geometry of nature). But note here an epistemological question: there is a category of "I don't know" that I also bundle in Extremistan for the sake of decision making—simply because I don't know much about the probabilistic structure or the role of large events.

-

The Map
-

Now let's see where the traps are:

-

First Quadrant: Simple binary decisions, in Mediocristan: Statistics does wonders. These situations are, unfortunately, more common in academia, laboratories, and games than real life—what I call the "ludic fallacy". In other words, these are the situations in casinos, games, dice, and we tend to study them because we are successful in modeling them.
-

Second Quadrant: Simple decisions, in Extremistan: some well known problem studied in the literature. Except of course that there are not many simple decisions in Extremistan.

-

Third Quadrant: Complex decisions in Mediocristan: Statistical methods work surprisingly well.

-

Fourth Quadrant: Complex decisions in Extremistan: Welcome to the Black Swan domain. Here is where your limits are. Do not base your decisions on statistically based claims. Or, alternatively, try to move your exposure type to make it third-quadrant style ("clipping tails").

-

....................

-

Related post: A focus on the exceptions that prove the rule

-

Related article: Shattering the Bell Curve

-

Related books:

-

The Black Swan: The Impact of the Highly Improbable

-

Fooled by Randomness: The Hidden Role of Chance in Life and in the Markets

-

The Misbehavior of Markets: A Fractal View of Risk, Ruin & Reward

-