Radar Perene / Archive / science

Survivorship bias: why 'winning' track records deceive

◦ Index methodology v2.2 (working papers with DOI). See the methodology.

Science

In the Second World War, the planes that returned from missions came back riddled with bullet holes in the wings and fuselage — and command wanted to armor exactly those spots. The statistician Abraham Wald replied with the observation that became a classroom staple: the visible holes marked where a plane could be hit and still come home. The spots that deserved armor were the ones that appeared intact on the survivors — because planes hit there never returned to be examined.

Survivorship bias is the error of drawing conclusions only from what survived to be observed — funds that did not close, companies that did not delist, series that were not discontinued — as if the vanished had never existed. The sample looks complete; what is missing is precisely the half that carries the lesson.

Eighty years later, financial markets repeat the air command's error on an industrial scale — and almost always without a Wald nearby.

Where the vanished hide

The bias operates in silence because absence leaves no trace in the spreadsheet. Three examples suffice.

The average return of a fund category, computed over the funds that exist today, excludes by construction all those that closed along the way — and funds close, frequently, for poor performance. The average of the living is systematically prettier than the average of the class that started.

The portfolio of "the decade's best stocks" suffers the same defect in reverse: picking today's winners and measuring the return they would have delivered guarantees the result before the test. Nobody, ten years ago, had the list of survivors.

And the stock index, benchmark of nearly every backtest, is itself a committee of survivors: delisted, merged or demoted companies leave the basket and the memory. A test run on the index's current composition, extended into the past, tests a portfolio that never existed.

There is also the deliberate version of the bias, bordering on a con: financial folklore records the newsletter scheme that mails opposite predictions to two groups of readers, discards the half that got it wrong, and repeats the cut for several rounds — at the end, a small group has received a perfect streak of correct calls and never learns it was selected, not informed. No serious manager operates this way; but every piece of material that displays only winners reproduces, in homeopathic doses, the same arithmetic.

Why the problem is bigger in Brazil

In large markets, the universe of stocks is wide enough to dilute a few disappearances. The Brazilian case is less comfortable: the set of companies effectively tradable on the B3 is small and has changed composition several times over recent decades — waves of IPOs, waves of delistings, long stretches of thinning out. Any two-decade study crosses, in practice, different universes under the same name.

The consequence for the research reader is direct: in a small, shifting sample, each survivor weighs more, and the bias that would be a rounding error in New York becomes a distortion in São Paulo. That is why the house treats the definition of the universe as a methodological decision, not a footnote: its studies declare which set of assets enters, over which period, under which inclusion rule — before any conclusion. The entry on market breadth shows that concern in operation: measuring how many assets take part in a move only makes sense if the denominator — who existed and traded on that date — is honest.

Rebuilding series from raw data, with each date's universe as it actually was, is expensive in time and invisible in the final product. The reader cannot tell the difference between a study that did this work and one that downloaded the survivors' ready-made series. Until the day they can.

The ruler for reading any "winning" series

Faced with any impressive track record — of a fund, a strategy, a portfolio — Wald's question remains the complete toolkit: who did not come back from the mission? How many of this manager's funds closed? How many stocks in this screen left the exchange along the way? How many sibling strategies were tested and abandoned?

When the material does not answer these questions, the omission is the answer: the track record describes the survivors, and about the vanished it knows nothing — nor should the reader pretend to.

Frequently asked questions

Is survivorship bias the same as p-hacking?

Relatives, not twins. P-hacking selects among tests; survivorship bias selects among data. In both cases, what disappears from the report is what would change the conclusion.

Are stock indices useless for research, then?

No — they are useful as long as they are treated as what they are: portfolios with entry and exit rules, not neutral portraits of the market. The error is projecting today's composition onto the past.

How is the bias corrected in practice?

With databases that preserve the dead (delisted assets, closed funds) and with the golden rule of rebuilding each date with that date's universe. It is laborious; that is exactly why it is so rarely done.

Does the bias only affect backtesters?

It affects any retrospective reading — including the biographies of celebrated investors: for every famous trajectory, there was a crowd of trajectories identical in method that ended in anonymity, outside the statistics.

---

If data can deceive through what is missing, the next step on this trail asks who checks the one who writes: What is peer review.

House reading: each date's declared universe shows in today's reading, in the Diário, and the memory that includes the vanished lives among the precedents, in the Atlas.

Checking whether a specific series carries this defect — and how much it changes the conclusion — is bench work, the kind the house does on commission.