Radar Perene / Archive / science

Publishing your own hit rate, even when it is not good

◦ Index methodology v2.2 (working papers with DOI). See the methodology.

Science

Notice what market analysis material tends to display: the call that anticipated the drop, the study that "predicted" the turn, the chart where the analyst's line hugs the world's line. Now notice what it never displays: the complete list. How many calls were made in total, how many missed, where the misses are. The showcase displays trophies; the accounting would display the entire season — and that is exactly why the accounting never appears.

This house keeps, as permanent editorial policy, the principle of publishing its own hit rate — the complete record of readings confronted with what actually happened — including in periods when the result does not flatter it. The specific number of any period matters less than the rule: the account is published because it was computed before knowing whether it would be flattering.

Why almost no one does this

It is not laziness — it is incentive. Whoever lives from selling conviction cannot afford to display their own error margin, because the error margin of any honest market reading is large. The result is a curious equilibrium: the whole industry errs routinely, everyone knows everyone errs, and yet the published material suggests infallibility by omission. The reader who wants to know the hit rate of whoever advises them simply cannot find the number — not because it does not exist, but because computing it and hiding it takes less effort than publishing it.

Showcase selection has well-studied formal relatives: it is survivorship bias applied to self-promotion. Among the red flags of financial pseudoscience, the exclusive display of hits is perhaps the most reliable — not because hitting is suspicious, but because the absence of the complete list is a choice, and choices inform.

What the policy demands — and what it costs

Publishing the whole rate changes the relationship with one's own error. The error stops being an accident to manage and becomes a line in the books: predictable, expected, recorded. This has a disciplining effect that precedes publication — whoever knows the account will be rendered avoids the showy call that would not survive the counting. The publishing policy works, in practice, as a brake on what gets claimed: every reading is written knowing it will enter the statistic.

The cost is evident and the house accepts it with open eyes: bad periods stay exposed, and a hurried reader may mistake the honesty of the account for the incompetence of whoever renders it. The bet is the reverse — that the reader who matters knows an imperfect, published hit rate informs more than an infallibility suggested by omission. Whoever promises the whole account cannot promise it will be pretty; they can promise it will be rendered.

The structural contrast, with no named villains

None of this requires bad faith on the other side. The analyst at an institution that distributes products operates under incentives that make the showcase rational — the subject has its own article in conflict of interest in bank research. The point here is not to accuse; it is to note that a published hit rate is a cheap test the reader can apply to any source: whoever renders the whole account accepted a cost; whoever does not, avoided it. Neither choice is neutral, and the difference between them does not depend on trusting anyone — it depends on looking for the list.

There is also an archive effect: the rate published period after period composes a series of its own, and the series informs more than any isolated number — it shows whether the operation improves, worsens or merely sways with the environment. One isolated bad period says little; a continuous accounting no one edited says much, including about the disposition of whoever keeps it.

Frequently asked questions

Where is the house's actual hit-rate number?

In the periodic accountability of the editorial production itself, confronting recorded readings with outcomes — not in this article, which is about the principle. Publishing the rule in one text and the number in another is deliberate: the rule is permanent; the number belongs to each period.

Doesn't a low hit rate demoralize whoever publishes it?

It depends on the ruler. In market readings, modest rates are the trade's normal state — the literature on forecasting in noisy environments has documented this for decades. What demoralizes is not the rate; it is the gap between the real rate and the infallibility the showcase suggests.

Is hit rate the right metric to evaluate analysis?

It is one among several, with known limits: hits and misses do not weigh the same, and a reading can be useful without being a verifiable "call". The house's policy is to publish the account with those caveats declared — what does not stand is the absence of an account.

How does a reader apply this test to other sources?

By looking for the complete list: the entire history of calls, dated, confronted with outcomes. If the material only offers highlights, the answer has already arrived — by omission.

---

Continue the trail: Reproducibility in action: what to do when a central statistic does not repeat →

House reading: today's reading is in the Diário; the archived episodes, in the Atlas.

Reading a third-party report with this ruler — the complete list against the showcase — is an exercise the house does in conversation, over concrete material.