Radar Perene / Archive / science

Red flags of financial pseudoscience

◦ Index methodology v2.2 (working papers with DOI). See the methodology.

Science

The most deceptive material in markets does not look like quackery — it looks like science. It has charts, it has percentages with two decimal places, it has the word "proven" and, increasingly, the word "backtest". Modern financial pseudoscience does not deny the scientific method; it wears its costume. And that is exactly why the reader's defense cannot be generic distrust — it needs to be a checklist.

Financial pseudoscience is any material that displays the form of evidence — numbers, charts, historical tests — without the conditions that make evidence reliable: identifiable data, described method, results checkable by third parties and declared limits. The quantitative disguise is not an accident; it is the product.

What follows is the filter the house itself applies before citing any source in its readings. It exists out of internal necessity — a publication that cites contaminated material contaminates itself — and it is described here without names, because the problem is a genre, not an author.

The house checklist

Are the data identifiable? The first question is the cheapest: which series, from which source, over which period. "Analysis of historical data" without those three answers is not analysis — it is assertion. Serious material names the source in a way that lets another researcher download the same file.

Was the period chosen or inherited? A sample window that begins and ends on round dates or at the start of the available series was probably inherited — a good sign. A window that begins at a market bottom and ends at a top was probably chosen, and window choice is the simplest way to manufacture a result without lying about any single number.

Is there an out-of-sample test? Any rule fitted to a historical period does well in that period — that is what "fitted" means. The scientific question is what happens in data the rule has never seen. Material that reports only in-sample performance is reporting the quality of its own fitting, not of the idea.

Are costs in the arithmetic? Brokerage, bid-ask spread, taxes, execution slippage. High-turnover strategies can display handsome curves that real costs would return to the ground. The absence of the word "costs" in material full of percentages is eloquent.

Is the result too good? High returns, tiny maximum drawdowns and clockwork consistency, simultaneously, describe a combination that decades of finance literature have documented as exceedingly rare — including among professionals with staff and infrastructure. Material that displays it casually asks the reader to believe the extraordinary without offering the extraordinary in proof.

Who profits if the reader believes? The last question is not about the chart — it is about the incentive of whoever shows it. A result that concludes exactly what the seller needed concluded deserves the slowest reading of all.

The central case: the course backtest

The example that ties the whole checklist together is a genre, common in promotional material for courses and strategies in Brazil: the backtest displayed as proof that "the rule works". The house sells no course and no strategy — it can describe the mechanism without conflict, and the mechanism deserves description because it is pedagogical.

The typical script fails the checklist in cascade. The data are rarely precisely identifiable; the period is frequently favorable to the rule on display; the out-of-sample test almost never appears; costs are omitted or treated as a detail. But the structural defect precedes all of those: the rule on display is the survivor of a selection the viewer does not see. Whoever built the material tried variants — parameter combinations, windows, indicators — and shows the one that worked in the past. That process, described in detail in this section's articles on overfitting and p-hacking, produces pretty curves with the same reliability with which a lottery produces winners: there is always one, and he never knows why.

That is why the course backtest almost never survives out of sample — not through the necessary bad faith of whoever displays it, but by construction. A rule selected for having won in the past carries the past embedded in itself; the future, which took no part in the selection, owes it nothing. The academic literature that tried to replicate hundreds of published "anomalies" — with declared data and methods, the opposite of the genre described here — has already found high rates of shrinkage or disappearance of effects after publication. What the invisible selection of a commercial product would yield, the reader can estimate.

What this filter is not

The checklist separates evidence from the staging of evidence; it does not separate good managers from bad ones, nor does it predict anyone's performance. Material that passes the six questions can be wrong all the same — honest science is frequently wrong, and this house's archive records its own verdicts that aged badly. The difference is that the honest error is visible and correctable; the staging is built so the error never shows.

Frequently asked questions

Is every backtest pseudoscience?

No — the backtest is a legitimate and central tool of quantitative research. What degrades it is the use: without identifiable data, without an out-of-sample test, without costs and without the count of how many variants were tried, it stops being a test and becomes visual rhetoric.

Isn't material with many numbers more reliable by definition?

Numerical density is not an indicator of reliability — it is an indicator of costume. The useful question is not how many numbers there are, but whether any of them can be checked by the reader.

Why are charts so persuasive?

A rising curve compresses years of supposed performance into a single visual stimulus, and the eye does not audit axes, windows or costs. The chart is the ideal format for transporting a conclusion without transporting its conditions.

Is there a seal or certification that solves the problem?

No seal replaces the six questions. Regulation imposes duties on whoever presents themselves as an analyst or manager, but the genre described here circulates in formats — courses, social networks, communities — that sit outside that perimeter most of the time.

---

The checklist protects against material that imitates a study. The trail's next step is the tool science uses to judge many studies at once — and a curious vacuum in the Brazilian literature: What is a meta-analysis.

House reading: the sources that survive this filter feed today's reading, in the Diário; the episodes in which the archive corrected itself are in the Atlas.

Running a specific piece of material through these six questions, document on the table, is the kind of second opinion the house gives on request.