Radar Perene / Archive / science

What peer review is — and why a refereed study counts for more

◦ Index methodology v2.2 (working papers with DOI). See the methodology.

Science

Two studies reach the reader in the same week and claim opposite things about the same market. Both have charts, both cite data, both sound competent. The reader has no laboratory to redo either one's calculations. What is there to lean on? Science lived with this problem for centuries, and the institutional answer it built is called peer review.

Peer review is the process in which a journal sends a study to other researchers in the same field — generally without revealing who wrote it and without revealing who reviews it — so that they hunt for errors, demand additional tests, and recommend publication or rejection. It is a quality filter applied by competitors; it is not a certificate of truth.

The definition carries the two halves this article develops: why the filter is worth something — and why it is not worth what many people credit it with.

How it works from the inside

The typical route at an economics or finance journal has well-marked stages. The text reaches the editor, who first decides whether it even joins the queue — a good share is returned at this initial screening, without referee reports. What passes goes to two or three referees: researchers in the field, unpaid, who sign anonymous reports pointing out methodological flaws, requesting missing tests, questioning interpretations.

The first answer is rarely "yes". The common outcome is a revision request: the author reworks the paper, replies point by point, and the text returns to the same referees — sometimes for more than one round. The whole process is measured in months, frequently in more than a year. Double anonymity, where adopted, has a precise function: the report should aim at the text, not at the prestige or obscurity of whoever wrote it.

It is a laborious, slow and imperfect mechanism — and it remains the most reliable one the scientific community has found to separate an examined argument from a merely published one.

What the stamp guarantees — and what it does not

A refereed study counts for more for a simple reason: people with the competence and the incentive to bring it down tried, and failed. The reader without a laboratory outsources to the referees the test they cannot run themselves. That is real information, and it is why the hierarchy matters: a hostile specialist's report is worth more than a sympathetic audience's applause.

What the stamp does not guarantee is truth. Refereed studies have been refuted, corrected and retracted; the replication crisis of recent decades showed that stamped results also fall when someone redoes the math. Review examines the argument and the tests presented — it does not rerun the experiment, does not audit the raw data, does not eliminate publication bias. The stamp says "this was examined with rigor"; it never says "this is right".

For the reader of market research, the practical ruler is this: refereed > unrefereed, all else equal — and neither exempts you from asking about method, sample, and what was left out of the report.

There is also the counterfeit of the stamp, which readers need to know about: so-called predatory journals, which charge the author and publish any text after a façade "review", sometimes within days. They mimic the system's appearance — grand name, journal website, even an invented impact factor — without performing the examination that gives the stamp its value. The defense is the usual one in science: check who edits, who has published there, and how long the evaluation took. An "accepted in 48 hours" says everything that matters.

Where the house sits in this funnel

Radar Perene's public output currently lives on the step before that filter: it consists of working papers — complete studies, published on Zenodo with DOIs, that have not yet passed through the referees' sieve. The structural study of the Ânima Index, for instance, is public under DOI 10.5281/zenodo.21327595, with its versions and corrections recorded in the repository itself, in view of any reader.

We say this with the ease of someone who knows the map: a working paper is a stage of the scientific route, not a shortcut around it. Each stage carries its degree of assurance, and declaring the stage is part of the product's honesty — a house study states on its cover exactly how much external scrutiny it has received, no more, no less.

Frequently asked questions

Are referees paid?

As a rule, no — reviewing is volunteer work, treated as a duty to the scientific community. It is one reason the system is slow, and one of the most debated topics in academic publishing today.

How long does peer review take?

It varies by field and journal; in economics and finance the full cycle is usually measured in many months, and exceeding a year is not exceptional. Slowness is the price of the examination — and the reason working papers exist.

Can a refereed study be wrong?

It can, and the history of science is full of examples. Review lowers the probability of gross error; it does not zero it. The definitive correction comes from another mechanism: replication by independent researchers.

Are reviewers always anonymous?

In the format dominant in economics, yes — single or double anonymity. Some journals are experimenting with open review, with signed, published reports; the debate between the two models is very much alive.

---

The referees' filter exists, in good part, to catch one specific defect that dominates quantitative research — and it is the next topic on this trail: Overfitting in backtests is not about trading, it is about method.

House reading: the daily product of this rigor is in today's reading, in the Diário.

Reading a specific study with this ruler — what its stamp guarantees and what it does not — is the kind of examination the house performs on request.