Radar Perene / Archive / science

What a meta-analysis is — and why finance in Brazil has almost none

◦ Index methodology v2.2 (working papers with DOI). See the methodology.

Science

When two studies on the same subject reach opposite conclusions, public debate picks the one confirming what it already believed and moves on. Science has a better answer for that tie — an entire genre of research dedicated to judging it. In medicine, it decides treatments; in psychology, it toppled famous effects. In finance in Brazil, it barely exists — and that absence says something about the state of the literature.

A meta-analysis is a study of studies: it gathers, through systematic search and declared criteria, all the research that tested the same hypothesis, and statistically combines their results into an aggregate effect — weighing sample size, quality and the possibility that negative results were never published at all.

The definition carries the two properties that make the genre the highest rung of empirical evidence. The first is the systematic search: the author does not pick the studies he knows or prefers — he declares in advance where he searched, with what terms and what inclusion criteria, so another researcher can redo the collection and arrive at the same pile. The second is the statistical combination: results are not summarized in prose ("most found a positive effect"); they are aggregated numerically, each study weighing according to its precision.

What a meta-analysis sees that a single study cannot

The genre's value lies in what only appears in the aggregate. An isolated study with a positive effect can be chance, a peculiar sample or a fortunate specification. Twenty studies, placed on the same ruler, reveal patterns none of them contains individually: whether the effect shrinks in the more recent studies; whether it appears only in the small-sample studies — the classic signature of publication bias; whether the results are distributed as if the negative ones had been shelved. Tools of the genre were designed precisely to detect that last hypothesis, because science's published archive is not a neutral sample of what was tested — it is a sample of what worked well enough to be accepted.

In international finance, the genre has produced sober, uncomfortable results: aggregations of hundreds of published estimates on anomalies and premia found average effects far smaller than the founding studies suggested, with a relevant share of the difference explained by selection of what gets published. It is the kind of conclusion no individual study could sustain alone.

The Brazilian vacuum

While preparing this section's agenda, the house did the miniature version of what the genre demands: it swept Portuguese-language search results, and the usual open databases, for meta-analyses on finance topics with Brazilian data — factor premia in the local market, announcement effects, seasonalities, cost of capital. The finding, documented in the agenda's internal records, was a near desert: the term "meta-análise" in Portuguese belongs, in practice, to health and education; in finance, what exists are narrative literature reviews, which summarize studies in prose without the genre's two pillars — systematic search and statistical aggregation.

The absence has honest explanations. Meta-analysis requires raw material: dozens of studies testing the same hypothesis, with statistics reported in comparable form. The Brazilian finance literature is smaller, more dispersed in methods and less given to direct replication — each author prefers varying the question to repeating the neighbor's, because replication earns less prestige. The result is a wide, shallow archive, hard to aggregate. Seen up close, the absence of meta-analyses is not just a missing genre: it is the visible symptom of a literature that does not yet accumulate upon itself.

For the reader of market research, the practical consequence is a ruler of modesty: on Brazilian topics, the honest answer to "what does the literature say" is almost always "a handful of studies, with different methods, saying partially different things" — and whoever asserts consensus is, most of the time, citing a single study.

Frequently asked questions

Are a meta-analysis and a literature review the same thing?

No. The narrative review summarizes studies chosen by the author, in prose; the meta-analysis systematically searches for everything that tested the hypothesis and combines the results numerically. The first informs; the second measures.

Can a meta-analysis be wrong?

It can — the genre inherits the defects of its raw material. If the aggregated studies share the same bias, the aggregation repeats it with greater confidence. The field's expression for this is blunt: garbage in, garbage out with a confidence interval.

What is publication bias?

The tendency of positive, significant results to be published more often than null results. Science's archive thus ends up more optimistic than the tested reality — and detecting that distortion is one of the meta-analysis's central functions.

Why does medicine have so many and finance so few?

Clinical trials are designed to be comparable — registered protocols, standardized outcomes — and the decisions that depend on them demand aggregation. In finance, each study designs its own test, and the comparability meta-analysis requires rarely exists from birth.

---

One question remains that no statistical method answers: who pays for the study — and what the payment does to the conclusion. It closes this trail: Conflicts of interest in bank research.

House reading: the state of the evidence, topic by topic, appears in today's reading, in the Diário.