Radar Perene / Archive / science

Who pays the bill if a market index errs: the institutional case for public self-assessment

◦ Index methodology v2.2 (working papers with DOI). See the methodology.

Science

When a medicine fails, there is pharmacovigilance, recall, civil liability. When a bridge fails, there is forensic engineering, a responsible engineer, a professional board. When a private market index fails — measures poorly, misclassifies a regime, suggests a state of the world that did not exist — strictly nothing happens to whoever published it. The bill exists, but it is paid at another address: that of the reader who adjusted their understanding of the market to that crooked ruler.

It is this asymmetry — the publisher's error being paid for by the reader — that constitutes the institutional case for periodic quantitative self-assessment: if no external instance presents the invoice to the index provider, the only correction available is for the provider to present it to itself, with criteria defined in advance, a fixed cadence and a recorded outcome. This house keeps that practice as routine — not as gesture, as calendar.

The anatomy of the bill no one collects

The damage from a bad index is real, but it has three properties that make it invisible to the usual accountability mechanisms. It is diffuse: it spreads across many readers in small doses, and diffuse damage generates no plaintiff. It is deniable: between the crooked ruler and the bad decision there are always layers of mediation dissolving the link — no one loses money "because of an index"; they lose it through decisions informed by it. And it is slow: a methodology that aged badly produces degraded readings for years before anyone notices.

Diffuse, deniable and slow damage is exactly the kind that markets and courts process poorly — and that, in other sectors, justified the creation of dedicated supervision. In the territory of private indicators, as this trail recorded in who audits the auditor, that supervision does not exist. The honest institutional question is not "who will collect?" — it is "what replaces collection where it will never come?".

What self-assessment needs in order not to be theater

Self-assessment is an easy word to discredit, and rightly so: the theatrical version — the house examines itself, approves itself and communicates satisfaction — abounds. The version the house practices is distinguished by three formal properties, all prior to the result. The criteria are quantitative and defined before the exercise, so the ruler is not drawn around the performance. The cadence is fixed, so the assessment happens also in the periods when no one would do it willingly. And the outcome is recorded with a date, so the series of assessments composes an auditable history, not a collection of scattered self-praise.

The grade of each exercise is not published, and the reason is the same one governing this whole trail: what gets published must be sustainable and interpretable indefinitely, and an internal grade without the full context of its criteria invites misreading more than verification. What is public is the commitment — the existence of the practice, its cadence and its consequence: the house materials the reader sees have passed through this periodic sieve, and what fails it does not continue as it was.

Self-assessment as signature, not exception

There is a simple test the reader can apply to any analysis or index provider: look for any evidence that the operation examines itself — declared criteria, errors admitted with dates, documented revisions. The complete absence of that layer is one of the most informative flags of financial pseudoscience, because it does not depend on judging the content: an operation that never records its own failure is claiming, by omission, a hit rate no real operation has.

The institutional case, in the end, is modest and therefore hard to refute: where the bill for error lands on the reader, periodic self-assessment is the minimum separating an index provider from an issuer of opinions with charts. It is not a guarantee of quality — it is the condition of its possibility.

Frequently asked questions

Isn't self-assessment without a public grade half-transparency?

It is transparency with a chosen object: the commitment and the consequence are public; the internal instrument is not. The alternative of publishing the grade without the full apparatus of criteria would produce a performative number — precisely the vice the practice wants to avoid.

What happens when the self-assessment finds a problem?

The same thing this trail described for statistics that do not repeat: the affected material does not continue as it was — it is revised, demoted or closed, and the change is recorded. When the problem touches already-public material, the revision is made in the open, as the preserved versions of the house's series document.

Why quantitative criteria, and not editorial judgment?

Because the assessor and the assessed are the same house. Qualitative judgment of oneself leans toward acquittal; numerical criteria defined in advance shrink the room for that complacency — not to zero, but enough for the exercise to have real cost.

Does this replace an external audit?

No, and the house does not claim it does. It replaces the absence of any audit — which is the actually existing alternative in the independent index sector.

---

Continue the trail: Conflict of interest in reverse: what changes when whoever calculates the index sells no product on it →

House reading: today's reading is in the Diário; the archived episodes, in the Atlas.

Designing a self-assessment exercise with criteria defined in advance, for the reader's own material, is bench conversation — the house has it often.