Radar Perene / Archive / science

16 spreadsheets, one hash: the public data package behind two indices

◦ Index methodology v2.2 (working papers with DOI). See the methodology.

Science

A chart persuades; a downloadable file invites distrust — and that is why the file is worth more. Almost every market analysis provider publishes the finished curve: pretty, edited, impossible to audit. What almost none publishes is the raw material, in a state that lets anyone rebuild the drawing or refute it. The difference between the two gestures is the difference between asking for trust and making it unnecessary.

Alongside two of its published indices, the house maintains on Zenodo a public data package — series, dated events and pre-registration material — in which every file is accompanied by a SHA-256 verification code. The hash makes integrity checkable by any reader: if a single byte of the file changes, the code changes with it, and tampering becomes visible without relying on anyone's word.

What a hash does for a non-technical reader

A hash is a mathematical fingerprint of a file. The reader does not need to understand the calculation to use its consequence: the code published next to the data is the promise that the data is exactly that data — not a version retouched later, not a spreadsheet silently "updated". Whoever downloads the file can generate the code on their own machine, with free tools, and compare. If it matches, the file is the original; if it does not, someone touched it.

In numbers: the package gathers 16 spreadsheets, each with its SHA-256 verification, deposited with its own DOI and preserved versioning in the repository.

For market data, this solves a silent problem: historical series are editable by nature, and the temptation to correct the past — deleting the number that aged badly, adjusting the window that turned ugly — leaves no trace when the data lives on a private server. Deposited with a hash in a third-party repository, the past is locked in: any correction requires a new, dated version, next to the old one.

Why a scientific repository, and not the house's own site

The package could live on the house's site — and it would be a weaker promise. On Zenodo, operated by CERN, the deposit gains three properties a private site does not offer: the DOI, which makes the set citable as an academic object; immutable versioning, which preserves every previous state; and independent custody, which outlives the house itself. If this site goes dark tomorrow, the package stays where it is.

There is also an ecosystem reason. Much of the raw material of the house's research comes from Brazilian open data — the path a series from Ipeadata travels until it becomes research has its own article. Feeding on public data and returning closed data interrupts the cycle that sustains it; returning the package open, verifiable and citable is the coherent counterpart.

What the package does not replace

Verifiable data is not open method. The package documents the inputs and dated events behind two indices; it does not document every construction decision — that boundary is the same one discussed in why the house opened one index's formula and not the others. Nor does the hash certify quality: it guarantees the file has not changed, not that the file is right. Integrity and merit are different audits — the package settles the first and exposes the material to the second.

Is that little? It is what separates an auditable claim from a presentation. And with this trail's account closed — indices with DOIs, one formula opened by criterion, one finding that survived a retest, data with verification — the question the next trail confronts remains: who watches the one doing all this, if no regulator demands any of it?

Citable also means citable against the house: whoever writes a study refuting one of these indices can use the package itself as evidence — and that possibility, far from being a tolerated risk, is the central argument. Published data that only serves to confirm its publisher is not open data; it is advertising in spreadsheet format. The package exists to be used in any direction, including the uncomfortable one.

Frequently asked questions

What exactly is in the package?

Data series, dated events and pre-registration material used by two of the house's published indices — 16 spreadsheets in total, each with SHA-256 verification, under DOI 10.5281/zenodo.21399426.

How does one check a hash in practice?

Modern operating systems ship the calculation in command-line tools, and there are free utilities with graphical interfaces. The procedure comes down to generating the code of the downloaded file and comparing it with the published one — any divergence exposes alteration.

Does a verified hash mean correct data?

No. The hash guarantees the file is identical to the deposited one — integrity, not merit. Collection or construction errors remain possible; the difference is that, with open data, they become findable by any reader.

Why do only two indices have a data package?

Because the deposit follows each study's publication stage. The package exists where public documentation already supported that step; each material's standard of openness follows the criterion discussed throughout this trail.

---

Continue the trail — and open the next one: Who audits the auditor: the problem of an index no one regulates from outside →

House reading: today's reading is in the Diário; the archived episodes, in the Atlas.

Assembling a verifiable package for a reader's own dataset is the kind of exercise the house discusses in conversation, outside the public article.