What we measure
- Extraction. For fifteen line items compiled from a filing: does that filing's own statement print the same number. Checked against the sentence that prints it.
- Claim support. For every claim: does it cite something this report collected, and does every number in it match the source it cites.
- Model proposals. For every proposed cell change: was it accepted or rejected by the analyst who reviewed it, counted per commit.
- Thesis calls. When an assumption breaks and the system says so, did the name then move by two percent, either way, by the close five days later.
- Abstentions. How many claims the verifier withheld, and how many reports were drafted without a language model.
On how many companies
The daily brief reports the filings of a handful of public companies, a line quoted from each and three headline figures, from primary sources only. Every measurement above is taken on the companies the terminal has models, reports or theses for, and that set is listed with the numbers. It grows with coverage, and the numbers are cumulative, with the count each one was taken over beside it.
When it abstains
The system does not fill gaps. It abstains, and says so, when any of these hold:
- The claim cites nothing this report collected, or a number no cited source contains.
- Two sources report the same figure for the same period and differ by more than two percent.
- The filing a figure came from is not stored, or does not print it: it counts neither way.
- The assumption it would touch has no observable metric behind it.
The abstention rate is published next to the hit rate. A high hit rate with a hidden abstention rate is not accuracy.
Where the numbers are
First measurements publish alongside the first briefs, with the next release. Until there is a number, there is no number on this page. What you will find here is the method, so that when the number appears you can see what it means and what it leaves out.
What this is not
A score without its breakdown. Every published figure carries the denominator, the company set and the abstentions beside it.
See also What changed today.