What we measure
- Extraction. For every fact taken from a filing, transcript or release: does it match the sentence it was taken from, exactly. Checked against the source span.
- Claim support. For every claim: is there an evidence span behind it, and does the span actually say what the claim says.
- Model proposals. For every proposed cell change: was it accepted, rejected or overridden by the analyst who reviewed it, and by how much.
- Thesis calls. When an assumption is marked weaker or broken, was it. Judged against what happened next, at the horizon the assumption named.
- Abstentions. How often the system declined to propose anything, and why.
On how many companies
The daily brief runs the full chain on a handful of public companies every morning, from primary sources only. Every measurement above is taken on that set, and the set is listed with each brief. It grows with coverage, and the numbers are reported per brief and cumulatively, so a good week cannot hide a bad one.
When it abstains
The system does not fill gaps. It abstains, and says so, when any of these hold:
- There is no primary source for the claim, only commentary about it.
- Two primary sources conflict and neither supersedes the other.
- A restatement is in flight and the earlier version has not yet closed.
- Extraction confidence for the span is below the threshold set for that source type.
- The assumption it would touch has no observable metric behind it.
The abstention rate is published next to the hit rate. A high hit rate with a hidden abstention rate is not accuracy.
Where the numbers are
First measurements publish alongside the first briefs, with the next release. Until there is a number, there is no number on this page. What you will find here is the method, so that when the number appears you can see what it means and what it leaves out.
What this is not
A score without its breakdown. Every published figure carries the denominator, the company set, the horizon and the abstentions beside it.
See also What changed today and the changelog.