card: five blocks, not four; label the ladder
Browse files
README.md
CHANGED
|
@@ -49,6 +49,9 @@ set can change; `main` is not a stable contract. Topos pins the SHA it was gated
|
|
| 49 |
snapshot_download("Dialogues/horos", revision="1873a1b667dc2d1bd630835f1f4f47025f3864d9")
|
| 50 |
```
|
| 51 |
|
|
|
|
|
|
|
|
|
|
| 52 |
```
|
| 53 |
labels β₯ tau_high, none low, nothing in band β ACT on the scope set (1..N scopes)
|
| 54 |
high none, nothing else β abstain: no personal data
|
|
@@ -58,7 +61,7 @@ nothing anywhere, none included β escalate to the pack's LLM (ign
|
|
| 58 |
|
| 59 |
## Report card
|
| 60 |
|
| 61 |
-
Every release publishes all
|
| 62 |
worse** β this is a standing contract, not a summary of a good run. Every row is versioned
|
| 63 |
and every cross-arm claim carries an interval. Each block earns its slot by having caught
|
| 64 |
something the others hid:
|
|
@@ -79,7 +82,7 @@ something the others hid:
|
|
| 79 |
`swallowed` and `escalated` stay separate throughout: they have opposite fixes, and
|
| 80 |
collapsing them into "not answered" hides which one you have.
|
| 81 |
|
| 82 |
-
Blocks 1β3 regenerate from one command in `topos-eval`
|
| 83 |
(`scripts/scope_head_report_card.py --head A --label v1 --head B --label v2`). That
|
| 84 |
repo is private, so the numbers here are auditable by us and take-it-or-leave-it for
|
| 85 |
you β the benchmark itself is what we'd have to open to change that.
|
|
|
|
| 49 |
snapshot_download("Dialogues/horos", revision="1873a1b667dc2d1bd630835f1f4f47025f3864d9")
|
| 50 |
```
|
| 51 |
|
| 52 |
+
**The ladder.** Four branches, because "no scopes" is two different states β the model
|
| 53 |
+
deciding there is no personal data here, and the model having no idea:
|
| 54 |
+
|
| 55 |
```
|
| 56 |
labels β₯ tau_high, none low, nothing in band β ACT on the scope set (1..N scopes)
|
| 57 |
high none, nothing else β abstain: no personal data
|
|
|
|
| 61 |
|
| 62 |
## Report card
|
| 63 |
|
| 64 |
+
Every release publishes all five blocks below, in full, **including the numbers that got
|
| 65 |
worse** β this is a standing contract, not a summary of a good run. Every row is versioned
|
| 66 |
and every cross-arm claim carries an interval. Each block earns its slot by having caught
|
| 67 |
something the others hid:
|
|
|
|
| 82 |
`swallowed` and `escalated` stay separate throughout: they have opposite fixes, and
|
| 83 |
collapsing them into "not answered" hides which one you have.
|
| 84 |
|
| 85 |
+
Blocks 1β3 and 5 regenerate from one command in `topos-eval`
|
| 86 |
(`scripts/scope_head_report_card.py --head A --label v1 --head B --label v2`). That
|
| 87 |
repo is private, so the numbers here are auditable by us and take-it-or-leave-it for
|
| 88 |
you β the benchmark itself is what we'd have to open to change that.
|