Burnmydays commited on
Commit ·
b40f536
1
Parent(s): d30f28f
update
Browse files
REVIEW.md
ADDED
|
@@ -0,0 +1,110 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# REVIEW — what changed this session, for your eyes
|
| 2 |
+
|
| 3 |
+
> Quick review file. Delete after you've read it.
|
| 4 |
+
|
| 5 |
+
---
|
| 6 |
+
|
| 7 |
+
## What the numbers look like now
|
| 8 |
+
|
| 9 |
+
```
|
| 10 |
+
=== CLAUDE (./sigrank) ===
|
| 11 |
+
source: ccusage claude
|
| 12 |
+
ledger R 2.65B · C 136.21M · I 2.90M · O 12.32M
|
| 13 |
+
SNR 0.810
|
| 14 |
+
10x DEV 2.96
|
| 15 |
+
velocity 4.25×
|
| 16 |
+
leverage 914×
|
| 17 |
+
$/1M $0.715
|
| 18 |
+
Υ yield 3,886
|
| 19 |
+
class Closed-Loop Kinetic · holds both axes
|
| 20 |
+
rank #2 of 8
|
| 21 |
+
|
| 22 |
+
=== CODEX (./sigrank --codex) ===
|
| 23 |
+
source: ccusage codex
|
| 24 |
+
⚠ estimated via turn-delta (cache_create from daily context growth)
|
| 25 |
+
ledger R 707.30M · C 26.17M · I 58.92M · O 4.01M
|
| 26 |
+
SNR 0.064
|
| 27 |
+
10x DEV 1.08
|
| 28 |
+
velocity 0.07×
|
| 29 |
+
leverage 12×
|
| 30 |
+
$/1M $0.561
|
| 31 |
+
Υ yield 1
|
| 32 |
+
class Archival Sponge · high reuse, low generation
|
| 33 |
+
rank #5 of 8
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
---
|
| 37 |
+
|
| 38 |
+
## Problem being solved
|
| 39 |
+
|
| 40 |
+
`ccusage --json` combines Claude + Codex + every other agent into one total.
|
| 41 |
+
When combined, Codex's large `inputTokens` (58.9M) tanked Υ quadratically.
|
| 42 |
+
Fix: run them separately — `ccusage claude --json` and `ccusage codex --json`.
|
| 43 |
+
|
| 44 |
+
---
|
| 45 |
+
|
| 46 |
+
## What was wrong with Codex before (and the fix)
|
| 47 |
+
|
| 48 |
+
**Bug 1 — detection miss (already fixed earlier):**
|
| 49 |
+
`is_codex_shape()` checked for `cached_input_tokens` (snake_case) but
|
| 50 |
+
`ccusage codex --json` emits `cachedInputTokens` (camelCase). Codex JSON
|
| 51 |
+
fell through to `parse_ccusage` — no split, no cache, raw 58.9M input.
|
| 52 |
+
Fix: check both cases.
|
| 53 |
+
|
| 54 |
+
**Bug 2 — anchor now uses turn-delta (CODEX.md item 1):**
|
| 55 |
+
Old: `est_fresh = 2 * output` (fixed 2:1).
|
| 56 |
+
New: with daily granularity present (we have 29 days of data), estimates
|
| 57 |
+
`cache_create` from per-day context growth deltas instead.
|
| 58 |
+
Fallback: if no daily data, uses Claude's measured I/O ratio (0.236:1)
|
| 59 |
+
instead of fixed 2:1 — grounded in your real data, not a constant.
|
| 60 |
+
|
| 61 |
+
---
|
| 62 |
+
|
| 63 |
+
## What the Codex numbers mean
|
| 64 |
+
|
| 65 |
+
| field | value | source |
|
| 66 |
+
|---|---|---|
|
| 67 |
+
| input (I) | 58.92M | `inputTokens` — fresh input directly from Codex JSON |
|
| 68 |
+
| output (O) | 4.01M | `outputTokens` + `reasoningOutputTokens` |
|
| 69 |
+
| cache_create (C) | 26.17M | **estimated** via turn-delta |
|
| 70 |
+
| cache_read (R) | 707.30M | `cachedInputTokens` — measured directly |
|
| 71 |
+
| cost | real ($0.561/1M) | from `costUSD` in the JSON |
|
| 72 |
+
|
| 73 |
+
The 707.3M cache reads are real and measured. Codex is reading a LOT of
|
| 74 |
+
cached context. It just generates very little output relative to input
|
| 75 |
+
(velocity 0.07×), so Υ is low. That's Codex's architecture, not a bug.
|
| 76 |
+
|
| 77 |
+
---
|
| 78 |
+
|
| 79 |
+
## Board marker (CODEX.md item 4)
|
| 80 |
+
|
| 81 |
+
Estimated rows (Codex-anchored) now show a `~` next to the operator name
|
| 82 |
+
in the leaderboard. Measured rows (real ccusage) show clean.
|
| 83 |
+
|
| 84 |
+
---
|
| 85 |
+
|
| 86 |
+
## What is NOT right (open questions for you)
|
| 87 |
+
|
| 88 |
+
1. **Codex `inputTokens` interpretation** — the turn-delta treats `inputTokens`
|
| 89 |
+
as already-fresh (not combined with reads). If OpenAI actually reports
|
| 90 |
+
combined fresh+cached in that field, the I value (58.92M) is too high
|
| 91 |
+
and Υ will be artificially low. Do you know which it is?
|
| 92 |
+
|
| 93 |
+
2. **Υ=1 for Codex** — with 58.9M fresh input and only 4M output, Υ is
|
| 94 |
+
nearly 0. That may be correct (Codex is a heavy reader, light generator)
|
| 95 |
+
or it may mean `inputTokens` is the combined figure and needs splitting.
|
| 96 |
+
|
| 97 |
+
3. **CODEX.md items 2 + 3 still open:**
|
| 98 |
+
- Item 2: `test_metrics.py` (pytest for canonical numbers)
|
| 99 |
+
- Item 3: self-cost via OpenAI per-1M pricing table in `parse_codex`
|
| 100 |
+
(cost already comes through from `costUSD`, so this may be done?)
|
| 101 |
+
|
| 102 |
+
---
|
| 103 |
+
|
| 104 |
+
## Files changed this session
|
| 105 |
+
|
| 106 |
+
| file | what changed |
|
| 107 |
+
|---|---|
|
| 108 |
+
| `ingest.py` | `is_codex_shape` → camelCase fix; `parse_codex` → turn-delta + io_ratio param; `ingest_meta` → io_ratio passthrough |
|
| 109 |
+
| `sigrank.py` | default changed to `ccusage claude --json`; `--codex` path fetches Claude ratio first |
|
| 110 |
+
| `app.py` | `html.escape(name)` XSS fix; `~` marker on estimated board rows |
|