Burnmydays commited on
Commit
b40f536
·
1 Parent(s): d30f28f
Files changed (1) hide show
  1. REVIEW.md +110 -0
REVIEW.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # REVIEW — what changed this session, for your eyes
2
+
3
+ > Quick review file. Delete after you've read it.
4
+
5
+ ---
6
+
7
+ ## What the numbers look like now
8
+
9
+ ```
10
+ === CLAUDE (./sigrank) ===
11
+ source: ccusage claude
12
+ ledger R 2.65B · C 136.21M · I 2.90M · O 12.32M
13
+ SNR 0.810
14
+ 10x DEV 2.96
15
+ velocity 4.25×
16
+ leverage 914×
17
+ $/1M $0.715
18
+ Υ yield 3,886
19
+ class Closed-Loop Kinetic · holds both axes
20
+ rank #2 of 8
21
+
22
+ === CODEX (./sigrank --codex) ===
23
+ source: ccusage codex
24
+ ⚠ estimated via turn-delta (cache_create from daily context growth)
25
+ ledger R 707.30M · C 26.17M · I 58.92M · O 4.01M
26
+ SNR 0.064
27
+ 10x DEV 1.08
28
+ velocity 0.07×
29
+ leverage 12×
30
+ $/1M $0.561
31
+ Υ yield 1
32
+ class Archival Sponge · high reuse, low generation
33
+ rank #5 of 8
34
+ ```
35
+
36
+ ---
37
+
38
+ ## Problem being solved
39
+
40
+ `ccusage --json` combines Claude + Codex + every other agent into one total.
41
+ When combined, Codex's large `inputTokens` (58.9M) tanked Υ quadratically.
42
+ Fix: run them separately — `ccusage claude --json` and `ccusage codex --json`.
43
+
44
+ ---
45
+
46
+ ## What was wrong with Codex before (and the fix)
47
+
48
+ **Bug 1 — detection miss (already fixed earlier):**
49
+ `is_codex_shape()` checked for `cached_input_tokens` (snake_case) but
50
+ `ccusage codex --json` emits `cachedInputTokens` (camelCase). Codex JSON
51
+ fell through to `parse_ccusage` — no split, no cache, raw 58.9M input.
52
+ Fix: check both cases.
53
+
54
+ **Bug 2 — anchor now uses turn-delta (CODEX.md item 1):**
55
+ Old: `est_fresh = 2 * output` (fixed 2:1).
56
+ New: with daily granularity present (we have 29 days of data), estimates
57
+ `cache_create` from per-day context growth deltas instead.
58
+ Fallback: if no daily data, uses Claude's measured I/O ratio (0.236:1)
59
+ instead of fixed 2:1 — grounded in your real data, not a constant.
60
+
61
+ ---
62
+
63
+ ## What the Codex numbers mean
64
+
65
+ | field | value | source |
66
+ |---|---|---|
67
+ | input (I) | 58.92M | `inputTokens` — fresh input directly from Codex JSON |
68
+ | output (O) | 4.01M | `outputTokens` + `reasoningOutputTokens` |
69
+ | cache_create (C) | 26.17M | **estimated** via turn-delta |
70
+ | cache_read (R) | 707.30M | `cachedInputTokens` — measured directly |
71
+ | cost | real ($0.561/1M) | from `costUSD` in the JSON |
72
+
73
+ The 707.3M cache reads are real and measured. Codex is reading a LOT of
74
+ cached context. It just generates very little output relative to input
75
+ (velocity 0.07×), so Υ is low. That's Codex's architecture, not a bug.
76
+
77
+ ---
78
+
79
+ ## Board marker (CODEX.md item 4)
80
+
81
+ Estimated rows (Codex-anchored) now show a `~` next to the operator name
82
+ in the leaderboard. Measured rows (real ccusage) show clean.
83
+
84
+ ---
85
+
86
+ ## What is NOT right (open questions for you)
87
+
88
+ 1. **Codex `inputTokens` interpretation** — the turn-delta treats `inputTokens`
89
+ as already-fresh (not combined with reads). If OpenAI actually reports
90
+ combined fresh+cached in that field, the I value (58.92M) is too high
91
+ and Υ will be artificially low. Do you know which it is?
92
+
93
+ 2. **Υ=1 for Codex** — with 58.9M fresh input and only 4M output, Υ is
94
+ nearly 0. That may be correct (Codex is a heavy reader, light generator)
95
+ or it may mean `inputTokens` is the combined figure and needs splitting.
96
+
97
+ 3. **CODEX.md items 2 + 3 still open:**
98
+ - Item 2: `test_metrics.py` (pytest for canonical numbers)
99
+ - Item 3: self-cost via OpenAI per-1M pricing table in `parse_codex`
100
+ (cost already comes through from `costUSD`, so this may be done?)
101
+
102
+ ---
103
+
104
+ ## Files changed this session
105
+
106
+ | file | what changed |
107
+ |---|---|
108
+ | `ingest.py` | `is_codex_shape` → camelCase fix; `parse_codex` → turn-delta + io_ratio param; `ingest_meta` → io_ratio passthrough |
109
+ | `sigrank.py` | default changed to `ccusage claude --json`; `--codex` path fetches Claude ratio first |
110
+ | `app.py` | `html.escape(name)` XSS fix; `~` marker on estimated board rows |