Burnmydays Claude Opus 4.8 (1M context) commited on
Commit
b92c3a7
·
1 Parent(s): 90eeb7a

feat: interactive leaderboard + clearer step-by-step instructions

Browse files

Leaderboard:
- "Rank by" control re-sorts the board by any column (Υ / SNR / 10x DEV /
velocity / leverage / $/1M; cost ascends so cheapest leads). board_html
gains a sort_key param; Υ bar always scales to Υ.
- "Operator profiles" picker opens any corpus operator's full profile + share
card (view_operator). Answers "where are the profiles / click a user".
- Hero stat-strip now dynamic (operator count + real lead factor), no stale 7.

Instructions (Clock Your Signal tab + README):
- Rewritten as a clean 4-step guide. Commands are copy-paste-safe — each on its
own line, NO inline `# comments` (those broke in zsh: `(measured)` globbed ->
"number expected"). Local importer marked optional/clone-only with prereqs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Files changed (2) hide show
  1. README.md +8 -4
  2. app.py +70 -26
README.md CHANGED
@@ -58,13 +58,17 @@ Three things: **what data, where to get it, where to put it.**
58
 
59
  **1 — What / where to get it.** SigRank needs four integers — `input`, `output`,
60
  `cache_create`, `cache_read`. [`ccusage`](https://github.com/ryoppippi/ccusage) reads
61
- them from your local logs. Run **one command per provider** — Claude and Codex are
62
- different operators, so measure them **separately**:
63
 
64
  ```
65
- ccusage claude --json # Claude Code
66
- ccusage codex --json # Codex
67
  ```
 
 
 
 
 
68
 
69
  > ⚠️ Don't use bare `ccusage --json` (no subcommand): it merges every agent into one
70
  > total, which inflates input and distorts the architecture read.
 
58
 
59
  **1 — What / where to get it.** SigRank needs four integers — `input`, `output`,
60
  `cache_create`, `cache_read`. [`ccusage`](https://github.com/ryoppippi/ccusage) reads
61
+ them from your local logs (needs Node.js). Run **one command per provider** — Claude and
62
+ Codex are different operators, so measure them **separately** (each command on its own line):
63
 
64
  ```
65
+ npx ccusage@latest claude --json
 
66
  ```
67
+ ```
68
+ npx ccusage@latest codex --json
69
+ ```
70
+
71
+ (If ccusage is installed globally, drop the prefix: `ccusage claude --json`.)
72
 
73
  > ⚠️ Don't use bare `ccusage --json` (no subcommand): it merges every agent into one
74
  > total, which inflates input and distorts the architecture read.
app.py CHANGED
@@ -41,13 +41,19 @@ def _cost_str(m):
41
  return f"{mark}${c:.2f}"
42
 
43
  # ---------- leaderboard (HTML hero, log-scaled Υ, cost column) ----------
44
- def board_html(extra=None):
 
 
 
 
45
  ops = operators()
46
  # dedup: if `extra` is already persisted, replace it so it shows once + highlighted
47
  rows=[(n,compute(*v)) for n,v in ops.items() if not (extra and n==extra[0])]
48
  if extra: rows.append(extra)
49
- rows.sort(key=lambda r:r[1]["yield"], reverse=True)
50
- ymax=rows[0][1]["yield"] or 1
 
 
51
  out=['<div class="moses-board">']
52
  out.append('<div class="mb-head"><span class="mb-rank">#</span>'
53
  '<span class="mb-op">operator</span>'
@@ -301,6 +307,23 @@ def run_ingest(blob, name, request: gr.Request):
301
  hits_html,
302
  board_html((name,m)))
303
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
304
  # ---------- UI ----------
305
  import os as _os
306
  _ON_SPACE = bool(_os.environ.get("SPACE_ID"))
@@ -311,50 +334,71 @@ def _build_demo():
311
  _b = gr.Blocks(css=CSS, theme=gr.themes.Base(), **_blocks_kw)
312
  except TypeError:
313
  _b = gr.Blocks(**_blocks_kw)
 
 
 
 
 
314
  with _b:
315
  with gr.Column(elem_id="moses-hero"):
316
  gr.HTML("<h1>MO\u00a7ES\u2122 SigRank</h1>"
317
  "<p>the diagnostic x-ray of the token economy \u00b7 ranked by \u03a5 (Net Volumetric Yield) \u00b7 volume can't buy rank</p>")
318
  gr.HTML('<div id="moses-stat-strip">'
319
- '<div>operators ranked <span>7</span></div>'
320
- '<div>MO\u00a7ES leads by <span>3,141\u00d7</span></div>'
321
  '<div>architecture beats budget</div>'
322
  '</div>')
323
 
324
  with gr.Tab("Leaderboard"):
325
  gr.Markdown("Ranked by **\u03a5 = (Cache\u00b7Output)/Input\u00b2**. Raw Read\u00b7Create\u00b7In\u00b7Out stacked under each operator. $/1M is blended cost \u2014 efficient architecture is also the cheapest.")
 
 
 
 
326
  gr.Markdown("*Corpus is curated \u2014 pasting your usage scores you live against the field but doesn't add you to the persisted board unless you're signed in via HuggingFace. $/1M is a list-price recompute (~); real cost shows when you paste your own ccusage. * = structural estimation.*", elem_id="moses-foot")
327
- gr.HTML(board_html())
 
 
 
 
 
328
 
329
  with gr.Tab("Clock Your Signal"):
330
- gr.Markdown("""**Get your operator profile \u2014 measure each provider separately.**
331
 
332
- **\u2460 Get your numbers.** Run one command per provider in your terminal:
 
 
 
333
  ```
334
- ccusage claude --json # Claude Code
335
- ccusage codex --json # Codex (estimated *)
336
  ```
337
- \u26a0\ufe0f **Run each provider on its own \u2014 never bare `ccusage --json`.** With no
338
- subcommand it merges every agent into one total, inflating input and distorting your
339
- architecture. Claude and Codex are different operators \u2014 measure them separately.
340
-
341
- Prefer zero copy-paste? Use the local importer (reads your usage on your machine):
342
  ```
343
- ./sigrank # Claude Code (measured)
344
- ./sigrank --codex # Codex (calibrated *)
345
- ./sigrank --all # each provider, one after another
346
  ```
 
 
 
 
 
 
347
 
348
- **\u2461 Paste it below.** Drop one provider's `ccusage` JSON in the box \u2014 we route it
349
- automatically. No JSON handy? Type four numbers in order:
350
- `input output cache_create cache_read`.
351
 
352
- *Codex reports no fresh-vs-cache input split, so its input is **estimated**: on its own it
353
- uses the AA-backed **2:1** baseline; if you also have a Claude profile it uses **your own
354
- Claude input:output ratio**. Estimated rows are flagged with \\*.*
355
 
356
- **\u2462 Sign in to save.** A HuggingFace login earns one persistent board entry + session
357
- history (Greatest Hits). Pasting without login is a live snapshot only.""")
 
 
 
 
 
 
 
358
  if _ON_SPACE:
359
  gr.LoginButton(elem_id="hf-login-btn")
360
  else:
 
41
  return f"{mark}${c:.2f}"
42
 
43
  # ---------- leaderboard (HTML hero, log-scaled Υ, cost column) ----------
44
+ # column label -> metrics key, used by the "Rank by" control on the board
45
+ SORT_LABELS = {"Υ yield": "yield", "SNR": "snr", "10x DEV": "dev10x",
46
+ "velocity": "velocity", "leverage": "leverage", "$/1M": "avg_cost_1m"}
47
+
48
+ def board_html(extra=None, sort_key="yield"):
49
  ops = operators()
50
  # dedup: if `extra` is already persisted, replace it so it shows once + highlighted
51
  rows=[(n,compute(*v)) for n,v in ops.items() if not (extra and n==extra[0])]
52
  if extra: rows.append(extra)
53
+ ymax=max((m["yield"] for _,m in rows), default=1) or 1 # Υ bar always scales to Υ
54
+ asc = sort_key == "avg_cost_1m" # cheapest leads for cost
55
+ rows.sort(key=lambda r:(r[1].get(sort_key) if r[1].get(sort_key) is not None
56
+ else float("-inf")), reverse=not asc)
57
  out=['<div class="moses-board">']
58
  out.append('<div class="mb-head"><span class="mb-rank">#</span>'
59
  '<span class="mb-op">operator</span>'
 
307
  hits_html,
308
  board_html((name,m)))
309
 
310
+ # ---------- interactive leaderboard helpers ----------
311
+ def resort_board(label):
312
+ """Re-render the board sorted by the chosen column (the 'Rank by' control)."""
313
+ return board_html(sort_key=SORT_LABELS.get(label, "yield"))
314
+
315
+ def view_operator(name):
316
+ """Render a corpus operator's full profile + share card (the 'open a profile' picker)."""
317
+ ops = operators()
318
+ if not name or name not in ops:
319
+ return "", ""
320
+ m = compute(*ops[name])
321
+ rows = sorted(((n, compute(*v)) for n, v in ops.items()),
322
+ key=lambda r: r[1]["yield"], reverse=True)
323
+ rank = next(i for i, (n, _) in enumerate(rows, 1) if n == name)
324
+ read = narrate(name, m, classify(m))
325
+ return profile_md(name, m, rank, len(rows), read), card_html(name, m, rank, len(rows), read)
326
+
327
  # ---------- UI ----------
328
  import os as _os
329
  _ON_SPACE = bool(_os.environ.get("SPACE_ID"))
 
334
  _b = gr.Blocks(css=CSS, theme=gr.themes.Base(), **_blocks_kw)
335
  except TypeError:
336
  _b = gr.Blocks(**_blocks_kw)
337
+ # dynamic hero stats (don't hardcode counts that drift when the corpus changes)
338
+ _ops_now = operators()
339
+ _names = list(_ops_now.keys())
340
+ _ys = sorted((compute(*v)["yield"] for v in _ops_now.values()), reverse=True)
341
+ _lead = (_ys[0] / _ys[1]) if len(_ys) > 1 and _ys[1] > 0 else 0.0
342
  with _b:
343
  with gr.Column(elem_id="moses-hero"):
344
  gr.HTML("<h1>MO\u00a7ES\u2122 SigRank</h1>"
345
  "<p>the diagnostic x-ray of the token economy \u00b7 ranked by \u03a5 (Net Volumetric Yield) \u00b7 volume can't buy rank</p>")
346
  gr.HTML('<div id="moses-stat-strip">'
347
+ f'<div>operators ranked <span>{len(_ops_now)}</span></div>'
348
+ f'<div>MO\u00a7ES leads by <span>{_lead:,.0f}\u00d7</span></div>'
349
  '<div>architecture beats budget</div>'
350
  '</div>')
351
 
352
  with gr.Tab("Leaderboard"):
353
  gr.Markdown("Ranked by **\u03a5 = (Cache\u00b7Output)/Input\u00b2**. Raw Read\u00b7Create\u00b7In\u00b7Out stacked under each operator. $/1M is blended cost \u2014 efficient architecture is also the cheapest.")
354
+ rank_by = gr.Radio(list(SORT_LABELS.keys()), value="\u03a5 yield",
355
+ label="Rank by \u2014 pick a column to see its leaders")
356
+ lb = gr.HTML(board_html())
357
+ rank_by.change(resort_board, rank_by, lb)
358
  gr.Markdown("*Corpus is curated \u2014 pasting your usage scores you live against the field but doesn't add you to the persisted board unless you're signed in via HuggingFace. $/1M is a list-price recompute (~); real cost shows when you paste your own ccusage. * = structural estimation.*", elem_id="moses-foot")
359
+
360
+ gr.Markdown("### Operator profiles")
361
+ op_pick = gr.Dropdown(_names, label="Open an operator's profile", value=None)
362
+ op_prof = gr.Markdown(elem_id="moses-profile")
363
+ op_card = gr.HTML()
364
+ op_pick.change(view_operator, op_pick, [op_prof, op_card])
365
 
366
  with gr.Tab("Clock Your Signal"):
367
+ gr.Markdown("""### Clock your signal \u2014 4 steps
368
 
369
+ **Step 1 \u00b7 Pull your usage.** In your terminal (needs [Node.js](https://nodejs.org)),
370
+ run **one** of these and copy the JSON it prints.
371
+
372
+ Claude Code:
373
  ```
374
+ npx ccusage@latest claude --json
 
375
  ```
376
+ Codex:
 
 
 
 
377
  ```
378
+ npx ccusage@latest codex --json
 
 
379
  ```
380
+ Already have ccusage installed? Drop the prefix and just run `ccusage claude --json`.
381
+ \u26a0\ufe0f Run Claude and Codex **separately** \u2014 never bare `ccusage --json` (no provider). That
382
+ merges every agent into one total and distorts your read.
383
+
384
+ **Step 2 \u00b7 Paste it in the box below** and press **Clock My Signal**. No JSON? Type four
385
+ numbers in order: `input output cache_create cache_read`.
386
 
387
+ **Step 3 \u00b7 Read your profile** \u2014 archetype, cascade, full metrics, and your live board
388
+ placement appear right below.
 
389
 
390
+ **Step 4 \u00b7 (optional) Sign in with HuggingFace** to save your entry to the board + keep
391
+ session history (Greatest Hits). Without login it's a live snapshot only.
 
392
 
393
+ ---
394
+ **Local importer (optional \u2014 only if you've cloned the repo).** From inside the repo
395
+ folder, run the command **on its own line** (don't paste any trailing notes):
396
+ ```
397
+ ./sigrank
398
+ ```
399
+ Use `./sigrank --codex` for Codex or `./sigrank --all` for both. Needs the repo + Node +
400
+ Python. *Codex input is estimated (`*`): alone \u2192 AA 2:1 baseline; with a Claude profile \u2192
401
+ your own Claude input:output ratio.*""")
402
  if _ON_SPACE:
403
  gr.LoginButton(elem_id="hf-login-btn")
404
  else: