Devin AI Deric J. McHenry commited on
Commit
d8929f9
·
1 Parent(s): 9e55406

Apply @GPU to narrate(), remove dead ingest(), expand README

Browse files
Files changed (4) hide show
  1. README.md +102 -3
  2. SCRATCHPAD.md +17 -0
  3. ingest.py +0 -11
  4. narrate.py +1 -0
README.md CHANGED
@@ -22,8 +22,29 @@ tags:
22
 
23
  A leaderboard that judges AI coding operators by **architecture, not budget**.
24
  Paste your token usage; get an operator profile with a tiny-model narration and
25
- your rank. The sort metric **Υ = (Cache·Output)/Input²** penalizes raw-input
26
- padding quadratically — volume can't buy rank.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
 
28
  ## What it does
29
  Paste `npx ccusage@latest --json` (Claude Code), `ccusage codex --json` (Codex),
@@ -37,6 +58,68 @@ or four numbers →
37
  narrates the operator read. It is **non-blocking**: if unavailable, a deterministic
38
  template is used and the app still works. Everything quantitative is pure computation.
39
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
40
  ## How the numbers work
41
  Four raw integers — `input`, `output`, `cache_create`, `cache_read` — drive all:
42
 
@@ -54,6 +137,20 @@ Four raw integers — `input`, `output`, `cache_create`, `cache_read` — drive
54
  Cascade (10x DEV) = transmission (O/I) × commitment (Create/O) × compounding
55
  (Read/Create); its log-sum is the exponent. By telescoping, 10^X = Leverage = C/I.
56
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
57
  ## Codex support
58
  Codex reports a *combined* input figure (fresh + cached) and never itemizes cache
59
  writes. SigRank splits it with the measured field anchor **input:output ≈ 2:1**
@@ -88,4 +185,6 @@ cheapest per token, by an order of magnitude.
88
  (Landauer, Ohmic dissipation); log Υ = X + log(Velocity). An analogy, not a
89
  microscopic-entropy derivation.
90
 
91
- Built for the HF/Gradio Build Small Hackathon · Thousand Token Wood 🍄 · MO§ES™
 
 
 
22
 
23
  A leaderboard that judges AI coding operators by **architecture, not budget**.
24
  Paste your token usage; get an operator profile with a tiny-model narration and
25
+ your rank. The ranking metric **Υ = (Cache·Output)/Input²** penalizes raw-input
26
+ padding quadratically — volume can't buy rank — but Υ is only the headline of a
27
+ larger metric system whose mathematical thesis is the **cascade decomposition**.
28
+
29
+ ## Origin
30
+ SigRank grew out of an earlier attempt to rank AI coding operators by *words* —
31
+ measuring commitment, output quality, and reuse through language rather than
32
+ numbers. That work produced a **conservation law of commitment**: the insight
33
+ that compounding architectures leave a measurable signature regardless of the
34
+ instrument used to observe them.
35
+
36
+ When token-level data became available via ccusage, the word-based framework
37
+ translated directly: commitment → `cache_create/output`, reuse → `cache_read/cache_create`,
38
+ transmission → `output/input`. The cascade identity `(O/I)×(C_w/O)×(C_r/C_w) = C_r/I`
39
+ is the token-domain expression of the same conservation law.
40
+
41
+ *[Full theoretical lineage — placeholder for extended write-up]*
42
+
43
+ ### Academic work & references
44
+ <!-- TODO: add links to the word-based ranking / conservation-law write-up,
45
+ any preprints, talks, or external references here. -->
46
+ - *[Conservation law of commitment — link placeholder]*
47
+ - *[Word-based operator ranking — link placeholder]*
48
 
49
  ## What it does
50
  Paste `npx ccusage@latest --json` (Claude Code), `ccusage codex --json` (Codex),
 
58
  narrates the operator read. It is **non-blocking**: if unavailable, a deterministic
59
  template is used and the app still works. Everything quantitative is pure computation.
60
 
61
+ ## The full metric system
62
+ Υ is the *ranking* metric. The *mathematical thesis* is the cascade decomposition
63
+ of leverage into three behavioral stages — Υ is what you sort by, the cascade is
64
+ what explains *why* an operator lands where it does.
65
+
66
+ ### The cascade as a diagnostic
67
+ Leverage `C_r/I` is not a single number; it factors into three compounding stages:
68
+
69
+ ```
70
+ (O/I) × (C_w/O) × (C_r/C_w) = C_r/I
71
+ │ │ │ │
72
+ transmission commitment compounding leverage
73
+ generate commit to reuse what
74
+ output cache was cached
75
+ ```
76
+
77
+ `10x DEV` is the **log₁₀** of that product — the amplification *exponent*. By
78
+ telescoping, `10^(10x DEV) = leverage = C_r/I`. The cascade tells you which stage
79
+ an operator is winning or losing on, not just the bottom line.
80
+
81
+ ### Operator archetypes (`classify()` in `app.py`)
82
+ Velocity (`O/I`) and leverage (`C_r/I`) place each operator in a behavioral class:
83
+
84
+ | archetype | signature |
85
+ |---|---|
86
+ | **Closed-Loop Kinetic** | holds both axes — velocity ≥ 1 *and* leverage ≥ 100 |
87
+ | **Archival Sponge** | high reuse, low generation — leverage ≥ 10, velocity < 1 |
88
+ | **Volatile Ingestor** | generates, doesn't retain — velocity ≥ 0.8, leverage < 2 |
89
+ | **Transient** | low on both axes |
90
+ | **Non-Compounding** | stateless pipe — no `cache_create`, so the cascade can't form |
91
+
92
+ ### Scale (V) — the volume axis
93
+ **V = log₁₀(total tokens)** measures how much volume an operator moves. Scale and
94
+ amplification are *independent* axes: a huge operator can have Υ ≈ 0, and a small
95
+ one can have high Υ. That independence is what produces the species map below.
96
+
97
+ ### Species / quadrants (`species_cards.md`)
98
+ Plot **Scale (V)** against **Amplification (Υ)** and operators sort into four
99
+ species by *which term of the math dominates* — not by who is "better":
100
+
101
+ ```
102
+ high amplification
103
+
104
+ CASCADE (stacks) │ (— rare / empty —)
105
+ low scale ◄───────┼───────► high scale
106
+ CONVERTER (I→O) │ THROUGHPUT (volume)
107
+ CACHE ARCHITECT │ (reuse)
108
+
109
+ low amplification
110
+ ```
111
+
112
+ | species | dominant term | signature |
113
+ |---|---|---|
114
+ | **Throughput** | raw volume | enormous total, Υ≈0 — scale without compounding |
115
+ | **Converter** | transmission `O/I` | high velocity, low leverage |
116
+ | **Cache Architect** | reuse `C/I` | high leverage, low velocity |
117
+ | **Cascade** | all three compound | low scale, high amplification |
118
+
119
+ MO§ES occupies the **empty quadrant** — low scale, high amplification. The claim
120
+ is *not* "top of a ladder"; it's that this region of the token economy is empty,
121
+ and the geometry of `(C·O)/I²` is what makes it empty.
122
+
123
  ## How the numbers work
124
  Four raw integers — `input`, `output`, `cache_create`, `cache_read` — drive all:
125
 
 
137
  Cascade (10x DEV) = transmission (O/I) × commitment (Create/O) × compounding
138
  (Read/Create); its log-sum is the exponent. By telescoping, 10^X = Leverage = C/I.
139
 
140
+ ## Benchmark convergence
141
+ SigRank's finding is corroborated by a second, independent instrument.
142
+
143
+ - **Artificial Analysis (AA)** benchmarks measure *models* at a **7:2:1**
144
+ (cache : input : output) token mix.
145
+ - **SigRank** measures *operators / users* from their own token logs.
146
+ - Both instruments converge on the same result: **cache-dominant architecture is
147
+ the most efficient AND the cheapest per token.**
148
+
149
+ The AA **7:2:1** ratio is the source of the **4.0 efficiency baseline** used in
150
+ the Efficiency metric: `(7 + 1) / 2 = 4.0`. Two independent instruments — model
151
+ benchmarks on one side, real user token ledgers on the other — landing on the
152
+ same architecture is the validation.
153
+
154
  ## Codex support
155
  Codex reports a *combined* input figure (fresh + cached) and never itemizes cache
156
  writes. SigRank splits it with the measured field anchor **input:output ≈ 2:1**
 
185
  (Landauer, Ohmic dissipation); log Υ = X + log(Velocity). An analogy, not a
186
  microscopic-entropy derivation.
187
 
188
+ Built for the HF/Gradio Build Small Hackathon · Thousand Token Wood 🍄 · MO§ES™
189
+ SigRank began as a word-based ranking of AI operators and a conservation law of
190
+ commitment, then translated into token-domain measurement.
SCRATCHPAD.md CHANGED
@@ -40,6 +40,23 @@ cd /Users/dericmchenry/Desktop/moses-sigrank
40
 
41
  ## WORK LOG (newest first)
42
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
  ### Claude/Opus session — 2026-06-15
44
  - [done] Verified baseline green (compile + canonical metrics print correct).
45
  - [done] FIX 6 — confirmed board CSS grid = 8 tracks, matches 8 header cells (theme.py:58). No code change needed; visual eyeball still owner's call.
 
40
 
41
  ## WORK LOG (newest first)
42
 
43
+ ### Devin session — 2026-06-15
44
+ - [done] GPU FIX — applied `@GPU` decorator to `narrate()` in narrate.py (line 60).
45
+ ZeroGPU needs the inference function decorated or GPU alloc never happens and the
46
+ model silently template-falls-back every call. `_try_load()` intentionally NOT
47
+ decorated. No-op fallback path verified (no torch → template, import OK).
48
+ - [done] DEAD-CODE FIX — removed unused `ingest()` from ingest.py (old lines 103-112).
49
+ Only `ingest_meta()` is called (app.py, sigrank.py). `ingest_meta()` unchanged.
50
+ - [done] README rewrite — kept YAML front matter + all math/formulas/SEED identical.
51
+ Added Origin (word-based ranking → conservation law of commitment → token domain,
52
+ with academic-links placeholder), "The full metric system" (cascade diagnostic,
53
+ 4 archetypes from classify(), Scale V, species/quadrant framing — MO§ES = empty
54
+ quadrant not a ladder), "Benchmark convergence" (AA 7:2:1 → 4.0 baseline = (7+1)/2,
55
+ two-instrument validation). Restructured so Υ is the ranking metric within the
56
+ broader system. Demo/Social placeholders untouched. Footer updated with origin.
57
+ - [done] Verify green: compile OK · `python3 metrics.py` → MO§ES Y 18436.98, lev
58
+ 2042.2, 10xDEV 3.31, $/1M 0.527 (unchanged).
59
+
60
  ### Claude/Opus session — 2026-06-15
61
  - [done] Verified baseline green (compile + canonical metrics print correct).
62
  - [done] FIX 6 — confirmed board CSS grid = 8 tracks, matches 8 header cells (theme.py:58). No code change needed; visual eyeball still owner's call.
ingest.py CHANGED
@@ -100,17 +100,6 @@ def parse_codex(text):
100
  "cost": tot["cost"] if tot["cost"] > 0 else None}
101
  return I, O, create, read, meta
102
 
103
- def ingest(text):
104
- """Returns (i,o,cw,cr)."""
105
- text=text.strip()
106
- if not text: raise ValueError("empty")
107
- if text[0] in "{[":
108
- d=json.loads(text)
109
- if is_codex_shape(d):
110
- i,o,cw,cr,_m = parse_codex(d); return i,o,cw,cr
111
- i,o,cw,cr,_c = parse_ccusage(text); return i,o,cw,cr
112
- return parse_four(text)
113
-
114
  def ingest_meta(text):
115
  """Returns (i,o,cw,cr,meta) with estimated/caveat/cost."""
116
  text=text.strip()
 
100
  "cost": tot["cost"] if tot["cost"] > 0 else None}
101
  return I, O, create, read, meta
102
 
 
 
 
 
 
 
 
 
 
 
 
103
  def ingest_meta(text):
104
  """Returns (i,o,cw,cr,meta) with estimated/caveat/cost."""
105
  text=text.strip()
narrate.py CHANGED
@@ -57,6 +57,7 @@ def _template(name, m, klass):
57
  f"converting input to output efficiently.")
58
  return f"**{klass}.** {body}"
59
 
 
60
  def narrate(name, m, klass):
61
  if not _try_load():
62
  return _template(name, m, klass)
 
57
  f"converting input to output efficiently.")
58
  return f"**{klass}.** {body}"
59
 
60
+ @GPU
61
  def narrate(name, m, klass):
62
  if not _try_load():
63
  return _template(name, m, klass)