Devin AI Deric J. McHenry commited on
Commit ·
d8929f9
1
Parent(s): 9e55406
Apply @GPU to narrate(), remove dead ingest(), expand README
Browse files- README.md +102 -3
- SCRATCHPAD.md +17 -0
- ingest.py +0 -11
- narrate.py +1 -0
README.md
CHANGED
|
@@ -22,8 +22,29 @@ tags:
|
|
| 22 |
|
| 23 |
A leaderboard that judges AI coding operators by **architecture, not budget**.
|
| 24 |
Paste your token usage; get an operator profile with a tiny-model narration and
|
| 25 |
-
your rank. The
|
| 26 |
-
padding quadratically — volume can't buy rank
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
## What it does
|
| 29 |
Paste `npx ccusage@latest --json` (Claude Code), `ccusage codex --json` (Codex),
|
|
@@ -37,6 +58,68 @@ or four numbers →
|
|
| 37 |
narrates the operator read. It is **non-blocking**: if unavailable, a deterministic
|
| 38 |
template is used and the app still works. Everything quantitative is pure computation.
|
| 39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
## How the numbers work
|
| 41 |
Four raw integers — `input`, `output`, `cache_create`, `cache_read` — drive all:
|
| 42 |
|
|
@@ -54,6 +137,20 @@ Four raw integers — `input`, `output`, `cache_create`, `cache_read` — drive
|
|
| 54 |
Cascade (10x DEV) = transmission (O/I) × commitment (Create/O) × compounding
|
| 55 |
(Read/Create); its log-sum is the exponent. By telescoping, 10^X = Leverage = C/I.
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
## Codex support
|
| 58 |
Codex reports a *combined* input figure (fresh + cached) and never itemizes cache
|
| 59 |
writes. SigRank splits it with the measured field anchor **input:output ≈ 2:1**
|
|
@@ -88,4 +185,6 @@ cheapest per token, by an order of magnitude.
|
|
| 88 |
(Landauer, Ohmic dissipation); log Υ = X + log(Velocity). An analogy, not a
|
| 89 |
microscopic-entropy derivation.
|
| 90 |
|
| 91 |
-
Built for the HF/Gradio Build Small Hackathon · Thousand Token Wood 🍄 · MO§ES™
|
|
|
|
|
|
|
|
|
| 22 |
|
| 23 |
A leaderboard that judges AI coding operators by **architecture, not budget**.
|
| 24 |
Paste your token usage; get an operator profile with a tiny-model narration and
|
| 25 |
+
your rank. The ranking metric **Υ = (Cache·Output)/Input²** penalizes raw-input
|
| 26 |
+
padding quadratically — volume can't buy rank — but Υ is only the headline of a
|
| 27 |
+
larger metric system whose mathematical thesis is the **cascade decomposition**.
|
| 28 |
+
|
| 29 |
+
## Origin
|
| 30 |
+
SigRank grew out of an earlier attempt to rank AI coding operators by *words* —
|
| 31 |
+
measuring commitment, output quality, and reuse through language rather than
|
| 32 |
+
numbers. That work produced a **conservation law of commitment**: the insight
|
| 33 |
+
that compounding architectures leave a measurable signature regardless of the
|
| 34 |
+
instrument used to observe them.
|
| 35 |
+
|
| 36 |
+
When token-level data became available via ccusage, the word-based framework
|
| 37 |
+
translated directly: commitment → `cache_create/output`, reuse → `cache_read/cache_create`,
|
| 38 |
+
transmission → `output/input`. The cascade identity `(O/I)×(C_w/O)×(C_r/C_w) = C_r/I`
|
| 39 |
+
is the token-domain expression of the same conservation law.
|
| 40 |
+
|
| 41 |
+
*[Full theoretical lineage — placeholder for extended write-up]*
|
| 42 |
+
|
| 43 |
+
### Academic work & references
|
| 44 |
+
<!-- TODO: add links to the word-based ranking / conservation-law write-up,
|
| 45 |
+
any preprints, talks, or external references here. -->
|
| 46 |
+
- *[Conservation law of commitment — link placeholder]*
|
| 47 |
+
- *[Word-based operator ranking — link placeholder]*
|
| 48 |
|
| 49 |
## What it does
|
| 50 |
Paste `npx ccusage@latest --json` (Claude Code), `ccusage codex --json` (Codex),
|
|
|
|
| 58 |
narrates the operator read. It is **non-blocking**: if unavailable, a deterministic
|
| 59 |
template is used and the app still works. Everything quantitative is pure computation.
|
| 60 |
|
| 61 |
+
## The full metric system
|
| 62 |
+
Υ is the *ranking* metric. The *mathematical thesis* is the cascade decomposition
|
| 63 |
+
of leverage into three behavioral stages — Υ is what you sort by, the cascade is
|
| 64 |
+
what explains *why* an operator lands where it does.
|
| 65 |
+
|
| 66 |
+
### The cascade as a diagnostic
|
| 67 |
+
Leverage `C_r/I` is not a single number; it factors into three compounding stages:
|
| 68 |
+
|
| 69 |
+
```
|
| 70 |
+
(O/I) × (C_w/O) × (C_r/C_w) = C_r/I
|
| 71 |
+
│ │ │ │
|
| 72 |
+
transmission commitment compounding leverage
|
| 73 |
+
generate commit to reuse what
|
| 74 |
+
output cache was cached
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
`10x DEV` is the **log₁₀** of that product — the amplification *exponent*. By
|
| 78 |
+
telescoping, `10^(10x DEV) = leverage = C_r/I`. The cascade tells you which stage
|
| 79 |
+
an operator is winning or losing on, not just the bottom line.
|
| 80 |
+
|
| 81 |
+
### Operator archetypes (`classify()` in `app.py`)
|
| 82 |
+
Velocity (`O/I`) and leverage (`C_r/I`) place each operator in a behavioral class:
|
| 83 |
+
|
| 84 |
+
| archetype | signature |
|
| 85 |
+
|---|---|
|
| 86 |
+
| **Closed-Loop Kinetic** | holds both axes — velocity ≥ 1 *and* leverage ≥ 100 |
|
| 87 |
+
| **Archival Sponge** | high reuse, low generation — leverage ≥ 10, velocity < 1 |
|
| 88 |
+
| **Volatile Ingestor** | generates, doesn't retain — velocity ≥ 0.8, leverage < 2 |
|
| 89 |
+
| **Transient** | low on both axes |
|
| 90 |
+
| **Non-Compounding** | stateless pipe — no `cache_create`, so the cascade can't form |
|
| 91 |
+
|
| 92 |
+
### Scale (V) — the volume axis
|
| 93 |
+
**V = log₁₀(total tokens)** measures how much volume an operator moves. Scale and
|
| 94 |
+
amplification are *independent* axes: a huge operator can have Υ ≈ 0, and a small
|
| 95 |
+
one can have high Υ. That independence is what produces the species map below.
|
| 96 |
+
|
| 97 |
+
### Species / quadrants (`species_cards.md`)
|
| 98 |
+
Plot **Scale (V)** against **Amplification (Υ)** and operators sort into four
|
| 99 |
+
species by *which term of the math dominates* — not by who is "better":
|
| 100 |
+
|
| 101 |
+
```
|
| 102 |
+
high amplification
|
| 103 |
+
│
|
| 104 |
+
CASCADE (stacks) │ (— rare / empty —)
|
| 105 |
+
low scale ◄───────┼───────► high scale
|
| 106 |
+
CONVERTER (I→O) │ THROUGHPUT (volume)
|
| 107 |
+
CACHE ARCHITECT │ (reuse)
|
| 108 |
+
│
|
| 109 |
+
low amplification
|
| 110 |
+
```
|
| 111 |
+
|
| 112 |
+
| species | dominant term | signature |
|
| 113 |
+
|---|---|---|
|
| 114 |
+
| **Throughput** | raw volume | enormous total, Υ≈0 — scale without compounding |
|
| 115 |
+
| **Converter** | transmission `O/I` | high velocity, low leverage |
|
| 116 |
+
| **Cache Architect** | reuse `C/I` | high leverage, low velocity |
|
| 117 |
+
| **Cascade** | all three compound | low scale, high amplification |
|
| 118 |
+
|
| 119 |
+
MO§ES occupies the **empty quadrant** — low scale, high amplification. The claim
|
| 120 |
+
is *not* "top of a ladder"; it's that this region of the token economy is empty,
|
| 121 |
+
and the geometry of `(C·O)/I²` is what makes it empty.
|
| 122 |
+
|
| 123 |
## How the numbers work
|
| 124 |
Four raw integers — `input`, `output`, `cache_create`, `cache_read` — drive all:
|
| 125 |
|
|
|
|
| 137 |
Cascade (10x DEV) = transmission (O/I) × commitment (Create/O) × compounding
|
| 138 |
(Read/Create); its log-sum is the exponent. By telescoping, 10^X = Leverage = C/I.
|
| 139 |
|
| 140 |
+
## Benchmark convergence
|
| 141 |
+
SigRank's finding is corroborated by a second, independent instrument.
|
| 142 |
+
|
| 143 |
+
- **Artificial Analysis (AA)** benchmarks measure *models* at a **7:2:1**
|
| 144 |
+
(cache : input : output) token mix.
|
| 145 |
+
- **SigRank** measures *operators / users* from their own token logs.
|
| 146 |
+
- Both instruments converge on the same result: **cache-dominant architecture is
|
| 147 |
+
the most efficient AND the cheapest per token.**
|
| 148 |
+
|
| 149 |
+
The AA **7:2:1** ratio is the source of the **4.0 efficiency baseline** used in
|
| 150 |
+
the Efficiency metric: `(7 + 1) / 2 = 4.0`. Two independent instruments — model
|
| 151 |
+
benchmarks on one side, real user token ledgers on the other — landing on the
|
| 152 |
+
same architecture is the validation.
|
| 153 |
+
|
| 154 |
## Codex support
|
| 155 |
Codex reports a *combined* input figure (fresh + cached) and never itemizes cache
|
| 156 |
writes. SigRank splits it with the measured field anchor **input:output ≈ 2:1**
|
|
|
|
| 185 |
(Landauer, Ohmic dissipation); log Υ = X + log(Velocity). An analogy, not a
|
| 186 |
microscopic-entropy derivation.
|
| 187 |
|
| 188 |
+
Built for the HF/Gradio Build Small Hackathon · Thousand Token Wood 🍄 · MO§ES™ —
|
| 189 |
+
SigRank began as a word-based ranking of AI operators and a conservation law of
|
| 190 |
+
commitment, then translated into token-domain measurement.
|
SCRATCHPAD.md
CHANGED
|
@@ -40,6 +40,23 @@ cd /Users/dericmchenry/Desktop/moses-sigrank
|
|
| 40 |
|
| 41 |
## WORK LOG (newest first)
|
| 42 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 43 |
### Claude/Opus session — 2026-06-15
|
| 44 |
- [done] Verified baseline green (compile + canonical metrics print correct).
|
| 45 |
- [done] FIX 6 — confirmed board CSS grid = 8 tracks, matches 8 header cells (theme.py:58). No code change needed; visual eyeball still owner's call.
|
|
|
|
| 40 |
|
| 41 |
## WORK LOG (newest first)
|
| 42 |
|
| 43 |
+
### Devin session — 2026-06-15
|
| 44 |
+
- [done] GPU FIX — applied `@GPU` decorator to `narrate()` in narrate.py (line 60).
|
| 45 |
+
ZeroGPU needs the inference function decorated or GPU alloc never happens and the
|
| 46 |
+
model silently template-falls-back every call. `_try_load()` intentionally NOT
|
| 47 |
+
decorated. No-op fallback path verified (no torch → template, import OK).
|
| 48 |
+
- [done] DEAD-CODE FIX — removed unused `ingest()` from ingest.py (old lines 103-112).
|
| 49 |
+
Only `ingest_meta()` is called (app.py, sigrank.py). `ingest_meta()` unchanged.
|
| 50 |
+
- [done] README rewrite — kept YAML front matter + all math/formulas/SEED identical.
|
| 51 |
+
Added Origin (word-based ranking → conservation law of commitment → token domain,
|
| 52 |
+
with academic-links placeholder), "The full metric system" (cascade diagnostic,
|
| 53 |
+
4 archetypes from classify(), Scale V, species/quadrant framing — MO§ES = empty
|
| 54 |
+
quadrant not a ladder), "Benchmark convergence" (AA 7:2:1 → 4.0 baseline = (7+1)/2,
|
| 55 |
+
two-instrument validation). Restructured so Υ is the ranking metric within the
|
| 56 |
+
broader system. Demo/Social placeholders untouched. Footer updated with origin.
|
| 57 |
+
- [done] Verify green: compile OK · `python3 metrics.py` → MO§ES Y 18436.98, lev
|
| 58 |
+
2042.2, 10xDEV 3.31, $/1M 0.527 (unchanged).
|
| 59 |
+
|
| 60 |
### Claude/Opus session — 2026-06-15
|
| 61 |
- [done] Verified baseline green (compile + canonical metrics print correct).
|
| 62 |
- [done] FIX 6 — confirmed board CSS grid = 8 tracks, matches 8 header cells (theme.py:58). No code change needed; visual eyeball still owner's call.
|
ingest.py
CHANGED
|
@@ -100,17 +100,6 @@ def parse_codex(text):
|
|
| 100 |
"cost": tot["cost"] if tot["cost"] > 0 else None}
|
| 101 |
return I, O, create, read, meta
|
| 102 |
|
| 103 |
-
def ingest(text):
|
| 104 |
-
"""Returns (i,o,cw,cr)."""
|
| 105 |
-
text=text.strip()
|
| 106 |
-
if not text: raise ValueError("empty")
|
| 107 |
-
if text[0] in "{[":
|
| 108 |
-
d=json.loads(text)
|
| 109 |
-
if is_codex_shape(d):
|
| 110 |
-
i,o,cw,cr,_m = parse_codex(d); return i,o,cw,cr
|
| 111 |
-
i,o,cw,cr,_c = parse_ccusage(text); return i,o,cw,cr
|
| 112 |
-
return parse_four(text)
|
| 113 |
-
|
| 114 |
def ingest_meta(text):
|
| 115 |
"""Returns (i,o,cw,cr,meta) with estimated/caveat/cost."""
|
| 116 |
text=text.strip()
|
|
|
|
| 100 |
"cost": tot["cost"] if tot["cost"] > 0 else None}
|
| 101 |
return I, O, create, read, meta
|
| 102 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 103 |
def ingest_meta(text):
|
| 104 |
"""Returns (i,o,cw,cr,meta) with estimated/caveat/cost."""
|
| 105 |
text=text.strip()
|
narrate.py
CHANGED
|
@@ -57,6 +57,7 @@ def _template(name, m, klass):
|
|
| 57 |
f"converting input to output efficiently.")
|
| 58 |
return f"**{klass}.** {body}"
|
| 59 |
|
|
|
|
| 60 |
def narrate(name, m, klass):
|
| 61 |
if not _try_load():
|
| 62 |
return _template(name, m, klass)
|
|
|
|
| 57 |
f"converting input to output efficiently.")
|
| 58 |
return f"**{klass}.** {body}"
|
| 59 |
|
| 60 |
+
@GPU
|
| 61 |
def narrate(name, m, klass):
|
| 62 |
if not _try_load():
|
| 63 |
return _template(name, m, klass)
|