Retract input-injection claim; correct batch-size pooling error
Browse files
README.md
CHANGED
|
@@ -35,7 +35,7 @@ digit relabelling. Metric: board accuracy, all 81 cells correct.
|
|
| 35 |
| greedy MRV | β | β | 5.90% |
|
| 36 |
| autoregressive prefix-LM | 2.4M | β | 0.30% |
|
| 37 |
| direct recurrent classifier* | 212k | 10.5% | 31.00% |
|
| 38 |
-
| recurrent, no input injection | 212k | β | 94.10% |
|
| 39 |
| feed-forward diffusion, MDLM | 37.9M | 4.40% | 88.15% |
|
| 40 |
| feed-forward diffusion, schedule-matched | 37.9M | 40.10% | 88.90% |
|
| 41 |
| 32 distinct layers, no weight sharing | 6.34M | 70.20% | 94.50% |
|
|
@@ -50,20 +50,32 @@ evidence against their method**, and it is not used as a comparison.
|
|
| 50 |
|
| 51 |
## Ablations
|
| 52 |
|
| 53 |
-
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
-
|
| 56 |
-
|---|---|---|
|
| 57 |
-
| feed-forward, 12 layers, 37.9M | 88.90% | β |
|
| 58 |
-
| feed-forward, **32 layers**, 6.34M | 88.95% | **+0.05** |
|
| 59 |
-
| + weight sharing (injection **off**), 212k | 94.10% | **+5.15** |
|
| 60 |
-
| + input injection | **99.70%** | **+5.60** |
|
| 61 |
|
| 62 |
-
|
| 63 |
-
|
| 64 |
-
|
| 65 |
-
|
| 66 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
|
| 68 |
Removing either from the full method: deep supervision **β1.95**, schedule
|
| 69 |
matching **β3.15**.
|
|
@@ -91,8 +103,15 @@ supervision is dominated by copying the given clues. Low loss, no capability.
|
|
| 91 |
clue range sits inside the Recurrent Transformer's 17β34 but excludes the
|
| 92 |
hardest 17-clue instances, so 99.70% at 212k reads as *competitive with* their
|
| 93 |
99.5% at 211k, not better.
|
| 94 |
-
- **Input injection is
|
| 95 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 96 |
- The SATNet split is not used: constraint propagation alone solves 100% of it.
|
| 97 |
|
| 98 |
## Negative results, reported
|
|
|
|
| 35 |
| greedy MRV | β | β | 5.90% |
|
| 36 |
| autoregressive prefix-LM | 2.4M | β | 0.30% |
|
| 37 |
| direct recurrent classifier* | 212k | 10.5% | 31.00% |
|
| 38 |
+
| recurrent, no input injection (batch 64) | 212k | β | 94.10% |
|
| 39 |
| feed-forward diffusion, MDLM | 37.9M | 4.40% | 88.15% |
|
| 40 |
| feed-forward diffusion, schedule-matched | 37.9M | 40.10% | 88.90% |
|
| 41 |
| 32 distinct layers, no weight sharing | 6.34M | 70.20% | 94.50% |
|
|
|
|
| 50 |
|
| 51 |
## Ablations
|
| 52 |
|
| 53 |
+
**Compare only within a batch size.** An earlier version of this card pooled a
|
| 54 |
+
32-layer run at batch 256 (94.50%) with one at batch 64 (83.40%) into a single
|
| 55 |
+
"88.95%", and drew two conclusions from the artefact. Both were wrong and are
|
| 56 |
+
corrected here.
|
| 57 |
|
| 58 |
+
**At batch 256:**
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 59 |
|
| 60 |
+
| step | params | board | gain |
|
| 61 |
+
|---|---|---|---|
|
| 62 |
+
| feed-forward, d=512, 12 layers | 37.9M | 88.90% (n=2) | β |
|
| 63 |
+
| feed-forward, d=128, 32 layers | 6.34M | 94.50% (n=1) | **+5.60** |
|
| 64 |
+
| + weight sharing, d=128, 1 layer Γ 32 | **212k** | **99.70%** (n=2) | **+5.20** |
|
| 65 |
+
|
| 66 |
+
Parameters fall monotonically, accuracy rises monotonically: 37.9M β 6.34M β
|
| 67 |
+
212k against 88.90% β 94.50% β 99.70%.
|
| 68 |
+
|
| 69 |
+
**At batch 64, injection off on both sides:**
|
| 70 |
+
|
| 71 |
+
| step | params | board | gain |
|
| 72 |
+
|---|---|---|---|
|
| 73 |
+
| feed-forward, d=128, 32 layers | 6.34M | 83.40% (n=1) | β |
|
| 74 |
+
| + weight sharing | **212k** | **94.10%** (n=2) | **+10.70** |
|
| 75 |
+
|
| 76 |
+
**Corrections.** ~~"Depth alone is worth nothing (+0.05)"~~ β wrong and backwards;
|
| 77 |
+
at matched batch size the deeper, narrower, 6Γ smaller model gains **+5.60**.
|
| 78 |
+
~~"Input injection is worth +5.60"~~ β **retracted**, see below.
|
| 79 |
|
| 80 |
Removing either from the full method: deep supervision **β1.95**, schedule
|
| 81 |
matching **β3.15**.
|
|
|
|
| 103 |
clue range sits inside the Recurrent Transformer's 17β34 but excludes the
|
| 104 |
hardest 17-clue instances, so 99.70% at 212k reads as *competitive with* their
|
| 105 |
99.5% at 211k, not better.
|
| 106 |
+
- **Input injection is retracted as a contribution.** In a matched test β
|
| 107 |
+
identical architecture, batch, and step count, with R-MDM's step embedding on
|
| 108 |
+
both sides β it is worth **+0.05 points** (96.85% β 96.90%, best decode). The
|
| 109 |
+
previously claimed +5.60 compared an injection-ON arm at batch 256 against an
|
| 110 |
+
injection-OFF arm at batch 64; it measured batch size. The single injection-ON
|
| 111 |
+
seed lands inside the two-seed injection-OFF range at five of six decoding
|
| 112 |
+
budgets. All three arms stopped at step 80k of 110k on a wall-clock limit, and
|
| 113 |
+
the ON side is n=1; we could not add seeds, as the allocation had 3,030
|
| 114 |
+
billing-minutes left against ~3,600 for one run.
|
| 115 |
- The SATNet split is not used: constraint propagation alone solves 100% of it.
|
| 116 |
|
| 117 |
## Negative results, reported
|