Recurrent masked diffusion: results, code, card
Browse filesThis view is limited to 50 files because it contains too many changes. See raw diff
- README.md +56 -91
- figures/compare_recurrent.json +119 -0
- figures/mol_fair.json +106 -0
- figures/sudoku_compare.json +119 -0
- figures/sudoku_hard.json +86 -0
- figures/sudoku_remask.json +190 -0
- figures/sudoku_rsweep.json +42 -0
- figures/sudoku_validate.json +68 -0
- results/abl-deep32L-s0.json +141 -0
- results/abl-nodeepsup-s0.json +141 -0
- results/abl-nodeepsup-s1.json +141 -0
- results/base-ar-s0.json +36 -0
- results/base-directrec-s0.json +142 -0
- results/base-directrec-s1.json +142 -0
- results/fix-arprefix-s0.json +37 -0
- results/fix-directall-s0.json +143 -0
- results/fix-directall-s1.json +143 -0
- results/rec1-full-s0.json +140 -0
- results/rec1-full-s1.json +140 -0
- results/rec1-mdlm-s0.json +140 -0
- results/rec1-mdlm-s1.json +140 -0
- results/sudh-full-s0.json +137 -0
- results/sudh-full-s1.json +137 -0
- results/sudh-mdlm-s0.json +137 -0
- results/sudh-mdlm-s1.json +137 -0
- slurm/abl.sh +38 -0
- slurm/b_gpu-a100_1g.5gb --gres=gpu:a100_1g.5gb:1.sh +14 -0
- slurm/b_gpu-h100 --gpus=h100:1.sh +14 -0
- slurm/base.sh +40 -0
- slurm/bh.sh +14 -0
- slurm/bm.sh +14 -0
- slurm/cmpall.sh +15 -0
- slurm/fix.sh +33 -0
- slurm/genhard.sh +16 -0
- slurm/inj.sh +40 -0
- slurm/inj2.sh +41 -0
- slurm/mol1.sh +38 -0
- slurm/molfair.sh +15 -0
- slurm/rec1.sh +35 -0
- slurm/rsw.sh +15 -0
- slurm/sud1.sh +38 -0
- slurm/sudcompare.sh +15 -0
- slurm/sudhard.sh +15 -0
- slurm/sudhard_train.sh +37 -0
- slurm/sudremask.sh +15 -0
- src/bench.py +38 -0
- src/diffusion.py +114 -0
- src/gen_hard_sudoku.py +200 -0
- src/model.py +71 -0
- src/sudoku.py +50 -0
README.md
CHANGED
|
@@ -3,120 +3,85 @@ license: apache-2.0
|
|
| 3 |
tags:
|
| 4 |
- masked-diffusion
|
| 5 |
- discrete-diffusion
|
| 6 |
-
- parallel-decoding
|
| 7 |
- constraint-satisfaction
|
| 8 |
-
- text8
|
| 9 |
-
- qm9
|
| 10 |
- sudoku
|
|
|
|
| 11 |
- from-scratch
|
| 12 |
---
|
| 13 |
|
| 14 |
-
#
|
| 15 |
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
|
|
|
|
|
|
| 19 |
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
---
|
| 23 |
-
|
| 24 |
-
## The claim
|
| 25 |
-
|
| 26 |
-
Parallel decoding in masked diffusion loses accuracy for two reasons that need
|
| 27 |
-
**opposite** fixes, and the literature treats them as one.
|
| 28 |
-
|
| 29 |
-
| | failure mode | fixable by |
|
| 30 |
-
|---|---|---|
|
| 31 |
-
| **(i)** | the fully-masked state is never trained — a *K*-pass decode starts at `t = 1`, and continuous `t ~ U(0,1)` assigns that state probability **zero** | **training only** |
|
| 32 |
-
| **(ii)** | committed positions carry real mutual information — parallel decoding samples the product of marginals when the truth is the joint | **inference only** |
|
| 33 |
-
|
| 34 |
-
Prior work applies inference-time fixes to both, which is why those methods
|
| 35 |
-
flatline on arithmetic, where (ii) is identically zero.
|
| 36 |
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
For a data distribution `P` uniform on a family `F`, let `V*` be the one-pass
|
| 40 |
-
validity achievable from the **true** marginals. Then
|
| 41 |
-
|
| 42 |
-
```
|
| 43 |
-
V* ≥ 2^(−TC), TC = D_KL( P ‖ ∏ᵢ pᵢ ), equality iff ∏ᵢ pᵢ is constant on F
|
| 44 |
-
```
|
| 45 |
-
|
| 46 |
-
**Every bit of total correlation among jointly committed variables at most halves
|
| 47 |
-
the one-pass success probability.**
|
| 48 |
|
| 49 |
-
|
| 50 |
-
checked against Cayley's formula (spanning trees at n = 5, 6, 7 → 125, 1296,
|
| 51 |
-
16807):
|
| 52 |
|
| 53 |
-
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
| tree (n=6) | 3.435 | **9.2488%** | **9.2488%** | **1.00000** |
|
| 57 |
-
| matching (n=6) | 6.922 | **0.8246%** | **0.8246%** | **1.00000** |
|
| 58 |
-
| 2-regular (n=7) | 10.423 | **0.0728%** | **0.0728%** | **1.00000** |
|
| 59 |
|
| 60 |
-
|
| 61 |
-
the fixed-edge-count families, and fails precisely where edge counts vary — which
|
| 62 |
-
is what the Jensen step requires.
|
| 63 |
|
| 64 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 65 |
|
| 66 |
-
|
| 67 |
-
|
| 68 |
-
|
| 69 |
-
| text8, NFE 16 | 116.88 gen-ppl | **92.74** (−20.7%) |
|
| 70 |
-
| text8, validation BPC | 1.7292 | **1.7010** |
|
| 71 |
|
| 72 |
-
|
| 73 |
-
so it is not bought by trading one for the other.
|
| 74 |
|
| 75 |
-
|
| 76 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 77 |
|
| 78 |
-
##
|
| 79 |
|
| 80 |
-
|
| 81 |
-
relative encoding the attention score depends only on `(i − j)`, and identical
|
| 82 |
-
value vectors give identical outputs at every position. **Position-dependent
|
| 83 |
-
marginals are inexpressible at any training budget.**
|
| 84 |
|
| 85 |
-
|
|
| 86 |
-
|---|---|---|
|
| 87 |
-
|
|
| 88 |
-
| marginals **constant** (control) | **−0.024 pts** | **+0.000 pts** |
|
| 89 |
|
| 90 |
-
|
| 91 |
-
out capacity; the control rules out "absolute is simply better". Predicted 5/5
|
| 92 |
-
before measurement, and the treated arm lands at **100.3% of `V*`** — at the
|
| 93 |
-
bound, not merely higher.
|
| 94 |
|
| 95 |
-
##
|
| 96 |
|
| 97 |
-
|
| 98 |
-
|
|
|
|
|
|
|
| 99 |
|
| 100 |
-
|
| 101 |
|
| 102 |
-
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
| 109 |
-
Two passes exceed the published number; 32 passes solve **all 1,000** test
|
| 110 |
-
puzzles exactly. One pass — the entire grid committed at once — solves 87.85%.
|
| 111 |
-
|
| 112 |
-
Leakage audited first: 0/1000 test puzzles in training, 0/1000 test solutions in
|
| 113 |
-
training, and 0/1000 test solutions matching a training solution even up to digit
|
| 114 |
-
relabelling. All 10,000 solutions are distinct.
|
| 115 |
-
|
| 116 |
-
**Scoped honestly:** this comes from the *plain MDLM baseline* with fixed-`K`
|
| 117 |
-
decoding. Neither finding above contributes — the standard formulation already
|
| 118 |
-
saturates this benchmark. It is a result for masked diffusion on SATNet Sudoku,
|
| 119 |
-
not evidence for our method.
|
| 120 |
|
| 121 |
## Negative results, reported
|
| 122 |
|
|
|
|
| 3 |
tags:
|
| 4 |
- masked-diffusion
|
| 5 |
- discrete-diffusion
|
|
|
|
| 6 |
- constraint-satisfaction
|
|
|
|
|
|
|
| 7 |
- sudoku
|
| 8 |
+
- recurrent-transformer
|
| 9 |
- from-scratch
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Recurrent Masked Diffusion for Constraint Satisfaction
|
| 13 |
|
| 14 |
+
**Constraint propagation is an iterative algorithm; masked diffusion models are
|
| 15 |
+
trained as fixed-depth denoisers. Making the denoiser recurrent — one
|
| 16 |
+
weight-shared block applied many times, supervised at every application —
|
| 17 |
+
reaches 99.70% on hard Sudoku with 212k parameters, where a 37.9M-parameter
|
| 18 |
+
feed-forward denoiser reaches 88.90%.**
|
| 19 |
|
| 20 |
+
Trained from scratch. No pretrained weights, no pretrained tokenizer.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
+
Code: https://github.com/GIND123/MSML612
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
+
## Benchmark
|
|
|
|
|
|
|
| 25 |
|
| 26 |
+
1,000 held-out minimal Sudoku puzzles, 20–30 clues (mean 24.4). Uniqueness
|
| 27 |
+
verified after **every** cell removal; train/test solutions disjoint even up to
|
| 28 |
+
digit relabelling. Metric: board accuracy, all 81 cells correct.
|
|
|
|
|
|
|
|
|
|
| 29 |
|
| 30 |
+
## Results
|
|
|
|
|
|
|
| 31 |
|
| 32 |
+
| method | params | 1 pass | best |
|
| 33 |
+
|---|---|---|---|
|
| 34 |
+
| constraint propagation (no search) | – | – | 3.00% |
|
| 35 |
+
| greedy MRV | – | – | 5.90% |
|
| 36 |
+
| autoregressive prefix-LM | 2.4M | – | 0.30% |
|
| 37 |
+
| direct recurrent classifier* | 212k | 10.5% | 31.00% |
|
| 38 |
+
| feed-forward diffusion, MDLM | 37.9M | 4.40% | 88.15% |
|
| 39 |
+
| feed-forward diffusion, schedule-matched | 37.9M | 40.10% | 88.90% |
|
| 40 |
+
| 32 distinct layers, no weight sharing | 6.34M | 70.20% | 94.50% |
|
| 41 |
+
| recurrent diffusion, MDLM | 212k | 87.35% | 96.55% |
|
| 42 |
+
| recurrent, no deep supervision | 212k | 93.35% | 97.75% |
|
| 43 |
+
| **recurrent diffusion, full method** | **212k** | **97.35%** | **99.70%** |
|
| 44 |
+
| full backtracking search | – | – | 100% (423 nodes) |
|
| 45 |
|
| 46 |
+
\* Our reimplementation of the published recurrent classifier reaches 31%
|
| 47 |
+
against their reported 99.5%. **That is a failed reproduction on our side, not
|
| 48 |
+
evidence against their method**, and it is not used as a comparison.
|
|
|
|
|
|
|
| 49 |
|
| 50 |
+
## Ablations
|
|
|
|
| 51 |
|
| 52 |
+
| ingredient | gain |
|
| 53 |
+
|---|---|
|
| 54 |
+
| recurrence vs feed-forward | +10.80 |
|
| 55 |
+
| weight sharing vs 32 distinct layers (matched depth) | +3.75 |
|
| 56 |
+
| schedule matching | +3.15 |
|
| 57 |
+
| deep supervision | +1.95 |
|
| 58 |
|
| 59 |
+
## Recurrence extrapolates past its training depth
|
| 60 |
|
| 61 |
+
Trained at R = 32, evaluated elsewhere without retraining:
|
|
|
|
|
|
|
|
|
|
| 62 |
|
| 63 |
+
| R | 1 | 4 | 8 | 16 | **32** | **64** | 128 |
|
| 64 |
+
|---|---|---|---|---|---|---|---|
|
| 65 |
+
| board | 0.10% | 48.40% | 82.00% | 92.40% | **98.30%** | **99.40%** | 99.10% / 39.70% |
|
|
|
|
| 66 |
|
| 67 |
+
2× extrapolation is reliable; 4× collapses on one seed.
|
|
|
|
|
|
|
|
|
|
| 68 |
|
| 69 |
+
## The instructive failure
|
| 70 |
|
| 71 |
+
Both non-diffusion baselines reach **lower training loss** than our method and
|
| 72 |
+
solve almost nothing — AR at 0.0486 loss gives 0.30% boards, direct all-cell
|
| 73 |
+
supervision at 0.0722 gives 55%. AR's errors compound unrecoverably; all-cell
|
| 74 |
+
supervision is dominated by copying the given clues. Low loss, no capability.
|
| 75 |
|
| 76 |
+
## Honest limits
|
| 77 |
|
| 78 |
+
- Published comparison is across **different puzzle distributions**: our 20–30
|
| 79 |
+
clue range sits inside the Recurrent Transformer's 17–34 but excludes the
|
| 80 |
+
hardest 17-clue instances, so 99.70% at 212k reads as *competitive with* their
|
| 81 |
+
99.5% at 211k, not better.
|
| 82 |
+
- **Input injection is not separated from weight sharing** in the 32-layer
|
| 83 |
+
comparison; the isolating run is in progress.
|
| 84 |
+
- The SATNet split is not used: constraint propagation alone solves 100% of it.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
## Negative results, reported
|
| 87 |
|
figures/compare_recurrent.json
ADDED
|
@@ -0,0 +1,119 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"n": 1000,
|
| 3 |
+
"clues": {
|
| 4 |
+
"min": 21,
|
| 5 |
+
"max": 28,
|
| 6 |
+
"mean": 24.318
|
| 7 |
+
},
|
| 8 |
+
"classical": {
|
| 9 |
+
"propagation": 0.03,
|
| 10 |
+
"greedy_mrv": 0.059,
|
| 11 |
+
"backtracking": 1.0,
|
| 12 |
+
"backtrack_nodes_mean": 423.126,
|
| 13 |
+
"backtrack_nodes_median": 211.5
|
| 14 |
+
},
|
| 15 |
+
"diffusion": {
|
| 16 |
+
"recurrent-full": {
|
| 17 |
+
"1": {
|
| 18 |
+
"board": 0.974,
|
| 19 |
+
"per_seed": [
|
| 20 |
+
0.974,
|
| 21 |
+
0.974
|
| 22 |
+
]
|
| 23 |
+
},
|
| 24 |
+
"4": {
|
| 25 |
+
"board": 0.985,
|
| 26 |
+
"per_seed": [
|
| 27 |
+
0.982,
|
| 28 |
+
0.988
|
| 29 |
+
]
|
| 30 |
+
},
|
| 31 |
+
"8": {
|
| 32 |
+
"board": 0.9895,
|
| 33 |
+
"per_seed": [
|
| 34 |
+
0.987,
|
| 35 |
+
0.992
|
| 36 |
+
]
|
| 37 |
+
},
|
| 38 |
+
"16": {
|
| 39 |
+
"board": 0.992,
|
| 40 |
+
"per_seed": [
|
| 41 |
+
0.99,
|
| 42 |
+
0.994
|
| 43 |
+
]
|
| 44 |
+
},
|
| 45 |
+
"32": {
|
| 46 |
+
"board": 0.995,
|
| 47 |
+
"per_seed": [
|
| 48 |
+
0.993,
|
| 49 |
+
0.997
|
| 50 |
+
]
|
| 51 |
+
},
|
| 52 |
+
"61": {
|
| 53 |
+
"board": 0.9955,
|
| 54 |
+
"per_seed": [
|
| 55 |
+
0.994,
|
| 56 |
+
0.997
|
| 57 |
+
],
|
| 58 |
+
"by_clue": {
|
| 59 |
+
"22": 0.9767441860465116,
|
| 60 |
+
"23": 1.0,
|
| 61 |
+
"24": 0.9942028985507246,
|
| 62 |
+
"25": 0.9964539007092199,
|
| 63 |
+
"26": 0.984251968503937
|
| 64 |
+
}
|
| 65 |
+
}
|
| 66 |
+
},
|
| 67 |
+
"recurrent-mdlm": {
|
| 68 |
+
"1": {
|
| 69 |
+
"board": 0.8705,
|
| 70 |
+
"per_seed": [
|
| 71 |
+
0.911,
|
| 72 |
+
0.83
|
| 73 |
+
]
|
| 74 |
+
},
|
| 75 |
+
"4": {
|
| 76 |
+
"board": 0.9155,
|
| 77 |
+
"per_seed": [
|
| 78 |
+
0.938,
|
| 79 |
+
0.893
|
| 80 |
+
]
|
| 81 |
+
},
|
| 82 |
+
"8": {
|
| 83 |
+
"board": 0.9315,
|
| 84 |
+
"per_seed": [
|
| 85 |
+
0.94,
|
| 86 |
+
0.923
|
| 87 |
+
]
|
| 88 |
+
},
|
| 89 |
+
"16": {
|
| 90 |
+
"board": 0.947,
|
| 91 |
+
"per_seed": [
|
| 92 |
+
0.951,
|
| 93 |
+
0.943
|
| 94 |
+
]
|
| 95 |
+
},
|
| 96 |
+
"32": {
|
| 97 |
+
"board": 0.949,
|
| 98 |
+
"per_seed": [
|
| 99 |
+
0.954,
|
| 100 |
+
0.944
|
| 101 |
+
]
|
| 102 |
+
},
|
| 103 |
+
"61": {
|
| 104 |
+
"board": 0.9515,
|
| 105 |
+
"per_seed": [
|
| 106 |
+
0.957,
|
| 107 |
+
0.946
|
| 108 |
+
],
|
| 109 |
+
"by_clue": {
|
| 110 |
+
"22": 0.9534883720930233,
|
| 111 |
+
"23": 0.9723756906077348,
|
| 112 |
+
"24": 0.9652173913043478,
|
| 113 |
+
"25": 0.9468085106382979,
|
| 114 |
+
"26": 0.952755905511811
|
| 115 |
+
}
|
| 116 |
+
}
|
| 117 |
+
}
|
| 118 |
+
}
|
| 119 |
+
}
|
figures/mol_fair.json
ADDED
|
@@ -0,0 +1,106 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"matched-ape": {
|
| 3 |
+
"1": [
|
| 4 |
+
0.3076171875,
|
| 5 |
+
0.30078125
|
| 6 |
+
],
|
| 7 |
+
"4": [
|
| 8 |
+
0.5634765625,
|
| 9 |
+
0.5458984375
|
| 10 |
+
],
|
| 11 |
+
"8": [
|
| 12 |
+
0.70068359375,
|
| 13 |
+
0.7294921875
|
| 14 |
+
],
|
| 15 |
+
"16": [
|
| 16 |
+
0.8759765625,
|
| 17 |
+
0.86572265625
|
| 18 |
+
],
|
| 19 |
+
"32": [
|
| 20 |
+
0.97119140625,
|
| 21 |
+
0.970703125
|
| 22 |
+
],
|
| 23 |
+
"45": [
|
| 24 |
+
0.9951171875,
|
| 25 |
+
0.998046875
|
| 26 |
+
]
|
| 27 |
+
},
|
| 28 |
+
"matched-rope": {
|
| 29 |
+
"1": [
|
| 30 |
+
0.173828125,
|
| 31 |
+
0.17236328125
|
| 32 |
+
],
|
| 33 |
+
"4": [
|
| 34 |
+
0.40771484375,
|
| 35 |
+
0.4462890625
|
| 36 |
+
],
|
| 37 |
+
"8": [
|
| 38 |
+
0.6943359375,
|
| 39 |
+
0.5380859375
|
| 40 |
+
],
|
| 41 |
+
"16": [
|
| 42 |
+
0.74072265625,
|
| 43 |
+
0.57958984375
|
| 44 |
+
],
|
| 45 |
+
"32": [
|
| 46 |
+
0.7822265625,
|
| 47 |
+
0.62939453125
|
| 48 |
+
],
|
| 49 |
+
"45": [
|
| 50 |
+
0.7880859375,
|
| 51 |
+
0.705078125
|
| 52 |
+
]
|
| 53 |
+
},
|
| 54 |
+
"mdlm-ape": {
|
| 55 |
+
"1": [
|
| 56 |
+
0.3115234375,
|
| 57 |
+
0.30810546875
|
| 58 |
+
],
|
| 59 |
+
"4": [
|
| 60 |
+
0.5458984375,
|
| 61 |
+
0.5400390625
|
| 62 |
+
],
|
| 63 |
+
"8": [
|
| 64 |
+
0.7041015625,
|
| 65 |
+
0.72314453125
|
| 66 |
+
],
|
| 67 |
+
"16": [
|
| 68 |
+
0.85693359375,
|
| 69 |
+
0.86181640625
|
| 70 |
+
],
|
| 71 |
+
"32": [
|
| 72 |
+
0.9638671875,
|
| 73 |
+
0.9609375
|
| 74 |
+
],
|
| 75 |
+
"45": [
|
| 76 |
+
0.99853515625,
|
| 77 |
+
0.998046875
|
| 78 |
+
]
|
| 79 |
+
},
|
| 80 |
+
"mdlm-rope": {
|
| 81 |
+
"1": [
|
| 82 |
+
0.18359375,
|
| 83 |
+
0.166015625
|
| 84 |
+
],
|
| 85 |
+
"4": [
|
| 86 |
+
0.4169921875,
|
| 87 |
+
0.41845703125
|
| 88 |
+
],
|
| 89 |
+
"8": [
|
| 90 |
+
0.7265625,
|
| 91 |
+
0.68505859375
|
| 92 |
+
],
|
| 93 |
+
"16": [
|
| 94 |
+
0.7861328125,
|
| 95 |
+
0.75732421875
|
| 96 |
+
],
|
| 97 |
+
"32": [
|
| 98 |
+
0.92578125,
|
| 99 |
+
0.75
|
| 100 |
+
],
|
| 101 |
+
"45": [
|
| 102 |
+
0.97607421875,
|
| 103 |
+
0.806640625
|
| 104 |
+
]
|
| 105 |
+
}
|
| 106 |
+
}
|
figures/sudoku_compare.json
ADDED
|
@@ -0,0 +1,119 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"n": 1000,
|
| 3 |
+
"clues": {
|
| 4 |
+
"min": 21,
|
| 5 |
+
"max": 28,
|
| 6 |
+
"mean": 24.318
|
| 7 |
+
},
|
| 8 |
+
"classical": {
|
| 9 |
+
"propagation": 0.03,
|
| 10 |
+
"greedy_mrv": 0.059,
|
| 11 |
+
"backtracking": 1.0,
|
| 12 |
+
"backtrack_nodes_mean": 423.126,
|
| 13 |
+
"backtrack_nodes_median": 211.5
|
| 14 |
+
},
|
| 15 |
+
"diffusion": {
|
| 16 |
+
"full": {
|
| 17 |
+
"1": {
|
| 18 |
+
"board": 0.40349999999999997,
|
| 19 |
+
"per_seed": [
|
| 20 |
+
0.421,
|
| 21 |
+
0.386
|
| 22 |
+
]
|
| 23 |
+
},
|
| 24 |
+
"4": {
|
| 25 |
+
"board": 0.7925,
|
| 26 |
+
"per_seed": [
|
| 27 |
+
0.786,
|
| 28 |
+
0.799
|
| 29 |
+
]
|
| 30 |
+
},
|
| 31 |
+
"8": {
|
| 32 |
+
"board": 0.8445,
|
| 33 |
+
"per_seed": [
|
| 34 |
+
0.84,
|
| 35 |
+
0.849
|
| 36 |
+
]
|
| 37 |
+
},
|
| 38 |
+
"16": {
|
| 39 |
+
"board": 0.855,
|
| 40 |
+
"per_seed": [
|
| 41 |
+
0.842,
|
| 42 |
+
0.868
|
| 43 |
+
]
|
| 44 |
+
},
|
| 45 |
+
"32": {
|
| 46 |
+
"board": 0.868,
|
| 47 |
+
"per_seed": [
|
| 48 |
+
0.861,
|
| 49 |
+
0.875
|
| 50 |
+
]
|
| 51 |
+
},
|
| 52 |
+
"61": {
|
| 53 |
+
"board": 0.894,
|
| 54 |
+
"per_seed": [
|
| 55 |
+
0.891,
|
| 56 |
+
0.897
|
| 57 |
+
],
|
| 58 |
+
"by_clue": {
|
| 59 |
+
"22": 0.8372093023255814,
|
| 60 |
+
"23": 0.9171270718232044,
|
| 61 |
+
"24": 0.8898550724637682,
|
| 62 |
+
"25": 0.8936170212765957,
|
| 63 |
+
"26": 0.889763779527559
|
| 64 |
+
}
|
| 65 |
+
}
|
| 66 |
+
},
|
| 67 |
+
"mdlm": {
|
| 68 |
+
"1": {
|
| 69 |
+
"board": 0.046,
|
| 70 |
+
"per_seed": [
|
| 71 |
+
0.057,
|
| 72 |
+
0.035
|
| 73 |
+
]
|
| 74 |
+
},
|
| 75 |
+
"4": {
|
| 76 |
+
"board": 0.709,
|
| 77 |
+
"per_seed": [
|
| 78 |
+
0.727,
|
| 79 |
+
0.691
|
| 80 |
+
]
|
| 81 |
+
},
|
| 82 |
+
"8": {
|
| 83 |
+
"board": 0.783,
|
| 84 |
+
"per_seed": [
|
| 85 |
+
0.798,
|
| 86 |
+
0.768
|
| 87 |
+
]
|
| 88 |
+
},
|
| 89 |
+
"16": {
|
| 90 |
+
"board": 0.843,
|
| 91 |
+
"per_seed": [
|
| 92 |
+
0.861,
|
| 93 |
+
0.825
|
| 94 |
+
]
|
| 95 |
+
},
|
| 96 |
+
"32": {
|
| 97 |
+
"board": 0.8645,
|
| 98 |
+
"per_seed": [
|
| 99 |
+
0.864,
|
| 100 |
+
0.865
|
| 101 |
+
]
|
| 102 |
+
},
|
| 103 |
+
"61": {
|
| 104 |
+
"board": 0.874,
|
| 105 |
+
"per_seed": [
|
| 106 |
+
0.884,
|
| 107 |
+
0.864
|
| 108 |
+
],
|
| 109 |
+
"by_clue": {
|
| 110 |
+
"22": 0.8372093023255814,
|
| 111 |
+
"23": 0.9116022099447514,
|
| 112 |
+
"24": 0.8956521739130435,
|
| 113 |
+
"25": 0.900709219858156,
|
| 114 |
+
"26": 0.8031496062992126
|
| 115 |
+
}
|
| 116 |
+
}
|
| 117 |
+
}
|
| 118 |
+
}
|
| 119 |
+
}
|
figures/sudoku_hard.json
ADDED
|
@@ -0,0 +1,86 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"full-aug|30": {
|
| 3 |
+
"fixedK_1": 0.0,
|
| 4 |
+
"fixedK_8": 0.5249999999999999,
|
| 5 |
+
"fixedK_45": 0.7208333333333333,
|
| 6 |
+
"entbudget_0.01": 0.11666666666666667,
|
| 7 |
+
"entbudget_0.2": 0.008333333333333333
|
| 8 |
+
},
|
| 9 |
+
"full-aug|26": {
|
| 10 |
+
"fixedK_1": 0.0,
|
| 11 |
+
"fixedK_8": 0.18333333333333332,
|
| 12 |
+
"fixedK_45": 0.3375,
|
| 13 |
+
"entbudget_0.01": 0.024999999999999998,
|
| 14 |
+
"entbudget_0.2": 0.004166666666666667
|
| 15 |
+
},
|
| 16 |
+
"full-aug|22": {
|
| 17 |
+
"fixedK_1": 0.0,
|
| 18 |
+
"fixedK_8": 0.12083333333333333,
|
| 19 |
+
"fixedK_45": 0.26666666666666666,
|
| 20 |
+
"entbudget_0.01": 0.004166666666666667,
|
| 21 |
+
"entbudget_0.2": 0.0
|
| 22 |
+
},
|
| 23 |
+
"full-plain|30": {
|
| 24 |
+
"fixedK_1": 0.0,
|
| 25 |
+
"fixedK_8": 0.5375,
|
| 26 |
+
"fixedK_45": 0.6458333333333333,
|
| 27 |
+
"entbudget_0.01": 0.04583333333333334,
|
| 28 |
+
"entbudget_0.2": 0.0
|
| 29 |
+
},
|
| 30 |
+
"full-plain|26": {
|
| 31 |
+
"fixedK_1": 0.0,
|
| 32 |
+
"fixedK_8": 0.23333333333333334,
|
| 33 |
+
"fixedK_45": 0.37916666666666665,
|
| 34 |
+
"entbudget_0.01": 0.008333333333333333,
|
| 35 |
+
"entbudget_0.2": 0.004166666666666667
|
| 36 |
+
},
|
| 37 |
+
"full-plain|22": {
|
| 38 |
+
"fixedK_1": 0.0,
|
| 39 |
+
"fixedK_8": 0.075,
|
| 40 |
+
"fixedK_45": 0.19583333333333333,
|
| 41 |
+
"entbudget_0.01": 0.0125,
|
| 42 |
+
"entbudget_0.2": 0.0
|
| 43 |
+
},
|
| 44 |
+
"mdlm-aug|30": {
|
| 45 |
+
"fixedK_1": 0.0,
|
| 46 |
+
"fixedK_8": 0.4916666666666667,
|
| 47 |
+
"fixedK_45": 0.6791666666666667,
|
| 48 |
+
"entbudget_0.01": 0.475,
|
| 49 |
+
"entbudget_0.2": 0.16249999999999998
|
| 50 |
+
},
|
| 51 |
+
"mdlm-aug|26": {
|
| 52 |
+
"fixedK_1": 0.0,
|
| 53 |
+
"fixedK_8": 0.2125,
|
| 54 |
+
"fixedK_45": 0.33333333333333337,
|
| 55 |
+
"entbudget_0.01": 0.15833333333333333,
|
| 56 |
+
"entbudget_0.2": 0.008333333333333333
|
| 57 |
+
},
|
| 58 |
+
"mdlm-aug|22": {
|
| 59 |
+
"fixedK_1": 0.0,
|
| 60 |
+
"fixedK_8": 0.10416666666666666,
|
| 61 |
+
"fixedK_45": 0.16666666666666666,
|
| 62 |
+
"entbudget_0.01": 0.12083333333333333,
|
| 63 |
+
"entbudget_0.2": 0.0125
|
| 64 |
+
},
|
| 65 |
+
"mdlm-plain|30": {
|
| 66 |
+
"fixedK_1": 0.0,
|
| 67 |
+
"fixedK_8": 0.39583333333333337,
|
| 68 |
+
"fixedK_45": 0.5875,
|
| 69 |
+
"entbudget_0.01": 0.20833333333333331,
|
| 70 |
+
"entbudget_0.2": 0.020833333333333332
|
| 71 |
+
},
|
| 72 |
+
"mdlm-plain|26": {
|
| 73 |
+
"fixedK_1": 0.0,
|
| 74 |
+
"fixedK_8": 0.14583333333333331,
|
| 75 |
+
"fixedK_45": 0.2875,
|
| 76 |
+
"entbudget_0.01": 0.06666666666666667,
|
| 77 |
+
"entbudget_0.2": 0.0
|
| 78 |
+
},
|
| 79 |
+
"mdlm-plain|22": {
|
| 80 |
+
"fixedK_1": 0.0,
|
| 81 |
+
"fixedK_8": 0.03333333333333333,
|
| 82 |
+
"fixedK_45": 0.13333333333333333,
|
| 83 |
+
"entbudget_0.01": 0.016666666666666666,
|
| 84 |
+
"entbudget_0.2": 0.0
|
| 85 |
+
}
|
| 86 |
+
}
|
figures/sudoku_remask.json
ADDED
|
@@ -0,0 +1,190 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"sudh-full-s0": {
|
| 3 |
+
"fixedK_16": {
|
| 4 |
+
"board": 0.85,
|
| 5 |
+
"cell": 0.9471260717688155,
|
| 6 |
+
"nfe": 16.0
|
| 7 |
+
},
|
| 8 |
+
"fixedK_61": {
|
| 9 |
+
"board": 0.877,
|
| 10 |
+
"cell": 0.9471966409089305,
|
| 11 |
+
"nfe": 59.5
|
| 12 |
+
},
|
| 13 |
+
"entbudget": {
|
| 14 |
+
"board": 0.882,
|
| 15 |
+
"cell": 0.9523305458522988,
|
| 16 |
+
"nfe": 34.5
|
| 17 |
+
},
|
| 18 |
+
"verifier_b32_r4": {
|
| 19 |
+
"board": 0.886,
|
| 20 |
+
"cell": 0.9486433082812886,
|
| 21 |
+
"nfe": 154.75
|
| 22 |
+
},
|
| 23 |
+
"verifier_b32_r8": {
|
| 24 |
+
"board": 0.886,
|
| 25 |
+
"cell": 0.9505486750643943,
|
| 26 |
+
"nfe": 183.75
|
| 27 |
+
},
|
| 28 |
+
"verifier_b61_r4": {
|
| 29 |
+
"board": 0.884,
|
| 30 |
+
"cell": 0.9470555026287004,
|
| 31 |
+
"nfe": 217.25
|
| 32 |
+
},
|
| 33 |
+
"verifier_b61_r8": {
|
| 34 |
+
"board": 0.884,
|
| 35 |
+
"cell": 0.9467379414981828,
|
| 36 |
+
"nfe": 253.75
|
| 37 |
+
},
|
| 38 |
+
"verifier_narrow": {
|
| 39 |
+
"board": 0.878,
|
| 40 |
+
"cell": 0.9480611128753396,
|
| 41 |
+
"nfe": 89.5
|
| 42 |
+
},
|
| 43 |
+
"random_remask_control": {
|
| 44 |
+
"board": 0.888,
|
| 45 |
+
"cell": 0.9505486750643943,
|
| 46 |
+
"nfe": 174.25
|
| 47 |
+
}
|
| 48 |
+
},
|
| 49 |
+
"sudh-full-s1": {
|
| 50 |
+
"fixedK_16": {
|
| 51 |
+
"board": 0.865,
|
| 52 |
+
"cell": 0.9489608694118062,
|
| 53 |
+
"nfe": 16.0
|
| 54 |
+
},
|
| 55 |
+
"fixedK_61": {
|
| 56 |
+
"board": 0.898,
|
| 57 |
+
"cell": 0.9554179457323313,
|
| 58 |
+
"nfe": 59.5
|
| 59 |
+
},
|
| 60 |
+
"entbudget": {
|
| 61 |
+
"board": 0.887,
|
| 62 |
+
"cell": 0.9549063194664973,
|
| 63 |
+
"nfe": 30.0
|
| 64 |
+
},
|
| 65 |
+
"verifier_b32_r4": {
|
| 66 |
+
"board": 0.895,
|
| 67 |
+
"cell": 0.9561942062735965,
|
| 68 |
+
"nfe": 151.25
|
| 69 |
+
},
|
| 70 |
+
"verifier_b32_r8": {
|
| 71 |
+
"board": 0.896,
|
| 72 |
+
"cell": 0.9564941251190854,
|
| 73 |
+
"nfe": 179.75
|
| 74 |
+
},
|
| 75 |
+
"verifier_b61_r4": {
|
| 76 |
+
"board": 0.9,
|
| 77 |
+
"cell": 0.9574644507956671,
|
| 78 |
+
"nfe": 212.25
|
| 79 |
+
},
|
| 80 |
+
"verifier_b61_r8": {
|
| 81 |
+
"board": 0.895,
|
| 82 |
+
"cell": 0.9557707914329064,
|
| 83 |
+
"nfe": 241.75
|
| 84 |
+
},
|
| 85 |
+
"verifier_narrow": {
|
| 86 |
+
"board": 0.897,
|
| 87 |
+
"cell": 0.9576938005010409,
|
| 88 |
+
"nfe": 83.75
|
| 89 |
+
},
|
| 90 |
+
"random_remask_control": {
|
| 91 |
+
"board": 0.899,
|
| 92 |
+
"cell": 0.9584700610423063,
|
| 93 |
+
"nfe": 174.25
|
| 94 |
+
}
|
| 95 |
+
},
|
| 96 |
+
"sudh-mdlm-s0": {
|
| 97 |
+
"fixedK_16": {
|
| 98 |
+
"board": 0.86,
|
| 99 |
+
"cell": 0.9477435517448219,
|
| 100 |
+
"nfe": 16.0
|
| 101 |
+
},
|
| 102 |
+
"fixedK_61": {
|
| 103 |
+
"board": 0.881,
|
| 104 |
+
"cell": 0.9500017642285029,
|
| 105 |
+
"nfe": 59.5
|
| 106 |
+
},
|
| 107 |
+
"entbudget": {
|
| 108 |
+
"board": 0.892,
|
| 109 |
+
"cell": 0.955506157157475,
|
| 110 |
+
"nfe": 26.75
|
| 111 |
+
},
|
| 112 |
+
"verifier_b32_r4": {
|
| 113 |
+
"board": 0.887,
|
| 114 |
+
"cell": 0.9510603013302283,
|
| 115 |
+
"nfe": 153.25
|
| 116 |
+
},
|
| 117 |
+
"verifier_b32_r8": {
|
| 118 |
+
"board": 0.899,
|
| 119 |
+
"cell": 0.9541477012102607,
|
| 120 |
+
"nfe": 184.0
|
| 121 |
+
},
|
| 122 |
+
"verifier_b61_r4": {
|
| 123 |
+
"board": 0.893,
|
| 124 |
+
"cell": 0.9520482692918387,
|
| 125 |
+
"nfe": 214.0
|
| 126 |
+
},
|
| 127 |
+
"verifier_b61_r8": {
|
| 128 |
+
"board": 0.886,
|
| 129 |
+
"cell": 0.9489432271267775,
|
| 130 |
+
"nfe": 240.0
|
| 131 |
+
},
|
| 132 |
+
"verifier_narrow": {
|
| 133 |
+
"board": 0.886,
|
| 134 |
+
"cell": 0.9512720087505734,
|
| 135 |
+
"nfe": 88.0
|
| 136 |
+
},
|
| 137 |
+
"random_remask_control": {
|
| 138 |
+
"board": 0.889,
|
| 139 |
+
"cell": 0.9519777001517237,
|
| 140 |
+
"nfe": 174.75
|
| 141 |
+
}
|
| 142 |
+
},
|
| 143 |
+
"sudh-mdlm-s1": {
|
| 144 |
+
"fixedK_16": {
|
| 145 |
+
"board": 0.807,
|
| 146 |
+
"cell": 0.9294132175999436,
|
| 147 |
+
"nfe": 16.0
|
| 148 |
+
},
|
| 149 |
+
"fixedK_61": {
|
| 150 |
+
"board": 0.873,
|
| 151 |
+
"cell": 0.9472848523340743,
|
| 152 |
+
"nfe": 59.5
|
| 153 |
+
},
|
| 154 |
+
"entbudget": {
|
| 155 |
+
"board": 0.874,
|
| 156 |
+
"cell": 0.9481669665855121,
|
| 157 |
+
"nfe": 23.5
|
| 158 |
+
},
|
| 159 |
+
"verifier_b32_r4": {
|
| 160 |
+
"board": 0.871,
|
| 161 |
+
"cell": 0.9441445255989556,
|
| 162 |
+
"nfe": 155.25
|
| 163 |
+
},
|
| 164 |
+
"verifier_b32_r8": {
|
| 165 |
+
"board": 0.873,
|
| 166 |
+
"cell": 0.9436505416181504,
|
| 167 |
+
"nfe": 189.5
|
| 168 |
+
},
|
| 169 |
+
"verifier_b61_r4": {
|
| 170 |
+
"board": 0.876,
|
| 171 |
+
"cell": 0.9451148512755372,
|
| 172 |
+
"nfe": 223.25
|
| 173 |
+
},
|
| 174 |
+
"verifier_b61_r8": {
|
| 175 |
+
"board": 0.88,
|
| 176 |
+
"cell": 0.9460146078120039,
|
| 177 |
+
"nfe": 243.75
|
| 178 |
+
},
|
| 179 |
+
"verifier_narrow": {
|
| 180 |
+
"board": 0.871,
|
| 181 |
+
"cell": 0.9458029003916587,
|
| 182 |
+
"nfe": 89.75
|
| 183 |
+
},
|
| 184 |
+
"random_remask_control": {
|
| 185 |
+
"board": 0.866,
|
| 186 |
+
"cell": 0.9417275325500158,
|
| 187 |
+
"nfe": 175.5
|
| 188 |
+
}
|
| 189 |
+
}
|
| 190 |
+
}
|
figures/sudoku_rsweep.json
ADDED
|
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"rec1-full-s0": {
|
| 3 |
+
"1": 0.001,
|
| 4 |
+
"2": 0.054,
|
| 5 |
+
"4": 0.484,
|
| 6 |
+
"8": 0.82,
|
| 7 |
+
"16": 0.924,
|
| 8 |
+
"32": 0.983,
|
| 9 |
+
"64": 0.994,
|
| 10 |
+
"128": 0.991
|
| 11 |
+
},
|
| 12 |
+
"rec1-full-s1": {
|
| 13 |
+
"1": 0.006,
|
| 14 |
+
"2": 0.061,
|
| 15 |
+
"4": 0.482,
|
| 16 |
+
"8": 0.815,
|
| 17 |
+
"16": 0.924,
|
| 18 |
+
"32": 0.982,
|
| 19 |
+
"64": 0.992,
|
| 20 |
+
"128": 0.397
|
| 21 |
+
},
|
| 22 |
+
"rec1-mdlm-s0": {
|
| 23 |
+
"1": 0.002,
|
| 24 |
+
"2": 0.053,
|
| 25 |
+
"4": 0.407,
|
| 26 |
+
"8": 0.799,
|
| 27 |
+
"16": 0.896,
|
| 28 |
+
"32": 0.933,
|
| 29 |
+
"64": 0.95,
|
| 30 |
+
"128": 0.949
|
| 31 |
+
},
|
| 32 |
+
"rec1-mdlm-s1": {
|
| 33 |
+
"1": 0.0,
|
| 34 |
+
"2": 0.024,
|
| 35 |
+
"4": 0.366,
|
| 36 |
+
"8": 0.769,
|
| 37 |
+
"16": 0.889,
|
| 38 |
+
"32": 0.916,
|
| 39 |
+
"64": 0.938,
|
| 40 |
+
"128": 0.935
|
| 41 |
+
}
|
| 42 |
+
}
|
figures/sudoku_validate.json
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"classical_propagation": 1.0,
|
| 3 |
+
"full/aug": {
|
| 4 |
+
"mean": 1.0,
|
| 5 |
+
"per_seed": [
|
| 6 |
+
1.0,
|
| 7 |
+
1.0
|
| 8 |
+
],
|
| 9 |
+
"ci": [
|
| 10 |
+
0.9961731014136095,
|
| 11 |
+
1.0
|
| 12 |
+
],
|
| 13 |
+
"p_vs_satnet": 3.576990874029317e-08
|
| 14 |
+
},
|
| 15 |
+
"full/plain": {
|
| 16 |
+
"mean": 1.0,
|
| 17 |
+
"per_seed": [
|
| 18 |
+
1.0,
|
| 19 |
+
1.0
|
| 20 |
+
],
|
| 21 |
+
"ci": [
|
| 22 |
+
0.9961731014136095,
|
| 23 |
+
1.0
|
| 24 |
+
],
|
| 25 |
+
"p_vs_satnet": 3.576990874029317e-08
|
| 26 |
+
},
|
| 27 |
+
"mdlm/aug": {
|
| 28 |
+
"mean": 1.0,
|
| 29 |
+
"per_seed": [
|
| 30 |
+
1.0,
|
| 31 |
+
1.0
|
| 32 |
+
],
|
| 33 |
+
"ci": [
|
| 34 |
+
0.9961731014136095,
|
| 35 |
+
1.0
|
| 36 |
+
],
|
| 37 |
+
"p_vs_satnet": 3.576990874029317e-08
|
| 38 |
+
},
|
| 39 |
+
"mdlm/plain": {
|
| 40 |
+
"mean": 1.0,
|
| 41 |
+
"per_seed": [
|
| 42 |
+
1.0,
|
| 43 |
+
1.0
|
| 44 |
+
],
|
| 45 |
+
"ci": [
|
| 46 |
+
0.9961731014136095,
|
| 47 |
+
1.0
|
| 48 |
+
],
|
| 49 |
+
"p_vs_satnet": 3.576990874029317e-08
|
| 50 |
+
},
|
| 51 |
+
"harder": {
|
| 52 |
+
"30": {
|
| 53 |
+
"n": 120,
|
| 54 |
+
"clues": 30.0,
|
| 55 |
+
"classical_propagation": 0.275
|
| 56 |
+
},
|
| 57 |
+
"26": {
|
| 58 |
+
"n": 120,
|
| 59 |
+
"clues": 26.033333333333335,
|
| 60 |
+
"classical_propagation": 0.041666666666666664
|
| 61 |
+
},
|
| 62 |
+
"22": {
|
| 63 |
+
"n": 120,
|
| 64 |
+
"clues": 23.583333333333332,
|
| 65 |
+
"classical_propagation": 0.03333333333333333
|
| 66 |
+
}
|
| 67 |
+
}
|
| 68 |
+
}
|
results/abl-deep32L-s0.json
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "full",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"recurrent": false,
|
| 8 |
+
"R": 32,
|
| 9 |
+
"no_deep_sup": false,
|
| 10 |
+
"R_infer": 0,
|
| 11 |
+
"root": "data/sudoku",
|
| 12 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 13 |
+
"augment": true,
|
| 14 |
+
"d": 128,
|
| 15 |
+
"layers": 32,
|
| 16 |
+
"heads": 4,
|
| 17 |
+
"bs": 256,
|
| 18 |
+
"steps": 120000,
|
| 19 |
+
"lr": 0.0003,
|
| 20 |
+
"warmup": 3000,
|
| 21 |
+
"seed": 0,
|
| 22 |
+
"eval_bs": 250,
|
| 23 |
+
"ckpt_every": 10000,
|
| 24 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/abl-deep32L-s0"
|
| 25 |
+
},
|
| 26 |
+
"decode": {
|
| 27 |
+
"fixedK_1": {
|
| 28 |
+
"board": 0.702,
|
| 29 |
+
"cell": 0.9354115945097209,
|
| 30 |
+
"valid_sudoku": 0.702,
|
| 31 |
+
"clues_kept": 1.0,
|
| 32 |
+
"nfe": 1.0
|
| 33 |
+
},
|
| 34 |
+
"fixedK_2": {
|
| 35 |
+
"board": 0.832,
|
| 36 |
+
"cell": 0.9493666419674677,
|
| 37 |
+
"valid_sudoku": 0.832,
|
| 38 |
+
"clues_kept": 1.0,
|
| 39 |
+
"nfe": 2.0
|
| 40 |
+
},
|
| 41 |
+
"fixedK_4": {
|
| 42 |
+
"board": 0.865,
|
| 43 |
+
"cell": 0.9574115239405808,
|
| 44 |
+
"valid_sudoku": 0.865,
|
| 45 |
+
"clues_kept": 1.0,
|
| 46 |
+
"nfe": 4.0
|
| 47 |
+
},
|
| 48 |
+
"fixedK_8": {
|
| 49 |
+
"board": 0.906,
|
| 50 |
+
"cell": 0.9639391694012208,
|
| 51 |
+
"valid_sudoku": 0.906,
|
| 52 |
+
"clues_kept": 1.0,
|
| 53 |
+
"nfe": 8.0
|
| 54 |
+
},
|
| 55 |
+
"fixedK_16": {
|
| 56 |
+
"board": 0.911,
|
| 57 |
+
"cell": 0.9625807134540065,
|
| 58 |
+
"valid_sudoku": 0.911,
|
| 59 |
+
"clues_kept": 1.0,
|
| 60 |
+
"nfe": 16.0
|
| 61 |
+
},
|
| 62 |
+
"fixedK_32": {
|
| 63 |
+
"board": 0.918,
|
| 64 |
+
"cell": 0.9660386013196429,
|
| 65 |
+
"valid_sudoku": 0.918,
|
| 66 |
+
"clues_kept": 1.0,
|
| 67 |
+
"nfe": 32.0
|
| 68 |
+
},
|
| 69 |
+
"fixedK_61": {
|
| 70 |
+
"board": 0.928,
|
| 71 |
+
"cell": 0.9694082777601355,
|
| 72 |
+
"valid_sudoku": 0.928,
|
| 73 |
+
"clues_kept": 1.0,
|
| 74 |
+
"nfe": 59.5
|
| 75 |
+
},
|
| 76 |
+
"entbudget_0.01": {
|
| 77 |
+
"board": 0.945,
|
| 78 |
+
"cell": 0.9751596626795103,
|
| 79 |
+
"valid_sudoku": 0.945,
|
| 80 |
+
"clues_kept": 1.0,
|
| 81 |
+
"nfe": 45.25
|
| 82 |
+
},
|
| 83 |
+
"entbudget_0.05": {
|
| 84 |
+
"board": 0.944,
|
| 85 |
+
"cell": 0.9752478741046541,
|
| 86 |
+
"valid_sudoku": 0.944,
|
| 87 |
+
"clues_kept": 1.0,
|
| 88 |
+
"nfe": 33.25
|
| 89 |
+
},
|
| 90 |
+
"entbudget_0.2": {
|
| 91 |
+
"board": 0.933,
|
| 92 |
+
"cell": 0.9735189301718359,
|
| 93 |
+
"valid_sudoku": 0.933,
|
| 94 |
+
"clues_kept": 1.0,
|
| 95 |
+
"nfe": 24.75
|
| 96 |
+
},
|
| 97 |
+
"entbudget_0.5": {
|
| 98 |
+
"board": 0.922,
|
| 99 |
+
"cell": 0.9706961645672347,
|
| 100 |
+
"valid_sudoku": 0.922,
|
| 101 |
+
"clues_kept": 1.0,
|
| 102 |
+
"nfe": 16.25
|
| 103 |
+
},
|
| 104 |
+
"entbudget_1.0": {
|
| 105 |
+
"board": 0.873,
|
| 106 |
+
"cell": 0.9671147807063971,
|
| 107 |
+
"valid_sudoku": 0.873,
|
| 108 |
+
"clues_kept": 1.0,
|
| 109 |
+
"nfe": 10.25
|
| 110 |
+
},
|
| 111 |
+
"entbudget_2.0": {
|
| 112 |
+
"board": 0.823,
|
| 113 |
+
"cell": 0.9554532303023887,
|
| 114 |
+
"valid_sudoku": 0.823,
|
| 115 |
+
"clues_kept": 1.0,
|
| 116 |
+
"nfe": 8.0
|
| 117 |
+
},
|
| 118 |
+
"conf_0.9": {
|
| 119 |
+
"board": 0.837,
|
| 120 |
+
"cell": 0.9656328287639815,
|
| 121 |
+
"valid_sudoku": 0.837,
|
| 122 |
+
"clues_kept": 1.0,
|
| 123 |
+
"nfe": 14.75
|
| 124 |
+
},
|
| 125 |
+
"conf_0.99": {
|
| 126 |
+
"board": 0.931,
|
| 127 |
+
"cell": 0.9755477929501429,
|
| 128 |
+
"valid_sudoku": 0.931,
|
| 129 |
+
"clues_kept": 1.0,
|
| 130 |
+
"nfe": 27.5
|
| 131 |
+
},
|
| 132 |
+
"conf_0.999": {
|
| 133 |
+
"board": 0.942,
|
| 134 |
+
"cell": 0.9744186867083025,
|
| 135 |
+
"valid_sudoku": 0.942,
|
| 136 |
+
"clues_kept": 1.0,
|
| 137 |
+
"nfe": 41.0
|
| 138 |
+
}
|
| 139 |
+
},
|
| 140 |
+
"best_board": 0.945
|
| 141 |
+
}
|
results/abl-nodeepsup-s0.json
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "full",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"recurrent": true,
|
| 8 |
+
"R": 32,
|
| 9 |
+
"no_deep_sup": true,
|
| 10 |
+
"R_infer": 64,
|
| 11 |
+
"root": "data/sudoku",
|
| 12 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 13 |
+
"augment": true,
|
| 14 |
+
"d": 128,
|
| 15 |
+
"layers": 1,
|
| 16 |
+
"heads": 4,
|
| 17 |
+
"bs": 256,
|
| 18 |
+
"steps": 120000,
|
| 19 |
+
"lr": 0.0003,
|
| 20 |
+
"warmup": 3000,
|
| 21 |
+
"seed": 0,
|
| 22 |
+
"eval_bs": 250,
|
| 23 |
+
"ckpt_every": 10000,
|
| 24 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/abl-nodeepsup-s0"
|
| 25 |
+
},
|
| 26 |
+
"decode": {
|
| 27 |
+
"fixedK_1": {
|
| 28 |
+
"board": 0.932,
|
| 29 |
+
"cell": 0.9677146183973748,
|
| 30 |
+
"valid_sudoku": 0.932,
|
| 31 |
+
"clues_kept": 1.0,
|
| 32 |
+
"nfe": 1.0
|
| 33 |
+
},
|
| 34 |
+
"fixedK_2": {
|
| 35 |
+
"board": 0.935,
|
| 36 |
+
"cell": 0.9694964891852793,
|
| 37 |
+
"valid_sudoku": 0.935,
|
| 38 |
+
"clues_kept": 1.0,
|
| 39 |
+
"nfe": 2.0
|
| 40 |
+
},
|
| 41 |
+
"fixedK_4": {
|
| 42 |
+
"board": 0.94,
|
| 43 |
+
"cell": 0.9735012878868071,
|
| 44 |
+
"valid_sudoku": 0.94,
|
| 45 |
+
"clues_kept": 1.0,
|
| 46 |
+
"nfe": 4.0
|
| 47 |
+
},
|
| 48 |
+
"fixedK_8": {
|
| 49 |
+
"board": 0.952,
|
| 50 |
+
"cell": 0.9796408030768146,
|
| 51 |
+
"valid_sudoku": 0.952,
|
| 52 |
+
"clues_kept": 1.0,
|
| 53 |
+
"nfe": 8.0
|
| 54 |
+
},
|
| 55 |
+
"fixedK_16": {
|
| 56 |
+
"board": 0.957,
|
| 57 |
+
"cell": 0.9813344624395752,
|
| 58 |
+
"valid_sudoku": 0.957,
|
| 59 |
+
"clues_kept": 1.0,
|
| 60 |
+
"nfe": 16.0
|
| 61 |
+
},
|
| 62 |
+
"fixedK_32": {
|
| 63 |
+
"board": 0.964,
|
| 64 |
+
"cell": 0.9835926749232561,
|
| 65 |
+
"valid_sudoku": 0.964,
|
| 66 |
+
"clues_kept": 1.0,
|
| 67 |
+
"nfe": 32.0
|
| 68 |
+
},
|
| 69 |
+
"fixedK_61": {
|
| 70 |
+
"board": 0.97,
|
| 71 |
+
"cell": 0.985268692000988,
|
| 72 |
+
"valid_sudoku": 0.97,
|
| 73 |
+
"clues_kept": 1.0,
|
| 74 |
+
"nfe": 59.5
|
| 75 |
+
},
|
| 76 |
+
"entbudget_0.01": {
|
| 77 |
+
"board": 0.963,
|
| 78 |
+
"cell": 0.9857097491267068,
|
| 79 |
+
"valid_sudoku": 0.963,
|
| 80 |
+
"clues_kept": 1.0,
|
| 81 |
+
"nfe": 35.75
|
| 82 |
+
},
|
| 83 |
+
"entbudget_0.05": {
|
| 84 |
+
"board": 0.97,
|
| 85 |
+
"cell": 0.9853569034261318,
|
| 86 |
+
"valid_sudoku": 0.97,
|
| 87 |
+
"clues_kept": 1.0,
|
| 88 |
+
"nfe": 24.75
|
| 89 |
+
},
|
| 90 |
+
"entbudget_0.2": {
|
| 91 |
+
"board": 0.966,
|
| 92 |
+
"cell": 0.9850040577255567,
|
| 93 |
+
"valid_sudoku": 0.966,
|
| 94 |
+
"clues_kept": 1.0,
|
| 95 |
+
"nfe": 26.0
|
| 96 |
+
},
|
| 97 |
+
"entbudget_0.5": {
|
| 98 |
+
"board": 0.961,
|
| 99 |
+
"cell": 0.9856391799865919,
|
| 100 |
+
"valid_sudoku": 0.961,
|
| 101 |
+
"clues_kept": 1.0,
|
| 102 |
+
"nfe": 22.75
|
| 103 |
+
},
|
| 104 |
+
"entbudget_1.0": {
|
| 105 |
+
"board": 0.956,
|
| 106 |
+
"cell": 0.9811051127342013,
|
| 107 |
+
"valid_sudoku": 0.956,
|
| 108 |
+
"clues_kept": 1.0,
|
| 109 |
+
"nfe": 14.5
|
| 110 |
+
},
|
| 111 |
+
"entbudget_2.0": {
|
| 112 |
+
"board": 0.958,
|
| 113 |
+
"cell": 0.982798772096962,
|
| 114 |
+
"valid_sudoku": 0.958,
|
| 115 |
+
"clues_kept": 1.0,
|
| 116 |
+
"nfe": 8.0
|
| 117 |
+
},
|
| 118 |
+
"conf_0.9": {
|
| 119 |
+
"board": 0.959,
|
| 120 |
+
"cell": 0.9802229984827635,
|
| 121 |
+
"valid_sudoku": 0.959,
|
| 122 |
+
"clues_kept": 1.0,
|
| 123 |
+
"nfe": 14.0
|
| 124 |
+
},
|
| 125 |
+
"conf_0.99": {
|
| 126 |
+
"board": 0.973,
|
| 127 |
+
"cell": 0.9887618644366818,
|
| 128 |
+
"valid_sudoku": 0.973,
|
| 129 |
+
"clues_kept": 1.0,
|
| 130 |
+
"nfe": 24.25
|
| 131 |
+
},
|
| 132 |
+
"conf_0.999": {
|
| 133 |
+
"board": 0.968,
|
| 134 |
+
"cell": 0.9864860096679722,
|
| 135 |
+
"valid_sudoku": 0.968,
|
| 136 |
+
"clues_kept": 1.0,
|
| 137 |
+
"nfe": 32.25
|
| 138 |
+
}
|
| 139 |
+
},
|
| 140 |
+
"best_board": 0.973
|
| 141 |
+
}
|
results/abl-nodeepsup-s1.json
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "full",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"recurrent": true,
|
| 8 |
+
"R": 32,
|
| 9 |
+
"no_deep_sup": true,
|
| 10 |
+
"R_infer": 64,
|
| 11 |
+
"root": "data/sudoku",
|
| 12 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 13 |
+
"augment": true,
|
| 14 |
+
"d": 128,
|
| 15 |
+
"layers": 1,
|
| 16 |
+
"heads": 4,
|
| 17 |
+
"bs": 256,
|
| 18 |
+
"steps": 120000,
|
| 19 |
+
"lr": 0.0003,
|
| 20 |
+
"warmup": 3000,
|
| 21 |
+
"seed": 1,
|
| 22 |
+
"eval_bs": 250,
|
| 23 |
+
"ckpt_every": 10000,
|
| 24 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/abl-nodeepsup-s1"
|
| 25 |
+
},
|
| 26 |
+
"decode": {
|
| 27 |
+
"fixedK_1": {
|
| 28 |
+
"board": 0.935,
|
| 29 |
+
"cell": 0.9744892558484175,
|
| 30 |
+
"valid_sudoku": 0.935,
|
| 31 |
+
"clues_kept": 1.0,
|
| 32 |
+
"nfe": 1.0
|
| 33 |
+
},
|
| 34 |
+
"fixedK_2": {
|
| 35 |
+
"board": 0.94,
|
| 36 |
+
"cell": 0.9745951095585901,
|
| 37 |
+
"valid_sudoku": 0.94,
|
| 38 |
+
"clues_kept": 1.0,
|
| 39 |
+
"nfe": 2.0
|
| 40 |
+
},
|
| 41 |
+
"fixedK_4": {
|
| 42 |
+
"board": 0.956,
|
| 43 |
+
"cell": 0.9811756818743164,
|
| 44 |
+
"valid_sudoku": 0.956,
|
| 45 |
+
"clues_kept": 1.0,
|
| 46 |
+
"nfe": 4.0
|
| 47 |
+
},
|
| 48 |
+
"fixedK_8": {
|
| 49 |
+
"board": 0.968,
|
| 50 |
+
"cell": 0.9852157651459017,
|
| 51 |
+
"valid_sudoku": 0.968,
|
| 52 |
+
"clues_kept": 1.0,
|
| 53 |
+
"nfe": 8.0
|
| 54 |
+
},
|
| 55 |
+
"fixedK_16": {
|
| 56 |
+
"board": 0.972,
|
| 57 |
+
"cell": 0.9868917822236336,
|
| 58 |
+
"valid_sudoku": 0.972,
|
| 59 |
+
"clues_kept": 1.0,
|
| 60 |
+
"nfe": 16.0
|
| 61 |
+
},
|
| 62 |
+
"fixedK_32": {
|
| 63 |
+
"board": 0.975,
|
| 64 |
+
"cell": 0.9883031650259342,
|
| 65 |
+
"valid_sudoku": 0.975,
|
| 66 |
+
"clues_kept": 1.0,
|
| 67 |
+
"nfe": 32.0
|
| 68 |
+
},
|
| 69 |
+
"fixedK_61": {
|
| 70 |
+
"board": 0.976,
|
| 71 |
+
"cell": 0.9873328393493526,
|
| 72 |
+
"valid_sudoku": 0.976,
|
| 73 |
+
"clues_kept": 1.0,
|
| 74 |
+
"nfe": 59.5
|
| 75 |
+
},
|
| 76 |
+
"entbudget_0.01": {
|
| 77 |
+
"board": 0.982,
|
| 78 |
+
"cell": 0.9916904837514555,
|
| 79 |
+
"valid_sudoku": 0.982,
|
| 80 |
+
"clues_kept": 1.0,
|
| 81 |
+
"nfe": 31.25
|
| 82 |
+
},
|
| 83 |
+
"entbudget_0.05": {
|
| 84 |
+
"board": 0.98,
|
| 85 |
+
"cell": 0.9899968243886949,
|
| 86 |
+
"valid_sudoku": 0.98,
|
| 87 |
+
"clues_kept": 1.0,
|
| 88 |
+
"nfe": 22.5
|
| 89 |
+
},
|
| 90 |
+
"entbudget_0.2": {
|
| 91 |
+
"board": 0.979,
|
| 92 |
+
"cell": 0.9905966620796726,
|
| 93 |
+
"valid_sudoku": 0.979,
|
| 94 |
+
"clues_kept": 1.0,
|
| 95 |
+
"nfe": 19.5
|
| 96 |
+
},
|
| 97 |
+
"entbudget_0.5": {
|
| 98 |
+
"board": 0.975,
|
| 99 |
+
"cell": 0.9885148724462792,
|
| 100 |
+
"valid_sudoku": 0.975,
|
| 101 |
+
"clues_kept": 1.0,
|
| 102 |
+
"nfe": 12.5
|
| 103 |
+
},
|
| 104 |
+
"entbudget_1.0": {
|
| 105 |
+
"board": 0.961,
|
| 106 |
+
"cell": 0.9830634063723933,
|
| 107 |
+
"valid_sudoku": 0.961,
|
| 108 |
+
"clues_kept": 1.0,
|
| 109 |
+
"nfe": 8.0
|
| 110 |
+
},
|
| 111 |
+
"entbudget_2.0": {
|
| 112 |
+
"board": 0.949,
|
| 113 |
+
"cell": 0.9780353551391976,
|
| 114 |
+
"valid_sudoku": 0.949,
|
| 115 |
+
"clues_kept": 1.0,
|
| 116 |
+
"nfe": 7.0
|
| 117 |
+
},
|
| 118 |
+
"conf_0.9": {
|
| 119 |
+
"board": 0.959,
|
| 120 |
+
"cell": 0.9837338132034861,
|
| 121 |
+
"valid_sudoku": 0.959,
|
| 122 |
+
"clues_kept": 1.0,
|
| 123 |
+
"nfe": 8.5
|
| 124 |
+
},
|
| 125 |
+
"conf_0.99": {
|
| 126 |
+
"board": 0.974,
|
| 127 |
+
"cell": 0.989044140997142,
|
| 128 |
+
"valid_sudoku": 0.974,
|
| 129 |
+
"clues_kept": 1.0,
|
| 130 |
+
"nfe": 20.5
|
| 131 |
+
},
|
| 132 |
+
"conf_0.999": {
|
| 133 |
+
"board": 0.979,
|
| 134 |
+
"cell": 0.9893969866977171,
|
| 135 |
+
"valid_sudoku": 0.979,
|
| 136 |
+
"clues_kept": 1.0,
|
| 137 |
+
"nfe": 30.0
|
| 138 |
+
}
|
| 139 |
+
},
|
| 140 |
+
"best_board": 0.982
|
| 141 |
+
}
|
results/base-ar-s0.json
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"objective": "ar",
|
| 5 |
+
"K": 8,
|
| 6 |
+
"lam": 1.0,
|
| 7 |
+
"pe": "ape",
|
| 8 |
+
"recurrent": false,
|
| 9 |
+
"R": 32,
|
| 10 |
+
"no_deep_sup": false,
|
| 11 |
+
"R_infer": 0,
|
| 12 |
+
"root": "data/sudoku",
|
| 13 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 14 |
+
"augment": true,
|
| 15 |
+
"d": 128,
|
| 16 |
+
"layers": 12,
|
| 17 |
+
"heads": 4,
|
| 18 |
+
"bs": 256,
|
| 19 |
+
"steps": 120000,
|
| 20 |
+
"lr": 0.0003,
|
| 21 |
+
"warmup": 3000,
|
| 22 |
+
"seed": 0,
|
| 23 |
+
"eval_bs": 250,
|
| 24 |
+
"ckpt_every": 10000,
|
| 25 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/base-ar-s0"
|
| 26 |
+
},
|
| 27 |
+
"decode": {
|
| 28 |
+
"ar": {
|
| 29 |
+
"board": 0.0,
|
| 30 |
+
"cell": 0.16823683003422604,
|
| 31 |
+
"valid_sudoku": 0.0,
|
| 32 |
+
"clues_kept": 1.0,
|
| 33 |
+
"nfe": 81.0
|
| 34 |
+
}
|
| 35 |
+
}
|
| 36 |
+
}
|
results/base-directrec-s0.json
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"objective": "direct",
|
| 5 |
+
"K": 8,
|
| 6 |
+
"lam": 1.0,
|
| 7 |
+
"pe": "ape",
|
| 8 |
+
"recurrent": true,
|
| 9 |
+
"R": 32,
|
| 10 |
+
"no_deep_sup": false,
|
| 11 |
+
"R_infer": 64,
|
| 12 |
+
"root": "data/sudoku",
|
| 13 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 14 |
+
"augment": true,
|
| 15 |
+
"d": 128,
|
| 16 |
+
"layers": 1,
|
| 17 |
+
"heads": 4,
|
| 18 |
+
"bs": 256,
|
| 19 |
+
"steps": 120000,
|
| 20 |
+
"lr": 0.0003,
|
| 21 |
+
"warmup": 3000,
|
| 22 |
+
"seed": 0,
|
| 23 |
+
"eval_bs": 250,
|
| 24 |
+
"ckpt_every": 10000,
|
| 25 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/base-directrec-s0"
|
| 26 |
+
},
|
| 27 |
+
"decode": {
|
| 28 |
+
"fixedK_1": {
|
| 29 |
+
"board": 0.358,
|
| 30 |
+
"cell": 0.8523870011643908,
|
| 31 |
+
"valid_sudoku": 0.358,
|
| 32 |
+
"clues_kept": 1.0,
|
| 33 |
+
"nfe": 1.0
|
| 34 |
+
},
|
| 35 |
+
"fixedK_2": {
|
| 36 |
+
"board": 0.077,
|
| 37 |
+
"cell": 0.7139127059736777,
|
| 38 |
+
"valid_sudoku": 0.077,
|
| 39 |
+
"clues_kept": 1.0,
|
| 40 |
+
"nfe": 2.0
|
| 41 |
+
},
|
| 42 |
+
"fixedK_4": {
|
| 43 |
+
"board": 0.0,
|
| 44 |
+
"cell": 0.5832715853357326,
|
| 45 |
+
"valid_sudoku": 0.0,
|
| 46 |
+
"clues_kept": 1.0,
|
| 47 |
+
"nfe": 4.0
|
| 48 |
+
},
|
| 49 |
+
"fixedK_8": {
|
| 50 |
+
"board": 0.0,
|
| 51 |
+
"cell": 0.574468085106383,
|
| 52 |
+
"valid_sudoku": 0.0,
|
| 53 |
+
"clues_kept": 1.0,
|
| 54 |
+
"nfe": 8.0
|
| 55 |
+
},
|
| 56 |
+
"fixedK_16": {
|
| 57 |
+
"board": 0.0,
|
| 58 |
+
"cell": 0.5897286616562577,
|
| 59 |
+
"valid_sudoku": 0.0,
|
| 60 |
+
"clues_kept": 1.0,
|
| 61 |
+
"nfe": 16.0
|
| 62 |
+
},
|
| 63 |
+
"fixedK_32": {
|
| 64 |
+
"board": 0.0,
|
| 65 |
+
"cell": 0.5926043541159451,
|
| 66 |
+
"valid_sudoku": 0.0,
|
| 67 |
+
"clues_kept": 1.0,
|
| 68 |
+
"nfe": 32.0
|
| 69 |
+
},
|
| 70 |
+
"fixedK_61": {
|
| 71 |
+
"board": 0.0,
|
| 72 |
+
"cell": 0.5926396386860026,
|
| 73 |
+
"valid_sudoku": 0.0,
|
| 74 |
+
"clues_kept": 1.0,
|
| 75 |
+
"nfe": 59.5
|
| 76 |
+
},
|
| 77 |
+
"entbudget_0.01": {
|
| 78 |
+
"board": 0.129,
|
| 79 |
+
"cell": 0.760117850463992,
|
| 80 |
+
"valid_sudoku": 0.129,
|
| 81 |
+
"clues_kept": 1.0,
|
| 82 |
+
"nfe": 57.0
|
| 83 |
+
},
|
| 84 |
+
"entbudget_0.05": {
|
| 85 |
+
"board": 0.318,
|
| 86 |
+
"cell": 0.8145795843477647,
|
| 87 |
+
"valid_sudoku": 0.318,
|
| 88 |
+
"clues_kept": 1.0,
|
| 89 |
+
"nfe": 51.25
|
| 90 |
+
},
|
| 91 |
+
"entbudget_0.2": {
|
| 92 |
+
"board": 0.381,
|
| 93 |
+
"cell": 0.8257471507709678,
|
| 94 |
+
"valid_sudoku": 0.381,
|
| 95 |
+
"clues_kept": 1.0,
|
| 96 |
+
"nfe": 41.75
|
| 97 |
+
},
|
| 98 |
+
"entbudget_0.5": {
|
| 99 |
+
"board": 0.39,
|
| 100 |
+
"cell": 0.8347623584206626,
|
| 101 |
+
"valid_sudoku": 0.39,
|
| 102 |
+
"clues_kept": 1.0,
|
| 103 |
+
"nfe": 32.5
|
| 104 |
+
},
|
| 105 |
+
"entbudget_1.0": {
|
| 106 |
+
"board": 0.381,
|
| 107 |
+
"cell": 0.8299812991778696,
|
| 108 |
+
"valid_sudoku": 0.381,
|
| 109 |
+
"clues_kept": 1.0,
|
| 110 |
+
"nfe": 26.5
|
| 111 |
+
},
|
| 112 |
+
"entbudget_2.0": {
|
| 113 |
+
"board": 0.38,
|
| 114 |
+
"cell": 0.8337391058889947,
|
| 115 |
+
"valid_sudoku": 0.38,
|
| 116 |
+
"clues_kept": 1.0,
|
| 117 |
+
"nfe": 20.0
|
| 118 |
+
},
|
| 119 |
+
"conf_0.9": {
|
| 120 |
+
"board": 0.401,
|
| 121 |
+
"cell": 0.8617550545146607,
|
| 122 |
+
"valid_sudoku": 0.401,
|
| 123 |
+
"clues_kept": 1.0,
|
| 124 |
+
"nfe": 25.75
|
| 125 |
+
},
|
| 126 |
+
"conf_0.99": {
|
| 127 |
+
"board": 0.409,
|
| 128 |
+
"cell": 0.8534278959810875,
|
| 129 |
+
"valid_sudoku": 0.409,
|
| 130 |
+
"clues_kept": 1.0,
|
| 131 |
+
"nfe": 44.25
|
| 132 |
+
},
|
| 133 |
+
"conf_0.999": {
|
| 134 |
+
"board": 0.329,
|
| 135 |
+
"cell": 0.8302282911682721,
|
| 136 |
+
"valid_sudoku": 0.329,
|
| 137 |
+
"clues_kept": 1.0,
|
| 138 |
+
"nfe": 54.0
|
| 139 |
+
}
|
| 140 |
+
},
|
| 141 |
+
"best_board": 0.409
|
| 142 |
+
}
|
results/base-directrec-s1.json
ADDED
|
@@ -0,0 +1,142 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"objective": "direct",
|
| 5 |
+
"K": 8,
|
| 6 |
+
"lam": 1.0,
|
| 7 |
+
"pe": "ape",
|
| 8 |
+
"recurrent": true,
|
| 9 |
+
"R": 32,
|
| 10 |
+
"no_deep_sup": false,
|
| 11 |
+
"R_infer": 64,
|
| 12 |
+
"root": "data/sudoku",
|
| 13 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 14 |
+
"augment": true,
|
| 15 |
+
"d": 128,
|
| 16 |
+
"layers": 1,
|
| 17 |
+
"heads": 4,
|
| 18 |
+
"bs": 256,
|
| 19 |
+
"steps": 120000,
|
| 20 |
+
"lr": 0.0003,
|
| 21 |
+
"warmup": 3000,
|
| 22 |
+
"seed": 1,
|
| 23 |
+
"eval_bs": 250,
|
| 24 |
+
"ckpt_every": 10000,
|
| 25 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/base-directrec-s1"
|
| 26 |
+
},
|
| 27 |
+
"decode": {
|
| 28 |
+
"fixedK_1": {
|
| 29 |
+
"board": 0.015,
|
| 30 |
+
"cell": 0.7376415793373557,
|
| 31 |
+
"valid_sudoku": 0.015,
|
| 32 |
+
"clues_kept": 1.0,
|
| 33 |
+
"nfe": 1.0
|
| 34 |
+
},
|
| 35 |
+
"fixedK_2": {
|
| 36 |
+
"board": 0.009,
|
| 37 |
+
"cell": 0.7314314950072334,
|
| 38 |
+
"valid_sudoku": 0.009,
|
| 39 |
+
"clues_kept": 1.0,
|
| 40 |
+
"nfe": 2.0
|
| 41 |
+
},
|
| 42 |
+
"fixedK_4": {
|
| 43 |
+
"board": 0.003,
|
| 44 |
+
"cell": 0.7721146042835468,
|
| 45 |
+
"valid_sudoku": 0.003,
|
| 46 |
+
"clues_kept": 1.0,
|
| 47 |
+
"nfe": 4.0
|
| 48 |
+
},
|
| 49 |
+
"fixedK_8": {
|
| 50 |
+
"board": 0.005,
|
| 51 |
+
"cell": 0.7732437105253872,
|
| 52 |
+
"valid_sudoku": 0.005,
|
| 53 |
+
"clues_kept": 1.0,
|
| 54 |
+
"nfe": 8.0
|
| 55 |
+
},
|
| 56 |
+
"fixedK_16": {
|
| 57 |
+
"board": 0.0,
|
| 58 |
+
"cell": 0.7650753325570728,
|
| 59 |
+
"valid_sudoku": 0.0,
|
| 60 |
+
"clues_kept": 1.0,
|
| 61 |
+
"nfe": 16.0
|
| 62 |
+
},
|
| 63 |
+
"fixedK_32": {
|
| 64 |
+
"board": 0.002,
|
| 65 |
+
"cell": 0.7653223245474754,
|
| 66 |
+
"valid_sudoku": 0.002,
|
| 67 |
+
"clues_kept": 1.0,
|
| 68 |
+
"nfe": 32.0
|
| 69 |
+
},
|
| 70 |
+
"fixedK_61": {
|
| 71 |
+
"board": 0.003,
|
| 72 |
+
"cell": 0.7670689107653224,
|
| 73 |
+
"valid_sudoku": 0.003,
|
| 74 |
+
"clues_kept": 1.0,
|
| 75 |
+
"nfe": 59.5
|
| 76 |
+
},
|
| 77 |
+
"entbudget_0.01": {
|
| 78 |
+
"board": 0.008,
|
| 79 |
+
"cell": 0.7843759923785328,
|
| 80 |
+
"valid_sudoku": 0.008,
|
| 81 |
+
"clues_kept": 1.0,
|
| 82 |
+
"nfe": 53.75
|
| 83 |
+
},
|
| 84 |
+
"entbudget_0.05": {
|
| 85 |
+
"board": 0.037,
|
| 86 |
+
"cell": 0.7805652588123214,
|
| 87 |
+
"valid_sudoku": 0.037,
|
| 88 |
+
"clues_kept": 1.0,
|
| 89 |
+
"nfe": 46.0
|
| 90 |
+
},
|
| 91 |
+
"entbudget_0.2": {
|
| 92 |
+
"board": 0.06,
|
| 93 |
+
"cell": 0.7796831445608835,
|
| 94 |
+
"valid_sudoku": 0.06,
|
| 95 |
+
"clues_kept": 1.0,
|
| 96 |
+
"nfe": 39.0
|
| 97 |
+
},
|
| 98 |
+
"entbudget_0.5": {
|
| 99 |
+
"board": 0.063,
|
| 100 |
+
"cell": 0.7810592427931266,
|
| 101 |
+
"valid_sudoku": 0.063,
|
| 102 |
+
"clues_kept": 1.0,
|
| 103 |
+
"nfe": 29.5
|
| 104 |
+
},
|
| 105 |
+
"entbudget_1.0": {
|
| 106 |
+
"board": 0.064,
|
| 107 |
+
"cell": 0.7823647718852546,
|
| 108 |
+
"valid_sudoku": 0.064,
|
| 109 |
+
"clues_kept": 1.0,
|
| 110 |
+
"nfe": 22.75
|
| 111 |
+
},
|
| 112 |
+
"entbudget_2.0": {
|
| 113 |
+
"board": 0.057,
|
| 114 |
+
"cell": 0.7639462263152323,
|
| 115 |
+
"valid_sudoku": 0.057,
|
| 116 |
+
"clues_kept": 1.0,
|
| 117 |
+
"nfe": 15.0
|
| 118 |
+
},
|
| 119 |
+
"conf_0.9": {
|
| 120 |
+
"board": 0.061,
|
| 121 |
+
"cell": 0.7680039518718464,
|
| 122 |
+
"valid_sudoku": 0.061,
|
| 123 |
+
"clues_kept": 1.0,
|
| 124 |
+
"nfe": 24.5
|
| 125 |
+
},
|
| 126 |
+
"conf_0.99": {
|
| 127 |
+
"board": 0.066,
|
| 128 |
+
"cell": 0.7839702198228714,
|
| 129 |
+
"valid_sudoku": 0.066,
|
| 130 |
+
"clues_kept": 1.0,
|
| 131 |
+
"nfe": 40.0
|
| 132 |
+
},
|
| 133 |
+
"conf_0.999": {
|
| 134 |
+
"board": 0.057,
|
| 135 |
+
"cell": 0.7903920115733389,
|
| 136 |
+
"valid_sudoku": 0.057,
|
| 137 |
+
"clues_kept": 1.0,
|
| 138 |
+
"nfe": 49.25
|
| 139 |
+
}
|
| 140 |
+
},
|
| 141 |
+
"best_board": 0.066
|
| 142 |
+
}
|
results/fix-arprefix-s0.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"all_cells": false,
|
| 5 |
+
"objective": "ar",
|
| 6 |
+
"K": 8,
|
| 7 |
+
"lam": 1.0,
|
| 8 |
+
"pe": "ape",
|
| 9 |
+
"recurrent": false,
|
| 10 |
+
"R": 32,
|
| 11 |
+
"no_deep_sup": false,
|
| 12 |
+
"R_infer": 0,
|
| 13 |
+
"root": "data/sudoku",
|
| 14 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 15 |
+
"augment": true,
|
| 16 |
+
"d": 128,
|
| 17 |
+
"layers": 12,
|
| 18 |
+
"heads": 4,
|
| 19 |
+
"bs": 256,
|
| 20 |
+
"steps": 110000,
|
| 21 |
+
"lr": 0.0003,
|
| 22 |
+
"warmup": 3000,
|
| 23 |
+
"seed": 0,
|
| 24 |
+
"eval_bs": 250,
|
| 25 |
+
"ckpt_every": 10000,
|
| 26 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/fix-arprefix-s0"
|
| 27 |
+
},
|
| 28 |
+
"decode": {
|
| 29 |
+
"ar": {
|
| 30 |
+
"board": 0.003,
|
| 31 |
+
"cell": 0.430683462122014,
|
| 32 |
+
"valid_sudoku": 0.043,
|
| 33 |
+
"clues_kept": 0.208,
|
| 34 |
+
"nfe": 81.0
|
| 35 |
+
}
|
| 36 |
+
}
|
| 37 |
+
}
|
results/fix-directall-s0.json
ADDED
|
@@ -0,0 +1,143 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"all_cells": true,
|
| 5 |
+
"objective": "direct",
|
| 6 |
+
"K": 8,
|
| 7 |
+
"lam": 1.0,
|
| 8 |
+
"pe": "ape",
|
| 9 |
+
"recurrent": true,
|
| 10 |
+
"R": 32,
|
| 11 |
+
"no_deep_sup": false,
|
| 12 |
+
"R_infer": 64,
|
| 13 |
+
"root": "data/sudoku",
|
| 14 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 15 |
+
"augment": true,
|
| 16 |
+
"d": 128,
|
| 17 |
+
"layers": 1,
|
| 18 |
+
"heads": 4,
|
| 19 |
+
"bs": 256,
|
| 20 |
+
"steps": 110000,
|
| 21 |
+
"lr": 0.0003,
|
| 22 |
+
"warmup": 3000,
|
| 23 |
+
"seed": 0,
|
| 24 |
+
"eval_bs": 250,
|
| 25 |
+
"ckpt_every": 10000,
|
| 26 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/fix-directall-s0"
|
| 27 |
+
},
|
| 28 |
+
"decode": {
|
| 29 |
+
"fixedK_1": {
|
| 30 |
+
"board": 0.105,
|
| 31 |
+
"cell": 0.6825270809075191,
|
| 32 |
+
"valid_sudoku": 0.105,
|
| 33 |
+
"clues_kept": 1.0,
|
| 34 |
+
"nfe": 1.0
|
| 35 |
+
},
|
| 36 |
+
"fixedK_2": {
|
| 37 |
+
"board": 0.154,
|
| 38 |
+
"cell": 0.7321371864083837,
|
| 39 |
+
"valid_sudoku": 0.154,
|
| 40 |
+
"clues_kept": 1.0,
|
| 41 |
+
"nfe": 2.0
|
| 42 |
+
},
|
| 43 |
+
"fixedK_4": {
|
| 44 |
+
"board": 0.147,
|
| 45 |
+
"cell": 0.7680568787269327,
|
| 46 |
+
"valid_sudoku": 0.147,
|
| 47 |
+
"clues_kept": 1.0,
|
| 48 |
+
"nfe": 4.0
|
| 49 |
+
},
|
| 50 |
+
"fixedK_8": {
|
| 51 |
+
"board": 0.259,
|
| 52 |
+
"cell": 0.8128506404149466,
|
| 53 |
+
"valid_sudoku": 0.259,
|
| 54 |
+
"clues_kept": 1.0,
|
| 55 |
+
"nfe": 8.0
|
| 56 |
+
},
|
| 57 |
+
"fixedK_16": {
|
| 58 |
+
"board": 0.318,
|
| 59 |
+
"cell": 0.8437775660703575,
|
| 60 |
+
"valid_sudoku": 0.318,
|
| 61 |
+
"clues_kept": 1.0,
|
| 62 |
+
"nfe": 16.0
|
| 63 |
+
},
|
| 64 |
+
"fixedK_32": {
|
| 65 |
+
"board": 0.398,
|
| 66 |
+
"cell": 0.8605553791327053,
|
| 67 |
+
"valid_sudoku": 0.398,
|
| 68 |
+
"clues_kept": 1.0,
|
| 69 |
+
"nfe": 32.0
|
| 70 |
+
},
|
| 71 |
+
"fixedK_61": {
|
| 72 |
+
"board": 0.452,
|
| 73 |
+
"cell": 0.881620267457041,
|
| 74 |
+
"valid_sudoku": 0.452,
|
| 75 |
+
"clues_kept": 1.0,
|
| 76 |
+
"nfe": 59.5
|
| 77 |
+
},
|
| 78 |
+
"entbudget_0.01": {
|
| 79 |
+
"board": 0.501,
|
| 80 |
+
"cell": 0.893017183585618,
|
| 81 |
+
"valid_sudoku": 0.501,
|
| 82 |
+
"clues_kept": 1.0,
|
| 83 |
+
"nfe": 58.5
|
| 84 |
+
},
|
| 85 |
+
"entbudget_0.05": {
|
| 86 |
+
"board": 0.525,
|
| 87 |
+
"cell": 0.9052609293955753,
|
| 88 |
+
"valid_sudoku": 0.525,
|
| 89 |
+
"clues_kept": 1.0,
|
| 90 |
+
"nfe": 57.75
|
| 91 |
+
},
|
| 92 |
+
"entbudget_0.2": {
|
| 93 |
+
"board": 0.549,
|
| 94 |
+
"cell": 0.9146819096009315,
|
| 95 |
+
"valid_sudoku": 0.549,
|
| 96 |
+
"clues_kept": 1.0,
|
| 97 |
+
"nfe": 56.5
|
| 98 |
+
},
|
| 99 |
+
"entbudget_0.5": {
|
| 100 |
+
"board": 0.493,
|
| 101 |
+
"cell": 0.9018383260999965,
|
| 102 |
+
"valid_sudoku": 0.493,
|
| 103 |
+
"clues_kept": 1.0,
|
| 104 |
+
"nfe": 55.0
|
| 105 |
+
},
|
| 106 |
+
"entbudget_1.0": {
|
| 107 |
+
"board": 0.432,
|
| 108 |
+
"cell": 0.905578490526093,
|
| 109 |
+
"valid_sudoku": 0.432,
|
| 110 |
+
"clues_kept": 1.0,
|
| 111 |
+
"nfe": 49.0
|
| 112 |
+
},
|
| 113 |
+
"entbudget_2.0": {
|
| 114 |
+
"board": 0.322,
|
| 115 |
+
"cell": 0.8803676652199993,
|
| 116 |
+
"valid_sudoku": 0.322,
|
| 117 |
+
"clues_kept": 1.0,
|
| 118 |
+
"nfe": 32.75
|
| 119 |
+
},
|
| 120 |
+
"conf_0.9": {
|
| 121 |
+
"board": 0.33,
|
| 122 |
+
"cell": 0.9106241840443174,
|
| 123 |
+
"valid_sudoku": 0.33,
|
| 124 |
+
"clues_kept": 1.0,
|
| 125 |
+
"nfe": 54.75
|
| 126 |
+
},
|
| 127 |
+
"conf_0.99": {
|
| 128 |
+
"board": 0.551,
|
| 129 |
+
"cell": 0.9083306869905791,
|
| 130 |
+
"valid_sudoku": 0.551,
|
| 131 |
+
"clues_kept": 1.0,
|
| 132 |
+
"nfe": 58.0
|
| 133 |
+
},
|
| 134 |
+
"conf_0.999": {
|
| 135 |
+
"board": 0.532,
|
| 136 |
+
"cell": 0.906954588758336,
|
| 137 |
+
"valid_sudoku": 0.532,
|
| 138 |
+
"clues_kept": 1.0,
|
| 139 |
+
"nfe": 58.75
|
| 140 |
+
}
|
| 141 |
+
},
|
| 142 |
+
"best_board": 0.551
|
| 143 |
+
}
|
results/fix-directall-s1.json
ADDED
|
@@ -0,0 +1,143 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"all_cells": true,
|
| 5 |
+
"objective": "direct",
|
| 6 |
+
"K": 8,
|
| 7 |
+
"lam": 1.0,
|
| 8 |
+
"pe": "ape",
|
| 9 |
+
"recurrent": true,
|
| 10 |
+
"R": 32,
|
| 11 |
+
"no_deep_sup": false,
|
| 12 |
+
"R_infer": 64,
|
| 13 |
+
"root": "data/sudoku",
|
| 14 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 15 |
+
"augment": true,
|
| 16 |
+
"d": 128,
|
| 17 |
+
"layers": 1,
|
| 18 |
+
"heads": 4,
|
| 19 |
+
"bs": 256,
|
| 20 |
+
"steps": 110000,
|
| 21 |
+
"lr": 0.0003,
|
| 22 |
+
"warmup": 3000,
|
| 23 |
+
"seed": 1,
|
| 24 |
+
"eval_bs": 250,
|
| 25 |
+
"ckpt_every": 10000,
|
| 26 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/fix-directall-s1"
|
| 27 |
+
},
|
| 28 |
+
"decode": {
|
| 29 |
+
"fixedK_1": {
|
| 30 |
+
"board": 0.094,
|
| 31 |
+
"cell": 0.5431353868953107,
|
| 32 |
+
"valid_sudoku": 0.094,
|
| 33 |
+
"clues_kept": 1.0,
|
| 34 |
+
"nfe": 1.0
|
| 35 |
+
},
|
| 36 |
+
"fixedK_2": {
|
| 37 |
+
"board": 0.084,
|
| 38 |
+
"cell": 0.5583606788751279,
|
| 39 |
+
"valid_sudoku": 0.084,
|
| 40 |
+
"clues_kept": 1.0,
|
| 41 |
+
"nfe": 2.0
|
| 42 |
+
},
|
| 43 |
+
"fixedK_4": {
|
| 44 |
+
"board": 0.03,
|
| 45 |
+
"cell": 0.5073568328569916,
|
| 46 |
+
"valid_sudoku": 0.03,
|
| 47 |
+
"clues_kept": 1.0,
|
| 48 |
+
"nfe": 4.0
|
| 49 |
+
},
|
| 50 |
+
"fixedK_8": {
|
| 51 |
+
"board": 0.007,
|
| 52 |
+
"cell": 0.5128965103560214,
|
| 53 |
+
"valid_sudoku": 0.007,
|
| 54 |
+
"clues_kept": 1.0,
|
| 55 |
+
"nfe": 8.0
|
| 56 |
+
},
|
| 57 |
+
"fixedK_16": {
|
| 58 |
+
"board": 0.01,
|
| 59 |
+
"cell": 0.5899050845065453,
|
| 60 |
+
"valid_sudoku": 0.01,
|
| 61 |
+
"clues_kept": 1.0,
|
| 62 |
+
"nfe": 16.0
|
| 63 |
+
},
|
| 64 |
+
"fixedK_32": {
|
| 65 |
+
"board": 0.007,
|
| 66 |
+
"cell": 0.6276948590381426,
|
| 67 |
+
"valid_sudoku": 0.007,
|
| 68 |
+
"clues_kept": 1.0,
|
| 69 |
+
"nfe": 32.0
|
| 70 |
+
},
|
| 71 |
+
"fixedK_61": {
|
| 72 |
+
"board": 0.014,
|
| 73 |
+
"cell": 0.6588158498288699,
|
| 74 |
+
"valid_sudoku": 0.014,
|
| 75 |
+
"clues_kept": 1.0,
|
| 76 |
+
"nfe": 59.5
|
| 77 |
+
},
|
| 78 |
+
"entbudget_0.01": {
|
| 79 |
+
"board": 0.088,
|
| 80 |
+
"cell": 0.715923926466956,
|
| 81 |
+
"valid_sudoku": 0.088,
|
| 82 |
+
"clues_kept": 1.0,
|
| 83 |
+
"nfe": 58.25
|
| 84 |
+
},
|
| 85 |
+
"entbudget_0.05": {
|
| 86 |
+
"board": 0.138,
|
| 87 |
+
"cell": 0.7457393881655552,
|
| 88 |
+
"valid_sudoku": 0.138,
|
| 89 |
+
"clues_kept": 1.0,
|
| 90 |
+
"nfe": 57.75
|
| 91 |
+
},
|
| 92 |
+
"entbudget_0.2": {
|
| 93 |
+
"board": 0.187,
|
| 94 |
+
"cell": 0.7710913517518789,
|
| 95 |
+
"valid_sudoku": 0.187,
|
| 96 |
+
"clues_kept": 1.0,
|
| 97 |
+
"nfe": 58.0
|
| 98 |
+
},
|
| 99 |
+
"entbudget_0.5": {
|
| 100 |
+
"board": 0.212,
|
| 101 |
+
"cell": 0.7857344483257471,
|
| 102 |
+
"valid_sudoku": 0.212,
|
| 103 |
+
"clues_kept": 1.0,
|
| 104 |
+
"nfe": 57.0
|
| 105 |
+
},
|
| 106 |
+
"entbudget_1.0": {
|
| 107 |
+
"board": 0.214,
|
| 108 |
+
"cell": 0.788504287075262,
|
| 109 |
+
"valid_sudoku": 0.214,
|
| 110 |
+
"clues_kept": 1.0,
|
| 111 |
+
"nfe": 57.0
|
| 112 |
+
},
|
| 113 |
+
"entbudget_2.0": {
|
| 114 |
+
"board": 0.189,
|
| 115 |
+
"cell": 0.8019829928372323,
|
| 116 |
+
"valid_sudoku": 0.189,
|
| 117 |
+
"clues_kept": 1.0,
|
| 118 |
+
"nfe": 57.5
|
| 119 |
+
},
|
| 120 |
+
"conf_0.9": {
|
| 121 |
+
"board": 0.194,
|
| 122 |
+
"cell": 0.7980840478458769,
|
| 123 |
+
"valid_sudoku": 0.194,
|
| 124 |
+
"clues_kept": 1.0,
|
| 125 |
+
"nfe": 57.75
|
| 126 |
+
},
|
| 127 |
+
"conf_0.99": {
|
| 128 |
+
"board": 0.213,
|
| 129 |
+
"cell": 0.7785893228891005,
|
| 130 |
+
"valid_sudoku": 0.213,
|
| 131 |
+
"clues_kept": 1.0,
|
| 132 |
+
"nfe": 58.0
|
| 133 |
+
},
|
| 134 |
+
"conf_0.999": {
|
| 135 |
+
"board": 0.149,
|
| 136 |
+
"cell": 0.7357009279841925,
|
| 137 |
+
"valid_sudoku": 0.149,
|
| 138 |
+
"clues_kept": 1.0,
|
| 139 |
+
"nfe": 58.25
|
| 140 |
+
}
|
| 141 |
+
},
|
| 142 |
+
"best_board": 0.214
|
| 143 |
+
}
|
results/rec1-full-s0.json
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "full",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"recurrent": true,
|
| 8 |
+
"R": 32,
|
| 9 |
+
"R_infer": 64,
|
| 10 |
+
"root": "data/sudoku",
|
| 11 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 12 |
+
"augment": true,
|
| 13 |
+
"d": 128,
|
| 14 |
+
"layers": 1,
|
| 15 |
+
"heads": 4,
|
| 16 |
+
"bs": 256,
|
| 17 |
+
"steps": 120000,
|
| 18 |
+
"lr": 0.0003,
|
| 19 |
+
"warmup": 3000,
|
| 20 |
+
"seed": 0,
|
| 21 |
+
"eval_bs": 250,
|
| 22 |
+
"ckpt_every": 10000,
|
| 23 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/rec1-full-s0"
|
| 24 |
+
},
|
| 25 |
+
"decode": {
|
| 26 |
+
"fixedK_1": {
|
| 27 |
+
"board": 0.974,
|
| 28 |
+
"cell": 0.9903849546593275,
|
| 29 |
+
"valid_sudoku": 0.974,
|
| 30 |
+
"clues_kept": 1.0,
|
| 31 |
+
"nfe": 1.0
|
| 32 |
+
},
|
| 33 |
+
"fixedK_2": {
|
| 34 |
+
"board": 0.975,
|
| 35 |
+
"cell": 0.9906319466497301,
|
| 36 |
+
"valid_sudoku": 0.975,
|
| 37 |
+
"clues_kept": 1.0,
|
| 38 |
+
"nfe": 2.0
|
| 39 |
+
},
|
| 40 |
+
"fixedK_4": {
|
| 41 |
+
"board": 0.981,
|
| 42 |
+
"cell": 0.9929254437034685,
|
| 43 |
+
"valid_sudoku": 0.981,
|
| 44 |
+
"clues_kept": 1.0,
|
| 45 |
+
"nfe": 4.0
|
| 46 |
+
},
|
| 47 |
+
"fixedK_8": {
|
| 48 |
+
"board": 0.985,
|
| 49 |
+
"cell": 0.9943721110758266,
|
| 50 |
+
"valid_sudoku": 0.985,
|
| 51 |
+
"clues_kept": 1.0,
|
| 52 |
+
"nfe": 8.0
|
| 53 |
+
},
|
| 54 |
+
"fixedK_16": {
|
| 55 |
+
"board": 0.992,
|
| 56 |
+
"cell": 0.9960481281535585,
|
| 57 |
+
"valid_sudoku": 0.992,
|
| 58 |
+
"clues_kept": 1.0,
|
| 59 |
+
"nfe": 16.0
|
| 60 |
+
},
|
| 61 |
+
"fixedK_32": {
|
| 62 |
+
"board": 0.993,
|
| 63 |
+
"cell": 0.9976182915211178,
|
| 64 |
+
"valid_sudoku": 0.993,
|
| 65 |
+
"clues_kept": 1.0,
|
| 66 |
+
"nfe": 32.0
|
| 67 |
+
},
|
| 68 |
+
"fixedK_61": {
|
| 69 |
+
"board": 0.995,
|
| 70 |
+
"cell": 0.9980946332168943,
|
| 71 |
+
"valid_sudoku": 0.995,
|
| 72 |
+
"clues_kept": 1.0,
|
| 73 |
+
"nfe": 59.5
|
| 74 |
+
},
|
| 75 |
+
"entbudget_0.01": {
|
| 76 |
+
"board": 0.996,
|
| 77 |
+
"cell": 0.9975653646660315,
|
| 78 |
+
"valid_sudoku": 0.996,
|
| 79 |
+
"clues_kept": 1.0,
|
| 80 |
+
"nfe": 16.0
|
| 81 |
+
},
|
| 82 |
+
"entbudget_0.05": {
|
| 83 |
+
"board": 0.993,
|
| 84 |
+
"cell": 0.9970360961151689,
|
| 85 |
+
"valid_sudoku": 0.993,
|
| 86 |
+
"clues_kept": 1.0,
|
| 87 |
+
"nfe": 20.5
|
| 88 |
+
},
|
| 89 |
+
"entbudget_0.2": {
|
| 90 |
+
"board": 0.992,
|
| 91 |
+
"cell": 0.9954835750326382,
|
| 92 |
+
"valid_sudoku": 0.992,
|
| 93 |
+
"clues_kept": 1.0,
|
| 94 |
+
"nfe": 13.75
|
| 95 |
+
},
|
| 96 |
+
"entbudget_0.5": {
|
| 97 |
+
"board": 0.992,
|
| 98 |
+
"cell": 0.9982886983522106,
|
| 99 |
+
"valid_sudoku": 0.992,
|
| 100 |
+
"clues_kept": 1.0,
|
| 101 |
+
"nfe": 12.0
|
| 102 |
+
},
|
| 103 |
+
"entbudget_1.0": {
|
| 104 |
+
"board": 0.986,
|
| 105 |
+
"cell": 0.9937722733848489,
|
| 106 |
+
"valid_sudoku": 0.986,
|
| 107 |
+
"clues_kept": 1.0,
|
| 108 |
+
"nfe": 8.0
|
| 109 |
+
},
|
| 110 |
+
"entbudget_2.0": {
|
| 111 |
+
"board": 0.98,
|
| 112 |
+
"cell": 0.9918316220316855,
|
| 113 |
+
"valid_sudoku": 0.98,
|
| 114 |
+
"clues_kept": 1.0,
|
| 115 |
+
"nfe": 4.75
|
| 116 |
+
},
|
| 117 |
+
"conf_0.9": {
|
| 118 |
+
"board": 0.982,
|
| 119 |
+
"cell": 0.9933135739741011,
|
| 120 |
+
"valid_sudoku": 0.982,
|
| 121 |
+
"clues_kept": 1.0,
|
| 122 |
+
"nfe": 12.25
|
| 123 |
+
},
|
| 124 |
+
"conf_0.99": {
|
| 125 |
+
"board": 0.992,
|
| 126 |
+
"cell": 0.9959246321583571,
|
| 127 |
+
"valid_sudoku": 0.992,
|
| 128 |
+
"clues_kept": 1.0,
|
| 129 |
+
"nfe": 13.75
|
| 130 |
+
},
|
| 131 |
+
"conf_0.999": {
|
| 132 |
+
"board": 0.993,
|
| 133 |
+
"cell": 0.9963304047140186,
|
| 134 |
+
"valid_sudoku": 0.993,
|
| 135 |
+
"clues_kept": 1.0,
|
| 136 |
+
"nfe": 21.75
|
| 137 |
+
}
|
| 138 |
+
},
|
| 139 |
+
"best_board": 0.996
|
| 140 |
+
}
|
results/rec1-full-s1.json
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "full",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"recurrent": true,
|
| 8 |
+
"R": 32,
|
| 9 |
+
"R_infer": 64,
|
| 10 |
+
"root": "data/sudoku",
|
| 11 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 12 |
+
"augment": true,
|
| 13 |
+
"d": 128,
|
| 14 |
+
"layers": 1,
|
| 15 |
+
"heads": 4,
|
| 16 |
+
"bs": 256,
|
| 17 |
+
"steps": 120000,
|
| 18 |
+
"lr": 0.0003,
|
| 19 |
+
"warmup": 3000,
|
| 20 |
+
"seed": 1,
|
| 21 |
+
"eval_bs": 250,
|
| 22 |
+
"ckpt_every": 10000,
|
| 23 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/rec1-full-s1"
|
| 24 |
+
},
|
| 25 |
+
"decode": {
|
| 26 |
+
"fixedK_1": {
|
| 27 |
+
"board": 0.973,
|
| 28 |
+
"cell": 0.9908612963551039,
|
| 29 |
+
"valid_sudoku": 0.973,
|
| 30 |
+
"clues_kept": 1.0,
|
| 31 |
+
"nfe": 1.0
|
| 32 |
+
},
|
| 33 |
+
"fixedK_2": {
|
| 34 |
+
"board": 0.973,
|
| 35 |
+
"cell": 0.9926078825729509,
|
| 36 |
+
"valid_sudoku": 0.973,
|
| 37 |
+
"clues_kept": 1.0,
|
| 38 |
+
"nfe": 2.0
|
| 39 |
+
},
|
| 40 |
+
"fixedK_4": {
|
| 41 |
+
"board": 0.986,
|
| 42 |
+
"cell": 0.9944073956458841,
|
| 43 |
+
"valid_sudoku": 0.986,
|
| 44 |
+
"clues_kept": 1.0,
|
| 45 |
+
"nfe": 4.0
|
| 46 |
+
},
|
| 47 |
+
"fixedK_8": {
|
| 48 |
+
"board": 0.992,
|
| 49 |
+
"cell": 0.9958011361631558,
|
| 50 |
+
"valid_sudoku": 0.992,
|
| 51 |
+
"clues_kept": 1.0,
|
| 52 |
+
"nfe": 8.0
|
| 53 |
+
},
|
| 54 |
+
"fixedK_16": {
|
| 55 |
+
"board": 0.992,
|
| 56 |
+
"cell": 0.9961186972936735,
|
| 57 |
+
"valid_sudoku": 0.992,
|
| 58 |
+
"clues_kept": 1.0,
|
| 59 |
+
"nfe": 16.0
|
| 60 |
+
},
|
| 61 |
+
"fixedK_32": {
|
| 62 |
+
"board": 0.998,
|
| 63 |
+
"cell": 0.9995060160191948,
|
| 64 |
+
"valid_sudoku": 0.998,
|
| 65 |
+
"clues_kept": 1.0,
|
| 66 |
+
"nfe": 32.0
|
| 67 |
+
},
|
| 68 |
+
"fixedK_61": {
|
| 69 |
+
"board": 0.997,
|
| 70 |
+
"cell": 0.9989591051833033,
|
| 71 |
+
"valid_sudoku": 0.997,
|
| 72 |
+
"clues_kept": 1.0,
|
| 73 |
+
"nfe": 59.5
|
| 74 |
+
},
|
| 75 |
+
"entbudget_0.01": {
|
| 76 |
+
"board": 0.996,
|
| 77 |
+
"cell": 0.9984298366324407,
|
| 78 |
+
"valid_sudoku": 0.996,
|
| 79 |
+
"clues_kept": 1.0,
|
| 80 |
+
"nfe": 14.75
|
| 81 |
+
},
|
| 82 |
+
"entbudget_0.05": {
|
| 83 |
+
"board": 0.994,
|
| 84 |
+
"cell": 0.9970184538301401,
|
| 85 |
+
"valid_sudoku": 0.994,
|
| 86 |
+
"clues_kept": 1.0,
|
| 87 |
+
"nfe": 16.75
|
| 88 |
+
},
|
| 89 |
+
"entbudget_0.2": {
|
| 90 |
+
"board": 0.993,
|
| 91 |
+
"cell": 0.9975830069510603,
|
| 92 |
+
"valid_sudoku": 0.993,
|
| 93 |
+
"clues_kept": 1.0,
|
| 94 |
+
"nfe": 10.75
|
| 95 |
+
},
|
| 96 |
+
"entbudget_0.5": {
|
| 97 |
+
"board": 0.993,
|
| 98 |
+
"cell": 0.9971948766804276,
|
| 99 |
+
"valid_sudoku": 0.993,
|
| 100 |
+
"clues_kept": 1.0,
|
| 101 |
+
"nfe": 7.0
|
| 102 |
+
},
|
| 103 |
+
"entbudget_1.0": {
|
| 104 |
+
"board": 0.991,
|
| 105 |
+
"cell": 0.9968243886948237,
|
| 106 |
+
"valid_sudoku": 0.991,
|
| 107 |
+
"clues_kept": 1.0,
|
| 108 |
+
"nfe": 6.0
|
| 109 |
+
},
|
| 110 |
+
"entbudget_2.0": {
|
| 111 |
+
"board": 0.981,
|
| 112 |
+
"cell": 0.9932606471190149,
|
| 113 |
+
"valid_sudoku": 0.981,
|
| 114 |
+
"clues_kept": 1.0,
|
| 115 |
+
"nfe": 4.25
|
| 116 |
+
},
|
| 117 |
+
"conf_0.9": {
|
| 118 |
+
"board": 0.988,
|
| 119 |
+
"cell": 0.9962774778589323,
|
| 120 |
+
"valid_sudoku": 0.988,
|
| 121 |
+
"clues_kept": 1.0,
|
| 122 |
+
"nfe": 6.5
|
| 123 |
+
},
|
| 124 |
+
"conf_0.99": {
|
| 125 |
+
"board": 0.994,
|
| 126 |
+
"cell": 0.9969831692600826,
|
| 127 |
+
"valid_sudoku": 0.994,
|
| 128 |
+
"clues_kept": 1.0,
|
| 129 |
+
"nfe": 11.75
|
| 130 |
+
},
|
| 131 |
+
"conf_0.999": {
|
| 132 |
+
"board": 0.995,
|
| 133 |
+
"cell": 0.9978476412264916,
|
| 134 |
+
"valid_sudoku": 0.995,
|
| 135 |
+
"clues_kept": 1.0,
|
| 136 |
+
"nfe": 18.0
|
| 137 |
+
}
|
| 138 |
+
},
|
| 139 |
+
"best_board": 0.998
|
| 140 |
+
}
|
results/rec1-mdlm-s0.json
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"recurrent": true,
|
| 8 |
+
"R": 32,
|
| 9 |
+
"R_infer": 64,
|
| 10 |
+
"root": "data/sudoku",
|
| 11 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 12 |
+
"augment": true,
|
| 13 |
+
"d": 128,
|
| 14 |
+
"layers": 1,
|
| 15 |
+
"heads": 4,
|
| 16 |
+
"bs": 256,
|
| 17 |
+
"steps": 120000,
|
| 18 |
+
"lr": 0.0003,
|
| 19 |
+
"warmup": 3000,
|
| 20 |
+
"seed": 0,
|
| 21 |
+
"eval_bs": 250,
|
| 22 |
+
"ckpt_every": 10000,
|
| 23 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/rec1-mdlm-s0"
|
| 24 |
+
},
|
| 25 |
+
"decode": {
|
| 26 |
+
"fixedK_1": {
|
| 27 |
+
"board": 0.912,
|
| 28 |
+
"cell": 0.9668148618609083,
|
| 29 |
+
"valid_sudoku": 0.912,
|
| 30 |
+
"clues_kept": 1.0,
|
| 31 |
+
"nfe": 1.0
|
| 32 |
+
},
|
| 33 |
+
"fixedK_2": {
|
| 34 |
+
"board": 0.925,
|
| 35 |
+
"cell": 0.971842913094104,
|
| 36 |
+
"valid_sudoku": 0.925,
|
| 37 |
+
"clues_kept": 1.0,
|
| 38 |
+
"nfe": 2.0
|
| 39 |
+
},
|
| 40 |
+
"fixedK_4": {
|
| 41 |
+
"board": 0.94,
|
| 42 |
+
"cell": 0.973871775872411,
|
| 43 |
+
"valid_sudoku": 0.94,
|
| 44 |
+
"clues_kept": 1.0,
|
| 45 |
+
"nfe": 4.0
|
| 46 |
+
},
|
| 47 |
+
"fixedK_8": {
|
| 48 |
+
"board": 0.944,
|
| 49 |
+
"cell": 0.978370558554744,
|
| 50 |
+
"valid_sudoku": 0.944,
|
| 51 |
+
"clues_kept": 1.0,
|
| 52 |
+
"nfe": 8.0
|
| 53 |
+
},
|
| 54 |
+
"fixedK_16": {
|
| 55 |
+
"board": 0.949,
|
| 56 |
+
"cell": 0.9784058431248015,
|
| 57 |
+
"valid_sudoku": 0.949,
|
| 58 |
+
"clues_kept": 1.0,
|
| 59 |
+
"nfe": 16.0
|
| 60 |
+
},
|
| 61 |
+
"fixedK_32": {
|
| 62 |
+
"board": 0.955,
|
| 63 |
+
"cell": 0.9792526728061819,
|
| 64 |
+
"valid_sudoku": 0.955,
|
| 65 |
+
"clues_kept": 1.0,
|
| 66 |
+
"nfe": 32.0
|
| 67 |
+
},
|
| 68 |
+
"fixedK_61": {
|
| 69 |
+
"board": 0.959,
|
| 70 |
+
"cell": 0.9808934053138563,
|
| 71 |
+
"valid_sudoku": 0.959,
|
| 72 |
+
"clues_kept": 1.0,
|
| 73 |
+
"nfe": 59.5
|
| 74 |
+
},
|
| 75 |
+
"entbudget_0.01": {
|
| 76 |
+
"board": 0.971,
|
| 77 |
+
"cell": 0.9863448713877422,
|
| 78 |
+
"valid_sudoku": 0.971,
|
| 79 |
+
"clues_kept": 1.0,
|
| 80 |
+
"nfe": 36.5
|
| 81 |
+
},
|
| 82 |
+
"entbudget_0.05": {
|
| 83 |
+
"board": 0.958,
|
| 84 |
+
"cell": 0.9812109664443739,
|
| 85 |
+
"valid_sudoku": 0.958,
|
| 86 |
+
"clues_kept": 1.0,
|
| 87 |
+
"nfe": 31.0
|
| 88 |
+
},
|
| 89 |
+
"entbudget_0.2": {
|
| 90 |
+
"board": 0.962,
|
| 91 |
+
"cell": 0.9824988532514731,
|
| 92 |
+
"valid_sudoku": 0.962,
|
| 93 |
+
"clues_kept": 1.0,
|
| 94 |
+
"nfe": 23.75
|
| 95 |
+
},
|
| 96 |
+
"entbudget_0.5": {
|
| 97 |
+
"board": 0.962,
|
| 98 |
+
"cell": 0.982869341237077,
|
| 99 |
+
"valid_sudoku": 0.962,
|
| 100 |
+
"clues_kept": 1.0,
|
| 101 |
+
"nfe": 19.25
|
| 102 |
+
},
|
| 103 |
+
"entbudget_1.0": {
|
| 104 |
+
"board": 0.948,
|
| 105 |
+
"cell": 0.978370558554744,
|
| 106 |
+
"valid_sudoku": 0.948,
|
| 107 |
+
"clues_kept": 1.0,
|
| 108 |
+
"nfe": 13.25
|
| 109 |
+
},
|
| 110 |
+
"entbudget_2.0": {
|
| 111 |
+
"board": 0.942,
|
| 112 |
+
"cell": 0.9765357609117533,
|
| 113 |
+
"valid_sudoku": 0.942,
|
| 114 |
+
"clues_kept": 1.0,
|
| 115 |
+
"nfe": 7.5
|
| 116 |
+
},
|
| 117 |
+
"conf_0.9": {
|
| 118 |
+
"board": 0.946,
|
| 119 |
+
"cell": 0.9811403973042588,
|
| 120 |
+
"valid_sudoku": 0.946,
|
| 121 |
+
"clues_kept": 1.0,
|
| 122 |
+
"nfe": 19.5
|
| 123 |
+
},
|
| 124 |
+
"conf_0.99": {
|
| 125 |
+
"board": 0.96,
|
| 126 |
+
"cell": 0.9817755195652941,
|
| 127 |
+
"valid_sudoku": 0.96,
|
| 128 |
+
"clues_kept": 1.0,
|
| 129 |
+
"nfe": 28.25
|
| 130 |
+
},
|
| 131 |
+
"conf_0.999": {
|
| 132 |
+
"board": 0.962,
|
| 133 |
+
"cell": 0.9814756007198052,
|
| 134 |
+
"valid_sudoku": 0.962,
|
| 135 |
+
"clues_kept": 1.0,
|
| 136 |
+
"nfe": 37.5
|
| 137 |
+
}
|
| 138 |
+
},
|
| 139 |
+
"best_board": 0.971
|
| 140 |
+
}
|
results/rec1-mdlm-s1.json
ADDED
|
@@ -0,0 +1,140 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"recurrent": true,
|
| 8 |
+
"R": 32,
|
| 9 |
+
"R_infer": 64,
|
| 10 |
+
"root": "data/sudoku",
|
| 11 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 12 |
+
"augment": true,
|
| 13 |
+
"d": 128,
|
| 14 |
+
"layers": 1,
|
| 15 |
+
"heads": 4,
|
| 16 |
+
"bs": 256,
|
| 17 |
+
"steps": 120000,
|
| 18 |
+
"lr": 0.0003,
|
| 19 |
+
"warmup": 3000,
|
| 20 |
+
"seed": 1,
|
| 21 |
+
"eval_bs": 250,
|
| 22 |
+
"ckpt_every": 10000,
|
| 23 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/rec1-mdlm-s1"
|
| 24 |
+
},
|
| 25 |
+
"decode": {
|
| 26 |
+
"fixedK_1": {
|
| 27 |
+
"board": 0.835,
|
| 28 |
+
"cell": 0.9497900568081578,
|
| 29 |
+
"valid_sudoku": 0.835,
|
| 30 |
+
"clues_kept": 1.0,
|
| 31 |
+
"nfe": 1.0
|
| 32 |
+
},
|
| 33 |
+
"fixedK_2": {
|
| 34 |
+
"board": 0.855,
|
| 35 |
+
"cell": 0.9570586782400057,
|
| 36 |
+
"valid_sudoku": 0.855,
|
| 37 |
+
"clues_kept": 1.0,
|
| 38 |
+
"nfe": 2.0
|
| 39 |
+
},
|
| 40 |
+
"fixedK_4": {
|
| 41 |
+
"board": 0.898,
|
| 42 |
+
"cell": 0.962333721463604,
|
| 43 |
+
"valid_sudoku": 0.898,
|
| 44 |
+
"clues_kept": 1.0,
|
| 45 |
+
"nfe": 4.0
|
| 46 |
+
},
|
| 47 |
+
"fixedK_8": {
|
| 48 |
+
"board": 0.923,
|
| 49 |
+
"cell": 0.9695141314703081,
|
| 50 |
+
"valid_sudoku": 0.923,
|
| 51 |
+
"clues_kept": 1.0,
|
| 52 |
+
"nfe": 8.0
|
| 53 |
+
},
|
| 54 |
+
"fixedK_16": {
|
| 55 |
+
"board": 0.94,
|
| 56 |
+
"cell": 0.9763593380614657,
|
| 57 |
+
"valid_sudoku": 0.94,
|
| 58 |
+
"clues_kept": 1.0,
|
| 59 |
+
"nfe": 16.0
|
| 60 |
+
},
|
| 61 |
+
"fixedK_32": {
|
| 62 |
+
"board": 0.944,
|
| 63 |
+
"cell": 0.9770826717476447,
|
| 64 |
+
"valid_sudoku": 0.944,
|
| 65 |
+
"clues_kept": 1.0,
|
| 66 |
+
"nfe": 32.0
|
| 67 |
+
},
|
| 68 |
+
"fixedK_61": {
|
| 69 |
+
"board": 0.947,
|
| 70 |
+
"cell": 0.9754595815249991,
|
| 71 |
+
"valid_sudoku": 0.947,
|
| 72 |
+
"clues_kept": 1.0,
|
| 73 |
+
"nfe": 59.5
|
| 74 |
+
},
|
| 75 |
+
"entbudget_0.01": {
|
| 76 |
+
"board": 0.947,
|
| 77 |
+
"cell": 0.9762005574962069,
|
| 78 |
+
"valid_sudoku": 0.947,
|
| 79 |
+
"clues_kept": 1.0,
|
| 80 |
+
"nfe": 52.25
|
| 81 |
+
},
|
| 82 |
+
"entbudget_0.05": {
|
| 83 |
+
"board": 0.96,
|
| 84 |
+
"cell": 0.9831869023675947,
|
| 85 |
+
"valid_sudoku": 0.96,
|
| 86 |
+
"clues_kept": 1.0,
|
| 87 |
+
"nfe": 40.5
|
| 88 |
+
},
|
| 89 |
+
"entbudget_0.2": {
|
| 90 |
+
"board": 0.946,
|
| 91 |
+
"cell": 0.9753360855297978,
|
| 92 |
+
"valid_sudoku": 0.946,
|
| 93 |
+
"clues_kept": 1.0,
|
| 94 |
+
"nfe": 32.25
|
| 95 |
+
},
|
| 96 |
+
"entbudget_0.5": {
|
| 97 |
+
"board": 0.944,
|
| 98 |
+
"cell": 0.9752655163896828,
|
| 99 |
+
"valid_sudoku": 0.944,
|
| 100 |
+
"clues_kept": 1.0,
|
| 101 |
+
"nfe": 23.0
|
| 102 |
+
},
|
| 103 |
+
"entbudget_1.0": {
|
| 104 |
+
"board": 0.925,
|
| 105 |
+
"cell": 0.9730425884760594,
|
| 106 |
+
"valid_sudoku": 0.925,
|
| 107 |
+
"clues_kept": 1.0,
|
| 108 |
+
"nfe": 12.0
|
| 109 |
+
},
|
| 110 |
+
"entbudget_2.0": {
|
| 111 |
+
"board": 0.889,
|
| 112 |
+
"cell": 0.9628453477294379,
|
| 113 |
+
"valid_sudoku": 0.889,
|
| 114 |
+
"clues_kept": 1.0,
|
| 115 |
+
"nfe": 8.5
|
| 116 |
+
},
|
| 117 |
+
"conf_0.9": {
|
| 118 |
+
"board": 0.884,
|
| 119 |
+
"cell": 0.9695141314703081,
|
| 120 |
+
"valid_sudoku": 0.884,
|
| 121 |
+
"clues_kept": 1.0,
|
| 122 |
+
"nfe": 15.25
|
| 123 |
+
},
|
| 124 |
+
"conf_0.99": {
|
| 125 |
+
"board": 0.946,
|
| 126 |
+
"cell": 0.976482834056667,
|
| 127 |
+
"valid_sudoku": 0.946,
|
| 128 |
+
"clues_kept": 1.0,
|
| 129 |
+
"nfe": 35.0
|
| 130 |
+
},
|
| 131 |
+
"conf_0.999": {
|
| 132 |
+
"board": 0.956,
|
| 133 |
+
"cell": 0.9794820225115557,
|
| 134 |
+
"valid_sudoku": 0.956,
|
| 135 |
+
"clues_kept": 1.0,
|
| 136 |
+
"nfe": 48.25
|
| 137 |
+
}
|
| 138 |
+
},
|
| 139 |
+
"best_board": 0.96
|
| 140 |
+
}
|
results/sudh-full-s0.json
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "full",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"root": "data/sudoku",
|
| 8 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 9 |
+
"augment": true,
|
| 10 |
+
"d": 512,
|
| 11 |
+
"layers": 12,
|
| 12 |
+
"heads": 8,
|
| 13 |
+
"bs": 256,
|
| 14 |
+
"steps": 150000,
|
| 15 |
+
"lr": 0.0001,
|
| 16 |
+
"warmup": 4000,
|
| 17 |
+
"seed": 0,
|
| 18 |
+
"eval_bs": 200,
|
| 19 |
+
"ckpt_every": 10000,
|
| 20 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/sudh-full-s0"
|
| 21 |
+
},
|
| 22 |
+
"decode": {
|
| 23 |
+
"fixedK_1": {
|
| 24 |
+
"board": 0.419,
|
| 25 |
+
"cell": 0.8971101937122896,
|
| 26 |
+
"valid_sudoku": 0.419,
|
| 27 |
+
"clues_kept": 1.0,
|
| 28 |
+
"nfe": 1.0
|
| 29 |
+
},
|
| 30 |
+
"fixedK_2": {
|
| 31 |
+
"board": 0.703,
|
| 32 |
+
"cell": 0.9253025651882432,
|
| 33 |
+
"valid_sudoku": 0.703,
|
| 34 |
+
"clues_kept": 1.0,
|
| 35 |
+
"nfe": 2.0
|
| 36 |
+
},
|
| 37 |
+
"fixedK_4": {
|
| 38 |
+
"board": 0.792,
|
| 39 |
+
"cell": 0.9413394022793832,
|
| 40 |
+
"valid_sudoku": 0.792,
|
| 41 |
+
"clues_kept": 1.0,
|
| 42 |
+
"nfe": 4.0
|
| 43 |
+
},
|
| 44 |
+
"fixedK_8": {
|
| 45 |
+
"board": 0.847,
|
| 46 |
+
"cell": 0.9466850146430966,
|
| 47 |
+
"valid_sudoku": 0.847,
|
| 48 |
+
"clues_kept": 1.0,
|
| 49 |
+
"nfe": 8.0
|
| 50 |
+
},
|
| 51 |
+
"fixedK_16": {
|
| 52 |
+
"board": 0.853,
|
| 53 |
+
"cell": 0.9441268833139268,
|
| 54 |
+
"valid_sudoku": 0.853,
|
| 55 |
+
"clues_kept": 1.0,
|
| 56 |
+
"nfe": 16.0
|
| 57 |
+
},
|
| 58 |
+
"fixedK_32": {
|
| 59 |
+
"board": 0.87,
|
| 60 |
+
"cell": 0.9467908683532691,
|
| 61 |
+
"valid_sudoku": 0.87,
|
| 62 |
+
"clues_kept": 1.0,
|
| 63 |
+
"nfe": 32.0
|
| 64 |
+
},
|
| 65 |
+
"fixedK_61": {
|
| 66 |
+
"board": 0.883,
|
| 67 |
+
"cell": 0.9494901379626689,
|
| 68 |
+
"valid_sudoku": 0.883,
|
| 69 |
+
"clues_kept": 1.0,
|
| 70 |
+
"nfe": 59.4
|
| 71 |
+
},
|
| 72 |
+
"entbudget_0.01": {
|
| 73 |
+
"board": 0.876,
|
| 74 |
+
"cell": 0.9501605447937617,
|
| 75 |
+
"valid_sudoku": 0.876,
|
| 76 |
+
"clues_kept": 1.0,
|
| 77 |
+
"nfe": 30.6
|
| 78 |
+
},
|
| 79 |
+
"entbudget_0.05": {
|
| 80 |
+
"board": 0.866,
|
| 81 |
+
"cell": 0.9512190818954871,
|
| 82 |
+
"valid_sudoku": 0.866,
|
| 83 |
+
"clues_kept": 1.0,
|
| 84 |
+
"nfe": 23.6
|
| 85 |
+
},
|
| 86 |
+
"entbudget_0.2": {
|
| 87 |
+
"board": 0.795,
|
| 88 |
+
"cell": 0.9393105395010761,
|
| 89 |
+
"valid_sudoku": 0.795,
|
| 90 |
+
"clues_kept": 1.0,
|
| 91 |
+
"nfe": 17.4
|
| 92 |
+
},
|
| 93 |
+
"entbudget_0.5": {
|
| 94 |
+
"board": 0.731,
|
| 95 |
+
"cell": 0.9327652517554074,
|
| 96 |
+
"valid_sudoku": 0.731,
|
| 97 |
+
"clues_kept": 1.0,
|
| 98 |
+
"nfe": 10.4
|
| 99 |
+
},
|
| 100 |
+
"entbudget_1.0": {
|
| 101 |
+
"board": 0.636,
|
| 102 |
+
"cell": 0.9199745951095586,
|
| 103 |
+
"valid_sudoku": 0.636,
|
| 104 |
+
"clues_kept": 1.0,
|
| 105 |
+
"nfe": 7.8
|
| 106 |
+
},
|
| 107 |
+
"entbudget_2.0": {
|
| 108 |
+
"board": 0.531,
|
| 109 |
+
"cell": 0.9122119896969055,
|
| 110 |
+
"valid_sudoku": 0.531,
|
| 111 |
+
"clues_kept": 1.0,
|
| 112 |
+
"nfe": 5.4
|
| 113 |
+
},
|
| 114 |
+
"conf_0.9": {
|
| 115 |
+
"board": 0.582,
|
| 116 |
+
"cell": 0.9171341872199287,
|
| 117 |
+
"valid_sudoku": 0.582,
|
| 118 |
+
"clues_kept": 1.0,
|
| 119 |
+
"nfe": 7.8
|
| 120 |
+
},
|
| 121 |
+
"conf_0.99": {
|
| 122 |
+
"board": 0.81,
|
| 123 |
+
"cell": 0.9423802970960798,
|
| 124 |
+
"valid_sudoku": 0.81,
|
| 125 |
+
"clues_kept": 1.0,
|
| 126 |
+
"nfe": 22.6
|
| 127 |
+
},
|
| 128 |
+
"conf_0.999": {
|
| 129 |
+
"board": 0.866,
|
| 130 |
+
"cell": 0.9504604636392505,
|
| 131 |
+
"valid_sudoku": 0.866,
|
| 132 |
+
"clues_kept": 1.0,
|
| 133 |
+
"nfe": 30.0
|
| 134 |
+
}
|
| 135 |
+
},
|
| 136 |
+
"best_board": 0.883
|
| 137 |
+
}
|
results/sudh-full-s1.json
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "full",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"root": "data/sudoku",
|
| 8 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 9 |
+
"augment": true,
|
| 10 |
+
"d": 512,
|
| 11 |
+
"layers": 12,
|
| 12 |
+
"heads": 8,
|
| 13 |
+
"bs": 256,
|
| 14 |
+
"steps": 150000,
|
| 15 |
+
"lr": 0.0001,
|
| 16 |
+
"warmup": 4000,
|
| 17 |
+
"seed": 1,
|
| 18 |
+
"eval_bs": 200,
|
| 19 |
+
"ckpt_every": 10000,
|
| 20 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/sudh-full-s1"
|
| 21 |
+
},
|
| 22 |
+
"decode": {
|
| 23 |
+
"fixedK_1": {
|
| 24 |
+
"board": 0.383,
|
| 25 |
+
"cell": 0.8876362866518471,
|
| 26 |
+
"valid_sudoku": 0.383,
|
| 27 |
+
"clues_kept": 1.0,
|
| 28 |
+
"nfe": 1.0
|
| 29 |
+
},
|
| 30 |
+
"fixedK_2": {
|
| 31 |
+
"board": 0.682,
|
| 32 |
+
"cell": 0.9200804488197312,
|
| 33 |
+
"valid_sudoku": 0.682,
|
| 34 |
+
"clues_kept": 1.0,
|
| 35 |
+
"nfe": 2.0
|
| 36 |
+
},
|
| 37 |
+
"fixedK_4": {
|
| 38 |
+
"board": 0.801,
|
| 39 |
+
"cell": 0.9390635475106736,
|
| 40 |
+
"valid_sudoku": 0.801,
|
| 41 |
+
"clues_kept": 1.0,
|
| 42 |
+
"nfe": 4.0
|
| 43 |
+
},
|
| 44 |
+
"fixedK_8": {
|
| 45 |
+
"board": 0.851,
|
| 46 |
+
"cell": 0.9509720899050845,
|
| 47 |
+
"valid_sudoku": 0.851,
|
| 48 |
+
"clues_kept": 1.0,
|
| 49 |
+
"nfe": 8.0
|
| 50 |
+
},
|
| 51 |
+
"fixedK_16": {
|
| 52 |
+
"board": 0.87,
|
| 53 |
+
"cell": 0.9515719275960622,
|
| 54 |
+
"valid_sudoku": 0.87,
|
| 55 |
+
"clues_kept": 1.0,
|
| 56 |
+
"nfe": 16.0
|
| 57 |
+
},
|
| 58 |
+
"fixedK_32": {
|
| 59 |
+
"board": 0.886,
|
| 60 |
+
"cell": 0.9570233936699482,
|
| 61 |
+
"valid_sudoku": 0.886,
|
| 62 |
+
"clues_kept": 1.0,
|
| 63 |
+
"nfe": 32.0
|
| 64 |
+
},
|
| 65 |
+
"fixedK_61": {
|
| 66 |
+
"board": 0.884,
|
| 67 |
+
"cell": 0.9524716841325288,
|
| 68 |
+
"valid_sudoku": 0.884,
|
| 69 |
+
"clues_kept": 1.0,
|
| 70 |
+
"nfe": 59.4
|
| 71 |
+
},
|
| 72 |
+
"entbudget_0.01": {
|
| 73 |
+
"board": 0.895,
|
| 74 |
+
"cell": 0.9573409548004658,
|
| 75 |
+
"valid_sudoku": 0.895,
|
| 76 |
+
"clues_kept": 1.0,
|
| 77 |
+
"nfe": 30.4
|
| 78 |
+
},
|
| 79 |
+
"entbudget_0.05": {
|
| 80 |
+
"board": 0.851,
|
| 81 |
+
"cell": 0.9468437952083554,
|
| 82 |
+
"valid_sudoku": 0.851,
|
| 83 |
+
"clues_kept": 1.0,
|
| 84 |
+
"nfe": 20.2
|
| 85 |
+
},
|
| 86 |
+
"entbudget_0.2": {
|
| 87 |
+
"board": 0.788,
|
| 88 |
+
"cell": 0.9315479340884232,
|
| 89 |
+
"valid_sudoku": 0.788,
|
| 90 |
+
"clues_kept": 1.0,
|
| 91 |
+
"nfe": 13.6
|
| 92 |
+
},
|
| 93 |
+
"entbudget_0.5": {
|
| 94 |
+
"board": 0.697,
|
| 95 |
+
"cell": 0.919215976853322,
|
| 96 |
+
"valid_sudoku": 0.697,
|
| 97 |
+
"clues_kept": 1.0,
|
| 98 |
+
"nfe": 9.8
|
| 99 |
+
},
|
| 100 |
+
"entbudget_1.0": {
|
| 101 |
+
"board": 0.604,
|
| 102 |
+
"cell": 0.9126001199675382,
|
| 103 |
+
"valid_sudoku": 0.604,
|
| 104 |
+
"clues_kept": 1.0,
|
| 105 |
+
"nfe": 6.4
|
| 106 |
+
},
|
| 107 |
+
"entbudget_2.0": {
|
| 108 |
+
"board": 0.482,
|
| 109 |
+
"cell": 0.9062136127871282,
|
| 110 |
+
"valid_sudoku": 0.482,
|
| 111 |
+
"clues_kept": 1.0,
|
| 112 |
+
"nfe": 5.0
|
| 113 |
+
},
|
| 114 |
+
"conf_0.9": {
|
| 115 |
+
"board": 0.538,
|
| 116 |
+
"cell": 0.9054549945308916,
|
| 117 |
+
"valid_sudoku": 0.538,
|
| 118 |
+
"clues_kept": 1.0,
|
| 119 |
+
"nfe": 7.8
|
| 120 |
+
},
|
| 121 |
+
"conf_0.99": {
|
| 122 |
+
"board": 0.777,
|
| 123 |
+
"cell": 0.9316890723686532,
|
| 124 |
+
"valid_sudoku": 0.777,
|
| 125 |
+
"clues_kept": 1.0,
|
| 126 |
+
"nfe": 17.0
|
| 127 |
+
},
|
| 128 |
+
"conf_0.999": {
|
| 129 |
+
"board": 0.884,
|
| 130 |
+
"cell": 0.9553473765922162,
|
| 131 |
+
"valid_sudoku": 0.884,
|
| 132 |
+
"clues_kept": 1.0,
|
| 133 |
+
"nfe": 26.2
|
| 134 |
+
}
|
| 135 |
+
},
|
| 136 |
+
"best_board": 0.895
|
| 137 |
+
}
|
results/sudh-mdlm-s0.json
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"root": "data/sudoku",
|
| 8 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 9 |
+
"augment": true,
|
| 10 |
+
"d": 512,
|
| 11 |
+
"layers": 12,
|
| 12 |
+
"heads": 8,
|
| 13 |
+
"bs": 256,
|
| 14 |
+
"steps": 150000,
|
| 15 |
+
"lr": 0.0001,
|
| 16 |
+
"warmup": 4000,
|
| 17 |
+
"seed": 0,
|
| 18 |
+
"eval_bs": 200,
|
| 19 |
+
"ckpt_every": 10000,
|
| 20 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/sudh-mdlm-s0"
|
| 21 |
+
},
|
| 22 |
+
"decode": {
|
| 23 |
+
"fixedK_1": {
|
| 24 |
+
"board": 0.05,
|
| 25 |
+
"cell": 0.7941851028545217,
|
| 26 |
+
"valid_sudoku": 0.05,
|
| 27 |
+
"clues_kept": 1.0,
|
| 28 |
+
"nfe": 1.0
|
| 29 |
+
},
|
| 30 |
+
"fixedK_2": {
|
| 31 |
+
"board": 0.511,
|
| 32 |
+
"cell": 0.8766980699340179,
|
| 33 |
+
"valid_sudoku": 0.511,
|
| 34 |
+
"clues_kept": 1.0,
|
| 35 |
+
"nfe": 2.0
|
| 36 |
+
},
|
| 37 |
+
"fixedK_4": {
|
| 38 |
+
"board": 0.71,
|
| 39 |
+
"cell": 0.9216682544723193,
|
| 40 |
+
"valid_sudoku": 0.71,
|
| 41 |
+
"clues_kept": 1.0,
|
| 42 |
+
"nfe": 4.0
|
| 43 |
+
},
|
| 44 |
+
"fixedK_8": {
|
| 45 |
+
"board": 0.807,
|
| 46 |
+
"cell": 0.9365407007515614,
|
| 47 |
+
"valid_sudoku": 0.807,
|
| 48 |
+
"clues_kept": 1.0,
|
| 49 |
+
"nfe": 8.0
|
| 50 |
+
},
|
| 51 |
+
"fixedK_16": {
|
| 52 |
+
"board": 0.856,
|
| 53 |
+
"cell": 0.9451501358455947,
|
| 54 |
+
"valid_sudoku": 0.856,
|
| 55 |
+
"clues_kept": 1.0,
|
| 56 |
+
"nfe": 16.0
|
| 57 |
+
},
|
| 58 |
+
"fixedK_32": {
|
| 59 |
+
"board": 0.872,
|
| 60 |
+
"cell": 0.947020218058643,
|
| 61 |
+
"valid_sudoku": 0.872,
|
| 62 |
+
"clues_kept": 1.0,
|
| 63 |
+
"nfe": 32.0
|
| 64 |
+
},
|
| 65 |
+
"fixedK_61": {
|
| 66 |
+
"board": 0.879,
|
| 67 |
+
"cell": 0.948537454571116,
|
| 68 |
+
"valid_sudoku": 0.879,
|
| 69 |
+
"clues_kept": 1.0,
|
| 70 |
+
"nfe": 59.4
|
| 71 |
+
},
|
| 72 |
+
"entbudget_0.01": {
|
| 73 |
+
"board": 0.89,
|
| 74 |
+
"cell": 0.9501252602237041,
|
| 75 |
+
"valid_sudoku": 0.89,
|
| 76 |
+
"clues_kept": 1.0,
|
| 77 |
+
"nfe": 24.6
|
| 78 |
+
},
|
| 79 |
+
"entbudget_0.05": {
|
| 80 |
+
"board": 0.872,
|
| 81 |
+
"cell": 0.9483786740058573,
|
| 82 |
+
"valid_sudoku": 0.872,
|
| 83 |
+
"clues_kept": 1.0,
|
| 84 |
+
"nfe": 22.2
|
| 85 |
+
},
|
| 86 |
+
"entbudget_0.2": {
|
| 87 |
+
"board": 0.856,
|
| 88 |
+
"cell": 0.9515542853110335,
|
| 89 |
+
"valid_sudoku": 0.856,
|
| 90 |
+
"clues_kept": 1.0,
|
| 91 |
+
"nfe": 16.2
|
| 92 |
+
},
|
| 93 |
+
"entbudget_0.5": {
|
| 94 |
+
"board": 0.766,
|
| 95 |
+
"cell": 0.9330298860308387,
|
| 96 |
+
"valid_sudoku": 0.766,
|
| 97 |
+
"clues_kept": 1.0,
|
| 98 |
+
"nfe": 11.2
|
| 99 |
+
},
|
| 100 |
+
"entbudget_1.0": {
|
| 101 |
+
"board": 0.647,
|
| 102 |
+
"cell": 0.9091951589569881,
|
| 103 |
+
"valid_sudoku": 0.647,
|
| 104 |
+
"clues_kept": 1.0,
|
| 105 |
+
"nfe": 7.8
|
| 106 |
+
},
|
| 107 |
+
"entbudget_2.0": {
|
| 108 |
+
"board": 0.44,
|
| 109 |
+
"cell": 0.8767333545040754,
|
| 110 |
+
"valid_sudoku": 0.44,
|
| 111 |
+
"clues_kept": 1.0,
|
| 112 |
+
"nfe": 5.0
|
| 113 |
+
},
|
| 114 |
+
"conf_0.9": {
|
| 115 |
+
"board": 0.413,
|
| 116 |
+
"cell": 0.8837726262305494,
|
| 117 |
+
"valid_sudoku": 0.413,
|
| 118 |
+
"clues_kept": 1.0,
|
| 119 |
+
"nfe": 9.6
|
| 120 |
+
},
|
| 121 |
+
"conf_0.99": {
|
| 122 |
+
"board": 0.847,
|
| 123 |
+
"cell": 0.9466144455029816,
|
| 124 |
+
"valid_sudoku": 0.847,
|
| 125 |
+
"clues_kept": 1.0,
|
| 126 |
+
"nfe": 20.0
|
| 127 |
+
},
|
| 128 |
+
"conf_0.999": {
|
| 129 |
+
"board": 0.89,
|
| 130 |
+
"cell": 0.9560354257083378,
|
| 131 |
+
"valid_sudoku": 0.89,
|
| 132 |
+
"clues_kept": 1.0,
|
| 133 |
+
"nfe": 24.6
|
| 134 |
+
}
|
| 135 |
+
},
|
| 136 |
+
"best_board": 0.89
|
| 137 |
+
}
|
results/sudh-mdlm-s1.json
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"args": {
|
| 3 |
+
"mode": "mdlm",
|
| 4 |
+
"K": 8,
|
| 5 |
+
"lam": 1.0,
|
| 6 |
+
"pe": "ape",
|
| 7 |
+
"root": "data/sudoku",
|
| 8 |
+
"hard": "/scratch/zt1/project/msml612/user/govind02/pcmd/data/sudoku_hard",
|
| 9 |
+
"augment": true,
|
| 10 |
+
"d": 512,
|
| 11 |
+
"layers": 12,
|
| 12 |
+
"heads": 8,
|
| 13 |
+
"bs": 256,
|
| 14 |
+
"steps": 150000,
|
| 15 |
+
"lr": 0.0001,
|
| 16 |
+
"warmup": 4000,
|
| 17 |
+
"seed": 1,
|
| 18 |
+
"eval_bs": 200,
|
| 19 |
+
"ckpt_every": 10000,
|
| 20 |
+
"out": "/scratch/zt1/project/msml612/user/govind02/pcmd/runs/sudh-mdlm-s1"
|
| 21 |
+
},
|
| 22 |
+
"decode": {
|
| 23 |
+
"fixedK_1": {
|
| 24 |
+
"board": 0.038,
|
| 25 |
+
"cell": 0.7791185914399633,
|
| 26 |
+
"valid_sudoku": 0.038,
|
| 27 |
+
"clues_kept": 1.0,
|
| 28 |
+
"nfe": 1.0
|
| 29 |
+
},
|
| 30 |
+
"fixedK_2": {
|
| 31 |
+
"board": 0.455,
|
| 32 |
+
"cell": 0.8588970043400022,
|
| 33 |
+
"valid_sudoku": 0.455,
|
| 34 |
+
"clues_kept": 1.0,
|
| 35 |
+
"nfe": 2.0
|
| 36 |
+
},
|
| 37 |
+
"fixedK_4": {
|
| 38 |
+
"board": 0.696,
|
| 39 |
+
"cell": 0.9148406901661903,
|
| 40 |
+
"valid_sudoku": 0.696,
|
| 41 |
+
"clues_kept": 1.0,
|
| 42 |
+
"nfe": 4.0
|
| 43 |
+
},
|
| 44 |
+
"fixedK_8": {
|
| 45 |
+
"board": 0.779,
|
| 46 |
+
"cell": 0.9273314279665502,
|
| 47 |
+
"valid_sudoku": 0.779,
|
| 48 |
+
"clues_kept": 1.0,
|
| 49 |
+
"nfe": 8.0
|
| 50 |
+
},
|
| 51 |
+
"fixedK_16": {
|
| 52 |
+
"board": 0.822,
|
| 53 |
+
"cell": 0.9361878550509862,
|
| 54 |
+
"valid_sudoku": 0.822,
|
| 55 |
+
"clues_kept": 1.0,
|
| 56 |
+
"nfe": 16.0
|
| 57 |
+
},
|
| 58 |
+
"fixedK_32": {
|
| 59 |
+
"board": 0.858,
|
| 60 |
+
"cell": 0.9440210296037542,
|
| 61 |
+
"valid_sudoku": 0.858,
|
| 62 |
+
"clues_kept": 1.0,
|
| 63 |
+
"nfe": 32.0
|
| 64 |
+
},
|
| 65 |
+
"fixedK_61": {
|
| 66 |
+
"board": 0.873,
|
| 67 |
+
"cell": 0.9453794855509685,
|
| 68 |
+
"valid_sudoku": 0.873,
|
| 69 |
+
"clues_kept": 1.0,
|
| 70 |
+
"nfe": 59.4
|
| 71 |
+
},
|
| 72 |
+
"entbudget_0.01": {
|
| 73 |
+
"board": 0.869,
|
| 74 |
+
"cell": 0.9446737941498183,
|
| 75 |
+
"valid_sudoku": 0.869,
|
| 76 |
+
"clues_kept": 1.0,
|
| 77 |
+
"nfe": 26.6
|
| 78 |
+
},
|
| 79 |
+
"entbudget_0.05": {
|
| 80 |
+
"board": 0.865,
|
| 81 |
+
"cell": 0.9473554214741894,
|
| 82 |
+
"valid_sudoku": 0.865,
|
| 83 |
+
"clues_kept": 1.0,
|
| 84 |
+
"nfe": 21.2
|
| 85 |
+
},
|
| 86 |
+
"entbudget_0.2": {
|
| 87 |
+
"board": 0.817,
|
| 88 |
+
"cell": 0.9398750926219964,
|
| 89 |
+
"valid_sudoku": 0.817,
|
| 90 |
+
"clues_kept": 1.0,
|
| 91 |
+
"nfe": 17.8
|
| 92 |
+
},
|
| 93 |
+
"entbudget_0.5": {
|
| 94 |
+
"board": 0.752,
|
| 95 |
+
"cell": 0.9303129741364101,
|
| 96 |
+
"valid_sudoku": 0.752,
|
| 97 |
+
"clues_kept": 1.0,
|
| 98 |
+
"nfe": 9.6
|
| 99 |
+
},
|
| 100 |
+
"entbudget_1.0": {
|
| 101 |
+
"board": 0.602,
|
| 102 |
+
"cell": 0.8949578349387812,
|
| 103 |
+
"valid_sudoku": 0.602,
|
| 104 |
+
"clues_kept": 1.0,
|
| 105 |
+
"nfe": 7.2
|
| 106 |
+
},
|
| 107 |
+
"entbudget_2.0": {
|
| 108 |
+
"board": 0.425,
|
| 109 |
+
"cell": 0.8687413993860484,
|
| 110 |
+
"valid_sudoku": 0.425,
|
| 111 |
+
"clues_kept": 1.0,
|
| 112 |
+
"nfe": 5.0
|
| 113 |
+
},
|
| 114 |
+
"conf_0.9": {
|
| 115 |
+
"board": 0.402,
|
| 116 |
+
"cell": 0.8780918104512897,
|
| 117 |
+
"valid_sudoku": 0.402,
|
| 118 |
+
"clues_kept": 1.0,
|
| 119 |
+
"nfe": 11.0
|
| 120 |
+
},
|
| 121 |
+
"conf_0.99": {
|
| 122 |
+
"board": 0.815,
|
| 123 |
+
"cell": 0.9398045234818814,
|
| 124 |
+
"valid_sudoku": 0.815,
|
| 125 |
+
"clues_kept": 1.0,
|
| 126 |
+
"nfe": 21.6
|
| 127 |
+
},
|
| 128 |
+
"conf_0.999": {
|
| 129 |
+
"board": 0.861,
|
| 130 |
+
"cell": 0.9431389153523164,
|
| 131 |
+
"valid_sudoku": 0.861,
|
| 132 |
+
"clues_kept": 1.0,
|
| 133 |
+
"nfe": 26.4
|
| 134 |
+
}
|
| 135 |
+
},
|
| 136 |
+
"best_board": 0.873
|
| 137 |
+
}
|
slurm/abl.sh
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Ablations that test the mechanism rather than decorate the result.
|
| 3 |
+
#
|
| 4 |
+
# 0,1 recurrent WITHOUT deep supervision -> is per-step supervision the
|
| 5 |
+
# ingredient, or is recurrence alone enough?
|
| 6 |
+
# 2,3 32 DISTINCT layers, d=128, no weight sharing -> is it the depth, or the
|
| 7 |
+
# weight sharing? Same depth, ~32x the parameters.
|
| 8 |
+
# 4,5 R=8 instead of 32 -> how much recurrence does the task actually need?
|
| 9 |
+
#
|
| 10 |
+
# All use the schedule-matched objective, which is the arm that reached 99.70%.
|
| 11 |
+
#SBATCH -J abl
|
| 12 |
+
#SBATCH -p gpu-h100
|
| 13 |
+
#SBATCH --gpus=h100:1
|
| 14 |
+
#SBATCH -c 8
|
| 15 |
+
#SBATCH --mem=48g
|
| 16 |
+
#SBATCH -t 5:00:00
|
| 17 |
+
#SBATCH -o logs/abl-%A_%a.out
|
| 18 |
+
#SBATCH --array=0-5%6
|
| 19 |
+
source /etc/profile
|
| 20 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 21 |
+
module load python/3.10.10
|
| 22 |
+
export PYTHONPATH=
|
| 23 |
+
cd $SCR/pcmd/src
|
| 24 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 25 |
+
S=$((I % 2))
|
| 26 |
+
COMMON="--mode full --seed $S --pe ape --hard $SCR/pcmd/data/sudoku_hard --augment \
|
| 27 |
+
--heads 4 --bs 256 --steps 120000 --lr 3e-4 --warmup 3000 --K 8 --lam 1.0 \
|
| 28 |
+
--eval_bs 250 --ckpt_every 10000"
|
| 29 |
+
case $((I / 2)) in
|
| 30 |
+
0) NAME="abl-nodeepsup-s$S"
|
| 31 |
+
ARGS="--recurrent --R 32 --R_infer 64 --no_deep_sup --d 128 --layers 1" ;;
|
| 32 |
+
1) NAME="abl-deep32L-s$S"
|
| 33 |
+
ARGS="--d 128 --layers 32" ;;
|
| 34 |
+
2) NAME="abl-R8-s$S"
|
| 35 |
+
ARGS="--recurrent --R 8 --R_infer 16 --d 128 --layers 1" ;;
|
| 36 |
+
esac
|
| 37 |
+
echo "=== $NAME ==="
|
| 38 |
+
$SCR/hf_env/bin/python -u train_sudoku.py $ARGS $COMMON --out $SCR/pcmd/runs/$NAME
|
slurm/b_gpu-a100_1g.5gb --gres=gpu:a100_1g.5gb:1.sh
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J bench-gpu-a100_1g.5gb --gres=gpu:a100_1g.5gb:1
|
| 3 |
+
#SBATCH -p gpu-a100_1g.5gb --gres=gpu:a100_1g.5gb:1
|
| 4 |
+
#SBATCH
|
| 5 |
+
#SBATCH -c 4
|
| 6 |
+
#SBATCH --mem=16g
|
| 7 |
+
#SBATCH -t 0:20:00
|
| 8 |
+
#SBATCH -o logs/bench-gpu-a100_1g.5gb --gres=gpu:a100_1g.5gb:1-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u bench.py
|
slurm/b_gpu-h100 --gpus=h100:1.sh
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J bench-gpu-h100 --gpus=h100:1
|
| 3 |
+
#SBATCH -p gpu-h100 --gpus=h100:1
|
| 4 |
+
#SBATCH
|
| 5 |
+
#SBATCH -c 4
|
| 6 |
+
#SBATCH --mem=16g
|
| 7 |
+
#SBATCH -t 0:20:00
|
| 8 |
+
#SBATCH -o logs/bench-gpu-h100 --gpus=h100:1-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u bench.py
|
slurm/base.sh
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# The baselines that can overturn the paper's framing.
|
| 3 |
+
#
|
| 4 |
+
# 0,1 DIRECT + RECURRENT: predict every blank from the puzzle, no masking
|
| 5 |
+
# schedule, recurrent with deep supervision. This is the Recurrent
|
| 6 |
+
# Transformer setup reimplemented in our codebase - the control that asks
|
| 7 |
+
# whether the DIFFUSION framing earns its place at all. If it matches
|
| 8 |
+
# 99.70%, diffusion contributes nothing and the paper's framing collapses.
|
| 9 |
+
# 2,3 DIRECT + FEED-FORWARD: same, without recurrence. Isolates recurrence
|
| 10 |
+
# from the objective.
|
| 11 |
+
# 4,5 AUTOREGRESSIVE: left-to-right cell filling at matched size. Asks whether
|
| 12 |
+
# any iterative model would do, making the generative framing irrelevant.
|
| 13 |
+
#SBATCH -J base
|
| 14 |
+
#SBATCH -p gpu-h100
|
| 15 |
+
#SBATCH --gpus=h100:1
|
| 16 |
+
#SBATCH -c 8
|
| 17 |
+
#SBATCH --mem=48g
|
| 18 |
+
#SBATCH -t 5:00:00
|
| 19 |
+
#SBATCH -o logs/base-%A_%a.out
|
| 20 |
+
#SBATCH --array=0-5%6
|
| 21 |
+
source /etc/profile
|
| 22 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 23 |
+
module load python/3.10.10
|
| 24 |
+
export PYTHONPATH=
|
| 25 |
+
cd $SCR/pcmd/src
|
| 26 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 27 |
+
S=$((I % 2))
|
| 28 |
+
COMMON="--seed $S --pe ape --hard $SCR/pcmd/data/sudoku_hard --augment \
|
| 29 |
+
--heads 4 --bs 256 --steps 120000 --lr 3e-4 --warmup 3000 \
|
| 30 |
+
--eval_bs 250 --ckpt_every 10000"
|
| 31 |
+
case $((I / 2)) in
|
| 32 |
+
0) NAME="base-directrec-s$S"
|
| 33 |
+
ARGS="--objective direct --recurrent --R 32 --R_infer 64 --d 128 --layers 1" ;;
|
| 34 |
+
1) NAME="base-directff-s$S"
|
| 35 |
+
ARGS="--objective direct --d 128 --layers 12" ;;
|
| 36 |
+
2) NAME="base-ar-s$S"
|
| 37 |
+
ARGS="--objective ar --d 128 --layers 12" ;;
|
| 38 |
+
esac
|
| 39 |
+
echo "=== $NAME ==="
|
| 40 |
+
$SCR/hf_env/bin/python -u train_sudoku.py $ARGS $COMMON --out $SCR/pcmd/runs/$NAME
|
slurm/bh.sh
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J benchH
|
| 3 |
+
#SBATCH -p gpu-h100
|
| 4 |
+
#SBATCH --gpus=h100:1
|
| 5 |
+
#SBATCH -c 4
|
| 6 |
+
#SBATCH --mem=16g
|
| 7 |
+
#SBATCH -t 0:20:00
|
| 8 |
+
#SBATCH -o logs/benchH-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u bench.py
|
slurm/bm.sh
ADDED
|
@@ -0,0 +1,14 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J benchM
|
| 3 |
+
#SBATCH -p gpu-a100_1g.5gb
|
| 4 |
+
#SBATCH --gres=gpu:a100_1g.5gb:1
|
| 5 |
+
#SBATCH -c 4
|
| 6 |
+
#SBATCH --mem=16g
|
| 7 |
+
#SBATCH -t 0:20:00
|
| 8 |
+
#SBATCH -o logs/benchM-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u bench.py
|
slurm/cmpall.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J cmpall
|
| 3 |
+
#SBATCH -p gpu-h100
|
| 4 |
+
#SBATCH --gpus=h100:1
|
| 5 |
+
#SBATCH -c 16
|
| 6 |
+
#SBATCH --mem=48g
|
| 7 |
+
#SBATCH -t 2:00:00
|
| 8 |
+
#SBATCH -o logs/cmpall-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u sudoku_compare.py --hard $SCR/pcmd/data/sudoku_hard \
|
| 15 |
+
--runs $SCR/pcmd/runs --pattern 'rec1-*' --out $SCR/pcmd/figures/compare_recurrent.json
|
slurm/fix.sh
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Properly specified baselines, replacing two broken ones.
|
| 3 |
+
#
|
| 4 |
+
# 0 DIRECT + all-cell supervision: the published recurrent classifier trains
|
| 5 |
+
# on all 81 cells, not just the blanks. Our blanks-only version plateaued at
|
| 6 |
+
# loss 0.34 against diffusion's 0.15, which is the likeliest cause.
|
| 7 |
+
# 1 same, second seed - the first attempt had huge seed variance (40.9 / 6.6)
|
| 8 |
+
# 2 AR PREFIX-LM: [puzzle][solution], causal, loss on the solution half. The
|
| 9 |
+
# earlier in-place causal version was mis-specified: Sudoku clues are
|
| 10 |
+
# scattered, so a causal model could not attend to clues on its right.
|
| 11 |
+
#SBATCH -J fix
|
| 12 |
+
#SBATCH -p gpu-h100
|
| 13 |
+
#SBATCH --gpus=h100:1
|
| 14 |
+
#SBATCH -c 8
|
| 15 |
+
#SBATCH --mem=48g
|
| 16 |
+
#SBATCH -t 2:00:00
|
| 17 |
+
#SBATCH -o logs/fix-%A_%a.out
|
| 18 |
+
#SBATCH --array=0-2%3
|
| 19 |
+
source /etc/profile
|
| 20 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 21 |
+
module load python/3.10.10
|
| 22 |
+
export PYTHONPATH=
|
| 23 |
+
cd $SCR/pcmd/src
|
| 24 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 25 |
+
COMMON="--pe ape --hard $SCR/pcmd/data/sudoku_hard --augment --heads 4 --bs 256 \
|
| 26 |
+
--steps 110000 --lr 3e-4 --warmup 3000 --eval_bs 250 --ckpt_every 10000"
|
| 27 |
+
case $I in
|
| 28 |
+
0) NAME="fix-directall-s0"; ARGS="--objective direct --all_cells --recurrent --R 32 --R_infer 64 --d 128 --layers 1 --seed 0" ;;
|
| 29 |
+
1) NAME="fix-directall-s1"; ARGS="--objective direct --all_cells --recurrent --R 32 --R_infer 64 --d 128 --layers 1 --seed 1" ;;
|
| 30 |
+
2) NAME="fix-arprefix-s0"; ARGS="--objective ar --d 128 --layers 12 --seed 0" ;;
|
| 31 |
+
esac
|
| 32 |
+
echo "=== $NAME ==="
|
| 33 |
+
$SCR/hf_env/bin/python -u train_sudoku.py $ARGS $COMMON --out $SCR/pcmd/runs/$NAME
|
slurm/genhard.sh
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# CPU-only: generate the hard Sudoku benchmark. No GPU needed, so this costs
|
| 3 |
+
# almost nothing against the class GPU allocation.
|
| 4 |
+
#SBATCH -J genhard
|
| 5 |
+
#SBATCH -p standard
|
| 6 |
+
#SBATCH -c 64
|
| 7 |
+
#SBATCH --mem=64g
|
| 8 |
+
#SBATCH -t 3:00:00
|
| 9 |
+
#SBATCH -o logs/genhard-%j.out
|
| 10 |
+
source /etc/profile
|
| 11 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 12 |
+
module load python/3.10.10
|
| 13 |
+
export PYTHONPATH=
|
| 14 |
+
cd $SCR/pcmd/src
|
| 15 |
+
$SCR/hf_env/bin/python -u gen_hard_sudoku.py --n_train 100000 --n_test 1000 \
|
| 16 |
+
--workers 64 --out $SCR/pcmd/data/sudoku_hard
|
slurm/inj.sh
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Closes the one acknowledged gap in the ablation.
|
| 3 |
+
#
|
| 4 |
+
# The headline comparison - recurrent 99.70% against 32 distinct layers 94.50% -
|
| 5 |
+
# conflates TWO differences: weight sharing, and re-supplying the input at every
|
| 6 |
+
# application. --no_inject holds architecture, depth, parameter count and
|
| 7 |
+
# objective fixed and changes only the injection, which isolates them.
|
| 8 |
+
#
|
| 9 |
+
# no-injection stays near 98% -> weight sharing is the mechanism
|
| 10 |
+
# no-injection falls toward 94% -> injection is, and our explanation is wrong
|
| 11 |
+
#
|
| 12 |
+
# Runs on a 5GB MIG slice: at 212k parameters and 81-token sequences the
|
| 13 |
+
# optimiser state is a few MB, and that partition bills at 9/min against the
|
| 14 |
+
# H100's 144 - 16x more compute from the same remaining allocation.
|
| 15 |
+
#SBATCH -J inj
|
| 16 |
+
#SBATCH -p gpu-a100_1g.5gb
|
| 17 |
+
#SBATCH --gres=gpu:a100_1g.5gb:1
|
| 18 |
+
#SBATCH -c 4
|
| 19 |
+
#SBATCH --mem=16g
|
| 20 |
+
#SBATCH -t 6:00:00
|
| 21 |
+
#SBATCH -o logs/inj-%A_%a.out
|
| 22 |
+
#SBATCH --array=0-4%5
|
| 23 |
+
source /etc/profile
|
| 24 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 25 |
+
module load python/3.10.10
|
| 26 |
+
export PYTHONPATH=
|
| 27 |
+
cd $SCR/pcmd/src
|
| 28 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 29 |
+
COMMON="--mode full --pe ape --hard $SCR/pcmd/data/sudoku_hard --augment \
|
| 30 |
+
--heads 4 --bs 256 --steps 110000 --lr 3e-4 --warmup 3000 --K 8 --lam 1.0 \
|
| 31 |
+
--eval_bs 200 --ckpt_every 8000"
|
| 32 |
+
case $I in
|
| 33 |
+
0) NAME="inj-noinject-s0"; ARGS="--recurrent --R 32 --R_infer 64 --no_inject --d 128 --layers 1 --seed 0" ;;
|
| 34 |
+
1) NAME="inj-noinject-s1"; ARGS="--recurrent --R 32 --R_infer 64 --no_inject --d 128 --layers 1 --seed 1" ;;
|
| 35 |
+
2) NAME="inj-deep32L-s1"; ARGS="--d 128 --layers 32 --seed 1" ;;
|
| 36 |
+
3) NAME="inj-rec-s2"; ARGS="--recurrent --R 32 --R_infer 64 --d 128 --layers 1 --seed 2" ;;
|
| 37 |
+
4) NAME="inj-rec-s3"; ARGS="--recurrent --R 32 --R_infer 64 --d 128 --layers 1 --seed 3" ;;
|
| 38 |
+
esac
|
| 39 |
+
echo "=== $NAME ==="
|
| 40 |
+
$SCR/hf_env/bin/python -u train_sudoku.py $ARGS $COMMON --out $SCR/pcmd/runs/$NAME
|
slurm/inj2.sh
ADDED
|
@@ -0,0 +1,41 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Closes the one acknowledged gap in the ablation.
|
| 3 |
+
#
|
| 4 |
+
# The headline comparison - recurrent 99.70% against 32 distinct layers 94.50% -
|
| 5 |
+
# conflates TWO differences: weight sharing, and re-supplying the input at every
|
| 6 |
+
# application. --no_inject holds architecture, depth, parameter count and
|
| 7 |
+
# objective fixed and changes only the injection, which isolates them.
|
| 8 |
+
#
|
| 9 |
+
# no-injection stays near 98% -> weight sharing is the mechanism
|
| 10 |
+
# no-injection falls toward 94% -> injection is, and our explanation is wrong
|
| 11 |
+
#
|
| 12 |
+
# Runs on a 5GB MIG slice: at 212k parameters and 81-token sequences the
|
| 13 |
+
# optimiser state is a few MB, and that partition bills at 9/min against the
|
| 14 |
+
# H100's 144 - 16x more compute from the same remaining allocation.
|
| 15 |
+
#SBATCH -J inj
|
| 16 |
+
#SBATCH -p gpu-a100_1g.5gb
|
| 17 |
+
#SBATCH --gres=gpu:a100_1g.5gb:1
|
| 18 |
+
#SBATCH -c 4
|
| 19 |
+
#SBATCH --mem=16g
|
| 20 |
+
#SBATCH -t 6:00:00
|
| 21 |
+
#SBATCH -o logs/inj-%A_%a.out
|
| 22 |
+
#SBATCH --array=0-2%3
|
| 23 |
+
source /etc/profile
|
| 24 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 25 |
+
module load python/3.10.10
|
| 26 |
+
export PYTHONPATH=
|
| 27 |
+
export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True
|
| 28 |
+
cd $SCR/pcmd/src
|
| 29 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 30 |
+
COMMON="--mode full --pe ape --hard $SCR/pcmd/data/sudoku_hard --augment \
|
| 31 |
+
--heads 4 --bs 64 --steps 110000 --lr 3e-4 --warmup 3000 --K 8 --lam 1.0 \
|
| 32 |
+
--eval_bs 200 --ckpt_every 8000"
|
| 33 |
+
case $I in
|
| 34 |
+
0) NAME="inj-noinject-s0"; ARGS="--recurrent --R 32 --R_infer 64 --no_inject --d 128 --layers 1 --seed 0" ;;
|
| 35 |
+
1) NAME="inj-noinject-s1"; ARGS="--recurrent --R 32 --R_infer 64 --no_inject --d 128 --layers 1 --seed 1" ;;
|
| 36 |
+
2) NAME="inj-deep32L-s1"; ARGS="--d 128 --layers 32 --seed 1" ;;
|
| 37 |
+
3) NAME="inj-rec-s2"; ARGS="--recurrent --R 32 --R_infer 64 --d 128 --layers 1 --seed 2" ;;
|
| 38 |
+
4) NAME="inj-rec-s3"; ARGS="--recurrent --R 32 --R_infer 64 --d 128 --layers 1 --seed 3" ;;
|
| 39 |
+
esac
|
| 40 |
+
echo "=== $NAME ==="
|
| 41 |
+
$SCR/hf_env/bin/python -u train_sudoku.py $ARGS $COMMON --out $SCR/pcmd/runs/$NAME
|
slurm/mol1.sh
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# QM9 molecular generation - validity retained as denoising steps fall.
|
| 3 |
+
#
|
| 4 |
+
# This is the axis the discrete graph-diffusion literature competes on. DiGress
|
| 5 |
+
# and its successors use hundreds of denoising steps; the question is how much
|
| 6 |
+
# validity survives at few steps, and that depends on the training schedule and
|
| 7 |
+
# the decoding rule rather than on model scale - which is why it is winnable at
|
| 8 |
+
# this budget when text8 bits-per-character is not.
|
| 9 |
+
#
|
| 10 |
+
# Both of the project's findings are under test here at once, and QM9 is the
|
| 11 |
+
# strongest possible case for the second. Its marginals are extremely
|
| 12 |
+
# position-dependent: node slot 8 is NEVER occupied, and edge slots range from
|
| 13 |
+
# never-bonded to always-bonded, giving a mean total-variation distance of 0.435
|
| 14 |
+
# between a slot and the average slot. Finding 2 therefore predicts that a
|
| 15 |
+
# relative-only encoding badly caps few-step generation here, and that absolute
|
| 16 |
+
# position information produces a large gain. Registered before the runs.
|
| 17 |
+
#SBATCH -J mol1
|
| 18 |
+
#SBATCH -p gpu-h100
|
| 19 |
+
#SBATCH --gpus=h100:1
|
| 20 |
+
#SBATCH -c 8
|
| 21 |
+
#SBATCH --mem=48g
|
| 22 |
+
#SBATCH -t 3:00:00
|
| 23 |
+
#SBATCH -o logs/mol1-%A_%a.out
|
| 24 |
+
#SBATCH --array=0-7%8
|
| 25 |
+
source /etc/profile
|
| 26 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 27 |
+
module load python/3.10.10
|
| 28 |
+
export PYTHONPATH=
|
| 29 |
+
cd $SCR/pcmd/src
|
| 30 |
+
MODES=(mdlm matched); PES=(rope ape); SEEDS=(0 1)
|
| 31 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 32 |
+
M=${MODES[$((I / 4))]}; P=${PES[$(((I % 4) / 2))]}; S=${SEEDS[$((I % 2))]}
|
| 33 |
+
NAME="mol1-$M-$P-s$S"
|
| 34 |
+
echo "=== $NAME ==="
|
| 35 |
+
$SCR/hf_env/bin/python -u train_mol.py --mode $M --pe $P --seed $S --K 8 \
|
| 36 |
+
--root $SCR/pcmd/data/qm9 --d 384 --layers 8 --heads 8 --bs 256 \
|
| 37 |
+
--steps 60000 --lr 1e-4 --warmup 2000 --n_gen 4096 --gen_bs 512 \
|
| 38 |
+
--out $SCR/pcmd/runs/$NAME
|
slurm/molfair.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J molfair
|
| 3 |
+
#SBATCH -p gpu-h100
|
| 4 |
+
#SBATCH --gpus=h100:1
|
| 5 |
+
#SBATCH -c 8
|
| 6 |
+
#SBATCH --mem=48g
|
| 7 |
+
#SBATCH -t 2:00:00
|
| 8 |
+
#SBATCH -o logs/molfair-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u mol_fair.py --runs $SCR/pcmd/runs --root $SCR/pcmd/data/qm9 \
|
| 15 |
+
--n_gen 2048 --gen_bs 512 --out $SCR/pcmd/figures/mol_fair.json
|
slurm/rec1.sh
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Recurrent masked diffusion with deep supervision, on hard Sudoku.
|
| 3 |
+
#
|
| 4 |
+
# The gap this targets is measured, not guessed. Our 37.86M-parameter 12-layer
|
| 5 |
+
# denoiser reaches 89.4%; Yang et al. (2023) reach 99.5% on a comparable split
|
| 6 |
+
# with 211k parameters - 180x smaller - by applying ONE block 32 times with loss
|
| 7 |
+
# at every recurrence. Constraint propagation is iterative, and a fixed-depth
|
| 8 |
+
# network cannot express thirty rounds of it at any width.
|
| 9 |
+
#
|
| 10 |
+
# Config follows theirs (d=128, 1 layer, 4 heads, R=32) so the comparison is
|
| 11 |
+
# about the diffusion framing rather than about scale. Inference runs R=64,
|
| 12 |
+
# more than training, which deep supervision is what makes possible.
|
| 13 |
+
#SBATCH -J rec1
|
| 14 |
+
#SBATCH -p gpu-h100
|
| 15 |
+
#SBATCH --gpus=h100:1
|
| 16 |
+
#SBATCH -c 8
|
| 17 |
+
#SBATCH --mem=48g
|
| 18 |
+
#SBATCH -t 5:00:00
|
| 19 |
+
#SBATCH -o logs/rec1-%A_%a.out
|
| 20 |
+
#SBATCH --array=0-3%4
|
| 21 |
+
source /etc/profile
|
| 22 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 23 |
+
module load python/3.10.10
|
| 24 |
+
export PYTHONPATH=
|
| 25 |
+
cd $SCR/pcmd/src
|
| 26 |
+
MODES=(mdlm full); SEEDS=(0 1)
|
| 27 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 28 |
+
M=${MODES[$((I / 2))]}; S=${SEEDS[$((I % 2))]}
|
| 29 |
+
NAME="rec1-$M-s$S"
|
| 30 |
+
echo "=== $NAME ==="
|
| 31 |
+
$SCR/hf_env/bin/python -u train_sudoku.py --recurrent --R 32 --R_infer 64 \
|
| 32 |
+
--mode $M --seed $S --pe ape --hard $SCR/pcmd/data/sudoku_hard --augment \
|
| 33 |
+
--d 128 --layers 1 --heads 4 --bs 256 --steps 120000 --lr 3e-4 --warmup 3000 \
|
| 34 |
+
--K 8 --lam 1.0 --eval_bs 250 --ckpt_every 10000 \
|
| 35 |
+
--out $SCR/pcmd/runs/$NAME
|
slurm/rsw.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J rsweep
|
| 3 |
+
#SBATCH -p gpu-h100
|
| 4 |
+
#SBATCH --gpus=h100:1
|
| 5 |
+
#SBATCH -c 8
|
| 6 |
+
#SBATCH --mem=48g
|
| 7 |
+
#SBATCH -t 2:00:00
|
| 8 |
+
#SBATCH -o logs/rsweep-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u sudoku_rsweep.py --hard $SCR/pcmd/data/sudoku_hard \
|
| 15 |
+
--runs $SCR/pcmd/runs --pattern 'rec1-*' --out $SCR/pcmd/figures/sudoku_rsweep.json
|
slurm/sud1.sh
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Sudoku on the SATNet benchmark (Wang et al., ICML 2019), 9000/1000 split,
|
| 3 |
+
# scored by BOARD-level accuracy - all 81 cells correct - which is the published
|
| 4 |
+
# metric. SATNet reports 98.3%.
|
| 5 |
+
#
|
| 6 |
+
# The registered prediction: Sudoku has a unique answer, so TC = 0 and there is
|
| 7 |
+
# no information-theoretic obstacle at all; the whole difficulty is mode (i) plus
|
| 8 |
+
# the ORDER of commitment. An entropy-budgeted decoder commits the forced cells
|
| 9 |
+
# first, recomputes and repeats, which IS constraint propagation. A fixed-K
|
| 10 |
+
# decoder commits a fixed share per pass regardless of certainty. The gap between
|
| 11 |
+
# them should be the largest in this project.
|
| 12 |
+
#
|
| 13 |
+
# Augmentation is by digit relabelling, a genuine Sudoku symmetry that leaves the
|
| 14 |
+
# constraint structure untouched. Both arms are run because SATNet did not
|
| 15 |
+
# augment, so the un-augmented number is the like-for-like one.
|
| 16 |
+
#SBATCH -J sud1
|
| 17 |
+
#SBATCH -p gpu-h100
|
| 18 |
+
#SBATCH --gpus=h100:1
|
| 19 |
+
#SBATCH -c 8
|
| 20 |
+
#SBATCH --mem=48g
|
| 21 |
+
#SBATCH -t 4:00:00
|
| 22 |
+
#SBATCH -o logs/sud1-%A_%a.out
|
| 23 |
+
#SBATCH --array=0-7%4
|
| 24 |
+
source /etc/profile
|
| 25 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 26 |
+
module load python/3.10.10
|
| 27 |
+
export PYTHONPATH=
|
| 28 |
+
cd $SCR/pcmd/src
|
| 29 |
+
MODES=(mdlm full); AUG=(plain aug); SEEDS=(0 1)
|
| 30 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 31 |
+
M=${MODES[$((I / 4))]}; A=${AUG[$(((I % 4) / 2))]}; S=${SEEDS[$((I % 2))]}
|
| 32 |
+
FLAG=""; [ "$A" = "aug" ] && FLAG="--augment"
|
| 33 |
+
NAME="sud1-$M-$A-s$S"
|
| 34 |
+
echo "=== $NAME ==="
|
| 35 |
+
$SCR/hf_env/bin/python -u train_sudoku.py --mode $M $FLAG --seed $S --pe ape \
|
| 36 |
+
--root $SCR/pcmd/data/sudoku --d 512 --layers 10 --heads 8 --bs 128 \
|
| 37 |
+
--steps 60000 --lr 1e-4 --warmup 2000 --K 8 --lam 1.0 --eval_bs 200 \
|
| 38 |
+
--out $SCR/pcmd/runs/$NAME
|
slurm/sudcompare.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J sudcmp
|
| 3 |
+
#SBATCH -p gpu-h100
|
| 4 |
+
#SBATCH --gpus=h100:1
|
| 5 |
+
#SBATCH -c 16
|
| 6 |
+
#SBATCH --mem=48g
|
| 7 |
+
#SBATCH -t 2:00:00
|
| 8 |
+
#SBATCH -o logs/sudcmp-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u sudoku_compare.py --hard $SCR/pcmd/data/sudoku_hard \
|
| 15 |
+
--runs $SCR/pcmd/runs --out $SCR/pcmd/figures/sudoku_compare.json
|
slurm/sudhard.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J sudhard
|
| 3 |
+
#SBATCH -p gpu-h100
|
| 4 |
+
#SBATCH --gpus=h100:1
|
| 5 |
+
#SBATCH -c 8
|
| 6 |
+
#SBATCH --mem=48g
|
| 7 |
+
#SBATCH -t 1:30:00
|
| 8 |
+
#SBATCH -o logs/sudhard-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u sudoku_hard.py --runs $SCR/pcmd/runs --figs $SCR/pcmd/figures \
|
| 15 |
+
--out $SCR/pcmd/figures/sudoku_hard.json
|
slurm/sudhard_train.sh
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
# Hard Sudoku: minimal puzzles at ~24 clues, where search-free constraint
|
| 3 |
+
# propagation solves about 4%. Unlike the SATNet split - which a classical
|
| 4 |
+
# solver saturates at 100% - this discriminates.
|
| 5 |
+
#
|
| 6 |
+
# Every puzzle was verified to have exactly one solution at generation time, and
|
| 7 |
+
# train/test solutions are disjoint even up to digit relabelling (an earlier
|
| 8 |
+
# generator leaked 767/1000 and the assertion caught it).
|
| 9 |
+
#
|
| 10 |
+
# Baselines to beat, measured on this exact test set:
|
| 11 |
+
# constraint propagation, no search : ~4%
|
| 12 |
+
# backtracking search : 100%, but it is search
|
| 13 |
+
# The question is what a masked diffusion model reaches in a fixed, small number
|
| 14 |
+
# of forward passes.
|
| 15 |
+
#SBATCH -J sudhard
|
| 16 |
+
#SBATCH -p gpu-h100
|
| 17 |
+
#SBATCH --gpus=h100:1
|
| 18 |
+
#SBATCH -c 8
|
| 19 |
+
#SBATCH --mem=48g
|
| 20 |
+
#SBATCH -t 4:00:00
|
| 21 |
+
#SBATCH -o logs/sudhard-%A_%a.out
|
| 22 |
+
#SBATCH --array=0-3%4
|
| 23 |
+
source /etc/profile
|
| 24 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 25 |
+
module load python/3.10.10
|
| 26 |
+
export PYTHONPATH=
|
| 27 |
+
cd $SCR/pcmd/src
|
| 28 |
+
MODES=(mdlm full); SEEDS=(0 1)
|
| 29 |
+
I=$SLURM_ARRAY_TASK_ID
|
| 30 |
+
M=${MODES[$((I / 2))]}; S=${SEEDS[$((I % 2))]}
|
| 31 |
+
NAME="sudh-$M-s$S"
|
| 32 |
+
echo "=== $NAME ==="
|
| 33 |
+
$SCR/hf_env/bin/python -u train_sudoku.py --mode $M --seed $S --pe ape \
|
| 34 |
+
--hard $SCR/pcmd/data/sudoku_hard --augment \
|
| 35 |
+
--d 512 --layers 12 --heads 8 --bs 256 --steps 150000 --lr 1e-4 --warmup 4000 \
|
| 36 |
+
--K 8 --lam 1.0 --eval_bs 200 --ckpt_every 10000 \
|
| 37 |
+
--out $SCR/pcmd/runs/$NAME
|
slurm/sudremask.sh
ADDED
|
@@ -0,0 +1,15 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/bash
|
| 2 |
+
#SBATCH -J sudrm
|
| 3 |
+
#SBATCH -p gpu-h100
|
| 4 |
+
#SBATCH --gpus=h100:1
|
| 5 |
+
#SBATCH -c 8
|
| 6 |
+
#SBATCH --mem=48g
|
| 7 |
+
#SBATCH -t 2:00:00
|
| 8 |
+
#SBATCH -o logs/sudrm-%j.out
|
| 9 |
+
source /etc/profile
|
| 10 |
+
SCR=/scratch/zt1/project/msml612/user/$USER
|
| 11 |
+
module load python/3.10.10
|
| 12 |
+
export PYTHONPATH=
|
| 13 |
+
cd $SCR/pcmd/src
|
| 14 |
+
$SCR/hf_env/bin/python -u sudoku_remask.py --hard $SCR/pcmd/data/sudoku_hard \
|
| 15 |
+
--runs $SCR/pcmd/runs --pattern 'sudh-*' --out $SCR/pcmd/figures/sudoku_remask.json
|
src/bench.py
ADDED
|
@@ -0,0 +1,38 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Throughput of the ACTUAL Sudoku model on whatever GPU this job got.
|
| 2 |
+
|
| 3 |
+
The class allocation bills an H100 at 144/min and a 5GB MIG slice at 9/min - 16x
|
| 4 |
+
apart. That only matters if the MIG slice is less than 16x slower on OUR
|
| 5 |
+
workload, which is 81-token sequences and a ~32M-parameter model: a size where an
|
| 6 |
+
H100 is largely idle. Measured rather than assumed.
|
| 7 |
+
"""
|
| 8 |
+
import os, sys, time, torch
|
| 9 |
+
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
| 10 |
+
from model import Transformer
|
| 11 |
+
import diffusion as dfn
|
| 12 |
+
from sudoku import CELLS, SudokuTokenizer
|
| 13 |
+
|
| 14 |
+
dev = "cuda"
|
| 15 |
+
tok = SudokuTokenizer()
|
| 16 |
+
m = Transformer(len(tok), 512, 10, 8, "ape", causal=False, max_len=CELLS + 8).to(dev)
|
| 17 |
+
opt = torch.optim.AdamW(m.parameters(), lr=1e-4)
|
| 18 |
+
x = torch.randint(0, 9, (128, CELLS), device=dev)
|
| 19 |
+
a = torch.rand(128, CELLS, device=dev) < 0.55
|
| 20 |
+
|
| 21 |
+
for _ in range(10): # warm up
|
| 22 |
+
with torch.autocast("cuda", dtype=torch.bfloat16):
|
| 23 |
+
loss = dfn.pmd_loss(m, x, a, tok, "full", 8, 1.0)
|
| 24 |
+
opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
|
| 25 |
+
torch.cuda.synchronize()
|
| 26 |
+
|
| 27 |
+
t0 = time.time()
|
| 28 |
+
N = 100
|
| 29 |
+
for _ in range(N):
|
| 30 |
+
with torch.autocast("cuda", dtype=torch.bfloat16):
|
| 31 |
+
loss = dfn.pmd_loss(m, x, a, tok, "full", 8, 1.0)
|
| 32 |
+
opt.zero_grad(set_to_none=True); loss.backward(); opt.step()
|
| 33 |
+
torch.cuda.synchronize()
|
| 34 |
+
el = time.time() - t0
|
| 35 |
+
name = torch.cuda.get_device_name(0)
|
| 36 |
+
print(f"GPU={name}")
|
| 37 |
+
print(f" {N/el:.1f} steps/sec ({el/N*1000:.1f} ms/step)")
|
| 38 |
+
print(f" 60k steps would take {60000*el/N/60:.1f} min")
|
src/diffusion.py
CHANGED
|
@@ -808,3 +808,117 @@ def cond_decode_budget(model, x0, amask, tok, steps, device, condition=True):
|
|
| 808 |
if rem.any() and pred is not None:
|
| 809 |
x[rem] = pred[rem]
|
| 810 |
return x
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 808 |
if rem.any() and pred is not None:
|
| 809 |
x[rem] = pred[rem]
|
| 810 |
return x
|
| 811 |
+
|
| 812 |
+
|
| 813 |
+
# --------------------------------------------- verifier-guided remasking ---
|
| 814 |
+
# Masked diffusion decoding is ABSORBING: once a position is unmasked it can
|
| 815 |
+
# never be revised. On a constraint problem that is fatal - three wrong cells out
|
| 816 |
+
# of fifty-seven destroy a whole board, which is exactly the observed failure
|
| 817 |
+
# mode (94.9% of cells correct but only 87.9% of boards).
|
| 818 |
+
#
|
| 819 |
+
# When the task has a checkable constraint, the repair signal is exact rather
|
| 820 |
+
# than heuristic: we know precisely which positions conflict. Remask only those,
|
| 821 |
+
# re-decode them against the surviving context, and repeat. This is the diffusion
|
| 822 |
+
# analogue of the propagate-detect-backtrack loop every classical solver runs,
|
| 823 |
+
# and unlike a confidence heuristic it never remasks a position that is provably
|
| 824 |
+
# fine.
|
| 825 |
+
#
|
| 826 |
+
# The per-position remask counter is what keeps it terminating: a cell that has
|
| 827 |
+
# already been reopened `max_revisits` times is frozen, so two mutually
|
| 828 |
+
# inconsistent cells cannot flip each other forever.
|
| 829 |
+
|
| 830 |
+
@torch.no_grad()
|
| 831 |
+
def verifier_remask_decode(model, x, tok, device, violation_fn, base_steps=16,
|
| 832 |
+
rounds=8, max_revisits=3, fillable=None,
|
| 833 |
+
expand_peers=None, allowed=None, temperature=1.0):
|
| 834 |
+
"""Decode, then repeatedly remask whatever a verifier says is wrong.
|
| 835 |
+
|
| 836 |
+
violation_fn(x) -> bool mask of positions involved in a violation, where x
|
| 837 |
+
holds the current token ids. `expand_peers`, if given, maps a violation mask
|
| 838 |
+
to a wider mask (for Sudoku: the conflicting cells plus their units), which
|
| 839 |
+
gives the model room to re-solve the local subproblem rather than re-guess a
|
| 840 |
+
single cell in an unchanged context.
|
| 841 |
+
|
| 842 |
+
Returns (x, nfe) with nfe counting every forward pass, so the cost of the
|
| 843 |
+
repair rounds is charged honestly against the fixed-K budgets it is compared
|
| 844 |
+
with.
|
| 845 |
+
"""
|
| 846 |
+
x = x.clone().to(device)
|
| 847 |
+
can = (x == tok.mask) if fillable is None else fillable.to(device)
|
| 848 |
+
B, L = x.shape
|
| 849 |
+
|
| 850 |
+
x, nfe = fixed_k_decode(model, x, tok, device, base_steps, fillable=can,
|
| 851 |
+
allowed=allowed, temperature=temperature)
|
| 852 |
+
revisits = torch.zeros((B, L), dtype=torch.long, device=device)
|
| 853 |
+
|
| 854 |
+
for _ in range(rounds):
|
| 855 |
+
bad = violation_fn(x).to(device) & can
|
| 856 |
+
if expand_peers is not None:
|
| 857 |
+
bad = expand_peers(bad) & can
|
| 858 |
+
# a position already reopened too often is frozen, or two conflicting
|
| 859 |
+
# cells would keep overwriting each other indefinitely
|
| 860 |
+
bad = bad & (revisits < max_revisits)
|
| 861 |
+
if not bad.any():
|
| 862 |
+
break
|
| 863 |
+
revisits = revisits + bad.long()
|
| 864 |
+
x = torch.where(bad, torch.full_like(x, tok.mask), x)
|
| 865 |
+
# re-decode only the reopened positions, against everything still held
|
| 866 |
+
steps = max(1, min(base_steps, int(bad.sum(1).max())))
|
| 867 |
+
x, n2 = fixed_k_decode(model, x, tok, device, steps, fillable=bad,
|
| 868 |
+
allowed=allowed, temperature=temperature)
|
| 869 |
+
nfe += n2
|
| 870 |
+
return x, nfe
|
| 871 |
+
|
| 872 |
+
|
| 873 |
+
def recurrent_cond_loss(model, x0, amask, tok, mode="mdlm", K=8, lam=1.0,
|
| 874 |
+
recurrences=None, deep_sup=True):
|
| 875 |
+
"""Conditional masked-diffusion loss with DEEP SUPERVISION over recurrences.
|
| 876 |
+
|
| 877 |
+
Loss is taken at every recurrence rather than only the last. That is the
|
| 878 |
+
ingredient Yang et al. (2023) identify as critical for constraint
|
| 879 |
+
satisfaction: it forces every application of the shared block to be a valid
|
| 880 |
+
one-step refinement, which in turn is what lets inference run more
|
| 881 |
+
recurrences than training.
|
| 882 |
+
|
| 883 |
+
Later recurrences are weighted more heavily - a linear ramp - so the final
|
| 884 |
+
output still receives the strongest signal while the early ones are kept
|
| 885 |
+
honest.
|
| 886 |
+
"""
|
| 887 |
+
B, L = x0.shape
|
| 888 |
+
dev = x0.device
|
| 889 |
+
if mode == "matched" or mode == "full":
|
| 890 |
+
t = torch.randint(1, K + 1, (B,), device=dev).float() / K
|
| 891 |
+
else:
|
| 892 |
+
t = torch.rand(B, device=dev).clamp_min(1.0 / L)
|
| 893 |
+
|
| 894 |
+
m = _cond_mask(x0, amask, tok, t)
|
| 895 |
+
xt = torch.where(m, torch.full_like(x0, tok.mask), x0)
|
| 896 |
+
outs = model(xt, recurrences=recurrences, return_all=True)
|
| 897 |
+
|
| 898 |
+
R = len(outs)
|
| 899 |
+
if not deep_sup:
|
| 900 |
+
outs = outs[-1:] # ablation: supervise only the last
|
| 901 |
+
total, wsum = 0.0, 0.0
|
| 902 |
+
for r, logits in enumerate(outs):
|
| 903 |
+
w = (r + 1) / len(outs) if deep_sup else 1.0
|
| 904 |
+
ce = F.cross_entropy(logits.reshape(-1, logits.size(-1)),
|
| 905 |
+
x0.reshape(-1), reduction="none").view(B, L)
|
| 906 |
+
total = total + w * ((ce * m).sum(1) / m.sum(1).clamp_min(1)).mean()
|
| 907 |
+
wsum += w
|
| 908 |
+
loss = total / wsum
|
| 909 |
+
|
| 910 |
+
if mode == "full":
|
| 911 |
+
# the t=1 term, also deep-supervised
|
| 912 |
+
xf = torch.where(amask, torch.full_like(x0, tok.mask), x0)
|
| 913 |
+
outs_f = model(xf, recurrences=recurrences, return_all=True)
|
| 914 |
+
if not deep_sup:
|
| 915 |
+
outs_f = outs_f[-1:]
|
| 916 |
+
tot_f, ws = 0.0, 0.0
|
| 917 |
+
for r, logits in enumerate(outs_f):
|
| 918 |
+
w = (r + 1) / len(outs_f) if deep_sup else 1.0
|
| 919 |
+
ce = F.cross_entropy(logits.reshape(-1, logits.size(-1)),
|
| 920 |
+
x0.reshape(-1), reduction="none").view(B, L)
|
| 921 |
+
tot_f = tot_f + w * ((ce * amask).sum(1) / amask.sum(1).clamp_min(1)).mean()
|
| 922 |
+
ws += w
|
| 923 |
+
loss = loss + lam * tot_f / ws
|
| 924 |
+
return loss
|
src/gen_hard_sudoku.py
ADDED
|
@@ -0,0 +1,200 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Generate a HARD Sudoku benchmark, with every puzzle verified unique.
|
| 2 |
+
|
| 3 |
+
The SATNet split is saturated - search-free constraint propagation solves
|
| 4 |
+
1000/1000 of it - so nothing measured there discriminates between methods. This
|
| 5 |
+
builds instances that do.
|
| 6 |
+
|
| 7 |
+
Each puzzle is MINIMAL: cells are removed from a completed grid in random order
|
| 8 |
+
and a removal is kept only if the solution is still unique, verified by an
|
| 9 |
+
exhaustive counter that stops at two. The result carries roughly 19-30 clues
|
| 10 |
+
(mean 24), where constraint propagation alone solves about 4%. No puzzle is accepted without
|
| 11 |
+
that uniqueness check, so board accuracy remains a well-defined quantity.
|
| 12 |
+
|
| 13 |
+
Solutions come from a randomised backtracking fill of an EMPTY grid, which
|
| 14 |
+
reaches the full space of roughly 6.7e21 valid grids.
|
| 15 |
+
|
| 16 |
+
An earlier version applied the Sudoku symmetry group to a single seed grid
|
| 17 |
+
instead. That was much faster and completely wrong: every grid it produced lay
|
| 18 |
+
in one equivalence class, whose size modulo digit relabelling is only about 3.4
|
| 19 |
+
million, so 101,000 draws collided constantly and 767 of 1000 test solutions
|
| 20 |
+
already appeared in training. The assertion below caught it. It is kept, and it
|
| 21 |
+
is the reason the slower fill is used.
|
| 22 |
+
|
| 23 |
+
Train and test are built from DISJOINT solution grids and the script asserts
|
| 24 |
+
that no test solution appears in training, in any relabelling.
|
| 25 |
+
"""
|
| 26 |
+
import argparse, os, sys, time
|
| 27 |
+
from multiprocessing import Pool
|
| 28 |
+
|
| 29 |
+
import numpy as np
|
| 30 |
+
|
| 31 |
+
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
| 32 |
+
from sudoku import CELLS, GRID
|
| 33 |
+
|
| 34 |
+
PEERS = None
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
def _peers():
|
| 38 |
+
out = []
|
| 39 |
+
for i in range(CELLS):
|
| 40 |
+
r, c = divmod(i, GRID)
|
| 41 |
+
br, bc = 3 * (r // 3), 3 * (c // 3)
|
| 42 |
+
s = set()
|
| 43 |
+
for k in range(GRID):
|
| 44 |
+
s.add(r * GRID + k)
|
| 45 |
+
s.add(k * GRID + c)
|
| 46 |
+
for dr in range(3):
|
| 47 |
+
for dc in range(3):
|
| 48 |
+
s.add((br + dr) * GRID + bc + dc)
|
| 49 |
+
s.discard(i)
|
| 50 |
+
out.append(sorted(s))
|
| 51 |
+
return out
|
| 52 |
+
|
| 53 |
+
|
| 54 |
+
PEERS = _peers()
|
| 55 |
+
|
| 56 |
+
|
| 57 |
+
|
| 58 |
+
def candidates(g):
|
| 59 |
+
cand = [0x1FF if v == 0 else (1 << (v - 1)) for v in g]
|
| 60 |
+
for i, v in enumerate(g):
|
| 61 |
+
if v:
|
| 62 |
+
bit = 1 << (v - 1)
|
| 63 |
+
for p in PEERS[i]:
|
| 64 |
+
if cand[p] & bit:
|
| 65 |
+
cand[p] &= ~bit
|
| 66 |
+
if cand[p] == 0:
|
| 67 |
+
return None
|
| 68 |
+
return cand
|
| 69 |
+
|
| 70 |
+
|
| 71 |
+
def count_solutions(grid, limit=2):
|
| 72 |
+
g = [int(v) for v in grid]
|
| 73 |
+
|
| 74 |
+
def rec():
|
| 75 |
+
cand = candidates(g)
|
| 76 |
+
if cand is None:
|
| 77 |
+
return 0
|
| 78 |
+
best, bi = 10, -1
|
| 79 |
+
for i in range(CELLS):
|
| 80 |
+
if g[i] == 0:
|
| 81 |
+
n = bin(cand[i]).count("1")
|
| 82 |
+
if n < best:
|
| 83 |
+
best, bi = n, i
|
| 84 |
+
if n == 1:
|
| 85 |
+
break
|
| 86 |
+
if bi < 0:
|
| 87 |
+
return 1
|
| 88 |
+
tot = 0
|
| 89 |
+
for d in range(9):
|
| 90 |
+
if cand[bi] & (1 << d):
|
| 91 |
+
g[bi] = d + 1
|
| 92 |
+
tot += rec()
|
| 93 |
+
g[bi] = 0
|
| 94 |
+
if tot >= limit:
|
| 95 |
+
return tot
|
| 96 |
+
return tot
|
| 97 |
+
|
| 98 |
+
return rec()
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
def random_solution(rng):
|
| 102 |
+
"""A uniformly-ish random completed grid, by randomised backtracking fill.
|
| 103 |
+
|
| 104 |
+
The previous version applied the Sudoku symmetry group to a single seed grid.
|
| 105 |
+
Every grid it produced therefore lay in ONE equivalence class, whose size
|
| 106 |
+
modulo digit relabelling is only about 3.4 million - so drawing 101,000
|
| 107 |
+
samples collided constantly, and 767 of 1000 test solutions turned out to
|
| 108 |
+
already appear in training. Filling an empty grid instead reaches the full
|
| 109 |
+
space of roughly 6.7e21 valid grids, where collisions are impossible in
|
| 110 |
+
practice.
|
| 111 |
+
"""
|
| 112 |
+
g = [0] * CELLS
|
| 113 |
+
while True:
|
| 114 |
+
cand = candidates(g)
|
| 115 |
+
if cand is None:
|
| 116 |
+
return None
|
| 117 |
+
best, bi = 10, -1
|
| 118 |
+
for i in range(CELLS):
|
| 119 |
+
if g[i] == 0:
|
| 120 |
+
n = bin(cand[i]).count("1")
|
| 121 |
+
if n < best:
|
| 122 |
+
best, bi = n, i
|
| 123 |
+
if bi < 0:
|
| 124 |
+
return np.array(g, dtype=np.int64)
|
| 125 |
+
digits = [d + 1 for d in range(9) if cand[bi] & (1 << d)]
|
| 126 |
+
rng.shuffle(digits)
|
| 127 |
+
placed = False
|
| 128 |
+
for d in digits:
|
| 129 |
+
g[bi] = d
|
| 130 |
+
if candidates(g) is not None:
|
| 131 |
+
placed = True
|
| 132 |
+
break
|
| 133 |
+
g[bi] = 0
|
| 134 |
+
if not placed:
|
| 135 |
+
# dead end: restart rather than backtrack; restarts are rare and a
|
| 136 |
+
# fresh fill is cheaper than unwinding a deep search
|
| 137 |
+
g = [0] * CELLS
|
| 138 |
+
|
| 139 |
+
|
| 140 |
+
def make_one(seed):
|
| 141 |
+
rng = np.random.default_rng(seed)
|
| 142 |
+
sol = None
|
| 143 |
+
while sol is None:
|
| 144 |
+
sol = random_solution(rng)
|
| 145 |
+
g = [int(v) for v in sol]
|
| 146 |
+
for c in rng.permutation(CELLS):
|
| 147 |
+
keep = g[c]
|
| 148 |
+
g[c] = 0
|
| 149 |
+
if count_solutions(g, 2) != 1:
|
| 150 |
+
g[c] = keep
|
| 151 |
+
return np.array(g, dtype=np.int64), sol
|
| 152 |
+
|
| 153 |
+
|
| 154 |
+
def canon(g):
|
| 155 |
+
m, out = {}, []
|
| 156 |
+
for v in g:
|
| 157 |
+
if v not in m:
|
| 158 |
+
m[v] = len(m) + 1
|
| 159 |
+
out.append(m[v])
|
| 160 |
+
return bytes(out)
|
| 161 |
+
|
| 162 |
+
|
| 163 |
+
if __name__ == "__main__":
|
| 164 |
+
p = argparse.ArgumentParser()
|
| 165 |
+
p.add_argument("--n_train", type=int, default=100000)
|
| 166 |
+
p.add_argument("--n_test", type=int, default=1000)
|
| 167 |
+
p.add_argument("--workers", type=int, default=64)
|
| 168 |
+
p.add_argument("--out", default="data/sudoku_hard")
|
| 169 |
+
a = p.parse_args()
|
| 170 |
+
os.makedirs(a.out, exist_ok=True)
|
| 171 |
+
|
| 172 |
+
total = a.n_train + a.n_test
|
| 173 |
+
t0 = time.time()
|
| 174 |
+
with Pool(a.workers) as pool:
|
| 175 |
+
got = pool.map(make_one, range(total), chunksize=32)
|
| 176 |
+
P = np.stack([g for g, _ in got])
|
| 177 |
+
S = np.stack([s for _, s in got])
|
| 178 |
+
print(f"generated {len(P)} minimal puzzles in {time.time()-t0:.0f}s", flush=True)
|
| 179 |
+
print(f"clues: min={(P>0).sum(1).min()} max={(P>0).sum(1).max()} "
|
| 180 |
+
f"mean={(P>0).sum(1).mean():.1f}", flush=True)
|
| 181 |
+
|
| 182 |
+
Ptr, Str = P[:a.n_train], S[:a.n_train]
|
| 183 |
+
Pte, Ste = P[a.n_train:], S[a.n_train:]
|
| 184 |
+
|
| 185 |
+
# a test solution must not appear in training, even up to digit relabelling
|
| 186 |
+
tr = {canon(r) for r in Str}
|
| 187 |
+
leak = sum(canon(r) in tr for r in Ste)
|
| 188 |
+
print(f"test solutions also in training (up to relabelling): {leak}", flush=True)
|
| 189 |
+
assert leak == 0, "leakage between train and test"
|
| 190 |
+
|
| 191 |
+
# every puzzle must still admit exactly one solution
|
| 192 |
+
bad = sum(count_solutions(Pte[i], 2) != 1 for i in range(0, len(Pte), 25))
|
| 193 |
+
print(f"test puzzles without a unique solution (sampled): {bad}", flush=True)
|
| 194 |
+
assert bad == 0
|
| 195 |
+
|
| 196 |
+
np.save(os.path.join(a.out, "train_puz.npy"), Ptr)
|
| 197 |
+
np.save(os.path.join(a.out, "train_sol.npy"), Str)
|
| 198 |
+
np.save(os.path.join(a.out, "test_puz.npy"), Pte)
|
| 199 |
+
np.save(os.path.join(a.out, "test_sol.npy"), Ste)
|
| 200 |
+
print(f"saved to {a.out}: train {Ptr.shape} test {Pte.shape}", flush=True)
|
src/model.py
CHANGED
|
@@ -162,3 +162,74 @@ class Transformer(nn.Module):
|
|
| 162 |
|
| 163 |
def n_params(self):
|
| 164 |
return sum(p.numel() for p in self.parameters() if p.requires_grad)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 162 |
|
| 163 |
def n_params(self):
|
| 164 |
return sum(p.numel() for p in self.parameters() if p.requires_grad)
|
| 165 |
+
|
| 166 |
+
|
| 167 |
+
class RecurrentDenoiser(nn.Module):
|
| 168 |
+
"""A weight-shared block applied R times, supervised at every step.
|
| 169 |
+
|
| 170 |
+
Motivated by a measurement, not a hunch. On hard Sudoku a 37.86M-parameter
|
| 171 |
+
12-layer denoiser reaches 89.4%, while the Recurrent Transformer of Yang et
|
| 172 |
+
al. (2023) reaches 99.5% on a comparable split with 211k parameters - 180x
|
| 173 |
+
smaller. The difference is not capacity, it is that constraint propagation is
|
| 174 |
+
an ITERATIVE algorithm: solving a hard Sudoku takes tens of rounds of
|
| 175 |
+
"eliminate, propagate, repeat". A fixed 12-layer feedforward network cannot
|
| 176 |
+
express thirty rounds of propagation at any width; one block applied thirty
|
| 177 |
+
times can.
|
| 178 |
+
|
| 179 |
+
This is the same argument THEORY.md makes for addition - that a K-pass decode
|
| 180 |
+
has effective depth L*K and "borrows depth from the decoding loop" - applied
|
| 181 |
+
to the architecture instead of to the decoder.
|
| 182 |
+
|
| 183 |
+
Two details carry the method:
|
| 184 |
+
|
| 185 |
+
INPUT INJECTION. The token embedding is re-added at every recurrence.
|
| 186 |
+
Without it the input signal decays through thirty applications of the same
|
| 187 |
+
block and the model forgets the clues it is solving for.
|
| 188 |
+
|
| 189 |
+
DEEP SUPERVISION. Training loss is taken at EVERY recurrence, not only the
|
| 190 |
+
last. That forces each application to make progress on its own rather than
|
| 191 |
+
letting the stack learn one entangled thirty-step function, and it is what
|
| 192 |
+
lets inference run MORE recurrences than training - the per-step objective
|
| 193 |
+
is the same at every step, so the map is iterable beyond where it was fit.
|
| 194 |
+
"""
|
| 195 |
+
|
| 196 |
+
def __init__(self, vocab, d=128, n_layers=1, n_heads=4, pe="ape",
|
| 197 |
+
max_len=128, recurrences=32, hidden_mult=4):
|
| 198 |
+
super().__init__()
|
| 199 |
+
self.pe_kind, self.causal = pe, False
|
| 200 |
+
self.recurrences = recurrences
|
| 201 |
+
self.emb = nn.Embedding(vocab, d)
|
| 202 |
+
if pe == "ape":
|
| 203 |
+
self.pos = nn.Embedding(max_len, d)
|
| 204 |
+
elif pe == "sin":
|
| 205 |
+
self.register_buffer("pos_sin", sinusoidal(max_len, d), persistent=False)
|
| 206 |
+
self.blocks = nn.ModuleList([Block(d, n_heads, pe, False, max_len)
|
| 207 |
+
for _ in range(n_layers)])
|
| 208 |
+
self.norm = nn.LayerNorm(d)
|
| 209 |
+
self.head = nn.Linear(d, vocab, bias=False)
|
| 210 |
+
self.apply(Transformer._init)
|
| 211 |
+
|
| 212 |
+
def _inject(self, idx):
|
| 213 |
+
x = self.emb(idx)
|
| 214 |
+
if self.pe_kind == "ape":
|
| 215 |
+
x = x + self.pos(torch.arange(idx.shape[1], device=idx.device))[None]
|
| 216 |
+
elif self.pe_kind == "sin":
|
| 217 |
+
x = x + self.pos_sin[torch.arange(idx.shape[1], device=idx.device)][None]
|
| 218 |
+
return x
|
| 219 |
+
|
| 220 |
+
def forward(self, idx, pad_mask=None, pos_ids=None, budget=None,
|
| 221 |
+
recurrences=None, return_all=False):
|
| 222 |
+
"""Returns final logits, or the list of logits from every recurrence."""
|
| 223 |
+
R = recurrences or self.recurrences
|
| 224 |
+
inp = self._inject(idx)
|
| 225 |
+
h = inp
|
| 226 |
+
outs = []
|
| 227 |
+
for _ in range(R):
|
| 228 |
+
h = h + inp # input injection: keep the clues alive
|
| 229 |
+
for b in self.blocks:
|
| 230 |
+
h = b(h, pad_mask, pos_ids)
|
| 231 |
+
outs.append(self.head(self.norm(h)))
|
| 232 |
+
return outs if return_all else outs[-1]
|
| 233 |
+
|
| 234 |
+
def n_params(self):
|
| 235 |
+
return sum(p.numel() for p in self.parameters() if p.requires_grad)
|
src/sudoku.py
CHANGED
|
@@ -142,3 +142,53 @@ if __name__ == "__main__":
|
|
| 142 |
tr = {bytes(r) for r in Xtr[::3]}
|
| 143 |
ov = sum(bytes(r) in tr for r in Xte)
|
| 144 |
print(f"test puzzles also seen in the training sample: {ov} (must be 0)")
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 142 |
tr = {bytes(r) for r in Xtr[::3]}
|
| 143 |
ov = sum(bytes(r) in tr for r in Xte)
|
| 144 |
print(f"test puzzles also seen in the training sample: {ov} (must be 0)")
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
def _unit_list():
|
| 148 |
+
us = []
|
| 149 |
+
for r in range(GRID):
|
| 150 |
+
us.append([r * GRID + c for c in range(GRID)])
|
| 151 |
+
for c in range(GRID):
|
| 152 |
+
us.append([r * GRID + c for r in range(GRID)])
|
| 153 |
+
for br in range(0, GRID, 3):
|
| 154 |
+
for bc in range(0, GRID, 3):
|
| 155 |
+
us.append([(br + dr) * GRID + bc + dc for dr in range(3) for dc in range(3)])
|
| 156 |
+
return us
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
UNIT_LIST = _unit_list()
|
| 160 |
+
|
| 161 |
+
|
| 162 |
+
def violation_mask(x):
|
| 163 |
+
"""Positions involved in a row/column/box duplicate. Exact, not heuristic.
|
| 164 |
+
|
| 165 |
+
x is (B, 81) of token ids 0-8. A cell is flagged when its value appears more
|
| 166 |
+
than once in any unit it belongs to. A solved grid flags nothing, so this
|
| 167 |
+
doubles as the stopping condition for verifier-guided remasking.
|
| 168 |
+
"""
|
| 169 |
+
import torch
|
| 170 |
+
B = x.shape[0]
|
| 171 |
+
bad = torch.zeros_like(x, dtype=torch.bool)
|
| 172 |
+
for unit in UNIT_LIST:
|
| 173 |
+
idx = torch.tensor(unit, device=x.device)
|
| 174 |
+
vals = x[:, idx] # (B, 9)
|
| 175 |
+
same = vals[:, :, None] == vals[:, None, :] # (B, 9, 9)
|
| 176 |
+
dup = same.sum(-1) > 1 # appears more than once
|
| 177 |
+
bad[:, idx] |= dup
|
| 178 |
+
return bad
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
def expand_to_units(bad):
|
| 182 |
+
"""Widen a violation mask to every cell sharing a unit with a flagged cell.
|
| 183 |
+
|
| 184 |
+
Re-deciding a single conflicting cell in an otherwise unchanged context tends
|
| 185 |
+
to reproduce the same value. Reopening its row, column and box gives the
|
| 186 |
+
model a subproblem it can actually re-solve.
|
| 187 |
+
"""
|
| 188 |
+
import torch
|
| 189 |
+
out = torch.zeros_like(bad)
|
| 190 |
+
for unit in UNIT_LIST:
|
| 191 |
+
idx = torch.tensor(unit, device=bad.device)
|
| 192 |
+
hit = bad[:, idx].any(1, keepdim=True) # (B, 1)
|
| 193 |
+
out[:, idx] |= hit
|
| 194 |
+
return out
|