Cukinator commited on
Commit
5315130
·
verified ·
1 Parent(s): 07898d2

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +28 -6
README.md CHANGED
@@ -20,17 +20,39 @@ Each run is stored in its own folder and contains a single `model.pt` file. Thes
20
 
21
  ## Repository contents
22
 
23
- Models from the ablation grid include:
24
 
 
25
  | Path | Description |
26
  |---|---|
27
- | `run_01/model.pt` | Transformer BPE FP16 (Baseline) |
28
- | `run_02/model.pt` | Transformer Byte FP16 (Best performer) |
 
29
  | `run_03/model.pt` | MLGRU Byte FP16 |
 
 
 
 
 
 
 
 
30
  | `run_07/model.pt` | CPU-1 COMPLETE (V3 Ternary with DeleteGate) |
31
- | `run_09/model.pt` | CPU-1 + PFNet |
32
- | `run_14_r2/model.pt` | Small 10M Byte (Round 2) |
33
- | ... and all other runs | (Check the folder structure for the full list) |
 
 
 
 
 
 
 
 
 
 
 
 
34
 
35
  ## How to test them locally
36
 
 
20
 
21
  ## Repository contents
22
 
23
+ Models from the ablation grid include all trained variants across Round 1 and Round 2:
24
 
25
+ ### FP16 Baselines
26
  | Path | Description |
27
  |---|---|
28
+ | `run_01/model.pt` | Transformer BPE (16K vocab) FP16 |
29
+ | `run_02a_byte_only_heads/model.pt` | Transformer Byte FP16 (Independent heads) |
30
+ | `run_02/model.pt` | Transformer Byte FP16 (LocalByteDecoder) — **Best Performer** |
31
  | `run_03/model.pt` | MLGRU Byte FP16 |
32
+
33
+ ### 50M Ternary Chain (V3 and Ablations)
34
+ | Path | Description |
35
+ |---|---|
36
+ | `run_04/model.pt` | MLGRU Byte Ternary |
37
+ | `run_05/model.pt` | MLGRU Byte Ternary + FPResidual (CPU-1 core) |
38
+ | `run_05b_kernel_strict/model.pt` | CPU-1 core without W_o |
39
+ | `run_06/model.pt` | CPU-1 core + Bolmo (Dynamic patches) |
40
  | `run_07/model.pt` | CPU-1 COMPLETE (V3 Ternary with DeleteGate) |
41
+ | `run_08/model.pt` | Folded Transformer Byte Ternary |
42
+ | `run_09/model.pt` | CPU-1 COMPLETE + PFNet |
43
+ | `run_10/model.pt` | CPU-1 COMPLETE + PFNet + Per-Channel Decay |
44
+
45
+ ### 10M Scale Runs (Ternary)
46
+ | Path | Description |
47
+ |---|---|
48
+ | `run_13/model.pt` | 10M BPE + Ternary (Round 1 - 2 tok/p) |
49
+ | `run_13_r2/model.pt` | 10M BPE + Ternary (Round 2 - 15 tok/p) |
50
+ | `run_14/model.pt` | 10M Byte + Ternary (Round 1 - 2 tok/p) |
51
+ | `run_14_r2/model.pt` | 10M Byte + Ternary (Round 2 - 15 tok/p) |
52
+ | `run_15/model.pt` | 10M Byte + Ternary + EmbeddingAligner (Round 1) |
53
+ | `run_15_r2/model.pt` | 10M Byte + Ternary + EmbeddingAligner (Round 2) |
54
+ | `run_16/model.pt` | 10M Byte + Ternary (No teacher logprobs) (Round 1) |
55
+ | `run_16_r2/model.pt` | 10M Byte + Ternary (No teacher logprobs) (Round 2) |
56
 
57
  ## How to test them locally
58