a3ro-dev commited on
Commit
cf20500
·
verified ·
1 Parent(s): e7b7213

card: diagrams; credit Pranav Ramakrishnan by name

Browse files
Files changed (5) hide show
  1. .gitattributes +3 -0
  2. README.md +7 -1
  3. assets/how.png +3 -0
  4. assets/results.png +3 -0
  5. assets/train.png +3 -0
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ assets/how.png filter=lfs diff=lfs merge=lfs -text
37
+ assets/results.png filter=lfs diff=lfs merge=lfs -text
38
+ assets/train.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -36,8 +36,12 @@ Code, training pipeline and paper: [github.com/a3ro-dev/reflex-2048](https://git
36
 
37
  This is not a record. It's a small, cheap data point.
38
 
 
 
39
  ## Results
40
 
 
 
41
  10,000 fresh games, greedy argmax policy, end-of-schedule checkpoint (not selected by eval score):
42
 
43
  | mean | median | max | 2048 | 4096 | 8192 | 16384 | 32768 | 65536 |
@@ -101,6 +105,8 @@ The checkpoint is a plain `torch.save` dict (`cfg`, `model` state dict, `step`,
101
 
102
  ## Training
103
 
 
 
104
  - **Loss:** cross-entropy to the teacher's move plus a 0.25× Huber loss on log return-to-go. Random D4 symmetry per sample.
105
  - **Data:** 360M depth-3 and 100M depth-4 teacher self-play decisions.
106
  - **Schedule:** 261M samples total. The model starts from a first run's step-6000 checkpoint, taken before any DAgger data reached the trainer. It then trains 212M samples on depth-3/depth-4 data at 50/50 with AdamW, peak lr 5e-4 and a 1.9 h cosine schedule, in bf16 on one A100.
@@ -109,4 +115,4 @@ The teacher is GPL-3.0 code. The repo ships only a patch script for it, not the
109
 
110
  ## Thanks
111
 
112
- The RunPod credits that paid for all the compute came from [pranavram98](https://www.linkedin.com/in/pranavram98/). Thank you.
 
36
 
37
  This is not a record. It's a small, cheap data point.
38
 
39
+ ![one look, one move: a real board from move 1,422 becomes 21 tokens, goes through 8 transformer blocks, and comes out as move probabilities, down 0.77 and left 0.23](assets/how.png)
40
+
41
  ## Results
42
 
43
+ ![how far it gets: share of 10,000 games that build each tile, 2048 96.6%, 4096 86.4%, 8192 63.3%, 16384 30.0%, 32768 3.9%, 65536 0%; mean score 164,395 against 412,785 to 625,898 for other agents](assets/results.png)
44
+
45
  10,000 fresh games, greedy argmax policy, end-of-schedule checkpoint (not selected by eval score):
46
 
47
  | mean | median | max | 2048 | 4096 | 8192 | 16384 | 32768 | 65536 |
 
105
 
106
  ## Training
107
 
108
+ ![how it learned: the teacher plays, 460M of its moves, copy the teacher, reflex.pt; a DAgger detour dropped play from 56,652 to 4,188](assets/train.png)
109
+
110
  - **Loss:** cross-entropy to the teacher's move plus a 0.25× Huber loss on log return-to-go. Random D4 symmetry per sample.
111
  - **Data:** 360M depth-3 and 100M depth-4 teacher self-play decisions.
112
  - **Schedule:** 261M samples total. The model starts from a first run's step-6000 checkpoint, taken before any DAgger data reached the trainer. It then trains 212M samples on depth-3/depth-4 data at 50/50 with AdamW, peak lr 5e-4 and a 1.9 h cosine schedule, in bf16 on one A100.
 
115
 
116
  ## Thanks
117
 
118
+ The RunPod credits that paid for all the compute came from [Pranav Ramakrishnan](https://www.linkedin.com/in/pranavram98/). Thank you.
assets/how.png ADDED

Git LFS Details

  • SHA256: 25f5f366327746123bb72b858eabe800d40dac475cd8c6211f041977172ab605
  • Pointer size: 131 Bytes
  • Size of remote file: 130 kB
assets/results.png ADDED

Git LFS Details

  • SHA256: 9eb3a891d05e24234b2b4ddc31eba607beee4c915f9b8ac82dfe37c34c3cf481
  • Pointer size: 131 Bytes
  • Size of remote file: 140 kB
assets/train.png ADDED

Git LFS Details

  • SHA256: 0124f95bdd4034a99ba206a977eaa9e5a59f718a114f089fdbbb8e2ffac8e538
  • Pointer size: 131 Bytes
  • Size of remote file: 170 kB