a3ro-dev commited on
Commit
0b70b58
·
verified ·
1 Parent(s): cf20500

card: architecture diagrams

Browse files
.gitattributes CHANGED
@@ -36,3 +36,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
36
  assets/how.png filter=lfs diff=lfs merge=lfs -text
37
  assets/results.png filter=lfs diff=lfs merge=lfs -text
38
  assets/train.png filter=lfs diff=lfs merge=lfs -text
 
 
 
36
  assets/how.png filter=lfs diff=lfs merge=lfs -text
37
  assets/results.png filter=lfs diff=lfs merge=lfs -text
38
  assets/train.png filter=lfs diff=lfs merge=lfs -text
39
+ assets/arch.png filter=lfs diff=lfs merge=lfs -text
40
+ assets/moves.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -99,6 +99,10 @@ The checkpoint is a plain `torch.save` dict (`cfg`, `model` state dict, `step`,
99
 
100
  ## Model
101
 
 
 
 
 
102
  - **Tokens:** 16 tile tokens (tile-exponent + row + column embeddings), 4 action tokens built from each move's afterstate (legality, reward, empties, max tile and corner, monotonicity, smoothness, merges, follow-up rewards), and 1 value token.
103
  - **Network:** 8 pre-norm bidirectional transformer blocks, d=384, 12 heads, MLP ratio 4. One logit per action token, with illegal moves masked.
104
  - **Action order:** 0=UP, 1=DOWN, 2=LEFT, 3=RIGHT.
 
99
 
100
  ## Model
101
 
102
+ ![inside the model: 16 tile tokens, 4 move tokens and 1 value token go through 8 pre-norm blocks of 12-head attention and a 384-1536-384 MLP, then a policy head with one logit per move token and a value head used only in training](assets/arch.png)
103
+
104
+ ![what a move token knows: the four afterstates of a real board; up and down both merge the 512s for 1,028 points, only down keeps the 2048 in its corner, and the model puts 0.77 on down](assets/moves.png)
105
+
106
  - **Tokens:** 16 tile tokens (tile-exponent + row + column embeddings), 4 action tokens built from each move's afterstate (legality, reward, empties, max tile and corner, monotonicity, smoothness, merges, follow-up rewards), and 1 value token.
107
  - **Network:** 8 pre-norm bidirectional transformer blocks, d=384, 12 heads, MLP ratio 4. One logit per action token, with illegal moves masked.
108
  - **Action order:** 0=UP, 1=DOWN, 2=LEFT, 3=RIGHT.
assets/arch.png ADDED

Git LFS Details

  • SHA256: ca6b6103a348f5ac12dc6c4092195a51dcf6df9a671927ea478879b386957aca
  • Pointer size: 131 Bytes
  • Size of remote file: 189 kB
assets/how.png CHANGED

Git LFS Details

  • SHA256: 25f5f366327746123bb72b858eabe800d40dac475cd8c6211f041977172ab605
  • Pointer size: 131 Bytes
  • Size of remote file: 130 kB

Git LFS Details

  • SHA256: eec6d962df4218396b924647279685f524b9346e9fe3a6b3e3f2493eee64eb92
  • Pointer size: 131 Bytes
  • Size of remote file: 130 kB
assets/moves.png ADDED

Git LFS Details

  • SHA256: 4ecb77a8e344cd9ed400dca2a630b4afd2d4d051eb66255911be5cbb65c7d76a
  • Pointer size: 131 Bytes
  • Size of remote file: 182 kB
assets/results.png CHANGED

Git LFS Details

  • SHA256: 9eb3a891d05e24234b2b4ddc31eba607beee4c915f9b8ac82dfe37c34c3cf481
  • Pointer size: 131 Bytes
  • Size of remote file: 140 kB

Git LFS Details

  • SHA256: 11f0d070be7fe6542b1082bc6e8c9221f54549f1da8bfa5391b36e67bb59105b
  • Pointer size: 131 Bytes
  • Size of remote file: 140 kB
assets/train.png CHANGED

Git LFS Details

  • SHA256: 0124f95bdd4034a99ba206a977eaa9e5a59f718a114f089fdbbb8e2ffac8e538
  • Pointer size: 131 Bytes
  • Size of remote file: 170 kB

Git LFS Details

  • SHA256: 4d142100dbc10d71fc569aa9b4c8ce0807b64b551fa168bd95d42a5f4f4c4ff0
  • Pointer size: 131 Bytes
  • Size of remote file: 170 kB