Chess Challenge submission by corentincaris

README.md

@@ -7,20 +7,20 @@ tags:
 license: mit
 ---
 
-# baseline
+# chess-baseline
 
 Chess model submitted to the LLM Course Chess Challenge.
 
 ## Submission Info
 
-- **Submitted by**: [Chiensaucisse67](https://huggingface.co/Chiensaucisse67)
-- **Parameters**: 909,824
+- **Submitted by**: [corentincaris](https://huggingface.co/corentincaris)
+- **Parameters**: 950,496
 - **Organization**: LLM-course
 
 ## Model Details
 
 - **Architecture**: Chess Transformer (GPT-style)
-- **Vocab size**: 1682
-- **Embedding dim**: 128
-- **Layers**: 4
-- **Heads**: 4
+- **Vocab size**: 4074
+- **Embedding dim**: 96
+- **Layers**: 5
+- **Heads**: 8

config.json

@@ -9,12 +9,13 @@
   "layer_norm_epsilon": 1e-05,
   "model_type": "chess_transformer",
   "n_ctx": 256,
-  "n_embd": 128,
-  "n_head": 4,
+  "n_embd": 96,
+  "n_head": 8,
   "n_inner": 384,
-  "n_layer": 4,
+  "n_layer": 5,
   "pad_token_id": 0,
   "tie_weights": true,
-  "transformers_version": "4.57.1",
-  "vocab_size": 1682
+  "transformers_version": "4.57.5",
+  "use_rope": true,
+  "vocab_size": 4074
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:07d869fcc7d97586752e194160bf38ac43c2aa4e2114a0a63a9cc7a45c919932
-size 3643696
+oid sha256:434ea59b434ecb57f72b3159455b5d84d7d66e663b8ab86077a0a3abb1366299
+size 3807840