Update README.md
Browse files
README.md
CHANGED
|
@@ -3,4 +3,17 @@ license: apache-2.0
|
|
| 3 |
---
|
| 4 |
# MarinbadGPT
|
| 5 |
|
| 6 |
-
MarinbadGPT
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
| 4 |
# MarinbadGPT
|
| 5 |
|
| 6 |
+
MarinbadGPT is a language model based on HuggingFaceTB's SmolLM-135M architecture, finely trained on a corpus of **Marinbad** games. The aim of this model is to generate games of Marinbad and play them against a human player.
|
| 7 |
+
|
| 8 |
+
## Model Training
|
| 9 |
+
|
| 10 |
+
The training of MarinbadGPT was conducted on a high-performance computing infrastructure utilizing NVIDIA H100 GPUs, renowned for their power in the field of deep learning.
|
| 11 |
+
|
| 12 |
+
**Training Configuration:**
|
| 13 |
+
|
| 14 |
+
* **Infrastructure:** 2x NVIDIA H100 (80GB HBM3)
|
| 15 |
+
* **Duration:** 1 hour
|
| 16 |
+
* **Optimizer:** AdamW
|
| 17 |
+
* **Learning Rate:** 3e-4
|
| 18 |
+
* **Batch Size:** Micro batch size of 128, with gradient accumulation steps of 8, resulting in an effective batch size of 1024
|
| 19 |
+
* **Warmup Steps:** 100
|