MistyozAI
/

CosmicFish-300M

Text Generation

Model card Files Files and versions

akkiisfrommars commited on 9 days ago

Commit

4527e0b

·

verified ·

1 Parent(s): 3064365

Update chat.py

Files changed (1) hide show

chat.py +1 -1

chat.py CHANGED Viewed

@@ -60,7 +60,7 @@ class CosmicConfig:
                  vocab_size=50257,
                  block_size=2048,
                  n_layer=24,
-                 n_head=16,
                  n_embd=960,
                  bias=True,
                  dropout=0.0,

                  vocab_size=50257,
                  block_size=2048,
                  n_layer=24,
+                 n_head=24,
                  n_embd=960,
                  bias=True,
                  dropout=0.0,