MistyozAI
/

CosmicFish-300M

Text Generation

Model card Files Files and versions

akkiisfrommars commited on 10 days ago

Commit

d872440

·

verified ·

1 Parent(s): 4527e0b

Update modeling_cosmicfish.py

Files changed (1) hide show

modeling_cosmicfish.py +1 -1

modeling_cosmicfish.py CHANGED Viewed

@@ -11,7 +11,7 @@ class CosmicConfig:
                  vocab_size=50257,
                  block_size=2048,
                  n_layer=24,
-                 n_head=16,
                  n_embd=960,
                  bias=True,
                  dropout=0.0,  # Always 0 for inference

                  vocab_size=50257,
                  block_size=2048,
                  n_layer=24,
+                 n_head=24,
                  n_embd=960,
                  bias=True,
                  dropout=0.0,  # Always 0 for inference