LM-Parallel
/

jax-reference-gpt2-2x-s

Model card Files Files and versions

Jiayi-Pan commited on Jan 15, 2025

Commit

f45a551

·

verified ·

1 Parent(s): 441e6e2

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -15,7 +15,7 @@
   "n_embd": 1024,
   "n_inner": 4096,
   "n_head": 16,
-  "n_layer": 16,
   "n_positions": 4096,
   "resid_pdrop": 0.1,
   "vocab_size": 50257

   "n_embd": 1024,
   "n_inner": 4096,
   "n_head": 16,
+  "n_layer": 32,
   "n_positions": 4096,
   "resid_pdrop": 0.1,
   "vocab_size": 50257