Hugging Face's logo

apple
/
DCLM-7B

Transformers
Safetensors
openlm
Model card Files
xet
 Community
11
vaishaal commited on
Commit
3959b33
·
verified ·
1 Parent(s): 29da0e7

Update config.json

Browse files
Files changed (1) hide show
  1. config.json +1 -1
config.json CHANGED
@@ -10,7 +10,7 @@
10
  "attn_seq_scalar": null,
11
  "attn_seq_scalar_alpha": null,
12
  "dim": 4096,
13
- "ffn_type": "swiglu",
14
  "model": "open_lm_7b",
15
  "norm_type": "gain_only_lp_layer_norm",
16
  "moe_capacity_factor": 1.25,
 
10
  "attn_seq_scalar": null,
11
  "attn_seq_scalar_alpha": null,
12
  "dim": 4096,
13
+ "ffn_type": "swiglu_torch",
14
  "model": "open_lm_7b",
15
  "norm_type": "gain_only_lp_layer_norm",
16
  "moe_capacity_factor": 1.25,