bigscience
/

bloom-7b1

Text Generation

text-generation-inference

Model card Files Files and versions

Muennighoff commited on Aug 13, 2022

Commit

fb716d3

·

1 Parent(s): 59facdf

Correct pretraining tp

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -20,7 +20,7 @@
   "n_layer": 30,
   "num_attention_heads": 32,
   "offset_alibi": 100,
-  "pretraining_tp": 4,
   "seq_length": 2048,
   "skip_bias_add": true,
   "skip_bias_add_qkv": false,

   "n_layer": 30,
   "num_attention_heads": 32,
   "offset_alibi": 100,
+  "pretraining_tp": 1,
   "seq_length": 2048,
   "skip_bias_add": true,
   "skip_bias_add_qkv": false,