OEvortex
/

HelpingAI-3B

Text Generation

Model card Files Files and versions

Abhaykoul commited on Mar 24, 2024

Commit

aeb3b62

·

verified ·

1 Parent(s): dacc032

Update configuration_HelpingAI.py

Files changed (1) hide show

configuration_HelpingAI.py +1 -1

configuration_HelpingAI.py CHANGED Viewed

@@ -34,7 +34,7 @@ class HelpingAIConfig(PretrainedConfig):
         use_cache=True,
         hidden_activation=None,
         rope_theta=10000,
-        rope_pct=0.25
         attention_bias=False,
         attention_dropout=0.0,
         num_experts_per_tok=2,

         use_cache=True,
         hidden_activation=None,
         rope_theta=10000,
+        rope_pct=0.25,
         attention_bias=False,
         attention_dropout=0.0,
         num_experts_per_tok=2,