Model save

README.md

@@ -14,7 +14,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 3.8060
+- Loss: 3.8026
 
 ## Model description
 
@@ -46,13 +46,13 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss |
 |:-------------:|:------:|:----:|:---------------:|
-| 4.7593        | 0.1280 | 1000 | 4.7435          |
-| 4.3215        | 0.2560 | 2000 | 4.2979          |
-| 4.151         | 0.3840 | 3000 | 4.1439          |
-| 4.0331        | 0.5120 | 4000 | 4.0196          |
-| 3.9306        | 0.6400 | 5000 | 3.9140          |
-| 3.8629        | 0.7680 | 6000 | 3.8403          |
-| 3.8104        | 0.8959 | 7000 | 3.8060          |
+| 4.7501        | 0.1280 | 1000 | 4.7370          |
+| 4.3175        | 0.2560 | 2000 | 4.2985          |
+| 4.1499        | 0.3840 | 3000 | 4.1480          |
+| 4.0267        | 0.5120 | 4000 | 4.0196          |
+| 3.9244        | 0.6400 | 5000 | 3.9129          |
+| 3.8518        | 0.7680 | 6000 | 3.8355          |
+| 3.8           | 0.8959 | 7000 | 3.8026          |
 
 
 ### Framework versions

config.json

@@ -3,7 +3,7 @@
     "NeoLLMForCausalLM"
   ],
   "attention_bias": false,
-  "attention_dropout": 0.1,
+  "attention_dropout": 0.0,
   "auto_map": {
     "AutoConfig": "configuration_unified.UnifiedModelConfig",
     "AutoModel": "modeling_unified.UnifiedModel",

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b04d8de8cd98d92dc0749d52adcc6cfc6ecc80baa5c8ff9269723a0b2713724e
+oid sha256:3e7e551f9f876e76d97539cc4140bf1ae834c05420963faf498bd4246d14d521
 size 250512472

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e00d7a5cd61299a0886f2712be3e43745ace37274e803b22cc66adb488cf2ff6
+oid sha256:e6c80d7861bc23f70972c20beea5d56ac23c3a3ffe6cf135f2ddd5703f530c14
 size 5969