End of training

README.md

@@ -1,6 +1,6 @@
 ---
-license: other
-base_model: facebook/opt-350m
+license: apache-2.0
+base_model: openlm-research/open_llama_3b
 tags:
 - generated_from_trainer
 model-index:
@@ -13,9 +13,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 # langauage-modelfacebookopt
 
-This model is a fine-tuned version of [facebook/opt-350m](https://huggingface.co/facebook/opt-350m) on the None dataset.
+This model is a fine-tuned version of [openlm-research/open_llama_3b](https://huggingface.co/openlm-research/open_llama_3b) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 3.2257
+- Loss: 2.8124
 
 ## Model description
 
@@ -40,15 +40,13 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 3
+- num_epochs: 1
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 3.2607        | 1.0   | 2293 | 3.2278          |
-| 3.2592        | 2.0   | 4586 | 3.2263          |
-| 3.2514        | 3.0   | 6879 | 3.2257          |
+| 2.772         | 1.0   | 1192 | 2.8124          |
 
 
 ### Framework versions

adapter_config.json

@@ -1,6 +1,6 @@
 {
   "auto_mapping": null,
-  "base_model_name_or_path": "facebook/opt-350m",
+  "base_model_name_or_path": "openlm-research/open_llama_3b",
   "bias": "none",
   "fan_in_fan_out": false,
   "inference_mode": false,
@@ -15,8 +15,7 @@
   "revision": null,
   "target_modules": [
     "q_proj",
-    "v_proj",
-    "k_proj"
+    "v_proj"
   ],
   "task_type": "CAUSAL_LM"
 }

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b70be66bc65cccab9d17ec1879dafd458e3a3a3f3b0b88ebb95de8ba086d1bde
-size 18927498
+oid sha256:b7c797e68ce04530953062e31d5ecc5cffb7dea3011af36fec66f1711bfe0daf
+size 42636138

generation_config.json

@@ -1,7 +1,7 @@
 {
   "_from_model_config": true,
-  "bos_token_id": 2,
+  "bos_token_id": 1,
   "eos_token_id": 2,
-  "pad_token_id": 1,
+  "pad_token_id": 0,
   "transformers_version": "4.34.1"
 }

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:755a0255878889d5556e94bbedbedade740b4b8d44662dedc558fc9829f5b783
+oid sha256:c9c4185d4549dd4148c986679a8f6fcfcfee7297539f567e8f32605237262432
 size 4536