azma-OpenHermes-2.5-Mistral-7B-agent-v1

README.md

@@ -18,7 +18,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [teknium/OpenHermes-2.5-Mistral-7B](https://huggingface.co/teknium/OpenHermes-2.5-Mistral-7B) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.6195
+- Loss: 0.4192
 
 ## Model description
 
@@ -37,7 +37,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0002
+- learning_rate: 2e-05
 - train_batch_size: 2
 - eval_batch_size: 2
 - seed: 42
@@ -46,7 +46,7 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: cosine
 - lr_scheduler_warmup_ratio: 0.03
-- lr_scheduler_warmup_steps: 50
+- lr_scheduler_warmup_steps: 100
 - num_epochs: 1
 - mixed_precision_training: Native AMP
 
@@ -54,18 +54,11 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 0.9186        | 0.08  | 10   | 0.7712          |
-| 0.5375        | 0.15  | 20   | 0.6183          |
-| 0.5409        | 0.23  | 30   | 0.6055          |
-| 0.635         | 0.31  | 40   | 0.6214          |
-| 0.5464        | 0.39  | 50   | 0.6498          |
-| 0.5474        | 0.46  | 60   | 0.6732          |
-| 0.4542        | 0.54  | 70   | 0.6435          |
-| 0.5981        | 0.62  | 80   | 0.6585          |
-| 0.5791        | 0.7   | 90   | 0.6484          |
-| 0.4875        | 0.77  | 100  | 0.6360          |
-| 0.4654        | 0.85  | 110  | 0.6309          |
-| 0.4344        | 0.93  | 120  | 0.6195          |
+| 1.3147        | 0.17  | 25   | 1.1471          |
+| 0.6178        | 0.34  | 50   | 0.5957          |
+| 0.4326        | 0.51  | 75   | 0.4810          |
+| 0.3723        | 0.67  | 100  | 0.4367          |
+| 0.348         | 0.84  | 125  | 0.4192          |
 
 
 ### Framework versions

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:643f145ef59ff6990158ed473a50e0a3d4f63269138d9ab025bdc2f1ccd9161f
+oid sha256:680b9c6b183759a7adf655f0859c97d3ed7de40d702d79161945a6e01a56a8e5
 size 859991408