End of training

README.md

@@ -0,0 +1,76 @@
+---
+license: apache-2.0
+library_name: peft
+tags:
+- trl
+- dpo
+- generated_from_trainer
+base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
+model-index:
+- name: fununun
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# fununun
+
+This model is a fine-tuned version of [TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) on an unknown dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.6908
+- Rewards/chosen: 0.0015
+- Rewards/rejected: -0.0032
+- Rewards/accuracies: 0.7176
+- Rewards/margins: 0.0047
+- Logps/rejected: -197.2385
+- Logps/chosen: -235.0630
+- Logits/rejected: -3.0691
+- Logits/chosen: -3.1037
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 5e-07
+- train_batch_size: 2
+- eval_batch_size: 2
+- seed: 42
+- gradient_accumulation_steps: 16
+- total_train_batch_size: 32
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 20
+- training_steps: 100
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
+|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
+| 0.693         | 0.04  | 20   | 0.6927          | 0.0001         | -0.0007          | 0.5614             | 0.0009          | -197.2139      | -235.0765    | -3.0688         | -3.1035       |
+| 0.6922        | 0.07  | 40   | 0.6919          | 0.0007         | -0.0017          | 0.6440             | 0.0024          | -197.2236      | -235.0704    | -3.0690         | -3.1036       |
+| 0.6913        | 0.11  | 60   | 0.6913          | 0.0011         | -0.0025          | 0.6886             | 0.0037          | -197.2319      | -235.0664    | -3.0691         | -3.1037       |
+| 0.6909        | 0.15  | 80   | 0.6909          | 0.0014         | -0.0030          | 0.7098             | 0.0044          | -197.2367      | -235.0639    | -3.0691         | -3.1037       |
+| 0.6906        | 0.19  | 100  | 0.6908          | 0.0015         | -0.0032          | 0.7176             | 0.0047          | -197.2385      | -235.0630    | -3.0691         | -3.1037       |
+
+
+### Framework versions
+
+- PEFT 0.7.1
+- Transformers 4.36.2
+- Pytorch 2.1.0+cu121
+- Datasets 2.16.1
+- Tokenizers 0.15.0

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f79288f6df2addc359d37611b2ea6803f2a9fd04486b552a9dbadff2b8f0b623
+oid sha256:1fdb0e6a7f2b8dbb74a544f65375ce519674e7c64c38aa5cce19cb6f83e5027c
 size 50503544

runs/Jan10_06-19-39_da9d9452362f/events.out.tfevents.1704867739.da9d9452362f.217.0

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:91e11faa7c06d1957ee18c6aaf1150223a819850067fc0460a7692f566eca3a6
-size 11332
+oid sha256:fc000860a7ef75c22818d48d5c484e018aac8b676427a71c00ac5d1169ed10f9
+size 15003