End of training

README.md

@@ -14,24 +14,9 @@ model-index:
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 should probably proofread and complete it, then remove this comment. -->
 
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/algo-llm/huggingface/runs/g44k7xr4)
-[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="200" height="32"/>](https://wandb.ai/algo-llm/huggingface/runs/g44k7xr4)
 # orpo-phi3
 
 This model is a fine-tuned version of [microsoft/Phi-3-mini-4k-instruct](https://huggingface.co/microsoft/Phi-3-mini-4k-instruct) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 4.1677
-- Rewards/chosen: -0.4133
-- Rewards/rejected: -0.4133
-- Rewards/accuracies: 0.0
-- Rewards/margins: 0.0
-- Logps/rejected: -4.1330
-- Logps/chosen: -4.1330
-- Logits/rejected: 24.1632
-- Logits/chosen: 24.1632
-- Nll Loss: 4.0984
-- Log Odds Ratio: -0.6931
-- Log Odds Chosen: 0.0
 
 ## Model description
 
@@ -51,10 +36,10 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 8e-06
-- train_batch_size: 4
-- eval_batch_size: 4
+- train_batch_size: 8
+- eval_batch_size: 8
 - seed: 42
-- gradient_accumulation_steps: 4
+- gradient_accumulation_steps: 2
 - total_train_batch_size: 16
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
@@ -63,15 +48,12 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen | Nll Loss | Log Odds Ratio | Log Odds Chosen |
-|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|:--------:|:--------------:|:---------------:|
-| 3.1975        | 1.0   | 1    | 4.1677          | -0.4133        | -0.4133          | 0.0                | 0.0             | -4.1330        | -4.1330      | 24.1632         | 24.1632       | 4.0984   | -0.6931        | 0.0             |
 
 
 ### Framework versions
 
 - PEFT 0.11.1
-- Transformers 4.42.3
-- Pytorch 2.1.2
+- Transformers 4.41.2
+- Pytorch 2.3.0+cu121
 - Datasets 2.20.0
 - Tokenizers 0.19.1

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:abc3a10390c80112f04fe1a06945116271d77c3735d7f6b40139dcd3e3cc70d3
+oid sha256:9a6c454e02b540706ec74c8763f61229c8ae6d334f4b18db403b9715ed02696b
 size 887450008

runs/Jul17_06-26-41_37d7a5970965/events.out.tfevents.1721197603.37d7a5970965.872.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c7b951e28302700597a0e7799107dc90f58b4f3cf3a946cea27a0f5ca58389c4
+size 6385

tokenizer_config.json

@@ -1,7 +1,6 @@
 {
   "add_bos_token": false,
   "add_eos_token": false,
-  "add_prefix_space": null,
   "added_tokens_decoder": {
     "0": {
       "content": "<unk>",

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9bc70753d74fcc8a5b520f4bc5d1b8b4c3abe3ed55fb535acdb010b983ca005b
+oid sha256:16e5147f1c412a171ca98f721ab261ca64e62756d8ea62b48c637bb727c83756
 size 5432