Hugging Face's logo

gitpullpull
/
D-CoT-models

TensorBoard
Safetensors
Model card Files
xet
 Metrics Community
D-CoT-models / run_20251218_140435 /README.md
gitpullpull's picture
gitpullpull
Update run_20251218_140435/README.md
e0f2afc verified
preview code
|
raw
history blame contribute delete
210 kB
metadata 
base_model: Unsloth/qwen3-8b
library_name: peft
model_name: output_orpo
tags:
  - base_model:adapter:Unsloth/qwen3-8b
  - lora
  - orpo
  - transformers
  - trl
  - unsloth
licence: license
pipeline_tag: text-generation

Model Card for output_orpo

This model is a fine-tuned version of Unsloth/qwen3-8b. It has been trained using TRL.

Quick start 

from transformers import pipeline

question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="None", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])

Training procedure

This model was trained with ORPO, a method introduced in ORPO: Monolithic Preference Optimization without Reference Model.

Framework versions

  • PEFT 0.18.0
  • TRL: 0.24.0
  • Transformers: 4.57.1
  • Pytorch: 2.9.0+cu128
  • Datasets: 4.3.0
  • Tokenizers: 0.22.1

Citations

Cite ORPO as:

@article{hong2024orpo,
    title        = {{ORPO: Monolithic Preference Optimization without Reference Model}},
    author       = {Jiwoo Hong and Noah Lee and James Thorne},
    year         = 2024,
    eprint       = {arXiv:2403.07691}
}

Cite TRL as:

@misc{vonwerra2022trl,
    title        = {{TRL: Transformer Reinforcement Learning}},
    author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
    year         = 2020,
    journal      = {GitHub repository},
    publisher    = {GitHub},
    howpublished = {\url{https://github.com/huggingface/trl}}
}

ๅˆ†ๆžใ—ใฆ unsloth@e150a2d13ef8:/workspace$ git clone https://github.com/gitpullpull/Introspective_Temperature Cloning into 'Introspective_Temperature'... remote: Enumerating objects: 24, done. remote: Counting objects: 100% (24/24), done. remote: Compressing objects: 100% (17/17), done. remote: Total 24 (delta 6), reused 21 (delta 3), pack-reused 0 (from 0) Receiving objects: 100% (24/24), 9.12 MiB | 20.25 MiB/s, done. Resolving deltas: 100% (6/6), done. unsloth@e150a2d13ef8:/workspace$ cd Introspective_Temperature/ unsloth@e150a2d13ef8:/workspace/Introspective_Temperature$ bash run_job.sh  === Job Started at 20251218_140407 === Logs will be saved to: training_log_20251218_140407.txt [1/2] Installing dependencies... Requirement already satisfied: pandas in /opt/conda/lib/python3.11/site-packages (2.3.3) Requirement already satisfied: matplotlib in /opt/conda/lib/python3.11/site-packages (3.10.8) Requirement already satisfied: huggingface_hub in /opt/conda/lib/python3.11/site-packages (0.36.0) Requirement already satisfied: numpy>=1.23.2 in /opt/conda/lib/python3.11/site-packages (from pandas) (2.2.6) Requirement already satisfied: python-dateutil>=2.8.2 in /opt/conda/lib/python3.11/site-packages (from pandas) (2.9.0.post0) Requirement already satisfied: pytz>=2020.1 in /opt/conda/lib/python3.11/site-packages (from pandas) (2025.2) Requirement already satisfied: tzdata>=2022.7 in /opt/conda/lib/python3.11/site-packages (from pandas) (2025.2) Requirement already satisfied: contourpy>=1.0.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (1.3.3) Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (0.12.1) Requirement already satisfied: fonttools>=4.22.0 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (4.61.0) Requirement already satisfied: kiwisolver>=1.3.1 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (1.4.9) Requirement already satisfied: packaging>=20.0 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (25.0) Requirement already satisfied: pillow>=8 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (11.3.0) Requirement already satisfied: pyparsing>=3 in /opt/conda/lib/python3.11/site-packages (from matplotlib) (3.2.5) Requirement already satisfied: filelock in /opt/conda/lib/python3.11/site-packages (from huggingface_hub) (3.20.0) Requirement already satisfied: fsspec>=2023.5.0 in /opt/conda/lib/python3.11/site-packages (from huggingface_hub) (2025.3.0) Requirement already satisfied: pyyaml>=5.1 in /opt/conda/lib/python3.11/site-packages (from huggingface_hub) (6.0.3) Requirement already satisfied: requests in /opt/conda/lib/python3.11/site-packages (from huggingface_hub) (2.32.5) Requirement already satisfied: tqdm>=4.42.1 in /opt/conda/lib/python3.11/site-packages (from huggingface_hub) (4.67.1) Requirement already satisfied: typing-extensions>=3.7.4.3 in /opt/conda/lib/python3.11/site-packages (from huggingface_hub) (4.15.0) Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /opt/conda/lib/python3.11/site-packages (from huggingface_hub) (1.2.0) Requirement already satisfied: six>=1.5 in /opt/conda/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.17.0) Requirement already satisfied: charset_normalizer<4,>=2 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface_hub) (3.4.4) Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface_hub) (3.11) Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface_hub) (2.6.2) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.11/site-packages (from requests->huggingface_hub) (2025.11.12) [2/2] Starting Train-ORPO.py... ๐Ÿฆฅ Unsloth: Will patch your computer to enable 2x faster free finetuning. TMA benchmarks will be running without grid constant TMA descriptor. ๐Ÿฆฅ Unsloth Zoo will now patch everything to make training faster! Unsloth: FBGEMM on the current GPU cannot load - will switch to Triton kernels [unsloth_zoo.log|WARNING]Unsloth: Failed to import trl openenv: No module named 'trl.experimental.openenv' Run Timestamp: 20251218_140435 Hugging Face logged in. Target Repo: gitpullpull/Introspective_Temperature_test Loading model: Unsloth/qwen3-8b... ==((====))==  Unsloth 2025.12.4: Fast Qwen3 patching. Transformers: 4.57.1. vLLM: 0.11.2.    \   /|    NVIDIA GeForce RTX 5090. Num GPUs = 1. Max memory: 31.367 GB. Platform: Linux. O^O/ _/ \    Torch: 2.9.0+cu128. CUDA: 12.0. CUDA Toolkit: 12.8. Triton: 3.5.0 \        /    Bfloat16 = TRUE. FA [Xformers = 0.0.33.post1. FA2 = False]  "--"     Free license: http://github.com/unslothai/unsloth Unsloth: Fast downloading is enabled - ignore downloading bars which are red colored! The following TP rules were not applied on any of the layers: {'layers..self_attn.q_proj': 'colwise', 'layers..self_attn.k_proj': 'colwise', 'layers..self_attn.v_proj': 'colwise', 'layers..self_attn.o_proj': 'rowwise', 'layers..mlp.gate_proj': 'colwise', 'layers..mlp.up_proj': 'colwise', 'layers..mlp.down_proj': 'rowwise'} The following layers were not sharded: lm_head.weight, model.embed_tokens.weight, model.layers..self_attn.k_norm.weight, model.layers..post_attention_layernorm.weight, model.layers..self_attn.q_norm.weight, model.norm.weight, model.layers.*.input_layernorm.weight Loading checkpoint shards: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 4/4 [00:01<00:00,  2.46it/s] Unsloth: Dropout = 0 is supported for fast patching. You are using dropout = 0.05. Unsloth will patch all other layers, except LoRA matrices, causing a performance hit. Unsloth 2025.12.4 patched 36 layers with 0 QKV layers, 0 O layers and 0 MLP layers. Generating train split: 2897 examples [00:00, 48596.01 examples/s] Filter: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 2897/2897 [00:00<00:00, 45470.16 examples/s] Map: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 2897/2897 [00:00<00:00, 5254.68 examples/s] Map (num_proc=64): 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 2897/2897 [00:07<00:00, 373.47 examples/s] Map (num_proc=64): 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 2897/2897 [00:16<00:00, 170.57 examples/s] Map (num_proc=64): 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 2897/2897 [00:16<00:00, 174.46 examples/s] Lion 8bit ORPOใƒˆใƒฌใƒผใƒ‹ใƒณใ‚ฐใ‚’้–‹ๅง‹ใ—ใพใ™ (LR: 1e-06)... The model is already on multiple devices. Skipping the move to device specified in args. ==((====))==  Unsloth - 2x faster free finetuning | Num GPUs used = 1    \   /|    Num examples = 2,897 | Num Epochs = 1 | Total steps = 363 O^O/ _/ \    Batch size per device = 1 | Gradient accumulation steps = 8 \        /    Data Parallel GPUs = 1 | Total batch size (1 x 8 x 1) = 8  "--"     Trainable parameters = 87,293,952 of 8,278,029,312 (1.05% trained)   0%|                                                                                                                                                                                                                                     | 0/363 [00:00<?, ?it/s]Unsloth: Will smartly offload gradients to save VRAM! {'loss': 2.1834, 'grad_norm': 0.5197265148162842, 'learning_rate': 0.0, 'rewards/chosen': -0.199848935008049, 'rewards/rejected': -0.23799772560596466, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03814878687262535, 'logps/rejected': -2.3799774646759033, 'logps/chosen': -1.998489260673523, 'logits/rejected': -0.10779725015163422, 'logits/chosen': -0.5456271767616272, 'nll_loss': 2.1329703330993652, 'log_odds_ratio': -0.5042418837547302, 'log_odds_chosen': 0.4317269027233124, 'epoch': 0.0} {'loss': 2.2463, 'grad_norm': 0.7380698323249817, 'learning_rate': 2.7027027027027028e-08, 'rewards/chosen': -0.20794589817523956, 'rewards/rejected': -0.24737600982189178, 'rewards/accuracies': 1.0, 'rewards/margins': 0.039430104196071625, 'logps/rejected': -2.473759889602661, 'logps/chosen': -2.0794589519500732, 'logits/rejected': -0.07251911610364914, 'logits/chosen': -0.6594648361206055, 'nll_loss': 2.196385383605957, 'log_odds_ratio': -0.4988963305950165, 'log_odds_chosen': 0.4411306381225586, 'epoch': 0.01} {'loss': 2.1674, 'grad_norm': 0.6366102695465088, 'learning_rate': 5.4054054054054056e-08, 'rewards/chosen': -0.20124749839305878, 'rewards/rejected': -0.24806547164916992, 'rewards/accuracies': 1.0, 'rewards/margins': 0.046817973256111145, 'logps/rejected': -2.480654716491699, 'logps/chosen': -2.012474775314331, 'logits/rejected': -0.04276692494750023, 'logits/chosen': -0.7439606785774231, 'nll_loss': 2.1203911304473877, 'log_odds_ratio': -0.46964502334594727, 'log_odds_chosen': 0.5236693620681763, 'epoch': 0.01} {'loss': 2.1537, 'grad_norm': 0.5186454653739929, 'learning_rate': 8.108108108108108e-08, 'rewards/chosen': -0.20060904324054718, 'rewards/rejected': -0.24304211139678955, 'rewards/accuracies': 1.0, 'rewards/margins': 0.042433060705661774, 'logps/rejected': -2.4304208755493164, 'logps/chosen': -2.0060904026031494, 'logits/rejected': -0.0752713680267334, 'logits/chosen': -0.5739610195159912, 'nll_loss': 2.1045989990234375, 'log_odds_ratio': -0.4913516938686371, 'log_odds_chosen': 0.4754515290260315, 'epoch': 0.01} {'loss': 2.1184, 'grad_norm': 0.5336912274360657, 'learning_rate': 1.0810810810810811e-07, 'rewards/chosen': -0.19649039208889008, 'rewards/rejected': -0.23827451467514038, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04178408533334732, 'logps/rejected': -2.382744789123535, 'logps/chosen': -1.9649040699005127, 'logits/rejected': -0.12771189212799072, 'logits/chosen': -0.5785455703735352, 'nll_loss': 2.0687689781188965, 'log_odds_ratio': -0.49613577127456665, 'log_odds_chosen': 0.4733606278896332, 'epoch': 0.01} {'loss': 2.1777, 'grad_norm': 0.5385575890541077, 'learning_rate': 1.3513513513513515e-07, 'rewards/chosen': -0.20121026039123535, 'rewards/rejected': -0.23618288338184357, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034972622990608215, 'logps/rejected': -2.3618290424346924, 'logps/chosen': -2.0121028423309326, 'logits/rejected': -0.22749024629592896, 'logits/chosen': -0.5337327718734741, 'nll_loss': 2.125922203063965, 'log_odds_ratio': -0.5181410312652588, 'log_odds_chosen': 0.3951405882835388, 'epoch': 0.02} {'loss': 2.237, 'grad_norm': 0.5466561913490295, 'learning_rate': 1.6216216216216215e-07, 'rewards/chosen': -0.20744195580482483, 'rewards/rejected': -0.23086859285831451, 'rewards/accuracies': 0.875, 'rewards/margins': 0.023426644504070282, 'logps/rejected': -2.308685779571533, 'logps/chosen': -2.0744194984436035, 'logits/rejected': -0.23400373756885529, 'logits/chosen': -0.5413729548454285, 'nll_loss': 2.1792986392974854, 'log_odds_ratio': -0.5772624015808105, 'log_odds_chosen': 0.2610325217247009, 'epoch': 0.02} {'loss': 2.247, 'grad_norm': 0.5931817889213562, 'learning_rate': 1.891891891891892e-07, 'rewards/chosen': -0.20818378031253815, 'rewards/rejected': -0.24534404277801514, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03716025501489639, 'logps/rejected': -2.4534404277801514, 'logps/chosen': -2.0818376541137695, 'logits/rejected': -0.042557284235954285, 'logits/chosen': -0.4540097415447235, 'nll_loss': 2.1960983276367188, 'log_odds_ratio': -0.5085166692733765, 'log_odds_chosen': 0.41551724076271057, 'epoch': 0.02} {'loss': 2.1468, 'grad_norm': 0.6041191816329956, 'learning_rate': 2.1621621621621622e-07, 'rewards/chosen': -0.19715292751789093, 'rewards/rejected': -0.25645241141319275, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05929948389530182, 'logps/rejected': -2.5645241737365723, 'logps/chosen': -1.9715290069580078, 'logits/rejected': -0.16427314281463623, 'logits/chosen': -0.7538988590240479, 'nll_loss': 2.104363441467285, 'log_odds_ratio': -0.42459195852279663, 'log_odds_chosen': 0.6630181670188904, 'epoch': 0.02} {'loss': 2.2565, 'grad_norm': 0.738123893737793, 'learning_rate': 2.4324324324324326e-07, 'rewards/chosen': -0.20911525189876556, 'rewards/rejected': -0.24895140528678894, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03983616083860397, 'logps/rejected': -2.489513874053955, 'logps/chosen': -2.0911521911621094, 'logits/rejected': -0.23782339692115784, 'logits/chosen': -0.7369757890701294, 'nll_loss': 2.2063329219818115, 'log_odds_ratio': -0.5019431710243225, 'log_odds_chosen': 0.44313472509384155, 'epoch': 0.03} {'loss': 2.2245, 'grad_norm': 0.66302490234375, 'learning_rate': 2.702702702702703e-07, 'rewards/chosen': -0.20552299916744232, 'rewards/rejected': -0.25372591614723206, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04820293188095093, 'logps/rejected': -2.537259340286255, 'logps/chosen': -2.055230140686035, 'logits/rejected': -0.14047637581825256, 'logits/chosen': -0.6730350255966187, 'nll_loss': 2.1777424812316895, 'log_odds_ratio': -0.4679567217826843, 'log_odds_chosen': 0.5352292060852051, 'epoch': 0.03} {'loss': 2.1445, 'grad_norm': 0.5557176470756531, 'learning_rate': 2.972972972972973e-07, 'rewards/chosen': -0.19575071334838867, 'rewards/rejected': -0.23148740828037262, 'rewards/accuracies': 0.75, 'rewards/margins': 0.03573668748140335, 'logps/rejected': -2.3148741722106934, 'logps/chosen': -1.9575071334838867, 'logits/rejected': -0.006882909685373306, 'logits/chosen': -0.5875533223152161, 'nll_loss': 2.092282772064209, 'log_odds_ratio': -0.5220299959182739, 'log_odds_chosen': 0.4027867913246155, 'epoch': 0.03} {'loss': 2.0677, 'grad_norm': 0.46403586864471436, 'learning_rate': 3.243243243243243e-07, 'rewards/chosen': -0.1889842301607132, 'rewards/rejected': -0.23196667432785034, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04298242926597595, 'logps/rejected': -2.3196663856506348, 'logps/chosen': -1.88984215259552, 'logits/rejected': -0.08394598215818405, 'logits/chosen': -0.4917553961277008, 'nll_loss': 2.019725799560547, 'log_odds_ratio': -0.479642391204834, 'log_odds_chosen': 0.4915584325790405, 'epoch': 0.04} {'loss': 2.233, 'grad_norm': 0.6981692314147949, 'learning_rate': 3.5135135135135134e-07, 'rewards/chosen': -0.2086106836795807, 'rewards/rejected': -0.2506310045719147, 'rewards/accuracies': 0.875, 'rewards/margins': 0.0420202761888504, 'logps/rejected': -2.506309747695923, 'logps/chosen': -2.086107015609741, 'logits/rejected': -0.013176449574530125, 'logits/chosen': -0.687742292881012, 'nll_loss': 2.183150053024292, 'log_odds_ratio': -0.4980979561805725, 'log_odds_chosen': 0.4690040051937103, 'epoch': 0.04} {'loss': 2.1671, 'grad_norm': 0.7050800323486328, 'learning_rate': 3.783783783783784e-07, 'rewards/chosen': -0.20061349868774414, 'rewards/rejected': -0.2514050006866455, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05079151317477226, 'logps/rejected': -2.514050006866455, 'logps/chosen': -2.0061349868774414, 'logits/rejected': 0.1294655054807663, 'logits/chosen': -0.7780197262763977, 'nll_loss': 2.1222198009490967, 'log_odds_ratio': -0.448469340801239, 'log_odds_chosen': 0.5713032484054565, 'epoch': 0.04} {'loss': 2.0746, 'grad_norm': 0.49071159958839417, 'learning_rate': 4.054054054054054e-07, 'rewards/chosen': -0.1896301507949829, 'rewards/rejected': -0.22880229353904724, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03917212039232254, 'logps/rejected': -2.288022994995117, 'logps/chosen': -1.8963017463684082, 'logits/rejected': 0.05348123610019684, 'logits/chosen': -0.6061773300170898, 'nll_loss': 2.0244240760803223, 'log_odds_ratio': -0.5018619894981384, 'log_odds_chosen': 0.4512079954147339, 'epoch': 0.04} {'loss': 2.1345, 'grad_norm': 0.5817921161651611, 'learning_rate': 4.3243243243243244e-07, 'rewards/chosen': -0.19876189529895782, 'rewards/rejected': -0.24516057968139648, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04639868065714836, 'logps/rejected': -2.451605796813965, 'logps/chosen': -1.987619161605835, 'logits/rejected': 0.055297549813985825, 'logits/chosen': -0.5998549461364746, 'nll_loss': 2.0870776176452637, 'log_odds_ratio': -0.47411778569221497, 'log_odds_chosen': 0.5201853513717651, 'epoch': 0.05} {'loss': 2.1659, 'grad_norm': 0.6028745174407959, 'learning_rate': 4.594594594594595e-07, 'rewards/chosen': -0.20205730199813843, 'rewards/rejected': -0.2591531574726105, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05709584802389145, 'logps/rejected': -2.59153151512146, 'logps/chosen': -2.0205729007720947, 'logits/rejected': -0.0934261828660965, 'logits/chosen': -0.6498495936393738, 'nll_loss': 2.122877359390259, 'log_odds_ratio': -0.4299185276031494, 'log_odds_chosen': 0.6340630054473877, 'epoch': 0.05} {'loss': 2.2221, 'grad_norm': 0.6153688430786133, 'learning_rate': 4.864864864864865e-07, 'rewards/chosen': -0.20559124648571014, 'rewards/rejected': -0.22657965123653412, 'rewards/accuracies': 1.0, 'rewards/margins': 0.02098841220140457, 'logps/rejected': -2.265796661376953, 'logps/chosen': -2.055912494659424, 'logits/rejected': -0.058345817029476166, 'logits/chosen': -0.6359980702400208, 'nll_loss': 2.163560152053833, 'log_odds_ratio': -0.5856046676635742, 'log_odds_chosen': 0.23762521147727966, 'epoch': 0.05} {'loss': 2.1563, 'grad_norm': 0.5243191123008728, 'learning_rate': 5.135135135135134e-07, 'rewards/chosen': -0.19716015458106995, 'rewards/rejected': -0.2358742207288742, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03871406614780426, 'logps/rejected': -2.3587422370910645, 'logps/chosen': -1.9716014862060547, 'logits/rejected': 0.012299822643399239, 'logits/chosen': -0.5481252670288086, 'nll_loss': 2.1058452129364014, 'log_odds_ratio': -0.5044845342636108, 'log_odds_chosen': 0.43903401494026184, 'epoch': 0.06} {'loss': 2.2806, 'grad_norm': 0.6418594717979431, 'learning_rate': 5.405405405405406e-07, 'rewards/chosen': -0.2136019915342331, 'rewards/rejected': -0.2454715520143509, 'rewards/accuracies': 0.75, 'rewards/margins': 0.03186953812837601, 'logps/rejected': -2.4547152519226074, 'logps/chosen': -2.1360199451446533, 'logits/rejected': -0.03464129567146301, 'logits/chosen': -0.45990100502967834, 'nll_loss': 2.2266147136688232, 'log_odds_ratio': -0.5401824712753296, 'log_odds_chosen': 0.3529980778694153, 'epoch': 0.06} {'loss': 2.1345, 'grad_norm': 0.573546290397644, 'learning_rate': 5.675675675675675e-07, 'rewards/chosen': -0.19800050556659698, 'rewards/rejected': -0.24254897236824036, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04454846680164337, 'logps/rejected': -2.425489664077759, 'logps/chosen': -1.9800050258636475, 'logits/rejected': -0.18484465777873993, 'logits/chosen': -0.6077824831008911, 'nll_loss': 2.0856926441192627, 'log_odds_ratio': -0.48850372433662415, 'log_odds_chosen': 0.5019404888153076, 'epoch': 0.06} {'loss': 2.1462, 'grad_norm': 0.5493528246879578, 'learning_rate': 5.945945945945947e-07, 'rewards/chosen': -0.19641812145709991, 'rewards/rejected': -0.23188185691833496, 'rewards/accuracies': 1.0, 'rewards/margins': 0.035463735461235046, 'logps/rejected': -2.3188183307647705, 'logps/chosen': -1.9641809463500977, 'logits/rejected': -0.3585430979728699, 'logits/chosen': -0.5691142082214355, 'nll_loss': 2.093902587890625, 'log_odds_ratio': -0.523137629032135, 'log_odds_chosen': 0.39711329340934753, 'epoch': 0.06} {'loss': 2.1365, 'grad_norm': 0.5700852870941162, 'learning_rate': 6.216216216216216e-07, 'rewards/chosen': -0.19673511385917664, 'rewards/rejected': -0.24291826784610748, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04618315398693085, 'logps/rejected': -2.429182767868042, 'logps/chosen': -1.967350959777832, 'logits/rejected': 0.039508990943431854, 'logits/chosen': -0.5802191495895386, 'nll_loss': 2.0896944999694824, 'log_odds_ratio': -0.4676600396633148, 'log_odds_chosen': 0.5202399492263794, 'epoch': 0.07} {'loss': 2.1586, 'grad_norm': 0.7011380791664124, 'learning_rate': 6.486486486486486e-07, 'rewards/chosen': -0.19984383881092072, 'rewards/rejected': -0.2588566839694977, 'rewards/accuracies': 1.0, 'rewards/margins': 0.059012845158576965, 'logps/rejected': -2.588566780090332, 'logps/chosen': -1.9984382390975952, 'logits/rejected': -0.06181307137012482, 'logits/chosen': -0.649218738079071, 'nll_loss': 2.1160356998443604, 'log_odds_ratio': -0.4259224534034729, 'log_odds_chosen': 0.6565573215484619, 'epoch': 0.07} {'loss': 2.1955, 'grad_norm': 0.6201441884040833, 'learning_rate': 6.756756756756756e-07, 'rewards/chosen': -0.20192717015743256, 'rewards/rejected': -0.23566550016403198, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03373831510543823, 'logps/rejected': -2.356654644012451, 'logps/chosen': -2.0192718505859375, 'logits/rejected': -0.11418405175209045, 'logits/chosen': -0.5880062580108643, 'nll_loss': 2.142634153366089, 'log_odds_ratio': -0.5289822816848755, 'log_odds_chosen': 0.3787028193473816, 'epoch': 0.07} {'loss': 2.0463, 'grad_norm': 0.44776439666748047, 'learning_rate': 7.027027027027027e-07, 'rewards/chosen': -0.18659432232379913, 'rewards/rejected': -0.22744019329547882, 'rewards/accuracies': 1.0, 'rewards/margins': 0.040845878422260284, 'logps/rejected': -2.274401903152466, 'logps/chosen': -1.8659433126449585, 'logits/rejected': -0.17443624138832092, 'logits/chosen': -0.43663290143013, 'nll_loss': 1.9967215061187744, 'log_odds_ratio': -0.49579283595085144, 'log_odds_chosen': 0.4692346453666687, 'epoch': 0.07} {'loss': 2.1246, 'grad_norm': 0.5262994170188904, 'learning_rate': 7.297297297297297e-07, 'rewards/chosen': -0.19571705162525177, 'rewards/rejected': -0.24248044192790985, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04676342010498047, 'logps/rejected': -2.424804449081421, 'logps/chosen': -1.9571702480316162, 'logits/rejected': -0.21029916405677795, 'logits/chosen': -0.5315351486206055, 'nll_loss': 2.077768325805664, 'log_odds_ratio': -0.4684979319572449, 'log_odds_chosen': 0.5275386571884155, 'epoch': 0.08} {'loss': 2.2042, 'grad_norm': 0.5570821762084961, 'learning_rate': 7.567567567567568e-07, 'rewards/chosen': -0.2046632468700409, 'rewards/rejected': -0.24716021120548248, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04249695688486099, 'logps/rejected': -2.471601724624634, 'logps/chosen': -2.0466325283050537, 'logits/rejected': -0.034404173493385315, 'logits/chosen': -0.5677704811096191, 'nll_loss': 2.1550400257110596, 'log_odds_ratio': -0.49199897050857544, 'log_odds_chosen': 0.4756891429424286, 'epoch': 0.08} {'loss': 2.1758, 'grad_norm': 0.6109534502029419, 'learning_rate': 7.837837837837838e-07, 'rewards/chosen': -0.20272573828697205, 'rewards/rejected': -0.2607854902744293, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05805976688861847, 'logps/rejected': -2.6078548431396484, 'logps/chosen': -2.027257204055786, 'logits/rejected': -0.004539853893220425, 'logits/chosen': -0.5536071062088013, 'nll_loss': 2.1329245567321777, 'log_odds_ratio': -0.42898693680763245, 'log_odds_chosen': 0.645262598991394, 'epoch': 0.08} {'loss': 2.1855, 'grad_norm': 0.6070998907089233, 'learning_rate': 8.108108108108108e-07, 'rewards/chosen': -0.20168517529964447, 'rewards/rejected': -0.2516200542449951, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04993489012122154, 'logps/rejected': -2.5162007808685303, 'logps/chosen': -2.0168516635894775, 'logits/rejected': -0.06864497065544128, 'logits/chosen': -0.5867283344268799, 'nll_loss': 2.1396615505218506, 'log_odds_ratio': -0.45878711342811584, 'log_odds_chosen': 0.5565178990364075, 'epoch': 0.09} {'loss': 2.1852, 'grad_norm': 0.4846738278865814, 'learning_rate': 8.378378378378377e-07, 'rewards/chosen': -0.20135138928890228, 'rewards/rejected': -0.24021016061306, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03885875269770622, 'logps/rejected': -2.402101516723633, 'logps/chosen': -2.0135138034820557, 'logits/rejected': 0.003593362867832184, 'logits/chosen': -0.4310506582260132, 'nll_loss': 2.134814500808716, 'log_odds_ratio': -0.5039945840835571, 'log_odds_chosen': 0.43661972880363464, 'epoch': 0.09} {'loss': 2.0956, 'grad_norm': 0.5441243648529053, 'learning_rate': 8.648648648648649e-07, 'rewards/chosen': -0.19284063577651978, 'rewards/rejected': -0.24360083043575287, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05076019838452339, 'logps/rejected': -2.4360082149505615, 'logps/chosen': -1.9284062385559082, 'logits/rejected': -0.058118633925914764, 'logits/chosen': -0.4675900936126709, 'nll_loss': 2.0503551959991455, 'log_odds_ratio': -0.45256343483924866, 'log_odds_chosen': 0.5726721286773682, 'epoch': 0.09} {'loss': 2.0767, 'grad_norm': 0.4893564283847809, 'learning_rate': 8.918918918918918e-07, 'rewards/chosen': -0.1925690770149231, 'rewards/rejected': -0.24649673700332642, 'rewards/accuracies': 1.0, 'rewards/margins': 0.053927645087242126, 'logps/rejected': -2.4649672508239746, 'logps/chosen': -1.9256908893585205, 'logits/rejected': -0.08722685277462006, 'logits/chosen': -0.41903308033943176, 'nll_loss': 2.0315608978271484, 'log_odds_ratio': -0.45129159092903137, 'log_odds_chosen': 0.6038653254508972, 'epoch': 0.09} {'loss': 2.1551, 'grad_norm': 0.6620936393737793, 'learning_rate': 9.18918918918919e-07, 'rewards/chosen': -0.1982971578836441, 'rewards/rejected': -0.23442938923835754, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03613222762942314, 'logps/rejected': -2.3442938327789307, 'logps/chosen': -1.982971429824829, 'logits/rejected': -0.20862913131713867, 'logits/chosen': -0.8187747001647949, 'nll_loss': 2.1038753986358643, 'log_odds_ratio': -0.512043833732605, 'log_odds_chosen': 0.41170692443847656, 'epoch': 0.1} {'loss': 2.1381, 'grad_norm': 0.6175864338874817, 'learning_rate': 9.459459459459459e-07, 'rewards/chosen': -0.1978471428155899, 'rewards/rejected': -0.252175897359848, 'rewards/accuracies': 1.0, 'rewards/margins': 0.054328739643096924, 'logps/rejected': -2.521758794784546, 'logps/chosen': -1.9784715175628662, 'logits/rejected': 0.013947159051895142, 'logits/chosen': -0.6989039182662964, 'nll_loss': 2.093674659729004, 'log_odds_ratio': -0.4442310333251953, 'log_odds_chosen': 0.6069755554199219, 'epoch': 0.1} {'loss': 2.2946, 'grad_norm': 0.6760265827178955, 'learning_rate': 9.72972972972973e-07, 'rewards/chosen': -0.2120981216430664, 'rewards/rejected': -0.26638612151145935, 'rewards/accuracies': 1.0, 'rewards/margins': 0.054287999868392944, 'logps/rejected': -2.6638612747192383, 'logps/chosen': -2.120981216430664, 'logits/rejected': -0.18748612701892853, 'logits/chosen': -0.5475363731384277, 'nll_loss': 2.2500131130218506, 'log_odds_ratio': -0.445417582988739, 'log_odds_chosen': 0.5962284803390503, 'epoch': 0.1} {'loss': 2.1348, 'grad_norm': 0.5492844581604004, 'learning_rate': 1e-06, 'rewards/chosen': -0.1951310932636261, 'rewards/rejected': -0.23313407599925995, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03800297528505325, 'logps/rejected': -2.3313405513763428, 'logps/chosen': -1.9513108730316162, 'logits/rejected': -0.1553649604320526, 'logits/chosen': -0.564774751663208, 'nll_loss': 2.0843911170959473, 'log_odds_ratio': -0.5045007467269897, 'log_odds_chosen': 0.429279625415802, 'epoch': 0.1} {'loss': 2.2348, 'grad_norm': 0.6983578205108643, 'learning_rate': 9.999767832624e-07, 'rewards/chosen': -0.20831823348999023, 'rewards/rejected': -0.25330057740211487, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04498231038451195, 'logps/rejected': -2.533005475997925, 'logps/chosen': -2.0831823348999023, 'logits/rejected': 0.011857733130455017, 'logits/chosen': -0.49002426862716675, 'nll_loss': 2.1848011016845703, 'log_odds_ratio': -0.49974578619003296, 'log_odds_chosen': 0.49402672052383423, 'epoch': 0.11} {'loss': 2.2778, 'grad_norm': 0.7787635922431946, 'learning_rate': 9.999071352056673e-07, 'rewards/chosen': -0.21053989231586456, 'rewards/rejected': -0.23543494939804077, 'rewards/accuracies': 0.75, 'rewards/margins': 0.02489505149424076, 'logps/rejected': -2.3543496131896973, 'logps/chosen': -2.1053991317749023, 'logits/rejected': -0.16685396432876587, 'logits/chosen': -0.6959272623062134, 'nll_loss': 2.2204456329345703, 'log_odds_ratio': -0.5739508271217346, 'log_odds_chosen': 0.2771000862121582, 'epoch': 0.11} {'loss': 2.0893, 'grad_norm': 0.4911040961742401, 'learning_rate': 9.99791062297805e-07, 'rewards/chosen': -0.19237489998340607, 'rewards/rejected': -0.2355954945087433, 'rewards/accuracies': 1.0, 'rewards/margins': 0.043220579624176025, 'logps/rejected': -2.355954647064209, 'logps/chosen': -1.9237489700317383, 'logits/rejected': -0.11792521923780441, 'logits/chosen': -0.5469129681587219, 'nll_loss': 2.0404934883117676, 'log_odds_ratio': -0.48782065510749817, 'log_odds_chosen': 0.48859286308288574, 'epoch': 0.11} {'loss': 2.2876, 'grad_norm': 0.6618607640266418, 'learning_rate': 9.996285753181497e-07, 'rewards/chosen': -0.21178597211837769, 'rewards/rejected': -0.24373027682304382, 'rewards/accuracies': 0.875, 'rewards/margins': 0.031944289803504944, 'logps/rejected': -2.437302589416504, 'logps/chosen': -2.1178598403930664, 'logits/rejected': -0.18673820793628693, 'logits/chosen': -0.6230701804161072, 'nll_loss': 2.2335362434387207, 'log_odds_ratio': -0.5404328107833862, 'log_odds_chosen': 0.3562512695789337, 'epoch': 0.12} {'loss': 2.1586, 'grad_norm': 0.5659067034721375, 'learning_rate': 9.99419689356372e-07, 'rewards/chosen': -0.19770362973213196, 'rewards/rejected': -0.2348427027463913, 'rewards/accuracies': 0.875, 'rewards/margins': 0.037139080464839935, 'logps/rejected': -2.3484270572662354, 'logps/chosen': -1.9770363569259644, 'logits/rejected': -0.1797933280467987, 'logits/chosen': -0.5873526930809021, 'nll_loss': 2.107292652130127, 'log_odds_ratio': -0.5130196809768677, 'log_odds_chosen': 0.4233945906162262, 'epoch': 0.12} {'loss': 2.2314, 'grad_norm': 0.5917359590530396, 'learning_rate': 9.99164423811074e-07, 'rewards/chosen': -0.2068721055984497, 'rewards/rejected': -0.23819389939308167, 'rewards/accuracies': 0.75, 'rewards/margins': 0.031321801245212555, 'logps/rejected': -2.381938934326172, 'logps/chosen': -2.068720817565918, 'logits/rejected': -0.20721937716007233, 'logits/chosen': -0.5411754846572876, 'nll_loss': 2.1769025325775146, 'log_odds_ratio': -0.5445480942726135, 'log_odds_chosen': 0.3507523834705353, 'epoch': 0.12} {'loss': 2.1234, 'grad_norm': 0.5997647643089294, 'learning_rate': 9.988628023879881e-07, 'rewards/chosen': -0.19727663695812225, 'rewards/rejected': -0.254878431558609, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05760182440280914, 'logps/rejected': -2.5487842559814453, 'logps/chosen': -1.972766399383545, 'logits/rejected': -0.3857523500919342, 'logits/chosen': -0.6800805926322937, 'nll_loss': 2.080449342727661, 'log_odds_ratio': -0.42948102951049805, 'log_odds_chosen': 0.6433253884315491, 'epoch': 0.12} {'loss': 2.0084, 'grad_norm': 0.5174111127853394, 'learning_rate': 9.985148530977764e-07, 'rewards/chosen': -0.18502694368362427, 'rewards/rejected': -0.2338823527097702, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04885541647672653, 'logps/rejected': -2.3388235569000244, 'logps/chosen': -1.8502693176269531, 'logits/rejected': -0.14804458618164062, 'logits/chosen': -0.5617026090621948, 'nll_loss': 1.9625658988952637, 'log_odds_ratio': -0.4585229754447937, 'log_odds_chosen': 0.5635263919830322, 'epoch': 0.13} {'loss': 2.2222, 'grad_norm': 0.5050185322761536, 'learning_rate': 9.981206082534287e-07, 'rewards/chosen': -0.20518088340759277, 'rewards/rejected': -0.24147498607635498, 'rewards/accuracies': 0.875, 'rewards/margins': 0.036294080317020416, 'logps/rejected': -2.41474986076355, 'logps/chosen': -2.0518088340759277, 'logits/rejected': -0.16735760867595673, 'logits/chosen': -0.4673263728618622, 'nll_loss': 2.1696157455444336, 'log_odds_ratio': -0.525580883026123, 'log_odds_chosen': 0.4026612937450409, 'epoch': 0.13} {'loss': 2.29, 'grad_norm': 0.6147828102111816, 'learning_rate': 9.976801044672607e-07, 'rewards/chosen': -0.21270211040973663, 'rewards/rejected': -0.23001064360141754, 'rewards/accuracies': 0.875, 'rewards/margins': 0.0173085518181324, 'logps/rejected': -2.3001065254211426, 'logps/chosen': -2.127021074295044, 'logits/rejected': -0.11841636896133423, 'logits/chosen': -0.4836263060569763, 'nll_loss': 2.2268035411834717, 'log_odds_ratio': -0.6318742632865906, 'log_odds_chosen': 0.18149018287658691, 'epoch': 0.13} {'loss': 2.2133, 'grad_norm': 0.6266311407089233, 'learning_rate': 9.97193382647516e-07, 'rewards/chosen': -0.20640523731708527, 'rewards/rejected': -0.26410865783691406, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0577034056186676, 'logps/rejected': -2.6410865783691406, 'logps/chosen': -2.064052104949951, 'logits/rejected': -0.030999187380075455, 'logits/chosen': -0.5783393383026123, 'nll_loss': 2.169518232345581, 'log_odds_ratio': -0.43759316205978394, 'log_odds_chosen': 0.6347547769546509, 'epoch': 0.14} {'loss': 2.1689, 'grad_norm': 0.6744863390922546, 'learning_rate': 9.966604879945656e-07, 'rewards/chosen': -0.20237436890602112, 'rewards/rejected': -0.2534412443637848, 'rewards/accuracies': 0.875, 'rewards/margins': 0.05106687173247337, 'logps/rejected': -2.534412384033203, 'logps/chosen': -2.0237436294555664, 'logits/rejected': -0.027224615216255188, 'logits/chosen': -0.6663930416107178, 'nll_loss': 2.1217703819274902, 'log_odds_ratio': -0.47156062722206116, 'log_odds_chosen': 0.5657740831375122, 'epoch': 0.14} {'loss': 2.2087, 'grad_norm': 0.6870261430740356, 'learning_rate': 9.96081469996711e-07, 'rewards/chosen': -0.20443923771381378, 'rewards/rejected': -0.24769122898578644, 'rewards/accuracies': 1.0, 'rewards/margins': 0.043251991271972656, 'logps/rejected': -2.476912498474121, 'logps/chosen': -2.0443923473358154, 'logits/rejected': -0.31325966119766235, 'logits/chosen': -0.5758293271064758, 'nll_loss': 2.1601643562316895, 'log_odds_ratio': -0.48495402932167053, 'log_odds_chosen': 0.4837898910045624, 'epoch': 0.14} {'loss': 2.1486, 'grad_norm': 0.5728748440742493, 'learning_rate': 9.954563824255877e-07, 'rewards/chosen': -0.1991468220949173, 'rewards/rejected': -0.23732483386993408, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03817800432443619, 'logps/rejected': -2.3732481002807617, 'logps/chosen': -1.9914681911468506, 'logits/rejected': -0.1452914923429489, 'logits/chosen': -0.5012402534484863, 'nll_loss': 2.098066806793213, 'log_odds_ratio': -0.5058196783065796, 'log_odds_chosen': 0.43097832798957825, 'epoch': 0.14} {'loss': 2.136, 'grad_norm': 0.6349831819534302, 'learning_rate': 9.947852833311724e-07, 'rewards/chosen': -0.19578438997268677, 'rewards/rejected': -0.22938966751098633, 'rewards/accuracies': 0.875, 'rewards/margins': 0.033605292439460754, 'logps/rejected': -2.2938966751098633, 'logps/chosen': -1.957843542098999, 'logits/rejected': -0.23492318391799927, 'logits/chosen': -0.643978476524353, 'nll_loss': 2.083264112472534, 'log_odds_ratio': -0.526985228061676, 'log_odds_chosen': 0.3815571665763855, 'epoch': 0.15} {'loss': 2.1273, 'grad_norm': 0.5989128351211548, 'learning_rate': 9.94068235036391e-07, 'rewards/chosen': -0.19467462599277496, 'rewards/rejected': -0.22964024543762207, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034965626895427704, 'logps/rejected': -2.2964022159576416, 'logps/chosen': -1.9467459917068481, 'logits/rejected': -0.17834247648715973, 'logits/chosen': -0.6117141842842102, 'nll_loss': 2.0749144554138184, 'log_odds_ratio': -0.5237154364585876, 'log_odds_chosen': 0.39456459879875183, 'epoch': 0.15} {'loss': 2.1823, 'grad_norm': 0.5899688005447388, 'learning_rate': 9.933053041313323e-07, 'rewards/chosen': -0.2004241794347763, 'rewards/rejected': -0.2352052479982376, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03478105366230011, 'logps/rejected': -2.3520522117614746, 'logps/chosen': -2.004241704940796, 'logits/rejected': -0.11192372441291809, 'logits/chosen': -0.6809389591217041, 'nll_loss': 2.130378246307373, 'log_odds_ratio': -0.5195820927619934, 'log_odds_chosen': 0.3930458724498749, 'epoch': 0.15} {'loss': 2.1764, 'grad_norm': 0.7014657258987427, 'learning_rate': 9.924965614670628e-07, 'rewards/chosen': -0.2005104273557663, 'rewards/rejected': -0.26001468300819397, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05950424075126648, 'logps/rejected': -2.600146770477295, 'logps/chosen': -2.0051043033599854, 'logits/rejected': -0.11418459564447403, 'logits/chosen': -0.6904681324958801, 'nll_loss': 2.133692502975464, 'log_odds_ratio': -0.42668840289115906, 'log_odds_chosen': 0.6613426804542542, 'epoch': 0.15} {'loss': 2.0278, 'grad_norm': 0.5079272389411926, 'learning_rate': 9.916420821490472e-07, 'rewards/chosen': -0.18575115501880646, 'rewards/rejected': -0.2229008823633194, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03714972361922264, 'logps/rejected': -2.229008674621582, 'logps/chosen': -1.8575114011764526, 'logits/rejected': -0.05774221196770668, 'logits/chosen': -0.5487760901451111, 'nll_loss': 1.9763325452804565, 'log_odds_ratio': -0.5145986080169678, 'log_odds_chosen': 0.4238623380661011, 'epoch': 0.16} {'loss': 2.172, 'grad_norm': 0.6976705193519592, 'learning_rate': 9.90741945530174e-07, 'rewards/chosen': -0.20053252577781677, 'rewards/rejected': -0.2459195852279663, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04538704454898834, 'logps/rejected': -2.459195852279663, 'logps/chosen': -2.0053253173828125, 'logits/rejected': -0.12501533329486847, 'logits/chosen': -0.5908156037330627, 'nll_loss': 2.123382091522217, 'log_odds_ratio': -0.4859912693500519, 'log_odds_chosen': 0.5120624303817749, 'epoch': 0.16} {'loss': 2.1691, 'grad_norm': 0.6146124601364136, 'learning_rate': 9.89796235203386e-07, 'rewards/chosen': -0.1990443915128708, 'rewards/rejected': -0.22475537657737732, 'rewards/accuracies': 1.0, 'rewards/margins': 0.025710972025990486, 'logps/rejected': -2.247553825378418, 'logps/chosen': -1.9904438257217407, 'logits/rejected': -0.20498931407928467, 'logits/chosen': -0.5347757339477539, 'nll_loss': 2.1127238273620605, 'log_odds_ratio': -0.5638565421104431, 'log_odds_chosen': 0.29176169633865356, 'epoch': 0.16} {'loss': 2.1964, 'grad_norm': 0.5901596546173096, 'learning_rate': 9.888050389939172e-07, 'rewards/chosen': -0.20238588750362396, 'rewards/rejected': -0.2355508804321289, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03316497802734375, 'logps/rejected': -2.355509042739868, 'logps/chosen': -2.0238590240478516, 'logits/rejected': 0.09010139852762222, 'logits/chosen': -0.5264938473701477, 'nll_loss': 2.143430233001709, 'log_odds_ratio': -0.5292580723762512, 'log_odds_chosen': 0.37348029017448425, 'epoch': 0.17} {'loss': 2.0812, 'grad_norm': 0.5306107997894287, 'learning_rate': 9.877684489511365e-07, 'rewards/chosen': -0.19372530281543732, 'rewards/rejected': -0.2540232241153717, 'rewards/accuracies': 1.0, 'rewards/margins': 0.060297898948192596, 'logps/rejected': -2.540231943130493, 'logps/chosen': -1.9372531175613403, 'logits/rejected': -0.17208705842494965, 'logits/chosen': -0.5884420871734619, 'nll_loss': 2.0395634174346924, 'log_odds_ratio': -0.41606950759887695, 'log_odds_chosen': 0.6772121787071228, 'epoch': 0.17} {'loss': 2.2753, 'grad_norm': 0.708717942237854, 'learning_rate': 9.866865613400006e-07, 'rewards/chosen': -0.21111147105693817, 'rewards/rejected': -0.2476184070110321, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03650694340467453, 'logps/rejected': -2.476184129714966, 'logps/chosen': -2.111114740371704, 'logits/rejected': 0.13652700185775757, 'logits/chosen': -0.5416155457496643, 'nll_loss': 2.2235543727874756, 'log_odds_ratio': -0.51793372631073, 'log_odds_chosen': 0.40706658363342285, 'epoch': 0.17} {'loss': 2.1671, 'grad_norm': 0.5680522918701172, 'learning_rate': 9.855594766321122e-07, 'rewards/chosen': -0.20177365839481354, 'rewards/rejected': -0.24775256216526031, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04597889631986618, 'logps/rejected': -2.4775257110595703, 'logps/chosen': -2.0177366733551025, 'logits/rejected': -0.11401443928480148, 'logits/chosen': -0.5442768335342407, 'nll_loss': 2.1197335720062256, 'log_odds_ratio': -0.4737292230129242, 'log_odds_chosen': 0.5157344341278076, 'epoch': 0.17} {'loss': 2.1343, 'grad_norm': 0.5254662036895752, 'learning_rate': 9.843872994963912e-07, 'rewards/chosen': -0.19873185455799103, 'rewards/rejected': -0.250969260931015, 'rewards/accuracies': 1.0, 'rewards/margins': 0.052237413823604584, 'logps/rejected': -2.509692668914795, 'logps/chosen': -1.987318515777588, 'logits/rejected': -0.27343085408210754, 'logits/chosen': -0.6353936791419983, 'nll_loss': 2.089053153991699, 'log_odds_ratio': -0.4526694416999817, 'log_odds_chosen': 0.581967830657959, 'epoch': 0.18} {'loss': 2.0987, 'grad_norm': 0.5754149556159973, 'learning_rate': 9.831701387893532e-07, 'rewards/chosen': -0.19162523746490479, 'rewards/rejected': -0.22321158647537231, 'rewards/accuracies': 0.875, 'rewards/margins': 0.031586356461048126, 'logps/rejected': -2.2321159839630127, 'logps/chosen': -1.9162522554397583, 'logits/rejected': -0.029723022133111954, 'logits/chosen': -0.5239946842193604, 'nll_loss': 2.0453133583068848, 'log_odds_ratio': -0.5337969064712524, 'log_odds_chosen': 0.3633577227592468, 'epoch': 0.18} {'loss': 2.0493, 'grad_norm': 0.5621442198753357, 'learning_rate': 9.819081075450013e-07, 'rewards/chosen': -0.1882205605506897, 'rewards/rejected': -0.23634123802185059, 'rewards/accuracies': 0.875, 'rewards/margins': 0.048120684921741486, 'logps/rejected': -2.363412380218506, 'logps/chosen': -1.882205605506897, 'logits/rejected': -0.14726145565509796, 'logits/chosen': -0.7125363945960999, 'nll_loss': 2.000981092453003, 'log_odds_ratio': -0.4831118583679199, 'log_odds_chosen': 0.5473266839981079, 'epoch': 0.18} {'loss': 2.173, 'grad_norm': 0.5734560489654541, 'learning_rate': 9.80601322964329e-07, 'rewards/chosen': -0.20293188095092773, 'rewards/rejected': -0.2324599325656891, 'rewards/accuracies': 1.0, 'rewards/margins': 0.029528062790632248, 'logps/rejected': -2.324599504470825, 'logps/chosen': -2.0293185710906982, 'logits/rejected': 0.03345927596092224, 'logits/chosen': -0.5026360750198364, 'nll_loss': 2.1185953617095947, 'log_odds_ratio': -0.5444394946098328, 'log_odds_chosen': 0.33386579155921936, 'epoch': 0.19} {'loss': 2.239, 'grad_norm': 0.6155676245689392, 'learning_rate': 9.792499064044342e-07, 'rewards/chosen': -0.2048148363828659, 'rewards/rejected': -0.24466755986213684, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03985270857810974, 'logps/rejected': -2.4466750621795654, 'logps/chosen': -2.0481481552124023, 'logits/rejected': 0.08332100510597229, 'logits/chosen': -0.373102605342865, 'nll_loss': 2.188358783721924, 'log_odds_ratio': -0.5067868232727051, 'log_odds_chosen': 0.4463171362876892, 'epoch': 0.19} {'loss': 2.2866, 'grad_norm': 0.7794233560562134, 'learning_rate': 9.778539833672523e-07, 'rewards/chosen': -0.21268144249916077, 'rewards/rejected': -0.26152101159095764, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04883957654237747, 'logps/rejected': -2.6152100563049316, 'logps/chosen': -2.126814365386963, 'logits/rejected': 0.006563026458024979, 'logits/chosen': -0.6216379404067993, 'nll_loss': 2.2394604682922363, 'log_odds_ratio': -0.4709131121635437, 'log_odds_chosen': 0.5371280908584595, 'epoch': 0.19} {'loss': 2.2046, 'grad_norm': 0.55165696144104, 'learning_rate': 9.764136834878985e-07, 'rewards/chosen': -0.20604537427425385, 'rewards/rejected': -0.260577529668808, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05453217402100563, 'logps/rejected': -2.6057753562927246, 'logps/chosen': -2.0604536533355713, 'logits/rejected': -0.26690828800201416, 'logits/chosen': -0.4978086054325104, 'nll_loss': 2.159515619277954, 'log_odds_ratio': -0.4510645270347595, 'log_odds_chosen': 0.6038047075271606, 'epoch': 0.19} {'loss': 2.2189, 'grad_norm': 0.7500314116477966, 'learning_rate': 9.749291405226304e-07, 'rewards/chosen': -0.203950434923172, 'rewards/rejected': -0.24402251839637756, 'rewards/accuracies': 1.0, 'rewards/margins': 0.040072064846754074, 'logps/rejected': -2.440225124359131, 'logps/chosen': -2.0395045280456543, 'logits/rejected': -0.13603514432907104, 'logits/chosen': -0.5977927446365356, 'nll_loss': 2.1687891483306885, 'log_odds_ratio': -0.5012515187263489, 'log_odds_chosen': 0.44653454422950745, 'epoch': 0.2} {'loss': 2.1054, 'grad_norm': 0.7076852917671204, 'learning_rate': 9.734004923364256e-07, 'rewards/chosen': -0.19468611478805542, 'rewards/rejected': -0.24599000811576843, 'rewards/accuracies': 0.875, 'rewards/margins': 0.05130389332771301, 'logps/rejected': -2.459900140762329, 'logps/chosen': -1.9468610286712646, 'logits/rejected': -0.14227402210235596, 'logits/chosen': -0.5302853584289551, 'nll_loss': 2.0594077110290527, 'log_odds_ratio': -0.4596526026725769, 'log_odds_chosen': 0.5996865630149841, 'epoch': 0.2} {'loss': 2.2568, 'grad_norm': 0.6143236756324768, 'learning_rate': 9.718278808901796e-07, 'rewards/chosen': -0.21002618968486786, 'rewards/rejected': -0.2455887794494629, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03556258603930473, 'logps/rejected': -2.455887794494629, 'logps/chosen': -2.100261688232422, 'logits/rejected': -0.058280639350414276, 'logits/chosen': -0.43135014176368713, 'nll_loss': 2.205214738845825, 'log_odds_ratio': -0.5162422060966492, 'log_odds_chosen': 0.396225243806839, 'epoch': 0.2} {'loss': 2.1793, 'grad_norm': 0.519507110118866, 'learning_rate': 9.702114522275216e-07, 'rewards/chosen': -0.19940906763076782, 'rewards/rejected': -0.2350345253944397, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03562544658780098, 'logps/rejected': -2.3503451347351074, 'logps/chosen': -1.9940909147262573, 'logits/rejected': -0.1517806351184845, 'logits/chosen': -0.4134013056755066, 'nll_loss': 2.127819538116455, 'log_odds_ratio': -0.5151816010475159, 'log_odds_chosen': 0.4036938548088074, 'epoch': 0.2} {'loss': 2.2057, 'grad_norm': 0.782429575920105, 'learning_rate': 9.685513564612519e-07, 'rewards/chosen': -0.20245149731636047, 'rewards/rejected': -0.2466682344675064, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04421672224998474, 'logps/rejected': -2.4666824340820312, 'logps/chosen': -2.024515151977539, 'logits/rejected': -0.06243826448917389, 'logits/chosen': -0.6792095899581909, 'nll_loss': 2.1572229862213135, 'log_odds_ratio': -0.48521819710731506, 'log_odds_chosen': 0.49272727966308594, 'epoch': 0.21} {'loss': 2.2481, 'grad_norm': 0.6543245911598206, 'learning_rate': 9.66847747759402e-07, 'rewards/chosen': -0.2105741947889328, 'rewards/rejected': -0.25357770919799805, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04300351440906525, 'logps/rejected': -2.5357770919799805, 'logps/chosen': -2.1057422161102295, 'logits/rejected': -0.22477136552333832, 'logits/chosen': -0.4802244007587433, 'nll_loss': 2.1991918087005615, 'log_odds_ratio': -0.48933419585227966, 'log_odds_chosen': 0.47571614384651184, 'epoch': 0.21} {'loss': 2.2666, 'grad_norm': 0.6939597725868225, 'learning_rate': 9.651007843309163e-07, 'rewards/chosen': -0.2088371068239212, 'rewards/rejected': -0.24642688035964966, 'rewards/accuracies': 0.875, 'rewards/margins': 0.037589773535728455, 'logps/rejected': -2.464268922805786, 'logps/chosen': -2.0883710384368896, 'logits/rejected': -0.16751326620578766, 'logits/chosen': -0.6152012944221497, 'nll_loss': 2.2142295837402344, 'log_odds_ratio': -0.5240318775177002, 'log_odds_chosen': 0.41576477885246277, 'epoch': 0.21} {'loss': 2.266, 'grad_norm': 0.6390596628189087, 'learning_rate': 9.63310628410961e-07, 'rewards/chosen': -0.21064873039722443, 'rewards/rejected': -0.2456912249326706, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03504248708486557, 'logps/rejected': -2.456912040710449, 'logps/chosen': -2.106487274169922, 'logits/rejected': 0.048835985362529755, 'logits/chosen': -0.4170103073120117, 'nll_loss': 2.212411880493164, 'log_odds_ratio': -0.5355044007301331, 'log_odds_chosen': 0.3899291455745697, 'epoch': 0.22} {'loss': 2.1759, 'grad_norm': 0.5294399857521057, 'learning_rate': 9.614774462458572e-07, 'rewards/chosen': -0.19899196922779083, 'rewards/rejected': -0.24037230014801025, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04138034209609032, 'logps/rejected': -2.4037232398986816, 'logps/chosen': -1.989919662475586, 'logits/rejected': -0.08842866122722626, 'logits/chosen': -0.43251708149909973, 'nll_loss': 2.1269915103912354, 'log_odds_ratio': -0.4890137314796448, 'log_odds_chosen': 0.466531366109848, 'epoch': 0.22} {'loss': 2.1511, 'grad_norm': 0.6490361094474792, 'learning_rate': 9.596014080776421e-07, 'rewards/chosen': -0.197781041264534, 'rewards/rejected': -0.23293854296207428, 'rewards/accuracies': 1.0, 'rewards/margins': 0.035157497972249985, 'logps/rejected': -2.329385280609131, 'logps/chosen': -1.9778105020523071, 'logits/rejected': -0.10723835974931717, 'logits/chosen': -0.5656427145004272, 'nll_loss': 2.098690986633301, 'log_odds_ratio': -0.524588406085968, 'log_odds_chosen': 0.39948511123657227, 'epoch': 0.22} {'loss': 2.1555, 'grad_norm': 0.5923272967338562, 'learning_rate': 9.576826881282595e-07, 'rewards/chosen': -0.19872736930847168, 'rewards/rejected': -0.2328701764345169, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03414280712604523, 'logps/rejected': -2.3287017345428467, 'logps/chosen': -1.9872735738754272, 'logits/rejected': -0.004090195521712303, 'logits/chosen': -0.4615509808063507, 'nll_loss': 2.1033740043640137, 'log_odds_ratio': -0.521623432636261, 'log_odds_chosen': 0.38708534836769104, 'epoch': 0.22} {'loss': 2.2536, 'grad_norm': 0.6121114492416382, 'learning_rate': 9.55721464583379e-07, 'rewards/chosen': -0.20732098817825317, 'rewards/rejected': -0.24214079976081848, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03481980785727501, 'logps/rejected': -2.421408176422119, 'logps/chosen': -2.073209762573242, 'logits/rejected': -0.051301341503858566, 'logits/chosen': -0.5602201819419861, 'nll_loss': 2.2012758255004883, 'log_odds_ratio': -0.5230833888053894, 'log_odds_chosen': 0.3890568017959595, 'epoch': 0.23} {'loss': 2.1898, 'grad_norm': 0.6667964458465576, 'learning_rate': 9.53717919575851e-07, 'rewards/chosen': -0.20237311720848083, 'rewards/rejected': -0.2616180181503296, 'rewards/accuracies': 1.0, 'rewards/margins': 0.059244900941848755, 'logps/rejected': -2.616180181503296, 'logps/chosen': -2.023730993270874, 'logits/rejected': -0.1270349621772766, 'logits/chosen': -0.5727198123931885, 'nll_loss': 2.1473617553710938, 'log_odds_ratio': -0.42426633834838867, 'log_odds_chosen': 0.6576822400093079, 'epoch': 0.23} {'loss': 2.2156, 'grad_norm': 0.693790853023529, 'learning_rate': 9.516722391687902e-07, 'rewards/chosen': -0.2029997855424881, 'rewards/rejected': -0.2422809600830078, 'rewards/accuracies': 1.0, 'rewards/margins': 0.039281152188777924, 'logps/rejected': -2.422809362411499, 'logps/chosen': -2.0299980640411377, 'logits/rejected': -0.06375281512737274, 'logits/chosen': -0.5797449350357056, 'nll_loss': 2.165597915649414, 'log_odds_ratio': -0.500329315662384, 'log_odds_chosen': 0.44280722737312317, 'epoch': 0.23} {'loss': 2.0991, 'grad_norm': 0.5140150189399719, 'learning_rate': 9.495846133382972e-07, 'rewards/chosen': -0.19280238449573517, 'rewards/rejected': -0.24602143466472626, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05321904271841049, 'logps/rejected': -2.460214138031006, 'logps/chosen': -1.9280236959457397, 'logits/rejected': -0.1540786623954773, 'logits/chosen': -0.6015070080757141, 'nll_loss': 2.0539848804473877, 'log_odds_ratio': -0.45164743065834045, 'log_odds_chosen': 0.5956349968910217, 'epoch': 0.23} {'loss': 2.22, 'grad_norm': 0.6148126721382141, 'learning_rate': 9.474552359558165e-07, 'rewards/chosen': -0.20516757667064667, 'rewards/rejected': -0.25938117504119873, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05421356484293938, 'logps/rejected': -2.593811511993408, 'logps/chosen': -2.051675796508789, 'logits/rejected': -0.22703289985656738, 'logits/chosen': -0.42590850591659546, 'nll_loss': 2.1741926670074463, 'log_odds_ratio': -0.4580463767051697, 'log_odds_chosen': 0.5965796709060669, 'epoch': 0.24} {'loss': 2.1023, 'grad_norm': 0.5130142569541931, 'learning_rate': 9.452843047701322e-07, 'rewards/chosen': -0.19355332851409912, 'rewards/rejected': -0.24127590656280518, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04772257059812546, 'logps/rejected': -2.4127590656280518, 'logps/chosen': -1.9355332851409912, 'logits/rejected': -0.19751112163066864, 'logits/chosen': -0.4120473861694336, 'nll_loss': 2.0553040504455566, 'log_odds_ratio': -0.46959275007247925, 'log_odds_chosen': 0.5396848917007446, 'epoch': 0.24} {'loss': 2.0608, 'grad_norm': 0.5595256686210632, 'learning_rate': 9.430720213890029e-07, 'rewards/chosen': -0.18794748187065125, 'rewards/rejected': -0.24248260259628296, 'rewards/accuracies': 0.75, 'rewards/margins': 0.05453511327505112, 'logps/rejected': -2.42482590675354, 'logps/chosen': -1.8794748783111572, 'logits/rejected': -0.12474207580089569, 'logits/chosen': -0.6118651032447815, 'nll_loss': 2.0134880542755127, 'log_odds_ratio': -0.4729953408241272, 'log_odds_chosen': 0.613072395324707, 'epoch': 0.24} {'loss': 2.1261, 'grad_norm': 0.4620777368545532, 'learning_rate': 9.408185912604394e-07, 'rewards/chosen': -0.1959434449672699, 'rewards/rejected': -0.2544253170490265, 'rewards/accuracies': 1.0, 'rewards/margins': 0.058481886982917786, 'logps/rejected': -2.544253349304199, 'logps/chosen': -1.9594345092773438, 'logits/rejected': -0.2336420714855194, 'logits/chosen': -0.4233023524284363, 'nll_loss': 2.083322286605835, 'log_odds_ratio': -0.4278123378753662, 'log_odds_chosen': 0.6550320982933044, 'epoch': 0.25} {'loss': 2.147, 'grad_norm': 0.5894656181335449, 'learning_rate': 9.385242236536259e-07, 'rewards/chosen': -0.19820062816143036, 'rewards/rejected': -0.25223690271377563, 'rewards/accuracies': 1.0, 'rewards/margins': 0.054036274552345276, 'logps/rejected': -2.522369146347046, 'logps/chosen': -1.9820064306259155, 'logits/rejected': 0.0337841734290123, 'logits/chosen': -0.588950514793396, 'nll_loss': 2.1032943725585938, 'log_odds_ratio': -0.43741244077682495, 'log_odds_chosen': 0.6060933470726013, 'epoch': 0.25} {'loss': 2.1238, 'grad_norm': 0.5386000275611877, 'learning_rate': 9.36189131639485e-07, 'rewards/chosen': -0.1960836499929428, 'rewards/rejected': -0.2363474816083908, 'rewards/accuracies': 0.875, 'rewards/margins': 0.040263839066028595, 'logps/rejected': -2.3634746074676514, 'logps/chosen': -1.960836410522461, 'logits/rejected': -0.03825013339519501, 'logits/chosen': -0.4553697109222412, 'nll_loss': 2.0735952854156494, 'log_odds_ratio': -0.5021163821220398, 'log_odds_chosen': 0.4552825093269348, 'epoch': 0.25} {'loss': 2.1638, 'grad_norm': 0.5585065484046936, 'learning_rate': 9.338135320708911e-07, 'rewards/chosen': -0.20036004483699799, 'rewards/rejected': -0.24209626019001007, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04173620790243149, 'logps/rejected': -2.4209625720977783, 'logps/chosen': -2.0036003589630127, 'logits/rejected': -0.3970990478992462, 'logits/chosen': -0.4344261884689331, 'nll_loss': 2.1146421432495117, 'log_odds_ratio': -0.4915333390235901, 'log_odds_chosen': 0.4675421714782715, 'epoch': 0.25} {'loss': 2.1803, 'grad_norm': 0.6691899299621582, 'learning_rate': 9.313976455625314e-07, 'rewards/chosen': -0.20110921561717987, 'rewards/rejected': -0.24887408316135406, 'rewards/accuracies': 0.875, 'rewards/margins': 0.047764845192432404, 'logps/rejected': -2.4887406826019287, 'logps/chosen': -2.011092185974121, 'logits/rejected': -0.08899018168449402, 'logits/chosen': -0.5215790867805481, 'nll_loss': 2.133345603942871, 'log_odds_ratio': -0.4698356091976166, 'log_odds_chosen': 0.5349908471107483, 'epoch': 0.26} {'loss': 2.0468, 'grad_norm': 0.5345549583435059, 'learning_rate': 9.289416964704185e-07, 'rewards/chosen': -0.18881818652153015, 'rewards/rejected': -0.23036324977874756, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0415450744330883, 'logps/rejected': -2.3036322593688965, 'logps/chosen': -1.8881816864013672, 'logits/rejected': -0.12612032890319824, 'logits/chosen': -0.646896481513977, 'nll_loss': 1.9974749088287354, 'log_odds_ratio': -0.4930804371833801, 'log_odds_chosen': 0.47327086329460144, 'epoch': 0.26} {'loss': 2.0786, 'grad_norm': 0.4902014434337616, 'learning_rate': 9.264459128710549e-07, 'rewards/chosen': -0.19082944095134735, 'rewards/rejected': -0.22408147156238556, 'rewards/accuracies': 1.0, 'rewards/margins': 0.033252015709877014, 'logps/rejected': -2.240814447402954, 'logps/chosen': -1.908294439315796, 'logits/rejected': -0.09607210755348206, 'logits/chosen': -0.4130203127861023, 'nll_loss': 2.02600359916687, 'log_odds_ratio': -0.5257529616355896, 'log_odds_chosen': 0.37959468364715576, 'epoch': 0.26} {'loss': 2.2571, 'grad_norm': 0.5475373268127441, 'learning_rate': 9.239105265402525e-07, 'rewards/chosen': -0.20786696672439575, 'rewards/rejected': -0.2350083291530609, 'rewards/accuracies': 1.0, 'rewards/margins': 0.027141369879245758, 'logps/rejected': -2.350083112716675, 'logps/chosen': -2.078669548034668, 'logits/rejected': -0.046746112406253815, 'logits/chosen': -0.44069942831993103, 'nll_loss': 2.2013750076293945, 'log_odds_ratio': -0.5576225519180298, 'log_odds_chosen': 0.30384886264801025, 'epoch': 0.27} {'loss': 2.1207, 'grad_norm': 0.547146737575531, 'learning_rate': 9.213357729316076e-07, 'rewards/chosen': -0.19446319341659546, 'rewards/rejected': -0.24830466508865356, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05384145677089691, 'logps/rejected': -2.483046770095825, 'logps/chosen': -1.9446320533752441, 'logits/rejected': -0.017504265531897545, 'logits/chosen': -0.3572068214416504, 'nll_loss': 2.075225353240967, 'log_odds_ratio': -0.455092191696167, 'log_odds_chosen': 0.5971243381500244, 'epoch': 0.27} {'loss': 2.1936, 'grad_norm': 0.6391997337341309, 'learning_rate': 9.187218911546361e-07, 'rewards/chosen': -0.20261992514133453, 'rewards/rejected': -0.24062804877758026, 'rewards/accuracies': 1.0, 'rewards/margins': 0.038008153438568115, 'logps/rejected': -2.406280517578125, 'logps/chosen': -2.0261991024017334, 'logits/rejected': -0.14603319764137268, 'logits/chosen': -0.4863595962524414, 'nll_loss': 2.142874002456665, 'log_odds_ratio': -0.5071159601211548, 'log_odds_chosen': 0.42829424142837524, 'epoch': 0.27} {'loss': 2.2289, 'grad_norm': 0.6325942277908325, 'learning_rate': 9.160691239525673e-07, 'rewards/chosen': -0.2076399028301239, 'rewards/rejected': -0.2607300281524658, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05309012532234192, 'logps/rejected': -2.6072998046875, 'logps/chosen': -2.076399087905884, 'logits/rejected': -0.2302314043045044, 'logits/chosen': -0.5181315541267395, 'nll_loss': 2.1842758655548096, 'log_odds_ratio': -0.44585251808166504, 'log_odds_chosen': 0.5877907276153564, 'epoch': 0.27} {'loss': 2.1386, 'grad_norm': 0.5501412749290466, 'learning_rate': 9.133777176798012e-07, 'rewards/chosen': -0.1947542428970337, 'rewards/rejected': -0.2268042117357254, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03204997628927231, 'logps/rejected': -2.2680420875549316, 'logps/chosen': -1.947542428970337, 'logits/rejected': -0.014902745373547077, 'logits/chosen': -0.586296021938324, 'nll_loss': 2.0854897499084473, 'log_odds_ratio': -0.531266987323761, 'log_odds_chosen': 0.3657677173614502, 'epoch': 0.28} {'loss': 2.2256, 'grad_norm': 0.5406647324562073, 'learning_rate': 9.10647922279031e-07, 'rewards/chosen': -0.2051105797290802, 'rewards/rejected': -0.23875710368156433, 'rewards/accuracies': 0.875, 'rewards/margins': 0.033646512776613235, 'logps/rejected': -2.387570858001709, 'logps/chosen': -2.0511059761047363, 'logits/rejected': -0.06091095507144928, 'logits/chosen': -0.3845860958099365, 'nll_loss': 2.172375440597534, 'log_odds_ratio': -0.5326068997383118, 'log_odds_chosen': 0.3731037974357605, 'epoch': 0.28} {'loss': 2.0868, 'grad_norm': 0.5214187502861023, 'learning_rate': 9.078799912580303e-07, 'rewards/chosen': -0.18962456285953522, 'rewards/rejected': -0.24267259240150452, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05304800719022751, 'logps/rejected': -2.426725387573242, 'logps/chosen': -1.8962457180023193, 'logits/rejected': -0.11495180428028107, 'logits/chosen': -0.5061873197555542, 'nll_loss': 2.0424931049346924, 'log_odds_ratio': -0.443120539188385, 'log_odds_chosen': 0.6036267280578613, 'epoch': 0.28} {'loss': 2.1286, 'grad_norm': 0.575564980506897, 'learning_rate': 9.050741816661126e-07, 'rewards/chosen': -0.19487696886062622, 'rewards/rejected': -0.2277163714170456, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03283938020467758, 'logps/rejected': -2.277163505554199, 'logps/chosen': -1.9487695693969727, 'logits/rejected': -0.16316528618335724, 'logits/chosen': -0.5737372040748596, 'nll_loss': 2.075375556945801, 'log_odds_ratio': -0.5326889157295227, 'log_odds_chosen': 0.372819721698761, 'epoch': 0.28} {'loss': 2.0482, 'grad_norm': 0.4972224533557892, 'learning_rate': 9.022307540702576e-07, 'rewards/chosen': -0.1859932690858841, 'rewards/rejected': -0.22202520072460175, 'rewards/accuracies': 1.0, 'rewards/margins': 0.036031924188137054, 'logps/rejected': -2.2202517986297607, 'logps/chosen': -1.859932780265808, 'logits/rejected': -0.024050669744610786, 'logits/chosen': -0.5097107291221619, 'nll_loss': 1.9969029426574707, 'log_odds_ratio': -0.512852668762207, 'log_odds_chosen': 0.41442954540252686, 'epoch': 0.29} {'loss': 2.0585, 'grad_norm': 0.4960035979747772, 'learning_rate': 8.993499725309148e-07, 'rewards/chosen': -0.18945816159248352, 'rewards/rejected': -0.23582102358341217, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04636286199092865, 'logps/rejected': -2.3582100868225098, 'logps/chosen': -1.8945815563201904, 'logits/rejected': -0.06721704453229904, 'logits/chosen': -0.42354345321655273, 'nll_loss': 2.011746644973755, 'log_odds_ratio': -0.4678184688091278, 'log_odds_chosen': 0.5264571309089661, 'epoch': 0.29} {'loss': 2.0632, 'grad_norm': 0.5499945282936096, 'learning_rate': 8.964321045774806e-07, 'rewards/chosen': -0.19003938138484955, 'rewards/rejected': -0.24120232462882996, 'rewards/accuracies': 0.875, 'rewards/margins': 0.05116293579339981, 'logps/rejected': -2.4120233058929443, 'logps/chosen': -1.9003938436508179, 'logits/rejected': -0.043670542538166046, 'logits/chosen': -0.48419511318206787, 'nll_loss': 2.017177104949951, 'log_odds_ratio': -0.46018439531326294, 'log_odds_chosen': 0.5790612101554871, 'epoch': 0.29} {'loss': 2.1426, 'grad_norm': 0.5568175911903381, 'learning_rate': 8.934774211834537e-07, 'rewards/chosen': -0.1978786587715149, 'rewards/rejected': -0.23785711824893951, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03997847065329552, 'logps/rejected': -2.3785712718963623, 'logps/chosen': -1.9787864685058594, 'logits/rejected': -0.08241380751132965, 'logits/chosen': -0.4519391357898712, 'nll_loss': 2.0928850173950195, 'log_odds_ratio': -0.4972607493400574, 'log_odds_chosen': 0.45170602202415466, 'epoch': 0.3} {'loss': 2.1647, 'grad_norm': 0.5856341123580933, 'learning_rate': 8.904861967412701e-07, 'rewards/chosen': -0.2012743055820465, 'rewards/rejected': -0.24806730449199677, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04679298400878906, 'logps/rejected': -2.480672836303711, 'logps/chosen': -2.0127432346343994, 'logits/rejected': -0.13722169399261475, 'logits/chosen': -0.4344129264354706, 'nll_loss': 2.1175570487976074, 'log_odds_ratio': -0.47116899490356445, 'log_odds_chosen': 0.5225868821144104, 'epoch': 0.3} {'loss': 2.1829, 'grad_norm': 0.6680328845977783, 'learning_rate': 8.87458709036822e-07, 'rewards/chosen': -0.20367543399333954, 'rewards/rejected': -0.26599156856536865, 'rewards/accuracies': 1.0, 'rewards/margins': 0.06231612712144852, 'logps/rejected': -2.6599154472351074, 'logps/chosen': -2.0367543697357178, 'logits/rejected': -0.2857436239719391, 'logits/chosen': -0.5091022849082947, 'nll_loss': 2.1406257152557373, 'log_odds_ratio': -0.4224277436733246, 'log_odds_chosen': 0.6864393949508667, 'epoch': 0.3} {'loss': 2.1606, 'grad_norm': 0.5433963537216187, 'learning_rate': 8.843952392236593e-07, 'rewards/chosen': -0.19623397290706635, 'rewards/rejected': -0.225574791431427, 'rewards/accuracies': 0.75, 'rewards/margins': 0.029340829700231552, 'logps/rejected': -2.2557480335235596, 'logps/chosen': -1.9623396396636963, 'logits/rejected': -0.14656810462474823, 'logits/chosen': -0.36713624000549316, 'nll_loss': 2.1045403480529785, 'log_odds_ratio': -0.5607271790504456, 'log_odds_chosen': 0.33481231331825256, 'epoch': 0.3} {'loss': 2.043, 'grad_norm': 0.5330284833908081, 'learning_rate': 8.812960717968818e-07, 'rewards/chosen': -0.18650837242603302, 'rewards/rejected': -0.2264942228794098, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03998585417866707, 'logps/rejected': -2.264942169189453, 'logps/chosen': -1.8650836944580078, 'logits/rejected': -0.11987907439470291, 'logits/chosen': -0.4940492808818817, 'nll_loss': 1.9932986497879028, 'log_odds_ratio': -0.4966501295566559, 'log_odds_chosen': 0.46152007579803467, 'epoch': 0.31} {'loss': 2.0869, 'grad_norm': 0.47368451952934265, 'learning_rate': 8.781614945667168e-07, 'rewards/chosen': -0.1914043426513672, 'rewards/rejected': -0.23168428242206573, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04027996212244034, 'logps/rejected': -2.316842794418335, 'logps/chosen': -1.9140433073043823, 'logits/rejected': -0.09748263657093048, 'logits/chosen': -0.3089265525341034, 'nll_loss': 2.0364115238189697, 'log_odds_ratio': -0.5047125220298767, 'log_odds_chosen': 0.4548957347869873, 'epoch': 0.31} {'loss': 2.1749, 'grad_norm': 0.6315337419509888, 'learning_rate': 8.749917986317928e-07, 'rewards/chosen': -0.1994360387325287, 'rewards/rejected': -0.23994943499565125, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04051339626312256, 'logps/rejected': -2.399494171142578, 'logps/chosen': -1.9943602085113525, 'logits/rejected': 0.00903545506298542, 'logits/chosen': -0.5219292640686035, 'nll_loss': 2.125378131866455, 'log_odds_ratio': -0.49500221014022827, 'log_odds_chosen': 0.45678791403770447, 'epoch': 0.31} {'loss': 2.1542, 'grad_norm': 0.5757876038551331, 'learning_rate': 8.717872783521047e-07, 'rewards/chosen': -0.1979493647813797, 'rewards/rejected': -0.2500247061252594, 'rewards/accuracies': 0.875, 'rewards/margins': 0.05207531899213791, 'logps/rejected': -2.500247001647949, 'logps/chosen': -1.9794937372207642, 'logits/rejected': -0.1346980631351471, 'logits/chosen': -0.38971248269081116, 'nll_loss': 2.1087803840637207, 'log_odds_ratio': -0.45457175374031067, 'log_odds_chosen': 0.583101749420166, 'epoch': 0.31} {'loss': 2.1491, 'grad_norm': 0.4948897957801819, 'learning_rate': 8.685482313216783e-07, 'rewards/chosen': -0.19936421513557434, 'rewards/rejected': -0.23769225180149078, 'rewards/accuracies': 1.0, 'rewards/margins': 0.038328029215335846, 'logps/rejected': -2.376922607421875, 'logps/chosen': -1.9936423301696777, 'logits/rejected': -0.16718950867652893, 'logits/chosen': -0.2275230586528778, 'nll_loss': 2.0988006591796875, 'log_odds_ratio': -0.5028841495513916, 'log_odds_chosen': 0.43383458256721497, 'epoch': 0.32} {'loss': 2.1521, 'grad_norm': 0.5214484930038452, 'learning_rate': 8.652749583409339e-07, 'rewards/chosen': -0.19792446494102478, 'rewards/rejected': -0.23307493329048157, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03515046834945679, 'logps/rejected': -2.330749273300171, 'logps/chosen': -1.979244589805603, 'logits/rejected': -0.07613816857337952, 'logits/chosen': -0.3641853928565979, 'nll_loss': 2.0998451709747314, 'log_odds_ratio': -0.522826075553894, 'log_odds_chosen': 0.40033382177352905, 'epoch': 0.32} {'loss': 2.233, 'grad_norm': 0.5448740124702454, 'learning_rate': 8.619677633887508e-07, 'rewards/chosen': -0.2061443328857422, 'rewards/rejected': -0.23936225473880768, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03321794420480728, 'logps/rejected': -2.393622636795044, 'logps/chosen': -2.061443328857422, 'logits/rejected': -0.17828039824962616, 'logits/chosen': -0.3629532754421234, 'nll_loss': 2.1793932914733887, 'log_odds_ratio': -0.5365285277366638, 'log_odds_chosen': 0.37107452750205994, 'epoch': 0.32} {'loss': 2.1266, 'grad_norm': 0.5429673194885254, 'learning_rate': 8.586269535942385e-07, 'rewards/chosen': -0.19498948752880096, 'rewards/rejected': -0.22681477665901184, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03182529658079147, 'logps/rejected': -2.2681477069854736, 'logps/chosen': -1.9498947858810425, 'logits/rejected': -0.4250182509422302, 'logits/chosen': -0.5306198596954346, 'nll_loss': 2.0730719566345215, 'log_odds_ratio': -0.535487949848175, 'log_odds_chosen': 0.36125749349594116, 'epoch': 0.33} {'loss': 2.2909, 'grad_norm': 0.6338194012641907, 'learning_rate': 8.552528392082146e-07, 'rewards/chosen': -0.21171432733535767, 'rewards/rejected': -0.25223207473754883, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04051773250102997, 'logps/rejected': -2.522320508956909, 'logps/chosen': -2.117143154144287, 'logits/rejected': -0.0979476124048233, 'logits/chosen': -0.2504444420337677, 'nll_loss': 2.2413687705993652, 'log_odds_ratio': -0.4954701364040375, 'log_odds_chosen': 0.45459601283073425, 'epoch': 0.33} {'loss': 2.0269, 'grad_norm': 0.48553842306137085, 'learning_rate': 8.518457335743924e-07, 'rewards/chosen': -0.18635164201259613, 'rewards/rejected': -0.235719695687294, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04936806112527847, 'logps/rejected': -2.3571970462799072, 'logps/chosen': -1.8635163307189941, 'logits/rejected': -0.02024952881038189, 'logits/chosen': -0.4194352924823761, 'nll_loss': 1.981484293937683, 'log_odds_ratio': -0.4539083242416382, 'log_odds_chosen': 0.5631441473960876, 'epoch': 0.33} {'loss': 2.0688, 'grad_norm': 0.5075821280479431, 'learning_rate': 8.48405953100282e-07, 'rewards/chosen': -0.19163572788238525, 'rewards/rejected': -0.23868180811405182, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04704609513282776, 'logps/rejected': -2.3868184089660645, 'logps/chosen': -1.9163572788238525, 'logits/rejected': -0.09555709362030029, 'logits/chosen': -0.4761042892932892, 'nll_loss': 2.022080898284912, 'log_odds_ratio': -0.46748292446136475, 'log_odds_chosen': 0.5342307686805725, 'epoch': 0.33} {'loss': 2.1379, 'grad_norm': 0.4722713232040405, 'learning_rate': 8.449338172278058e-07, 'rewards/chosen': -0.195204496383667, 'rewards/rejected': -0.23235377669334412, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03714926540851593, 'logps/rejected': -2.3235373497009277, 'logps/chosen': -1.95204496383667, 'logits/rejected': -0.14001092314720154, 'logits/chosen': -0.3123060464859009, 'nll_loss': 2.0863630771636963, 'log_odds_ratio': -0.5151209235191345, 'log_odds_chosen': 0.42079102993011475, 'epoch': 0.34} {'loss': 2.2738, 'grad_norm': 0.563771665096283, 'learning_rate': 8.414296484036338e-07, 'rewards/chosen': -0.2117709368467331, 'rewards/rejected': -0.2364516258239746, 'rewards/accuracies': 1.0, 'rewards/margins': 0.02468070387840271, 'logps/rejected': -2.364516496658325, 'logps/chosen': -2.1177093982696533, 'logits/rejected': -0.04408363625407219, 'logits/chosen': -0.3733835816383362, 'nll_loss': 2.2168140411376953, 'log_odds_ratio': -0.5697895884513855, 'log_odds_chosen': 0.27497193217277527, 'epoch': 0.34} {'loss': 2.2261, 'grad_norm': 0.716310977935791, 'learning_rate': 8.378937720492383e-07, 'rewards/chosen': -0.2059776484966278, 'rewards/rejected': -0.24121545255184174, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03523779660463333, 'logps/rejected': -2.4121546745300293, 'logps/chosen': -2.059776544570923, 'logits/rejected': 0.044096484780311584, 'logits/chosen': -0.6494585275650024, 'nll_loss': 2.173640727996826, 'log_odds_ratio': -0.5247021317481995, 'log_odds_chosen': 0.3946109414100647, 'epoch': 0.34} {'loss': 2.0951, 'grad_norm': 0.5385156273841858, 'learning_rate': 8.343265165306734e-07, 'rewards/chosen': -0.19126594066619873, 'rewards/rejected': -0.2468159794807434, 'rewards/accuracies': 1.0, 'rewards/margins': 0.055550046265125275, 'logps/rejected': -2.4681596755981445, 'logps/chosen': -1.9126592874526978, 'logits/rejected': -0.38548457622528076, 'logits/chosen': -0.4387221932411194, 'nll_loss': 2.0500893592834473, 'log_odds_ratio': -0.44978296756744385, 'log_odds_chosen': 0.6251612901687622, 'epoch': 0.35} {'loss': 2.1001, 'grad_norm': 0.545859158039093, 'learning_rate': 8.307282131280804e-07, 'rewards/chosen': -0.19255836308002472, 'rewards/rejected': -0.2445870041847229, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05202864110469818, 'logps/rejected': -2.4458699226379395, 'logps/chosen': -1.9255836009979248, 'logits/rejected': -0.2851545214653015, 'logits/chosen': -0.44923073053359985, 'nll_loss': 2.0555202960968018, 'log_odds_ratio': -0.44584840536117554, 'log_odds_chosen': 0.5865803956985474, 'epoch': 0.35} {'loss': 2.2761, 'grad_norm': 0.5951908826828003, 'learning_rate': 8.27099196004923e-07, 'rewards/chosen': -0.2090090662240982, 'rewards/rejected': -0.23857367038726807, 'rewards/accuracies': 0.875, 'rewards/margins': 0.02956460230052471, 'logps/rejected': -2.3857364654541016, 'logps/chosen': -2.09009051322937, 'logits/rejected': 0.10464876145124435, 'logits/chosen': -0.31916385889053345, 'nll_loss': 2.220856189727783, 'log_odds_ratio': -0.5520588159561157, 'log_odds_chosen': 0.3327331244945526, 'epoch': 0.35} {'loss': 2.1149, 'grad_norm': 0.5234466195106506, 'learning_rate': 8.23439802176954e-07, 'rewards/chosen': -0.19356943666934967, 'rewards/rejected': -0.2396468222141266, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04607734829187393, 'logps/rejected': -2.396468162536621, 'logps/chosen': -1.9356945753097534, 'logits/rejected': -0.07198012620210648, 'logits/chosen': -0.3624882400035858, 'nll_loss': 2.0666897296905518, 'log_odds_ratio': -0.48196426033973694, 'log_odds_chosen': 0.5177618861198425, 'epoch': 0.35} {'loss': 2.1926, 'grad_norm': 0.5753765106201172, 'learning_rate': 8.19750371480919e-07, 'rewards/chosen': -0.2017781138420105, 'rewards/rejected': -0.23294886946678162, 'rewards/accuracies': 0.75, 'rewards/margins': 0.031170757487416267, 'logps/rejected': -2.329488754272461, 'logps/chosen': -2.0177810192108154, 'logits/rejected': -0.013083010911941528, 'logits/chosen': -0.4609984755516052, 'nll_loss': 2.1379547119140625, 'log_odds_ratio': -0.5461571216583252, 'log_odds_chosen': 0.35112565755844116, 'epoch': 0.36} {'loss': 2.1233, 'grad_norm': 0.5582403540611267, 'learning_rate': 8.160312465429952e-07, 'rewards/chosen': -0.194510355591774, 'rewards/rejected': -0.25350093841552734, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05899059772491455, 'logps/rejected': -2.5350096225738525, 'logps/chosen': -1.945103645324707, 'logits/rejected': -0.20930448174476624, 'logits/chosen': -0.46288540959358215, 'nll_loss': 2.0805039405822754, 'log_odds_ratio': -0.4275791049003601, 'log_odds_chosen': 0.6578480005264282, 'epoch': 0.36} {'loss': 2.1695, 'grad_norm': 0.589810848236084, 'learning_rate': 8.122827727469737e-07, 'rewards/chosen': -0.2011301964521408, 'rewards/rejected': -0.2541583776473999, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05302818864583969, 'logps/rejected': -2.541584014892578, 'logps/chosen': -2.0113017559051514, 'logits/rejected': -0.0018861368298530579, 'logits/chosen': -0.32802432775497437, 'nll_loss': 2.1246747970581055, 'log_odds_ratio': -0.44798266887664795, 'log_odds_chosen': 0.5904497504234314, 'epoch': 0.36} {'loss': 2.1143, 'grad_norm': 0.6079692840576172, 'learning_rate': 8.085052982021847e-07, 'rewards/chosen': -0.19396479427814484, 'rewards/rejected': -0.24543817341327667, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05147337168455124, 'logps/rejected': -2.4543814659118652, 'logps/chosen': -1.9396477937698364, 'logits/rejected': -0.2528253197669983, 'logits/chosen': -0.46243995428085327, 'nll_loss': 2.068951368331909, 'log_odds_ratio': -0.4538944363594055, 'log_odds_chosen': 0.5775200128555298, 'epoch': 0.36} {'loss': 2.138, 'grad_norm': 0.5014004111289978, 'learning_rate': 8.046991737111695e-07, 'rewards/chosen': -0.1962708979845047, 'rewards/rejected': -0.2392961084842682, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04302523285150528, 'logps/rejected': -2.392961263656616, 'logps/chosen': -1.9627089500427246, 'logits/rejected': -0.011914369650185108, 'logits/chosen': -0.42011186480522156, 'nll_loss': 2.089262008666992, 'log_odds_ratio': -0.48756104707717896, 'log_odds_chosen': 0.48234912753105164, 'epoch': 0.37} {'loss': 2.1428, 'grad_norm': 0.5740203261375427, 'learning_rate': 8.008647527371022e-07, 'rewards/chosen': -0.19665035605430603, 'rewards/rejected': -0.24090173840522766, 'rewards/accuracies': 1.0, 'rewards/margins': 0.044251397252082825, 'logps/rejected': -2.409017324447632, 'logps/chosen': -1.966503620147705, 'logits/rejected': -0.4352371096611023, 'logits/chosen': -0.5055060982704163, 'nll_loss': 2.0950794219970703, 'log_odds_ratio': -0.4775448441505432, 'log_odds_chosen': 0.5004888772964478, 'epoch': 0.37} {'loss': 2.0367, 'grad_norm': 0.44002586603164673, 'learning_rate': 7.970023913709651e-07, 'rewards/chosen': -0.1857527196407318, 'rewards/rejected': -0.2367796003818512, 'rewards/accuracies': 1.0, 'rewards/margins': 0.051026877015829086, 'logps/rejected': -2.367795944213867, 'logps/chosen': -1.857527256011963, 'logits/rejected': -0.261316180229187, 'logits/chosen': -0.29188770055770874, 'nll_loss': 1.99098539352417, 'log_odds_ratio': -0.45719826221466064, 'log_odds_chosen': 0.5827584862709045, 'epoch': 0.37} {'loss': 2.1561, 'grad_norm': 0.5160555243492126, 'learning_rate': 7.931124482984801e-07, 'rewards/chosen': -0.20039717853069305, 'rewards/rejected': -0.2447955459356308, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04439838230609894, 'logps/rejected': -2.44795560836792, 'logps/chosen': -2.003971576690674, 'logits/rejected': 0.061051055788993835, 'logits/chosen': -0.422038733959198, 'nll_loss': 2.1079189777374268, 'log_odds_ratio': -0.4820690155029297, 'log_odds_chosen': 0.49602457880973816, 'epoch': 0.38} {'loss': 2.037, 'grad_norm': 0.4655568599700928, 'learning_rate': 7.891952847667972e-07, 'rewards/chosen': -0.18628200888633728, 'rewards/rejected': -0.23243649303913116, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04615447670221329, 'logps/rejected': -2.3243649005889893, 'logps/chosen': -1.862820029258728, 'logits/rejected': -0.14115437865257263, 'logits/chosen': -0.3394045829772949, 'nll_loss': 1.9898964166641235, 'log_odds_ratio': -0.4714434742927551, 'log_odds_chosen': 0.5271226763725281, 'epoch': 0.38} {'loss': 2.1244, 'grad_norm': 0.4939066469669342, 'learning_rate': 7.85251264550948e-07, 'rewards/chosen': -0.195757195353508, 'rewards/rejected': -0.2314586043357849, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03570141643285751, 'logps/rejected': -2.3145861625671387, 'logps/chosen': -1.9575718641281128, 'logits/rejected': -0.1826004981994629, 'logits/chosen': -0.3042515516281128, 'nll_loss': 2.0729963779449463, 'log_odds_ratio': -0.5139645934104919, 'log_odds_chosen': 0.4060474634170532, 'epoch': 0.38} {'loss': 2.206, 'grad_norm': 0.5114497542381287, 'learning_rate': 7.812807539200621e-07, 'rewards/chosen': -0.20425887405872345, 'rewards/rejected': -0.2417077273130417, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03744884580373764, 'logps/rejected': -2.4170773029327393, 'logps/chosen': -2.042588710784912, 'logits/rejected': -0.031729258596897125, 'logits/chosen': -0.24258123338222504, 'nll_loss': 2.1546831130981445, 'log_odds_ratio': -0.5130881667137146, 'log_odds_chosen': 0.42092645168304443, 'epoch': 0.38} {'loss': 2.0275, 'grad_norm': 0.45012837648391724, 'learning_rate': 7.772841216033532e-07, 'rewards/chosen': -0.18509985506534576, 'rewards/rejected': -0.22159941494464874, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03649955242872238, 'logps/rejected': -2.215993881225586, 'logps/chosen': -1.8509984016418457, 'logits/rejected': -0.06629861146211624, 'logits/chosen': -0.43615201115608215, 'nll_loss': 1.9760750532150269, 'log_odds_ratio': -0.5143554210662842, 'log_odds_chosen': 0.417616069316864, 'epoch': 0.39} {'loss': 2.0445, 'grad_norm': 0.4387165904045105, 'learning_rate': 7.732617387558768e-07, 'rewards/chosen': -0.18875840306282043, 'rewards/rejected': -0.23721879720687866, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04846040904521942, 'logps/rejected': -2.372188091278076, 'logps/chosen': -1.8875839710235596, 'logits/rejected': -0.32880979776382446, 'logits/chosen': -0.30067747831344604, 'nll_loss': 1.997605800628662, 'log_odds_ratio': -0.469321608543396, 'log_odds_chosen': 0.5448954701423645, 'epoch': 0.39} {'loss': 2.0458, 'grad_norm': 0.5102280378341675, 'learning_rate': 7.69213978924061e-07, 'rewards/chosen': -0.18841463327407837, 'rewards/rejected': -0.24185076355934143, 'rewards/accuracies': 0.875, 'rewards/margins': 0.053436145186424255, 'logps/rejected': -2.4185075759887695, 'logps/chosen': -1.8841460943222046, 'logits/rejected': -0.1547757089138031, 'logits/chosen': -0.5158487558364868, 'nll_loss': 2.0015206336975098, 'log_odds_ratio': -0.4432864785194397, 'log_odds_chosen': 0.6055910587310791, 'epoch': 0.39} {'loss': 2.0609, 'grad_norm': 0.5141972899436951, 'learning_rate': 7.651412180110175e-07, 'rewards/chosen': -0.1856735795736313, 'rewards/rejected': -0.2130412757396698, 'rewards/accuracies': 1.0, 'rewards/margins': 0.027367670089006424, 'logps/rejected': -2.1304125785827637, 'logps/chosen': -1.8567359447479248, 'logits/rejected': -0.22623799741268158, 'logits/chosen': -0.4066213369369507, 'nll_loss': 2.00547456741333, 'log_odds_ratio': -0.554594099521637, 'log_odds_chosen': 0.3155019283294678, 'epoch': 0.39} {'loss': 2.1363, 'grad_norm': 0.5515993237495422, 'learning_rate': 7.610438342416319e-07, 'rewards/chosen': -0.1983896642923355, 'rewards/rejected': -0.24396765232086182, 'rewards/accuracies': 1.0, 'rewards/margins': 0.045577969402074814, 'logps/rejected': -2.439676284790039, 'logps/chosen': -1.9838966131210327, 'logits/rejected': -0.2101307213306427, 'logits/chosen': -0.5050479769706726, 'nll_loss': 2.088998556137085, 'log_odds_ratio': -0.47270989418029785, 'log_odds_chosen': 0.5125759840011597, 'epoch': 0.4} {'loss': 2.1104, 'grad_norm': 0.6079866886138916, 'learning_rate': 7.569222081274395e-07, 'rewards/chosen': -0.19400662183761597, 'rewards/rejected': -0.24247224628925323, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04846561700105667, 'logps/rejected': -2.424722194671631, 'logps/chosen': -1.9400660991668701, 'logits/rejected': -0.051476310938596725, 'logits/chosen': -0.6153779029846191, 'nll_loss': 2.0644783973693848, 'log_odds_ratio': -0.4596479535102844, 'log_odds_chosen': 0.5498825311660767, 'epoch': 0.4} {'loss': 2.0674, 'grad_norm': 0.4408237040042877, 'learning_rate': 7.527767224312882e-07, 'rewards/chosen': -0.1905336081981659, 'rewards/rejected': -0.23589414358139038, 'rewards/accuracies': 1.0, 'rewards/margins': 0.045360542833805084, 'logps/rejected': -2.3589413166046143, 'logps/chosen': -1.9053359031677246, 'logits/rejected': -0.06426948308944702, 'logits/chosen': -0.42787209153175354, 'nll_loss': 2.0203795433044434, 'log_odds_ratio': -0.4705893397331238, 'log_odds_chosen': 0.5145049095153809, 'epoch': 0.4} {'loss': 2.16, 'grad_norm': 0.4828256070613861, 'learning_rate': 7.486077621317925e-07, 'rewards/chosen': -0.20035697519779205, 'rewards/rejected': -0.24468713998794556, 'rewards/accuracies': 1.0, 'rewards/margins': 0.044330157339572906, 'logps/rejected': -2.446871280670166, 'logps/chosen': -2.0035698413848877, 'logits/rejected': -0.15828007459640503, 'logits/chosen': -0.2916801869869232, 'nll_loss': 2.1111345291137695, 'log_odds_ratio': -0.488892138004303, 'log_odds_chosen': 0.49437594413757324, 'epoch': 0.41} {'loss': 1.9515, 'grad_norm': 0.46756893396377563, 'learning_rate': 7.444157143875819e-07, 'rewards/chosen': -0.17716486752033234, 'rewards/rejected': -0.24373489618301392, 'rewards/accuracies': 1.0, 'rewards/margins': 0.06657001376152039, 'logps/rejected': -2.4373488426208496, 'logps/chosen': -1.7716487646102905, 'logits/rejected': -0.38097241520881653, 'logits/chosen': -0.6170058250427246, 'nll_loss': 1.9118363857269287, 'log_odds_ratio': -0.3968402147293091, 'log_odds_chosen': 0.7566857933998108, 'epoch': 0.41} {'loss': 2.0902, 'grad_norm': 0.5957549810409546, 'learning_rate': 7.402009685013462e-07, 'rewards/chosen': -0.19221307337284088, 'rewards/rejected': -0.2370673269033432, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04485424980521202, 'logps/rejected': -2.370673418045044, 'logps/chosen': -1.9221307039260864, 'logits/rejected': -0.10341174900531769, 'logits/chosen': -0.46232783794403076, 'nll_loss': 2.0427207946777344, 'log_odds_ratio': -0.4749317467212677, 'log_odds_chosen': 0.5112757682800293, 'epoch': 0.41} {'loss': 2.2278, 'grad_norm': 0.5625369548797607, 'learning_rate': 7.359639158836827e-07, 'rewards/chosen': -0.2085065394639969, 'rewards/rejected': -0.2557412385940552, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04723470285534859, 'logps/rejected': -2.557412624359131, 'logps/chosen': -2.0850653648376465, 'logits/rejected': -0.08281365036964417, 'logits/chosen': -0.2808252274990082, 'nll_loss': 2.180760383605957, 'log_odds_ratio': -0.4707946479320526, 'log_odds_chosen': 0.5237774848937988, 'epoch': 0.41} {'loss': 2.1062, 'grad_norm': 0.5004662871360779, 'learning_rate': 7.317049500167465e-07, 'rewards/chosen': -0.19358593225479126, 'rewards/rejected': -0.23361001908779144, 'rewards/accuracies': 1.0, 'rewards/margins': 0.040024079382419586, 'logps/rejected': -2.3361001014709473, 'logps/chosen': -1.935859203338623, 'logits/rejected': -0.08140935748815536, 'logits/chosen': -0.29819634556770325, 'nll_loss': 2.056227922439575, 'log_odds_ratio': -0.4992949068546295, 'log_odds_chosen': 0.4554722011089325, 'epoch': 0.42} {'loss': 2.137, 'grad_norm': 0.5074776411056519, 'learning_rate': 7.274244664177097e-07, 'rewards/chosen': -0.19829802215099335, 'rewards/rejected': -0.24422214925289154, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04592413455247879, 'logps/rejected': -2.4422216415405273, 'logps/chosen': -1.9829801321029663, 'logits/rejected': -0.24845655262470245, 'logits/chosen': -0.31135305762290955, 'nll_loss': 2.0889265537261963, 'log_odds_ratio': -0.4809737801551819, 'log_odds_chosen': 0.5130349397659302, 'epoch': 0.42} {'loss': 2.0708, 'grad_norm': 0.5020032525062561, 'learning_rate': 7.231228626020303e-07, 'rewards/chosen': -0.1908215582370758, 'rewards/rejected': -0.24285900592803955, 'rewards/accuracies': 1.0, 'rewards/margins': 0.052037447690963745, 'logps/rejected': -2.4285898208618164, 'logps/chosen': -1.9082156419754028, 'logits/rejected': -0.07742983847856522, 'logits/chosen': -0.360279381275177, 'nll_loss': 2.026240348815918, 'log_odds_ratio': -0.4452877938747406, 'log_odds_chosen': 0.5971647500991821, 'epoch': 0.42} {'loss': 2.0629, 'grad_norm': 0.4719047248363495, 'learning_rate': 7.188005380465364e-07, 'rewards/chosen': -0.18679876625537872, 'rewards/rejected': -0.2189595103263855, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03216075152158737, 'logps/rejected': -2.1895949840545654, 'logps/chosen': -1.8679873943328857, 'logits/rejected': -0.17365439236164093, 'logits/chosen': -0.36417311429977417, 'nll_loss': 2.009333372116089, 'log_odds_ratio': -0.5354775190353394, 'log_odds_chosen': 0.3675041198730469, 'epoch': 0.43} {'loss': 2.1127, 'grad_norm': 0.5571091771125793, 'learning_rate': 7.144578941523282e-07, 'rewards/chosen': -0.19241367280483246, 'rewards/rejected': -0.24405038356781006, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0516367070376873, 'logps/rejected': -2.4405038356781006, 'logps/chosen': -1.9241366386413574, 'logits/rejected': -0.028011344373226166, 'logits/chosen': -0.34637928009033203, 'nll_loss': 2.0673089027404785, 'log_odds_ratio': -0.45410656929016113, 'log_odds_chosen': 0.5827471017837524, 'epoch': 0.43} {'loss': 2.1742, 'grad_norm': 0.507463812828064, 'learning_rate': 7.100953342075009e-07, 'rewards/chosen': -0.20070505142211914, 'rewards/rejected': -0.22861552238464355, 'rewards/accuracies': 0.875, 'rewards/margins': 0.027910443022847176, 'logps/rejected': -2.2861552238464355, 'logps/chosen': -2.0070505142211914, 'logits/rejected': -0.1983020305633545, 'logits/chosen': -0.2541901171207428, 'nll_loss': 2.118666410446167, 'log_odds_ratio': -0.5548453330993652, 'log_odds_chosen': 0.31627729535102844, 'epoch': 0.43} {'loss': 2.0937, 'grad_norm': 0.47030338644981384, 'learning_rate': 7.057132633496923e-07, 'rewards/chosen': -0.1912306398153305, 'rewards/rejected': -0.23054589331150055, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03931526839733124, 'logps/rejected': -2.3054590225219727, 'logps/chosen': -1.912306308746338, 'logits/rejected': -0.07695013284683228, 'logits/chosen': -0.3302999436855316, 'nll_loss': 2.0438175201416016, 'log_odds_ratio': -0.49846184253692627, 'log_odds_chosen': 0.44779300689697266, 'epoch': 0.43} {'loss': 2.0497, 'grad_norm': 0.4665156900882721, 'learning_rate': 7.013120885284598e-07, 'rewards/chosen': -0.1880425363779068, 'rewards/rejected': -0.22466430068016052, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03662176430225372, 'logps/rejected': -2.24664306640625, 'logps/chosen': -1.8804253339767456, 'logits/rejected': -0.12284443527460098, 'logits/chosen': -0.4476760923862457, 'nll_loss': 1.9972330331802368, 'log_odds_ratio': -0.525128960609436, 'log_odds_chosen': 0.41706544160842896, 'epoch': 0.44} {'loss': 2.238, 'grad_norm': 0.5610730648040771, 'learning_rate': 6.968922184674867e-07, 'rewards/chosen': -0.20575681328773499, 'rewards/rejected': -0.23407329618930817, 'rewards/accuracies': 1.0, 'rewards/margins': 0.028316490352153778, 'logps/rejected': -2.340733289718628, 'logps/chosen': -2.057568073272705, 'logits/rejected': -0.021672092378139496, 'logits/chosen': -0.277768611907959, 'nll_loss': 2.183201313018799, 'log_odds_ratio': -0.5479089021682739, 'log_odds_chosen': 0.32004255056381226, 'epoch': 0.44} {'loss': 2.0379, 'grad_norm': 0.4623549282550812, 'learning_rate': 6.924540636266272e-07, 'rewards/chosen': -0.18401743471622467, 'rewards/rejected': -0.22281770408153534, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03880026936531067, 'logps/rejected': -2.2281768321990967, 'logps/chosen': -1.8401743173599243, 'logits/rejected': -0.10325874388217926, 'logits/chosen': -0.2480657398700714, 'nll_loss': 1.987722635269165, 'log_odds_ratio': -0.5021770596504211, 'log_odds_chosen': 0.44459542632102966, 'epoch': 0.44} {'loss': 2.085, 'grad_norm': 0.4706402122974396, 'learning_rate': 6.879980361637865e-07, 'rewards/chosen': -0.19227181375026703, 'rewards/rejected': -0.2310340255498886, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03876221179962158, 'logps/rejected': -2.310340166091919, 'logps/chosen': -1.9227181673049927, 'logits/rejected': -0.14531859755516052, 'logits/chosen': -0.2594497501850128, 'nll_loss': 2.034846544265747, 'log_odds_ratio': -0.5012038350105286, 'log_odds_chosen': 0.4396761953830719, 'epoch': 0.44} {'loss': 2.1421, 'grad_norm': 0.5076239109039307, 'learning_rate': 6.83524549896646e-07, 'rewards/chosen': -0.19800883531570435, 'rewards/rejected': -0.24419891834259033, 'rewards/accuracies': 1.0, 'rewards/margins': 0.046190083026885986, 'logps/rejected': -2.4419891834259033, 'logps/chosen': -1.980088233947754, 'logits/rejected': -0.18334615230560303, 'logits/chosen': -0.4366263449192047, 'nll_loss': 2.0949079990386963, 'log_odds_ratio': -0.4721822738647461, 'log_odds_chosen': 0.5196196436882019, 'epoch': 0.45} {'loss': 2.0855, 'grad_norm': 0.5030075907707214, 'learning_rate': 6.790340202642331e-07, 'rewards/chosen': -0.19148677587509155, 'rewards/rejected': -0.2361053079366684, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04461853206157684, 'logps/rejected': -2.361052989959717, 'logps/chosen': -1.914867639541626, 'logits/rejected': -0.20123453438282013, 'logits/chosen': -0.37305155396461487, 'nll_loss': 2.0372049808502197, 'log_odds_ratio': -0.4829539954662323, 'log_odds_chosen': 0.5108202695846558, 'epoch': 0.45} {'loss': 1.9987, 'grad_norm': 0.4399136006832123, 'learning_rate': 6.745268642883404e-07, 'rewards/chosen': -0.18217025697231293, 'rewards/rejected': -0.22611698508262634, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04394673556089401, 'logps/rejected': -2.261169672012329, 'logps/chosen': -1.8217023611068726, 'logits/rejected': -0.25371578335762024, 'logits/chosen': -0.31777992844581604, 'nll_loss': 1.9510999917984009, 'log_odds_ratio': -0.47626441717147827, 'log_odds_chosen': 0.5083030462265015, 'epoch': 0.45} {'loss': 2.1813, 'grad_norm': 0.45623862743377686, 'learning_rate': 6.700035005347982e-07, 'rewards/chosen': -0.2008546143770218, 'rewards/rejected': -0.25288069248199463, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05202607810497284, 'logps/rejected': -2.5288069248199463, 'logps/chosen': -2.0085461139678955, 'logits/rejected': -0.1132553368806839, 'logits/chosen': -0.25216957926750183, 'nll_loss': 2.1363625526428223, 'log_odds_ratio': -0.4491708278656006, 'log_odds_chosen': 0.5798279047012329, 'epoch': 0.46} {'loss': 2.1739, 'grad_norm': 0.5783258676528931, 'learning_rate': 6.654643490746041e-07, 'rewards/chosen': -0.19943203032016754, 'rewards/rejected': -0.2360106259584427, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03657860308885574, 'logps/rejected': -2.3601064682006836, 'logps/chosen': -1.994320273399353, 'logits/rejected': -0.16523872315883636, 'logits/chosen': -0.46417152881622314, 'nll_loss': 2.1212868690490723, 'log_odds_ratio': -0.5265771746635437, 'log_odds_chosen': 0.4070507884025574, 'epoch': 0.46} {'loss': 2.1152, 'grad_norm': 0.48164114356040955, 'learning_rate': 6.609098314449115e-07, 'rewards/chosen': -0.1924610137939453, 'rewards/rejected': -0.23838523030281067, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04592423141002655, 'logps/rejected': -2.383852005004883, 'logps/chosen': -1.9246100187301636, 'logits/rejected': 0.05244123563170433, 'logits/chosen': -0.19338321685791016, 'nll_loss': 2.0679879188537598, 'log_odds_ratio': -0.4722740054130554, 'log_odds_chosen': 0.5174700021743774, 'epoch': 0.46} {'loss': 2.0875, 'grad_norm': 0.49352937936782837, 'learning_rate': 6.563403706098832e-07, 'rewards/chosen': -0.1905830204486847, 'rewards/rejected': -0.24507898092269897, 'rewards/accuracies': 0.875, 'rewards/margins': 0.054495953023433685, 'logps/rejected': -2.4507896900177, 'logps/chosen': -1.9058302640914917, 'logits/rejected': -0.13914185762405396, 'logits/chosen': -0.33679214119911194, 'nll_loss': 2.0423457622528076, 'log_odds_ratio': -0.45163875818252563, 'log_odds_chosen': 0.6114981174468994, 'epoch': 0.46} {'loss': 2.0798, 'grad_norm': 0.4441838562488556, 'learning_rate': 6.517563909214119e-07, 'rewards/chosen': -0.19123899936676025, 'rewards/rejected': -0.2352985292673111, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04405952990055084, 'logps/rejected': -2.352985143661499, 'logps/chosen': -1.912389874458313, 'logits/rejected': -0.1331631988286972, 'logits/chosen': -0.27010205388069153, 'nll_loss': 2.0319559574127197, 'log_odds_ratio': -0.47856655716896057, 'log_odds_chosen': 0.5005499124526978, 'epoch': 0.47} {'loss': 2.0483, 'grad_norm': 0.499590128660202, 'learning_rate': 6.47158318079712e-07, 'rewards/chosen': -0.18735414743423462, 'rewards/rejected': -0.2361140102148056, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04875984787940979, 'logps/rejected': -2.361140012741089, 'logps/chosen': -1.8735414743423462, 'logits/rejected': -0.2863028347492218, 'logits/chosen': -0.39645957946777344, 'nll_loss': 2.000643730163574, 'log_odds_ratio': -0.4767185151576996, 'log_odds_chosen': 0.5453026294708252, 'epoch': 0.47} {'loss': 2.143, 'grad_norm': 0.5450389981269836, 'learning_rate': 6.425465790937861e-07, 'rewards/chosen': -0.195481076836586, 'rewards/rejected': -0.23471572995185852, 'rewards/accuracies': 1.0, 'rewards/margins': 0.039234623312950134, 'logps/rejected': -2.3471572399139404, 'logps/chosen': -1.9548108577728271, 'logits/rejected': 0.0022758357226848602, 'logits/chosen': -0.3227909505367279, 'nll_loss': 2.0925097465515137, 'log_odds_ratio': -0.5045577883720398, 'log_odds_chosen': 0.44236356019973755, 'epoch': 0.47} {'loss': 2.0709, 'grad_norm': 0.5155457854270935, 'learning_rate': 6.379216022417695e-07, 'rewards/chosen': -0.18866465985774994, 'rewards/rejected': -0.23667012155056, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04800545424222946, 'logps/rejected': -2.366701126098633, 'logps/chosen': -1.8866467475891113, 'logits/rejected': -0.024255044758319855, 'logits/chosen': -0.4189146161079407, 'nll_loss': 2.025205612182617, 'log_odds_ratio': -0.4568098783493042, 'log_odds_chosen': 0.5473636984825134, 'epoch': 0.47} {'loss': 2.081, 'grad_norm': 0.5155901908874512, 'learning_rate': 6.332838170311585e-07, 'rewards/chosen': -0.19039152562618256, 'rewards/rejected': -0.23025237023830414, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03986084833741188, 'logps/rejected': -2.3025238513946533, 'logps/chosen': -1.9039151668548584, 'logits/rejected': -0.13081157207489014, 'logits/chosen': -0.3719104826450348, 'nll_loss': 2.0305662155151367, 'log_odds_ratio': -0.5040324330329895, 'log_odds_chosen': 0.45225203037261963, 'epoch': 0.48} {'loss': 2.2208, 'grad_norm': 0.5719819664955139, 'learning_rate': 6.286336541589223e-07, 'rewards/chosen': -0.20402835309505463, 'rewards/rejected': -0.2249561846256256, 'rewards/accuracies': 0.625, 'rewards/margins': 0.020927807316184044, 'logps/rejected': -2.2495615482330322, 'logps/chosen': -2.040283441543579, 'logits/rejected': -0.1406262069940567, 'logits/chosen': -0.37860414385795593, 'nll_loss': 2.161356210708618, 'log_odds_ratio': -0.5945478677749634, 'log_odds_chosen': 0.23404161632061005, 'epoch': 0.48} {'loss': 2.0894, 'grad_norm': 0.4385439157485962, 'learning_rate': 6.239715454715053e-07, 'rewards/chosen': -0.19338127970695496, 'rewards/rejected': -0.23564325273036957, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04226197302341461, 'logps/rejected': -2.3564324378967285, 'logps/chosen': -1.9338128566741943, 'logits/rejected': -0.25715842843055725, 'logits/chosen': -0.2991381883621216, 'nll_loss': 2.0407304763793945, 'log_odds_ratio': -0.48626744747161865, 'log_odds_chosen': 0.47704774141311646, 'epoch': 0.48} {'loss': 2.0786, 'grad_norm': 0.5189127922058105, 'learning_rate': 6.192979239247242e-07, 'rewards/chosen': -0.19110207259655, 'rewards/rejected': -0.22788234055042267, 'rewards/accuracies': 1.0, 'rewards/margins': 0.036780260503292084, 'logps/rejected': -2.2788233757019043, 'logps/chosen': -1.9110206365585327, 'logits/rejected': -0.3438834547996521, 'logits/chosen': -0.45005002617836, 'nll_loss': 2.0272610187530518, 'log_odds_ratio': -0.513340950012207, 'log_odds_chosen': 0.41903528571128845, 'epoch': 0.49} {'loss': 2.145, 'grad_norm': 0.5391824841499329, 'learning_rate': 6.146132235435591e-07, 'rewards/chosen': -0.19622071087360382, 'rewards/rejected': -0.2364424616098404, 'rewards/accuracies': 1.0, 'rewards/margins': 0.040221765637397766, 'logps/rejected': -2.364424467086792, 'logps/chosen': -1.9622068405151367, 'logits/rejected': -0.13199269771575928, 'logits/chosen': -0.3154350221157074, 'nll_loss': 2.094515562057495, 'log_odds_ratio': -0.5045695304870605, 'log_odds_chosen': 0.4565315842628479, 'epoch': 0.49} {'loss': 2.0695, 'grad_norm': 0.5079851150512695, 'learning_rate': 6.099178793818478e-07, 'rewards/chosen': -0.18969543278217316, 'rewards/rejected': -0.2526519298553467, 'rewards/accuracies': 1.0, 'rewards/margins': 0.06295648217201233, 'logps/rejected': -2.526519298553467, 'logps/chosen': -1.8969542980194092, 'logits/rejected': -0.18183393776416779, 'logits/chosen': -0.4896236062049866, 'nll_loss': 2.0289804935455322, 'log_odds_ratio': -0.4052090644836426, 'log_odds_chosen': 0.7125115990638733, 'epoch': 0.49} {'loss': 2.1241, 'grad_norm': 0.42423373460769653, 'learning_rate': 6.052123274818841e-07, 'rewards/chosen': -0.19779011607170105, 'rewards/rejected': -0.2437371015548706, 'rewards/accuracies': 1.0, 'rewards/margins': 0.045946985483169556, 'logps/rejected': -2.437370777130127, 'logps/chosen': -1.9779009819030762, 'logits/rejected': -0.1859840303659439, 'logits/chosen': -0.22320327162742615, 'nll_loss': 2.0772347450256348, 'log_odds_ratio': -0.4682368338108063, 'log_odds_chosen': 0.518886923789978, 'epoch': 0.49} {'loss': 2.1479, 'grad_norm': 0.5107156038284302, 'learning_rate': 6.004970048339225e-07, 'rewards/chosen': -0.19648662209510803, 'rewards/rejected': -0.2271694540977478, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03068285994231701, 'logps/rejected': -2.2716946601867676, 'logps/chosen': -1.964866042137146, 'logits/rejected': -0.13514721393585205, 'logits/chosen': -0.34241271018981934, 'nll_loss': 2.0939929485321045, 'log_odds_ratio': -0.538905918598175, 'log_odds_chosen': 0.3471386730670929, 'epoch': 0.5} {'loss': 2.1414, 'grad_norm': 0.4654123783111572, 'learning_rate': 5.957723493355976e-07, 'rewards/chosen': -0.19399675726890564, 'rewards/rejected': -0.2346663475036621, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04066961258649826, 'logps/rejected': -2.3466637134552, 'logps/chosen': -1.9399676322937012, 'logits/rejected': -0.024188008159399033, 'logits/chosen': -0.21615736186504364, 'nll_loss': 2.0918264389038086, 'log_odds_ratio': -0.49577149748802185, 'log_odds_chosen': 0.4619404077529907, 'epoch': 0.5} {'loss': 2.1393, 'grad_norm': 0.4790768027305603, 'learning_rate': 5.910387997512573e-07, 'rewards/chosen': -0.19683094322681427, 'rewards/rejected': -0.24322962760925293, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04639867693185806, 'logps/rejected': -2.43229603767395, 'logps/chosen': -1.9683092832565308, 'logits/rejected': -0.45565640926361084, 'logits/chosen': -0.271729052066803, 'nll_loss': 2.091820001602173, 'log_odds_ratio': -0.47471052408218384, 'log_odds_chosen': 0.519871175289154, 'epoch': 0.5} {'loss': 2.0347, 'grad_norm': 0.48759087920188904, 'learning_rate': 5.862967956712159e-07, 'rewards/chosen': -0.18550390005111694, 'rewards/rejected': -0.22921758890151978, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04371367394924164, 'logps/rejected': -2.292175769805908, 'logps/chosen': -1.855039119720459, 'logits/rejected': 0.008702604100108147, 'logits/chosen': -0.3652348518371582, 'nll_loss': 1.9861171245574951, 'log_odds_ratio': -0.48592039942741394, 'log_odds_chosen': 0.4985675811767578, 'epoch': 0.51} {'loss': 2.0622, 'grad_norm': 0.4419066607952118, 'learning_rate': 5.815467774709313e-07, 'rewards/chosen': -0.18986095488071442, 'rewards/rejected': -0.23583440482616425, 'rewards/accuracies': 1.0, 'rewards/margins': 0.045973438769578934, 'logps/rejected': -2.3583438396453857, 'logps/chosen': -1.8986095190048218, 'logits/rejected': -0.20873025059700012, 'logits/chosen': -0.32331582903862, 'nll_loss': 2.0154149532318115, 'log_odds_ratio': -0.46782034635543823, 'log_odds_chosen': 0.5235296487808228, 'epoch': 0.51} {'loss': 1.9619, 'grad_norm': 0.40547627210617065, 'learning_rate': 5.767891862701081e-07, 'rewards/chosen': -0.17969751358032227, 'rewards/rejected': -0.22778378427028656, 'rewards/accuracies': 1.0, 'rewards/margins': 0.048086270689964294, 'logps/rejected': -2.2778377532958984, 'logps/chosen': -1.796975016593933, 'logits/rejected': -0.1256704479455948, 'logits/chosen': -0.3630341589450836, 'nll_loss': 1.915809988975525, 'log_odds_ratio': -0.46065008640289307, 'log_odds_chosen': 0.5548373460769653, 'epoch': 0.51} {'loss': 2.1194, 'grad_norm': 0.5163060426712036, 'learning_rate': 5.720244638917323e-07, 'rewards/chosen': -0.19638122618198395, 'rewards/rejected': -0.22940419614315033, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03302296996116638, 'logps/rejected': -2.2940421104431152, 'logps/chosen': -1.9638121128082275, 'logits/rejected': -0.07070097327232361, 'logits/chosen': -0.22042424976825714, 'nll_loss': 2.066725254058838, 'log_odds_ratio': -0.5264884233474731, 'log_odds_chosen': 0.37705615162849426, 'epoch': 0.51} {'loss': 2.1004, 'grad_norm': 0.47353076934814453, 'learning_rate': 5.672530528210404e-07, 'rewards/chosen': -0.19184976816177368, 'rewards/rejected': -0.2148369401693344, 'rewards/accuracies': 1.0, 'rewards/margins': 0.022987185046076775, 'logps/rejected': -2.148369550704956, 'logps/chosen': -1.9184975624084473, 'logits/rejected': -0.006437892094254494, 'logits/chosen': -0.2964926064014435, 'nll_loss': 2.042937994003296, 'log_odds_ratio': -0.5742963552474976, 'log_odds_chosen': 0.265331506729126, 'epoch': 0.52} {'loss': 2.184, 'grad_norm': 0.5041413903236389, 'learning_rate': 5.624753961644281e-07, 'rewards/chosen': -0.20171691477298737, 'rewards/rejected': -0.2374822348356247, 'rewards/accuracies': 1.0, 'rewards/margins': 0.035765327513217926, 'logps/rejected': -2.3748221397399902, 'logps/chosen': -2.017169237136841, 'logits/rejected': -0.14603643119335175, 'logits/chosen': -0.19358986616134644, 'nll_loss': 2.1322598457336426, 'log_odds_ratio': -0.5178303718566895, 'log_odds_chosen': 0.4017038941383362, 'epoch': 0.52} {'loss': 2.1302, 'grad_norm': 0.46041780710220337, 'learning_rate': 5.576919376082989e-07, 'rewards/chosen': -0.19662585854530334, 'rewards/rejected': -0.23327937722206116, 'rewards/accuracies': 1.0, 'rewards/margins': 0.036653533577919006, 'logps/rejected': -2.332793712615967, 'logps/chosen': -1.9662584066390991, 'logits/rejected': -0.13594995439052582, 'logits/chosen': -0.2839770019054413, 'nll_loss': 2.0789811611175537, 'log_odds_ratio': -0.5125820636749268, 'log_odds_chosen': 0.4141581654548645, 'epoch': 0.52} {'loss': 2.0948, 'grad_norm': 0.47012194991111755, 'learning_rate': 5.529031213778614e-07, 'rewards/chosen': -0.18962793052196503, 'rewards/rejected': -0.23211446404457092, 'rewards/accuracies': 1.0, 'rewards/margins': 0.042486537247896194, 'logps/rejected': -2.3211445808410645, 'logps/chosen': -1.896279215812683, 'logits/rejected': -0.23288729786872864, 'logits/chosen': -0.389356791973114, 'nll_loss': 2.0464413166046143, 'log_odds_ratio': -0.4840419888496399, 'log_odds_chosen': 0.4835112690925598, 'epoch': 0.52} {'loss': 2.1912, 'grad_norm': 0.5491785407066345, 'learning_rate': 5.481093921958749e-07, 'rewards/chosen': -0.20216712355613708, 'rewards/rejected': -0.24666264653205872, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04449552297592163, 'logps/rejected': -2.4666261672973633, 'logps/chosen': -2.0216710567474365, 'logits/rejected': -0.15095578134059906, 'logits/chosen': -0.38450419902801514, 'nll_loss': 2.1433069705963135, 'log_odds_ratio': -0.47897040843963623, 'log_odds_chosen': 0.4995926320552826, 'epoch': 0.53} {'loss': 2.0587, 'grad_norm': 0.48054239153862, 'learning_rate': 5.433111952413494e-07, 'rewards/chosen': -0.19027060270309448, 'rewards/rejected': -0.23220594227313995, 'rewards/accuracies': 1.0, 'rewards/margins': 0.041935332119464874, 'logps/rejected': -2.322059392929077, 'logps/chosen': -1.9027060270309448, 'logits/rejected': -0.18638935685157776, 'logits/chosen': -0.26365604996681213, 'nll_loss': 2.0102057456970215, 'log_odds_ratio': -0.48475128412246704, 'log_odds_chosen': 0.4785778820514679, 'epoch': 0.53} {'loss': 2.056, 'grad_norm': 0.41837820410728455, 'learning_rate': 5.385089761082039e-07, 'rewards/chosen': -0.1887839138507843, 'rewards/rejected': -0.23233118653297424, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04354727268218994, 'logps/rejected': -2.3233118057250977, 'logps/chosen': -1.8878390789031982, 'logits/rejected': -0.1336251199245453, 'logits/chosen': -0.26745283603668213, 'nll_loss': 2.008148670196533, 'log_odds_ratio': -0.47896653413772583, 'log_odds_chosen': 0.4972219169139862, 'epoch': 0.53} {'loss': 2.0681, 'grad_norm': 0.45828357338905334, 'learning_rate': 5.33703180763884e-07, 'rewards/chosen': -0.1887953132390976, 'rewards/rejected': -0.2296655774116516, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04087024927139282, 'logps/rejected': -2.2966556549072266, 'logps/chosen': -1.887953281402588, 'logits/rejected': -0.07468175888061523, 'logits/chosen': -0.3924446702003479, 'nll_loss': 2.018134593963623, 'log_odds_ratio': -0.4991610050201416, 'log_odds_chosen': 0.46731892228126526, 'epoch': 0.54} {'loss': 2.1089, 'grad_norm': 0.4831075072288513, 'learning_rate': 5.288942555079478e-07, 'rewards/chosen': -0.19115132093429565, 'rewards/rejected': -0.25091320276260376, 'rewards/accuracies': 1.0, 'rewards/margins': 0.059761881828308105, 'logps/rejected': -2.509132146835327, 'logps/chosen': -1.911513090133667, 'logits/rejected': -0.0966208279132843, 'logits/chosen': -0.3334307074546814, 'nll_loss': 2.063706159591675, 'log_odds_ratio': -0.4520753026008606, 'log_odds_chosen': 0.6895397901535034, 'epoch': 0.54} {'loss': 2.0735, 'grad_norm': 0.42661550641059875, 'learning_rate': 5.240826469306186e-07, 'rewards/chosen': -0.18927910923957825, 'rewards/rejected': -0.22394056618213654, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034661464393138885, 'logps/rejected': -2.239405632019043, 'logps/chosen': -1.8927909135818481, 'logits/rejected': -0.5069546103477478, 'logits/chosen': -0.3010050058364868, 'nll_loss': 2.021390199661255, 'log_odds_ratio': -0.5208821892738342, 'log_odds_chosen': 0.39442360401153564, 'epoch': 0.54} {'loss': 2.1202, 'grad_norm': 0.511133074760437, 'learning_rate': 5.192688018713113e-07, 'rewards/chosen': -0.19366243481636047, 'rewards/rejected': -0.23165374994277954, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03799131512641907, 'logps/rejected': -2.316537618637085, 'logps/chosen': -1.93662428855896, 'logits/rejected': -0.14694276452064514, 'logits/chosen': -0.29611679911613464, 'nll_loss': 2.0698323249816895, 'log_odds_ratio': -0.5032349228858948, 'log_odds_chosen': 0.43289878964424133, 'epoch': 0.54} {'loss': 2.0922, 'grad_norm': 0.45973795652389526, 'learning_rate': 5.144531673771363e-07, 'rewards/chosen': -0.19230739772319794, 'rewards/rejected': -0.2343052327632904, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04199780896306038, 'logps/rejected': -2.343052387237549, 'logps/chosen': -1.9230740070343018, 'logits/rejected': -0.18779116868972778, 'logits/chosen': -0.17253297567367554, 'nll_loss': 2.043295383453369, 'log_odds_ratio': -0.4891282320022583, 'log_odds_chosen': 0.4756642282009125, 'epoch': 0.55} {'loss': 2.0449, 'grad_norm': 0.4573759138584137, 'learning_rate': 5.096361906613835e-07, 'rewards/chosen': -0.18643978238105774, 'rewards/rejected': -0.22135013341903687, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03491034358739853, 'logps/rejected': -2.213501453399658, 'logps/chosen': -1.8643978834152222, 'logits/rejected': -0.29722410440444946, 'logits/chosen': -0.3204929530620575, 'nll_loss': 1.9916300773620605, 'log_odds_ratio': -0.5324212312698364, 'log_odds_chosen': 0.3954029977321625, 'epoch': 0.55} {'loss': 2.0796, 'grad_norm': 0.478037029504776, 'learning_rate': 5.048183190619903e-07, 'rewards/chosen': -0.1899263709783554, 'rewards/rejected': -0.23657621443271637, 'rewards/accuracies': 1.0, 'rewards/margins': 0.046649858355522156, 'logps/rejected': -2.36576247215271, 'logps/chosen': -1.899263620376587, 'logits/rejected': -0.18273036181926727, 'logits/chosen': -0.29242661595344543, 'nll_loss': 2.032811164855957, 'log_odds_ratio': -0.4676993787288666, 'log_odds_chosen': 0.5303144454956055, 'epoch': 0.55} {'loss': 2.1284, 'grad_norm': 0.4479018747806549, 'learning_rate': 5e-07, 'rewards/chosen': -0.1952238380908966, 'rewards/rejected': -0.2308758646249771, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0356520339846611, 'logps/rejected': -2.308758497238159, 'logps/chosen': -1.9522382020950317, 'logits/rejected': -0.3748874068260193, 'logits/chosen': -0.20333202183246613, 'nll_loss': 2.076796531677246, 'log_odds_ratio': -0.5163565874099731, 'log_odds_chosen': 0.40334001183509827, 'epoch': 0.56} {'loss': 2.0662, 'grad_norm': 0.4593195617198944, 'learning_rate': 4.951816809380097e-07, 'rewards/chosen': -0.18836651742458344, 'rewards/rejected': -0.22832421958446503, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03995770215988159, 'logps/rejected': -2.2832422256469727, 'logps/chosen': -1.8836652040481567, 'logits/rejected': -0.28211909532546997, 'logits/chosen': -0.374197393655777, 'nll_loss': 2.0162971019744873, 'log_odds_ratio': -0.4991394877433777, 'log_odds_chosen': 0.4540878236293793, 'epoch': 0.56} {'loss': 2.111, 'grad_norm': 0.4635930359363556, 'learning_rate': 4.903638093386167e-07, 'rewards/chosen': -0.19495271146297455, 'rewards/rejected': -0.23999318480491638, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04504048079252243, 'logps/rejected': -2.3999316692352295, 'logps/chosen': -1.9495270252227783, 'logits/rejected': -0.31064000725746155, 'logits/chosen': -0.26355335116386414, 'nll_loss': 2.063145875930786, 'log_odds_ratio': -0.4782922863960266, 'log_odds_chosen': 0.509470522403717, 'epoch': 0.56} {'loss': 2.0928, 'grad_norm': 0.44470760226249695, 'learning_rate': 4.855468326228638e-07, 'rewards/chosen': -0.19127273559570312, 'rewards/rejected': -0.24635626375675201, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05508355423808098, 'logps/rejected': -2.4635629653930664, 'logps/chosen': -1.9127272367477417, 'logits/rejected': -0.2821475565433502, 'logits/chosen': -0.3340487480163574, 'nll_loss': 2.049443244934082, 'log_odds_ratio': -0.4331510663032532, 'log_odds_chosen': 0.620209813117981, 'epoch': 0.56} {'loss': 2.0765, 'grad_norm': 0.4645419716835022, 'learning_rate': 4.807311981286888e-07, 'rewards/chosen': -0.19042205810546875, 'rewards/rejected': -0.22294369339942932, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03252164646983147, 'logps/rejected': -2.2294371128082275, 'logps/chosen': -1.9042205810546875, 'logits/rejected': -0.0387892983853817, 'logits/chosen': -0.2799780070781708, 'nll_loss': 2.0238375663757324, 'log_odds_ratio': -0.5261833667755127, 'log_odds_chosen': 0.3717229962348938, 'epoch': 0.57} {'loss': 2.0672, 'grad_norm': 0.44044730067253113, 'learning_rate': 4.7591735306938134e-07, 'rewards/chosen': -0.18773600459098816, 'rewards/rejected': -0.23853006958961487, 'rewards/accuracies': 0.875, 'rewards/margins': 0.0507940798997879, 'logps/rejected': -2.385300636291504, 'logps/chosen': -1.8773598670959473, 'logits/rejected': -0.3041171431541443, 'logits/chosen': -0.2254553735256195, 'nll_loss': 2.020278215408325, 'log_odds_ratio': -0.46954020857810974, 'log_odds_chosen': 0.5784755349159241, 'epoch': 0.57} {'loss': 2.2192, 'grad_norm': 0.4391086995601654, 'learning_rate': 4.7110574449205214e-07, 'rewards/chosen': -0.2064468413591385, 'rewards/rejected': -0.2524523437023163, 'rewards/accuracies': 1.0, 'rewards/margins': 0.046005476266145706, 'logps/rejected': -2.5245234966278076, 'logps/chosen': -2.0644686222076416, 'logits/rejected': -0.22331276535987854, 'logits/chosen': -0.15283432602882385, 'nll_loss': 2.171563148498535, 'log_odds_ratio': -0.4766671359539032, 'log_odds_chosen': 0.5138649344444275, 'epoch': 0.57} {'loss': 2.15, 'grad_norm': 0.46477314829826355, 'learning_rate': 4.6629681923611603e-07, 'rewards/chosen': -0.19834566116333008, 'rewards/rejected': -0.23482660949230194, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03648097440600395, 'logps/rejected': -2.348266124725342, 'logps/chosen': -1.9834563732147217, 'logits/rejected': -0.15614847838878632, 'logits/chosen': -0.10923075675964355, 'nll_loss': 2.098912477493286, 'log_odds_ratio': -0.5109040141105652, 'log_odds_chosen': 0.41258686780929565, 'epoch': 0.57} {'loss': 2.0429, 'grad_norm': 0.4983419179916382, 'learning_rate': 4.614910238917963e-07, 'rewards/chosen': -0.186179518699646, 'rewards/rejected': -0.22444197535514832, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03826242685317993, 'logps/rejected': -2.244419574737549, 'logps/chosen': -1.8617953062057495, 'logits/rejected': -0.14922872185707092, 'logits/chosen': -0.37217968702316284, 'nll_loss': 1.9927852153778076, 'log_odds_ratio': -0.5007336735725403, 'log_odds_chosen': 0.440628319978714, 'epoch': 0.58} {'loss': 2.0876, 'grad_norm': 0.4707476794719696, 'learning_rate': 4.5668880475865067e-07, 'rewards/chosen': -0.19095300137996674, 'rewards/rejected': -0.23211316764354706, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04116016626358032, 'logps/rejected': -2.321131706237793, 'logps/chosen': -1.9095300436019897, 'logits/rejected': 0.020261432975530624, 'logits/chosen': -0.2454988956451416, 'nll_loss': 2.038007974624634, 'log_odds_ratio': -0.49586039781570435, 'log_odds_chosen': 0.46539944410324097, 'epoch': 0.58} {'loss': 2.0762, 'grad_norm': 0.369704008102417, 'learning_rate': 4.5189060780412515e-07, 'rewards/chosen': -0.18902254104614258, 'rewards/rejected': -0.22649873793125153, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03747619688510895, 'logps/rejected': -2.2649874687194824, 'logps/chosen': -1.8902254104614258, 'logits/rejected': -0.021833106875419617, 'logits/chosen': -0.1828247755765915, 'nll_loss': 2.0231916904449463, 'log_odds_ratio': -0.5302607417106628, 'log_odds_chosen': 0.423761248588562, 'epoch': 0.58} {'loss': 1.9792, 'grad_norm': 0.39925074577331543, 'learning_rate': 4.4709687862213864e-07, 'rewards/chosen': -0.1815415918827057, 'rewards/rejected': -0.23315729200839996, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05161570757627487, 'logps/rejected': -2.3315727710723877, 'logps/chosen': -1.8154157400131226, 'logits/rejected': -0.41044506430625916, 'logits/chosen': -0.2861301302909851, 'nll_loss': 1.9341650009155273, 'log_odds_ratio': -0.4505179226398468, 'log_odds_chosen': 0.5902334451675415, 'epoch': 0.59} {'loss': 2.0255, 'grad_norm': 0.38932037353515625, 'learning_rate': 4.4230806239170113e-07, 'rewards/chosen': -0.18713994324207306, 'rewards/rejected': -0.23042459785938263, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04328465461730957, 'logps/rejected': -2.304245948791504, 'logps/chosen': -1.8713995218276978, 'logits/rejected': -0.3000808656215668, 'logits/chosen': -0.14587503671646118, 'nll_loss': 1.9769891500473022, 'log_odds_ratio': -0.48540055751800537, 'log_odds_chosen': 0.49800512194633484, 'epoch': 0.59} {'loss': 2.1077, 'grad_norm': 0.4413486421108246, 'learning_rate': 4.3752460383557194e-07, 'rewards/chosen': -0.19498810172080994, 'rewards/rejected': -0.24114246666431427, 'rewards/accuracies': 0.875, 'rewards/margins': 0.046154364943504333, 'logps/rejected': -2.4114248752593994, 'logps/chosen': -1.9498810768127441, 'logits/rejected': -0.11889218538999557, 'logits/chosen': -0.2637350261211395, 'nll_loss': 2.059854507446289, 'log_odds_ratio': -0.4785919189453125, 'log_odds_chosen': 0.5172650218009949, 'epoch': 0.59} {'loss': 2.1841, 'grad_norm': 0.4996843636035919, 'learning_rate': 4.3274694717895964e-07, 'rewards/chosen': -0.19931714236736298, 'rewards/rejected': -0.23626123368740082, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03694409132003784, 'logps/rejected': -2.36261248588562, 'logps/chosen': -1.9931713342666626, 'logits/rejected': -0.066367506980896, 'logits/chosen': -0.18713760375976562, 'nll_loss': 2.132643222808838, 'log_odds_ratio': -0.5150564908981323, 'log_odds_chosen': 0.41755494475364685, 'epoch': 0.59} {'loss': 2.0009, 'grad_norm': 0.4241478145122528, 'learning_rate': 4.2797553610826797e-07, 'rewards/chosen': -0.1824950873851776, 'rewards/rejected': -0.22439813613891602, 'rewards/accuracies': 1.0, 'rewards/margins': 0.041903071105480194, 'logps/rejected': -2.24398136138916, 'logps/chosen': -1.8249506950378418, 'logits/rejected': -0.01856943964958191, 'logits/chosen': -0.31950169801712036, 'nll_loss': 1.9520447254180908, 'log_odds_ratio': -0.48890987038612366, 'log_odds_chosen': 0.48448869585990906, 'epoch': 0.6} {'loss': 2.1157, 'grad_norm': 0.4482674300670624, 'learning_rate': 4.2321081372989186e-07, 'rewards/chosen': -0.1915510594844818, 'rewards/rejected': -0.2215559035539627, 'rewards/accuracies': 0.75, 'rewards/margins': 0.030004845932126045, 'logps/rejected': -2.2155590057373047, 'logps/chosen': -1.9155105352401733, 'logits/rejected': -0.11674074083566666, 'logits/chosen': -0.2806357741355896, 'nll_loss': 2.0606534481048584, 'log_odds_ratio': -0.5502303242683411, 'log_odds_chosen': 0.34753182530403137, 'epoch': 0.6} {'loss': 2.092, 'grad_norm': 0.5122972130775452, 'learning_rate': 4.184532225290686e-07, 'rewards/chosen': -0.19241386651992798, 'rewards/rejected': -0.2317504584789276, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03933658450841904, 'logps/rejected': -2.317504644393921, 'logps/chosen': -1.9241386651992798, 'logits/rejected': -0.1805831342935562, 'logits/chosen': -0.29193422198295593, 'nll_loss': 2.0420260429382324, 'log_odds_ratio': -0.4998216927051544, 'log_odds_chosen': 0.4445059299468994, 'epoch': 0.6} {'loss': 2.043, 'grad_norm': 0.4178394079208374, 'learning_rate': 4.13703204328784e-07, 'rewards/chosen': -0.18720358610153198, 'rewards/rejected': -0.23234832286834717, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0451447069644928, 'logps/rejected': -2.3234829902648926, 'logps/chosen': -1.8720359802246094, 'logits/rejected': -0.2523457705974579, 'logits/chosen': -0.3372417986392975, 'nll_loss': 1.9955538511276245, 'log_odds_ratio': -0.4747515618801117, 'log_odds_chosen': 0.5128558874130249, 'epoch': 0.6} {'loss': 2.1533, 'grad_norm': 0.5029446482658386, 'learning_rate': 4.089612002487428e-07, 'rewards/chosen': -0.19919249415397644, 'rewards/rejected': -0.23357556760311127, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034383051097393036, 'logps/rejected': -2.3357558250427246, 'logps/chosen': -1.9919250011444092, 'logits/rejected': -0.2113722860813141, 'logits/chosen': -0.19269907474517822, 'nll_loss': 2.0988759994506836, 'log_odds_ratio': -0.5439907312393188, 'log_odds_chosen': 0.3836578130722046, 'epoch': 0.61} {'loss': 2.0517, 'grad_norm': 0.4052739441394806, 'learning_rate': 4.042276506644024e-07, 'rewards/chosen': -0.18867139518260956, 'rewards/rejected': -0.2378152757883072, 'rewards/accuracies': 1.0, 'rewards/margins': 0.049143895506858826, 'logps/rejected': -2.378152847290039, 'logps/chosen': -1.8867138624191284, 'logits/rejected': -0.10660164058208466, 'logits/chosen': -0.1459711492061615, 'nll_loss': 2.0061984062194824, 'log_odds_ratio': -0.4547596871852875, 'log_odds_chosen': 0.5609914660453796, 'epoch': 0.61} {'loss': 2.1057, 'grad_norm': 0.40826407074928284, 'learning_rate': 3.995029951660776e-07, 'rewards/chosen': -0.19388024508953094, 'rewards/rejected': -0.24753732979297638, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05365708842873573, 'logps/rejected': -2.4753732681274414, 'logps/chosen': -1.9388023614883423, 'logits/rejected': -0.14933869242668152, 'logits/chosen': -0.23413187265396118, 'nll_loss': 2.0613319873809814, 'log_odds_ratio': -0.44388866424560547, 'log_odds_chosen': 0.6102659106254578, 'epoch': 0.61} {'loss': 2.074, 'grad_norm': 0.40428611636161804, 'learning_rate': 3.9478767251811595e-07, 'rewards/chosen': -0.19040776789188385, 'rewards/rejected': -0.22454842925071716, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03414066135883331, 'logps/rejected': -2.2454843521118164, 'logps/chosen': -1.9040776491165161, 'logits/rejected': -0.4330897927284241, 'logits/chosen': -0.2519416809082031, 'nll_loss': 2.0216562747955322, 'log_odds_ratio': -0.5237889289855957, 'log_odds_chosen': 0.38443854451179504, 'epoch': 0.62} {'loss': 2.0232, 'grad_norm': 0.41897815465927124, 'learning_rate': 3.9008212061815207e-07, 'rewards/chosen': -0.18474040925502777, 'rewards/rejected': -0.2265615463256836, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04182111844420433, 'logps/rejected': -2.265615224838257, 'logps/chosen': -1.8474040031433105, 'logits/rejected': -0.16496577858924866, 'logits/chosen': -0.24099406599998474, 'nll_loss': 1.9748239517211914, 'log_odds_ratio': -0.48397549986839294, 'log_odds_chosen': 0.48172813653945923, 'epoch': 0.62} {'loss': 1.9669, 'grad_norm': 0.3934837281703949, 'learning_rate': 3.853867764564409e-07, 'rewards/chosen': -0.18030327558517456, 'rewards/rejected': -0.228864848613739, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04856157302856445, 'logps/rejected': -2.2886483669281006, 'logps/chosen': -1.803032636642456, 'logits/rejected': -0.3277437686920166, 'logits/chosen': -0.34043368697166443, 'nll_loss': 1.9212396144866943, 'log_odds_ratio': -0.4562433063983917, 'log_odds_chosen': 0.5578025579452515, 'epoch': 0.62} {'loss': 2.1831, 'grad_norm': 0.4615204334259033, 'learning_rate': 3.8070207607527585e-07, 'rewards/chosen': -0.20035582780838013, 'rewards/rejected': -0.2344210296869278, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034065183252096176, 'logps/rejected': -2.344210147857666, 'logps/chosen': -2.0035581588745117, 'logits/rejected': -0.15861357748508453, 'logits/chosen': -0.09884040057659149, 'nll_loss': 2.130826950073242, 'log_odds_ratio': -0.5226789712905884, 'log_odds_chosen': 0.3832344710826874, 'epoch': 0.62} {'loss': 2.2513, 'grad_norm': 0.5194337368011475, 'learning_rate': 3.7602845452849463e-07, 'rewards/chosen': -0.2091406285762787, 'rewards/rejected': -0.25869280099868774, 'rewards/accuracies': 1.0, 'rewards/margins': 0.049552179872989655, 'logps/rejected': -2.586927890777588, 'logps/chosen': -2.0914061069488525, 'logits/rejected': -0.17510032653808594, 'logits/chosen': -0.19283336400985718, 'nll_loss': 2.2046549320220947, 'log_odds_ratio': -0.466709703207016, 'log_odds_chosen': 0.547633171081543, 'epoch': 0.63} {'loss': 2.1851, 'grad_norm': 0.4143467843532562, 'learning_rate': 3.7136634584107783e-07, 'rewards/chosen': -0.201212540268898, 'rewards/rejected': -0.22754555940628052, 'rewards/accuracies': 0.875, 'rewards/margins': 0.026333047077059746, 'logps/rejected': -2.2754557132720947, 'logps/chosen': -2.012125253677368, 'logits/rejected': -0.28935497999191284, 'logits/chosen': -0.08032770454883575, 'nll_loss': 2.128629684448242, 'log_odds_ratio': -0.5645455121994019, 'log_odds_chosen': 0.29817482829093933, 'epoch': 0.63} {'loss': 2.079, 'grad_norm': 0.4432436525821686, 'learning_rate': 3.6671618296884143e-07, 'rewards/chosen': -0.189656600356102, 'rewards/rejected': -0.2258012592792511, 'rewards/accuracies': 0.875, 'rewards/margins': 0.036144666373729706, 'logps/rejected': -2.2580127716064453, 'logps/chosen': -1.8965659141540527, 'logits/rejected': -0.21302977204322815, 'logits/chosen': -0.2652166783809662, 'nll_loss': 2.0267221927642822, 'log_odds_ratio': -0.5227192640304565, 'log_odds_chosen': 0.4121524393558502, 'epoch': 0.63} {'loss': 1.938, 'grad_norm': 0.4152255058288574, 'learning_rate': 3.6207839775823047e-07, 'rewards/chosen': -0.17441876232624054, 'rewards/rejected': -0.22215408086776733, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04773533344268799, 'logps/rejected': -2.221540927886963, 'logps/chosen': -1.744187593460083, 'logits/rejected': -0.3200719356536865, 'logits/chosen': -0.4695712625980377, 'nll_loss': 1.8914943933486938, 'log_odds_ratio': -0.46520212292671204, 'log_odds_chosen': 0.5571652054786682, 'epoch': 0.64} {'loss': 2.0915, 'grad_norm': 0.4536137282848358, 'learning_rate': 3.57453420906214e-07, 'rewards/chosen': -0.19137263298034668, 'rewards/rejected': -0.2133842557668686, 'rewards/accuracies': 0.875, 'rewards/margins': 0.022011613473296165, 'logps/rejected': -2.133842706680298, 'logps/chosen': -1.913726568222046, 'logits/rejected': -0.2664189040660858, 'logits/chosen': -0.3376019597053528, 'nll_loss': 2.033724069595337, 'log_odds_ratio': -0.5782453417778015, 'log_odds_chosen': 0.25317129492759705, 'epoch': 0.64} {'loss': 2.153, 'grad_norm': 0.5167147517204285, 'learning_rate': 3.5284168192028805e-07, 'rewards/chosen': -0.19809167087078094, 'rewards/rejected': -0.2453472763299942, 'rewards/accuracies': 1.0, 'rewards/margins': 0.047255612909793854, 'logps/rejected': -2.453472852706909, 'logps/chosen': -1.9809167385101318, 'logits/rejected': -0.3100574314594269, 'logits/chosen': -0.24234242737293243, 'nll_loss': 2.10510516166687, 'log_odds_ratio': -0.4787348508834839, 'log_odds_chosen': 0.5250340700149536, 'epoch': 0.64} {'loss': 2.0912, 'grad_norm': 0.41366466879844666, 'learning_rate': 3.482436090785882e-07, 'rewards/chosen': -0.19174233078956604, 'rewards/rejected': -0.23675590753555298, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04501357674598694, 'logps/rejected': -2.3675589561462402, 'logps/chosen': -1.917423129081726, 'logits/rejected': -0.3802124261856079, 'logits/chosen': -0.26800814270973206, 'nll_loss': 2.042955160140991, 'log_odds_ratio': -0.48293790221214294, 'log_odds_chosen': 0.5098401308059692, 'epoch': 0.64} {'loss': 2.1174, 'grad_norm': 0.4223003387451172, 'learning_rate': 3.4365962939011693e-07, 'rewards/chosen': -0.19584694504737854, 'rewards/rejected': -0.23475554585456848, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03890858590602875, 'logps/rejected': -2.34755539894104, 'logps/chosen': -1.9584696292877197, 'logits/rejected': -0.3638402819633484, 'logits/chosen': -0.1607169806957245, 'nll_loss': 2.067427158355713, 'log_odds_ratio': -0.5000528693199158, 'log_odds_chosen': 0.4395076036453247, 'epoch': 0.65} {'loss': 2.1205, 'grad_norm': 0.44036707282066345, 'learning_rate': 3.3909016855508867e-07, 'rewards/chosen': -0.1928165853023529, 'rewards/rejected': -0.22500671446323395, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03219012916088104, 'logps/rejected': -2.2500669956207275, 'logps/chosen': -1.9281657934188843, 'logits/rejected': -0.21828660368919373, 'logits/chosen': -0.2543387711048126, 'nll_loss': 2.0675907135009766, 'log_odds_ratio': -0.5295863747596741, 'log_odds_chosen': 0.3687521815299988, 'epoch': 0.65} {'loss': 2.0196, 'grad_norm': 0.42869260907173157, 'learning_rate': 3.345356509253958e-07, 'rewards/chosen': -0.18679141998291016, 'rewards/rejected': -0.22405396401882172, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03726254403591156, 'logps/rejected': -2.24053955078125, 'logps/chosen': -1.867914080619812, 'logits/rejected': -0.32511040568351746, 'logits/chosen': -0.28990602493286133, 'nll_loss': 1.968214511871338, 'log_odds_ratio': -0.513598620891571, 'log_odds_chosen': 0.4268803298473358, 'epoch': 0.65} {'loss': 1.9985, 'grad_norm': 0.43179935216903687, 'learning_rate': 3.299964994652017e-07, 'rewards/chosen': -0.18077686429023743, 'rewards/rejected': -0.21621198952198029, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03543510288000107, 'logps/rejected': -2.1621196269989014, 'logps/chosen': -1.8077688217163086, 'logits/rejected': -0.21216286718845367, 'logits/chosen': -0.29947933554649353, 'nll_loss': 1.945885181427002, 'log_odds_ratio': -0.5262683033943176, 'log_odds_chosen': 0.40741828083992004, 'epoch': 0.65} {'loss': 2.1357, 'grad_norm': 0.5112394690513611, 'learning_rate': 3.2547313571165967e-07, 'rewards/chosen': -0.19549569487571716, 'rewards/rejected': -0.23675595223903656, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04126022756099701, 'logps/rejected': -2.3675594329833984, 'logps/chosen': -1.954957127571106, 'logits/rejected': -0.1256256252527237, 'logits/chosen': -0.39401593804359436, 'nll_loss': 2.0854828357696533, 'log_odds_ratio': -0.5017648935317993, 'log_odds_chosen': 0.46831125020980835, 'epoch': 0.66} {'loss': 2.0979, 'grad_norm': 0.44042789936065674, 'learning_rate': 3.209659797357669e-07, 'rewards/chosen': -0.19284175336360931, 'rewards/rejected': -0.23844575881958008, 'rewards/accuracies': 1.0, 'rewards/margins': 0.045604005455970764, 'logps/rejected': -2.384457588195801, 'logps/chosen': -1.928417444229126, 'logits/rejected': -0.04683329910039902, 'logits/chosen': -0.18000362813472748, 'nll_loss': 2.05072021484375, 'log_odds_ratio': -0.4714072644710541, 'log_odds_chosen': 0.5182204246520996, 'epoch': 0.66} {'loss': 2.0502, 'grad_norm': 0.43828555941581726, 'learning_rate': 3.1647545010335395e-07, 'rewards/chosen': -0.1882575899362564, 'rewards/rejected': -0.2474672943353653, 'rewards/accuracies': 1.0, 'rewards/margins': 0.059209711849689484, 'logps/rejected': -2.47467303276062, 'logps/chosen': -1.8825758695602417, 'logits/rejected': -0.1823115348815918, 'logits/chosen': -0.3079659342765808, 'nll_loss': 2.007534980773926, 'log_odds_ratio': -0.4267301559448242, 'log_odds_chosen': 0.6641589403152466, 'epoch': 0.66} {'loss': 2.0244, 'grad_norm': 0.3798598349094391, 'learning_rate': 3.120019638362136e-07, 'rewards/chosen': -0.1864730715751648, 'rewards/rejected': -0.22846123576164246, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04198816418647766, 'logps/rejected': -2.2846121788024902, 'logps/chosen': -1.864730715751648, 'logits/rejected': -0.10241278260946274, 'logits/chosen': -0.11574582755565643, 'nll_loss': 1.9758269786834717, 'log_odds_ratio': -0.48572006821632385, 'log_odds_chosen': 0.4808882772922516, 'epoch': 0.67} {'loss': 1.9902, 'grad_norm': 0.4033946990966797, 'learning_rate': 3.075459363733727e-07, 'rewards/chosen': -0.1827787160873413, 'rewards/rejected': -0.23323185741901398, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05045313015580177, 'logps/rejected': -2.3323185443878174, 'logps/chosen': -1.8277872800827026, 'logits/rejected': -0.16377201676368713, 'logits/chosen': -0.21031953394412994, 'nll_loss': 1.945261836051941, 'log_odds_ratio': -0.4489781856536865, 'log_odds_chosen': 0.5772659182548523, 'epoch': 0.67} {'loss': 2.1594, 'grad_norm': 0.4857328534126282, 'learning_rate': 3.031077815325132e-07, 'rewards/chosen': -0.19921672344207764, 'rewards/rejected': -0.24898964166641235, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04977291822433472, 'logps/rejected': -2.489896297454834, 'logps/chosen': -1.9921672344207764, 'logits/rejected': -0.19354675710201263, 'logits/chosen': -0.16459237039089203, 'nll_loss': 2.1121487617492676, 'log_odds_ratio': -0.4721669554710388, 'log_odds_chosen': 0.5548877716064453, 'epoch': 0.67} {'loss': 2.0185, 'grad_norm': 0.3858998417854309, 'learning_rate': 2.9868791147154025e-07, 'rewards/chosen': -0.18433226644992828, 'rewards/rejected': -0.21892650425434113, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034594256430864334, 'logps/rejected': -2.189265012741089, 'logps/chosen': -1.843322515487671, 'logits/rejected': -0.2074287384748459, 'logits/chosen': -0.12661641836166382, 'nll_loss': 1.9665818214416504, 'log_odds_ratio': -0.5191612839698792, 'log_odds_chosen': 0.39727193117141724, 'epoch': 0.67} {'loss': 2.0762, 'grad_norm': 0.4435907006263733, 'learning_rate': 2.942867366503077e-07, 'rewards/chosen': -0.19041216373443604, 'rewards/rejected': -0.2381822019815445, 'rewards/accuracies': 1.0, 'rewards/margins': 0.047770045697689056, 'logps/rejected': -2.381821870803833, 'logps/chosen': -1.9041216373443604, 'logits/rejected': -0.09750137478113174, 'logits/chosen': -0.2335578054189682, 'nll_loss': 2.0292389392852783, 'log_odds_ratio': -0.4697742462158203, 'log_odds_chosen': 0.539937436580658, 'epoch': 0.68} {'loss': 2.1306, 'grad_norm': 0.40660807490348816, 'learning_rate': 2.8990466579249917e-07, 'rewards/chosen': -0.19583265483379364, 'rewards/rejected': -0.23237979412078857, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03654715418815613, 'logps/rejected': -2.323798179626465, 'logps/chosen': -1.9583265781402588, 'logits/rejected': -0.20848733186721802, 'logits/chosen': -0.15573078393936157, 'nll_loss': 2.078730583190918, 'log_odds_ratio': -0.5189868211746216, 'log_odds_chosen': 0.41202157735824585, 'epoch': 0.68} {'loss': 1.9834, 'grad_norm': 0.39858436584472656, 'learning_rate': 2.8554210584767184e-07, 'rewards/chosen': -0.1804186999797821, 'rewards/rejected': -0.21945743262767792, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03903874009847641, 'logps/rejected': -2.1945743560791016, 'logps/chosen': -1.8041869401931763, 'logits/rejected': -0.33112263679504395, 'logits/chosen': -0.31871944665908813, 'nll_loss': 1.933552622795105, 'log_odds_ratio': -0.49805763363838196, 'log_odds_chosen': 0.4532400071620941, 'epoch': 0.68} {'loss': 2.1136, 'grad_norm': 0.4925353527069092, 'learning_rate': 2.811994619534637e-07, 'rewards/chosen': -0.1942918300628662, 'rewards/rejected': -0.2318544238805771, 'rewards/accuracies': 1.0, 'rewards/margins': 0.037562571465969086, 'logps/rejected': -2.3185441493988037, 'logps/chosen': -1.9429184198379517, 'logits/rejected': -0.07596518099308014, 'logits/chosen': -0.24002224206924438, 'nll_loss': 2.0626585483551025, 'log_odds_ratio': -0.5094651579856873, 'log_odds_chosen': 0.4248819351196289, 'epoch': 0.68} {'loss': 1.9745, 'grad_norm': 0.44815462827682495, 'learning_rate': 2.768771373979697e-07, 'rewards/chosen': -0.17851634323596954, 'rewards/rejected': -0.23117467761039734, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0526583231985569, 'logps/rejected': -2.31174635887146, 'logps/chosen': -1.785163402557373, 'logits/rejected': -0.15260514616966248, 'logits/chosen': -0.2777319550514221, 'nll_loss': 1.9289604425430298, 'log_odds_ratio': -0.455441415309906, 'log_odds_chosen': 0.6093379855155945, 'epoch': 0.69} {'loss': 2.1236, 'grad_norm': 0.4060370624065399, 'learning_rate': 2.725755335822903e-07, 'rewards/chosen': -0.19416242837905884, 'rewards/rejected': -0.2420864999294281, 'rewards/accuracies': 1.0, 'rewards/margins': 0.047924067825078964, 'logps/rejected': -2.420865058898926, 'logps/chosen': -1.9416241645812988, 'logits/rejected': -0.3438076674938202, 'logits/chosen': -0.1876389980316162, 'nll_loss': 2.076474189758301, 'log_odds_ratio': -0.47121572494506836, 'log_odds_chosen': 0.5381920337677002, 'epoch': 0.69} {'loss': 2.0204, 'grad_norm': 0.462128609418869, 'learning_rate': 2.6829504998325345e-07, 'rewards/chosen': -0.1823728233575821, 'rewards/rejected': -0.2334706038236618, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05109778791666031, 'logps/rejected': -2.3347060680389404, 'logps/chosen': -1.8237282037734985, 'logits/rejected': -0.2352999448776245, 'logits/chosen': -0.27919477224349976, 'nll_loss': 1.9752583503723145, 'log_odds_ratio': -0.45154690742492676, 'log_odds_chosen': 0.5822583436965942, 'epoch': 0.69} {'loss': 2.0929, 'grad_norm': 0.46492475271224976, 'learning_rate': 2.640360841163174e-07, 'rewards/chosen': -0.19320616126060486, 'rewards/rejected': -0.23003314435482025, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03682699054479599, 'logps/rejected': -2.3003315925598145, 'logps/chosen': -1.9320614337921143, 'logits/rejected': -0.2511332929134369, 'logits/chosen': -0.2801150977611542, 'nll_loss': 2.042011260986328, 'log_odds_ratio': -0.508406937122345, 'log_odds_chosen': 0.4209150969982147, 'epoch': 0.7} {'loss': 2.1045, 'grad_norm': 0.4857609272003174, 'learning_rate': 2.597990314986538e-07, 'rewards/chosen': -0.19238559901714325, 'rewards/rejected': -0.22503602504730225, 'rewards/accuracies': 0.875, 'rewards/margins': 0.032650426030159, 'logps/rejected': -2.2503604888916016, 'logps/chosen': -1.9238560199737549, 'logits/rejected': -0.19061926007270813, 'logits/chosen': -0.23271705210208893, 'nll_loss': 2.0513229370117188, 'log_odds_ratio': -0.531346321105957, 'log_odds_chosen': 0.3778655529022217, 'epoch': 0.7} {'loss': 2.0596, 'grad_norm': 0.40434175729751587, 'learning_rate': 2.5558428561241816e-07, 'rewards/chosen': -0.18846413493156433, 'rewards/rejected': -0.22514067590236664, 'rewards/accuracies': 1.0, 'rewards/margins': 0.036676548421382904, 'logps/rejected': -2.251406669616699, 'logps/chosen': -1.884641408920288, 'logits/rejected': -0.293675035238266, 'logits/chosen': -0.14151108264923096, 'nll_loss': 2.008186101913452, 'log_odds_ratio': -0.5145137906074524, 'log_odds_chosen': 0.41764160990715027, 'epoch': 0.7} {'loss': 2.0929, 'grad_norm': 0.4174916446208954, 'learning_rate': 2.5139223786820744e-07, 'rewards/chosen': -0.19182339310646057, 'rewards/rejected': -0.23702432215213776, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04520091414451599, 'logps/rejected': -2.3702430725097656, 'logps/chosen': -1.9182339906692505, 'logits/rejected': -0.3382716178894043, 'logits/chosen': -0.2591693699359894, 'nll_loss': 2.045783519744873, 'log_odds_ratio': -0.4715559780597687, 'log_odds_chosen': 0.5133525133132935, 'epoch': 0.7} {'loss': 2.0681, 'grad_norm': 0.40420272946357727, 'learning_rate': 2.472232775687119e-07, 'rewards/chosen': -0.19031447172164917, 'rewards/rejected': -0.2256007343530655, 'rewards/accuracies': 0.875, 'rewards/margins': 0.035286273807287216, 'logps/rejected': -2.256007432937622, 'logps/chosen': -1.9031445980072021, 'logits/rejected': -0.15366941690444946, 'logits/chosen': -0.2077058106660843, 'nll_loss': 2.015625476837158, 'log_odds_ratio': -0.5246070623397827, 'log_odds_chosen': 0.40430134534835815, 'epoch': 0.71} {'loss': 2.1358, 'grad_norm': 0.44793635606765747, 'learning_rate': 2.430777918725606e-07, 'rewards/chosen': -0.19426265358924866, 'rewards/rejected': -0.24272982776165009, 'rewards/accuracies': 1.0, 'rewards/margins': 0.048467181622982025, 'logps/rejected': -2.4272983074188232, 'logps/chosen': -1.9426264762878418, 'logits/rejected': -0.09219225496053696, 'logits/chosen': -0.2710830867290497, 'nll_loss': 2.089747905731201, 'log_odds_ratio': -0.46093201637268066, 'log_odds_chosen': 0.548711895942688, 'epoch': 0.71} {'loss': 2.1488, 'grad_norm': 0.47333893179893494, 'learning_rate': 2.3895616575836806e-07, 'rewards/chosen': -0.19584107398986816, 'rewards/rejected': -0.23111367225646973, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03527258336544037, 'logps/rejected': -2.3111367225646973, 'logps/chosen': -1.9584107398986816, 'logits/rejected': -0.07071331143379211, 'logits/chosen': -0.17184937000274658, 'nll_loss': 2.096987009048462, 'log_odds_ratio': -0.517660915851593, 'log_odds_chosen': 0.4012176990509033, 'epoch': 0.71} {'loss': 2.1166, 'grad_norm': 0.467342346906662, 'learning_rate': 2.348587819889825e-07, 'rewards/chosen': -0.1944408416748047, 'rewards/rejected': -0.21991188824176788, 'rewards/accuracies': 1.0, 'rewards/margins': 0.025471050292253494, 'logps/rejected': -2.1991188526153564, 'logps/chosen': -1.9444085359573364, 'logits/rejected': -0.1637982726097107, 'logits/chosen': -0.18664903938770294, 'nll_loss': 2.0604355335235596, 'log_odds_ratio': -0.5616452097892761, 'log_odds_chosen': 0.29258960485458374, 'epoch': 0.72} {'loss': 1.9385, 'grad_norm': 0.40307050943374634, 'learning_rate': 2.3078602107593897e-07, 'rewards/chosen': -0.1764669567346573, 'rewards/rejected': -0.22104518115520477, 'rewards/accuracies': 1.0, 'rewards/margins': 0.044578224420547485, 'logps/rejected': -2.21045184135437, 'logps/chosen': -1.76466965675354, 'logits/rejected': -0.06134866178035736, 'logits/chosen': -0.34735918045043945, 'nll_loss': 1.8912420272827148, 'log_odds_ratio': -0.4730759859085083, 'log_odds_chosen': 0.5154957175254822, 'epoch': 0.72} {'loss': 2.0366, 'grad_norm': 0.40177178382873535, 'learning_rate': 2.267382612441231e-07, 'rewards/chosen': -0.1846449077129364, 'rewards/rejected': -0.2185257375240326, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03388083726167679, 'logps/rejected': -2.1852574348449707, 'logps/chosen': -1.8464490175247192, 'logits/rejected': 0.005903229117393494, 'logits/chosen': -0.1719050407409668, 'nll_loss': 1.98414146900177, 'log_odds_ratio': -0.5248652100563049, 'log_odds_chosen': 0.3897167444229126, 'epoch': 0.72} {'loss': 2.034, 'grad_norm': 0.41757282614707947, 'learning_rate': 2.2271587839664668e-07, 'rewards/chosen': -0.18827658891677856, 'rewards/rejected': -0.23392753303050995, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04565096274018288, 'logps/rejected': -2.339275360107422, 'logps/chosen': -1.8827658891677856, 'logits/rejected': -0.11123887449502945, 'logits/chosen': -0.26425325870513916, 'nll_loss': 1.9866759777069092, 'log_odds_ratio': -0.4728841483592987, 'log_odds_chosen': 0.5213944911956787, 'epoch': 0.72} {'loss': 2.1231, 'grad_norm': 0.46502548456192017, 'learning_rate': 2.1871924607993797e-07, 'rewards/chosen': -0.19347071647644043, 'rewards/rejected': -0.23199334740638733, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03852261230349541, 'logps/rejected': -2.3199334144592285, 'logps/chosen': -1.9347072839736938, 'logits/rejected': -0.05642605200409889, 'logits/chosen': -0.15488485991954803, 'nll_loss': 2.072688341140747, 'log_odds_ratio': -0.504368245601654, 'log_odds_chosen': 0.43557122349739075, 'epoch': 0.73} {'loss': 2.074, 'grad_norm': 0.44662994146347046, 'learning_rate': 2.1474873544905203e-07, 'rewards/chosen': -0.18936744332313538, 'rewards/rejected': -0.22044254839420319, 'rewards/accuracies': 1.0, 'rewards/margins': 0.031075095757842064, 'logps/rejected': -2.204425573348999, 'logps/chosen': -1.8936744928359985, 'logits/rejected': -0.0033029038459062576, 'logits/chosen': -0.18652108311653137, 'nll_loss': 2.020807981491089, 'log_odds_ratio': -0.5319563150405884, 'log_odds_chosen': 0.3577326536178589, 'epoch': 0.73} {'loss': 2.0875, 'grad_norm': 0.47074761986732483, 'learning_rate': 2.1080471523320277e-07, 'rewards/chosen': -0.1920810043811798, 'rewards/rejected': -0.2258332371711731, 'rewards/accuracies': 1.0, 'rewards/margins': 0.033752232789993286, 'logps/rejected': -2.2583324909210205, 'logps/chosen': -1.9208102226257324, 'logits/rejected': -0.19018438458442688, 'logits/chosen': -0.23264719545841217, 'nll_loss': 2.0348596572875977, 'log_odds_ratio': -0.5263108015060425, 'log_odds_chosen': 0.38798317313194275, 'epoch': 0.73} {'loss': 2.1498, 'grad_norm': 0.4252146780490875, 'learning_rate': 2.0688755170151994e-07, 'rewards/chosen': -0.19821108877658844, 'rewards/rejected': -0.2321494072675705, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03393830358982086, 'logps/rejected': -2.3214941024780273, 'logps/chosen': -1.9821109771728516, 'logits/rejected': -0.22396771609783173, 'logits/chosen': -0.19508197903633118, 'nll_loss': 2.0970942974090576, 'log_odds_ratio': -0.5270596146583557, 'log_odds_chosen': 0.38214442133903503, 'epoch': 0.73} {'loss': 2.0399, 'grad_norm': 0.4351484477519989, 'learning_rate': 2.029976086290347e-07, 'rewards/chosen': -0.18674813210964203, 'rewards/rejected': -0.24815186858177185, 'rewards/accuracies': 1.0, 'rewards/margins': 0.06140372157096863, 'logps/rejected': -2.481518507003784, 'logps/chosen': -1.8674814701080322, 'logits/rejected': -0.4311271905899048, 'logits/chosen': -0.17970114946365356, 'nll_loss': 1.9990986585617065, 'log_odds_ratio': -0.4082562327384949, 'log_odds_chosen': 0.6953536868095398, 'epoch': 0.74} {'loss': 2.1002, 'grad_norm': 0.4341726005077362, 'learning_rate': 1.991352472628978e-07, 'rewards/chosen': -0.1931062638759613, 'rewards/rejected': -0.23620708286762238, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04310084134340286, 'logps/rejected': -2.3620707988739014, 'logps/chosen': -1.9310623407363892, 'logits/rejected': -0.10476130247116089, 'logits/chosen': -0.13388441503047943, 'nll_loss': 2.051971197128296, 'log_odds_ratio': -0.4817885756492615, 'log_odds_chosen': 0.48897361755371094, 'epoch': 0.74} {'loss': 2.0796, 'grad_norm': 0.410519003868103, 'learning_rate': 1.9530082628883055e-07, 'rewards/chosen': -0.18986576795578003, 'rewards/rejected': -0.23129220306873322, 'rewards/accuracies': 1.0, 'rewards/margins': 0.041426435112953186, 'logps/rejected': -2.312922239303589, 'logps/chosen': -1.8986576795578003, 'logits/rejected': -0.27936267852783203, 'logits/chosen': -0.1638665646314621, 'nll_loss': 2.0300209522247314, 'log_odds_ratio': -0.4956197142601013, 'log_odds_chosen': 0.47287672758102417, 'epoch': 0.74} {'loss': 2.0295, 'grad_norm': 0.3971397280693054, 'learning_rate': 1.9149470179781529e-07, 'rewards/chosen': -0.18639934062957764, 'rewards/rejected': -0.21733081340789795, 'rewards/accuracies': 1.0, 'rewards/margins': 0.030931470915675163, 'logps/rejected': -2.1733081340789795, 'logps/chosen': -1.8639934062957764, 'logits/rejected': -0.2334275245666504, 'logits/chosen': -0.2508685290813446, 'nll_loss': 1.975880742073059, 'log_odds_ratio': -0.5360889434814453, 'log_odds_chosen': 0.3566720187664032, 'epoch': 0.75} {'loss': 2.1673, 'grad_norm': 0.5120834708213806, 'learning_rate': 1.877172272530264e-07, 'rewards/chosen': -0.19745893776416779, 'rewards/rejected': -0.24759721755981445, 'rewards/accuracies': 1.0, 'rewards/margins': 0.050138264894485474, 'logps/rejected': -2.4759721755981445, 'logps/chosen': -1.9745893478393555, 'logits/rejected': 0.013759253546595573, 'logits/chosen': -0.26217329502105713, 'nll_loss': 2.122096061706543, 'log_odds_ratio': -0.4525200128555298, 'log_odds_chosen': 0.5644389390945435, 'epoch': 0.75} {'loss': 2.0233, 'grad_norm': 0.3965185284614563, 'learning_rate': 1.8396875345700496e-07, 'rewards/chosen': -0.1845950186252594, 'rewards/rejected': -0.2227669507265091, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03817194700241089, 'logps/rejected': -2.2276697158813477, 'logps/chosen': -1.8459501266479492, 'logits/rejected': -0.2431364506483078, 'logits/chosen': -0.2846404016017914, 'nll_loss': 1.9729148149490356, 'log_odds_ratio': -0.5040175318717957, 'log_odds_chosen': 0.4397854804992676, 'epoch': 0.75} {'loss': 2.0506, 'grad_norm': 0.40360215306282043, 'learning_rate': 1.8024962851908105e-07, 'rewards/chosen': -0.18986110389232635, 'rewards/rejected': -0.2448522448539734, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05499115586280823, 'logps/rejected': -2.4485225677490234, 'logps/chosen': -1.898611068725586, 'logits/rejected': -0.21754667162895203, 'logits/chosen': -0.17764055728912354, 'nll_loss': 2.0071144104003906, 'log_odds_ratio': -0.4344263970851898, 'log_odds_chosen': 0.6198650598526001, 'epoch': 0.75} {'loss': 2.0072, 'grad_norm': 0.3847593069076538, 'learning_rate': 1.76560197823046e-07, 'rewards/chosen': -0.18321974575519562, 'rewards/rejected': -0.21579232811927795, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03257260471582413, 'logps/rejected': -2.157923460006714, 'logps/chosen': -1.8321974277496338, 'logits/rejected': -0.26148176193237305, 'logits/chosen': -0.3020249307155609, 'nll_loss': 1.954557180404663, 'log_odds_ratio': -0.5261546969413757, 'log_odds_chosen': 0.3786264657974243, 'epoch': 0.76} {'loss': 2.1037, 'grad_norm': 0.47011545300483704, 'learning_rate': 1.7290080399507717e-07, 'rewards/chosen': -0.19168490171432495, 'rewards/rejected': -0.22653940320014954, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03485449403524399, 'logps/rejected': -2.2653939723968506, 'logps/chosen': -1.9168490171432495, 'logits/rejected': -0.01877725124359131, 'logits/chosen': -0.2686113715171814, 'nll_loss': 2.051600217819214, 'log_odds_ratio': -0.5213245749473572, 'log_odds_chosen': 0.3963688015937805, 'epoch': 0.76} {'loss': 1.9662, 'grad_norm': 0.401557594537735, 'learning_rate': 1.6927178687191952e-07, 'rewards/chosen': -0.17967242002487183, 'rewards/rejected': -0.21166963875293732, 'rewards/accuracies': 1.0, 'rewards/margins': 0.031997211277484894, 'logps/rejected': -2.116696357727051, 'logps/chosen': -1.7967240810394287, 'logits/rejected': -0.18011616170406342, 'logits/chosen': -0.2824167311191559, 'nll_loss': 1.9133520126342773, 'log_odds_ratio': -0.5283534526824951, 'log_odds_chosen': 0.37216082215309143, 'epoch': 0.76} {'loss': 1.9706, 'grad_norm': 0.45957428216934204, 'learning_rate': 1.656734834693266e-07, 'rewards/chosen': -0.17964611947536469, 'rewards/rejected': -0.23418167233467102, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05453556776046753, 'logps/rejected': -2.3418169021606445, 'logps/chosen': -1.7964612245559692, 'logits/rejected': -0.1111484244465828, 'logits/chosen': -0.2980007529258728, 'nll_loss': 1.9274060726165771, 'log_odds_ratio': -0.4314519166946411, 'log_odds_chosen': 0.625089704990387, 'epoch': 0.76} {'loss': 2.1419, 'grad_norm': 0.44963356852531433, 'learning_rate': 1.6210622795076167e-07, 'rewards/chosen': -0.1944456845521927, 'rewards/rejected': -0.2343103587627411, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03986464440822601, 'logps/rejected': -2.3431031703948975, 'logps/chosen': -1.944456934928894, 'logits/rejected': -0.10728436708450317, 'logits/chosen': -0.1990780085325241, 'nll_loss': 2.091618537902832, 'log_odds_ratio': -0.5030001401901245, 'log_odds_chosen': 0.45046642422676086, 'epoch': 0.77} {'loss': 2.1694, 'grad_norm': 0.4373071491718292, 'learning_rate': 1.5857035159636622e-07, 'rewards/chosen': -0.19991669058799744, 'rewards/rejected': -0.23591114580631256, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03599446266889572, 'logps/rejected': -2.3591115474700928, 'logps/chosen': -1.9991668462753296, 'logits/rejected': -0.2680577039718628, 'logits/chosen': -0.01834922283887863, 'nll_loss': 2.117833375930786, 'log_odds_ratio': -0.5157204866409302, 'log_odds_chosen': 0.4049380123615265, 'epoch': 0.77} {'loss': 2.0928, 'grad_norm': 0.45289215445518494, 'learning_rate': 1.5506618277219408e-07, 'rewards/chosen': -0.19300661981105804, 'rewards/rejected': -0.24362774193286896, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05062113702297211, 'logps/rejected': -2.436277389526367, 'logps/chosen': -1.9300659894943237, 'logits/rejected': -0.2676662504673004, 'logits/chosen': -0.27342498302459717, 'nll_loss': 2.0461032390594482, 'log_odds_ratio': -0.4670514762401581, 'log_odds_chosen': 0.571408748626709, 'epoch': 0.77} {'loss': 2.1004, 'grad_norm': 0.40905460715293884, 'learning_rate': 1.5159404689971796e-07, 'rewards/chosen': -0.19346947968006134, 'rewards/rejected': -0.22194482386112213, 'rewards/accuracies': 1.0, 'rewards/margins': 0.028475342318415642, 'logps/rejected': -2.2194483280181885, 'logps/chosen': -1.934694766998291, 'logits/rejected': -0.10185020416975021, 'logits/chosen': -0.2184755951166153, 'nll_loss': 2.045550584793091, 'log_odds_ratio': -0.5482557415962219, 'log_odds_chosen': 0.3241649270057678, 'epoch': 0.78} {'loss': 2.0319, 'grad_norm': 0.4077555537223816, 'learning_rate': 1.481542664256075e-07, 'rewards/chosen': -0.18646378815174103, 'rewards/rejected': -0.24395956099033356, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05749577283859253, 'logps/rejected': -2.4395954608917236, 'logps/chosen': -1.8646379709243774, 'logits/rejected': -0.507286548614502, 'logits/chosen': -0.2367272973060608, 'nll_loss': 1.9888720512390137, 'log_odds_ratio': -0.43051740527153015, 'log_odds_chosen': 0.650532603263855, 'epoch': 0.78} {'loss': 2.0688, 'grad_norm': 0.423104852437973, 'learning_rate': 1.447471607917854e-07, 'rewards/chosen': -0.18955425918102264, 'rewards/rejected': -0.2261371612548828, 'rewards/accuracies': 0.75, 'rewards/margins': 0.03658292070031166, 'logps/rejected': -2.261371612548828, 'logps/chosen': -1.8955426216125488, 'logits/rejected': -0.2856886386871338, 'logits/chosen': -0.23630788922309875, 'nll_loss': 2.0160558223724365, 'log_odds_ratio': -0.5273401737213135, 'log_odds_chosen': 0.4105510711669922, 'epoch': 0.78} {'loss': 2.1646, 'grad_norm': 0.45282772183418274, 'learning_rate': 1.413730464057616e-07, 'rewards/chosen': -0.19959735870361328, 'rewards/rejected': -0.23508770763874054, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03549034148454666, 'logps/rejected': -2.350877046585083, 'logps/chosen': -1.9959735870361328, 'logits/rejected': -0.14882172644138336, 'logits/chosen': -0.14440934360027313, 'nll_loss': 2.112144708633423, 'log_odds_ratio': -0.5250266194343567, 'log_odds_chosen': 0.39813441038131714, 'epoch': 0.78} {'loss': 2.0658, 'grad_norm': 0.42741671204566956, 'learning_rate': 1.3803223661124935e-07, 'rewards/chosen': -0.1898500621318817, 'rewards/rejected': -0.22631105780601501, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03646095469594002, 'logps/rejected': -2.2631101608276367, 'logps/chosen': -1.898500680923462, 'logits/rejected': -0.29025012254714966, 'logits/chosen': -0.20184141397476196, 'nll_loss': 2.0147459506988525, 'log_odds_ratio': -0.5100723505020142, 'log_odds_chosen': 0.416368305683136, 'epoch': 0.79} {'loss': 1.9882, 'grad_norm': 0.35584133863449097, 'learning_rate': 1.3472504165906612e-07, 'rewards/chosen': -0.18227334320545197, 'rewards/rejected': -0.216158926486969, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03388558700680733, 'logps/rejected': -2.1615891456604004, 'logps/chosen': -1.8227334022521973, 'logits/rejected': -0.28870323300361633, 'logits/chosen': -0.19002895057201385, 'nll_loss': 1.9361411333084106, 'log_odds_ratio': -0.5203800201416016, 'log_odds_chosen': 0.39061859250068665, 'epoch': 0.79} {'loss': 1.9996, 'grad_norm': 0.39489269256591797, 'learning_rate': 1.3145176867832165e-07, 'rewards/chosen': -0.18206286430358887, 'rewards/rejected': -0.2295069843530655, 'rewards/accuracies': 1.0, 'rewards/margins': 0.047444120049476624, 'logps/rejected': -2.295069932937622, 'logps/chosen': -1.8206287622451782, 'logits/rejected': -0.12423109263181686, 'logits/chosen': -0.08378874510526657, 'nll_loss': 1.952966570854187, 'log_odds_ratio': -0.4665833115577698, 'log_odds_chosen': 0.5395733714103699, 'epoch': 0.79} {'loss': 2.1875, 'grad_norm': 0.4502512216567993, 'learning_rate': 1.2821272164789543e-07, 'rewards/chosen': -0.20250937342643738, 'rewards/rejected': -0.2261410504579544, 'rewards/accuracies': 0.875, 'rewards/margins': 0.023631669580936432, 'logps/rejected': -2.2614104747772217, 'logps/chosen': -2.0250937938690186, 'logits/rejected': -0.14970630407333374, 'logits/chosen': -0.26246315240859985, 'nll_loss': 2.1301777362823486, 'log_odds_ratio': -0.5727296471595764, 'log_odds_chosen': 0.26743370294570923, 'epoch': 0.8} {'loss': 2.177, 'grad_norm': 0.4383101165294647, 'learning_rate': 1.2500820136820733e-07, 'rewards/chosen': -0.19984503090381622, 'rewards/rejected': -0.24775493144989014, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04790988564491272, 'logps/rejected': -2.4775490760803223, 'logps/chosen': -1.9984502792358398, 'logits/rejected': -0.18432113528251648, 'logits/chosen': -0.01988196186721325, 'nll_loss': 2.1300911903381348, 'log_odds_ratio': -0.4690496325492859, 'log_odds_chosen': 0.5365539789199829, 'epoch': 0.8} {'loss': 2.0907, 'grad_norm': 0.45397064089775085, 'learning_rate': 1.2183850543328312e-07, 'rewards/chosen': -0.19052539765834808, 'rewards/rejected': -0.2160310447216034, 'rewards/accuracies': 1.0, 'rewards/margins': 0.02550562657415867, 'logps/rejected': -2.1603102684020996, 'logps/chosen': -1.9052541255950928, 'logits/rejected': -0.1190246194601059, 'logits/chosen': -0.20800651609897614, 'nll_loss': 2.0344083309173584, 'log_odds_ratio': -0.5629580616950989, 'log_odds_chosen': 0.2914395332336426, 'epoch': 0.8} {'loss': 2.1042, 'grad_norm': 0.41133707761764526, 'learning_rate': 1.187039282031182e-07, 'rewards/chosen': -0.19467462599277496, 'rewards/rejected': -0.23198355734348297, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0373089462518692, 'logps/rejected': -2.319835662841797, 'logps/chosen': -1.9467461109161377, 'logits/rejected': -0.11638610064983368, 'logits/chosen': -0.13887159526348114, 'nll_loss': 2.0530924797058105, 'log_odds_ratio': -0.5111529231071472, 'log_odds_chosen': 0.42272239923477173, 'epoch': 0.8} {'loss': 1.9973, 'grad_norm': 0.417402982711792, 'learning_rate': 1.1560476077634069e-07, 'rewards/chosen': -0.18048414587974548, 'rewards/rejected': -0.22771945595741272, 'rewards/accuracies': 1.0, 'rewards/margins': 0.047235310077667236, 'logps/rejected': -2.2771944999694824, 'logps/chosen': -1.8048415184020996, 'logits/rejected': 0.04298261180520058, 'logits/chosen': -0.14703384041786194, 'nll_loss': 1.951300024986267, 'log_odds_ratio': -0.4601641297340393, 'log_odds_chosen': 0.544741153717041, 'epoch': 0.81} {'loss': 2.126, 'grad_norm': 0.45477494597435, 'learning_rate': 1.1254129096317805e-07, 'rewards/chosen': -0.19333416223526, 'rewards/rejected': -0.22766299545764923, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03432881087064743, 'logps/rejected': -2.27662992477417, 'logps/chosen': -1.9333416223526, 'logits/rejected': -0.22473666071891785, 'logits/chosen': -0.13202102482318878, 'nll_loss': 2.0736703872680664, 'log_odds_ratio': -0.523013174533844, 'log_odds_chosen': 0.3906061053276062, 'epoch': 0.81} {'loss': 2.022, 'grad_norm': 0.45089539885520935, 'learning_rate': 1.0951380325872977e-07, 'rewards/chosen': -0.18353353440761566, 'rewards/rejected': -0.22949153184890747, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04595799744129181, 'logps/rejected': -2.294915199279785, 'logps/chosen': -1.835335373878479, 'logits/rejected': -0.12991823256015778, 'logits/chosen': -0.227797731757164, 'nll_loss': 1.9751490354537964, 'log_odds_ratio': -0.4683087170124054, 'log_odds_chosen': 0.5258044600486755, 'epoch': 0.81} {'loss': 2.0407, 'grad_norm': 0.42870861291885376, 'learning_rate': 1.0652257881654624e-07, 'rewards/chosen': -0.1846594363451004, 'rewards/rejected': -0.23002836108207703, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04536893218755722, 'logps/rejected': -2.300283670425415, 'logps/chosen': -1.8465943336486816, 'logits/rejected': -0.15594279766082764, 'logits/chosen': -0.31071117520332336, 'nll_loss': 1.9935269355773926, 'log_odds_ratio': -0.47181975841522217, 'log_odds_chosen': 0.5187486410140991, 'epoch': 0.81} {'loss': 2.0123, 'grad_norm': 0.41282153129577637, 'learning_rate': 1.0356789542251936e-07, 'rewards/chosen': -0.185934916138649, 'rewards/rejected': -0.24323897063732147, 'rewards/accuracies': 1.0, 'rewards/margins': 0.057304054498672485, 'logps/rejected': -2.432389736175537, 'logps/chosen': -1.859349250793457, 'logits/rejected': -0.3672432601451874, 'logits/chosen': -0.11298441886901855, 'nll_loss': 1.9697067737579346, 'log_odds_ratio': -0.4256685674190521, 'log_odds_chosen': 0.648618757724762, 'epoch': 0.82} {'loss': 2.1397, 'grad_norm': 0.45917996764183044, 'learning_rate': 1.006500274690853e-07, 'rewards/chosen': -0.19684001803398132, 'rewards/rejected': -0.24166302382946014, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04482300952076912, 'logps/rejected': -2.416630268096924, 'logps/chosen': -1.9684001207351685, 'logits/rejected': -0.17968234419822693, 'logits/chosen': -0.16094689071178436, 'nll_loss': 2.091897487640381, 'log_odds_ratio': -0.4776912033557892, 'log_odds_chosen': 0.5037029385566711, 'epoch': 0.82} {'loss': 1.9928, 'grad_norm': 0.39604610204696655, 'learning_rate': 9.776924592974256e-08, 'rewards/chosen': -0.18277277052402496, 'rewards/rejected': -0.2336917221546173, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05091894418001175, 'logps/rejected': -2.3369171619415283, 'logps/chosen': -1.8277275562286377, 'logits/rejected': -0.3255308270454407, 'logits/chosen': -0.23677986860275269, 'nll_loss': 1.9474081993103027, 'log_odds_ratio': -0.45360007882118225, 'log_odds_chosen': 0.5864183902740479, 'epoch': 0.82} {'loss': 2.0551, 'grad_norm': 0.4423990249633789, 'learning_rate': 9.492581833388735e-08, 'rewards/chosen': -0.1884804368019104, 'rewards/rejected': -0.23121139407157898, 'rewards/accuracies': 1.0, 'rewards/margins': 0.042730942368507385, 'logps/rejected': -2.3121137619018555, 'logps/chosen': -1.8848044872283936, 'logits/rejected': -0.30755069851875305, 'logits/chosen': -0.2521671950817108, 'nll_loss': 2.0062382221221924, 'log_odds_ratio': -0.4888085126876831, 'log_odds_chosen': 0.4847714900970459, 'epoch': 0.83} {'loss': 2.0882, 'grad_norm': 0.46994680166244507, 'learning_rate': 9.212000874196952e-08, 'rewards/chosen': -0.1924719512462616, 'rewards/rejected': -0.2473946213722229, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0549226775765419, 'logps/rejected': -2.4739463329315186, 'logps/chosen': -1.9247195720672607, 'logits/rejected': -0.30806779861450195, 'logits/chosen': -0.32599297165870667, 'nll_loss': 2.044715642929077, 'log_odds_ratio': -0.4352167248725891, 'log_odds_chosen': 0.6198639273643494, 'epoch': 0.83} {'loss': 2.0637, 'grad_norm': 0.3998923599720001, 'learning_rate': 8.935207772096903e-08, 'rewards/chosen': -0.19146327674388885, 'rewards/rejected': -0.2445802092552185, 'rewards/accuracies': 1.0, 'rewards/margins': 0.053116947412490845, 'logps/rejected': -2.4458022117614746, 'logps/chosen': -1.914632797241211, 'logits/rejected': -0.4473310112953186, 'logits/chosen': -0.2712499499320984, 'nll_loss': 2.0193285942077637, 'log_odds_ratio': -0.4437808096408844, 'log_odds_chosen': 0.5991203784942627, 'epoch': 0.83} {'loss': 1.9577, 'grad_norm': 0.3971911370754242, 'learning_rate': 8.662228232019875e-08, 'rewards/chosen': -0.17802958190441132, 'rewards/rejected': -0.23111258447170258, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05308301001787186, 'logps/rejected': -2.3111257553100586, 'logps/chosen': -1.7802956104278564, 'logits/rejected': -0.11423099786043167, 'logits/chosen': -0.3155173063278198, 'nll_loss': 1.9136923551559448, 'log_odds_ratio': -0.44052228331565857, 'log_odds_chosen': 0.6176374554634094, 'epoch': 0.83} {'loss': 2.1014, 'grad_norm': 0.4174393117427826, 'learning_rate': 8.393087604743283e-08, 'rewards/chosen': -0.19350114464759827, 'rewards/rejected': -0.22750572860240936, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034004583954811096, 'logps/rejected': -2.275057315826416, 'logps/chosen': -1.935011386871338, 'logits/rejected': -0.0011536306701600552, 'logits/chosen': -0.18575572967529297, 'nll_loss': 2.049394369125366, 'log_odds_ratio': -0.5197128057479858, 'log_odds_chosen': 0.38905197381973267, 'epoch': 0.84} {'loss': 2.0462, 'grad_norm': 0.45562076568603516, 'learning_rate': 8.127810884536402e-08, 'rewards/chosen': -0.18564678728580475, 'rewards/rejected': -0.23439012467861176, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0487433560192585, 'logps/rejected': -2.3439013957977295, 'logps/chosen': -1.856467843055725, 'logits/rejected': -0.1494869887828827, 'logits/chosen': -0.13820932805538177, 'nll_loss': 2.0004019737243652, 'log_odds_ratio': -0.4578395485877991, 'log_odds_chosen': 0.5559534430503845, 'epoch': 0.84} {'loss': 2.2194, 'grad_norm': 0.4482463300228119, 'learning_rate': 7.866422706839238e-08, 'rewards/chosen': -0.20293305814266205, 'rewards/rejected': -0.23376816511154175, 'rewards/accuracies': 1.0, 'rewards/margins': 0.030835090205073357, 'logps/rejected': -2.337681531906128, 'logps/chosen': -2.0293304920196533, 'logits/rejected': -0.12020088732242584, 'logits/chosen': -0.2583235800266266, 'nll_loss': 2.1653995513916016, 'log_odds_ratio': -0.5404360890388489, 'log_odds_chosen': 0.348733127117157, 'epoch': 0.84} {'loss': 2.0508, 'grad_norm': 0.39394766092300415, 'learning_rate': 7.608947345974759e-08, 'rewards/chosen': -0.1864737868309021, 'rewards/rejected': -0.24632208049297333, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05984831228852272, 'logps/rejected': -2.4632208347320557, 'logps/chosen': -1.8647377490997314, 'logits/rejected': -0.3051375448703766, 'logits/chosen': -0.17396017909049988, 'nll_loss': 2.00935959815979, 'log_odds_ratio': -0.41480177640914917, 'log_odds_chosen': 0.6800714135169983, 'epoch': 0.85} {'loss': 2.0888, 'grad_norm': 0.4351319968700409, 'learning_rate': 7.355408712894507e-08, 'rewards/chosen': -0.19018332660198212, 'rewards/rejected': -0.22755271196365356, 'rewards/accuracies': 1.0, 'rewards/margins': 0.037369392812252045, 'logps/rejected': -2.275527000427246, 'logps/chosen': -1.9018332958221436, 'logits/rejected': -0.030818209052085876, 'logits/chosen': -0.1508183628320694, 'nll_loss': 2.037964105606079, 'log_odds_ratio': -0.5087683200836182, 'log_odds_chosen': 0.4267576038837433, 'epoch': 0.85} {'loss': 1.9927, 'grad_norm': 0.46579980850219727, 'learning_rate': 7.105830352958142e-08, 'rewards/chosen': -0.181413933634758, 'rewards/rejected': -0.22429785132408142, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04288391023874283, 'logps/rejected': -2.24297833442688, 'logps/chosen': -1.8141393661499023, 'logits/rejected': -0.06965890526771545, 'logits/chosen': -0.30391234159469604, 'nll_loss': 1.9442522525787354, 'log_odds_ratio': -0.48423653841018677, 'log_odds_chosen': 0.49559804797172546, 'epoch': 0.85} {'loss': 2.1235, 'grad_norm': 0.515749454498291, 'learning_rate': 6.86023544374686e-08, 'rewards/chosen': -0.19380497932434082, 'rewards/rejected': -0.2131102830171585, 'rewards/accuracies': 0.75, 'rewards/margins': 0.01930529810488224, 'logps/rejected': -2.1311028003692627, 'logps/chosen': -1.9380499124526978, 'logits/rejected': -0.06818941235542297, 'logits/chosen': -0.27339303493499756, 'nll_loss': 2.0640058517456055, 'log_odds_ratio': -0.5952900648117065, 'log_odds_chosen': 0.22447392344474792, 'epoch': 0.85} {'loss': 2.088, 'grad_norm': 0.4221779704093933, 'learning_rate': 6.618646792910893e-08, 'rewards/chosen': -0.19178378582000732, 'rewards/rejected': -0.23588792979717255, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04410414397716522, 'logps/rejected': -2.3588790893554688, 'logps/chosen': -1.9178377389907837, 'logits/rejected': -0.3814562261104584, 'logits/chosen': -0.173689603805542, 'nll_loss': 2.0399904251098633, 'log_odds_ratio': -0.480294406414032, 'log_odds_chosen': 0.5058144927024841, 'epoch': 0.86} {'loss': 2.1795, 'grad_norm': 0.4498308300971985, 'learning_rate': 6.381086836051497e-08, 'rewards/chosen': -0.19893160462379456, 'rewards/rejected': -0.22166214883327484, 'rewards/accuracies': 0.75, 'rewards/margins': 0.022730525583028793, 'logps/rejected': -2.2166213989257812, 'logps/chosen': -1.9893162250518799, 'logits/rejected': -0.19249173998832703, 'logits/chosen': -0.13349135220050812, 'nll_loss': 2.1212141513824463, 'log_odds_ratio': -0.5829892754554749, 'log_odds_chosen': 0.2599206268787384, 'epoch': 0.86} {'loss': 2.0394, 'grad_norm': 0.4042651057243347, 'learning_rate': 6.147577634637413e-08, 'rewards/chosen': -0.1846935898065567, 'rewards/rejected': -0.23884299397468567, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05414938926696777, 'logps/rejected': -2.388429880142212, 'logps/chosen': -1.8469359874725342, 'logits/rejected': -0.2573563754558563, 'logits/chosen': -0.17402523756027222, 'nll_loss': 1.9960887432098389, 'log_odds_ratio': -0.43272557854652405, 'log_odds_chosen': 0.6165303587913513, 'epoch': 0.86} {'loss': 1.9984, 'grad_norm': 0.38300517201423645, 'learning_rate': 5.918140873956062e-08, 'rewards/chosen': -0.1843004673719406, 'rewards/rejected': -0.22771507501602173, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04341460019350052, 'logps/rejected': -2.2771506309509277, 'logps/chosen': -1.8430047035217285, 'logits/rejected': -0.23760297894477844, 'logits/chosen': -0.17489324510097504, 'nll_loss': 1.9504523277282715, 'log_odds_ratio': -0.47913581132888794, 'log_odds_chosen': 0.4959363341331482, 'epoch': 0.86} {'loss': 2.0904, 'grad_norm': 0.4202573299407959, 'learning_rate': 5.692797861099718e-08, 'rewards/chosen': -0.19210916757583618, 'rewards/rejected': -0.2406761199235916, 'rewards/accuracies': 1.0, 'rewards/margins': 0.048566970974206924, 'logps/rejected': -2.406761407852173, 'logps/chosen': -1.9210916757583618, 'logits/rejected': -0.13348816335201263, 'logits/chosen': -0.08470579981803894, 'nll_loss': 2.0447049140930176, 'log_odds_ratio': -0.4572814404964447, 'log_odds_chosen': 0.5489151477813721, 'epoch': 0.87} {'loss': 2.0665, 'grad_norm': 0.39355114102363586, 'learning_rate': 5.471569522986774e-08, 'rewards/chosen': -0.18986515700817108, 'rewards/rejected': -0.2330189049243927, 'rewards/accuracies': 0.875, 'rewards/margins': 0.043153755366802216, 'logps/rejected': -2.3301892280578613, 'logps/chosen': -1.8986515998840332, 'logits/rejected': -0.18721967935562134, 'logits/chosen': -0.1918748915195465, 'nll_loss': 2.0170679092407227, 'log_odds_ratio': -0.49415305256843567, 'log_odds_chosen': 0.4892618954181671, 'epoch': 0.87} {'loss': 2.1284, 'grad_norm': 0.4295710325241089, 'learning_rate': 5.25447640441834e-08, 'rewards/chosen': -0.19495448470115662, 'rewards/rejected': -0.23239898681640625, 'rewards/accuracies': 0.875, 'rewards/margins': 0.037444520741701126, 'logps/rejected': -2.3239901065826416, 'logps/chosen': -1.949544906616211, 'logits/rejected': -0.12590187788009644, 'logits/chosen': -0.17091691493988037, 'nll_loss': 2.077145576477051, 'log_odds_ratio': -0.5125213265419006, 'log_odds_chosen': 0.42472338676452637, 'epoch': 0.87} {'loss': 2.139, 'grad_norm': 0.49030354619026184, 'learning_rate': 5.041538666170281e-08, 'rewards/chosen': -0.1960122138261795, 'rewards/rejected': -0.23068848252296448, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03467627614736557, 'logps/rejected': -2.306885004043579, 'logps/chosen': -1.9601221084594727, 'logits/rejected': -0.12990185618400574, 'logits/chosen': -0.3289051353931427, 'nll_loss': 2.087029218673706, 'log_odds_ratio': -0.5200437307357788, 'log_odds_chosen': 0.39470869302749634, 'epoch': 0.88} {'loss': 2.1739, 'grad_norm': 0.4893982410430908, 'learning_rate': 4.832776083120982e-08, 'rewards/chosen': -0.20057085156440735, 'rewards/rejected': -0.23888179659843445, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03831096366047859, 'logps/rejected': -2.38881778717041, 'logps/chosen': -2.0057082176208496, 'logits/rejected': -0.21032114326953888, 'logits/chosen': -0.1508714258670807, 'nll_loss': 2.1231908798217773, 'log_odds_ratio': -0.5068095922470093, 'log_odds_chosen': 0.4297412931919098, 'epoch': 0.88} {'loss': 2.0171, 'grad_norm': 0.3878088593482971, 'learning_rate': 4.6282080424148886e-08, 'rewards/chosen': -0.1853950470685959, 'rewards/rejected': -0.23637409508228302, 'rewards/accuracies': 1.0, 'rewards/margins': 0.050979048013687134, 'logps/rejected': -2.363740921020508, 'logps/chosen': -1.8539503812789917, 'logits/rejected': -0.38495659828186035, 'logits/chosen': -0.20354528725147247, 'nll_loss': 1.9717295169830322, 'log_odds_ratio': -0.45386186242103577, 'log_odds_chosen': 0.5795797109603882, 'epoch': 0.88} {'loss': 2.1125, 'grad_norm': 0.4000735580921173, 'learning_rate': 4.427853541662091e-08, 'rewards/chosen': -0.1945798248052597, 'rewards/rejected': -0.22210662066936493, 'rewards/accuracies': 1.0, 'rewards/margins': 0.027526797726750374, 'logps/rejected': -2.2210662364959717, 'logps/chosen': -1.9457982778549194, 'logits/rejected': -0.2006971538066864, 'logits/chosen': -0.06682717800140381, 'nll_loss': 2.0570363998413086, 'log_odds_ratio': -0.5543741583824158, 'log_odds_chosen': 0.31460365653038025, 'epoch': 0.88} {'loss': 2.0726, 'grad_norm': 0.37986594438552856, 'learning_rate': 4.2317311871740644e-08, 'rewards/chosen': -0.18993327021598816, 'rewards/rejected': -0.23954033851623535, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04960707575082779, 'logps/rejected': -2.3954031467437744, 'logps/chosen': -1.8993326425552368, 'logits/rejected': -0.38316231966018677, 'logits/chosen': -0.08007542788982391, 'nll_loss': 2.025761604309082, 'log_odds_ratio': -0.4679988920688629, 'log_odds_chosen': 0.5543714165687561, 'epoch': 0.89} {'loss': 2.0483, 'grad_norm': 0.4185676574707031, 'learning_rate': 4.039859192235778e-08, 'rewards/chosen': -0.18619689345359802, 'rewards/rejected': -0.2132142037153244, 'rewards/accuracies': 0.875, 'rewards/margins': 0.027017315849661827, 'logps/rejected': -2.1321418285369873, 'logps/chosen': -1.861968994140625, 'logits/rejected': -0.11621041595935822, 'logits/chosen': -0.14175063371658325, 'nll_loss': 1.992612600326538, 'log_odds_ratio': -0.5566931366920471, 'log_odds_chosen': 0.31169041991233826, 'epoch': 0.89} {'loss': 2.1359, 'grad_norm': 0.4270247519016266, 'learning_rate': 3.852255375414271e-08, 'rewards/chosen': -0.1940581053495407, 'rewards/rejected': -0.22706599533557892, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03300788253545761, 'logps/rejected': -2.270659923553467, 'logps/chosen': -1.9405810832977295, 'logits/rejected': -0.2174735814332962, 'logits/chosen': -0.2114194631576538, 'nll_loss': 2.081935405731201, 'log_odds_ratio': -0.5399094223976135, 'log_odds_chosen': 0.3725894093513489, 'epoch': 0.89} {'loss': 2.1772, 'grad_norm': 0.4771701693534851, 'learning_rate': 3.668937158903901e-08, 'rewards/chosen': -0.20157849788665771, 'rewards/rejected': -0.2531397044658661, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05156121402978897, 'logps/rejected': -2.5313968658447266, 'logps/chosen': -2.015784740447998, 'logits/rejected': -0.1700681895017624, 'logits/chosen': -0.2463836818933487, 'nll_loss': 2.1319868564605713, 'log_odds_ratio': -0.45257288217544556, 'log_odds_chosen': 0.5787175297737122, 'epoch': 0.89} {'loss': 2.0642, 'grad_norm': 0.42737582325935364, 'learning_rate': 3.489921566908371e-08, 'rewards/chosen': -0.18919143080711365, 'rewards/rejected': -0.23052623867988586, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04133479669690132, 'logps/rejected': -2.305262327194214, 'logps/chosen': -1.8919143676757812, 'logits/rejected': -0.27947261929512024, 'logits/chosen': -0.16491183638572693, 'nll_loss': 2.015066385269165, 'log_odds_ratio': -0.4908846318721771, 'log_odds_chosen': 0.4688037931919098, 'epoch': 0.9} {'loss': 2.1874, 'grad_norm': 0.4799785017967224, 'learning_rate': 3.3152252240598086e-08, 'rewards/chosen': -0.2000555843114853, 'rewards/rejected': -0.230727881193161, 'rewards/accuracies': 1.0, 'rewards/margins': 0.030672287568449974, 'logps/rejected': -2.307278633117676, 'logps/chosen': -2.0005557537078857, 'logits/rejected': 0.09353294223546982, 'logits/chosen': -0.14339762926101685, 'nll_loss': 2.1336636543273926, 'log_odds_ratio': -0.5378261208534241, 'log_odds_chosen': 0.3481528162956238, 'epoch': 0.9} {'loss': 2.1447, 'grad_norm': 0.4857260584831238, 'learning_rate': 3.144864353874804e-08, 'rewards/chosen': -0.19487199187278748, 'rewards/rejected': -0.23177646100521088, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0369044728577137, 'logps/rejected': -2.3177647590637207, 'logps/chosen': -1.9487199783325195, 'logits/rejected': -0.03285370022058487, 'logits/chosen': -0.24292851984500885, 'nll_loss': 2.093341827392578, 'log_odds_ratio': -0.5131241083145142, 'log_odds_chosen': 0.41800910234451294, 'epoch': 0.9} {'loss': 1.9433, 'grad_norm': 0.43344756960868835, 'learning_rate': 2.978854777247841e-08, 'rewards/chosen': -0.1790362149477005, 'rewards/rejected': -0.2297755479812622, 'rewards/accuracies': 1.0, 'rewards/margins': 0.05073932558298111, 'logps/rejected': -2.297755479812622, 'logps/chosen': -1.790362000465393, 'logits/rejected': -0.31945887207984924, 'logits/chosen': -0.26334643363952637, 'nll_loss': 1.8982830047607422, 'log_odds_ratio': -0.4501224160194397, 'log_odds_chosen': 0.5819824934005737, 'epoch': 0.91} {'loss': 2.0441, 'grad_norm': 0.42716139554977417, 'learning_rate': 2.817211910982037e-08, 'rewards/chosen': -0.18751075863838196, 'rewards/rejected': -0.24470902979373932, 'rewards/accuracies': 1.0, 'rewards/margins': 0.057198263704776764, 'logps/rejected': -2.4470901489257812, 'logps/chosen': -1.8751076459884644, 'logits/rejected': -0.15542913973331451, 'logits/chosen': -0.202769935131073, 'nll_loss': 2.0014681816101074, 'log_odds_ratio': -0.42589282989501953, 'log_odds_chosen': 0.649154543876648, 'epoch': 0.91} {'loss': 2.1396, 'grad_norm': 0.4449755847454071, 'learning_rate': 2.6599507663574384e-08, 'rewards/chosen': -0.19731488823890686, 'rewards/rejected': -0.23602209985256195, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03870721161365509, 'logps/rejected': -2.3602209091186523, 'logps/chosen': -1.9731489419937134, 'logits/rejected': -0.1446068286895752, 'logits/chosen': -0.14027515053749084, 'nll_loss': 2.0895612239837646, 'log_odds_ratio': -0.4999912977218628, 'log_odds_chosen': 0.43763425946235657, 'epoch': 0.91} {'loss': 2.0324, 'grad_norm': 0.4197101891040802, 'learning_rate': 2.5070859477369642e-08, 'rewards/chosen': -0.18657900393009186, 'rewards/rejected': -0.23098163306713104, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04440262168645859, 'logps/rejected': -2.309816360473633, 'logps/chosen': -1.8657901287078857, 'logits/rejected': -0.3043823838233948, 'logits/chosen': -0.0747680813074112, 'nll_loss': 1.9844887256622314, 'log_odds_ratio': -0.47924864292144775, 'log_odds_chosen': 0.5072610974311829, 'epoch': 0.91} {'loss': 2.0877, 'grad_norm': 0.3947368562221527, 'learning_rate': 2.358631651210141e-08, 'rewards/chosen': -0.19082088768482208, 'rewards/rejected': -0.24037298560142517, 'rewards/accuracies': 1.0, 'rewards/margins': 0.049552083015441895, 'logps/rejected': -2.4037296772003174, 'logps/chosen': -1.908208966255188, 'logits/rejected': -0.3236241340637207, 'logits/chosen': -0.23719505965709686, 'nll_loss': 2.0417838096618652, 'log_odds_ratio': -0.45878544449806213, 'log_odds_chosen': 0.5603442788124084, 'epoch': 0.92} {'loss': 2.0087, 'grad_norm': 0.4243737459182739, 'learning_rate': 2.214601663274762e-08, 'rewards/chosen': -0.18259567022323608, 'rewards/rejected': -0.21251866221427917, 'rewards/accuracies': 0.875, 'rewards/margins': 0.02992301806807518, 'logps/rejected': -2.1251866817474365, 'logps/chosen': -1.8259567022323608, 'logits/rejected': -0.14712917804718018, 'logits/chosen': -0.3143972158432007, 'nll_loss': 1.9534207582473755, 'log_odds_ratio': -0.5531291365623474, 'log_odds_chosen': 0.3393900990486145, 'epoch': 0.92} {'loss': 2.0948, 'grad_norm': 0.37499305605888367, 'learning_rate': 2.0750093595565733e-08, 'rewards/chosen': -0.19290241599082947, 'rewards/rejected': -0.23633918166160583, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04343675822019577, 'logps/rejected': -2.363391876220703, 'logps/chosen': -1.9290242195129395, 'logits/rejected': -0.2299581617116928, 'logits/chosen': -0.14541703462600708, 'nll_loss': 2.0463147163391113, 'log_odds_ratio': -0.4850914180278778, 'log_odds_chosen': 0.49258458614349365, 'epoch': 0.92} {'loss': 2.1732, 'grad_norm': 0.44641590118408203, 'learning_rate': 1.9398677035671218e-08, 'rewards/chosen': -0.20193640887737274, 'rewards/rejected': -0.23952704668045044, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03759066015481949, 'logps/rejected': -2.395270586013794, 'logps/chosen': -2.0193638801574707, 'logits/rejected': -0.17076365649700165, 'logits/chosen': -0.1494864821434021, 'nll_loss': 2.122241973876953, 'log_odds_ratio': -0.5091593861579895, 'log_odds_chosen': 0.42436647415161133, 'epoch': 0.93} {'loss': 2.087, 'grad_norm': 0.4320031702518463, 'learning_rate': 1.8091892454998593e-08, 'rewards/chosen': -0.19114135205745697, 'rewards/rejected': -0.2323937565088272, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04125239700078964, 'logps/rejected': -2.32393741607666, 'logps/chosen': -1.911413550376892, 'logits/rejected': -0.10504031181335449, 'logits/chosen': -0.10691366344690323, 'nll_loss': 2.037943124771118, 'log_odds_ratio': -0.4904363751411438, 'log_odds_chosen': 0.4674716889858246, 'epoch': 0.93} {'loss': 1.9905, 'grad_norm': 0.396973580121994, 'learning_rate': 1.682986121064689e-08, 'rewards/chosen': -0.181071937084198, 'rewards/rejected': -0.20561034977436066, 'rewards/accuracies': 0.875, 'rewards/margins': 0.024538416415452957, 'logps/rejected': -2.056103229522705, 'logps/chosen': -1.8107192516326904, 'logits/rejected': -0.11924867331981659, 'logits/chosen': -0.2744186520576477, 'nll_loss': 1.934165596961975, 'log_odds_ratio': -0.5638278722763062, 'log_odds_chosen': 0.2887546718120575, 'epoch': 0.93} {'loss': 2.0447, 'grad_norm': 0.4459056556224823, 'learning_rate': 1.5612700503608967e-08, 'rewards/chosen': -0.18574512004852295, 'rewards/rejected': -0.22552061080932617, 'rewards/accuracies': 1.0, 'rewards/margins': 0.039775483310222626, 'logps/rejected': -2.2552061080932617, 'logps/chosen': -1.8574512004852295, 'logits/rejected': -0.37708672881126404, 'logits/chosen': -0.36018478870391846, 'nll_loss': 1.9939401149749756, 'log_odds_ratio': -0.5071058869361877, 'log_odds_chosen': 0.45278072357177734, 'epoch': 0.93} {'loss': 1.9768, 'grad_norm': 0.40658360719680786, 'learning_rate': 1.444052336788787e-08, 'rewards/chosen': -0.18056757748126984, 'rewards/rejected': -0.2195272445678711, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03895965963602066, 'logps/rejected': -2.195272445678711, 'logps/chosen': -1.8056758642196655, 'logits/rejected': -0.2852480709552765, 'logits/chosen': -0.12710537016391754, 'nll_loss': 1.926626205444336, 'log_odds_ratio': -0.5013195872306824, 'log_odds_chosen': 0.4479530453681946, 'epoch': 0.94} {'loss': 2.0094, 'grad_norm': 0.38530224561691284, 'learning_rate': 1.3313438659999399e-08, 'rewards/chosen': -0.18249857425689697, 'rewards/rejected': -0.2275277078151703, 'rewards/accuracies': 0.875, 'rewards/margins': 0.04502914100885391, 'logps/rejected': -2.2752771377563477, 'logps/chosen': -1.8249857425689697, 'logits/rejected': -0.1850556582212448, 'logits/chosen': -0.14443586766719818, 'nll_loss': 1.9606778621673584, 'log_odds_ratio': -0.4869002103805542, 'log_odds_chosen': 0.5152425765991211, 'epoch': 0.94} {'loss': 2.0469, 'grad_norm': 0.45364004373550415, 'learning_rate': 1.223155104886342e-08, 'rewards/chosen': -0.18658114969730377, 'rewards/rejected': -0.2210913896560669, 'rewards/accuracies': 1.0, 'rewards/margins': 0.034510232508182526, 'logps/rejected': -2.21091365814209, 'logps/chosen': -1.8658114671707153, 'logits/rejected': -0.06684684753417969, 'logits/chosen': -0.24809373915195465, 'nll_loss': 1.9946162700653076, 'log_odds_ratio': -0.5225122570991516, 'log_odds_chosen': 0.39692723751068115, 'epoch': 0.94} {'loss': 1.9827, 'grad_norm': 0.39424437284469604, 'learning_rate': 1.119496100608297e-08, 'rewards/chosen': -0.17855985462665558, 'rewards/rejected': -0.21128283441066742, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03272298350930214, 'logps/rejected': -2.112828254699707, 'logps/chosen': -1.7855983972549438, 'logits/rejected': -0.2984020709991455, 'logits/chosen': -0.2839021384716034, 'nll_loss': 1.930551290512085, 'log_odds_ratio': -0.5215116739273071, 'log_odds_chosen': 0.3901040554046631, 'epoch': 0.94} {'loss': 2.0737, 'grad_norm': 0.42831146717071533, 'learning_rate': 1.0203764796614056e-08, 'rewards/chosen': -0.19064341485500336, 'rewards/rejected': -0.23443713784217834, 'rewards/accuracies': 0.875, 'rewards/margins': 0.043793708086013794, 'logps/rejected': -2.3443710803985596, 'logps/chosen': -1.9064340591430664, 'logits/rejected': -0.22582583129405975, 'logits/chosen': -0.2962571084499359, 'nll_loss': 2.0255472660064697, 'log_odds_ratio': -0.481975793838501, 'log_odds_chosen': 0.4961273670196533, 'epoch': 0.95} {'loss': 2.056, 'grad_norm': 0.4325099587440491, 'learning_rate': 9.258054469825972e-09, 'rewards/chosen': -0.18755343556404114, 'rewards/rejected': -0.230577290058136, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04302385449409485, 'logps/rejected': -2.3057727813720703, 'logps/chosen': -1.8755344152450562, 'logits/rejected': -0.17450091242790222, 'logits/chosen': -0.17820492386817932, 'nll_loss': 2.0078787803649902, 'log_odds_ratio': -0.48156440258026123, 'log_odds_chosen': 0.49040326476097107, 'epoch': 0.95} {'loss': 2.1447, 'grad_norm': 0.4622589647769928, 'learning_rate': 8.3579178509528e-09, 'rewards/chosen': -0.19416610896587372, 'rewards/rejected': -0.24332500994205475, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04915890470147133, 'logps/rejected': -2.4332499504089355, 'logps/chosen': -1.9416611194610596, 'logits/rejected': -0.1634458750486374, 'logits/chosen': -0.11451496928930283, 'nll_loss': 2.098790168762207, 'log_odds_ratio': -0.4589051902294159, 'log_odds_chosen': 0.5557539463043213, 'epoch': 0.95} {'loss': 1.9615, 'grad_norm': 0.42654889822006226, 'learning_rate': 7.503438532937168e-09, 'rewards/chosen': -0.17952464520931244, 'rewards/rejected': -0.22983895242214203, 'rewards/accuracies': 1.0, 'rewards/margins': 0.050314296036958694, 'logps/rejected': -2.298389434814453, 'logps/chosen': -1.7952463626861572, 'logits/rejected': -0.22434376180171967, 'logits/chosen': -0.3199605941772461, 'nll_loss': 1.9161052703857422, 'log_odds_ratio': -0.4537157118320465, 'log_odds_chosen': 0.5813964605331421, 'epoch': 0.96} {'loss': 2.1994, 'grad_norm': 0.4929260313510895, 'learning_rate': 6.694695868667554e-09, 'rewards/chosen': -0.20295149087905884, 'rewards/rejected': -0.224810391664505, 'rewards/accuracies': 0.875, 'rewards/margins': 0.021858885884284973, 'logps/rejected': -2.2481038570404053, 'logps/chosen': -2.029514789581299, 'logits/rejected': -0.20021158456802368, 'logits/chosen': -0.24652338027954102, 'nll_loss': 2.141369342803955, 'log_odds_ratio': -0.5799821615219116, 'log_odds_chosen': 0.24821370840072632, 'epoch': 0.96} {'loss': 2.0628, 'grad_norm': 0.4227042496204376, 'learning_rate': 5.931764963608865e-09, 'rewards/chosen': -0.1869337260723114, 'rewards/rejected': -0.22021916508674622, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03328544646501541, 'logps/rejected': -2.2021918296813965, 'logps/chosen': -1.8693372011184692, 'logits/rejected': 0.02388381026685238, 'logits/chosen': -0.20029394328594208, 'nll_loss': 2.010028600692749, 'log_odds_ratio': -0.5276705026626587, 'log_odds_chosen': 0.38164404034614563, 'epoch': 0.96} {'loss': 2.0468, 'grad_norm': 0.40184086561203003, 'learning_rate': 5.214716668827557e-09, 'rewards/chosen': -0.18679484724998474, 'rewards/rejected': -0.23385639488697052, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04706154763698578, 'logps/rejected': -2.338563919067383, 'logps/chosen': -1.8679484128952026, 'logits/rejected': -0.25241750478744507, 'logits/chosen': -0.1988634169101715, 'nll_loss': 1.9996461868286133, 'log_odds_ratio': -0.4719686806201935, 'log_odds_chosen': 0.5314087271690369, 'epoch': 0.96} {'loss': 1.9971, 'grad_norm': 0.4056241512298584, 'learning_rate': 4.543617574412184e-09, 'rewards/chosen': -0.1817009598016739, 'rewards/rejected': -0.22090913355350494, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03920818120241165, 'logps/rejected': -2.2090914249420166, 'logps/chosen': -1.8170093297958374, 'logits/rejected': -0.09356098622083664, 'logits/chosen': -0.18360307812690735, 'nll_loss': 1.947261095046997, 'log_odds_ratio': -0.4986838698387146, 'log_odds_chosen': 0.45314475893974304, 'epoch': 0.97} {'loss': 2.1481, 'grad_norm': 0.4642251133918762, 'learning_rate': 3.9185300032889e-09, 'rewards/chosen': -0.19764922559261322, 'rewards/rejected': -0.2325756698846817, 'rewards/accuracies': 0.875, 'rewards/margins': 0.034926433116197586, 'logps/rejected': -2.325756788253784, 'logps/chosen': -1.9764922857284546, 'logits/rejected': -0.2326173335313797, 'logits/chosen': -0.2754601836204529, 'nll_loss': 2.0957164764404297, 'log_odds_ratio': -0.5235860347747803, 'log_odds_chosen': 0.3950684368610382, 'epoch': 0.97} {'loss': 2.0549, 'grad_norm': 0.4343326687812805, 'learning_rate': 3.3395120054343086e-09, 'rewards/chosen': -0.18860295414924622, 'rewards/rejected': -0.22324594855308533, 'rewards/accuracies': 0.875, 'rewards/margins': 0.034643009305000305, 'logps/rejected': -2.232459545135498, 'logps/chosen': -1.886029601097107, 'logits/rejected': -0.18407291173934937, 'logits/chosen': -0.24127677083015442, 'nll_loss': 2.002397060394287, 'log_odds_ratio': -0.525107741355896, 'log_odds_chosen': 0.3908619284629822, 'epoch': 0.97} {'loss': 2.0918, 'grad_norm': 0.44272759556770325, 'learning_rate': 2.8066173524839977e-09, 'rewards/chosen': -0.19302789866924286, 'rewards/rejected': -0.22968368232250214, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03665578365325928, 'logps/rejected': -2.2968368530273438, 'logps/chosen': -1.9302788972854614, 'logits/rejected': -0.15310092270374298, 'logits/chosen': -0.24375535547733307, 'nll_loss': 2.040801525115967, 'log_odds_ratio': -0.5094950199127197, 'log_odds_chosen': 0.4167912006378174, 'epoch': 0.97} {'loss': 2.1045, 'grad_norm': 0.45615145564079285, 'learning_rate': 2.3198955327393686e-09, 'rewards/chosen': -0.1922983080148697, 'rewards/rejected': -0.23313231766223907, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04083399474620819, 'logps/rejected': -2.3313231468200684, 'logps/chosen': -1.922983169555664, 'logits/rejected': -0.11756740510463715, 'logits/chosen': -0.23265661299228668, 'nll_loss': 2.0555341243743896, 'log_odds_ratio': -0.4895220994949341, 'log_odds_chosen': 0.4656871557235718, 'epoch': 0.98} {'loss': 1.9018, 'grad_norm': 0.4014591872692108, 'learning_rate': 1.8793917465713684e-09, 'rewards/chosen': -0.17173448204994202, 'rewards/rejected': -0.22362574934959412, 'rewards/accuracies': 1.0, 'rewards/margins': 0.0518912598490715, 'logps/rejected': -2.236257314682007, 'logps/chosen': -1.7173447608947754, 'logits/rejected': -0.1527189016342163, 'logits/chosen': -0.1957464963197708, 'nll_loss': 1.8569546937942505, 'log_odds_ratio': -0.44817936420440674, 'log_odds_chosen': 0.6040939688682556, 'epoch': 0.98} {'loss': 2.0743, 'grad_norm': 0.40029025077819824, 'learning_rate': 1.4851469022233997e-09, 'rewards/chosen': -0.18896281719207764, 'rewards/rejected': -0.222550168633461, 'rewards/accuracies': 0.875, 'rewards/margins': 0.03358734771609306, 'logps/rejected': -2.225501775741577, 'logps/chosen': -1.889628291130066, 'logits/rejected': -0.47735050320625305, 'logits/chosen': -0.27820494771003723, 'nll_loss': 2.021350860595703, 'log_odds_ratio': -0.529025673866272, 'log_odds_chosen': 0.3815869390964508, 'epoch': 0.98} {'loss': 2.1139, 'grad_norm': 0.40846526622772217, 'learning_rate': 1.1371976120118088e-09, 'rewards/chosen': -0.19486680626869202, 'rewards/rejected': -0.2388983815908432, 'rewards/accuracies': 1.0, 'rewards/margins': 0.04403157904744148, 'logps/rejected': -2.388983726501465, 'logps/chosen': -1.9486680030822754, 'logits/rejected': -0.14713740348815918, 'logits/chosen': -0.09253039211034775, 'nll_loss': 2.0657119750976562, 'log_odds_ratio': -0.4815439283847809, 'log_odds_chosen': 0.4965031147003174, 'epoch': 0.99} {'loss': 1.9848, 'grad_norm': 0.4720138609409332, 'learning_rate': 8.35576188926046e-10, 'rewards/chosen': -0.17912402749061584, 'rewards/rejected': -0.21588559448719025, 'rewards/accuracies': 1.0, 'rewards/margins': 0.036761559545993805, 'logps/rejected': -2.158855676651001, 'logps/chosen': -1.7912402153015137, 'logits/rejected': 0.027382859960198402, 'logits/chosen': -0.4898189902305603, 'nll_loss': 1.9342451095581055, 'log_odds_ratio': -0.5054396390914917, 'log_odds_chosen': 0.43012499809265137, 'epoch': 0.99} {'loss': 2.0238, 'grad_norm': 0.42207953333854675, 'learning_rate': 5.80310643627957e-10, 'rewards/chosen': -0.18451660871505737, 'rewards/rejected': -0.2227260023355484, 'rewards/accuracies': 1.0, 'rewards/margins': 0.03820939362049103, 'logps/rejected': -2.227259874343872, 'logps/chosen': -1.8451660871505737, 'logits/rejected': -0.22522786259651184, 'logits/chosen': -0.212925985455513, 'nll_loss': 1.9736899137496948, 'log_odds_ratio': -0.5013801455497742, 'log_odds_chosen': 0.4414942264556885, 'epoch': 0.99} {'loss': 2.05, 'grad_norm': 0.43415895104408264, 'learning_rate': 3.71424681850141e-10, 'rewards/chosen': -0.1866944432258606, 'rewards/rejected': -0.2458716332912445, 'rewards/accuracies': 0.875, 'rewards/margins': 0.05917719751596451, 'logps/rejected': -2.45871639251709, 'logps/chosen': -1.8669443130493164, 'logits/rejected': -0.3631182312965393, 'logits/chosen': -0.20278087258338928, 'nll_loss': 2.0075032711029053, 'log_odds_ratio': -0.42513564229011536, 'log_odds_chosen': 0.6658082008361816, 'epoch': 0.99} {'loss': 1.9504, 'grad_norm': 0.37619003653526306, 'learning_rate': 2.0893770219493344e-10, 'rewards/chosen': -0.1759258508682251, 'rewards/rejected': -0.21549852192401886, 'rewards/accuracies': 1.0, 'rewards/margins': 0.039572667330503464, 'logps/rejected': -2.154985189437866, 'logps/chosen': -1.759258508682251, 'logits/rejected': -0.09165595471858978, 'logits/chosen': -0.29233649373054504, 'nll_loss': 1.900978446006775, 'log_odds_ratio': -0.49466732144355774, 'log_odds_chosen': 0.4615989327430725, 'epoch': 1.0} {'loss': 2.0612, 'grad_norm': 0.41035768389701843, 'learning_rate': 9.286479433257e-11, 'rewards/chosen': -0.18897803127765656, 'rewards/rejected': -0.23044171929359436, 'rewards/accuracies': 1.0, 'rewards/margins': 0.041463691741228104, 'logps/rejected': -2.304417133331299, 'logps/chosen': -1.8897802829742432, 'logits/rejected': -0.2605111002922058, 'logits/chosen': -0.2844846844673157, 'nll_loss': 2.011777877807617, 'log_odds_ratio': -0.4945923686027527, 'log_odds_chosen': 0.46994221210479736, 'epoch': 1.0} {'loss': 2.1647, 'grad_norm': 0.5401268601417542, 'learning_rate': 2.321673760002918e-11, 'rewards/chosen': -0.19792285561561584, 'rewards/rejected': -0.21805314719676971, 'rewards/accuracies': 1.0, 'rewards/margins': 0.02013029158115387, 'logps/rejected': -2.1805315017700195, 'logps/chosen': -1.9792284965515137, 'logits/rejected': -0.25849246978759766, 'logits/chosen': -0.319807767868042, 'nll_loss': 2.1062676906585693, 'log_odds_ratio': -0.5846931338310242, 'log_odds_chosen': 0.2301175892353058, 'epoch': 1.0} {'train_runtime': 2774.0244, 'train_samples_per_second': 1.044, 'train_steps_per_second': 0.131, 'train_loss': 2.112552823114001, 'epoch': 1.0}                                                                                                                    100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 363/363 [46:14<00:00,  7.64s/it] ใƒญใƒผใ‚ซใƒซไฟๅญ˜ไธญ... Plots saved. Uploading to Hugging Face: gitpullpull/Introspective_Temperature_test Destination folder: run_20251218_140435 Processing Files (10 / 10)    : 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  811MB /  811MB,  110MB/s   New Data Upload               : 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  620MB /  620MB, 83.8MB/s     ...eckpoint-363/optimizer.pt: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 89.0MB / 89.0MB               ...kpoint-363/tokenizer.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 11.4MB / 11.4MB               ...utput_orpo/tokenizer.json: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 11.4MB / 11.4MB               ...ckpoint-363/rng_state.pth: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 14.6kB / 14.6kB               ...adapter_model.safetensors: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  349MB /  349MB               ...adapter_model.safetensors: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  349MB /  349MB               ...eckpoint-363/scheduler.pt: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 1.47kB / 1.47kB               ...int-363/training_args.bin: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 6.22kB / 6.22kB               ...66923.e150a2d13ef8.1700.0: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  317kB /  317kB               ...plots_20251218_140435.png: 100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ|  149kB /  149kB             Upload Job Finished Successfully! Saved to: https://huggingface.co/gitpullpull/Introspective_Temperature_test/tree/main/run_20251218_140435 ๅฎŒไบ†ใ€‚ === Job Finished === unsloth@e150a2d13ef8:/workspace/Introspective_Temperature$