Instructions to use SoonOk/SFTWeAgents with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SoonOk/SFTWeAgents with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SoonOk/SFTWeAgents")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SoonOk/SFTWeAgents")
model = AutoModelForCausalLM.from_pretrained("SoonOk/SFTWeAgents", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SoonOk/SFTWeAgents with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SoonOk/SFTWeAgents"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SoonOk/SFTWeAgents",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/SoonOk/SFTWeAgents

SGLang

How to use SoonOk/SFTWeAgents with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SoonOk/SFTWeAgents" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SoonOk/SFTWeAgents",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SoonOk/SFTWeAgents" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SoonOk/SFTWeAgents",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use SoonOk/SFTWeAgents with Docker Model Runner:
```
docker model run hf.co/SoonOk/SFTWeAgents
```

SoonOk commited on Apr 20

Commit

4e052ac

verified ·

1 Parent(s): 54d4dc2

Upload folder using huggingface_hub

Browse files

Files changed (17) hide show

.gitattributes +1 -0
makto_v1/added_tokens.json +24 -0
makto_v1/chat_template.jinja +54 -0
makto_v1/config.json +58 -0
makto_v1/generation_config.json +13 -0
makto_v1/merges.txt +0 -0
makto_v1/model-00001-of-00004.safetensors +3 -0
makto_v1/model-00002-of-00004.safetensors +3 -0
makto_v1/model-00003-of-00004.safetensors +3 -0
makto_v1/model-00004-of-00004.safetensors +3 -0
makto_v1/model.safetensors.index.json +347 -0
makto_v1/special_tokens_map.json +31 -0
makto_v1/tokenizer.json +3 -0
makto_v1/tokenizer_config.json +207 -0
makto_v1/trainer_state.json +3034 -0
makto_v1/training_args.bin +3 -0
makto_v1/vocab.json +0 -0

.gitattributes CHANGED Viewed

@@ -45,3 +45,4 @@ kto_dataset_train_samples.json filter=lfs diff=lfs merge=lfs -text
 kto_dataset_no_recon.json filter=lfs diff=lfs merge=lfs -text
 kto_dataset_true_recon.json filter=lfs diff=lfs merge=lfs -text
 makto.json filter=lfs diff=lfs merge=lfs -text

 kto_dataset_no_recon.json filter=lfs diff=lfs merge=lfs -text
 kto_dataset_true_recon.json filter=lfs diff=lfs merge=lfs -text
 makto.json filter=lfs diff=lfs merge=lfs -text
+makto_v1/tokenizer.json filter=lfs diff=lfs merge=lfs -text

makto_v1/added_tokens.json ADDED Viewed

	@@ -0,0 +1,24 @@

+{
+  "</tool_call>": 151658,
+  "<tool_call>": 151657,
+  "<|box_end|>": 151649,
+  "<|box_start|>": 151648,
+  "<|endoftext|>": 151643,
+  "<|file_sep|>": 151664,
+  "<|fim_middle|>": 151660,
+  "<|fim_pad|>": 151662,
+  "<|fim_prefix|>": 151659,
+  "<|fim_suffix|>": 151661,
+  "<|im_end|>": 151645,
+  "<|im_start|>": 151644,
+  "<|image_pad|>": 151655,
+  "<|object_ref_end|>": 151647,
+  "<|object_ref_start|>": 151646,
+  "<|quad_end|>": 151651,
+  "<|quad_start|>": 151650,
+  "<|repo_name|>": 151663,
+  "<|video_pad|>": 151656,
+  "<|vision_end|>": 151653,
+  "<|vision_pad|>": 151654,
+  "<|vision_start|>": 151652
+}

makto_v1/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,54 @@

+{%- if tools %}
+    {{- '<|im_start|>system\n' }}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- messages[0]['content'] }}
+    {%- else %}
+        {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}
+    {%- endif %}
+    {{- "\n\n# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
+    {%- for tool in tools %}
+        {{- "\n" }}
+        {{- tool | tojson }}
+    {%- endfor %}
+    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
+{%- else %}
+    {%- if messages[0]['role'] == 'system' %}
+        {{- '<|im_start|>system\n' + messages[0]['content'] + '<|im_end|>\n' }}
+    {%- else %}
+        {{- '<|im_start|>system\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n' }}
+    {%- endif %}
+{%- endif %}
+{%- for message in messages %}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) or (message.role == "assistant" and not message.tool_calls) %}
+        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>' + '\n' }}
+    {%- elif message.role == "assistant" %}
+        {{- '<|im_start|>' + message.role }}
+        {%- if message.content %}
+            {{- '\n' + message.content }}
+        {%- endif %}
+        {%- for tool_call in message.tool_calls %}
+            {%- if tool_call.function is defined %}
+                {%- set tool_call = tool_call.function %}
+            {%- endif %}
+            {{- '\n<tool_call>\n{"name": "' }}
+            {{- tool_call.name }}
+            {{- '", "arguments": ' }}
+            {{- tool_call.arguments | tojson }}
+            {{- '}\n</tool_call>' }}
+        {%- endfor %}
+        {{- '<|im_end|>\n' }}
+    {%- elif message.role == "tool" %}
+        {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != "tool") %}
+            {{- '<|im_start|>user' }}
+        {%- endif %}
+        {{- '\n<tool_response>\n' }}
+        {{- message.content }}
+        {{- '\n</tool_response>' }}
+        {%- if loop.last or (messages[loop.index0 + 1].role != "tool") %}
+            {{- '<|im_end|>\n' }}
+        {%- endif %}
+    {%- endif %}
+{%- endfor %}
+{%- if add_generation_prompt %}
+    {{- '<|im_start|>assistant\n' }}
+{%- endif %}

makto_v1/config.json ADDED Viewed

	@@ -0,0 +1,58 @@

+{
+  "architectures": [
+    "Qwen2ForCausalLM"
+  ],
+  "attention_dropout": 0.0,
+  "dtype": "float16",
+  "eos_token_id": 151645,
+  "hidden_act": "silu",
+  "hidden_size": 3584,
+  "initializer_range": 0.02,
+  "intermediate_size": 18944,
+  "layer_types": [
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention",
+    "full_attention"
+  ],
+  "max_position_embeddings": 32768,
+  "max_window_layers": 28,
+  "model_type": "qwen2",
+  "num_attention_heads": 28,
+  "num_hidden_layers": 28,
+  "num_key_value_heads": 4,
+  "pad_token_id": 151643,
+  "rms_norm_eps": 1e-06,
+  "rope_scaling": null,
+  "rope_theta": 1000000.0,
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "4.57.1",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 152064
+}

makto_v1/generation_config.json ADDED Viewed

	@@ -0,0 +1,13 @@

+{
+  "do_sample": true,
+  "eos_token_id": [
+    151645,
+    151643
+  ],
+  "pad_token_id": 151643,
+  "repetition_penalty": 1.05,
+  "temperature": 0.7,
+  "top_k": 20,
+  "top_p": 0.8,
+  "transformers_version": "4.57.1"
+}

makto_v1/merges.txt ADDED Viewed

The diff for this file is too large to render. See raw diff

makto_v1/model-00001-of-00004.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0572433d9a9fc35dee80e4a7a8e75ecf6aa63acec9265209238a72e8412edbbe
+size 4877660672

makto_v1/model-00002-of-00004.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a302210b4e37412e03258f9350e54560e99bc1828acd8c9ef184126e4d467c8c
+size 4932750888

makto_v1/model-00003-of-00004.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:dc7c507f22e7a4bc05aa1cce230dada3a203f0d6f465bbeabcdf744bb908fdd9
+size 4330865088

makto_v1/model-00004-of-00004.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:88bcbc578ea0f4a9b0e64830d98d53d58d129a1f78273fef8c4f0bdbf446f216
+size 1089994880

makto_v1/model.safetensors.index.json ADDED Viewed

	@@ -0,0 +1,347 @@

+{
+  "metadata": {
+    "total_parameters": 333312,
+    "total_size": 15231233024
+  },
+  "weight_map": {
+    "lm_head.weight": "model-00004-of-00004.safetensors",
+    "model.embed_tokens.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.10.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.18.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.19.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.19.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.2.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.20.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.input_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.k_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.q_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.v_proj.bias": "model-00003-of-00004.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "model-00003-of-00004.safetensors",
+    "model.layers.3.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.input_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.8.self_attn.k_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.q_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.v_proj.bias": "model-00001-of-00004.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "model-00001-of-00004.safetensors",
+    "model.layers.9.input_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.k_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.q_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.v_proj.bias": "model-00002-of-00004.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "model-00002-of-00004.safetensors",
+    "model.norm.weight": "model-00003-of-00004.safetensors"
+  }
+}

makto_v1/special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,31 @@

+{
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "eos_token": {
+    "content": "<|im_end|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "<|endoftext|>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

makto_v1/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9c5ae00e602b8860cbd784ba82a8aa14e8feecec692e7076590d014d7b7fdafa
+size 11421896

makto_v1/tokenizer_config.json ADDED Viewed

	@@ -0,0 +1,207 @@

+{
+  "add_bos_token": false,
+  "add_prefix_space": false,
+  "added_tokens_decoder": {
+    "151643": {
+      "content": "<|endoftext|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151644": {
+      "content": "<|im_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151645": {
+      "content": "<|im_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151646": {
+      "content": "<|object_ref_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151647": {
+      "content": "<|object_ref_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151648": {
+      "content": "<|box_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151649": {
+      "content": "<|box_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151650": {
+      "content": "<|quad_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151651": {
+      "content": "<|quad_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151652": {
+      "content": "<|vision_start|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151653": {
+      "content": "<|vision_end|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151654": {
+      "content": "<|vision_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151655": {
+      "content": "<|image_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151656": {
+      "content": "<|video_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "151657": {
+      "content": "<tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151658": {
+      "content": "</tool_call>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151659": {
+      "content": "<|fim_prefix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151660": {
+      "content": "<|fim_middle|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151661": {
+      "content": "<|fim_suffix|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151662": {
+      "content": "<|fim_pad|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151663": {
+      "content": "<|repo_name|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "151664": {
+      "content": "<|file_sep|>",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "additional_special_tokens": [
+    "<|im_start|>",
+    "<|im_end|>",
+    "<|object_ref_start|>",
+    "<|object_ref_end|>",
+    "<|box_start|>",
+    "<|box_end|>",
+    "<|quad_start|>",
+    "<|quad_end|>",
+    "<|vision_start|>",
+    "<|vision_end|>",
+    "<|vision_pad|>",
+    "<|image_pad|>",
+    "<|video_pad|>"
+  ],
+  "bos_token": null,
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "errors": "replace",
+  "extra_special_tokens": {},
+  "model_max_length": 131072,
+  "pad_token": "<|endoftext|>",
+  "split_special_tokens": false,
+  "tokenizer_class": "Qwen2Tokenizer",
+  "unk_token": null
+}

makto_v1/trainer_state.json ADDED Viewed

	@@ -0,0 +1,3034 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 8.298910223144784,
+  "eval_steps": 500,
+  "global_step": 2000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.04151530877010898,
+      "grad_norm": 132.4568328857422,
+      "kl": 0.0,
+      "learning_rate": 6.224066390041493e-09,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -168.5890625,
+      "logps/rejected": -251.4546875,
+      "loss": 0.5,
+      "rewards/chosen": 0.0,
+      "rewards/margins": 0.0,
+      "rewards/rejected": 0.0,
+      "step": 10
+    },
+    {
+      "epoch": 0.08303061754021795,
+      "grad_norm": 187.47601318359375,
+      "kl": 0.03632812574505806,
+      "learning_rate": 2.6970954356846473e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -167.965625,
+      "logps/rejected": -244.0390625,
+      "loss": 0.4971,
+      "rewards/chosen": 0.011016559600830079,
+      "rewards/margins": 0.01890573501586914,
+      "rewards/rejected": -0.007889175415039062,
+      "step": 20
+    },
+    {
+      "epoch": 0.12454592631032693,
+      "grad_norm": 72.57302856445312,
+      "kl": 0.33984375,
+      "learning_rate": 4.771784232365145e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -166.2703125,
+      "logps/rejected": -254.025,
+      "loss": 0.4565,
+      "rewards/chosen": 0.16999893188476561,
+      "rewards/margins": 0.2705129623413086,
+      "rewards/rejected": -0.10051403045654297,
+      "step": 30
+    },
+    {
+      "epoch": 0.1660612350804359,
+      "grad_norm": 100.96675872802734,
+      "kl": 0.24257811903953552,
+      "learning_rate": 6.846473029045644e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -159.671875,
+      "logps/rejected": -258.6359375,
+      "loss": 0.2918,
+      "rewards/chosen": 0.7255096435546875,
+      "rewards/margins": 1.4680633544921875,
+      "rewards/rejected": -0.7425537109375,
+      "step": 40
+    },
+    {
+      "epoch": 0.2075765438505449,
+      "grad_norm": 25.185272216796875,
+      "kl": 0.44023436307907104,
+      "learning_rate": 8.921161825726141e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -145.25546875,
+      "logps/rejected": -275.2671875,
+      "loss": 0.1066,
+      "rewards/chosen": 1.682635498046875,
+      "rewards/margins": 2316800.8826354984,
+      "rewards/rejected": -2316799.2,
+      "step": 50
+    },
+    {
+      "epoch": 0.24909185262065386,
+      "grad_norm": 1.8716069459915161,
+      "kl": 0.0021484375465661287,
+      "learning_rate": 1.0995850622406639e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -119.45,
+      "logps/rejected": -309.778125,
+      "loss": 0.0102,
+      "rewards/chosen": 4.002685546875,
+      "rewards/margins": 9.1927978515625,
+      "rewards/rejected": -5.1901123046875,
+      "step": 60
+    },
+    {
+      "epoch": 0.29060716139076287,
+      "grad_norm": 0.09964268654584885,
+      "kl": 0.16572265326976776,
+      "learning_rate": 1.3070539419087136e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -96.8203125,
+      "logps/rejected": -338.1375,
+      "loss": 0.0015,
+      "rewards/chosen": 5.47841796875,
+      "rewards/margins": 12.020117187499999,
+      "rewards/rejected": -6.54169921875,
+      "step": 70
+    },
+    {
+      "epoch": 0.3321224701608718,
+      "grad_norm": 0.03205466270446777,
+      "kl": 0.0,
+      "learning_rate": 1.5145228215767634e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -76.798046875,
+      "logps/rejected": -371.21875,
+      "loss": 0.0001,
+      "rewards/chosen": 6.976416015625,
+      "rewards/margins": 16.564697265625,
+      "rewards/rejected": -9.58828125,
+      "step": 80
+    },
+    {
+      "epoch": 0.3736377789309808,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.721991701244813e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -63.2125,
+      "logps/rejected": -388.109375,
+      "loss": 0.0,
+      "rewards/chosen": 7.11220703125,
+      "rewards/margins": 18.68125,
+      "rewards/rejected": -11.56904296875,
+      "step": 90
+    },
+    {
+      "epoch": 0.4151530877010898,
+      "grad_norm": 0.0,
+      "kl": 0.06220703199505806,
+      "learning_rate": 1.9294605809128631e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -59.933984375,
+      "logps/rejected": -384.340625,
+      "loss": 0.0,
+      "rewards/chosen": 7.99345703125,
+      "rewards/margins": 18.1388671875,
+      "rewards/rejected": -10.14541015625,
+      "step": 100
+    },
+    {
+      "epoch": 0.4566683964711988,
+      "grad_norm": 0.0,
+      "kl": 0.03007812425494194,
+      "learning_rate": 2.1369294605809126e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -59.248828125,
+      "logps/rejected": -392.021875,
+      "loss": 0.0,
+      "rewards/chosen": 8.7330078125,
+      "rewards/margins": 19.453515625,
+      "rewards/rejected": -10.7205078125,
+      "step": 110
+    },
+    {
+      "epoch": 0.49818370524130773,
+      "grad_norm": 0.0,
+      "kl": 0.04677734524011612,
+      "learning_rate": 2.3443983402489626e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.1859375,
+      "logps/rejected": -380.440625,
+      "loss": 0.0,
+      "rewards/chosen": 8.69755859375,
+      "rewards/margins": 18.490234375,
+      "rewards/rejected": -9.79267578125,
+      "step": 120
+    },
+    {
+      "epoch": 0.5396990140114167,
+      "grad_norm": 0.0,
+      "kl": 0.0002929687616415322,
+      "learning_rate": 2.551867219917012e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -60.96640625,
+      "logps/rejected": -393.29375,
+      "loss": 0.0,
+      "rewards/chosen": 8.54423828125,
+      "rewards/margins": 19.678906249999997,
+      "rewards/rejected": -11.13466796875,
+      "step": 130
+    },
+    {
+      "epoch": 0.5812143227815257,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.759336099585062e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.43046875,
+      "logps/rejected": -394.375,
+      "loss": 0.0,
+      "rewards/chosen": 8.45693359375,
+      "rewards/margins": 19.6638671875,
+      "rewards/rejected": -11.20693359375,
+      "step": 140
+    },
+    {
+      "epoch": 0.6227296315516346,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.966804979253112e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.4625,
+      "logps/rejected": -397.975,
+      "loss": 0.0,
+      "rewards/chosen": 8.54521484375,
+      "rewards/margins": 19.72744140625,
+      "rewards/rejected": -11.1822265625,
+      "step": 150
+    },
+    {
+      "epoch": 0.6642449403217436,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.1742738589211616e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.2078125,
+      "logps/rejected": -396.053125,
+      "loss": 0.0,
+      "rewards/chosen": 8.15185546875,
+      "rewards/margins": 20.4986328125,
+      "rewards/rejected": -12.34677734375,
+      "step": 160
+    },
+    {
+      "epoch": 0.7057602490918526,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.3817427385892117e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -60.707421875,
+      "logps/rejected": -398.50625,
+      "loss": 0.0,
+      "rewards/chosen": 8.2712890625,
+      "rewards/margins": 19.8142578125,
+      "rewards/rejected": -11.54296875,
+      "step": 170
+    },
+    {
+      "epoch": 0.7472755578619616,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.589211618257261e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.514453125,
+      "logps/rejected": -394.521875,
+      "loss": 0.0,
+      "rewards/chosen": 8.2978515625,
+      "rewards/margins": 19.995703125,
+      "rewards/rejected": -11.6978515625,
+      "step": 180
+    },
+    {
+      "epoch": 0.7887908666320705,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.7966804979253106e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.275,
+      "logps/rejected": -399.25625,
+      "loss": 0.0004,
+      "rewards/chosen": 7.46494140625,
+      "rewards/margins": 19.26240234375,
+      "rewards/rejected": -11.7974609375,
+      "step": 190
+    },
+    {
+      "epoch": 0.8303061754021795,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.004149377593361e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -59.777734375,
+      "logps/rejected": -393.48125,
+      "loss": 0.0,
+      "rewards/chosen": 9.08603515625,
+      "rewards/margins": 20.08662109375,
+      "rewards/rejected": -11.0005859375,
+      "step": 200
+    },
+    {
+      "epoch": 0.8718214841722886,
+      "grad_norm": 0.007175118196755648,
+      "kl": 0.03310547024011612,
+      "learning_rate": 4.2116182572614107e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -61.14609375,
+      "logps/rejected": -391.25,
+      "loss": 0.0,
+      "rewards/chosen": 8.869921875,
+      "rewards/margins": 18.82626953125,
+      "rewards/rejected": -9.95634765625,
+      "step": 210
+    },
+    {
+      "epoch": 0.9133367929423976,
+      "grad_norm": 0.0,
+      "kl": 0.33320313692092896,
+      "learning_rate": 4.41908713692946e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.293359375,
+      "logps/rejected": -376.175,
+      "loss": 0.0,
+      "rewards/chosen": 8.47998046875,
+      "rewards/margins": 17.5115234375,
+      "rewards/rejected": -9.03154296875,
+      "step": 220
+    },
+    {
+      "epoch": 0.9548521017125065,
+      "grad_norm": 0.0,
+      "kl": 0.695996105670929,
+      "learning_rate": 4.62655601659751e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.915625,
+      "logps/rejected": -371.834375,
+      "loss": 0.0003,
+      "rewards/chosen": 8.96708984375,
+      "rewards/margins": 19.18515625,
+      "rewards/rejected": -10.21806640625,
+      "step": 230
+    },
+    {
+      "epoch": 0.9963674104826155,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.83402489626556e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.111328125,
+      "logps/rejected": -402.821875,
+      "loss": 0.0,
+      "rewards/chosen": 8.88525390625,
+      "rewards/margins": 19.94462890625,
+      "rewards/rejected": -11.059375,
+      "step": 240
+    },
+    {
+      "epoch": 1.037363777893098,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.999989510607367e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -59.644382911392405,
+      "logps/rejected": -413.93037974683546,
+      "loss": 0.0001,
+      "rewards/chosen": 8.279568829113924,
+      "rewards/margins": 20.64418512658228,
+      "rewards/rejected": -12.364616297468354,
+      "step": 250
+    },
+    {
+      "epoch": 1.078879086663207,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.999622391107413e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.2484375,
+      "logps/rejected": -413.240625,
+      "loss": 0.0,
+      "rewards/chosen": 8.775390625,
+      "rewards/margins": 23.0369140625,
+      "rewards/rejected": -14.2615234375,
+      "step": 260
+    },
+    {
+      "epoch": 1.120394395433316,
+      "grad_norm": 0.008085768669843674,
+      "kl": 0.0,
+      "learning_rate": 4.998730889994357e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.14609375,
+      "logps/rejected": -426.475,
+      "loss": 0.0,
+      "rewards/chosen": 8.68017578125,
+      "rewards/margins": 22.3830078125,
+      "rewards/rejected": -13.70283203125,
+      "step": 270
+    },
+    {
+      "epoch": 1.161909704203425,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.997315194291163e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.7390625,
+      "logps/rejected": -419.390625,
+      "loss": 0.0,
+      "rewards/chosen": 8.890625,
+      "rewards/margins": 20.6443359375,
+      "rewards/rejected": -11.7537109375,
+      "step": 280
+    },
+    {
+      "epoch": 1.203425012973534,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.995375600988607e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.975,
+      "logps/rejected": -427.60625,
+      "loss": 0.0,
+      "rewards/chosen": 9.2767578125,
+      "rewards/margins": 2133841.0767578124,
+      "rewards/rejected": -2133831.8,
+      "step": 290
+    },
+    {
+      "epoch": 1.244940321743643,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.992912516982977e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.59140625,
+      "logps/rejected": -418.1125,
+      "loss": 0.0,
+      "rewards/chosen": 8.633203125,
+      "rewards/margins": 22.89931640625,
+      "rewards/rejected": -14.26611328125,
+      "step": 300
+    },
+    {
+      "epoch": 1.286455630513752,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.989926458990703e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.058984375,
+      "logps/rejected": -422.371875,
+      "loss": 0.0,
+      "rewards/chosen": 8.55595703125,
+      "rewards/margins": 22.6380859375,
+      "rewards/rejected": -14.08212890625,
+      "step": 310
+    },
+    {
+      "epoch": 1.327970939283861,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.986418053439967e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.62421875,
+      "logps/rejected": -422.45625,
+      "loss": 0.0,
+      "rewards/chosen": 8.54091796875,
+      "rewards/margins": 21.5828125,
+      "rewards/rejected": -13.04189453125,
+      "step": 320
+    },
+    {
+      "epoch": 1.36948624805397,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.982388036339281e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.719921875,
+      "logps/rejected": -428.234375,
+      "loss": 0.0,
+      "rewards/chosen": 8.3095703125,
+      "rewards/margins": 23.1138671875,
+      "rewards/rejected": -14.804296875,
+      "step": 330
+    },
+    {
+      "epoch": 1.411001556824079,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.977837253123093e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.39453125,
+      "logps/rejected": -427.778125,
+      "loss": 0.0,
+      "rewards/chosen": 8.0908203125,
+      "rewards/margins": 21.8474609375,
+      "rewards/rejected": -13.756640625,
+      "step": 340
+    },
+    {
+      "epoch": 1.4525168655941878,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.972766658474416e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.5515625,
+      "logps/rejected": -431.903125,
+      "loss": 0.0,
+      "rewards/chosen": 8.4337890625,
+      "rewards/margins": 22.44033203125,
+      "rewards/rejected": -14.00654296875,
+      "step": 350
+    },
+    {
+      "epoch": 1.4940321743642968,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.967177316124562e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.41171875,
+      "logps/rejected": -420.146875,
+      "loss": 0.0,
+      "rewards/chosen": 9.06220703125,
+      "rewards/margins": 21.434375000000003,
+      "rewards/rejected": -12.37216796875,
+      "step": 360
+    },
+    {
+      "epoch": 1.5355474831344058,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.96107039862998e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -59.240625,
+      "logps/rejected": -427.140625,
+      "loss": 0.0,
+      "rewards/chosen": 8.4953125,
+      "rewards/margins": 22.3705078125,
+      "rewards/rejected": -13.8751953125,
+      "step": 370
+    },
+    {
+      "epoch": 1.5770627919045148,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.954447187126277e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.04375,
+      "logps/rejected": -424.1375,
+      "loss": 0.0,
+      "rewards/chosen": 8.42724609375,
+      "rewards/margins": 22.11142578125,
+      "rewards/rejected": -13.6841796875,
+      "step": 380
+    },
+    {
+      "epoch": 1.6185781006746238,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.947309071059454e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.012109375,
+      "logps/rejected": -422.68125,
+      "loss": 0.0,
+      "rewards/chosen": 8.7861328125,
+      "rewards/margins": 21.86669921875,
+      "rewards/rejected": -13.08056640625,
+      "step": 390
+    },
+    {
+      "epoch": 1.6600934094447326,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.939657547894417e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.5640625,
+      "logps/rejected": -421.50625,
+      "loss": 0.0,
+      "rewards/chosen": 8.09853515625,
+      "rewards/margins": 22.8310546875,
+      "rewards/rejected": -14.73251953125,
+      "step": 400
+    },
+    {
+      "epoch": 1.7016087182148416,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.931494222800844e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -59.571875,
+      "logps/rejected": -427.103125,
+      "loss": 0.0,
+      "rewards/chosen": 8.24189453125,
+      "rewards/margins": 21.720507812500003,
+      "rewards/rejected": -13.47861328125,
+      "step": 410
+    },
+    {
+      "epoch": 1.7431240269849506,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.922820808316432e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.427734375,
+      "logps/rejected": -423.03125,
+      "loss": 0.0,
+      "rewards/chosen": 8.62509765625,
+      "rewards/margins": 22.10634765625,
+      "rewards/rejected": -13.48125,
+      "step": 420
+    },
+    {
+      "epoch": 1.7846393357550596,
+      "grad_norm": 0.030060525983572006,
+      "kl": 0.0,
+      "learning_rate": 4.913639123987647e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.20078125,
+      "logps/rejected": -428.26875,
+      "loss": 0.0,
+      "rewards/chosen": 7.60302734375,
+      "rewards/margins": 21.67802734375,
+      "rewards/rejected": -14.075,
+      "step": 430
+    },
+    {
+      "epoch": 1.8261546445251686,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.903951095987997e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.110546875,
+      "logps/rejected": -420.6625,
+      "loss": 0.0,
+      "rewards/chosen": 9.16064453125,
+      "rewards/margins": 22.630078125,
+      "rewards/rejected": -13.46943359375,
+      "step": 440
+    },
+    {
+      "epoch": 1.8676699532952776,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.893758756713965e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.281640625,
+      "logps/rejected": -432.89375,
+      "loss": 0.0,
+      "rewards/chosen": 9.04521484375,
+      "rewards/margins": 22.1927734375,
+      "rewards/rejected": -13.14755859375,
+      "step": 450
+    },
+    {
+      "epoch": 1.9091852620653866,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.883064244358634e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.083203125,
+      "logps/rejected": -424.7625,
+      "loss": 0.0,
+      "rewards/chosen": 8.64970703125,
+      "rewards/margins": 21.421093749999997,
+      "rewards/rejected": -12.77138671875,
+      "step": 460
+    },
+    {
+      "epoch": 1.9507005708354956,
+      "grad_norm": 0.018523748964071274,
+      "kl": 0.0,
+      "learning_rate": 4.871869802463133e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.148046875,
+      "logps/rejected": -427.046875,
+      "loss": 0.0,
+      "rewards/chosen": 9.2419921875,
+      "rewards/margins": 23.36220703125,
+      "rewards/rejected": -14.12021484375,
+      "step": 470
+    },
+    {
+      "epoch": 1.9922158796056046,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.860177779445976e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.4734375,
+      "logps/rejected": -432.665625,
+      "loss": 0.0,
+      "rewards/chosen": 8.90703125,
+      "rewards/margins": 22.8521484375,
+      "rewards/rejected": -13.9451171875,
+      "step": 480
+    },
+    {
+      "epoch": 2.0332122470160874,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.847990628110396e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.453322784810126,
+      "logps/rejected": -429.35759493670884,
+      "loss": 0.0,
+      "rewards/chosen": 8.609473892405063,
+      "rewards/margins": 21.73012262658228,
+      "rewards/rejected": -13.120648734177216,
+      "step": 490
+    },
+    {
+      "epoch": 2.074727555786196,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.835310905129797e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.3546875,
+      "logps/rejected": -422.85625,
+      "loss": 0.0,
+      "rewards/chosen": 8.8333984375,
+      "rewards/margins": 23.8111328125,
+      "rewards/rejected": -14.977734375,
+      "step": 500
+    },
+    {
+      "epoch": 2.116242864556305,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.822141270511389e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.230859375,
+      "logps/rejected": -429.203125,
+      "loss": 0.0,
+      "rewards/chosen": 8.699609375,
+      "rewards/margins": 23.47412109375,
+      "rewards/rejected": -14.77451171875,
+      "step": 510
+    },
+    {
+      "epoch": 2.157758173326414,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.808484487038171e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.476953125,
+      "logps/rejected": -427.296875,
+      "loss": 0.0,
+      "rewards/chosen": 9.33251953125,
+      "rewards/margins": 21.38388671875,
+      "rewards/rejected": -12.0513671875,
+      "step": 520
+    },
+    {
+      "epoch": 2.199273482096523,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.79434341968934e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.183203125,
+      "logps/rejected": -429.365625,
+      "loss": 0.0,
+      "rewards/chosen": 9.31181640625,
+      "rewards/margins": 2146953.1118164062,
+      "rewards/rejected": -2146943.8,
+      "step": 530
+    },
+    {
+      "epoch": 2.240788790866632,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.779721035039259e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.186328125,
+      "logps/rejected": -424.915625,
+      "loss": 0.0,
+      "rewards/chosen": 8.61201171875,
+      "rewards/margins": 23.39619140625,
+      "rewards/rejected": -14.7841796875,
+      "step": 540
+    },
+    {
+      "epoch": 2.282304099636741,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.7646204006351234e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.524609375,
+      "logps/rejected": -428.603125,
+      "loss": 0.0,
+      "rewards/chosen": 9.11591796875,
+      "rewards/margins": 23.6212890625,
+      "rewards/rejected": -14.50537109375,
+      "step": 550
+    },
+    {
+      "epoch": 2.32381940840685,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.74904468435343e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.7046875,
+      "logps/rejected": -426.175,
+      "loss": 0.0,
+      "rewards/chosen": 8.7958984375,
+      "rewards/margins": 21.5349609375,
+      "rewards/rejected": -12.7390625,
+      "step": 560
+    },
+    {
+      "epoch": 2.365334717176959,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.732997153735412e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.576171875,
+      "logps/rejected": -435.496875,
+      "loss": 0.0,
+      "rewards/chosen": 8.2650390625,
+      "rewards/margins": 23.9505859375,
+      "rewards/rejected": -15.685546875,
+      "step": 570
+    },
+    {
+      "epoch": 2.406850025947068,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.7164811753015577e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.655859375,
+      "logps/rejected": -432.36875,
+      "loss": 0.0,
+      "rewards/chosen": 7.7580078125,
+      "rewards/margins": 21.476171875,
+      "rewards/rejected": -13.7181640625,
+      "step": 580
+    },
+    {
+      "epoch": 2.448365334717177,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.699500213845367e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.793359375,
+      "logps/rejected": -436.865625,
+      "loss": 0.0,
+      "rewards/chosen": 8.4578125,
+      "rewards/margins": 23.114453124999997,
+      "rewards/rejected": -14.656640625,
+      "step": 590
+    },
+    {
+      "epoch": 2.489880643487286,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.6820578317064917e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.936328125,
+      "logps/rejected": -428.140625,
+      "loss": 0.0,
+      "rewards/chosen": 9.27197265625,
+      "rewards/margins": 22.27119140625,
+      "rewards/rejected": -12.99921875,
+      "step": 600
+    },
+    {
+      "epoch": 2.531395952257395,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.6641576880234155e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.475390625,
+      "logps/rejected": -428.628125,
+      "loss": 0.0,
+      "rewards/chosen": 8.649609375,
+      "rewards/margins": 22.516015625,
+      "rewards/rejected": -13.86640625,
+      "step": 610
+    },
+    {
+      "epoch": 2.572911261027504,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.6458035379658187e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.3859375,
+      "logps/rejected": -431.04375,
+      "loss": 0.0,
+      "rewards/chosen": 8.33798828125,
+      "rewards/margins": 22.91220703125,
+      "rewards/rejected": -14.57421875,
+      "step": 620
+    },
+    {
+      "epoch": 2.614426569797613,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.626999231946808e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.731640625,
+      "logps/rejected": -429.471875,
+      "loss": 0.0,
+      "rewards/chosen": 8.68994140625,
+      "rewards/margins": 21.95966796875,
+      "rewards/rejected": -13.2697265625,
+      "step": 630
+    },
+    {
+      "epoch": 2.655941878567722,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.6077487148151573e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.594921875,
+      "logps/rejected": -427.74375,
+      "loss": 0.0,
+      "rewards/chosen": 8.40380859375,
+      "rewards/margins": 23.739453125,
+      "rewards/rejected": -15.33564453125,
+      "step": 640
+    },
+    {
+      "epoch": 2.6974571873378306,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.588056025027739e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -59.0046875,
+      "logps/rejected": -431.9,
+      "loss": 0.0,
+      "rewards/chosen": 8.10400390625,
+      "rewards/margins": 22.35126953125,
+      "rewards/rejected": -14.247265625,
+      "step": 650
+    },
+    {
+      "epoch": 2.73897249610794,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.567925293802322e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.812890625,
+      "logps/rejected": -431.221875,
+      "loss": 0.0,
+      "rewards/chosen": 8.9552734375,
+      "rewards/margins": 22.76328125,
+      "rewards/rejected": -13.8080078125,
+      "step": 660
+    },
+    {
+      "epoch": 2.7804878048780486,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.547360744250906e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.68984375,
+      "logps/rejected": -433.571875,
+      "loss": 0.0,
+      "rewards/chosen": 7.765234375,
+      "rewards/margins": 22.14375,
+      "rewards/rejected": -14.378515625,
+      "step": 670
+    },
+    {
+      "epoch": 2.822003113648158,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.526366690493777e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.06328125,
+      "logps/rejected": -423.225,
+      "loss": 0.0,
+      "rewards/chosen": 9.05380859375,
+      "rewards/margins": 22.520800781250003,
+      "rewards/rejected": -13.4669921875,
+      "step": 680
+    },
+    {
+      "epoch": 2.8635184224182666,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.504947536754475e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.7875,
+      "logps/rejected": -436.025,
+      "loss": 0.0,
+      "rewards/chosen": 8.8158203125,
+      "rewards/margins": 23.167187499999997,
+      "rewards/rejected": -14.3513671875,
+      "step": 690
+    },
+    {
+      "epoch": 2.9050337311883756,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.483107776435855e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.624609375,
+      "logps/rejected": -428.325,
+      "loss": 0.0,
+      "rewards/chosen": 8.69375,
+      "rewards/margins": 21.64794921875,
+      "rewards/rejected": -12.95419921875,
+      "step": 700
+    },
+    {
+      "epoch": 2.9465490399584846,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.4608519911774424e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.81953125,
+      "logps/rejected": -432.34375,
+      "loss": 0.0,
+      "rewards/chosen": 9.27255859375,
+      "rewards/margins": 23.01220703125,
+      "rewards/rejected": -13.7396484375,
+      "step": 710
+    },
+    {
+      "epoch": 2.9880643487285936,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.438184849894276e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.7765625,
+      "logps/rejected": -435.315625,
+      "loss": 0.0,
+      "rewards/chosen": 8.94521484375,
+      "rewards/margins": 23.417578125,
+      "rewards/rejected": -14.47236328125,
+      "step": 720
+    },
+    {
+      "epoch": 3.0290607161390763,
+      "grad_norm": 0.008966372348368168,
+      "kl": 0.0,
+      "learning_rate": 4.415111107797445e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.625791139240505,
+      "logps/rejected": -432.05379746835445,
+      "loss": 0.0,
+      "rewards/chosen": 8.636075949367088,
+      "rewards/margins": 22.248615506329113,
+      "rewards/rejected": -13.612539556962025,
+      "step": 730
+    },
+    {
+      "epoch": 3.0705760249091854,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.391635605396517e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.3921875,
+      "logps/rejected": -428.571875,
+      "loss": 0.0,
+      "rewards/chosen": 8.9939453125,
+      "rewards/margins": 23.55712890625,
+      "rewards/rejected": -14.56318359375,
+      "step": 740
+    },
+    {
+      "epoch": 3.1120913336792944,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.367763267484077e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.937109375,
+      "logps/rejected": -429.821875,
+      "loss": 0.0,
+      "rewards/chosen": 8.7171875,
+      "rewards/margins": 24.287109375,
+      "rewards/rejected": -15.569921875,
+      "step": 750
+    },
+    {
+      "epoch": 3.1536066424494034,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.3434991021025837e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.877734375,
+      "logps/rejected": -432.840625,
+      "loss": 0.0,
+      "rewards/chosen": 9.2078125,
+      "rewards/margins": 21.5162109375,
+      "rewards/rejected": -12.3083984375,
+      "step": 760
+    },
+    {
+      "epoch": 3.1951219512195124,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.3188481994937574e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.520703125,
+      "logps/rejected": -432.2125,
+      "loss": 0.0,
+      "rewards/chosen": 9.61484375,
+      "rewards/margins": 2173173.21484375,
+      "rewards/rejected": -2173163.6,
+      "step": 770
+    },
+    {
+      "epoch": 3.2366372599896214,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.293815731030732e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.91015625,
+      "logps/rejected": -429.971875,
+      "loss": 0.0,
+      "rewards/chosen": 8.62470703125,
+      "rewards/margins": 23.114453125,
+      "rewards/rejected": -14.48974609375,
+      "step": 780
+    },
+    {
+      "epoch": 3.2781525687597304,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.2684069481331765e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.614453125,
+      "logps/rejected": -433.184375,
+      "loss": 0.0,
+      "rewards/chosen": 9.28623046875,
+      "rewards/margins": 24.15517578125,
+      "rewards/rejected": -14.8689453125,
+      "step": 790
+    },
+    {
+      "epoch": 3.319667877529839,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.2426271811656336e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.441015625,
+      "logps/rejected": -432.33125,
+      "loss": 0.0,
+      "rewards/chosen": 8.53427734375,
+      "rewards/margins": 22.44697265625,
+      "rewards/rejected": -13.9126953125,
+      "step": 800
+    },
+    {
+      "epoch": 3.361183186299948,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.21648183831929e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.10390625,
+      "logps/rejected": -436.0125,
+      "loss": 0.0,
+      "rewards/chosen": 8.1349609375,
+      "rewards/margins": 23.6640625,
+      "rewards/rejected": -15.5291015625,
+      "step": 810
+    },
+    {
+      "epoch": 3.402698495070057,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.189976404477426e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.3453125,
+      "logps/rejected": -439.559375,
+      "loss": 0.0,
+      "rewards/chosen": 7.7779296875,
+      "rewards/margins": 21.8373046875,
+      "rewards/rejected": -14.059375,
+      "step": 820
+    },
+    {
+      "epoch": 3.444213803840166,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.163116440064769e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.809765625,
+      "logps/rejected": -439.396875,
+      "loss": 0.0,
+      "rewards/chosen": 8.62890625,
+      "rewards/margins": 23.3876953125,
+      "rewards/rejected": -14.7587890625,
+      "step": 830
+    },
+    {
+      "epoch": 3.485729112610275,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.135907579881006e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.483203125,
+      "logps/rejected": -432.490625,
+      "loss": 0.0,
+      "rewards/chosen": 9.29111328125,
+      "rewards/margins": 22.81826171875,
+      "rewards/rejected": -13.5271484375,
+      "step": 840
+    },
+    {
+      "epoch": 3.527244421380384,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.10835553191869e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.9578125,
+      "logps/rejected": -431.303125,
+      "loss": 0.0,
+      "rewards/chosen": 8.8224609375,
+      "rewards/margins": 22.7369140625,
+      "rewards/rejected": -13.914453125,
+      "step": 850
+    },
+    {
+      "epoch": 3.568759730150493,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.080466076165793e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.930859375,
+      "logps/rejected": -436.76875,
+      "loss": 0.0,
+      "rewards/chosen": 8.4865234375,
+      "rewards/margins": 23.223437500000003,
+      "rewards/rejected": -14.7369140625,
+      "step": 860
+    },
+    {
+      "epoch": 3.610275038920602,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.052245063393154e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.96640625,
+      "logps/rejected": -431.709375,
+      "loss": 0.0,
+      "rewards/chosen": 8.2875,
+      "rewards/margins": 22.9720703125,
+      "rewards/rejected": -14.6845703125,
+      "step": 870
+    },
+    {
+      "epoch": 3.651790347690711,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.023698413927081e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.4359375,
+      "logps/rejected": -431.9625,
+      "loss": 0.0,
+      "rewards/chosen": 8.56435546875,
+      "rewards/margins": 23.27294921875,
+      "rewards/rejected": -14.70859375,
+      "step": 880
+    },
+    {
+      "epoch": 3.69330565646082,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.994832116407358e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.3375,
+      "logps/rejected": -435.071875,
+      "loss": 0.0,
+      "rewards/chosen": 8.2638671875,
+      "rewards/margins": 23.1302734375,
+      "rewards/rejected": -14.86640625,
+      "step": 890
+    },
+    {
+      "epoch": 3.734820965230929,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.965652226530923e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.98515625,
+      "logps/rejected": -437.28125,
+      "loss": 0.0,
+      "rewards/chosen": 8.69609375,
+      "rewards/margins": 23.01796875,
+      "rewards/rejected": -14.321875,
+      "step": 900
+    },
+    {
+      "epoch": 3.776336274001038,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.936164865781474e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.778125,
+      "logps/rejected": -435.471875,
+      "loss": 0.0,
+      "rewards/chosen": 8.06259765625,
+      "rewards/margins": 22.26669921875,
+      "rewards/rejected": -14.2041015625,
+      "step": 910
+    },
+    {
+      "epoch": 3.817851582771147,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.906376220145284e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.884375,
+      "logps/rejected": -430.15,
+      "loss": 0.0,
+      "rewards/chosen": 8.7865234375,
+      "rewards/margins": 22.80751953125,
+      "rewards/rejected": -14.02099609375,
+      "step": 920
+    },
+    {
+      "epoch": 3.8593668915412556,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.8762925388134734e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.378515625,
+      "logps/rejected": -438.1375,
+      "loss": 0.0,
+      "rewards/chosen": 8.84580078125,
+      "rewards/margins": 23.32529296875,
+      "rewards/rejected": -14.4794921875,
+      "step": 930
+    },
+    {
+      "epoch": 3.900882200311365,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.8459201328710285e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.1109375,
+      "logps/rejected": -432.6,
+      "loss": 0.0,
+      "rewards/chosen": 8.7173828125,
+      "rewards/margins": 22.101367187500003,
+      "rewards/rejected": -13.383984375,
+      "step": 940
+    },
+    {
+      "epoch": 3.9423975090814736,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.815265373972836e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.32421875,
+      "logps/rejected": -433.5875,
+      "loss": 0.0,
+      "rewards/chosen": 9.56171875,
+      "rewards/margins": 23.294921875,
+      "rewards/rejected": -13.733203125,
+      "step": 950
+    },
+    {
+      "epoch": 3.983912817851583,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.784334693007007e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.55078125,
+      "logps/rejected": -441.434375,
+      "loss": 0.0,
+      "rewards/chosen": 8.694140625,
+      "rewards/margins": 23.6884765625,
+      "rewards/rejected": -14.9943359375,
+      "step": 960
+    },
+    {
+      "epoch": 4.024909185262065,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.753134578745782e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.31566455696203,
+      "logps/rejected": -435.2689873417722,
+      "loss": 0.0,
+      "rewards/chosen": 8.785007911392405,
+      "rewards/margins": 22.73674841772152,
+      "rewards/rejected": -13.951740506329115,
+      "step": 970
+    },
+    {
+      "epoch": 4.066424494032175,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.7216715764842827e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.51484375,
+      "logps/rejected": -432.1625,
+      "loss": 0.0,
+      "rewards/chosen": 8.85234375,
+      "rewards/margins": 23.15478515625,
+      "rewards/rejected": -14.30244140625,
+      "step": 980
+    },
+    {
+      "epoch": 4.107939802802283,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.689952286667416e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.45546875,
+      "logps/rejected": -429.734375,
+      "loss": 0.0,
+      "rewards/chosen": 8.7328125,
+      "rewards/margins": 24.483984375,
+      "rewards/rejected": -15.751171875,
+      "step": 990
+    },
+    {
+      "epoch": 4.149455111572392,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.6579833635052013e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.623046875,
+      "logps/rejected": -435.946875,
+      "loss": 0.0,
+      "rewards/chosen": 9.20107421875,
+      "rewards/margins": 22.15849609375,
+      "rewards/rejected": -12.957421875,
+      "step": 1000
+    },
+    {
+      "epoch": 4.190970420342501,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.625771513576824e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.094921875,
+      "logps/rejected": -436.74375,
+      "loss": 0.0,
+      "rewards/chosen": 9.64638671875,
+      "rewards/margins": 2186284.0463867188,
+      "rewards/rejected": -2186274.4,
+      "step": 1010
+    },
+    {
+      "epoch": 4.23248572911261,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.593323494423696e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.10234375,
+      "logps/rejected": -430.65625,
+      "loss": 0.0,
+      "rewards/chosen": 8.788671875,
+      "rewards/margins": 23.4951171875,
+      "rewards/rejected": -14.7064453125,
+      "step": 1020
+    },
+    {
+      "epoch": 4.274001037882719,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.560646113131836e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.8328125,
+      "logps/rejected": -436.03125,
+      "loss": 0.0,
+      "rewards/chosen": 9.16591796875,
+      "rewards/margins": 24.45615234375,
+      "rewards/rejected": -15.290234375,
+      "step": 1030
+    },
+    {
+      "epoch": 4.315516346652828,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.5277462249038415e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.340234375,
+      "logps/rejected": -437.125,
+      "loss": 0.0,
+      "rewards/chosen": 8.396484375,
+      "rewards/margins": 22.7353515625,
+      "rewards/rejected": -14.3388671875,
+      "step": 1040
+    },
+    {
+      "epoch": 4.357031655422937,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.4946307316207826e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.0265625,
+      "logps/rejected": -437.515625,
+      "loss": 0.0,
+      "rewards/chosen": 8.56611328125,
+      "rewards/margins": 23.64619140625,
+      "rewards/rejected": -15.080078125,
+      "step": 1050
+    },
+    {
+      "epoch": 4.398546964193046,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.4613065803942906e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.63125,
+      "logps/rejected": -442.49375,
+      "loss": 0.0,
+      "rewards/chosen": 7.8025390625,
+      "rewards/margins": 22.5173828125,
+      "rewards/rejected": -14.71484375,
+      "step": 1060
+    },
+    {
+      "epoch": 4.440062272963155,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.4277807621091624e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.81640625,
+      "logps/rejected": -440.6625,
+      "loss": 0.0,
+      "rewards/chosen": 8.341015625,
+      "rewards/margins": 23.3091796875,
+      "rewards/rejected": -14.9681640625,
+      "step": 1070
+    },
+    {
+      "epoch": 4.481577581733264,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.394060309956786e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.913671875,
+      "logps/rejected": -436.83125,
+      "loss": 0.0,
+      "rewards/chosen": 9.330859375,
+      "rewards/margins": 23.29296875,
+      "rewards/rejected": -13.962109375,
+      "step": 1080
+    },
+    {
+      "epoch": 4.523092890503373,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.360152297959686e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.336328125,
+      "logps/rejected": -435.10625,
+      "loss": 0.0,
+      "rewards/chosen": 8.8078125,
+      "rewards/margins": 22.4814453125,
+      "rewards/rejected": -13.6736328125,
+      "step": 1090
+    },
+    {
+      "epoch": 4.564608199273482,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.326063839487501e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.49453125,
+      "logps/rejected": -439.28125,
+      "loss": 0.0,
+      "rewards/chosen": 8.76396484375,
+      "rewards/margins": 24.19541015625,
+      "rewards/rejected": -15.4314453125,
+      "step": 1100
+    },
+    {
+      "epoch": 4.606123508043591,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.2918020857647135e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.661328125,
+      "logps/rejected": -433.9,
+      "loss": 0.0,
+      "rewards/chosen": 7.90029296875,
+      "rewards/margins": 22.62099609375,
+      "rewards/rejected": -14.720703125,
+      "step": 1110
+    },
+    {
+      "epoch": 4.6476388168137,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.257374224370436e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.54296875,
+      "logps/rejected": -434.896875,
+      "loss": 0.0,
+      "rewards/chosen": 8.8478515625,
+      "rewards/margins": 23.24375,
+      "rewards/rejected": -14.3958984375,
+      "step": 1120
+    },
+    {
+      "epoch": 4.689154125583809,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.2227874777305664e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.783203125,
+      "logps/rejected": -435.821875,
+      "loss": 0.0,
+      "rewards/chosen": 8.14619140625,
+      "rewards/margins": 23.46865234375,
+      "rewards/rejected": -15.3224609375,
+      "step": 1130
+    },
+    {
+      "epoch": 4.730669434353918,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.188049101602638e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.450390625,
+      "logps/rejected": -440.34375,
+      "loss": 0.0,
+      "rewards/chosen": 8.70439453125,
+      "rewards/margins": 23.24814453125,
+      "rewards/rejected": -14.54375,
+      "step": 1140
+    },
+    {
+      "epoch": 4.772184743124027,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.1531663835536767e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.584765625,
+      "logps/rejected": -439.196875,
+      "loss": 0.0,
+      "rewards/chosen": 8.338671875,
+      "rewards/margins": 22.749609375,
+      "rewards/rejected": -14.4109375,
+      "step": 1150
+    },
+    {
+      "epoch": 4.813700051894136,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.118146641431378e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.916015625,
+      "logps/rejected": -432.1,
+      "loss": 0.0,
+      "rewards/chosen": 8.66708984375,
+      "rewards/margins": 23.12548828125,
+      "rewards/rejected": -14.4583984375,
+      "step": 1160
+    },
+    {
+      "epoch": 4.8552153606642445,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.0829972218289445e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.41171875,
+      "logps/rejected": -441.5,
+      "loss": 0.0,
+      "rewards/chosen": 8.599609375,
+      "rewards/margins": 23.2998046875,
+      "rewards/rejected": -14.7001953125,
+      "step": 1170
+    },
+    {
+      "epoch": 4.896730669434354,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.0477254985438803e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.587890625,
+      "logps/rejected": -434.490625,
+      "loss": 0.0,
+      "rewards/chosen": 8.99931640625,
+      "rewards/margins": 22.34404296875,
+      "rewards/rejected": -13.3447265625,
+      "step": 1180
+    },
+    {
+      "epoch": 4.9382459782044625,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 3.0123388710310876e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.934375,
+      "logps/rejected": -435.915625,
+      "loss": 0.0,
+      "rewards/chosen": 9.44716796875,
+      "rewards/margins": 23.35107421875,
+      "rewards/rejected": -13.90390625,
+      "step": 1190
+    },
+    {
+      "epoch": 4.979761286974572,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.9768447628505744e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.5484375,
+      "logps/rejected": -443.378125,
+      "loss": 0.0,
+      "rewards/chosen": 8.9732421875,
+      "rewards/margins": 24.3615234375,
+      "rewards/rejected": -15.38828125,
+      "step": 1200
+    },
+    {
+      "epoch": 5.020757654385054,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.941250620110107e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.15941455696203,
+      "logps/rejected": -441.3481012658228,
+      "loss": 0.0,
+      "rewards/chosen": 8.509988132911392,
+      "rewards/margins": 22.26829509493671,
+      "rewards/rejected": -13.758306962025317,
+      "step": 1210
+    },
+    {
+      "epoch": 5.062272963155164,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.905563909903138e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.30703125,
+      "logps/rejected": -430.825,
+      "loss": 0.0,
+      "rewards/chosen": 9.2783203125,
+      "rewards/margins": 23.89697265625,
+      "rewards/rejected": -14.61865234375,
+      "step": 1220
+    },
+    {
+      "epoch": 5.103788271925272,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.8697921187423233e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.29609375,
+      "logps/rejected": -433.93125,
+      "loss": 0.0,
+      "rewards/chosen": 8.59677734375,
+      "rewards/margins": 24.571582031250003,
+      "rewards/rejected": -15.9748046875,
+      "step": 1230
+    },
+    {
+      "epoch": 5.145303580695382,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.833942750988968e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.79375,
+      "logps/rejected": -436.0875,
+      "loss": 0.0,
+      "rewards/chosen": 9.20283203125,
+      "rewards/margins": 22.530566406250003,
+      "rewards/rejected": -13.327734375,
+      "step": 1240
+    },
+    {
+      "epoch": 5.18681888946549,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.798023327278733e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.024609375,
+      "logps/rejected": -440.65,
+      "loss": 0.0,
+      "rewards/chosen": 9.36552734375,
+      "rewards/margins": 23.12119140625,
+      "rewards/rejected": -13.7556640625,
+      "step": 1250
+    },
+    {
+      "epoch": 5.2283341982356,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.76204138294392e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.083203125,
+      "logps/rejected": -433.44375,
+      "loss": 0.0,
+      "rewards/chosen": 8.9607421875,
+      "rewards/margins": 2199394.3607421876,
+      "rewards/rejected": -2199385.4,
+      "step": 1260
+    },
+    {
+      "epoch": 5.269849507005708,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.726004466432678e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.461328125,
+      "logps/rejected": -435.31875,
+      "loss": 0.0,
+      "rewards/chosen": 9.04677734375,
+      "rewards/margins": 24.70908203125,
+      "rewards/rejected": -15.6623046875,
+      "step": 1270
+    },
+    {
+      "epoch": 5.311364815775818,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.6899201377254606e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.4,
+      "logps/rejected": -441.125,
+      "loss": 0.0,
+      "rewards/chosen": 8.2662109375,
+      "rewards/margins": 22.722460937500003,
+      "rewards/rejected": -14.45625,
+      "step": 1280
+    },
+    {
+      "epoch": 5.352880124545926,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.653795966749051e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.733984375,
+      "logps/rejected": -437.9,
+      "loss": 0.0,
+      "rewards/chosen": 8.8494140625,
+      "rewards/margins": 23.6392578125,
+      "rewards/rejected": -14.78984375,
+      "step": 1290
+    },
+    {
+      "epoch": 5.394395433316035,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.6176395317885215e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.358984375,
+      "logps/rejected": -442.2375,
+      "loss": 0.0,
+      "rewards/chosen": 8.07041015625,
+      "rewards/margins": 22.96103515625,
+      "rewards/rejected": -14.890625,
+      "step": 1300
+    },
+    {
+      "epoch": 5.435910742086144,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.5814584178974215e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.30625,
+      "logps/rejected": -442.35,
+      "loss": 0.0,
+      "rewards/chosen": 8.20556640625,
+      "rewards/margins": 23.52705078125,
+      "rewards/rejected": -15.321484375,
+      "step": 1310
+    },
+    {
+      "epoch": 5.477426050856253,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.545260215306552e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.909765625,
+      "logps/rejected": -441.915625,
+      "loss": 0.0,
+      "rewards/chosen": 9.357421875,
+      "rewards/margins": 23.463671875,
+      "rewards/rejected": -14.10625,
+      "step": 1320
+    },
+    {
+      "epoch": 5.518941359626362,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.5090525178316565e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.54765625,
+      "logps/rejected": -437.69375,
+      "loss": 0.0,
+      "rewards/chosen": 8.8115234375,
+      "rewards/margins": 22.658203125,
+      "rewards/rejected": -13.8466796875,
+      "step": 1330
+    },
+    {
+      "epoch": 5.560456668396471,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.472842921280357e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.5453125,
+      "logps/rejected": -438.059375,
+      "loss": 0.0,
+      "rewards/chosen": 8.769140625,
+      "rewards/margins": 23.8595703125,
+      "rewards/rejected": -15.0904296875,
+      "step": 1340
+    },
+    {
+      "epoch": 5.60197197716658,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.4366390218586705e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.751953125,
+      "logps/rejected": -438.825,
+      "loss": 0.0,
+      "rewards/chosen": 7.47724609375,
+      "rewards/margins": 22.64541015625,
+      "rewards/rejected": -15.1681640625,
+      "step": 1350
+    },
+    {
+      "epoch": 5.643487285936689,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.4004484145774466e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.194140625,
+      "logps/rejected": -437.415625,
+      "loss": 0.0,
+      "rewards/chosen": 8.998828125,
+      "rewards/margins": 23.770703124999997,
+      "rewards/rejected": -14.771875,
+      "step": 1360
+    },
+    {
+      "epoch": 5.685002594706798,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.3642786916590504e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.728515625,
+      "logps/rejected": -438.390625,
+      "loss": 0.0,
+      "rewards/chosen": 8.293359375,
+      "rewards/margins": 23.49375,
+      "rewards/rejected": -15.200390625,
+      "step": 1370
+    },
+    {
+      "epoch": 5.726517903476907,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.328137440944638e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.558984375,
+      "logps/rejected": -440.021875,
+      "loss": 0.0,
+      "rewards/chosen": 9.01044921875,
+      "rewards/margins": 23.94482421875,
+      "rewards/rejected": -14.934375,
+      "step": 1380
+    },
+    {
+      "epoch": 5.768033212247016,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.2920322443023414e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.25078125,
+      "logps/rejected": -438.846875,
+      "loss": 0.0,
+      "rewards/chosen": 8.04638671875,
+      "rewards/margins": 22.32841796875,
+      "rewards/rejected": -14.28203125,
+      "step": 1390
+    },
+    {
+      "epoch": 5.809548521017125,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.255970676036712e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.161328125,
+      "logps/rejected": -437.2,
+      "loss": 0.0,
+      "rewards/chosen": 8.37216796875,
+      "rewards/margins": 23.49521484375,
+      "rewards/rejected": -15.123046875,
+      "step": 1400
+    },
+    {
+      "epoch": 5.851063829787234,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.2199603012997527e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.688671875,
+      "logps/rejected": -441.709375,
+      "loss": 0.0,
+      "rewards/chosen": 8.783203125,
+      "rewards/margins": 23.58515625,
+      "rewards/rejected": -14.801953125,
+      "step": 1410
+    },
+    {
+      "epoch": 5.892579138557343,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.184008674503867e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.2765625,
+      "logps/rejected": -437.63125,
+      "loss": 0.0,
+      "rewards/chosen": 9.280078125,
+      "rewards/margins": 22.5248046875,
+      "rewards/rejected": -13.2447265625,
+      "step": 1420
+    },
+    {
+      "epoch": 5.9340944473274515,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.1481233377370588e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.459375,
+      "logps/rejected": -437.8125,
+      "loss": 0.0,
+      "rewards/chosen": 9.15703125,
+      "rewards/margins": 22.924609375,
+      "rewards/rejected": -13.767578125,
+      "step": 1430
+    },
+    {
+      "epoch": 5.975609756097561,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.1123118191807232e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.565625,
+      "logps/rejected": -441.365625,
+      "loss": 0.0,
+      "rewards/chosen": 9.16650390625,
+      "rewards/margins": 24.88935546875,
+      "rewards/rejected": -15.7228515625,
+      "step": 1440
+    },
+    {
+      "epoch": 6.016606123508043,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.0765816315303532e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.12856012658228,
+      "logps/rejected": -445.24050632911394,
+      "loss": 0.0,
+      "rewards/chosen": 8.624505537974683,
+      "rewards/margins": 22.528777689873415,
+      "rewards/rejected": -13.904272151898734,
+      "step": 1450
+    },
+    {
+      "epoch": 6.058121432278153,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.0409402704194904e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.18984375,
+      "logps/rejected": -432.2875,
+      "loss": 0.0,
+      "rewards/chosen": 9.14765625,
+      "rewards/margins": 23.389453125000003,
+      "rewards/rejected": -14.241796875,
+      "step": 1460
+    },
+    {
+      "epoch": 6.099636741048261,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 2.0053952128472592e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.054296875,
+      "logps/rejected": -432.309375,
+      "loss": 0.0,
+      "rewards/chosen": 8.3177734375,
+      "rewards/margins": 24.859765625,
+      "rewards/rejected": -16.5419921875,
+      "step": 1470
+    },
+    {
+      "epoch": 6.141152049818371,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.9699539156098133e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.35546875,
+      "logps/rejected": -442.309375,
+      "loss": 0.0,
+      "rewards/chosen": 9.515625,
+      "rewards/margins": 23.059765625,
+      "rewards/rejected": -13.544140625,
+      "step": 1480
+    },
+    {
+      "epoch": 6.182667358588479,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.9346238137360105e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.08046875,
+      "logps/rejected": -439.61875,
+      "loss": 0.0,
+      "rewards/chosen": 9.3615234375,
+      "rewards/margins": 22.60546875,
+      "rewards/rejected": -13.2439453125,
+      "step": 1490
+    },
+    {
+      "epoch": 6.224182667358589,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.899412318927666e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.1890625,
+      "logps/rejected": -436.075,
+      "loss": 0.0,
+      "rewards/chosen": 8.95849609375,
+      "rewards/margins": 2212503.3584960937,
+      "rewards/rejected": -2212494.4,
+      "step": 1500
+    },
+    {
+      "epoch": 6.265697976128697,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.86432681800469e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.323828125,
+      "logps/rejected": -436.259375,
+      "loss": 0.0,
+      "rewards/chosen": 9.04599609375,
+      "rewards/margins": 24.49697265625,
+      "rewards/rejected": -15.4509765625,
+      "step": 1510
+    },
+    {
+      "epoch": 6.307213284898807,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.8293746713554506e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.584765625,
+      "logps/rejected": -441.534375,
+      "loss": 0.0,
+      "rewards/chosen": 8.42900390625,
+      "rewards/margins": 23.038769531249997,
+      "rewards/rejected": -14.609765625,
+      "step": 1520
+    },
+    {
+      "epoch": 6.348728593668915,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.7945632113926785e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.46796875,
+      "logps/rejected": -437.25625,
+      "loss": 0.0,
+      "rewards/chosen": 8.8142578125,
+      "rewards/margins": 23.949609375,
+      "rewards/rejected": -15.1353515625,
+      "step": 1530
+    },
+    {
+      "epoch": 6.390243902439025,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.75989974101524e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.712109375,
+      "logps/rejected": -445.45,
+      "loss": 0.0,
+      "rewards/chosen": 7.95322265625,
+      "rewards/margins": 23.17822265625,
+      "rewards/rejected": -15.225,
+      "step": 1540
+    },
+    {
+      "epoch": 6.431759211209133,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.725391532076107e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.2296875,
+      "logps/rejected": -444.109375,
+      "loss": 0.0,
+      "rewards/chosen": 8.07529296875,
+      "rewards/margins": 23.04638671875,
+      "rewards/rejected": -14.97109375,
+      "step": 1550
+    },
+    {
+      "epoch": 6.473274519979243,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.6910458238568348e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.74921875,
+      "logps/rejected": -445.26875,
+      "loss": 0.0,
+      "rewards/chosen": 9.6302734375,
+      "rewards/margins": 23.798046874999997,
+      "rewards/rejected": -14.1677734375,
+      "step": 1560
+    },
+    {
+      "epoch": 6.514789828749351,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.6568698215488708e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.045703125,
+      "logps/rejected": -436.596875,
+      "loss": 0.0,
+      "rewards/chosen": 8.55966796875,
+      "rewards/margins": 23.00830078125,
+      "rewards/rejected": -14.4486328125,
+      "step": 1570
+    },
+    {
+      "epoch": 6.556305137519461,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.6228706947420257e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.472265625,
+      "logps/rejected": -438.95,
+      "loss": 0.0,
+      "rewards/chosen": 8.76123046875,
+      "rewards/margins": 23.66220703125,
+      "rewards/rejected": -14.9009765625,
+      "step": 1580
+    },
+    {
+      "epoch": 6.597820446289569,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.5890555759203973e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.26640625,
+      "logps/rejected": -440.375,
+      "loss": 0.0,
+      "rewards/chosen": 7.763671875,
+      "rewards/margins": 22.582421875,
+      "rewards/rejected": -14.81875,
+      "step": 1590
+    },
+    {
+      "epoch": 6.639335755059678,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.5554315589660932e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.46328125,
+      "logps/rejected": -439.01875,
+      "loss": 0.0,
+      "rewards/chosen": 9.001171875,
+      "rewards/margins": 24.100390625000003,
+      "rewards/rejected": -15.09921875,
+      "step": 1600
+    },
+    {
+      "epoch": 6.680851063829787,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.5220056976710432e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -58.163671875,
+      "logps/rejected": -439.484375,
+      "loss": 0.0,
+      "rewards/chosen": 8.3013671875,
+      "rewards/margins": 23.587890625,
+      "rewards/rejected": -15.2865234375,
+      "step": 1610
+    },
+    {
+      "epoch": 6.722366372599896,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.4887850042572242e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.42109375,
+      "logps/rejected": -440.946875,
+      "loss": 0.0,
+      "rewards/chosen": 8.8693359375,
+      "rewards/margins": 23.6337890625,
+      "rewards/rejected": -14.764453125,
+      "step": 1620
+    },
+    {
+      "epoch": 6.763881681370005,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.455776447905606e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.123828125,
+      "logps/rejected": -439.4,
+      "loss": 0.0,
+      "rewards/chosen": 8.18681640625,
+      "rewards/margins": 23.551464843749997,
+      "rewards/rejected": -15.3646484375,
+      "step": 1630
+    },
+    {
+      "epoch": 6.805396990140114,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.4229869532941265e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.4140625,
+      "logps/rejected": -436.46875,
+      "loss": 0.0,
+      "rewards/chosen": 7.9869140625,
+      "rewards/margins": 23.1359375,
+      "rewards/rejected": -15.1490234375,
+      "step": 1640
+    },
+    {
+      "epoch": 6.846912298910223,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.390423399145005e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.818359375,
+      "logps/rejected": -443.3625,
+      "loss": 0.0,
+      "rewards/chosen": 9.0294921875,
+      "rewards/margins": 23.2962890625,
+      "rewards/rejected": -14.266796875,
+      "step": 1650
+    },
+    {
+      "epoch": 6.888427607680332,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.358092616781689e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.6515625,
+      "logps/rejected": -440.925,
+      "loss": 0.0,
+      "rewards/chosen": 9.171875,
+      "rewards/margins": 22.953125,
+      "rewards/rejected": -13.78125,
+      "step": 1660
+    },
+    {
+      "epoch": 6.929942916450441,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.3260013886957536e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.4,
+      "logps/rejected": -438.96875,
+      "loss": 0.0,
+      "rewards/chosen": 9.43759765625,
+      "rewards/margins": 22.85263671875,
+      "rewards/rejected": -13.4150390625,
+      "step": 1670
+    },
+    {
+      "epoch": 6.97145822522055,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.294156447124041e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.90078125,
+      "logps/rejected": -440.3125,
+      "loss": 0.0,
+      "rewards/chosen": 8.8833984375,
+      "rewards/margins": 24.6234375,
+      "rewards/rejected": -15.7400390625,
+      "step": 1680
+    },
+    {
+      "epoch": 7.012454592631033,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.262564472636338e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.210443037974684,
+      "logps/rejected": -446.9367088607595,
+      "loss": 0.0,
+      "rewards/chosen": 8.921776107594937,
+      "rewards/margins": 23.11619857594937,
+      "rewards/rejected": -14.19442246835443,
+      "step": 1690
+    },
+    {
+      "epoch": 7.053969901401142,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.2312320927339053e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.343359375,
+      "logps/rejected": -434.45,
+      "loss": 0.0,
+      "rewards/chosen": 9.015625,
+      "rewards/margins": 23.36875,
+      "rewards/rejected": -14.353125,
+      "step": 1700
+    },
+    {
+      "epoch": 7.09548521017125,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.200165880459123e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.309765625,
+      "logps/rejected": -433.73125,
+      "loss": 0.0,
+      "rewards/chosen": 8.315625,
+      "rewards/margins": 24.880468750000002,
+      "rewards/rejected": -16.56484375,
+      "step": 1710
+    },
+    {
+      "epoch": 7.13700051894136,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.1693723530165772e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.632421875,
+      "logps/rejected": -441.75625,
+      "loss": 0.0,
+      "rewards/chosen": 9.3783203125,
+      "rewards/margins": 23.195703125,
+      "rewards/rejected": -13.8173828125,
+      "step": 1720
+    },
+    {
+      "epoch": 7.178515827711468,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.1388579704058457e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.860546875,
+      "logps/rejected": -439.965625,
+      "loss": 0.0,
+      "rewards/chosen": 9.3494140625,
+      "rewards/margins": 22.3892578125,
+      "rewards/rejected": -13.03984375,
+      "step": 1730
+    },
+    {
+      "epoch": 7.220031136481578,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.108629134066296e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.313671875,
+      "logps/rejected": -437.840625,
+      "loss": 0.0,
+      "rewards/chosen": 8.825,
+      "rewards/margins": 2212504.4250000003,
+      "rewards/rejected": -2212495.6,
+      "step": 1740
+    },
+    {
+      "epoch": 7.261546445251686,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.078692185534159e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.49765625,
+      "logps/rejected": -434.309375,
+      "loss": 0.0,
+      "rewards/chosen": 9.344140625,
+      "rewards/margins": 25.0677734375,
+      "rewards/rejected": -15.7236328125,
+      "step": 1750
+    },
+    {
+      "epoch": 7.303061754021796,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.0490534051121808e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.96953125,
+      "logps/rejected": -444.521875,
+      "loss": 0.0,
+      "rewards/chosen": 8.55732421875,
+      "rewards/margins": 23.45693359375,
+      "rewards/rejected": -14.899609375,
+      "step": 1760
+    },
+    {
+      "epoch": 7.344577062791904,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 1.019719010552107e-07,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.471484375,
+      "logps/rejected": -436.584375,
+      "loss": 0.0,
+      "rewards/chosen": 8.81484375,
+      "rewards/margins": 23.71328125,
+      "rewards/rejected": -14.8984375,
+      "step": 1770
+    },
+    {
+      "epoch": 7.386092371562014,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.90695155750299e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.3984375,
+      "logps/rejected": -446.9625,
+      "loss": 0.0,
+      "rewards/chosen": 8.10048828125,
+      "rewards/margins": 23.87900390625,
+      "rewards/rejected": -15.778515625,
+      "step": 1780
+    },
+    {
+      "epoch": 7.427607680332122,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.619879294567459e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.53203125,
+      "logps/rejected": -441.90625,
+      "loss": 0.0,
+      "rewards/chosen": 7.6572265625,
+      "rewards/margins": 22.3876953125,
+      "rewards/rejected": -14.73046875,
+      "step": 1790
+    },
+    {
+      "epoch": 7.469122989102232,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.336033539977312e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.80390625,
+      "logps/rejected": -446.228125,
+      "loss": 0.0,
+      "rewards/chosen": 9.64072265625,
+      "rewards/margins": 24.61689453125,
+      "rewards/rejected": -14.976171875,
+      "step": 1800
+    },
+    {
+      "epoch": 7.51063829787234,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 9.055473840124517e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.40234375,
+      "logps/rejected": -440.203125,
+      "loss": 0.0,
+      "rewards/chosen": 8.8470703125,
+      "rewards/margins": 22.6546875,
+      "rewards/rejected": -13.8076171875,
+      "step": 1810
+    },
+    {
+      "epoch": 7.55215360664245,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.778259052038248e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.129296875,
+      "logps/rejected": -439.65625,
+      "loss": 0.0,
+      "rewards/chosen": 8.33759765625,
+      "rewards/margins": 23.311816406250003,
+      "rewards/rejected": -14.97421875,
+      "step": 1820
+    },
+    {
+      "epoch": 7.593668915412558,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.50444733103759e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.0296875,
+      "logps/rejected": -441.346875,
+      "loss": 0.0,
+      "rewards/chosen": 8.6201171875,
+      "rewards/margins": 23.2560546875,
+      "rewards/rejected": -14.6359375,
+      "step": 1830
+    },
+    {
+      "epoch": 7.635184224182668,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 8.234096118531478e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.198828125,
+      "logps/rejected": -437.803125,
+      "loss": 0.0,
+      "rewards/chosen": 8.58095703125,
+      "rewards/margins": 23.43388671875,
+      "rewards/rejected": -14.8529296875,
+      "step": 1840
+    },
+    {
+      "epoch": 7.676699532952776,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.967262129968377e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.703515625,
+      "logps/rejected": -437.525,
+      "loss": 0.0,
+      "rewards/chosen": 8.17451171875,
+      "rewards/margins": 24.18349609375,
+      "rewards/rejected": -16.008984375,
+      "step": 1850
+    },
+    {
+      "epoch": 7.718214841722885,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.704001342938285e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.475390625,
+      "logps/rejected": -442.2125,
+      "loss": 0.0,
+      "rewards/chosen": 8.72392578125,
+      "rewards/margins": 23.56005859375,
+      "rewards/rejected": -14.8361328125,
+      "step": 1860
+    },
+    {
+      "epoch": 7.759730150492994,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.444368985429502e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.555078125,
+      "logps/rejected": -441.846875,
+      "loss": 0.0,
+      "rewards/chosen": 8.44189453125,
+      "rewards/margins": 23.68095703125,
+      "rewards/rejected": -15.2390625,
+      "step": 1870
+    },
+    {
+      "epoch": 7.801245459263104,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 7.188419524242667e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.173046875,
+      "logps/rejected": -439.278125,
+      "loss": 0.0,
+      "rewards/chosen": 7.99853515625,
+      "rewards/margins": 22.91826171875,
+      "rewards/rejected": -14.9197265625,
+      "step": 1880
+    },
+    {
+      "epoch": 7.842760768033212,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.936206653564472e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -57.181640625,
+      "logps/rejected": -442.08125,
+      "loss": 0.0,
+      "rewards/chosen": 8.89619140625,
+      "rewards/margins": 23.23193359375,
+      "rewards/rejected": -14.3357421875,
+      "step": 1890
+    },
+    {
+      "epoch": 7.884276076803321,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.687783283703466e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.98046875,
+      "logps/rejected": -441.953125,
+      "loss": 0.0,
+      "rewards/chosen": 9.32265625,
+      "rewards/margins": 23.374609375,
+      "rewards/rejected": -14.051953125,
+      "step": 1900
+    },
+    {
+      "epoch": 7.92579138557343,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.443201529990299e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.859375,
+      "logps/rejected": -440.440625,
+      "loss": 0.0,
+      "rewards/chosen": 9.15498046875,
+      "rewards/margins": 22.85517578125,
+      "rewards/rejected": -13.7001953125,
+      "step": 1910
+    },
+    {
+      "epoch": 7.967306694343539,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 6.202512701844756e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.1625,
+      "logps/rejected": -437.36875,
+      "loss": 0.0,
+      "rewards/chosen": 9.1548828125,
+      "rewards/margins": 24.6205078125,
+      "rewards/rejected": -15.465625,
+      "step": 1920
+    },
+    {
+      "epoch": 8.008303061754022,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.965767292011814e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.936708860759495,
+      "logps/rejected": -447.37025316455697,
+      "loss": 0.0,
+      "rewards/chosen": 8.511570411392405,
+      "rewards/margins": 22.623121044303797,
+      "rewards/rejected": -14.111550632911392,
+      "step": 1930
+    },
+    {
+      "epoch": 8.04981837052413,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.733014965969091e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -56.451171875,
+      "logps/rejected": -438.26875,
+      "loss": 0.0,
+      "rewards/chosen": 9.00244140625,
+      "rewards/margins": 23.33095703125,
+      "rewards/rejected": -14.328515625,
+      "step": 1940
+    },
+    {
+      "epoch": 8.09133367929424,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.5043045515077827e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.31171875,
+      "logps/rejected": -434.296875,
+      "loss": 0.0,
+      "rewards/chosen": 8.59873046875,
+      "rewards/margins": 25.21201171875,
+      "rewards/rejected": -16.61328125,
+      "step": 1950
+    },
+    {
+      "epoch": 8.13284898806435,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.2796840284893703e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.348828125,
+      "logps/rejected": -440.35625,
+      "loss": 0.0,
+      "rewards/chosen": 9.0904296875,
+      "rewards/margins": 22.8658203125,
+      "rewards/rejected": -13.775390625,
+      "step": 1960
+    },
+    {
+      "epoch": 8.174364296834458,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 5.059200518780177e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.9859375,
+      "logps/rejected": -438.6875,
+      "loss": 0.0,
+      "rewards/chosen": 9.78017578125,
+      "rewards/margins": 23.330761718749997,
+      "rewards/rejected": -13.5505859375,
+      "step": 1970
+    },
+    {
+      "epoch": 8.215879605604567,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.8429002763659604e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -53.984375,
+      "logps/rejected": -443.065625,
+      "loss": 0.0,
+      "rewards/chosen": 8.719140625,
+      "rewards/margins": 2212504.519140625,
+      "rewards/rejected": -2212495.8,
+      "step": 1980
+    },
+    {
+      "epoch": 8.257394914374675,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.630828677648557e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -55.964453125,
+      "logps/rejected": -432.21875,
+      "loss": 0.0,
+      "rewards/chosen": 9.16728515625,
+      "rewards/margins": 24.916699218749997,
+      "rewards/rejected": -15.7494140625,
+      "step": 1990
+    },
+    {
+      "epoch": 8.298910223144784,
+      "grad_norm": 0.0,
+      "kl": 0.0,
+      "learning_rate": 4.423030211926598e-08,
+      "logits/chosen": NaN,
+      "logits/rejected": NaN,
+      "logps/chosen": -54.16953125,
+      "logps/rejected": -443.203125,
+      "loss": 0.0,
+      "rewards/chosen": 8.7251953125,
+      "rewards/margins": 23.1640625,
+      "rewards/rejected": -14.4388671875,
+      "step": 2000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 2410,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 10,
+  "save_steps": 200,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 2,
+  "trial_name": null,
+  "trial_params": null
+}

makto_v1/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e80fafb7df95cb118fca345ba4cb29e4d68f072d9f7f2e1decb1daa3a21a1b76
+size 7441

makto_v1/vocab.json ADDED Viewed

The diff for this file is too large to render. See raw diff