Instructions to use kth8/gemma-3-270m-it-OpenCode-Title-Generator with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use kth8/gemma-3-270m-it-OpenCode-Title-Generator with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="kth8/gemma-3-270m-it-OpenCode-Title-Generator")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("kth8/gemma-3-270m-it-OpenCode-Title-Generator")
model = AutoModelForCausalLM.from_pretrained("kth8/gemma-3-270m-it-OpenCode-Title-Generator")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use kth8/gemma-3-270m-it-OpenCode-Title-Generator with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "kth8/gemma-3-270m-it-OpenCode-Title-Generator"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kth8/gemma-3-270m-it-OpenCode-Title-Generator",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/kth8/gemma-3-270m-it-OpenCode-Title-Generator

SGLang

How to use kth8/gemma-3-270m-it-OpenCode-Title-Generator with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "kth8/gemma-3-270m-it-OpenCode-Title-Generator" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kth8/gemma-3-270m-it-OpenCode-Title-Generator",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "kth8/gemma-3-270m-it-OpenCode-Title-Generator" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "kth8/gemma-3-270m-it-OpenCode-Title-Generator",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use kth8/gemma-3-270m-it-OpenCode-Title-Generator with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kth8/gemma-3-270m-it-OpenCode-Title-Generator to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for kth8/gemma-3-270m-it-OpenCode-Title-Generator to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for kth8/gemma-3-270m-it-OpenCode-Title-Generator to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="kth8/gemma-3-270m-it-OpenCode-Title-Generator",
    max_seq_length=2048,
)

Docker Model Runner
How to use kth8/gemma-3-270m-it-OpenCode-Title-Generator with Docker Model Runner:
```
docker model run hf.co/kth8/gemma-3-270m-it-OpenCode-Title-Generator
```

kth8 commited on 3 days ago

Commit

a29fd5f

verified ·

1 Parent(s): 767981d

Upload folder using huggingface_hub

Browse files

Files changed (6) hide show

README.md +28 -48
chat_template.jinja +3 -0
model.safetensors +1 -1
train/log.json +780 -758
train/training_loss.png +0 -0
train/validation_loss.png +0 -0

README.md CHANGED Viewed

@@ -21,27 +21,6 @@ Trained with the exact system prompt OpenCode's [title agent uses](https://raw.g
 ## Usage example
-Point to this model with `small_model` in `opencode.jsonc` file.
-```json
-{
-  "$schema": "https://opencode.ai/config.json",
-  "provider": {
-    "title": {
-      "npm": "@ai-sdk/openai-compatible",
-      "options": {
-        "baseURL": "http://127.0.0.1:8080/v1",
-        "apiKey": "not-needed"
-      },
-      "models": {
-        "generator": {}
-      }
-    }
-  },
-  "small_model": "title/generator"
-}
-```
 **System prompt**
 ```
 You are a title generator. You output ONLY a thread title. Nothing else.
@@ -121,38 +100,39 @@ Student course passing growth calculation
 - Weight decay: 0.01
 ## Training stats
-- Date: 2026-05-27T13:32:50.844050
 - GPU: NVIDIA A100-SXM4-40GB
-- Peak VRAM usage: 12.129 GB
-- Global step: 1588
-- Training runtime (seconds): 1573.3242
-- Best validation loss: 1.3689830303192139
 | Step | Training Loss | Validation Loss |
-|------|--------------|----------------|
-| 79   | 1.916200     | 1.783801       |
-| 158  | 1.725300     | 1.744159       |
-| 237  | 1.693900     | 1.640494       |
-| 316  | 1.628300     | 1.608212       |
-| 395  | 1.535700     | 1.557622       |
-| 474  | 1.525200     | 1.579373       |
-| 553  | 1.465500     | 1.528539       |
-| 632  | 1.447900     | 1.489644       |
-| 711  | 1.572100     | 1.488969       |
-| 790  | 1.528400     | 1.472376       |
-| 869  | 1.497800     | 1.438234       |
-| 948  | 1.476900     | 1.431505       |
-| 1027 | 1.387800     | 1.412816       |
-| 1106 | 1.369100     | 1.401051       |
-| 1185 | 1.286400     | 1.391667       |
-| 1264 | 1.406300     | 1.379098       |
-| 1343 | 1.412500     | 1.374640       |
-| 1422 | 1.321700     | 1.371226       |
-| 1501 | 1.383900     | 1.368983       |
-| 1580 | 1.337300     | 1.369141       |
 ## Framework versions
-- Unsloth: 2026.5.8
 - TRL: 0.22.2
 - Transformers: 4.56.2
 - Pytorch: 2.11.0+cu128

 ## Usage example
 **System prompt**
 ```
 You are a title generator. You output ONLY a thread title. Nothing else.
 - Weight decay: 0.01
 ## Training stats
+- Date: 2026-06-01T11:04:43.747952
 - GPU: NVIDIA A100-SXM4-40GB
+- Peak VRAM usage: 12.15 GB
+- Global step: 1607
+- Training runtime (seconds): 1590.5658
+- Best validation loss: 1.408400058746338
 | Step | Training Loss | Validation Loss |
+|------|---------------|-----------------|
+| 0    | No log        | 5.064917        |
+| 80   | 1.672600      | 1.848531        |
+| 160  | 1.695400      | 1.742237        |
+| 240  | 1.751600      | 1.726482        |
+| 320  | 1.427200      | 1.663712        |
+| 400  | 1.550400      | 1.609400        |
+| 480  | 1.559000      | 1.573220        |
+| 560  | 1.471900      | 1.572365        |
+| 640  | 1.538100      | 1.539643        |
+| 720  | 1.485500      | 1.515100        |
+| 800  | 1.391200      | 1.486133        |
+| 880  | 1.390600      | 1.473583        |
+| 960  | 1.405300      | 1.461052        |
+| 1040 | 1.392000      | 1.450962        |
+| 1120 | 1.521300      | 1.440739        |
+| 1200 | 1.438300      | 1.431336        |
+| 1280 | 1.336900      | 1.418500        |
+| 1360 | 1.375000      | 1.413560        |
+| 1440 | 1.342100      | 1.408760        |
+| 1520 | 1.309400      | 1.408400        |
+| 1600 | 1.428100      | 1.409352        |
 ## Framework versions
+- Unsloth: 2026.5.9
 - TRL: 0.22.2
 - Transformers: 4.56.2
 - Pytorch: 2.11.0+cu128

chat_template.jinja CHANGED Viewed

@@ -16,6 +16,9 @@
     {%- set loop_messages = messages -%}
 {%- endif -%}
 {%- for message in loop_messages -%}
     {%- if (message['role'] == 'assistant') -%}
         {%- set role = "model" -%}
     {%- else -%}

     {%- set loop_messages = messages -%}
 {%- endif -%}
 {%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
     {%- if (message['role'] == 'assistant') -%}
         {%- set role = "model" -%}
     {%- else -%}

model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6a6bbc6d0e985a9c71352fdf27e960db747ca2d5d83dbbde6df8fed839f0a2d4
 size 536223056

 version https://git-lfs.github.com/spec/v1
+oid sha256:5d0871bbb0474c330eaa27c51c4dbe2d73a4ea4909837452845236d3f0d7882f
 size 536223056

train/log.json CHANGED Viewed

@@ -1,1277 +1,1299 @@
 [
   {
-    "loss": 4.6142,
-    "grad_norm": 33.659332275390625,
     "learning_rate": 1.8e-05,
-    "epoch": 0.006297229219143577,
     "step": 10
   },
   {
-    "loss": 2.6482,
-    "grad_norm": 14.412582397460938,
     "learning_rate": 3.8e-05,
-    "epoch": 0.012594458438287154,
     "step": 20
   },
   {
-    "loss": 2.2011,
-    "grad_norm": 9.777717590332031,
     "learning_rate": 5.8e-05,
-    "epoch": 0.018891687657430732,
     "step": 30
   },
   {
-    "loss": 1.9662,
-    "grad_norm": 17.46149253845215,
     "learning_rate": 7.800000000000001e-05,
-    "epoch": 0.02518891687657431,
     "step": 40
   },
   {
-    "loss": 1.8247,
-    "grad_norm": 9.683545112609863,
     "learning_rate": 9.8e-05,
-    "epoch": 0.031486146095717885,
     "step": 50
   },
   {
-    "loss": 1.8919,
-    "grad_norm": 10.557292938232422,
     "learning_rate": 0.000118,
-    "epoch": 0.037783375314861464,
     "step": 60
   },
   {
-    "loss": 1.9162,
-    "grad_norm": 11.902077674865723,
     "learning_rate": 0.000138,
-    "epoch": 0.04408060453400504,
     "step": 70
   },
   {
-    "eval_loss": 1.7838011980056763,
-    "eval_runtime": 33.8157,
-    "eval_samples_per_second": 15.348,
-    "eval_steps_per_second": 3.844,
-    "epoch": 0.04974811083123426,
-    "step": 79
   },
   {
-    "loss": 1.8701,
-    "grad_norm": 8.326041221618652,
-    "learning_rate": 0.00015800000000000002,
-    "epoch": 0.05037783375314862,
     "step": 80
   },
   {
-    "loss": 1.7494,
-    "grad_norm": 8.273054122924805,
     "learning_rate": 0.00017800000000000002,
-    "epoch": 0.05667506297229219,
     "step": 90
   },
   {
-    "loss": 1.7557,
-    "grad_norm": 10.647675514221191,
     "learning_rate": 0.00019800000000000002,
-    "epoch": 0.06297229219143577,
     "step": 100
   },
   {
-    "loss": 1.7535,
-    "grad_norm": 6.604005813598633,
-    "learning_rate": 0.0001999819475629623,
-    "epoch": 0.06926952141057935,
     "step": 110
   },
   {
-    "loss": 1.7317,
-    "grad_norm": 6.132753372192383,
-    "learning_rate": 0.00019991955244474347,
-    "epoch": 0.07556675062972293,
     "step": 120
   },
   {
-    "loss": 1.6196,
-    "grad_norm": 5.765186786651611,
-    "learning_rate": 0.0001998126195808803,
-    "epoch": 0.0818639798488665,
     "step": 130
   },
   {
-    "loss": 1.5384,
-    "grad_norm": 5.294787883758545,
-    "learning_rate": 0.00019966119663520412,
-    "epoch": 0.08816120906801007,
     "step": 140
   },
   {
-    "loss": 1.7253,
-    "grad_norm": 8.26528549194336,
-    "learning_rate": 0.00019946535110237904,
-    "epoch": 0.09445843828715365,
     "step": 150
   },
   {
-    "eval_loss": 1.7441585063934326,
-    "eval_runtime": 14.4715,
-    "eval_samples_per_second": 35.864,
-    "eval_steps_per_second": 8.983,
-    "epoch": 0.09949622166246852,
-    "step": 158
   },
   {
-    "loss": 1.7126,
-    "grad_norm": 5.565060138702393,
-    "learning_rate": 0.00019922517027781746,
-    "epoch": 0.10075566750629723,
     "step": 160
   },
   {
-    "loss": 1.8884,
-    "grad_norm": 6.4396867752075195,
-    "learning_rate": 0.000198940761218769,
-    "epoch": 0.1070528967254408,
     "step": 170
   },
   {
-    "loss": 1.7091,
-    "grad_norm": 5.561753273010254,
-    "learning_rate": 0.0001986122506966014,
-    "epoch": 0.11335012594458438,
     "step": 180
   },
   {
-    "loss": 1.7571,
-    "grad_norm": 4.833555221557617,
-    "learning_rate": 0.000198239785140294,
-    "epoch": 0.11964735516372796,
     "step": 190
   },
   {
-    "loss": 1.6005,
-    "grad_norm": 5.930907249450684,
-    "learning_rate": 0.000197823530571169,
-    "epoch": 0.12594458438287154,
     "step": 200
   },
   {
-    "loss": 1.7721,
-    "grad_norm": 5.551156520843506,
-    "learning_rate": 0.00019736367252888974,
-    "epoch": 0.13224181360201512,
     "step": 210
   },
   {
-    "loss": 1.6479,
-    "grad_norm": 5.802966117858887,
-    "learning_rate": 0.000196860415988759,
-    "epoch": 0.1385390428211587,
     "step": 220
   },
   {
-    "loss": 1.6939,
-    "grad_norm": 5.381985664367676,
-    "learning_rate": 0.00019631398527035422,
-    "epoch": 0.14483627204030228,
     "step": 230
   },
   {
-    "eval_loss": 1.6404942274093628,
-    "eval_runtime": 14.4847,
-    "eval_samples_per_second": 35.831,
-    "eval_steps_per_second": 8.975,
-    "epoch": 0.14924433249370278,
-    "step": 237
   },
   {
-    "loss": 1.6613,
-    "grad_norm": 5.851009368896484,
-    "learning_rate": 0.00019572462393754005,
-    "epoch": 0.15113350125944586,
     "step": 240
   },
   {
-    "loss": 1.6204,
-    "grad_norm": 5.324190139770508,
-    "learning_rate": 0.00019509259468990323,
-    "epoch": 0.1574307304785894,
     "step": 250
   },
   {
-    "loss": 1.5024,
-    "grad_norm": 6.643704891204834,
-    "learning_rate": 0.00019441817924565786,
-    "epoch": 0.163727959697733,
     "step": 260
   },
   {
-    "loss": 1.7425,
-    "grad_norm": 5.175152778625488,
-    "learning_rate": 0.00019370167821607353,
-    "epoch": 0.17002518891687657,
     "step": 270
   },
   {
-    "loss": 1.6053,
-    "grad_norm": 5.162317276000977,
-    "learning_rate": 0.00019294341097148194,
-    "epoch": 0.17632241813602015,
     "step": 280
   },
   {
-    "loss": 1.6262,
-    "grad_norm": 4.8840179443359375,
-    "learning_rate": 0.0001921437154989221,
-    "epoch": 0.18261964735516373,
     "step": 290
   },
   {
-    "loss": 1.6046,
-    "grad_norm": 5.020822048187256,
-    "learning_rate": 0.0001913029482514873,
-    "epoch": 0.1889168765743073,
     "step": 300
   },
   {
-    "loss": 1.6283,
-    "grad_norm": 4.799527645111084,
-    "learning_rate": 0.00019042148398944116,
-    "epoch": 0.1952141057934509,
     "step": 310
   },
   {
-    "eval_loss": 1.6082123517990112,
-    "eval_runtime": 14.6548,
-    "eval_samples_per_second": 35.415,
-    "eval_steps_per_second": 8.871,
-    "epoch": 0.19899244332493704,
-    "step": 316
   },
   {
-    "loss": 1.6391,
-    "grad_norm": 5.666315078735352,
-    "learning_rate": 0.0001894997156131734,
-    "epoch": 0.20151133501259447,
     "step": 320
   },
   {
-    "loss": 1.6637,
-    "grad_norm": 5.880155086517334,
-    "learning_rate": 0.00018853805398807004,
-    "epoch": 0.20780856423173805,
     "step": 330
   },
   {
-    "loss": 1.444,
-    "grad_norm": 5.846175193786621,
-    "learning_rate": 0.00018753692776137584,
-    "epoch": 0.2141057934508816,
     "step": 340
   },
   {
-    "loss": 1.625,
-    "grad_norm": 5.362205982208252,
-    "learning_rate": 0.00018649678317113084,
-    "epoch": 0.22040302267002518,
     "step": 350
   },
   {
-    "loss": 1.6577,
-    "grad_norm": 4.623821258544922,
-    "learning_rate": 0.0001854180838472659,
-    "epoch": 0.22670025188916876,
     "step": 360
   },
   {
-    "loss": 1.5298,
-    "grad_norm": 4.931164741516113,
-    "learning_rate": 0.00018430131060494616,
-    "epoch": 0.23299748110831234,
     "step": 370
   },
   {
-    "loss": 1.5142,
-    "grad_norm": 5.050358295440674,
-    "learning_rate": 0.00018314696123025454,
-    "epoch": 0.23929471032745592,
     "step": 380
   },
   {
-    "loss": 1.5357,
-    "grad_norm": 4.571812629699707,
-    "learning_rate": 0.00018195555025831022,
-    "epoch": 0.2455919395465995,
     "step": 390
   },
   {
-    "eval_loss": 1.5576220750808716,
-    "eval_runtime": 14.7082,
-    "eval_samples_per_second": 35.286,
-    "eval_steps_per_second": 8.839,
-    "epoch": 0.24874055415617127,
-    "step": 395
   },
   {
-    "loss": 1.5179,
-    "grad_norm": 4.708597183227539,
-    "learning_rate": 0.00018072760874392215,
-    "epoch": 0.2518891687657431,
     "step": 400
   },
   {
-    "loss": 1.5781,
-    "grad_norm": 4.262819766998291,
-    "learning_rate": 0.00017946368402487845,
-    "epoch": 0.25818639798488663,
     "step": 410
   },
   {
-    "loss": 1.4886,
-    "grad_norm": 5.044946193695068,
-    "learning_rate": 0.00017816433947797858,
-    "epoch": 0.26448362720403024,
     "step": 420
   },
   {
-    "loss": 1.4781,
-    "grad_norm": 5.743882656097412,
-    "learning_rate": 0.00017683015426791566,
-    "epoch": 0.2707808564231738,
     "step": 430
   },
   {
-    "loss": 1.5219,
-    "grad_norm": 5.393696308135986,
-    "learning_rate": 0.00017546172308912213,
-    "epoch": 0.2770780856423174,
     "step": 440
   },
   {
-    "loss": 1.5132,
-    "grad_norm": 7.644551753997803,
-    "learning_rate": 0.0001740596559006929,
-    "epoch": 0.28337531486146095,
     "step": 450
   },
   {
-    "loss": 1.4883,
-    "grad_norm": 5.462601661682129,
-    "learning_rate": 0.00017262457765450457,
-    "epoch": 0.28967254408060455,
     "step": 460
   },
   {
-    "loss": 1.5252,
-    "grad_norm": 4.247793197631836,
-    "learning_rate": 0.000171157128016652,
-    "epoch": 0.2959697732997481,
     "step": 470
   },
   {
-    "eval_loss": 1.5793730020523071,
-    "eval_runtime": 14.5182,
-    "eval_samples_per_second": 35.748,
-    "eval_steps_per_second": 8.954,
-    "epoch": 0.29848866498740556,
-    "step": 474
   },
   {
-    "loss": 1.6033,
-    "grad_norm": 6.310419082641602,
-    "learning_rate": 0.00016965796108232598,
-    "epoch": 0.3022670025188917,
     "step": 480
   },
   {
-    "loss": 1.6718,
-    "grad_norm": 5.100507736206055,
-    "learning_rate": 0.00016812774508425933,
-    "epoch": 0.30856423173803527,
     "step": 490
   },
   {
-    "loss": 1.5078,
-    "grad_norm": 4.1262946128845215,
-    "learning_rate": 0.00016656716209487174,
-    "epoch": 0.3148614609571788,
     "step": 500
   },
   {
-    "loss": 1.4854,
-    "grad_norm": 5.230551719665527,
-    "learning_rate": 0.00016497690772224522,
-    "epoch": 0.3211586901763224,
     "step": 510
   },
   {
-    "loss": 1.5091,
-    "grad_norm": 5.111631393432617,
-    "learning_rate": 0.00016335769080006674,
-    "epoch": 0.327455919395466,
     "step": 520
   },
   {
-    "loss": 1.5427,
-    "grad_norm": 4.551891803741455,
-    "learning_rate": 0.00016171023307167545,
-    "epoch": 0.3337531486146096,
     "step": 530
   },
   {
-    "loss": 1.5839,
-    "grad_norm": 4.708298683166504,
-    "learning_rate": 0.00016003526886835553,
-    "epoch": 0.34005037783375314,
     "step": 540
   },
   {
-    "loss": 1.4655,
-    "grad_norm": 4.417643070220947,
-    "learning_rate": 0.00015833354478201844,
-    "epoch": 0.34634760705289674,
     "step": 550
   },
   {
-    "eval_loss": 1.5285385847091675,
-    "eval_runtime": 14.6291,
-    "eval_samples_per_second": 35.477,
-    "eval_steps_per_second": 8.886,
-    "epoch": 0.3482367758186398,
-    "step": 553
   },
   {
-    "loss": 1.5909,
-    "grad_norm": 5.512932300567627,
-    "learning_rate": 0.00015660581933241993,
-    "epoch": 0.3526448362720403,
     "step": 560
   },
   {
-    "loss": 1.5667,
-    "grad_norm": 5.218059539794922,
-    "learning_rate": 0.00015485286262906044,
-    "epoch": 0.3589420654911839,
     "step": 570
   },
   {
-    "loss": 1.584,
-    "grad_norm": 5.250601291656494,
-    "learning_rate": 0.00015307545602791955,
-    "epoch": 0.36523929471032746,
     "step": 580
   },
   {
-    "loss": 1.5065,
-    "grad_norm": 5.601760387420654,
-    "learning_rate": 0.00015127439178317745,
-    "epoch": 0.371536523929471,
     "step": 590
   },
   {
-    "loss": 1.4472,
-    "grad_norm": 4.8602986335754395,
-    "learning_rate": 0.00014945047269407876,
-    "epoch": 0.3778337531486146,
     "step": 600
   },
   {
-    "loss": 1.5242,
-    "grad_norm": 3.2672348022460938,
-    "learning_rate": 0.0001476045117470958,
-    "epoch": 0.38413098236775817,
     "step": 610
   },
   {
-    "loss": 1.4048,
-    "grad_norm": 4.925168037414551,
-    "learning_rate": 0.0001457373317535515,
-    "epoch": 0.3904282115869018,
     "step": 620
   },
   {
-    "loss": 1.4479,
-    "grad_norm": 5.590626239776611,
-    "learning_rate": 0.00014384976498286252,
-    "epoch": 0.3967254408060453,
     "step": 630
   },
   {
-    "eval_loss": 1.489644169807434,
-    "eval_runtime": 14.4216,
-    "eval_samples_per_second": 35.988,
-    "eval_steps_per_second": 9.014,
-    "epoch": 0.3979848866498741,
-    "step": 632
   },
   {
-    "loss": 1.4313,
-    "grad_norm": 5.052401065826416,
-    "learning_rate": 0.00014194265279156675,
-    "epoch": 0.40302267002518893,
     "step": 640
   },
   {
-    "loss": 1.5194,
-    "grad_norm": 3.9724292755126953,
-    "learning_rate": 0.00014001684524830057,
-    "epoch": 0.4093198992443325,
     "step": 650
   },
   {
-    "loss": 1.4166,
-    "grad_norm": 4.177152156829834,
-    "learning_rate": 0.0001380732007548925,
-    "epoch": 0.4156171284634761,
     "step": 660
   },
   {
-    "loss": 1.5784,
-    "grad_norm": 4.706490516662598,
-    "learning_rate": 0.00013611258566374257,
-    "epoch": 0.42191435768261965,
     "step": 670
   },
   {
-    "loss": 1.3634,
-    "grad_norm": 4.3738017082214355,
-    "learning_rate": 0.00013413587389165784,
-    "epoch": 0.4282115869017632,
     "step": 680
   },
   {
-    "loss": 1.5313,
-    "grad_norm": 3.9545352458953857,
-    "learning_rate": 0.00013214394653031616,
-    "epoch": 0.4345088161209068,
     "step": 690
   },
   {
-    "loss": 1.3985,
-    "grad_norm": 4.201825141906738,
-    "learning_rate": 0.000130137691453532,
-    "epoch": 0.44080604534005036,
     "step": 700
   },
   {
-    "loss": 1.5721,
-    "grad_norm": 5.4546966552734375,
-    "learning_rate": 0.0001281180029214988,
-    "epoch": 0.44710327455919396,
     "step": 710
   },
   {
-    "eval_loss": 1.4889687299728394,
-    "eval_runtime": 14.577,
-    "eval_samples_per_second": 35.604,
-    "eval_steps_per_second": 8.918,
-    "epoch": 0.4477329974811083,
-    "step": 711
   },
   {
-    "loss": 1.4477,
-    "grad_norm": 5.406968116760254,
-    "learning_rate": 0.00012608578118218535,
-    "epoch": 0.4534005037783375,
     "step": 720
   },
   {
-    "loss": 1.4616,
-    "grad_norm": 4.538527965545654,
-    "learning_rate": 0.00012404193207006246,
-    "epoch": 0.4596977329974811,
     "step": 730
   },
   {
-    "loss": 1.5133,
-    "grad_norm": 4.29402494430542,
-    "learning_rate": 0.00012198736660234009,
-    "epoch": 0.4659949622166247,
     "step": 740
   },
   {
-    "loss": 1.4501,
-    "grad_norm": 3.8199307918548584,
-    "learning_rate": 0.0001199230005728942,
-    "epoch": 0.4722921914357683,
     "step": 750
   },
   {
-    "loss": 1.5288,
-    "grad_norm": 6.067654132843018,
-    "learning_rate": 0.00011784975414406426,
-    "epoch": 0.47858942065491183,
     "step": 760
   },
   {
-    "loss": 1.3803,
-    "grad_norm": 5.480033874511719,
-    "learning_rate": 0.00011576855143650371,
-    "epoch": 0.48488664987405544,
     "step": 770
   },
   {
-    "loss": 1.5517,
-    "grad_norm": 4.182514190673828,
-    "learning_rate": 0.00011368032011726635,
-    "epoch": 0.491183879093199,
     "step": 780
   },
   {
-    "loss": 1.5284,
-    "grad_norm": 4.433547496795654,
-    "learning_rate": 0.00011158599098631137,
-    "epoch": 0.49748110831234255,
     "step": 790
   },
   {
-    "eval_loss": 1.4723758697509766,
-    "eval_runtime": 14.538,
-    "eval_samples_per_second": 35.7,
-    "eval_steps_per_second": 8.942,
-    "epoch": 0.49748110831234255,
-    "step": 790
   },
   {
-    "loss": 1.5369,
-    "grad_norm": 4.8588433265686035,
-    "learning_rate": 0.00010948649756161246,
-    "epoch": 0.5037783375314862,
     "step": 800
   },
   {
-    "loss": 1.4349,
-    "grad_norm": 4.88569450378418,
-    "learning_rate": 0.00010738277566305514,
-    "epoch": 0.5100755667506297,
     "step": 810
   },
   {
-    "loss": 1.6126,
-    "grad_norm": 5.098734378814697,
-    "learning_rate": 0.0001052757629953082,
-    "epoch": 0.5163727959697733,
     "step": 820
   },
   {
-    "loss": 1.5043,
-    "grad_norm": 4.012162208557129,
-    "learning_rate": 0.00010316639872985472,
-    "epoch": 0.5226700251889169,
     "step": 830
   },
   {
-    "loss": 1.4438,
-    "grad_norm": 4.813101768493652,
-    "learning_rate": 0.00010105562308636973,
-    "epoch": 0.5289672544080605,
     "step": 840
   },
   {
-    "loss": 1.434,
-    "grad_norm": 3.8592841625213623,
-    "learning_rate": 9.894437691363028e-05,
-    "epoch": 0.535264483627204,
     "step": 850
   },
   {
-    "loss": 1.4978,
-    "grad_norm": 5.364985942840576,
-    "learning_rate": 9.683360127014529e-05,
-    "epoch": 0.5415617128463476,
     "step": 860
   },
   {
-    "eval_loss": 1.4382336139678955,
-    "eval_runtime": 14.714,
-    "eval_samples_per_second": 35.272,
-    "eval_steps_per_second": 8.835,
-    "epoch": 0.5472292191435768,
-    "step": 869
   },
   {
-    "loss": 1.4885,
-    "grad_norm": 5.363029479980469,
-    "learning_rate": 9.47242370046918e-05,
-    "epoch": 0.5478589420654912,
-    "step": 870
   },
   {
-    "loss": 1.5044,
-    "grad_norm": 4.980364799499512,
-    "learning_rate": 9.261722433694485e-05,
-    "epoch": 0.5541561712846348,
     "step": 880
   },
   {
-    "loss": 1.353,
-    "grad_norm": 4.4146952629089355,
-    "learning_rate": 9.051350243838756e-05,
-    "epoch": 0.5604534005037783,
     "step": 890
   },
   {
-    "loss": 1.3262,
-    "grad_norm": 5.358567237854004,
-    "learning_rate": 8.841400901368864e-05,
-    "epoch": 0.5667506297229219,
     "step": 900
   },
   {
-    "loss": 1.5514,
-    "grad_norm": 4.715535640716553,
-    "learning_rate": 8.631967988273366e-05,
-    "epoch": 0.5730478589420654,
     "step": 910
   },
   {
-    "loss": 1.4533,
-    "grad_norm": 4.502087116241455,
-    "learning_rate": 8.423144856349631e-05,
-    "epoch": 0.5793450881612091,
     "step": 920
   },
   {
-    "loss": 1.4378,
-    "grad_norm": 4.85370397567749,
-    "learning_rate": 8.215024585593577e-05,
-    "epoch": 0.5856423173803527,
     "step": 930
   },
   {
-    "loss": 1.4769,
-    "grad_norm": 4.3678297996521,
-    "learning_rate": 8.007699942710581e-05,
-    "epoch": 0.5919395465994962,
     "step": 940
   },
   {
-    "eval_loss": 1.4315046072006226,
-    "eval_runtime": 14.4016,
-    "eval_samples_per_second": 36.038,
-    "eval_steps_per_second": 9.027,
-    "epoch": 0.5969773299748111,
-    "step": 948
   },
   {
-    "loss": 1.3736,
-    "grad_norm": 4.551608085632324,
-    "learning_rate": 7.801263339765994e-05,
-    "epoch": 0.5982367758186398,
-    "step": 950
   },
   {
-    "loss": 1.4156,
-    "grad_norm": 3.7702224254608154,
-    "learning_rate": 7.595806792993759e-05,
-    "epoch": 0.6045340050377834,
     "step": 960
   },
   {
-    "loss": 1.3623,
-    "grad_norm": 4.440454483032227,
-    "learning_rate": 7.391421881781468e-05,
-    "epoch": 0.610831234256927,
     "step": 970
   },
   {
-    "loss": 1.38,
-    "grad_norm": 4.850752830505371,
-    "learning_rate": 7.188199707850122e-05,
-    "epoch": 0.6171284634760705,
     "step": 980
   },
   {
-    "loss": 1.3101,
-    "grad_norm": 3.9209938049316406,
-    "learning_rate": 6.986230854646804e-05,
-    "epoch": 0.6234256926952141,
     "step": 990
   },
   {
-    "loss": 1.3677,
-    "grad_norm": 4.586505889892578,
-    "learning_rate": 6.785605346968386e-05,
-    "epoch": 0.6297229219143576,
     "step": 1000
   },
   {
-    "loss": 1.4159,
-    "grad_norm": 4.723811149597168,
-    "learning_rate": 6.586412610834221e-05,
-    "epoch": 0.6360201511335013,
     "step": 1010
   },
   {
-    "loss": 1.3878,
-    "grad_norm": 4.9766154289245605,
-    "learning_rate": 6.388741433625746e-05,
-    "epoch": 0.6423173803526449,
     "step": 1020
   },
   {
-    "eval_loss": 1.4128155708312988,
-    "eval_runtime": 14.5331,
-    "eval_samples_per_second": 35.712,
-    "eval_steps_per_second": 8.945,
-    "epoch": 0.6467254408060453,
-    "step": 1027
   },
   {
-    "loss": 1.3711,
-    "grad_norm": 4.173240661621094,
-    "learning_rate": 6.192679924510752e-05,
-    "epoch": 0.6486146095717884,
-    "step": 1030
   },
   {
-    "loss": 1.408,
-    "grad_norm": 5.662805557250977,
-    "learning_rate": 5.998315475169942e-05,
-    "epoch": 0.654911838790932,
     "step": 1040
   },
   {
-    "loss": 1.4101,
-    "grad_norm": 4.8551716804504395,
-    "learning_rate": 5.8057347208433266e-05,
-    "epoch": 0.6612090680100756,
     "step": 1050
   },
   {
-    "loss": 1.2795,
-    "grad_norm": 4.329833030700684,
-    "learning_rate": 5.615023501713752e-05,
-    "epoch": 0.6675062972292192,
     "step": 1060
   },
   {
-    "loss": 1.4149,
-    "grad_norm": 3.759012460708618,
-    "learning_rate": 5.4262668246448475e-05,
-    "epoch": 0.6738035264483627,
     "step": 1070
   },
   {
-    "loss": 1.3728,
-    "grad_norm": 4.901814937591553,
-    "learning_rate": 5.239548825290419e-05,
-    "epoch": 0.6801007556675063,
     "step": 1080
   },
   {
-    "loss": 1.4085,
-    "grad_norm": 4.735474109649658,
-    "learning_rate": 5.0549527305921265e-05,
-    "epoch": 0.6863979848866498,
     "step": 1090
   },
   {
-    "loss": 1.3691,
-    "grad_norm": 3.6761767864227295,
-    "learning_rate": 4.872560821682256e-05,
-    "epoch": 0.6926952141057935,
     "step": 1100
   },
   {
-    "eval_loss": 1.4010512828826904,
-    "eval_runtime": 14.5256,
-    "eval_samples_per_second": 35.73,
-    "eval_steps_per_second": 8.95,
-    "epoch": 0.6964735516372796,
-    "step": 1106
   },
   {
-    "loss": 1.3057,
-    "grad_norm": 4.493793964385986,
-    "learning_rate": 4.692454397208048e-05,
-    "epoch": 0.698992443324937,
-    "step": 1110
   },
   {
-    "loss": 1.3721,
-    "grad_norm": 5.266867160797119,
-    "learning_rate": 4.5147137370939595e-05,
-    "epoch": 0.7052896725440806,
     "step": 1120
   },
   {
-    "loss": 1.4906,
-    "grad_norm": 4.867866039276123,
-    "learning_rate": 4.339418066758008e-05,
-    "epoch": 0.7115869017632241,
     "step": 1130
   },
   {
-    "loss": 1.4071,
-    "grad_norm": 4.608040809631348,
-    "learning_rate": 4.166645521798157e-05,
-    "epoch": 0.7178841309823678,
     "step": 1140
   },
   {
-    "loss": 1.3585,
-    "grad_norm": 7.612720012664795,
-    "learning_rate": 3.99647311316445e-05,
-    "epoch": 0.7241813602015114,
     "step": 1150
   },
   {
-    "loss": 1.4131,
-    "grad_norm": 5.223732948303223,
-    "learning_rate": 3.828976692832458e-05,
-    "epoch": 0.7304785894206549,
     "step": 1160
   },
   {
-    "loss": 1.3566,
-    "grad_norm": 4.113969802856445,
-    "learning_rate": 3.6642309199933275e-05,
-    "epoch": 0.7367758186397985,
     "step": 1170
   },
   {
-    "loss": 1.2864,
-    "grad_norm": 3.8852932453155518,
-    "learning_rate": 3.502309227775482e-05,
-    "epoch": 0.743073047858942,
     "step": 1180
   },
   {
-    "eval_loss": 1.391667366027832,
-    "eval_runtime": 14.5167,
-    "eval_samples_per_second": 35.752,
-    "eval_steps_per_second": 8.955,
-    "epoch": 0.7462216624685138,
-    "step": 1185
   },
   {
-    "loss": 1.416,
-    "grad_norm": 4.738650798797607,
-    "learning_rate": 3.343283790512829e-05,
-    "epoch": 0.7493702770780857,
-    "step": 1190
   },
   {
-    "loss": 1.4496,
-    "grad_norm": 4.808592319488525,
-    "learning_rate": 3.1872254915740686e-05,
-    "epoch": 0.7556675062972292,
     "step": 1200
   },
   {
-    "loss": 1.3199,
-    "grad_norm": 5.310007095336914,
-    "learning_rate": 3.0342038917674064e-05,
-    "epoch": 0.7619647355163728,
     "step": 1210
   },
   {
-    "loss": 1.4116,
-    "grad_norm": 3.6367766857147217,
-    "learning_rate": 2.8842871983347998e-05,
-    "epoch": 0.7682619647355163,
     "step": 1220
   },
   {
-    "loss": 1.4192,
-    "grad_norm": 4.783074378967285,
-    "learning_rate": 2.7375422345495448e-05,
-    "epoch": 0.77455919395466,
     "step": 1230
   },
   {
-    "loss": 1.294,
-    "grad_norm": 5.352371692657471,
-    "learning_rate": 2.5940344099307146e-05,
-    "epoch": 0.7808564231738035,
     "step": 1240
   },
   {
-    "loss": 1.493,
-    "grad_norm": 4.887784957885742,
-    "learning_rate": 2.45382769108779e-05,
-    "epoch": 0.7871536523929471,
     "step": 1250
   },
   {
-    "loss": 1.4063,
-    "grad_norm": 5.015107154846191,
-    "learning_rate": 2.3169845732084338e-05,
-    "epoch": 0.7934508816120907,
     "step": 1260
   },
   {
-    "eval_loss": 1.3790984153747559,
-    "eval_runtime": 14.5192,
-    "eval_samples_per_second": 35.746,
-    "eval_steps_per_second": 8.954,
-    "epoch": 0.7959697732997482,
-    "step": 1264
   },
   {
-    "loss": 1.4125,
-    "grad_norm": 4.3769917488098145,
-    "learning_rate": 2.1835660522021417e-05,
-    "epoch": 0.7997481108312342,
-    "step": 1270
   },
   {
-    "loss": 1.3623,
-    "grad_norm": 4.965677261352539,
-    "learning_rate": 2.0536315975121544e-05,
-    "epoch": 0.8060453400503779,
     "step": 1280
   },
   {
-    "loss": 1.3702,
-    "grad_norm": 5.39815616607666,
-    "learning_rate": 1.927239125607788e-05,
-    "epoch": 0.8123425692695214,
     "step": 1290
   },
   {
-    "loss": 1.3878,
-    "grad_norm": 4.974949836730957,
-    "learning_rate": 1.8044449741689794e-05,
-    "epoch": 0.818639798488665,
     "step": 1300
   },
   {
-    "loss": 1.4957,
-    "grad_norm": 4.583181381225586,
-    "learning_rate": 1.6853038769745467e-05,
-    "epoch": 0.8249370277078085,
     "step": 1310
   },
   {
-    "loss": 1.4018,
-    "grad_norm": 5.503086566925049,
-    "learning_rate": 1.5698689395053835e-05,
-    "epoch": 0.8312342569269522,
     "step": 1320
   },
   {
-    "loss": 1.2619,
-    "grad_norm": 4.569085121154785,
-    "learning_rate": 1.4581916152734132e-05,
-    "epoch": 0.8375314861460957,
     "step": 1330
   },
   {
-    "loss": 1.4125,
-    "grad_norm": 5.070247173309326,
-    "learning_rate": 1.3503216828869192e-05,
-    "epoch": 0.8438287153652393,
     "step": 1340
   },
   {
-    "eval_loss": 1.3746401071548462,
-    "eval_runtime": 14.702,
-    "eval_samples_per_second": 35.301,
-    "eval_steps_per_second": 8.842,
-    "epoch": 0.8457178841309824,
-    "step": 1343
   },
   {
-    "loss": 1.405,
-    "grad_norm": 4.505115032196045,
-    "learning_rate": 1.2463072238624185e-05,
-    "epoch": 0.8501259445843828,
-    "step": 1350
   },
   {
-    "loss": 1.2935,
-    "grad_norm": 3.9875564575195312,
-    "learning_rate": 1.146194601192998e-05,
-    "epoch": 0.8564231738035264,
     "step": 1360
   },
   {
-    "loss": 1.3752,
-    "grad_norm": 4.175727844238281,
-    "learning_rate": 1.0500284386826597e-05,
-    "epoch": 0.8627204030226701,
     "step": 1370
   },
   {
-    "loss": 1.4002,
-    "grad_norm": 3.8710429668426514,
-    "learning_rate": 9.578516010558847e-06,
-    "epoch": 0.8690176322418136,
     "step": 1380
   },
   {
-    "loss": 1.2572,
-    "grad_norm": 39.77766418457031,
-    "learning_rate": 8.697051748512708e-06,
-    "epoch": 0.8753148614609572,
     "step": 1390
   },
   {
-    "loss": 1.3341,
-    "grad_norm": 5.350057601928711,
-    "learning_rate": 7.856284501077926e-06,
-    "epoch": 0.8816120906801007,
     "step": 1400
   },
   {
-    "loss": 1.3181,
-    "grad_norm": 4.475919246673584,
-    "learning_rate": 7.056589028518068e-06,
-    "epoch": 0.8879093198992444,
     "step": 1410
   },
   {
-    "loss": 1.3217,
-    "grad_norm": 4.369590759277344,
-    "learning_rate": 6.2983217839264756e-06,
-    "epoch": 0.8942065491183879,
     "step": 1420
   },
   {
-    "eval_loss": 1.37122642993927,
-    "eval_runtime": 14.501,
-    "eval_samples_per_second": 35.791,
-    "eval_steps_per_second": 8.965,
-    "epoch": 0.8954659949622166,
-    "step": 1422
   },
   {
-    "loss": 1.3551,
-    "grad_norm": 4.054989337921143,
-    "learning_rate": 5.581820754342137e-06,
-    "epoch": 0.9005037783375315,
-    "step": 1430
   },
   {
-    "loss": 1.2635,
-    "grad_norm": 4.094259738922119,
-    "learning_rate": 4.907405310096769e-06,
-    "epoch": 0.906801007556675,
     "step": 1440
   },
   {
-    "loss": 1.3872,
-    "grad_norm": 4.884560585021973,
-    "learning_rate": 4.275376062459946e-06,
-    "epoch": 0.9130982367758187,
     "step": 1450
   },
   {
-    "loss": 1.2567,
-    "grad_norm": 4.7786641120910645,
-    "learning_rate": 3.6860147296457816e-06,
-    "epoch": 0.9193954659949622,
     "step": 1460
   },
   {
-    "loss": 1.4019,
-    "grad_norm": 4.289524078369141,
-    "learning_rate": 3.1395840112410034e-06,
-    "epoch": 0.9256926952141058,
     "step": 1470
   },
   {
-    "loss": 1.3127,
-    "grad_norm": 4.1973490715026855,
-    "learning_rate": 2.636327471110289e-06,
-    "epoch": 0.9319899244332494,
     "step": 1480
   },
   {
-    "loss": 1.2502,
-    "grad_norm": 4.9555206298828125,
-    "learning_rate": 2.1764694288310184e-06,
-    "epoch": 0.9382871536523929,
     "step": 1490
   },
   {
-    "loss": 1.3839,
-    "grad_norm": 5.079888343811035,
-    "learning_rate": 1.7602148597060108e-06,
-    "epoch": 0.9445843828715366,
     "step": 1500
   },
   {
-    "eval_loss": 1.3689830303192139,
-    "eval_runtime": 14.5025,
-    "eval_samples_per_second": 35.787,
-    "eval_steps_per_second": 8.964,
-    "epoch": 0.9452141057934509,
-    "step": 1501
   },
   {
-    "loss": 1.4754,
-    "grad_norm": 4.302427768707275,
-    "learning_rate": 1.3877493033986223e-06,
-    "epoch": 0.9508816120906801,
-    "step": 1510
   },
   {
-    "loss": 1.3091,
-    "grad_norm": 4.7720866203308105,
-    "learning_rate": 1.0592387812310311e-06,
-    "epoch": 0.9571788413098237,
     "step": 1520
   },
   {
-    "loss": 1.2316,
-    "grad_norm": 4.849659442901611,
-    "learning_rate": 7.748297221825573e-07,
-    "epoch": 0.9634760705289672,
     "step": 1530
   },
   {
-    "loss": 1.2809,
-    "grad_norm": 4.5142621994018555,
-    "learning_rate": 5.346488976209596e-07,
-    "epoch": 0.9697732997481109,
     "step": 1540
   },
   {
-    "loss": 1.2613,
-    "grad_norm": 4.6826019287109375,
-    "learning_rate": 3.3880336479590325e-07,
-    "epoch": 0.9760705289672544,
     "step": 1550
   },
   {
-    "loss": 1.3195,
-    "grad_norm": 4.848607540130615,
-    "learning_rate": 1.8738041911969818e-07,
-    "epoch": 0.982367758186398,
     "step": 1560
   },
   {
-    "loss": 1.3067,
-    "grad_norm": 4.836756706237793,
-    "learning_rate": 8.044755525652425e-08,
-    "epoch": 0.9886649874055415,
     "step": 1570
   },
   {
-    "loss": 1.3373,
-    "grad_norm": 4.4072771072387695,
-    "learning_rate": 1.8052437037707758e-08,
-    "epoch": 0.9949622166246851,
     "step": 1580
   },
   {
-    "eval_loss": 1.3691413402557373,
-    "eval_runtime": 14.5693,
-    "eval_samples_per_second": 35.623,
-    "eval_steps_per_second": 8.923,
-    "epoch": 0.9949622166246851,
-    "step": 1580
   },
   {
-    "train_runtime": 1573.3242,
-    "train_samples_per_second": 16.149,
-    "train_steps_per_second": 1.009,
-    "total_flos": 1.159913454336e+16,
-    "train_loss": 1.5156250816748784,
     "epoch": 1.0,
-    "step": 1588
   }
 ]

 [
   {
+    "eval_loss": 5.064917087554932,
+    "eval_runtime": 82.8434,
+    "eval_samples_per_second": 6.337,
+    "eval_steps_per_second": 1.593,
+    "epoch": 0,
+    "step": 0
+  },
+  {
+    "loss": 4.4134,
+    "grad_norm": 34.756309509277344,
     "learning_rate": 1.8e-05,
+    "epoch": 0.006222775357809583,
     "step": 10
   },
   {
+    "loss": 2.8965,
+    "grad_norm": 16.411876678466797,
     "learning_rate": 3.8e-05,
+    "epoch": 0.012445550715619166,
     "step": 20
   },
   {
+    "loss": 2.1559,
+    "grad_norm": 10.711593627929688,
     "learning_rate": 5.8e-05,
+    "epoch": 0.018668326073428748,
     "step": 30
   },
   {
+    "loss": 2.0065,
+    "grad_norm": 9.655144691467285,
     "learning_rate": 7.800000000000001e-05,
+    "epoch": 0.024891101431238332,
     "step": 40
   },
   {
+    "loss": 1.9285,
+    "grad_norm": 10.14815902709961,
     "learning_rate": 9.8e-05,
+    "epoch": 0.031113876789047916,
     "step": 50
   },
   {
+    "loss": 1.905,
+    "grad_norm": 9.267602920532227,
     "learning_rate": 0.000118,
+    "epoch": 0.037336652146857496,
     "step": 60
   },
   {
+    "loss": 1.8057,
+    "grad_norm": 9.699400901794434,
     "learning_rate": 0.000138,
+    "epoch": 0.043559427504667084,
     "step": 70
   },
   {
+    "loss": 1.6726,
+    "grad_norm": 7.2970871925354,
+    "learning_rate": 0.00015800000000000002,
+    "epoch": 0.049782202862476664,
+    "step": 80
   },
   {
+    "eval_loss": 1.848530650138855,
+    "eval_runtime": 14.1069,
+    "eval_samples_per_second": 37.216,
+    "eval_steps_per_second": 9.357,
+    "epoch": 0.049782202862476664,
     "step": 80
   },
   {
+    "loss": 1.7914,
+    "grad_norm": 7.156599998474121,
     "learning_rate": 0.00017800000000000002,
+    "epoch": 0.056004978220286245,
     "step": 90
   },
   {
+    "loss": 1.8229,
+    "grad_norm": 7.823815822601318,
     "learning_rate": 0.00019800000000000002,
+    "epoch": 0.06222775357809583,
     "step": 100
   },
   {
+    "loss": 1.9023,
+    "grad_norm": 8.658170700073242,
+    "learning_rate": 0.00019998239988424423,
+    "epoch": 0.06845052893590542,
     "step": 110
   },
   {
+    "loss": 1.8214,
+    "grad_norm": 6.837601184844971,
+    "learning_rate": 0.0001999215679316913,
+    "epoch": 0.07467330429371499,
     "step": 120
   },
   {
+    "loss": 1.8212,
+    "grad_norm": 6.444572448730469,
+    "learning_rate": 0.00019981731328627206,
+    "epoch": 0.08089607965152458,
     "step": 130
   },
   {
+    "loss": 1.6969,
+    "grad_norm": 7.096024036407471,
+    "learning_rate": 0.00019966968125369522,
+    "epoch": 0.08711885500933417,
     "step": 140
   },
   {
+    "loss": 1.7863,
+    "grad_norm": 6.71297550201416,
+    "learning_rate": 0.00019947873599008388,
+    "epoch": 0.09334163036714374,
     "step": 150
   },
   {
+    "loss": 1.6954,
+    "grad_norm": 6.715601921081543,
+    "learning_rate": 0.00019924456047409517,
+    "epoch": 0.09956440572495333,
+    "step": 160
   },
   {
+    "eval_loss": 1.7422373294830322,
+    "eval_runtime": 13.8482,
+    "eval_samples_per_second": 37.911,
+    "eval_steps_per_second": 9.532,
+    "epoch": 0.09956440572495333,
     "step": 160
   },
   {
+    "loss": 1.7892,
+    "grad_norm": 6.36358118057251,
+    "learning_rate": 0.00019896725647086072,
+    "epoch": 0.10578718108276292,
     "step": 170
   },
   {
+    "loss": 1.7179,
+    "grad_norm": 8.09688949584961,
+    "learning_rate": 0.0001986469444877626,
+    "epoch": 0.11200995644057249,
     "step": 180
   },
   {
+    "loss": 1.6861,
+    "grad_norm": 5.653729438781738,
+    "learning_rate": 0.0001982837637220647,
+    "epoch": 0.11823273179838208,
     "step": 190
   },
   {
+    "loss": 1.6671,
+    "grad_norm": 6.865845203399658,
+    "learning_rate": 0.00019787787200042223,
+    "epoch": 0.12445550715619166,
     "step": 200
   },
   {
+    "loss": 1.7291,
+    "grad_norm": 5.955836296081543,
+    "learning_rate": 0.00019742944571029517,
+    "epoch": 0.13067828251400124,
     "step": 210
   },
   {
+    "loss": 1.626,
+    "grad_norm": 5.953787803649902,
+    "learning_rate": 0.00019693867972329598,
+    "epoch": 0.13690105787181084,
     "step": 220
   },
   {
+    "loss": 1.5731,
+    "grad_norm": 5.453970909118652,
+    "learning_rate": 0.00019640578731050488,
+    "epoch": 0.1431238332296204,
     "step": 230
   },
   {
+    "loss": 1.7516,
+    "grad_norm": 6.274188041687012,
+    "learning_rate": 0.00019583100004978886,
+    "epoch": 0.14934660858742999,
+    "step": 240
   },
   {
+    "eval_loss": 1.726481556892395,
+    "eval_runtime": 14.0063,
+    "eval_samples_per_second": 37.483,
+    "eval_steps_per_second": 9.424,
+    "epoch": 0.14934660858742999,
     "step": 240
   },
   {
+    "loss": 1.603,
+    "grad_norm": 5.389549732208252,
+    "learning_rate": 0.00019521456772516552,
+    "epoch": 0.1555693839452396,
     "step": 250
   },
   {
+    "loss": 1.6531,
+    "grad_norm": 5.605257987976074,
+    "learning_rate": 0.0001945567582182551,
+    "epoch": 0.16179215930304916,
     "step": 260
   },
   {
+    "loss": 1.6709,
+    "grad_norm": 6.4445343017578125,
+    "learning_rate": 0.00019385785739186746,
+    "epoch": 0.16801493466085873,
     "step": 270
   },
   {
+    "loss": 1.8325,
+    "grad_norm": 4.806204319000244,
+    "learning_rate": 0.0001931181689657756,
+    "epoch": 0.17423771001866833,
     "step": 280
   },
   {
+    "loss": 1.7735,
+    "grad_norm": 5.748836994171143,
+    "learning_rate": 0.00019233801438472875,
+    "epoch": 0.1804604853764779,
     "step": 290
   },
   {
+    "loss": 1.7027,
+    "grad_norm": 4.680629730224609,
+    "learning_rate": 0.00019151773267876273,
+    "epoch": 0.18668326073428748,
     "step": 300
   },
   {
+    "loss": 1.5229,
+    "grad_norm": 5.180474758148193,
+    "learning_rate": 0.0001906576803158686,
+    "epoch": 0.19290603609209708,
     "step": 310
   },
   {
+    "loss": 1.4272,
+    "grad_norm": 4.142552852630615,
+    "learning_rate": 0.00018975823104708313,
+    "epoch": 0.19912881144990666,
+    "step": 320
   },
   {
+    "eval_loss": 1.6637120246887207,
+    "eval_runtime": 14.0189,
+    "eval_samples_per_second": 37.45,
+    "eval_steps_per_second": 9.416,
+    "epoch": 0.19912881144990666,
     "step": 320
   },
   {
+    "loss": 1.591,
+    "grad_norm": 5.900393486022949,
+    "learning_rate": 0.00018881977574406838,
+    "epoch": 0.20535158680771623,
     "step": 330
   },
   {
+    "loss": 1.681,
+    "grad_norm": 5.877817153930664,
+    "learning_rate": 0.00018784272222925198,
+    "epoch": 0.21157436216552583,
     "step": 340
   },
   {
+    "loss": 1.5724,
+    "grad_norm": 4.48579216003418,
+    "learning_rate": 0.00018682749509860012,
+    "epoch": 0.2177971375233354,
     "step": 350
   },
   {
+    "loss": 1.6694,
+    "grad_norm": 6.317138195037842,
+    "learning_rate": 0.00018577453553710215,
+    "epoch": 0.22401991288114498,
     "step": 360
   },
   {
+    "loss": 1.585,
+    "grad_norm": 5.043068885803223,
+    "learning_rate": 0.00018468430112704573,
+    "epoch": 0.23024268823895458,
     "step": 370
   },
   {
+    "loss": 1.4904,
+    "grad_norm": 4.478468894958496,
+    "learning_rate": 0.00018355726564916628,
+    "epoch": 0.23646546359676415,
     "step": 380
   },
   {
+    "loss": 1.5321,
+    "grad_norm": 6.474656105041504,
+    "learning_rate": 0.00018239391887675722,
+    "epoch": 0.24268823895457373,
     "step": 390
   },
   {
+    "loss": 1.5504,
+    "grad_norm": 5.6627726554870605,
+    "learning_rate": 0.00018119476636283018,
+    "epoch": 0.24891101431238333,
+    "step": 400
   },
   {
+    "eval_loss": 1.6094003915786743,
+    "eval_runtime": 13.9871,
+    "eval_samples_per_second": 37.535,
+    "eval_steps_per_second": 9.437,
+    "epoch": 0.24891101431238333,
     "step": 400
   },
   {
+    "loss": 1.5178,
+    "grad_norm": 3.671290397644043,
+    "learning_rate": 0.00017996032922041797,
+    "epoch": 0.2551337896701929,
     "step": 410
   },
   {
+    "loss": 1.5326,
+    "grad_norm": 4.7752814292907715,
+    "learning_rate": 0.00017869114389611575,
+    "epoch": 0.2613565650280025,
     "step": 420
   },
   {
+    "loss": 1.6097,
+    "grad_norm": 4.629915237426758,
+    "learning_rate": 0.00017738776193695853,
+    "epoch": 0.26757934038581205,
     "step": 430
   },
   {
+    "loss": 1.5883,
+    "grad_norm": 4.744765281677246,
+    "learning_rate": 0.00017605074975073664,
+    "epoch": 0.2738021157436217,
     "step": 440
   },
   {
+    "loss": 1.5456,
+    "grad_norm": 5.553370952606201,
+    "learning_rate": 0.00017468068835985325,
+    "epoch": 0.28002489110143125,
     "step": 450
   },
   {
+    "loss": 1.635,
+    "grad_norm": 4.544879913330078,
+    "learning_rate": 0.00017327817314883055,
+    "epoch": 0.2862476664592408,
     "step": 460
   },
   {
+    "loss": 1.3816,
+    "grad_norm": 4.572506427764893,
+    "learning_rate": 0.00017184381360557498,
+    "epoch": 0.2924704418170504,
     "step": 470
   },
   {
+    "loss": 1.559,
+    "grad_norm": 5.5629119873046875,
+    "learning_rate": 0.00017037823305651343,
+    "epoch": 0.29869321717485997,
+    "step": 480
   },
   {
+    "eval_loss": 1.573219656944275,
+    "eval_runtime": 13.9728,
+    "eval_samples_per_second": 37.573,
+    "eval_steps_per_second": 9.447,
+    "epoch": 0.29869321717485997,
     "step": 480
   },
   {
+    "loss": 1.5028,
+    "grad_norm": 4.841222763061523,
+    "learning_rate": 0.0001688820683957156,
+    "epoch": 0.30491599253266954,
     "step": 490
   },
   {
+    "loss": 1.5322,
+    "grad_norm": 4.653014183044434,
+    "learning_rate": 0.00016735596980812047,
+    "epoch": 0.3111387678904792,
     "step": 500
   },
   {
+    "loss": 1.5097,
+    "grad_norm": 5.121198654174805,
+    "learning_rate": 0.0001658006004869867,
+    "epoch": 0.31736154324828875,
     "step": 510
   },
   {
+    "loss": 1.6415,
+    "grad_norm": 5.429177761077881,
+    "learning_rate": 0.00016421663634569046,
+    "epoch": 0.3235843186060983,
     "step": 520
   },
   {
+    "loss": 1.5053,
+    "grad_norm": 6.6233673095703125,
+    "learning_rate": 0.00016260476572399496,
+    "epoch": 0.3298070939639079,
     "step": 530
   },
   {
+    "loss": 1.4254,
+    "grad_norm": 4.580263137817383,
+    "learning_rate": 0.00016096568908892047,
+    "epoch": 0.33602986932171747,
     "step": 540
   },
   {
+    "loss": 1.5475,
+    "grad_norm": 5.61928129196167,
+    "learning_rate": 0.00015930011873034375,
+    "epoch": 0.3422526446795271,
     "step": 550
   },
   {
+    "loss": 1.4719,
+    "grad_norm": 5.081675052642822,
+    "learning_rate": 0.00015760877845145995,
+    "epoch": 0.34847542003733667,
+    "step": 560
   },
   {
+    "eval_loss": 1.572365403175354,
+    "eval_runtime": 14.0923,
+    "eval_samples_per_second": 37.254,
+    "eval_steps_per_second": 9.367,
+    "epoch": 0.34847542003733667,
     "step": 560
   },
   {
+    "loss": 1.5566,
+    "grad_norm": 4.780381679534912,
+    "learning_rate": 0.00015589240325424088,
+    "epoch": 0.35469819539514624,
     "step": 570
   },
   {
+    "loss": 1.5206,
+    "grad_norm": 4.883270740509033,
+    "learning_rate": 0.00015415173902002703,
+    "epoch": 0.3609209707529558,
     "step": 580
   },
   {
+    "loss": 1.3941,
+    "grad_norm": 6.177889823913574,
+    "learning_rate": 0.00015238754218539156,
+    "epoch": 0.3671437461107654,
     "step": 590
   },
   {
+    "loss": 1.4619,
+    "grad_norm": 4.226222038269043,
+    "learning_rate": 0.00015060057941341718,
+    "epoch": 0.37336652146857496,
     "step": 600
   },
   {
+    "loss": 1.3478,
+    "grad_norm": 5.7228102684021,
+    "learning_rate": 0.00014879162726052928,
+    "epoch": 0.3795892968263846,
     "step": 610
   },
   {
+    "loss": 1.5298,
+    "grad_norm": 4.754924774169922,
+    "learning_rate": 0.0001469614718390295,
+    "epoch": 0.38581207218419417,
     "step": 620
   },
   {
+    "loss": 1.4809,
+    "grad_norm": 5.708168983459473,
+    "learning_rate": 0.00014511090847547643,
+    "epoch": 0.39203484754200374,
     "step": 630
   },
   {
+    "loss": 1.5381,
+    "grad_norm": 5.013388633728027,
+    "learning_rate": 0.00014324074136506284,
+    "epoch": 0.3982576228998133,
+    "step": 640
   },
   {
+    "eval_loss": 1.5396426916122437,
+    "eval_runtime": 13.9405,
+    "eval_samples_per_second": 37.66,
+    "eval_steps_per_second": 9.469,
+    "epoch": 0.3982576228998133,
     "step": 640
   },
   {
+    "loss": 1.4894,
+    "grad_norm": 5.063615322113037,
+    "learning_rate": 0.00014135178322213765,
+    "epoch": 0.4044803982576229,
     "step": 650
   },
   {
+    "loss": 1.5547,
+    "grad_norm": 4.9530744552612305,
+    "learning_rate": 0.00013944485492702716,
+    "epoch": 0.41070317361543246,
     "step": 660
   },
   {
+    "loss": 1.4292,
+    "grad_norm": 4.819840908050537,
+    "learning_rate": 0.00013752078516930652,
+    "epoch": 0.4169259489732421,
     "step": 670
   },
   {
+    "loss": 1.556,
+    "grad_norm": 4.666694641113281,
+    "learning_rate": 0.00013558041008767798,
+    "epoch": 0.42314872433105166,
     "step": 680
   },
   {
+    "loss": 1.5216,
+    "grad_norm": 5.643951416015625,
+    "learning_rate": 0.00013362457290661215,
+    "epoch": 0.42937149968886124,
     "step": 690
   },
   {
+    "loss": 1.5768,
+    "grad_norm": 6.549987316131592,
+    "learning_rate": 0.00013165412356990955,
+    "epoch": 0.4355942750466708,
     "step": 700
   },
   {
+    "loss": 1.4482,
+    "grad_norm": 5.201975345611572,
+    "learning_rate": 0.0001296699183713427,
+    "epoch": 0.4418170504044804,
     "step": 710
   },
   {
+    "loss": 1.4855,
+    "grad_norm": 4.325445652008057,
+    "learning_rate": 0.0001276728195825383,
+    "epoch": 0.44803982576228996,
+    "step": 720
   },
   {
+    "eval_loss": 1.5151004791259766,
+    "eval_runtime": 13.9174,
+    "eval_samples_per_second": 37.723,
+    "eval_steps_per_second": 9.485,
+    "epoch": 0.44803982576228996,
     "step": 720
   },
   {
+    "loss": 1.4242,
+    "grad_norm": 6.4271931648254395,
+    "learning_rate": 0.00012566369507826175,
+    "epoch": 0.4542626011200996,
     "step": 730
   },
   {
+    "loss": 1.544,
+    "grad_norm": 4.974382400512695,
+    "learning_rate": 0.00012364341795926683,
+    "epoch": 0.46048537647790916,
     "step": 740
   },
   {
+    "loss": 1.3544,
+    "grad_norm": 4.475541114807129,
+    "learning_rate": 0.00012161286617287419,
+    "epoch": 0.46670815183571873,
     "step": 750
   },
   {
+    "loss": 1.5565,
+    "grad_norm": 4.763009548187256,
+    "learning_rate": 0.00011957292213144385,
+    "epoch": 0.4729309271935283,
     "step": 760
   },
   {
+    "loss": 1.4961,
+    "grad_norm": 5.890961647033691,
+    "learning_rate": 0.00011752447232890702,
+    "epoch": 0.4791537025513379,
     "step": 770
   },
   {
+    "loss": 1.4762,
+    "grad_norm": 5.207292556762695,
+    "learning_rate": 0.00011546840695552466,
+    "epoch": 0.48537647790914745,
     "step": 780
   },
   {
+    "loss": 1.5207,
+    "grad_norm": 4.962773323059082,
+    "learning_rate": 0.0001134056195110393,
+    "epoch": 0.4915992532669571,
     "step": 790
   },
   {
+    "loss": 1.3912,
+    "grad_norm": 4.377303123474121,
+    "learning_rate": 0.00011133700641638891,
+    "epoch": 0.49782202862476665,
+    "step": 800
   },
   {
+    "eval_loss": 1.4861326217651367,
+    "eval_runtime": 13.9515,
+    "eval_samples_per_second": 37.63,
+    "eval_steps_per_second": 9.461,
+    "epoch": 0.49782202862476665,
     "step": 800
   },
   {
+    "loss": 1.3795,
+    "grad_norm": 4.376067161560059,
+    "learning_rate": 0.0001092634666241513,
+    "epoch": 0.5040448039825762,
     "step": 810
   },
   {
+    "loss": 1.4531,
+    "grad_norm": 4.802002429962158,
+    "learning_rate": 0.00010718590122788821,
+    "epoch": 0.5102675793403858,
     "step": 820
   },
   {
+    "loss": 1.435,
+    "grad_norm": 4.865792751312256,
+    "learning_rate": 0.00010510521307055914,
+    "epoch": 0.5164903546981954,
     "step": 830
   },
   {
+    "loss": 1.4412,
+    "grad_norm": 5.106067180633545,
+    "learning_rate": 0.000103022306352175,
+    "epoch": 0.522713130056005,
     "step": 840
   },
   {
+    "loss": 1.398,
+    "grad_norm": 4.740355491638184,
+    "learning_rate": 0.00010093808623686165,
+    "epoch": 0.5289359054138145,
     "step": 850
   },
   {
+    "loss": 1.5138,
+    "grad_norm": 4.870340347290039,
+    "learning_rate": 9.88534584595051e-05,
+    "epoch": 0.5351586807716241,
     "step": 860
   },
   {
+    "loss": 1.4865,
+    "grad_norm": 4.641928195953369,
+    "learning_rate": 9.676932893214805e-05,
+    "epoch": 0.5413814561294338,
+    "step": 870
   },
   {
+    "loss": 1.3906,
+    "grad_norm": 6.179795742034912,
+    "learning_rate": 9.46866033503098e-05,
+    "epoch": 0.5476042314872434,
+    "step": 880
   },
   {
+    "eval_loss": 1.473583459854126,
+    "eval_runtime": 13.9209,
+    "eval_samples_per_second": 37.713,
+    "eval_steps_per_second": 9.482,
+    "epoch": 0.5476042314872434,
     "step": 880
   },
   {
+    "loss": 1.5219,
+    "grad_norm": 5.138612270355225,
+    "learning_rate": 9.260618679940025e-05,
+    "epoch": 0.5538270068450529,
     "step": 890
   },
   {
+    "loss": 1.4397,
+    "grad_norm": 4.696632385253906,
+    "learning_rate": 9.05289833613988e-05,
+    "epoch": 0.5600497822028625,
     "step": 900
   },
   {
+    "loss": 1.5194,
+    "grad_norm": 5.26527738571167,
+    "learning_rate": 8.845589572196961e-05,
+    "epoch": 0.5662725575606721,
     "step": 910
   },
   {
+    "loss": 1.388,
+    "grad_norm": 4.302639961242676,
+    "learning_rate": 8.638782477818334e-05,
+    "epoch": 0.5724953329184816,
     "step": 920
   },
   {
+    "loss": 1.3796,
+    "grad_norm": 5.462881565093994,
+    "learning_rate": 8.432566924701659e-05,
+    "epoch": 0.5787181082762912,
     "step": 930
   },
   {
+    "loss": 1.4552,
+    "grad_norm": 5.315045356750488,
+    "learning_rate": 8.227032527479806e-05,
+    "epoch": 0.5849408836341008,
     "step": 940
   },
   {
+    "loss": 1.3473,
+    "grad_norm": 4.484452247619629,
+    "learning_rate": 8.022268604777271e-05,
+    "epoch": 0.5911636589919104,
+    "step": 950
   },
   {
+    "loss": 1.4053,
+    "grad_norm": 4.7990312576293945,
+    "learning_rate": 7.818364140395137e-05,
+    "epoch": 0.5973864343497199,
+    "step": 960
   },
   {
+    "eval_loss": 1.4610518217086792,
+    "eval_runtime": 14.0919,
+    "eval_samples_per_second": 37.255,
+    "eval_steps_per_second": 9.367,
+    "epoch": 0.5973864343497199,
     "step": 960
   },
   {
+    "loss": 1.4184,
+    "grad_norm": 5.96822452545166,
+    "learning_rate": 7.615407744641619e-05,
+    "epoch": 0.6036092097075295,
     "step": 970
   },
   {
+    "loss": 1.495,
+    "grad_norm": 4.883596420288086,
+    "learning_rate": 7.413487615824847e-05,
+    "epoch": 0.6098319850653391,
     "step": 980
   },
   {
+    "loss": 1.4563,
+    "grad_norm": 4.199883460998535,
+    "learning_rate": 7.212691501924753e-05,
+    "epoch": 0.6160547604231488,
     "step": 990
   },
   {
+    "loss": 1.4791,
+    "grad_norm": 4.85598611831665,
+    "learning_rate": 7.013106662460604e-05,
+    "epoch": 0.6222775357809583,
     "step": 1000
   },
   {
+    "loss": 1.4621,
+    "grad_norm": 5.302894115447998,
+    "learning_rate": 6.81481983057085e-05,
+    "epoch": 0.6285003111387679,
     "step": 1010
   },
   {
+    "loss": 1.3599,
+    "grad_norm": 4.943319797515869,
+    "learning_rate": 6.617917175321669e-05,
+    "epoch": 0.6347230864965775,
     "step": 1020
   },
   {
+    "loss": 1.2883,
+    "grad_norm": 6.421422958374023,
+    "learning_rate": 6.422484264260698e-05,
+    "epoch": 0.6409458618543871,
+    "step": 1030
   },
   {
+    "loss": 1.392,
+    "grad_norm": 4.642984867095947,
+    "learning_rate": 6.228606026232118e-05,
+    "epoch": 0.6471686372121966,
+    "step": 1040
   },
   {
+    "eval_loss": 1.450961709022522,
+    "eval_runtime": 13.9604,
+    "eval_samples_per_second": 37.606,
+    "eval_steps_per_second": 9.455,
+    "epoch": 0.6471686372121966,
     "step": 1040
   },
   {
+    "loss": 1.3627,
+    "grad_norm": 4.882229328155518,
+    "learning_rate": 6.0363667144693105e-05,
+    "epoch": 0.6533914125700062,
     "step": 1050
   },
   {
+    "loss": 1.4695,
+    "grad_norm": 4.268496990203857,
+    "learning_rate": 5.845849869981137e-05,
+    "epoch": 0.6596141879278158,
     "step": 1060
   },
   {
+    "loss": 1.2919,
+    "grad_norm": 4.607876777648926,
+    "learning_rate": 5.657138285247687e-05,
+    "epoch": 0.6658369632856254,
     "step": 1070
   },
   {
+    "loss": 1.3095,
+    "grad_norm": 5.3004937171936035,
+    "learning_rate": 5.4703139682413586e-05,
+    "epoch": 0.6720597386434349,
     "step": 1080
   },
   {
+    "loss": 1.4059,
+    "grad_norm": 4.090771675109863,
+    "learning_rate": 5.285458106788807e-05,
+    "epoch": 0.6782825140012445,
     "step": 1090
   },
   {
+    "loss": 1.2986,
+    "grad_norm": 4.546812057495117,
+    "learning_rate": 5.10265103328937e-05,
+    "epoch": 0.6845052893590542,
     "step": 1100
   },
   {
+    "loss": 1.3874,
+    "grad_norm": 4.813508987426758,
+    "learning_rate": 4.921972189805154e-05,
+    "epoch": 0.6907280647168638,
+    "step": 1110
   },
   {
+    "loss": 1.5213,
+    "grad_norm": 4.514693737030029,
+    "learning_rate": 4.7435000935381115e-05,
+    "epoch": 0.6969508400746733,
+    "step": 1120
   },
   {
+    "eval_loss": 1.4407389163970947,
+    "eval_runtime": 14.4277,
+    "eval_samples_per_second": 36.388,
+    "eval_steps_per_second": 9.149,
+    "epoch": 0.6969508400746733,
     "step": 1120
   },
   {
+    "loss": 1.3087,
+    "grad_norm": 4.339325428009033,
+    "learning_rate": 4.567312302708965e-05,
+    "epoch": 0.7031736154324829,
     "step": 1130
   },
   {
+    "loss": 1.5258,
+    "grad_norm": 7.012087821960449,
+    "learning_rate": 4.393485382852935e-05,
+    "epoch": 0.7093963907902925,
     "step": 1140
   },
   {
+    "loss": 1.2682,
+    "grad_norm": 4.673589706420898,
+    "learning_rate": 4.2220948735467967e-05,
+    "epoch": 0.7156191661481021,
     "step": 1150
   },
   {
+    "loss": 1.3627,
+    "grad_norm": 5.091920852661133,
+    "learning_rate": 4.053215255581844e-05,
+    "epoch": 0.7218419415059116,
     "step": 1160
   },
   {
+    "loss": 1.5111,
+    "grad_norm": 4.602088928222656,
+    "learning_rate": 3.886919918596894e-05,
+    "epoch": 0.7280647168637212,
     "step": 1170
   },
   {
+    "loss": 1.3743,
+    "grad_norm": 4.942876815795898,
+    "learning_rate": 3.723281129185574e-05,
+    "epoch": 0.7342874922215308,
     "step": 1180
   },
   {
+    "loss": 1.3959,
+    "grad_norm": 4.139978885650635,
+    "learning_rate": 3.562369999491536e-05,
+    "epoch": 0.7405102675793404,
+    "step": 1190
   },
   {
+    "loss": 1.4383,
+    "grad_norm": 5.013376235961914,
+    "learning_rate": 3.4042564563054526e-05,
+    "epoch": 0.7467330429371499,
+    "step": 1200
   },
   {
+    "eval_loss": 1.4313360452651978,
+    "eval_runtime": 14.3246,
+    "eval_samples_per_second": 36.65,
+    "eval_steps_per_second": 9.215,
+    "epoch": 0.7467330429371499,
     "step": 1200
   },
   {
+    "loss": 1.3621,
+    "grad_norm": 5.042903900146484,
+    "learning_rate": 3.249009210677054e-05,
+    "epoch": 0.7529558182949595,
     "step": 1210
   },
   {
+    "loss": 1.4528,
+    "grad_norm": 4.191718101501465,
+    "learning_rate": 3.096695728055536e-05,
+    "epoch": 0.7591785936527692,
     "step": 1220
   },
   {
+    "loss": 1.326,
+    "grad_norm": 4.906574249267578,
+    "learning_rate": 2.9473821989712625e-05,
+    "epoch": 0.7654013690105788,
     "step": 1230
   },
   {
+    "loss": 1.3792,
+    "grad_norm": 5.21110200881958,
+    "learning_rate": 2.801133510271463e-05,
+    "epoch": 0.7716241443683883,
     "step": 1240
   },
   {
+    "loss": 1.3634,
+    "grad_norm": 5.0998430252075195,
+    "learning_rate": 2.6580132169225335e-05,
+    "epoch": 0.7778469197261979,
     "step": 1250
   },
   {
+    "loss": 1.378,
+    "grad_norm": 5.857067108154297,
+    "learning_rate": 2.5180835143910732e-05,
+    "epoch": 0.7840696950840075,
     "step": 1260
   },
   {
+    "loss": 1.2895,
+    "grad_norm": 4.248624801635742,
+    "learning_rate": 2.3814052116157492e-05,
+    "epoch": 0.790292470441817,
+    "step": 1270
   },
   {
+    "loss": 1.3369,
+    "grad_norm": 4.634941577911377,
+    "learning_rate": 2.248037704581686e-05,
+    "epoch": 0.7965152457996266,
+    "step": 1280
   },
   {
+    "eval_loss": 1.4184999465942383,
+    "eval_runtime": 14.0409,
+    "eval_samples_per_second": 37.391,
+    "eval_steps_per_second": 9.401,
+    "epoch": 0.7965152457996266,
     "step": 1280
   },
   {
+    "loss": 1.2569,
+    "grad_norm": 4.819410800933838,
+    "learning_rate": 2.1180389505089004e-05,
+    "epoch": 0.8027380211574362,
     "step": 1290
   },
   {
+    "loss": 1.2557,
+    "grad_norm": 4.907857894897461,
+    "learning_rate": 1.9914654426659374e-05,
+    "epoch": 0.8089607965152458,
     "step": 1300
   },
   {
+    "loss": 1.4703,
+    "grad_norm": 4.4456000328063965,
+    "learning_rate": 1.8683721858197366e-05,
+    "epoch": 0.8151835718730553,
     "step": 1310
   },
   {
+    "loss": 1.359,
+    "grad_norm": 5.10971736907959,
+    "learning_rate": 1.7488126723323183e-05,
+    "epoch": 0.8214063472308649,
     "step": 1320
   },
   {
+    "loss": 1.3467,
+    "grad_norm": 4.865292549133301,
+    "learning_rate": 1.632838858914747e-05,
+    "epoch": 0.8276291225886746,
     "step": 1330
   },
   {
+    "loss": 1.41,
+    "grad_norm": 4.803621292114258,
+    "learning_rate": 1.5205011440483929e-05,
+    "epoch": 0.8338518979464842,
     "step": 1340
   },
   {
+    "loss": 1.4062,
+    "grad_norm": 4.7581377029418945,
+    "learning_rate": 1.4118483460834064e-05,
+    "epoch": 0.8400746733042938,
+    "step": 1350
   },
   {
+    "loss": 1.375,
+    "grad_norm": 5.33804988861084,
+    "learning_rate": 1.3069276820237997e-05,
+    "epoch": 0.8462974486621033,
+    "step": 1360
   },
   {
+    "eval_loss": 1.4135597944259644,
+    "eval_runtime": 14.1252,
+    "eval_samples_per_second": 37.168,
+    "eval_steps_per_second": 9.345,
+    "epoch": 0.8462974486621033,
     "step": 1360
   },
   {
+    "loss": 1.4474,
+    "grad_norm": 4.493839263916016,
+    "learning_rate": 1.2057847470084993e-05,
+    "epoch": 0.8525202240199129,
     "step": 1370
   },
   {
+    "loss": 1.3765,
+    "grad_norm": 3.7897987365722656,
+    "learning_rate": 1.108463494497135e-05,
+    "epoch": 0.8587429993777225,
     "step": 1380
   },
   {
+    "loss": 1.3583,
+    "grad_norm": 3.9571006298065186,
+    "learning_rate": 1.0150062171693076e-05,
+    "epoch": 0.864965774735532,
     "step": 1390
   },
   {
+    "loss": 1.4705,
+    "grad_norm": 4.610692501068115,
+    "learning_rate": 9.254535285455334e-06,
+    "epoch": 0.8711885500933416,
     "step": 1400
   },
   {
+    "loss": 1.3517,
+    "grad_norm": 4.550515651702881,
+    "learning_rate": 8.398443453379267e-06,
+    "epoch": 0.8774113254511512,
     "step": 1410
   },
   {
+    "loss": 1.2454,
+    "grad_norm": 4.898622512817383,
+    "learning_rate": 7.582158705382581e-06,
+    "epoch": 0.8836341008089608,
     "step": 1420
   },
   {
+    "loss": 1.3031,
+    "grad_norm": 4.423801422119141,
+    "learning_rate": 6.806035772507169e-06,
+    "epoch": 0.8898568761667703,
+    "step": 1430
   },
   {
+    "loss": 1.3421,
+    "grad_norm": 3.7221221923828125,
+    "learning_rate": 6.070411932764586e-06,
+    "epoch": 0.8960796515245799,
+    "step": 1440
   },
   {
+    "eval_loss": 1.4087599515914917,
+    "eval_runtime": 14.0409,
+    "eval_samples_per_second": 37.391,
+    "eval_steps_per_second": 9.401,
+    "epoch": 0.8960796515245799,
     "step": 1440
   },
   {
+    "loss": 1.3722,
+    "grad_norm": 5.566176414489746,
+    "learning_rate": 5.375606864565785e-06,
+    "epoch": 0.9023024268823896,
     "step": 1450
   },
   {
+    "loss": 1.4635,
+    "grad_norm": 5.002288818359375,
+    "learning_rate": 4.721922507799248e-06,
+    "epoch": 0.9085252022401992,
     "step": 1460
   },
   {
+    "loss": 1.2757,
+    "grad_norm": 3.9484496116638184,
+    "learning_rate": 4.10964293261763e-06,
+    "epoch": 0.9147479775980087,
     "step": 1470
   },
   {
+    "loss": 1.3995,
+    "grad_norm": 5.582644939422607,
+    "learning_rate": 3.5390342159900223e-06,
+    "epoch": 0.9209707529558183,
     "step": 1480
   },
   {
+    "loss": 1.3435,
+    "grad_norm": 4.107953071594238,
+    "learning_rate": 3.0103443260734554e-06,
+    "epoch": 0.9271935283136279,
     "step": 1490
   },
   {
+    "loss": 1.4009,
+    "grad_norm": 4.991889953613281,
+    "learning_rate": 2.5238030144539737e-06,
+    "epoch": 0.9334163036714375,
     "step": 1500
   },
   {
+    "loss": 1.4222,
+    "grad_norm": 4.622715473175049,
+    "learning_rate": 2.079621716303959e-06,
+    "epoch": 0.939639079029247,
+    "step": 1510
   },
   {
+    "loss": 1.3094,
+    "grad_norm": 3.724646806716919,
+    "learning_rate": 1.6779934584992718e-06,
+    "epoch": 0.9458618543870566,
+    "step": 1520
   },
   {
+    "eval_loss": 1.408400058746338,
+    "eval_runtime": 14.2644,
+    "eval_samples_per_second": 36.805,
+    "eval_steps_per_second": 9.254,
+    "epoch": 0.9458618543870566,
     "step": 1520
   },
   {
+    "loss": 1.2012,
+    "grad_norm": 5.017258644104004,
+    "learning_rate": 1.3190927757358973e-06,
+    "epoch": 0.9520846297448662,
     "step": 1530
   },
   {
+    "loss": 1.4522,
+    "grad_norm": 5.213080883026123,
+    "learning_rate": 1.0030756346829151e-06,
+    "epoch": 0.9583074051026758,
     "step": 1540
   },
   {
+    "loss": 1.4536,
+    "grad_norm": 4.179771900177002,
+    "learning_rate": 7.300793662043282e-07,
+    "epoch": 0.9645301804604853,
     "step": 1550
   },
   {
+    "loss": 1.3803,
+    "grad_norm": 4.139328479766846,
+    "learning_rate": 5.002226056795123e-07,
+    "epoch": 0.9707529558182949,
     "step": 1560
   },
   {
+    "loss": 1.4377,
+    "grad_norm": 5.55445671081543,
+    "learning_rate": 3.1360524144810055e-07,
+    "epoch": 0.9769757311761046,
     "step": 1570
   },
   {
+    "loss": 1.3551,
+    "grad_norm": 6.75304651260376,
+    "learning_rate": 1.703083714017617e-07,
+    "epoch": 0.9831985065339142,
     "step": 1580
   },
   {
+    "loss": 1.4783,
+    "grad_norm": 5.284483432769775,
+    "learning_rate": 7.039426774164693e-08,
+    "epoch": 0.9894212818917237,
+    "step": 1590
+  },
+  {
+    "loss": 1.4281,
+    "grad_norm": 5.420683860778809,
+    "learning_rate": 1.3906349916881222e-08,
+    "epoch": 0.9956440572495333,
+    "step": 1600
+  },
+  {
+    "eval_loss": 1.4093519449234009,
+    "eval_runtime": 14.2389,
+    "eval_samples_per_second": 36.871,
+    "eval_steps_per_second": 9.27,
+    "epoch": 0.9956440572495333,
+    "step": 1600
   },
   {
+    "train_runtime": 1590.5658,
+    "train_samples_per_second": 16.162,
+    "train_steps_per_second": 1.01,
+    "total_flos": 1.172369019552e+16,
+    "train_loss": 1.5245253444830675,
     "epoch": 1.0,
+    "step": 1607
   }
 ]

train/training_loss.png CHANGED Viewed

train/validation_loss.png CHANGED Viewed