Instructions to use adpretko/train-riscv-O2_epoch3_AMD with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use adpretko/train-riscv-O2_epoch3_AMD with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="adpretko/train-riscv-O2_epoch3_AMD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("adpretko/train-riscv-O2_epoch3_AMD")
model = AutoModelForCausalLM.from_pretrained("adpretko/train-riscv-O2_epoch3_AMD")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use adpretko/train-riscv-O2_epoch3_AMD with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "adpretko/train-riscv-O2_epoch3_AMD"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adpretko/train-riscv-O2_epoch3_AMD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/adpretko/train-riscv-O2_epoch3_AMD

SGLang

How to use adpretko/train-riscv-O2_epoch3_AMD with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "adpretko/train-riscv-O2_epoch3_AMD" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adpretko/train-riscv-O2_epoch3_AMD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "adpretko/train-riscv-O2_epoch3_AMD" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "adpretko/train-riscv-O2_epoch3_AMD",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use adpretko/train-riscv-O2_epoch3_AMD with Docker Model Runner:
```
docker model run hf.co/adpretko/train-riscv-O2_epoch3_AMD
```

adpretko commited on Nov 6, 2025

Commit

0285a83

verified ·

1 Parent(s): fec6877

End of training

Browse files

Files changed (5) hide show

README.md +2 -1
all_results.json +8 -0
train_results.json +8 -0
trainer_state.json +2759 -0
training_loss.png +0 -0

README.md CHANGED Viewed

@@ -3,6 +3,7 @@ library_name: transformers
 base_model: adpretko/train-riscv-O2_epoch1and2
 tags:
 - llama-factory
 - generated_from_trainer
 model-index:
 - name: train-riscv-O2_epoch3_AMD
@@ -14,7 +15,7 @@ should probably proofread and complete it, then remove this comment. -->
 # train-riscv-O2_epoch3_AMD
-This model is a fine-tuned version of [adpretko/train-riscv-O2_epoch1and2](https://huggingface.co/adpretko/train-riscv-O2_epoch1and2) on the None dataset.
 ## Model description

 base_model: adpretko/train-riscv-O2_epoch1and2
 tags:
 - llama-factory
+- full
 - generated_from_trainer
 model-index:
 - name: train-riscv-O2_epoch3_AMD
 # train-riscv-O2_epoch3_AMD
+This model is a fine-tuned version of [adpretko/train-riscv-O2_epoch1and2](https://huggingface.co/adpretko/train-riscv-O2_epoch1and2) on the AnghaBench-risc-o2-full_part_00, the AnghaBench-risc-o2-full_part_01, the AnghaBench-risc-o2-full_part_02, the AnghaBench-risc-o2-full_part_03, the AnghaBench-risc-o2-full_part_04, the AnghaBench-risc-o2-full_part_05, the AnghaBench-risc-o2-full_part_06, the AnghaBench-risc-o2-full_part_07, the AnghaBench-risc-o2-full_part_08, the AnghaBench-risc-o2-full_part_09, the AnghaBench-risc-o2-full_part_10, the AnghaBench-risc-o2-full_part_11, the AnghaBench-risc-o2-full_part_12, the AnghaBench-risc-o2-full_part_13, the AnghaBench-risc-o2-full_part_14, the AnghaBench-risc-o2-full_part_15, the AnghaBench-risc-o2-full_part_16, the AnghaBench-risc-o2-full_part_17, the AnghaBench-risc-o2-full_part_18, the AnghaBench-risc-o2-full_part_19, the AnghaBench-risc-o2-full_part_20, the AnghaBench-risc-o2-full_part_21, the AnghaBench-risc-o2-full_part_22, the AnghaBench-risc-o2-full_part_23, the AnghaBench-risc-o2-full_part_24, the AnghaBench-risc-o2-full_part_25, the AnghaBench-risc-o2-full_part_26, the AnghaBench-risc-o2-full_part_27, the AnghaBench-risc-o2-full_part_28, the AnghaBench-risc-o2-full_part_29, the AnghaBench-risc-o2-full_part_30, the AnghaBench-risc-o2-full_part_31, the AnghaBench-risc-o2-full_part_32, the AnghaBench-risc-o2-full_part_33, the AnghaBench-risc-o2-full_part_34, the AnghaBench-risc-o2-full_part_35, the AnghaBench-risc-o2-full_part_36, the AnghaBench-risc-o2-full_part_37, the AnghaBench-risc-o2-full_part_38, the AnghaBench-risc-o2-full_part_39, the AnghaBench-risc-o2-full_part_40, the AnghaBench-risc-o2-full_part_41, the AnghaBench-risc-o2-full_part_42, the AnghaBench-risc-o2-full_part_43, the AnghaBench-risc-o2-full_part_44, the AnghaBench-risc-o2-full_part_45, the AnghaBench-risc-o2-full_part_46, the AnghaBench-risc-o2-full_part_47, the AnghaBench-risc-o2-full_part_48 and the AnghaBench-risc-o2-full_part_49 datasets.
 ## Model description

all_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "epoch": 2.0,
+    "total_flos": 6.320832928623572e+19,
+    "train_loss": 0.009204940595273436,
+    "train_runtime": 219891.592,
+    "train_samples_per_second": 9.047,
+    "train_steps_per_second": 0.018
+}

train_results.json ADDED Viewed

	@@ -0,0 +1,8 @@

+{
+    "epoch": 2.0,
+    "total_flos": 6.320832928623572e+19,
+    "train_loss": 0.009204940595273436,
+    "train_runtime": 219891.592,
+    "train_samples_per_second": 9.047,
+    "train_steps_per_second": 0.018
+}

trainer_state.json ADDED Viewed

	@@ -0,0 +1,2759 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.0,
+  "eval_steps": 500,
+  "global_step": 3886,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 0.0051473426843392095,
+      "grad_norm": 0.05525032430887222,
+      "learning_rate": 4.6272493573264783e-07,
+      "loss": 0.0106,
+      "step": 10
+    },
+    {
+      "epoch": 0.010294685368678419,
+      "grad_norm": 0.05362959951162338,
+      "learning_rate": 9.768637532133676e-07,
+      "loss": 0.0106,
+      "step": 20
+    },
+    {
+      "epoch": 0.01544202805301763,
+      "grad_norm": 0.05603431165218353,
+      "learning_rate": 1.4910025706940876e-06,
+      "loss": 0.0105,
+      "step": 30
+    },
+    {
+      "epoch": 0.020589370737356838,
+      "grad_norm": 0.0636371374130249,
+      "learning_rate": 2.0051413881748076e-06,
+      "loss": 0.0113,
+      "step": 40
+    },
+    {
+      "epoch": 0.02573671342169605,
+      "grad_norm": 0.06462942063808441,
+      "learning_rate": 2.519280205655527e-06,
+      "loss": 0.0102,
+      "step": 50
+    },
+    {
+      "epoch": 0.03088405610603526,
+      "grad_norm": 0.0556134358048439,
+      "learning_rate": 3.033419023136247e-06,
+      "loss": 0.0111,
+      "step": 60
+    },
+    {
+      "epoch": 0.036031398790374466,
+      "grad_norm": 0.057751912623643875,
+      "learning_rate": 3.547557840616967e-06,
+      "loss": 0.0109,
+      "step": 70
+    },
+    {
+      "epoch": 0.041178741474713676,
+      "grad_norm": 0.059322893619537354,
+      "learning_rate": 4.0616966580976866e-06,
+      "loss": 0.0112,
+      "step": 80
+    },
+    {
+      "epoch": 0.04632608415905289,
+      "grad_norm": 0.05676131322979927,
+      "learning_rate": 4.575835475578407e-06,
+      "loss": 0.0113,
+      "step": 90
+    },
+    {
+      "epoch": 0.0514734268433921,
+      "grad_norm": 0.06342212855815887,
+      "learning_rate": 5.0899742930591265e-06,
+      "loss": 0.0115,
+      "step": 100
+    },
+    {
+      "epoch": 0.05662076952773131,
+      "grad_norm": 0.05883905664086342,
+      "learning_rate": 5.604113110539846e-06,
+      "loss": 0.011,
+      "step": 110
+    },
+    {
+      "epoch": 0.06176811221207052,
+      "grad_norm": 0.06119426712393761,
+      "learning_rate": 6.1182519280205664e-06,
+      "loss": 0.0112,
+      "step": 120
+    },
+    {
+      "epoch": 0.06691545489640972,
+      "grad_norm": 0.06628178805112839,
+      "learning_rate": 6.632390745501286e-06,
+      "loss": 0.0114,
+      "step": 130
+    },
+    {
+      "epoch": 0.07206279758074893,
+      "grad_norm": 0.06540685147047043,
+      "learning_rate": 7.1465295629820055e-06,
+      "loss": 0.0116,
+      "step": 140
+    },
+    {
+      "epoch": 0.07721014026508814,
+      "grad_norm": 0.06296146661043167,
+      "learning_rate": 7.660668380462726e-06,
+      "loss": 0.0112,
+      "step": 150
+    },
+    {
+      "epoch": 0.08235748294942735,
+      "grad_norm": 0.06895332783460617,
+      "learning_rate": 8.174807197943445e-06,
+      "loss": 0.0115,
+      "step": 160
+    },
+    {
+      "epoch": 0.08750482563376656,
+      "grad_norm": 0.08407478034496307,
+      "learning_rate": 8.688946015424165e-06,
+      "loss": 0.0126,
+      "step": 170
+    },
+    {
+      "epoch": 0.09265216831810578,
+      "grad_norm": 0.06299558281898499,
+      "learning_rate": 9.203084832904885e-06,
+      "loss": 0.0116,
+      "step": 180
+    },
+    {
+      "epoch": 0.097799511002445,
+      "grad_norm": 0.06793331354856491,
+      "learning_rate": 9.717223650385606e-06,
+      "loss": 0.0115,
+      "step": 190
+    },
+    {
+      "epoch": 0.1029468536867842,
+      "grad_norm": 0.07241130620241165,
+      "learning_rate": 1.0231362467866324e-05,
+      "loss": 0.0114,
+      "step": 200
+    },
+    {
+      "epoch": 0.10809419637112341,
+      "grad_norm": 0.0763164609670639,
+      "learning_rate": 1.0745501285347045e-05,
+      "loss": 0.0119,
+      "step": 210
+    },
+    {
+      "epoch": 0.11324153905546262,
+      "grad_norm": 0.06727232038974762,
+      "learning_rate": 1.1259640102827764e-05,
+      "loss": 0.0116,
+      "step": 220
+    },
+    {
+      "epoch": 0.11838888173980183,
+      "grad_norm": 0.09049773216247559,
+      "learning_rate": 1.1773778920308486e-05,
+      "loss": 0.0122,
+      "step": 230
+    },
+    {
+      "epoch": 0.12353622442414104,
+      "grad_norm": 0.07317323982715607,
+      "learning_rate": 1.2287917737789203e-05,
+      "loss": 0.012,
+      "step": 240
+    },
+    {
+      "epoch": 0.12868356710848025,
+      "grad_norm": 0.07453487813472748,
+      "learning_rate": 1.2802056555269925e-05,
+      "loss": 0.0118,
+      "step": 250
+    },
+    {
+      "epoch": 0.13383090979281945,
+      "grad_norm": 0.06471797078847885,
+      "learning_rate": 1.3316195372750644e-05,
+      "loss": 0.0114,
+      "step": 260
+    },
+    {
+      "epoch": 0.13897825247715867,
+      "grad_norm": 0.08005302399396896,
+      "learning_rate": 1.3830334190231362e-05,
+      "loss": 0.012,
+      "step": 270
+    },
+    {
+      "epoch": 0.14412559516149787,
+      "grad_norm": 0.07167572528123856,
+      "learning_rate": 1.4344473007712083e-05,
+      "loss": 0.0119,
+      "step": 280
+    },
+    {
+      "epoch": 0.1492729378458371,
+      "grad_norm": 0.08343254774808884,
+      "learning_rate": 1.4858611825192803e-05,
+      "loss": 0.0126,
+      "step": 290
+    },
+    {
+      "epoch": 0.15442028053017628,
+      "grad_norm": 0.07320424169301987,
+      "learning_rate": 1.5372750642673522e-05,
+      "loss": 0.0123,
+      "step": 300
+    },
+    {
+      "epoch": 0.1595676232145155,
+      "grad_norm": 0.07763416320085526,
+      "learning_rate": 1.5886889460154242e-05,
+      "loss": 0.0122,
+      "step": 310
+    },
+    {
+      "epoch": 0.1647149658988547,
+      "grad_norm": 0.07546375691890717,
+      "learning_rate": 1.640102827763496e-05,
+      "loss": 0.0128,
+      "step": 320
+    },
+    {
+      "epoch": 0.16986230858319393,
+      "grad_norm": 0.07282799482345581,
+      "learning_rate": 1.6915167095115684e-05,
+      "loss": 0.0129,
+      "step": 330
+    },
+    {
+      "epoch": 0.17500965126753312,
+      "grad_norm": 0.07360665500164032,
+      "learning_rate": 1.74293059125964e-05,
+      "loss": 0.0123,
+      "step": 340
+    },
+    {
+      "epoch": 0.18015699395187235,
+      "grad_norm": 0.07590780407190323,
+      "learning_rate": 1.7943444730077123e-05,
+      "loss": 0.0128,
+      "step": 350
+    },
+    {
+      "epoch": 0.18530433663621157,
+      "grad_norm": 0.06358295679092407,
+      "learning_rate": 1.8457583547557843e-05,
+      "loss": 0.0124,
+      "step": 360
+    },
+    {
+      "epoch": 0.19045167932055077,
+      "grad_norm": 0.10541003942489624,
+      "learning_rate": 1.8971722365038563e-05,
+      "loss": 0.013,
+      "step": 370
+    },
+    {
+      "epoch": 0.19559902200489,
+      "grad_norm": 0.06901050359010696,
+      "learning_rate": 1.9485861182519282e-05,
+      "loss": 0.0129,
+      "step": 380
+    },
+    {
+      "epoch": 0.20074636468922918,
+      "grad_norm": 0.0681707039475441,
+      "learning_rate": 2e-05,
+      "loss": 0.0131,
+      "step": 390
+    },
+    {
+      "epoch": 0.2058937073735684,
+      "grad_norm": 0.07603519409894943,
+      "learning_rate": 1.999959647024453e-05,
+      "loss": 0.0135,
+      "step": 400
+    },
+    {
+      "epoch": 0.2110410500579076,
+      "grad_norm": 0.06607502698898315,
+      "learning_rate": 1.999838591354537e-05,
+      "loss": 0.0125,
+      "step": 410
+    },
+    {
+      "epoch": 0.21618839274224683,
+      "grad_norm": 0.08145863562822342,
+      "learning_rate": 1.9996368427601652e-05,
+      "loss": 0.0129,
+      "step": 420
+    },
+    {
+      "epoch": 0.22133573542658602,
+      "grad_norm": 0.06165160983800888,
+      "learning_rate": 1.9993544175236498e-05,
+      "loss": 0.0124,
+      "step": 430
+    },
+    {
+      "epoch": 0.22648307811092525,
+      "grad_norm": 0.07120924443006516,
+      "learning_rate": 1.998991338438388e-05,
+      "loss": 0.013,
+      "step": 440
+    },
+    {
+      "epoch": 0.23163042079526444,
+      "grad_norm": 0.0669078603386879,
+      "learning_rate": 1.9985476348070224e-05,
+      "loss": 0.0121,
+      "step": 450
+    },
+    {
+      "epoch": 0.23677776347960366,
+      "grad_norm": 3.4496610164642334,
+      "learning_rate": 1.9980233424390773e-05,
+      "loss": 0.021,
+      "step": 460
+    },
+    {
+      "epoch": 0.24192510616394286,
+      "grad_norm": 0.11881168931722641,
+      "learning_rate": 1.9974185036480664e-05,
+      "loss": 0.0277,
+      "step": 470
+    },
+    {
+      "epoch": 0.24707244884828208,
+      "grad_norm": 0.068561851978302,
+      "learning_rate": 1.9967331672480798e-05,
+      "loss": 0.0164,
+      "step": 480
+    },
+    {
+      "epoch": 0.2522197915326213,
+      "grad_norm": 0.06290870159864426,
+      "learning_rate": 1.995967388549843e-05,
+      "loss": 0.0138,
+      "step": 490
+    },
+    {
+      "epoch": 0.2573671342169605,
+      "grad_norm": 0.06344795227050781,
+      "learning_rate": 1.9951212293562547e-05,
+      "loss": 0.0137,
+      "step": 500
+    },
+    {
+      "epoch": 0.2625144769012997,
+      "grad_norm": 0.06535464525222778,
+      "learning_rate": 1.994194757957397e-05,
+      "loss": 0.013,
+      "step": 510
+    },
+    {
+      "epoch": 0.2676618195856389,
+      "grad_norm": 0.06390328705310822,
+      "learning_rate": 1.9931880491250263e-05,
+      "loss": 0.0124,
+      "step": 520
+    },
+    {
+      "epoch": 0.27280916226997814,
+      "grad_norm": 0.06637264043092728,
+      "learning_rate": 1.992101184106535e-05,
+      "loss": 0.0125,
+      "step": 530
+    },
+    {
+      "epoch": 0.27795650495431734,
+      "grad_norm": 0.06743142008781433,
+      "learning_rate": 1.990934250618399e-05,
+      "loss": 0.0132,
+      "step": 540
+    },
+    {
+      "epoch": 0.28310384763865654,
+      "grad_norm": 0.053368110209703445,
+      "learning_rate": 1.989687342839095e-05,
+      "loss": 0.0117,
+      "step": 550
+    },
+    {
+      "epoch": 0.28825119032299573,
+      "grad_norm": 0.058148909360170364,
+      "learning_rate": 1.9883605614015014e-05,
+      "loss": 0.0121,
+      "step": 560
+    },
+    {
+      "epoch": 0.293398533007335,
+      "grad_norm": 0.060990240424871445,
+      "learning_rate": 1.986954013384776e-05,
+      "loss": 0.0116,
+      "step": 570
+    },
+    {
+      "epoch": 0.2985458756916742,
+      "grad_norm": 0.06001332029700279,
+      "learning_rate": 1.9854678123057144e-05,
+      "loss": 0.0122,
+      "step": 580
+    },
+    {
+      "epoch": 0.3036932183760134,
+      "grad_norm": 0.06884918361902237,
+      "learning_rate": 1.9839020781095873e-05,
+      "loss": 0.0116,
+      "step": 590
+    },
+    {
+      "epoch": 0.30884056106035257,
+      "grad_norm": 0.058082133531570435,
+      "learning_rate": 1.9822569371604637e-05,
+      "loss": 0.012,
+      "step": 600
+    },
+    {
+      "epoch": 0.3139879037446918,
+      "grad_norm": 0.053921736776828766,
+      "learning_rate": 1.9805325222310072e-05,
+      "loss": 0.0108,
+      "step": 610
+    },
+    {
+      "epoch": 0.319135246429031,
+      "grad_norm": 0.059762243181467056,
+      "learning_rate": 1.9787289724917657e-05,
+      "loss": 0.0117,
+      "step": 620
+    },
+    {
+      "epoch": 0.3242825891133702,
+      "grad_norm": 0.052163220942020416,
+      "learning_rate": 1.9768464334999352e-05,
+      "loss": 0.0108,
+      "step": 630
+    },
+    {
+      "epoch": 0.3294299317977094,
+      "grad_norm": 0.054961204528808594,
+      "learning_rate": 1.974885057187617e-05,
+      "loss": 0.0114,
+      "step": 640
+    },
+    {
+      "epoch": 0.33457727448204866,
+      "grad_norm": 0.06342387944459915,
+      "learning_rate": 1.9728450018495506e-05,
+      "loss": 0.0116,
+      "step": 650
+    },
+    {
+      "epoch": 0.33972461716638785,
+      "grad_norm": 0.05155491828918457,
+      "learning_rate": 1.9707264321303427e-05,
+      "loss": 0.0115,
+      "step": 660
+    },
+    {
+      "epoch": 0.34487195985072705,
+      "grad_norm": 0.05698103830218315,
+      "learning_rate": 1.9685295190111777e-05,
+      "loss": 0.0113,
+      "step": 670
+    },
+    {
+      "epoch": 0.35001930253506625,
+      "grad_norm": 0.06619592010974884,
+      "learning_rate": 1.9662544397960182e-05,
+      "loss": 0.0113,
+      "step": 680
+    },
+    {
+      "epoch": 0.3551666452194055,
+      "grad_norm": 0.055313605815172195,
+      "learning_rate": 1.9639013780972958e-05,
+      "loss": 0.0113,
+      "step": 690
+    },
+    {
+      "epoch": 0.3603139879037447,
+      "grad_norm": 0.06306542456150055,
+      "learning_rate": 1.961470523821093e-05,
+      "loss": 0.0115,
+      "step": 700
+    },
+    {
+      "epoch": 0.3654613305880839,
+      "grad_norm": 0.05840103700757027,
+      "learning_rate": 1.9589620731518166e-05,
+      "loss": 0.0111,
+      "step": 710
+    },
+    {
+      "epoch": 0.37060867327242314,
+      "grad_norm": 0.05483512952923775,
+      "learning_rate": 1.956376228536363e-05,
+      "loss": 0.0109,
+      "step": 720
+    },
+    {
+      "epoch": 0.37575601595676233,
+      "grad_norm": 0.05598577484488487,
+      "learning_rate": 1.953713198667782e-05,
+      "loss": 0.0116,
+      "step": 730
+    },
+    {
+      "epoch": 0.38090335864110153,
+      "grad_norm": 0.05302765220403671,
+      "learning_rate": 1.950973198468431e-05,
+      "loss": 0.0109,
+      "step": 740
+    },
+    {
+      "epoch": 0.3860507013254407,
+      "grad_norm": 0.05226908251643181,
+      "learning_rate": 1.9481564490726327e-05,
+      "loss": 0.0116,
+      "step": 750
+    },
+    {
+      "epoch": 0.39119804400978,
+      "grad_norm": 0.06026271730661392,
+      "learning_rate": 1.9452631778088262e-05,
+      "loss": 0.0109,
+      "step": 760
+    },
+    {
+      "epoch": 0.3963453866941192,
+      "grad_norm": 0.058383017778396606,
+      "learning_rate": 1.94229361818122e-05,
+      "loss": 0.011,
+      "step": 770
+    },
+    {
+      "epoch": 0.40149272937845837,
+      "grad_norm": 0.05417947843670845,
+      "learning_rate": 1.9392480098509488e-05,
+      "loss": 0.0111,
+      "step": 780
+    },
+    {
+      "epoch": 0.40664007206279756,
+      "grad_norm": 0.06370815634727478,
+      "learning_rate": 1.9361265986167292e-05,
+      "loss": 0.0112,
+      "step": 790
+    },
+    {
+      "epoch": 0.4117874147471368,
+      "grad_norm": 0.05550752207636833,
+      "learning_rate": 1.9329296363950237e-05,
+      "loss": 0.0111,
+      "step": 800
+    },
+    {
+      "epoch": 0.416934757431476,
+      "grad_norm": 0.054070789366960526,
+      "learning_rate": 1.929657381199709e-05,
+      "loss": 0.0113,
+      "step": 810
+    },
+    {
+      "epoch": 0.4220821001158152,
+      "grad_norm": 0.06153961271047592,
+      "learning_rate": 1.9263100971212533e-05,
+      "loss": 0.0115,
+      "step": 820
+    },
+    {
+      "epoch": 0.4272294428001544,
+      "grad_norm": 0.05265296623110771,
+      "learning_rate": 1.922888054305401e-05,
+      "loss": 0.0114,
+      "step": 830
+    },
+    {
+      "epoch": 0.43237678548449365,
+      "grad_norm": 0.051895588636398315,
+      "learning_rate": 1.9193915289313726e-05,
+      "loss": 0.011,
+      "step": 840
+    },
+    {
+      "epoch": 0.43752412816883285,
+      "grad_norm": 0.05822490155696869,
+      "learning_rate": 1.9158208031895738e-05,
+      "loss": 0.0108,
+      "step": 850
+    },
+    {
+      "epoch": 0.44267147085317204,
+      "grad_norm": 0.05402348190546036,
+      "learning_rate": 1.9121761652588217e-05,
+      "loss": 0.011,
+      "step": 860
+    },
+    {
+      "epoch": 0.44781881353751124,
+      "grad_norm": 0.05292493477463722,
+      "learning_rate": 1.9084579092830873e-05,
+      "loss": 0.0105,
+      "step": 870
+    },
+    {
+      "epoch": 0.4529661562218505,
+      "grad_norm": 0.052194658666849136,
+      "learning_rate": 1.904666335347755e-05,
+      "loss": 0.0097,
+      "step": 880
+    },
+    {
+      "epoch": 0.4581134989061897,
+      "grad_norm": 0.06219863146543503,
+      "learning_rate": 1.900801749455406e-05,
+      "loss": 0.0104,
+      "step": 890
+    },
+    {
+      "epoch": 0.4632608415905289,
+      "grad_norm": 0.0552273727953434,
+      "learning_rate": 1.8968644635011192e-05,
+      "loss": 0.0108,
+      "step": 900
+    },
+    {
+      "epoch": 0.4684081842748681,
+      "grad_norm": 0.05805448442697525,
+      "learning_rate": 1.8928547952473037e-05,
+      "loss": 0.0109,
+      "step": 910
+    },
+    {
+      "epoch": 0.47355552695920733,
+      "grad_norm": 0.05885449796915054,
+      "learning_rate": 1.8887730682980484e-05,
+      "loss": 0.0101,
+      "step": 920
+    },
+    {
+      "epoch": 0.4787028696435465,
+      "grad_norm": 0.06097600609064102,
+      "learning_rate": 1.8846196120730095e-05,
+      "loss": 0.011,
+      "step": 930
+    },
+    {
+      "epoch": 0.4838502123278857,
+      "grad_norm": 0.057253919541835785,
+      "learning_rate": 1.8803947617808217e-05,
+      "loss": 0.0113,
+      "step": 940
+    },
+    {
+      "epoch": 0.4889975550122249,
+      "grad_norm": 0.05612657591700554,
+      "learning_rate": 1.8760988583920457e-05,
+      "loss": 0.0105,
+      "step": 950
+    },
+    {
+      "epoch": 0.49414489769656417,
+      "grad_norm": 0.05309102311730385,
+      "learning_rate": 1.8717322486116513e-05,
+      "loss": 0.0103,
+      "step": 960
+    },
+    {
+      "epoch": 0.49929224038090336,
+      "grad_norm": 0.06416105479001999,
+      "learning_rate": 1.867295284851033e-05,
+      "loss": 0.0103,
+      "step": 970
+    },
+    {
+      "epoch": 0.5044395830652426,
+      "grad_norm": 0.060254644602537155,
+      "learning_rate": 1.8627883251995712e-05,
+      "loss": 0.0108,
+      "step": 980
+    },
+    {
+      "epoch": 0.5095869257495818,
+      "grad_norm": 0.057798177003860474,
+      "learning_rate": 1.858211733395731e-05,
+      "loss": 0.0098,
+      "step": 990
+    },
+    {
+      "epoch": 0.514734268433921,
+      "grad_norm": 0.05069541186094284,
+      "learning_rate": 1.8535658787977076e-05,
+      "loss": 0.0108,
+      "step": 1000
+    },
+    {
+      "epoch": 0.5198816111182601,
+      "grad_norm": 0.05629874765872955,
+      "learning_rate": 1.848851136353614e-05,
+      "loss": 0.0101,
+      "step": 1010
+    },
+    {
+      "epoch": 0.5250289538025994,
+      "grad_norm": 0.05504068732261658,
+      "learning_rate": 1.8440678865712237e-05,
+      "loss": 0.0098,
+      "step": 1020
+    },
+    {
+      "epoch": 0.5301762964869386,
+      "grad_norm": 0.053771983832120895,
+      "learning_rate": 1.8392165154872596e-05,
+      "loss": 0.0101,
+      "step": 1030
+    },
+    {
+      "epoch": 0.5353236391712778,
+      "grad_norm": 0.058356136083602905,
+      "learning_rate": 1.8342974146362397e-05,
+      "loss": 0.0102,
+      "step": 1040
+    },
+    {
+      "epoch": 0.540470981855617,
+      "grad_norm": 0.05251409858465195,
+      "learning_rate": 1.8293109810188756e-05,
+      "loss": 0.0105,
+      "step": 1050
+    },
+    {
+      "epoch": 0.5456183245399563,
+      "grad_norm": 0.05582083389163017,
+      "learning_rate": 1.8242576170700356e-05,
+      "loss": 0.0108,
+      "step": 1060
+    },
+    {
+      "epoch": 0.5507656672242954,
+      "grad_norm": 0.054003242403268814,
+      "learning_rate": 1.8191377306262635e-05,
+      "loss": 0.0104,
+      "step": 1070
+    },
+    {
+      "epoch": 0.5559130099086347,
+      "grad_norm": 0.05103457346558571,
+      "learning_rate": 1.813951734892864e-05,
+      "loss": 0.0102,
+      "step": 1080
+    },
+    {
+      "epoch": 0.5610603525929739,
+      "grad_norm": 0.04666399583220482,
+      "learning_rate": 1.808700048410555e-05,
+      "loss": 0.0104,
+      "step": 1090
+    },
+    {
+      "epoch": 0.5662076952773131,
+      "grad_norm": 0.05002720654010773,
+      "learning_rate": 1.80338309502169e-05,
+      "loss": 0.0103,
+      "step": 1100
+    },
+    {
+      "epoch": 0.5713550379616523,
+      "grad_norm": 0.0568714365363121,
+      "learning_rate": 1.798001303836048e-05,
+      "loss": 0.0107,
+      "step": 1110
+    },
+    {
+      "epoch": 0.5765023806459915,
+      "grad_norm": 0.051760654896497726,
+      "learning_rate": 1.792555109196205e-05,
+      "loss": 0.0109,
+      "step": 1120
+    },
+    {
+      "epoch": 0.5816497233303307,
+      "grad_norm": 0.05566855147480965,
+      "learning_rate": 1.7870449506424802e-05,
+      "loss": 0.0102,
+      "step": 1130
+    },
+    {
+      "epoch": 0.58679706601467,
+      "grad_norm": 0.058662049472332,
+      "learning_rate": 1.7814712728774602e-05,
+      "loss": 0.0098,
+      "step": 1140
+    },
+    {
+      "epoch": 0.5919444086990091,
+      "grad_norm": 0.04951569065451622,
+      "learning_rate": 1.7758345257301097e-05,
+      "loss": 0.01,
+      "step": 1150
+    },
+    {
+      "epoch": 0.5970917513833484,
+      "grad_norm": 0.05501884967088699,
+      "learning_rate": 1.770135164119468e-05,
+      "loss": 0.0106,
+      "step": 1160
+    },
+    {
+      "epoch": 0.6022390940676876,
+      "grad_norm": 0.04806562513113022,
+      "learning_rate": 1.7643736480179353e-05,
+      "loss": 0.0099,
+      "step": 1170
+    },
+    {
+      "epoch": 0.6073864367520267,
+      "grad_norm": 0.056878462433815,
+      "learning_rate": 1.7585504424141483e-05,
+      "loss": 0.0101,
+      "step": 1180
+    },
+    {
+      "epoch": 0.612533779436366,
+      "grad_norm": 0.05829343572258949,
+      "learning_rate": 1.752666017275453e-05,
+      "loss": 0.0104,
+      "step": 1190
+    },
+    {
+      "epoch": 0.6176811221207051,
+      "grad_norm": 0.054884593933820724,
+      "learning_rate": 1.7467208475099777e-05,
+      "loss": 0.0102,
+      "step": 1200
+    },
+    {
+      "epoch": 0.6228284648050444,
+      "grad_norm": 0.07677657157182693,
+      "learning_rate": 1.740715412928302e-05,
+      "loss": 0.0101,
+      "step": 1210
+    },
+    {
+      "epoch": 0.6279758074893836,
+      "grad_norm": 0.058270618319511414,
+      "learning_rate": 1.734650198204736e-05,
+      "loss": 0.0099,
+      "step": 1220
+    },
+    {
+      "epoch": 0.6331231501737228,
+      "grad_norm": 0.04892059788107872,
+      "learning_rate": 1.7285256928382024e-05,
+      "loss": 0.0098,
+      "step": 1230
+    },
+    {
+      "epoch": 0.638270492858062,
+      "grad_norm": 0.05252835527062416,
+      "learning_rate": 1.7223423911127315e-05,
+      "loss": 0.0112,
+      "step": 1240
+    },
+    {
+      "epoch": 0.6434178355424013,
+      "grad_norm": 0.05630745738744736,
+      "learning_rate": 1.7161007920575705e-05,
+      "loss": 0.0098,
+      "step": 1250
+    },
+    {
+      "epoch": 0.6485651782267404,
+      "grad_norm": 0.051849983632564545,
+      "learning_rate": 1.709801399406907e-05,
+      "loss": 0.0097,
+      "step": 1260
+    },
+    {
+      "epoch": 0.6537125209110797,
+      "grad_norm": 0.05386548116803169,
+      "learning_rate": 1.7034447215592168e-05,
+      "loss": 0.0097,
+      "step": 1270
+    },
+    {
+      "epoch": 0.6588598635954188,
+      "grad_norm": 0.051140256226062775,
+      "learning_rate": 1.6970312715362304e-05,
+      "loss": 0.0101,
+      "step": 1280
+    },
+    {
+      "epoch": 0.6640072062797581,
+      "grad_norm": 0.047512467950582504,
+      "learning_rate": 1.6905615669415326e-05,
+      "loss": 0.0105,
+      "step": 1290
+    },
+    {
+      "epoch": 0.6691545489640973,
+      "grad_norm": 0.051020972430706024,
+      "learning_rate": 1.684036129918786e-05,
+      "loss": 0.0098,
+      "step": 1300
+    },
+    {
+      "epoch": 0.6743018916484365,
+      "grad_norm": 0.04566548392176628,
+      "learning_rate": 1.6774554871095918e-05,
+      "loss": 0.0095,
+      "step": 1310
+    },
+    {
+      "epoch": 0.6794492343327757,
+      "grad_norm": 0.05308493971824646,
+      "learning_rate": 1.6708201696109857e-05,
+      "loss": 0.0102,
+      "step": 1320
+    },
+    {
+      "epoch": 0.684596577017115,
+      "grad_norm": 0.046334341168403625,
+      "learning_rate": 1.6641307129325785e-05,
+      "loss": 0.0105,
+      "step": 1330
+    },
+    {
+      "epoch": 0.6897439197014541,
+      "grad_norm": 0.059736642986536026,
+      "learning_rate": 1.657387656953333e-05,
+      "loss": 0.0103,
+      "step": 1340
+    },
+    {
+      "epoch": 0.6948912623857934,
+      "grad_norm": 0.055522654205560684,
+      "learning_rate": 1.6505915458779958e-05,
+      "loss": 0.01,
+      "step": 1350
+    },
+    {
+      "epoch": 0.7000386050701325,
+      "grad_norm": 0.05038890987634659,
+      "learning_rate": 1.6437429281931744e-05,
+      "loss": 0.0094,
+      "step": 1360
+    },
+    {
+      "epoch": 0.7051859477544717,
+      "grad_norm": 0.05290698632597923,
+      "learning_rate": 1.636842356623073e-05,
+      "loss": 0.0098,
+      "step": 1370
+    },
+    {
+      "epoch": 0.710333290438811,
+      "grad_norm": 0.053712453693151474,
+      "learning_rate": 1.6298903880848836e-05,
+      "loss": 0.0102,
+      "step": 1380
+    },
+    {
+      "epoch": 0.7154806331231501,
+      "grad_norm": 0.04889325425028801,
+      "learning_rate": 1.6228875836438386e-05,
+      "loss": 0.0099,
+      "step": 1390
+    },
+    {
+      "epoch": 0.7206279758074894,
+      "grad_norm": 0.05055889114737511,
+      "learning_rate": 1.615834508467931e-05,
+      "loss": 0.0095,
+      "step": 1400
+    },
+    {
+      "epoch": 0.7257753184918286,
+      "grad_norm": 0.05285302549600601,
+      "learning_rate": 1.608731731782301e-05,
+      "loss": 0.0101,
+      "step": 1410
+    },
+    {
+      "epoch": 0.7309226611761678,
+      "grad_norm": 0.05686409771442413,
+      "learning_rate": 1.6015798268232964e-05,
+      "loss": 0.0102,
+      "step": 1420
+    },
+    {
+      "epoch": 0.736070003860507,
+      "grad_norm": 0.04935956746339798,
+      "learning_rate": 1.5943793707922086e-05,
+      "loss": 0.0093,
+      "step": 1430
+    },
+    {
+      "epoch": 0.7412173465448463,
+      "grad_norm": 0.050160281360149384,
+      "learning_rate": 1.5871309448086903e-05,
+      "loss": 0.0104,
+      "step": 1440
+    },
+    {
+      "epoch": 0.7463646892291854,
+      "grad_norm": 0.05499512329697609,
+      "learning_rate": 1.5798351338638548e-05,
+      "loss": 0.0097,
+      "step": 1450
+    },
+    {
+      "epoch": 0.7515120319135247,
+      "grad_norm": 0.05228967219591141,
+      "learning_rate": 1.5724925267730625e-05,
+      "loss": 0.0103,
+      "step": 1460
+    },
+    {
+      "epoch": 0.7566593745978638,
+      "grad_norm": 0.0571601577103138,
+      "learning_rate": 1.5651037161284032e-05,
+      "loss": 0.0102,
+      "step": 1470
+    },
+    {
+      "epoch": 0.7618067172822031,
+      "grad_norm": 0.05126621946692467,
+      "learning_rate": 1.5576692982508665e-05,
+      "loss": 0.0098,
+      "step": 1480
+    },
+    {
+      "epoch": 0.7669540599665423,
+      "grad_norm": 0.050559673458337784,
+      "learning_rate": 1.550189873142219e-05,
+      "loss": 0.0099,
+      "step": 1490
+    },
+    {
+      "epoch": 0.7721014026508815,
+      "grad_norm": 0.047047846019268036,
+      "learning_rate": 1.542666044436577e-05,
+      "loss": 0.0097,
+      "step": 1500
+    },
+    {
+      "epoch": 0.7772487453352207,
+      "grad_norm": 0.051877353340387344,
+      "learning_rate": 1.5350984193516917e-05,
+      "loss": 0.01,
+      "step": 1510
+    },
+    {
+      "epoch": 0.78239608801956,
+      "grad_norm": 0.05931961163878441,
+      "learning_rate": 1.5274876086399444e-05,
+      "loss": 0.0096,
+      "step": 1520
+    },
+    {
+      "epoch": 0.7875434307038991,
+      "grad_norm": 0.04609573632478714,
+      "learning_rate": 1.5198342265390511e-05,
+      "loss": 0.0095,
+      "step": 1530
+    },
+    {
+      "epoch": 0.7926907733882383,
+      "grad_norm": 0.05874916538596153,
+      "learning_rate": 1.512138890722494e-05,
+      "loss": 0.0099,
+      "step": 1540
+    },
+    {
+      "epoch": 0.7978381160725775,
+      "grad_norm": 0.05315364524722099,
+      "learning_rate": 1.5044022222496684e-05,
+      "loss": 0.01,
+      "step": 1550
+    },
+    {
+      "epoch": 0.8029854587569167,
+      "grad_norm": 0.04722389206290245,
+      "learning_rate": 1.4966248455157622e-05,
+      "loss": 0.0096,
+      "step": 1560
+    },
+    {
+      "epoch": 0.808132801441256,
+      "grad_norm": 0.0479586161673069,
+      "learning_rate": 1.4888073882013616e-05,
+      "loss": 0.0094,
+      "step": 1570
+    },
+    {
+      "epoch": 0.8132801441255951,
+      "grad_norm": 0.051096055656671524,
+      "learning_rate": 1.480950481221795e-05,
+      "loss": 0.0096,
+      "step": 1580
+    },
+    {
+      "epoch": 0.8184274868099344,
+      "grad_norm": 0.0522165410220623,
+      "learning_rate": 1.4730547586762114e-05,
+      "loss": 0.0093,
+      "step": 1590
+    },
+    {
+      "epoch": 0.8235748294942736,
+      "grad_norm": 0.05063074454665184,
+      "learning_rate": 1.4651208577964099e-05,
+      "loss": 0.0101,
+      "step": 1600
+    },
+    {
+      "epoch": 0.8287221721786128,
+      "grad_norm": 0.060858119279146194,
+      "learning_rate": 1.4571494188954058e-05,
+      "loss": 0.01,
+      "step": 1610
+    },
+    {
+      "epoch": 0.833869514862952,
+      "grad_norm": 0.040989868342876434,
+      "learning_rate": 1.4491410853157577e-05,
+      "loss": 0.0095,
+      "step": 1620
+    },
+    {
+      "epoch": 0.8390168575472912,
+      "grad_norm": 0.04905058071017265,
+      "learning_rate": 1.4410965033776439e-05,
+      "loss": 0.0099,
+      "step": 1630
+    },
+    {
+      "epoch": 0.8441642002316304,
+      "grad_norm": 0.04507426545023918,
+      "learning_rate": 1.4330163223267005e-05,
+      "loss": 0.01,
+      "step": 1640
+    },
+    {
+      "epoch": 0.8493115429159697,
+      "grad_norm": 0.048070088028907776,
+      "learning_rate": 1.4249011942816245e-05,
+      "loss": 0.0099,
+      "step": 1650
+    },
+    {
+      "epoch": 0.8544588856003088,
+      "grad_norm": 0.04860364645719528,
+      "learning_rate": 1.416751774181543e-05,
+      "loss": 0.0097,
+      "step": 1660
+    },
+    {
+      "epoch": 0.859606228284648,
+      "grad_norm": 0.047335829585790634,
+      "learning_rate": 1.4085687197331559e-05,
+      "loss": 0.0097,
+      "step": 1670
+    },
+    {
+      "epoch": 0.8647535709689873,
+      "grad_norm": 0.04517504945397377,
+      "learning_rate": 1.4003526913576555e-05,
+      "loss": 0.0103,
+      "step": 1680
+    },
+    {
+      "epoch": 0.8699009136533264,
+      "grad_norm": 0.04708757624030113,
+      "learning_rate": 1.392104352137426e-05,
+      "loss": 0.0091,
+      "step": 1690
+    },
+    {
+      "epoch": 0.8750482563376657,
+      "grad_norm": 0.04715834558010101,
+      "learning_rate": 1.3838243677625292e-05,
+      "loss": 0.0099,
+      "step": 1700
+    },
+    {
+      "epoch": 0.8801955990220048,
+      "grad_norm": 0.052369069308042526,
+      "learning_rate": 1.3755134064769788e-05,
+      "loss": 0.0097,
+      "step": 1710
+    },
+    {
+      "epoch": 0.8853429417063441,
+      "grad_norm": 0.04586135223507881,
+      "learning_rate": 1.36717213902481e-05,
+      "loss": 0.0093,
+      "step": 1720
+    },
+    {
+      "epoch": 0.8904902843906833,
+      "grad_norm": 0.04993924871087074,
+      "learning_rate": 1.3588012385959464e-05,
+      "loss": 0.0101,
+      "step": 1730
+    },
+    {
+      "epoch": 0.8956376270750225,
+      "grad_norm": 0.05076312646269798,
+      "learning_rate": 1.350401380771868e-05,
+      "loss": 0.0096,
+      "step": 1740
+    },
+    {
+      "epoch": 0.9007849697593617,
+      "grad_norm": 0.04548332095146179,
+      "learning_rate": 1.3419732434710895e-05,
+      "loss": 0.009,
+      "step": 1750
+    },
+    {
+      "epoch": 0.905932312443701,
+      "grad_norm": 0.05302567407488823,
+      "learning_rate": 1.333517506894448e-05,
+      "loss": 0.0098,
+      "step": 1760
+    },
+    {
+      "epoch": 0.9110796551280401,
+      "grad_norm": 0.04806780815124512,
+      "learning_rate": 1.3250348534702063e-05,
+      "loss": 0.0096,
+      "step": 1770
+    },
+    {
+      "epoch": 0.9162269978123794,
+      "grad_norm": 0.044754717499017715,
+      "learning_rate": 1.316525967798976e-05,
+      "loss": 0.0095,
+      "step": 1780
+    },
+    {
+      "epoch": 0.9213743404967185,
+      "grad_norm": 0.048751894384622574,
+      "learning_rate": 1.307991536598469e-05,
+      "loss": 0.01,
+      "step": 1790
+    },
+    {
+      "epoch": 0.9265216831810578,
+      "grad_norm": 0.051805585622787476,
+      "learning_rate": 1.2994322486480716e-05,
+      "loss": 0.0094,
+      "step": 1800
+    },
+    {
+      "epoch": 0.931669025865397,
+      "grad_norm": 0.047136593610048294,
+      "learning_rate": 1.2908487947332588e-05,
+      "loss": 0.0092,
+      "step": 1810
+    },
+    {
+      "epoch": 0.9368163685497362,
+      "grad_norm": 0.05186924338340759,
+      "learning_rate": 1.2822418675898428e-05,
+      "loss": 0.01,
+      "step": 1820
+    },
+    {
+      "epoch": 0.9419637112340754,
+      "grad_norm": 0.046143535524606705,
+      "learning_rate": 1.273612161848064e-05,
+      "loss": 0.0094,
+      "step": 1830
+    },
+    {
+      "epoch": 0.9471110539184147,
+      "grad_norm": 0.0523681677877903,
+      "learning_rate": 1.2649603739765323e-05,
+      "loss": 0.0092,
+      "step": 1840
+    },
+    {
+      "epoch": 0.9522583966027538,
+      "grad_norm": 0.047227270901203156,
+      "learning_rate": 1.2562872022260168e-05,
+      "loss": 0.0097,
+      "step": 1850
+    },
+    {
+      "epoch": 0.957405739287093,
+      "grad_norm": 0.05158701911568642,
+      "learning_rate": 1.2475933465730923e-05,
+      "loss": 0.0091,
+      "step": 1860
+    },
+    {
+      "epoch": 0.9625530819714323,
+      "grad_norm": 0.05671868845820427,
+      "learning_rate": 1.238879508663648e-05,
+      "loss": 0.0094,
+      "step": 1870
+    },
+    {
+      "epoch": 0.9677004246557714,
+      "grad_norm": 0.05596312880516052,
+      "learning_rate": 1.2301463917562602e-05,
+      "loss": 0.0094,
+      "step": 1880
+    },
+    {
+      "epoch": 0.9728477673401107,
+      "grad_norm": 0.050950005650520325,
+      "learning_rate": 1.2213947006654347e-05,
+      "loss": 0.0095,
+      "step": 1890
+    },
+    {
+      "epoch": 0.9779951100244498,
+      "grad_norm": 0.05468609929084778,
+      "learning_rate": 1.212625141704725e-05,
+      "loss": 0.0099,
+      "step": 1900
+    },
+    {
+      "epoch": 0.9831424527087891,
+      "grad_norm": 0.04101032391190529,
+      "learning_rate": 1.2038384226297272e-05,
+      "loss": 0.0093,
+      "step": 1910
+    },
+    {
+      "epoch": 0.9882897953931283,
+      "grad_norm": 0.04567647725343704,
+      "learning_rate": 1.1950352525809619e-05,
+      "loss": 0.0093,
+      "step": 1920
+    },
+    {
+      "epoch": 0.9934371380774675,
+      "grad_norm": 0.05419466271996498,
+      "learning_rate": 1.18621634202664e-05,
+      "loss": 0.0093,
+      "step": 1930
+    },
+    {
+      "epoch": 0.9985844807618067,
+      "grad_norm": 0.04440421238541603,
+      "learning_rate": 1.1773824027053256e-05,
+      "loss": 0.0093,
+      "step": 1940
+    },
+    {
+      "epoch": 1.0036031398790375,
+      "grad_norm": 0.04989992827177048,
+      "learning_rate": 1.1685341475684935e-05,
+      "loss": 0.0081,
+      "step": 1950
+    },
+    {
+      "epoch": 1.0087504825633766,
+      "grad_norm": 0.047590870410203934,
+      "learning_rate": 1.15967229072299e-05,
+      "loss": 0.008,
+      "step": 1960
+    },
+    {
+      "epoch": 1.0138978252477158,
+      "grad_norm": 0.044245149940252304,
+      "learning_rate": 1.1507975473734002e-05,
+      "loss": 0.008,
+      "step": 1970
+    },
+    {
+      "epoch": 1.0190451679320551,
+      "grad_norm": 0.04547454044222832,
+      "learning_rate": 1.141910633764327e-05,
+      "loss": 0.0077,
+      "step": 1980
+    },
+    {
+      "epoch": 1.0241925106163943,
+      "grad_norm": 0.04556995630264282,
+      "learning_rate": 1.1330122671225855e-05,
+      "loss": 0.008,
+      "step": 1990
+    },
+    {
+      "epoch": 1.0293398533007334,
+      "grad_norm": 0.0485275536775589,
+      "learning_rate": 1.1241031655993188e-05,
+      "loss": 0.0081,
+      "step": 2000
+    },
+    {
+      "epoch": 1.0344871959850728,
+      "grad_norm": 0.04903995245695114,
+      "learning_rate": 1.1151840482120386e-05,
+      "loss": 0.008,
+      "step": 2010
+    },
+    {
+      "epoch": 1.039634538669412,
+      "grad_norm": 0.046454910188913345,
+      "learning_rate": 1.1062556347865967e-05,
+      "loss": 0.0081,
+      "step": 2020
+    },
+    {
+      "epoch": 1.044781881353751,
+      "grad_norm": 0.04765252768993378,
+      "learning_rate": 1.0973186458990902e-05,
+      "loss": 0.0076,
+      "step": 2030
+    },
+    {
+      "epoch": 1.0499292240380904,
+      "grad_norm": 0.050241731107234955,
+      "learning_rate": 1.0883738028177069e-05,
+      "loss": 0.0081,
+      "step": 2040
+    },
+    {
+      "epoch": 1.0550765667224296,
+      "grad_norm": 0.050933465361595154,
+      "learning_rate": 1.0794218274445155e-05,
+      "loss": 0.0082,
+      "step": 2050
+    },
+    {
+      "epoch": 1.0602239094067687,
+      "grad_norm": 0.04815370962023735,
+      "learning_rate": 1.0704634422572029e-05,
+      "loss": 0.0082,
+      "step": 2060
+    },
+    {
+      "epoch": 1.065371252091108,
+      "grad_norm": 0.04359019547700882,
+      "learning_rate": 1.0614993702507649e-05,
+      "loss": 0.0083,
+      "step": 2070
+    },
+    {
+      "epoch": 1.0705185947754472,
+      "grad_norm": 0.05031314119696617,
+      "learning_rate": 1.0525303348791599e-05,
+      "loss": 0.0077,
+      "step": 2080
+    },
+    {
+      "epoch": 1.0756659374597863,
+      "grad_norm": 0.04779091849923134,
+      "learning_rate": 1.0435570599969168e-05,
+      "loss": 0.0081,
+      "step": 2090
+    },
+    {
+      "epoch": 1.0808132801441257,
+      "grad_norm": 0.04763912782073021,
+      "learning_rate": 1.0345802698007198e-05,
+      "loss": 0.0083,
+      "step": 2100
+    },
+    {
+      "epoch": 1.0859606228284648,
+      "grad_norm": 0.048106785863637924,
+      "learning_rate": 1.0256006887709593e-05,
+      "loss": 0.0074,
+      "step": 2110
+    },
+    {
+      "epoch": 1.091107965512804,
+      "grad_norm": 0.04768507555127144,
+      "learning_rate": 1.016619041613263e-05,
+      "loss": 0.008,
+      "step": 2120
+    },
+    {
+      "epoch": 1.0962553081971431,
+      "grad_norm": 0.04941874369978905,
+      "learning_rate": 1.0076360532000069e-05,
+      "loss": 0.0081,
+      "step": 2130
+    },
+    {
+      "epoch": 1.1014026508814825,
+      "grad_norm": 0.043673593550920486,
+      "learning_rate": 9.986524485118152e-06,
+      "loss": 0.0076,
+      "step": 2140
+    },
+    {
+      "epoch": 1.1065499935658216,
+      "grad_norm": 0.04693906381726265,
+      "learning_rate": 9.896689525790474e-06,
+      "loss": 0.0085,
+      "step": 2150
+    },
+    {
+      "epoch": 1.1116973362501608,
+      "grad_norm": 0.04276665300130844,
+      "learning_rate": 9.80686290423288e-06,
+      "loss": 0.0075,
+      "step": 2160
+    },
+    {
+      "epoch": 1.1168446789345001,
+      "grad_norm": 0.04190380498766899,
+      "learning_rate": 9.717051869988287e-06,
+      "loss": 0.0079,
+      "step": 2170
+    },
+    {
+      "epoch": 1.1219920216188393,
+      "grad_norm": 0.05057093873620033,
+      "learning_rate": 9.627263671341638e-06,
+      "loss": 0.0077,
+      "step": 2180
+    },
+    {
+      "epoch": 1.1271393643031784,
+      "grad_norm": 0.050689809024333954,
+      "learning_rate": 9.537505554734901e-06,
+      "loss": 0.0081,
+      "step": 2190
+    },
+    {
+      "epoch": 1.1322867069875178,
+      "grad_norm": 0.043263569474220276,
+      "learning_rate": 9.447784764182247e-06,
+      "loss": 0.0077,
+      "step": 2200
+    },
+    {
+      "epoch": 1.137434049671857,
+      "grad_norm": 0.0434611551463604,
+      "learning_rate": 9.358108540685406e-06,
+      "loss": 0.0075,
+      "step": 2210
+    },
+    {
+      "epoch": 1.142581392356196,
+      "grad_norm": 0.052013982087373734,
+      "learning_rate": 9.268484121649289e-06,
+      "loss": 0.008,
+      "step": 2220
+    },
+    {
+      "epoch": 1.1477287350405354,
+      "grad_norm": 0.04599529504776001,
+      "learning_rate": 9.178918740297877e-06,
+      "loss": 0.0089,
+      "step": 2230
+    },
+    {
+      "epoch": 1.1528760777248745,
+      "grad_norm": 0.04090885445475578,
+      "learning_rate": 9.08941962509045e-06,
+      "loss": 0.0078,
+      "step": 2240
+    },
+    {
+      "epoch": 1.1580234204092137,
+      "grad_norm": 0.04642123356461525,
+      "learning_rate": 8.999993999138227e-06,
+      "loss": 0.0083,
+      "step": 2250
+    },
+    {
+      "epoch": 1.1631707630935528,
+      "grad_norm": 0.04511793702840805,
+      "learning_rate": 8.91064907962141e-06,
+      "loss": 0.0075,
+      "step": 2260
+    },
+    {
+      "epoch": 1.1683181057778922,
+      "grad_norm": 0.04354127123951912,
+      "learning_rate": 8.821392077206695e-06,
+      "loss": 0.0079,
+      "step": 2270
+    },
+    {
+      "epoch": 1.1734654484622313,
+      "grad_norm": 0.04541221261024475,
+      "learning_rate": 8.732230195465353e-06,
+      "loss": 0.0077,
+      "step": 2280
+    },
+    {
+      "epoch": 1.1786127911465707,
+      "grad_norm": 0.04809452220797539,
+      "learning_rate": 8.64317063029186e-06,
+      "loss": 0.0079,
+      "step": 2290
+    },
+    {
+      "epoch": 1.1837601338309098,
+      "grad_norm": 0.04383910447359085,
+      "learning_rate": 8.554220569323117e-06,
+      "loss": 0.008,
+      "step": 2300
+    },
+    {
+      "epoch": 1.188907476515249,
+      "grad_norm": 0.04871731251478195,
+      "learning_rate": 8.465387191358398e-06,
+      "loss": 0.0079,
+      "step": 2310
+    },
+    {
+      "epoch": 1.1940548191995881,
+      "grad_norm": 0.042991671711206436,
+      "learning_rate": 8.376677665779968e-06,
+      "loss": 0.0078,
+      "step": 2320
+    },
+    {
+      "epoch": 1.1992021618839275,
+      "grad_norm": 0.04335131123661995,
+      "learning_rate": 8.288099151974446e-06,
+      "loss": 0.0079,
+      "step": 2330
+    },
+    {
+      "epoch": 1.2043495045682666,
+      "grad_norm": 0.04826260730624199,
+      "learning_rate": 8.199658798755048e-06,
+      "loss": 0.008,
+      "step": 2340
+    },
+    {
+      "epoch": 1.2094968472526058,
+      "grad_norm": 0.046739570796489716,
+      "learning_rate": 8.111363743784587e-06,
+      "loss": 0.0079,
+      "step": 2350
+    },
+    {
+      "epoch": 1.2146441899369451,
+      "grad_norm": 0.05227670818567276,
+      "learning_rate": 8.023221112999459e-06,
+      "loss": 0.0076,
+      "step": 2360
+    },
+    {
+      "epoch": 1.2197915326212843,
+      "grad_norm": 0.041948165744543076,
+      "learning_rate": 7.935238020034505e-06,
+      "loss": 0.0069,
+      "step": 2370
+    },
+    {
+      "epoch": 1.2249388753056234,
+      "grad_norm": 0.04497140645980835,
+      "learning_rate": 7.847421565648929e-06,
+      "loss": 0.0078,
+      "step": 2380
+    },
+    {
+      "epoch": 1.2300862179899628,
+      "grad_norm": 0.045373767614364624,
+      "learning_rate": 7.759778837153205e-06,
+      "loss": 0.0083,
+      "step": 2390
+    },
+    {
+      "epoch": 1.235233560674302,
+      "grad_norm": 0.04437263309955597,
+      "learning_rate": 7.672316907837084e-06,
+      "loss": 0.0076,
+      "step": 2400
+    },
+    {
+      "epoch": 1.240380903358641,
+      "grad_norm": 0.043433234095573425,
+      "learning_rate": 7.585042836398763e-06,
+      "loss": 0.0076,
+      "step": 2410
+    },
+    {
+      "epoch": 1.2455282460429804,
+      "grad_norm": 0.05035654082894325,
+      "learning_rate": 7.497963666375185e-06,
+      "loss": 0.0081,
+      "step": 2420
+    },
+    {
+      "epoch": 1.2506755887273195,
+      "grad_norm": 0.048377279192209244,
+      "learning_rate": 7.411086425573584e-06,
+      "loss": 0.0079,
+      "step": 2430
+    },
+    {
+      "epoch": 1.2558229314116587,
+      "grad_norm": 0.04650159552693367,
+      "learning_rate": 7.324418125504305e-06,
+      "loss": 0.008,
+      "step": 2440
+    },
+    {
+      "epoch": 1.2609702740959978,
+      "grad_norm": 0.045357707887887955,
+      "learning_rate": 7.23796576081494e-06,
+      "loss": 0.0081,
+      "step": 2450
+    },
+    {
+      "epoch": 1.2661176167803372,
+      "grad_norm": 0.043975580483675,
+      "learning_rate": 7.151736308725802e-06,
+      "loss": 0.0073,
+      "step": 2460
+    },
+    {
+      "epoch": 1.2712649594646763,
+      "grad_norm": 0.04668817296624184,
+      "learning_rate": 7.065736728466832e-06,
+      "loss": 0.0077,
+      "step": 2470
+    },
+    {
+      "epoch": 1.2764123021490157,
+      "grad_norm": 0.04745522886514664,
+      "learning_rate": 6.979973960715958e-06,
+      "loss": 0.0081,
+      "step": 2480
+    },
+    {
+      "epoch": 1.2815596448333548,
+      "grad_norm": 0.0445983000099659,
+      "learning_rate": 6.894454927038907e-06,
+      "loss": 0.0072,
+      "step": 2490
+    },
+    {
+      "epoch": 1.286706987517694,
+      "grad_norm": 0.043321456760168076,
+      "learning_rate": 6.809186529330639e-06,
+      "loss": 0.0079,
+      "step": 2500
+    },
+    {
+      "epoch": 1.291854330202033,
+      "grad_norm": 0.04421858489513397,
+      "learning_rate": 6.724175649258287e-06,
+      "loss": 0.0083,
+      "step": 2510
+    },
+    {
+      "epoch": 1.2970016728863725,
+      "grad_norm": 0.0411141961812973,
+      "learning_rate": 6.6394291477057736e-06,
+      "loss": 0.0076,
+      "step": 2520
+    },
+    {
+      "epoch": 1.3021490155707116,
+      "grad_norm": 0.05308463051915169,
+      "learning_rate": 6.554953864220115e-06,
+      "loss": 0.0079,
+      "step": 2530
+    },
+    {
+      "epoch": 1.3072963582550507,
+      "grad_norm": 0.04345130920410156,
+      "learning_rate": 6.4707566164594105e-06,
+      "loss": 0.0076,
+      "step": 2540
+    },
+    {
+      "epoch": 1.3124437009393901,
+      "grad_norm": 0.046729203313589096,
+      "learning_rate": 6.386844199642619e-06,
+      "loss": 0.0077,
+      "step": 2550
+    },
+    {
+      "epoch": 1.3175910436237293,
+      "grad_norm": 0.046459197998046875,
+      "learning_rate": 6.30322338600114e-06,
+      "loss": 0.0083,
+      "step": 2560
+    },
+    {
+      "epoch": 1.3227383863080684,
+      "grad_norm": 0.04731113836169243,
+      "learning_rate": 6.219900924232279e-06,
+      "loss": 0.0072,
+      "step": 2570
+    },
+    {
+      "epoch": 1.3278857289924075,
+      "grad_norm": 0.04013852775096893,
+      "learning_rate": 6.136883538954561e-06,
+      "loss": 0.0079,
+      "step": 2580
+    },
+    {
+      "epoch": 1.333033071676747,
+      "grad_norm": 0.046594880521297455,
+      "learning_rate": 6.054177930165017e-06,
+      "loss": 0.0072,
+      "step": 2590
+    },
+    {
+      "epoch": 1.338180414361086,
+      "grad_norm": 0.04768165200948715,
+      "learning_rate": 5.971790772698467e-06,
+      "loss": 0.0075,
+      "step": 2600
+    },
+    {
+      "epoch": 1.3433277570454254,
+      "grad_norm": 0.046321555972099304,
+      "learning_rate": 5.889728715688814e-06,
+      "loss": 0.0076,
+      "step": 2610
+    },
+    {
+      "epoch": 1.3484750997297645,
+      "grad_norm": 0.04485670477151871,
+      "learning_rate": 5.807998382032414e-06,
+      "loss": 0.0077,
+      "step": 2620
+    },
+    {
+      "epoch": 1.3536224424141037,
+      "grad_norm": 0.046425607055425644,
+      "learning_rate": 5.726606367853581e-06,
+      "loss": 0.0078,
+      "step": 2630
+    },
+    {
+      "epoch": 1.3587697850984428,
+      "grad_norm": 0.051463935524225235,
+      "learning_rate": 5.645559241972231e-06,
+      "loss": 0.0081,
+      "step": 2640
+    },
+    {
+      "epoch": 1.3639171277827822,
+      "grad_norm": 0.04258693754673004,
+      "learning_rate": 5.56486354537374e-06,
+      "loss": 0.0077,
+      "step": 2650
+    },
+    {
+      "epoch": 1.3690644704671213,
+      "grad_norm": 0.04762498661875725,
+      "learning_rate": 5.484525790681052e-06,
+      "loss": 0.0077,
+      "step": 2660
+    },
+    {
+      "epoch": 1.3742118131514607,
+      "grad_norm": 0.04822554439306259,
+      "learning_rate": 5.404552461629069e-06,
+      "loss": 0.0075,
+      "step": 2670
+    },
+    {
+      "epoch": 1.3793591558357998,
+      "grad_norm": 0.04345494881272316,
+      "learning_rate": 5.324950012541372e-06,
+      "loss": 0.0079,
+      "step": 2680
+    },
+    {
+      "epoch": 1.384506498520139,
+      "grad_norm": 0.044225189834833145,
+      "learning_rate": 5.245724867809326e-06,
+      "loss": 0.0077,
+      "step": 2690
+    },
+    {
+      "epoch": 1.389653841204478,
+      "grad_norm": 0.04013178497552872,
+      "learning_rate": 5.166883421373583e-06,
+      "loss": 0.0077,
+      "step": 2700
+    },
+    {
+      "epoch": 1.3948011838888175,
+      "grad_norm": 0.04217646270990372,
+      "learning_rate": 5.088432036208075e-06,
+      "loss": 0.0074,
+      "step": 2710
+    },
+    {
+      "epoch": 1.3999485265731566,
+      "grad_norm": 0.04379292577505112,
+      "learning_rate": 5.010377043806443e-06,
+      "loss": 0.0067,
+      "step": 2720
+    },
+    {
+      "epoch": 1.4050958692574957,
+      "grad_norm": 0.04153648018836975,
+      "learning_rate": 4.932724743671089e-06,
+      "loss": 0.0074,
+      "step": 2730
+    },
+    {
+      "epoch": 1.410243211941835,
+      "grad_norm": 0.04640040919184685,
+      "learning_rate": 4.8554814028047605e-06,
+      "loss": 0.0075,
+      "step": 2740
+    },
+    {
+      "epoch": 1.4153905546261742,
+      "grad_norm": 0.04649236798286438,
+      "learning_rate": 4.778653255204733e-06,
+      "loss": 0.008,
+      "step": 2750
+    },
+    {
+      "epoch": 1.4205378973105134,
+      "grad_norm": 0.04218636080622673,
+      "learning_rate": 4.702246501359739e-06,
+      "loss": 0.0071,
+      "step": 2760
+    },
+    {
+      "epoch": 1.4256852399948525,
+      "grad_norm": 0.0437491312623024,
+      "learning_rate": 4.626267307749519e-06,
+      "loss": 0.0074,
+      "step": 2770
+    },
+    {
+      "epoch": 1.4308325826791919,
+      "grad_norm": 0.047346778213977814,
+      "learning_rate": 4.550721806347146e-06,
+      "loss": 0.0077,
+      "step": 2780
+    },
+    {
+      "epoch": 1.435979925363531,
+      "grad_norm": 0.04050770774483681,
+      "learning_rate": 4.475616094124171e-06,
+      "loss": 0.0076,
+      "step": 2790
+    },
+    {
+      "epoch": 1.4411272680478704,
+      "grad_norm": 0.04215119779109955,
+      "learning_rate": 4.40095623255853e-06,
+      "loss": 0.0074,
+      "step": 2800
+    },
+    {
+      "epoch": 1.4462746107322095,
+      "grad_norm": 0.04186023771762848,
+      "learning_rate": 4.3267482471453574e-06,
+      "loss": 0.0076,
+      "step": 2810
+    },
+    {
+      "epoch": 1.4514219534165487,
+      "grad_norm": 0.04239216446876526,
+      "learning_rate": 4.252998126910696e-06,
+      "loss": 0.0077,
+      "step": 2820
+    },
+    {
+      "epoch": 1.4565692961008878,
+      "grad_norm": 0.04196004569530487,
+      "learning_rate": 4.179711823928141e-06,
+      "loss": 0.0075,
+      "step": 2830
+    },
+    {
+      "epoch": 1.4617166387852272,
+      "grad_norm": 0.04251675307750702,
+      "learning_rate": 4.106895252838478e-06,
+      "loss": 0.0074,
+      "step": 2840
+    },
+    {
+      "epoch": 1.4668639814695663,
+      "grad_norm": 0.04271869733929634,
+      "learning_rate": 4.034554290372332e-06,
+      "loss": 0.0074,
+      "step": 2850
+    },
+    {
+      "epoch": 1.4720113241539057,
+      "grad_norm": 0.04729824885725975,
+      "learning_rate": 3.96269477487588e-06,
+      "loss": 0.0081,
+      "step": 2860
+    },
+    {
+      "epoch": 1.4771586668382448,
+      "grad_norm": 0.04521729424595833,
+      "learning_rate": 3.891322505839672e-06,
+      "loss": 0.0075,
+      "step": 2870
+    },
+    {
+      "epoch": 1.482306009522584,
+      "grad_norm": 0.04185229167342186,
+      "learning_rate": 3.820443243430555e-06,
+      "loss": 0.0076,
+      "step": 2880
+    },
+    {
+      "epoch": 1.487453352206923,
+      "grad_norm": 0.04580387473106384,
+      "learning_rate": 3.750062708026814e-06,
+      "loss": 0.0078,
+      "step": 2890
+    },
+    {
+      "epoch": 1.4926006948912625,
+      "grad_norm": 0.04715467616915703,
+      "learning_rate": 3.680186579756507e-06,
+      "loss": 0.0078,
+      "step": 2900
+    },
+    {
+      "epoch": 1.4977480375756016,
+      "grad_norm": 0.047138504683971405,
+      "learning_rate": 3.6108204980390116e-06,
+      "loss": 0.0077,
+      "step": 2910
+    },
+    {
+      "epoch": 1.502895380259941,
+      "grad_norm": 0.04406086727976799,
+      "learning_rate": 3.5419700611299356e-06,
+      "loss": 0.0071,
+      "step": 2920
+    },
+    {
+      "epoch": 1.50804272294428,
+      "grad_norm": 0.04638756439089775,
+      "learning_rate": 3.4736408256692757e-06,
+      "loss": 0.0079,
+      "step": 2930
+    },
+    {
+      "epoch": 1.5131900656286192,
+      "grad_norm": 0.04099031910300255,
+      "learning_rate": 3.4058383062329534e-06,
+      "loss": 0.007,
+      "step": 2940
+    },
+    {
+      "epoch": 1.5183374083129584,
+      "grad_norm": 0.042310506105422974,
+      "learning_rate": 3.3385679748877996e-06,
+      "loss": 0.0073,
+      "step": 2950
+    },
+    {
+      "epoch": 1.5234847509972975,
+      "grad_norm": 0.042861271649599075,
+      "learning_rate": 3.2718352607498814e-06,
+      "loss": 0.0072,
+      "step": 2960
+    },
+    {
+      "epoch": 1.5286320936816369,
+      "grad_norm": 0.04568492993712425,
+      "learning_rate": 3.205645549546361e-06,
+      "loss": 0.0075,
+      "step": 2970
+    },
+    {
+      "epoch": 1.533779436365976,
+      "grad_norm": 0.03910849615931511,
+      "learning_rate": 3.140004183180837e-06,
+      "loss": 0.0076,
+      "step": 2980
+    },
+    {
+      "epoch": 1.5389267790503154,
+      "grad_norm": 0.044579919427633286,
+      "learning_rate": 3.074916459302211e-06,
+      "loss": 0.0077,
+      "step": 2990
+    },
+    {
+      "epoch": 1.5440741217346545,
+      "grad_norm": 0.04029650613665581,
+      "learning_rate": 3.010387630877142e-06,
+      "loss": 0.0071,
+      "step": 3000
+    },
+    {
+      "epoch": 1.5492214644189937,
+      "grad_norm": 0.039773497730493546,
+      "learning_rate": 2.946422905766103e-06,
+      "loss": 0.0079,
+      "step": 3010
+    },
+    {
+      "epoch": 1.5543688071033328,
+      "grad_norm": 0.041781287640333176,
+      "learning_rate": 2.8830274463030705e-06,
+      "loss": 0.0074,
+      "step": 3020
+    },
+    {
+      "epoch": 1.5595161497876722,
+      "grad_norm": 0.04066796600818634,
+      "learning_rate": 2.820206368878894e-06,
+      "loss": 0.0073,
+      "step": 3030
+    },
+    {
+      "epoch": 1.5646634924720113,
+      "grad_norm": 0.037947360426187515,
+      "learning_rate": 2.7579647435283773e-06,
+      "loss": 0.0075,
+      "step": 3040
+    },
+    {
+      "epoch": 1.5698108351563507,
+      "grad_norm": 0.0416950099170208,
+      "learning_rate": 2.696307593521088e-06,
+      "loss": 0.0075,
+      "step": 3050
+    },
+    {
+      "epoch": 1.5749581778406898,
+      "grad_norm": 0.043909162282943726,
+      "learning_rate": 2.6352398949559697e-06,
+      "loss": 0.0074,
+      "step": 3060
+    },
+    {
+      "epoch": 1.580105520525029,
+      "grad_norm": 0.04362725093960762,
+      "learning_rate": 2.5747665763597016e-06,
+      "loss": 0.0078,
+      "step": 3070
+    },
+    {
+      "epoch": 1.585252863209368,
+      "grad_norm": 0.04155511409044266,
+      "learning_rate": 2.514892518288988e-06,
+      "loss": 0.0075,
+      "step": 3080
+    },
+    {
+      "epoch": 1.5904002058937072,
+      "grad_norm": 0.04423890262842178,
+      "learning_rate": 2.4556225529366316e-06,
+      "loss": 0.0073,
+      "step": 3090
+    },
+    {
+      "epoch": 1.5955475485780466,
+      "grad_norm": 0.04012097790837288,
+      "learning_rate": 2.3969614637415474e-06,
+      "loss": 0.0078,
+      "step": 3100
+    },
+    {
+      "epoch": 1.600694891262386,
+      "grad_norm": 0.03919919952750206,
+      "learning_rate": 2.338913985002739e-06,
+      "loss": 0.0071,
+      "step": 3110
+    },
+    {
+      "epoch": 1.605842233946725,
+      "grad_norm": 0.04075808823108673,
+      "learning_rate": 2.281484801497186e-06,
+      "loss": 0.0072,
+      "step": 3120
+    },
+    {
+      "epoch": 1.6109895766310642,
+      "grad_norm": 0.04051820561289787,
+      "learning_rate": 2.224678548101762e-06,
+      "loss": 0.0068,
+      "step": 3130
+    },
+    {
+      "epoch": 1.6161369193154034,
+      "grad_norm": 0.046087682247161865,
+      "learning_rate": 2.168499809419177e-06,
+      "loss": 0.0071,
+      "step": 3140
+    },
+    {
+      "epoch": 1.6212842619997425,
+      "grad_norm": 0.04052680358290672,
+      "learning_rate": 2.1129531194079656e-06,
+      "loss": 0.0072,
+      "step": 3150
+    },
+    {
+      "epoch": 1.6264316046840819,
+      "grad_norm": 0.043879568576812744,
+      "learning_rate": 2.0580429610165763e-06,
+      "loss": 0.0073,
+      "step": 3160
+    },
+    {
+      "epoch": 1.631578947368421,
+      "grad_norm": 0.042183198034763336,
+      "learning_rate": 2.0037737658215684e-06,
+      "loss": 0.0074,
+      "step": 3170
+    },
+    {
+      "epoch": 1.6367262900527604,
+      "grad_norm": 0.041050445288419724,
+      "learning_rate": 1.950149913669953e-06,
+      "loss": 0.0074,
+      "step": 3180
+    },
+    {
+      "epoch": 1.6418736327370995,
+      "grad_norm": 0.04309527203440666,
+      "learning_rate": 1.89717573232572e-06,
+      "loss": 0.0072,
+      "step": 3190
+    },
+    {
+      "epoch": 1.6470209754214387,
+      "grad_norm": 0.0433724969625473,
+      "learning_rate": 1.844855497120558e-06,
+      "loss": 0.0074,
+      "step": 3200
+    },
+    {
+      "epoch": 1.6521683181057778,
+      "grad_norm": 0.0432274304330349,
+      "learning_rate": 1.7931934306088082e-06,
+      "loss": 0.007,
+      "step": 3210
+    },
+    {
+      "epoch": 1.657315660790117,
+      "grad_norm": 0.04388746619224548,
+      "learning_rate": 1.7421937022266921e-06,
+      "loss": 0.0075,
+      "step": 3220
+    },
+    {
+      "epoch": 1.6624630034744563,
+      "grad_norm": 0.04156123474240303,
+      "learning_rate": 1.691860427955785e-06,
+      "loss": 0.0074,
+      "step": 3230
+    },
+    {
+      "epoch": 1.6676103461587957,
+      "grad_norm": 0.04092007502913475,
+      "learning_rate": 1.6421976699908636e-06,
+      "loss": 0.0073,
+      "step": 3240
+    },
+    {
+      "epoch": 1.6727576888431348,
+      "grad_norm": 0.038688093423843384,
+      "learning_rate": 1.5932094364120453e-06,
+      "loss": 0.0066,
+      "step": 3250
+    },
+    {
+      "epoch": 1.677905031527474,
+      "grad_norm": 0.04466381296515465,
+      "learning_rate": 1.544899680861306e-06,
+      "loss": 0.0073,
+      "step": 3260
+    },
+    {
+      "epoch": 1.683052374211813,
+      "grad_norm": 0.04036378487944603,
+      "learning_rate": 1.4972723022234214e-06,
+      "loss": 0.0072,
+      "step": 3270
+    },
+    {
+      "epoch": 1.6881997168961522,
+      "grad_norm": 0.04854420945048332,
+      "learning_rate": 1.45033114431128e-06,
+      "loss": 0.0076,
+      "step": 3280
+    },
+    {
+      "epoch": 1.6933470595804916,
+      "grad_norm": 0.04195775091648102,
+      "learning_rate": 1.4040799955556771e-06,
+      "loss": 0.0073,
+      "step": 3290
+    },
+    {
+      "epoch": 1.6984944022648307,
+      "grad_norm": 0.042864393442869186,
+      "learning_rate": 1.3585225886995613e-06,
+      "loss": 0.0072,
+      "step": 3300
+    },
+    {
+      "epoch": 1.70364174494917,
+      "grad_norm": 0.04399369657039642,
+      "learning_rate": 1.3136626004967834e-06,
+      "loss": 0.0076,
+      "step": 3310
+    },
+    {
+      "epoch": 1.7087890876335092,
+      "grad_norm": 0.04049016907811165,
+      "learning_rate": 1.2695036514153569e-06,
+      "loss": 0.0072,
+      "step": 3320
+    },
+    {
+      "epoch": 1.7139364303178484,
+      "grad_norm": 0.0450235940515995,
+      "learning_rate": 1.2260493053452672e-06,
+      "loss": 0.0077,
+      "step": 3330
+    },
+    {
+      "epoch": 1.7190837730021875,
+      "grad_norm": 0.04058736562728882,
+      "learning_rate": 1.1833030693108426e-06,
+      "loss": 0.0071,
+      "step": 3340
+    },
+    {
+      "epoch": 1.7242311156865269,
+      "grad_norm": 0.0449027344584465,
+      "learning_rate": 1.1412683931877177e-06,
+      "loss": 0.0074,
+      "step": 3350
+    },
+    {
+      "epoch": 1.729378458370866,
+      "grad_norm": 0.040831901133060455,
+      "learning_rate": 1.0999486694244087e-06,
+      "loss": 0.0071,
+      "step": 3360
+    },
+    {
+      "epoch": 1.7345258010552054,
+      "grad_norm": 0.04613341763615608,
+      "learning_rate": 1.0593472327685183e-06,
+      "loss": 0.0074,
+      "step": 3370
+    },
+    {
+      "epoch": 1.7396731437395445,
+      "grad_norm": 0.04327157512307167,
+      "learning_rate": 1.0194673599976134e-06,
+      "loss": 0.0073,
+      "step": 3380
+    },
+    {
+      "epoch": 1.7448204864238837,
+      "grad_norm": 0.04192129895091057,
+      "learning_rate": 9.803122696547473e-07,
+      "loss": 0.0069,
+      "step": 3390
+    },
+    {
+      "epoch": 1.7499678291082228,
+      "grad_norm": 0.04375963658094406,
+      "learning_rate": 9.418851217887337e-07,
+      "loss": 0.0069,
+      "step": 3400
+    },
+    {
+      "epoch": 1.755115171792562,
+      "grad_norm": 0.04598482698202133,
+      "learning_rate": 9.041890176990876e-07,
+      "loss": 0.0074,
+      "step": 3410
+    },
+    {
+      "epoch": 1.7602625144769013,
+      "grad_norm": 0.040907781571149826,
+      "learning_rate": 8.672269996857375e-07,
+      "loss": 0.0067,
+      "step": 3420
+    },
+    {
+      "epoch": 1.7654098571612407,
+      "grad_norm": 0.040340472012758255,
+      "learning_rate": 8.310020508035055e-07,
+      "loss": 0.0071,
+      "step": 3430
+    },
+    {
+      "epoch": 1.7705571998455798,
+      "grad_norm": 0.0423114113509655,
+      "learning_rate": 7.955170946213442e-07,
+      "loss": 0.0072,
+      "step": 3440
+    },
+    {
+      "epoch": 1.775704542529919,
+      "grad_norm": 0.04236366227269173,
+      "learning_rate": 7.607749949863907e-07,
+      "loss": 0.0074,
+      "step": 3450
+    },
+    {
+      "epoch": 1.780851885214258,
+      "grad_norm": 0.04394996911287308,
+      "learning_rate": 7.267785557928398e-07,
+      "loss": 0.0072,
+      "step": 3460
+    },
+    {
+      "epoch": 1.7859992278985972,
+      "grad_norm": 0.04455741122364998,
+      "learning_rate": 6.935305207556509e-07,
+      "loss": 0.0071,
+      "step": 3470
+    },
+    {
+      "epoch": 1.7911465705829366,
+      "grad_norm": 0.04342462122440338,
+      "learning_rate": 6.610335731891116e-07,
+      "loss": 0.0069,
+      "step": 3480
+    },
+    {
+      "epoch": 1.7962939132672757,
+      "grad_norm": 0.03930308297276497,
+      "learning_rate": 6.292903357902846e-07,
+      "loss": 0.0073,
+      "step": 3490
+    },
+    {
+      "epoch": 1.801441255951615,
+      "grad_norm": 0.04408276826143265,
+      "learning_rate": 5.983033704273355e-07,
+      "loss": 0.0075,
+      "step": 3500
+    },
+    {
+      "epoch": 1.8065885986359542,
+      "grad_norm": 0.04385414719581604,
+      "learning_rate": 5.680751779327742e-07,
+      "loss": 0.0074,
+      "step": 3510
+    },
+    {
+      "epoch": 1.8117359413202934,
+      "grad_norm": 0.04157879576086998,
+      "learning_rate": 5.386081979016256e-07,
+      "loss": 0.0072,
+      "step": 3520
+    },
+    {
+      "epoch": 1.8168832840046325,
+      "grad_norm": 0.044605404138565063,
+      "learning_rate": 5.099048084945401e-07,
+      "loss": 0.0071,
+      "step": 3530
+    },
+    {
+      "epoch": 1.8220306266889719,
+      "grad_norm": 0.04070171341300011,
+      "learning_rate": 4.819673262458591e-07,
+      "loss": 0.0072,
+      "step": 3540
+    },
+    {
+      "epoch": 1.827177969373311,
+      "grad_norm": 0.04037825018167496,
+      "learning_rate": 4.547980058766566e-07,
+      "loss": 0.0071,
+      "step": 3550
+    },
+    {
+      "epoch": 1.8323253120576504,
+      "grad_norm": 0.039735108613967896,
+      "learning_rate": 4.283990401127769e-07,
+      "loss": 0.007,
+      "step": 3560
+    },
+    {
+      "epoch": 1.8374726547419895,
+      "grad_norm": 0.0628032460808754,
+      "learning_rate": 4.027725595078591e-07,
+      "loss": 0.0071,
+      "step": 3570
+    },
+    {
+      "epoch": 1.8426199974263286,
+      "grad_norm": 0.040935881435871124,
+      "learning_rate": 3.779206322713913e-07,
+      "loss": 0.0074,
+      "step": 3580
+    },
+    {
+      "epoch": 1.8477673401106678,
+      "grad_norm": 0.043412309139966965,
+      "learning_rate": 3.5384526410180085e-07,
+      "loss": 0.0071,
+      "step": 3590
+    },
+    {
+      "epoch": 1.852914682795007,
+      "grad_norm": 0.044377975165843964,
+      "learning_rate": 3.305483980245716e-07,
+      "loss": 0.0075,
+      "step": 3600
+    },
+    {
+      "epoch": 1.8580620254793463,
+      "grad_norm": 0.038890235126018524,
+      "learning_rate": 3.080319142354393e-07,
+      "loss": 0.0069,
+      "step": 3610
+    },
+    {
+      "epoch": 1.8632093681636857,
+      "grad_norm": 0.04343342036008835,
+      "learning_rate": 2.8629762994864195e-07,
+      "loss": 0.0067,
+      "step": 3620
+    },
+    {
+      "epoch": 1.8683567108480248,
+      "grad_norm": 0.04095729440450668,
+      "learning_rate": 2.653472992502648e-07,
+      "loss": 0.0071,
+      "step": 3630
+    },
+    {
+      "epoch": 1.873504053532364,
+      "grad_norm": 0.04180029407143593,
+      "learning_rate": 2.4518261295667255e-07,
+      "loss": 0.0072,
+      "step": 3640
+    },
+    {
+      "epoch": 1.878651396216703,
+      "grad_norm": 0.04427026957273483,
+      "learning_rate": 2.2580519847805182e-07,
+      "loss": 0.0072,
+      "step": 3650
+    },
+    {
+      "epoch": 1.8837987389010422,
+      "grad_norm": 0.04394019767642021,
+      "learning_rate": 2.072166196870673e-07,
+      "loss": 0.0069,
+      "step": 3660
+    },
+    {
+      "epoch": 1.8889460815853816,
+      "grad_norm": 0.0422549843788147,
+      "learning_rate": 1.894183767926494e-07,
+      "loss": 0.0071,
+      "step": 3670
+    },
+    {
+      "epoch": 1.8940934242697207,
+      "grad_norm": 0.03970884904265404,
+      "learning_rate": 1.724119062189189e-07,
+      "loss": 0.007,
+      "step": 3680
+    },
+    {
+      "epoch": 1.89924076695406,
+      "grad_norm": 0.043532587587833405,
+      "learning_rate": 1.561985804892585e-07,
+      "loss": 0.0073,
+      "step": 3690
+    },
+    {
+      "epoch": 1.9043881096383992,
+      "grad_norm": 0.038174182176589966,
+      "learning_rate": 1.4077970811554153e-07,
+      "loss": 0.007,
+      "step": 3700
+    },
+    {
+      "epoch": 1.9095354523227384,
+      "grad_norm": 0.03825508803129196,
+      "learning_rate": 1.2615653349252743e-07,
+      "loss": 0.0074,
+      "step": 3710
+    },
+    {
+      "epoch": 1.9146827950070775,
+      "grad_norm": 0.04140858352184296,
+      "learning_rate": 1.1233023679743326e-07,
+      "loss": 0.0073,
+      "step": 3720
+    },
+    {
+      "epoch": 1.9198301376914169,
+      "grad_norm": 0.042516909539699554,
+      "learning_rate": 9.930193389468323e-08,
+      "loss": 0.007,
+      "step": 3730
+    },
+    {
+      "epoch": 1.924977480375756,
+      "grad_norm": 0.04423694312572479,
+      "learning_rate": 8.707267624585292e-08,
+      "loss": 0.0073,
+      "step": 3740
+    },
+    {
+      "epoch": 1.9301248230600954,
+      "grad_norm": 0.04059486463665962,
+      "learning_rate": 7.564345082481383e-08,
+      "loss": 0.0068,
+      "step": 3750
+    },
+    {
+      "epoch": 1.9352721657444345,
+      "grad_norm": 0.04558904469013214,
+      "learning_rate": 6.501518003807272e-08,
+      "loss": 0.0073,
+      "step": 3760
+    },
+    {
+      "epoch": 1.9404195084287736,
+      "grad_norm": 0.0388142392039299,
+      "learning_rate": 5.518872165033329e-08,
+      "loss": 0.0071,
+      "step": 3770
+    },
+    {
+      "epoch": 1.9455668511131128,
+      "grad_norm": 0.04608234018087387,
+      "learning_rate": 4.6164868715263825e-08,
+      "loss": 0.0073,
+      "step": 3780
+    },
+    {
+      "epoch": 1.950714193797452,
+      "grad_norm": 0.040703512728214264,
+      "learning_rate": 3.79443495114995e-08,
+      "loss": 0.0071,
+      "step": 3790
+    },
+    {
+      "epoch": 1.9558615364817913,
+      "grad_norm": 0.04268708452582359,
+      "learning_rate": 3.052782748386052e-08,
+      "loss": 0.0072,
+      "step": 3800
+    },
+    {
+      "epoch": 1.9610088791661306,
+      "grad_norm": 0.04412174969911575,
+      "learning_rate": 2.3915901189811576e-08,
+      "loss": 0.0069,
+      "step": 3810
+    },
+    {
+      "epoch": 1.9661562218504698,
+      "grad_norm": 0.0402398481965065,
+      "learning_rate": 1.8109104251151645e-08,
+      "loss": 0.0069,
+      "step": 3820
+    },
+    {
+      "epoch": 1.971303564534809,
+      "grad_norm": 0.04533049836754799,
+      "learning_rate": 1.310790531095063e-08,
+      "loss": 0.0067,
+      "step": 3830
+    },
+    {
+      "epoch": 1.976450907219148,
+      "grad_norm": 0.043354544788599014,
+      "learning_rate": 8.9127079957263e-09,
+      "loss": 0.0071,
+      "step": 3840
+    },
+    {
+      "epoch": 1.9815982499034872,
+      "grad_norm": 0.039460960775613785,
+      "learning_rate": 5.523850882866999e-09,
+      "loss": 0.0072,
+      "step": 3850
+    },
+    {
+      "epoch": 1.9867455925878266,
+      "grad_norm": 0.04111147299408913,
+      "learning_rate": 2.941607473311292e-09,
+      "loss": 0.0071,
+      "step": 3860
+    },
+    {
+      "epoch": 1.9918929352721657,
+      "grad_norm": 0.0409269742667675,
+      "learning_rate": 1.166186169466732e-09,
+      "loss": 0.0076,
+      "step": 3870
+    },
+    {
+      "epoch": 1.997040277956505,
+      "grad_norm": 0.036945004016160965,
+      "learning_rate": 1.9773025839997518e-10,
+      "loss": 0.0076,
+      "step": 3880
+    },
+    {
+      "epoch": 2.0,
+      "step": 3886,
+      "total_flos": 6.320832928623572e+19,
+      "train_loss": 0.009204940595273436,
+      "train_runtime": 219891.592,
+      "train_samples_per_second": 9.047,
+      "train_steps_per_second": 0.018
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 3886,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 2,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 6.320832928623572e+19,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

training_loss.png ADDED Viewed