Instructions to use WeiboAI/VibeThinker-3B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use WeiboAI/VibeThinker-3B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="WeiboAI/VibeThinker-3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForMultimodalLM

tokenizer = AutoTokenizer.from_pretrained("WeiboAI/VibeThinker-3B")
model = AutoModelForMultimodalLM.from_pretrained("WeiboAI/VibeThinker-3B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use WeiboAI/VibeThinker-3B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "WeiboAI/VibeThinker-3B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WeiboAI/VibeThinker-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/WeiboAI/VibeThinker-3B

SGLang

How to use WeiboAI/VibeThinker-3B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "WeiboAI/VibeThinker-3B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WeiboAI/VibeThinker-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "WeiboAI/VibeThinker-3B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WeiboAI/VibeThinker-3B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use WeiboAI/VibeThinker-3B with Docker Model Runner:
```
docker model run hf.co/WeiboAI/VibeThinker-3B
```

VibeThinker-3B / trainer_state.json

YinZhiBin

Upload VibeThinker-3B model files

d788e9b verified 2 days ago

Raw

History Blame Contribute Delete

122 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 4.0,
	"eval_steps": 500,
	"global_step": 704,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.005681818181818182,
	"grad_norm": 0.6539334923896951,
	"learning_rate": 0.0,
	"loss": 0.7303,
	"step": 1
	},
	{
	"epoch": 0.011363636363636364,
	"grad_norm": 0.7362861971573373,
	"learning_rate": 2.2727272727272729e-07,
	"loss": 0.791,
	"step": 2
	},
	{
	"epoch": 0.017045454545454544,
	"grad_norm": 0.7161466289578443,
	"learning_rate": 4.5454545454545457e-07,
	"loss": 0.7752,
	"step": 3
	},
	{
	"epoch": 0.022727272727272728,
	"grad_norm": 0.6612023910349915,
	"learning_rate": 6.818181818181818e-07,
	"loss": 0.796,
	"step": 4
	},
	{
	"epoch": 0.028409090909090908,
	"grad_norm": 0.6357445627946944,
	"learning_rate": 9.090909090909091e-07,
	"loss": 0.7376,
	"step": 5
	},
	{
	"epoch": 0.03409090909090909,
	"grad_norm": 0.6755782649331866,
	"learning_rate": 1.1363636363636364e-06,
	"loss": 0.7405,
	"step": 6
	},
	{
	"epoch": 0.03977272727272727,
	"grad_norm": 0.6211153562695977,
	"learning_rate": 1.3636363636363636e-06,
	"loss": 0.7387,
	"step": 7
	},
	{
	"epoch": 0.045454545454545456,
	"grad_norm": 0.6661789022359752,
	"learning_rate": 1.590909090909091e-06,
	"loss": 0.7514,
	"step": 8
	},
	{
	"epoch": 0.05113636363636364,
	"grad_norm": 0.6177949136554609,
	"learning_rate": 1.8181818181818183e-06,
	"loss": 0.7747,
	"step": 9
	},
	{
	"epoch": 0.056818181818181816,
	"grad_norm": 0.5778981092595771,
	"learning_rate": 2.0454545454545457e-06,
	"loss": 0.7499,
	"step": 10
	},
	{
	"epoch": 0.0625,
	"grad_norm": 0.5524784357129678,
	"learning_rate": 2.2727272727272728e-06,
	"loss": 0.7556,
	"step": 11
	},
	{
	"epoch": 0.06818181818181818,
	"grad_norm": 0.5522955577138747,
	"learning_rate": 2.5e-06,
	"loss": 0.7691,
	"step": 12
	},
	{
	"epoch": 0.07386363636363637,
	"grad_norm": 0.4429302688913553,
	"learning_rate": 2.7272727272727272e-06,
	"loss": 0.7215,
	"step": 13
	},
	{
	"epoch": 0.07954545454545454,
	"grad_norm": 0.4177273616201185,
	"learning_rate": 2.954545454545455e-06,
	"loss": 0.7595,
	"step": 14
	},
	{
	"epoch": 0.08522727272727272,
	"grad_norm": 0.3771885123401612,
	"learning_rate": 3.181818181818182e-06,
	"loss": 0.7252,
	"step": 15
	},
	{
	"epoch": 0.09090909090909091,
	"grad_norm": 0.40058969410351514,
	"learning_rate": 3.409090909090909e-06,
	"loss": 0.75,
	"step": 16
	},
	{
	"epoch": 0.09659090909090909,
	"grad_norm": 0.3893262571109241,
	"learning_rate": 3.6363636363636366e-06,
	"loss": 0.7521,
	"step": 17
	},
	{
	"epoch": 0.10227272727272728,
	"grad_norm": 0.4240034029133202,
	"learning_rate": 3.863636363636364e-06,
	"loss": 0.7758,
	"step": 18
	},
	{
	"epoch": 0.10795454545454546,
	"grad_norm": 0.4019100085908253,
	"learning_rate": 4.0909090909090915e-06,
	"loss": 0.7579,
	"step": 19
	},
	{
	"epoch": 0.11363636363636363,
	"grad_norm": 0.42981758870904435,
	"learning_rate": 4.3181818181818185e-06,
	"loss": 0.7639,
	"step": 20
	},
	{
	"epoch": 0.11931818181818182,
	"grad_norm": 0.51337588681513,
	"learning_rate": 4.5454545454545455e-06,
	"loss": 0.7645,
	"step": 21
	},
	{
	"epoch": 0.125,
	"grad_norm": 0.4395409517891011,
	"learning_rate": 4.772727272727273e-06,
	"loss": 0.7644,
	"step": 22
	},
	{
	"epoch": 0.13068181818181818,
	"grad_norm": 0.4289913207985502,
	"learning_rate": 5e-06,
	"loss": 0.757,
	"step": 23
	},
	{
	"epoch": 0.13636363636363635,
	"grad_norm": 0.46134611062777053,
	"learning_rate": 5.2272727272727274e-06,
	"loss": 0.7995,
	"step": 24
	},
	{
	"epoch": 0.14204545454545456,
	"grad_norm": 0.43847023885397474,
	"learning_rate": 5.4545454545454545e-06,
	"loss": 0.75,
	"step": 25
	},
	{
	"epoch": 0.14772727272727273,
	"grad_norm": 0.32320162542498704,
	"learning_rate": 5.681818181818183e-06,
	"loss": 0.7598,
	"step": 26
	},
	{
	"epoch": 0.1534090909090909,
	"grad_norm": 0.39127601413927054,
	"learning_rate": 5.90909090909091e-06,
	"loss": 0.7823,
	"step": 27
	},
	{
	"epoch": 0.1590909090909091,
	"grad_norm": 0.31227571661934783,
	"learning_rate": 6.136363636363637e-06,
	"loss": 0.7525,
	"step": 28
	},
	{
	"epoch": 0.16477272727272727,
	"grad_norm": 0.34430216268136543,
	"learning_rate": 6.363636363636364e-06,
	"loss": 0.7264,
	"step": 29
	},
	{
	"epoch": 0.17045454545454544,
	"grad_norm": 0.32443224162841167,
	"learning_rate": 6.590909090909091e-06,
	"loss": 0.765,
	"step": 30
	},
	{
	"epoch": 0.17613636363636365,
	"grad_norm": 0.3051787985519309,
	"learning_rate": 6.818181818181818e-06,
	"loss": 0.7395,
	"step": 31
	},
	{
	"epoch": 0.18181818181818182,
	"grad_norm": 0.31563281676103977,
	"learning_rate": 7.045454545454546e-06,
	"loss": 0.7045,
	"step": 32
	},
	{
	"epoch": 0.1875,
	"grad_norm": 0.3356677925995578,
	"learning_rate": 7.272727272727273e-06,
	"loss": 0.7262,
	"step": 33
	},
	{
	"epoch": 0.19318181818181818,
	"grad_norm": 0.27327166597526453,
	"learning_rate": 7.500000000000001e-06,
	"loss": 0.7514,
	"step": 34
	},
	{
	"epoch": 0.19886363636363635,
	"grad_norm": 0.31022754668150926,
	"learning_rate": 7.727272727272727e-06,
	"loss": 0.7505,
	"step": 35
	},
	{
	"epoch": 0.20454545454545456,
	"grad_norm": 0.27781199140261953,
	"learning_rate": 7.954545454545455e-06,
	"loss": 0.7505,
	"step": 36
	},
	{
	"epoch": 0.21022727272727273,
	"grad_norm": 0.2623337663808676,
	"learning_rate": 8.181818181818183e-06,
	"loss": 0.7518,
	"step": 37
	},
	{
	"epoch": 0.2159090909090909,
	"grad_norm": 0.27645930308342953,
	"learning_rate": 8.40909090909091e-06,
	"loss": 0.7607,
	"step": 38
	},
	{
	"epoch": 0.2215909090909091,
	"grad_norm": 0.2679059151688725,
	"learning_rate": 8.636363636363637e-06,
	"loss": 0.7673,
	"step": 39
	},
	{
	"epoch": 0.22727272727272727,
	"grad_norm": 0.2586016761066041,
	"learning_rate": 8.863636363636365e-06,
	"loss": 0.7222,
	"step": 40
	},
	{
	"epoch": 0.23295454545454544,
	"grad_norm": 0.29451162430265443,
	"learning_rate": 9.090909090909091e-06,
	"loss": 0.7293,
	"step": 41
	},
	{
	"epoch": 0.23863636363636365,
	"grad_norm": 0.25409403633283223,
	"learning_rate": 9.318181818181819e-06,
	"loss": 0.7036,
	"step": 42
	},
	{
	"epoch": 0.24431818181818182,
	"grad_norm": 0.2634137220146159,
	"learning_rate": 9.545454545454547e-06,
	"loss": 0.7075,
	"step": 43
	},
	{
	"epoch": 0.25,
	"grad_norm": 0.2540592656236238,
	"learning_rate": 9.772727272727273e-06,
	"loss": 0.7276,
	"step": 44
	},
	{
	"epoch": 0.2556818181818182,
	"grad_norm": 0.2627808401472948,
	"learning_rate": 1e-05,
	"loss": 0.7215,
	"step": 45
	},
	{
	"epoch": 0.26136363636363635,
	"grad_norm": 0.2455732295364075,
	"learning_rate": 1.0227272727272729e-05,
	"loss": 0.7191,
	"step": 46
	},
	{
	"epoch": 0.26704545454545453,
	"grad_norm": 0.25527128850655173,
	"learning_rate": 1.0454545454545455e-05,
	"loss": 0.7208,
	"step": 47
	},
	{
	"epoch": 0.2727272727272727,
	"grad_norm": 0.25066863183778093,
	"learning_rate": 1.0681818181818183e-05,
	"loss": 0.7418,
	"step": 48
	},
	{
	"epoch": 0.2784090909090909,
	"grad_norm": 0.2500040412250409,
	"learning_rate": 1.0909090909090909e-05,
	"loss": 0.7607,
	"step": 49
	},
	{
	"epoch": 0.2840909090909091,
	"grad_norm": 0.2554332076422905,
	"learning_rate": 1.1136363636363637e-05,
	"loss": 0.7614,
	"step": 50
	},
	{
	"epoch": 0.2897727272727273,
	"grad_norm": 0.2540573323466542,
	"learning_rate": 1.1363636363636366e-05,
	"loss": 0.7552,
	"step": 51
	},
	{
	"epoch": 0.29545454545454547,
	"grad_norm": 0.24152725346501944,
	"learning_rate": 1.1590909090909093e-05,
	"loss": 0.7371,
	"step": 52
	},
	{
	"epoch": 0.30113636363636365,
	"grad_norm": 0.24748178254735026,
	"learning_rate": 1.181818181818182e-05,
	"loss": 0.7277,
	"step": 53
	},
	{
	"epoch": 0.3068181818181818,
	"grad_norm": 0.21712849459903497,
	"learning_rate": 1.2045454545454547e-05,
	"loss": 0.7451,
	"step": 54
	},
	{
	"epoch": 0.3125,
	"grad_norm": 0.24646941350994314,
	"learning_rate": 1.2272727272727274e-05,
	"loss": 0.735,
	"step": 55
	},
	{
	"epoch": 0.3181818181818182,
	"grad_norm": 0.2512290694169805,
	"learning_rate": 1.25e-05,
	"loss": 0.761,
	"step": 56
	},
	{
	"epoch": 0.32386363636363635,
	"grad_norm": 0.2461067349529032,
	"learning_rate": 1.2727272727272728e-05,
	"loss": 0.7272,
	"step": 57
	},
	{
	"epoch": 0.32954545454545453,
	"grad_norm": 0.23374263228979594,
	"learning_rate": 1.2954545454545455e-05,
	"loss": 0.7345,
	"step": 58
	},
	{
	"epoch": 0.3352272727272727,
	"grad_norm": 0.2457367427804197,
	"learning_rate": 1.3181818181818183e-05,
	"loss": 0.7307,
	"step": 59
	},
	{
	"epoch": 0.3409090909090909,
	"grad_norm": 0.22217836170777067,
	"learning_rate": 1.3409090909090909e-05,
	"loss": 0.7459,
	"step": 60
	},
	{
	"epoch": 0.3465909090909091,
	"grad_norm": 0.2684616021449928,
	"learning_rate": 1.3636363636363637e-05,
	"loss": 0.7249,
	"step": 61
	},
	{
	"epoch": 0.3522727272727273,
	"grad_norm": 0.26482500740784093,
	"learning_rate": 1.3863636363636366e-05,
	"loss": 0.7753,
	"step": 62
	},
	{
	"epoch": 0.35795454545454547,
	"grad_norm": 0.22037094695010664,
	"learning_rate": 1.4090909090909092e-05,
	"loss": 0.708,
	"step": 63
	},
	{
	"epoch": 0.36363636363636365,
	"grad_norm": 0.21741108669857576,
	"learning_rate": 1.431818181818182e-05,
	"loss": 0.7242,
	"step": 64
	},
	{
	"epoch": 0.3693181818181818,
	"grad_norm": 0.22729261341964987,
	"learning_rate": 1.4545454545454546e-05,
	"loss": 0.7192,
	"step": 65
	},
	{
	"epoch": 0.375,
	"grad_norm": 0.22609367717066553,
	"learning_rate": 1.4772727272727274e-05,
	"loss": 0.7346,
	"step": 66
	},
	{
	"epoch": 0.3806818181818182,
	"grad_norm": 0.2517917272998053,
	"learning_rate": 1.5000000000000002e-05,
	"loss": 0.7053,
	"step": 67
	},
	{
	"epoch": 0.38636363636363635,
	"grad_norm": 0.23575647329775867,
	"learning_rate": 1.5227272727272728e-05,
	"loss": 0.7139,
	"step": 68
	},
	{
	"epoch": 0.39204545454545453,
	"grad_norm": 0.2605621593407242,
	"learning_rate": 1.5454545454545454e-05,
	"loss": 0.7079,
	"step": 69
	},
	{
	"epoch": 0.3977272727272727,
	"grad_norm": 0.24154405450856214,
	"learning_rate": 1.5681818181818182e-05,
	"loss": 0.7224,
	"step": 70
	},
	{
	"epoch": 0.4034090909090909,
	"grad_norm": 0.2697097736642708,
	"learning_rate": 1.590909090909091e-05,
	"loss": 0.7315,
	"step": 71
	},
	{
	"epoch": 0.4090909090909091,
	"grad_norm": 0.22549274558442894,
	"learning_rate": 1.6136363636363638e-05,
	"loss": 0.7214,
	"step": 72
	},
	{
	"epoch": 0.4147727272727273,
	"grad_norm": 0.23365342578387163,
	"learning_rate": 1.6363636363636366e-05,
	"loss": 0.7342,
	"step": 73
	},
	{
	"epoch": 0.42045454545454547,
	"grad_norm": 0.25682029083953883,
	"learning_rate": 1.6590909090909094e-05,
	"loss": 0.7466,
	"step": 74
	},
	{
	"epoch": 0.42613636363636365,
	"grad_norm": 0.24880268273648679,
	"learning_rate": 1.681818181818182e-05,
	"loss": 0.7417,
	"step": 75
	},
	{
	"epoch": 0.4318181818181818,
	"grad_norm": 0.22354489882525846,
	"learning_rate": 1.7045454545454546e-05,
	"loss": 0.7416,
	"step": 76
	},
	{
	"epoch": 0.4375,
	"grad_norm": 0.21959893112636297,
	"learning_rate": 1.7272727272727274e-05,
	"loss": 0.7451,
	"step": 77
	},
	{
	"epoch": 0.4431818181818182,
	"grad_norm": 0.22505968595761988,
	"learning_rate": 1.7500000000000002e-05,
	"loss": 0.7064,
	"step": 78
	},
	{
	"epoch": 0.44886363636363635,
	"grad_norm": 0.2415063898841816,
	"learning_rate": 1.772727272727273e-05,
	"loss": 0.7535,
	"step": 79
	},
	{
	"epoch": 0.45454545454545453,
	"grad_norm": 0.2440124982709127,
	"learning_rate": 1.7954545454545454e-05,
	"loss": 0.7722,
	"step": 80
	},
	{
	"epoch": 0.4602272727272727,
	"grad_norm": 0.27227864126562756,
	"learning_rate": 1.8181818181818182e-05,
	"loss": 0.7378,
	"step": 81
	},
	{
	"epoch": 0.4659090909090909,
	"grad_norm": 0.2533070274173587,
	"learning_rate": 1.840909090909091e-05,
	"loss": 0.7326,
	"step": 82
	},
	{
	"epoch": 0.4715909090909091,
	"grad_norm": 0.2801926646527799,
	"learning_rate": 1.8636363636363638e-05,
	"loss": 0.7149,
	"step": 83
	},
	{
	"epoch": 0.4772727272727273,
	"grad_norm": 0.23284641010919663,
	"learning_rate": 1.8863636363636366e-05,
	"loss": 0.7687,
	"step": 84
	},
	{
	"epoch": 0.48295454545454547,
	"grad_norm": 0.22858799800301324,
	"learning_rate": 1.9090909090909094e-05,
	"loss": 0.7246,
	"step": 85
	},
	{
	"epoch": 0.48863636363636365,
	"grad_norm": 0.2680207710064355,
	"learning_rate": 1.931818181818182e-05,
	"loss": 0.7304,
	"step": 86
	},
	{
	"epoch": 0.4943181818181818,
	"grad_norm": 0.28289497574550504,
	"learning_rate": 1.9545454545454546e-05,
	"loss": 0.7221,
	"step": 87
	},
	{
	"epoch": 0.5,
	"grad_norm": 0.2330908800500943,
	"learning_rate": 1.9772727272727274e-05,
	"loss": 0.7269,
	"step": 88
	},
	{
	"epoch": 0.5056818181818182,
	"grad_norm": 0.21528663885202565,
	"learning_rate": 2e-05,
	"loss": 0.7296,
	"step": 89
	},
	{
	"epoch": 0.5113636363636364,
	"grad_norm": 0.24349634478751364,
	"learning_rate": 1.9999921406920624e-05,
	"loss": 0.732,
	"step": 90
	},
	{
	"epoch": 0.5170454545454546,
	"grad_norm": 0.24946797023957654,
	"learning_rate": 1.99996856289191e-05,
	"loss": 0.748,
	"step": 91
	},
	{
	"epoch": 0.5227272727272727,
	"grad_norm": 0.2460157070303222,
	"learning_rate": 1.9999292669705243e-05,
	"loss": 0.745,
	"step": 92
	},
	{
	"epoch": 0.5284090909090909,
	"grad_norm": 0.23974363562452397,
	"learning_rate": 1.9998742535462015e-05,
	"loss": 0.7303,
	"step": 93
	},
	{
	"epoch": 0.5340909090909091,
	"grad_norm": 0.2447891892794692,
	"learning_rate": 1.9998035234845414e-05,
	"loss": 0.7191,
	"step": 94
	},
	{
	"epoch": 0.5397727272727273,
	"grad_norm": 0.23792748234451203,
	"learning_rate": 1.999717077898436e-05,
	"loss": 0.743,
	"step": 95
	},
	{
	"epoch": 0.5454545454545454,
	"grad_norm": 0.2620667294397206,
	"learning_rate": 1.9996149181480506e-05,
	"loss": 0.7811,
	"step": 96
	},
	{
	"epoch": 0.5511363636363636,
	"grad_norm": 0.24957242146448744,
	"learning_rate": 1.999497045840802e-05,
	"loss": 0.8047,
	"step": 97
	},
	{
	"epoch": 0.5568181818181818,
	"grad_norm": 0.24493775100284407,
	"learning_rate": 1.999363462831335e-05,
	"loss": 0.7408,
	"step": 98
	},
	{
	"epoch": 0.5625,
	"grad_norm": 0.25788850223987986,
	"learning_rate": 1.9992141712214908e-05,
	"loss": 0.7402,
	"step": 99
	},
	{
	"epoch": 0.5681818181818182,
	"grad_norm": 0.2268504545061795,
	"learning_rate": 1.999049173360276e-05,
	"loss": 0.7426,
	"step": 100
	},
	{
	"epoch": 0.5738636363636364,
	"grad_norm": 0.24698918238591733,
	"learning_rate": 1.9988684718438252e-05,
	"loss": 0.7759,
	"step": 101
	},
	{
	"epoch": 0.5795454545454546,
	"grad_norm": 0.25387546769444963,
	"learning_rate": 1.9986720695153588e-05,
	"loss": 0.7507,
	"step": 102
	},
	{
	"epoch": 0.5852272727272727,
	"grad_norm": 0.21576778898157,
	"learning_rate": 1.9984599694651404e-05,
	"loss": 0.7319,
	"step": 103
	},
	{
	"epoch": 0.5909090909090909,
	"grad_norm": 0.22207749833387033,
	"learning_rate": 1.9982321750304256e-05,
	"loss": 0.744,
	"step": 104
	},
	{
	"epoch": 0.5965909090909091,
	"grad_norm": 0.2588353719031421,
	"learning_rate": 1.9979886897954125e-05,
	"loss": 0.7394,
	"step": 105
	},
	{
	"epoch": 0.6022727272727273,
	"grad_norm": 0.21537827080360136,
	"learning_rate": 1.9977295175911832e-05,
	"loss": 0.7452,
	"step": 106
	},
	{
	"epoch": 0.6079545454545454,
	"grad_norm": 0.25611308453281817,
	"learning_rate": 1.9974546624956435e-05,
	"loss": 0.7534,
	"step": 107
	},
	{
	"epoch": 0.6136363636363636,
	"grad_norm": 0.24080492675977055,
	"learning_rate": 1.99716412883346e-05,
	"loss": 0.7457,
	"step": 108
	},
	{
	"epoch": 0.6193181818181818,
	"grad_norm": 0.24811942574106713,
	"learning_rate": 1.9968579211759907e-05,
	"loss": 0.7408,
	"step": 109
	},
	{
	"epoch": 0.625,
	"grad_norm": 0.23946442659249176,
	"learning_rate": 1.9965360443412143e-05,
	"loss": 0.7518,
	"step": 110
	},
	{
	"epoch": 0.6306818181818182,
	"grad_norm": 0.2608612543256031,
	"learning_rate": 1.996198503393654e-05,
	"loss": 0.7778,
	"step": 111
	},
	{
	"epoch": 0.6363636363636364,
	"grad_norm": 0.21775672753130904,
	"learning_rate": 1.995845303644297e-05,
	"loss": 0.6716,
	"step": 112
	},
	{
	"epoch": 0.6420454545454546,
	"grad_norm": 0.22671833402266955,
	"learning_rate": 1.995476450650512e-05,
	"loss": 0.7302,
	"step": 113
	},
	{
	"epoch": 0.6477272727272727,
	"grad_norm": 0.2603863357434813,
	"learning_rate": 1.9950919502159608e-05,
	"loss": 0.7318,
	"step": 114
	},
	{
	"epoch": 0.6534090909090909,
	"grad_norm": 0.26505433111911075,
	"learning_rate": 1.994691808390508e-05,
	"loss": 0.7357,
	"step": 115
	},
	{
	"epoch": 0.6590909090909091,
	"grad_norm": 0.23050888218913693,
	"learning_rate": 1.9942760314701257e-05,
	"loss": 0.756,
	"step": 116
	},
	{
	"epoch": 0.6647727272727273,
	"grad_norm": 0.23286827785494774,
	"learning_rate": 1.9938446259967928e-05,
	"loss": 0.7399,
	"step": 117
	},
	{
	"epoch": 0.6704545454545454,
	"grad_norm": 0.2506590530312723,
	"learning_rate": 1.9933975987583954e-05,
	"loss": 0.7661,
	"step": 118
	},
	{
	"epoch": 0.6761363636363636,
	"grad_norm": 0.23298066567805087,
	"learning_rate": 1.992934956788615e-05,
	"loss": 0.7554,
	"step": 119
	},
	{
	"epoch": 0.6818181818181818,
	"grad_norm": 0.25503855445575174,
	"learning_rate": 1.9924567073668236e-05,
	"loss": 0.7515,
	"step": 120
	},
	{
	"epoch": 0.6875,
	"grad_norm": 0.24488517121594475,
	"learning_rate": 1.9919628580179647e-05,
	"loss": 0.7034,
	"step": 121
	},
	{
	"epoch": 0.6931818181818182,
	"grad_norm": 0.2574174897942862,
	"learning_rate": 1.9914534165124367e-05,
	"loss": 0.7649,
	"step": 122
	},
	{
	"epoch": 0.6988636363636364,
	"grad_norm": 0.23872990732826432,
	"learning_rate": 1.990928390865971e-05,
	"loss": 0.722,
	"step": 123
	},
	{
	"epoch": 0.7045454545454546,
	"grad_norm": 0.2652594681578002,
	"learning_rate": 1.9903877893395046e-05,
	"loss": 0.7176,
	"step": 124
	},
	{
	"epoch": 0.7102272727272727,
	"grad_norm": 0.24767695313847904,
	"learning_rate": 1.989831620439052e-05,
	"loss": 0.7397,
	"step": 125
	},
	{
	"epoch": 0.7159090909090909,
	"grad_norm": 0.2304739081322821,
	"learning_rate": 1.9892598929155686e-05,
	"loss": 0.699,
	"step": 126
	},
	{
	"epoch": 0.7215909090909091,
	"grad_norm": 0.25576584510320144,
	"learning_rate": 1.988672615764816e-05,
	"loss": 0.7729,
	"step": 127
	},
	{
	"epoch": 0.7272727272727273,
	"grad_norm": 0.21530066363340297,
	"learning_rate": 1.9880697982272182e-05,
	"loss": 0.7366,
	"step": 128
	},
	{
	"epoch": 0.7329545454545454,
	"grad_norm": 0.21496223861980898,
	"learning_rate": 1.987451449787718e-05,
	"loss": 0.7283,
	"step": 129
	},
	{
	"epoch": 0.7386363636363636,
	"grad_norm": 0.22317164121017025,
	"learning_rate": 1.9868175801756264e-05,
	"loss": 0.7324,
	"step": 130
	},
	{
	"epoch": 0.7443181818181818,
	"grad_norm": 0.21301735312819622,
	"learning_rate": 1.9861681993644685e-05,
	"loss": 0.7339,
	"step": 131
	},
	{
	"epoch": 0.75,
	"grad_norm": 0.27777692494345224,
	"learning_rate": 1.985503317571831e-05,
	"loss": 0.7125,
	"step": 132
	},
	{
	"epoch": 0.7556818181818182,
	"grad_norm": 0.22971874145882867,
	"learning_rate": 1.984822945259196e-05,
	"loss": 0.7411,
	"step": 133
	},
	{
	"epoch": 0.7613636363636364,
	"grad_norm": 0.239723168388915,
	"learning_rate": 1.98412709313178e-05,
	"loss": 0.7696,
	"step": 134
	},
	{
	"epoch": 0.7670454545454546,
	"grad_norm": 0.2537411388295359,
	"learning_rate": 1.9834157721383646e-05,
	"loss": 0.7208,
	"step": 135
	},
	{
	"epoch": 0.7727272727272727,
	"grad_norm": 0.21859611673997784,
	"learning_rate": 1.9826889934711223e-05,
	"loss": 0.7127,
	"step": 136
	},
	{
	"epoch": 0.7784090909090909,
	"grad_norm": 0.25462148717362965,
	"learning_rate": 1.981946768565444e-05,
	"loss": 0.7254,
	"step": 137
	},
	{
	"epoch": 0.7840909090909091,
	"grad_norm": 0.22700318445173676,
	"learning_rate": 1.9811891090997567e-05,
	"loss": 0.7368,
	"step": 138
	},
	{
	"epoch": 0.7897727272727273,
	"grad_norm": 0.23366974243945487,
	"learning_rate": 1.9804160269953385e-05,
	"loss": 0.7597,
	"step": 139
	},
	{
	"epoch": 0.7954545454545454,
	"grad_norm": 0.21872118677377747,
	"learning_rate": 1.979627534416135e-05,
	"loss": 0.7175,
	"step": 140
	},
	{
	"epoch": 0.8011363636363636,
	"grad_norm": 0.247438491157152,
	"learning_rate": 1.978823643768564e-05,
	"loss": 0.7287,
	"step": 141
	},
	{
	"epoch": 0.8068181818181818,
	"grad_norm": 0.23499325541824231,
	"learning_rate": 1.9780043677013233e-05,
	"loss": 0.7441,
	"step": 142
	},
	{
	"epoch": 0.8125,
	"grad_norm": 0.26493606388847807,
	"learning_rate": 1.9771697191051887e-05,
	"loss": 0.7425,
	"step": 143
	},
	{
	"epoch": 0.8181818181818182,
	"grad_norm": 0.24740040115604717,
	"learning_rate": 1.9763197111128137e-05,
	"loss": 0.7239,
	"step": 144
	},
	{
	"epoch": 0.8238636363636364,
	"grad_norm": 0.2329746775584324,
	"learning_rate": 1.9754543570985218e-05,
	"loss": 0.7501,
	"step": 145
	},
	{
	"epoch": 0.8295454545454546,
	"grad_norm": 0.3288373802185708,
	"learning_rate": 1.974573670678096e-05,
	"loss": 0.7776,
	"step": 146
	},
	{
	"epoch": 0.8352272727272727,
	"grad_norm": 0.22042149429151925,
	"learning_rate": 1.9736776657085655e-05,
	"loss": 0.7415,
	"step": 147
	},
	{
	"epoch": 0.8409090909090909,
	"grad_norm": 0.257622489260105,
	"learning_rate": 1.9727663562879858e-05,
	"loss": 0.7319,
	"step": 148
	},
	{
	"epoch": 0.8465909090909091,
	"grad_norm": 0.26380004698577236,
	"learning_rate": 1.9718397567552183e-05,
	"loss": 0.7171,
	"step": 149
	},
	{
	"epoch": 0.8522727272727273,
	"grad_norm": 0.5258645999158631,
	"learning_rate": 1.970897881689705e-05,
	"loss": 0.7401,
	"step": 150
	},
	{
	"epoch": 0.8579545454545454,
	"grad_norm": 0.2358177160647429,
	"learning_rate": 1.9699407459112375e-05,
	"loss": 0.7467,
	"step": 151
	},
	{
	"epoch": 0.8636363636363636,
	"grad_norm": 0.21270404732689172,
	"learning_rate": 1.968968364479726e-05,
	"loss": 0.7212,
	"step": 152
	},
	{
	"epoch": 0.8693181818181818,
	"grad_norm": 0.21897205490499316,
	"learning_rate": 1.96798075269496e-05,
	"loss": 0.7423,
	"step": 153
	},
	{
	"epoch": 0.875,
	"grad_norm": 0.22541908161316654,
	"learning_rate": 1.9669779260963694e-05,
	"loss": 0.6994,
	"step": 154
	},
	{
	"epoch": 0.8806818181818182,
	"grad_norm": 0.2193393868128554,
	"learning_rate": 1.9659599004627793e-05,
	"loss": 0.755,
	"step": 155
	},
	{
	"epoch": 0.8863636363636364,
	"grad_norm": 0.21938068626252985,
	"learning_rate": 1.9649266918121616e-05,
	"loss": 0.7382,
	"step": 156
	},
	{
	"epoch": 0.8920454545454546,
	"grad_norm": 0.23292541897440228,
	"learning_rate": 1.9638783164013825e-05,
	"loss": 0.7229,
	"step": 157
	},
	{
	"epoch": 0.8977272727272727,
	"grad_norm": 0.24467184526288463,
	"learning_rate": 1.962814790725948e-05,
	"loss": 0.7461,
	"step": 158
	},
	{
	"epoch": 0.9034090909090909,
	"grad_norm": 0.23980595373195004,
	"learning_rate": 1.961736131519744e-05,
	"loss": 0.7215,
	"step": 159
	},
	{
	"epoch": 0.9090909090909091,
	"grad_norm": 0.215049987253943,
	"learning_rate": 1.960642355754772e-05,
	"loss": 0.7315,
	"step": 160
	},
	{
	"epoch": 0.9147727272727273,
	"grad_norm": 0.2308832469797205,
	"learning_rate": 1.9595334806408832e-05,
	"loss": 0.7657,
	"step": 161
	},
	{
	"epoch": 0.9204545454545454,
	"grad_norm": 0.2358227594429107,
	"learning_rate": 1.9584095236255064e-05,
	"loss": 0.7467,
	"step": 162
	},
	{
	"epoch": 0.9261363636363636,
	"grad_norm": 0.23878339754803615,
	"learning_rate": 1.9572705023933753e-05,
	"loss": 0.7605,
	"step": 163
	},
	{
	"epoch": 0.9318181818181818,
	"grad_norm": 0.20539739372657043,
	"learning_rate": 1.9561164348662488e-05,
	"loss": 0.7277,
	"step": 164
	},
	{
	"epoch": 0.9375,
	"grad_norm": 0.22801578236594316,
	"learning_rate": 1.95494733920263e-05,
	"loss": 0.7274,
	"step": 165
	},
	{
	"epoch": 0.9431818181818182,
	"grad_norm": 0.215982699623725,
	"learning_rate": 1.9537632337974787e-05,
	"loss": 0.7683,
	"step": 166
	},
	{
	"epoch": 0.9488636363636364,
	"grad_norm": 0.25960780964246777,
	"learning_rate": 1.9525641372819248e-05,
	"loss": 0.7411,
	"step": 167
	},
	{
	"epoch": 0.9545454545454546,
	"grad_norm": 0.26868415505321463,
	"learning_rate": 1.9513500685229724e-05,
	"loss": 0.754,
	"step": 168
	},
	{
	"epoch": 0.9602272727272727,
	"grad_norm": 0.2744420696848421,
	"learning_rate": 1.9501210466232047e-05,
	"loss": 0.7572,
	"step": 169
	},
	{
	"epoch": 0.9659090909090909,
	"grad_norm": 0.22015812515164712,
	"learning_rate": 1.9488770909204823e-05,
	"loss": 0.7278,
	"step": 170
	},
	{
	"epoch": 0.9715909090909091,
	"grad_norm": 0.25536803176832584,
	"learning_rate": 1.9476182209876406e-05,
	"loss": 0.7753,
	"step": 171
	},
	{
	"epoch": 0.9772727272727273,
	"grad_norm": 0.2645350016268184,
	"learning_rate": 1.9463444566321798e-05,
	"loss": 0.7139,
	"step": 172
	},
	{
	"epoch": 0.9829545454545454,
	"grad_norm": 0.22766347808488796,
	"learning_rate": 1.9450558178959537e-05,
	"loss": 0.7431,
	"step": 173
	},
	{
	"epoch": 0.9886363636363636,
	"grad_norm": 0.23073303338018075,
	"learning_rate": 1.9437523250548566e-05,
	"loss": 0.7181,
	"step": 174
	},
	{
	"epoch": 0.9943181818181818,
	"grad_norm": 0.24405702816221608,
	"learning_rate": 1.9424339986185007e-05,
	"loss": 0.7534,
	"step": 175
	},
	{
	"epoch": 1.0,
	"grad_norm": 0.22106804964821963,
	"learning_rate": 1.941100859329896e-05,
	"loss": 0.6965,
	"step": 176
	},
	{
	"epoch": 1.0056818181818181,
	"grad_norm": 0.3075328621892957,
	"learning_rate": 1.9397529281651226e-05,
	"loss": 0.7095,
	"step": 177
	},
	{
	"epoch": 1.0113636363636365,
	"grad_norm": 0.3074326960974043,
	"learning_rate": 1.938390226333002e-05,
	"loss": 0.6764,
	"step": 178
	},
	{
	"epoch": 1.0170454545454546,
	"grad_norm": 0.21560938740987112,
	"learning_rate": 1.9370127752747623e-05,
	"loss": 0.6811,
	"step": 179
	},
	{
	"epoch": 1.0227272727272727,
	"grad_norm": 0.2693235484256499,
	"learning_rate": 1.9356205966637e-05,
	"loss": 0.6927,
	"step": 180
	},
	{
	"epoch": 1.0284090909090908,
	"grad_norm": 0.274773219752413,
	"learning_rate": 1.934213712404842e-05,
	"loss": 0.7087,
	"step": 181
	},
	{
	"epoch": 1.0340909090909092,
	"grad_norm": 0.22907290010393921,
	"learning_rate": 1.932792144634597e-05,
	"loss": 0.7036,
	"step": 182
	},
	{
	"epoch": 1.0397727272727273,
	"grad_norm": 0.299829281456353,
	"learning_rate": 1.9313559157204106e-05,
	"loss": 0.6739,
	"step": 183
	},
	{
	"epoch": 1.0454545454545454,
	"grad_norm": 0.28070384466307274,
	"learning_rate": 1.9299050482604116e-05,
	"loss": 0.7122,
	"step": 184
	},
	{
	"epoch": 1.0511363636363635,
	"grad_norm": 0.25071511079956166,
	"learning_rate": 1.9284395650830566e-05,
	"loss": 0.6785,
	"step": 185
	},
	{
	"epoch": 1.0568181818181819,
	"grad_norm": 0.29018939256803955,
	"learning_rate": 1.9269594892467716e-05,
	"loss": 0.7199,
	"step": 186
	},
	{
	"epoch": 1.0625,
	"grad_norm": 0.24309962888338685,
	"learning_rate": 1.925464844039587e-05,
	"loss": 0.7137,
	"step": 187
	},
	{
	"epoch": 1.0681818181818181,
	"grad_norm": 0.2697926394543293,
	"learning_rate": 1.9239556529787756e-05,
	"loss": 0.6768,
	"step": 188
	},
	{
	"epoch": 1.0738636363636365,
	"grad_norm": 0.25152536982642754,
	"learning_rate": 1.922431939810477e-05,
	"loss": 0.6945,
	"step": 189
	},
	{
	"epoch": 1.0795454545454546,
	"grad_norm": 0.23082913478475675,
	"learning_rate": 1.920893728509328e-05,
	"loss": 0.7283,
	"step": 190
	},
	{
	"epoch": 1.0852272727272727,
	"grad_norm": 0.24855030464430225,
	"learning_rate": 1.919341043278084e-05,
	"loss": 0.6827,
	"step": 191
	},
	{
	"epoch": 1.0909090909090908,
	"grad_norm": 0.2313234616127128,
	"learning_rate": 1.9177739085472385e-05,
	"loss": 0.6866,
	"step": 192
	},
	{
	"epoch": 1.0965909090909092,
	"grad_norm": 0.2251811060865089,
	"learning_rate": 1.9161923489746376e-05,
	"loss": 0.7248,
	"step": 193
	},
	{
	"epoch": 1.1022727272727273,
	"grad_norm": 0.2393343709505617,
	"learning_rate": 1.9145963894450944e-05,
	"loss": 0.6961,
	"step": 194
	},
	{
	"epoch": 1.1079545454545454,
	"grad_norm": 0.2578547134649496,
	"learning_rate": 1.912986055069994e-05,
	"loss": 0.7144,
	"step": 195
	},
	{
	"epoch": 1.1136363636363635,
	"grad_norm": 0.2700184794194847,
	"learning_rate": 1.9113613711869018e-05,
	"loss": 0.6768,
	"step": 196
	},
	{
	"epoch": 1.1193181818181819,
	"grad_norm": 0.2542360178470223,
	"learning_rate": 1.9097223633591643e-05,
	"loss": 0.7116,
	"step": 197
	},
	{
	"epoch": 1.125,
	"grad_norm": 0.2414370942840572,
	"learning_rate": 1.9080690573755027e-05,
	"loss": 0.7137,
	"step": 198
	},
	{
	"epoch": 1.1306818181818181,
	"grad_norm": 0.2585882095705147,
	"learning_rate": 1.9064014792496135e-05,
	"loss": 0.6545,
	"step": 199
	},
	{
	"epoch": 1.1363636363636362,
	"grad_norm": 0.23468513874240207,
	"learning_rate": 1.9047196552197548e-05,
	"loss": 0.6414,
	"step": 200
	},
	{
	"epoch": 1.1420454545454546,
	"grad_norm": 0.2238281842728056,
	"learning_rate": 1.903023611748335e-05,
	"loss": 0.6939,
	"step": 201
	},
	{
	"epoch": 1.1477272727272727,
	"grad_norm": 0.24072842250537277,
	"learning_rate": 1.9013133755214955e-05,
	"loss": 0.7228,
	"step": 202
	},
	{
	"epoch": 1.1534090909090908,
	"grad_norm": 0.2526377656467856,
	"learning_rate": 1.8995889734486926e-05,
	"loss": 0.6801,
	"step": 203
	},
	{
	"epoch": 1.1590909090909092,
	"grad_norm": 0.2596001775762996,
	"learning_rate": 1.8978504326622727e-05,
	"loss": 0.6917,
	"step": 204
	},
	{
	"epoch": 1.1647727272727273,
	"grad_norm": 0.23270783644317258,
	"learning_rate": 1.8960977805170445e-05,
	"loss": 0.6983,
	"step": 205
	},
	{
	"epoch": 1.1704545454545454,
	"grad_norm": 0.2655585422471245,
	"learning_rate": 1.8943310445898516e-05,
	"loss": 0.7455,
	"step": 206
	},
	{
	"epoch": 1.1761363636363638,
	"grad_norm": 0.20553338334652346,
	"learning_rate": 1.8925502526791354e-05,
	"loss": 0.6958,
	"step": 207
	},
	{
	"epoch": 1.1818181818181819,
	"grad_norm": 0.23374990847887048,
	"learning_rate": 1.8907554328044994e-05,
	"loss": 0.701,
	"step": 208
	},
	{
	"epoch": 1.1875,
	"grad_norm": 0.23808518745375035,
	"learning_rate": 1.8889466132062685e-05,
	"loss": 0.6965,
	"step": 209
	},
	{
	"epoch": 1.1931818181818181,
	"grad_norm": 0.22859263506595168,
	"learning_rate": 1.8871238223450433e-05,
	"loss": 0.6945,
	"step": 210
	},
	{
	"epoch": 1.1988636363636362,
	"grad_norm": 0.2359807237871379,
	"learning_rate": 1.8852870889012542e-05,
	"loss": 0.696,
	"step": 211
	},
	{
	"epoch": 1.2045454545454546,
	"grad_norm": 0.22146505308955905,
	"learning_rate": 1.883436441774708e-05,
	"loss": 0.7121,
	"step": 212
	},
	{
	"epoch": 1.2102272727272727,
	"grad_norm": 0.22825649015879565,
	"learning_rate": 1.8815719100841345e-05,
	"loss": 0.6854,
	"step": 213
	},
	{
	"epoch": 1.2159090909090908,
	"grad_norm": 0.24160214306672273,
	"learning_rate": 1.8796935231667287e-05,
	"loss": 0.6962,
	"step": 214
	},
	{
	"epoch": 1.2215909090909092,
	"grad_norm": 0.23247718981071797,
	"learning_rate": 1.8778013105776886e-05,
	"loss": 0.6888,
	"step": 215
	},
	{
	"epoch": 1.2272727272727273,
	"grad_norm": 0.2296407938774432,
	"learning_rate": 1.8758953020897488e-05,
	"loss": 0.717,
	"step": 216
	},
	{
	"epoch": 1.2329545454545454,
	"grad_norm": 0.23668931518912045,
	"learning_rate": 1.8739755276927153e-05,
	"loss": 0.6625,
	"step": 217
	},
	{
	"epoch": 1.2386363636363638,
	"grad_norm": 0.2196067564982769,
	"learning_rate": 1.8720420175929905e-05,
	"loss": 0.6899,
	"step": 218
	},
	{
	"epoch": 1.2443181818181819,
	"grad_norm": 0.2167386012067358,
	"learning_rate": 1.8700948022130993e-05,
	"loss": 0.6848,
	"step": 219
	},
	{
	"epoch": 1.25,
	"grad_norm": 0.26054841448633276,
	"learning_rate": 1.8681339121912104e-05,
	"loss": 0.7163,
	"step": 220
	},
	{
	"epoch": 1.2556818181818181,
	"grad_norm": 0.217485443863006,
	"learning_rate": 1.8661593783806547e-05,
	"loss": 0.7094,
	"step": 221
	},
	{
	"epoch": 1.2613636363636362,
	"grad_norm": 0.25352308297980314,
	"learning_rate": 1.8641712318494377e-05,
	"loss": 0.6777,
	"step": 222
	},
	{
	"epoch": 1.2670454545454546,
	"grad_norm": 0.26676327814921935,
	"learning_rate": 1.8621695038797542e-05,
	"loss": 0.659,
	"step": 223
	},
	{
	"epoch": 1.2727272727272727,
	"grad_norm": 0.2445773037285135,
	"learning_rate": 1.860154225967493e-05,
	"loss": 0.711,
	"step": 224
	},
	{
	"epoch": 1.2784090909090908,
	"grad_norm": 0.2621241936269718,
	"learning_rate": 1.8581254298217423e-05,
	"loss": 0.7116,
	"step": 225
	},
	{
	"epoch": 1.2840909090909092,
	"grad_norm": 0.2278213779152499,
	"learning_rate": 1.856083147364291e-05,
	"loss": 0.7112,
	"step": 226
	},
	{
	"epoch": 1.2897727272727273,
	"grad_norm": 0.2761059812936266,
	"learning_rate": 1.854027410729127e-05,
	"loss": 0.6976,
	"step": 227
	},
	{
	"epoch": 1.2954545454545454,
	"grad_norm": 0.24201677354279896,
	"learning_rate": 1.8519582522619307e-05,
	"loss": 0.7312,
	"step": 228
	},
	{
	"epoch": 1.3011363636363638,
	"grad_norm": 0.22342014116661876,
	"learning_rate": 1.849875704519565e-05,
	"loss": 0.6526,
	"step": 229
	},
	{
	"epoch": 1.3068181818181819,
	"grad_norm": 0.24093678295358437,
	"learning_rate": 1.8477798002695664e-05,
	"loss": 0.7048,
	"step": 230
	},
	{
	"epoch": 1.3125,
	"grad_norm": 0.2320873629901675,
	"learning_rate": 1.845670572489626e-05,
	"loss": 0.6709,
	"step": 231
	},
	{
	"epoch": 1.3181818181818181,
	"grad_norm": 0.25065055721625384,
	"learning_rate": 1.8435480543670728e-05,
	"loss": 0.6812,
	"step": 232
	},
	{
	"epoch": 1.3238636363636362,
	"grad_norm": 0.24255538578581526,
	"learning_rate": 1.84141227929835e-05,
	"loss": 0.6908,
	"step": 233
	},
	{
	"epoch": 1.3295454545454546,
	"grad_norm": 0.25769505763856887,
	"learning_rate": 1.8392632808884906e-05,
	"loss": 0.6414,
	"step": 234
	},
	{
	"epoch": 1.3352272727272727,
	"grad_norm": 0.24007713957110563,
	"learning_rate": 1.8371010929505884e-05,
	"loss": 0.7265,
	"step": 235
	},
	{
	"epoch": 1.3409090909090908,
	"grad_norm": 0.24040848112708096,
	"learning_rate": 1.834925749505265e-05,
	"loss": 0.6799,
	"step": 236
	},
	{
	"epoch": 1.3465909090909092,
	"grad_norm": 0.21808192428190298,
	"learning_rate": 1.8327372847801366e-05,
	"loss": 0.701,
	"step": 237
	},
	{
	"epoch": 1.3522727272727273,
	"grad_norm": 0.25546771943978364,
	"learning_rate": 1.830535733209274e-05,
	"loss": 0.7046,
	"step": 238
	},
	{
	"epoch": 1.3579545454545454,
	"grad_norm": 0.2038956806061383,
	"learning_rate": 1.8283211294326593e-05,
	"loss": 0.7138,
	"step": 239
	},
	{
	"epoch": 1.3636363636363638,
	"grad_norm": 0.2701691948326091,
	"learning_rate": 1.8260935082956458e-05,
	"loss": 0.7041,
	"step": 240
	},
	{
	"epoch": 1.3693181818181819,
	"grad_norm": 0.25479083208670755,
	"learning_rate": 1.823852904848403e-05,
	"loss": 0.7053,
	"step": 241
	},
	{
	"epoch": 1.375,
	"grad_norm": 0.2370431774375721,
	"learning_rate": 1.821599354345371e-05,
	"loss": 0.7287,
	"step": 242
	},
	{
	"epoch": 1.3806818181818181,
	"grad_norm": 0.36933996081261683,
	"learning_rate": 1.8193328922447032e-05,
	"loss": 0.7182,
	"step": 243
	},
	{
	"epoch": 1.3863636363636362,
	"grad_norm": 0.22998708362886527,
	"learning_rate": 1.8170535542077065e-05,
	"loss": 0.7004,
	"step": 244
	},
	{
	"epoch": 1.3920454545454546,
	"grad_norm": 0.26907455069754355,
	"learning_rate": 1.8147613760982858e-05,
	"loss": 0.708,
	"step": 245
	},
	{
	"epoch": 1.3977272727272727,
	"grad_norm": 0.2980927125379044,
	"learning_rate": 1.8124563939823724e-05,
	"loss": 0.6832,
	"step": 246
	},
	{
	"epoch": 1.4034090909090908,
	"grad_norm": 0.24360950110652738,
	"learning_rate": 1.8101386441273636e-05,
	"loss": 0.7291,
	"step": 247
	},
	{
	"epoch": 1.4090909090909092,
	"grad_norm": 0.2440002259357998,
	"learning_rate": 1.8078081630015466e-05,
	"loss": 0.7202,
	"step": 248
	},
	{
	"epoch": 1.4147727272727273,
	"grad_norm": 0.25243909135743026,
	"learning_rate": 1.8054649872735277e-05,
	"loss": 0.7113,
	"step": 249
	},
	{
	"epoch": 1.4204545454545454,
	"grad_norm": 0.23901763773765677,
	"learning_rate": 1.8031091538116545e-05,
	"loss": 0.6869,
	"step": 250
	},
	{
	"epoch": 1.4261363636363638,
	"grad_norm": 0.28227769218983706,
	"learning_rate": 1.8007406996834362e-05,
	"loss": 0.7232,
	"step": 251
	},
	{
	"epoch": 1.4318181818181819,
	"grad_norm": 0.24314631245211815,
	"learning_rate": 1.7983596621549593e-05,
	"loss": 0.7215,
	"step": 252
	},
	{
	"epoch": 1.4375,
	"grad_norm": 0.25267308627191304,
	"learning_rate": 1.7959660786903014e-05,
	"loss": 0.7189,
	"step": 253
	},
	{
	"epoch": 1.4431818181818181,
	"grad_norm": 0.2330894386426013,
	"learning_rate": 1.7935599869509443e-05,
	"loss": 0.681,
	"step": 254
	},
	{
	"epoch": 1.4488636363636362,
	"grad_norm": 0.25165192416307597,
	"learning_rate": 1.791141424795177e-05,
	"loss": 0.6939,
	"step": 255
	},
	{
	"epoch": 1.4545454545454546,
	"grad_norm": 0.23180022765024763,
	"learning_rate": 1.7887104302775042e-05,
	"loss": 0.7026,
	"step": 256
	},
	{
	"epoch": 1.4602272727272727,
	"grad_norm": 0.2705476370432157,
	"learning_rate": 1.7862670416480448e-05,
	"loss": 0.6768,
	"step": 257
	},
	{
	"epoch": 1.4659090909090908,
	"grad_norm": 0.26355908006485196,
	"learning_rate": 1.7838112973519314e-05,
	"loss": 0.6684,
	"step": 258
	},
	{
	"epoch": 1.4715909090909092,
	"grad_norm": 0.2615896253371567,
	"learning_rate": 1.7813432360287045e-05,
	"loss": 0.6936,
	"step": 259
	},
	{
	"epoch": 1.4772727272727273,
	"grad_norm": 0.24838929388619757,
	"learning_rate": 1.7788628965117053e-05,
	"loss": 0.6798,
	"step": 260
	},
	{
	"epoch": 1.4829545454545454,
	"grad_norm": 0.2475299036207677,
	"learning_rate": 1.776370317827465e-05,
	"loss": 0.693,
	"step": 261
	},
	{
	"epoch": 1.4886363636363638,
	"grad_norm": 0.23975582905349616,
	"learning_rate": 1.7738655391950897e-05,
	"loss": 0.7068,
	"step": 262
	},
	{
	"epoch": 1.4943181818181819,
	"grad_norm": 0.22825993815390608,
	"learning_rate": 1.7713486000256425e-05,
	"loss": 0.7009,
	"step": 263
	},
	{
	"epoch": 1.5,
	"grad_norm": 0.23398811574933598,
	"learning_rate": 1.7688195399215265e-05,
	"loss": 0.7356,
	"step": 264
	},
	{
	"epoch": 1.5056818181818183,
	"grad_norm": 0.25376692096145753,
	"learning_rate": 1.766278398675859e-05,
	"loss": 0.6743,
	"step": 265
	},
	{
	"epoch": 1.5113636363636362,
	"grad_norm": 0.25729406494025636,
	"learning_rate": 1.7637252162718468e-05,
	"loss": 0.6863,
	"step": 266
	},
	{
	"epoch": 1.5170454545454546,
	"grad_norm": 0.25708402905609273,
	"learning_rate": 1.7611600328821557e-05,
	"loss": 0.6864,
	"step": 267
	},
	{
	"epoch": 1.5227272727272727,
	"grad_norm": 0.24112295365219877,
	"learning_rate": 1.758582888868279e-05,
	"loss": 0.6634,
	"step": 268
	},
	{
	"epoch": 1.5284090909090908,
	"grad_norm": 0.24748237345676505,
	"learning_rate": 1.7559938247799043e-05,
	"loss": 0.6804,
	"step": 269
	},
	{
	"epoch": 1.5340909090909092,
	"grad_norm": 0.26459530586991326,
	"learning_rate": 1.7533928813542716e-05,
	"loss": 0.7001,
	"step": 270
	},
	{
	"epoch": 1.5397727272727273,
	"grad_norm": 0.22173413467695058,
	"learning_rate": 1.7507800995155366e-05,
	"loss": 0.6931,
	"step": 271
	},
	{
	"epoch": 1.5454545454545454,
	"grad_norm": 0.2380170322873207,
	"learning_rate": 1.748155520374123e-05,
	"loss": 0.6726,
	"step": 272
	},
	{
	"epoch": 1.5511363636363638,
	"grad_norm": 0.2545109481502012,
	"learning_rate": 1.745519185226079e-05,
	"loss": 0.6811,
	"step": 273
	},
	{
	"epoch": 1.5568181818181817,
	"grad_norm": 0.25398972526123387,
	"learning_rate": 1.742871135552425e-05,
	"loss": 0.6789,
	"step": 274
	},
	{
	"epoch": 1.5625,
	"grad_norm": 0.24083319089482572,
	"learning_rate": 1.740211413018501e-05,
	"loss": 0.7202,
	"step": 275
	},
	{
	"epoch": 1.5681818181818183,
	"grad_norm": 0.23245106430586177,
	"learning_rate": 1.737540059473314e-05,
	"loss": 0.7242,
	"step": 276
	},
	{
	"epoch": 1.5738636363636362,
	"grad_norm": 0.23416302735276057,
	"learning_rate": 1.7348571169488758e-05,
	"loss": 0.7368,
	"step": 277
	},
	{
	"epoch": 1.5795454545454546,
	"grad_norm": 0.2398405721463988,
	"learning_rate": 1.732162627659544e-05,
	"loss": 0.6911,
	"step": 278
	},
	{
	"epoch": 1.5852272727272727,
	"grad_norm": 0.24537353416709734,
	"learning_rate": 1.7294566340013563e-05,
	"loss": 0.6753,
	"step": 279
	},
	{
	"epoch": 1.5909090909090908,
	"grad_norm": 0.2453550458309433,
	"learning_rate": 1.7267391785513648e-05,
	"loss": 0.6965,
	"step": 280
	},
	{
	"epoch": 1.5965909090909092,
	"grad_norm": 0.270626558530016,
	"learning_rate": 1.7240103040669653e-05,
	"loss": 0.6447,
	"step": 281
	},
	{
	"epoch": 1.6022727272727273,
	"grad_norm": 0.23962881054711577,
	"learning_rate": 1.7212700534852248e-05,
	"loss": 0.7053,
	"step": 282
	},
	{
	"epoch": 1.6079545454545454,
	"grad_norm": 0.24662637449948943,
	"learning_rate": 1.7185184699222055e-05,
	"loss": 0.6977,
	"step": 283
	},
	{
	"epoch": 1.6136363636363638,
	"grad_norm": 0.24919698912030872,
	"learning_rate": 1.7157555966722867e-05,
	"loss": 0.6868,
	"step": 284
	},
	{
	"epoch": 1.6193181818181817,
	"grad_norm": 0.23486089980185718,
	"learning_rate": 1.7129814772074843e-05,
	"loss": 0.7272,
	"step": 285
	},
	{
	"epoch": 1.625,
	"grad_norm": 0.24464537962896693,
	"learning_rate": 1.710196155176765e-05,
	"loss": 0.7213,
	"step": 286
	},
	{
	"epoch": 1.6306818181818183,
	"grad_norm": 0.22342013890335025,
	"learning_rate": 1.707399674405361e-05,
	"loss": 0.736,
	"step": 287
	},
	{
	"epoch": 1.6363636363636362,
	"grad_norm": 0.21908869161670677,
	"learning_rate": 1.704592078894081e-05,
	"loss": 0.7061,
	"step": 288
	},
	{
	"epoch": 1.6420454545454546,
	"grad_norm": 0.23814136201629713,
	"learning_rate": 1.7017734128186152e-05,
	"loss": 0.7001,
	"step": 289
	},
	{
	"epoch": 1.6477272727272727,
	"grad_norm": 0.2333682887979006,
	"learning_rate": 1.698943720528843e-05,
	"loss": 0.711,
	"step": 290
	},
	{
	"epoch": 1.6534090909090908,
	"grad_norm": 0.235654159685315,
	"learning_rate": 1.696103046548134e-05,
	"loss": 0.7143,
	"step": 291
	},
	{
	"epoch": 1.6590909090909092,
	"grad_norm": 0.24551558482368455,
	"learning_rate": 1.6932514355726478e-05,
	"loss": 0.695,
	"step": 292
	},
	{
	"epoch": 1.6647727272727273,
	"grad_norm": 0.22482952296139788,
	"learning_rate": 1.6903889324706298e-05,
	"loss": 0.7267,
	"step": 293
	},
	{
	"epoch": 1.6704545454545454,
	"grad_norm": 0.24729482355944715,
	"learning_rate": 1.687515582281707e-05,
	"loss": 0.7224,
	"step": 294
	},
	{
	"epoch": 1.6761363636363638,
	"grad_norm": 0.27601790803517107,
	"learning_rate": 1.6846314302161765e-05,
	"loss": 0.7156,
	"step": 295
	},
	{
	"epoch": 1.6818181818181817,
	"grad_norm": 0.2711191827961018,
	"learning_rate": 1.6817365216542977e-05,
	"loss": 0.6932,
	"step": 296
	},
	{
	"epoch": 1.6875,
	"grad_norm": 0.27759124212956277,
	"learning_rate": 1.6788309021455752e-05,
	"loss": 0.6773,
	"step": 297
	},
	{
	"epoch": 1.6931818181818183,
	"grad_norm": 0.2910347544996426,
	"learning_rate": 1.675914617408045e-05,
	"loss": 0.7115,
	"step": 298
	},
	{
	"epoch": 1.6988636363636362,
	"grad_norm": 0.2438477554338481,
	"learning_rate": 1.6729877133275516e-05,
	"loss": 0.6826,
	"step": 299
	},
	{
	"epoch": 1.7045454545454546,
	"grad_norm": 0.2027619806909821,
	"learning_rate": 1.670050235957029e-05,
	"loss": 0.7174,
	"step": 300
	},
	{
	"epoch": 1.7102272727272727,
	"grad_norm": 0.22950919856957447,
	"learning_rate": 1.6671022315157755e-05,
	"loss": 0.7187,
	"step": 301
	},
	{
	"epoch": 1.7159090909090908,
	"grad_norm": 0.2667651340141666,
	"learning_rate": 1.664143746388725e-05,
	"loss": 0.7261,
	"step": 302
	},
	{
	"epoch": 1.7215909090909092,
	"grad_norm": 0.25042079846103943,
	"learning_rate": 1.6611748271257185e-05,
	"loss": 0.7299,
	"step": 303
	},
	{
	"epoch": 1.7272727272727273,
	"grad_norm": 0.2735384411601353,
	"learning_rate": 1.6581955204407724e-05,
	"loss": 0.7082,
	"step": 304
	},
	{
	"epoch": 1.7329545454545454,
	"grad_norm": 0.24949913329652224,
	"learning_rate": 1.65520587321134e-05,
	"loss": 0.6924,
	"step": 305
	},
	{
	"epoch": 1.7386363636363638,
	"grad_norm": 0.23649495318583158,
	"learning_rate": 1.6522059324775786e-05,
	"loss": 0.66,
	"step": 306
	},
	{
	"epoch": 1.7443181818181817,
	"grad_norm": 0.2484850926789353,
	"learning_rate": 1.6491957454416073e-05,
	"loss": 0.7115,
	"step": 307
	},
	{
	"epoch": 1.75,
	"grad_norm": 0.21536790251302598,
	"learning_rate": 1.6461753594667616e-05,
	"loss": 0.7512,
	"step": 308
	},
	{
	"epoch": 1.7556818181818183,
	"grad_norm": 0.2657344638268922,
	"learning_rate": 1.6431448220768527e-05,
	"loss": 0.6985,
	"step": 309
	},
	{
	"epoch": 1.7613636363636362,
	"grad_norm": 0.23610163030643044,
	"learning_rate": 1.6401041809554182e-05,
	"loss": 0.7029,
	"step": 310
	},
	{
	"epoch": 1.7670454545454546,
	"grad_norm": 0.25919505870200205,
	"learning_rate": 1.6370534839449693e-05,
	"loss": 0.6939,
	"step": 311
	},
	{
	"epoch": 1.7727272727272727,
	"grad_norm": 0.26874864847927193,
	"learning_rate": 1.6339927790462415e-05,
	"loss": 0.7049,
	"step": 312
	},
	{
	"epoch": 1.7784090909090908,
	"grad_norm": 0.22742311933441617,
	"learning_rate": 1.630922114417438e-05,
	"loss": 0.6958,
	"step": 313
	},
	{
	"epoch": 1.7840909090909092,
	"grad_norm": 0.24155808757448036,
	"learning_rate": 1.6278415383734715e-05,
	"loss": 0.6808,
	"step": 314
	},
	{
	"epoch": 1.7897727272727273,
	"grad_norm": 0.22559376773130285,
	"learning_rate": 1.6247510993852035e-05,
	"loss": 0.7027,
	"step": 315
	},
	{
	"epoch": 1.7954545454545454,
	"grad_norm": 0.23591171424190074,
	"learning_rate": 1.6216508460786843e-05,
	"loss": 0.6582,
	"step": 316
	},
	{
	"epoch": 1.8011363636363638,
	"grad_norm": 0.21522692956557832,
	"learning_rate": 1.618540827234385e-05,
	"loss": 0.7242,
	"step": 317
	},
	{
	"epoch": 1.8068181818181817,
	"grad_norm": 0.23886497309276813,
	"learning_rate": 1.6154210917864308e-05,
	"loss": 0.7302,
	"step": 318
	},
	{
	"epoch": 1.8125,
	"grad_norm": 0.27506620002712084,
	"learning_rate": 1.6122916888218327e-05,
	"loss": 0.6726,
	"step": 319
	},
	{
	"epoch": 1.8181818181818183,
	"grad_norm": 0.2427785599778038,
	"learning_rate": 1.609152667579712e-05,
	"loss": 0.7009,
	"step": 320
	},
	{
	"epoch": 1.8238636363636362,
	"grad_norm": 0.2666819674634428,
	"learning_rate": 1.606004077450529e-05,
	"loss": 0.7025,
	"step": 321
	},
	{
	"epoch": 1.8295454545454546,
	"grad_norm": 0.25366630458720835,
	"learning_rate": 1.6028459679753036e-05,
	"loss": 0.6995,
	"step": 322
	},
	{
	"epoch": 1.8352272727272727,
	"grad_norm": 0.2660405074948153,
	"learning_rate": 1.5996783888448358e-05,
	"loss": 0.6594,
	"step": 323
	},
	{
	"epoch": 1.8409090909090908,
	"grad_norm": 0.24524759974385674,
	"learning_rate": 1.5965013898989253e-05,
	"loss": 0.6985,
	"step": 324
	},
	{
	"epoch": 1.8465909090909092,
	"grad_norm": 0.25468397323436187,
	"learning_rate": 1.593315021125586e-05,
	"loss": 0.7122,
	"step": 325
	},
	{
	"epoch": 1.8522727272727273,
	"grad_norm": 0.2596933970661363,
	"learning_rate": 1.5901193326602603e-05,
	"loss": 0.6718,
	"step": 326
	},
	{
	"epoch": 1.8579545454545454,
	"grad_norm": 0.2315799202118947,
	"learning_rate": 1.5869143747850296e-05,
	"loss": 0.731,
	"step": 327
	},
	{
	"epoch": 1.8636363636363638,
	"grad_norm": 0.22577168084948318,
	"learning_rate": 1.583700197927824e-05,
	"loss": 0.7218,
	"step": 328
	},
	{
	"epoch": 1.8693181818181817,
	"grad_norm": 0.22387948368805072,
	"learning_rate": 1.580476852661627e-05,
	"loss": 0.7143,
	"step": 329
	},
	{
	"epoch": 1.875,
	"grad_norm": 0.25384938132009893,
	"learning_rate": 1.577244389703683e-05,
	"loss": 0.6594,
	"step": 330
	},
	{
	"epoch": 1.8806818181818183,
	"grad_norm": 0.2405481875896758,
	"learning_rate": 1.574002859914695e-05,
	"loss": 0.7036,
	"step": 331
	},
	{
	"epoch": 1.8863636363636362,
	"grad_norm": 0.21606836375528762,
	"learning_rate": 1.570752314298029e-05,
	"loss": 0.6847,
	"step": 332
	},
	{
	"epoch": 1.8920454545454546,
	"grad_norm": 0.2670272086020513,
	"learning_rate": 1.567492803998908e-05,
	"loss": 0.6944,
	"step": 333
	},
	{
	"epoch": 1.8977272727272727,
	"grad_norm": 0.22141759643196818,
	"learning_rate": 1.5642243803036086e-05,
	"loss": 0.7014,
	"step": 334
	},
	{
	"epoch": 1.9034090909090908,
	"grad_norm": 0.2575087823938948,
	"learning_rate": 1.5609470946386532e-05,
	"loss": 0.7425,
	"step": 335
	},
	{
	"epoch": 1.9090909090909092,
	"grad_norm": 0.24966780403243188,
	"learning_rate": 1.557660998570003e-05,
	"loss": 0.6745,
	"step": 336
	},
	{
	"epoch": 1.9147727272727273,
	"grad_norm": 0.23937055151608005,
	"learning_rate": 1.5543661438022443e-05,
	"loss": 0.6877,
	"step": 337
	},
	{
	"epoch": 1.9204545454545454,
	"grad_norm": 0.23081290484858946,
	"learning_rate": 1.5510625821777757e-05,
	"loss": 0.6855,
	"step": 338
	},
	{
	"epoch": 1.9261363636363638,
	"grad_norm": 0.23093462136566367,
	"learning_rate": 1.5477503656759932e-05,
	"loss": 0.6786,
	"step": 339
	},
	{
	"epoch": 1.9318181818181817,
	"grad_norm": 0.22826938322630316,
	"learning_rate": 1.544429546412471e-05,
	"loss": 0.6753,
	"step": 340
	},
	{
	"epoch": 1.9375,
	"grad_norm": 0.24749853083595733,
	"learning_rate": 1.5411001766381423e-05,
	"loss": 0.6929,
	"step": 341
	},
	{
	"epoch": 1.9431818181818183,
	"grad_norm": 0.2581371582325157,
	"learning_rate": 1.5377623087384774e-05,
	"loss": 0.6845,
	"step": 342
	},
	{
	"epoch": 1.9488636363636362,
	"grad_norm": 0.227987425444293,
	"learning_rate": 1.5344159952326585e-05,
	"loss": 0.7117,
	"step": 343
	},
	{
	"epoch": 1.9545454545454546,
	"grad_norm": 0.23408883726308855,
	"learning_rate": 1.5310612887727546e-05,
	"loss": 0.6958,
	"step": 344
	},
	{
	"epoch": 1.9602272727272727,
	"grad_norm": 0.23115888733209755,
	"learning_rate": 1.527698242142892e-05,
	"loss": 0.7239,
	"step": 345
	},
	{
	"epoch": 1.9659090909090908,
	"grad_norm": 0.22856334848021354,
	"learning_rate": 1.5243269082584244e-05,
	"loss": 0.6934,
	"step": 346
	},
	{
	"epoch": 1.9715909090909092,
	"grad_norm": 0.24031285284143297,
	"learning_rate": 1.5209473401650994e-05,
	"loss": 0.6937,
	"step": 347
	},
	{
	"epoch": 1.9772727272727273,
	"grad_norm": 0.21522131082384216,
	"learning_rate": 1.517559591038225e-05,
	"loss": 0.7043,
	"step": 348
	},
	{
	"epoch": 1.9829545454545454,
	"grad_norm": 0.23041738413650129,
	"learning_rate": 1.5141637141818332e-05,
	"loss": 0.7118,
	"step": 349
	},
	{
	"epoch": 1.9886363636363638,
	"grad_norm": 0.25810504246223814,
	"learning_rate": 1.510759763027839e-05,
	"loss": 0.6715,
	"step": 350
	},
	{
	"epoch": 1.9943181818181817,
	"grad_norm": 0.24034889129124723,
	"learning_rate": 1.507347791135202e-05,
	"loss": 0.6922,
	"step": 351
	},
	{
	"epoch": 2.0,
	"grad_norm": 0.24885824826045777,
	"learning_rate": 1.5039278521890834e-05,
	"loss": 0.6456,
	"step": 352
	},
	{
	"epoch": 2.0056818181818183,
	"grad_norm": 0.3472143230017057,
	"learning_rate": 1.5005000000000001e-05,
	"loss": 0.6537,
	"step": 353
	},
	{
	"epoch": 2.0113636363636362,
	"grad_norm": 0.2905492777484498,
	"learning_rate": 1.4970642885029792e-05,
	"loss": 0.6434,
	"step": 354
	},
	{
	"epoch": 2.0170454545454546,
	"grad_norm": 0.2526340752597525,
	"learning_rate": 1.493620771756708e-05,
	"loss": 0.6537,
	"step": 355
	},
	{
	"epoch": 2.022727272727273,
	"grad_norm": 0.30365778738329285,
	"learning_rate": 1.4901695039426856e-05,
	"loss": 0.6645,
	"step": 356
	},
	{
	"epoch": 2.028409090909091,
	"grad_norm": 0.33570470235900024,
	"learning_rate": 1.4867105393643684e-05,
	"loss": 0.6685,
	"step": 357
	},
	{
	"epoch": 2.034090909090909,
	"grad_norm": 0.27865381817223606,
	"learning_rate": 1.4832439324463166e-05,
	"loss": 0.6674,
	"step": 358
	},
	{
	"epoch": 2.039772727272727,
	"grad_norm": 0.3179033530231687,
	"learning_rate": 1.4797697377333368e-05,
	"loss": 0.638,
	"step": 359
	},
	{
	"epoch": 2.0454545454545454,
	"grad_norm": 0.3007819226005066,
	"learning_rate": 1.4762880098896264e-05,
	"loss": 0.6404,
	"step": 360
	},
	{
	"epoch": 2.0511363636363638,
	"grad_norm": 0.26601610092892064,
	"learning_rate": 1.4727988036979101e-05,
	"loss": 0.6306,
	"step": 361
	},
	{
	"epoch": 2.0568181818181817,
	"grad_norm": 0.27679916801332566,
	"learning_rate": 1.4693021740585802e-05,
	"loss": 0.6213,
	"step": 362
	},
	{
	"epoch": 2.0625,
	"grad_norm": 0.27534906441217527,
	"learning_rate": 1.4657981759888325e-05,
	"loss": 0.6499,
	"step": 363
	},
	{
	"epoch": 2.0681818181818183,
	"grad_norm": 0.2767046904034387,
	"learning_rate": 1.4622868646217991e-05,
	"loss": 0.644,
	"step": 364
	},
	{
	"epoch": 2.0738636363636362,
	"grad_norm": 0.2967916818336686,
	"learning_rate": 1.4587682952056832e-05,
	"loss": 0.645,
	"step": 365
	},
	{
	"epoch": 2.0795454545454546,
	"grad_norm": 0.30558770844411454,
	"learning_rate": 1.455242523102888e-05,
	"loss": 0.6394,
	"step": 366
	},
	{
	"epoch": 2.085227272727273,
	"grad_norm": 0.27836659091810884,
	"learning_rate": 1.4517096037891471e-05,
	"loss": 0.6681,
	"step": 367
	},
	{
	"epoch": 2.090909090909091,
	"grad_norm": 0.23122759138150856,
	"learning_rate": 1.4481695928526497e-05,
	"loss": 0.6395,
	"step": 368
	},
	{
	"epoch": 2.096590909090909,
	"grad_norm": 0.2855389983604095,
	"learning_rate": 1.4446225459931685e-05,
	"loss": 0.624,
	"step": 369
	},
	{
	"epoch": 2.102272727272727,
	"grad_norm": 0.24456545721165454,
	"learning_rate": 1.4410685190211804e-05,
	"loss": 0.6589,
	"step": 370
	},
	{
	"epoch": 2.1079545454545454,
	"grad_norm": 0.23377139828857593,
	"learning_rate": 1.43750756785699e-05,
	"loss": 0.6591,
	"step": 371
	},
	{
	"epoch": 2.1136363636363638,
	"grad_norm": 0.2603161646383973,
	"learning_rate": 1.4339397485298513e-05,
	"loss": 0.6464,
	"step": 372
	},
	{
	"epoch": 2.1193181818181817,
	"grad_norm": 0.30920512362898006,
	"learning_rate": 1.4303651171770827e-05,
	"loss": 0.6505,
	"step": 373
	},
	{
	"epoch": 2.125,
	"grad_norm": 0.2705705551940828,
	"learning_rate": 1.4267837300431853e-05,
	"loss": 0.6285,
	"step": 374
	},
	{
	"epoch": 2.1306818181818183,
	"grad_norm": 0.2564176839723421,
	"learning_rate": 1.423195643478959e-05,
	"loss": 0.6592,
	"step": 375
	},
	{
	"epoch": 2.1363636363636362,
	"grad_norm": 0.2566409624702578,
	"learning_rate": 1.4196009139406143e-05,
	"loss": 0.6803,
	"step": 376
	},
	{
	"epoch": 2.1420454545454546,
	"grad_norm": 0.2468121993715405,
	"learning_rate": 1.4159995979888848e-05,
	"loss": 0.6367,
	"step": 377
	},
	{
	"epoch": 2.147727272727273,
	"grad_norm": 0.26707076409596897,
	"learning_rate": 1.412391752288137e-05,
	"loss": 0.6195,
	"step": 378
	},
	{
	"epoch": 2.153409090909091,
	"grad_norm": 0.2474851029212172,
	"learning_rate": 1.4087774336054785e-05,
	"loss": 0.6321,
	"step": 379
	},
	{
	"epoch": 2.159090909090909,
	"grad_norm": 0.28301842366317254,
	"learning_rate": 1.4051566988098653e-05,
	"loss": 0.6744,
	"step": 380
	},
	{
	"epoch": 2.164772727272727,
	"grad_norm": 0.3121997904066233,
	"learning_rate": 1.4015296048712072e-05,
	"loss": 0.661,
	"step": 381
	},
	{
	"epoch": 2.1704545454545454,
	"grad_norm": 0.23055713609356548,
	"learning_rate": 1.39789620885947e-05,
	"loss": 0.6225,
	"step": 382
	},
	{
	"epoch": 2.1761363636363638,
	"grad_norm": 0.2750865546819273,
	"learning_rate": 1.3942565679437796e-05,
	"loss": 0.6483,
	"step": 383
	},
	{
	"epoch": 2.1818181818181817,
	"grad_norm": 0.26453266223438976,
	"learning_rate": 1.3906107393915203e-05,
	"loss": 0.6774,
	"step": 384
	},
	{
	"epoch": 2.1875,
	"grad_norm": 0.25098561577112793,
	"learning_rate": 1.3869587805674356e-05,
	"loss": 0.6675,
	"step": 385
	},
	{
	"epoch": 2.1931818181818183,
	"grad_norm": 0.2323227272699586,
	"learning_rate": 1.3833007489327248e-05,
	"loss": 0.6075,
	"step": 386
	},
	{
	"epoch": 2.1988636363636362,
	"grad_norm": 0.27562285580882023,
	"learning_rate": 1.3796367020441385e-05,
	"loss": 0.6379,
	"step": 387
	},
	{
	"epoch": 2.2045454545454546,
	"grad_norm": 0.23695503689165387,
	"learning_rate": 1.375966697553074e-05,
	"loss": 0.6446,
	"step": 388
	},
	{
	"epoch": 2.210227272727273,
	"grad_norm": 0.28244912058357574,
	"learning_rate": 1.3722907932046672e-05,
	"loss": 0.6532,
	"step": 389
	},
	{
	"epoch": 2.215909090909091,
	"grad_norm": 0.2781825023586537,
	"learning_rate": 1.3686090468368844e-05,
	"loss": 0.6309,
	"step": 390
	},
	{
	"epoch": 2.221590909090909,
	"grad_norm": 0.27149113889748666,
	"learning_rate": 1.3649215163796125e-05,
	"loss": 0.6651,
	"step": 391
	},
	{
	"epoch": 2.227272727272727,
	"grad_norm": 0.2662019499322451,
	"learning_rate": 1.3612282598537474e-05,
	"loss": 0.6724,
	"step": 392
	},
	{
	"epoch": 2.2329545454545454,
	"grad_norm": 0.269107093261393,
	"learning_rate": 1.3575293353702801e-05,
	"loss": 0.6453,
	"step": 393
	},
	{
	"epoch": 2.2386363636363638,
	"grad_norm": 0.2317166804803217,
	"learning_rate": 1.3538248011293844e-05,
	"loss": 0.663,
	"step": 394
	},
	{
	"epoch": 2.2443181818181817,
	"grad_norm": 0.23950186259478626,
	"learning_rate": 1.3501147154194996e-05,
	"loss": 0.6549,
	"step": 395
	},
	{
	"epoch": 2.25,
	"grad_norm": 0.24863453959495244,
	"learning_rate": 1.3463991366164129e-05,
	"loss": 0.6455,
	"step": 396
	},
	{
	"epoch": 2.2556818181818183,
	"grad_norm": 0.2398993409950357,
	"learning_rate": 1.3426781231823434e-05,
	"loss": 0.6855,
	"step": 397
	},
	{
	"epoch": 2.2613636363636362,
	"grad_norm": 0.2865339875478833,
	"learning_rate": 1.3389517336650188e-05,
	"loss": 0.6421,
	"step": 398
	},
	{
	"epoch": 2.2670454545454546,
	"grad_norm": 0.23353095290990664,
	"learning_rate": 1.3352200266967574e-05,
	"loss": 0.6708,
	"step": 399
	},
	{
	"epoch": 2.2727272727272725,
	"grad_norm": 0.2586652478045742,
	"learning_rate": 1.3314830609935436e-05,
	"loss": 0.6696,
	"step": 400
	},
	{
	"epoch": 2.278409090909091,
	"grad_norm": 0.27425018042631255,
	"learning_rate": 1.3277408953541045e-05,
	"loss": 0.6377,
	"step": 401
	},
	{
	"epoch": 2.284090909090909,
	"grad_norm": 0.2515724142988181,
	"learning_rate": 1.3239935886589842e-05,
	"loss": 0.6454,
	"step": 402
	},
	{
	"epoch": 2.2897727272727275,
	"grad_norm": 0.233009186035962,
	"learning_rate": 1.3202411998696198e-05,
	"loss": 0.6311,
	"step": 403
	},
	{
	"epoch": 2.2954545454545454,
	"grad_norm": 0.24698221444425036,
	"learning_rate": 1.3164837880274102e-05,
	"loss": 0.6333,
	"step": 404
	},
	{
	"epoch": 2.3011363636363638,
	"grad_norm": 0.23428236265875174,
	"learning_rate": 1.3127214122527887e-05,
	"loss": 0.6897,
	"step": 405
	},
	{
	"epoch": 2.3068181818181817,
	"grad_norm": 0.266394308083812,
	"learning_rate": 1.3089541317442942e-05,
	"loss": 0.6921,
	"step": 406
	},
	{
	"epoch": 2.3125,
	"grad_norm": 0.2674905014374322,
	"learning_rate": 1.3051820057776372e-05,
	"loss": 0.6807,
	"step": 407
	},
	{
	"epoch": 2.3181818181818183,
	"grad_norm": 0.2137087245142042,
	"learning_rate": 1.3014050937047689e-05,
	"loss": 0.6737,
	"step": 408
	},
	{
	"epoch": 2.3238636363636362,
	"grad_norm": 0.2783808097297248,
	"learning_rate": 1.2976234549529468e-05,
	"loss": 0.665,
	"step": 409
	},
	{
	"epoch": 2.3295454545454546,
	"grad_norm": 0.27566128321399636,
	"learning_rate": 1.2938371490237991e-05,
	"loss": 0.6366,
	"step": 410
	},
	{
	"epoch": 2.3352272727272725,
	"grad_norm": 0.3415326527044872,
	"learning_rate": 1.2900462354923895e-05,
	"loss": 0.585,
	"step": 411
	},
	{
	"epoch": 2.340909090909091,
	"grad_norm": 0.2369928048251193,
	"learning_rate": 1.2862507740062791e-05,
	"loss": 0.6702,
	"step": 412
	},
	{
	"epoch": 2.346590909090909,
	"grad_norm": 0.2571030373557976,
	"learning_rate": 1.2824508242845883e-05,
	"loss": 0.6701,
	"step": 413
	},
	{
	"epoch": 2.3522727272727275,
	"grad_norm": 0.2512438181119513,
	"learning_rate": 1.2786464461170571e-05,
	"loss": 0.6692,
	"step": 414
	},
	{
	"epoch": 2.3579545454545454,
	"grad_norm": 0.2616630220647511,
	"learning_rate": 1.2748376993631036e-05,
	"loss": 0.6621,
	"step": 415
	},
	{
	"epoch": 2.3636363636363638,
	"grad_norm": 0.23941324421675933,
	"learning_rate": 1.2710246439508832e-05,
	"loss": 0.6317,
	"step": 416
	},
	{
	"epoch": 2.3693181818181817,
	"grad_norm": 0.2713010551644654,
	"learning_rate": 1.267207339876345e-05,
	"loss": 0.6192,
	"step": 417
	},
	{
	"epoch": 2.375,
	"grad_norm": 0.24726457351413972,
	"learning_rate": 1.2633858472022889e-05,
	"loss": 0.6676,
	"step": 418
	},
	{
	"epoch": 2.3806818181818183,
	"grad_norm": 0.26136247748851726,
	"learning_rate": 1.2595602260574183e-05,
	"loss": 0.6499,
	"step": 419
	},
	{
	"epoch": 2.3863636363636362,
	"grad_norm": 0.26887173138466497,
	"learning_rate": 1.2557305366353967e-05,
	"loss": 0.6593,
	"step": 420
	},
	{
	"epoch": 2.3920454545454546,
	"grad_norm": 0.2631680357232149,
	"learning_rate": 1.2518968391938981e-05,
	"loss": 0.6706,
	"step": 421
	},
	{
	"epoch": 2.3977272727272725,
	"grad_norm": 0.2848070851069287,
	"learning_rate": 1.2480591940536614e-05,
	"loss": 0.6982,
	"step": 422
	},
	{
	"epoch": 2.403409090909091,
	"grad_norm": 0.24569635875430837,
	"learning_rate": 1.2442176615975387e-05,
	"loss": 0.6631,
	"step": 423
	},
	{
	"epoch": 2.409090909090909,
	"grad_norm": 0.2619994813640326,
	"learning_rate": 1.2403723022695474e-05,
	"loss": 0.6333,
	"step": 424
	},
	{
	"epoch": 2.4147727272727275,
	"grad_norm": 0.2576830187826504,
	"learning_rate": 1.236523176573918e-05,
	"loss": 0.6275,
	"step": 425
	},
	{
	"epoch": 2.4204545454545454,
	"grad_norm": 0.2518171273338542,
	"learning_rate": 1.2326703450741417e-05,
	"loss": 0.6469,
	"step": 426
	},
	{
	"epoch": 2.4261363636363638,
	"grad_norm": 0.2567792775010768,
	"learning_rate": 1.2288138683920195e-05,
	"loss": 0.6877,
	"step": 427
	},
	{
	"epoch": 2.4318181818181817,
	"grad_norm": 0.2790955555100393,
	"learning_rate": 1.2249538072067053e-05,
	"loss": 0.6597,
	"step": 428
	},
	{
	"epoch": 2.4375,
	"grad_norm": 0.26042685066726207,
	"learning_rate": 1.2210902222537542e-05,
	"loss": 0.6666,
	"step": 429
	},
	{
	"epoch": 2.4431818181818183,
	"grad_norm": 0.2972919255075205,
	"learning_rate": 1.217223174324165e-05,
	"loss": 0.669,
	"step": 430
	},
	{
	"epoch": 2.4488636363636362,
	"grad_norm": 0.296605885406187,
	"learning_rate": 1.2133527242634238e-05,
	"loss": 0.6471,
	"step": 431
	},
	{
	"epoch": 2.4545454545454546,
	"grad_norm": 0.23188269573687592,
	"learning_rate": 1.2094789329705477e-05,
	"loss": 0.6805,
	"step": 432
	},
	{
	"epoch": 2.4602272727272725,
	"grad_norm": 0.25792613970879996,
	"learning_rate": 1.2056018613971256e-05,
	"loss": 0.6693,
	"step": 433
	},
	{
	"epoch": 2.465909090909091,
	"grad_norm": 0.2572454630749856,
	"learning_rate": 1.2017215705463594e-05,
	"loss": 0.656,
	"step": 434
	},
	{
	"epoch": 2.471590909090909,
	"grad_norm": 0.25764577452086906,
	"learning_rate": 1.1978381214721045e-05,
	"loss": 0.6947,
	"step": 435
	},
	{
	"epoch": 2.4772727272727275,
	"grad_norm": 0.2742258829899158,
	"learning_rate": 1.1939515752779079e-05,
	"loss": 0.6399,
	"step": 436
	},
	{
	"epoch": 2.4829545454545454,
	"grad_norm": 0.24837908631806496,
	"learning_rate": 1.19006199311605e-05,
	"loss": 0.6451,
	"step": 437
	},
	{
	"epoch": 2.4886363636363638,
	"grad_norm": 0.24284893820699507,
	"learning_rate": 1.1861694361865779e-05,
	"loss": 0.6546,
	"step": 438
	},
	{
	"epoch": 2.4943181818181817,
	"grad_norm": 0.25069400329076413,
	"learning_rate": 1.182273965736345e-05,
	"loss": 0.6927,
	"step": 439
	},
	{
	"epoch": 2.5,
	"grad_norm": 0.2678083063093823,
	"learning_rate": 1.1783756430580493e-05,
	"loss": 0.6458,
	"step": 440
	},
	{
	"epoch": 2.5056818181818183,
	"grad_norm": 0.2632227195432009,
	"learning_rate": 1.1744745294892636e-05,
	"loss": 0.6883,
	"step": 441
	},
	{
	"epoch": 2.5113636363636362,
	"grad_norm": 0.26749750535263866,
	"learning_rate": 1.170570686411476e-05,
	"loss": 0.6107,
	"step": 442
	},
	{
	"epoch": 2.5170454545454546,
	"grad_norm": 0.26518288203134016,
	"learning_rate": 1.1666641752491199e-05,
	"loss": 0.6554,
	"step": 443
	},
	{
	"epoch": 2.5227272727272725,
	"grad_norm": 0.25816776240956607,
	"learning_rate": 1.1627550574686108e-05,
	"loss": 0.6789,
	"step": 444
	},
	{
	"epoch": 2.528409090909091,
	"grad_norm": 0.2559320715852198,
	"learning_rate": 1.1588433945773768e-05,
	"loss": 0.6276,
	"step": 445
	},
	{
	"epoch": 2.534090909090909,
	"grad_norm": 0.27400320554594415,
	"learning_rate": 1.1549292481228916e-05,
	"loss": 0.618,
	"step": 446
	},
	{
	"epoch": 2.5397727272727275,
	"grad_norm": 0.29496304343373836,
	"learning_rate": 1.151012679691706e-05,
	"loss": 0.6192,
	"step": 447
	},
	{
	"epoch": 2.5454545454545454,
	"grad_norm": 0.23075707125564887,
	"learning_rate": 1.1470937509084803e-05,
	"loss": 0.6782,
	"step": 448
	},
	{
	"epoch": 2.5511363636363638,
	"grad_norm": 0.25077940754287287,
	"learning_rate": 1.1431725234350121e-05,
	"loss": 0.6476,
	"step": 449
	},
	{
	"epoch": 2.5568181818181817,
	"grad_norm": 0.25593578748476414,
	"learning_rate": 1.1392490589692673e-05,
	"loss": 0.6491,
	"step": 450
	},
	{
	"epoch": 2.5625,
	"grad_norm": 0.2836656623532132,
	"learning_rate": 1.1353234192444105e-05,
	"loss": 0.6797,
	"step": 451
	},
	{
	"epoch": 2.5681818181818183,
	"grad_norm": 0.23632872123077125,
	"learning_rate": 1.1313956660278318e-05,
	"loss": 0.6677,
	"step": 452
	},
	{
	"epoch": 2.5738636363636362,
	"grad_norm": 0.3064012678998771,
	"learning_rate": 1.1274658611201757e-05,
	"loss": 0.6818,
	"step": 453
	},
	{
	"epoch": 2.5795454545454546,
	"grad_norm": 0.2518715115807263,
	"learning_rate": 1.1235340663543693e-05,
	"loss": 0.6528,
	"step": 454
	},
	{
	"epoch": 2.5852272727272725,
	"grad_norm": 0.24451149455899293,
	"learning_rate": 1.1196003435946478e-05,
	"loss": 0.6358,
	"step": 455
	},
	{
	"epoch": 2.590909090909091,
	"grad_norm": 0.24090850928675675,
	"learning_rate": 1.115664754735584e-05,
	"loss": 0.6746,
	"step": 456
	},
	{
	"epoch": 2.596590909090909,
	"grad_norm": 0.25790436969252234,
	"learning_rate": 1.1117273617011102e-05,
	"loss": 0.6172,
	"step": 457
	},
	{
	"epoch": 2.6022727272727275,
	"grad_norm": 0.25762960688401837,
	"learning_rate": 1.107788226443548e-05,
	"loss": 0.6799,
	"step": 458
	},
	{
	"epoch": 2.6079545454545454,
	"grad_norm": 0.23519013067633623,
	"learning_rate": 1.1038474109426308e-05,
	"loss": 0.6174,
	"step": 459
	},
	{
	"epoch": 2.6136363636363638,
	"grad_norm": 0.23246208241242247,
	"learning_rate": 1.0999049772045298e-05,
	"loss": 0.652,
	"step": 460
	},
	{
	"epoch": 2.6193181818181817,
	"grad_norm": 0.25829977721916203,
	"learning_rate": 1.0959609872608782e-05,
	"loss": 0.7086,
	"step": 461
	},
	{
	"epoch": 2.625,
	"grad_norm": 0.29983059755966684,
	"learning_rate": 1.092015503167796e-05,
	"loss": 0.651,
	"step": 462
	},
	{
	"epoch": 2.6306818181818183,
	"grad_norm": 0.2703967474851342,
	"learning_rate": 1.0880685870049107e-05,
	"loss": 0.6379,
	"step": 463
	},
	{
	"epoch": 2.6363636363636362,
	"grad_norm": 0.24131039046544242,
	"learning_rate": 1.0841203008743837e-05,
	"loss": 0.6362,
	"step": 464
	},
	{
	"epoch": 2.6420454545454546,
	"grad_norm": 0.285443576595251,
	"learning_rate": 1.0801707068999317e-05,
	"loss": 0.6311,
	"step": 465
	},
	{
	"epoch": 2.6477272727272725,
	"grad_norm": 0.2356164799947745,
	"learning_rate": 1.07621986722585e-05,
	"loss": 0.5848,
	"step": 466
	},
	{
	"epoch": 2.653409090909091,
	"grad_norm": 0.25853157677363164,
	"learning_rate": 1.0722678440160332e-05,
	"loss": 0.6486,
	"step": 467
	},
	{
	"epoch": 2.659090909090909,
	"grad_norm": 0.23462103848103283,
	"learning_rate": 1.068314699452999e-05,
	"loss": 0.6368,
	"step": 468
	},
	{
	"epoch": 2.6647727272727275,
	"grad_norm": 0.2335632622858092,
	"learning_rate": 1.064360495736908e-05,
	"loss": 0.6601,
	"step": 469
	},
	{
	"epoch": 2.6704545454545454,
	"grad_norm": 0.27738458044785463,
	"learning_rate": 1.0604052950845867e-05,
	"loss": 0.6152,
	"step": 470
	},
	{
	"epoch": 2.6761363636363638,
	"grad_norm": 0.30886101654086384,
	"learning_rate": 1.0564491597285473e-05,
	"loss": 0.6351,
	"step": 471
	},
	{
	"epoch": 2.6818181818181817,
	"grad_norm": 0.25615188663287575,
	"learning_rate": 1.0524921519160094e-05,
	"loss": 0.6263,
	"step": 472
	},
	{
	"epoch": 2.6875,
	"grad_norm": 0.22368968365520642,
	"learning_rate": 1.0485343339079188e-05,
	"loss": 0.6825,
	"step": 473
	},
	{
	"epoch": 2.6931818181818183,
	"grad_norm": 0.24061009749478196,
	"learning_rate": 1.0445757679779709e-05,
	"loss": 0.645,
	"step": 474
	},
	{
	"epoch": 2.6988636363636362,
	"grad_norm": 0.2605033914696535,
	"learning_rate": 1.0406165164116282e-05,
	"loss": 0.6598,
	"step": 475
	},
	{
	"epoch": 2.7045454545454546,
	"grad_norm": 0.2753620371131539,
	"learning_rate": 1.0366566415051416e-05,
	"loss": 0.6463,
	"step": 476
	},
	{
	"epoch": 2.7102272727272725,
	"grad_norm": 0.27287635796854715,
	"learning_rate": 1.0326962055645698e-05,
	"loss": 0.6377,
	"step": 477
	},
	{
	"epoch": 2.715909090909091,
	"grad_norm": 0.27456173870796785,
	"learning_rate": 1.0287352709047986e-05,
	"loss": 0.673,
	"step": 478
	},
	{
	"epoch": 2.721590909090909,
	"grad_norm": 0.22760094980146184,
	"learning_rate": 1.0247738998485615e-05,
	"loss": 0.6607,
	"step": 479
	},
	{
	"epoch": 2.7272727272727275,
	"grad_norm": 0.2349020434393742,
	"learning_rate": 1.0208121547254581e-05,
	"loss": 0.6517,
	"step": 480
	},
	{
	"epoch": 2.7329545454545454,
	"grad_norm": 0.238810339709502,
	"learning_rate": 1.0168500978709732e-05,
	"loss": 0.6537,
	"step": 481
	},
	{
	"epoch": 2.7386363636363638,
	"grad_norm": 0.2796706340429125,
	"learning_rate": 1.0128877916254976e-05,
	"loss": 0.6875,
	"step": 482
	},
	{
	"epoch": 2.7443181818181817,
	"grad_norm": 0.23675671202353749,
	"learning_rate": 1.008925298333345e-05,
	"loss": 0.6571,
	"step": 483
	},
	{
	"epoch": 2.75,
	"grad_norm": 0.258205467847281,
	"learning_rate": 1.0049626803417724e-05,
	"loss": 0.6678,
	"step": 484
	},
	{
	"epoch": 2.7556818181818183,
	"grad_norm": 0.23272101276831392,
	"learning_rate": 1.0009999999999999e-05,
	"loss": 0.655,
	"step": 485
	},
	{
	"epoch": 2.7613636363636362,
	"grad_norm": 0.250779274046665,
	"learning_rate": 9.970373196582274e-06,
	"loss": 0.6594,
	"step": 486
	},
	{
	"epoch": 2.7670454545454546,
	"grad_norm": 0.24655381075747243,
	"learning_rate": 9.930747016666554e-06,
	"loss": 0.6752,
	"step": 487
	},
	{
	"epoch": 2.7727272727272725,
	"grad_norm": 0.2861262066450717,
	"learning_rate": 9.891122083745025e-06,
	"loss": 0.6664,
	"step": 488
	},
	{
	"epoch": 2.778409090909091,
	"grad_norm": 0.23859296347923306,
	"learning_rate": 9.851499021290269e-06,
	"loss": 0.6479,
	"step": 489
	},
	{
	"epoch": 2.784090909090909,
	"grad_norm": 0.28451435934695163,
	"learning_rate": 9.811878452745424e-06,
	"loss": 0.6741,
	"step": 490
	},
	{
	"epoch": 2.7897727272727275,
	"grad_norm": 0.23705223062420253,
	"learning_rate": 9.772261001514388e-06,
	"loss": 0.7084,
	"step": 491
	},
	{
	"epoch": 2.7954545454545454,
	"grad_norm": 0.2522487982254347,
	"learning_rate": 9.732647290952015e-06,
	"loss": 0.6641,
	"step": 492
	},
	{
	"epoch": 2.8011363636363638,
	"grad_norm": 0.258372122541777,
	"learning_rate": 9.693037944354307e-06,
	"loss": 0.6331,
	"step": 493
	},
	{
	"epoch": 2.8068181818181817,
	"grad_norm": 0.2567627181228032,
	"learning_rate": 9.653433584948587e-06,
	"loss": 0.6439,
	"step": 494
	},
	{
	"epoch": 2.8125,
	"grad_norm": 0.22327709911877947,
	"learning_rate": 9.61383483588372e-06,
	"loss": 0.6525,
	"step": 495
	},
	{
	"epoch": 2.8181818181818183,
	"grad_norm": 0.23531080467004545,
	"learning_rate": 9.574242320220295e-06,
	"loss": 0.6797,
	"step": 496
	},
	{
	"epoch": 2.8238636363636362,
	"grad_norm": 0.28726152130238697,
	"learning_rate": 9.534656660920817e-06,
	"loss": 0.6798,
	"step": 497
	},
	{
	"epoch": 2.8295454545454546,
	"grad_norm": 0.2502351943282993,
	"learning_rate": 9.495078480839911e-06,
	"loss": 0.6585,
	"step": 498
	},
	{
	"epoch": 2.8352272727272725,
	"grad_norm": 0.4638928010399934,
	"learning_rate": 9.45550840271453e-06,
	"loss": 0.6625,
	"step": 499
	},
	{
	"epoch": 2.840909090909091,
	"grad_norm": 0.247466081687593,
	"learning_rate": 9.415947049154136e-06,
	"loss": 0.6611,
	"step": 500
	},
	{
	"epoch": 2.846590909090909,
	"grad_norm": 0.24136165076647917,
	"learning_rate": 9.376395042630921e-06,
	"loss": 0.6672,
	"step": 501
	},
	{
	"epoch": 2.8522727272727275,
	"grad_norm": 0.2456294117004471,
	"learning_rate": 9.336853005470012e-06,
	"loss": 0.616,
	"step": 502
	},
	{
	"epoch": 2.8579545454545454,
	"grad_norm": 0.24611183240687112,
	"learning_rate": 9.297321559839671e-06,
	"loss": 0.645,
	"step": 503
	},
	{
	"epoch": 2.8636363636363638,
	"grad_norm": 0.25214373787937217,
	"learning_rate": 9.257801327741502e-06,
	"loss": 0.6633,
	"step": 504
	},
	{
	"epoch": 2.8693181818181817,
	"grad_norm": 0.26140049364946705,
	"learning_rate": 9.218292931000683e-06,
	"loss": 0.6688,
	"step": 505
	},
	{
	"epoch": 2.875,
	"grad_norm": 0.2545303455506609,
	"learning_rate": 9.178796991256166e-06,
	"loss": 0.6276,
	"step": 506
	},
	{
	"epoch": 2.8806818181818183,
	"grad_norm": 0.2983499833972796,
	"learning_rate": 9.139314129950894e-06,
	"loss": 0.6364,
	"step": 507
	},
	{
	"epoch": 2.8863636363636362,
	"grad_norm": 0.2517347433565825,
	"learning_rate": 9.09984496832204e-06,
	"loss": 0.6457,
	"step": 508
	},
	{
	"epoch": 2.8920454545454546,
	"grad_norm": 0.22523497306252882,
	"learning_rate": 9.060390127391217e-06,
	"loss": 0.6437,
	"step": 509
	},
	{
	"epoch": 2.8977272727272725,
	"grad_norm": 0.2745150487518302,
	"learning_rate": 9.020950227954705e-06,
	"loss": 0.6519,
	"step": 510
	},
	{
	"epoch": 2.903409090909091,
	"grad_norm": 0.24788049577175192,
	"learning_rate": 8.981525890573695e-06,
	"loss": 0.6295,
	"step": 511
	},
	{
	"epoch": 2.909090909090909,
	"grad_norm": 0.25759378890258916,
	"learning_rate": 8.942117735564526e-06,
	"loss": 0.626,
	"step": 512
	},
	{
	"epoch": 2.9147727272727275,
	"grad_norm": 0.2660966119852049,
	"learning_rate": 8.902726382988901e-06,
	"loss": 0.6536,
	"step": 513
	},
	{
	"epoch": 2.9204545454545454,
	"grad_norm": 0.23714630572548454,
	"learning_rate": 8.863352452644162e-06,
	"loss": 0.6352,
	"step": 514
	},
	{
	"epoch": 2.9261363636363638,
	"grad_norm": 0.3052304209761072,
	"learning_rate": 8.823996564053524e-06,
	"loss": 0.64,
	"step": 515
	},
	{
	"epoch": 2.9318181818181817,
	"grad_norm": 0.25121887713106983,
	"learning_rate": 8.78465933645631e-06,
	"loss": 0.6644,
	"step": 516
	},
	{
	"epoch": 2.9375,
	"grad_norm": 0.2598225711845343,
	"learning_rate": 8.745341388798244e-06,
	"loss": 0.6701,
	"step": 517
	},
	{
	"epoch": 2.9431818181818183,
	"grad_norm": 0.23621332537780398,
	"learning_rate": 8.706043339721688e-06,
	"loss": 0.6492,
	"step": 518
	},
	{
	"epoch": 2.9488636363636362,
	"grad_norm": 0.22278876013770876,
	"learning_rate": 8.666765807555898e-06,
	"loss": 0.6621,
	"step": 519
	},
	{
	"epoch": 2.9545454545454546,
	"grad_norm": 0.21913653553879306,
	"learning_rate": 8.627509410307328e-06,
	"loss": 0.679,
	"step": 520
	},
	{
	"epoch": 2.9602272727272725,
	"grad_norm": 0.29031682244544094,
	"learning_rate": 8.588274765649884e-06,
	"loss": 0.6702,
	"step": 521
	},
	{
	"epoch": 2.965909090909091,
	"grad_norm": 0.2244735406811807,
	"learning_rate": 8.549062490915197e-06,
	"loss": 0.6517,
	"step": 522
	},
	{
	"epoch": 2.971590909090909,
	"grad_norm": 0.31356995719486913,
	"learning_rate": 8.50987320308294e-06,
	"loss": 0.6316,
	"step": 523
	},
	{
	"epoch": 2.9772727272727275,
	"grad_norm": 0.2853161387217062,
	"learning_rate": 8.470707518771091e-06,
	"loss": 0.6705,
	"step": 524
	},
	{
	"epoch": 2.9829545454545454,
	"grad_norm": 0.24659539530797978,
	"learning_rate": 8.431566054226234e-06,
	"loss": 0.678,
	"step": 525
	},
	{
	"epoch": 2.9886363636363638,
	"grad_norm": 0.22692824895093408,
	"learning_rate": 8.392449425313893e-06,
	"loss": 0.6398,
	"step": 526
	},
	{
	"epoch": 2.9943181818181817,
	"grad_norm": 0.2813609414823193,
	"learning_rate": 8.353358247508804e-06,
	"loss": 0.6603,
	"step": 527
	},
	{
	"epoch": 3.0,
	"grad_norm": 0.23746281518991996,
	"learning_rate": 8.314293135885246e-06,
	"loss": 0.6474,
	"step": 528
	},
	{
	"epoch": 3.0056818181818183,
	"grad_norm": 0.33715522303658735,
	"learning_rate": 8.275254705107367e-06,
	"loss": 0.6107,
	"step": 529
	},
	{
	"epoch": 3.0113636363636362,
	"grad_norm": 0.275107520589631,
	"learning_rate": 8.236243569419513e-06,
	"loss": 0.6373,
	"step": 530
	},
	{
	"epoch": 3.0170454545454546,
	"grad_norm": 0.2658244054808944,
	"learning_rate": 8.197260342636551e-06,
	"loss": 0.6259,
	"step": 531
	},
	{
	"epoch": 3.022727272727273,
	"grad_norm": 0.24207881138544574,
	"learning_rate": 8.158305638134226e-06,
	"loss": 0.6379,
	"step": 532
	},
	{
	"epoch": 3.028409090909091,
	"grad_norm": 0.2975245966297416,
	"learning_rate": 8.119380068839507e-06,
	"loss": 0.6375,
	"step": 533
	},
	{
	"epoch": 3.034090909090909,
	"grad_norm": 0.3500993989900972,
	"learning_rate": 8.080484247220922e-06,
	"loss": 0.6291,
	"step": 534
	},
	{
	"epoch": 3.039772727272727,
	"grad_norm": 0.3540569781546958,
	"learning_rate": 8.041618785278959e-06,
	"loss": 0.6474,
	"step": 535
	},
	{
	"epoch": 3.0454545454545454,
	"grad_norm": 0.2767884116491314,
	"learning_rate": 8.002784294536405e-06,
	"loss": 0.6209,
	"step": 536
	},
	{
	"epoch": 3.0511363636363638,
	"grad_norm": 0.2730525185559992,
	"learning_rate": 7.963981386028747e-06,
	"loss": 0.6025,
	"step": 537
	},
	{
	"epoch": 3.0568181818181817,
	"grad_norm": 0.2672918797676767,
	"learning_rate": 7.925210670294524e-06,
	"loss": 0.6099,
	"step": 538
	},
	{
	"epoch": 3.0625,
	"grad_norm": 0.2498727126563379,
	"learning_rate": 7.886472757365763e-06,
	"loss": 0.6327,
	"step": 539
	},
	{
	"epoch": 3.0681818181818183,
	"grad_norm": 0.2757136103812718,
	"learning_rate": 7.847768256758353e-06,
	"loss": 0.6129,
	"step": 540
	},
	{
	"epoch": 3.0738636363636362,
	"grad_norm": 0.29836430654259016,
	"learning_rate": 7.80909777746246e-06,
	"loss": 0.6106,
	"step": 541
	},
	{
	"epoch": 3.0795454545454546,
	"grad_norm": 0.2922353325852422,
	"learning_rate": 7.770461927932948e-06,
	"loss": 0.6189,
	"step": 542
	},
	{
	"epoch": 3.085227272727273,
	"grad_norm": 0.26118207759465745,
	"learning_rate": 7.731861316079812e-06,
	"loss": 0.61,
	"step": 543
	},
	{
	"epoch": 3.090909090909091,
	"grad_norm": 0.26782409784383515,
	"learning_rate": 7.693296549258586e-06,
	"loss": 0.6095,
	"step": 544
	},
	{
	"epoch": 3.096590909090909,
	"grad_norm": 0.27483603381883653,
	"learning_rate": 7.654768234260821e-06,
	"loss": 0.5815,
	"step": 545
	},
	{
	"epoch": 3.102272727272727,
	"grad_norm": 0.2553662093748128,
	"learning_rate": 7.616276977304529e-06,
	"loss": 0.648,
	"step": 546
	},
	{
	"epoch": 3.1079545454545454,
	"grad_norm": 0.2529080144402227,
	"learning_rate": 7.577823384024614e-06,
	"loss": 0.6123,
	"step": 547
	},
	{
	"epoch": 3.1136363636363638,
	"grad_norm": 0.2666839686283043,
	"learning_rate": 7.539408059463388e-06,
	"loss": 0.6495,
	"step": 548
	},
	{
	"epoch": 3.1193181818181817,
	"grad_norm": 0.27740631756789835,
	"learning_rate": 7.50103160806102e-06,
	"loss": 0.6047,
	"step": 549
	},
	{
	"epoch": 3.125,
	"grad_norm": 0.30736439133887294,
	"learning_rate": 7.462694633646035e-06,
	"loss": 0.6197,
	"step": 550
	},
	{
	"epoch": 3.1306818181818183,
	"grad_norm": 0.25847774331916945,
	"learning_rate": 7.424397739425817e-06,
	"loss": 0.6226,
	"step": 551
	},
	{
	"epoch": 3.1363636363636362,
	"grad_norm": 0.2748937356518825,
	"learning_rate": 7.386141527977117e-06,
	"loss": 0.637,
	"step": 552
	},
	{
	"epoch": 3.1420454545454546,
	"grad_norm": 0.25207922704395636,
	"learning_rate": 7.347926601236552e-06,
	"loss": 0.6121,
	"step": 553
	},
	{
	"epoch": 3.147727272727273,
	"grad_norm": 0.2768359128189639,
	"learning_rate": 7.30975356049117e-06,
	"loss": 0.5966,
	"step": 554
	},
	{
	"epoch": 3.153409090909091,
	"grad_norm": 0.2763274885397192,
	"learning_rate": 7.27162300636897e-06,
	"loss": 0.6283,
	"step": 555
	},
	{
	"epoch": 3.159090909090909,
	"grad_norm": 0.27965092197116476,
	"learning_rate": 7.2335355388294335e-06,
	"loss": 0.6055,
	"step": 556
	},
	{
	"epoch": 3.164772727272727,
	"grad_norm": 0.2829014638161376,
	"learning_rate": 7.195491757154117e-06,
	"loss": 0.5964,
	"step": 557
	},
	{
	"epoch": 3.1704545454545454,
	"grad_norm": 0.27813212640421314,
	"learning_rate": 7.157492259937212e-06,
	"loss": 0.6291,
	"step": 558
	},
	{
	"epoch": 3.1761363636363638,
	"grad_norm": 0.23480358487936978,
	"learning_rate": 7.119537645076109e-06,
	"loss": 0.6223,
	"step": 559
	},
	{
	"epoch": 3.1818181818181817,
	"grad_norm": 0.2719493705771613,
	"learning_rate": 7.08162850976201e-06,
	"loss": 0.5976,
	"step": 560
	},
	{
	"epoch": 3.1875,
	"grad_norm": 0.24253147387750687,
	"learning_rate": 7.043765450470538e-06,
	"loss": 0.6323,
	"step": 561
	},
	{
	"epoch": 3.1931818181818183,
	"grad_norm": 0.26179000833789534,
	"learning_rate": 7.005949062952313e-06,
	"loss": 0.5825,
	"step": 562
	},
	{
	"epoch": 3.1988636363636362,
	"grad_norm": 0.2786921127118021,
	"learning_rate": 6.968179942223631e-06,
	"loss": 0.64,
	"step": 563
	},
	{
	"epoch": 3.2045454545454546,
	"grad_norm": 0.27006583269125817,
	"learning_rate": 6.930458682557062e-06,
	"loss": 0.6137,
	"step": 564
	},
	{
	"epoch": 3.210227272727273,
	"grad_norm": 0.25587564378663175,
	"learning_rate": 6.892785877472116e-06,
	"loss": 0.5977,
	"step": 565
	},
	{
	"epoch": 3.215909090909091,
	"grad_norm": 0.26521482568318483,
	"learning_rate": 6.855162119725903e-06,
	"loss": 0.5748,
	"step": 566
	},
	{
	"epoch": 3.221590909090909,
	"grad_norm": 0.2508823350291074,
	"learning_rate": 6.817588001303804e-06,
	"loss": 0.6259,
	"step": 567
	},
	{
	"epoch": 3.227272727272727,
	"grad_norm": 0.25685608239044205,
	"learning_rate": 6.780064113410157e-06,
	"loss": 0.6215,
	"step": 568
	},
	{
	"epoch": 3.2329545454545454,
	"grad_norm": 0.2929886694411762,
	"learning_rate": 6.742591046458958e-06,
	"loss": 0.5737,
	"step": 569
	},
	{
	"epoch": 3.2386363636363638,
	"grad_norm": 0.239917869651524,
	"learning_rate": 6.705169390064565e-06,
	"loss": 0.606,
	"step": 570
	},
	{
	"epoch": 3.2443181818181817,
	"grad_norm": 0.2475217330296139,
	"learning_rate": 6.667799733032429e-06,
	"loss": 0.6395,
	"step": 571
	},
	{
	"epoch": 3.25,
	"grad_norm": 0.2772118720610513,
	"learning_rate": 6.630482663349815e-06,
	"loss": 0.6037,
	"step": 572
	},
	{
	"epoch": 3.2556818181818183,
	"grad_norm": 0.25556844504750864,
	"learning_rate": 6.59321876817657e-06,
	"loss": 0.6001,
	"step": 573
	},
	{
	"epoch": 3.2613636363636362,
	"grad_norm": 0.27350419143108884,
	"learning_rate": 6.556008633835874e-06,
	"loss": 0.5938,
	"step": 574
	},
	{
	"epoch": 3.2670454545454546,
	"grad_norm": 0.2418817541977412,
	"learning_rate": 6.5188528458050094e-06,
	"loss": 0.6156,
	"step": 575
	},
	{
	"epoch": 3.2727272727272725,
	"grad_norm": 0.2730671762683675,
	"learning_rate": 6.481751988706157e-06,
	"loss": 0.6135,
	"step": 576
	},
	{
	"epoch": 3.278409090909091,
	"grad_norm": 0.2509238275933269,
	"learning_rate": 6.444706646297202e-06,
	"loss": 0.6311,
	"step": 577
	},
	{
	"epoch": 3.284090909090909,
	"grad_norm": 0.2734402776459457,
	"learning_rate": 6.4077174014625295e-06,
	"loss": 0.5758,
	"step": 578
	},
	{
	"epoch": 3.2897727272727275,
	"grad_norm": 0.2864449825077938,
	"learning_rate": 6.370784836203875e-06,
	"loss": 0.6016,
	"step": 579
	},
	{
	"epoch": 3.2954545454545454,
	"grad_norm": 0.28409720916177283,
	"learning_rate": 6.3339095316311585e-06,
	"loss": 0.6069,
	"step": 580
	},
	{
	"epoch": 3.3011363636363638,
	"grad_norm": 0.25778427958261996,
	"learning_rate": 6.29709206795333e-06,
	"loss": 0.5959,
	"step": 581
	},
	{
	"epoch": 3.3068181818181817,
	"grad_norm": 0.27890406747874336,
	"learning_rate": 6.260333024469261e-06,
	"loss": 0.6073,
	"step": 582
	},
	{
	"epoch": 3.3125,
	"grad_norm": 0.2662579155749672,
	"learning_rate": 6.223632979558617e-06,
	"loss": 0.6191,
	"step": 583
	},
	{
	"epoch": 3.3181818181818183,
	"grad_norm": 0.2740712274285668,
	"learning_rate": 6.186992510672755e-06,
	"loss": 0.6175,
	"step": 584
	},
	{
	"epoch": 3.3238636363636362,
	"grad_norm": 0.24751155639667363,
	"learning_rate": 6.150412194325645e-06,
	"loss": 0.6063,
	"step": 585
	},
	{
	"epoch": 3.3295454545454546,
	"grad_norm": 0.25006252516585226,
	"learning_rate": 6.1138926060848e-06,
	"loss": 0.5784,
	"step": 586
	},
	{
	"epoch": 3.3352272727272725,
	"grad_norm": 0.23020394917654904,
	"learning_rate": 6.077434320562207e-06,
	"loss": 0.6238,
	"step": 587
	},
	{
	"epoch": 3.340909090909091,
	"grad_norm": 0.2537521706343083,
	"learning_rate": 6.0410379114053e-06,
	"loss": 0.6647,
	"step": 588
	},
	{
	"epoch": 3.346590909090909,
	"grad_norm": 0.25361512691242893,
	"learning_rate": 6.004703951287931e-06,
	"loss": 0.5999,
	"step": 589
	},
	{
	"epoch": 3.3522727272727275,
	"grad_norm": 0.25476494335605665,
	"learning_rate": 5.968433011901349e-06,
	"loss": 0.5864,
	"step": 590
	},
	{
	"epoch": 3.3579545454545454,
	"grad_norm": 0.2468910338187795,
	"learning_rate": 5.9322256639452195e-06,
	"loss": 0.6241,
	"step": 591
	},
	{
	"epoch": 3.3636363636363638,
	"grad_norm": 0.24023469878807002,
	"learning_rate": 5.8960824771186355e-06,
	"loss": 0.6301,
	"step": 592
	},
	{
	"epoch": 3.3693181818181817,
	"grad_norm": 0.26412938823785315,
	"learning_rate": 5.8600040201111565e-06,
	"loss": 0.6285,
	"step": 593
	},
	{
	"epoch": 3.375,
	"grad_norm": 0.25164854512639345,
	"learning_rate": 5.82399086059386e-06,
	"loss": 0.6184,
	"step": 594
	},
	{
	"epoch": 3.3806818181818183,
	"grad_norm": 0.302680957857416,
	"learning_rate": 5.788043565210415e-06,
	"loss": 0.5911,
	"step": 595
	},
	{
	"epoch": 3.3863636363636362,
	"grad_norm": 0.2531141469331557,
	"learning_rate": 5.7521626995681525e-06,
	"loss": 0.6188,
	"step": 596
	},
	{
	"epoch": 3.3920454545454546,
	"grad_norm": 0.24786943442709114,
	"learning_rate": 5.716348828229176e-06,
	"loss": 0.618,
	"step": 597
	},
	{
	"epoch": 3.3977272727272725,
	"grad_norm": 0.24602531386071005,
	"learning_rate": 5.680602514701486e-06,
	"loss": 0.64,
	"step": 598
	},
	{
	"epoch": 3.403409090909091,
	"grad_norm": 0.2424967436497295,
	"learning_rate": 5.6449243214301e-06,
	"loss": 0.6103,
	"step": 599
	},
	{
	"epoch": 3.409090909090909,
	"grad_norm": 0.2778320938878234,
	"learning_rate": 5.6093148097881985e-06,
	"loss": 0.6362,
	"step": 600
	},
	{
	"epoch": 3.4147727272727275,
	"grad_norm": 0.9893806703731987,
	"learning_rate": 5.573774540068316e-06,
	"loss": 0.6133,
	"step": 601
	},
	{
	"epoch": 3.4204545454545454,
	"grad_norm": 0.224933468129408,
	"learning_rate": 5.5383040714735025e-06,
	"loss": 0.5996,
	"step": 602
	},
	{
	"epoch": 3.4261363636363638,
	"grad_norm": 0.2683286564605203,
	"learning_rate": 5.502903962108532e-06,
	"loss": 0.6282,
	"step": 603
	},
	{
	"epoch": 3.4318181818181817,
	"grad_norm": 0.2763798397106102,
	"learning_rate": 5.46757476897112e-06,
	"loss": 0.6385,
	"step": 604
	},
	{
	"epoch": 3.4375,
	"grad_norm": 0.26480157804014764,
	"learning_rate": 5.4323170479431705e-06,
	"loss": 0.5933,
	"step": 605
	},
	{
	"epoch": 3.4431818181818183,
	"grad_norm": 0.2636362462680493,
	"learning_rate": 5.397131353782014e-06,
	"loss": 0.6051,
	"step": 606
	},
	{
	"epoch": 3.4488636363636362,
	"grad_norm": 0.2527434792717813,
	"learning_rate": 5.362018240111676e-06,
	"loss": 0.6096,
	"step": 607
	},
	{
	"epoch": 3.4545454545454546,
	"grad_norm": 0.22910947503920823,
	"learning_rate": 5.326978259414198e-06,
	"loss": 0.6568,
	"step": 608
	},
	{
	"epoch": 3.4602272727272725,
	"grad_norm": 0.23531525368042022,
	"learning_rate": 5.292011963020901e-06,
	"loss": 0.599,
	"step": 609
	},
	{
	"epoch": 3.465909090909091,
	"grad_norm": 0.26647799843491227,
	"learning_rate": 5.257119901103735e-06,
	"loss": 0.6534,
	"step": 610
	},
	{
	"epoch": 3.471590909090909,
	"grad_norm": 0.2460386249176432,
	"learning_rate": 5.2223026226666365e-06,
	"loss": 0.6253,
	"step": 611
	},
	{
	"epoch": 3.4772727272727275,
	"grad_norm": 0.2492517977808207,
	"learning_rate": 5.187560675536839e-06,
	"loss": 0.6158,
	"step": 612
	},
	{
	"epoch": 3.4829545454545454,
	"grad_norm": 0.268173380827287,
	"learning_rate": 5.15289460635632e-06,
	"loss": 0.5914,
	"step": 613
	},
	{
	"epoch": 3.4886363636363638,
	"grad_norm": 0.23838550950095033,
	"learning_rate": 5.118304960573149e-06,
	"loss": 0.6315,
	"step": 614
	},
	{
	"epoch": 3.4943181818181817,
	"grad_norm": 0.281993567528547,
	"learning_rate": 5.083792282432921e-06,
	"loss": 0.6008,
	"step": 615
	},
	{
	"epoch": 3.5,
	"grad_norm": 0.2574379065229267,
	"learning_rate": 5.049357114970214e-06,
	"loss": 0.6085,
	"step": 616
	},
	{
	"epoch": 3.5056818181818183,
	"grad_norm": 1.7302653406129704,
	"learning_rate": 5.015000000000002e-06,
	"loss": 0.5996,
	"step": 617
	},
	{
	"epoch": 3.5113636363636362,
	"grad_norm": 0.24042453821407192,
	"learning_rate": 4.980721478109167e-06,
	"loss": 0.6378,
	"step": 618
	},
	{
	"epoch": 3.5170454545454546,
	"grad_norm": 0.25812070558816935,
	"learning_rate": 4.946522088647982e-06,
	"loss": 0.6482,
	"step": 619
	},
	{
	"epoch": 3.5227272727272725,
	"grad_norm": 0.2378154297460634,
	"learning_rate": 4.912402369721615e-06,
	"loss": 0.6295,
	"step": 620
	},
	{
	"epoch": 3.528409090909091,
	"grad_norm": 0.23171893996983065,
	"learning_rate": 4.878362858181674e-06,
	"loss": 0.5934,
	"step": 621
	},
	{
	"epoch": 3.534090909090909,
	"grad_norm": 0.2653039830945405,
	"learning_rate": 4.84440408961775e-06,
	"loss": 0.6306,
	"step": 622
	},
	{
	"epoch": 3.5397727272727275,
	"grad_norm": 0.2400050425716777,
	"learning_rate": 4.81052659834901e-06,
	"loss": 0.6273,
	"step": 623
	},
	{
	"epoch": 3.5454545454545454,
	"grad_norm": 0.28275367617242414,
	"learning_rate": 4.776730917415761e-06,
	"loss": 0.5967,
	"step": 624
	},
	{
	"epoch": 3.5511363636363638,
	"grad_norm": 0.26416007931691776,
	"learning_rate": 4.743017578571081e-06,
	"loss": 0.5759,
	"step": 625
	},
	{
	"epoch": 3.5568181818181817,
	"grad_norm": 0.3311537677008043,
	"learning_rate": 4.709387112272456e-06,
	"loss": 0.6124,
	"step": 626
	},
	{
	"epoch": 3.5625,
	"grad_norm": 0.22944557358959208,
	"learning_rate": 4.675840047673419e-06,
	"loss": 0.6047,
	"step": 627
	},
	{
	"epoch": 3.5681818181818183,
	"grad_norm": 0.266395335548406,
	"learning_rate": 4.642376912615229e-06,
	"loss": 0.6255,
	"step": 628
	},
	{
	"epoch": 3.5738636363636362,
	"grad_norm": 0.2546461736682591,
	"learning_rate": 4.608998233618584e-06,
	"loss": 0.5917,
	"step": 629
	},
	{
	"epoch": 3.5795454545454546,
	"grad_norm": 0.24685522883436725,
	"learning_rate": 4.575704535875295e-06,
	"loss": 0.6149,
	"step": 630
	},
	{
	"epoch": 3.5852272727272725,
	"grad_norm": 0.24644710894145763,
	"learning_rate": 4.542496343240072e-06,
	"loss": 0.6438,
	"step": 631
	},
	{
	"epoch": 3.590909090909091,
	"grad_norm": 0.230690440966595,
	"learning_rate": 4.509374178222246e-06,
	"loss": 0.6012,
	"step": 632
	},
	{
	"epoch": 3.596590909090909,
	"grad_norm": 0.24341741680358914,
	"learning_rate": 4.476338561977559e-06,
	"loss": 0.6204,
	"step": 633
	},
	{
	"epoch": 3.6022727272727275,
	"grad_norm": 0.28085422004109406,
	"learning_rate": 4.443390014299972e-06,
	"loss": 0.6037,
	"step": 634
	},
	{
	"epoch": 3.6079545454545454,
	"grad_norm": 0.2490824110427276,
	"learning_rate": 4.4105290536134685e-06,
	"loss": 0.6172,
	"step": 635
	},
	{
	"epoch": 3.6136363636363638,
	"grad_norm": 0.22595910328043572,
	"learning_rate": 4.377756196963917e-06,
	"loss": 0.6294,
	"step": 636
	},
	{
	"epoch": 3.6193181818181817,
	"grad_norm": 0.24724180065915813,
	"learning_rate": 4.345071960010922e-06,
	"loss": 0.6346,
	"step": 637
	},
	{
	"epoch": 3.625,
	"grad_norm": 0.23836110299004593,
	"learning_rate": 4.312476857019708e-06,
	"loss": 0.5833,
	"step": 638
	},
	{
	"epoch": 3.6306818181818183,
	"grad_norm": 0.24676927764067763,
	"learning_rate": 4.279971400853052e-06,
	"loss": 0.6488,
	"step": 639
	},
	{
	"epoch": 3.6363636363636362,
	"grad_norm": 0.23506987772840007,
	"learning_rate": 4.247556102963174e-06,
	"loss": 0.6129,
	"step": 640
	},
	{
	"epoch": 3.6420454545454546,
	"grad_norm": 0.292077057081307,
	"learning_rate": 4.215231473383732e-06,
	"loss": 0.6069,
	"step": 641
	},
	{
	"epoch": 3.6477272727272725,
	"grad_norm": 0.25450453897027653,
	"learning_rate": 4.182998020721766e-06,
	"loss": 0.6329,
	"step": 642
	},
	{
	"epoch": 3.653409090909091,
	"grad_norm": 0.24162646216132147,
	"learning_rate": 4.150856252149704e-06,
	"loss": 0.6442,
	"step": 643
	},
	{
	"epoch": 3.659090909090909,
	"grad_norm": 0.24494295417958842,
	"learning_rate": 4.118806673397399e-06,
	"loss": 0.6508,
	"step": 644
	},
	{
	"epoch": 3.6647727272727275,
	"grad_norm": 0.2905864509174215,
	"learning_rate": 4.086849788744144e-06,
	"loss": 0.6298,
	"step": 645
	},
	{
	"epoch": 3.6704545454545454,
	"grad_norm": 0.24711866529738705,
	"learning_rate": 4.054986101010749e-06,
	"loss": 0.6109,
	"step": 646
	},
	{
	"epoch": 3.6761363636363638,
	"grad_norm": 0.2541715276168137,
	"learning_rate": 4.023216111551644e-06,
	"loss": 0.6093,
	"step": 647
	},
	{
	"epoch": 3.6818181818181817,
	"grad_norm": 0.3190429620157626,
	"learning_rate": 3.991540320246967e-06,
	"loss": 0.5906,
	"step": 648
	},
	{
	"epoch": 3.6875,
	"grad_norm": 0.2648242722592236,
	"learning_rate": 3.959959225494712e-06,
	"loss": 0.6461,
	"step": 649
	},
	{
	"epoch": 3.6931818181818183,
	"grad_norm": 0.2525561621384859,
	"learning_rate": 3.928473324202881e-06,
	"loss": 0.5814,
	"step": 650
	},
	{
	"epoch": 3.6988636363636362,
	"grad_norm": 0.2939149507471348,
	"learning_rate": 3.897083111781677e-06,
	"loss": 0.6186,
	"step": 651
	},
	{
	"epoch": 3.7045454545454546,
	"grad_norm": 0.23286752185787438,
	"learning_rate": 3.865789082135696e-06,
	"loss": 0.6268,
	"step": 652
	},
	{
	"epoch": 3.7102272727272725,
	"grad_norm": 0.28618226009377634,
	"learning_rate": 3.834591727656153e-06,
	"loss": 0.6311,
	"step": 653
	},
	{
	"epoch": 3.715909090909091,
	"grad_norm": 0.25677863654333516,
	"learning_rate": 3.8034915392131592e-06,
	"loss": 0.6373,
	"step": 654
	},
	{
	"epoch": 3.721590909090909,
	"grad_norm": 0.27903372674977994,
	"learning_rate": 3.7724890061479678e-06,
	"loss": 0.6034,
	"step": 655
	},
	{
	"epoch": 3.7272727272727275,
	"grad_norm": 0.27761808194211035,
	"learning_rate": 3.741584616265288e-06,
	"loss": 0.6372,
	"step": 656
	},
	{
	"epoch": 3.7329545454545454,
	"grad_norm": 0.24183987681213975,
	"learning_rate": 3.7107788558256235e-06,
	"loss": 0.5951,
	"step": 657
	},
	{
	"epoch": 3.7386363636363638,
	"grad_norm": 0.2613900857473471,
	"learning_rate": 3.6800722095375847e-06,
	"loss": 0.6035,
	"step": 658
	},
	{
	"epoch": 3.7443181818181817,
	"grad_norm": 0.2292541646627328,
	"learning_rate": 3.64946516055031e-06,
	"loss": 0.6314,
	"step": 659
	},
	{
	"epoch": 3.75,
	"grad_norm": 0.3005150835688057,
	"learning_rate": 3.618958190445823e-06,
	"loss": 0.6088,
	"step": 660
	},
	{
	"epoch": 3.7556818181818183,
	"grad_norm": 0.24442922409031267,
	"learning_rate": 3.5885517792314724e-06,
	"loss": 0.6116,
	"step": 661
	},
	{
	"epoch": 3.7613636363636362,
	"grad_norm": 0.25526983618570476,
	"learning_rate": 3.5582464053323878e-06,
	"loss": 0.6027,
	"step": 662
	},
	{
	"epoch": 3.7670454545454546,
	"grad_norm": 0.22609712271640642,
	"learning_rate": 3.5280425455839335e-06,
	"loss": 0.6309,
	"step": 663
	},
	{
	"epoch": 3.7727272727272725,
	"grad_norm": 0.2629847318048887,
	"learning_rate": 3.4979406752242124e-06,
	"loss": 0.6172,
	"step": 664
	},
	{
	"epoch": 3.778409090909091,
	"grad_norm": 0.2804818295898827,
	"learning_rate": 3.467941267886603e-06,
	"loss": 0.6292,
	"step": 665
	},
	{
	"epoch": 3.784090909090909,
	"grad_norm": 0.25788132495292804,
	"learning_rate": 3.438044795592283e-06,
	"loss": 0.5867,
	"step": 666
	},
	{
	"epoch": 3.7897727272727275,
	"grad_norm": 0.24986094236004233,
	"learning_rate": 3.4082517287428173e-06,
	"loss": 0.6145,
	"step": 667
	},
	{
	"epoch": 3.7954545454545454,
	"grad_norm": 0.2422288952252709,
	"learning_rate": 3.3785625361127517e-06,
	"loss": 0.6268,
	"step": 668
	},
	{
	"epoch": 3.8011363636363638,
	"grad_norm": 0.2532441425324105,
	"learning_rate": 3.348977684842248e-06,
	"loss": 0.6469,
	"step": 669
	},
	{
	"epoch": 3.8068181818181817,
	"grad_norm": 0.26384481042825847,
	"learning_rate": 3.319497640429713e-06,
	"loss": 0.5938,
	"step": 670
	},
	{
	"epoch": 3.8125,
	"grad_norm": 0.25353848058343487,
	"learning_rate": 3.2901228667244874e-06,
	"loss": 0.6041,
	"step": 671
	},
	{
	"epoch": 3.8181818181818183,
	"grad_norm": 0.26716123342070275,
	"learning_rate": 3.2608538259195543e-06,
	"loss": 0.6206,
	"step": 672
	},
	{
	"epoch": 3.8238636363636362,
	"grad_norm": 0.24902271674653384,
	"learning_rate": 3.231690978544251e-06,
	"loss": 0.5847,
	"step": 673
	},
	{
	"epoch": 3.8295454545454546,
	"grad_norm": 0.23250804366288846,
	"learning_rate": 3.202634783457027e-06,
	"loss": 0.5856,
	"step": 674
	},
	{
	"epoch": 3.8352272727272725,
	"grad_norm": 0.24765150390012558,
	"learning_rate": 3.1736856978382385e-06,
	"loss": 0.6225,
	"step": 675
	},
	{
	"epoch": 3.840909090909091,
	"grad_norm": 0.25546359188020296,
	"learning_rate": 3.144844177182936e-06,
	"loss": 0.621,
	"step": 676
	},
	{
	"epoch": 3.846590909090909,
	"grad_norm": 0.23344232708212276,
	"learning_rate": 3.116110675293703e-06,
	"loss": 0.6362,
	"step": 677
	},
	{
	"epoch": 3.8522727272727275,
	"grad_norm": 0.2737529012015266,
	"learning_rate": 3.0874856442735214e-06,
	"loss": 0.5821,
	"step": 678
	},
	{
	"epoch": 3.8579545454545454,
	"grad_norm": 0.2546759586107607,
	"learning_rate": 3.0589695345186587e-06,
	"loss": 0.6378,
	"step": 679
	},
	{
	"epoch": 3.8636363636363638,
	"grad_norm": 0.2693807112476373,
	"learning_rate": 3.0305627947115722e-06,
	"loss": 0.6026,
	"step": 680
	},
	{
	"epoch": 3.8693181818181817,
	"grad_norm": 0.27972595371865355,
	"learning_rate": 3.0022658718138496e-06,
	"loss": 0.5954,
	"step": 681
	},
	{
	"epoch": 3.875,
	"grad_norm": 0.24586326030908112,
	"learning_rate": 2.9740792110591967e-06,
	"loss": 0.6251,
	"step": 682
	},
	{
	"epoch": 3.8806818181818183,
	"grad_norm": 0.23444414550566664,
	"learning_rate": 2.9460032559463915e-06,
	"loss": 0.6204,
	"step": 683
	},
	{
	"epoch": 3.8863636363636362,
	"grad_norm": 0.2573176440511936,
	"learning_rate": 2.918038448232352e-06,
	"loss": 0.5921,
	"step": 684
	},
	{
	"epoch": 3.8920454545454546,
	"grad_norm": 0.2504263555606992,
	"learning_rate": 2.890185227925163e-06,
	"loss": 0.6417,
	"step": 685
	},
	{
	"epoch": 3.8977272727272725,
	"grad_norm": 0.25592584634032867,
	"learning_rate": 2.8624440332771346e-06,
	"loss": 0.6134,
	"step": 686
	},
	{
	"epoch": 3.903409090909091,
	"grad_norm": 0.22239686409523537,
	"learning_rate": 2.834815300777949e-06,
	"loss": 0.638,
	"step": 687
	},
	{
	"epoch": 3.909090909090909,
	"grad_norm": 0.28114208710948113,
	"learning_rate": 2.8072994651477576e-06,
	"loss": 0.6135,
	"step": 688
	},
	{
	"epoch": 3.9147727272727275,
	"grad_norm": 0.37516661814594865,
	"learning_rate": 2.7798969593303487e-06,
	"loss": 0.6057,
	"step": 689
	},
	{
	"epoch": 3.9204545454545454,
	"grad_norm": 0.2434478116075633,
	"learning_rate": 2.7526082144863554e-06,
	"loss": 0.6129,
	"step": 690
	},
	{
	"epoch": 3.9261363636363638,
	"grad_norm": 0.26339101087702566,
	"learning_rate": 2.725433659986442e-06,
	"loss": 0.5881,
	"step": 691
	},
	{
	"epoch": 3.9318181818181817,
	"grad_norm": 0.2444962872075247,
	"learning_rate": 2.6983737234045653e-06,
	"loss": 0.6248,
	"step": 692
	},
	{
	"epoch": 3.9375,
	"grad_norm": 0.2334586335548691,
	"learning_rate": 2.6714288305112434e-06,
	"loss": 0.6058,
	"step": 693
	},
	{
	"epoch": 3.9431818181818183,
	"grad_norm": 0.2774562050262344,
	"learning_rate": 2.6445994052668615e-06,
	"loss": 0.6249,
	"step": 694
	},
	{
	"epoch": 3.9488636363636362,
	"grad_norm": 0.26185376647771846,
	"learning_rate": 2.617885869814992e-06,
	"loss": 0.6208,
	"step": 695
	},
	{
	"epoch": 3.9545454545454546,
	"grad_norm": 0.2392435129163743,
	"learning_rate": 2.5912886444757547e-06,
	"loss": 0.6005,
	"step": 696
	},
	{
	"epoch": 3.9602272727272725,
	"grad_norm": 0.24058741614781873,
	"learning_rate": 2.564808147739211e-06,
	"loss": 0.6382,
	"step": 697
	},
	{
	"epoch": 3.965909090909091,
	"grad_norm": 0.23649313036102357,
	"learning_rate": 2.538444796258771e-06,
	"loss": 0.6118,
	"step": 698
	},
	{
	"epoch": 3.971590909090909,
	"grad_norm": 0.24242650086544323,
	"learning_rate": 2.512199004844636e-06,
	"loss": 0.6384,
	"step": 699
	},
	{
	"epoch": 3.9772727272727275,
	"grad_norm": 0.24877327138195238,
	"learning_rate": 2.4860711864572844e-06,
	"loss": 0.6008,
	"step": 700
	},
	{
	"epoch": 3.9829545454545454,
	"grad_norm": 0.23886822022380147,
	"learning_rate": 2.4600617522009612e-06,
	"loss": 0.6178,
	"step": 701
	},
	{
	"epoch": 3.9886363636363638,
	"grad_norm": 0.2368692720169352,
	"learning_rate": 2.4341711113172124e-06,
	"loss": 0.5966,
	"step": 702
	},
	{
	"epoch": 3.9943181818181817,
	"grad_norm": 0.2470873237355616,
	"learning_rate": 2.4083996711784456e-06,
	"loss": 0.6164,
	"step": 703
	},
	{
	"epoch": 4.0,
	"grad_norm": 0.2495252077120968,
	"learning_rate": 2.3827478372815333e-06,
	"loss": 0.6018,
	"step": 704
	}
	],
	"logging_steps": 1,
	"max_steps": 880,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 5,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": false
	},
	"attributes": {}
	}
	},
	"total_flos": 1859126053699584.0,
	"train_batch_size": 2,
	"trial_name": null,
	"trial_params": null
	}