Instructions to use itsmehyperbee/random15kSFTQwen2.5-3B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use itsmehyperbee/random15kSFTQwen2.5-3B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="itsmehyperbee/random15kSFTQwen2.5-3B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("itsmehyperbee/random15kSFTQwen2.5-3B-Instruct")
model = AutoModelForCausalLM.from_pretrained("itsmehyperbee/random15kSFTQwen2.5-3B-Instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use itsmehyperbee/random15kSFTQwen2.5-3B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "itsmehyperbee/random15kSFTQwen2.5-3B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "itsmehyperbee/random15kSFTQwen2.5-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/itsmehyperbee/random15kSFTQwen2.5-3B-Instruct

SGLang

How to use itsmehyperbee/random15kSFTQwen2.5-3B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "itsmehyperbee/random15kSFTQwen2.5-3B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "itsmehyperbee/random15kSFTQwen2.5-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "itsmehyperbee/random15kSFTQwen2.5-3B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "itsmehyperbee/random15kSFTQwen2.5-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use itsmehyperbee/random15kSFTQwen2.5-3B-Instruct with Docker Model Runner:
```
docker model run hf.co/itsmehyperbee/random15kSFTQwen2.5-3B-Instruct
```

random15kSFTQwen2.5-3B-Instruct / trainer_state.json

itsmehyperbee

Upload folder using huggingface_hub

f348f86 verified 10 months ago

Raw

History Blame Contribute Delete

89.7 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 5.0,
	"eval_steps": 500,
	"global_step": 515,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.009791921664626682,
	"grad_norm": 4.937262535095215,
	"learning_rate": 5e-06,
	"loss": 0.9202,
	"step": 1
	},
	{
	"epoch": 0.019583843329253364,
	"grad_norm": 3.8881704807281494,
	"learning_rate": 4.999953484900578e-06,
	"loss": 0.7318,
	"step": 2
	},
	{
	"epoch": 0.02937576499388005,
	"grad_norm": 2.42814564704895,
	"learning_rate": 4.999813941333237e-06,
	"loss": 0.8649,
	"step": 3
	},
	{
	"epoch": 0.03916768665850673,
	"grad_norm": 1.798976182937622,
	"learning_rate": 4.999581374490681e-06,
	"loss": 0.8175,
	"step": 4
	},
	{
	"epoch": 0.04895960832313342,
	"grad_norm": 2.1165692806243896,
	"learning_rate": 4.999255793027207e-06,
	"loss": 0.8809,
	"step": 5
	},
	{
	"epoch": 0.0587515299877601,
	"grad_norm": 2.0665972232818604,
	"learning_rate": 4.998837209058379e-06,
	"loss": 0.7402,
	"step": 6
	},
	{
	"epoch": 0.06854345165238677,
	"grad_norm": 1.9621387720108032,
	"learning_rate": 4.998325638160576e-06,
	"loss": 0.8053,
	"step": 7
	},
	{
	"epoch": 0.07833537331701346,
	"grad_norm": 1.8001136779785156,
	"learning_rate": 4.997721099370416e-06,
	"loss": 0.7659,
	"step": 8
	},
	{
	"epoch": 0.08812729498164015,
	"grad_norm": 1.560722827911377,
	"learning_rate": 4.997023615184044e-06,
	"loss": 0.7254,
	"step": 9
	},
	{
	"epoch": 0.09791921664626684,
	"grad_norm": 1.6357344388961792,
	"learning_rate": 4.996233211556295e-06,
	"loss": 0.9604,
	"step": 10
	},
	{
	"epoch": 0.10771113831089352,
	"grad_norm": 1.5878242254257202,
	"learning_rate": 4.995349917899735e-06,
	"loss": 0.8585,
	"step": 11
	},
	{
	"epoch": 0.1175030599755202,
	"grad_norm": 1.1256458759307861,
	"learning_rate": 4.994373767083556e-06,
	"loss": 0.8329,
	"step": 12
	},
	{
	"epoch": 0.12729498164014688,
	"grad_norm": 1.1497397422790527,
	"learning_rate": 4.99330479543236e-06,
	"loss": 0.6592,
	"step": 13
	},
	{
	"epoch": 0.13708690330477355,
	"grad_norm": 1.147639513015747,
	"learning_rate": 4.992143042724805e-06,
	"loss": 0.6779,
	"step": 14
	},
	{
	"epoch": 0.14687882496940025,
	"grad_norm": 1.0602340698242188,
	"learning_rate": 4.990888552192126e-06,
	"loss": 0.8156,
	"step": 15
	},
	{
	"epoch": 0.15667074663402691,
	"grad_norm": 1.006097435951233,
	"learning_rate": 4.989541370516523e-06,
	"loss": 0.6821,
	"step": 16
	},
	{
	"epoch": 0.1664626682986536,
	"grad_norm": 1.0098599195480347,
	"learning_rate": 4.9881015478294294e-06,
	"loss": 0.6265,
	"step": 17
	},
	{
	"epoch": 0.1762545899632803,
	"grad_norm": 1.3986536264419556,
	"learning_rate": 4.98656913770964e-06,
	"loss": 0.9192,
	"step": 18
	},
	{
	"epoch": 0.18604651162790697,
	"grad_norm": 1.1685140132904053,
	"learning_rate": 4.984944197181324e-06,
	"loss": 0.7399,
	"step": 19
	},
	{
	"epoch": 0.19583843329253367,
	"grad_norm": 1.2679662704467773,
	"learning_rate": 4.983226786711895e-06,
	"loss": 0.8148,
	"step": 20
	},
	{
	"epoch": 0.20563035495716034,
	"grad_norm": 1.0894488096237183,
	"learning_rate": 4.98141697020977e-06,
	"loss": 0.759,
	"step": 21
	},
	{
	"epoch": 0.21542227662178703,
	"grad_norm": 1.1688951253890991,
	"learning_rate": 4.979514815021984e-06,
	"loss": 0.6333,
	"step": 22
	},
	{
	"epoch": 0.2252141982864137,
	"grad_norm": 0.910540759563446,
	"learning_rate": 4.9775203919316864e-06,
	"loss": 0.6401,
	"step": 23
	},
	{
	"epoch": 0.2350061199510404,
	"grad_norm": 1.0755398273468018,
	"learning_rate": 4.975433775155509e-06,
	"loss": 0.9768,
	"step": 24
	},
	{
	"epoch": 0.24479804161566707,
	"grad_norm": 0.8792560696601868,
	"learning_rate": 4.973255042340801e-06,
	"loss": 0.8438,
	"step": 25
	},
	{
	"epoch": 0.25458996328029376,
	"grad_norm": 1.0656226873397827,
	"learning_rate": 4.970984274562741e-06,
	"loss": 0.7518,
	"step": 26
	},
	{
	"epoch": 0.26438188494492043,
	"grad_norm": 0.9469810724258423,
	"learning_rate": 4.968621556321319e-06,
	"loss": 0.8339,
	"step": 27
	},
	{
	"epoch": 0.2741738066095471,
	"grad_norm": 1.0511175394058228,
	"learning_rate": 4.966166975538197e-06,
	"loss": 0.6094,
	"step": 28
	},
	{
	"epoch": 0.2839657282741738,
	"grad_norm": 0.8811876773834229,
	"learning_rate": 4.963620623553428e-06,
	"loss": 0.6981,
	"step": 29
	},
	{
	"epoch": 0.2937576499388005,
	"grad_norm": 0.9535372853279114,
	"learning_rate": 4.960982595122064e-06,
	"loss": 0.6527,
	"step": 30
	},
	{
	"epoch": 0.30354957160342716,
	"grad_norm": 1.0013355016708374,
	"learning_rate": 4.958252988410631e-06,
	"loss": 0.6456,
	"step": 31
	},
	{
	"epoch": 0.31334149326805383,
	"grad_norm": 1.1226270198822021,
	"learning_rate": 4.955431904993471e-06,
	"loss": 0.69,
	"step": 32
	},
	{
	"epoch": 0.32313341493268055,
	"grad_norm": 0.8976802825927734,
	"learning_rate": 4.952519449848962e-06,
	"loss": 0.695,
	"step": 33
	},
	{
	"epoch": 0.3329253365973072,
	"grad_norm": 0.9635095596313477,
	"learning_rate": 4.9495157313556185e-06,
	"loss": 0.7557,
	"step": 34
	},
	{
	"epoch": 0.3427172582619339,
	"grad_norm": 0.9229223728179932,
	"learning_rate": 4.946420861288051e-06,
	"loss": 0.662,
	"step": 35
	},
	{
	"epoch": 0.3525091799265606,
	"grad_norm": 0.8999955654144287,
	"learning_rate": 4.943234954812812e-06,
	"loss": 0.6456,
	"step": 36
	},
	{
	"epoch": 0.3623011015911873,
	"grad_norm": 0.9576444029808044,
	"learning_rate": 4.939958130484106e-06,
	"loss": 0.8278,
	"step": 37
	},
	{
	"epoch": 0.37209302325581395,
	"grad_norm": 0.8263678550720215,
	"learning_rate": 4.93659051023938e-06,
	"loss": 0.8871,
	"step": 38
	},
	{
	"epoch": 0.3818849449204406,
	"grad_norm": 0.8772329688072205,
	"learning_rate": 4.933132219394786e-06,
	"loss": 0.695,
	"step": 39
	},
	{
	"epoch": 0.39167686658506734,
	"grad_norm": 1.0509618520736694,
	"learning_rate": 4.92958338664052e-06,
	"loss": 0.9088,
	"step": 40
	},
	{
	"epoch": 0.401468788249694,
	"grad_norm": 0.9522665143013,
	"learning_rate": 4.925944144036027e-06,
	"loss": 0.7337,
	"step": 41
	},
	{
	"epoch": 0.4112607099143207,
	"grad_norm": 0.8179937601089478,
	"learning_rate": 4.922214627005092e-06,
	"loss": 0.6799,
	"step": 42
	},
	{
	"epoch": 0.42105263157894735,
	"grad_norm": 0.851662278175354,
	"learning_rate": 4.918394974330801e-06,
	"loss": 0.727,
	"step": 43
	},
	{
	"epoch": 0.43084455324357407,
	"grad_norm": 0.8003877401351929,
	"learning_rate": 4.9144853281503715e-06,
	"loss": 0.7412,
	"step": 44
	},
	{
	"epoch": 0.44063647490820074,
	"grad_norm": 0.7761635184288025,
	"learning_rate": 4.91048583394987e-06,
	"loss": 0.5825,
	"step": 45
	},
	{
	"epoch": 0.4504283965728274,
	"grad_norm": 0.7423298954963684,
	"learning_rate": 4.90639664055879e-06,
	"loss": 0.5448,
	"step": 46
	},
	{
	"epoch": 0.4602203182374541,
	"grad_norm": 0.9305451512336731,
	"learning_rate": 4.902217900144524e-06,
	"loss": 0.5969,
	"step": 47
	},
	{
	"epoch": 0.4700122399020808,
	"grad_norm": 1.3245203495025635,
	"learning_rate": 4.8979497682066916e-06,
	"loss": 0.789,
	"step": 48
	},
	{
	"epoch": 0.47980416156670747,
	"grad_norm": 0.7262954115867615,
	"learning_rate": 4.8935924035713564e-06,
	"loss": 0.5776,
	"step": 49
	},
	{
	"epoch": 0.48959608323133413,
	"grad_norm": 0.8691399097442627,
	"learning_rate": 4.889145968385121e-06,
	"loss": 0.752,
	"step": 50
	},
	{
	"epoch": 0.49938800489596086,
	"grad_norm": 0.8003021478652954,
	"learning_rate": 4.884610628109082e-06,
	"loss": 0.617,
	"step": 51
	},
	{
	"epoch": 0.5091799265605875,
	"grad_norm": 0.8315463662147522,
	"learning_rate": 4.879986551512684e-06,
	"loss": 0.7293,
	"step": 52
	},
	{
	"epoch": 0.5189718482252142,
	"grad_norm": 1.0837576389312744,
	"learning_rate": 4.875273910667434e-06,
	"loss": 0.9608,
	"step": 53
	},
	{
	"epoch": 0.5287637698898409,
	"grad_norm": 0.8256236910820007,
	"learning_rate": 4.870472880940496e-06,
	"loss": 0.6044,
	"step": 54
	},
	{
	"epoch": 0.5385556915544676,
	"grad_norm": 0.8332642912864685,
	"learning_rate": 4.865583640988173e-06,
	"loss": 0.6544,
	"step": 55
	},
	{
	"epoch": 0.5483476132190942,
	"grad_norm": 0.7978657484054565,
	"learning_rate": 4.860606372749247e-06,
	"loss": 0.5414,
	"step": 56
	},
	{
	"epoch": 0.5581395348837209,
	"grad_norm": 0.9077686071395874,
	"learning_rate": 4.855541261438223e-06,
	"loss": 0.745,
	"step": 57
	},
	{
	"epoch": 0.5679314565483476,
	"grad_norm": 0.8141072392463684,
	"learning_rate": 4.850388495538423e-06,
	"loss": 0.6731,
	"step": 58
	},
	{
	"epoch": 0.5777233782129743,
	"grad_norm": 0.6969121694564819,
	"learning_rate": 4.8451482667949836e-06,
	"loss": 0.6933,
	"step": 59
	},
	{
	"epoch": 0.587515299877601,
	"grad_norm": 0.8899105191230774,
	"learning_rate": 4.839820770207714e-06,
	"loss": 0.7177,
	"step": 60
	},
	{
	"epoch": 0.5973072215422277,
	"grad_norm": 0.8271994590759277,
	"learning_rate": 4.83440620402384e-06,
	"loss": 0.6168,
	"step": 61
	},
	{
	"epoch": 0.6070991432068543,
	"grad_norm": 0.9924188852310181,
	"learning_rate": 4.828904769730628e-06,
	"loss": 0.7943,
	"step": 62
	},
	{
	"epoch": 0.616891064871481,
	"grad_norm": 0.9602711796760559,
	"learning_rate": 4.82331667204789e-06,
	"loss": 0.9227,
	"step": 63
	},
	{
	"epoch": 0.6266829865361077,
	"grad_norm": 0.8672987222671509,
	"learning_rate": 4.8176421189203605e-06,
	"loss": 0.5994,
	"step": 64
	},
	{
	"epoch": 0.6364749082007344,
	"grad_norm": 0.9110500812530518,
	"learning_rate": 4.811881321509964e-06,
	"loss": 0.8647,
	"step": 65
	},
	{
	"epoch": 0.6462668298653611,
	"grad_norm": 0.9391680359840393,
	"learning_rate": 4.806034494187949e-06,
	"loss": 0.6962,
	"step": 66
	},
	{
	"epoch": 0.6560587515299877,
	"grad_norm": 1.0803498029708862,
	"learning_rate": 4.80010185452692e-06,
	"loss": 0.8844,
	"step": 67
	},
	{
	"epoch": 0.6658506731946144,
	"grad_norm": 0.8144680857658386,
	"learning_rate": 4.794083623292737e-06,
	"loss": 0.7663,
	"step": 68
	},
	{
	"epoch": 0.6756425948592412,
	"grad_norm": 0.8452160954475403,
	"learning_rate": 4.7879800244362975e-06,
	"loss": 0.7186,
	"step": 69
	},
	{
	"epoch": 0.6854345165238678,
	"grad_norm": 0.747094988822937,
	"learning_rate": 4.781791285085209e-06,
	"loss": 0.6717,
	"step": 70
	},
	{
	"epoch": 0.6952264381884945,
	"grad_norm": 0.9193806648254395,
	"learning_rate": 4.775517635535332e-06,
	"loss": 0.8848,
	"step": 71
	},
	{
	"epoch": 0.7050183598531212,
	"grad_norm": 0.7738409638404846,
	"learning_rate": 4.769159309242213e-06,
	"loss": 0.6146,
	"step": 72
	},
	{
	"epoch": 0.7148102815177478,
	"grad_norm": 0.9756547212600708,
	"learning_rate": 4.762716542812395e-06,
	"loss": 0.8656,
	"step": 73
	},
	{
	"epoch": 0.7246022031823746,
	"grad_norm": 0.8940049409866333,
	"learning_rate": 4.756189575994614e-06,
	"loss": 0.6799,
	"step": 74
	},
	{
	"epoch": 0.7343941248470012,
	"grad_norm": 0.7465048432350159,
	"learning_rate": 4.7495786516708806e-06,
	"loss": 0.5871,
	"step": 75
	},
	{
	"epoch": 0.7441860465116279,
	"grad_norm": 0.9098663330078125,
	"learning_rate": 4.742884015847436e-06,
	"loss": 0.7138,
	"step": 76
	},
	{
	"epoch": 0.7539779681762546,
	"grad_norm": 0.8581227660179138,
	"learning_rate": 4.7361059176456e-06,
	"loss": 0.6612,
	"step": 77
	},
	{
	"epoch": 0.7637698898408812,
	"grad_norm": 0.7587230801582336,
	"learning_rate": 4.7292446092925016e-06,
	"loss": 0.6883,
	"step": 78
	},
	{
	"epoch": 0.773561811505508,
	"grad_norm": 1.2049251794815063,
	"learning_rate": 4.722300346111695e-06,
	"loss": 0.7712,
	"step": 79
	},
	{
	"epoch": 0.7833537331701347,
	"grad_norm": 0.7951218485832214,
	"learning_rate": 4.715273386513651e-06,
	"loss": 0.5629,
	"step": 80
	},
	{
	"epoch": 0.7931456548347613,
	"grad_norm": 0.9955035448074341,
	"learning_rate": 4.708163991986152e-06,
	"loss": 0.8486,
	"step": 81
	},
	{
	"epoch": 0.802937576499388,
	"grad_norm": 0.8821019530296326,
	"learning_rate": 4.700972427084551e-06,
	"loss": 0.6695,
	"step": 82
	},
	{
	"epoch": 0.8127294981640147,
	"grad_norm": 0.7500548958778381,
	"learning_rate": 4.693698959421935e-06,
	"loss": 0.7501,
	"step": 83
	},
	{
	"epoch": 0.8225214198286414,
	"grad_norm": 0.9788134098052979,
	"learning_rate": 4.686343859659158e-06,
	"loss": 0.7674,
	"step": 84
	},
	{
	"epoch": 0.8323133414932681,
	"grad_norm": 0.8246988654136658,
	"learning_rate": 4.67890740149478e-06,
	"loss": 0.6613,
	"step": 85
	},
	{
	"epoch": 0.8421052631578947,
	"grad_norm": 1.0771559476852417,
	"learning_rate": 4.671389861654873e-06,
	"loss": 0.6919,
	"step": 86
	},
	{
	"epoch": 0.8518971848225214,
	"grad_norm": 0.8286159038543701,
	"learning_rate": 4.6637915198827265e-06,
	"loss": 0.6084,
	"step": 87
	},
	{
	"epoch": 0.8616891064871481,
	"grad_norm": 0.7630783915519714,
	"learning_rate": 4.65611265892844e-06,
	"loss": 0.793,
	"step": 88
	},
	{
	"epoch": 0.8714810281517748,
	"grad_norm": 0.884671688079834,
	"learning_rate": 4.648353564538397e-06,
	"loss": 0.66,
	"step": 89
	},
	{
	"epoch": 0.8812729498164015,
	"grad_norm": 0.7917746901512146,
	"learning_rate": 4.640514525444637e-06,
	"loss": 0.5262,
	"step": 90
	},
	{
	"epoch": 0.8910648714810282,
	"grad_norm": 0.8289669156074524,
	"learning_rate": 4.632595833354105e-06,
	"loss": 0.5832,
	"step": 91
	},
	{
	"epoch": 0.9008567931456548,
	"grad_norm": 0.7501598000526428,
	"learning_rate": 4.6245977829378e-06,
	"loss": 0.6683,
	"step": 92
	},
	{
	"epoch": 0.9106487148102815,
	"grad_norm": 0.8565434813499451,
	"learning_rate": 4.616520671819812e-06,
	"loss": 0.7477,
	"step": 93
	},
	{
	"epoch": 0.9204406364749081,
	"grad_norm": 1.0656261444091797,
	"learning_rate": 4.608364800566241e-06,
	"loss": 0.5368,
	"step": 94
	},
	{
	"epoch": 0.9302325581395349,
	"grad_norm": 1.0636756420135498,
	"learning_rate": 4.600130472674017e-06,
	"loss": 0.8167,
	"step": 95
	},
	{
	"epoch": 0.9400244798041616,
	"grad_norm": 0.962367057800293,
	"learning_rate": 4.5918179945596055e-06,
	"loss": 0.6549,
	"step": 96
	},
	{
	"epoch": 0.9498164014687882,
	"grad_norm": 0.8308187127113342,
	"learning_rate": 4.583427675547602e-06,
	"loss": 0.5565,
	"step": 97
	},
	{
	"epoch": 0.9596083231334149,
	"grad_norm": 0.9818964600563049,
	"learning_rate": 4.574959827859226e-06,
	"loss": 0.7916,
	"step": 98
	},
	{
	"epoch": 0.9694002447980417,
	"grad_norm": 0.777848482131958,
	"learning_rate": 4.566414766600698e-06,
	"loss": 0.506,
	"step": 99
	},
	{
	"epoch": 0.9791921664626683,
	"grad_norm": 0.8741338849067688,
	"learning_rate": 4.557792809751519e-06,
	"loss": 0.5291,
	"step": 100
	},
	{
	"epoch": 0.988984088127295,
	"grad_norm": 0.9098804593086243,
	"learning_rate": 4.549094278152631e-06,
	"loss": 0.923,
	"step": 101
	},
	{
	"epoch": 0.9987760097919217,
	"grad_norm": 0.9387841820716858,
	"learning_rate": 4.540319495494486e-06,
	"loss": 0.7088,
	"step": 102
	},
	{
	"epoch": 1.0,
	"grad_norm": 1.920072078704834,
	"learning_rate": 4.531468788304992e-06,
	"loss": 0.5303,
	"step": 103
	},
	{
	"epoch": 1.0097919216646267,
	"grad_norm": 0.7750993371009827,
	"learning_rate": 4.522542485937369e-06,
	"loss": 0.5491,
	"step": 104
	},
	{
	"epoch": 1.0195838433292534,
	"grad_norm": 0.7841798663139343,
	"learning_rate": 4.513540920557892e-06,
	"loss": 0.6706,
	"step": 105
	},
	{
	"epoch": 1.02937576499388,
	"grad_norm": 1.0083625316619873,
	"learning_rate": 4.504464427133527e-06,
	"loss": 0.5901,
	"step": 106
	},
	{
	"epoch": 1.0391676866585067,
	"grad_norm": 0.9725245833396912,
	"learning_rate": 4.49531334341947e-06,
	"loss": 0.7538,
	"step": 107
	},
	{
	"epoch": 1.0489596083231334,
	"grad_norm": 0.9139071702957153,
	"learning_rate": 4.486088009946575e-06,
	"loss": 0.6618,
	"step": 108
	},
	{
	"epoch": 1.0587515299877601,
	"grad_norm": 0.7784703373908997,
	"learning_rate": 4.476788770008685e-06,
	"loss": 0.5991,
	"step": 109
	},
	{
	"epoch": 1.0685434516523868,
	"grad_norm": 1.2311869859695435,
	"learning_rate": 4.467415969649858e-06,
	"loss": 0.7144,
	"step": 110
	},
	{
	"epoch": 1.0783353733170133,
	"grad_norm": 0.9871697425842285,
	"learning_rate": 4.457969957651485e-06,
	"loss": 0.7294,
	"step": 111
	},
	{
	"epoch": 1.08812729498164,
	"grad_norm": 0.7750707268714905,
	"learning_rate": 4.448451085519314e-06,
	"loss": 0.4198,
	"step": 112
	},
	{
	"epoch": 1.0979192166462668,
	"grad_norm": 0.802689790725708,
	"learning_rate": 4.438859707470376e-06,
	"loss": 0.6132,
	"step": 113
	},
	{
	"epoch": 1.1077111383108935,
	"grad_norm": 1.0714514255523682,
	"learning_rate": 4.42919618041979e-06,
	"loss": 0.7338,
	"step": 114
	},
	{
	"epoch": 1.1175030599755202,
	"grad_norm": 0.846858024597168,
	"learning_rate": 4.419460863967496e-06,
	"loss": 0.7716,
	"step": 115
	},
	{
	"epoch": 1.127294981640147,
	"grad_norm": 0.788904070854187,
	"learning_rate": 4.409654120384863e-06,
	"loss": 0.5973,
	"step": 116
	},
	{
	"epoch": 1.1370869033047735,
	"grad_norm": 1.1219301223754883,
	"learning_rate": 4.399776314601212e-06,
	"loss": 0.6006,
	"step": 117
	},
	{
	"epoch": 1.1468788249694002,
	"grad_norm": 0.7849307656288147,
	"learning_rate": 4.3898278141902396e-06,
	"loss": 0.5086,
	"step": 118
	},
	{
	"epoch": 1.156670746634027,
	"grad_norm": 0.84114009141922,
	"learning_rate": 4.3798089893563335e-06,
	"loss": 0.6369,
	"step": 119
	},
	{
	"epoch": 1.1664626682986536,
	"grad_norm": 0.8064510822296143,
	"learning_rate": 4.3697202129208e-06,
	"loss": 0.6868,
	"step": 120
	},
	{
	"epoch": 1.1762545899632804,
	"grad_norm": 0.8554649353027344,
	"learning_rate": 4.35956186030799e-06,
	"loss": 0.6017,
	"step": 121
	},
	{
	"epoch": 1.1860465116279069,
	"grad_norm": 0.8291575908660889,
	"learning_rate": 4.34933430953133e-06,
	"loss": 0.5997,
	"step": 122
	},
	{
	"epoch": 1.1958384332925336,
	"grad_norm": 0.9356364607810974,
	"learning_rate": 4.339037941179253e-06,
	"loss": 0.782,
	"step": 123
	},
	{
	"epoch": 1.2056303549571603,
	"grad_norm": 1.0298223495483398,
	"learning_rate": 4.328673138401036e-06,
	"loss": 0.817,
	"step": 124
	},
	{
	"epoch": 1.215422276621787,
	"grad_norm": 0.8074529767036438,
	"learning_rate": 4.318240286892544e-06,
	"loss": 0.5649,
	"step": 125
	},
	{
	"epoch": 1.2252141982864138,
	"grad_norm": 0.7682766318321228,
	"learning_rate": 4.307739774881878e-06,
	"loss": 0.6684,
	"step": 126
	},
	{
	"epoch": 1.2350061199510405,
	"grad_norm": 0.7936257123947144,
	"learning_rate": 4.297171993114927e-06,
	"loss": 0.644,
	"step": 127
	},
	{
	"epoch": 1.244798041615667,
	"grad_norm": 0.87111896276474,
	"learning_rate": 4.286537334840825e-06,
	"loss": 0.6471,
	"step": 128
	},
	{
	"epoch": 1.2545899632802937,
	"grad_norm": 0.9743096232414246,
	"learning_rate": 4.275836195797323e-06,
	"loss": 0.5829,
	"step": 129
	},
	{
	"epoch": 1.2643818849449204,
	"grad_norm": 1.1330357789993286,
	"learning_rate": 4.265068974196056e-06,
	"loss": 0.8836,
	"step": 130
	},
	{
	"epoch": 1.2741738066095472,
	"grad_norm": 0.7714349031448364,
	"learning_rate": 4.254236070707734e-06,
	"loss": 0.521,
	"step": 131
	},
	{
	"epoch": 1.2839657282741739,
	"grad_norm": 0.8136262893676758,
	"learning_rate": 4.24333788844722e-06,
	"loss": 0.5759,
	"step": 132
	},
	{
	"epoch": 1.2937576499388004,
	"grad_norm": 0.8456746935844421,
	"learning_rate": 4.23237483295854e-06,
	"loss": 0.7977,
	"step": 133
	},
	{
	"epoch": 1.303549571603427,
	"grad_norm": 0.7905974984169006,
	"learning_rate": 4.221347312199788e-06,
	"loss": 0.5306,
	"step": 134
	},
	{
	"epoch": 1.3133414932680538,
	"grad_norm": 0.8234582543373108,
	"learning_rate": 4.2102557365279435e-06,
	"loss": 0.537,
	"step": 135
	},
	{
	"epoch": 1.3231334149326806,
	"grad_norm": 0.747821569442749,
	"learning_rate": 4.199100518683601e-06,
	"loss": 0.5548,
	"step": 136
	},
	{
	"epoch": 1.3329253365973073,
	"grad_norm": 0.9524441957473755,
	"learning_rate": 4.187882073775615e-06,
	"loss": 0.7255,
	"step": 137
	},
	{
	"epoch": 1.342717258261934,
	"grad_norm": 0.9252872467041016,
	"learning_rate": 4.17660081926565e-06,
	"loss": 0.8311,
	"step": 138
	},
	{
	"epoch": 1.3525091799265607,
	"grad_norm": 0.8391897082328796,
	"learning_rate": 4.165257174952647e-06,
	"loss": 0.8801,
	"step": 139
	},
	{
	"epoch": 1.3623011015911872,
	"grad_norm": 1.0331575870513916,
	"learning_rate": 4.1538515629571985e-06,
	"loss": 0.619,
	"step": 140
	},
	{
	"epoch": 1.372093023255814,
	"grad_norm": 0.9103237986564636,
	"learning_rate": 4.142384407705846e-06,
	"loss": 0.7313,
	"step": 141
	},
	{
	"epoch": 1.3818849449204407,
	"grad_norm": 0.8347257375717163,
	"learning_rate": 4.130856135915282e-06,
	"loss": 0.5443,
	"step": 142
	},
	{
	"epoch": 1.3916768665850674,
	"grad_norm": 0.6917163729667664,
	"learning_rate": 4.119267176576475e-06,
	"loss": 0.5891,
	"step": 143
	},
	{
	"epoch": 1.401468788249694,
	"grad_norm": 0.8349971175193787,
	"learning_rate": 4.107617960938702e-06,
	"loss": 0.5991,
	"step": 144
	},
	{
	"epoch": 1.4112607099143206,
	"grad_norm": 0.8106833100318909,
	"learning_rate": 4.095908922493499e-06,
	"loss": 0.6126,
	"step": 145
	},
	{
	"epoch": 1.4210526315789473,
	"grad_norm": 1.021196722984314,
	"learning_rate": 4.084140496958539e-06,
	"loss": 0.7304,
	"step": 146
	},
	{
	"epoch": 1.430844553243574,
	"grad_norm": 0.7839908003807068,
	"learning_rate": 4.072313122261406e-06,
	"loss": 0.5419,
	"step": 147
	},
	{
	"epoch": 1.4406364749082008,
	"grad_norm": 0.9130982160568237,
	"learning_rate": 4.0604272385233105e-06,
	"loss": 0.5285,
	"step": 148
	},
	{
	"epoch": 1.4504283965728275,
	"grad_norm": 0.8416780829429626,
	"learning_rate": 4.048483288042703e-06,
	"loss": 0.5503,
	"step": 149
	},
	{
	"epoch": 1.460220318237454,
	"grad_norm": 0.9285712838172913,
	"learning_rate": 4.036481715278818e-06,
	"loss": 0.5466,
	"step": 150
	},
	{
	"epoch": 1.4700122399020807,
	"grad_norm": 0.7227409482002258,
	"learning_rate": 4.024422966835137e-06,
	"loss": 0.5007,
	"step": 151
	},
	{
	"epoch": 1.4798041615667075,
	"grad_norm": 1.0086003541946411,
	"learning_rate": 4.0123074914427635e-06,
	"loss": 0.5577,
	"step": 152
	},
	{
	"epoch": 1.4895960832313342,
	"grad_norm": 0.8262959122657776,
	"learning_rate": 4.000135739943735e-06,
	"loss": 0.6587,
	"step": 153
	},
	{
	"epoch": 1.499388004895961,
	"grad_norm": 0.9099556803703308,
	"learning_rate": 3.987908165274233e-06,
	"loss": 0.608,
	"step": 154
	},
	{
	"epoch": 1.5091799265605874,
	"grad_norm": 0.8146927952766418,
	"learning_rate": 3.975625222447742e-06,
	"loss": 0.5707,
	"step": 155
	},
	{
	"epoch": 1.5189718482252141,
	"grad_norm": 1.010314702987671,
	"learning_rate": 3.963287368538105e-06,
	"loss": 0.6462,
	"step": 156
	},
	{
	"epoch": 1.5287637698898409,
	"grad_norm": 0.7961241006851196,
	"learning_rate": 3.9508950626625244e-06,
	"loss": 0.4594,
	"step": 157
	},
	{
	"epoch": 1.5385556915544676,
	"grad_norm": 0.8801811337471008,
	"learning_rate": 3.9384487659644716e-06,
	"loss": 0.8903,
	"step": 158
	},
	{
	"epoch": 1.5483476132190943,
	"grad_norm": 0.9089103937149048,
	"learning_rate": 3.925948941596528e-06,
	"loss": 0.7168,
	"step": 159
	},
	{
	"epoch": 1.558139534883721,
	"grad_norm": 1.2202116250991821,
	"learning_rate": 3.913396054703155e-06,
	"loss": 0.776,
	"step": 160
	},
	{
	"epoch": 1.5679314565483478,
	"grad_norm": 0.9282280206680298,
	"learning_rate": 3.900790572403376e-06,
	"loss": 0.6796,
	"step": 161
	},
	{
	"epoch": 1.5777233782129743,
	"grad_norm": 0.7951483726501465,
	"learning_rate": 3.8881329637734e-06,
	"loss": 0.4882,
	"step": 162
	},
	{
	"epoch": 1.587515299877601,
	"grad_norm": 0.8813434839248657,
	"learning_rate": 3.875423699829168e-06,
	"loss": 0.6126,
	"step": 163
	},
	{
	"epoch": 1.5973072215422277,
	"grad_norm": 0.7947882413864136,
	"learning_rate": 3.862663253508822e-06,
	"loss": 0.5212,
	"step": 164
	},
	{
	"epoch": 1.6070991432068542,
	"grad_norm": 0.9579887986183167,
	"learning_rate": 3.849852099655102e-06,
	"loss": 0.5142,
	"step": 165
	},
	{
	"epoch": 1.616891064871481,
	"grad_norm": 0.7665782570838928,
	"learning_rate": 3.836990714997686e-06,
	"loss": 0.5428,
	"step": 166
	},
	{
	"epoch": 1.6266829865361077,
	"grad_norm": 0.8969327211380005,
	"learning_rate": 3.824079578135442e-06,
	"loss": 0.5935,
	"step": 167
	},
	{
	"epoch": 1.6364749082007344,
	"grad_norm": 0.922908365726471,
	"learning_rate": 3.811119169518624e-06,
	"loss": 0.67,
	"step": 168
	},
	{
	"epoch": 1.646266829865361,
	"grad_norm": 0.8614731431007385,
	"learning_rate": 3.7981099714309856e-06,
	"loss": 0.7935,
	"step": 169
	},
	{
	"epoch": 1.6560587515299878,
	"grad_norm": 0.785549521446228,
	"learning_rate": 3.7850524679718424e-06,
	"loss": 0.4286,
	"step": 170
	},
	{
	"epoch": 1.6658506731946146,
	"grad_norm": 0.9235135912895203,
	"learning_rate": 3.7719471450380518e-06,
	"loss": 0.6131,
	"step": 171
	},
	{
	"epoch": 1.6756425948592413,
	"grad_norm": 0.9928593039512634,
	"learning_rate": 3.758794490305932e-06,
	"loss": 0.6065,
	"step": 172
	},
	{
	"epoch": 1.6854345165238678,
	"grad_norm": 1.1369634866714478,
	"learning_rate": 3.745594993213118e-06,
	"loss": 0.556,
	"step": 173
	},
	{
	"epoch": 1.6952264381884945,
	"grad_norm": 0.7624911665916443,
	"learning_rate": 3.7323491449403444e-06,
	"loss": 0.6322,
	"step": 174
	},
	{
	"epoch": 1.7050183598531212,
	"grad_norm": 0.9393674731254578,
	"learning_rate": 3.719057438393172e-06,
	"loss": 0.7003,
	"step": 175
	},
	{
	"epoch": 1.7148102815177477,
	"grad_norm": 0.7659545540809631,
	"learning_rate": 3.7057203681836407e-06,
	"loss": 0.518,
	"step": 176
	},
	{
	"epoch": 1.7246022031823744,
	"grad_norm": 0.8991119265556335,
	"learning_rate": 3.692338430611869e-06,
	"loss": 0.5977,
	"step": 177
	},
	{
	"epoch": 1.7343941248470012,
	"grad_norm": 0.9369416832923889,
	"learning_rate": 3.6789121236475818e-06,
	"loss": 0.5339,
	"step": 178
	},
	{
	"epoch": 1.744186046511628,
	"grad_norm": 1.080082654953003,
	"learning_rate": 3.665441946911582e-06,
	"loss": 0.6629,
	"step": 179
	},
	{
	"epoch": 1.7539779681762546,
	"grad_norm": 0.7540098428726196,
	"learning_rate": 3.6519284016571567e-06,
	"loss": 0.5146,
	"step": 180
	},
	{
	"epoch": 1.7637698898408813,
	"grad_norm": 0.9198381304740906,
	"learning_rate": 3.638371990751428e-06,
	"loss": 0.6441,
	"step": 181
	},
	{
	"epoch": 1.773561811505508,
	"grad_norm": 0.825372576713562,
	"learning_rate": 3.6247732186566365e-06,
	"loss": 0.6664,
	"step": 182
	},
	{
	"epoch": 1.7833537331701348,
	"grad_norm": 0.789925754070282,
	"learning_rate": 3.61113259141137e-06,
	"loss": 0.6116,
	"step": 183
	},
	{
	"epoch": 1.7931456548347613,
	"grad_norm": 0.9787645936012268,
	"learning_rate": 3.5974506166117355e-06,
	"loss": 0.5526,
	"step": 184
	},
	{
	"epoch": 1.802937576499388,
	"grad_norm": 0.8744960427284241,
	"learning_rate": 3.583727803392468e-06,
	"loss": 0.6993,
	"step": 185
	},
	{
	"epoch": 1.8127294981640147,
	"grad_norm": 0.9353811144828796,
	"learning_rate": 3.5699646624079824e-06,
	"loss": 0.4717,
	"step": 186
	},
	{
	"epoch": 1.8225214198286412,
	"grad_norm": 1.1998875141143799,
	"learning_rate": 3.556161705813378e-06,
	"loss": 0.6727,
	"step": 187
	},
	{
	"epoch": 1.832313341493268,
	"grad_norm": 0.9249100685119629,
	"learning_rate": 3.542319447245372e-06,
	"loss": 0.6676,
	"step": 188
	},
	{
	"epoch": 1.8421052631578947,
	"grad_norm": 0.8355156779289246,
	"learning_rate": 3.528438401803192e-06,
	"loss": 0.5393,
	"step": 189
	},
	{
	"epoch": 1.8518971848225214,
	"grad_norm": 0.9889106750488281,
	"learning_rate": 3.5145190860294043e-06,
	"loss": 0.6742,
	"step": 190
	},
	{
	"epoch": 1.8616891064871481,
	"grad_norm": 0.9249429702758789,
	"learning_rate": 3.500562017890695e-06,
	"loss": 0.6702,
	"step": 191
	},
	{
	"epoch": 1.8714810281517749,
	"grad_norm": 0.9629391431808472,
	"learning_rate": 3.4865677167585942e-06,
	"loss": 0.706,
	"step": 192
	},
	{
	"epoch": 1.8812729498164016,
	"grad_norm": 0.8838512301445007,
	"learning_rate": 3.472536703390148e-06,
	"loss": 0.7781,
	"step": 193
	},
	{
	"epoch": 1.8910648714810283,
	"grad_norm": 1.0404340028762817,
	"learning_rate": 3.4584694999085424e-06,
	"loss": 0.5921,
	"step": 194
	},
	{
	"epoch": 1.9008567931456548,
	"grad_norm": 0.854124903678894,
	"learning_rate": 3.4443666297836715e-06,
	"loss": 0.6825,
	"step": 195
	},
	{
	"epoch": 1.9106487148102815,
	"grad_norm": 0.8705180883407593,
	"learning_rate": 3.430228617812661e-06,
	"loss": 0.5781,
	"step": 196
	},
	{
	"epoch": 1.920440636474908,
	"grad_norm": 0.9270431995391846,
	"learning_rate": 3.416055990100336e-06,
	"loss": 0.4722,
	"step": 197
	},
	{
	"epoch": 1.9302325581395348,
	"grad_norm": 0.7962782382965088,
	"learning_rate": 3.401849274039647e-06,
	"loss": 0.5866,
	"step": 198
	},
	{
	"epoch": 1.9400244798041615,
	"grad_norm": 0.923992395401001,
	"learning_rate": 3.387608998292041e-06,
	"loss": 0.5312,
	"step": 199
	},
	{
	"epoch": 1.9498164014687882,
	"grad_norm": 0.8443131446838379,
	"learning_rate": 3.373335692767793e-06,
	"loss": 0.5156,
	"step": 200
	},
	{
	"epoch": 1.959608323133415,
	"grad_norm": 0.9666808843612671,
	"learning_rate": 3.3590298886062833e-06,
	"loss": 0.7684,
	"step": 201
	},
	{
	"epoch": 1.9694002447980417,
	"grad_norm": 0.896936297416687,
	"learning_rate": 3.3446921181562326e-06,
	"loss": 0.5143,
	"step": 202
	},
	{
	"epoch": 1.9791921664626684,
	"grad_norm": 0.7992851734161377,
	"learning_rate": 3.330322914955897e-06,
	"loss": 0.6279,
	"step": 203
	},
	{
	"epoch": 1.988984088127295,
	"grad_norm": 0.8046123385429382,
	"learning_rate": 3.315922813713209e-06,
	"loss": 0.6242,
	"step": 204
	},
	{
	"epoch": 1.9987760097919218,
	"grad_norm": 0.8214249610900879,
	"learning_rate": 3.301492350285879e-06,
	"loss": 0.6508,
	"step": 205
	},
	{
	"epoch": 2.0,
	"grad_norm": 2.8854775428771973,
	"learning_rate": 3.2870320616614626e-06,
	"loss": 1.1335,
	"step": 206
	},
	{
	"epoch": 2.0097919216646267,
	"grad_norm": 1.1363986730575562,
	"learning_rate": 3.272542485937369e-06,
	"loss": 0.5857,
	"step": 207
	},
	{
	"epoch": 2.0195838433292534,
	"grad_norm": 0.9117214679718018,
	"learning_rate": 3.2580241623008426e-06,
	"loss": 0.4788,
	"step": 208
	},
	{
	"epoch": 2.02937576499388,
	"grad_norm": 0.9110085368156433,
	"learning_rate": 3.243477631008897e-06,
	"loss": 0.7518,
	"step": 209
	},
	{
	"epoch": 2.039167686658507,
	"grad_norm": 0.7805931568145752,
	"learning_rate": 3.228903433368212e-06,
	"loss": 0.6359,
	"step": 210
	},
	{
	"epoch": 2.0489596083231336,
	"grad_norm": 0.7718285918235779,
	"learning_rate": 3.21430211171499e-06,
	"loss": 0.6026,
	"step": 211
	},
	{
	"epoch": 2.05875152998776,
	"grad_norm": 1.000604271888733,
	"learning_rate": 3.1996742093947724e-06,
	"loss": 0.65,
	"step": 212
	},
	{
	"epoch": 2.0685434516523866,
	"grad_norm": 1.1084641218185425,
	"learning_rate": 3.185020270742225e-06,
	"loss": 0.4994,
	"step": 213
	},
	{
	"epoch": 2.0783353733170133,
	"grad_norm": 0.8272696137428284,
	"learning_rate": 3.1703408410608777e-06,
	"loss": 0.565,
	"step": 214
	},
	{
	"epoch": 2.08812729498164,
	"grad_norm": 0.8151060342788696,
	"learning_rate": 3.155636466602836e-06,
	"loss": 0.6464,
	"step": 215
	},
	{
	"epoch": 2.097919216646267,
	"grad_norm": 0.919487714767456,
	"learning_rate": 3.1409076945484513e-06,
	"loss": 0.6338,
	"step": 216
	},
	{
	"epoch": 2.1077111383108935,
	"grad_norm": 0.874394416809082,
	"learning_rate": 3.1261550729859602e-06,
	"loss": 0.6066,
	"step": 217
	},
	{
	"epoch": 2.1175030599755202,
	"grad_norm": 1.0532450675964355,
	"learning_rate": 3.1113791508910913e-06,
	"loss": 0.8078,
	"step": 218
	},
	{
	"epoch": 2.127294981640147,
	"grad_norm": 0.8656905293464661,
	"learning_rate": 3.096580478106631e-06,
	"loss": 0.4196,
	"step": 219
	},
	{
	"epoch": 2.1370869033047737,
	"grad_norm": 0.9572075605392456,
	"learning_rate": 3.0817596053219697e-06,
	"loss": 0.602,
	"step": 220
	},
	{
	"epoch": 2.1468788249694004,
	"grad_norm": 0.8143496513366699,
	"learning_rate": 3.066917084052603e-06,
	"loss": 0.4429,
	"step": 221
	},
	{
	"epoch": 2.1566707466340267,
	"grad_norm": 0.9006687998771667,
	"learning_rate": 3.0520534666196134e-06,
	"loss": 0.6761,
	"step": 222
	},
	{
	"epoch": 2.1664626682986534,
	"grad_norm": 1.1604130268096924,
	"learning_rate": 3.037169306129115e-06,
	"loss": 0.5316,
	"step": 223
	},
	{
	"epoch": 2.17625458996328,
	"grad_norm": 1.090090036392212,
	"learning_rate": 3.0222651564516715e-06,
	"loss": 0.5979,
	"step": 224
	},
	{
	"epoch": 2.186046511627907,
	"grad_norm": 1.104018211364746,
	"learning_rate": 3.0073415722016875e-06,
	"loss": 0.5426,
	"step": 225
	},
	{
	"epoch": 2.1958384332925336,
	"grad_norm": 0.9593565464019775,
	"learning_rate": 2.9923991087167657e-06,
	"loss": 0.6325,
	"step": 226
	},
	{
	"epoch": 2.2056303549571603,
	"grad_norm": 0.8623105883598328,
	"learning_rate": 2.977438322037046e-06,
	"loss": 0.4547,
	"step": 227
	},
	{
	"epoch": 2.215422276621787,
	"grad_norm": 0.8678483366966248,
	"learning_rate": 2.9624597688845126e-06,
	"loss": 0.5359,
	"step": 228
	},
	{
	"epoch": 2.2252141982864138,
	"grad_norm": 0.9539102911949158,
	"learning_rate": 2.9474640066422757e-06,
	"loss": 0.7405,
	"step": 229
	},
	{
	"epoch": 2.2350061199510405,
	"grad_norm": 0.8902791142463684,
	"learning_rate": 2.9324515933338343e-06,
	"loss": 0.4307,
	"step": 230
	},
	{
	"epoch": 2.244798041615667,
	"grad_norm": 0.9679104089736938,
	"learning_rate": 2.917423087602306e-06,
	"loss": 0.5503,
	"step": 231
	},
	{
	"epoch": 2.254589963280294,
	"grad_norm": 0.7148194909095764,
	"learning_rate": 2.9023790486896404e-06,
	"loss": 0.4171,
	"step": 232
	},
	{
	"epoch": 2.26438188494492,
	"grad_norm": 0.8577345609664917,
	"learning_rate": 2.887320036415811e-06,
	"loss": 0.4699,
	"step": 233
	},
	{
	"epoch": 2.274173806609547,
	"grad_norm": 1.0629149675369263,
	"learning_rate": 2.87224661115798e-06,
	"loss": 0.6527,
	"step": 234
	},
	{
	"epoch": 2.2839657282741737,
	"grad_norm": 1.0864779949188232,
	"learning_rate": 2.8571593338296473e-06,
	"loss": 0.6231,
	"step": 235
	},
	{
	"epoch": 2.2937576499388004,
	"grad_norm": 0.8141933083534241,
	"learning_rate": 2.842058765859776e-06,
	"loss": 0.7346,
	"step": 236
	},
	{
	"epoch": 2.303549571603427,
	"grad_norm": 0.8865804076194763,
	"learning_rate": 2.826945469171903e-06,
	"loss": 0.6919,
	"step": 237
	},
	{
	"epoch": 2.313341493268054,
	"grad_norm": 0.9080017805099487,
	"learning_rate": 2.8118200061632273e-06,
	"loss": 0.5312,
	"step": 238
	},
	{
	"epoch": 2.3231334149326806,
	"grad_norm": 0.6919491291046143,
	"learning_rate": 2.7966829396836804e-06,
	"loss": 0.4845,
	"step": 239
	},
	{
	"epoch": 2.3329253365973073,
	"grad_norm": 0.7912428975105286,
	"learning_rate": 2.781534833014985e-06,
	"loss": 0.6016,
	"step": 240
	},
	{
	"epoch": 2.342717258261934,
	"grad_norm": 0.9763979911804199,
	"learning_rate": 2.7663762498496905e-06,
	"loss": 0.6039,
	"step": 241
	},
	{
	"epoch": 2.3525091799265607,
	"grad_norm": 0.8945314884185791,
	"learning_rate": 2.7512077542702005e-06,
	"loss": 0.4668,
	"step": 242
	},
	{
	"epoch": 2.3623011015911874,
	"grad_norm": 0.8270300626754761,
	"learning_rate": 2.736029910727777e-06,
	"loss": 0.5889,
	"step": 243
	},
	{
	"epoch": 2.3720930232558137,
	"grad_norm": 0.9622857570648193,
	"learning_rate": 2.720843284021543e-06,
	"loss": 0.5138,
	"step": 244
	},
	{
	"epoch": 2.3818849449204405,
	"grad_norm": 0.8391791582107544,
	"learning_rate": 2.705648439277459e-06,
	"loss": 0.6747,
	"step": 245
	},
	{
	"epoch": 2.391676866585067,
	"grad_norm": 0.870780885219574,
	"learning_rate": 2.6904459419272955e-06,
	"loss": 0.5906,
	"step": 246
	},
	{
	"epoch": 2.401468788249694,
	"grad_norm": 0.8945671319961548,
	"learning_rate": 2.6752363576875933e-06,
	"loss": 0.7187,
	"step": 247
	},
	{
	"epoch": 2.4112607099143206,
	"grad_norm": 0.7328640222549438,
	"learning_rate": 2.660020252538611e-06,
	"loss": 0.4277,
	"step": 248
	},
	{
	"epoch": 2.4210526315789473,
	"grad_norm": 0.7584955096244812,
	"learning_rate": 2.6447981927032634e-06,
	"loss": 0.5469,
	"step": 249
	},
	{
	"epoch": 2.430844553243574,
	"grad_norm": 0.7713475227355957,
	"learning_rate": 2.629570744626052e-06,
	"loss": 0.5101,
	"step": 250
	},
	{
	"epoch": 2.440636474908201,
	"grad_norm": 0.9404786229133606,
	"learning_rate": 2.614338474951987e-06,
	"loss": 0.6086,
	"step": 251
	},
	{
	"epoch": 2.4504283965728275,
	"grad_norm": 1.0046964883804321,
	"learning_rate": 2.5991019505054965e-06,
	"loss": 0.6417,
	"step": 252
	},
	{
	"epoch": 2.4602203182374542,
	"grad_norm": 0.8127304911613464,
	"learning_rate": 2.5838617382693415e-06,
	"loss": 0.577,
	"step": 253
	},
	{
	"epoch": 2.470012239902081,
	"grad_norm": 0.7843394875526428,
	"learning_rate": 2.5686184053635127e-06,
	"loss": 0.4163,
	"step": 254
	},
	{
	"epoch": 2.4798041615667072,
	"grad_norm": 0.8961329460144043,
	"learning_rate": 2.5533725190241255e-06,
	"loss": 0.5953,
	"step": 255
	},
	{
	"epoch": 2.489596083231334,
	"grad_norm": 0.8873429894447327,
	"learning_rate": 2.538124646582315e-06,
	"loss": 0.6428,
	"step": 256
	},
	{
	"epoch": 2.4993880048959607,
	"grad_norm": 0.956925094127655,
	"learning_rate": 2.522875355443124e-06,
	"loss": 0.5783,
	"step": 257
	},
	{
	"epoch": 2.5091799265605874,
	"grad_norm": 1.177071452140808,
	"learning_rate": 2.507625213064386e-06,
	"loss": 0.709,
	"step": 258
	},
	{
	"epoch": 2.518971848225214,
	"grad_norm": 0.9448588490486145,
	"learning_rate": 2.492374786935614e-06,
	"loss": 0.6386,
	"step": 259
	},
	{
	"epoch": 2.528763769889841,
	"grad_norm": 0.8508694767951965,
	"learning_rate": 2.4771246445568763e-06,
	"loss": 0.5688,
	"step": 260
	},
	{
	"epoch": 2.5385556915544676,
	"grad_norm": 0.9512906074523926,
	"learning_rate": 2.4618753534176854e-06,
	"loss": 0.6807,
	"step": 261
	},
	{
	"epoch": 2.5483476132190943,
	"grad_norm": 0.9921900629997253,
	"learning_rate": 2.4466274809758757e-06,
	"loss": 0.6139,
	"step": 262
	},
	{
	"epoch": 2.558139534883721,
	"grad_norm": 0.9332875609397888,
	"learning_rate": 2.431381594636488e-06,
	"loss": 0.4453,
	"step": 263
	},
	{
	"epoch": 2.5679314565483478,
	"grad_norm": 0.8664917349815369,
	"learning_rate": 2.4161382617306585e-06,
	"loss": 0.5011,
	"step": 264
	},
	{
	"epoch": 2.5777233782129745,
	"grad_norm": 0.8101782202720642,
	"learning_rate": 2.4008980494945044e-06,
	"loss": 0.4799,
	"step": 265
	},
	{
	"epoch": 2.5875152998776008,
	"grad_norm": 1.2083535194396973,
	"learning_rate": 2.385661525048014e-06,
	"loss": 0.4982,
	"step": 266
	},
	{
	"epoch": 2.597307221542228,
	"grad_norm": 0.9032344222068787,
	"learning_rate": 2.3704292553739487e-06,
	"loss": 0.6806,
	"step": 267
	},
	{
	"epoch": 2.607099143206854,
	"grad_norm": 1.1612454652786255,
	"learning_rate": 2.3552018072967375e-06,
	"loss": 1.0154,
	"step": 268
	},
	{
	"epoch": 2.616891064871481,
	"grad_norm": 0.8985655903816223,
	"learning_rate": 2.3399797474613894e-06,
	"loss": 0.5293,
	"step": 269
	},
	{
	"epoch": 2.6266829865361077,
	"grad_norm": 0.7207229137420654,
	"learning_rate": 2.324763642312407e-06,
	"loss": 0.5278,
	"step": 270
	},
	{
	"epoch": 2.6364749082007344,
	"grad_norm": 0.9060878753662109,
	"learning_rate": 2.3095540580727054e-06,
	"loss": 0.5756,
	"step": 271
	},
	{
	"epoch": 2.646266829865361,
	"grad_norm": 1.1018798351287842,
	"learning_rate": 2.294351560722542e-06,
	"loss": 0.6635,
	"step": 272
	},
	{
	"epoch": 2.656058751529988,
	"grad_norm": 0.7798595428466797,
	"learning_rate": 2.279156715978457e-06,
	"loss": 0.7283,
	"step": 273
	},
	{
	"epoch": 2.6658506731946146,
	"grad_norm": 0.8930055499076843,
	"learning_rate": 2.263970089272223e-06,
	"loss": 0.4787,
	"step": 274
	},
	{
	"epoch": 2.6756425948592413,
	"grad_norm": 0.8998661637306213,
	"learning_rate": 2.2487922457298007e-06,
	"loss": 0.7048,
	"step": 275
	},
	{
	"epoch": 2.685434516523868,
	"grad_norm": 0.9193498492240906,
	"learning_rate": 2.2336237501503103e-06,
	"loss": 0.4771,
	"step": 276
	},
	{
	"epoch": 2.6952264381884943,
	"grad_norm": 0.9432845711708069,
	"learning_rate": 2.2184651669850164e-06,
	"loss": 0.691,
	"step": 277
	},
	{
	"epoch": 2.7050183598531214,
	"grad_norm": 0.8951868414878845,
	"learning_rate": 2.20331706031632e-06,
	"loss": 0.5048,
	"step": 278
	},
	{
	"epoch": 2.7148102815177477,
	"grad_norm": 0.9745557904243469,
	"learning_rate": 2.1881799938367735e-06,
	"loss": 0.6715,
	"step": 279
	},
	{
	"epoch": 2.7246022031823744,
	"grad_norm": 0.7149460315704346,
	"learning_rate": 2.173054530828098e-06,
	"loss": 0.6103,
	"step": 280
	},
	{
	"epoch": 2.734394124847001,
	"grad_norm": 0.8981413841247559,
	"learning_rate": 2.157941234140225e-06,
	"loss": 0.6661,
	"step": 281
	},
	{
	"epoch": 2.744186046511628,
	"grad_norm": 0.8984580039978027,
	"learning_rate": 2.142840666170354e-06,
	"loss": 0.4824,
	"step": 282
	},
	{
	"epoch": 2.7539779681762546,
	"grad_norm": 0.7966169714927673,
	"learning_rate": 2.1277533888420203e-06,
	"loss": 0.6415,
	"step": 283
	},
	{
	"epoch": 2.7637698898408813,
	"grad_norm": 0.8332808017730713,
	"learning_rate": 2.1126799635841897e-06,
	"loss": 0.5478,
	"step": 284
	},
	{
	"epoch": 2.773561811505508,
	"grad_norm": 0.8504648804664612,
	"learning_rate": 2.0976209513103604e-06,
	"loss": 0.5287,
	"step": 285
	},
	{
	"epoch": 2.783353733170135,
	"grad_norm": 0.7362974286079407,
	"learning_rate": 2.0825769123976954e-06,
	"loss": 0.4472,
	"step": 286
	},
	{
	"epoch": 2.7931456548347615,
	"grad_norm": 1.0003184080123901,
	"learning_rate": 2.0675484066661666e-06,
	"loss": 0.7951,
	"step": 287
	},
	{
	"epoch": 2.802937576499388,
	"grad_norm": 0.9405409693717957,
	"learning_rate": 2.0525359933577243e-06,
	"loss": 0.678,
	"step": 288
	},
	{
	"epoch": 2.812729498164015,
	"grad_norm": 0.861124575138092,
	"learning_rate": 2.0375402311154886e-06,
	"loss": 0.6691,
	"step": 289
	},
	{
	"epoch": 2.8225214198286412,
	"grad_norm": 0.8977060317993164,
	"learning_rate": 2.022561677962955e-06,
	"loss": 0.5294,
	"step": 290
	},
	{
	"epoch": 2.832313341493268,
	"grad_norm": 0.9480942487716675,
	"learning_rate": 2.0076008912832355e-06,
	"loss": 0.5251,
	"step": 291
	},
	{
	"epoch": 2.8421052631578947,
	"grad_norm": 0.9462578296661377,
	"learning_rate": 1.9926584277983134e-06,
	"loss": 0.5192,
	"step": 292
	},
	{
	"epoch": 2.8518971848225214,
	"grad_norm": 0.845146894454956,
	"learning_rate": 1.9777348435483285e-06,
	"loss": 0.6675,
	"step": 293
	},
	{
	"epoch": 2.861689106487148,
	"grad_norm": 0.7902911305427551,
	"learning_rate": 1.9628306938708857e-06,
	"loss": 0.5263,
	"step": 294
	},
	{
	"epoch": 2.871481028151775,
	"grad_norm": 0.8663612008094788,
	"learning_rate": 1.947946533380387e-06,
	"loss": 0.4762,
	"step": 295
	},
	{
	"epoch": 2.8812729498164016,
	"grad_norm": 0.8681756854057312,
	"learning_rate": 1.933082915947398e-06,
	"loss": 0.4717,
	"step": 296
	},
	{
	"epoch": 2.8910648714810283,
	"grad_norm": 0.7455431222915649,
	"learning_rate": 1.9182403946780316e-06,
	"loss": 0.5162,
	"step": 297
	},
	{
	"epoch": 2.900856793145655,
	"grad_norm": 0.7615146636962891,
	"learning_rate": 1.903419521893369e-06,
	"loss": 0.5126,
	"step": 298
	},
	{
	"epoch": 2.9106487148102813,
	"grad_norm": 0.9317947030067444,
	"learning_rate": 1.8886208491089095e-06,
	"loss": 0.5979,
	"step": 299
	},
	{
	"epoch": 2.920440636474908,
	"grad_norm": 0.7302988171577454,
	"learning_rate": 1.8738449270140404e-06,
	"loss": 0.5472,
	"step": 300
	},
	{
	"epoch": 2.9302325581395348,
	"grad_norm": 0.9182233214378357,
	"learning_rate": 1.8590923054515504e-06,
	"loss": 0.5403,
	"step": 301
	},
	{
	"epoch": 2.9400244798041615,
	"grad_norm": 0.9749161601066589,
	"learning_rate": 1.8443635333971643e-06,
	"loss": 0.5792,
	"step": 302
	},
	{
	"epoch": 2.949816401468788,
	"grad_norm": 0.8248334527015686,
	"learning_rate": 1.8296591589391227e-06,
	"loss": 0.5705,
	"step": 303
	},
	{
	"epoch": 2.959608323133415,
	"grad_norm": 0.8621752262115479,
	"learning_rate": 1.8149797292577757e-06,
	"loss": 0.5825,
	"step": 304
	},
	{
	"epoch": 2.9694002447980417,
	"grad_norm": 0.7498932480812073,
	"learning_rate": 1.8003257906052284e-06,
	"loss": 0.468,
	"step": 305
	},
	{
	"epoch": 2.9791921664626684,
	"grad_norm": 0.8821313977241516,
	"learning_rate": 1.7856978882850112e-06,
	"loss": 0.4225,
	"step": 306
	},
	{
	"epoch": 2.988984088127295,
	"grad_norm": 0.9697312712669373,
	"learning_rate": 1.771096566631788e-06,
	"loss": 0.6226,
	"step": 307
	},
	{
	"epoch": 2.998776009791922,
	"grad_norm": 0.6282007098197937,
	"learning_rate": 1.7565223689911038e-06,
	"loss": 0.3947,
	"step": 308
	},
	{
	"epoch": 3.0,
	"grad_norm": 2.0759425163269043,
	"learning_rate": 1.741975837699158e-06,
	"loss": 0.3904,
	"step": 309
	},
	{
	"epoch": 3.0097919216646267,
	"grad_norm": 0.7113844156265259,
	"learning_rate": 1.7274575140626318e-06,
	"loss": 0.4556,
	"step": 310
	},
	{
	"epoch": 3.0195838433292534,
	"grad_norm": 0.7638390064239502,
	"learning_rate": 1.7129679383385384e-06,
	"loss": 0.4479,
	"step": 311
	},
	{
	"epoch": 3.02937576499388,
	"grad_norm": 0.8067305088043213,
	"learning_rate": 1.698507649714121e-06,
	"loss": 0.5066,
	"step": 312
	},
	{
	"epoch": 3.039167686658507,
	"grad_norm": 0.7775254249572754,
	"learning_rate": 1.6840771862867922e-06,
	"loss": 0.6543,
	"step": 313
	},
	{
	"epoch": 3.0489596083231336,
	"grad_norm": 0.7211126685142517,
	"learning_rate": 1.6696770850441036e-06,
	"loss": 0.3549,
	"step": 314
	},
	{
	"epoch": 3.05875152998776,
	"grad_norm": 1.0748693943023682,
	"learning_rate": 1.6553078818437678e-06,
	"loss": 0.4546,
	"step": 315
	},
	{
	"epoch": 3.0685434516523866,
	"grad_norm": 0.9343094229698181,
	"learning_rate": 1.6409701113937182e-06,
	"loss": 0.5258,
	"step": 316
	},
	{
	"epoch": 3.0783353733170133,
	"grad_norm": 0.8405662178993225,
	"learning_rate": 1.626664307232207e-06,
	"loss": 0.4435,
	"step": 317
	},
	{
	"epoch": 3.08812729498164,
	"grad_norm": 0.7024164199829102,
	"learning_rate": 1.6123910017079591e-06,
	"loss": 0.3879,
	"step": 318
	},
	{
	"epoch": 3.097919216646267,
	"grad_norm": 0.9318003058433533,
	"learning_rate": 1.598150725960354e-06,
	"loss": 0.5145,
	"step": 319
	},
	{
	"epoch": 3.1077111383108935,
	"grad_norm": 0.8346633315086365,
	"learning_rate": 1.583944009899665e-06,
	"loss": 0.4832,
	"step": 320
	},
	{
	"epoch": 3.1175030599755202,
	"grad_norm": 0.8605434894561768,
	"learning_rate": 1.5697713821873401e-06,
	"loss": 0.4306,
	"step": 321
	},
	{
	"epoch": 3.127294981640147,
	"grad_norm": 0.8628739714622498,
	"learning_rate": 1.555633370216329e-06,
	"loss": 0.4909,
	"step": 322
	},
	{
	"epoch": 3.1370869033047737,
	"grad_norm": 0.8787243962287903,
	"learning_rate": 1.5415305000914587e-06,
	"loss": 0.7018,
	"step": 323
	},
	{
	"epoch": 3.1468788249694004,
	"grad_norm": 0.8786433935165405,
	"learning_rate": 1.5274632966098527e-06,
	"loss": 0.6932,
	"step": 324
	},
	{
	"epoch": 3.1566707466340267,
	"grad_norm": 0.7585502862930298,
	"learning_rate": 1.5134322832414066e-06,
	"loss": 0.5931,
	"step": 325
	},
	{
	"epoch": 3.1664626682986534,
	"grad_norm": 0.9854638576507568,
	"learning_rate": 1.499437982109305e-06,
	"loss": 0.6011,
	"step": 326
	},
	{
	"epoch": 3.17625458996328,
	"grad_norm": 0.925818920135498,
	"learning_rate": 1.4854809139705961e-06,
	"loss": 0.5138,
	"step": 327
	},
	{
	"epoch": 3.186046511627907,
	"grad_norm": 0.7765573263168335,
	"learning_rate": 1.4715615981968088e-06,
	"loss": 0.5687,
	"step": 328
	},
	{
	"epoch": 3.1958384332925336,
	"grad_norm": 0.908775269985199,
	"learning_rate": 1.4576805527546293e-06,
	"loss": 0.5769,
	"step": 329
	},
	{
	"epoch": 3.2056303549571603,
	"grad_norm": 0.8740766644477844,
	"learning_rate": 1.4438382941866224e-06,
	"loss": 0.4665,
	"step": 330
	},
	{
	"epoch": 3.215422276621787,
	"grad_norm": 0.7475005984306335,
	"learning_rate": 1.430035337592018e-06,
	"loss": 0.7062,
	"step": 331
	},
	{
	"epoch": 3.2252141982864138,
	"grad_norm": 0.851504385471344,
	"learning_rate": 1.4162721966075323e-06,
	"loss": 0.4735,
	"step": 332
	},
	{
	"epoch": 3.2350061199510405,
	"grad_norm": 0.9121419191360474,
	"learning_rate": 1.4025493833882645e-06,
	"loss": 0.4578,
	"step": 333
	},
	{
	"epoch": 3.244798041615667,
	"grad_norm": 1.0339323282241821,
	"learning_rate": 1.3888674085886302e-06,
	"loss": 0.6386,
	"step": 334
	},
	{
	"epoch": 3.254589963280294,
	"grad_norm": 0.7751194834709167,
	"learning_rate": 1.3752267813433645e-06,
	"loss": 0.4681,
	"step": 335
	},
	{
	"epoch": 3.26438188494492,
	"grad_norm": 0.8739397525787354,
	"learning_rate": 1.3616280092485719e-06,
	"loss": 0.5172,
	"step": 336
	},
	{
	"epoch": 3.274173806609547,
	"grad_norm": 0.7891517877578735,
	"learning_rate": 1.3480715983428433e-06,
	"loss": 0.3973,
	"step": 337
	},
	{
	"epoch": 3.2839657282741737,
	"grad_norm": 0.9016935229301453,
	"learning_rate": 1.334558053088419e-06,
	"loss": 0.5252,
	"step": 338
	},
	{
	"epoch": 3.2937576499388004,
	"grad_norm": 1.0618005990982056,
	"learning_rate": 1.3210878763524186e-06,
	"loss": 0.4908,
	"step": 339
	},
	{
	"epoch": 3.303549571603427,
	"grad_norm": 0.8356817960739136,
	"learning_rate": 1.307661569388132e-06,
	"loss": 0.3408,
	"step": 340
	},
	{
	"epoch": 3.313341493268054,
	"grad_norm": 0.8103477954864502,
	"learning_rate": 1.2942796318163595e-06,
	"loss": 0.5309,
	"step": 341
	},
	{
	"epoch": 3.3231334149326806,
	"grad_norm": 0.9388753175735474,
	"learning_rate": 1.2809425616068288e-06,
	"loss": 0.6679,
	"step": 342
	},
	{
	"epoch": 3.3329253365973073,
	"grad_norm": 0.7531409859657288,
	"learning_rate": 1.2676508550596562e-06,
	"loss": 0.541,
	"step": 343
	},
	{
	"epoch": 3.342717258261934,
	"grad_norm": 0.9515266418457031,
	"learning_rate": 1.2544050067868834e-06,
	"loss": 0.5489,
	"step": 344
	},
	{
	"epoch": 3.3525091799265607,
	"grad_norm": 0.9053857326507568,
	"learning_rate": 1.2412055096940692e-06,
	"loss": 0.7047,
	"step": 345
	},
	{
	"epoch": 3.3623011015911874,
	"grad_norm": 0.8061960935592651,
	"learning_rate": 1.2280528549619487e-06,
	"loss": 0.6067,
	"step": 346
	},
	{
	"epoch": 3.3720930232558137,
	"grad_norm": 0.9094511866569519,
	"learning_rate": 1.2149475320281578e-06,
	"loss": 0.7628,
	"step": 347
	},
	{
	"epoch": 3.3818849449204405,
	"grad_norm": 0.7460764646530151,
	"learning_rate": 1.2018900285690148e-06,
	"loss": 0.4011,
	"step": 348
	},
	{
	"epoch": 3.391676866585067,
	"grad_norm": 0.9510339498519897,
	"learning_rate": 1.188880830481377e-06,
	"loss": 0.6808,
	"step": 349
	},
	{
	"epoch": 3.401468788249694,
	"grad_norm": 1.0849850177764893,
	"learning_rate": 1.1759204218645577e-06,
	"loss": 0.6753,
	"step": 350
	},
	{
	"epoch": 3.4112607099143206,
	"grad_norm": 0.8478190898895264,
	"learning_rate": 1.1630092850023148e-06,
	"loss": 0.5099,
	"step": 351
	},
	{
	"epoch": 3.4210526315789473,
	"grad_norm": 0.8813092112541199,
	"learning_rate": 1.1501479003448992e-06,
	"loss": 0.605,
	"step": 352
	},
	{
	"epoch": 3.430844553243574,
	"grad_norm": 0.8529421091079712,
	"learning_rate": 1.1373367464911798e-06,
	"loss": 0.553,
	"step": 353
	},
	{
	"epoch": 3.440636474908201,
	"grad_norm": 0.9015612602233887,
	"learning_rate": 1.1245763001708326e-06,
	"loss": 0.4921,
	"step": 354
	},
	{
	"epoch": 3.4504283965728275,
	"grad_norm": 1.0200309753417969,
	"learning_rate": 1.1118670362266003e-06,
	"loss": 0.4484,
	"step": 355
	},
	{
	"epoch": 3.4602203182374542,
	"grad_norm": 0.7840336561203003,
	"learning_rate": 1.0992094275966256e-06,
	"loss": 0.4734,
	"step": 356
	},
	{
	"epoch": 3.470012239902081,
	"grad_norm": 0.9333547949790955,
	"learning_rate": 1.0866039452968464e-06,
	"loss": 0.555,
	"step": 357
	},
	{
	"epoch": 3.4798041615667072,
	"grad_norm": 0.9499984979629517,
	"learning_rate": 1.074051058403472e-06,
	"loss": 0.4556,
	"step": 358
	},
	{
	"epoch": 3.489596083231334,
	"grad_norm": 0.7713193893432617,
	"learning_rate": 1.061551234035529e-06,
	"loss": 0.658,
	"step": 359
	},
	{
	"epoch": 3.4993880048959607,
	"grad_norm": 0.8462610244750977,
	"learning_rate": 1.0491049373374762e-06,
	"loss": 0.5014,
	"step": 360
	},
	{
	"epoch": 3.5091799265605874,
	"grad_norm": 0.8270758986473083,
	"learning_rate": 1.0367126314618946e-06,
	"loss": 0.5163,
	"step": 361
	},
	{
	"epoch": 3.518971848225214,
	"grad_norm": 0.8598589301109314,
	"learning_rate": 1.024374777552258e-06,
	"loss": 0.979,
	"step": 362
	},
	{
	"epoch": 3.528763769889841,
	"grad_norm": 0.9260916113853455,
	"learning_rate": 1.0120918347257669e-06,
	"loss": 0.6514,
	"step": 363
	},
	{
	"epoch": 3.5385556915544676,
	"grad_norm": 1.0365149974822998,
	"learning_rate": 9.998642600562664e-07,
	"loss": 0.6321,
	"step": 364
	},
	{
	"epoch": 3.5483476132190943,
	"grad_norm": 0.7237308025360107,
	"learning_rate": 9.876925085572365e-07,
	"loss": 0.5488,
	"step": 365
	},
	{
	"epoch": 3.558139534883721,
	"grad_norm": 0.8265156149864197,
	"learning_rate": 9.755770331648642e-07,
	"loss": 0.5855,
	"step": 366
	},
	{
	"epoch": 3.5679314565483478,
	"grad_norm": 0.9144641160964966,
	"learning_rate": 9.635182847211827e-07,
	"loss": 0.5351,
	"step": 367
	},
	{
	"epoch": 3.5777233782129745,
	"grad_norm": 0.940665066242218,
	"learning_rate": 9.51516711957298e-07,
	"loss": 0.5807,
	"step": 368
	},
	{
	"epoch": 3.5875152998776008,
	"grad_norm": 0.8486706614494324,
	"learning_rate": 9.395727614766903e-07,
	"loss": 0.5703,
	"step": 369
	},
	{
	"epoch": 3.597307221542228,
	"grad_norm": 1.0740798711776733,
	"learning_rate": 9.276868777385942e-07,
	"loss": 0.4291,
	"step": 370
	},
	{
	"epoch": 3.607099143206854,
	"grad_norm": 0.8575485944747925,
	"learning_rate": 9.158595030414621e-07,
	"loss": 0.4149,
	"step": 371
	},
	{
	"epoch": 3.616891064871481,
	"grad_norm": 0.8195849657058716,
	"learning_rate": 9.040910775065015e-07,
	"loss": 0.4979,
	"step": 372
	},
	{
	"epoch": 3.6266829865361077,
	"grad_norm": 0.7140581011772156,
	"learning_rate": 8.923820390612991e-07,
	"loss": 0.4553,
	"step": 373
	},
	{
	"epoch": 3.6364749082007344,
	"grad_norm": 0.944703996181488,
	"learning_rate": 8.807328234235254e-07,
	"loss": 0.4933,
	"step": 374
	},
	{
	"epoch": 3.646266829865361,
	"grad_norm": 0.8490734696388245,
	"learning_rate": 8.691438640847177e-07,
	"loss": 0.5576,
	"step": 375
	},
	{
	"epoch": 3.656058751529988,
	"grad_norm": 1.0354382991790771,
	"learning_rate": 8.576155922941548e-07,
	"loss": 0.4687,
	"step": 376
	},
	{
	"epoch": 3.6658506731946146,
	"grad_norm": 0.952894926071167,
	"learning_rate": 8.461484370428025e-07,
	"loss": 0.6824,
	"step": 377
	},
	{
	"epoch": 3.6756425948592413,
	"grad_norm": 0.856124997138977,
	"learning_rate": 8.347428250473541e-07,
	"loss": 0.3794,
	"step": 378
	},
	{
	"epoch": 3.685434516523868,
	"grad_norm": 0.8288161754608154,
	"learning_rate": 8.233991807343497e-07,
	"loss": 0.6997,
	"step": 379
	},
	{
	"epoch": 3.6952264381884943,
	"grad_norm": 0.8297052383422852,
	"learning_rate": 8.12117926224385e-07,
	"loss": 0.4001,
	"step": 380
	},
	{
	"epoch": 3.7050183598531214,
	"grad_norm": 0.8977110385894775,
	"learning_rate": 8.008994813163995e-07,
	"loss": 0.641,
	"step": 381
	},
	{
	"epoch": 3.7148102815177477,
	"grad_norm": 1.0157899856567383,
	"learning_rate": 7.897442634720576e-07,
	"loss": 0.504,
	"step": 382
	},
	{
	"epoch": 3.7246022031823744,
	"grad_norm": 0.9636339545249939,
	"learning_rate": 7.786526878002126e-07,
	"loss": 0.7636,
	"step": 383
	},
	{
	"epoch": 3.734394124847001,
	"grad_norm": 0.8743162155151367,
	"learning_rate": 7.6762516704146e-07,
	"loss": 0.6405,
	"step": 384
	},
	{
	"epoch": 3.744186046511628,
	"grad_norm": 0.9365859031677246,
	"learning_rate": 7.566621115527811e-07,
	"loss": 0.688,
	"step": 385
	},
	{
	"epoch": 3.7539779681762546,
	"grad_norm": 0.9409651160240173,
	"learning_rate": 7.457639292922675e-07,
	"loss": 0.7227,
	"step": 386
	},
	{
	"epoch": 3.7637698898408813,
	"grad_norm": 0.835321843624115,
	"learning_rate": 7.349310258039441e-07,
	"loss": 0.3902,
	"step": 387
	},
	{
	"epoch": 3.773561811505508,
	"grad_norm": 0.991841733455658,
	"learning_rate": 7.241638042026783e-07,
	"loss": 0.5005,
	"step": 388
	},
	{
	"epoch": 3.783353733170135,
	"grad_norm": 0.8466964960098267,
	"learning_rate": 7.134626651591758e-07,
	"loss": 0.5818,
	"step": 389
	},
	{
	"epoch": 3.7931456548347615,
	"grad_norm": 0.825508713722229,
	"learning_rate": 7.028280068850734e-07,
	"loss": 0.4976,
	"step": 390
	},
	{
	"epoch": 3.802937576499388,
	"grad_norm": 0.9017467498779297,
	"learning_rate": 6.922602251181221e-07,
	"loss": 0.454,
	"step": 391
	},
	{
	"epoch": 3.812729498164015,
	"grad_norm": 0.9352346658706665,
	"learning_rate": 6.817597131074566e-07,
	"loss": 0.5703,
	"step": 392
	},
	{
	"epoch": 3.8225214198286412,
	"grad_norm": 0.9042191505432129,
	"learning_rate": 6.713268615989654e-07,
	"loss": 0.4789,
	"step": 393
	},
	{
	"epoch": 3.832313341493268,
	"grad_norm": 0.8832695484161377,
	"learning_rate": 6.609620588207474e-07,
	"loss": 0.4636,
	"step": 394
	},
	{
	"epoch": 3.8421052631578947,
	"grad_norm": 0.7026119232177734,
	"learning_rate": 6.506656904686698e-07,
	"loss": 0.4476,
	"step": 395
	},
	{
	"epoch": 3.8518971848225214,
	"grad_norm": 1.0028407573699951,
	"learning_rate": 6.4043813969201e-07,
	"loss": 0.7347,
	"step": 396
	},
	{
	"epoch": 3.861689106487148,
	"grad_norm": 0.9018222689628601,
	"learning_rate": 6.302797870792007e-07,
	"loss": 0.4932,
	"step": 397
	},
	{
	"epoch": 3.871481028151775,
	"grad_norm": 1.017398715019226,
	"learning_rate": 6.201910106436673e-07,
	"loss": 0.4939,
	"step": 398
	},
	{
	"epoch": 3.8812729498164016,
	"grad_norm": 0.9321378469467163,
	"learning_rate": 6.101721858097606e-07,
	"loss": 0.5258,
	"step": 399
	},
	{
	"epoch": 3.8910648714810283,
	"grad_norm": 0.6798187494277954,
	"learning_rate": 6.00223685398788e-07,
	"loss": 0.4351,
	"step": 400
	},
	{
	"epoch": 3.900856793145655,
	"grad_norm": 0.7866200804710388,
	"learning_rate": 5.903458796151382e-07,
	"loss": 0.536,
	"step": 401
	},
	{
	"epoch": 3.9106487148102813,
	"grad_norm": 0.9600208401679993,
	"learning_rate": 5.80539136032505e-07,
	"loss": 0.7666,
	"step": 402
	},
	{
	"epoch": 3.920440636474908,
	"grad_norm": 0.9808923602104187,
	"learning_rate": 5.708038195802098e-07,
	"loss": 0.6936,
	"step": 403
	},
	{
	"epoch": 3.9302325581395348,
	"grad_norm": 0.8592504858970642,
	"learning_rate": 5.61140292529625e-07,
	"loss": 0.3843,
	"step": 404
	},
	{
	"epoch": 3.9400244798041615,
	"grad_norm": 0.8602292537689209,
	"learning_rate": 5.515489144806862e-07,
	"loss": 0.4275,
	"step": 405
	},
	{
	"epoch": 3.949816401468788,
	"grad_norm": 0.8622450828552246,
	"learning_rate": 5.420300423485167e-07,
	"loss": 0.6017,
	"step": 406
	},
	{
	"epoch": 3.959608323133415,
	"grad_norm": 0.8687506914138794,
	"learning_rate": 5.325840303501431e-07,
	"loss": 0.5231,
	"step": 407
	},
	{
	"epoch": 3.9694002447980417,
	"grad_norm": 0.7728466987609863,
	"learning_rate": 5.232112299913151e-07,
	"loss": 0.4827,
	"step": 408
	},
	{
	"epoch": 3.9791921664626684,
	"grad_norm": 0.7771996855735779,
	"learning_rate": 5.139119900534259e-07,
	"loss": 0.5156,
	"step": 409
	},
	{
	"epoch": 3.988984088127295,
	"grad_norm": 0.8858612179756165,
	"learning_rate": 5.046866565805311e-07,
	"loss": 0.522,
	"step": 410
	},
	{
	"epoch": 3.998776009791922,
	"grad_norm": 0.9791521430015564,
	"learning_rate": 4.95535572866474e-07,
	"loss": 0.4726,
	"step": 411
	},
	{
	"epoch": 4.0,
	"grad_norm": 5.095057964324951,
	"learning_rate": 4.864590794421092e-07,
	"loss": 0.3991,
	"step": 412
	},
	{
	"epoch": 4.009791921664626,
	"grad_norm": 0.7647070288658142,
	"learning_rate": 4.774575140626317e-07,
	"loss": 0.5232,
	"step": 413
	},
	{
	"epoch": 4.0195838433292534,
	"grad_norm": 0.9652845859527588,
	"learning_rate": 4.6853121169500914e-07,
	"loss": 0.4453,
	"step": 414
	},
	{
	"epoch": 4.02937576499388,
	"grad_norm": 0.8679951429367065,
	"learning_rate": 4.5968050450551527e-07,
	"loss": 0.5705,
	"step": 415
	},
	{
	"epoch": 4.039167686658507,
	"grad_norm": 0.8676388263702393,
	"learning_rate": 4.5090572184736863e-07,
	"loss": 0.5859,
	"step": 416
	},
	{
	"epoch": 4.048959608323133,
	"grad_norm": 0.8352739214897156,
	"learning_rate": 4.422071902484812e-07,
	"loss": 0.5812,
	"step": 417
	},
	{
	"epoch": 4.05875152998776,
	"grad_norm": 0.7597355842590332,
	"learning_rate": 4.335852333993018e-07,
	"loss": 0.5188,
	"step": 418
	},
	{
	"epoch": 4.068543451652387,
	"grad_norm": 0.7206275463104248,
	"learning_rate": 4.2504017214077374e-07,
	"loss": 0.4629,
	"step": 419
	},
	{
	"epoch": 4.078335373317014,
	"grad_norm": 0.7692583799362183,
	"learning_rate": 4.165723244523978e-07,
	"loss": 0.6966,
	"step": 420
	},
	{
	"epoch": 4.08812729498164,
	"grad_norm": 0.8363829255104065,
	"learning_rate": 4.0818200544039484e-07,
	"loss": 0.4098,
	"step": 421
	},
	{
	"epoch": 4.097919216646267,
	"grad_norm": 0.8537617921829224,
	"learning_rate": 3.998695273259834e-07,
	"loss": 0.5483,
	"step": 422
	},
	{
	"epoch": 4.1077111383108935,
	"grad_norm": 0.7518119812011719,
	"learning_rate": 3.9163519943375973e-07,
	"loss": 0.4535,
	"step": 423
	},
	{
	"epoch": 4.11750305997552,
	"grad_norm": 0.8763237595558167,
	"learning_rate": 3.834793281801891e-07,
	"loss": 0.4935,
	"step": 424
	},
	{
	"epoch": 4.127294981640147,
	"grad_norm": 0.7164304256439209,
	"learning_rate": 3.754022170622007e-07,
	"loss": 0.4924,
	"step": 425
	},
	{
	"epoch": 4.137086903304773,
	"grad_norm": 0.7896924018859863,
	"learning_rate": 3.6740416664589634e-07,
	"loss": 0.5304,
	"step": 426
	},
	{
	"epoch": 4.1468788249694,
	"grad_norm": 0.8500013947486877,
	"learning_rate": 3.594854745553636e-07,
	"loss": 0.7045,
	"step": 427
	},
	{
	"epoch": 4.156670746634027,
	"grad_norm": 0.9297462105751038,
	"learning_rate": 3.516464354616031e-07,
	"loss": 0.6637,
	"step": 428
	},
	{
	"epoch": 4.166462668298654,
	"grad_norm": 1.220871090888977,
	"learning_rate": 3.43887341071561e-07,
	"loss": 0.5235,
	"step": 429
	},
	{
	"epoch": 4.17625458996328,
	"grad_norm": 0.9784387946128845,
	"learning_rate": 3.3620848011727437e-07,
	"loss": 0.5594,
	"step": 430
	},
	{
	"epoch": 4.186046511627907,
	"grad_norm": 0.721200704574585,
	"learning_rate": 3.2861013834512844e-07,
	"loss": 0.4662,
	"step": 431
	},
	{
	"epoch": 4.195838433292534,
	"grad_norm": 0.9451988935470581,
	"learning_rate": 3.2109259850522045e-07,
	"loss": 0.7258,
	"step": 432
	},
	{
	"epoch": 4.205630354957161,
	"grad_norm": 0.8058119416236877,
	"learning_rate": 3.1365614034084224e-07,
	"loss": 0.3451,
	"step": 433
	},
	{
	"epoch": 4.215422276621787,
	"grad_norm": 0.8161913752555847,
	"learning_rate": 3.0630104057806616e-07,
	"loss": 0.4309,
	"step": 434
	},
	{
	"epoch": 4.225214198286413,
	"grad_norm": 0.9032533764839172,
	"learning_rate": 2.9902757291544905e-07,
	"loss": 0.4457,
	"step": 435
	},
	{
	"epoch": 4.2350061199510405,
	"grad_norm": 1.0546553134918213,
	"learning_rate": 2.9183600801384853e-07,
	"loss": 0.5218,
	"step": 436
	},
	{
	"epoch": 4.244798041615667,
	"grad_norm": 0.7201629281044006,
	"learning_rate": 2.8472661348634883e-07,
	"loss": 0.3597,
	"step": 437
	},
	{
	"epoch": 4.254589963280294,
	"grad_norm": 0.7832567095756531,
	"learning_rate": 2.776996538883062e-07,
	"loss": 0.7423,
	"step": 438
	},
	{
	"epoch": 4.26438188494492,
	"grad_norm": 0.7669157981872559,
	"learning_rate": 2.707553907074989e-07,
	"loss": 0.3548,
	"step": 439
	},
	{
	"epoch": 4.274173806609547,
	"grad_norm": 0.796465277671814,
	"learning_rate": 2.638940823544012e-07,
	"loss": 0.3967,
	"step": 440
	},
	{
	"epoch": 4.283965728274174,
	"grad_norm": 0.8338704109191895,
	"learning_rate": 2.57115984152565e-07,
	"loss": 0.5426,
	"step": 441
	},
	{
	"epoch": 4.293757649938801,
	"grad_norm": 0.9683997631072998,
	"learning_rate": 2.504213483291193e-07,
	"loss": 0.4077,
	"step": 442
	},
	{
	"epoch": 4.303549571603427,
	"grad_norm": 0.8337029218673706,
	"learning_rate": 2.43810424005386e-07,
	"loss": 0.5351,
	"step": 443
	},
	{
	"epoch": 4.313341493268053,
	"grad_norm": 0.9775717258453369,
	"learning_rate": 2.3728345718760622e-07,
	"loss": 0.4849,
	"step": 444
	},
	{
	"epoch": 4.3231334149326806,
	"grad_norm": 0.8828545808792114,
	"learning_rate": 2.3084069075778758e-07,
	"loss": 0.4333,
	"step": 445
	},
	{
	"epoch": 4.332925336597307,
	"grad_norm": 0.844749391078949,
	"learning_rate": 2.2448236446466847e-07,
	"loss": 0.5903,
	"step": 446
	},
	{
	"epoch": 4.342717258261934,
	"grad_norm": 1.0013747215270996,
	"learning_rate": 2.1820871491479102e-07,
	"loss": 0.663,
	"step": 447
	},
	{
	"epoch": 4.35250917992656,
	"grad_norm": 0.7944239377975464,
	"learning_rate": 2.1201997556370284e-07,
	"loss": 0.5328,
	"step": 448
	},
	{
	"epoch": 4.3623011015911874,
	"grad_norm": 0.9357317090034485,
	"learning_rate": 2.059163767072639e-07,
	"loss": 0.3807,
	"step": 449
	},
	{
	"epoch": 4.372093023255814,
	"grad_norm": 0.8000408411026001,
	"learning_rate": 1.9989814547308056e-07,
	"loss": 0.4997,
	"step": 450
	},
	{
	"epoch": 4.381884944920441,
	"grad_norm": 0.898949921131134,
	"learning_rate": 1.9396550581205208e-07,
	"loss": 0.4768,
	"step": 451
	},
	{
	"epoch": 4.391676866585067,
	"grad_norm": 0.9039142727851868,
	"learning_rate": 1.8811867849003684e-07,
	"loss": 0.4327,
	"step": 452
	},
	{
	"epoch": 4.401468788249694,
	"grad_norm": 0.9020278453826904,
	"learning_rate": 1.8235788107963948e-07,
	"loss": 0.4165,
	"step": 453
	},
	{
	"epoch": 4.411260709914321,
	"grad_norm": 0.7534590363502502,
	"learning_rate": 1.7668332795211074e-07,
	"loss": 0.3673,
	"step": 454
	},
	{
	"epoch": 4.421052631578947,
	"grad_norm": 0.9269835948944092,
	"learning_rate": 1.7109523026937302e-07,
	"loss": 0.5881,
	"step": 455
	},
	{
	"epoch": 4.430844553243574,
	"grad_norm": 1.2964661121368408,
	"learning_rate": 1.6559379597616136e-07,
	"loss": 0.8225,
	"step": 456
	},
	{
	"epoch": 4.4406364749082,
	"grad_norm": 0.8568935394287109,
	"learning_rate": 1.6017922979228662e-07,
	"loss": 0.5303,
	"step": 457
	},
	{
	"epoch": 4.4504283965728275,
	"grad_norm": 0.9542502164840698,
	"learning_rate": 1.5485173320501673e-07,
	"loss": 0.5347,
	"step": 458
	},
	{
	"epoch": 4.460220318237454,
	"grad_norm": 0.8366877436637878,
	"learning_rate": 1.4961150446157759e-07,
	"loss": 0.4857,
	"step": 459
	},
	{
	"epoch": 4.470012239902081,
	"grad_norm": 0.8118438124656677,
	"learning_rate": 1.444587385617785e-07,
	"loss": 0.655,
	"step": 460
	},
	{
	"epoch": 4.479804161566707,
	"grad_norm": 0.9384212493896484,
	"learning_rate": 1.3939362725075344e-07,
	"loss": 0.6283,
	"step": 461
	},
	{
	"epoch": 4.489596083231334,
	"grad_norm": 0.8418795466423035,
	"learning_rate": 1.3441635901182803e-07,
	"loss": 0.3867,
	"step": 462
	},
	{
	"epoch": 4.499388004895961,
	"grad_norm": 0.9226901531219482,
	"learning_rate": 1.2952711905950377e-07,
	"loss": 0.6693,
	"step": 463
	},
	{
	"epoch": 4.509179926560588,
	"grad_norm": 0.8896486163139343,
	"learning_rate": 1.2472608933256637e-07,
	"loss": 0.5253,
	"step": 464
	},
	{
	"epoch": 4.518971848225214,
	"grad_norm": 0.9425986409187317,
	"learning_rate": 1.2001344848731612e-07,
	"loss": 0.3955,
	"step": 465
	},
	{
	"epoch": 4.52876376988984,
	"grad_norm": 0.8691738247871399,
	"learning_rate": 1.1538937189091825e-07,
	"loss": 0.6987,
	"step": 466
	},
	{
	"epoch": 4.538555691554468,
	"grad_norm": 0.9938181638717651,
	"learning_rate": 1.1085403161488012e-07,
	"loss": 0.678,
	"step": 467
	},
	{
	"epoch": 4.548347613219094,
	"grad_norm": 0.7933505773544312,
	"learning_rate": 1.0640759642864401e-07,
	"loss": 0.3979,
	"step": 468
	},
	{
	"epoch": 4.558139534883721,
	"grad_norm": 0.8107869625091553,
	"learning_rate": 1.0205023179330975e-07,
	"loss": 0.567,
	"step": 469
	},
	{
	"epoch": 4.567931456548347,
	"grad_norm": 0.818139910697937,
	"learning_rate": 9.778209985547682e-08,
	"loss": 0.45,
	"step": 470
	},
	{
	"epoch": 4.5777233782129745,
	"grad_norm": 0.9956123232841492,
	"learning_rate": 9.360335944121029e-08,
	"loss": 0.4543,
	"step": 471
	},
	{
	"epoch": 4.587515299877601,
	"grad_norm": 0.7484842538833618,
	"learning_rate": 8.951416605013114e-08,
	"loss": 0.5314,
	"step": 472
	},
	{
	"epoch": 4.597307221542228,
	"grad_norm": 0.768964946269989,
	"learning_rate": 8.55146718496283e-08,
	"loss": 0.5993,
	"step": 473
	},
	{
	"epoch": 4.607099143206854,
	"grad_norm": 0.8783057332038879,
	"learning_rate": 8.160502566919942e-08,
	"loss": 0.5261,
	"step": 474
	},
	{
	"epoch": 4.616891064871481,
	"grad_norm": 0.8336430191993713,
	"learning_rate": 7.778537299490796e-08,
	"loss": 0.4712,
	"step": 475
	},
	{
	"epoch": 4.626682986536108,
	"grad_norm": 0.853015124797821,
	"learning_rate": 7.405585596397314e-08,
	"loss": 0.5546,
	"step": 476
	},
	{
	"epoch": 4.636474908200734,
	"grad_norm": 0.7003260254859924,
	"learning_rate": 7.041661335948024e-08,
	"loss": 0.496,
	"step": 477
	},
	{
	"epoch": 4.646266829865361,
	"grad_norm": 0.839685320854187,
	"learning_rate": 6.68677806052137e-08,
	"loss": 0.5368,
	"step": 478
	},
	{
	"epoch": 4.656058751529987,
	"grad_norm": 0.8594149947166443,
	"learning_rate": 6.340948976062023e-08,
	"loss": 0.7643,
	"step": 479
	},
	{
	"epoch": 4.6658506731946146,
	"grad_norm": 1.0430092811584473,
	"learning_rate": 6.004186951589414e-08,
	"loss": 0.6026,
	"step": 480
	},
	{
	"epoch": 4.675642594859241,
	"grad_norm": 0.720399022102356,
	"learning_rate": 5.6765045187187614e-08,
	"loss": 0.5163,
	"step": 481
	},
	{
	"epoch": 4.685434516523868,
	"grad_norm": 0.9541524648666382,
	"learning_rate": 5.3579138711948587e-08,
	"loss": 0.5079,
	"step": 482
	},
	{
	"epoch": 4.695226438188494,
	"grad_norm": 0.8264009356498718,
	"learning_rate": 5.048426864438183e-08,
	"loss": 0.3677,
	"step": 483
	},
	{
	"epoch": 4.7050183598531214,
	"grad_norm": 0.8248957991600037,
	"learning_rate": 4.7480550151038365e-08,
	"loss": 0.4881,
	"step": 484
	},
	{
	"epoch": 4.714810281517748,
	"grad_norm": 0.8793155550956726,
	"learning_rate": 4.4568095006529975e-08,
	"loss": 0.539,
	"step": 485
	},
	{
	"epoch": 4.724602203182375,
	"grad_norm": 0.8505308628082275,
	"learning_rate": 4.174701158936895e-08,
	"loss": 0.4469,
	"step": 486
	},
	{
	"epoch": 4.734394124847001,
	"grad_norm": 0.7341795563697815,
	"learning_rate": 3.9017404877935986e-08,
	"loss": 0.4151,
	"step": 487
	},
	{
	"epoch": 4.7441860465116275,
	"grad_norm": 0.8461063504219055,
	"learning_rate": 3.637937644657308e-08,
	"loss": 0.6125,
	"step": 488
	},
	{
	"epoch": 4.753977968176255,
	"grad_norm": 0.9181951880455017,
	"learning_rate": 3.3833024461803756e-08,
	"loss": 0.5113,
	"step": 489
	},
	{
	"epoch": 4.763769889840881,
	"grad_norm": 0.8345598578453064,
	"learning_rate": 3.1378443678680706e-08,
	"loss": 0.4516,
	"step": 490
	},
	{
	"epoch": 4.773561811505508,
	"grad_norm": 0.9937892556190491,
	"learning_rate": 2.9015725437259724e-08,
	"loss": 0.5444,
	"step": 491
	},
	{
	"epoch": 4.783353733170134,
	"grad_norm": 0.987704336643219,
	"learning_rate": 2.6744957659199376e-08,
	"loss": 0.6802,
	"step": 492
	},
	{
	"epoch": 4.7931456548347615,
	"grad_norm": 0.7462158203125,
	"learning_rate": 2.4566224844491393e-08,
	"loss": 0.426,
	"step": 493
	},
	{
	"epoch": 4.802937576499388,
	"grad_norm": 0.764529824256897,
	"learning_rate": 2.247960806831373e-08,
	"loss": 0.5006,
	"step": 494
	},
	{
	"epoch": 4.812729498164015,
	"grad_norm": 0.9468876123428345,
	"learning_rate": 2.0485184978016604e-08,
	"loss": 0.6549,
	"step": 495
	},
	{
	"epoch": 4.822521419828641,
	"grad_norm": 1.0024148225784302,
	"learning_rate": 1.8583029790230356e-08,
	"loss": 0.6364,
	"step": 496
	},
	{
	"epoch": 4.832313341493268,
	"grad_norm": 0.6664872169494629,
	"learning_rate": 1.677321328810516e-08,
	"loss": 0.4718,
	"step": 497
	},
	{
	"epoch": 4.842105263157895,
	"grad_norm": 0.8330218195915222,
	"learning_rate": 1.5055802818676745e-08,
	"loss": 0.5102,
	"step": 498
	},
	{
	"epoch": 4.851897184822521,
	"grad_norm": 0.7899358868598938,
	"learning_rate": 1.3430862290359781e-08,
	"loss": 0.5603,
	"step": 499
	},
	{
	"epoch": 4.861689106487148,
	"grad_norm": 0.818114161491394,
	"learning_rate": 1.1898452170570618e-08,
	"loss": 0.4545,
	"step": 500
	},
	{
	"epoch": 4.871481028151774,
	"grad_norm": 0.9684895873069763,
	"learning_rate": 1.0458629483476868e-08,
	"loss": 0.7103,
	"step": 501
	},
	{
	"epoch": 4.881272949816402,
	"grad_norm": 0.8995057940483093,
	"learning_rate": 9.111447807874374e-09,
	"loss": 0.7959,
	"step": 502
	},
	{
	"epoch": 4.891064871481028,
	"grad_norm": 0.962817370891571,
	"learning_rate": 7.856957275194921e-09,
	"loss": 0.5464,
	"step": 503
	},
	{
	"epoch": 4.900856793145655,
	"grad_norm": 0.8113439679145813,
	"learning_rate": 6.6952045676405005e-09,
	"loss": 0.3896,
	"step": 504
	},
	{
	"epoch": 4.910648714810281,
	"grad_norm": 0.8609779477119446,
	"learning_rate": 5.62623291644443e-09,
	"loss": 0.4916,
	"step": 505
	},
	{
	"epoch": 4.9204406364749085,
	"grad_norm": 0.9280630350112915,
	"learning_rate": 4.6500821002654075e-09,
	"loss": 0.5505,
	"step": 506
	},
	{
	"epoch": 4.930232558139535,
	"grad_norm": 0.9593523740768433,
	"learning_rate": 3.766788443705094e-09,
	"loss": 0.4648,
	"step": 507
	},
	{
	"epoch": 4.940024479804162,
	"grad_norm": 0.8477063179016113,
	"learning_rate": 2.976384815957245e-09,
	"loss": 0.4508,
	"step": 508
	},
	{
	"epoch": 4.949816401468788,
	"grad_norm": 0.886053740978241,
	"learning_rate": 2.278900629584524e-09,
	"loss": 0.593,
	"step": 509
	},
	{
	"epoch": 4.9596083231334145,
	"grad_norm": 0.8786177039146423,
	"learning_rate": 1.6743618394238215e-09,
	"loss": 0.4909,
	"step": 510
	},
	{
	"epoch": 4.969400244798042,
	"grad_norm": 0.7327588796615601,
	"learning_rate": 1.1627909416211947e-09,
	"loss": 0.528,
	"step": 511
	},
	{
	"epoch": 4.979192166462668,
	"grad_norm": 0.7525851130485535,
	"learning_rate": 7.442069727930934e-10,
	"loss": 0.5381,
	"step": 512
	},
	{
	"epoch": 4.988984088127295,
	"grad_norm": 0.8296193480491638,
	"learning_rate": 4.186255093194258e-10,
	"loss": 0.4302,
	"step": 513
	},
	{
	"epoch": 4.998776009791921,
	"grad_norm": 0.8766030073165894,
	"learning_rate": 1.8605866676374428e-10,
	"loss": 0.5562,
	"step": 514
	},
	{
	"epoch": 5.0,
	"grad_norm": 3.7876617908477783,
	"learning_rate": 4.651509942193988e-11,
	"loss": 0.3239,
	"step": 515
	},
	{
	"epoch": 5.0,
	"step": 515,
	"total_flos": 41105816813568.0,
	"train_loss": 0.5990637808170134,
	"train_runtime": 15307.5317,
	"train_samples_per_second": 0.267,
	"train_steps_per_second": 0.034
	}
	],
	"logging_steps": 1,
	"max_steps": 515,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 5,
	"save_steps": 100,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 41105816813568.0,
	"train_batch_size": 1,
	"trial_name": null,
	"trial_params": null
	}