Instructions to use rish3on3AI/chem-Phi-Mini with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use rish3on3AI/chem-Phi-Mini with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/phi-4-mini-reasoning-unsloth-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "rish3on3AI/chem-Phi-Mini")

Transformers

How to use rish3on3AI/chem-Phi-Mini with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="rish3on3AI/chem-Phi-Mini")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("rish3on3AI/chem-Phi-Mini", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use rish3on3AI/chem-Phi-Mini with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "rish3on3AI/chem-Phi-Mini"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rish3on3AI/chem-Phi-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/rish3on3AI/chem-Phi-Mini

SGLang

How to use rish3on3AI/chem-Phi-Mini with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "rish3on3AI/chem-Phi-Mini" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rish3on3AI/chem-Phi-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "rish3on3AI/chem-Phi-Mini" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "rish3on3AI/chem-Phi-Mini",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use rish3on3AI/chem-Phi-Mini with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for rish3on3AI/chem-Phi-Mini to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for rish3on3AI/chem-Phi-Mini to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for rish3on3AI/chem-Phi-Mini to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="rish3on3AI/chem-Phi-Mini",
    max_seq_length=2048,
)

Docker Model Runner
How to use rish3on3AI/chem-Phi-Mini with Docker Model Runner:
```
docker model run hf.co/rish3on3AI/chem-Phi-Mini
```

chem-Phi-Mini / checkpoint-5000 /trainer_state.json

rish3on3AI

Upload folder using huggingface_hub

a47865b verified about 2 months ago

Raw

History Blame Contribute Delete

97.4 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 0.05838018343053634,
	"eval_steps": 500,
	"global_step": 5000,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.00011676036686107268,
	"grad_norm": 59.875431060791016,
	"learning_rate": 1.6000000000000003e-05,
	"loss": 5.634319305419922,
	"step": 10
	},
	{
	"epoch": 0.00023352073372214535,
	"grad_norm": 14.255208015441895,
	"learning_rate": 3.6e-05,
	"loss": 1.5329483985900878,
	"step": 20
	},
	{
	"epoch": 0.00035028110058321805,
	"grad_norm": 0.09891340881586075,
	"learning_rate": 5.6000000000000006e-05,
	"loss": 0.07387322783470154,
	"step": 30
	},
	{
	"epoch": 0.0004670414674442907,
	"grad_norm": 0.012327014468610287,
	"learning_rate": 7.6e-05,
	"loss": 5.6700489949434996e-05,
	"step": 40
	},
	{
	"epoch": 0.0005838018343053634,
	"grad_norm": 0.007001669146120548,
	"learning_rate": 9.6e-05,
	"loss": 1.7879287770483643e-05,
	"step": 50
	},
	{
	"epoch": 0.0007005622011664361,
	"grad_norm": 0.004531091079115868,
	"learning_rate": 0.000116,
	"loss": 1.258878328371793e-05,
	"step": 60
	},
	{
	"epoch": 0.0008173225680275087,
	"grad_norm": 0.003342111362144351,
	"learning_rate": 0.00013600000000000003,
	"loss": 8.95144257810898e-06,
	"step": 70
	},
	{
	"epoch": 0.0009340829348885814,
	"grad_norm": 0.003412243677303195,
	"learning_rate": 0.00015600000000000002,
	"loss": 9.027028863783925e-06,
	"step": 80
	},
	{
	"epoch": 0.001050843301749654,
	"grad_norm": 0.003643080359324813,
	"learning_rate": 0.00017600000000000002,
	"loss": 8.901499677449465e-06,
	"step": 90
	},
	{
	"epoch": 0.0011676036686107268,
	"grad_norm": 0.005081272684037685,
	"learning_rate": 0.000196,
	"loss": 1.0010588448494673e-05,
	"step": 100
	},
	{
	"epoch": 0.0012843640354717994,
	"grad_norm": 0.0034726918675005436,
	"learning_rate": 0.00019972881355932205,
	"loss": 1.3041248894296587e-05,
	"step": 110
	},
	{
	"epoch": 0.0014011244023328722,
	"grad_norm": 0.004254758358001709,
	"learning_rate": 0.00019938983050847458,
	"loss": 1.2646359391510486e-05,
	"step": 120
	},
	{
	"epoch": 0.0015178847691939448,
	"grad_norm": 0.0073653459548950195,
	"learning_rate": 0.00019905084745762712,
	"loss": 1.571593020344153e-05,
	"step": 130
	},
	{
	"epoch": 0.0016346451360550174,
	"grad_norm": 0.010147612541913986,
	"learning_rate": 0.00019871186440677968,
	"loss": 2.0644562027882785e-05,
	"step": 140
	},
	{
	"epoch": 0.0017514055029160902,
	"grad_norm": 0.019865401089191437,
	"learning_rate": 0.0001983728813559322,
	"loss": 4.4391825213097036e-05,
	"step": 150
	},
	{
	"epoch": 0.0018681658697771628,
	"grad_norm": 0.005128371529281139,
	"learning_rate": 0.00019803389830508477,
	"loss": 2.0386229152791202e-05,
	"step": 160
	},
	{
	"epoch": 0.0019849262366382356,
	"grad_norm": 0.006253004539757967,
	"learning_rate": 0.0001976949152542373,
	"loss": 2.618969592731446e-05,
	"step": 170
	},
	{
	"epoch": 0.002101686603499308,
	"grad_norm": 0.02808883599936962,
	"learning_rate": 0.00019735593220338984,
	"loss": 3.143970388919115e-05,
	"step": 180
	},
	{
	"epoch": 0.002218446970360381,
	"grad_norm": 0.008802235126495361,
	"learning_rate": 0.0001970169491525424,
	"loss": 5.302483914420009e-05,
	"step": 190
	},
	{
	"epoch": 0.0023352073372214536,
	"grad_norm": 0.0010030085686594248,
	"learning_rate": 0.00019667796610169493,
	"loss": 5.503819556906819e-06,
	"step": 200
	},
	{
	"epoch": 0.0024519677040825264,
	"grad_norm": 0.0016146366251632571,
	"learning_rate": 0.00019633898305084746,
	"loss": 4.235541564412415e-06,
	"step": 210
	},
	{
	"epoch": 0.002568728070943599,
	"grad_norm": 0.0015124814817681909,
	"learning_rate": 0.000196,
	"loss": 3.0034610972506925e-06,
	"step": 220
	},
	{
	"epoch": 0.0026854884378046716,
	"grad_norm": 0.0011274752905592322,
	"learning_rate": 0.00019566101694915255,
	"loss": 2.2700047338730656e-06,
	"step": 230
	},
	{
	"epoch": 0.0028022488046657444,
	"grad_norm": 0.0017322940984740853,
	"learning_rate": 0.0001953220338983051,
	"loss": 2.7529582439456136e-06,
	"step": 240
	},
	{
	"epoch": 0.002919009171526817,
	"grad_norm": 0.0038653130177408457,
	"learning_rate": 0.00019498305084745762,
	"loss": 3.1220744858728723e-06,
	"step": 250
	},
	{
	"epoch": 0.0030357695383878896,
	"grad_norm": 0.0014339711051434278,
	"learning_rate": 0.00019464406779661018,
	"loss": 2.4247039618785495e-06,
	"step": 260
	},
	{
	"epoch": 0.0031525299052489624,
	"grad_norm": 0.002295655431225896,
	"learning_rate": 0.0001943050847457627,
	"loss": 3.04567874991335e-06,
	"step": 270
	},
	{
	"epoch": 0.003269290272110035,
	"grad_norm": 0.008011543191969395,
	"learning_rate": 0.00019396610169491527,
	"loss": 3.7598518247250466e-06,
	"step": 280
	},
	{
	"epoch": 0.0033860506389711076,
	"grad_norm": 0.003250898327678442,
	"learning_rate": 0.0001936271186440678,
	"loss": 4.587964940583333e-06,
	"step": 290
	},
	{
	"epoch": 0.0035028110058321804,
	"grad_norm": 0.008879482746124268,
	"learning_rate": 0.00019328813559322034,
	"loss": 1.9185926066711546e-05,
	"step": 300
	},
	{
	"epoch": 0.003619571372693253,
	"grad_norm": 78.78023529052734,
	"learning_rate": 0.0001929491525423729,
	"loss": 0.13886371850967408,
	"step": 310
	},
	{
	"epoch": 0.0037363317395543256,
	"grad_norm": 0.07044398039579391,
	"learning_rate": 0.00019261016949152543,
	"loss": 0.10051707029342652,
	"step": 320
	},
	{
	"epoch": 0.0038530921064153984,
	"grad_norm": 0.15833403170108795,
	"learning_rate": 0.000192271186440678,
	"loss": 0.017947301268577576,
	"step": 330
	},
	{
	"epoch": 0.003969852473276471,
	"grad_norm": 0.0070645627565681934,
	"learning_rate": 0.00019193220338983052,
	"loss": 7.719181012362241e-05,
	"step": 340
	},
	{
	"epoch": 0.004086612840137544,
	"grad_norm": 0.0017116001108661294,
	"learning_rate": 0.00019159322033898306,
	"loss": 1.865742960944772e-05,
	"step": 350
	},
	{
	"epoch": 0.004203373206998616,
	"grad_norm": 0.001340084825642407,
	"learning_rate": 0.00019125423728813561,
	"loss": 7.437381282215938e-06,
	"step": 360
	},
	{
	"epoch": 0.004320133573859689,
	"grad_norm": 0.0008336898172274232,
	"learning_rate": 0.00019091525423728815,
	"loss": 3.6651814298238606e-06,
	"step": 370
	},
	{
	"epoch": 0.004436893940720762,
	"grad_norm": 0.00022103996889200062,
	"learning_rate": 0.00019057627118644068,
	"loss": 2.676965232240036e-06,
	"step": 380
	},
	{
	"epoch": 0.004553654307581834,
	"grad_norm": 0.00012676029291469604,
	"learning_rate": 0.00019023728813559324,
	"loss": 2.325788045709487e-06,
	"step": 390
	},
	{
	"epoch": 0.004670414674442907,
	"grad_norm": 8.621117740403861e-05,
	"learning_rate": 0.00018989830508474577,
	"loss": 1.8469903807272202e-06,
	"step": 400
	},
	{
	"epoch": 0.00478717504130398,
	"grad_norm": 6.617301551159471e-05,
	"learning_rate": 0.00018955932203389833,
	"loss": 1.6815065464470536e-06,
	"step": 410
	},
	{
	"epoch": 0.004903935408165053,
	"grad_norm": 0.00010414897406008095,
	"learning_rate": 0.00018922033898305084,
	"loss": 1.6302668882417492e-06,
	"step": 420
	},
	{
	"epoch": 0.005020695775026125,
	"grad_norm": 4.913287557428703e-05,
	"learning_rate": 0.0001888813559322034,
	"loss": 1.6286450772895477e-06,
	"step": 430
	},
	{
	"epoch": 0.005137456141887198,
	"grad_norm": 0.00019105675164610147,
	"learning_rate": 0.00018854237288135593,
	"loss": 1.6238565876847133e-06,
	"step": 440
	},
	{
	"epoch": 0.005254216508748271,
	"grad_norm": 6.603466317756101e-05,
	"learning_rate": 0.0001882033898305085,
	"loss": 1.6175361452042126e-06,
	"step": 450
	},
	{
	"epoch": 0.005370976875609343,
	"grad_norm": 2.6533391064731404e-05,
	"learning_rate": 0.00018786440677966102,
	"loss": 1.6475098163937218e-06,
	"step": 460
	},
	{
	"epoch": 0.005487737242470416,
	"grad_norm": 6.834692612756044e-05,
	"learning_rate": 0.00018752542372881356,
	"loss": 1.5935082046780736e-06,
	"step": 470
	},
	{
	"epoch": 0.005604497609331489,
	"grad_norm": 6.407107866834849e-05,
	"learning_rate": 0.00018718644067796612,
	"loss": 1.6839179806993342e-06,
	"step": 480
	},
	{
	"epoch": 0.005721257976192561,
	"grad_norm": 3.699313310789876e-05,
	"learning_rate": 0.00018684745762711865,
	"loss": 1.711418735794723e-06,
	"step": 490
	},
	{
	"epoch": 0.005838018343053634,
	"grad_norm": 0.00010490009299246594,
	"learning_rate": 0.00018650847457627118,
	"loss": 1.6201638572965749e-06,
	"step": 500
	},
	{
	"epoch": 0.005954778709914707,
	"grad_norm": 3.698803629959002e-05,
	"learning_rate": 0.00018616949152542374,
	"loss": 1.61662683240138e-06,
	"step": 510
	},
	{
	"epoch": 0.006071539076775779,
	"grad_norm": 7.622056000400335e-05,
	"learning_rate": 0.00018583050847457627,
	"loss": 1.6264759324258193e-06,
	"step": 520
	},
	{
	"epoch": 0.006188299443636852,
	"grad_norm": 6.441948789870366e-05,
	"learning_rate": 0.00018549152542372883,
	"loss": 1.682136280578561e-06,
	"step": 530
	},
	{
	"epoch": 0.006305059810497925,
	"grad_norm": 0.00021724995167460293,
	"learning_rate": 0.00018515254237288137,
	"loss": 1.6840473108459265e-06,
	"step": 540
	},
	{
	"epoch": 0.006421820177358997,
	"grad_norm": 7.526912668254226e-05,
	"learning_rate": 0.0001848135593220339,
	"loss": 1.7155130990431645e-06,
	"step": 550
	},
	{
	"epoch": 0.00653858054422007,
	"grad_norm": 0.0002523708972148597,
	"learning_rate": 0.00018447457627118646,
	"loss": 1.7206137272296474e-06,
	"step": 560
	},
	{
	"epoch": 0.006655340911081143,
	"grad_norm": 0.00011283948697382584,
	"learning_rate": 0.000184135593220339,
	"loss": 1.6726349713280798e-06,
	"step": 570
	},
	{
	"epoch": 0.006772101277942215,
	"grad_norm": 1.4866308447381016e-05,
	"learning_rate": 0.00018379661016949153,
	"loss": 1.6677884559612722e-06,
	"step": 580
	},
	{
	"epoch": 0.006888861644803288,
	"grad_norm": 6.323176785372198e-05,
	"learning_rate": 0.00018345762711864406,
	"loss": 1.6571942978771403e-06,
	"step": 590
	},
	{
	"epoch": 0.007005622011664361,
	"grad_norm": 5.835434785694815e-05,
	"learning_rate": 0.00018311864406779662,
	"loss": 1.644503754505422e-06,
	"step": 600
	},
	{
	"epoch": 0.007122382378525433,
	"grad_norm": 0.00016602610412519425,
	"learning_rate": 0.00018277966101694918,
	"loss": 1.6589550796197726e-06,
	"step": 610
	},
	{
	"epoch": 0.007239142745386506,
	"grad_norm": 5.203962791711092e-05,
	"learning_rate": 0.0001824406779661017,
	"loss": 1.6293291992042213e-06,
	"step": 620
	},
	{
	"epoch": 0.007355903112247579,
	"grad_norm": 7.588268636027351e-05,
	"learning_rate": 0.00018210169491525424,
	"loss": 1.613848507986404e-06,
	"step": 630
	},
	{
	"epoch": 0.007472663479108651,
	"grad_norm": 0.0002376928023295477,
	"learning_rate": 0.00018176271186440678,
	"loss": 1.587349470355548e-06,
	"step": 640
	},
	{
	"epoch": 0.007589423845969724,
	"grad_norm": 6.04057167947758e-05,
	"learning_rate": 0.00018142372881355934,
	"loss": 1.5923547834972852e-06,
	"step": 650
	},
	{
	"epoch": 0.007706184212830797,
	"grad_norm": 6.623761146329343e-05,
	"learning_rate": 0.0001810847457627119,
	"loss": 1.6252573914243839e-06,
	"step": 660
	},
	{
	"epoch": 0.00782294457969187,
	"grad_norm": 5.2295028581283987e-05,
	"learning_rate": 0.0001807457627118644,
	"loss": 1.5565537978545763e-06,
	"step": 670
	},
	{
	"epoch": 0.007939704946552942,
	"grad_norm": 1.0837538866326213e-05,
	"learning_rate": 0.00018040677966101696,
	"loss": 1.6090023564174772e-06,
	"step": 680
	},
	{
	"epoch": 0.008056465313414014,
	"grad_norm": 9.154729013971519e-06,
	"learning_rate": 0.0001800677966101695,
	"loss": 1.6676300219842234e-06,
	"step": 690
	},
	{
	"epoch": 0.008173225680275087,
	"grad_norm": 7.765877671772614e-05,
	"learning_rate": 0.00017972881355932205,
	"loss": 1.5882900697761214e-06,
	"step": 700
	},
	{
	"epoch": 0.00828998604713616,
	"grad_norm": 1.9532657461240888e-05,
	"learning_rate": 0.0001793898305084746,
	"loss": 1.5790859833941794e-06,
	"step": 710
	},
	{
	"epoch": 0.008406746413997232,
	"grad_norm": 0.00010468939581187442,
	"learning_rate": 0.00017905084745762712,
	"loss": 1.5944267943268642e-06,
	"step": 720
	},
	{
	"epoch": 0.008523506780858305,
	"grad_norm": 0.0002512343635316938,
	"learning_rate": 0.00017871186440677968,
	"loss": 1.5406461898237466e-06,
	"step": 730
	},
	{
	"epoch": 0.008640267147719378,
	"grad_norm": 4.805830758414231e-05,
	"learning_rate": 0.0001783728813559322,
	"loss": 1.681893081695307e-06,
	"step": 740
	},
	{
	"epoch": 0.00875702751458045,
	"grad_norm": 0.00020604948804248124,
	"learning_rate": 0.00017803389830508474,
	"loss": 1.6168803995242343e-06,
	"step": 750
	},
	{
	"epoch": 0.008873787881441523,
	"grad_norm": 9.445300383958966e-05,
	"learning_rate": 0.0001776949152542373,
	"loss": 1.6993661120068282e-06,
	"step": 760
	},
	{
	"epoch": 0.008990548248302596,
	"grad_norm": 6.204142118804157e-05,
	"learning_rate": 0.00017735593220338984,
	"loss": 1.5745516066090204e-06,
	"step": 770
	},
	{
	"epoch": 0.009107308615163668,
	"grad_norm": 0.00016777920245658606,
	"learning_rate": 0.0001770169491525424,
	"loss": 1.6554306057514622e-06,
	"step": 780
	},
	{
	"epoch": 0.009224068982024741,
	"grad_norm": 0.00014272049884311855,
	"learning_rate": 0.0001766779661016949,
	"loss": 1.552332287246827e-06,
	"step": 790
	},
	{
	"epoch": 0.009340829348885814,
	"grad_norm": 0.00011927623563678935,
	"learning_rate": 0.00017633898305084746,
	"loss": 1.5156711015151814e-06,
	"step": 800
	},
	{
	"epoch": 0.009457589715746888,
	"grad_norm": 6.487468635896221e-05,
	"learning_rate": 0.00017600000000000002,
	"loss": 1.5687646737205796e-06,
	"step": 810
	},
	{
	"epoch": 0.00957435008260796,
	"grad_norm": 6.103752184571931e-06,
	"learning_rate": 0.00017566101694915256,
	"loss": 1.6446090739918873e-06,
	"step": 820
	},
	{
	"epoch": 0.009691110449469032,
	"grad_norm": 1.1832606105599552e-05,
	"learning_rate": 0.0001753220338983051,
	"loss": 1.545727536722552e-06,
	"step": 830
	},
	{
	"epoch": 0.009807870816330106,
	"grad_norm": 3.0457977118203416e-05,
	"learning_rate": 0.00017498305084745762,
	"loss": 1.6163508917088621e-06,
	"step": 840
	},
	{
	"epoch": 0.009924631183191177,
	"grad_norm": 0.00013204464630689472,
	"learning_rate": 0.00017464406779661018,
	"loss": 1.619108661543578e-06,
	"step": 850
	},
	{
	"epoch": 0.01004139155005225,
	"grad_norm": 0.0001281007134821266,
	"learning_rate": 0.0001743050847457627,
	"loss": 1.6555333786527626e-06,
	"step": 860
	},
	{
	"epoch": 0.010158151916913324,
	"grad_norm": 1.5468805941054597e-05,
	"learning_rate": 0.00017396610169491527,
	"loss": 1.5782845366629771e-06,
	"step": 870
	},
	{
	"epoch": 0.010274912283774395,
	"grad_norm": 6.472590757766739e-05,
	"learning_rate": 0.0001736271186440678,
	"loss": 1.6427136870333925e-06,
	"step": 880
	},
	{
	"epoch": 0.010391672650635468,
	"grad_norm": 0.00016957020852714777,
	"learning_rate": 0.00017328813559322034,
	"loss": 1.6775775293353945e-06,
	"step": 890
	},
	{
	"epoch": 0.010508433017496542,
	"grad_norm": 0.00026253293617628515,
	"learning_rate": 0.0001729491525423729,
	"loss": 1.6508824046468361e-06,
	"step": 900
	},
	{
	"epoch": 0.010625193384357613,
	"grad_norm": 6.529199163196608e-05,
	"learning_rate": 0.00017261016949152543,
	"loss": 1.5743760741315782e-06,
	"step": 910
	},
	{
	"epoch": 0.010741953751218686,
	"grad_norm": 0.0,
	"learning_rate": 0.00017227118644067796,
	"loss": 1.5706827980466187e-06,
	"step": 920
	},
	{
	"epoch": 0.01085871411807976,
	"grad_norm": 0.0,
	"learning_rate": 0.00017193220338983052,
	"loss": 1.4936667867004871e-06,
	"step": 930
	},
	{
	"epoch": 0.010975474484940831,
	"grad_norm": 9.25576314330101e-05,
	"learning_rate": 0.00017159322033898306,
	"loss": 1.5477988199563697e-06,
	"step": 940
	},
	{
	"epoch": 0.011092234851801904,
	"grad_norm": 0.00013144953118171543,
	"learning_rate": 0.00017125423728813562,
	"loss": 1.5690189684391954e-06,
	"step": 950
	},
	{
	"epoch": 0.011208995218662978,
	"grad_norm": 0.00015978094597812742,
	"learning_rate": 0.00017091525423728815,
	"loss": 1.492923365731258e-06,
	"step": 960
	},
	{
	"epoch": 0.01132575558552405,
	"grad_norm": 7.87453682278283e-05,
	"learning_rate": 0.00017057627118644068,
	"loss": 1.5079906916071196e-06,
	"step": 970
	},
	{
	"epoch": 0.011442515952385122,
	"grad_norm": 0.0,
	"learning_rate": 0.00017023728813559324,
	"loss": 1.5082824575074484e-06,
	"step": 980
	},
	{
	"epoch": 0.011559276319246196,
	"grad_norm": 9.658994531491771e-05,
	"learning_rate": 0.00016989830508474577,
	"loss": 1.5206737771222834e-06,
	"step": 990
	},
	{
	"epoch": 0.011676036686107267,
	"grad_norm": 0.00039425029535777867,
	"learning_rate": 0.0001695593220338983,
	"loss": 1.6318292182404548e-06,
	"step": 1000
	},
	{
	"epoch": 0.01179279705296834,
	"grad_norm": 6.258842040551826e-05,
	"learning_rate": 0.00016922033898305084,
	"loss": 1.6430853065685368e-06,
	"step": 1010
	},
	{
	"epoch": 0.011909557419829414,
	"grad_norm": 9.773509373189881e-05,
	"learning_rate": 0.0001688813559322034,
	"loss": 1.635014996281825e-06,
	"step": 1020
	},
	{
	"epoch": 0.012026317786690485,
	"grad_norm": 9.781518747331575e-05,
	"learning_rate": 0.00016854237288135596,
	"loss": 1.6510937712155283e-06,
	"step": 1030
	},
	{
	"epoch": 0.012143078153551558,
	"grad_norm": 6.78331489325501e-05,
	"learning_rate": 0.00016820338983050847,
	"loss": 1.4550299056281802e-06,
	"step": 1040
	},
	{
	"epoch": 0.012259838520412632,
	"grad_norm": 9.950463572749868e-05,
	"learning_rate": 0.00016786440677966103,
	"loss": 1.4592896150134037e-06,
	"step": 1050
	},
	{
	"epoch": 0.012376598887273703,
	"grad_norm": 0.00010552265302976593,
	"learning_rate": 0.00016752542372881356,
	"loss": 1.4062128684599883e-06,
	"step": 1060
	},
	{
	"epoch": 0.012493359254134776,
	"grad_norm": 1.2324165254540276e-05,
	"learning_rate": 0.00016718644067796612,
	"loss": 1.3861051229469012e-06,
	"step": 1070
	},
	{
	"epoch": 0.01261011962099585,
	"grad_norm": 0.00023415149189531803,
	"learning_rate": 0.00016684745762711865,
	"loss": 1.487008103140397e-06,
	"step": 1080
	},
	{
	"epoch": 0.012726879987856921,
	"grad_norm": 3.0040808269404806e-05,
	"learning_rate": 0.00016650847457627118,
	"loss": 1.3674427464138716e-06,
	"step": 1090
	},
	{
	"epoch": 0.012843640354717994,
	"grad_norm": 6.606012902921066e-05,
	"learning_rate": 0.00016616949152542374,
	"loss": 1.3812194083584473e-06,
	"step": 1100
	},
	{
	"epoch": 0.012960400721579068,
	"grad_norm": 5.7821172958938405e-05,
	"learning_rate": 0.00016583050847457628,
	"loss": 1.3413139640761074e-06,
	"step": 1110
	},
	{
	"epoch": 0.01307716108844014,
	"grad_norm": 7.803161861374974e-05,
	"learning_rate": 0.0001654915254237288,
	"loss": 1.3541534826799762e-06,
	"step": 1120
	},
	{
	"epoch": 0.013193921455301212,
	"grad_norm": 6.889037467772141e-05,
	"learning_rate": 0.00016515254237288137,
	"loss": 1.3373861293075607e-06,
	"step": 1130
	},
	{
	"epoch": 0.013310681822162286,
	"grad_norm": 9.310171299148351e-05,
	"learning_rate": 0.0001648135593220339,
	"loss": 1.3587457942776382e-06,
	"step": 1140
	},
	{
	"epoch": 0.013427442189023357,
	"grad_norm": 0.0,
	"learning_rate": 0.00016447457627118646,
	"loss": 1.3045552805124316e-06,
	"step": 1150
	},
	{
	"epoch": 0.01354420255588443,
	"grad_norm": 1.8477974663255736e-05,
	"learning_rate": 0.000164135593220339,
	"loss": 1.3706468962482176e-06,
	"step": 1160
	},
	{
	"epoch": 0.013660962922745504,
	"grad_norm": 2.0946725271642208e-05,
	"learning_rate": 0.00016379661016949153,
	"loss": 1.341822735412279e-06,
	"step": 1170
	},
	{
	"epoch": 0.013777723289606575,
	"grad_norm": 0.00018352564075030386,
	"learning_rate": 0.0001634576271186441,
	"loss": 1.4251672837417573e-06,
	"step": 1180
	},
	{
	"epoch": 0.013894483656467648,
	"grad_norm": 0.00010090215801028535,
	"learning_rate": 0.00016311864406779662,
	"loss": 1.360269288852578e-06,
	"step": 1190
	},
	{
	"epoch": 0.014011244023328722,
	"grad_norm": 6.751793989678845e-05,
	"learning_rate": 0.00016277966101694918,
	"loss": 1.3718395166506525e-06,
	"step": 1200
	},
	{
	"epoch": 0.014128004390189793,
	"grad_norm": 6.73890026519075e-05,
	"learning_rate": 0.00016244067796610169,
	"loss": 1.3808265975967514e-06,
	"step": 1210
	},
	{
	"epoch": 0.014244764757050866,
	"grad_norm": 7.155471394071355e-05,
	"learning_rate": 0.00016210169491525424,
	"loss": 1.3531963304558304e-06,
	"step": 1220
	},
	{
	"epoch": 0.01436152512391194,
	"grad_norm": 8.340043132193387e-05,
	"learning_rate": 0.00016176271186440678,
	"loss": 1.4715641555085312e-06,
	"step": 1230
	},
	{
	"epoch": 0.014478285490773011,
	"grad_norm": 5.85219140702975e-06,
	"learning_rate": 0.00016142372881355934,
	"loss": 1.4763120816496667e-06,
	"step": 1240
	},
	{
	"epoch": 0.014595045857634084,
	"grad_norm": 9.948112710844725e-05,
	"learning_rate": 0.00016108474576271187,
	"loss": 1.4198940334608778e-06,
	"step": 1250
	},
	{
	"epoch": 0.014711806224495158,
	"grad_norm": 0.00010508869308978319,
	"learning_rate": 0.0001607457627118644,
	"loss": 1.456945301470114e-06,
	"step": 1260
	},
	{
	"epoch": 0.01482856659135623,
	"grad_norm": 0.00010197787923971191,
	"learning_rate": 0.00016040677966101696,
	"loss": 1.4444390217249747e-06,
	"step": 1270
	},
	{
	"epoch": 0.014945326958217302,
	"grad_norm": 8.006284770090133e-05,
	"learning_rate": 0.0001600677966101695,
	"loss": 1.4245255442801862e-06,
	"step": 1280
	},
	{
	"epoch": 0.015062087325078376,
	"grad_norm": 2.938463512691669e-05,
	"learning_rate": 0.00015972881355932203,
	"loss": 1.355502627120586e-06,
	"step": 1290
	},
	{
	"epoch": 0.015178847691939447,
	"grad_norm": 6.846284668426961e-05,
	"learning_rate": 0.0001593898305084746,
	"loss": 1.4384363566932734e-06,
	"step": 1300
	},
	{
	"epoch": 0.01529560805880052,
	"grad_norm": 7.272188668139279e-05,
	"learning_rate": 0.00015905084745762712,
	"loss": 1.3579057849710807e-06,
	"step": 1310
	},
	{
	"epoch": 0.015412368425661594,
	"grad_norm": 1.7946014850167558e-05,
	"learning_rate": 0.00015871186440677968,
	"loss": 1.3938626580056734e-06,
	"step": 1320
	},
	{
	"epoch": 0.015529128792522667,
	"grad_norm": 6.652858428424224e-05,
	"learning_rate": 0.00015837288135593221,
	"loss": 1.350030015601078e-06,
	"step": 1330
	},
	{
	"epoch": 0.01564588915938374,
	"grad_norm": 1.5056033589644358e-05,
	"learning_rate": 0.00015803389830508475,
	"loss": 1.4097900020715315e-06,
	"step": 1340
	},
	{
	"epoch": 0.01576264952624481,
	"grad_norm": 0.00012858258560299873,
	"learning_rate": 0.0001576949152542373,
	"loss": 1.3804547961626668e-06,
	"step": 1350
	},
	{
	"epoch": 0.015879409893105885,
	"grad_norm": 4.996855295757996e-06,
	"learning_rate": 0.00015735593220338984,
	"loss": 1.3450860024022404e-06,
	"step": 1360
	},
	{
	"epoch": 0.015996170259966958,
	"grad_norm": 8.589975186623633e-05,
	"learning_rate": 0.00015701694915254237,
	"loss": 1.3399458111962304e-06,
	"step": 1370
	},
	{
	"epoch": 0.016112930626828028,
	"grad_norm": 0.0001891898864414543,
	"learning_rate": 0.0001566779661016949,
	"loss": 1.3313266208569986e-06,
	"step": 1380
	},
	{
	"epoch": 0.0162296909936891,
	"grad_norm": 0.0,
	"learning_rate": 0.00015633898305084746,
	"loss": 1.4106865819485393e-06,
	"step": 1390
	},
	{
	"epoch": 0.016346451360550174,
	"grad_norm": 6.900495645822957e-05,
	"learning_rate": 0.00015600000000000002,
	"loss": 1.3441473129205407e-06,
	"step": 1400
	},
	{
	"epoch": 0.016463211727411248,
	"grad_norm": 9.959421004168689e-05,
	"learning_rate": 0.00015566101694915256,
	"loss": 1.3834467608830892e-06,
	"step": 1410
	},
	{
	"epoch": 0.01657997209427232,
	"grad_norm": 6.697992648696527e-05,
	"learning_rate": 0.0001553220338983051,
	"loss": 1.357650944555644e-06,
	"step": 1420
	},
	{
	"epoch": 0.016696732461133394,
	"grad_norm": 0.0001277877454413101,
	"learning_rate": 0.00015498305084745762,
	"loss": 1.396657171426341e-06,
	"step": 1430
	},
	{
	"epoch": 0.016813492827994464,
	"grad_norm": 0.00010542531526880339,
	"learning_rate": 0.00015464406779661018,
	"loss": 1.4042208931641653e-06,
	"step": 1440
	},
	{
	"epoch": 0.016930253194855537,
	"grad_norm": 0.00013882535859011114,
	"learning_rate": 0.00015430508474576274,
	"loss": 1.4621442460338586e-06,
	"step": 1450
	},
	{
	"epoch": 0.01704701356171661,
	"grad_norm": 6.771282642148435e-05,
	"learning_rate": 0.00015396610169491525,
	"loss": 1.4194434697856196e-06,
	"step": 1460
	},
	{
	"epoch": 0.017163773928577684,
	"grad_norm": 0.00010310515062883496,
	"learning_rate": 0.0001536271186440678,
	"loss": 1.503728526586201e-06,
	"step": 1470
	},
	{
	"epoch": 0.017280534295438757,
	"grad_norm": 2.718402538448572e-05,
	"learning_rate": 0.00015328813559322034,
	"loss": 1.4018172805663197e-06,
	"step": 1480
	},
	{
	"epoch": 0.01739729466229983,
	"grad_norm": 3.286205173935741e-05,
	"learning_rate": 0.0001529491525423729,
	"loss": 1.4310914593806956e-06,
	"step": 1490
	},
	{
	"epoch": 0.0175140550291609,
	"grad_norm": 8.035649807425216e-05,
	"learning_rate": 0.00015261016949152543,
	"loss": 1.4795346032769885e-06,
	"step": 1500
	},
	{
	"epoch": 0.017630815396021973,
	"grad_norm": 7.678180554648861e-05,
	"learning_rate": 0.00015227118644067797,
	"loss": 1.4396316146303434e-06,
	"step": 1510
	},
	{
	"epoch": 0.017747575762883046,
	"grad_norm": 2.948004475911148e-05,
	"learning_rate": 0.00015193220338983053,
	"loss": 1.4612438462791034e-06,
	"step": 1520
	},
	{
	"epoch": 0.01786433612974412,
	"grad_norm": 0.00013761305308435112,
	"learning_rate": 0.00015159322033898306,
	"loss": 1.3847372429154347e-06,
	"step": 1530
	},
	{
	"epoch": 0.017981096496605193,
	"grad_norm": 0.00016148357826750726,
	"learning_rate": 0.0001512542372881356,
	"loss": 1.4071320947550704e-06,
	"step": 1540
	},
	{
	"epoch": 0.018097856863466266,
	"grad_norm": 0.00012403611617628485,
	"learning_rate": 0.00015091525423728815,
	"loss": 1.4144216947897804e-06,
	"step": 1550
	},
	{
	"epoch": 0.018214617230327336,
	"grad_norm": 0.0,
	"learning_rate": 0.00015057627118644068,
	"loss": 1.3889767615182792e-06,
	"step": 1560
	},
	{
	"epoch": 0.01833137759718841,
	"grad_norm": 5.628773578791879e-05,
	"learning_rate": 0.00015023728813559324,
	"loss": 1.4211836059985216e-06,
	"step": 1570
	},
	{
	"epoch": 0.018448137964049482,
	"grad_norm": 0.00012825976591557264,
	"learning_rate": 0.00014989830508474575,
	"loss": 1.4366208233695943e-06,
	"step": 1580
	},
	{
	"epoch": 0.018564898330910556,
	"grad_norm": 1.7427068087272346e-05,
	"learning_rate": 0.0001495593220338983,
	"loss": 1.4261461728892755e-06,
	"step": 1590
	},
	{
	"epoch": 0.01868165869777163,
	"grad_norm": 1.4448479305428918e-05,
	"learning_rate": 0.00014922033898305087,
	"loss": 1.3885482985642739e-06,
	"step": 1600
	},
	{
	"epoch": 0.018798419064632702,
	"grad_norm": 0.0001871474669314921,
	"learning_rate": 0.0001488813559322034,
	"loss": 1.46058046084363e-06,
	"step": 1610
	},
	{
	"epoch": 0.018915179431493775,
	"grad_norm": 6.522086187032983e-05,
	"learning_rate": 0.00014854237288135593,
	"loss": 1.5170177903200965e-06,
	"step": 1620
	},
	{
	"epoch": 0.019031939798354845,
	"grad_norm": 0.00026542512932792306,
	"learning_rate": 0.00014820338983050847,
	"loss": 1.6691523342160507e-06,
	"step": 1630
	},
	{
	"epoch": 0.01914870016521592,
	"grad_norm": 0.0002182667376473546,
	"learning_rate": 0.00014786440677966103,
	"loss": 1.684337439655792e-06,
	"step": 1640
	},
	{
	"epoch": 0.01926546053207699,
	"grad_norm": 0.000918116420507431,
	"learning_rate": 0.00014752542372881356,
	"loss": 1.8373031707596965e-06,
	"step": 1650
	},
	{
	"epoch": 0.019382220898938065,
	"grad_norm": 0.00011826390255009755,
	"learning_rate": 0.00014718644067796612,
	"loss": 1.6845940990606323e-06,
	"step": 1660
	},
	{
	"epoch": 0.019498981265799138,
	"grad_norm": 0.0002123541198670864,
	"learning_rate": 0.00014684745762711865,
	"loss": 1.6323769159498624e-06,
	"step": 1670
	},
	{
	"epoch": 0.01961574163266021,
	"grad_norm": 8.740282646613196e-05,
	"learning_rate": 0.00014650847457627119,
	"loss": 1.7147473045042717e-06,
	"step": 1680
	},
	{
	"epoch": 0.01973250199952128,
	"grad_norm": 8.509032340953127e-05,
	"learning_rate": 0.00014616949152542375,
	"loss": 1.6850821339176036e-06,
	"step": 1690
	},
	{
	"epoch": 0.019849262366382354,
	"grad_norm": 0.0002514809893909842,
	"learning_rate": 0.00014583050847457628,
	"loss": 1.794596028048545e-06,
	"step": 1700
	},
	{
	"epoch": 0.019966022733243428,
	"grad_norm": 0.00019716132374014705,
	"learning_rate": 0.0001454915254237288,
	"loss": 1.8356367945671082e-06,
	"step": 1710
	},
	{
	"epoch": 0.0200827831001045,
	"grad_norm": 0.00015515851555392146,
	"learning_rate": 0.00014515254237288137,
	"loss": 2.0529379980871456e-06,
	"step": 1720
	},
	{
	"epoch": 0.020199543466965574,
	"grad_norm": 8.209195948438719e-05,
	"learning_rate": 0.0001448135593220339,
	"loss": 1.93539908650564e-06,
	"step": 1730
	},
	{
	"epoch": 0.020316303833826647,
	"grad_norm": 0.00023831179714761674,
	"learning_rate": 0.00014447457627118646,
	"loss": 1.9934388546971606e-06,
	"step": 1740
	},
	{
	"epoch": 0.020433064200687717,
	"grad_norm": 7.010973058640957e-05,
	"learning_rate": 0.000144135593220339,
	"loss": 2.090843736368697e-06,
	"step": 1750
	},
	{
	"epoch": 0.02054982456754879,
	"grad_norm": 9.501949534751475e-05,
	"learning_rate": 0.00014379661016949153,
	"loss": 1.8209009795100429e-06,
	"step": 1760
	},
	{
	"epoch": 0.020666584934409864,
	"grad_norm": 6.99472293490544e-05,
	"learning_rate": 0.0001434576271186441,
	"loss": 1.8174021533923224e-06,
	"step": 1770
	},
	{
	"epoch": 0.020783345301270937,
	"grad_norm": 7.495882164221257e-05,
	"learning_rate": 0.00014311864406779662,
	"loss": 1.781230457709171e-06,
	"step": 1780
	},
	{
	"epoch": 0.02090010566813201,
	"grad_norm": 0.00020178320119157434,
	"learning_rate": 0.00014277966101694915,
	"loss": 1.7899070371640847e-06,
	"step": 1790
	},
	{
	"epoch": 0.021016866034993083,
	"grad_norm": 0.00014918761735316366,
	"learning_rate": 0.0001424406779661017,
	"loss": 1.7568227121955715e-06,
	"step": 1800
	},
	{
	"epoch": 0.021133626401854153,
	"grad_norm": 1.9064373191213235e-05,
	"learning_rate": 0.00014210169491525425,
	"loss": 1.7985246813623235e-06,
	"step": 1810
	},
	{
	"epoch": 0.021250386768715226,
	"grad_norm": 0.00012646097457036376,
	"learning_rate": 0.0001417627118644068,
	"loss": 1.8512891983846202e-06,
	"step": 1820
	},
	{
	"epoch": 0.0213671471355763,
	"grad_norm": 0.00017406328697688878,
	"learning_rate": 0.0001414237288135593,
	"loss": 1.8382943380856887e-06,
	"step": 1830
	},
	{
	"epoch": 0.021483907502437373,
	"grad_norm": 1.7785694581107236e-05,
	"learning_rate": 0.00014108474576271187,
	"loss": 1.9178693037247286e-06,
	"step": 1840
	},
	{
	"epoch": 0.021600667869298446,
	"grad_norm": 0.0002715683658607304,
	"learning_rate": 0.0001407457627118644,
	"loss": 1.962191890925169e-06,
	"step": 1850
	},
	{
	"epoch": 0.02171742823615952,
	"grad_norm": 0.00039442337583750486,
	"learning_rate": 0.00014040677966101696,
	"loss": 2.1291600205586294e-06,
	"step": 1860
	},
	{
	"epoch": 0.02183418860302059,
	"grad_norm": 0.00044150848407298326,
	"learning_rate": 0.0001400677966101695,
	"loss": 2.294996193086263e-06,
	"step": 1870
	},
	{
	"epoch": 0.021950948969881662,
	"grad_norm": 0.0009703988907858729,
	"learning_rate": 0.00013972881355932203,
	"loss": 3.0259834602475166e-06,
	"step": 1880
	},
	{
	"epoch": 0.022067709336742736,
	"grad_norm": 0.0004824126372113824,
	"learning_rate": 0.0001393898305084746,
	"loss": 3.0716255423612893e-06,
	"step": 1890
	},
	{
	"epoch": 0.02218446970360381,
	"grad_norm": 0.0009920555166900158,
	"learning_rate": 0.00013905084745762712,
	"loss": 3.824635132332332e-06,
	"step": 1900
	},
	{
	"epoch": 0.022301230070464882,
	"grad_norm": 0.00033314264146611094,
	"learning_rate": 0.00013871186440677966,
	"loss": 2.861424582079053e-06,
	"step": 1910
	},
	{
	"epoch": 0.022417990437325955,
	"grad_norm": 0.0003477221471257508,
	"learning_rate": 0.00013837288135593222,
	"loss": 2.431200300634373e-06,
	"step": 1920
	},
	{
	"epoch": 0.022534750804187025,
	"grad_norm": 0.00010379795276094228,
	"learning_rate": 0.00013803389830508475,
	"loss": 2.044691973424051e-06,
	"step": 1930
	},
	{
	"epoch": 0.0226515111710481,
	"grad_norm": 2.0585646780091338e-05,
	"learning_rate": 0.0001376949152542373,
	"loss": 1.7322965504718013e-06,
	"step": 1940
	},
	{
	"epoch": 0.02276827153790917,
	"grad_norm": 0.00014282431220635772,
	"learning_rate": 0.00013735593220338984,
	"loss": 1.7086696971091441e-06,
	"step": 1950
	},
	{
	"epoch": 0.022885031904770245,
	"grad_norm": 0.00010931365250144154,
	"learning_rate": 0.00013701694915254237,
	"loss": 1.687408257566858e-06,
	"step": 1960
	},
	{
	"epoch": 0.023001792271631318,
	"grad_norm": 6.643973028985783e-05,
	"learning_rate": 0.00013667796610169493,
	"loss": 1.6163905456778593e-06,
	"step": 1970
	},
	{
	"epoch": 0.02311855263849239,
	"grad_norm": 9.641289943829179e-05,
	"learning_rate": 0.00013633898305084747,
	"loss": 1.5819381587789393e-06,
	"step": 1980
	},
	{
	"epoch": 0.02323531300535346,
	"grad_norm": 6.208644481375813e-05,
	"learning_rate": 0.00013600000000000003,
	"loss": 1.5344317944254727e-06,
	"step": 1990
	},
	{
	"epoch": 0.023352073372214534,
	"grad_norm": 5.661721388605656e-06,
	"learning_rate": 0.00013566101694915253,
	"loss": 1.54183799168095e-06,
	"step": 2000
	},
	{
	"epoch": 0.023468833739075608,
	"grad_norm": 6.737840158166364e-05,
	"learning_rate": 0.00013535593220338985,
	"loss": 1.5509236618527212e-06,
	"step": 2010
	},
	{
	"epoch": 0.02358559410593668,
	"grad_norm": 8.187992352759466e-05,
	"learning_rate": 0.00013501694915254238,
	"loss": 1.5634514056728221e-06,
	"step": 2020
	},
	{
	"epoch": 0.023702354472797754,
	"grad_norm": 2.138152194675058e-05,
	"learning_rate": 0.0001346779661016949,
	"loss": 1.5208880540740211e-06,
	"step": 2030
	},
	{
	"epoch": 0.023819114839658827,
	"grad_norm": 5.823801075166557e-06,
	"learning_rate": 0.00013433898305084747,
	"loss": 1.5027135304990225e-06,
	"step": 2040
	},
	{
	"epoch": 0.023935875206519897,
	"grad_norm": 0.00018770445603877306,
	"learning_rate": 0.000134,
	"loss": 1.523056926089339e-06,
	"step": 2050
	},
	{
	"epoch": 0.02405263557338097,
	"grad_norm": 0.0001309715589741245,
	"learning_rate": 0.00013366101694915254,
	"loss": 1.5320663806051016e-06,
	"step": 2060
	},
	{
	"epoch": 0.024169395940242044,
	"grad_norm": 8.546516619389877e-05,
	"learning_rate": 0.0001333220338983051,
	"loss": 1.6208254237426444e-06,
	"step": 2070
	},
	{
	"epoch": 0.024286156307103117,
	"grad_norm": 0.00023168772167991847,
	"learning_rate": 0.00013298305084745763,
	"loss": 1.6410729585913942e-06,
	"step": 2080
	},
	{
	"epoch": 0.02440291667396419,
	"grad_norm": 0.0003386707976460457,
	"learning_rate": 0.0001326440677966102,
	"loss": 1.819970020733308e-06,
	"step": 2090
	},
	{
	"epoch": 0.024519677040825263,
	"grad_norm": 1.7884540284285322e-05,
	"learning_rate": 0.0001323050847457627,
	"loss": 1.6884436263353563e-06,
	"step": 2100
	},
	{
	"epoch": 0.024636437407686333,
	"grad_norm": 9.627863619243726e-05,
	"learning_rate": 0.00013196610169491526,
	"loss": 1.740954212436918e-06,
	"step": 2110
	},
	{
	"epoch": 0.024753197774547406,
	"grad_norm": 0.00016727342153899372,
	"learning_rate": 0.00013162711864406782,
	"loss": 1.822072226786986e-06,
	"step": 2120
	},
	{
	"epoch": 0.02486995814140848,
	"grad_norm": 0.0001092248858185485,
	"learning_rate": 0.00013128813559322035,
	"loss": 1.8791550246533007e-06,
	"step": 2130
	},
	{
	"epoch": 0.024986718508269553,
	"grad_norm": 0.00042687152745202184,
	"learning_rate": 0.00013094915254237288,
	"loss": 2.346935980312992e-06,
	"step": 2140
	},
	{
	"epoch": 0.025103478875130626,
	"grad_norm": 0.005975034087896347,
	"learning_rate": 0.0001306101694915254,
	"loss": 3.751758049475029e-06,
	"step": 2150
	},
	{
	"epoch": 0.0252202392419917,
	"grad_norm": 0.0002233933482784778,
	"learning_rate": 0.00013027118644067797,
	"loss": 2.2646194338449276e-06,
	"step": 2160
	},
	{
	"epoch": 0.025336999608852773,
	"grad_norm": 3.1968527764547616e-05,
	"learning_rate": 0.00012993220338983053,
	"loss": 1.8630344129633158e-06,
	"step": 2170
	},
	{
	"epoch": 0.025453759975713842,
	"grad_norm": 0.00013106416736263782,
	"learning_rate": 0.00012959322033898307,
	"loss": 1.6936999600147829e-06,
	"step": 2180
	},
	{
	"epoch": 0.025570520342574916,
	"grad_norm": 0.00012995705765206367,
	"learning_rate": 0.0001292542372881356,
	"loss": 1.5745910786790773e-06,
	"step": 2190
	},
	{
	"epoch": 0.02568728070943599,
	"grad_norm": 2.0045872588525526e-05,
	"learning_rate": 0.00012891525423728813,
	"loss": 1.5972793335095048e-06,
	"step": 2200
	},
	{
	"epoch": 0.025804041076297062,
	"grad_norm": 3.028578430530615e-05,
	"learning_rate": 0.0001285762711864407,
	"loss": 1.6016567315091379e-06,
	"step": 2210
	},
	{
	"epoch": 0.025920801443158135,
	"grad_norm": 0.0001439986372133717,
	"learning_rate": 0.00012823728813559322,
	"loss": 1.5990770407370292e-06,
	"step": 2220
	},
	{
	"epoch": 0.02603756181001921,
	"grad_norm": 2.008549199672416e-05,
	"learning_rate": 0.00012789830508474576,
	"loss": 1.6530904758838005e-06,
	"step": 2230
	},
	{
	"epoch": 0.02615432217688028,
	"grad_norm": 0.00010255297820549458,
	"learning_rate": 0.00012755932203389832,
	"loss": 1.629211510589812e-06,
	"step": 2240
	},
	{
	"epoch": 0.02627108254374135,
	"grad_norm": 0.00012852097279392183,
	"learning_rate": 0.00012722033898305085,
	"loss": 1.636833076190669e-06,
	"step": 2250
	},
	{
	"epoch": 0.026387842910602425,
	"grad_norm": 2.7599591703619808e-05,
	"learning_rate": 0.0001268813559322034,
	"loss": 1.6537753253942355e-06,
	"step": 2260
	},
	{
	"epoch": 0.026504603277463498,
	"grad_norm": 0.0001625096338102594,
	"learning_rate": 0.00012654237288135594,
	"loss": 1.6621592294541187e-06,
	"step": 2270
	},
	{
	"epoch": 0.02662136364432457,
	"grad_norm": 8.66148984641768e-05,
	"learning_rate": 0.00012620338983050847,
	"loss": 1.6177982615772635e-06,
	"step": 2280
	},
	{
	"epoch": 0.026738124011185645,
	"grad_norm": 9.50808243942447e-05,
	"learning_rate": 0.00012586440677966103,
	"loss": 1.6551437511225231e-06,
	"step": 2290
	},
	{
	"epoch": 0.026854884378046714,
	"grad_norm": 0.00011600083962548524,
	"learning_rate": 0.00012552542372881357,
	"loss": 1.6295243767672218e-06,
	"step": 2300
	},
	{
	"epoch": 0.026971644744907788,
	"grad_norm": 0.00030933969537727535,
	"learning_rate": 0.0001251864406779661,
	"loss": 1.7880485756904818e-06,
	"step": 2310
	},
	{
	"epoch": 0.02708840511176886,
	"grad_norm": 0.0001261148281628266,
	"learning_rate": 0.00012484745762711866,
	"loss": 1.8836488379747607e-06,
	"step": 2320
	},
	{
	"epoch": 0.027205165478629934,
	"grad_norm": 0.0007181658875197172,
	"learning_rate": 0.0001245084745762712,
	"loss": 2.163295903301332e-06,
	"step": 2330
	},
	{
	"epoch": 0.027321925845491007,
	"grad_norm": 0.00026710331439971924,
	"learning_rate": 0.00012416949152542375,
	"loss": 2.493400825187564e-06,
	"step": 2340
	},
	{
	"epoch": 0.02743868621235208,
	"grad_norm": 6.985353684285656e-05,
	"learning_rate": 0.00012383050847457626,
	"loss": 2.357956691412255e-06,
	"step": 2350
	},
	{
	"epoch": 0.02755544657921315,
	"grad_norm": 0.0004108547291252762,
	"learning_rate": 0.00012349152542372882,
	"loss": 2.3179190975497477e-06,
	"step": 2360
	},
	{
	"epoch": 0.027672206946074224,
	"grad_norm": 0.0004732685047201812,
	"learning_rate": 0.00012315254237288135,
	"loss": 2.3334543584496716e-06,
	"step": 2370
	},
	{
	"epoch": 0.027788967312935297,
	"grad_norm": 0.00011048079613829032,
	"learning_rate": 0.0001228135593220339,
	"loss": 2.450296233291738e-06,
	"step": 2380
	},
	{
	"epoch": 0.02790572767979637,
	"grad_norm": 0.00017211842350661755,
	"learning_rate": 0.00012247457627118644,
	"loss": 2.3725997380097397e-06,
	"step": 2390
	},
	{
	"epoch": 0.028022488046657443,
	"grad_norm": 0.00044060120126232505,
	"learning_rate": 0.00012213559322033898,
	"loss": 2.3969145331648178e-06,
	"step": 2400
	},
	{
	"epoch": 0.028139248413518517,
	"grad_norm": 5.874623457202688e-05,
	"learning_rate": 0.00012179661016949154,
	"loss": 1.7556896636961028e-06,
	"step": 2410
	},
	{
	"epoch": 0.028256008780379586,
	"grad_norm": 0.00011153460218338296,
	"learning_rate": 0.00012145762711864408,
	"loss": 1.7029842638294213e-06,
	"step": 2420
	},
	{
	"epoch": 0.02837276914724066,
	"grad_norm": 0.00011762795475078747,
	"learning_rate": 0.0001211186440677966,
	"loss": 1.7962009223992937e-06,
	"step": 2430
	},
	{
	"epoch": 0.028489529514101733,
	"grad_norm": 0.0004375619755592197,
	"learning_rate": 0.00012077966101694915,
	"loss": 2.118217526003718e-06,
	"step": 2440
	},
	{
	"epoch": 0.028606289880962806,
	"grad_norm": 0.0005378103232942522,
	"learning_rate": 0.0001204406779661017,
	"loss": 3.5935878258896992e-06,
	"step": 2450
	},
	{
	"epoch": 0.02872305024782388,
	"grad_norm": 0.0005620762822218239,
	"learning_rate": 0.00012010169491525425,
	"loss": 4.4984088162891565e-06,
	"step": 2460
	},
	{
	"epoch": 0.028839810614684953,
	"grad_norm": 0.0021721464581787586,
	"learning_rate": 0.0001197627118644068,
	"loss": 5.6281347497133535e-06,
	"step": 2470
	},
	{
	"epoch": 0.028956570981546022,
	"grad_norm": 0.0015677689807489514,
	"learning_rate": 0.00011942372881355932,
	"loss": 5.305397280608304e-06,
	"step": 2480
	},
	{
	"epoch": 0.029073331348407096,
	"grad_norm": 0.001409961492754519,
	"learning_rate": 0.00011908474576271187,
	"loss": 7.273031951626763e-06,
	"step": 2490
	},
	{
	"epoch": 0.02919009171526817,
	"grad_norm": 0.0038958522491157055,
	"learning_rate": 0.00011874576271186441,
	"loss": 1.9833706028293817e-05,
	"step": 2500
	},
	{
	"epoch": 0.029306852082129242,
	"grad_norm": 0.0010552983731031418,
	"learning_rate": 0.00011840677966101696,
	"loss": 8.511309715686366e-06,
	"step": 2510
	},
	{
	"epoch": 0.029423612448990315,
	"grad_norm": 0.0018230744171887636,
	"learning_rate": 0.00011806779661016949,
	"loss": 5.045130819780752e-06,
	"step": 2520
	},
	{
	"epoch": 0.02954037281585139,
	"grad_norm": 0.0023112548515200615,
	"learning_rate": 0.00011772881355932204,
	"loss": 2.0276603754609825e-05,
	"step": 2530
	},
	{
	"epoch": 0.02965713318271246,
	"grad_norm": 0.619185745716095,
	"learning_rate": 0.00011738983050847458,
	"loss": 7.167774019762873e-05,
	"step": 2540
	},
	{
	"epoch": 0.02977389354957353,
	"grad_norm": 2.633053779602051,
	"learning_rate": 0.00011708474576271187,
	"loss": 0.3269044399261475,
	"step": 2550
	},
	{
	"epoch": 0.029890653916434605,
	"grad_norm": 78.88597869873047,
	"learning_rate": 0.00011677966101694915,
	"loss": 0.1233400583267212,
	"step": 2560
	},
	{
	"epoch": 0.030007414283295678,
	"grad_norm": 70.32923889160156,
	"learning_rate": 0.00011644067796610169,
	"loss": 0.18804351091384888,
	"step": 2570
	},
	{
	"epoch": 0.03012417465015675,
	"grad_norm": 0.061514776200056076,
	"learning_rate": 0.00011610169491525424,
	"loss": 0.5769382953643799,
	"step": 2580
	},
	{
	"epoch": 0.030240935017017825,
	"grad_norm": 0.0027894896920770407,
	"learning_rate": 0.0001157627118644068,
	"loss": 0.0004052900243550539,
	"step": 2590
	},
	{
	"epoch": 0.030357695383878894,
	"grad_norm": 0.002143974881619215,
	"learning_rate": 0.00011542372881355932,
	"loss": 0.0006741383578628302,
	"step": 2600
	},
	{
	"epoch": 0.030474455750739968,
	"grad_norm": 0.00237264740280807,
	"learning_rate": 0.00011508474576271186,
	"loss": 2.506431774236262e-05,
	"step": 2610
	},
	{
	"epoch": 0.03059121611760104,
	"grad_norm": 0.00042826461140066385,
	"learning_rate": 0.00011474576271186441,
	"loss": 8.783912198850885e-06,
	"step": 2620
	},
	{
	"epoch": 0.030707976484462114,
	"grad_norm": 0.0003991922130808234,
	"learning_rate": 0.00011440677966101696,
	"loss": 5.77748978685122e-06,
	"step": 2630
	},
	{
	"epoch": 0.030824736851323187,
	"grad_norm": 0.00013551986194215715,
	"learning_rate": 0.00011406779661016949,
	"loss": 4.757464193971828e-06,
	"step": 2640
	},
	{
	"epoch": 0.03094149721818426,
	"grad_norm": 0.0001976866042241454,
	"learning_rate": 0.00011372881355932204,
	"loss": 3.982192720286548e-06,
	"step": 2650
	},
	{
	"epoch": 0.031058257585045334,
	"grad_norm": 0.00016013045387808233,
	"learning_rate": 0.00011338983050847458,
	"loss": 3.4830805816454814e-06,
	"step": 2660
	},
	{
	"epoch": 0.031175017951906404,
	"grad_norm": 0.00014570701750926673,
	"learning_rate": 0.00011305084745762713,
	"loss": 3.5921588278142735e-06,
	"step": 2670
	},
	{
	"epoch": 0.03129177831876748,
	"grad_norm": 0.00011156608525197953,
	"learning_rate": 0.00011271186440677967,
	"loss": 3.2941421522991733e-06,
	"step": 2680
	},
	{
	"epoch": 0.031408538685628554,
	"grad_norm": 0.00021060730796307325,
	"learning_rate": 0.00011237288135593221,
	"loss": 3.206400288036093e-06,
	"step": 2690
	},
	{
	"epoch": 0.03152529905248962,
	"grad_norm": 0.00014615162217523903,
	"learning_rate": 0.00011203389830508475,
	"loss": 3.0735405744053424e-06,
	"step": 2700
	},
	{
	"epoch": 0.03164205941935069,
	"grad_norm": 8.24721428216435e-06,
	"learning_rate": 0.0001116949152542373,
	"loss": 2.964516716019716e-06,
	"step": 2710
	},
	{
	"epoch": 0.03175881978621177,
	"grad_norm": 9.397340181749314e-05,
	"learning_rate": 0.00011135593220338985,
	"loss": 2.933578798547387e-06,
	"step": 2720
	},
	{
	"epoch": 0.03187558015307284,
	"grad_norm": 8.81102605490014e-05,
	"learning_rate": 0.00011101694915254236,
	"loss": 2.7795455025625416e-06,
	"step": 2730
	},
	{
	"epoch": 0.031992340519933916,
	"grad_norm": 0.00010666410526027903,
	"learning_rate": 0.00011067796610169492,
	"loss": 2.8226597351022066e-06,
	"step": 2740
	},
	{
	"epoch": 0.032109100886794986,
	"grad_norm": 7.704042945988476e-05,
	"learning_rate": 0.00011033898305084747,
	"loss": 2.9511858883779498e-06,
	"step": 2750
	},
	{
	"epoch": 0.032225861253656056,
	"grad_norm": 2.1477530026459135e-05,
	"learning_rate": 0.00011000000000000002,
	"loss": 2.6188161427853627e-06,
	"step": 2760
	},
	{
	"epoch": 0.03234262162051713,
	"grad_norm": 1.7250757082365453e-05,
	"learning_rate": 0.00010966101694915254,
	"loss": 2.622844112920575e-06,
	"step": 2770
	},
	{
	"epoch": 0.0324593819873782,
	"grad_norm": 0.00013947318075224757,
	"learning_rate": 0.00010932203389830508,
	"loss": 2.5740868295542895e-06,
	"step": 2780
	},
	{
	"epoch": 0.03257614235423928,
	"grad_norm": 6.799829134251922e-05,
	"learning_rate": 0.00010898305084745763,
	"loss": 2.5692570488899946e-06,
	"step": 2790
	},
	{
	"epoch": 0.03269290272110035,
	"grad_norm": 8.138755219988525e-06,
	"learning_rate": 0.00010864406779661019,
	"loss": 2.6186939066974445e-06,
	"step": 2800
	},
	{
	"epoch": 0.032809663087961426,
	"grad_norm": 4.546531727100955e-06,
	"learning_rate": 0.00010830508474576271,
	"loss": 2.671362381079234e-06,
	"step": 2810
	},
	{
	"epoch": 0.032926423454822495,
	"grad_norm": 2.267390300403349e-05,
	"learning_rate": 0.00010796610169491525,
	"loss": 2.7489344574860296e-06,
	"step": 2820
	},
	{
	"epoch": 0.033043183821683565,
	"grad_norm": 0.00013649604807142168,
	"learning_rate": 0.0001076271186440678,
	"loss": 2.5407276552869008e-06,
	"step": 2830
	},
	{
	"epoch": 0.03315994418854464,
	"grad_norm": 7.409998215734959e-05,
	"learning_rate": 0.00010728813559322035,
	"loss": 2.3465197955374604e-06,
	"step": 2840
	},
	{
	"epoch": 0.03327670455540571,
	"grad_norm": 1.3193653103371616e-05,
	"learning_rate": 0.00010694915254237288,
	"loss": 2.431076609354932e-06,
	"step": 2850
	},
	{
	"epoch": 0.03339346492226679,
	"grad_norm": 7.322274177568033e-05,
	"learning_rate": 0.00010661016949152543,
	"loss": 2.3664873879170044e-06,
	"step": 2860
	},
	{
	"epoch": 0.03351022528912786,
	"grad_norm": 1.4847065358480904e-05,
	"learning_rate": 0.00010627118644067797,
	"loss": 2.424820922897197e-06,
	"step": 2870
	},
	{
	"epoch": 0.03362698565598893,
	"grad_norm": 1.4212756468623411e-05,
	"learning_rate": 0.00010593220338983052,
	"loss": 2.2911957785254343e-06,
	"step": 2880
	},
	{
	"epoch": 0.033743746022850005,
	"grad_norm": 6.81228857501992e-06,
	"learning_rate": 0.00010559322033898305,
	"loss": 2.3263673938345164e-06,
	"step": 2890
	},
	{
	"epoch": 0.033860506389711074,
	"grad_norm": 2.190048235206632e-06,
	"learning_rate": 0.0001052542372881356,
	"loss": 2.2670785256195815e-06,
	"step": 2900
	},
	{
	"epoch": 0.03397726675657215,
	"grad_norm": 6.678269073745469e-06,
	"learning_rate": 0.00010491525423728814,
	"loss": 2.264988870592788e-06,
	"step": 2910
	},
	{
	"epoch": 0.03409402712343322,
	"grad_norm": 7.456969615304843e-05,
	"learning_rate": 0.00010457627118644069,
	"loss": 2.175485496991314e-06,
	"step": 2920
	},
	{
	"epoch": 0.0342107874902943,
	"grad_norm": 1.9885351321136113e-06,
	"learning_rate": 0.00010423728813559324,
	"loss": 2.287617462570779e-06,
	"step": 2930
	},
	{
	"epoch": 0.03432754785715537,
	"grad_norm": 2.060169890683028e-06,
	"learning_rate": 0.00010389830508474576,
	"loss": 2.315424535481725e-06,
	"step": 2940
	},
	{
	"epoch": 0.03444430822401644,
	"grad_norm": 7.341393211390823e-05,
	"learning_rate": 0.00010355932203389832,
	"loss": 2.377605596848298e-06,
	"step": 2950
	},
	{
	"epoch": 0.034561068590877514,
	"grad_norm": 2.2036572318029357e-06,
	"learning_rate": 0.00010322033898305086,
	"loss": 2.2663543859380297e-06,
	"step": 2960
	},
	{
	"epoch": 0.034677828957738584,
	"grad_norm": 9.463338938076049e-05,
	"learning_rate": 0.00010288135593220341,
	"loss": 2.26012161874678e-06,
	"step": 2970
	},
	{
	"epoch": 0.03479458932459966,
	"grad_norm": 1.5640855053788982e-05,
	"learning_rate": 0.00010254237288135593,
	"loss": 2.0918838345096447e-06,
	"step": 2980
	},
	{
	"epoch": 0.03491134969146073,
	"grad_norm": 9.752279765962157e-06,
	"learning_rate": 0.00010220338983050847,
	"loss": 2.331857831450179e-06,
	"step": 2990
	},
	{
	"epoch": 0.0350281100583218,
	"grad_norm": 0.00010294884123140946,
	"learning_rate": 0.00010186440677966102,
	"loss": 2.1787878722534516e-06,
	"step": 3000
	},
	{
	"epoch": 0.03514487042518288,
	"grad_norm": 0.00010099232167704031,
	"learning_rate": 0.00010152542372881357,
	"loss": 2.174664223275613e-06,
	"step": 3010
	},
	{
	"epoch": 0.035261630792043946,
	"grad_norm": 7.355651177931577e-05,
	"learning_rate": 0.0001011864406779661,
	"loss": 2.1843752620043233e-06,
	"step": 3020
	},
	{
	"epoch": 0.03537839115890502,
	"grad_norm": 1.63390432135202e-05,
	"learning_rate": 0.00010084745762711865,
	"loss": 2.182403295591939e-06,
	"step": 3030
	},
	{
	"epoch": 0.03549515152576609,
	"grad_norm": 7.151766476454213e-05,
	"learning_rate": 0.00010050847457627119,
	"loss": 2.1563122572842985e-06,
	"step": 3040
	},
	{
	"epoch": 0.03561191189262717,
	"grad_norm": 2.100063056786894e-06,
	"learning_rate": 0.00010016949152542374,
	"loss": 2.141538425348699e-06,
	"step": 3050
	},
	{
	"epoch": 0.03572867225948824,
	"grad_norm": 0.00016249011969193816,
	"learning_rate": 9.983050847457628e-05,
	"loss": 2.136301736754831e-06,
	"step": 3060
	},
	{
	"epoch": 0.03584543262634931,
	"grad_norm": 6.550679972860962e-05,
	"learning_rate": 9.949152542372882e-05,
	"loss": 2.0307767044869252e-06,
	"step": 3070
	},
	{
	"epoch": 0.035962192993210386,
	"grad_norm": 1.0432325325382408e-05,
	"learning_rate": 9.915254237288136e-05,
	"loss": 2.1234629457467237e-06,
	"step": 3080
	},
	{
	"epoch": 0.036078953360071456,
	"grad_norm": 0.00010071093856822699,
	"learning_rate": 9.88135593220339e-05,
	"loss": 2.0928424419253134e-06,
	"step": 3090
	},
	{
	"epoch": 0.03619571372693253,
	"grad_norm": 5.0666458264458925e-05,
	"learning_rate": 9.847457627118644e-05,
	"loss": 2.1181087504373865e-06,
	"step": 3100
	},
	{
	"epoch": 0.0363124740937936,
	"grad_norm": 0.00012666780094150454,
	"learning_rate": 9.813559322033899e-05,
	"loss": 1.997594335989561e-06,
	"step": 3110
	},
	{
	"epoch": 0.03642923446065467,
	"grad_norm": 5.238919402472675e-05,
	"learning_rate": 9.779661016949154e-05,
	"loss": 2.063276042463258e-06,
	"step": 3120
	},
	{
	"epoch": 0.03654599482751575,
	"grad_norm": 7.314996764762327e-05,
	"learning_rate": 9.745762711864407e-05,
	"loss": 1.9893661374226214e-06,
	"step": 3130
	},
	{
	"epoch": 0.03666275519437682,
	"grad_norm": 6.618754014198203e-06,
	"learning_rate": 9.711864406779661e-05,
	"loss": 2.0250694433343596e-06,
	"step": 3140
	},
	{
	"epoch": 0.036779515561237895,
	"grad_norm": 5.7765271776588634e-05,
	"learning_rate": 9.677966101694915e-05,
	"loss": 2.039803803199902e-06,
	"step": 3150
	},
	{
	"epoch": 0.036896275928098965,
	"grad_norm": 5.030070315115154e-05,
	"learning_rate": 9.64406779661017e-05,
	"loss": 2.049379145319108e-06,
	"step": 3160
	},
	{
	"epoch": 0.03701303629496004,
	"grad_norm": 0.00014426227426156402,
	"learning_rate": 9.610169491525424e-05,
	"loss": 2.065716398647055e-06,
	"step": 3170
	},
	{
	"epoch": 0.03712979666182111,
	"grad_norm": 0.0001042195362970233,
	"learning_rate": 9.576271186440679e-05,
	"loss": 2.0669653167715295e-06,
	"step": 3180
	},
	{
	"epoch": 0.03724655702868218,
	"grad_norm": 3.6196935980115086e-05,
	"learning_rate": 9.542372881355933e-05,
	"loss": 2.0056630091858095e-06,
	"step": 3190
	},
	{
	"epoch": 0.03736331739554326,
	"grad_norm": 2.1439184365590336e-06,
	"learning_rate": 9.508474576271187e-05,
	"loss": 1.960424742719624e-06,
	"step": 3200
	},
	{
	"epoch": 0.03748007776240433,
	"grad_norm": 2.0883240722469054e-05,
	"learning_rate": 9.474576271186441e-05,
	"loss": 1.931091901496984e-06,
	"step": 3210
	},
	{
	"epoch": 0.037596838129265404,
	"grad_norm": 0.0001155544669018127,
	"learning_rate": 9.440677966101696e-05,
	"loss": 1.9896011508535595e-06,
	"step": 3220
	},
	{
	"epoch": 0.037713598496126474,
	"grad_norm": 6.560485871887067e-06,
	"learning_rate": 9.40677966101695e-05,
	"loss": 1.947508644661866e-06,
	"step": 3230
	},
	{
	"epoch": 0.03783035886298755,
	"grad_norm": 9.319369382865261e-06,
	"learning_rate": 9.372881355932204e-05,
	"loss": 2.071146809612401e-06,
	"step": 3240
	},
	{
	"epoch": 0.03794711922984862,
	"grad_norm": 8.688485831953585e-05,
	"learning_rate": 9.338983050847458e-05,
	"loss": 1.9745541067095475e-06,
	"step": 3250
	},
	{
	"epoch": 0.03806387959670969,
	"grad_norm": 2.1601556454697857e-06,
	"learning_rate": 9.305084745762712e-05,
	"loss": 1.968435935850721e-06,
	"step": 3260
	},
	{
	"epoch": 0.03818063996357077,
	"grad_norm": 2.1765763449366204e-06,
	"learning_rate": 9.271186440677968e-05,
	"loss": 1.980983688554261e-06,
	"step": 3270
	},
	{
	"epoch": 0.03829740033043184,
	"grad_norm": 1.0872706297959667e-05,
	"learning_rate": 9.237288135593221e-05,
	"loss": 1.938048990268726e-06,
	"step": 3280
	},
	{
	"epoch": 0.038414160697292914,
	"grad_norm": 4.855420411331579e-05,
	"learning_rate": 9.203389830508475e-05,
	"loss": 2.0312449123593978e-06,
	"step": 3290
	},
	{
	"epoch": 0.03853092106415398,
	"grad_norm": 0.0,
	"learning_rate": 9.169491525423729e-05,
	"loss": 1.9035969671676867e-06,
	"step": 3300
	},
	{
	"epoch": 0.03864768143101505,
	"grad_norm": 5.165005859453231e-05,
	"learning_rate": 9.135593220338983e-05,
	"loss": 1.897206311696209e-06,
	"step": 3310
	},
	{
	"epoch": 0.03876444179787613,
	"grad_norm": 5.7258494052803144e-05,
	"learning_rate": 9.101694915254238e-05,
	"loss": 1.961050838872325e-06,
	"step": 3320
	},
	{
	"epoch": 0.0388812021647372,
	"grad_norm": 1.8185855878982693e-05,
	"learning_rate": 9.067796610169493e-05,
	"loss": 1.9104940292891114e-06,
	"step": 3330
	},
	{
	"epoch": 0.038997962531598276,
	"grad_norm": 5.590146429312881e-06,
	"learning_rate": 9.033898305084746e-05,
	"loss": 1.927184530359227e-06,
	"step": 3340
	},
	{
	"epoch": 0.039114722898459346,
	"grad_norm": 7.191507756942883e-05,
	"learning_rate": 9e-05,
	"loss": 1.9486598830553705e-06,
	"step": 3350
	},
	{
	"epoch": 0.03923148326532042,
	"grad_norm": 6.619741179747507e-05,
	"learning_rate": 8.966101694915254e-05,
	"loss": 2.0193221644149164e-06,
	"step": 3360
	},
	{
	"epoch": 0.03934824363218149,
	"grad_norm": 5.084141321276547e-06,
	"learning_rate": 8.932203389830508e-05,
	"loss": 1.9432471162872387e-06,
	"step": 3370
	},
	{
	"epoch": 0.03946500399904256,
	"grad_norm": 5.648166552418843e-05,
	"learning_rate": 8.898305084745763e-05,
	"loss": 1.8662518414203078e-06,
	"step": 3380
	},
	{
	"epoch": 0.03958176436590364,
	"grad_norm": 7.099672802723944e-05,
	"learning_rate": 8.864406779661018e-05,
	"loss": 1.926773620652966e-06,
	"step": 3390
	},
	{
	"epoch": 0.03969852473276471,
	"grad_norm": 5.490633157023694e-06,
	"learning_rate": 8.830508474576271e-05,
	"loss": 2.003668669203762e-06,
	"step": 3400
	},
	{
	"epoch": 0.039815285099625786,
	"grad_norm": 4.8940302804112434e-05,
	"learning_rate": 8.796610169491526e-05,
	"loss": 1.951141894096509e-06,
	"step": 3410
	},
	{
	"epoch": 0.039932045466486855,
	"grad_norm": 5.092696665087715e-05,
	"learning_rate": 8.76271186440678e-05,
	"loss": 1.8971470126416534e-06,
	"step": 3420
	},
	{
	"epoch": 0.040048805833347925,
	"grad_norm": 4.840248220716603e-05,
	"learning_rate": 8.728813559322035e-05,
	"loss": 1.7974445654544979e-06,
	"step": 3430
	},
	{
	"epoch": 0.040165566200209,
	"grad_norm": 4.424721737450454e-06,
	"learning_rate": 8.694915254237288e-05,
	"loss": 1.9609513401519505e-06,
	"step": 3440
	},
	{
	"epoch": 0.04028232656707007,
	"grad_norm": 2.1310127067408757e-06,
	"learning_rate": 8.661016949152543e-05,
	"loss": 2.0018365830765107e-06,
	"step": 3450
	},
	{
	"epoch": 0.04039908693393115,
	"grad_norm": 7.513844320783392e-05,
	"learning_rate": 8.627118644067797e-05,
	"loss": 1.8380125766270794e-06,
	"step": 3460
	},
	{
	"epoch": 0.04051584730079222,
	"grad_norm": 4.8595178668620065e-05,
	"learning_rate": 8.593220338983051e-05,
	"loss": 1.8887243641074746e-06,
	"step": 3470
	},
	{
	"epoch": 0.040632607667653295,
	"grad_norm": 5.385456097428687e-05,
	"learning_rate": 8.559322033898305e-05,
	"loss": 1.940351830853615e-06,
	"step": 3480
	},
	{
	"epoch": 0.040749368034514365,
	"grad_norm": 0.00014665677736047655,
	"learning_rate": 8.52542372881356e-05,
	"loss": 1.8199347323388792e-06,
	"step": 3490
	},
	{
	"epoch": 0.040866128401375434,
	"grad_norm": 9.344716818304732e-05,
	"learning_rate": 8.491525423728815e-05,
	"loss": 1.8320717572350986e-06,
	"step": 3500
	},
	{
	"epoch": 0.04098288876823651,
	"grad_norm": 2.1979137727612397e-06,
	"learning_rate": 8.457627118644068e-05,
	"loss": 1.8974973500007763e-06,
	"step": 3510
	},
	{
	"epoch": 0.04109964913509758,
	"grad_norm": 5.3389358072308823e-05,
	"learning_rate": 8.423728813559323e-05,
	"loss": 1.8122955225408077e-06,
	"step": 3520
	},
	{
	"epoch": 0.04121640950195866,
	"grad_norm": 7.409727550111711e-05,
	"learning_rate": 8.389830508474577e-05,
	"loss": 1.8840732082026078e-06,
	"step": 3530
	},
	{
	"epoch": 0.04133316986881973,
	"grad_norm": 5.480295294546522e-05,
	"learning_rate": 8.355932203389832e-05,
	"loss": 1.8401231500320137e-06,
	"step": 3540
	},
	{
	"epoch": 0.0414499302356808,
	"grad_norm": 0.0,
	"learning_rate": 8.322033898305085e-05,
	"loss": 1.764593253028579e-06,
	"step": 3550
	},
	{
	"epoch": 0.041566690602541874,
	"grad_norm": 2.2396238819055725e-06,
	"learning_rate": 8.28813559322034e-05,
	"loss": 1.8283782992511987e-06,
	"step": 3560
	},
	{
	"epoch": 0.041683450969402944,
	"grad_norm": 9.503438195679337e-05,
	"learning_rate": 8.254237288135593e-05,
	"loss": 1.913698542921338e-06,
	"step": 3570
	},
	{
	"epoch": 0.04180021133626402,
	"grad_norm": 4.277124389773235e-06,
	"learning_rate": 8.220338983050848e-05,
	"loss": 1.8454575183568522e-06,
	"step": 3580
	},
	{
	"epoch": 0.04191697170312509,
	"grad_norm": 0.00010249170736642554,
	"learning_rate": 8.186440677966102e-05,
	"loss": 1.791014256014023e-06,
	"step": 3590
	},
	{
	"epoch": 0.04203373206998617,
	"grad_norm": 2.1688040305889444e-06,
	"learning_rate": 8.152542372881357e-05,
	"loss": 1.8011172869591973e-06,
	"step": 3600
	},
	{
	"epoch": 0.04215049243684724,
	"grad_norm": 6.839696288807318e-05,
	"learning_rate": 8.11864406779661e-05,
	"loss": 1.7736805602908134e-06,
	"step": 3610
	},
	{
	"epoch": 0.042267252803708306,
	"grad_norm": 7.180737156886607e-06,
	"learning_rate": 8.084745762711865e-05,
	"loss": 1.7980677512241527e-06,
	"step": 3620
	},
	{
	"epoch": 0.04238401317056938,
	"grad_norm": 6.958799076528521e-06,
	"learning_rate": 8.050847457627118e-05,
	"loss": 1.7815744286053814e-06,
	"step": 3630
	},
	{
	"epoch": 0.04250077353743045,
	"grad_norm": 6.6404591052560136e-06,
	"learning_rate": 8.016949152542374e-05,
	"loss": 1.8455340978107415e-06,
	"step": 3640
	},
	{
	"epoch": 0.04261753390429153,
	"grad_norm": 6.840089918114245e-05,
	"learning_rate": 7.983050847457627e-05,
	"loss": 1.7297694284934551e-06,
	"step": 3650
	},
	{
	"epoch": 0.0427342942711526,
	"grad_norm": 0.0,
	"learning_rate": 7.949152542372882e-05,
	"loss": 1.774441079760436e-06,
	"step": 3660
	},
	{
	"epoch": 0.04285105463801367,
	"grad_norm": 4.816091313841753e-05,
	"learning_rate": 7.915254237288135e-05,
	"loss": 1.8213042494608089e-06,
	"step": 3670
	},
	{
	"epoch": 0.042967815004874746,
	"grad_norm": 6.61901940475218e-05,
	"learning_rate": 7.88135593220339e-05,
	"loss": 1.796543801901862e-06,
	"step": 3680
	},
	{
	"epoch": 0.043084575371735816,
	"grad_norm": 2.207036914114724e-06,
	"learning_rate": 7.847457627118644e-05,
	"loss": 1.727639028104022e-06,
	"step": 3690
	},
	{
	"epoch": 0.04320133573859689,
	"grad_norm": 4.7766596253495663e-05,
	"learning_rate": 7.813559322033899e-05,
	"loss": 1.7181819202960469e-06,
	"step": 3700
	},
	{
	"epoch": 0.04331809610545796,
	"grad_norm": 9.358324314234778e-05,
	"learning_rate": 7.779661016949152e-05,
	"loss": 1.8226328393211588e-06,
	"step": 3710
	},
	{
	"epoch": 0.04343485647231904,
	"grad_norm": 4.626426743925549e-05,
	"learning_rate": 7.745762711864407e-05,
	"loss": 1.7751455743564292e-06,
	"step": 3720
	},
	{
	"epoch": 0.04355161683918011,
	"grad_norm": 0.0,
	"learning_rate": 7.711864406779662e-05,
	"loss": 1.760644772730302e-06,
	"step": 3730
	},
	{
	"epoch": 0.04366837720604118,
	"grad_norm": 5.273880105960416e-06,
	"learning_rate": 7.677966101694916e-05,
	"loss": 1.7392259906046092e-06,
	"step": 3740
	},
	{
	"epoch": 0.043785137572902255,
	"grad_norm": 0.0,
	"learning_rate": 7.644067796610171e-05,
	"loss": 1.8046715922537259e-06,
	"step": 3750
	},
	{
	"epoch": 0.043901897939763325,
	"grad_norm": 5.032142871641554e-05,
	"learning_rate": 7.610169491525424e-05,
	"loss": 1.821439764171373e-06,
	"step": 3760
	},
	{
	"epoch": 0.0440186583066244,
	"grad_norm": 2.19600315176649e-06,
	"learning_rate": 7.576271186440679e-05,
	"loss": 1.7603120795683936e-06,
	"step": 3770
	},
	{
	"epoch": 0.04413541867348547,
	"grad_norm": 0.0,
	"learning_rate": 7.542372881355932e-05,
	"loss": 1.7147014659713022e-06,
	"step": 3780
	},
	{
	"epoch": 0.04425217904034655,
	"grad_norm": 9.782256529433653e-05,
	"learning_rate": 7.508474576271187e-05,
	"loss": 1.7871834643301554e-06,
	"step": 3790
	},
	{
	"epoch": 0.04436893940720762,
	"grad_norm": 9.953362314263359e-05,
	"learning_rate": 7.474576271186441e-05,
	"loss": 1.7436228517908604e-06,
	"step": 3800
	},
	{
	"epoch": 0.04448569977406869,
	"grad_norm": 7.75048101786524e-05,
	"learning_rate": 7.440677966101696e-05,
	"loss": 1.7289874449488707e-06,
	"step": 3810
	},
	{
	"epoch": 0.044602460140929764,
	"grad_norm": 0.00010488975385669619,
	"learning_rate": 7.406779661016949e-05,
	"loss": 1.7794043742469512e-06,
	"step": 3820
	},
	{
	"epoch": 0.044719220507790834,
	"grad_norm": 2.131126166204922e-06,
	"learning_rate": 7.372881355932204e-05,
	"loss": 1.784406958904583e-06,
	"step": 3830
	},
	{
	"epoch": 0.04483598087465191,
	"grad_norm": 6.641692016273737e-05,
	"learning_rate": 7.338983050847457e-05,
	"loss": 1.8203621948487124e-06,
	"step": 3840
	},
	{
	"epoch": 0.04495274124151298,
	"grad_norm": 9.44149142014794e-05,
	"learning_rate": 7.305084745762713e-05,
	"loss": 1.7521047993795947e-06,
	"step": 3850
	},
	{
	"epoch": 0.04506950160837405,
	"grad_norm": 0.0,
	"learning_rate": 7.271186440677966e-05,
	"loss": 1.65750188898528e-06,
	"step": 3860
	},
	{
	"epoch": 0.04518626197523513,
	"grad_norm": 0.0,
	"learning_rate": 7.237288135593221e-05,
	"loss": 1.7596286852494813e-06,
	"step": 3870
	},
	{
	"epoch": 0.0453030223420962,
	"grad_norm": 5.394325853558257e-05,
	"learning_rate": 7.203389830508474e-05,
	"loss": 1.6870504623511807e-06,
	"step": 3880
	},
	{
	"epoch": 0.045419782708957274,
	"grad_norm": 2.284836000399082e-06,
	"learning_rate": 7.169491525423729e-05,
	"loss": 1.689726377662737e-06,
	"step": 3890
	},
	{
	"epoch": 0.04553654307581834,
	"grad_norm": 4.559060471365228e-05,
	"learning_rate": 7.135593220338984e-05,
	"loss": 1.7255473721888847e-06,
	"step": 3900
	},
	{
	"epoch": 0.04565330344267942,
	"grad_norm": 4.5206386857898906e-05,
	"learning_rate": 7.101694915254238e-05,
	"loss": 1.740222614898812e-06,
	"step": 3910
	},
	{
	"epoch": 0.04577006380954049,
	"grad_norm": 2.1293201371008763e-06,
	"learning_rate": 7.067796610169491e-05,
	"loss": 1.7261534594581463e-06,
	"step": 3920
	},
	{
	"epoch": 0.04588682417640156,
	"grad_norm": 1.0025795745605137e-05,
	"learning_rate": 7.033898305084746e-05,
	"loss": 1.8548162188380957e-06,
	"step": 3930
	},
	{
	"epoch": 0.046003584543262636,
	"grad_norm": 8.444023842457682e-05,
	"learning_rate": 7e-05,
	"loss": 1.645698466745671e-06,
	"step": 3940
	},
	{
	"epoch": 0.046120344910123706,
	"grad_norm": 6.808809121139348e-05,
	"learning_rate": 6.966101694915254e-05,
	"loss": 1.6807753127068281e-06,
	"step": 3950
	},
	{
	"epoch": 0.04623710527698478,
	"grad_norm": 6.510812818305567e-05,
	"learning_rate": 6.932203389830509e-05,
	"loss": 1.7703574485494756e-06,
	"step": 3960
	},
	{
	"epoch": 0.04635386564384585,
	"grad_norm": 1.5896932382020168e-05,
	"learning_rate": 6.898305084745763e-05,
	"loss": 1.7425098121748306e-06,
	"step": 3970
	},
	{
	"epoch": 0.04647062601070692,
	"grad_norm": 5.356021938496269e-05,
	"learning_rate": 6.864406779661017e-05,
	"loss": 1.7232412574230694e-06,
	"step": 3980
	},
	{
	"epoch": 0.046587386377568,
	"grad_norm": 4.5757722546113655e-05,
	"learning_rate": 6.830508474576271e-05,
	"loss": 1.657305983826518e-06,
	"step": 3990
	},
	{
	"epoch": 0.04670414674442907,
	"grad_norm": 0.00017296714941039681,
	"learning_rate": 6.796610169491526e-05,
	"loss": 1.7679916709312237e-06,
	"step": 4000
	},
	{
	"epoch": 0.046820907111290146,
	"grad_norm": 4.453680230653845e-05,
	"learning_rate": 6.76271186440678e-05,
	"loss": 1.641594644752331e-06,
	"step": 4010
	},
	{
	"epoch": 0.046937667478151215,
	"grad_norm": 8.189085747289937e-06,
	"learning_rate": 6.728813559322035e-05,
	"loss": 1.6620160749880597e-06,
	"step": 4020
	},
	{
	"epoch": 0.04705442784501229,
	"grad_norm": 2.0963232145732036e-06,
	"learning_rate": 6.694915254237288e-05,
	"loss": 1.6667652744217776e-06,
	"step": 4030
	},
	{
	"epoch": 0.04717118821187336,
	"grad_norm": 1.5700910807936452e-05,
	"learning_rate": 6.661016949152543e-05,
	"loss": 1.7349459085380658e-06,
	"step": 4040
	},
	{
	"epoch": 0.04728794857873443,
	"grad_norm": 0.0001244883897015825,
	"learning_rate": 6.627118644067796e-05,
	"loss": 1.6490987036377191e-06,
	"step": 4050
	},
	{
	"epoch": 0.04740470894559551,
	"grad_norm": 0.0,
	"learning_rate": 6.593220338983052e-05,
	"loss": 1.6804051483632065e-06,
	"step": 4060
	},
	{
	"epoch": 0.04752146931245658,
	"grad_norm": 0.00012983976921532303,
	"learning_rate": 6.559322033898306e-05,
	"loss": 1.6431975382147357e-06,
	"step": 4070
	},
	{
	"epoch": 0.047638229679317655,
	"grad_norm": 0.0,
	"learning_rate": 6.52542372881356e-05,
	"loss": 1.6927364413277245e-06,
	"step": 4080
	},
	{
	"epoch": 0.047754990046178725,
	"grad_norm": 9.585384395904839e-05,
	"learning_rate": 6.491525423728813e-05,
	"loss": 1.7203876268467865e-06,
	"step": 4090
	},
	{
	"epoch": 0.047871750413039794,
	"grad_norm": 8.841085946187377e-05,
	"learning_rate": 6.457627118644068e-05,
	"loss": 1.7156202375190332e-06,
	"step": 4100
	},
	{
	"epoch": 0.04798851077990087,
	"grad_norm": 9.684242832008749e-05,
	"learning_rate": 6.423728813559323e-05,
	"loss": 1.7227321222890168e-06,
	"step": 4110
	},
	{
	"epoch": 0.04810527114676194,
	"grad_norm": 0.0,
	"learning_rate": 6.389830508474577e-05,
	"loss": 1.6056968888733535e-06,
	"step": 4120
	},
	{
	"epoch": 0.04822203151362302,
	"grad_norm": 4.84180100102094e-06,
	"learning_rate": 6.35593220338983e-05,
	"loss": 1.6854257410159334e-06,
	"step": 4130
	},
	{
	"epoch": 0.04833879188048409,
	"grad_norm": 0.0,
	"learning_rate": 6.322033898305085e-05,
	"loss": 1.6812249668873846e-06,
	"step": 4140
	},
	{
	"epoch": 0.048455552247345164,
	"grad_norm": 0.0,
	"learning_rate": 6.288135593220339e-05,
	"loss": 1.6822612451505848e-06,
	"step": 4150
	},
	{
	"epoch": 0.048572312614206234,
	"grad_norm": 5.203986347623868e-06,
	"learning_rate": 6.254237288135593e-05,
	"loss": 1.697816878731828e-06,
	"step": 4160
	},
	{
	"epoch": 0.048689072981067304,
	"grad_norm": 9.379423136124387e-05,
	"learning_rate": 6.220338983050848e-05,
	"loss": 1.6019837858038955e-06,
	"step": 4170
	},
	{
	"epoch": 0.04880583334792838,
	"grad_norm": 2.1230739548627753e-06,
	"learning_rate": 6.186440677966102e-05,
	"loss": 1.6596117347944527e-06,
	"step": 4180
	},
	{
	"epoch": 0.04892259371478945,
	"grad_norm": 0.0,
	"learning_rate": 6.152542372881356e-05,
	"loss": 1.618065834918525e-06,
	"step": 4190
	},
	{
	"epoch": 0.04903935408165053,
	"grad_norm": 9.350182517664507e-05,
	"learning_rate": 6.11864406779661e-05,
	"loss": 1.6505833627888934e-06,
	"step": 4200
	},
	{
	"epoch": 0.0491561144485116,
	"grad_norm": 4.2700281483121216e-05,
	"learning_rate": 6.084745762711864e-05,
	"loss": 1.6859339666552842e-06,
	"step": 4210
	},
	{
	"epoch": 0.049272874815372666,
	"grad_norm": 0.00019017797603737563,
	"learning_rate": 6.050847457627119e-05,
	"loss": 1.6512869478901848e-06,
	"step": 4220
	},
	{
	"epoch": 0.04938963518223374,
	"grad_norm": 6.502112228190526e-05,
	"learning_rate": 6.016949152542373e-05,
	"loss": 1.5450377759407274e-06,
	"step": 4230
	},
	{
	"epoch": 0.04950639554909481,
	"grad_norm": 4.248663390171714e-05,
	"learning_rate": 5.9830508474576275e-05,
	"loss": 1.6007126760086976e-06,
	"step": 4240
	},
	{
	"epoch": 0.04962315591595589,
	"grad_norm": 0.0001021979987854138,
	"learning_rate": 5.9491525423728814e-05,
	"loss": 1.5740963135613128e-06,
	"step": 4250
	},
	{
	"epoch": 0.04973991628281696,
	"grad_norm": 0.0,
	"learning_rate": 5.915254237288136e-05,
	"loss": 1.7099528122344054e-06,
	"step": 4260
	},
	{
	"epoch": 0.049856676649678036,
	"grad_norm": 4.2370869778096676e-05,
	"learning_rate": 5.881355932203391e-05,
	"loss": 1.6406565919169225e-06,
	"step": 4270
	},
	{
	"epoch": 0.049973437016539106,
	"grad_norm": 4.322805398260243e-05,
	"learning_rate": 5.8474576271186446e-05,
	"loss": 1.5791374607942998e-06,
	"step": 4280
	},
	{
	"epoch": 0.050090197383400176,
	"grad_norm": 2.1999978798703523e-06,
	"learning_rate": 5.813559322033899e-05,
	"loss": 1.626175617275294e-06,
	"step": 4290
	},
	{
	"epoch": 0.05020695775026125,
	"grad_norm": 4.189540049992502e-05,
	"learning_rate": 5.7796610169491525e-05,
	"loss": 1.5821848137420603e-06,
	"step": 4300
	},
	{
	"epoch": 0.05032371811712232,
	"grad_norm": 6.657272024312988e-05,
	"learning_rate": 5.745762711864407e-05,
	"loss": 1.658791006775573e-06,
	"step": 4310
	},
	{
	"epoch": 0.0504404784839834,
	"grad_norm": 6.365696754073724e-05,
	"learning_rate": 5.711864406779661e-05,
	"loss": 1.6204114217543974e-06,
	"step": 4320
	},
	{
	"epoch": 0.05055723885084447,
	"grad_norm": 0.00016623760166112334,
	"learning_rate": 5.677966101694916e-05,
	"loss": 1.714504833216779e-06,
	"step": 4330
	},
	{
	"epoch": 0.050673999217705545,
	"grad_norm": 4.169995736447163e-05,
	"learning_rate": 5.64406779661017e-05,
	"loss": 1.6089798009488732e-06,
	"step": 4340
	},
	{
	"epoch": 0.050790759584566615,
	"grad_norm": 2.1216033928794786e-05,
	"learning_rate": 5.6101694915254243e-05,
	"loss": 1.643040195631329e-06,
	"step": 4350
	},
	{
	"epoch": 0.050907519951427685,
	"grad_norm": 2.1762675714853685e-06,
	"learning_rate": 5.576271186440678e-05,
	"loss": 1.6593365216976963e-06,
	"step": 4360
	},
	{
	"epoch": 0.05102428031828876,
	"grad_norm": 0.000107581727206707,
	"learning_rate": 5.542372881355933e-05,
	"loss": 1.6118299754452892e-06,
	"step": 4370
	},
	{
	"epoch": 0.05114104068514983,
	"grad_norm": 5.259860336082056e-05,
	"learning_rate": 5.508474576271186e-05,
	"loss": 1.5759527741465717e-06,
	"step": 4380
	},
	{
	"epoch": 0.05125780105201091,
	"grad_norm": 0.00010669691255316138,
	"learning_rate": 5.4745762711864415e-05,
	"loss": 1.5893943782430143e-06,
	"step": 4390
	},
	{
	"epoch": 0.05137456141887198,
	"grad_norm": 0.000116826115117874,
	"learning_rate": 5.440677966101695e-05,
	"loss": 1.5961008102749473e-06,
	"step": 4400
	},
	{
	"epoch": 0.05149132178573305,
	"grad_norm": 0.00015807825548108667,
	"learning_rate": 5.4067796610169494e-05,
	"loss": 1.6570907973800787e-06,
	"step": 4410
	},
	{
	"epoch": 0.051608082152594124,
	"grad_norm": 0.0,
	"learning_rate": 5.3728813559322034e-05,
	"loss": 1.5699330106144772e-06,
	"step": 4420
	},
	{
	"epoch": 0.051724842519455194,
	"grad_norm": 4.22196535510011e-06,
	"learning_rate": 5.338983050847458e-05,
	"loss": 1.5922509192023425e-06,
	"step": 4430
	},
	{
	"epoch": 0.05184160288631627,
	"grad_norm": 4.248529876349494e-05,
	"learning_rate": 5.305084745762712e-05,
	"loss": 1.5383542631752788e-06,
	"step": 4440
	},
	{
	"epoch": 0.05195836325317734,
	"grad_norm": 3.999047112301923e-05,
	"learning_rate": 5.2711864406779666e-05,
	"loss": 1.5818350220797583e-06,
	"step": 4450
	},
	{
	"epoch": 0.05207512362003842,
	"grad_norm": 5.459059320855886e-05,
	"learning_rate": 5.23728813559322e-05,
	"loss": 1.5423203876707702e-06,
	"step": 4460
	},
	{
	"epoch": 0.05219188398689949,
	"grad_norm": 5.246176806394942e-05,
	"learning_rate": 5.203389830508475e-05,
	"loss": 1.603524106030818e-06,
	"step": 4470
	},
	{
	"epoch": 0.05230864435376056,
	"grad_norm": 5.46584851690568e-06,
	"learning_rate": 5.1694915254237284e-05,
	"loss": 1.5377483578049577e-06,
	"step": 4480
	},
	{
	"epoch": 0.052425404720621634,
	"grad_norm": 4.1143452108372e-05,
	"learning_rate": 5.135593220338983e-05,
	"loss": 1.541577876196243e-06,
	"step": 4490
	},
	{
	"epoch": 0.0525421650874827,
	"grad_norm": 0.00018789821479003876,
	"learning_rate": 5.101694915254237e-05,
	"loss": 1.5388426618301309e-06,
	"step": 4500
	},
	{
	"epoch": 0.05265892545434378,
	"grad_norm": 0.00010700323764467612,
	"learning_rate": 5.0677966101694917e-05,
	"loss": 1.5483585229958407e-06,
	"step": 4510
	},
	{
	"epoch": 0.05277568582120485,
	"grad_norm": 5.43225251021795e-05,
	"learning_rate": 5.0338983050847456e-05,
	"loss": 1.5612771676387637e-06,
	"step": 4520
	},
	{
	"epoch": 0.05289244618806592,
	"grad_norm": 6.245359691092744e-05,
	"learning_rate": 5e-05,
	"loss": 1.6059664631029591e-06,
	"step": 4530
	},
	{
	"epoch": 0.053009206554926996,
	"grad_norm": 8.126306784106418e-05,
	"learning_rate": 4.966101694915254e-05,
	"loss": 1.5144147255341523e-06,
	"step": 4540
	},
	{
	"epoch": 0.053125966921788066,
	"grad_norm": 0.0,
	"learning_rate": 4.932203389830509e-05,
	"loss": 1.506343778601149e-06,
	"step": 4550
	},
	{
	"epoch": 0.05324272728864914,
	"grad_norm": 8.041210094233975e-05,
	"learning_rate": 4.898305084745763e-05,
	"loss": 1.554262235003989e-06,
	"step": 4560
	},
	{
	"epoch": 0.05335948765551021,
	"grad_norm": 3.988751268479973e-05,
	"learning_rate": 4.8644067796610174e-05,
	"loss": 1.54417721205391e-06,
	"step": 4570
	},
	{
	"epoch": 0.05347624802237129,
	"grad_norm": 5.1112729124724865e-06,
	"learning_rate": 4.8305084745762714e-05,
	"loss": 1.5159579561441205e-06,
	"step": 4580
	},
	{
	"epoch": 0.05359300838923236,
	"grad_norm": 0.0,
	"learning_rate": 4.796610169491525e-05,
	"loss": 1.5396437447634525e-06,
	"step": 4590
	},
	{
	"epoch": 0.05370976875609343,
	"grad_norm": 0.00011359211202943698,
	"learning_rate": 4.76271186440678e-05,
	"loss": 1.5359306416939945e-06,
	"step": 4600
	},
	{
	"epoch": 0.053826529122954506,
	"grad_norm": 8.150914072757587e-05,
	"learning_rate": 4.728813559322034e-05,
	"loss": 1.5157635971263516e-06,
	"step": 4610
	},
	{
	"epoch": 0.053943289489815575,
	"grad_norm": 3.945328717236407e-05,
	"learning_rate": 4.694915254237288e-05,
	"loss": 1.473337397328578e-06,
	"step": 4620
	},
	{
	"epoch": 0.05406004985667665,
	"grad_norm": 5.413919552665902e-06,
	"learning_rate": 4.6610169491525425e-05,
	"loss": 1.524340950709302e-06,
	"step": 4630
	},
	{
	"epoch": 0.05417681022353772,
	"grad_norm": 0.00017984810983762145,
	"learning_rate": 4.6271186440677964e-05,
	"loss": 1.469545986765297e-06,
	"step": 4640
	},
	{
	"epoch": 0.05429357059039879,
	"grad_norm": 0.0,
	"learning_rate": 4.593220338983051e-05,
	"loss": 1.4799027667322662e-06,
	"step": 4650
	},
	{
	"epoch": 0.05441033095725987,
	"grad_norm": 6.315819337032735e-05,
	"learning_rate": 4.559322033898305e-05,
	"loss": 1.495204651291715e-06,
	"step": 4660
	},
	{
	"epoch": 0.05452709132412094,
	"grad_norm": 4.7498702770099044e-05,
	"learning_rate": 4.5254237288135596e-05,
	"loss": 1.5366334991995244e-06,
	"step": 4670
	},
	{
	"epoch": 0.054643851690982015,
	"grad_norm": 8.457310468656942e-05,
	"learning_rate": 4.491525423728814e-05,
	"loss": 1.534151851956267e-06,
	"step": 4680
	},
	{
	"epoch": 0.054760612057843085,
	"grad_norm": 0.00012268357386346906,
	"learning_rate": 4.457627118644068e-05,
	"loss": 1.4569609447789844e-06,
	"step": 4690
	},
	{
	"epoch": 0.05487737242470416,
	"grad_norm": 0.00010966585978167132,
	"learning_rate": 4.423728813559322e-05,
	"loss": 1.5641469872207381e-06,
	"step": 4700
	},
	{
	"epoch": 0.05499413279156523,
	"grad_norm": 3.980071414844133e-05,
	"learning_rate": 4.389830508474577e-05,
	"loss": 1.464406796003459e-06,
	"step": 4710
	},
	{
	"epoch": 0.0551108931584263,
	"grad_norm": 5.441743633127771e-05,
	"learning_rate": 4.355932203389831e-05,
	"loss": 1.446505484636873e-06,
	"step": 4720
	},
	{
	"epoch": 0.05522765352528738,
	"grad_norm": 2.770342234725831e-06,
	"learning_rate": 4.3220338983050854e-05,
	"loss": 1.4363635273184626e-06,
	"step": 4730
	},
	{
	"epoch": 0.05534441389214845,
	"grad_norm": 8.978364348877221e-06,
	"learning_rate": 4.2881355932203394e-05,
	"loss": 1.482755487813847e-06,
	"step": 4740
	},
	{
	"epoch": 0.055461174259009524,
	"grad_norm": 0.00010691554052755237,
	"learning_rate": 4.254237288135593e-05,
	"loss": 1.4542250937665812e-06,
	"step": 4750
	},
	{
	"epoch": 0.055577934625870594,
	"grad_norm": 1.002875706035411e-05,
	"learning_rate": 4.220338983050848e-05,
	"loss": 1.4257126167649404e-06,
	"step": 4760
	},
	{
	"epoch": 0.05569469499273167,
	"grad_norm": 5.3997257055016235e-05,
	"learning_rate": 4.186440677966102e-05,
	"loss": 1.4622179151047022e-06,
	"step": 4770
	},
	{
	"epoch": 0.05581145535959274,
	"grad_norm": 8.037842053454369e-05,
	"learning_rate": 4.152542372881356e-05,
	"loss": 1.4390406249731313e-06,
	"step": 4780
	},
	{
	"epoch": 0.05592821572645381,
	"grad_norm": 5.411546953837387e-05,
	"learning_rate": 4.1186440677966105e-05,
	"loss": 1.5547482689726167e-06,
	"step": 4790
	},
	{
	"epoch": 0.05604497609331489,
	"grad_norm": 5.027368388255127e-06,
	"learning_rate": 4.0847457627118644e-05,
	"loss": 1.4949664546293207e-06,
	"step": 4800
	},
	{
	"epoch": 0.05616173646017596,
	"grad_norm": 0.0,
	"learning_rate": 4.050847457627119e-05,
	"loss": 1.4146327885100618e-06,
	"step": 4810
	},
	{
	"epoch": 0.05627849682703703,
	"grad_norm": 0.0,
	"learning_rate": 4.016949152542373e-05,
	"loss": 1.4047062904865014e-06,
	"step": 4820
	},
	{
	"epoch": 0.0563952571938981,
	"grad_norm": 0.0,
	"learning_rate": 3.983050847457627e-05,
	"loss": 1.378285014652647e-06,
	"step": 4830
	},
	{
	"epoch": 0.05651201756075917,
	"grad_norm": 6.339018727885559e-05,
	"learning_rate": 3.9491525423728816e-05,
	"loss": 1.4362078218255192e-06,
	"step": 4840
	},
	{
	"epoch": 0.05662877792762025,
	"grad_norm": 0.0,
	"learning_rate": 3.9152542372881355e-05,
	"loss": 1.3823691006109583e-06,
	"step": 4850
	},
	{
	"epoch": 0.05674553829448132,
	"grad_norm": 0.00011063187412219122,
	"learning_rate": 3.88135593220339e-05,
	"loss": 1.4159812963043804e-06,
	"step": 4860
	},
	{
	"epoch": 0.056862298661342396,
	"grad_norm": 3.853217276628129e-05,
	"learning_rate": 3.847457627118644e-05,
	"loss": 1.443984638171969e-06,
	"step": 4870
	},
	{
	"epoch": 0.056979059028203466,
	"grad_norm": 0.00016084953676909208,
	"learning_rate": 3.813559322033898e-05,
	"loss": 1.4580738024960739e-06,
	"step": 4880
	},
	{
	"epoch": 0.05709581939506454,
	"grad_norm": 0.0,
	"learning_rate": 3.779661016949153e-05,
	"loss": 1.385260657116305e-06,
	"step": 4890
	},
	{
	"epoch": 0.05721257976192561,
	"grad_norm": 3.9170579839264974e-05,
	"learning_rate": 3.745762711864407e-05,
	"loss": 1.444784902560059e-06,
	"step": 4900
	},
	{
	"epoch": 0.05732934012878668,
	"grad_norm": 0.00013803069305140525,
	"learning_rate": 3.711864406779661e-05,
	"loss": 1.3933115951658693e-06,
	"step": 4910
	},
	{
	"epoch": 0.05744610049564776,
	"grad_norm": 2.30526734412706e-06,
	"learning_rate": 3.677966101694915e-05,
	"loss": 1.3226872397353873e-06,
	"step": 4920
	},
	{
	"epoch": 0.05756286086250883,
	"grad_norm": 8.447455911664292e-05,
	"learning_rate": 3.644067796610169e-05,
	"loss": 1.3623964150610845e-06,
	"step": 4930
	},
	{
	"epoch": 0.057679621229369905,
	"grad_norm": 0.0,
	"learning_rate": 3.610169491525424e-05,
	"loss": 1.4465447748079896e-06,
	"step": 4940
	},
	{
	"epoch": 0.057796381596230975,
	"grad_norm": 0.0002463430864736438,
	"learning_rate": 3.5762711864406785e-05,
	"loss": 1.4583848496840802e-06,
	"step": 4950
	},
	{
	"epoch": 0.057913141963092045,
	"grad_norm": 8.406710549024865e-05,
	"learning_rate": 3.5423728813559324e-05,
	"loss": 1.3667741768585985e-06,
	"step": 4960
	},
	{
	"epoch": 0.05802990232995312,
	"grad_norm": 4.017229002784006e-05,
	"learning_rate": 3.508474576271187e-05,
	"loss": 1.3913191651226954e-06,
	"step": 4970
	},
	{
	"epoch": 0.05814666269681419,
	"grad_norm": 3.8721009332221e-05,
	"learning_rate": 3.474576271186441e-05,
	"loss": 1.4052521692065057e-06,
	"step": 4980
	},
	{
	"epoch": 0.05826342306367527,
	"grad_norm": 5.4408383221016265e-06,
	"learning_rate": 3.440677966101695e-05,
	"loss": 1.399311895511346e-06,
	"step": 4990
	},
	{
	"epoch": 0.05838018343053634,
	"grad_norm": 8.536592940799892e-05,
	"learning_rate": 3.4067796610169496e-05,
	"loss": 1.473233260185225e-06,
	"step": 5000
	}
	],
	"logging_steps": 10,
	"max_steps": 6000,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 1,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": false
	},
	"attributes": {}
	}
	},
	"total_flos": 6.20224905216e+16,
	"train_batch_size": 8,
	"trial_name": null,
	"trial_params": null
	}