Instructions to use Kaipengm2/u2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Kaipengm2/u2 with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("/inspire/hdd/global_user/zhengkaipeng-240108120123/models/OneReason-0.8B-pretrain-competition")
model = PeftModel.from_pretrained(base_model, "Kaipengm2/u2")

Transformers

How to use Kaipengm2/u2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Kaipengm2/u2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Kaipengm2/u2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Kaipengm2/u2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Kaipengm2/u2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kaipengm2/u2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Kaipengm2/u2

SGLang

How to use Kaipengm2/u2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Kaipengm2/u2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kaipengm2/u2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Kaipengm2/u2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Kaipengm2/u2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Kaipengm2/u2 with Docker Model Runner:
```
docker model run hf.co/Kaipengm2/u2
```

u2 / trainer_state.json

Kaipengm2

Upload folder using huggingface_hub

556e22c verified 28 days ago

Raw

History Blame Contribute Delete

88.5 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 1.0,
	"eval_steps": 500,
	"global_step": 2359,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.00211954217888936,
	"grad_norm": 1.478369951248169,
	"learning_rate": 5.6338028169014084e-06,
	"loss": 2.9127758026123045,
	"step": 5
	},
	{
	"epoch": 0.00423908435777872,
	"grad_norm": 1.115067720413208,
	"learning_rate": 1.267605633802817e-05,
	"loss": 2.930403137207031,
	"step": 10
	},
	{
	"epoch": 0.00635862653666808,
	"grad_norm": 0.5497398972511292,
	"learning_rate": 1.971830985915493e-05,
	"loss": 2.6963897705078126,
	"step": 15
	},
	{
	"epoch": 0.00847816871555744,
	"grad_norm": 0.2849062383174896,
	"learning_rate": 2.676056338028169e-05,
	"loss": 2.5970108032226564,
	"step": 20
	},
	{
	"epoch": 0.0105977108944468,
	"grad_norm": 0.3050392270088196,
	"learning_rate": 3.380281690140845e-05,
	"loss": 2.4680963516235352,
	"step": 25
	},
	{
	"epoch": 0.01271725307333616,
	"grad_norm": 0.3327608108520508,
	"learning_rate": 4.0845070422535214e-05,
	"loss": 2.4574920654296877,
	"step": 30
	},
	{
	"epoch": 0.01483679525222552,
	"grad_norm": 0.225258007645607,
	"learning_rate": 4.788732394366197e-05,
	"loss": 2.347016143798828,
	"step": 35
	},
	{
	"epoch": 0.01695633743111488,
	"grad_norm": 0.21145237982273102,
	"learning_rate": 5.492957746478874e-05,
	"loss": 2.2124860763549803,
	"step": 40
	},
	{
	"epoch": 0.01907587961000424,
	"grad_norm": 0.19001203775405884,
	"learning_rate": 6.197183098591549e-05,
	"loss": 2.1703458786010743,
	"step": 45
	},
	{
	"epoch": 0.0211954217888936,
	"grad_norm": 0.2154950201511383,
	"learning_rate": 6.901408450704226e-05,
	"loss": 2.101375961303711,
	"step": 50
	},
	{
	"epoch": 0.02331496396778296,
	"grad_norm": 0.1908937692642212,
	"learning_rate": 7.605633802816902e-05,
	"loss": 2.0583450317382814,
	"step": 55
	},
	{
	"epoch": 0.02543450614667232,
	"grad_norm": 0.1949082314968109,
	"learning_rate": 8.309859154929578e-05,
	"loss": 2.017146110534668,
	"step": 60
	},
	{
	"epoch": 0.02755404832556168,
	"grad_norm": 0.21955864131450653,
	"learning_rate": 9.014084507042254e-05,
	"loss": 1.9777019500732422,
	"step": 65
	},
	{
	"epoch": 0.02967359050445104,
	"grad_norm": 0.20840191841125488,
	"learning_rate": 9.718309859154931e-05,
	"loss": 1.9054256439208985,
	"step": 70
	},
	{
	"epoch": 0.0317931326833404,
	"grad_norm": 0.21708975732326508,
	"learning_rate": 9.99995758010305e-05,
	"loss": 1.8764774322509765,
	"step": 75
	},
	{
	"epoch": 0.03391267486222976,
	"grad_norm": 0.1861514002084732,
	"learning_rate": 9.999698350006063e-05,
	"loss": 1.8388587951660156,
	"step": 80
	},
	{
	"epoch": 0.03603221704111912,
	"grad_norm": 0.17940093576908112,
	"learning_rate": 9.999203468625017e-05,
	"loss": 1.8215919494628907,
	"step": 85
	},
	{
	"epoch": 0.03815175922000848,
	"grad_norm": 0.2362624704837799,
	"learning_rate": 9.998472959285203e-05,
	"loss": 1.7982364654541017,
	"step": 90
	},
	{
	"epoch": 0.04027130139889784,
	"grad_norm": 0.24312259256839752,
	"learning_rate": 9.997506856417793e-05,
	"loss": 1.757807159423828,
	"step": 95
	},
	{
	"epoch": 0.0423908435777872,
	"grad_norm": 0.21631494164466858,
	"learning_rate": 9.996305205558207e-05,
	"loss": 1.7292367935180664,
	"step": 100
	},
	{
	"epoch": 0.04451038575667656,
	"grad_norm": 0.24247734248638153,
	"learning_rate": 9.994868063343981e-05,
	"loss": 1.7515438079833985,
	"step": 105
	},
	{
	"epoch": 0.04662992793556592,
	"grad_norm": 0.24475762248039246,
	"learning_rate": 9.993195497512078e-05,
	"loss": 1.7488979339599608,
	"step": 110
	},
	{
	"epoch": 0.048749470114455275,
	"grad_norm": 0.20843763649463654,
	"learning_rate": 9.991287586895713e-05,
	"loss": 1.7254671096801757,
	"step": 115
	},
	{
	"epoch": 0.05086901229334464,
	"grad_norm": 0.21939566731452942,
	"learning_rate": 9.98914442142063e-05,
	"loss": 1.698301124572754,
	"step": 120
	},
	{
	"epoch": 0.052988554472233995,
	"grad_norm": 0.22747793793678284,
	"learning_rate": 9.986766102100861e-05,
	"loss": 1.7358760833740234,
	"step": 125
	},
	{
	"epoch": 0.05510809665112336,
	"grad_norm": 0.2305459976196289,
	"learning_rate": 9.98415274103397e-05,
	"loss": 1.6733268737792968,
	"step": 130
	},
	{
	"epoch": 0.057227638830012716,
	"grad_norm": 0.23201587796211243,
	"learning_rate": 9.981304461395764e-05,
	"loss": 1.6506608963012694,
	"step": 135
	},
	{
	"epoch": 0.05934718100890208,
	"grad_norm": 0.29317599534988403,
	"learning_rate": 9.978221397434496e-05,
	"loss": 1.679683303833008,
	"step": 140
	},
	{
	"epoch": 0.061466723187791436,
	"grad_norm": 0.2630968391895294,
	"learning_rate": 9.974903694464528e-05,
	"loss": 1.641597366333008,
	"step": 145
	},
	{
	"epoch": 0.0635862653666808,
	"grad_norm": 0.23136691749095917,
	"learning_rate": 9.971351508859488e-05,
	"loss": 1.6186656951904297,
	"step": 150
	},
	{
	"epoch": 0.06570580754557016,
	"grad_norm": 0.2308151125907898,
	"learning_rate": 9.967565008044894e-05,
	"loss": 1.647288703918457,
	"step": 155
	},
	{
	"epoch": 0.06782534972445951,
	"grad_norm": 0.22832302749156952,
	"learning_rate": 9.96354437049027e-05,
	"loss": 1.643804168701172,
	"step": 160
	},
	{
	"epoch": 0.06994489190334888,
	"grad_norm": 0.2323179692029953,
	"learning_rate": 9.959289785700732e-05,
	"loss": 1.610850715637207,
	"step": 165
	},
	{
	"epoch": 0.07206443408223824,
	"grad_norm": 0.2351847141981125,
	"learning_rate": 9.954801454208047e-05,
	"loss": 1.6312665939331055,
	"step": 170
	},
	{
	"epoch": 0.07418397626112759,
	"grad_norm": 0.24222879111766815,
	"learning_rate": 9.950079587561199e-05,
	"loss": 1.5695745468139648,
	"step": 175
	},
	{
	"epoch": 0.07630351844001695,
	"grad_norm": 0.2247307300567627,
	"learning_rate": 9.945124408316398e-05,
	"loss": 1.5752859115600586,
	"step": 180
	},
	{
	"epoch": 0.07842306061890632,
	"grad_norm": 0.28919321298599243,
	"learning_rate": 9.939936150026607e-05,
	"loss": 1.630772590637207,
	"step": 185
	},
	{
	"epoch": 0.08054260279779568,
	"grad_norm": 0.27556222677230835,
	"learning_rate": 9.934515057230523e-05,
	"loss": 1.6568859100341797,
	"step": 190
	},
	{
	"epoch": 0.08266214497668503,
	"grad_norm": 0.3280954360961914,
	"learning_rate": 9.928861385441058e-05,
	"loss": 1.5952363967895509,
	"step": 195
	},
	{
	"epoch": 0.0847816871555744,
	"grad_norm": 0.23287025094032288,
	"learning_rate": 9.922975401133293e-05,
	"loss": 1.6103595733642577,
	"step": 200
	},
	{
	"epoch": 0.08690122933446376,
	"grad_norm": 0.2420487403869629,
	"learning_rate": 9.916857381731914e-05,
	"loss": 1.5963797569274902,
	"step": 205
	},
	{
	"epoch": 0.08902077151335312,
	"grad_norm": 0.21538694202899933,
	"learning_rate": 9.910507615598146e-05,
	"loss": 1.5785039901733398,
	"step": 210
	},
	{
	"epoch": 0.09114031369224247,
	"grad_norm": 0.22233419120311737,
	"learning_rate": 9.903926402016153e-05,
	"loss": 1.5804765701293946,
	"step": 215
	},
	{
	"epoch": 0.09325985587113184,
	"grad_norm": 0.2901401221752167,
	"learning_rate": 9.897114051178934e-05,
	"loss": 1.5728794097900392,
	"step": 220
	},
	{
	"epoch": 0.0953793980500212,
	"grad_norm": 0.26263752579689026,
	"learning_rate": 9.890070884173706e-05,
	"loss": 1.5213123321533204,
	"step": 225
	},
	{
	"epoch": 0.09749894022891055,
	"grad_norm": 0.34815293550491333,
	"learning_rate": 9.882797232966764e-05,
	"loss": 1.5992424011230468,
	"step": 230
	},
	{
	"epoch": 0.09961848240779991,
	"grad_norm": 0.25123897194862366,
	"learning_rate": 9.875293440387843e-05,
	"loss": 1.561671257019043,
	"step": 235
	},
	{
	"epoch": 0.10173802458668928,
	"grad_norm": 0.236192986369133,
	"learning_rate": 9.86755986011395e-05,
	"loss": 1.6357194900512695,
	"step": 240
	},
	{
	"epoch": 0.10385756676557864,
	"grad_norm": 0.25594475865364075,
	"learning_rate": 9.859596856652699e-05,
	"loss": 1.5540438652038575,
	"step": 245
	},
	{
	"epoch": 0.10597710894446799,
	"grad_norm": 0.243271142244339,
	"learning_rate": 9.851404805325135e-05,
	"loss": 1.5619804382324218,
	"step": 250
	},
	{
	"epoch": 0.10809665112335735,
	"grad_norm": 0.27132245898246765,
	"learning_rate": 9.842984092248033e-05,
	"loss": 1.5923894882202148,
	"step": 255
	},
	{
	"epoch": 0.11021619330224672,
	"grad_norm": 0.2902625799179077,
	"learning_rate": 9.834335114315708e-05,
	"loss": 1.5520791053771972,
	"step": 260
	},
	{
	"epoch": 0.11233573548113608,
	"grad_norm": 0.21790118515491486,
	"learning_rate": 9.825458279181303e-05,
	"loss": 1.5991345405578614,
	"step": 265
	},
	{
	"epoch": 0.11445527766002543,
	"grad_norm": 0.31842270493507385,
	"learning_rate": 9.816354005237583e-05,
	"loss": 1.5432598114013671,
	"step": 270
	},
	{
	"epoch": 0.1165748198389148,
	"grad_norm": 0.26832494139671326,
	"learning_rate": 9.8070227215972e-05,
	"loss": 1.5321737289428712,
	"step": 275
	},
	{
	"epoch": 0.11869436201780416,
	"grad_norm": 0.2646670937538147,
	"learning_rate": 9.797464868072488e-05,
	"loss": 1.5474868774414063,
	"step": 280
	},
	{
	"epoch": 0.12081390419669351,
	"grad_norm": 0.19843275845050812,
	"learning_rate": 9.787680895154712e-05,
	"loss": 1.5311594009399414,
	"step": 285
	},
	{
	"epoch": 0.12293344637558287,
	"grad_norm": 0.22445574402809143,
	"learning_rate": 9.777671263992851e-05,
	"loss": 1.526435089111328,
	"step": 290
	},
	{
	"epoch": 0.12505298855447222,
	"grad_norm": 0.2522224187850952,
	"learning_rate": 9.767436446371856e-05,
	"loss": 1.5387699127197265,
	"step": 295
	},
	{
	"epoch": 0.1271725307333616,
	"grad_norm": 0.26532384753227234,
	"learning_rate": 9.756976924690412e-05,
	"loss": 1.5146759033203125,
	"step": 300
	},
	{
	"epoch": 0.12929207291225095,
	"grad_norm": 0.24386748671531677,
	"learning_rate": 9.746293191938203e-05,
	"loss": 1.4663808822631836,
	"step": 305
	},
	{
	"epoch": 0.13141161509114033,
	"grad_norm": 0.2265448272228241,
	"learning_rate": 9.735385751672679e-05,
	"loss": 1.5362167358398438,
	"step": 310
	},
	{
	"epoch": 0.13353115727002968,
	"grad_norm": 0.25418055057525635,
	"learning_rate": 9.724255117995316e-05,
	"loss": 1.5503853797912597,
	"step": 315
	},
	{
	"epoch": 0.13565069944891903,
	"grad_norm": 0.32841023802757263,
	"learning_rate": 9.712901815527386e-05,
	"loss": 1.52670259475708,
	"step": 320
	},
	{
	"epoch": 0.1377702416278084,
	"grad_norm": 0.2815428078174591,
	"learning_rate": 9.701326379385238e-05,
	"loss": 1.5199266433715821,
	"step": 325
	},
	{
	"epoch": 0.13988978380669775,
	"grad_norm": 0.27619239687919617,
	"learning_rate": 9.689529355155062e-05,
	"loss": 1.516659927368164,
	"step": 330
	},
	{
	"epoch": 0.1420093259855871,
	"grad_norm": 0.26593637466430664,
	"learning_rate": 9.677511298867184e-05,
	"loss": 1.5414225578308105,
	"step": 335
	},
	{
	"epoch": 0.14412886816447648,
	"grad_norm": 0.25387999415397644,
	"learning_rate": 9.66527277696986e-05,
	"loss": 1.46606502532959,
	"step": 340
	},
	{
	"epoch": 0.14624841034336583,
	"grad_norm": 0.2604225277900696,
	"learning_rate": 9.652814366302568e-05,
	"loss": 1.4718778610229493,
	"step": 345
	},
	{
	"epoch": 0.14836795252225518,
	"grad_norm": 0.25482121109962463,
	"learning_rate": 9.64013665406883e-05,
	"loss": 1.484701442718506,
	"step": 350
	},
	{
	"epoch": 0.15048749470114456,
	"grad_norm": 0.27724894881248474,
	"learning_rate": 9.627240237808531e-05,
	"loss": 1.5058162689208985,
	"step": 355
	},
	{
	"epoch": 0.1526070368800339,
	"grad_norm": 0.29061540961265564,
	"learning_rate": 9.614125725369747e-05,
	"loss": 1.5177499771118164,
	"step": 360
	},
	{
	"epoch": 0.15472657905892329,
	"grad_norm": 0.23264379799365997,
	"learning_rate": 9.600793734880113e-05,
	"loss": 1.4873903274536133,
	"step": 365
	},
	{
	"epoch": 0.15684612123781264,
	"grad_norm": 0.24515703320503235,
	"learning_rate": 9.587244894717674e-05,
	"loss": 1.4688379287719726,
	"step": 370
	},
	{
	"epoch": 0.15896566341670199,
	"grad_norm": 0.2518482208251953,
	"learning_rate": 9.573479843481272e-05,
	"loss": 1.4565875053405761,
	"step": 375
	},
	{
	"epoch": 0.16108520559559136,
	"grad_norm": 0.26230958104133606,
	"learning_rate": 9.559499229960451e-05,
	"loss": 1.5070652961730957,
	"step": 380
	},
	{
	"epoch": 0.1632047477744807,
	"grad_norm": 0.25381526350975037,
	"learning_rate": 9.545303713104868e-05,
	"loss": 1.5322324752807617,
	"step": 385
	},
	{
	"epoch": 0.16532428995337006,
	"grad_norm": 0.24196504056453705,
	"learning_rate": 9.530893961993244e-05,
	"loss": 1.4989935874938964,
	"step": 390
	},
	{
	"epoch": 0.16744383213225944,
	"grad_norm": 0.24316468834877014,
	"learning_rate": 9.516270655801826e-05,
	"loss": 1.4433355331420898,
	"step": 395
	},
	{
	"epoch": 0.1695633743111488,
	"grad_norm": 0.2673147916793823,
	"learning_rate": 9.50143448377237e-05,
	"loss": 1.4859644889831543,
	"step": 400
	},
	{
	"epoch": 0.17168291649003814,
	"grad_norm": 0.2447042614221573,
	"learning_rate": 9.486386145179663e-05,
	"loss": 1.510274887084961,
	"step": 405
	},
	{
	"epoch": 0.17380245866892752,
	"grad_norm": 0.24322748184204102,
	"learning_rate": 9.471126349298556e-05,
	"loss": 1.440256690979004,
	"step": 410
	},
	{
	"epoch": 0.17592200084781687,
	"grad_norm": 0.2688756585121155,
	"learning_rate": 9.455655815370542e-05,
	"loss": 1.502675437927246,
	"step": 415
	},
	{
	"epoch": 0.17804154302670624,
	"grad_norm": 0.26254403591156006,
	"learning_rate": 9.439975272569848e-05,
	"loss": 1.4326117515563965,
	"step": 420
	},
	{
	"epoch": 0.1801610852055956,
	"grad_norm": 0.25307175517082214,
	"learning_rate": 9.42408545996907e-05,
	"loss": 1.5270989418029786,
	"step": 425
	},
	{
	"epoch": 0.18228062738448494,
	"grad_norm": 0.2811579406261444,
	"learning_rate": 9.407987126504343e-05,
	"loss": 1.5156700134277343,
	"step": 430
	},
	{
	"epoch": 0.18440016956337432,
	"grad_norm": 0.26235663890838623,
	"learning_rate": 9.391681030940031e-05,
	"loss": 1.4990873336791992,
	"step": 435
	},
	{
	"epoch": 0.18651971174226367,
	"grad_norm": 0.2685093581676483,
	"learning_rate": 9.375167941832973e-05,
	"loss": 1.4492849349975585,
	"step": 440
	},
	{
	"epoch": 0.18863925392115302,
	"grad_norm": 0.2444113940000534,
	"learning_rate": 9.358448637496255e-05,
	"loss": 1.4470645904541015,
	"step": 445
	},
	{
	"epoch": 0.1907587961000424,
	"grad_norm": 0.2317148745059967,
	"learning_rate": 9.341523905962528e-05,
	"loss": 1.4810625076293946,
	"step": 450
	},
	{
	"epoch": 0.19287833827893175,
	"grad_norm": 0.250481516122818,
	"learning_rate": 9.32439454494686e-05,
	"loss": 1.4820423126220703,
	"step": 455
	},
	{
	"epoch": 0.1949978804578211,
	"grad_norm": 0.24248360097408295,
	"learning_rate": 9.307061361809141e-05,
	"loss": 1.4511442184448242,
	"step": 460
	},
	{
	"epoch": 0.19711742263671048,
	"grad_norm": 0.23305915296077728,
	"learning_rate": 9.289525173516036e-05,
	"loss": 1.4681825637817383,
	"step": 465
	},
	{
	"epoch": 0.19923696481559983,
	"grad_norm": 0.24631157517433167,
	"learning_rate": 9.271786806602465e-05,
	"loss": 1.4296730041503907,
	"step": 470
	},
	{
	"epoch": 0.2013565069944892,
	"grad_norm": 0.2306128889322281,
	"learning_rate": 9.253847097132655e-05,
	"loss": 1.471218967437744,
	"step": 475
	},
	{
	"epoch": 0.20347604917337855,
	"grad_norm": 0.280068039894104,
	"learning_rate": 9.235706890660733e-05,
	"loss": 1.4634878158569335,
	"step": 480
	},
	{
	"epoch": 0.2055955913522679,
	"grad_norm": 0.2586542069911957,
	"learning_rate": 9.217367042190872e-05,
	"loss": 1.4746664047241211,
	"step": 485
	},
	{
	"epoch": 0.20771513353115728,
	"grad_norm": 0.23478713631629944,
	"learning_rate": 9.19882841613699e-05,
	"loss": 1.442302131652832,
	"step": 490
	},
	{
	"epoch": 0.20983467571004663,
	"grad_norm": 0.2758229970932007,
	"learning_rate": 9.18009188628201e-05,
	"loss": 1.4378442764282227,
	"step": 495
	},
	{
	"epoch": 0.21195421788893598,
	"grad_norm": 0.2975175976753235,
	"learning_rate": 9.161158335736671e-05,
	"loss": 1.4621500015258788,
	"step": 500
	},
	{
	"epoch": 0.21407376006782536,
	"grad_norm": 0.33133307099342346,
	"learning_rate": 9.142028656897915e-05,
	"loss": 1.4451147079467774,
	"step": 505
	},
	{
	"epoch": 0.2161933022467147,
	"grad_norm": 0.27300840616226196,
	"learning_rate": 9.122703751406812e-05,
	"loss": 1.4552135467529297,
	"step": 510
	},
	{
	"epoch": 0.21831284442560406,
	"grad_norm": 0.23869898915290833,
	"learning_rate": 9.10318453010607e-05,
	"loss": 1.4278772354125977,
	"step": 515
	},
	{
	"epoch": 0.22043238660449344,
	"grad_norm": 0.24180854856967926,
	"learning_rate": 9.083471912997108e-05,
	"loss": 1.4868871688842773,
	"step": 520
	},
	{
	"epoch": 0.22255192878338279,
	"grad_norm": 0.30506250262260437,
	"learning_rate": 9.063566829196688e-05,
	"loss": 1.4626097679138184,
	"step": 525
	},
	{
	"epoch": 0.22467147096227216,
	"grad_norm": 0.25557592511177063,
	"learning_rate": 9.043470216893119e-05,
	"loss": 1.4360750198364258,
	"step": 530
	},
	{
	"epoch": 0.2267910131411615,
	"grad_norm": 0.21570295095443726,
	"learning_rate": 9.023183023302046e-05,
	"loss": 1.408976936340332,
	"step": 535
	},
	{
	"epoch": 0.22891055532005086,
	"grad_norm": 0.27088096737861633,
	"learning_rate": 9.002706204621803e-05,
	"loss": 1.4363481521606445,
	"step": 540
	},
	{
	"epoch": 0.23103009749894024,
	"grad_norm": 0.22459541261196136,
	"learning_rate": 8.982040725988336e-05,
	"loss": 1.443190574645996,
	"step": 545
	},
	{
	"epoch": 0.2331496396778296,
	"grad_norm": 0.29666635394096375,
	"learning_rate": 8.961187561429725e-05,
	"loss": 1.4185059547424317,
	"step": 550
	},
	{
	"epoch": 0.23526918185671894,
	"grad_norm": 0.2716991901397705,
	"learning_rate": 8.940147693820268e-05,
	"loss": 1.4261333465576171,
	"step": 555
	},
	{
	"epoch": 0.23738872403560832,
	"grad_norm": 0.27625572681427,
	"learning_rate": 8.918922114834156e-05,
	"loss": 1.4396458625793458,
	"step": 560
	},
	{
	"epoch": 0.23950826621449767,
	"grad_norm": 0.21192488074302673,
	"learning_rate": 8.897511824898734e-05,
	"loss": 1.4417417526245118,
	"step": 565
	},
	{
	"epoch": 0.24162780839338702,
	"grad_norm": 0.23892341554164886,
	"learning_rate": 8.875917833147347e-05,
	"loss": 1.4266895294189452,
	"step": 570
	},
	{
	"epoch": 0.2437473505722764,
	"grad_norm": 0.26707836985588074,
	"learning_rate": 8.854141157371771e-05,
	"loss": 1.4456998825073242,
	"step": 575
	},
	{
	"epoch": 0.24586689275116574,
	"grad_norm": 0.24583099782466888,
	"learning_rate": 8.832182823974256e-05,
	"loss": 1.4755363464355469,
	"step": 580
	},
	{
	"epoch": 0.24798643493005512,
	"grad_norm": 0.23366068303585052,
	"learning_rate": 8.810043867919131e-05,
	"loss": 1.4339129447937011,
	"step": 585
	},
	{
	"epoch": 0.25010597710894444,
	"grad_norm": 0.25772130489349365,
	"learning_rate": 8.787725332684034e-05,
	"loss": 1.4205987930297852,
	"step": 590
	},
	{
	"epoch": 0.2522255192878338,
	"grad_norm": 0.28989508748054504,
	"learning_rate": 8.765228270210724e-05,
	"loss": 1.4272231101989745,
	"step": 595
	},
	{
	"epoch": 0.2543450614667232,
	"grad_norm": 0.30134838819503784,
	"learning_rate": 8.742553740855506e-05,
	"loss": 1.4661161422729492,
	"step": 600
	},
	{
	"epoch": 0.2564646036456125,
	"grad_norm": 0.24564655125141144,
	"learning_rate": 8.719702813339248e-05,
	"loss": 1.4054734230041503,
	"step": 605
	},
	{
	"epoch": 0.2585841458245019,
	"grad_norm": 0.25341734290122986,
	"learning_rate": 8.696676564697008e-05,
	"loss": 1.4335659980773925,
	"step": 610
	},
	{
	"epoch": 0.2607036880033913,
	"grad_norm": 0.23098282516002655,
	"learning_rate": 8.673476080227276e-05,
	"loss": 1.401704978942871,
	"step": 615
	},
	{
	"epoch": 0.26282323018228065,
	"grad_norm": 0.2564055025577545,
	"learning_rate": 8.650102453440813e-05,
	"loss": 1.417249870300293,
	"step": 620
	},
	{
	"epoch": 0.26494277236117,
	"grad_norm": 0.28266844153404236,
	"learning_rate": 8.626556786009117e-05,
	"loss": 1.3447033882141113,
	"step": 625
	},
	{
	"epoch": 0.26706231454005935,
	"grad_norm": 0.26195573806762695,
	"learning_rate": 8.602840187712495e-05,
	"loss": 1.4243742942810058,
	"step": 630
	},
	{
	"epoch": 0.26918185671894873,
	"grad_norm": 0.23798225820064545,
	"learning_rate": 8.57895377638775e-05,
	"loss": 1.4327657699584961,
	"step": 635
	},
	{
	"epoch": 0.27130139889783805,
	"grad_norm": 0.2587537169456482,
	"learning_rate": 8.554898677875509e-05,
	"loss": 1.4313726425170898,
	"step": 640
	},
	{
	"epoch": 0.27342094107672743,
	"grad_norm": 0.26167142391204834,
	"learning_rate": 8.530676025967143e-05,
	"loss": 1.3799146652221679,
	"step": 645
	},
	{
	"epoch": 0.2755404832556168,
	"grad_norm": 0.27753326296806335,
	"learning_rate": 8.506286962351334e-05,
	"loss": 1.4059999465942383,
	"step": 650
	},
	{
	"epoch": 0.27766002543450613,
	"grad_norm": 0.3176511824131012,
	"learning_rate": 8.481732636560265e-05,
	"loss": 1.4076026916503905,
	"step": 655
	},
	{
	"epoch": 0.2797795676133955,
	"grad_norm": 0.2393917590379715,
	"learning_rate": 8.457014205915438e-05,
	"loss": 1.4270062446594238,
	"step": 660
	},
	{
	"epoch": 0.2818991097922849,
	"grad_norm": 0.24150854349136353,
	"learning_rate": 8.432132835473126e-05,
	"loss": 1.4393320083618164,
	"step": 665
	},
	{
	"epoch": 0.2840186519711742,
	"grad_norm": 0.28520679473876953,
	"learning_rate": 8.407089697969457e-05,
	"loss": 1.4433520317077637,
	"step": 670
	},
	{
	"epoch": 0.2861381941500636,
	"grad_norm": 0.26772400736808777,
	"learning_rate": 8.381885973765145e-05,
	"loss": 1.3935935020446777,
	"step": 675
	},
	{
	"epoch": 0.28825773632895296,
	"grad_norm": 0.2673088610172272,
	"learning_rate": 8.356522850789852e-05,
	"loss": 1.4048246383666991,
	"step": 680
	},
	{
	"epoch": 0.2903772785078423,
	"grad_norm": 0.25384509563446045,
	"learning_rate": 8.3310015244862e-05,
	"loss": 1.4248210906982421,
	"step": 685
	},
	{
	"epoch": 0.29249682068673166,
	"grad_norm": 0.2267899215221405,
	"learning_rate": 8.30532319775342e-05,
	"loss": 1.4328462600708007,
	"step": 690
	},
	{
	"epoch": 0.29461636286562104,
	"grad_norm": 0.22595034539699554,
	"learning_rate": 8.279489080890665e-05,
	"loss": 1.4718433380126954,
	"step": 695
	},
	{
	"epoch": 0.29673590504451036,
	"grad_norm": 0.3098070025444031,
	"learning_rate": 8.253500391539956e-05,
	"loss": 1.3861515045166015,
	"step": 700
	},
	{
	"epoch": 0.29885544722339974,
	"grad_norm": 0.26085492968559265,
	"learning_rate": 8.2273583546288e-05,
	"loss": 1.418048095703125,
	"step": 705
	},
	{
	"epoch": 0.3009749894022891,
	"grad_norm": 0.24011294543743134,
	"learning_rate": 8.201064202312441e-05,
	"loss": 1.401289939880371,
	"step": 710
	},
	{
	"epoch": 0.30309453158117844,
	"grad_norm": 0.26714739203453064,
	"learning_rate": 8.174619173915801e-05,
	"loss": 1.404327392578125,
	"step": 715
	},
	{
	"epoch": 0.3052140737600678,
	"grad_norm": 0.29608142375946045,
	"learning_rate": 8.148024515875057e-05,
	"loss": 1.360389804840088,
	"step": 720
	},
	{
	"epoch": 0.3073336159389572,
	"grad_norm": 0.2537589371204376,
	"learning_rate": 8.121281481678896e-05,
	"loss": 1.4211153030395507,
	"step": 725
	},
	{
	"epoch": 0.30945315811784657,
	"grad_norm": 0.24114106595516205,
	"learning_rate": 8.094391331809432e-05,
	"loss": 1.3666695594787597,
	"step": 730
	},
	{
	"epoch": 0.3115727002967359,
	"grad_norm": 0.2491806000471115,
	"learning_rate": 8.067355333682798e-05,
	"loss": 1.373825454711914,
	"step": 735
	},
	{
	"epoch": 0.31369224247562527,
	"grad_norm": 0.2774358093738556,
	"learning_rate": 8.040174761589406e-05,
	"loss": 1.3748723983764648,
	"step": 740
	},
	{
	"epoch": 0.31581178465451465,
	"grad_norm": 0.28923022747039795,
	"learning_rate": 8.012850896633887e-05,
	"loss": 1.4060026168823243,
	"step": 745
	},
	{
	"epoch": 0.31793132683340397,
	"grad_norm": 0.2848386764526367,
	"learning_rate": 7.985385026674712e-05,
	"loss": 1.4264955520629883,
	"step": 750
	},
	{
	"epoch": 0.32005086901229335,
	"grad_norm": 0.2752326726913452,
	"learning_rate": 7.957778446263485e-05,
	"loss": 1.392862319946289,
	"step": 755
	},
	{
	"epoch": 0.3221704111911827,
	"grad_norm": 0.2647860646247864,
	"learning_rate": 7.930032456583931e-05,
	"loss": 1.3425813674926759,
	"step": 760
	},
	{
	"epoch": 0.32428995337007205,
	"grad_norm": 0.2188040316104889,
	"learning_rate": 7.902148365390568e-05,
	"loss": 1.4169334411621093,
	"step": 765
	},
	{
	"epoch": 0.3264094955489614,
	"grad_norm": 0.2687230408191681,
	"learning_rate": 7.874127486947059e-05,
	"loss": 1.3839466094970703,
	"step": 770
	},
	{
	"epoch": 0.3285290377278508,
	"grad_norm": 0.2475503385066986,
	"learning_rate": 7.845971141964286e-05,
	"loss": 1.4165502548217774,
	"step": 775
	},
	{
	"epoch": 0.3306485799067401,
	"grad_norm": 0.2595086395740509,
	"learning_rate": 7.817680657538078e-05,
	"loss": 1.4441847801208496,
	"step": 780
	},
	{
	"epoch": 0.3327681220856295,
	"grad_norm": 0.23000121116638184,
	"learning_rate": 7.789257367086677e-05,
	"loss": 1.3525403022766114,
	"step": 785
	},
	{
	"epoch": 0.3348876642645189,
	"grad_norm": 0.2894885241985321,
	"learning_rate": 7.760702610287888e-05,
	"loss": 1.3827078819274903,
	"step": 790
	},
	{
	"epoch": 0.3370072064434082,
	"grad_norm": 0.24365928769111633,
	"learning_rate": 7.732017733015925e-05,
	"loss": 1.378408432006836,
	"step": 795
	},
	{
	"epoch": 0.3391267486222976,
	"grad_norm": 0.25980210304260254,
	"learning_rate": 7.703204087277988e-05,
	"loss": 1.363591194152832,
	"step": 800
	},
	{
	"epoch": 0.34124629080118696,
	"grad_norm": 0.2965616285800934,
	"learning_rate": 7.674263031150535e-05,
	"loss": 1.4055633544921875,
	"step": 805
	},
	{
	"epoch": 0.3433658329800763,
	"grad_norm": 0.3120691776275635,
	"learning_rate": 7.645195928715265e-05,
	"loss": 1.398946475982666,
	"step": 810
	},
	{
	"epoch": 0.34548537515896566,
	"grad_norm": 0.23403185606002808,
	"learning_rate": 7.616004149994836e-05,
	"loss": 1.3970661163330078,
	"step": 815
	},
	{
	"epoch": 0.34760491733785503,
	"grad_norm": 0.2513766586780548,
	"learning_rate": 7.586689070888284e-05,
	"loss": 1.3672683715820313,
	"step": 820
	},
	{
	"epoch": 0.34972445951674436,
	"grad_norm": 0.22340752184391022,
	"learning_rate": 7.557252073106175e-05,
	"loss": 1.4124885559082032,
	"step": 825
	},
	{
	"epoch": 0.35184400169563373,
	"grad_norm": 0.3091108798980713,
	"learning_rate": 7.527694544105479e-05,
	"loss": 1.3688016891479493,
	"step": 830
	},
	{
	"epoch": 0.3539635438745231,
	"grad_norm": 0.3515608608722687,
	"learning_rate": 7.498017877024175e-05,
	"loss": 1.3278549194335938,
	"step": 835
	},
	{
	"epoch": 0.3560830860534125,
	"grad_norm": 0.22320693731307983,
	"learning_rate": 7.468223470615593e-05,
	"loss": 1.383990478515625,
	"step": 840
	},
	{
	"epoch": 0.3582026282323018,
	"grad_norm": 0.2978534996509552,
	"learning_rate": 7.438312729182478e-05,
	"loss": 1.372886848449707,
	"step": 845
	},
	{
	"epoch": 0.3603221704111912,
	"grad_norm": 0.2522164583206177,
	"learning_rate": 7.40828706251081e-05,
	"loss": 1.3534553527832032,
	"step": 850
	},
	{
	"epoch": 0.36244171259008057,
	"grad_norm": 0.28909653425216675,
	"learning_rate": 7.378147885803348e-05,
	"loss": 1.3604212760925294,
	"step": 855
	},
	{
	"epoch": 0.3645612547689699,
	"grad_norm": 0.21767939627170563,
	"learning_rate": 7.347896619612932e-05,
	"loss": 1.3633522987365723,
	"step": 860
	},
	{
	"epoch": 0.36668079694785927,
	"grad_norm": 0.24926166236400604,
	"learning_rate": 7.317534689775528e-05,
	"loss": 1.3965436935424804,
	"step": 865
	},
	{
	"epoch": 0.36880033912674864,
	"grad_norm": 0.2501588761806488,
	"learning_rate": 7.287063527343021e-05,
	"loss": 1.3550969123840333,
	"step": 870
	},
	{
	"epoch": 0.37091988130563797,
	"grad_norm": 0.24204188585281372,
	"learning_rate": 7.256484568515769e-05,
	"loss": 1.3854890823364259,
	"step": 875
	},
	{
	"epoch": 0.37303942348452734,
	"grad_norm": 0.24696694314479828,
	"learning_rate": 7.225799254574904e-05,
	"loss": 1.4168305397033691,
	"step": 880
	},
	{
	"epoch": 0.3751589656634167,
	"grad_norm": 0.24754558503627777,
	"learning_rate": 7.195009031814413e-05,
	"loss": 1.4237482070922851,
	"step": 885
	},
	{
	"epoch": 0.37727850784230604,
	"grad_norm": 0.2518116235733032,
	"learning_rate": 7.164115351472948e-05,
	"loss": 1.349531650543213,
	"step": 890
	},
	{
	"epoch": 0.3793980500211954,
	"grad_norm": 0.28398391604423523,
	"learning_rate": 7.133119669665451e-05,
	"loss": 1.3038536071777345,
	"step": 895
	},
	{
	"epoch": 0.3815175922000848,
	"grad_norm": 0.2679619789123535,
	"learning_rate": 7.102023447314501e-05,
	"loss": 1.30383243560791,
	"step": 900
	},
	{
	"epoch": 0.3836371343789741,
	"grad_norm": 0.2988184988498688,
	"learning_rate": 7.070828150081466e-05,
	"loss": 1.3719066619873046,
	"step": 905
	},
	{
	"epoch": 0.3857566765578635,
	"grad_norm": 0.23092111945152283,
	"learning_rate": 7.03953524829742e-05,
	"loss": 1.3580788612365722,
	"step": 910
	},
	{
	"epoch": 0.3878762187367529,
	"grad_norm": 0.30076298117637634,
	"learning_rate": 7.008146216893845e-05,
	"loss": 1.3289698600769042,
	"step": 915
	},
	{
	"epoch": 0.3899957609156422,
	"grad_norm": 0.27456188201904297,
	"learning_rate": 6.976662535333107e-05,
	"loss": 1.3972248077392577,
	"step": 920
	},
	{
	"epoch": 0.3921153030945316,
	"grad_norm": 0.2861294150352478,
	"learning_rate": 6.94508568753873e-05,
	"loss": 1.3623567581176759,
	"step": 925
	},
	{
	"epoch": 0.39423484527342095,
	"grad_norm": 0.29730045795440674,
	"learning_rate": 6.91341716182545e-05,
	"loss": 1.381660270690918,
	"step": 930
	},
	{
	"epoch": 0.3963543874523103,
	"grad_norm": 0.2967582643032074,
	"learning_rate": 6.881658450829063e-05,
	"loss": 1.3973417282104492,
	"step": 935
	},
	{
	"epoch": 0.39847392963119965,
	"grad_norm": 0.24039551615715027,
	"learning_rate": 6.849811051436088e-05,
	"loss": 1.342219352722168,
	"step": 940
	},
	{
	"epoch": 0.40059347181008903,
	"grad_norm": 0.27450528740882874,
	"learning_rate": 6.817876464713192e-05,
	"loss": 1.3056873321533202,
	"step": 945
	},
	{
	"epoch": 0.4027130139889784,
	"grad_norm": 0.2570208013057709,
	"learning_rate": 6.785856195836458e-05,
	"loss": 1.3302719116210937,
	"step": 950
	},
	{
	"epoch": 0.40483255616786773,
	"grad_norm": 0.27621614933013916,
	"learning_rate": 6.75375175402043e-05,
	"loss": 1.3917129516601563,
	"step": 955
	},
	{
	"epoch": 0.4069520983467571,
	"grad_norm": 0.2792826294898987,
	"learning_rate": 6.721564652446986e-05,
	"loss": 1.3864983558654784,
	"step": 960
	},
	{
	"epoch": 0.4090716405256465,
	"grad_norm": 0.25702837109565735,
	"learning_rate": 6.689296408194018e-05,
	"loss": 1.3953963279724122,
	"step": 965
	},
	{
	"epoch": 0.4111911827045358,
	"grad_norm": 0.24588777124881744,
	"learning_rate": 6.656948542163918e-05,
	"loss": 1.372115421295166,
	"step": 970
	},
	{
	"epoch": 0.4133107248834252,
	"grad_norm": 0.25069954991340637,
	"learning_rate": 6.6245225790119e-05,
	"loss": 1.3625761032104493,
	"step": 975
	},
	{
	"epoch": 0.41543026706231456,
	"grad_norm": 0.2256464809179306,
	"learning_rate": 6.592020047074144e-05,
	"loss": 1.3400249481201172,
	"step": 980
	},
	{
	"epoch": 0.4175498092412039,
	"grad_norm": 0.41250014305114746,
	"learning_rate": 6.559442478295745e-05,
	"loss": 1.372917366027832,
	"step": 985
	},
	{
	"epoch": 0.41966935142009326,
	"grad_norm": 0.35147812962532043,
	"learning_rate": 6.526791408158524e-05,
	"loss": 1.3529234886169434,
	"step": 990
	},
	{
	"epoch": 0.42178889359898264,
	"grad_norm": 0.2732420265674591,
	"learning_rate": 6.494068375608646e-05,
	"loss": 1.3784616470336915,
	"step": 995
	},
	{
	"epoch": 0.42390843577787196,
	"grad_norm": 0.2583400309085846,
	"learning_rate": 6.461274922984086e-05,
	"loss": 1.3561012268066406,
	"step": 1000
	},
	{
	"epoch": 0.42602797795676134,
	"grad_norm": 0.30438780784606934,
	"learning_rate": 6.42841259594194e-05,
	"loss": 1.3605072021484375,
	"step": 1005
	},
	{
	"epoch": 0.4281475201356507,
	"grad_norm": 0.2518197000026703,
	"learning_rate": 6.395482943385567e-05,
	"loss": 1.3522664070129395,
	"step": 1010
	},
	{
	"epoch": 0.43026706231454004,
	"grad_norm": 0.3172878623008728,
	"learning_rate": 6.362487517391581e-05,
	"loss": 1.4028697967529298,
	"step": 1015
	},
	{
	"epoch": 0.4323866044934294,
	"grad_norm": 0.24216385185718536,
	"learning_rate": 6.32942787313671e-05,
	"loss": 1.3458670616149901,
	"step": 1020
	},
	{
	"epoch": 0.4345061466723188,
	"grad_norm": 0.26854652166366577,
	"learning_rate": 6.296305568824484e-05,
	"loss": 1.3897855758666993,
	"step": 1025
	},
	{
	"epoch": 0.4366256888512081,
	"grad_norm": 0.2709643244743347,
	"learning_rate": 6.263122165611796e-05,
	"loss": 1.383059024810791,
	"step": 1030
	},
	{
	"epoch": 0.4387452310300975,
	"grad_norm": 0.2364184558391571,
	"learning_rate": 6.229879227535318e-05,
	"loss": 1.342313289642334,
	"step": 1035
	},
	{
	"epoch": 0.44086477320898687,
	"grad_norm": 0.26499423384666443,
	"learning_rate": 6.19657832143779e-05,
	"loss": 1.3966899871826173,
	"step": 1040
	},
	{
	"epoch": 0.4429843153878762,
	"grad_norm": 0.28243476152420044,
	"learning_rate": 6.163221016894156e-05,
	"loss": 1.3298649787902832,
	"step": 1045
	},
	{
	"epoch": 0.44510385756676557,
	"grad_norm": 0.2845008075237274,
	"learning_rate": 6.129808886137603e-05,
	"loss": 1.394639015197754,
	"step": 1050
	},
	{
	"epoch": 0.44722339974565495,
	"grad_norm": 0.271884560585022,
	"learning_rate": 6.09634350398544e-05,
	"loss": 1.3739830017089845,
	"step": 1055
	},
	{
	"epoch": 0.4493429419245443,
	"grad_norm": 0.27650710940361023,
	"learning_rate": 6.062826447764883e-05,
	"loss": 1.366208553314209,
	"step": 1060
	},
	{
	"epoch": 0.45146248410343365,
	"grad_norm": 0.26297181844711304,
	"learning_rate": 6.0292592972387085e-05,
	"loss": 1.302414321899414,
	"step": 1065
	},
	{
	"epoch": 0.453582026282323,
	"grad_norm": 0.2801998257637024,
	"learning_rate": 5.9956436345307874e-05,
	"loss": 1.3412423133850098,
	"step": 1070
	},
	{
	"epoch": 0.4557015684612124,
	"grad_norm": 0.28200995922088623,
	"learning_rate": 5.9619810440515255e-05,
	"loss": 1.3836960792541504,
	"step": 1075
	},
	{
	"epoch": 0.4578211106401017,
	"grad_norm": 0.2644345462322235,
	"learning_rate": 5.928273112423177e-05,
	"loss": 1.3341361045837403,
	"step": 1080
	},
	{
	"epoch": 0.4599406528189911,
	"grad_norm": 0.31360724568367004,
	"learning_rate": 5.894521428405067e-05,
	"loss": 1.3307119369506837,
	"step": 1085
	},
	{
	"epoch": 0.4620601949978805,
	"grad_norm": 0.27192357182502747,
	"learning_rate": 5.8607275828187056e-05,
	"loss": 1.3375093460083007,
	"step": 1090
	},
	{
	"epoch": 0.4641797371767698,
	"grad_norm": 0.27361467480659485,
	"learning_rate": 5.826893168472807e-05,
	"loss": 1.3176691055297851,
	"step": 1095
	},
	{
	"epoch": 0.4662992793556592,
	"grad_norm": 0.23689980804920197,
	"learning_rate": 5.793019780088217e-05,
	"loss": 1.3547334671020508,
	"step": 1100
	},
	{
	"epoch": 0.46841882153454856,
	"grad_norm": 0.2812424302101135,
	"learning_rate": 5.759109014222752e-05,
	"loss": 1.2941896438598632,
	"step": 1105
	},
	{
	"epoch": 0.4705383637134379,
	"grad_norm": 0.25365373492240906,
	"learning_rate": 5.725162469195943e-05,
	"loss": 1.344375991821289,
	"step": 1110
	},
	{
	"epoch": 0.47265790589232726,
	"grad_norm": 0.2660122215747833,
	"learning_rate": 5.6911817450137e-05,
	"loss": 1.3773727416992188,
	"step": 1115
	},
	{
	"epoch": 0.47477744807121663,
	"grad_norm": 0.28968071937561035,
	"learning_rate": 5.6571684432929085e-05,
	"loss": 1.3161144256591797,
	"step": 1120
	},
	{
	"epoch": 0.47689699025010596,
	"grad_norm": 0.23999790847301483,
	"learning_rate": 5.62312416718593e-05,
	"loss": 1.3463720321655273,
	"step": 1125
	},
	{
	"epoch": 0.47901653242899533,
	"grad_norm": 0.25241681933403015,
	"learning_rate": 5.589050521305048e-05,
	"loss": 1.3290607452392578,
	"step": 1130
	},
	{
	"epoch": 0.4811360746078847,
	"grad_norm": 0.3081045150756836,
	"learning_rate": 5.554949111646834e-05,
	"loss": 1.3267099380493164,
	"step": 1135
	},
	{
	"epoch": 0.48325561678677403,
	"grad_norm": 0.3254645764827728,
	"learning_rate": 5.52082154551645e-05,
	"loss": 1.3080659866333009,
	"step": 1140
	},
	{
	"epoch": 0.4853751589656634,
	"grad_norm": 0.2733573317527771,
	"learning_rate": 5.4866694314518964e-05,
	"loss": 1.3987855911254883,
	"step": 1145
	},
	{
	"epoch": 0.4874947011445528,
	"grad_norm": 0.2580370008945465,
	"learning_rate": 5.45249437914819e-05,
	"loss": 1.2663162231445313,
	"step": 1150
	},
	{
	"epoch": 0.4896142433234421,
	"grad_norm": 0.2923078238964081,
	"learning_rate": 5.418297999381503e-05,
	"loss": 1.3100603103637696,
	"step": 1155
	},
	{
	"epoch": 0.4917337855023315,
	"grad_norm": 0.29695257544517517,
	"learning_rate": 5.384081903933235e-05,
	"loss": 1.3386902809143066,
	"step": 1160
	},
	{
	"epoch": 0.49385332768122087,
	"grad_norm": 0.25486746430397034,
	"learning_rate": 5.349847705514045e-05,
	"loss": 1.3110622406005858,
	"step": 1165
	},
	{
	"epoch": 0.49597286986011024,
	"grad_norm": 0.24641317129135132,
	"learning_rate": 5.3155970176878425e-05,
	"loss": 1.3458501815795898,
	"step": 1170
	},
	{
	"epoch": 0.49809241203899957,
	"grad_norm": 0.2538903057575226,
	"learning_rate": 5.2813314547957346e-05,
	"loss": 1.324117088317871,
	"step": 1175
	},
	{
	"epoch": 0.5002119542178889,
	"grad_norm": 0.3414819836616516,
	"learning_rate": 5.2470526318799365e-05,
	"loss": 1.3489215850830079,
	"step": 1180
	},
	{
	"epoch": 0.5023314963967783,
	"grad_norm": 0.2770753800868988,
	"learning_rate": 5.212762164607648e-05,
	"loss": 1.3533670425415039,
	"step": 1185
	},
	{
	"epoch": 0.5044510385756676,
	"grad_norm": 0.23823553323745728,
	"learning_rate": 5.178461669194903e-05,
	"loss": 1.2994348526000976,
	"step": 1190
	},
	{
	"epoch": 0.506570580754557,
	"grad_norm": 0.3061671853065491,
	"learning_rate": 5.1441527623303945e-05,
	"loss": 1.3628089904785157,
	"step": 1195
	},
	{
	"epoch": 0.5086901229334464,
	"grad_norm": 0.24663905799388885,
	"learning_rate": 5.109837061099274e-05,
	"loss": 1.3552839279174804,
	"step": 1200
	},
	{
	"epoch": 0.5108096651123357,
	"grad_norm": 0.29530245065689087,
	"learning_rate": 5.0755161829069294e-05,
	"loss": 1.3993040084838868,
	"step": 1205
	},
	{
	"epoch": 0.512929207291225,
	"grad_norm": 0.2994881570339203,
	"learning_rate": 5.041191745402758e-05,
	"loss": 1.2775969505310059,
	"step": 1210
	},
	{
	"epoch": 0.5150487494701145,
	"grad_norm": 0.26164939999580383,
	"learning_rate": 5.006865366403918e-05,
	"loss": 1.3317434310913085,
	"step": 1215
	},
	{
	"epoch": 0.5171682916490038,
	"grad_norm": 0.28463315963745117,
	"learning_rate": 4.972538663819073e-05,
	"loss": 1.320242214202881,
	"step": 1220
	},
	{
	"epoch": 0.5192878338278932,
	"grad_norm": 0.2733197510242462,
	"learning_rate": 4.9382132555721434e-05,
	"loss": 1.3351732254028321,
	"step": 1225
	},
	{
	"epoch": 0.5214073760067826,
	"grad_norm": 0.32395341992378235,
	"learning_rate": 4.9038907595260374e-05,
	"loss": 1.3229989051818847,
	"step": 1230
	},
	{
	"epoch": 0.5235269181856719,
	"grad_norm": 0.31577321887016296,
	"learning_rate": 4.869572793406408e-05,
	"loss": 1.2411699295043945,
	"step": 1235
	},
	{
	"epoch": 0.5256464603645613,
	"grad_norm": 0.2966740131378174,
	"learning_rate": 4.835260974725397e-05,
	"loss": 1.2873966217041015,
	"step": 1240
	},
	{
	"epoch": 0.5277660025434506,
	"grad_norm": 0.2768215239048004,
	"learning_rate": 4.800956920705394e-05,
	"loss": 1.3199604034423829,
	"step": 1245
	},
	{
	"epoch": 0.52988554472234,
	"grad_norm": 0.255959689617157,
	"learning_rate": 4.766662248202823e-05,
	"loss": 1.35555419921875,
	"step": 1250
	},
	{
	"epoch": 0.5320050869012294,
	"grad_norm": 0.2854427695274353,
	"learning_rate": 4.7323785736319244e-05,
	"loss": 1.285231590270996,
	"step": 1255
	},
	{
	"epoch": 0.5341246290801187,
	"grad_norm": 0.26254940032958984,
	"learning_rate": 4.6981075128885693e-05,
	"loss": 1.3078142166137696,
	"step": 1260
	},
	{
	"epoch": 0.536244171259008,
	"grad_norm": 0.26061853766441345,
	"learning_rate": 4.663850681274106e-05,
	"loss": 1.2951852798461914,
	"step": 1265
	},
	{
	"epoch": 0.5383637134378975,
	"grad_norm": 0.2561376392841339,
	"learning_rate": 4.629609693419215e-05,
	"loss": 1.3492764472961425,
	"step": 1270
	},
	{
	"epoch": 0.5404832556167868,
	"grad_norm": 0.2746264934539795,
	"learning_rate": 4.595386163207813e-05,
	"loss": 1.3047043800354003,
	"step": 1275
	},
	{
	"epoch": 0.5426027977956761,
	"grad_norm": 0.25904104113578796,
	"learning_rate": 4.561181703700986e-05,
	"loss": 1.2830739974975587,
	"step": 1280
	},
	{
	"epoch": 0.5447223399745655,
	"grad_norm": 0.2630980312824249,
	"learning_rate": 4.526997927060953e-05,
	"loss": 1.2768247604370118,
	"step": 1285
	},
	{
	"epoch": 0.5468418821534549,
	"grad_norm": 0.28676772117614746,
	"learning_rate": 4.492836444475093e-05,
	"loss": 1.3044819831848145,
	"step": 1290
	},
	{
	"epoch": 0.5489614243323442,
	"grad_norm": 0.3154017925262451,
	"learning_rate": 4.4586988660799885e-05,
	"loss": 1.3336353302001953,
	"step": 1295
	},
	{
	"epoch": 0.5510809665112336,
	"grad_norm": 0.21752913296222687,
	"learning_rate": 4.424586800885551e-05,
	"loss": 1.2934207916259766,
	"step": 1300
	},
	{
	"epoch": 0.5532005086901229,
	"grad_norm": 0.2799369990825653,
	"learning_rate": 4.390501856699169e-05,
	"loss": 1.2993745803833008,
	"step": 1305
	},
	{
	"epoch": 0.5553200508690123,
	"grad_norm": 0.271516352891922,
	"learning_rate": 4.356445640049941e-05,
	"loss": 1.3014348030090332,
	"step": 1310
	},
	{
	"epoch": 0.5574395930479017,
	"grad_norm": 0.2767527997493744,
	"learning_rate": 4.322419756112943e-05,
	"loss": 1.2999347686767577,
	"step": 1315
	},
	{
	"epoch": 0.559559135226791,
	"grad_norm": 0.2813481092453003,
	"learning_rate": 4.288425808633575e-05,
	"loss": 1.3840051651000977,
	"step": 1320
	},
	{
	"epoch": 0.5616786774056803,
	"grad_norm": 0.2867674231529236,
	"learning_rate": 4.2544653998519786e-05,
	"loss": 1.3272880554199218,
	"step": 1325
	},
	{
	"epoch": 0.5637982195845698,
	"grad_norm": 0.2314652055501938,
	"learning_rate": 4.2205401304275055e-05,
	"loss": 1.2865137100219726,
	"step": 1330
	},
	{
	"epoch": 0.5659177617634591,
	"grad_norm": 0.2834185063838959,
	"learning_rate": 4.1866515993632853e-05,
	"loss": 1.3238757133483887,
	"step": 1335
	},
	{
	"epoch": 0.5680373039423484,
	"grad_norm": 0.281909316778183,
	"learning_rate": 4.152801403930855e-05,
	"loss": 1.2689424514770509,
	"step": 1340
	},
	{
	"epoch": 0.5701568461212378,
	"grad_norm": 0.26909124851226807,
	"learning_rate": 4.11899113959487e-05,
	"loss": 1.31326904296875,
	"step": 1345
	},
	{
	"epoch": 0.5722763883001272,
	"grad_norm": 0.2861478924751282,
	"learning_rate": 4.085222399937913e-05,
	"loss": 1.3619144439697266,
	"step": 1350
	},
	{
	"epoch": 0.5743959304790165,
	"grad_norm": 0.28419825434684753,
	"learning_rate": 4.0514967765853815e-05,
	"loss": 1.279756450653076,
	"step": 1355
	},
	{
	"epoch": 0.5765154726579059,
	"grad_norm": 0.23340576887130737,
	"learning_rate": 4.017815859130461e-05,
	"loss": 1.2949078559875489,
	"step": 1360
	},
	{
	"epoch": 0.5786350148367952,
	"grad_norm": 0.2574683129787445,
	"learning_rate": 3.98418123505922e-05,
	"loss": 1.3117816925048829,
	"step": 1365
	},
	{
	"epoch": 0.5807545570156846,
	"grad_norm": 0.2542143762111664,
	"learning_rate": 3.950594489675763e-05,
	"loss": 1.3210847854614258,
	"step": 1370
	},
	{
	"epoch": 0.582874099194574,
	"grad_norm": 0.2623249590396881,
	"learning_rate": 3.917057206027534e-05,
	"loss": 1.312696647644043,
	"step": 1375
	},
	{
	"epoch": 0.5849936413734633,
	"grad_norm": 0.24822038412094116,
	"learning_rate": 3.883570964830692e-05,
	"loss": 1.2733299255371093,
	"step": 1380
	},
	{
	"epoch": 0.5871131835523526,
	"grad_norm": 0.28279241919517517,
	"learning_rate": 3.850137344395598e-05,
	"loss": 1.304712677001953,
	"step": 1385
	},
	{
	"epoch": 0.5892327257312421,
	"grad_norm": 0.30743899941444397,
	"learning_rate": 3.816757920552442e-05,
	"loss": 1.3324396133422851,
	"step": 1390
	},
	{
	"epoch": 0.5913522679101314,
	"grad_norm": 0.2944941222667694,
	"learning_rate": 3.783434266576958e-05,
	"loss": 1.308326530456543,
	"step": 1395
	},
	{
	"epoch": 0.5934718100890207,
	"grad_norm": 0.23847438395023346,
	"learning_rate": 3.750167953116272e-05,
	"loss": 1.3272653579711915,
	"step": 1400
	},
	{
	"epoch": 0.5955913522679102,
	"grad_norm": 0.28534844517707825,
	"learning_rate": 3.716960548114876e-05,
	"loss": 1.2522911071777343,
	"step": 1405
	},
	{
	"epoch": 0.5977108944467995,
	"grad_norm": 0.27947354316711426,
	"learning_rate": 3.68381361674072e-05,
	"loss": 1.322800064086914,
	"step": 1410
	},
	{
	"epoch": 0.5998304366256888,
	"grad_norm": 0.27901798486709595,
	"learning_rate": 3.6507287213114485e-05,
	"loss": 1.2732330322265626,
	"step": 1415
	},
	{
	"epoch": 0.6019499788045782,
	"grad_norm": 0.27319100499153137,
	"learning_rate": 3.617707421220758e-05,
	"loss": 1.2938933372497559,
	"step": 1420
	},
	{
	"epoch": 0.6040695209834676,
	"grad_norm": 0.2461337298154831,
	"learning_rate": 3.5847512728648985e-05,
	"loss": 1.3500941276550293,
	"step": 1425
	},
	{
	"epoch": 0.6061890631623569,
	"grad_norm": 0.26702332496643066,
	"learning_rate": 3.5518618295693176e-05,
	"loss": 1.3118473052978517,
	"step": 1430
	},
	{
	"epoch": 0.6083086053412463,
	"grad_norm": 0.2998215854167938,
	"learning_rate": 3.519040641515449e-05,
	"loss": 1.278294277191162,
	"step": 1435
	},
	{
	"epoch": 0.6104281475201356,
	"grad_norm": 0.29190096259117126,
	"learning_rate": 3.4862892556676395e-05,
	"loss": 1.3104103088378907,
	"step": 1440
	},
	{
	"epoch": 0.612547689699025,
	"grad_norm": 0.2719017267227173,
	"learning_rate": 3.4536092157002484e-05,
	"loss": 1.3252870559692382,
	"step": 1445
	},
	{
	"epoch": 0.6146672318779144,
	"grad_norm": 0.2539942264556885,
	"learning_rate": 3.421002061924876e-05,
	"loss": 1.3193702697753906,
	"step": 1450
	},
	{
	"epoch": 0.6167867740568037,
	"grad_norm": 0.26833513379096985,
	"learning_rate": 3.388469331217774e-05,
	"loss": 1.3424810409545898,
	"step": 1455
	},
	{
	"epoch": 0.6189063162356931,
	"grad_norm": 0.2887224555015564,
	"learning_rate": 3.356012556947405e-05,
	"loss": 1.3028847694396972,
	"step": 1460
	},
	{
	"epoch": 0.6210258584145825,
	"grad_norm": 0.28537675738334656,
	"learning_rate": 3.323633268902167e-05,
	"loss": 1.2761875152587892,
	"step": 1465
	},
	{
	"epoch": 0.6231454005934718,
	"grad_norm": 0.3113485872745514,
	"learning_rate": 3.291332993218292e-05,
	"loss": 1.3314291000366212,
	"step": 1470
	},
	{
	"epoch": 0.6252649427723612,
	"grad_norm": 0.23381048440933228,
	"learning_rate": 3.2591132523079174e-05,
	"loss": 1.3221202850341798,
	"step": 1475
	},
	{
	"epoch": 0.6273844849512505,
	"grad_norm": 0.2736342251300812,
	"learning_rate": 3.226975564787322e-05,
	"loss": 1.3375913619995117,
	"step": 1480
	},
	{
	"epoch": 0.6295040271301399,
	"grad_norm": 0.2420145571231842,
	"learning_rate": 3.194921445405359e-05,
	"loss": 1.299214744567871,
	"step": 1485
	},
	{
	"epoch": 0.6316235693090293,
	"grad_norm": 0.27538812160491943,
	"learning_rate": 3.162952404972052e-05,
	"loss": 1.344044303894043,
	"step": 1490
	},
	{
	"epoch": 0.6337431114879186,
	"grad_norm": 0.3411538302898407,
	"learning_rate": 3.131069950287393e-05,
	"loss": 1.2582563400268554,
	"step": 1495
	},
	{
	"epoch": 0.6358626536668079,
	"grad_norm": 0.28971365094184875,
	"learning_rate": 3.0992755840703195e-05,
	"loss": 1.2894617080688477,
	"step": 1500
	},
	{
	"epoch": 0.6379821958456974,
	"grad_norm": 0.2775018811225891,
	"learning_rate": 3.067570804887882e-05,
	"loss": 1.283580207824707,
	"step": 1505
	},
	{
	"epoch": 0.6401017380245867,
	"grad_norm": 0.2630194425582886,
	"learning_rate": 3.0359571070846216e-05,
	"loss": 1.2923145294189453,
	"step": 1510
	},
	{
	"epoch": 0.642221280203476,
	"grad_norm": 0.28774213790893555,
	"learning_rate": 3.004435980712129e-05,
	"loss": 1.2898404121398925,
	"step": 1515
	},
	{
	"epoch": 0.6443408223823655,
	"grad_norm": 0.28930774331092834,
	"learning_rate": 2.973008911458816e-05,
	"loss": 1.298238468170166,
	"step": 1520
	},
	{
	"epoch": 0.6464603645612548,
	"grad_norm": 0.27233803272247314,
	"learning_rate": 2.9416773805798918e-05,
	"loss": 1.253738784790039,
	"step": 1525
	},
	{
	"epoch": 0.6485799067401441,
	"grad_norm": 0.2648472785949707,
	"learning_rate": 2.910442864827549e-05,
	"loss": 1.2993535041809081,
	"step": 1530
	},
	{
	"epoch": 0.6506994489190335,
	"grad_norm": 0.2896256446838379,
	"learning_rate": 2.879306836381345e-05,
	"loss": 1.3196508407592773,
	"step": 1535
	},
	{
	"epoch": 0.6528189910979229,
	"grad_norm": 0.2507874071598053,
	"learning_rate": 2.8482707627788406e-05,
	"loss": 1.2362573623657227,
	"step": 1540
	},
	{
	"epoch": 0.6549385332768122,
	"grad_norm": 0.3429214060306549,
	"learning_rate": 2.8173361068464037e-05,
	"loss": 1.3394251823425294,
	"step": 1545
	},
	{
	"epoch": 0.6570580754557016,
	"grad_norm": 0.2614726722240448,
	"learning_rate": 2.786504326630276e-05,
	"loss": 1.2999580383300782,
	"step": 1550
	},
	{
	"epoch": 0.6591776176345909,
	"grad_norm": 0.2695824205875397,
	"learning_rate": 2.7557768753278524e-05,
	"loss": 1.2815800666809083,
	"step": 1555
	},
	{
	"epoch": 0.6612971598134803,
	"grad_norm": 0.27772441506385803,
	"learning_rate": 2.7251552012191762e-05,
	"loss": 1.2807681083679199,
	"step": 1560
	},
	{
	"epoch": 0.6634167019923697,
	"grad_norm": 0.26756763458251953,
	"learning_rate": 2.6946407475986864e-05,
	"loss": 1.3336042404174804,
	"step": 1565
	},
	{
	"epoch": 0.665536244171259,
	"grad_norm": 0.24804961681365967,
	"learning_rate": 2.6642349527071853e-05,
	"loss": 1.3213619232177733,
	"step": 1570
	},
	{
	"epoch": 0.6676557863501483,
	"grad_norm": 0.2715989053249359,
	"learning_rate": 2.6339392496640593e-05,
	"loss": 1.3158538818359375,
	"step": 1575
	},
	{
	"epoch": 0.6697753285290378,
	"grad_norm": 0.23300881683826447,
	"learning_rate": 2.603755066399718e-05,
	"loss": 1.326387882232666,
	"step": 1580
	},
	{
	"epoch": 0.6718948707079271,
	"grad_norm": 0.274876207113266,
	"learning_rate": 2.573683825588298e-05,
	"loss": 1.3208455085754394,
	"step": 1585
	},
	{
	"epoch": 0.6740144128868164,
	"grad_norm": 0.3012780547142029,
	"learning_rate": 2.5437269445806145e-05,
	"loss": 1.3030429840087892,
	"step": 1590
	},
	{
	"epoch": 0.6761339550657058,
	"grad_norm": 0.3057415187358856,
	"learning_rate": 2.513885835337344e-05,
	"loss": 1.2938409805297852,
	"step": 1595
	},
	{
	"epoch": 0.6782534972445952,
	"grad_norm": 0.3147355318069458,
	"learning_rate": 2.4841619043624807e-05,
	"loss": 1.2742310523986817,
	"step": 1600
	},
	{
	"epoch": 0.6803730394234845,
	"grad_norm": 0.28181126713752747,
	"learning_rate": 2.454556552637051e-05,
	"loss": 1.2890135765075683,
	"step": 1605
	},
	{
	"epoch": 0.6824925816023739,
	"grad_norm": 0.29839661717414856,
	"learning_rate": 2.425071175553067e-05,
	"loss": 1.3602640151977539,
	"step": 1610
	},
	{
	"epoch": 0.6846121237812632,
	"grad_norm": 0.27441298961639404,
	"learning_rate": 2.395707162847763e-05,
	"loss": 1.220123291015625,
	"step": 1615
	},
	{
	"epoch": 0.6867316659601526,
	"grad_norm": 0.2589101791381836,
	"learning_rate": 2.3664658985381e-05,
	"loss": 1.272838020324707,
	"step": 1620
	},
	{
	"epoch": 0.688851208139042,
	"grad_norm": 0.2562583088874817,
	"learning_rate": 2.3373487608555222e-05,
	"loss": 1.2698060989379882,
	"step": 1625
	},
	{
	"epoch": 0.6909707503179313,
	"grad_norm": 0.29641854763031006,
	"learning_rate": 2.308357122180999e-05,
	"loss": 1.2765254974365234,
	"step": 1630
	},
	{
	"epoch": 0.6930902924968206,
	"grad_norm": 0.26938116550445557,
	"learning_rate": 2.2794923489803504e-05,
	"loss": 1.2877408981323242,
	"step": 1635
	},
	{
	"epoch": 0.6952098346757101,
	"grad_norm": 0.2766175866127014,
	"learning_rate": 2.2507558017398263e-05,
	"loss": 1.2882837295532226,
	"step": 1640
	},
	{
	"epoch": 0.6973293768545994,
	"grad_norm": 0.3003990650177002,
	"learning_rate": 2.2221488349019903e-05,
	"loss": 1.2408750534057618,
	"step": 1645
	},
	{
	"epoch": 0.6994489190334887,
	"grad_norm": 0.297383189201355,
	"learning_rate": 2.1936727968018807e-05,
	"loss": 1.2793511390686034,
	"step": 1650
	},
	{
	"epoch": 0.7015684612123781,
	"grad_norm": 0.279326856136322,
	"learning_rate": 2.165329029603461e-05,
	"loss": 1.2509811401367188,
	"step": 1655
	},
	{
	"epoch": 0.7036880033912675,
	"grad_norm": 0.25482437014579773,
	"learning_rate": 2.1371188692363552e-05,
	"loss": 1.3125807762145996,
	"step": 1660
	},
	{
	"epoch": 0.7058075455701568,
	"grad_norm": 0.27757492661476135,
	"learning_rate": 2.1090436453328793e-05,
	"loss": 1.2483471870422362,
	"step": 1665
	},
	{
	"epoch": 0.7079270877490462,
	"grad_norm": 0.2942657172679901,
	"learning_rate": 2.0811046811653835e-05,
	"loss": 1.3040696144104005,
	"step": 1670
	},
	{
	"epoch": 0.7100466299279355,
	"grad_norm": 0.2963309586048126,
	"learning_rate": 2.053303293583867e-05,
	"loss": 1.335435390472412,
	"step": 1675
	},
	{
	"epoch": 0.712166172106825,
	"grad_norm": 0.2660171687602997,
	"learning_rate": 2.0256407929539202e-05,
	"loss": 1.3440280914306642,
	"step": 1680
	},
	{
	"epoch": 0.7142857142857143,
	"grad_norm": 0.23948104679584503,
	"learning_rate": 1.998118483094963e-05,
	"loss": 1.323198413848877,
	"step": 1685
	},
	{
	"epoch": 0.7164052564646036,
	"grad_norm": 0.2677803933620453,
	"learning_rate": 1.9707376612187873e-05,
	"loss": 1.3158408164978028,
	"step": 1690
	},
	{
	"epoch": 0.7185247986434931,
	"grad_norm": 0.24974216520786285,
	"learning_rate": 1.9434996178684167e-05,
	"loss": 1.1985395431518555,
	"step": 1695
	},
	{
	"epoch": 0.7206443408223824,
	"grad_norm": 0.2566191256046295,
	"learning_rate": 1.9164056368572846e-05,
	"loss": 1.2507144927978515,
	"step": 1700
	},
	{
	"epoch": 0.7227638830012717,
	"grad_norm": 0.26276081800460815,
	"learning_rate": 1.889456995208716e-05,
	"loss": 1.2876646041870117,
	"step": 1705
	},
	{
	"epoch": 0.7248834251801611,
	"grad_norm": 0.2921786904335022,
	"learning_rate": 1.8626549630957396e-05,
	"loss": 1.2243350982666015,
	"step": 1710
	},
	{
	"epoch": 0.7270029673590505,
	"grad_norm": 0.2412720024585724,
	"learning_rate": 1.836000803781228e-05,
	"loss": 1.2449832916259767,
	"step": 1715
	},
	{
	"epoch": 0.7291225095379398,
	"grad_norm": 0.2814440429210663,
	"learning_rate": 1.8094957735583463e-05,
	"loss": 1.3065194129943847,
	"step": 1720
	},
	{
	"epoch": 0.7312420517168292,
	"grad_norm": 0.33798617124557495,
	"learning_rate": 1.783141121691341e-05,
	"loss": 1.305231475830078,
	"step": 1725
	},
	{
	"epoch": 0.7333615938957185,
	"grad_norm": 0.24688974022865295,
	"learning_rate": 1.7569380903566674e-05,
	"loss": 1.250468635559082,
	"step": 1730
	},
	{
	"epoch": 0.7354811360746079,
	"grad_norm": 0.2482006996870041,
	"learning_rate": 1.7308879145844286e-05,
	"loss": 1.3213464736938476,
	"step": 1735
	},
	{
	"epoch": 0.7376006782534973,
	"grad_norm": 0.28195133805274963,
	"learning_rate": 1.7049918222001742e-05,
	"loss": 1.2562795639038087,
	"step": 1740
	},
	{
	"epoch": 0.7397202204323866,
	"grad_norm": 0.29843616485595703,
	"learning_rate": 1.679251033767023e-05,
	"loss": 1.2251798629760742,
	"step": 1745
	},
	{
	"epoch": 0.7418397626112759,
	"grad_norm": 0.2938304543495178,
	"learning_rate": 1.6536667625281433e-05,
	"loss": 1.3671794891357423,
	"step": 1750
	},
	{
	"epoch": 0.7439593047901654,
	"grad_norm": 0.25978606939315796,
	"learning_rate": 1.6282402143495568e-05,
	"loss": 1.2856554985046387,
	"step": 1755
	},
	{
	"epoch": 0.7460788469690547,
	"grad_norm": 0.2800081968307495,
	"learning_rate": 1.6029725876633085e-05,
	"loss": 1.2771218299865723,
	"step": 1760
	},
	{
	"epoch": 0.748198389147944,
	"grad_norm": 0.28945568203926086,
	"learning_rate": 1.5778650734109878e-05,
	"loss": 1.3087704658508301,
	"step": 1765
	},
	{
	"epoch": 0.7503179313268334,
	"grad_norm": 0.28071361780166626,
	"learning_rate": 1.552918854987577e-05,
	"loss": 1.3361966133117675,
	"step": 1770
	},
	{
	"epoch": 0.7524374735057228,
	"grad_norm": 0.29401445388793945,
	"learning_rate": 1.5281351081856974e-05,
	"loss": 1.2625712394714355,
	"step": 1775
	},
	{
	"epoch": 0.7545570156846121,
	"grad_norm": 0.29227736592292786,
	"learning_rate": 1.5035150011401727e-05,
	"loss": 1.315434455871582,
	"step": 1780
	},
	{
	"epoch": 0.7566765578635015,
	"grad_norm": 0.2883286774158478,
	"learning_rate": 1.4790596942729784e-05,
	"loss": 1.3468461990356446,
	"step": 1785
	},
	{
	"epoch": 0.7587961000423908,
	"grad_norm": 0.2792353928089142,
	"learning_rate": 1.454770340238546e-05,
	"loss": 1.2823726654052734,
	"step": 1790
	},
	{
	"epoch": 0.7609156422212802,
	"grad_norm": 0.3239692151546478,
	"learning_rate": 1.4306480838694402e-05,
	"loss": 1.2746034622192384,
	"step": 1795
	},
	{
	"epoch": 0.7630351844001696,
	"grad_norm": 0.2689991891384125,
	"learning_rate": 1.406694062122389e-05,
	"loss": 1.2611713409423828,
	"step": 1800
	},
	{
	"epoch": 0.7651547265790589,
	"grad_norm": 0.3099757134914398,
	"learning_rate": 1.3829094040247021e-05,
	"loss": 1.2869087219238282,
	"step": 1805
	},
	{
	"epoch": 0.7672742687579482,
	"grad_norm": 0.3021414875984192,
	"learning_rate": 1.3592952306210588e-05,
	"loss": 1.2934808731079102,
	"step": 1810
	},
	{
	"epoch": 0.7693938109368377,
	"grad_norm": 0.26889753341674805,
	"learning_rate": 1.335852654920663e-05,
	"loss": 1.292646884918213,
	"step": 1815
	},
	{
	"epoch": 0.771513353115727,
	"grad_norm": 0.27630671858787537,
	"learning_rate": 1.3125827818447862e-05,
	"loss": 1.2981810569763184,
	"step": 1820
	},
	{
	"epoch": 0.7736328952946163,
	"grad_norm": 0.30495333671569824,
	"learning_rate": 1.2894867081746942e-05,
	"loss": 1.3083015441894532,
	"step": 1825
	},
	{
	"epoch": 0.7757524374735058,
	"grad_norm": 0.3265323042869568,
	"learning_rate": 1.2665655224999434e-05,
	"loss": 1.3190043449401856,
	"step": 1830
	},
	{
	"epoch": 0.7778719796523951,
	"grad_norm": 0.3173474073410034,
	"learning_rate": 1.2438203051670772e-05,
	"loss": 1.2658998489379882,
	"step": 1835
	},
	{
	"epoch": 0.7799915218312844,
	"grad_norm": 0.2906636595726013,
	"learning_rate": 1.2212521282287092e-05,
	"loss": 1.2488012313842773,
	"step": 1840
	},
	{
	"epoch": 0.7821110640101738,
	"grad_norm": 0.3078494071960449,
	"learning_rate": 1.1988620553929863e-05,
	"loss": 1.2655397415161134,
	"step": 1845
	},
	{
	"epoch": 0.7842306061890632,
	"grad_norm": 0.30600544810295105,
	"learning_rate": 1.1766511419734577e-05,
	"loss": 1.2737188339233398,
	"step": 1850
	},
	{
	"epoch": 0.7863501483679525,
	"grad_norm": 0.32284632325172424,
	"learning_rate": 1.1546204348393363e-05,
	"loss": 1.2989439010620116,
	"step": 1855
	},
	{
	"epoch": 0.7884696905468419,
	"grad_norm": 0.2343946099281311,
	"learning_rate": 1.1327709723661517e-05,
	"loss": 1.2967771530151366,
	"step": 1860
	},
	{
	"epoch": 0.7905892327257312,
	"grad_norm": 0.26664406061172485,
	"learning_rate": 1.1111037843868094e-05,
	"loss": 1.3008729934692382,
	"step": 1865
	},
	{
	"epoch": 0.7927087749046206,
	"grad_norm": 0.24961388111114502,
	"learning_rate": 1.0896198921430579e-05,
	"loss": 1.2569337844848634,
	"step": 1870
	},
	{
	"epoch": 0.79482831708351,
	"grad_norm": 0.28111404180526733,
	"learning_rate": 1.0683203082373444e-05,
	"loss": 1.2984585762023926,
	"step": 1875
	},
	{
	"epoch": 0.7969478592623993,
	"grad_norm": 0.2607182562351227,
	"learning_rate": 1.047206036585095e-05,
	"loss": 1.3258838653564453,
	"step": 1880
	},
	{
	"epoch": 0.7990674014412886,
	"grad_norm": 0.2674267888069153,
	"learning_rate": 1.0262780723673931e-05,
	"loss": 1.2292760848999023,
	"step": 1885
	},
	{
	"epoch": 0.8011869436201781,
	"grad_norm": 0.37508147954940796,
	"learning_rate": 1.0055374019840785e-05,
	"loss": 1.2805218696594238,
	"step": 1890
	},
	{
	"epoch": 0.8033064857990674,
	"grad_norm": 0.36540335416793823,
	"learning_rate": 9.849850030072494e-06,
	"loss": 1.2833273887634278,
	"step": 1895
	},
	{
	"epoch": 0.8054260279779568,
	"grad_norm": 0.24529576301574707,
	"learning_rate": 9.646218441351867e-06,
	"loss": 1.2620270729064942,
	"step": 1900
	},
	{
	"epoch": 0.8075455701568461,
	"grad_norm": 0.26133260130882263,
	"learning_rate": 9.444488851467042e-06,
	"loss": 1.255943489074707,
	"step": 1905
	},
	{
	"epoch": 0.8096651123357355,
	"grad_norm": 0.30230292677879333,
	"learning_rate": 9.24467076855901e-06,
	"loss": 1.275054359436035,
	"step": 1910
	},
	{
	"epoch": 0.8117846545146249,
	"grad_norm": 0.25680676102638245,
	"learning_rate": 9.046773610673515e-06,
	"loss": 1.2673065185546875,
	"step": 1915
	},
	{
	"epoch": 0.8139041966935142,
	"grad_norm": 0.34163177013397217,
	"learning_rate": 8.850806705317183e-06,
	"loss": 1.260227870941162,
	"step": 1920
	},
	{
	"epoch": 0.8160237388724035,
	"grad_norm": 0.26306524872779846,
	"learning_rate": 8.656779289017814e-06,
	"loss": 1.2553665161132812,
	"step": 1925
	},
	{
	"epoch": 0.818143281051293,
	"grad_norm": 0.29241958260536194,
	"learning_rate": 8.464700506889078e-06,
	"loss": 1.3109891891479493,
	"step": 1930
	},
	{
	"epoch": 0.8202628232301823,
	"grad_norm": 0.30750298500061035,
	"learning_rate": 8.274579412199518e-06,
	"loss": 1.2717646598815917,
	"step": 1935
	},
	{
	"epoch": 0.8223823654090716,
	"grad_norm": 0.23530098795890808,
	"learning_rate": 8.086424965945755e-06,
	"loss": 1.2389122009277345,
	"step": 1940
	},
	{
	"epoch": 0.824501907587961,
	"grad_norm": 0.2277512401342392,
	"learning_rate": 7.90024603643017e-06,
	"loss": 1.2927206039428711,
	"step": 1945
	},
	{
	"epoch": 0.8266214497668504,
	"grad_norm": 0.32569536566734314,
	"learning_rate": 7.716051398842961e-06,
	"loss": 1.2816347122192382,
	"step": 1950
	},
	{
	"epoch": 0.8287409919457397,
	"grad_norm": 0.31003010272979736,
	"learning_rate": 7.5338497348484525e-06,
	"loss": 1.2516345977783203,
	"step": 1955
	},
	{
	"epoch": 0.8308605341246291,
	"grad_norm": 0.2880670130252838,
	"learning_rate": 7.353649632175957e-06,
	"loss": 1.270726203918457,
	"step": 1960
	},
	{
	"epoch": 0.8329800763035184,
	"grad_norm": 0.28343909978866577,
	"learning_rate": 7.175459584214989e-06,
	"loss": 1.3056310653686523,
	"step": 1965
	},
	{
	"epoch": 0.8350996184824078,
	"grad_norm": 0.2951664328575134,
	"learning_rate": 6.999287989614972e-06,
	"loss": 1.2573291778564453,
	"step": 1970
	},
	{
	"epoch": 0.8372191606612972,
	"grad_norm": 0.2489994466304779,
	"learning_rate": 6.825143151889357e-06,
	"loss": 1.298172378540039,
	"step": 1975
	},
	{
	"epoch": 0.8393387028401865,
	"grad_norm": 0.2669251561164856,
	"learning_rate": 6.65303327902424e-06,
	"loss": 1.292896556854248,
	"step": 1980
	},
	{
	"epoch": 0.8414582450190758,
	"grad_norm": 0.33183568716049194,
	"learning_rate": 6.482966483091546e-06,
	"loss": 1.2437992095947266,
	"step": 1985
	},
	{
	"epoch": 0.8435777871979653,
	"grad_norm": 0.31719595193862915,
	"learning_rate": 6.314950779866635e-06,
	"loss": 1.2581483840942382,
	"step": 1990
	},
	{
	"epoch": 0.8456973293768546,
	"grad_norm": 0.3390994071960449,
	"learning_rate": 6.148994088450482e-06,
	"loss": 1.27978458404541,
	"step": 1995
	},
	{
	"epoch": 0.8478168715557439,
	"grad_norm": 0.2966649532318115,
	"learning_rate": 5.985104230896516e-06,
	"loss": 1.2705857276916503,
	"step": 2000
	},
	{
	"epoch": 0.8499364137346334,
	"grad_norm": 0.24032042920589447,
	"learning_rate": 5.823288931841814e-06,
	"loss": 1.3069118499755858,
	"step": 2005
	},
	{
	"epoch": 0.8520559559135227,
	"grad_norm": 0.2740707993507385,
	"learning_rate": 5.663555818143118e-06,
	"loss": 1.232090377807617,
	"step": 2010
	},
	{
	"epoch": 0.854175498092412,
	"grad_norm": 0.25971028208732605,
	"learning_rate": 5.505912418517295e-06,
	"loss": 1.2706013679504395,
	"step": 2015
	},
	{
	"epoch": 0.8562950402713014,
	"grad_norm": 0.27701419591903687,
	"learning_rate": 5.350366163186521e-06,
	"loss": 1.260000991821289,
	"step": 2020
	},
	{
	"epoch": 0.8584145824501908,
	"grad_norm": 0.2975700795650482,
	"learning_rate": 5.196924383528057e-06,
	"loss": 1.3433919906616212,
	"step": 2025
	},
	{
	"epoch": 0.8605341246290801,
	"grad_norm": 0.2869032323360443,
	"learning_rate": 5.045594311728707e-06,
	"loss": 1.2693572044372559,
	"step": 2030
	},
	{
	"epoch": 0.8626536668079695,
	"grad_norm": 0.29436832666397095,
	"learning_rate": 4.896383080443934e-06,
	"loss": 1.2822356224060059,
	"step": 2035
	},
	{
	"epoch": 0.8647732089868588,
	"grad_norm": 0.31956785917282104,
	"learning_rate": 4.74929772246166e-06,
	"loss": 1.2516891479492187,
	"step": 2040
	},
	{
	"epoch": 0.8668927511657482,
	"grad_norm": 0.3107931613922119,
	"learning_rate": 4.60434517037085e-06,
	"loss": 1.298238754272461,
	"step": 2045
	},
	{
	"epoch": 0.8690122933446376,
	"grad_norm": 0.26913464069366455,
	"learning_rate": 4.4615322562346625e-06,
	"loss": 1.3168695449829102,
	"step": 2050
	},
	{
	"epoch": 0.8711318355235269,
	"grad_norm": 0.2527681887149811,
	"learning_rate": 4.320865711268518e-06,
	"loss": 1.239467239379883,
	"step": 2055
	},
	{
	"epoch": 0.8732513777024162,
	"grad_norm": 0.24979636073112488,
	"learning_rate": 4.182352165522807e-06,
	"loss": 1.28905029296875,
	"step": 2060
	},
	{
	"epoch": 0.8753709198813057,
	"grad_norm": 0.2744317948818207,
	"learning_rate": 4.045998147570379e-06,
	"loss": 1.2322834014892579,
	"step": 2065
	},
	{
	"epoch": 0.877490462060195,
	"grad_norm": 0.28092294931411743,
	"learning_rate": 3.911810084198825e-06,
	"loss": 1.2464519500732423,
	"step": 2070
	},
	{
	"epoch": 0.8796100042390843,
	"grad_norm": 0.28864502906799316,
	"learning_rate": 3.779794300107625e-06,
	"loss": 1.2705391883850097,
	"step": 2075
	},
	{
	"epoch": 0.8817295464179737,
	"grad_norm": 0.3669198453426361,
	"learning_rate": 3.6499570176099785e-06,
	"loss": 1.2647643089294434,
	"step": 2080
	},
	{
	"epoch": 0.8838490885968631,
	"grad_norm": 0.3103444278240204,
	"learning_rate": 3.5223043563395288e-06,
	"loss": 1.304600143432617,
	"step": 2085
	},
	{
	"epoch": 0.8859686307757524,
	"grad_norm": 0.27703604102134705,
	"learning_rate": 3.3968423329619915e-06,
	"loss": 1.2277642250061036,
	"step": 2090
	},
	{
	"epoch": 0.8880881729546418,
	"grad_norm": 0.2625238001346588,
	"learning_rate": 3.273576860891503e-06,
	"loss": 1.2695680618286134,
	"step": 2095
	},
	{
	"epoch": 0.8902077151335311,
	"grad_norm": 0.3185974061489105,
	"learning_rate": 3.1525137500119207e-06,
	"loss": 1.2830419540405273,
	"step": 2100
	},
	{
	"epoch": 0.8923272573124205,
	"grad_norm": 0.29703983664512634,
	"learning_rate": 3.033658706402992e-06,
	"loss": 1.2871177673339844,
	"step": 2105
	},
	{
	"epoch": 0.8944467994913099,
	"grad_norm": 0.3013726770877838,
	"learning_rate": 2.917017332071431e-06,
	"loss": 1.2571928024291992,
	"step": 2110
	},
	{
	"epoch": 0.8965663416701992,
	"grad_norm": 0.3186560571193695,
	"learning_rate": 2.80259512468683e-06,
	"loss": 1.2625058174133301,
	"step": 2115
	},
	{
	"epoch": 0.8986858838490887,
	"grad_norm": 0.30863144993782043,
	"learning_rate": 2.6903974773225702e-06,
	"loss": 1.2434380531311036,
	"step": 2120
	},
	{
	"epoch": 0.900805426027978,
	"grad_norm": 0.24899743497371674,
	"learning_rate": 2.5804296782016347e-06,
	"loss": 1.2562905311584474,
	"step": 2125
	},
	{
	"epoch": 0.9029249682068673,
	"grad_norm": 0.263972669839859,
	"learning_rate": 2.472696910447331e-06,
	"loss": 1.262008285522461,
	"step": 2130
	},
	{
	"epoch": 0.9050445103857567,
	"grad_norm": 0.3364119529724121,
	"learning_rate": 2.367204251838995e-06,
	"loss": 1.283153247833252,
	"step": 2135
	},
	{
	"epoch": 0.907164052564646,
	"grad_norm": 0.3495783805847168,
	"learning_rate": 2.2639566745727205e-06,
	"loss": 1.2852234840393066,
	"step": 2140
	},
	{
	"epoch": 0.9092835947435354,
	"grad_norm": 0.2861587107181549,
	"learning_rate": 2.1629590450269076e-06,
	"loss": 1.253425693511963,
	"step": 2145
	},
	{
	"epoch": 0.9114031369224248,
	"grad_norm": 0.223190575838089,
	"learning_rate": 2.0642161235329694e-06,
	"loss": 1.3334663391113282,
	"step": 2150
	},
	{
	"epoch": 0.9135226791013141,
	"grad_norm": 0.23863250017166138,
	"learning_rate": 1.9677325641509427e-06,
	"loss": 1.2898451805114746,
	"step": 2155
	},
	{
	"epoch": 0.9156422212802034,
	"grad_norm": 0.2902802526950836,
	"learning_rate": 1.873512914450104e-06,
	"loss": 1.276516819000244,
	"step": 2160
	},
	{
	"epoch": 0.9177617634590929,
	"grad_norm": 0.27276214957237244,
	"learning_rate": 1.7815616152946523e-06,
	"loss": 1.2869909286499024,
	"step": 2165
	},
	{
	"epoch": 0.9198813056379822,
	"grad_norm": 0.25340595841407776,
	"learning_rate": 1.6918830006344199e-06,
	"loss": 1.249582576751709,
	"step": 2170
	},
	{
	"epoch": 0.9220008478168715,
	"grad_norm": 0.28898558020591736,
	"learning_rate": 1.6044812973005308e-06,
	"loss": 1.2465517044067382,
	"step": 2175
	},
	{
	"epoch": 0.924120389995761,
	"grad_norm": 0.2674211263656616,
	"learning_rate": 1.5193606248062486e-06,
	"loss": 1.2785299301147461,
	"step": 2180
	},
	{
	"epoch": 0.9262399321746503,
	"grad_norm": 0.2752663195133209,
	"learning_rate": 1.4365249951527658e-06,
	"loss": 1.2554035186767578,
	"step": 2185
	},
	{
	"epoch": 0.9283594743535396,
	"grad_norm": 0.27620890736579895,
	"learning_rate": 1.3559783126401326e-06,
	"loss": 1.2389708518981934,
	"step": 2190
	},
	{
	"epoch": 0.930479016532429,
	"grad_norm": 0.2857415974140167,
	"learning_rate": 1.27772437368322e-06,
	"loss": 1.241176700592041,
	"step": 2195
	},
	{
	"epoch": 0.9325985587113184,
	"grad_norm": 0.3106928765773773,
	"learning_rate": 1.2017668666327753e-06,
	"loss": 1.327683448791504,
	"step": 2200
	},
	{
	"epoch": 0.9347181008902077,
	"grad_norm": 0.28668051958084106,
	"learning_rate": 1.128109371601621e-06,
	"loss": 1.2413459777832032,
	"step": 2205
	},
	{
	"epoch": 0.9368376430690971,
	"grad_norm": 0.297909677028656,
	"learning_rate": 1.0567553602958413e-06,
	"loss": 1.3241584777832032,
	"step": 2210
	},
	{
	"epoch": 0.9389571852479864,
	"grad_norm": 0.2803064286708832,
	"learning_rate": 9.877081958512224e-07,
	"loss": 1.2748098373413086,
	"step": 2215
	},
	{
	"epoch": 0.9410767274268758,
	"grad_norm": 0.3201870918273926,
	"learning_rate": 9.209711326746918e-07,
	"loss": 1.3116162300109864,
	"step": 2220
	},
	{
	"epoch": 0.9431962696057652,
	"grad_norm": 0.26600632071495056,
	"learning_rate": 8.565473162909509e-07,
	"loss": 1.2817995071411132,
	"step": 2225
	},
	{
	"epoch": 0.9453158117846545,
	"grad_norm": 0.3192591369152069,
	"learning_rate": 7.944397831941952e-07,
	"loss": 1.23384370803833,
	"step": 2230
	},
	{
	"epoch": 0.9474353539635438,
	"grad_norm": 0.27128636837005615,
	"learning_rate": 7.346514607050159e-07,
	"loss": 1.2136476516723633,
	"step": 2235
	},
	{
	"epoch": 0.9495548961424333,
	"grad_norm": 0.2583428919315338,
	"learning_rate": 6.771851668324225e-07,
	"loss": 1.2903619766235352,
	"step": 2240
	},
	{
	"epoch": 0.9516744383213226,
	"grad_norm": 0.24806281924247742,
	"learning_rate": 6.220436101410044e-07,
	"loss": 1.2967029571533204,
	"step": 2245
	},
	{
	"epoch": 0.9537939805002119,
	"grad_norm": 0.2721601724624634,
	"learning_rate": 5.692293896232936e-07,
	"loss": 1.2712093353271485,
	"step": 2250
	},
	{
	"epoch": 0.9559135226791013,
	"grad_norm": 0.3129066228866577,
	"learning_rate": 5.18744994577236e-07,
	"loss": 1.3387887954711915,
	"step": 2255
	},
	{
	"epoch": 0.9580330648579907,
	"grad_norm": 0.2757937014102936,
	"learning_rate": 4.705928044888952e-07,
	"loss": 1.279701042175293,
	"step": 2260
	},
	{
	"epoch": 0.96015260703688,
	"grad_norm": 0.2870910167694092,
	"learning_rate": 4.247750889202762e-07,
	"loss": 1.290435218811035,
	"step": 2265
	},
	{
	"epoch": 0.9622721492157694,
	"grad_norm": 0.27160516381263733,
	"learning_rate": 3.8129400740237186e-07,
	"loss": 1.263418197631836,
	"step": 2270
	},
	{
	"epoch": 0.9643916913946587,
	"grad_norm": 0.2892550230026245,
	"learning_rate": 3.4015160933335564e-07,
	"loss": 1.3052814483642579,
	"step": 2275
	},
	{
	"epoch": 0.9665112335735481,
	"grad_norm": 0.3261653482913971,
	"learning_rate": 3.013498338820031e-07,
	"loss": 1.2655174255371093,
	"step": 2280
	},
	{
	"epoch": 0.9686307757524375,
	"grad_norm": 0.2924319803714752,
	"learning_rate": 2.648905098962928e-07,
	"loss": 1.3226234436035156,
	"step": 2285
	},
	{
	"epoch": 0.9707503179313268,
	"grad_norm": 0.29761645197868347,
	"learning_rate": 2.3077535581719766e-07,
	"loss": 1.3013673782348634,
	"step": 2290
	},
	{
	"epoch": 0.9728698601102161,
	"grad_norm": 0.2872036397457123,
	"learning_rate": 1.9900597959770507e-07,
	"loss": 1.2685457229614259,
	"step": 2295
	},
	{
	"epoch": 0.9749894022891056,
	"grad_norm": 0.30427616834640503,
	"learning_rate": 1.6958387862701098e-07,
	"loss": 1.2520437240600586,
	"step": 2300
	},
	{
	"epoch": 0.9771089444679949,
	"grad_norm": 0.2846372425556183,
	"learning_rate": 1.4251043965994304e-07,
	"loss": 1.2421930313110352,
	"step": 2305
	},
	{
	"epoch": 0.9792284866468842,
	"grad_norm": 0.2917633056640625,
	"learning_rate": 1.1778693875162949e-07,
	"loss": 1.2915799140930175,
	"step": 2310
	},
	{
	"epoch": 0.9813480288257737,
	"grad_norm": 0.30620837211608887,
	"learning_rate": 9.541454119731397e-08,
	"loss": 1.2920777320861816,
	"step": 2315
	},
	{
	"epoch": 0.983467571004663,
	"grad_norm": 0.2834172248840332,
	"learning_rate": 7.539430147745496e-08,
	"loss": 1.268046760559082,
	"step": 2320
	},
	{
	"epoch": 0.9855871131835523,
	"grad_norm": 0.29660966992378235,
	"learning_rate": 5.772716320802119e-08,
	"loss": 1.279447078704834,
	"step": 2325
	},
	{
	"epoch": 0.9877066553624417,
	"grad_norm": 0.2827729880809784,
	"learning_rate": 4.241395909601598e-08,
	"loss": 1.2770563125610352,
	"step": 2330
	},
	{
	"epoch": 0.989826197541331,
	"grad_norm": 0.29589614272117615,
	"learning_rate": 2.945541090021986e-08,
	"loss": 1.3152960777282714,
	"step": 2335
	},
	{
	"epoch": 0.9919457397202205,
	"grad_norm": 0.3299049437046051,
	"learning_rate": 1.8852129397189942e-08,
	"loss": 1.2531587600708007,
	"step": 2340
	},
	{
	"epoch": 0.9940652818991098,
	"grad_norm": 0.26200127601623535,
	"learning_rate": 1.060461435246074e-08,
	"loss": 1.320154571533203,
	"step": 2345
	},
	{
	"epoch": 0.9961848240779991,
	"grad_norm": 0.3367665410041809,
	"learning_rate": 4.71325449697968e-09,
	"loss": 1.2206826210021973,
	"step": 2350
	},
	{
	"epoch": 0.9983043662568886,
	"grad_norm": 0.3022238612174988,
	"learning_rate": 1.178327508799537e-09,
	"loss": 1.2553547859191894,
	"step": 2355
	}
	],
	"logging_steps": 5,
	"max_steps": 2359,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 1,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 2.744030824717353e+17,
	"train_batch_size": 1,
	"trial_name": null,
	"trial_params": null
	}