Instructions to use simplex-ai-inc/LiteResearcher-4B-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use simplex-ai-inc/LiteResearcher-4B-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="simplex-ai-inc/LiteResearcher-4B-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("simplex-ai-inc/LiteResearcher-4B-SFT")
model = AutoModelForCausalLM.from_pretrained("simplex-ai-inc/LiteResearcher-4B-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use simplex-ai-inc/LiteResearcher-4B-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "simplex-ai-inc/LiteResearcher-4B-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "simplex-ai-inc/LiteResearcher-4B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/simplex-ai-inc/LiteResearcher-4B-SFT

SGLang

How to use simplex-ai-inc/LiteResearcher-4B-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "simplex-ai-inc/LiteResearcher-4B-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "simplex-ai-inc/LiteResearcher-4B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "simplex-ai-inc/LiteResearcher-4B-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "simplex-ai-inc/LiteResearcher-4B-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use simplex-ai-inc/LiteResearcher-4B-SFT with Docker Model Runner:
```
docker model run hf.co/simplex-ai-inc/LiteResearcher-4B-SFT
```

LiteResearcher-4B-SFT / trainer_state.json

wanlilll

Upload LiteResearcher-4B SFT cold-start checkpoint (Qwen3-4B, 1 epoch on 68.2k SFT data, max-len 64k, bs 128, step 533)

2c4087a verified about 1 month ago

Raw

History Blame Contribute Delete

93.3 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 1.0,
	"eval_steps": 500,
	"global_step": 533,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.001876172607879925,
	"grad_norm": 13.30275821685791,
	"learning_rate": 0.0,
	"loss": 1.1937,
	"step": 1
	},
	{
	"epoch": 0.00375234521575985,
	"grad_norm": 13.791933059692383,
	"learning_rate": 3.7037037037037036e-07,
	"loss": 1.1867,
	"step": 2
	},
	{
	"epoch": 0.005628517823639775,
	"grad_norm": 13.11504077911377,
	"learning_rate": 7.407407407407407e-07,
	"loss": 1.1638,
	"step": 3
	},
	{
	"epoch": 0.0075046904315197,
	"grad_norm": 13.519221305847168,
	"learning_rate": 1.111111111111111e-06,
	"loss": 1.1875,
	"step": 4
	},
	{
	"epoch": 0.009380863039399626,
	"grad_norm": 13.415589332580566,
	"learning_rate": 1.4814814814814815e-06,
	"loss": 1.2118,
	"step": 5
	},
	{
	"epoch": 0.01125703564727955,
	"grad_norm": 12.372065544128418,
	"learning_rate": 1.8518518518518519e-06,
	"loss": 1.1666,
	"step": 6
	},
	{
	"epoch": 0.013133208255159476,
	"grad_norm": 10.053466796875,
	"learning_rate": 2.222222222222222e-06,
	"loss": 1.0824,
	"step": 7
	},
	{
	"epoch": 0.0150093808630394,
	"grad_norm": 7.384225368499756,
	"learning_rate": 2.5925925925925925e-06,
	"loss": 1.0465,
	"step": 8
	},
	{
	"epoch": 0.016885553470919325,
	"grad_norm": 5.936990737915039,
	"learning_rate": 2.962962962962963e-06,
	"loss": 0.9748,
	"step": 9
	},
	{
	"epoch": 0.01876172607879925,
	"grad_norm": 5.118947982788086,
	"learning_rate": 3.3333333333333333e-06,
	"loss": 0.9498,
	"step": 10
	},
	{
	"epoch": 0.020637898686679174,
	"grad_norm": 2.2925024032592773,
	"learning_rate": 3.7037037037037037e-06,
	"loss": 0.8612,
	"step": 11
	},
	{
	"epoch": 0.0225140712945591,
	"grad_norm": 2.1615967750549316,
	"learning_rate": 4.074074074074074e-06,
	"loss": 0.8523,
	"step": 12
	},
	{
	"epoch": 0.024390243902439025,
	"grad_norm": 1.967605710029602,
	"learning_rate": 4.444444444444444e-06,
	"loss": 0.852,
	"step": 13
	},
	{
	"epoch": 0.02626641651031895,
	"grad_norm": 2.0044472217559814,
	"learning_rate": 4.814814814814815e-06,
	"loss": 0.8223,
	"step": 14
	},
	{
	"epoch": 0.028142589118198873,
	"grad_norm": 1.8968573808670044,
	"learning_rate": 5.185185185185185e-06,
	"loss": 0.784,
	"step": 15
	},
	{
	"epoch": 0.0300187617260788,
	"grad_norm": 2.0519533157348633,
	"learning_rate": 5.555555555555557e-06,
	"loss": 0.7962,
	"step": 16
	},
	{
	"epoch": 0.03189493433395872,
	"grad_norm": 1.8372418880462646,
	"learning_rate": 5.925925925925926e-06,
	"loss": 0.7928,
	"step": 17
	},
	{
	"epoch": 0.03377110694183865,
	"grad_norm": 1.6606271266937256,
	"learning_rate": 6.296296296296297e-06,
	"loss": 0.7741,
	"step": 18
	},
	{
	"epoch": 0.03564727954971857,
	"grad_norm": 1.4279577732086182,
	"learning_rate": 6.666666666666667e-06,
	"loss": 0.7726,
	"step": 19
	},
	{
	"epoch": 0.0375234521575985,
	"grad_norm": 1.1227573156356812,
	"learning_rate": 7.0370370370370375e-06,
	"loss": 0.7226,
	"step": 20
	},
	{
	"epoch": 0.039399624765478425,
	"grad_norm": 1.1973068714141846,
	"learning_rate": 7.4074074074074075e-06,
	"loss": 0.7286,
	"step": 21
	},
	{
	"epoch": 0.04127579737335835,
	"grad_norm": 1.053694248199463,
	"learning_rate": 7.77777777777778e-06,
	"loss": 0.7125,
	"step": 22
	},
	{
	"epoch": 0.043151969981238276,
	"grad_norm": 0.8389816284179688,
	"learning_rate": 8.148148148148148e-06,
	"loss": 0.7081,
	"step": 23
	},
	{
	"epoch": 0.0450281425891182,
	"grad_norm": 0.7549389004707336,
	"learning_rate": 8.518518518518519e-06,
	"loss": 0.6835,
	"step": 24
	},
	{
	"epoch": 0.04690431519699812,
	"grad_norm": 0.7767587304115295,
	"learning_rate": 8.888888888888888e-06,
	"loss": 0.6772,
	"step": 25
	},
	{
	"epoch": 0.04878048780487805,
	"grad_norm": 0.7495452165603638,
	"learning_rate": 9.25925925925926e-06,
	"loss": 0.6365,
	"step": 26
	},
	{
	"epoch": 0.05065666041275797,
	"grad_norm": 0.6553788185119629,
	"learning_rate": 9.62962962962963e-06,
	"loss": 0.6428,
	"step": 27
	},
	{
	"epoch": 0.0525328330206379,
	"grad_norm": 0.6172502636909485,
	"learning_rate": 1e-05,
	"loss": 0.6557,
	"step": 28
	},
	{
	"epoch": 0.054409005628517824,
	"grad_norm": 0.6201605796813965,
	"learning_rate": 1.037037037037037e-05,
	"loss": 0.6371,
	"step": 29
	},
	{
	"epoch": 0.05628517823639775,
	"grad_norm": 0.6146591901779175,
	"learning_rate": 1.0740740740740742e-05,
	"loss": 0.6211,
	"step": 30
	},
	{
	"epoch": 0.058161350844277676,
	"grad_norm": 0.6061451435089111,
	"learning_rate": 1.1111111111111113e-05,
	"loss": 0.6179,
	"step": 31
	},
	{
	"epoch": 0.0600375234521576,
	"grad_norm": 0.5912018418312073,
	"learning_rate": 1.1481481481481482e-05,
	"loss": 0.611,
	"step": 32
	},
	{
	"epoch": 0.06191369606003752,
	"grad_norm": 0.5214031934738159,
	"learning_rate": 1.1851851851851852e-05,
	"loss": 0.6025,
	"step": 33
	},
	{
	"epoch": 0.06378986866791744,
	"grad_norm": 0.5357693433761597,
	"learning_rate": 1.2222222222222224e-05,
	"loss": 0.6057,
	"step": 34
	},
	{
	"epoch": 0.06566604127579738,
	"grad_norm": 0.5372332334518433,
	"learning_rate": 1.2592592592592593e-05,
	"loss": 0.6085,
	"step": 35
	},
	{
	"epoch": 0.0675422138836773,
	"grad_norm": 0.5066092014312744,
	"learning_rate": 1.2962962962962964e-05,
	"loss": 0.5833,
	"step": 36
	},
	{
	"epoch": 0.06941838649155722,
	"grad_norm": 0.5455030798912048,
	"learning_rate": 1.3333333333333333e-05,
	"loss": 0.5954,
	"step": 37
	},
	{
	"epoch": 0.07129455909943715,
	"grad_norm": 0.4855195879936218,
	"learning_rate": 1.3703703703703706e-05,
	"loss": 0.5732,
	"step": 38
	},
	{
	"epoch": 0.07317073170731707,
	"grad_norm": 0.7693135142326355,
	"learning_rate": 1.4074074074074075e-05,
	"loss": 0.5762,
	"step": 39
	},
	{
	"epoch": 0.075046904315197,
	"grad_norm": 0.4799287021160126,
	"learning_rate": 1.4444444444444446e-05,
	"loss": 0.6114,
	"step": 40
	},
	{
	"epoch": 0.07692307692307693,
	"grad_norm": 0.4819142520427704,
	"learning_rate": 1.4814814814814815e-05,
	"loss": 0.5855,
	"step": 41
	},
	{
	"epoch": 0.07879924953095685,
	"grad_norm": 0.49574413895606995,
	"learning_rate": 1.5185185185185187e-05,
	"loss": 0.5762,
	"step": 42
	},
	{
	"epoch": 0.08067542213883677,
	"grad_norm": 0.4454074501991272,
	"learning_rate": 1.555555555555556e-05,
	"loss": 0.5777,
	"step": 43
	},
	{
	"epoch": 0.0825515947467167,
	"grad_norm": 0.44248902797698975,
	"learning_rate": 1.5925925925925926e-05,
	"loss": 0.5729,
	"step": 44
	},
	{
	"epoch": 0.08442776735459662,
	"grad_norm": 0.46763530373573303,
	"learning_rate": 1.6296296296296297e-05,
	"loss": 0.5741,
	"step": 45
	},
	{
	"epoch": 0.08630393996247655,
	"grad_norm": 0.42297661304473877,
	"learning_rate": 1.6666666666666667e-05,
	"loss": 0.5488,
	"step": 46
	},
	{
	"epoch": 0.08818011257035648,
	"grad_norm": 0.49024441838264465,
	"learning_rate": 1.7037037037037038e-05,
	"loss": 0.5786,
	"step": 47
	},
	{
	"epoch": 0.0900562851782364,
	"grad_norm": 0.4836975038051605,
	"learning_rate": 1.740740740740741e-05,
	"loss": 0.549,
	"step": 48
	},
	{
	"epoch": 0.09193245778611632,
	"grad_norm": 0.4239577054977417,
	"learning_rate": 1.7777777777777777e-05,
	"loss": 0.5448,
	"step": 49
	},
	{
	"epoch": 0.09380863039399624,
	"grad_norm": 0.4513685405254364,
	"learning_rate": 1.814814814814815e-05,
	"loss": 0.5412,
	"step": 50
	},
	{
	"epoch": 0.09568480300187618,
	"grad_norm": 0.45834019780158997,
	"learning_rate": 1.851851851851852e-05,
	"loss": 0.555,
	"step": 51
	},
	{
	"epoch": 0.0975609756097561,
	"grad_norm": 0.44011378288269043,
	"learning_rate": 1.888888888888889e-05,
	"loss": 0.5427,
	"step": 52
	},
	{
	"epoch": 0.09943714821763602,
	"grad_norm": 0.4496270418167114,
	"learning_rate": 1.925925925925926e-05,
	"loss": 0.5371,
	"step": 53
	},
	{
	"epoch": 0.10131332082551595,
	"grad_norm": 0.41629332304000854,
	"learning_rate": 1.962962962962963e-05,
	"loss": 0.5204,
	"step": 54
	},
	{
	"epoch": 0.10318949343339587,
	"grad_norm": 0.45103877782821655,
	"learning_rate": 2e-05,
	"loss": 0.5648,
	"step": 55
	},
	{
	"epoch": 0.1050656660412758,
	"grad_norm": 0.4260631501674652,
	"learning_rate": 1.999978492141723e-05,
	"loss": 0.5491,
	"step": 56
	},
	{
	"epoch": 0.10694183864915573,
	"grad_norm": 0.46286889910697937,
	"learning_rate": 1.999913969492067e-05,
	"loss": 0.521,
	"step": 57
	},
	{
	"epoch": 0.10881801125703565,
	"grad_norm": 0.43611228466033936,
	"learning_rate": 1.99980643482652e-05,
	"loss": 0.5296,
	"step": 58
	},
	{
	"epoch": 0.11069418386491557,
	"grad_norm": 0.4539852738380432,
	"learning_rate": 1.9996558927707637e-05,
	"loss": 0.5463,
	"step": 59
	},
	{
	"epoch": 0.1125703564727955,
	"grad_norm": 0.42966416478157043,
	"learning_rate": 1.9994623498004717e-05,
	"loss": 0.5446,
	"step": 60
	},
	{
	"epoch": 0.11444652908067542,
	"grad_norm": 0.4595656394958496,
	"learning_rate": 1.9992258142410335e-05,
	"loss": 0.5443,
	"step": 61
	},
	{
	"epoch": 0.11632270168855535,
	"grad_norm": 0.4296264052391052,
	"learning_rate": 1.9989462962671957e-05,
	"loss": 0.5388,
	"step": 62
	},
	{
	"epoch": 0.11819887429643527,
	"grad_norm": 0.45982134342193604,
	"learning_rate": 1.998623807902625e-05,
	"loss": 0.5281,
	"step": 63
	},
	{
	"epoch": 0.1200750469043152,
	"grad_norm": 0.4597987234592438,
	"learning_rate": 1.9982583630193882e-05,
	"loss": 0.5351,
	"step": 64
	},
	{
	"epoch": 0.12195121951219512,
	"grad_norm": 0.45049232244491577,
	"learning_rate": 1.9978499773373596e-05,
	"loss": 0.5393,
	"step": 65
	},
	{
	"epoch": 0.12382739212007504,
	"grad_norm": 0.438589870929718,
	"learning_rate": 1.997398668423542e-05,
	"loss": 0.5374,
	"step": 66
	},
	{
	"epoch": 0.12570356472795496,
	"grad_norm": 0.4221203327178955,
	"learning_rate": 1.9969044556913116e-05,
	"loss": 0.5198,
	"step": 67
	},
	{
	"epoch": 0.1275797373358349,
	"grad_norm": 0.4725799560546875,
	"learning_rate": 1.9963673603995833e-05,
	"loss": 0.5174,
	"step": 68
	},
	{
	"epoch": 0.1294559099437148,
	"grad_norm": 0.41669028997421265,
	"learning_rate": 1.9957874056518954e-05,
	"loss": 0.5306,
	"step": 69
	},
	{
	"epoch": 0.13133208255159476,
	"grad_norm": 0.47250062227249146,
	"learning_rate": 1.9951646163954177e-05,
	"loss": 0.5067,
	"step": 70
	},
	{
	"epoch": 0.13320825515947468,
	"grad_norm": 0.47961607575416565,
	"learning_rate": 1.9944990194198758e-05,
	"loss": 0.5372,
	"step": 71
	},
	{
	"epoch": 0.1350844277673546,
	"grad_norm": 0.4163620173931122,
	"learning_rate": 1.993790643356401e-05,
	"loss": 0.5054,
	"step": 72
	},
	{
	"epoch": 0.13696060037523453,
	"grad_norm": 0.47573086619377136,
	"learning_rate": 1.9930395186762967e-05,
	"loss": 0.529,
	"step": 73
	},
	{
	"epoch": 0.13883677298311445,
	"grad_norm": 0.43163445591926575,
	"learning_rate": 1.9922456776897295e-05,
	"loss": 0.4976,
	"step": 74
	},
	{
	"epoch": 0.14071294559099437,
	"grad_norm": 0.4349230229854584,
	"learning_rate": 1.991409154544338e-05,
	"loss": 0.4964,
	"step": 75
	},
	{
	"epoch": 0.1425891181988743,
	"grad_norm": 0.41780340671539307,
	"learning_rate": 1.9905299852237654e-05,
	"loss": 0.5256,
	"step": 76
	},
	{
	"epoch": 0.14446529080675422,
	"grad_norm": 0.4596555531024933,
	"learning_rate": 1.9896082075461093e-05,
	"loss": 0.5019,
	"step": 77
	},
	{
	"epoch": 0.14634146341463414,
	"grad_norm": 0.45016729831695557,
	"learning_rate": 1.9886438611622973e-05,
	"loss": 0.533,
	"step": 78
	},
	{
	"epoch": 0.14821763602251406,
	"grad_norm": 0.45367276668548584,
	"learning_rate": 1.9876369875543805e-05,
	"loss": 0.5198,
	"step": 79
	},
	{
	"epoch": 0.150093808630394,
	"grad_norm": 0.47471052408218384,
	"learning_rate": 1.986587630033748e-05,
	"loss": 0.5106,
	"step": 80
	},
	{
	"epoch": 0.15196998123827393,
	"grad_norm": 0.45293959975242615,
	"learning_rate": 1.9854958337392654e-05,
	"loss": 0.5015,
	"step": 81
	},
	{
	"epoch": 0.15384615384615385,
	"grad_norm": 0.43654966354370117,
	"learning_rate": 1.9843616456353333e-05,
	"loss": 0.5146,
	"step": 82
	},
	{
	"epoch": 0.15572232645403378,
	"grad_norm": 0.454272598028183,
	"learning_rate": 1.983185114509865e-05,
	"loss": 0.5256,
	"step": 83
	},
	{
	"epoch": 0.1575984990619137,
	"grad_norm": 0.4107024371623993,
	"learning_rate": 1.9819662909721905e-05,
	"loss": 0.5109,
	"step": 84
	},
	{
	"epoch": 0.15947467166979362,
	"grad_norm": 0.46822798252105713,
	"learning_rate": 1.9807052274508773e-05,
	"loss": 0.5215,
	"step": 85
	},
	{
	"epoch": 0.16135084427767354,
	"grad_norm": 0.43109920620918274,
	"learning_rate": 1.9794019781914764e-05,
	"loss": 0.5403,
	"step": 86
	},
	{
	"epoch": 0.16322701688555347,
	"grad_norm": 0.44587594270706177,
	"learning_rate": 1.9780565992541887e-05,
	"loss": 0.4877,
	"step": 87
	},
	{
	"epoch": 0.1651031894934334,
	"grad_norm": 0.45819446444511414,
	"learning_rate": 1.976669148511453e-05,
	"loss": 0.4943,
	"step": 88
	},
	{
	"epoch": 0.1669793621013133,
	"grad_norm": 0.44408440589904785,
	"learning_rate": 1.9752396856454576e-05,
	"loss": 0.5095,
	"step": 89
	},
	{
	"epoch": 0.16885553470919323,
	"grad_norm": 0.4661826193332672,
	"learning_rate": 1.973768272145571e-05,
	"loss": 0.5232,
	"step": 90
	},
	{
	"epoch": 0.17073170731707318,
	"grad_norm": 0.4267039895057678,
	"learning_rate": 1.9722549713057008e-05,
	"loss": 0.5069,
	"step": 91
	},
	{
	"epoch": 0.1726078799249531,
	"grad_norm": 0.4659229516983032,
	"learning_rate": 1.9706998482215654e-05,
	"loss": 0.4994,
	"step": 92
	},
	{
	"epoch": 0.17448405253283303,
	"grad_norm": 0.4425814747810364,
	"learning_rate": 1.9691029697878998e-05,
	"loss": 0.493,
	"step": 93
	},
	{
	"epoch": 0.17636022514071295,
	"grad_norm": 0.5046799182891846,
	"learning_rate": 1.9674644046955732e-05,
	"loss": 0.5022,
	"step": 94
	},
	{
	"epoch": 0.17823639774859287,
	"grad_norm": 0.4815562069416046,
	"learning_rate": 1.965784223428638e-05,
	"loss": 0.5086,
	"step": 95
	},
	{
	"epoch": 0.1801125703564728,
	"grad_norm": 0.463766872882843,
	"learning_rate": 1.9640624982612943e-05,
	"loss": 0.4984,
	"step": 96
	},
	{
	"epoch": 0.18198874296435272,
	"grad_norm": 0.4671774208545685,
	"learning_rate": 1.962299303254785e-05,
	"loss": 0.5002,
	"step": 97
	},
	{
	"epoch": 0.18386491557223264,
	"grad_norm": 0.4975290596485138,
	"learning_rate": 1.960494714254206e-05,
	"loss": 0.5194,
	"step": 98
	},
	{
	"epoch": 0.18574108818011256,
	"grad_norm": 0.4462299644947052,
	"learning_rate": 1.9586488088852465e-05,
	"loss": 0.5137,
	"step": 99
	},
	{
	"epoch": 0.18761726078799248,
	"grad_norm": 0.46154576539993286,
	"learning_rate": 1.9567616665508483e-05,
	"loss": 0.5029,
	"step": 100
	},
	{
	"epoch": 0.1894934333958724,
	"grad_norm": 0.4629541039466858,
	"learning_rate": 1.954833368427792e-05,
	"loss": 0.5122,
	"step": 101
	},
	{
	"epoch": 0.19136960600375236,
	"grad_norm": 0.4575340151786804,
	"learning_rate": 1.952863997463202e-05,
	"loss": 0.4964,
	"step": 102
	},
	{
	"epoch": 0.19324577861163228,
	"grad_norm": 0.43607306480407715,
	"learning_rate": 1.950853638370982e-05,
	"loss": 0.5048,
	"step": 103
	},
	{
	"epoch": 0.1951219512195122,
	"grad_norm": 0.45974987745285034,
	"learning_rate": 1.9488023776281695e-05,
	"loss": 0.5003,
	"step": 104
	},
	{
	"epoch": 0.19699812382739212,
	"grad_norm": 0.52689129114151,
	"learning_rate": 1.946710303471214e-05,
	"loss": 0.5149,
	"step": 105
	},
	{
	"epoch": 0.19887429643527205,
	"grad_norm": 0.5598635077476501,
	"learning_rate": 1.9445775058921856e-05,
	"loss": 0.5218,
	"step": 106
	},
	{
	"epoch": 0.20075046904315197,
	"grad_norm": 0.4801148474216461,
	"learning_rate": 1.9424040766348992e-05,
	"loss": 0.5,
	"step": 107
	},
	{
	"epoch": 0.2026266416510319,
	"grad_norm": 0.4354962110519409,
	"learning_rate": 1.9401901091909733e-05,
	"loss": 0.4922,
	"step": 108
	},
	{
	"epoch": 0.2045028142589118,
	"grad_norm": 0.49421778321266174,
	"learning_rate": 1.9379356987958022e-05,
	"loss": 0.5101,
	"step": 109
	},
	{
	"epoch": 0.20637898686679174,
	"grad_norm": 0.4634808599948883,
	"learning_rate": 1.9356409424244655e-05,
	"loss": 0.4723,
	"step": 110
	},
	{
	"epoch": 0.20825515947467166,
	"grad_norm": 0.45459961891174316,
	"learning_rate": 1.9333059387875527e-05,
	"loss": 0.4984,
	"step": 111
	},
	{
	"epoch": 0.2101313320825516,
	"grad_norm": 0.4482363164424896,
	"learning_rate": 1.930930788326918e-05,
	"loss": 0.4992,
	"step": 112
	},
	{
	"epoch": 0.21200750469043153,
	"grad_norm": 0.46492260694503784,
	"learning_rate": 1.9285155932113612e-05,
	"loss": 0.5049,
	"step": 113
	},
	{
	"epoch": 0.21388367729831145,
	"grad_norm": 0.4934901297092438,
	"learning_rate": 1.9260604573322298e-05,
	"loss": 0.511,
	"step": 114
	},
	{
	"epoch": 0.21575984990619138,
	"grad_norm": 0.47866374254226685,
	"learning_rate": 1.9235654862989537e-05,
	"loss": 0.4813,
	"step": 115
	},
	{
	"epoch": 0.2176360225140713,
	"grad_norm": 0.4691963493824005,
	"learning_rate": 1.921030787434499e-05,
	"loss": 0.5034,
	"step": 116
	},
	{
	"epoch": 0.21951219512195122,
	"grad_norm": 0.48848795890808105,
	"learning_rate": 1.9184564697707545e-05,
	"loss": 0.4827,
	"step": 117
	},
	{
	"epoch": 0.22138836772983114,
	"grad_norm": 0.48106610774993896,
	"learning_rate": 1.915842644043838e-05,
	"loss": 0.5097,
	"step": 118
	},
	{
	"epoch": 0.22326454033771106,
	"grad_norm": 0.46916061639785767,
	"learning_rate": 1.913189422689337e-05,
	"loss": 0.501,
	"step": 119
	},
	{
	"epoch": 0.225140712945591,
	"grad_norm": 0.46038028597831726,
	"learning_rate": 1.910496919837469e-05,
	"loss": 0.5161,
	"step": 120
	},
	{
	"epoch": 0.2270168855534709,
	"grad_norm": 0.4193981885910034,
	"learning_rate": 1.907765251308173e-05,
	"loss": 0.4696,
	"step": 121
	},
	{
	"epoch": 0.22889305816135083,
	"grad_norm": 0.47197026014328003,
	"learning_rate": 1.9049945346061292e-05,
	"loss": 0.5152,
	"step": 122
	},
	{
	"epoch": 0.23076923076923078,
	"grad_norm": 0.44787487387657166,
	"learning_rate": 1.902184888915701e-05,
	"loss": 0.4983,
	"step": 123
	},
	{
	"epoch": 0.2326454033771107,
	"grad_norm": 0.5235477685928345,
	"learning_rate": 1.8993364350958115e-05,
	"loss": 0.5041,
	"step": 124
	},
	{
	"epoch": 0.23452157598499063,
	"grad_norm": 0.4769859313964844,
	"learning_rate": 1.8964492956747427e-05,
	"loss": 0.5011,
	"step": 125
	},
	{
	"epoch": 0.23639774859287055,
	"grad_norm": 0.48972049355506897,
	"learning_rate": 1.8935235948448653e-05,
	"loss": 0.4742,
	"step": 126
	},
	{
	"epoch": 0.23827392120075047,
	"grad_norm": 0.4640520215034485,
	"learning_rate": 1.8905594584572973e-05,
	"loss": 0.5005,
	"step": 127
	},
	{
	"epoch": 0.2401500938086304,
	"grad_norm": 0.467355340719223,
	"learning_rate": 1.8875570140164895e-05,
	"loss": 0.5138,
	"step": 128
	},
	{
	"epoch": 0.24202626641651032,
	"grad_norm": 0.4687252342700958,
	"learning_rate": 1.8845163906747407e-05,
	"loss": 0.497,
	"step": 129
	},
	{
	"epoch": 0.24390243902439024,
	"grad_norm": 0.4493774175643921,
	"learning_rate": 1.8814377192266424e-05,
	"loss": 0.5025,
	"step": 130
	},
	{
	"epoch": 0.24577861163227016,
	"grad_norm": 0.4495251774787903,
	"learning_rate": 1.8783211321034537e-05,
	"loss": 0.5102,
	"step": 131
	},
	{
	"epoch": 0.24765478424015008,
	"grad_norm": 0.45046165585517883,
	"learning_rate": 1.8751667633674023e-05,
	"loss": 0.4922,
	"step": 132
	},
	{
	"epoch": 0.24953095684803,
	"grad_norm": 0.4332929849624634,
	"learning_rate": 1.87197474870592e-05,
	"loss": 0.488,
	"step": 133
	},
	{
	"epoch": 0.25140712945590993,
	"grad_norm": 0.4550478160381317,
	"learning_rate": 1.8687452254258046e-05,
	"loss": 0.512,
	"step": 134
	},
	{
	"epoch": 0.25328330206378985,
	"grad_norm": 0.4503070116043091,
	"learning_rate": 1.8654783324473138e-05,
	"loss": 0.5134,
	"step": 135
	},
	{
	"epoch": 0.2551594746716698,
	"grad_norm": 0.43286991119384766,
	"learning_rate": 1.8621742102981906e-05,
	"loss": 0.4567,
	"step": 136
	},
	{
	"epoch": 0.2570356472795497,
	"grad_norm": 0.43096885085105896,
	"learning_rate": 1.8588330011076167e-05,
	"loss": 0.5206,
	"step": 137
	},
	{
	"epoch": 0.2589118198874296,
	"grad_norm": 0.4638442099094391,
	"learning_rate": 1.855454848600099e-05,
	"loss": 0.5162,
	"step": 138
	},
	{
	"epoch": 0.2607879924953096,
	"grad_norm": 0.441213458776474,
	"learning_rate": 1.852039898089289e-05,
	"loss": 0.5092,
	"step": 139
	},
	{
	"epoch": 0.2626641651031895,
	"grad_norm": 0.4825282692909241,
	"learning_rate": 1.8485882964717292e-05,
	"loss": 0.4898,
	"step": 140
	},
	{
	"epoch": 0.26454033771106944,
	"grad_norm": 0.4525107145309448,
	"learning_rate": 1.845100192220537e-05,
	"loss": 0.4945,
	"step": 141
	},
	{
	"epoch": 0.26641651031894936,
	"grad_norm": 0.515526294708252,
	"learning_rate": 1.841575735379016e-05,
	"loss": 0.4843,
	"step": 142
	},
	{
	"epoch": 0.2682926829268293,
	"grad_norm": 0.4331778287887573,
	"learning_rate": 1.838015077554203e-05,
	"loss": 0.4641,
	"step": 143
	},
	{
	"epoch": 0.2701688555347092,
	"grad_norm": 0.47300004959106445,
	"learning_rate": 1.8344183719103454e-05,
	"loss": 0.4745,
	"step": 144
	},
	{
	"epoch": 0.27204502814258913,
	"grad_norm": 0.4185693860054016,
	"learning_rate": 1.8307857731623133e-05,
	"loss": 0.4667,
	"step": 145
	},
	{
	"epoch": 0.27392120075046905,
	"grad_norm": 0.44932979345321655,
	"learning_rate": 1.8271174375689454e-05,
	"loss": 0.5011,
	"step": 146
	},
	{
	"epoch": 0.275797373358349,
	"grad_norm": 0.4410933554172516,
	"learning_rate": 1.8234135229263258e-05,
	"loss": 0.4734,
	"step": 147
	},
	{
	"epoch": 0.2776735459662289,
	"grad_norm": 0.45480766892433167,
	"learning_rate": 1.819674188560997e-05,
	"loss": 0.5067,
	"step": 148
	},
	{
	"epoch": 0.2795497185741088,
	"grad_norm": 0.4583900272846222,
	"learning_rate": 1.8158995953231055e-05,
	"loss": 0.5051,
	"step": 149
	},
	{
	"epoch": 0.28142589118198874,
	"grad_norm": 0.421601802110672,
	"learning_rate": 1.8120899055794848e-05,
	"loss": 0.4795,
	"step": 150
	},
	{
	"epoch": 0.28330206378986866,
	"grad_norm": 0.41240212321281433,
	"learning_rate": 1.808245283206669e-05,
	"loss": 0.4904,
	"step": 151
	},
	{
	"epoch": 0.2851782363977486,
	"grad_norm": 0.4446975886821747,
	"learning_rate": 1.8043658935838436e-05,
	"loss": 0.4899,
	"step": 152
	},
	{
	"epoch": 0.2870544090056285,
	"grad_norm": 0.43199482560157776,
	"learning_rate": 1.800451903585734e-05,
	"loss": 0.4833,
	"step": 153
	},
	{
	"epoch": 0.28893058161350843,
	"grad_norm": 0.42450612783432007,
	"learning_rate": 1.796503481575424e-05,
	"loss": 0.4872,
	"step": 154
	},
	{
	"epoch": 0.29080675422138835,
	"grad_norm": 0.4116723835468292,
	"learning_rate": 1.7925207973971163e-05,
	"loss": 0.4952,
	"step": 155
	},
	{
	"epoch": 0.2926829268292683,
	"grad_norm": 0.4367542266845703,
	"learning_rate": 1.7885040223688234e-05,
	"loss": 0.488,
	"step": 156
	},
	{
	"epoch": 0.2945590994371482,
	"grad_norm": 0.47015565633773804,
	"learning_rate": 1.7844533292750025e-05,
	"loss": 0.4965,
	"step": 157
	},
	{
	"epoch": 0.2964352720450281,
	"grad_norm": 0.43937620520591736,
	"learning_rate": 1.7803688923591194e-05,
	"loss": 0.4935,
	"step": 158
	},
	{
	"epoch": 0.29831144465290804,
	"grad_norm": 0.4693063795566559,
	"learning_rate": 1.7762508873161542e-05,
	"loss": 0.4862,
	"step": 159
	},
	{
	"epoch": 0.300187617260788,
	"grad_norm": 0.444977730512619,
	"learning_rate": 1.7720994912850452e-05,
	"loss": 0.4888,
	"step": 160
	},
	{
	"epoch": 0.30206378986866794,
	"grad_norm": 0.4322124719619751,
	"learning_rate": 1.767914882841067e-05,
	"loss": 0.4875,
	"step": 161
	},
	{
	"epoch": 0.30393996247654786,
	"grad_norm": 0.4464256763458252,
	"learning_rate": 1.7636972419881507e-05,
	"loss": 0.4939,
	"step": 162
	},
	{
	"epoch": 0.3058161350844278,
	"grad_norm": 0.46564194560050964,
	"learning_rate": 1.759446750151139e-05,
	"loss": 0.4987,
	"step": 163
	},
	{
	"epoch": 0.3076923076923077,
	"grad_norm": 0.5087194442749023,
	"learning_rate": 1.755163590167985e-05,
	"loss": 0.4799,
	"step": 164
	},
	{
	"epoch": 0.30956848030018763,
	"grad_norm": 0.4272122085094452,
	"learning_rate": 1.7508479462818836e-05,
	"loss": 0.4919,
	"step": 165
	},
	{
	"epoch": 0.31144465290806755,
	"grad_norm": 0.506621778011322,
	"learning_rate": 1.7465000041333496e-05,
	"loss": 0.4846,
	"step": 166
	},
	{
	"epoch": 0.3133208255159475,
	"grad_norm": 0.4598137140274048,
	"learning_rate": 1.7421199507522297e-05,
	"loss": 0.4933,
	"step": 167
	},
	{
	"epoch": 0.3151969981238274,
	"grad_norm": 0.4636599123477936,
	"learning_rate": 1.7377079745496586e-05,
	"loss": 0.4898,
	"step": 168
	},
	{
	"epoch": 0.3170731707317073,
	"grad_norm": 0.48578599095344543,
	"learning_rate": 1.7332642653099542e-05,
	"loss": 0.5115,
	"step": 169
	},
	{
	"epoch": 0.31894934333958724,
	"grad_norm": 0.5075885653495789,
	"learning_rate": 1.7287890141824537e-05,
	"loss": 0.4839,
	"step": 170
	},
	{
	"epoch": 0.32082551594746717,
	"grad_norm": 0.48771244287490845,
	"learning_rate": 1.724282413673291e-05,
	"loss": 0.5032,
	"step": 171
	},
	{
	"epoch": 0.3227016885553471,
	"grad_norm": 0.5117051005363464,
	"learning_rate": 1.7197446576371163e-05,
	"loss": 0.4806,
	"step": 172
	},
	{
	"epoch": 0.324577861163227,
	"grad_norm": 0.48115479946136475,
	"learning_rate": 1.7151759412687573e-05,
	"loss": 0.4913,
	"step": 173
	},
	{
	"epoch": 0.32645403377110693,
	"grad_norm": 0.5075434446334839,
	"learning_rate": 1.7105764610948212e-05,
	"loss": 0.4902,
	"step": 174
	},
	{
	"epoch": 0.32833020637898686,
	"grad_norm": 0.4747563898563385,
	"learning_rate": 1.705946414965245e-05,
	"loss": 0.4934,
	"step": 175
	},
	{
	"epoch": 0.3302063789868668,
	"grad_norm": 0.45580369234085083,
	"learning_rate": 1.7012860020447797e-05,
	"loss": 0.5069,
	"step": 176
	},
	{
	"epoch": 0.3320825515947467,
	"grad_norm": 0.4514389932155609,
	"learning_rate": 1.6965954228044265e-05,
	"loss": 0.4985,
	"step": 177
	},
	{
	"epoch": 0.3339587242026266,
	"grad_norm": 0.4397890567779541,
	"learning_rate": 1.691874879012813e-05,
	"loss": 0.4543,
	"step": 178
	},
	{
	"epoch": 0.33583489681050654,
	"grad_norm": 0.47879987955093384,
	"learning_rate": 1.6871245737275124e-05,
	"loss": 0.4939,
	"step": 179
	},
	{
	"epoch": 0.33771106941838647,
	"grad_norm": 0.4909740090370178,
	"learning_rate": 1.682344711286311e-05,
	"loss": 0.4951,
	"step": 180
	},
	{
	"epoch": 0.3395872420262664,
	"grad_norm": 0.46630603075027466,
	"learning_rate": 1.677535497298416e-05,
	"loss": 0.4913,
	"step": 181
	},
	{
	"epoch": 0.34146341463414637,
	"grad_norm": 0.46876662969589233,
	"learning_rate": 1.6726971386356137e-05,
	"loss": 0.4815,
	"step": 182
	},
	{
	"epoch": 0.3433395872420263,
	"grad_norm": 0.4811256229877472,
	"learning_rate": 1.6678298434233687e-05,
	"loss": 0.4892,
	"step": 183
	},
	{
	"epoch": 0.3452157598499062,
	"grad_norm": 0.44975170493125916,
	"learning_rate": 1.6629338210318725e-05,
	"loss": 0.4848,
	"step": 184
	},
	{
	"epoch": 0.34709193245778613,
	"grad_norm": 0.47891831398010254,
	"learning_rate": 1.658009282067036e-05,
	"loss": 0.4767,
	"step": 185
	},
	{
	"epoch": 0.34896810506566606,
	"grad_norm": 0.4403041899204254,
	"learning_rate": 1.653056438361432e-05,
	"loss": 0.4866,
	"step": 186
	},
	{
	"epoch": 0.350844277673546,
	"grad_norm": 0.45113885402679443,
	"learning_rate": 1.648075502965181e-05,
	"loss": 0.4777,
	"step": 187
	},
	{
	"epoch": 0.3527204502814259,
	"grad_norm": 0.44296011328697205,
	"learning_rate": 1.643066690136789e-05,
	"loss": 0.4787,
	"step": 188
	},
	{
	"epoch": 0.3545966228893058,
	"grad_norm": 0.43343913555145264,
	"learning_rate": 1.638030215333928e-05,
	"loss": 0.4725,
	"step": 189
	},
	{
	"epoch": 0.35647279549718575,
	"grad_norm": 0.4752146005630493,
	"learning_rate": 1.6329662952041712e-05,
	"loss": 0.4972,
	"step": 190
	},
	{
	"epoch": 0.35834896810506567,
	"grad_norm": 0.47067368030548096,
	"learning_rate": 1.6278751475756712e-05,
	"loss": 0.4814,
	"step": 191
	},
	{
	"epoch": 0.3602251407129456,
	"grad_norm": 0.449448823928833,
	"learning_rate": 1.6227569914477914e-05,
	"loss": 0.4948,
	"step": 192
	},
	{
	"epoch": 0.3621013133208255,
	"grad_norm": 0.43309763073921204,
	"learning_rate": 1.617612046981685e-05,
	"loss": 0.4759,
	"step": 193
	},
	{
	"epoch": 0.36397748592870544,
	"grad_norm": 0.4724906384944916,
	"learning_rate": 1.6124405354908246e-05,
	"loss": 0.4779,
	"step": 194
	},
	{
	"epoch": 0.36585365853658536,
	"grad_norm": 0.4331618845462799,
	"learning_rate": 1.6072426794314836e-05,
	"loss": 0.4936,
	"step": 195
	},
	{
	"epoch": 0.3677298311444653,
	"grad_norm": 0.4011441469192505,
	"learning_rate": 1.602018702393164e-05,
	"loss": 0.4701,
	"step": 196
	},
	{
	"epoch": 0.3696060037523452,
	"grad_norm": 0.4599146246910095,
	"learning_rate": 1.5967688290889817e-05,
	"loss": 0.4584,
	"step": 197
	},
	{
	"epoch": 0.3714821763602251,
	"grad_norm": 0.4477120637893677,
	"learning_rate": 1.591493285345999e-05,
	"loss": 0.4932,
	"step": 198
	},
	{
	"epoch": 0.37335834896810505,
	"grad_norm": 0.4366087317466736,
	"learning_rate": 1.5861922980955092e-05,
	"loss": 0.4853,
	"step": 199
	},
	{
	"epoch": 0.37523452157598497,
	"grad_norm": 0.4719770848751068,
	"learning_rate": 1.5808660953632783e-05,
	"loss": 0.4881,
	"step": 200
	},
	{
	"epoch": 0.3771106941838649,
	"grad_norm": 0.4526340961456299,
	"learning_rate": 1.5755149062597332e-05,
	"loss": 0.4972,
	"step": 201
	},
	{
	"epoch": 0.3789868667917448,
	"grad_norm": 0.4521251618862152,
	"learning_rate": 1.5701389609701076e-05,
	"loss": 0.4854,
	"step": 202
	},
	{
	"epoch": 0.3808630393996248,
	"grad_norm": 0.41455745697021484,
	"learning_rate": 1.5647384907445402e-05,
	"loss": 0.486,
	"step": 203
	},
	{
	"epoch": 0.3827392120075047,
	"grad_norm": 0.47328656911849976,
	"learning_rate": 1.559313727888127e-05,
	"loss": 0.477,
	"step": 204
	},
	{
	"epoch": 0.38461538461538464,
	"grad_norm": 0.4337882697582245,
	"learning_rate": 1.5538649057509307e-05,
	"loss": 0.4757,
	"step": 205
	},
	{
	"epoch": 0.38649155722326456,
	"grad_norm": 0.4305742681026459,
	"learning_rate": 1.5483922587179386e-05,
	"loss": 0.4907,
	"step": 206
	},
	{
	"epoch": 0.3883677298311445,
	"grad_norm": 0.4382014870643616,
	"learning_rate": 1.5428960221989844e-05,
	"loss": 0.4805,
	"step": 207
	},
	{
	"epoch": 0.3902439024390244,
	"grad_norm": 0.4552824795246124,
	"learning_rate": 1.537376432618621e-05,
	"loss": 0.476,
	"step": 208
	},
	{
	"epoch": 0.3921200750469043,
	"grad_norm": 0.430683434009552,
	"learning_rate": 1.5318337274059492e-05,
	"loss": 0.496,
	"step": 209
	},
	{
	"epoch": 0.39399624765478425,
	"grad_norm": 0.46657851338386536,
	"learning_rate": 1.5262681449844047e-05,
	"loss": 0.466,
	"step": 210
	},
	{
	"epoch": 0.39587242026266417,
	"grad_norm": 0.42215755581855774,
	"learning_rate": 1.5206799247615038e-05,
	"loss": 0.4694,
	"step": 211
	},
	{
	"epoch": 0.3977485928705441,
	"grad_norm": 0.44483405351638794,
	"learning_rate": 1.5150693071185441e-05,
	"loss": 0.4625,
	"step": 212
	},
	{
	"epoch": 0.399624765478424,
	"grad_norm": 0.4508770704269409,
	"learning_rate": 1.5094365334002631e-05,
	"loss": 0.4721,
	"step": 213
	},
	{
	"epoch": 0.40150093808630394,
	"grad_norm": 0.47280606627464294,
	"learning_rate": 1.5037818459044589e-05,
	"loss": 0.4988,
	"step": 214
	},
	{
	"epoch": 0.40337711069418386,
	"grad_norm": 0.46489429473876953,
	"learning_rate": 1.498105487871566e-05,
	"loss": 0.4775,
	"step": 215
	},
	{
	"epoch": 0.4052532833020638,
	"grad_norm": 0.45847252011299133,
	"learning_rate": 1.4924077034741924e-05,
	"loss": 0.501,
	"step": 216
	},
	{
	"epoch": 0.4071294559099437,
	"grad_norm": 0.4636571407318115,
	"learning_rate": 1.4866887378066169e-05,
	"loss": 0.4603,
	"step": 217
	},
	{
	"epoch": 0.4090056285178236,
	"grad_norm": 0.4642573893070221,
	"learning_rate": 1.4809488368742458e-05,
	"loss": 0.452,
	"step": 218
	},
	{
	"epoch": 0.41088180112570355,
	"grad_norm": 0.42232292890548706,
	"learning_rate": 1.4751882475830302e-05,
	"loss": 0.4969,
	"step": 219
	},
	{
	"epoch": 0.41275797373358347,
	"grad_norm": 0.5324833989143372,
	"learning_rate": 1.4694072177288464e-05,
	"loss": 0.4563,
	"step": 220
	},
	{
	"epoch": 0.4146341463414634,
	"grad_norm": 0.4543067514896393,
	"learning_rate": 1.4636059959868363e-05,
	"loss": 0.4663,
	"step": 221
	},
	{
	"epoch": 0.4165103189493433,
	"grad_norm": 0.45857909321784973,
	"learning_rate": 1.4577848319007098e-05,
	"loss": 0.4857,
	"step": 222
	},
	{
	"epoch": 0.41838649155722324,
	"grad_norm": 0.4585043787956238,
	"learning_rate": 1.4519439758720108e-05,
	"loss": 0.4722,
	"step": 223
	},
	{
	"epoch": 0.4202626641651032,
	"grad_norm": 0.431410551071167,
	"learning_rate": 1.4460836791493477e-05,
	"loss": 0.4857,
	"step": 224
	},
	{
	"epoch": 0.42213883677298314,
	"grad_norm": 0.4164378345012665,
	"learning_rate": 1.4402041938175826e-05,
	"loss": 0.4703,
	"step": 225
	},
	{
	"epoch": 0.42401500938086306,
	"grad_norm": 0.43511542677879333,
	"learning_rate": 1.43430577278699e-05,
	"loss": 0.5023,
	"step": 226
	},
	{
	"epoch": 0.425891181988743,
	"grad_norm": 0.4781929552555084,
	"learning_rate": 1.428388669782377e-05,
	"loss": 0.489,
	"step": 227
	},
	{
	"epoch": 0.4277673545966229,
	"grad_norm": 0.38802462816238403,
	"learning_rate": 1.4224531393321696e-05,
	"loss": 0.4669,
	"step": 228
	},
	{
	"epoch": 0.42964352720450283,
	"grad_norm": 0.3962300419807434,
	"learning_rate": 1.4164994367574631e-05,
	"loss": 0.4001,
	"step": 229
	},
	{
	"epoch": 0.43151969981238275,
	"grad_norm": 0.44593545794487,
	"learning_rate": 1.41052781816104e-05,
	"loss": 0.4861,
	"step": 230
	},
	{
	"epoch": 0.4333958724202627,
	"grad_norm": 0.4363047182559967,
	"learning_rate": 1.4045385404163533e-05,
	"loss": 0.4897,
	"step": 231
	},
	{
	"epoch": 0.4352720450281426,
	"grad_norm": 0.4661344885826111,
	"learning_rate": 1.3985318611564761e-05,
	"loss": 0.4867,
	"step": 232
	},
	{
	"epoch": 0.4371482176360225,
	"grad_norm": 0.48936548829078674,
	"learning_rate": 1.3925080387630217e-05,
	"loss": 0.4686,
	"step": 233
	},
	{
	"epoch": 0.43902439024390244,
	"grad_norm": 0.4576188623905182,
	"learning_rate": 1.3864673323550264e-05,
	"loss": 0.4811,
	"step": 234
	},
	{
	"epoch": 0.44090056285178236,
	"grad_norm": 0.44022276997566223,
	"learning_rate": 1.3804100017778048e-05,
	"loss": 0.4866,
	"step": 235
	},
	{
	"epoch": 0.4427767354596623,
	"grad_norm": 0.48328202962875366,
	"learning_rate": 1.3743363075917723e-05,
	"loss": 0.4863,
	"step": 236
	},
	{
	"epoch": 0.4446529080675422,
	"grad_norm": 0.44896700978279114,
	"learning_rate": 1.3682465110612361e-05,
	"loss": 0.4797,
	"step": 237
	},
	{
	"epoch": 0.44652908067542213,
	"grad_norm": 0.4463822841644287,
	"learning_rate": 1.3621408741431583e-05,
	"loss": 0.4676,
	"step": 238
	},
	{
	"epoch": 0.44840525328330205,
	"grad_norm": 0.4772860109806061,
	"learning_rate": 1.3560196594758855e-05,
	"loss": 0.4581,
	"step": 239
	},
	{
	"epoch": 0.450281425891182,
	"grad_norm": 0.4344514012336731,
	"learning_rate": 1.3498831303678525e-05,
	"loss": 0.4658,
	"step": 240
	},
	{
	"epoch": 0.4521575984990619,
	"grad_norm": 0.4256344735622406,
	"learning_rate": 1.3437315507862568e-05,
	"loss": 0.4817,
	"step": 241
	},
	{
	"epoch": 0.4540337711069418,
	"grad_norm": 0.44355109333992004,
	"learning_rate": 1.3375651853457012e-05,
	"loss": 0.4938,
	"step": 242
	},
	{
	"epoch": 0.45590994371482174,
	"grad_norm": 0.39970898628234863,
	"learning_rate": 1.331384299296814e-05,
	"loss": 0.474,
	"step": 243
	},
	{
	"epoch": 0.45778611632270166,
	"grad_norm": 0.43661677837371826,
	"learning_rate": 1.325189158514838e-05,
	"loss": 0.4736,
	"step": 244
	},
	{
	"epoch": 0.4596622889305816,
	"grad_norm": 0.39525124430656433,
	"learning_rate": 1.3189800294881926e-05,
	"loss": 0.4699,
	"step": 245
	},
	{
	"epoch": 0.46153846153846156,
	"grad_norm": 0.4325285851955414,
	"learning_rate": 1.312757179307012e-05,
	"loss": 0.4885,
	"step": 246
	},
	{
	"epoch": 0.4634146341463415,
	"grad_norm": 0.403023362159729,
	"learning_rate": 1.3065208756516557e-05,
	"loss": 0.4775,
	"step": 247
	},
	{
	"epoch": 0.4652908067542214,
	"grad_norm": 0.434388667345047,
	"learning_rate": 1.3002713867811947e-05,
	"loss": 0.474,
	"step": 248
	},
	{
	"epoch": 0.46716697936210133,
	"grad_norm": 0.4568539559841156,
	"learning_rate": 1.2940089815218703e-05,
	"loss": 0.4998,
	"step": 249
	},
	{
	"epoch": 0.46904315196998125,
	"grad_norm": 0.43496668338775635,
	"learning_rate": 1.287733929255532e-05,
	"loss": 0.4753,
	"step": 250
	},
	{
	"epoch": 0.4709193245778612,
	"grad_norm": 0.38006827235221863,
	"learning_rate": 1.28144649990805e-05,
	"loss": 0.4618,
	"step": 251
	},
	{
	"epoch": 0.4727954971857411,
	"grad_norm": 0.40870386362075806,
	"learning_rate": 1.2751469639377028e-05,
	"loss": 0.4708,
	"step": 252
	},
	{
	"epoch": 0.474671669793621,
	"grad_norm": 0.40516600012779236,
	"learning_rate": 1.2688355923235439e-05,
	"loss": 0.4666,
	"step": 253
	},
	{
	"epoch": 0.47654784240150094,
	"grad_norm": 0.41616344451904297,
	"learning_rate": 1.2625126565537458e-05,
	"loss": 0.4774,
	"step": 254
	},
	{
	"epoch": 0.47842401500938087,
	"grad_norm": 0.41662484407424927,
	"learning_rate": 1.2561784286139214e-05,
	"loss": 0.4696,
	"step": 255
	},
	{
	"epoch": 0.4803001876172608,
	"grad_norm": 0.46140581369400024,
	"learning_rate": 1.2498331809754243e-05,
	"loss": 0.4584,
	"step": 256
	},
	{
	"epoch": 0.4821763602251407,
	"grad_norm": 0.38339847326278687,
	"learning_rate": 1.2434771865836281e-05,
	"loss": 0.4705,
	"step": 257
	},
	{
	"epoch": 0.48405253283302063,
	"grad_norm": 0.39309021830558777,
	"learning_rate": 1.2371107188461868e-05,
	"loss": 0.4826,
	"step": 258
	},
	{
	"epoch": 0.48592870544090055,
	"grad_norm": 0.4067246615886688,
	"learning_rate": 1.2307340516212713e-05,
	"loss": 0.4745,
	"step": 259
	},
	{
	"epoch": 0.4878048780487805,
	"grad_norm": 0.3975689113140106,
	"learning_rate": 1.2243474592057918e-05,
	"loss": 0.47,
	"step": 260
	},
	{
	"epoch": 0.4896810506566604,
	"grad_norm": 0.4230562448501587,
	"learning_rate": 1.2179512163235973e-05,
	"loss": 0.4664,
	"step": 261
	},
	{
	"epoch": 0.4915572232645403,
	"grad_norm": 0.4172329008579254,
	"learning_rate": 1.2115455981136587e-05,
	"loss": 0.4624,
	"step": 262
	},
	{
	"epoch": 0.49343339587242024,
	"grad_norm": 0.3934493064880371,
	"learning_rate": 1.2051308801182333e-05,
	"loss": 0.4741,
	"step": 263
	},
	{
	"epoch": 0.49530956848030017,
	"grad_norm": 0.40225809812545776,
	"learning_rate": 1.1987073382710123e-05,
	"loss": 0.4577,
	"step": 264
	},
	{
	"epoch": 0.4971857410881801,
	"grad_norm": 0.4090576469898224,
	"learning_rate": 1.1922752488852512e-05,
	"loss": 0.4688,
	"step": 265
	},
	{
	"epoch": 0.49906191369606,
	"grad_norm": 0.39649295806884766,
	"learning_rate": 1.1858348886418832e-05,
	"loss": 0.4637,
	"step": 266
	},
	{
	"epoch": 0.50093808630394,
	"grad_norm": 0.44001370668411255,
	"learning_rate": 1.1793865345776193e-05,
	"loss": 0.4708,
	"step": 267
	},
	{
	"epoch": 0.5028142589118199,
	"grad_norm": 0.41409698128700256,
	"learning_rate": 1.1729304640730303e-05,
	"loss": 0.4764,
	"step": 268
	},
	{
	"epoch": 0.5046904315196998,
	"grad_norm": 0.4101436138153076,
	"learning_rate": 1.1664669548406153e-05,
	"loss": 0.4701,
	"step": 269
	},
	{
	"epoch": 0.5065666041275797,
	"grad_norm": 0.4108476936817169,
	"learning_rate": 1.159996284912855e-05,
	"loss": 0.4764,
	"step": 270
	},
	{
	"epoch": 0.5084427767354597,
	"grad_norm": 0.40407615900039673,
	"learning_rate": 1.153518732630253e-05,
	"loss": 0.4875,
	"step": 271
	},
	{
	"epoch": 0.5103189493433395,
	"grad_norm": 0.3871546983718872,
	"learning_rate": 1.1470345766293627e-05,
	"loss": 0.4784,
	"step": 272
	},
	{
	"epoch": 0.5121951219512195,
	"grad_norm": 0.3768629729747772,
	"learning_rate": 1.1405440958308001e-05,
	"loss": 0.4168,
	"step": 273
	},
	{
	"epoch": 0.5140712945590994,
	"grad_norm": 0.3968166708946228,
	"learning_rate": 1.1340475694272479e-05,
	"loss": 0.4451,
	"step": 274
	},
	{
	"epoch": 0.5159474671669794,
	"grad_norm": 0.38803425431251526,
	"learning_rate": 1.1275452768714444e-05,
	"loss": 0.481,
	"step": 275
	},
	{
	"epoch": 0.5178236397748592,
	"grad_norm": 0.4098987579345703,
	"learning_rate": 1.121037497864163e-05,
	"loss": 0.4644,
	"step": 276
	},
	{
	"epoch": 0.5196998123827392,
	"grad_norm": 0.41665926575660706,
	"learning_rate": 1.1145245123421812e-05,
	"loss": 0.4752,
	"step": 277
	},
	{
	"epoch": 0.5215759849906192,
	"grad_norm": 0.435349702835083,
	"learning_rate": 1.108006600466238e-05,
	"loss": 0.452,
	"step": 278
	},
	{
	"epoch": 0.5234521575984991,
	"grad_norm": 0.4160681962966919,
	"learning_rate": 1.101484042608983e-05,
	"loss": 0.4857,
	"step": 279
	},
	{
	"epoch": 0.525328330206379,
	"grad_norm": 0.38691315054893494,
	"learning_rate": 1.0949571193429162e-05,
	"loss": 0.4847,
	"step": 280
	},
	{
	"epoch": 0.5272045028142589,
	"grad_norm": 0.4164392650127411,
	"learning_rate": 1.088426111428319e-05,
	"loss": 0.4807,
	"step": 281
	},
	{
	"epoch": 0.5290806754221389,
	"grad_norm": 0.4300763010978699,
	"learning_rate": 1.0818912998011766e-05,
	"loss": 0.4534,
	"step": 282
	},
	{
	"epoch": 0.5309568480300187,
	"grad_norm": 0.40870925784111023,
	"learning_rate": 1.0753529655610935e-05,
	"loss": 0.4695,
	"step": 283
	},
	{
	"epoch": 0.5328330206378987,
	"grad_norm": 0.3918152153491974,
	"learning_rate": 1.0688113899592025e-05,
	"loss": 0.454,
	"step": 284
	},
	{
	"epoch": 0.5347091932457786,
	"grad_norm": 0.4441761374473572,
	"learning_rate": 1.0622668543860653e-05,
	"loss": 0.4816,
	"step": 285
	},
	{
	"epoch": 0.5365853658536586,
	"grad_norm": 0.4223552346229553,
	"learning_rate": 1.0557196403595689e-05,
	"loss": 0.4685,
	"step": 286
	},
	{
	"epoch": 0.5384615384615384,
	"grad_norm": 0.42374157905578613,
	"learning_rate": 1.0491700295128163e-05,
	"loss": 0.4749,
	"step": 287
	},
	{
	"epoch": 0.5403377110694184,
	"grad_norm": 0.482738196849823,
	"learning_rate": 1.0426183035820112e-05,
	"loss": 0.4808,
	"step": 288
	},
	{
	"epoch": 0.5422138836772983,
	"grad_norm": 0.4222562313079834,
	"learning_rate": 1.0360647443943392e-05,
	"loss": 0.4537,
	"step": 289
	},
	{
	"epoch": 0.5440900562851783,
	"grad_norm": 0.4363473057746887,
	"learning_rate": 1.0295096338558447e-05,
	"loss": 0.4684,
	"step": 290
	},
	{
	"epoch": 0.5459662288930581,
	"grad_norm": 0.4176522493362427,
	"learning_rate": 1.0229532539393051e-05,
	"loss": 0.4707,
	"step": 291
	},
	{
	"epoch": 0.5478424015009381,
	"grad_norm": 0.41336387395858765,
	"learning_rate": 1.0163958866720996e-05,
	"loss": 0.4852,
	"step": 292
	},
	{
	"epoch": 0.549718574108818,
	"grad_norm": 0.40363791584968567,
	"learning_rate": 1.00983781412408e-05,
	"loss": 0.4735,
	"step": 293
	},
	{
	"epoch": 0.551594746716698,
	"grad_norm": 0.42629796266555786,
	"learning_rate": 1.0032793183954371e-05,
	"loss": 0.487,
	"step": 294
	},
	{
	"epoch": 0.5534709193245778,
	"grad_norm": 0.40003812313079834,
	"learning_rate": 9.967206816045634e-06,
	"loss": 0.4626,
	"step": 295
	},
	{
	"epoch": 0.5553470919324578,
	"grad_norm": 0.3810182809829712,
	"learning_rate": 9.901621858759203e-06,
	"loss": 0.467,
	"step": 296
	},
	{
	"epoch": 0.5572232645403377,
	"grad_norm": 0.4191839098930359,
	"learning_rate": 9.83604113327901e-06,
	"loss": 0.4804,
	"step": 297
	},
	{
	"epoch": 0.5590994371482176,
	"grad_norm": 0.3994486629962921,
	"learning_rate": 9.77046746060695e-06,
	"loss": 0.4788,
	"step": 298
	},
	{
	"epoch": 0.5609756097560976,
	"grad_norm": 0.41478341817855835,
	"learning_rate": 9.704903661441555e-06,
	"loss": 0.4678,
	"step": 299
	},
	{
	"epoch": 0.5628517823639775,
	"grad_norm": 0.3982663154602051,
	"learning_rate": 9.63935255605661e-06,
	"loss": 0.4609,
	"step": 300
	},
	{
	"epoch": 0.5647279549718575,
	"grad_norm": 0.3930562138557434,
	"learning_rate": 9.573816964179891e-06,
	"loss": 0.4724,
	"step": 301
	},
	{
	"epoch": 0.5666041275797373,
	"grad_norm": 0.38638681173324585,
	"learning_rate": 9.50829970487184e-06,
	"loss": 0.4375,
	"step": 302
	},
	{
	"epoch": 0.5684803001876173,
	"grad_norm": 0.3946895897388458,
	"learning_rate": 9.442803596404315e-06,
	"loss": 0.4746,
	"step": 303
	},
	{
	"epoch": 0.5703564727954972,
	"grad_norm": 0.42083805799484253,
	"learning_rate": 9.377331456139352e-06,
	"loss": 0.4894,
	"step": 304
	},
	{
	"epoch": 0.5722326454033771,
	"grad_norm": 0.3850444257259369,
	"learning_rate": 9.311886100407975e-06,
	"loss": 0.4835,
	"step": 305
	},
	{
	"epoch": 0.574108818011257,
	"grad_norm": 0.4170306324958801,
	"learning_rate": 9.246470344389065e-06,
	"loss": 0.4502,
	"step": 306
	},
	{
	"epoch": 0.575984990619137,
	"grad_norm": 0.5081507563591003,
	"learning_rate": 9.181087001988236e-06,
	"loss": 0.456,
	"step": 307
	},
	{
	"epoch": 0.5778611632270169,
	"grad_norm": 0.40114471316337585,
	"learning_rate": 9.115738885716812e-06,
	"loss": 0.4694,
	"step": 308
	},
	{
	"epoch": 0.5797373358348968,
	"grad_norm": 0.3917900025844574,
	"learning_rate": 9.050428806570841e-06,
	"loss": 0.4803,
	"step": 309
	},
	{
	"epoch": 0.5816135084427767,
	"grad_norm": 0.3974637985229492,
	"learning_rate": 8.985159573910173e-06,
	"loss": 0.4665,
	"step": 310
	},
	{
	"epoch": 0.5834896810506567,
	"grad_norm": 0.3936939537525177,
	"learning_rate": 8.919933995337624e-06,
	"loss": 0.4894,
	"step": 311
	},
	{
	"epoch": 0.5853658536585366,
	"grad_norm": 0.3804459869861603,
	"learning_rate": 8.85475487657819e-06,
	"loss": 0.4572,
	"step": 312
	},
	{
	"epoch": 0.5872420262664165,
	"grad_norm": 0.3730376362800598,
	"learning_rate": 8.789625021358371e-06,
	"loss": 0.4665,
	"step": 313
	},
	{
	"epoch": 0.5891181988742964,
	"grad_norm": 0.39923325181007385,
	"learning_rate": 8.724547231285558e-06,
	"loss": 0.4741,
	"step": 314
	},
	{
	"epoch": 0.5909943714821764,
	"grad_norm": 0.37232276797294617,
	"learning_rate": 8.659524305727524e-06,
	"loss": 0.4625,
	"step": 315
	},
	{
	"epoch": 0.5928705440900562,
	"grad_norm": 0.3921473026275635,
	"learning_rate": 8.594559041692004e-06,
	"loss": 0.4591,
	"step": 316
	},
	{
	"epoch": 0.5947467166979362,
	"grad_norm": 0.39849600195884705,
	"learning_rate": 8.529654233706378e-06,
	"loss": 0.4559,
	"step": 317
	},
	{
	"epoch": 0.5966228893058161,
	"grad_norm": 0.39225223660469055,
	"learning_rate": 8.464812673697468e-06,
	"loss": 0.4806,
	"step": 318
	},
	{
	"epoch": 0.5984990619136961,
	"grad_norm": 0.41534101963043213,
	"learning_rate": 8.400037150871452e-06,
	"loss": 0.4573,
	"step": 319
	},
	{
	"epoch": 0.600375234521576,
	"grad_norm": 0.364001989364624,
	"learning_rate": 8.33533045159385e-06,
	"loss": 0.4441,
	"step": 320
	},
	{
	"epoch": 0.6022514071294559,
	"grad_norm": 0.4032931327819824,
	"learning_rate": 8.270695359269699e-06,
	"loss": 0.4841,
	"step": 321
	},
	{
	"epoch": 0.6041275797373359,
	"grad_norm": 0.411393940448761,
	"learning_rate": 8.20613465422381e-06,
	"loss": 0.4595,
	"step": 322
	},
	{
	"epoch": 0.6060037523452158,
	"grad_norm": 0.38068366050720215,
	"learning_rate": 8.141651113581173e-06,
	"loss": 0.4695,
	"step": 323
	},
	{
	"epoch": 0.6078799249530957,
	"grad_norm": 0.40632885694503784,
	"learning_rate": 8.077247511147495e-06,
	"loss": 0.4987,
	"step": 324
	},
	{
	"epoch": 0.6097560975609756,
	"grad_norm": 0.42743155360221863,
	"learning_rate": 8.012926617289879e-06,
	"loss": 0.4571,
	"step": 325
	},
	{
	"epoch": 0.6116322701688556,
	"grad_norm": 0.3742930591106415,
	"learning_rate": 7.948691198817667e-06,
	"loss": 0.4709,
	"step": 326
	},
	{
	"epoch": 0.6135084427767354,
	"grad_norm": 0.38311901688575745,
	"learning_rate": 7.884544018863414e-06,
	"loss": 0.4672,
	"step": 327
	},
	{
	"epoch": 0.6153846153846154,
	"grad_norm": 0.4228517413139343,
	"learning_rate": 7.82048783676403e-06,
	"loss": 0.4779,
	"step": 328
	},
	{
	"epoch": 0.6172607879924953,
	"grad_norm": 0.4097955822944641,
	"learning_rate": 7.756525407942087e-06,
	"loss": 0.459,
	"step": 329
	},
	{
	"epoch": 0.6191369606003753,
	"grad_norm": 0.3821118175983429,
	"learning_rate": 7.692659483787292e-06,
	"loss": 0.4692,
	"step": 330
	},
	{
	"epoch": 0.6210131332082551,
	"grad_norm": 0.36977776885032654,
	"learning_rate": 7.628892811538138e-06,
	"loss": 0.4581,
	"step": 331
	},
	{
	"epoch": 0.6228893058161351,
	"grad_norm": 0.39426207542419434,
	"learning_rate": 7.565228134163719e-06,
	"loss": 0.4632,
	"step": 332
	},
	{
	"epoch": 0.624765478424015,
	"grad_norm": 0.37044960260391235,
	"learning_rate": 7.50166819024576e-06,
	"loss": 0.4805,
	"step": 333
	},
	{
	"epoch": 0.626641651031895,
	"grad_norm": 0.3636074960231781,
	"learning_rate": 7.43821571386079e-06,
	"loss": 0.4649,
	"step": 334
	},
	{
	"epoch": 0.6285178236397748,
	"grad_norm": 0.40089529752731323,
	"learning_rate": 7.374873434462546e-06,
	"loss": 0.4617,
	"step": 335
	},
	{
	"epoch": 0.6303939962476548,
	"grad_norm": 0.3862261474132538,
	"learning_rate": 7.311644076764565e-06,
	"loss": 0.4946,
	"step": 336
	},
	{
	"epoch": 0.6322701688555347,
	"grad_norm": 0.38654249906539917,
	"learning_rate": 7.248530360622975e-06,
	"loss": 0.4445,
	"step": 337
	},
	{
	"epoch": 0.6341463414634146,
	"grad_norm": 0.3740447461605072,
	"learning_rate": 7.1855350009195e-06,
	"loss": 0.4556,
	"step": 338
	},
	{
	"epoch": 0.6360225140712945,
	"grad_norm": 0.40742936730384827,
	"learning_rate": 7.12266070744468e-06,
	"loss": 0.4694,
	"step": 339
	},
	{
	"epoch": 0.6378986866791745,
	"grad_norm": 0.37905779480934143,
	"learning_rate": 7.0599101847813e-06,
	"loss": 0.4759,
	"step": 340
	},
	{
	"epoch": 0.6397748592870544,
	"grad_norm": 0.4220942556858063,
	"learning_rate": 6.9972861321880566e-06,
	"loss": 0.4764,
	"step": 341
	},
	{
	"epoch": 0.6416510318949343,
	"grad_norm": 0.3825709819793701,
	"learning_rate": 6.9347912434834455e-06,
	"loss": 0.4522,
	"step": 342
	},
	{
	"epoch": 0.6435272045028143,
	"grad_norm": 0.40397360920906067,
	"learning_rate": 6.872428206929887e-06,
	"loss": 0.4756,
	"step": 343
	},
	{
	"epoch": 0.6454033771106942,
	"grad_norm": 0.42121949791908264,
	"learning_rate": 6.810199705118081e-06,
	"loss": 0.4468,
	"step": 344
	},
	{
	"epoch": 0.6472795497185742,
	"grad_norm": 0.39659222960472107,
	"learning_rate": 6.748108414851622e-06,
	"loss": 0.4609,
	"step": 345
	},
	{
	"epoch": 0.649155722326454,
	"grad_norm": 0.3952244520187378,
	"learning_rate": 6.6861570070318595e-06,
	"loss": 0.4782,
	"step": 346
	},
	{
	"epoch": 0.651031894934334,
	"grad_norm": 0.399282306432724,
	"learning_rate": 6.624348146542991e-06,
	"loss": 0.4634,
	"step": 347
	},
	{
	"epoch": 0.6529080675422139,
	"grad_norm": 0.38961517810821533,
	"learning_rate": 6.562684492137435e-06,
	"loss": 0.4639,
	"step": 348
	},
	{
	"epoch": 0.6547842401500938,
	"grad_norm": 0.40523359179496765,
	"learning_rate": 6.501168696321476e-06,
	"loss": 0.4539,
	"step": 349
	},
	{
	"epoch": 0.6566604127579737,
	"grad_norm": 0.3791859447956085,
	"learning_rate": 6.439803405241149e-06,
	"loss": 0.4681,
	"step": 350
	},
	{
	"epoch": 0.6585365853658537,
	"grad_norm": 0.41422030329704285,
	"learning_rate": 6.378591258568421e-06,
	"loss": 0.4756,
	"step": 351
	},
	{
	"epoch": 0.6604127579737336,
	"grad_norm": 0.38266244530677795,
	"learning_rate": 6.317534889387638e-06,
	"loss": 0.4744,
	"step": 352
	},
	{
	"epoch": 0.6622889305816135,
	"grad_norm": 0.3789862096309662,
	"learning_rate": 6.2566369240822775e-06,
	"loss": 0.4568,
	"step": 353
	},
	{
	"epoch": 0.6641651031894934,
	"grad_norm": 0.36977142095565796,
	"learning_rate": 6.1958999822219535e-06,
	"loss": 0.446,
	"step": 354
	},
	{
	"epoch": 0.6660412757973734,
	"grad_norm": 0.41515210270881653,
	"learning_rate": 6.13532667644974e-06,
	"loss": 0.4588,
	"step": 355
	},
	{
	"epoch": 0.6679174484052532,
	"grad_norm": 0.3867998421192169,
	"learning_rate": 6.074919612369787e-06,
	"loss": 0.4644,
	"step": 356
	},
	{
	"epoch": 0.6697936210131332,
	"grad_norm": 0.3649778664112091,
	"learning_rate": 6.014681388435243e-06,
	"loss": 0.4827,
	"step": 357
	},
	{
	"epoch": 0.6716697936210131,
	"grad_norm": 0.3807336986064911,
	"learning_rate": 5.95461459583647e-06,
	"loss": 0.4676,
	"step": 358
	},
	{
	"epoch": 0.6735459662288931,
	"grad_norm": 0.39816009998321533,
	"learning_rate": 5.894721818389602e-06,
	"loss": 0.4713,
	"step": 359
	},
	{
	"epoch": 0.6754221388367729,
	"grad_norm": 0.40490055084228516,
	"learning_rate": 5.83500563242537e-06,
	"loss": 0.4609,
	"step": 360
	},
	{
	"epoch": 0.6772983114446529,
	"grad_norm": 0.3870190978050232,
	"learning_rate": 5.775468606678305e-06,
	"loss": 0.4515,
	"step": 361
	},
	{
	"epoch": 0.6791744840525328,
	"grad_norm": 0.3919334411621094,
	"learning_rate": 5.7161133021762315e-06,
	"loss": 0.459,
	"step": 362
	},
	{
	"epoch": 0.6810506566604128,
	"grad_norm": 0.40317437052726746,
	"learning_rate": 5.6569422721301034e-06,
	"loss": 0.4919,
	"step": 363
	},
	{
	"epoch": 0.6829268292682927,
	"grad_norm": 0.4027089774608612,
	"learning_rate": 5.597958061824179e-06,
	"loss": 0.471,
	"step": 364
	},
	{
	"epoch": 0.6848030018761726,
	"grad_norm": 0.3774213492870331,
	"learning_rate": 5.539163208506524e-06,
	"loss": 0.4667,
	"step": 365
	},
	{
	"epoch": 0.6866791744840526,
	"grad_norm": 0.3761104941368103,
	"learning_rate": 5.480560241279891e-06,
	"loss": 0.4737,
	"step": 366
	},
	{
	"epoch": 0.6885553470919324,
	"grad_norm": 0.37898918986320496,
	"learning_rate": 5.422151680992907e-06,
	"loss": 0.4838,
	"step": 367
	},
	{
	"epoch": 0.6904315196998124,
	"grad_norm": 0.3650558590888977,
	"learning_rate": 5.363940040131641e-06,
	"loss": 0.4556,
	"step": 368
	},
	{
	"epoch": 0.6923076923076923,
	"grad_norm": 0.41331934928894043,
	"learning_rate": 5.305927822711539e-06,
	"loss": 0.457,
	"step": 369
	},
	{
	"epoch": 0.6941838649155723,
	"grad_norm": 0.38073617219924927,
	"learning_rate": 5.2481175241697024e-06,
	"loss": 0.4598,
	"step": 370
	},
	{
	"epoch": 0.6960600375234521,
	"grad_norm": 0.40819063782691956,
	"learning_rate": 5.190511631257548e-06,
	"loss": 0.463,
	"step": 371
	},
	{
	"epoch": 0.6979362101313321,
	"grad_norm": 0.40303105115890503,
	"learning_rate": 5.133112621933831e-06,
	"loss": 0.4705,
	"step": 372
	},
	{
	"epoch": 0.699812382739212,
	"grad_norm": 0.3903728723526001,
	"learning_rate": 5.075922965258077e-06,
	"loss": 0.4478,
	"step": 373
	},
	{
	"epoch": 0.701688555347092,
	"grad_norm": 1.0995211601257324,
	"learning_rate": 5.018945121284342e-06,
	"loss": 0.4609,
	"step": 374
	},
	{
	"epoch": 0.7035647279549718,
	"grad_norm": 0.37805426120758057,
	"learning_rate": 4.962181540955412e-06,
	"loss": 0.439,
	"step": 375
	},
	{
	"epoch": 0.7054409005628518,
	"grad_norm": 0.3795923888683319,
	"learning_rate": 4.905634665997371e-06,
	"loss": 0.4805,
	"step": 376
	},
	{
	"epoch": 0.7073170731707317,
	"grad_norm": 0.3871743083000183,
	"learning_rate": 4.849306928814562e-06,
	"loss": 0.4763,
	"step": 377
	},
	{
	"epoch": 0.7091932457786116,
	"grad_norm": 0.3588029444217682,
	"learning_rate": 4.793200752384963e-06,
	"loss": 0.4555,
	"step": 378
	},
	{
	"epoch": 0.7110694183864915,
	"grad_norm": 0.39265263080596924,
	"learning_rate": 4.7373185501559575e-06,
	"loss": 0.464,
	"step": 379
	},
	{
	"epoch": 0.7129455909943715,
	"grad_norm": 0.3756472170352936,
	"learning_rate": 4.681662725940514e-06,
	"loss": 0.4712,
	"step": 380
	},
	{
	"epoch": 0.7148217636022514,
	"grad_norm": 0.38207897543907166,
	"learning_rate": 4.626235673813794e-06,
	"loss": 0.471,
	"step": 381
	},
	{
	"epoch": 0.7166979362101313,
	"grad_norm": 0.45319947600364685,
	"learning_rate": 4.57103977801016e-06,
	"loss": 0.4579,
	"step": 382
	},
	{
	"epoch": 0.7185741088180112,
	"grad_norm": 0.37751689553260803,
	"learning_rate": 4.516077412820622e-06,
	"loss": 0.4725,
	"step": 383
	},
	{
	"epoch": 0.7204502814258912,
	"grad_norm": 0.37137770652770996,
	"learning_rate": 4.461350942490701e-06,
	"loss": 0.4512,
	"step": 384
	},
	{
	"epoch": 0.7223264540337712,
	"grad_norm": 0.37328335642814636,
	"learning_rate": 4.40686272111873e-06,
	"loss": 0.4526,
	"step": 385
	},
	{
	"epoch": 0.724202626641651,
	"grad_norm": 0.4662545919418335,
	"learning_rate": 4.3526150925546005e-06,
	"loss": 0.4715,
	"step": 386
	},
	{
	"epoch": 0.726078799249531,
	"grad_norm": 0.3949073553085327,
	"learning_rate": 4.298610390298926e-06,
	"loss": 0.4645,
	"step": 387
	},
	{
	"epoch": 0.7279549718574109,
	"grad_norm": 0.3606877326965332,
	"learning_rate": 4.244850937402669e-06,
	"loss": 0.4627,
	"step": 388
	},
	{
	"epoch": 0.7298311444652908,
	"grad_norm": 0.3852671682834625,
	"learning_rate": 4.191339046367219e-06,
	"loss": 0.4689,
	"step": 389
	},
	{
	"epoch": 0.7317073170731707,
	"grad_norm": 0.4201098084449768,
	"learning_rate": 4.138077019044911e-06,
	"loss": 0.468,
	"step": 390
	},
	{
	"epoch": 0.7335834896810507,
	"grad_norm": 0.3923519551753998,
	"learning_rate": 4.085067146540015e-06,
	"loss": 0.4599,
	"step": 391
	},
	{
	"epoch": 0.7354596622889306,
	"grad_norm": 0.36301520466804504,
	"learning_rate": 4.032311709110185e-06,
	"loss": 0.455,
	"step": 392
	},
	{
	"epoch": 0.7373358348968105,
	"grad_norm": 0.3901977241039276,
	"learning_rate": 3.979812976068362e-06,
	"loss": 0.4511,
	"step": 393
	},
	{
	"epoch": 0.7392120075046904,
	"grad_norm": 0.3626657724380493,
	"learning_rate": 3.927573205685167e-06,
	"loss": 0.4614,
	"step": 394
	},
	{
	"epoch": 0.7410881801125704,
	"grad_norm": 0.3936521112918854,
	"learning_rate": 3.875594645091755e-06,
	"loss": 0.4673,
	"step": 395
	},
	{
	"epoch": 0.7429643527204502,
	"grad_norm": 0.38001248240470886,
	"learning_rate": 3.823879530183155e-06,
	"loss": 0.48,
	"step": 396
	},
	{
	"epoch": 0.7448405253283302,
	"grad_norm": 0.37876972556114197,
	"learning_rate": 3.7724300855220906e-06,
	"loss": 0.4646,
	"step": 397
	},
	{
	"epoch": 0.7467166979362101,
	"grad_norm": 0.41002485156059265,
	"learning_rate": 3.7212485242432885e-06,
	"loss": 0.4622,
	"step": 398
	},
	{
	"epoch": 0.7485928705440901,
	"grad_norm": 0.3855739235877991,
	"learning_rate": 3.670337047958288e-06,
	"loss": 0.4426,
	"step": 399
	},
	{
	"epoch": 0.7504690431519699,
	"grad_norm": 0.3790871202945709,
	"learning_rate": 3.619697846660719e-06,
	"loss": 0.4592,
	"step": 400
	},
	{
	"epoch": 0.7523452157598499,
	"grad_norm": 0.3793615698814392,
	"learning_rate": 3.5693330986321096e-06,
	"loss": 0.4674,
	"step": 401
	},
	{
	"epoch": 0.7542213883677298,
	"grad_norm": 0.38816037774086,
	"learning_rate": 3.5192449703481877e-06,
	"loss": 0.4758,
	"step": 402
	},
	{
	"epoch": 0.7560975609756098,
	"grad_norm": 0.380281001329422,
	"learning_rate": 3.4694356163856814e-06,
	"loss": 0.4584,
	"step": 403
	},
	{
	"epoch": 0.7579737335834896,
	"grad_norm": 0.3708191215991974,
	"learning_rate": 3.419907179329641e-06,
	"loss": 0.4686,
	"step": 404
	},
	{
	"epoch": 0.7598499061913696,
	"grad_norm": 0.36141806840896606,
	"learning_rate": 3.3706617896812775e-06,
	"loss": 0.4498,
	"step": 405
	},
	{
	"epoch": 0.7617260787992496,
	"grad_norm": 0.35164591670036316,
	"learning_rate": 3.3217015657663145e-06,
	"loss": 0.455,
	"step": 406
	},
	{
	"epoch": 0.7636022514071295,
	"grad_norm": 0.3529546856880188,
	"learning_rate": 3.273028613643866e-06,
	"loss": 0.453,
	"step": 407
	},
	{
	"epoch": 0.7654784240150094,
	"grad_norm": 0.38521793484687805,
	"learning_rate": 3.2246450270158425e-06,
	"loss": 0.4959,
	"step": 408
	},
	{
	"epoch": 0.7673545966228893,
	"grad_norm": 0.38005560636520386,
	"learning_rate": 3.1765528871368946e-06,
	"loss": 0.426,
	"step": 409
	},
	{
	"epoch": 0.7692307692307693,
	"grad_norm": 0.42967918515205383,
	"learning_rate": 3.128754262724879e-06,
	"loss": 0.4792,
	"step": 410
	},
	{
	"epoch": 0.7711069418386491,
	"grad_norm": 0.3669396638870239,
	"learning_rate": 3.0812512098718717e-06,
	"loss": 0.4602,
	"step": 411
	},
	{
	"epoch": 0.7729831144465291,
	"grad_norm": 0.3635663092136383,
	"learning_rate": 3.034045771955736e-06,
	"loss": 0.4546,
	"step": 412
	},
	{
	"epoch": 0.774859287054409,
	"grad_norm": 0.3650107979774475,
	"learning_rate": 2.9871399795522062e-06,
	"loss": 0.4647,
	"step": 413
	},
	{
	"epoch": 0.776735459662289,
	"grad_norm": 0.36175158619880676,
	"learning_rate": 2.9405358503475536e-06,
	"loss": 0.4506,
	"step": 414
	},
	{
	"epoch": 0.7786116322701688,
	"grad_norm": 0.3650931417942047,
	"learning_rate": 2.8942353890517894e-06,
	"loss": 0.4521,
	"step": 415
	},
	{
	"epoch": 0.7804878048780488,
	"grad_norm": 0.3675496578216553,
	"learning_rate": 2.848240587312433e-06,
	"loss": 0.4655,
	"step": 416
	},
	{
	"epoch": 0.7823639774859287,
	"grad_norm": 0.35920068621635437,
	"learning_rate": 2.80255342362884e-06,
	"loss": 0.4606,
	"step": 417
	},
	{
	"epoch": 0.7842401500938087,
	"grad_norm": 0.3604222238063812,
	"learning_rate": 2.7571758632670897e-06,
	"loss": 0.464,
	"step": 418
	},
	{
	"epoch": 0.7861163227016885,
	"grad_norm": 0.36000943183898926,
	"learning_rate": 2.7121098581754635e-06,
	"loss": 0.4562,
	"step": 419
	},
	{
	"epoch": 0.7879924953095685,
	"grad_norm": 0.3869664669036865,
	"learning_rate": 2.667357346900459e-06,
	"loss": 0.4606,
	"step": 420
	},
	{
	"epoch": 0.7898686679174484,
	"grad_norm": 0.35324493050575256,
	"learning_rate": 2.622920254503416e-06,
	"loss": 0.4647,
	"step": 421
	},
	{
	"epoch": 0.7917448405253283,
	"grad_norm": 0.3965626060962677,
	"learning_rate": 2.5788004924777065e-06,
	"loss": 0.4719,
	"step": 422
	},
	{
	"epoch": 0.7936210131332082,
	"grad_norm": 0.35837239027023315,
	"learning_rate": 2.5349999586665077e-06,
	"loss": 0.4597,
	"step": 423
	},
	{
	"epoch": 0.7954971857410882,
	"grad_norm": 0.3734018802642822,
	"learning_rate": 2.491520537181167e-06,
	"loss": 0.4627,
	"step": 424
	},
	{
	"epoch": 0.797373358348968,
	"grad_norm": 0.38226038217544556,
	"learning_rate": 2.448364098320154e-06,
	"loss": 0.4462,
	"step": 425
	},
	{
	"epoch": 0.799249530956848,
	"grad_norm": 0.34819310903549194,
	"learning_rate": 2.405532498488612e-06,
	"loss": 0.4405,
	"step": 426
	},
	{
	"epoch": 0.801125703564728,
	"grad_norm": 0.3812004625797272,
	"learning_rate": 2.3630275801184975e-06,
	"loss": 0.4623,
	"step": 427
	},
	{
	"epoch": 0.8030018761726079,
	"grad_norm": 0.33405259251594543,
	"learning_rate": 2.3208511715893334e-06,
	"loss": 0.4702,
	"step": 428
	},
	{
	"epoch": 0.8048780487804879,
	"grad_norm": 0.3457287847995758,
	"learning_rate": 2.2790050871495518e-06,
	"loss": 0.4543,
	"step": 429
	},
	{
	"epoch": 0.8067542213883677,
	"grad_norm": 0.37848466634750366,
	"learning_rate": 2.2374911268384614e-06,
	"loss": 0.4747,
	"step": 430
	},
	{
	"epoch": 0.8086303939962477,
	"grad_norm": 0.37516096234321594,
	"learning_rate": 2.196311076408808e-06,
	"loss": 0.4499,
	"step": 431
	},
	{
	"epoch": 0.8105065666041276,
	"grad_norm": 0.35740143060684204,
	"learning_rate": 2.1554667072499747e-06,
	"loss": 0.448,
	"step": 432
	},
	{
	"epoch": 0.8123827392120075,
	"grad_norm": 0.37683647871017456,
	"learning_rate": 2.114959776311767e-06,
	"loss": 0.4595,
	"step": 433
	},
	{
	"epoch": 0.8142589118198874,
	"grad_norm": 0.36013343930244446,
	"learning_rate": 2.074792026028841e-06,
	"loss": 0.457,
	"step": 434
	},
	{
	"epoch": 0.8161350844277674,
	"grad_norm": 0.37009820342063904,
	"learning_rate": 2.0349651842457606e-06,
	"loss": 0.4739,
	"step": 435
	},
	{
	"epoch": 0.8180112570356473,
	"grad_norm": 0.3644636571407318,
	"learning_rate": 1.995480964142663e-06,
	"loss": 0.4551,
	"step": 436
	},
	{
	"epoch": 0.8198874296435272,
	"grad_norm": 0.3416192829608917,
	"learning_rate": 1.9563410641615666e-06,
	"loss": 0.4597,
	"step": 437
	},
	{
	"epoch": 0.8217636022514071,
	"grad_norm": 0.3656933009624481,
	"learning_rate": 1.9175471679333146e-06,
	"loss": 0.4195,
	"step": 438
	},
	{
	"epoch": 0.8236397748592871,
	"grad_norm": 0.4020621180534363,
	"learning_rate": 1.8791009442051545e-06,
	"loss": 0.4751,
	"step": 439
	},
	{
	"epoch": 0.8255159474671669,
	"grad_norm": 0.3509070873260498,
	"learning_rate": 1.8410040467689472e-06,
	"loss": 0.4418,
	"step": 440
	},
	{
	"epoch": 0.8273921200750469,
	"grad_norm": 0.4642477333545685,
	"learning_rate": 1.8032581143900342e-06,
	"loss": 0.4648,
	"step": 441
	},
	{
	"epoch": 0.8292682926829268,
	"grad_norm": 0.3921687602996826,
	"learning_rate": 1.7658647707367438e-06,
	"loss": 0.4616,
	"step": 442
	},
	{
	"epoch": 0.8311444652908068,
	"grad_norm": 0.33472052216529846,
	"learning_rate": 1.7288256243105484e-06,
	"loss": 0.449,
	"step": 443
	},
	{
	"epoch": 0.8330206378986866,
	"grad_norm": 0.3562254011631012,
	"learning_rate": 1.6921422683768707e-06,
	"loss": 0.4402,
	"step": 444
	},
	{
	"epoch": 0.8348968105065666,
	"grad_norm": 0.38186293840408325,
	"learning_rate": 1.6558162808965495e-06,
	"loss": 0.454,
	"step": 445
	},
	{
	"epoch": 0.8367729831144465,
	"grad_norm": 0.35252490639686584,
	"learning_rate": 1.6198492244579723e-06,
	"loss": 0.4611,
	"step": 446
	},
	{
	"epoch": 0.8386491557223265,
	"grad_norm": 0.3606928288936615,
	"learning_rate": 1.5842426462098392e-06,
	"loss": 0.4572,
	"step": 447
	},
	{
	"epoch": 0.8405253283302064,
	"grad_norm": 0.3576737940311432,
	"learning_rate": 1.54899807779463e-06,
	"loss": 0.4785,
	"step": 448
	},
	{
	"epoch": 0.8424015009380863,
	"grad_norm": 0.35132601857185364,
	"learning_rate": 1.5141170352827094e-06,
	"loss": 0.4481,
	"step": 449
	},
	{
	"epoch": 0.8442776735459663,
	"grad_norm": 0.3655112683773041,
	"learning_rate": 1.479601019107113e-06,
	"loss": 0.4668,
	"step": 450
	},
	{
	"epoch": 0.8461538461538461,
	"grad_norm": 0.3495786190032959,
	"learning_rate": 1.4454515139990122e-06,
	"loss": 0.4671,
	"step": 451
	},
	{
	"epoch": 0.8480300187617261,
	"grad_norm": 0.36569535732269287,
	"learning_rate": 1.4116699889238372e-06,
	"loss": 0.4635,
	"step": 452
	},
	{
	"epoch": 0.849906191369606,
	"grad_norm": 0.34859490394592285,
	"learning_rate": 1.378257897018096e-06,
	"loss": 0.4679,
	"step": 453
	},
	{
	"epoch": 0.851782363977486,
	"grad_norm": 0.3567720055580139,
	"learning_rate": 1.3452166755268637e-06,
	"loss": 0.454,
	"step": 454
	},
	{
	"epoch": 0.8536585365853658,
	"grad_norm": 0.3588789999485016,
	"learning_rate": 1.312547745741959e-06,
	"loss": 0.4466,
	"step": 455
	},
	{
	"epoch": 0.8555347091932458,
	"grad_norm": 0.35184621810913086,
	"learning_rate": 1.2802525129408039e-06,
	"loss": 0.4478,
	"step": 456
	},
	{
	"epoch": 0.8574108818011257,
	"grad_norm": 0.3764590620994568,
	"learning_rate": 1.2483323663259805e-06,
	"loss": 0.4599,
	"step": 457
	},
	{
	"epoch": 0.8592870544090057,
	"grad_norm": 0.36193057894706726,
	"learning_rate": 1.2167886789654659e-06,
	"loss": 0.4555,
	"step": 458
	},
	{
	"epoch": 0.8611632270168855,
	"grad_norm": 0.35677453875541687,
	"learning_rate": 1.1856228077335775e-06,
	"loss": 0.4716,
	"step": 459
	},
	{
	"epoch": 0.8630393996247655,
	"grad_norm": 0.33563223481178284,
	"learning_rate": 1.154836093252597e-06,
	"loss": 0.4664,
	"step": 460
	},
	{
	"epoch": 0.8649155722326454,
	"grad_norm": 0.3576035499572754,
	"learning_rate": 1.1244298598351078e-06,
	"loss": 0.467,
	"step": 461
	},
	{
	"epoch": 0.8667917448405253,
	"grad_norm": 0.3751625120639801,
	"learning_rate": 1.094405415427029e-06,
	"loss": 0.447,
	"step": 462
	},
	{
	"epoch": 0.8686679174484052,
	"grad_norm": 0.348402202129364,
	"learning_rate": 1.064764051551349e-06,
	"loss": 0.4653,
	"step": 463
	},
	{
	"epoch": 0.8705440900562852,
	"grad_norm": 0.3535415530204773,
	"learning_rate": 1.035507043252577e-06,
	"loss": 0.4627,
	"step": 464
	},
	{
	"epoch": 0.8724202626641651,
	"grad_norm": 0.35580676794052124,
	"learning_rate": 1.0066356490418871e-06,
	"loss": 0.4415,
	"step": 465
	},
	{
	"epoch": 0.874296435272045,
	"grad_norm": 0.36892953515052795,
	"learning_rate": 9.781511108429908e-07,
	"loss": 0.4596,
	"step": 466
	},
	{
	"epoch": 0.8761726078799249,
	"grad_norm": 0.388686865568161,
	"learning_rate": 9.500546539387101e-07,
	"loss": 0.4672,
	"step": 467
	},
	{
	"epoch": 0.8780487804878049,
	"grad_norm": 0.3477744162082672,
	"learning_rate": 9.223474869182691e-07,
	"loss": 0.4467,
	"step": 468
	},
	{
	"epoch": 0.8799249530956847,
	"grad_norm": 0.5885300040245056,
	"learning_rate": 8.950308016253129e-07,
	"loss": 0.4668,
	"step": 469
	},
	{
	"epoch": 0.8818011257035647,
	"grad_norm": 0.3635028600692749,
	"learning_rate": 8.681057731066322e-07,
	"loss": 0.4509,
	"step": 470
	},
	{
	"epoch": 0.8836772983114447,
	"grad_norm": 0.34183740615844727,
	"learning_rate": 8.415735595616204e-07,
	"loss": 0.4445,
	"step": 471
	},
	{
	"epoch": 0.8855534709193246,
	"grad_norm": 0.38252541422843933,
	"learning_rate": 8.154353022924577e-07,
	"loss": 0.4486,
	"step": 472
	},
	{
	"epoch": 0.8874296435272045,
	"grad_norm": 0.3649146854877472,
	"learning_rate": 7.896921256550094e-07,
	"loss": 0.4594,
	"step": 473
	},
	{
	"epoch": 0.8893058161350844,
	"grad_norm": 0.3455508053302765,
	"learning_rate": 7.643451370104649e-07,
	"loss": 0.47,
	"step": 474
	},
	{
	"epoch": 0.8911819887429644,
	"grad_norm": 0.34994518756866455,
	"learning_rate": 7.393954266777048e-07,
	"loss": 0.4577,
	"step": 475
	},
	{
	"epoch": 0.8930581613508443,
	"grad_norm": 0.3708822429180145,
	"learning_rate": 7.148440678863921e-07,
	"loss": 0.4603,
	"step": 476
	},
	{
	"epoch": 0.8949343339587242,
	"grad_norm": 0.34789973497390747,
	"learning_rate": 6.906921167308212e-07,
	"loss": 0.4649,
	"step": 477
	},
	{
	"epoch": 0.8968105065666041,
	"grad_norm": 0.34732475876808167,
	"learning_rate": 6.669406121244748e-07,
	"loss": 0.4563,
	"step": 478
	},
	{
	"epoch": 0.8986866791744841,
	"grad_norm": 0.34246909618377686,
	"learning_rate": 6.435905757553463e-07,
	"loss": 0.4637,
	"step": 479
	},
	{
	"epoch": 0.900562851782364,
	"grad_norm": 0.35301247239112854,
	"learning_rate": 6.206430120419804e-07,
	"loss": 0.4415,
	"step": 480
	},
	{
	"epoch": 0.9024390243902439,
	"grad_norm": 0.34336552023887634,
	"learning_rate": 5.980989080902721e-07,
	"loss": 0.4515,
	"step": 481
	},
	{
	"epoch": 0.9043151969981238,
	"grad_norm": 0.34129807353019714,
	"learning_rate": 5.759592336510078e-07,
	"loss": 0.4414,
	"step": 482
	},
	{
	"epoch": 0.9061913696060038,
	"grad_norm": 0.34144312143325806,
	"learning_rate": 5.542249410781486e-07,
	"loss": 0.4503,
	"step": 483
	},
	{
	"epoch": 0.9080675422138836,
	"grad_norm": 0.3457910120487213,
	"learning_rate": 5.328969652878624e-07,
	"loss": 0.4474,
	"step": 484
	},
	{
	"epoch": 0.9099437148217636,
	"grad_norm": 0.35740694403648376,
	"learning_rate": 5.119762237183079e-07,
	"loss": 0.4555,
	"step": 485
	},
	{
	"epoch": 0.9118198874296435,
	"grad_norm": 0.3321978747844696,
	"learning_rate": 4.914636162901799e-07,
	"loss": 0.4427,
	"step": 486
	},
	{
	"epoch": 0.9136960600375235,
	"grad_norm": 0.3253515064716339,
	"learning_rate": 4.7136002536798176e-07,
	"loss": 0.4599,
	"step": 487
	},
	{
	"epoch": 0.9155722326454033,
	"grad_norm": 0.36756089329719543,
	"learning_rate": 4.516663157220846e-07,
	"loss": 0.4788,
	"step": 488
	},
	{
	"epoch": 0.9174484052532833,
	"grad_norm": 0.33447879552841187,
	"learning_rate": 4.323833344915185e-07,
	"loss": 0.4578,
	"step": 489
	},
	{
	"epoch": 0.9193245778611632,
	"grad_norm": 0.3380565047264099,
	"learning_rate": 4.135119111475383e-07,
	"loss": 0.4426,
	"step": 490
	},
	{
	"epoch": 0.9212007504690432,
	"grad_norm": 0.3644172251224518,
	"learning_rate": 3.9505285745794155e-07,
	"loss": 0.4602,
	"step": 491
	},
	{
	"epoch": 0.9230769230769231,
	"grad_norm": 0.358119398355484,
	"learning_rate": 3.770069674521515e-07,
	"loss": 0.445,
	"step": 492
	},
	{
	"epoch": 0.924953095684803,
	"grad_norm": 0.34130486845970154,
	"learning_rate": 3.5937501738705584e-07,
	"loss": 0.4582,
	"step": 493
	},
	{
	"epoch": 0.926829268292683,
	"grad_norm": 0.33966928720474243,
	"learning_rate": 3.421577657136221e-07,
	"loss": 0.4442,
	"step": 494
	},
	{
	"epoch": 0.9287054409005628,
	"grad_norm": 0.360805869102478,
	"learning_rate": 3.2535595304426735e-07,
	"loss": 0.467,
	"step": 495
	},
	{
	"epoch": 0.9305816135084428,
	"grad_norm": 0.33938154578208923,
	"learning_rate": 3.089703021210033e-07,
	"loss": 0.4574,
	"step": 496
	},
	{
	"epoch": 0.9324577861163227,
	"grad_norm": 0.3522200882434845,
	"learning_rate": 2.9300151778434593e-07,
	"loss": 0.4374,
	"step": 497
	},
	{
	"epoch": 0.9343339587242027,
	"grad_norm": 0.34403195977211,
	"learning_rate": 2.7745028694299513e-07,
	"loss": 0.4428,
	"step": 498
	},
	{
	"epoch": 0.9362101313320825,
	"grad_norm": 0.33853229880332947,
	"learning_rate": 2.6231727854428845e-07,
	"loss": 0.4576,
	"step": 499
	},
	{
	"epoch": 0.9380863039399625,
	"grad_norm": 0.34455257654190063,
	"learning_rate": 2.4760314354542737e-07,
	"loss": 0.444,
	"step": 500
	},
	{
	"epoch": 0.9399624765478424,
	"grad_norm": 0.3327702283859253,
	"learning_rate": 2.333085148854708e-07,
	"loss": 0.459,
	"step": 501
	},
	{
	"epoch": 0.9418386491557224,
	"grad_norm": 0.3427714407444,
	"learning_rate": 2.1943400745811473e-07,
	"loss": 0.4542,
	"step": 502
	},
	{
	"epoch": 0.9437148217636022,
	"grad_norm": 0.3703705966472626,
	"learning_rate": 2.059802180852377e-07,
	"loss": 0.4494,
	"step": 503
	},
	{
	"epoch": 0.9455909943714822,
	"grad_norm": 0.3655228018760681,
	"learning_rate": 1.9294772549122932e-07,
	"loss": 0.4553,
	"step": 504
	},
	{
	"epoch": 0.9474671669793621,
	"grad_norm": 0.34462401270866394,
	"learning_rate": 1.8033709027809655e-07,
	"loss": 0.4447,
	"step": 505
	},
	{
	"epoch": 0.949343339587242,
	"grad_norm": 0.3382973372936249,
	"learning_rate": 1.6814885490135103e-07,
	"loss": 0.4528,
	"step": 506
	},
	{
	"epoch": 0.9512195121951219,
	"grad_norm": 0.34772956371307373,
	"learning_rate": 1.5638354364666986e-07,
	"loss": 0.4587,
	"step": 507
	},
	{
	"epoch": 0.9530956848030019,
	"grad_norm": 0.34169095754623413,
	"learning_rate": 1.4504166260734698e-07,
	"loss": 0.4675,
	"step": 508
	},
	{
	"epoch": 0.9549718574108818,
	"grad_norm": 0.3375537395477295,
	"learning_rate": 1.3412369966252393e-07,
	"loss": 0.4462,
	"step": 509
	},
	{
	"epoch": 0.9568480300187617,
	"grad_norm": 0.3615083694458008,
	"learning_rate": 1.236301244561977e-07,
	"loss": 0.4585,
	"step": 510
	},
	{
	"epoch": 0.9587242026266416,
	"grad_norm": 0.3645174205303192,
	"learning_rate": 1.1356138837702702e-07,
	"loss": 0.4589,
	"step": 511
	},
	{
	"epoch": 0.9606003752345216,
	"grad_norm": 0.34688958525657654,
	"learning_rate": 1.039179245389077e-07,
	"loss": 0.4526,
	"step": 512
	},
	{
	"epoch": 0.9624765478424016,
	"grad_norm": 0.36116132140159607,
	"learning_rate": 9.470014776234881e-08,
	"loss": 0.4696,
	"step": 513
	},
	{
	"epoch": 0.9643527204502814,
	"grad_norm": 0.3596712052822113,
	"learning_rate": 8.590845455662133e-08,
	"loss": 0.4604,
	"step": 514
	},
	{
	"epoch": 0.9662288930581614,
	"grad_norm": 0.33651649951934814,
	"learning_rate": 7.754322310270846e-08,
	"loss": 0.477,
	"step": 515
	},
	{
	"epoch": 0.9681050656660413,
	"grad_norm": 0.3588255047798157,
	"learning_rate": 6.960481323703639e-08,
	"loss": 0.4392,
	"step": 516
	},
	{
	"epoch": 0.9699812382739212,
	"grad_norm": 0.37312379479408264,
	"learning_rate": 6.20935664359934e-08,
	"loss": 0.4527,
	"step": 517
	},
	{
	"epoch": 0.9718574108818011,
	"grad_norm": 0.4521438181400299,
	"learning_rate": 5.500980580124271e-08,
	"loss": 0.4626,
	"step": 518
	},
	{
	"epoch": 0.9737335834896811,
	"grad_norm": 0.34084129333496094,
	"learning_rate": 4.8353836045824666e-08,
	"loss": 0.4549,
	"step": 519
	},
	{
	"epoch": 0.975609756097561,
	"grad_norm": 0.3634425103664398,
	"learning_rate": 4.212594348104615e-08,
	"loss": 0.4578,
	"step": 520
	},
	{
	"epoch": 0.9774859287054409,
	"grad_norm": 0.3438330590724945,
	"learning_rate": 3.632639600416932e-08,
	"loss": 0.4671,
	"step": 521
	},
	{
	"epoch": 0.9793621013133208,
	"grad_norm": 0.3470819294452667,
	"learning_rate": 3.095544308688414e-08,
	"loss": 0.4696,
	"step": 522
	},
	{
	"epoch": 0.9812382739212008,
	"grad_norm": 0.37453538179397583,
	"learning_rate": 2.6013315764580326e-08,
	"loss": 0.4515,
	"step": 523
	},
	{
	"epoch": 0.9831144465290806,
	"grad_norm": 0.34173503518104553,
	"learning_rate": 2.1500226626404164e-08,
	"loss": 0.4752,
	"step": 524
	},
	{
	"epoch": 0.9849906191369606,
	"grad_norm": 0.3387591242790222,
	"learning_rate": 1.741636980612027e-08,
	"loss": 0.4478,
	"step": 525
	},
	{
	"epoch": 0.9868667917448405,
	"grad_norm": 0.3569246530532837,
	"learning_rate": 1.376192097375495e-08,
	"loss": 0.4673,
	"step": 526
	},
	{
	"epoch": 0.9887429643527205,
	"grad_norm": 0.3663165867328644,
	"learning_rate": 1.0537037328044453e-08,
	"loss": 0.457,
	"step": 527
	},
	{
	"epoch": 0.9906191369606003,
	"grad_norm": 0.34201619029045105,
	"learning_rate": 7.741857589668167e-09,
	"loss": 0.47,
	"step": 528
	},
	{
	"epoch": 0.9924953095684803,
	"grad_norm": 0.354859858751297,
	"learning_rate": 5.376501995286721e-09,
	"loss": 0.458,
	"step": 529
	},
	{
	"epoch": 0.9943714821763602,
	"grad_norm": 0.3580581247806549,
	"learning_rate": 3.4410722923650108e-09,
	"loss": 0.4543,
	"step": 530
	},
	{
	"epoch": 0.9962476547842402,
	"grad_norm": 0.351889431476593,
	"learning_rate": 1.9356517347990377e-09,
	"loss": 0.462,
	"step": 531
	},
	{
	"epoch": 0.99812382739212,
	"grad_norm": 0.35378509759902954,
	"learning_rate": 8.603050793332124e-10,
	"loss": 0.4505,
	"step": 532
	},
	{
	"epoch": 1.0,
	"grad_norm": 0.35387977957725525,
	"learning_rate": 2.1507858277258764e-10,
	"loss": 0.4469,
	"step": 533
	}
	],
	"logging_steps": 1,
	"max_steps": 533,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 1,
	"save_steps": 50,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 2.465935052576391e+19,
	"train_batch_size": 2,
	"trial_name": null,
	"trial_params": null
	}