Instructions to use FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth")
model = AutoModelForCausalLM.from_pretrained("FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth

SGLang

How to use FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth",
    max_seq_length=2048,
)

Docker Model Runner
How to use FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth with Docker Model Runner:
```
docker model run hf.co/FormlessAI/Qwen2.5-3B-Instruct-Code-Unsloth
```

Qwen2.5-3B-Instruct-Code-Unsloth / trainer_state.json

FormlessAI

Upload folder using huggingface_hub

8dd66e1 verified 5 days ago

Raw

History Blame Contribute Delete

87.8 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 2.0,
	"eval_steps": 30,
	"global_step": 294,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.006837606837606838,
	"grad_norm": 32.511837005615234,
	"learning_rate": 0.0,
	"loss": 1.2298,
	"num_input_tokens_seen": 35008,
	"step": 1,
	"train_runtime": 8.0497,
	"train_tokens_per_second": 4348.976
	},
	{
	"epoch": 0.013675213675213675,
	"grad_norm": 28.007068634033203,
	"learning_rate": 4.000000000000001e-06,
	"loss": 0.9948,
	"num_input_tokens_seen": 83776,
	"step": 2,
	"train_runtime": 12.057,
	"train_tokens_per_second": 6948.356
	},
	{
	"epoch": 0.020512820512820513,
	"grad_norm": 30.101133346557617,
	"learning_rate": 8.000000000000001e-06,
	"loss": 1.0103,
	"num_input_tokens_seen": 123392,
	"step": 3,
	"train_runtime": 15.3107,
	"train_tokens_per_second": 8059.208
	},
	{
	"epoch": 0.02735042735042735,
	"grad_norm": 19.826982498168945,
	"learning_rate": 1.2e-05,
	"loss": 0.7465,
	"num_input_tokens_seen": 163904,
	"step": 4,
	"train_runtime": 18.5797,
	"train_tokens_per_second": 8821.648
	},
	{
	"epoch": 0.03418803418803419,
	"grad_norm": 5.2462077140808105,
	"learning_rate": 1.6000000000000003e-05,
	"loss": 0.4954,
	"num_input_tokens_seen": 204512,
	"step": 5,
	"train_runtime": 21.9294,
	"train_tokens_per_second": 9325.92
	},
	{
	"epoch": 0.041025641025641026,
	"grad_norm": 6.633277893066406,
	"learning_rate": 2e-05,
	"loss": 0.5549,
	"num_input_tokens_seen": 241824,
	"step": 6,
	"train_runtime": 25.0124,
	"train_tokens_per_second": 9668.169
	},
	{
	"epoch": 0.04786324786324787,
	"grad_norm": 3.92435884475708,
	"learning_rate": 1.9930795847750867e-05,
	"loss": 0.447,
	"num_input_tokens_seen": 291392,
	"step": 7,
	"train_runtime": 28.9989,
	"train_tokens_per_second": 10048.39
	},
	{
	"epoch": 0.0547008547008547,
	"grad_norm": 2.5249392986297607,
	"learning_rate": 1.9861591695501733e-05,
	"loss": 0.5641,
	"num_input_tokens_seen": 325088,
	"step": 8,
	"train_runtime": 31.8438,
	"train_tokens_per_second": 10208.84
	},
	{
	"epoch": 0.06153846153846154,
	"grad_norm": 1.8566827774047852,
	"learning_rate": 1.9792387543252595e-05,
	"loss": 0.4856,
	"num_input_tokens_seen": 357280,
	"step": 9,
	"train_runtime": 34.5322,
	"train_tokens_per_second": 10346.275
	},
	{
	"epoch": 0.06837606837606838,
	"grad_norm": 2.148573160171509,
	"learning_rate": 1.972318339100346e-05,
	"loss": 0.5044,
	"num_input_tokens_seen": 397440,
	"step": 10,
	"train_runtime": 37.8542,
	"train_tokens_per_second": 10499.228
	},
	{
	"epoch": 0.07521367521367521,
	"grad_norm": 2.142800807952881,
	"learning_rate": 1.9653979238754327e-05,
	"loss": 0.464,
	"num_input_tokens_seen": 446592,
	"step": 11,
	"train_runtime": 44.1229,
	"train_tokens_per_second": 10121.553
	},
	{
	"epoch": 0.08205128205128205,
	"grad_norm": 1.9147850275039673,
	"learning_rate": 1.9584775086505192e-05,
	"loss": 0.4892,
	"num_input_tokens_seen": 477888,
	"step": 12,
	"train_runtime": 46.7615,
	"train_tokens_per_second": 10219.683
	},
	{
	"epoch": 0.08888888888888889,
	"grad_norm": 1.8965516090393066,
	"learning_rate": 1.9515570934256058e-05,
	"loss": 0.4817,
	"num_input_tokens_seen": 517056,
	"step": 13,
	"train_runtime": 49.9951,
	"train_tokens_per_second": 10342.129
	},
	{
	"epoch": 0.09572649572649573,
	"grad_norm": 1.7629674673080444,
	"learning_rate": 1.9446366782006923e-05,
	"loss": 0.516,
	"num_input_tokens_seen": 551104,
	"step": 14,
	"train_runtime": 52.8297,
	"train_tokens_per_second": 10431.717
	},
	{
	"epoch": 0.10256410256410256,
	"grad_norm": 1.6642876863479614,
	"learning_rate": 1.9377162629757786e-05,
	"loss": 0.455,
	"num_input_tokens_seen": 587104,
	"step": 15,
	"train_runtime": 55.844,
	"train_tokens_per_second": 10513.294
	},
	{
	"epoch": 0.1094017094017094,
	"grad_norm": 1.685402274131775,
	"learning_rate": 1.930795847750865e-05,
	"loss": 0.4607,
	"num_input_tokens_seen": 624800,
	"step": 16,
	"train_runtime": 59.0018,
	"train_tokens_per_second": 10589.5
	},
	{
	"epoch": 0.11623931623931624,
	"grad_norm": 1.7894681692123413,
	"learning_rate": 1.9238754325259517e-05,
	"loss": 0.4795,
	"num_input_tokens_seen": 667424,
	"step": 17,
	"train_runtime": 62.5071,
	"train_tokens_per_second": 10677.567
	},
	{
	"epoch": 0.12307692307692308,
	"grad_norm": 1.5134732723236084,
	"learning_rate": 1.9169550173010383e-05,
	"loss": 0.4488,
	"num_input_tokens_seen": 705056,
	"step": 18,
	"train_runtime": 65.654,
	"train_tokens_per_second": 10738.968
	},
	{
	"epoch": 0.12991452991452992,
	"grad_norm": 1.671798825263977,
	"learning_rate": 1.910034602076125e-05,
	"loss": 0.4473,
	"num_input_tokens_seen": 735200,
	"step": 19,
	"train_runtime": 68.188,
	"train_tokens_per_second": 10781.957
	},
	{
	"epoch": 0.13675213675213677,
	"grad_norm": 1.7231043577194214,
	"learning_rate": 1.9031141868512114e-05,
	"loss": 0.4886,
	"num_input_tokens_seen": 769856,
	"step": 20,
	"train_runtime": 71.1292,
	"train_tokens_per_second": 10823.354
	},
	{
	"epoch": 0.14358974358974358,
	"grad_norm": 1.4723550081253052,
	"learning_rate": 1.8961937716262976e-05,
	"loss": 0.3481,
	"num_input_tokens_seen": 846016,
	"step": 21,
	"train_runtime": 81.9017,
	"train_tokens_per_second": 10329.653
	},
	{
	"epoch": 0.15042735042735042,
	"grad_norm": 1.5541292428970337,
	"learning_rate": 1.8892733564013842e-05,
	"loss": 0.4992,
	"num_input_tokens_seen": 880352,
	"step": 22,
	"train_runtime": 84.7765,
	"train_tokens_per_second": 10384.389
	},
	{
	"epoch": 0.15726495726495726,
	"grad_norm": 1.5459015369415283,
	"learning_rate": 1.8823529411764708e-05,
	"loss": 0.4687,
	"num_input_tokens_seen": 920448,
	"step": 23,
	"train_runtime": 88.149,
	"train_tokens_per_second": 10441.956
	},
	{
	"epoch": 0.1641025641025641,
	"grad_norm": 1.7185068130493164,
	"learning_rate": 1.8754325259515573e-05,
	"loss": 0.4597,
	"num_input_tokens_seen": 966944,
	"step": 24,
	"train_runtime": 92.0191,
	"train_tokens_per_second": 10508.075
	},
	{
	"epoch": 0.17094017094017094,
	"grad_norm": 1.4897587299346924,
	"learning_rate": 1.868512110726644e-05,
	"loss": 0.4537,
	"num_input_tokens_seen": 1008384,
	"step": 25,
	"train_runtime": 95.4709,
	"train_tokens_per_second": 10562.217
	},
	{
	"epoch": 0.17777777777777778,
	"grad_norm": 1.5461437702178955,
	"learning_rate": 1.8615916955017305e-05,
	"loss": 0.4948,
	"num_input_tokens_seen": 1045088,
	"step": 26,
	"train_runtime": 98.5427,
	"train_tokens_per_second": 10605.434
	},
	{
	"epoch": 0.18461538461538463,
	"grad_norm": 1.5771409273147583,
	"learning_rate": 1.8546712802768167e-05,
	"loss": 0.5204,
	"num_input_tokens_seen": 1089184,
	"step": 27,
	"train_runtime": 102.2145,
	"train_tokens_per_second": 10655.869
	},
	{
	"epoch": 0.19145299145299147,
	"grad_norm": 1.4081662893295288,
	"learning_rate": 1.8477508650519033e-05,
	"loss": 0.452,
	"num_input_tokens_seen": 1131168,
	"step": 28,
	"train_runtime": 105.7371,
	"train_tokens_per_second": 10697.931
	},
	{
	"epoch": 0.19829059829059828,
	"grad_norm": 1.6867786645889282,
	"learning_rate": 1.8408304498269898e-05,
	"loss": 0.4906,
	"num_input_tokens_seen": 1164352,
	"step": 29,
	"train_runtime": 108.5496,
	"train_tokens_per_second": 10726.45
	},
	{
	"epoch": 0.20512820512820512,
	"grad_norm": 1.3786219358444214,
	"learning_rate": 1.833910034602076e-05,
	"loss": 0.4705,
	"num_input_tokens_seen": 1201440,
	"step": 30,
	"train_runtime": 111.6624,
	"train_tokens_per_second": 10759.575
	},
	{
	"epoch": 0.20512820512820512,
	"eval_loss": 0.4640962481498718,
	"eval_runtime": 6.7936,
	"eval_samples_per_second": 146.903,
	"eval_steps_per_second": 4.71,
	"num_input_tokens_seen": 1201440,
	"step": 30
	},
	{
	"epoch": 0.21196581196581196,
	"grad_norm": 1.7535614967346191,
	"learning_rate": 1.826989619377163e-05,
	"loss": 0.487,
	"num_input_tokens_seen": 1236704,
	"step": 31,
	"train_runtime": 139.6017,
	"train_tokens_per_second": 8858.806
	},
	{
	"epoch": 0.2188034188034188,
	"grad_norm": 1.4274615049362183,
	"learning_rate": 1.8200692041522492e-05,
	"loss": 0.4459,
	"num_input_tokens_seen": 1278464,
	"step": 32,
	"train_runtime": 143.0525,
	"train_tokens_per_second": 8937.028
	},
	{
	"epoch": 0.22564102564102564,
	"grad_norm": 1.527426838874817,
	"learning_rate": 1.8131487889273357e-05,
	"loss": 0.5151,
	"num_input_tokens_seen": 1315616,
	"step": 33,
	"train_runtime": 146.1773,
	"train_tokens_per_second": 9000.14
	},
	{
	"epoch": 0.23247863247863249,
	"grad_norm": 1.4509639739990234,
	"learning_rate": 1.8062283737024223e-05,
	"loss": 0.473,
	"num_input_tokens_seen": 1351200,
	"step": 34,
	"train_runtime": 149.1958,
	"train_tokens_per_second": 9056.555
	},
	{
	"epoch": 0.23931623931623933,
	"grad_norm": 1.574431300163269,
	"learning_rate": 1.799307958477509e-05,
	"loss": 0.4562,
	"num_input_tokens_seen": 1393888,
	"step": 35,
	"train_runtime": 152.7492,
	"train_tokens_per_second": 9125.335
	},
	{
	"epoch": 0.24615384615384617,
	"grad_norm": 1.421439290046692,
	"learning_rate": 1.792387543252595e-05,
	"loss": 0.4213,
	"num_input_tokens_seen": 1432000,
	"step": 36,
	"train_runtime": 155.9721,
	"train_tokens_per_second": 9181.127
	},
	{
	"epoch": 0.252991452991453,
	"grad_norm": 1.3346866369247437,
	"learning_rate": 1.785467128027682e-05,
	"loss": 0.4865,
	"num_input_tokens_seen": 1474912,
	"step": 37,
	"train_runtime": 159.5609,
	"train_tokens_per_second": 9243.568
	},
	{
	"epoch": 0.25982905982905985,
	"grad_norm": 1.401798129081726,
	"learning_rate": 1.7785467128027682e-05,
	"loss": 0.4983,
	"num_input_tokens_seen": 1514688,
	"step": 38,
	"train_runtime": 162.9161,
	"train_tokens_per_second": 9297.352
	},
	{
	"epoch": 0.26666666666666666,
	"grad_norm": 1.5963151454925537,
	"learning_rate": 1.7716262975778548e-05,
	"loss": 0.5577,
	"num_input_tokens_seen": 1553248,
	"step": 39,
	"train_runtime": 166.1397,
	"train_tokens_per_second": 9349.045
	},
	{
	"epoch": 0.27350427350427353,
	"grad_norm": 1.3875170946121216,
	"learning_rate": 1.7647058823529414e-05,
	"loss": 0.4095,
	"num_input_tokens_seen": 1593120,
	"step": 40,
	"train_runtime": 169.4554,
	"train_tokens_per_second": 9401.41
	},
	{
	"epoch": 0.28034188034188035,
	"grad_norm": 1.6025474071502686,
	"learning_rate": 1.757785467128028e-05,
	"loss": 0.5361,
	"num_input_tokens_seen": 1628256,
	"step": 41,
	"train_runtime": 172.428,
	"train_tokens_per_second": 9443.106
	},
	{
	"epoch": 0.28717948717948716,
	"grad_norm": 1.3887425661087036,
	"learning_rate": 1.750865051903114e-05,
	"loss": 0.4542,
	"num_input_tokens_seen": 1674016,
	"step": 42,
	"train_runtime": 176.2327,
	"train_tokens_per_second": 9498.898
	},
	{
	"epoch": 0.294017094017094,
	"grad_norm": 1.7252106666564941,
	"learning_rate": 1.743944636678201e-05,
	"loss": 0.4771,
	"num_input_tokens_seen": 1706944,
	"step": 43,
	"train_runtime": 179.0413,
	"train_tokens_per_second": 9533.802
	},
	{
	"epoch": 0.30085470085470084,
	"grad_norm": 1.5482698678970337,
	"learning_rate": 1.7370242214532873e-05,
	"loss": 0.4532,
	"num_input_tokens_seen": 1748128,
	"step": 44,
	"train_runtime": 182.4655,
	"train_tokens_per_second": 9580.595
	},
	{
	"epoch": 0.3076923076923077,
	"grad_norm": 1.6200870275497437,
	"learning_rate": 1.730103806228374e-05,
	"loss": 0.4823,
	"num_input_tokens_seen": 1780032,
	"step": 45,
	"train_runtime": 185.183,
	"train_tokens_per_second": 9612.285
	},
	{
	"epoch": 0.3145299145299145,
	"grad_norm": 1.4116896390914917,
	"learning_rate": 1.7231833910034604e-05,
	"loss": 0.4341,
	"num_input_tokens_seen": 1820128,
	"step": 46,
	"train_runtime": 188.5419,
	"train_tokens_per_second": 9653.708
	},
	{
	"epoch": 0.3213675213675214,
	"grad_norm": 1.6274487972259521,
	"learning_rate": 1.716262975778547e-05,
	"loss": 0.4756,
	"num_input_tokens_seen": 1853536,
	"step": 47,
	"train_runtime": 191.3682,
	"train_tokens_per_second": 9685.703
	},
	{
	"epoch": 0.3282051282051282,
	"grad_norm": 1.4281944036483765,
	"learning_rate": 1.7093425605536332e-05,
	"loss": 0.4333,
	"num_input_tokens_seen": 1890560,
	"step": 48,
	"train_runtime": 194.4859,
	"train_tokens_per_second": 9720.806
	},
	{
	"epoch": 0.335042735042735,
	"grad_norm": 1.411935567855835,
	"learning_rate": 1.70242214532872e-05,
	"loss": 0.4519,
	"num_input_tokens_seen": 1939648,
	"step": 49,
	"train_runtime": 198.5911,
	"train_tokens_per_second": 9767.042
	},
	{
	"epoch": 0.3418803418803419,
	"grad_norm": 1.4910825490951538,
	"learning_rate": 1.6955017301038063e-05,
	"loss": 0.4128,
	"num_input_tokens_seen": 1971584,
	"step": 50,
	"train_runtime": 201.3116,
	"train_tokens_per_second": 9793.692
	},
	{
	"epoch": 0.3487179487179487,
	"grad_norm": 1.67446768283844,
	"learning_rate": 1.688581314878893e-05,
	"loss": 0.5384,
	"num_input_tokens_seen": 2021536,
	"step": 51,
	"train_runtime": 205.4602,
	"train_tokens_per_second": 9839.062
	},
	{
	"epoch": 0.35555555555555557,
	"grad_norm": 1.500937581062317,
	"learning_rate": 1.6816608996539795e-05,
	"loss": 0.4733,
	"num_input_tokens_seen": 2066656,
	"step": 52,
	"train_runtime": 209.327,
	"train_tokens_per_second": 9872.859
	},
	{
	"epoch": 0.3623931623931624,
	"grad_norm": 1.6410099267959595,
	"learning_rate": 1.6747404844290657e-05,
	"loss": 0.4283,
	"num_input_tokens_seen": 2100160,
	"step": 53,
	"train_runtime": 212.1628,
	"train_tokens_per_second": 9898.815
	},
	{
	"epoch": 0.36923076923076925,
	"grad_norm": 1.507749080657959,
	"learning_rate": 1.6678200692041523e-05,
	"loss": 0.4679,
	"num_input_tokens_seen": 2133888,
	"step": 54,
	"train_runtime": 215.0512,
	"train_tokens_per_second": 9922.696
	},
	{
	"epoch": 0.37606837606837606,
	"grad_norm": 1.5565907955169678,
	"learning_rate": 1.6608996539792388e-05,
	"loss": 0.43,
	"num_input_tokens_seen": 2169472,
	"step": 55,
	"train_runtime": 218.0654,
	"train_tokens_per_second": 9948.72
	},
	{
	"epoch": 0.38290598290598293,
	"grad_norm": 1.380922555923462,
	"learning_rate": 1.6539792387543254e-05,
	"loss": 0.4838,
	"num_input_tokens_seen": 2214720,
	"step": 56,
	"train_runtime": 221.8928,
	"train_tokens_per_second": 9981.036
	},
	{
	"epoch": 0.38974358974358975,
	"grad_norm": 1.6342693567276,
	"learning_rate": 1.647058823529412e-05,
	"loss": 0.4654,
	"num_input_tokens_seen": 2251648,
	"step": 57,
	"train_runtime": 225.0239,
	"train_tokens_per_second": 10006.263
	},
	{
	"epoch": 0.39658119658119656,
	"grad_norm": 1.5192848443984985,
	"learning_rate": 1.6401384083044985e-05,
	"loss": 0.5026,
	"num_input_tokens_seen": 2284064,
	"step": 58,
	"train_runtime": 227.7635,
	"train_tokens_per_second": 10028.225
	},
	{
	"epoch": 0.40341880341880343,
	"grad_norm": 1.5925633907318115,
	"learning_rate": 1.6332179930795848e-05,
	"loss": 0.4585,
	"num_input_tokens_seen": 2312512,
	"step": 59,
	"train_runtime": 230.219,
	"train_tokens_per_second": 10044.836
	},
	{
	"epoch": 0.41025641025641024,
	"grad_norm": 1.4670740365982056,
	"learning_rate": 1.6262975778546713e-05,
	"loss": 0.434,
	"num_input_tokens_seen": 2348320,
	"step": 60,
	"train_runtime": 233.2567,
	"train_tokens_per_second": 10067.536
	},
	{
	"epoch": 0.41025641025641024,
	"eval_loss": 0.4576193392276764,
	"eval_runtime": 5.7904,
	"eval_samples_per_second": 172.354,
	"eval_steps_per_second": 5.526,
	"num_input_tokens_seen": 2348320,
	"step": 60
	},
	{
	"epoch": 0.4170940170940171,
	"grad_norm": 1.5287467241287231,
	"learning_rate": 1.619377162629758e-05,
	"loss": 0.4578,
	"num_input_tokens_seen": 2391232,
	"step": 61,
	"train_runtime": 259.4978,
	"train_tokens_per_second": 9214.845
	},
	{
	"epoch": 0.4239316239316239,
	"grad_norm": 1.507736325263977,
	"learning_rate": 1.6124567474048444e-05,
	"loss": 0.4129,
	"num_input_tokens_seen": 2428096,
	"step": 62,
	"train_runtime": 262.5473,
	"train_tokens_per_second": 9248.225
	},
	{
	"epoch": 0.4307692307692308,
	"grad_norm": 1.5525883436203003,
	"learning_rate": 1.605536332179931e-05,
	"loss": 0.4635,
	"num_input_tokens_seen": 2457504,
	"step": 63,
	"train_runtime": 265.0656,
	"train_tokens_per_second": 9271.305
	},
	{
	"epoch": 0.4376068376068376,
	"grad_norm": 1.6987532377243042,
	"learning_rate": 1.5986159169550176e-05,
	"loss": 0.5224,
	"num_input_tokens_seen": 2490496,
	"step": 64,
	"train_runtime": 267.8475,
	"train_tokens_per_second": 9298.187
	},
	{
	"epoch": 0.4444444444444444,
	"grad_norm": 1.3846668004989624,
	"learning_rate": 1.5916955017301038e-05,
	"loss": 0.4179,
	"num_input_tokens_seen": 2529888,
	"step": 65,
	"train_runtime": 271.0955,
	"train_tokens_per_second": 9332.093
	},
	{
	"epoch": 0.4512820512820513,
	"grad_norm": 1.4100641012191772,
	"learning_rate": 1.5847750865051904e-05,
	"loss": 0.4574,
	"num_input_tokens_seen": 2563744,
	"step": 66,
	"train_runtime": 273.9696,
	"train_tokens_per_second": 9357.767
	},
	{
	"epoch": 0.4581196581196581,
	"grad_norm": 1.4976650476455688,
	"learning_rate": 1.577854671280277e-05,
	"loss": 0.4836,
	"num_input_tokens_seen": 2607584,
	"step": 67,
	"train_runtime": 277.6271,
	"train_tokens_per_second": 9392.398
	},
	{
	"epoch": 0.46495726495726497,
	"grad_norm": 1.4736229181289673,
	"learning_rate": 1.5709342560553635e-05,
	"loss": 0.403,
	"num_input_tokens_seen": 2650848,
	"step": 68,
	"train_runtime": 281.2134,
	"train_tokens_per_second": 9426.463
	},
	{
	"epoch": 0.4717948717948718,
	"grad_norm": 1.4522730112075806,
	"learning_rate": 1.56401384083045e-05,
	"loss": 0.4548,
	"num_input_tokens_seen": 2678816,
	"step": 69,
	"train_runtime": 283.5945,
	"train_tokens_per_second": 9445.936
	},
	{
	"epoch": 0.47863247863247865,
	"grad_norm": 1.5067986249923706,
	"learning_rate": 1.5570934256055366e-05,
	"loss": 0.4432,
	"num_input_tokens_seen": 2717472,
	"step": 70,
	"train_runtime": 286.8288,
	"train_tokens_per_second": 9474.195
	},
	{
	"epoch": 0.48547008547008547,
	"grad_norm": 1.542819857597351,
	"learning_rate": 1.550173010380623e-05,
	"loss": 0.4915,
	"num_input_tokens_seen": 2754592,
	"step": 71,
	"train_runtime": 289.9357,
	"train_tokens_per_second": 9500.7
	},
	{
	"epoch": 0.49230769230769234,
	"grad_norm": 1.4620888233184814,
	"learning_rate": 1.5432525951557094e-05,
	"loss": 0.3876,
	"num_input_tokens_seen": 2799296,
	"step": 72,
	"train_runtime": 293.7189,
	"train_tokens_per_second": 9530.527
	},
	{
	"epoch": 0.49914529914529915,
	"grad_norm": 1.526044487953186,
	"learning_rate": 1.536332179930796e-05,
	"loss": 0.4648,
	"num_input_tokens_seen": 2828800,
	"step": 73,
	"train_runtime": 296.237,
	"train_tokens_per_second": 9549.112
	},
	{
	"epoch": 0.505982905982906,
	"grad_norm": 1.4033782482147217,
	"learning_rate": 1.5294117647058822e-05,
	"loss": 0.4114,
	"num_input_tokens_seen": 2876800,
	"step": 74,
	"train_runtime": 300.199,
	"train_tokens_per_second": 9582.978
	},
	{
	"epoch": 0.5128205128205128,
	"grad_norm": 1.6525285243988037,
	"learning_rate": 1.522491349480969e-05,
	"loss": 0.4523,
	"num_input_tokens_seen": 2924736,
	"step": 75,
	"train_runtime": 304.1673,
	"train_tokens_per_second": 9615.551
	},
	{
	"epoch": 0.5196581196581197,
	"grad_norm": 1.3782298564910889,
	"learning_rate": 1.5155709342560554e-05,
	"loss": 0.416,
	"num_input_tokens_seen": 2969600,
	"step": 76,
	"train_runtime": 307.8767,
	"train_tokens_per_second": 9645.42
	},
	{
	"epoch": 0.5264957264957265,
	"grad_norm": 1.2422502040863037,
	"learning_rate": 1.5086505190311421e-05,
	"loss": 0.3851,
	"num_input_tokens_seen": 3030912,
	"step": 77,
	"train_runtime": 313.0558,
	"train_tokens_per_second": 9681.699
	},
	{
	"epoch": 0.5333333333333333,
	"grad_norm": 1.7053712606430054,
	"learning_rate": 1.5017301038062285e-05,
	"loss": 0.5167,
	"num_input_tokens_seen": 3069312,
	"step": 78,
	"train_runtime": 316.2163,
	"train_tokens_per_second": 9706.369
	},
	{
	"epoch": 0.5401709401709401,
	"grad_norm": 1.57555091381073,
	"learning_rate": 1.494809688581315e-05,
	"loss": 0.5093,
	"num_input_tokens_seen": 3099456,
	"step": 79,
	"train_runtime": 318.7901,
	"train_tokens_per_second": 9722.56
	},
	{
	"epoch": 0.5470085470085471,
	"grad_norm": 1.528808355331421,
	"learning_rate": 1.4878892733564014e-05,
	"loss": 0.5011,
	"num_input_tokens_seen": 3137024,
	"step": 80,
	"train_runtime": 321.9542,
	"train_tokens_per_second": 9743.697
	},
	{
	"epoch": 0.5538461538461539,
	"grad_norm": 1.358077049255371,
	"learning_rate": 1.480968858131488e-05,
	"loss": 0.4043,
	"num_input_tokens_seen": 3183648,
	"step": 81,
	"train_runtime": 325.824,
	"train_tokens_per_second": 9771.066
	},
	{
	"epoch": 0.5606837606837607,
	"grad_norm": 1.7505600452423096,
	"learning_rate": 1.4740484429065744e-05,
	"loss": 0.4766,
	"num_input_tokens_seen": 3214592,
	"step": 82,
	"train_runtime": 328.4491,
	"train_tokens_per_second": 9787.183
	},
	{
	"epoch": 0.5675213675213675,
	"grad_norm": 1.6043431758880615,
	"learning_rate": 1.4671280276816611e-05,
	"loss": 0.4837,
	"num_input_tokens_seen": 3242720,
	"step": 83,
	"train_runtime": 330.8621,
	"train_tokens_per_second": 9800.82
	},
	{
	"epoch": 0.5743589743589743,
	"grad_norm": 1.3557583093643188,
	"learning_rate": 1.4602076124567475e-05,
	"loss": 0.4051,
	"num_input_tokens_seen": 3278816,
	"step": 84,
	"train_runtime": 333.921,
	"train_tokens_per_second": 9819.136
	},
	{
	"epoch": 0.5811965811965812,
	"grad_norm": 1.4810553789138794,
	"learning_rate": 1.4532871972318341e-05,
	"loss": 0.4385,
	"num_input_tokens_seen": 3315008,
	"step": 85,
	"train_runtime": 336.9782,
	"train_tokens_per_second": 9837.454
	},
	{
	"epoch": 0.588034188034188,
	"grad_norm": 1.520869493484497,
	"learning_rate": 1.4463667820069205e-05,
	"loss": 0.4663,
	"num_input_tokens_seen": 3349152,
	"step": 86,
	"train_runtime": 339.8919,
	"train_tokens_per_second": 9853.58
	},
	{
	"epoch": 0.5948717948717949,
	"grad_norm": 1.5246199369430542,
	"learning_rate": 1.439446366782007e-05,
	"loss": 0.4519,
	"num_input_tokens_seen": 3377696,
	"step": 87,
	"train_runtime": 342.3568,
	"train_tokens_per_second": 9866.012
	},
	{
	"epoch": 0.6017094017094017,
	"grad_norm": 1.554733157157898,
	"learning_rate": 1.4325259515570935e-05,
	"loss": 0.5046,
	"num_input_tokens_seen": 3414656,
	"step": 88,
	"train_runtime": 345.4265,
	"train_tokens_per_second": 9885.332
	},
	{
	"epoch": 0.6085470085470085,
	"grad_norm": 1.571320652961731,
	"learning_rate": 1.4256055363321802e-05,
	"loss": 0.4736,
	"num_input_tokens_seen": 3446080,
	"step": 89,
	"train_runtime": 348.0842,
	"train_tokens_per_second": 9900.134
	},
	{
	"epoch": 0.6153846153846154,
	"grad_norm": 1.624376893043518,
	"learning_rate": 1.4186851211072666e-05,
	"loss": 0.4512,
	"num_input_tokens_seen": 3475168,
	"step": 90,
	"train_runtime": 350.5802,
	"train_tokens_per_second": 9912.62
	},
	{
	"epoch": 0.6153846153846154,
	"eval_loss": 0.4554298520088196,
	"eval_runtime": 5.7133,
	"eval_samples_per_second": 174.681,
	"eval_steps_per_second": 5.601,
	"num_input_tokens_seen": 3475168,
	"step": 90
	},
	{
	"epoch": 0.6222222222222222,
	"grad_norm": 1.713934063911438,
	"learning_rate": 1.4117647058823532e-05,
	"loss": 0.4519,
	"num_input_tokens_seen": 3500032,
	"step": 91,
	"train_runtime": 376.436,
	"train_tokens_per_second": 9297.815
	},
	{
	"epoch": 0.629059829059829,
	"grad_norm": 1.5343419313430786,
	"learning_rate": 1.4048442906574396e-05,
	"loss": 0.4543,
	"num_input_tokens_seen": 3535136,
	"step": 92,
	"train_runtime": 379.3304,
	"train_tokens_per_second": 9319.412
	},
	{
	"epoch": 0.6358974358974359,
	"grad_norm": 1.4790829420089722,
	"learning_rate": 1.3979238754325261e-05,
	"loss": 0.439,
	"num_input_tokens_seen": 3564192,
	"step": 93,
	"train_runtime": 381.791,
	"train_tokens_per_second": 9335.453
	},
	{
	"epoch": 0.6427350427350428,
	"grad_norm": 1.435154914855957,
	"learning_rate": 1.3910034602076125e-05,
	"loss": 0.4486,
	"num_input_tokens_seen": 3600640,
	"step": 94,
	"train_runtime": 384.8183,
	"train_tokens_per_second": 9356.727
	},
	{
	"epoch": 0.6495726495726496,
	"grad_norm": 1.7755075693130493,
	"learning_rate": 1.3840830449826989e-05,
	"loss": 0.486,
	"num_input_tokens_seen": 3634336,
	"step": 95,
	"train_runtime": 387.6562,
	"train_tokens_per_second": 9375.153
	},
	{
	"epoch": 0.6564102564102564,
	"grad_norm": 1.542274832725525,
	"learning_rate": 1.3771626297577856e-05,
	"loss": 0.476,
	"num_input_tokens_seen": 3667328,
	"step": 96,
	"train_runtime": 390.429,
	"train_tokens_per_second": 9393.073
	},
	{
	"epoch": 0.6632478632478632,
	"grad_norm": 1.5652968883514404,
	"learning_rate": 1.370242214532872e-05,
	"loss": 0.4655,
	"num_input_tokens_seen": 3700224,
	"step": 97,
	"train_runtime": 393.1811,
	"train_tokens_per_second": 9410.992
	},
	{
	"epoch": 0.67008547008547,
	"grad_norm": 1.433912992477417,
	"learning_rate": 1.3633217993079586e-05,
	"loss": 0.4575,
	"num_input_tokens_seen": 3744448,
	"step": 98,
	"train_runtime": 396.8421,
	"train_tokens_per_second": 9435.611
	},
	{
	"epoch": 0.676923076923077,
	"grad_norm": 1.5182974338531494,
	"learning_rate": 1.356401384083045e-05,
	"loss": 0.4643,
	"num_input_tokens_seen": 3779936,
	"step": 99,
	"train_runtime": 399.8119,
	"train_tokens_per_second": 9454.287
	},
	{
	"epoch": 0.6837606837606838,
	"grad_norm": 1.5050530433654785,
	"learning_rate": 1.3494809688581316e-05,
	"loss": 0.4509,
	"num_input_tokens_seen": 3828608,
	"step": 100,
	"train_runtime": 403.7792,
	"train_tokens_per_second": 9481.935
	},
	{
	"epoch": 0.6905982905982906,
	"grad_norm": 1.3182984590530396,
	"learning_rate": 1.342560553633218e-05,
	"loss": 0.4183,
	"num_input_tokens_seen": 3881632,
	"step": 101,
	"train_runtime": 408.1485,
	"train_tokens_per_second": 9510.342
	},
	{
	"epoch": 0.6974358974358974,
	"grad_norm": 1.4368970394134521,
	"learning_rate": 1.3356401384083047e-05,
	"loss": 0.5363,
	"num_input_tokens_seen": 3911200,
	"step": 102,
	"train_runtime": 410.6666,
	"train_tokens_per_second": 9524.028
	},
	{
	"epoch": 0.7042735042735043,
	"grad_norm": 1.702073335647583,
	"learning_rate": 1.3287197231833911e-05,
	"loss": 0.4391,
	"num_input_tokens_seen": 3951424,
	"step": 103,
	"train_runtime": 414.0044,
	"train_tokens_per_second": 9544.4
	},
	{
	"epoch": 0.7111111111111111,
	"grad_norm": 1.484083652496338,
	"learning_rate": 1.3217993079584777e-05,
	"loss": 0.4622,
	"num_input_tokens_seen": 3984992,
	"step": 104,
	"train_runtime": 416.8268,
	"train_tokens_per_second": 9560.306
	},
	{
	"epoch": 0.717948717948718,
	"grad_norm": 1.5703684091567993,
	"learning_rate": 1.314878892733564e-05,
	"loss": 0.4656,
	"num_input_tokens_seen": 4022400,
	"step": 105,
	"train_runtime": 419.9371,
	"train_tokens_per_second": 9578.578
	},
	{
	"epoch": 0.7247863247863248,
	"grad_norm": 1.617999792098999,
	"learning_rate": 1.3079584775086506e-05,
	"loss": 0.475,
	"num_input_tokens_seen": 4056160,
	"step": 106,
	"train_runtime": 422.7581,
	"train_tokens_per_second": 9594.517
	},
	{
	"epoch": 0.7316239316239316,
	"grad_norm": 1.7811297178268433,
	"learning_rate": 1.301038062283737e-05,
	"loss": 0.517,
	"num_input_tokens_seen": 4092928,
	"step": 107,
	"train_runtime": 425.86,
	"train_tokens_per_second": 9610.97
	},
	{
	"epoch": 0.7384615384615385,
	"grad_norm": 1.3520305156707764,
	"learning_rate": 1.2941176470588238e-05,
	"loss": 0.4112,
	"num_input_tokens_seen": 4153440,
	"step": 108,
	"train_runtime": 431.5262,
	"train_tokens_per_second": 9625.002
	},
	{
	"epoch": 0.7452991452991453,
	"grad_norm": 1.4912413358688354,
	"learning_rate": 1.2871972318339102e-05,
	"loss": 0.4536,
	"num_input_tokens_seen": 4187744,
	"step": 109,
	"train_runtime": 434.4156,
	"train_tokens_per_second": 9639.948
	},
	{
	"epoch": 0.7521367521367521,
	"grad_norm": 1.4879883527755737,
	"learning_rate": 1.2802768166089967e-05,
	"loss": 0.4744,
	"num_input_tokens_seen": 4226496,
	"step": 110,
	"train_runtime": 437.6348,
	"train_tokens_per_second": 9657.586
	},
	{
	"epoch": 0.7589743589743589,
	"grad_norm": 1.5085774660110474,
	"learning_rate": 1.2733564013840831e-05,
	"loss": 0.5143,
	"num_input_tokens_seen": 4262176,
	"step": 111,
	"train_runtime": 440.6057,
	"train_tokens_per_second": 9673.447
	},
	{
	"epoch": 0.7658119658119659,
	"grad_norm": 1.4866958856582642,
	"learning_rate": 1.2664359861591697e-05,
	"loss": 0.3955,
	"num_input_tokens_seen": 4299200,
	"step": 112,
	"train_runtime": 443.6903,
	"train_tokens_per_second": 9689.643
	},
	{
	"epoch": 0.7726495726495727,
	"grad_norm": 1.7484487295150757,
	"learning_rate": 1.259515570934256e-05,
	"loss": 0.5021,
	"num_input_tokens_seen": 4339904,
	"step": 113,
	"train_runtime": 447.0564,
	"train_tokens_per_second": 9707.733
	},
	{
	"epoch": 0.7794871794871795,
	"grad_norm": 1.5065933465957642,
	"learning_rate": 1.2525951557093428e-05,
	"loss": 0.4726,
	"num_input_tokens_seen": 4369728,
	"step": 114,
	"train_runtime": 449.5826,
	"train_tokens_per_second": 9719.522
	},
	{
	"epoch": 0.7863247863247863,
	"grad_norm": 1.4008255004882812,
	"learning_rate": 1.2456747404844292e-05,
	"loss": 0.4908,
	"num_input_tokens_seen": 4407936,
	"step": 115,
	"train_runtime": 452.8133,
	"train_tokens_per_second": 9734.554
	},
	{
	"epoch": 0.7931623931623931,
	"grad_norm": 1.5314161777496338,
	"learning_rate": 1.2387543252595158e-05,
	"loss": 0.5017,
	"num_input_tokens_seen": 4439520,
	"step": 116,
	"train_runtime": 455.4893,
	"train_tokens_per_second": 9746.705
	},
	{
	"epoch": 0.8,
	"grad_norm": 1.3641289472579956,
	"learning_rate": 1.2318339100346022e-05,
	"loss": 0.4654,
	"num_input_tokens_seen": 4475552,
	"step": 117,
	"train_runtime": 458.5097,
	"train_tokens_per_second": 9761.084
	},
	{
	"epoch": 0.8068376068376069,
	"grad_norm": 1.6261942386627197,
	"learning_rate": 1.2249134948096886e-05,
	"loss": 0.4088,
	"num_input_tokens_seen": 4530112,
	"step": 118,
	"train_runtime": 463.158,
	"train_tokens_per_second": 9780.921
	},
	{
	"epoch": 0.8136752136752137,
	"grad_norm": 1.4655214548110962,
	"learning_rate": 1.2179930795847751e-05,
	"loss": 0.4371,
	"num_input_tokens_seen": 4559936,
	"step": 119,
	"train_runtime": 465.6803,
	"train_tokens_per_second": 9791.988
	},
	{
	"epoch": 0.8205128205128205,
	"grad_norm": 1.5850762128829956,
	"learning_rate": 1.2110726643598615e-05,
	"loss": 0.4721,
	"num_input_tokens_seen": 4601120,
	"step": 120,
	"train_runtime": 469.1647,
	"train_tokens_per_second": 9807.046
	},
	{
	"epoch": 0.8205128205128205,
	"eval_loss": 0.4540318250656128,
	"eval_runtime": 5.7037,
	"eval_samples_per_second": 174.974,
	"eval_steps_per_second": 5.61,
	"num_input_tokens_seen": 4601120,
	"step": 120
	},
	{
	"epoch": 0.8273504273504273,
	"grad_norm": 1.4847512245178223,
	"learning_rate": 1.2041522491349483e-05,
	"loss": 0.4827,
	"num_input_tokens_seen": 4635648,
	"step": 121,
	"train_runtime": 496.058,
	"train_tokens_per_second": 9344.972
	},
	{
	"epoch": 0.8341880341880342,
	"grad_norm": 1.4828547239303589,
	"learning_rate": 1.1972318339100347e-05,
	"loss": 0.4849,
	"num_input_tokens_seen": 4674560,
	"step": 122,
	"train_runtime": 499.2917,
	"train_tokens_per_second": 9362.382
	},
	{
	"epoch": 0.841025641025641,
	"grad_norm": 1.5140576362609863,
	"learning_rate": 1.1903114186851212e-05,
	"loss": 0.521,
	"num_input_tokens_seen": 4717344,
	"step": 123,
	"train_runtime": 502.8538,
	"train_tokens_per_second": 9381.145
	},
	{
	"epoch": 0.8478632478632478,
	"grad_norm": 1.1721268892288208,
	"learning_rate": 1.1833910034602076e-05,
	"loss": 0.3616,
	"num_input_tokens_seen": 4778304,
	"step": 124,
	"train_runtime": 507.9229,
	"train_tokens_per_second": 9407.537
	},
	{
	"epoch": 0.8547008547008547,
	"grad_norm": 1.3763750791549683,
	"learning_rate": 1.1764705882352942e-05,
	"loss": 0.4127,
	"num_input_tokens_seen": 4814464,
	"step": 125,
	"train_runtime": 510.9361,
	"train_tokens_per_second": 9422.83
	},
	{
	"epoch": 0.8615384615384616,
	"grad_norm": 1.4982727766036987,
	"learning_rate": 1.1695501730103806e-05,
	"loss": 0.433,
	"num_input_tokens_seen": 4860768,
	"step": 126,
	"train_runtime": 514.8181,
	"train_tokens_per_second": 9441.719
	},
	{
	"epoch": 0.8683760683760684,
	"grad_norm": 1.4128450155258179,
	"learning_rate": 1.1626297577854673e-05,
	"loss": 0.5173,
	"num_input_tokens_seen": 4900512,
	"step": 127,
	"train_runtime": 518.1154,
	"train_tokens_per_second": 9458.342
	},
	{
	"epoch": 0.8752136752136752,
	"grad_norm": 1.564786434173584,
	"learning_rate": 1.1557093425605537e-05,
	"loss": 0.4603,
	"num_input_tokens_seen": 4938752,
	"step": 128,
	"train_runtime": 521.2991,
	"train_tokens_per_second": 9473.931
	},
	{
	"epoch": 0.882051282051282,
	"grad_norm": 1.5176303386688232,
	"learning_rate": 1.1487889273356403e-05,
	"loss": 0.4633,
	"num_input_tokens_seen": 4979584,
	"step": 129,
	"train_runtime": 524.6889,
	"train_tokens_per_second": 9490.545
	},
	{
	"epoch": 0.8888888888888888,
	"grad_norm": 1.5156581401824951,
	"learning_rate": 1.1418685121107267e-05,
	"loss": 0.4556,
	"num_input_tokens_seen": 5017216,
	"step": 130,
	"train_runtime": 527.841,
	"train_tokens_per_second": 9505.166
	},
	{
	"epoch": 0.8957264957264958,
	"grad_norm": 1.4633028507232666,
	"learning_rate": 1.1349480968858132e-05,
	"loss": 0.3931,
	"num_input_tokens_seen": 5054016,
	"step": 131,
	"train_runtime": 530.9057,
	"train_tokens_per_second": 9519.612
	},
	{
	"epoch": 0.9025641025641026,
	"grad_norm": 1.8066245317459106,
	"learning_rate": 1.1280276816608996e-05,
	"loss": 0.4996,
	"num_input_tokens_seen": 5080544,
	"step": 132,
	"train_runtime": 533.1694,
	"train_tokens_per_second": 9528.95
	},
	{
	"epoch": 0.9094017094017094,
	"grad_norm": 1.4042011499404907,
	"learning_rate": 1.1211072664359864e-05,
	"loss": 0.4592,
	"num_input_tokens_seen": 5116960,
	"step": 133,
	"train_runtime": 536.2021,
	"train_tokens_per_second": 9542.969
	},
	{
	"epoch": 0.9162393162393162,
	"grad_norm": 1.4731768369674683,
	"learning_rate": 1.1141868512110728e-05,
	"loss": 0.4417,
	"num_input_tokens_seen": 5152288,
	"step": 134,
	"train_runtime": 539.1624,
	"train_tokens_per_second": 9556.096
	},
	{
	"epoch": 0.9230769230769231,
	"grad_norm": 1.2772272825241089,
	"learning_rate": 1.1072664359861593e-05,
	"loss": 0.4064,
	"num_input_tokens_seen": 5204448,
	"step": 135,
	"train_runtime": 543.4474,
	"train_tokens_per_second": 9576.729
	},
	{
	"epoch": 0.9299145299145299,
	"grad_norm": 1.6605689525604248,
	"learning_rate": 1.1003460207612457e-05,
	"loss": 0.4421,
	"num_input_tokens_seen": 5255488,
	"step": 136,
	"train_runtime": 547.7055,
	"train_tokens_per_second": 9595.463
	},
	{
	"epoch": 0.9367521367521368,
	"grad_norm": 1.5701137781143188,
	"learning_rate": 1.0934256055363323e-05,
	"loss": 0.4587,
	"num_input_tokens_seen": 5282848,
	"step": 137,
	"train_runtime": 550.0393,
	"train_tokens_per_second": 9604.491
	},
	{
	"epoch": 0.9435897435897436,
	"grad_norm": 1.4943722486495972,
	"learning_rate": 1.0865051903114187e-05,
	"loss": 0.4801,
	"num_input_tokens_seen": 5319456,
	"step": 138,
	"train_runtime": 553.1186,
	"train_tokens_per_second": 9617.207
	},
	{
	"epoch": 0.9504273504273504,
	"grad_norm": 1.5468835830688477,
	"learning_rate": 1.0795847750865054e-05,
	"loss": 0.4504,
	"num_input_tokens_seen": 5359296,
	"step": 139,
	"train_runtime": 556.4596,
	"train_tokens_per_second": 9631.06
	},
	{
	"epoch": 0.9572649572649573,
	"grad_norm": 1.409867763519287,
	"learning_rate": 1.0726643598615918e-05,
	"loss": 0.4377,
	"num_input_tokens_seen": 5416672,
	"step": 140,
	"train_runtime": 561.16,
	"train_tokens_per_second": 9652.634
	},
	{
	"epoch": 0.9641025641025641,
	"grad_norm": 1.3638184070587158,
	"learning_rate": 1.0657439446366782e-05,
	"loss": 0.3818,
	"num_input_tokens_seen": 5498976,
	"step": 141,
	"train_runtime": 568.811,
	"train_tokens_per_second": 9667.493
	},
	{
	"epoch": 0.9709401709401709,
	"grad_norm": 1.4294383525848389,
	"learning_rate": 1.0588235294117648e-05,
	"loss": 0.4468,
	"num_input_tokens_seen": 5532992,
	"step": 142,
	"train_runtime": 571.6619,
	"train_tokens_per_second": 9678.784
	},
	{
	"epoch": 0.9777777777777777,
	"grad_norm": 1.3784202337265015,
	"learning_rate": 1.0519031141868512e-05,
	"loss": 0.4106,
	"num_input_tokens_seen": 5585280,
	"step": 143,
	"train_runtime": 576.0752,
	"train_tokens_per_second": 9695.4
	},
	{
	"epoch": 0.9846153846153847,
	"grad_norm": 1.6159602403640747,
	"learning_rate": 1.0449826989619377e-05,
	"loss": 0.4474,
	"num_input_tokens_seen": 5622976,
	"step": 144,
	"train_runtime": 579.2137,
	"train_tokens_per_second": 9707.948
	},
	{
	"epoch": 0.9914529914529915,
	"grad_norm": 1.510590672492981,
	"learning_rate": 1.0380622837370241e-05,
	"loss": 0.4664,
	"num_input_tokens_seen": 5661472,
	"step": 145,
	"train_runtime": 582.3983,
	"train_tokens_per_second": 9720.962
	},
	{
	"epoch": 0.9982905982905983,
	"grad_norm": 1.3742411136627197,
	"learning_rate": 1.0311418685121109e-05,
	"loss": 0.4175,
	"num_input_tokens_seen": 5698240,
	"step": 146,
	"train_runtime": 585.4675,
	"train_tokens_per_second": 9732.803
	},
	{
	"epoch": 1.0,
	"grad_norm": 2.935847282409668,
	"learning_rate": 1.0242214532871973e-05,
	"loss": 0.4182,
	"num_input_tokens_seen": 5709310,
	"step": 147,
	"train_runtime": 587.7866,
	"train_tokens_per_second": 9713.235
	},
	{
	"epoch": 1.0068376068376068,
	"grad_norm": 1.3502393960952759,
	"learning_rate": 1.0173010380622838e-05,
	"loss": 0.3852,
	"num_input_tokens_seen": 5746654,
	"step": 148,
	"train_runtime": 590.9756,
	"train_tokens_per_second": 9724.013
	},
	{
	"epoch": 1.0136752136752136,
	"grad_norm": 1.4006216526031494,
	"learning_rate": 1.0103806228373702e-05,
	"loss": 0.3924,
	"num_input_tokens_seen": 5777726,
	"step": 149,
	"train_runtime": 593.6371,
	"train_tokens_per_second": 9732.758
	},
	{
	"epoch": 1.0205128205128204,
	"grad_norm": 1.4392833709716797,
	"learning_rate": 1.0034602076124568e-05,
	"loss": 0.4426,
	"num_input_tokens_seen": 5812574,
	"step": 150,
	"train_runtime": 596.583,
	"train_tokens_per_second": 9743.111
	},
	{
	"epoch": 1.0205128205128204,
	"eval_loss": 0.45281028747558594,
	"eval_runtime": 6.2291,
	"eval_samples_per_second": 160.216,
	"eval_steps_per_second": 5.137,
	"num_input_tokens_seen": 5812574,
	"step": 150
	},
	{
	"epoch": 1.0273504273504273,
	"grad_norm": 1.3533525466918945,
	"learning_rate": 9.965397923875434e-06,
	"loss": 0.4033,
	"num_input_tokens_seen": 5855934,
	"step": 151,
	"train_runtime": 626.0707,
	"train_tokens_per_second": 9353.471
	},
	{
	"epoch": 1.0341880341880343,
	"grad_norm": 1.3957024812698364,
	"learning_rate": 9.896193771626298e-06,
	"loss": 0.4154,
	"num_input_tokens_seen": 5883870,
	"step": 152,
	"train_runtime": 628.4362,
	"train_tokens_per_second": 9362.717
	},
	{
	"epoch": 1.041025641025641,
	"grad_norm": 1.2741293907165527,
	"learning_rate": 9.826989619377163e-06,
	"loss": 0.4497,
	"num_input_tokens_seen": 5932190,
	"step": 153,
	"train_runtime": 632.3789,
	"train_tokens_per_second": 9380.752
	},
	{
	"epoch": 1.047863247863248,
	"grad_norm": 1.3246592283248901,
	"learning_rate": 9.757785467128029e-06,
	"loss": 0.3884,
	"num_input_tokens_seen": 5981854,
	"step": 154,
	"train_runtime": 636.5436,
	"train_tokens_per_second": 9397.398
	},
	{
	"epoch": 1.0547008547008547,
	"grad_norm": 1.3506548404693604,
	"learning_rate": 9.688581314878893e-06,
	"loss": 0.3601,
	"num_input_tokens_seen": 6030590,
	"step": 155,
	"train_runtime": 640.5068,
	"train_tokens_per_second": 9415.341
	},
	{
	"epoch": 1.0615384615384615,
	"grad_norm": 1.298898458480835,
	"learning_rate": 9.619377162629759e-06,
	"loss": 0.4158,
	"num_input_tokens_seen": 6072030,
	"step": 156,
	"train_runtime": 643.9463,
	"train_tokens_per_second": 9429.405
	},
	{
	"epoch": 1.0683760683760684,
	"grad_norm": 1.4321287870407104,
	"learning_rate": 9.550173010380624e-06,
	"loss": 0.4496,
	"num_input_tokens_seen": 6107006,
	"step": 157,
	"train_runtime": 646.8903,
	"train_tokens_per_second": 9440.559
	},
	{
	"epoch": 1.0752136752136752,
	"grad_norm": 1.362835168838501,
	"learning_rate": 9.480968858131488e-06,
	"loss": 0.4082,
	"num_input_tokens_seen": 6153086,
	"step": 158,
	"train_runtime": 650.7394,
	"train_tokens_per_second": 9455.529
	},
	{
	"epoch": 1.082051282051282,
	"grad_norm": 1.332863211631775,
	"learning_rate": 9.411764705882354e-06,
	"loss": 0.4156,
	"num_input_tokens_seen": 6192222,
	"step": 159,
	"train_runtime": 653.9789,
	"train_tokens_per_second": 9468.535
	},
	{
	"epoch": 1.0888888888888888,
	"grad_norm": 1.3329721689224243,
	"learning_rate": 9.34256055363322e-06,
	"loss": 0.4446,
	"num_input_tokens_seen": 6234430,
	"step": 160,
	"train_runtime": 657.5263,
	"train_tokens_per_second": 9481.643
	},
	{
	"epoch": 1.0957264957264958,
	"grad_norm": 1.4851776361465454,
	"learning_rate": 9.273356401384083e-06,
	"loss": 0.4104,
	"num_input_tokens_seen": 6267262,
	"step": 161,
	"train_runtime": 660.3125,
	"train_tokens_per_second": 9491.358
	},
	{
	"epoch": 1.1025641025641026,
	"grad_norm": 1.3695839643478394,
	"learning_rate": 9.204152249134949e-06,
	"loss": 0.3741,
	"num_input_tokens_seen": 6301214,
	"step": 162,
	"train_runtime": 663.1514,
	"train_tokens_per_second": 9501.923
	},
	{
	"epoch": 1.1094017094017095,
	"grad_norm": 1.3758296966552734,
	"learning_rate": 9.134948096885815e-06,
	"loss": 0.3491,
	"num_input_tokens_seen": 6348382,
	"step": 163,
	"train_runtime": 667.043,
	"train_tokens_per_second": 9517.2
	},
	{
	"epoch": 1.1162393162393163,
	"grad_norm": 1.564186692237854,
	"learning_rate": 9.065743944636679e-06,
	"loss": 0.3936,
	"num_input_tokens_seen": 6380958,
	"step": 164,
	"train_runtime": 669.7803,
	"train_tokens_per_second": 9526.942
	},
	{
	"epoch": 1.123076923076923,
	"grad_norm": 1.4669296741485596,
	"learning_rate": 8.996539792387544e-06,
	"loss": 0.4147,
	"num_input_tokens_seen": 6420126,
	"step": 165,
	"train_runtime": 673.027,
	"train_tokens_per_second": 9539.18
	},
	{
	"epoch": 1.12991452991453,
	"grad_norm": 1.2374634742736816,
	"learning_rate": 8.92733564013841e-06,
	"loss": 0.3456,
	"num_input_tokens_seen": 6466430,
	"step": 166,
	"train_runtime": 676.9123,
	"train_tokens_per_second": 9552.833
	},
	{
	"epoch": 1.1367521367521367,
	"grad_norm": 1.3043692111968994,
	"learning_rate": 8.858131487889274e-06,
	"loss": 0.3613,
	"num_input_tokens_seen": 6500990,
	"step": 167,
	"train_runtime": 679.8269,
	"train_tokens_per_second": 9562.714
	},
	{
	"epoch": 1.1435897435897435,
	"grad_norm": 1.5178890228271484,
	"learning_rate": 8.78892733564014e-06,
	"loss": 0.4367,
	"num_input_tokens_seen": 6534718,
	"step": 168,
	"train_runtime": 682.6614,
	"train_tokens_per_second": 9572.414
	},
	{
	"epoch": 1.1504273504273503,
	"grad_norm": 1.3679075241088867,
	"learning_rate": 8.719723183391005e-06,
	"loss": 0.4432,
	"num_input_tokens_seen": 6586718,
	"step": 169,
	"train_runtime": 686.9418,
	"train_tokens_per_second": 9588.466
	},
	{
	"epoch": 1.1572649572649572,
	"grad_norm": 1.356967806816101,
	"learning_rate": 8.65051903114187e-06,
	"loss": 0.3925,
	"num_input_tokens_seen": 6618526,
	"step": 170,
	"train_runtime": 689.6389,
	"train_tokens_per_second": 9597.089
	},
	{
	"epoch": 1.1641025641025642,
	"grad_norm": 1.5362049341201782,
	"learning_rate": 8.581314878892735e-06,
	"loss": 0.4119,
	"num_input_tokens_seen": 6648830,
	"step": 171,
	"train_runtime": 692.1817,
	"train_tokens_per_second": 9605.614
	},
	{
	"epoch": 1.170940170940171,
	"grad_norm": 1.4113274812698364,
	"learning_rate": 8.5121107266436e-06,
	"loss": 0.4419,
	"num_input_tokens_seen": 6683454,
	"step": 172,
	"train_runtime": 695.0911,
	"train_tokens_per_second": 9615.22
	},
	{
	"epoch": 1.1777777777777778,
	"grad_norm": 1.4560883045196533,
	"learning_rate": 8.442906574394465e-06,
	"loss": 0.4328,
	"num_input_tokens_seen": 6717822,
	"step": 173,
	"train_runtime": 697.9616,
	"train_tokens_per_second": 9624.916
	},
	{
	"epoch": 1.1846153846153846,
	"grad_norm": 1.3433024883270264,
	"learning_rate": 8.373702422145328e-06,
	"loss": 0.3697,
	"num_input_tokens_seen": 6759774,
	"step": 174,
	"train_runtime": 701.4628,
	"train_tokens_per_second": 9636.683
	},
	{
	"epoch": 1.1914529914529914,
	"grad_norm": 1.3296136856079102,
	"learning_rate": 8.304498269896194e-06,
	"loss": 0.3546,
	"num_input_tokens_seen": 6802974,
	"step": 175,
	"train_runtime": 704.9944,
	"train_tokens_per_second": 9649.685
	},
	{
	"epoch": 1.1982905982905983,
	"grad_norm": 1.3747283220291138,
	"learning_rate": 8.23529411764706e-06,
	"loss": 0.3864,
	"num_input_tokens_seen": 6838686,
	"step": 176,
	"train_runtime": 707.9671,
	"train_tokens_per_second": 9659.61
	},
	{
	"epoch": 1.205128205128205,
	"grad_norm": 1.498785138130188,
	"learning_rate": 8.166089965397924e-06,
	"loss": 0.4183,
	"num_input_tokens_seen": 6873086,
	"step": 177,
	"train_runtime": 710.8358,
	"train_tokens_per_second": 9669.021
	},
	{
	"epoch": 1.2119658119658119,
	"grad_norm": 1.4664194583892822,
	"learning_rate": 8.09688581314879e-06,
	"loss": 0.3918,
	"num_input_tokens_seen": 6916254,
	"step": 178,
	"train_runtime": 714.3711,
	"train_tokens_per_second": 9681.599
	},
	{
	"epoch": 1.218803418803419,
	"grad_norm": 1.299894094467163,
	"learning_rate": 8.027681660899655e-06,
	"loss": 0.3753,
	"num_input_tokens_seen": 6957598,
	"step": 179,
	"train_runtime": 717.7521,
	"train_tokens_per_second": 9693.594
	},
	{
	"epoch": 1.2256410256410257,
	"grad_norm": 1.4512171745300293,
	"learning_rate": 7.958477508650519e-06,
	"loss": 0.4402,
	"num_input_tokens_seen": 6991550,
	"step": 180,
	"train_runtime": 720.5836,
	"train_tokens_per_second": 9702.621
	},
	{
	"epoch": 1.2256410256410257,
	"eval_loss": 0.4540201723575592,
	"eval_runtime": 5.6718,
	"eval_samples_per_second": 175.959,
	"eval_steps_per_second": 5.642,
	"num_input_tokens_seen": 6991550,
	"step": 180
	},
	{
	"epoch": 1.2324786324786325,
	"grad_norm": 1.360262393951416,
	"learning_rate": 7.889273356401385e-06,
	"loss": 0.3946,
	"num_input_tokens_seen": 7025758,
	"step": 181,
	"train_runtime": 747.3736,
	"train_tokens_per_second": 9400.598
	},
	{
	"epoch": 1.2393162393162394,
	"grad_norm": 1.4997259378433228,
	"learning_rate": 7.82006920415225e-06,
	"loss": 0.4529,
	"num_input_tokens_seen": 7064478,
	"step": 182,
	"train_runtime": 750.5493,
	"train_tokens_per_second": 9412.41
	},
	{
	"epoch": 1.2461538461538462,
	"grad_norm": 1.585199236869812,
	"learning_rate": 7.750865051903114e-06,
	"loss": 0.4257,
	"num_input_tokens_seen": 7089886,
	"step": 183,
	"train_runtime": 752.7622,
	"train_tokens_per_second": 9418.493
	},
	{
	"epoch": 1.252991452991453,
	"grad_norm": 1.3399189710617065,
	"learning_rate": 7.68166089965398e-06,
	"loss": 0.3918,
	"num_input_tokens_seen": 7134814,
	"step": 184,
	"train_runtime": 756.4288,
	"train_tokens_per_second": 9432.235
	},
	{
	"epoch": 1.2598290598290598,
	"grad_norm": 1.4995014667510986,
	"learning_rate": 7.612456747404845e-06,
	"loss": 0.481,
	"num_input_tokens_seen": 7178046,
	"step": 185,
	"train_runtime": 760.0338,
	"train_tokens_per_second": 9444.377
	},
	{
	"epoch": 1.2666666666666666,
	"grad_norm": 1.4340060949325562,
	"learning_rate": 7.5432525951557104e-06,
	"loss": 0.4483,
	"num_input_tokens_seen": 7212158,
	"step": 186,
	"train_runtime": 762.8937,
	"train_tokens_per_second": 9453.686
	},
	{
	"epoch": 1.2735042735042734,
	"grad_norm": 1.4529216289520264,
	"learning_rate": 7.474048442906575e-06,
	"loss": 0.4695,
	"num_input_tokens_seen": 7245886,
	"step": 187,
	"train_runtime": 765.7249,
	"train_tokens_per_second": 9462.779
	},
	{
	"epoch": 1.2803418803418802,
	"grad_norm": 1.3756847381591797,
	"learning_rate": 7.40484429065744e-06,
	"loss": 0.4277,
	"num_input_tokens_seen": 7287646,
	"step": 188,
	"train_runtime": 769.1612,
	"train_tokens_per_second": 9474.797
	},
	{
	"epoch": 1.287179487179487,
	"grad_norm": 1.4164284467697144,
	"learning_rate": 7.335640138408306e-06,
	"loss": 0.4475,
	"num_input_tokens_seen": 7326238,
	"step": 189,
	"train_runtime": 772.3229,
	"train_tokens_per_second": 9485.978
	},
	{
	"epoch": 1.294017094017094,
	"grad_norm": 1.3923799991607666,
	"learning_rate": 7.2664359861591705e-06,
	"loss": 0.3954,
	"num_input_tokens_seen": 7364894,
	"step": 190,
	"train_runtime": 775.5465,
	"train_tokens_per_second": 9496.393
	},
	{
	"epoch": 1.300854700854701,
	"grad_norm": 1.3876417875289917,
	"learning_rate": 7.197231833910035e-06,
	"loss": 0.4325,
	"num_input_tokens_seen": 7401534,
	"step": 191,
	"train_runtime": 778.6241,
	"train_tokens_per_second": 9505.915
	},
	{
	"epoch": 1.3076923076923077,
	"grad_norm": 1.337653398513794,
	"learning_rate": 7.128027681660901e-06,
	"loss": 0.3433,
	"num_input_tokens_seen": 7442494,
	"step": 192,
	"train_runtime": 782.0513,
	"train_tokens_per_second": 9516.631
	},
	{
	"epoch": 1.3145299145299145,
	"grad_norm": 1.3862982988357544,
	"learning_rate": 7.058823529411766e-06,
	"loss": 0.3943,
	"num_input_tokens_seen": 7487582,
	"step": 193,
	"train_runtime": 785.7613,
	"train_tokens_per_second": 9529.079
	},
	{
	"epoch": 1.3213675213675213,
	"grad_norm": 1.5185152292251587,
	"learning_rate": 6.989619377162631e-06,
	"loss": 0.3824,
	"num_input_tokens_seen": 7520734,
	"step": 194,
	"train_runtime": 788.5665,
	"train_tokens_per_second": 9537.222
	},
	{
	"epoch": 1.3282051282051281,
	"grad_norm": 1.3591375350952148,
	"learning_rate": 6.9204152249134946e-06,
	"loss": 0.3988,
	"num_input_tokens_seen": 7557854,
	"step": 195,
	"train_runtime": 791.673,
	"train_tokens_per_second": 9546.686
	},
	{
	"epoch": 1.335042735042735,
	"grad_norm": 1.5463569164276123,
	"learning_rate": 6.85121107266436e-06,
	"loss": 0.4155,
	"num_input_tokens_seen": 7594110,
	"step": 196,
	"train_runtime": 794.7163,
	"train_tokens_per_second": 9555.75
	},
	{
	"epoch": 1.341880341880342,
	"grad_norm": 1.3969792127609253,
	"learning_rate": 6.782006920415225e-06,
	"loss": 0.4054,
	"num_input_tokens_seen": 7634878,
	"step": 197,
	"train_runtime": 798.1263,
	"train_tokens_per_second": 9566.003
	},
	{
	"epoch": 1.3487179487179488,
	"grad_norm": 1.4635127782821655,
	"learning_rate": 6.71280276816609e-06,
	"loss": 0.3989,
	"num_input_tokens_seen": 7668894,
	"step": 198,
	"train_runtime": 801.0008,
	"train_tokens_per_second": 9574.14
	},
	{
	"epoch": 1.3555555555555556,
	"grad_norm": 1.4537571668624878,
	"learning_rate": 6.6435986159169555e-06,
	"loss": 0.4503,
	"num_input_tokens_seen": 7705598,
	"step": 199,
	"train_runtime": 804.0791,
	"train_tokens_per_second": 9583.135
	},
	{
	"epoch": 1.3623931623931624,
	"grad_norm": 1.5418137311935425,
	"learning_rate": 6.57439446366782e-06,
	"loss": 0.4482,
	"num_input_tokens_seen": 7737630,
	"step": 200,
	"train_runtime": 806.81,
	"train_tokens_per_second": 9590.399
	},
	{
	"epoch": 1.3692307692307693,
	"grad_norm": 1.512305736541748,
	"learning_rate": 6.505190311418685e-06,
	"loss": 0.4625,
	"num_input_tokens_seen": 7772446,
	"step": 201,
	"train_runtime": 809.7648,
	"train_tokens_per_second": 9598.399
	},
	{
	"epoch": 1.376068376068376,
	"grad_norm": 1.2927024364471436,
	"learning_rate": 6.435986159169551e-06,
	"loss": 0.434,
	"num_input_tokens_seen": 7825886,
	"step": 202,
	"train_runtime": 814.22,
	"train_tokens_per_second": 9611.513
	},
	{
	"epoch": 1.3829059829059829,
	"grad_norm": 1.3422515392303467,
	"learning_rate": 6.3667820069204156e-06,
	"loss": 0.4471,
	"num_input_tokens_seen": 7877566,
	"step": 203,
	"train_runtime": 818.5631,
	"train_tokens_per_second": 9623.652
	},
	{
	"epoch": 1.3897435897435897,
	"grad_norm": 1.5034905672073364,
	"learning_rate": 6.29757785467128e-06,
	"loss": 0.4317,
	"num_input_tokens_seen": 7915038,
	"step": 204,
	"train_runtime": 821.6717,
	"train_tokens_per_second": 9632.847
	},
	{
	"epoch": 1.3965811965811965,
	"grad_norm": 1.4573144912719727,
	"learning_rate": 6.228373702422146e-06,
	"loss": 0.4127,
	"num_input_tokens_seen": 7945950,
	"step": 205,
	"train_runtime": 824.2951,
	"train_tokens_per_second": 9639.691
	},
	{
	"epoch": 1.4034188034188033,
	"grad_norm": 1.424501657485962,
	"learning_rate": 6.159169550173011e-06,
	"loss": 0.4444,
	"num_input_tokens_seen": 7981854,
	"step": 206,
	"train_runtime": 827.3346,
	"train_tokens_per_second": 9647.674
	},
	{
	"epoch": 1.4102564102564101,
	"grad_norm": 1.52495276927948,
	"learning_rate": 6.089965397923876e-06,
	"loss": 0.4204,
	"num_input_tokens_seen": 8024286,
	"step": 207,
	"train_runtime": 830.8818,
	"train_tokens_per_second": 9657.554
	},
	{
	"epoch": 1.4170940170940172,
	"grad_norm": 1.3244974613189697,
	"learning_rate": 6.020761245674741e-06,
	"loss": 0.3899,
	"num_input_tokens_seen": 8062558,
	"step": 208,
	"train_runtime": 834.1195,
	"train_tokens_per_second": 9665.951
	},
	{
	"epoch": 1.423931623931624,
	"grad_norm": 1.4783118963241577,
	"learning_rate": 5.951557093425606e-06,
	"loss": 0.4001,
	"num_input_tokens_seen": 8097406,
	"step": 209,
	"train_runtime": 837.0288,
	"train_tokens_per_second": 9673.987
	},
	{
	"epoch": 1.4307692307692308,
	"grad_norm": 1.4629509449005127,
	"learning_rate": 5.882352941176471e-06,
	"loss": 0.4112,
	"num_input_tokens_seen": 8130526,
	"step": 210,
	"train_runtime": 839.833,
	"train_tokens_per_second": 9681.123
	},
	{
	"epoch": 1.4307692307692308,
	"eval_loss": 0.4539625346660614,
	"eval_runtime": 5.7425,
	"eval_samples_per_second": 173.792,
	"eval_steps_per_second": 5.572,
	"num_input_tokens_seen": 8130526,
	"step": 210
	},
	{
	"epoch": 1.4376068376068376,
	"grad_norm": 1.35781991481781,
	"learning_rate": 5.8131487889273366e-06,
	"loss": 0.376,
	"num_input_tokens_seen": 8170302,
	"step": 211,
	"train_runtime": 867.0208,
	"train_tokens_per_second": 9423.421
	},
	{
	"epoch": 1.4444444444444444,
	"grad_norm": 1.5642489194869995,
	"learning_rate": 5.743944636678201e-06,
	"loss": 0.4331,
	"num_input_tokens_seen": 8204958,
	"step": 212,
	"train_runtime": 869.9438,
	"train_tokens_per_second": 9431.596
	},
	{
	"epoch": 1.4512820512820512,
	"grad_norm": 1.4987717866897583,
	"learning_rate": 5.674740484429066e-06,
	"loss": 0.4854,
	"num_input_tokens_seen": 8244126,
	"step": 213,
	"train_runtime": 873.217,
	"train_tokens_per_second": 9441.097
	},
	{
	"epoch": 1.458119658119658,
	"grad_norm": 1.4428235292434692,
	"learning_rate": 5.605536332179932e-06,
	"loss": 0.4259,
	"num_input_tokens_seen": 8286910,
	"step": 214,
	"train_runtime": 876.729,
	"train_tokens_per_second": 9452.077
	},
	{
	"epoch": 1.464957264957265,
	"grad_norm": 1.5963438749313354,
	"learning_rate": 5.536332179930797e-06,
	"loss": 0.4945,
	"num_input_tokens_seen": 8322110,
	"step": 215,
	"train_runtime": 879.6762,
	"train_tokens_per_second": 9460.425
	},
	{
	"epoch": 1.471794871794872,
	"grad_norm": 1.3209023475646973,
	"learning_rate": 5.4671280276816615e-06,
	"loss": 0.5295,
	"num_input_tokens_seen": 8360894,
	"step": 216,
	"train_runtime": 882.8992,
	"train_tokens_per_second": 9469.818
	},
	{
	"epoch": 1.4786324786324787,
	"grad_norm": 1.3461143970489502,
	"learning_rate": 5.397923875432527e-06,
	"loss": 0.3801,
	"num_input_tokens_seen": 8393630,
	"step": 217,
	"train_runtime": 885.6709,
	"train_tokens_per_second": 9477.143
	},
	{
	"epoch": 1.4854700854700855,
	"grad_norm": 1.5508912801742554,
	"learning_rate": 5.328719723183391e-06,
	"loss": 0.479,
	"num_input_tokens_seen": 8422910,
	"step": 218,
	"train_runtime": 888.188,
	"train_tokens_per_second": 9483.251
	},
	{
	"epoch": 1.4923076923076923,
	"grad_norm": 1.4699424505233765,
	"learning_rate": 5.259515570934256e-06,
	"loss": 0.4195,
	"num_input_tokens_seen": 8458014,
	"step": 219,
	"train_runtime": 891.1228,
	"train_tokens_per_second": 9491.412
	},
	{
	"epoch": 1.4991452991452991,
	"grad_norm": 1.3626216650009155,
	"learning_rate": 5.190311418685121e-06,
	"loss": 0.4021,
	"num_input_tokens_seen": 8500670,
	"step": 220,
	"train_runtime": 894.6464,
	"train_tokens_per_second": 9501.709
	},
	{
	"epoch": 1.505982905982906,
	"grad_norm": 1.6453019380569458,
	"learning_rate": 5.121107266435986e-06,
	"loss": 0.4276,
	"num_input_tokens_seen": 8531038,
	"step": 221,
	"train_runtime": 897.2331,
	"train_tokens_per_second": 9508.162
	},
	{
	"epoch": 1.5128205128205128,
	"grad_norm": 1.3846372365951538,
	"learning_rate": 5.051903114186851e-06,
	"loss": 0.3921,
	"num_input_tokens_seen": 8562014,
	"step": 222,
	"train_runtime": 899.8779,
	"train_tokens_per_second": 9514.64
	},
	{
	"epoch": 1.5196581196581196,
	"grad_norm": 1.5681668519973755,
	"learning_rate": 4.982698961937717e-06,
	"loss": 0.4713,
	"num_input_tokens_seen": 8593054,
	"step": 223,
	"train_runtime": 902.506,
	"train_tokens_per_second": 9521.326
	},
	{
	"epoch": 1.5264957264957264,
	"grad_norm": 1.4132840633392334,
	"learning_rate": 4.913494809688582e-06,
	"loss": 0.3994,
	"num_input_tokens_seen": 8628254,
	"step": 224,
	"train_runtime": 905.4606,
	"train_tokens_per_second": 9529.133
	},
	{
	"epoch": 1.5333333333333332,
	"grad_norm": 1.4908711910247803,
	"learning_rate": 4.8442906574394464e-06,
	"loss": 0.4021,
	"num_input_tokens_seen": 8689630,
	"step": 225,
	"train_runtime": 910.705,
	"train_tokens_per_second": 9541.652
	},
	{
	"epoch": 1.54017094017094,
	"grad_norm": 1.1932868957519531,
	"learning_rate": 4.775086505190312e-06,
	"loss": 0.3898,
	"num_input_tokens_seen": 8734526,
	"step": 226,
	"train_runtime": 914.4429,
	"train_tokens_per_second": 9551.746
	},
	{
	"epoch": 1.547008547008547,
	"grad_norm": 1.253638744354248,
	"learning_rate": 4.705882352941177e-06,
	"loss": 0.4028,
	"num_input_tokens_seen": 8799582,
	"step": 227,
	"train_runtime": 919.89,
	"train_tokens_per_second": 9565.907
	},
	{
	"epoch": 1.5538461538461539,
	"grad_norm": 1.322104573249817,
	"learning_rate": 4.636678200692042e-06,
	"loss": 0.3858,
	"num_input_tokens_seen": 8840638,
	"step": 228,
	"train_runtime": 923.2956,
	"train_tokens_per_second": 9575.089
	},
	{
	"epoch": 1.5606837606837607,
	"grad_norm": 1.377384901046753,
	"learning_rate": 4.567474048442907e-06,
	"loss": 0.3577,
	"num_input_tokens_seen": 8895710,
	"step": 229,
	"train_runtime": 927.776,
	"train_tokens_per_second": 9588.209
	},
	{
	"epoch": 1.5675213675213675,
	"grad_norm": 1.4464075565338135,
	"learning_rate": 4.498269896193772e-06,
	"loss": 0.3797,
	"num_input_tokens_seen": 8940670,
	"step": 230,
	"train_runtime": 931.4748,
	"train_tokens_per_second": 9598.402
	},
	{
	"epoch": 1.5743589743589743,
	"grad_norm": 1.4492119550704956,
	"learning_rate": 4.429065743944637e-06,
	"loss": 0.3816,
	"num_input_tokens_seen": 8984350,
	"step": 231,
	"train_runtime": 935.1105,
	"train_tokens_per_second": 9607.795
	},
	{
	"epoch": 1.5811965811965814,
	"grad_norm": 1.6883492469787598,
	"learning_rate": 4.359861591695503e-06,
	"loss": 0.4917,
	"num_input_tokens_seen": 9019390,
	"step": 232,
	"train_runtime": 938.0738,
	"train_tokens_per_second": 9614.797
	},
	{
	"epoch": 1.5880341880341882,
	"grad_norm": 1.4682198762893677,
	"learning_rate": 4.2906574394463675e-06,
	"loss": 0.3746,
	"num_input_tokens_seen": 9059294,
	"step": 233,
	"train_runtime": 941.4183,
	"train_tokens_per_second": 9623.028
	},
	{
	"epoch": 1.594871794871795,
	"grad_norm": 1.3500458002090454,
	"learning_rate": 4.221453287197232e-06,
	"loss": 0.4348,
	"num_input_tokens_seen": 9097854,
	"step": 234,
	"train_runtime": 944.6072,
	"train_tokens_per_second": 9631.362
	},
	{
	"epoch": 1.6017094017094018,
	"grad_norm": 1.5531758069992065,
	"learning_rate": 4.152249134948097e-06,
	"loss": 0.4281,
	"num_input_tokens_seen": 9131518,
	"step": 235,
	"train_runtime": 947.4585,
	"train_tokens_per_second": 9637.908
	},
	{
	"epoch": 1.6085470085470086,
	"grad_norm": 1.503575325012207,
	"learning_rate": 4.083044982698962e-06,
	"loss": 0.4181,
	"num_input_tokens_seen": 9170014,
	"step": 236,
	"train_runtime": 950.6625,
	"train_tokens_per_second": 9645.919
	},
	{
	"epoch": 1.6153846153846154,
	"grad_norm": 1.5178425312042236,
	"learning_rate": 4.0138408304498275e-06,
	"loss": 0.4313,
	"num_input_tokens_seen": 9211198,
	"step": 237,
	"train_runtime": 954.0784,
	"train_tokens_per_second": 9654.551
	},
	{
	"epoch": 1.6222222222222222,
	"grad_norm": 1.4492536783218384,
	"learning_rate": 3.944636678200692e-06,
	"loss": 0.4175,
	"num_input_tokens_seen": 9240734,
	"step": 238,
	"train_runtime": 956.6363,
	"train_tokens_per_second": 9659.611
	},
	{
	"epoch": 1.629059829059829,
	"grad_norm": 1.4955025911331177,
	"learning_rate": 3.875432525951557e-06,
	"loss": 0.4145,
	"num_input_tokens_seen": 9279966,
	"step": 239,
	"train_runtime": 959.9009,
	"train_tokens_per_second": 9667.63
	},
	{
	"epoch": 1.6358974358974359,
	"grad_norm": 1.3603522777557373,
	"learning_rate": 3.8062283737024224e-06,
	"loss": 0.3571,
	"num_input_tokens_seen": 9318206,
	"step": 240,
	"train_runtime": 963.044,
	"train_tokens_per_second": 9675.784
	},
	{
	"epoch": 1.6358974358974359,
	"eval_loss": 0.4540530741214752,
	"eval_runtime": 5.7145,
	"eval_samples_per_second": 174.643,
	"eval_steps_per_second": 5.6,
	"num_input_tokens_seen": 9318206,
	"step": 240
	},
	{
	"epoch": 1.6427350427350427,
	"grad_norm": 1.3077824115753174,
	"learning_rate": 3.7370242214532876e-06,
	"loss": 0.3903,
	"num_input_tokens_seen": 9357278,
	"step": 241,
	"train_runtime": 989.9944,
	"train_tokens_per_second": 9451.849
	},
	{
	"epoch": 1.6495726495726495,
	"grad_norm": 1.4134607315063477,
	"learning_rate": 3.667820069204153e-06,
	"loss": 0.4299,
	"num_input_tokens_seen": 9401246,
	"step": 242,
	"train_runtime": 993.6355,
	"train_tokens_per_second": 9461.464
	},
	{
	"epoch": 1.6564102564102563,
	"grad_norm": 1.3824211359024048,
	"learning_rate": 3.5986159169550177e-06,
	"loss": 0.4513,
	"num_input_tokens_seen": 9439326,
	"step": 243,
	"train_runtime": 996.8489,
	"train_tokens_per_second": 9469.164
	},
	{
	"epoch": 1.6632478632478631,
	"grad_norm": 1.3873765468597412,
	"learning_rate": 3.529411764705883e-06,
	"loss": 0.4037,
	"num_input_tokens_seen": 9474398,
	"step": 244,
	"train_runtime": 999.7516,
	"train_tokens_per_second": 9476.752
	},
	{
	"epoch": 1.67008547008547,
	"grad_norm": 1.5342257022857666,
	"learning_rate": 3.4602076124567473e-06,
	"loss": 0.4479,
	"num_input_tokens_seen": 9504830,
	"step": 245,
	"train_runtime": 1002.3332,
	"train_tokens_per_second": 9482.704
	},
	{
	"epoch": 1.676923076923077,
	"grad_norm": 1.4302300214767456,
	"learning_rate": 3.3910034602076125e-06,
	"loss": 0.4137,
	"num_input_tokens_seen": 9550430,
	"step": 246,
	"train_runtime": 1006.0494,
	"train_tokens_per_second": 9493.003
	},
	{
	"epoch": 1.6837606837606838,
	"grad_norm": 1.516709566116333,
	"learning_rate": 3.3217993079584777e-06,
	"loss": 0.4088,
	"num_input_tokens_seen": 9583806,
	"step": 247,
	"train_runtime": 1008.8176,
	"train_tokens_per_second": 9500.038
	},
	{
	"epoch": 1.6905982905982906,
	"grad_norm": 1.4347656965255737,
	"learning_rate": 3.2525951557093425e-06,
	"loss": 0.3991,
	"num_input_tokens_seen": 9644798,
	"step": 248,
	"train_runtime": 1013.8333,
	"train_tokens_per_second": 9513.199
	},
	{
	"epoch": 1.6974358974358974,
	"grad_norm": 1.2900501489639282,
	"learning_rate": 3.1833910034602078e-06,
	"loss": 0.3854,
	"num_input_tokens_seen": 9699294,
	"step": 249,
	"train_runtime": 1018.4019,
	"train_tokens_per_second": 9524.034
	},
	{
	"epoch": 1.7042735042735044,
	"grad_norm": 1.4763895273208618,
	"learning_rate": 3.114186851211073e-06,
	"loss": 0.3875,
	"num_input_tokens_seen": 9738238,
	"step": 250,
	"train_runtime": 1021.6841,
	"train_tokens_per_second": 9531.554
	},
	{
	"epoch": 1.7111111111111112,
	"grad_norm": 1.5247527360916138,
	"learning_rate": 3.044982698961938e-06,
	"loss": 0.4356,
	"num_input_tokens_seen": 9774910,
	"step": 251,
	"train_runtime": 1024.7713,
	"train_tokens_per_second": 9538.626
	},
	{
	"epoch": 1.717948717948718,
	"grad_norm": 1.3282755613327026,
	"learning_rate": 2.975778546712803e-06,
	"loss": 0.3761,
	"num_input_tokens_seen": 9816158,
	"step": 252,
	"train_runtime": 1028.1704,
	"train_tokens_per_second": 9547.21
	},
	{
	"epoch": 1.7247863247863249,
	"grad_norm": 1.5729610919952393,
	"learning_rate": 2.9065743944636683e-06,
	"loss": 0.411,
	"num_input_tokens_seen": 9860414,
	"step": 253,
	"train_runtime": 1031.7918,
	"train_tokens_per_second": 9556.593
	},
	{
	"epoch": 1.7316239316239317,
	"grad_norm": 1.3676373958587646,
	"learning_rate": 2.837370242214533e-06,
	"loss": 0.3885,
	"num_input_tokens_seen": 9895518,
	"step": 254,
	"train_runtime": 1034.7479,
	"train_tokens_per_second": 9563.216
	},
	{
	"epoch": 1.7384615384615385,
	"grad_norm": 1.3123220205307007,
	"learning_rate": 2.7681660899653983e-06,
	"loss": 0.3553,
	"num_input_tokens_seen": 9931838,
	"step": 255,
	"train_runtime": 1037.7767,
	"train_tokens_per_second": 9570.304
	},
	{
	"epoch": 1.7452991452991453,
	"grad_norm": 1.4366247653961182,
	"learning_rate": 2.6989619377162636e-06,
	"loss": 0.42,
	"num_input_tokens_seen": 9975582,
	"step": 256,
	"train_runtime": 1041.3876,
	"train_tokens_per_second": 9579.125
	},
	{
	"epoch": 1.7521367521367521,
	"grad_norm": 1.330451488494873,
	"learning_rate": 2.629757785467128e-06,
	"loss": 0.3794,
	"num_input_tokens_seen": 10009150,
	"step": 257,
	"train_runtime": 1044.2188,
	"train_tokens_per_second": 9585.3
	},
	{
	"epoch": 1.758974358974359,
	"grad_norm": 1.1759202480316162,
	"learning_rate": 2.560553633217993e-06,
	"loss": 0.3586,
	"num_input_tokens_seen": 10082110,
	"step": 258,
	"train_runtime": 1050.4616,
	"train_tokens_per_second": 9597.79
	},
	{
	"epoch": 1.7658119658119658,
	"grad_norm": 1.5406081676483154,
	"learning_rate": 2.4913494809688584e-06,
	"loss": 0.4238,
	"num_input_tokens_seen": 10116126,
	"step": 259,
	"train_runtime": 1053.3071,
	"train_tokens_per_second": 9604.156
	},
	{
	"epoch": 1.7726495726495726,
	"grad_norm": 1.5332951545715332,
	"learning_rate": 2.4221453287197232e-06,
	"loss": 0.4283,
	"num_input_tokens_seen": 10152958,
	"step": 260,
	"train_runtime": 1056.3542,
	"train_tokens_per_second": 9611.32
	},
	{
	"epoch": 1.7794871794871794,
	"grad_norm": 1.3916773796081543,
	"learning_rate": 2.3529411764705885e-06,
	"loss": 0.4203,
	"num_input_tokens_seen": 10190046,
	"step": 261,
	"train_runtime": 1059.4345,
	"train_tokens_per_second": 9618.382
	},
	{
	"epoch": 1.7863247863247862,
	"grad_norm": 1.3282185792922974,
	"learning_rate": 2.2837370242214537e-06,
	"loss": 0.3668,
	"num_input_tokens_seen": 10230334,
	"step": 262,
	"train_runtime": 1062.7483,
	"train_tokens_per_second": 9626.3
	},
	{
	"epoch": 1.793162393162393,
	"grad_norm": 1.3999195098876953,
	"learning_rate": 2.2145328719723185e-06,
	"loss": 0.3823,
	"num_input_tokens_seen": 10262302,
	"step": 263,
	"train_runtime": 1065.4169,
	"train_tokens_per_second": 9632.194
	},
	{
	"epoch": 1.8,
	"grad_norm": 1.483365535736084,
	"learning_rate": 2.1453287197231837e-06,
	"loss": 0.437,
	"num_input_tokens_seen": 10306046,
	"step": 264,
	"train_runtime": 1069.0582,
	"train_tokens_per_second": 9640.304
	},
	{
	"epoch": 1.8068376068376069,
	"grad_norm": 1.5923428535461426,
	"learning_rate": 2.0761245674740485e-06,
	"loss": 0.4307,
	"num_input_tokens_seen": 10335582,
	"step": 265,
	"train_runtime": 1071.5688,
	"train_tokens_per_second": 9645.281
	},
	{
	"epoch": 1.8136752136752137,
	"grad_norm": 1.4312759637832642,
	"learning_rate": 2.0069204152249138e-06,
	"loss": 0.4697,
	"num_input_tokens_seen": 10369214,
	"step": 266,
	"train_runtime": 1074.4014,
	"train_tokens_per_second": 9651.154
	},
	{
	"epoch": 1.8205128205128205,
	"grad_norm": 1.4313794374465942,
	"learning_rate": 1.9377162629757786e-06,
	"loss": 0.4429,
	"num_input_tokens_seen": 10415998,
	"step": 267,
	"train_runtime": 1078.2698,
	"train_tokens_per_second": 9659.918
	},
	{
	"epoch": 1.8273504273504273,
	"grad_norm": 1.440711259841919,
	"learning_rate": 1.8685121107266438e-06,
	"loss": 0.427,
	"num_input_tokens_seen": 10457470,
	"step": 268,
	"train_runtime": 1081.7021,
	"train_tokens_per_second": 9667.607
	},
	{
	"epoch": 1.8341880341880343,
	"grad_norm": 1.5918058156967163,
	"learning_rate": 1.7993079584775088e-06,
	"loss": 0.4871,
	"num_input_tokens_seen": 10496382,
	"step": 269,
	"train_runtime": 1084.9586,
	"train_tokens_per_second": 9674.454
	},
	{
	"epoch": 1.8410256410256411,
	"grad_norm": 1.4235551357269287,
	"learning_rate": 1.7301038062283736e-06,
	"loss": 0.5227,
	"num_input_tokens_seen": 10529278,
	"step": 270,
	"train_runtime": 1087.7757,
	"train_tokens_per_second": 9679.64
	},
	{
	"epoch": 1.8410256410256411,
	"eval_loss": 0.45376113057136536,
	"eval_runtime": 5.8063,
	"eval_samples_per_second": 171.881,
	"eval_steps_per_second": 5.511,
	"num_input_tokens_seen": 10529278,
	"step": 270
	},
	{
	"epoch": 1.847863247863248,
	"grad_norm": 1.3479825258255005,
	"learning_rate": 1.6608996539792389e-06,
	"loss": 0.4222,
	"num_input_tokens_seen": 10573694,
	"step": 271,
	"train_runtime": 1115.6749,
	"train_tokens_per_second": 9477.397
	},
	{
	"epoch": 1.8547008547008548,
	"grad_norm": 1.4445607662200928,
	"learning_rate": 1.5916955017301039e-06,
	"loss": 0.4388,
	"num_input_tokens_seen": 10615710,
	"step": 272,
	"train_runtime": 1119.1547,
	"train_tokens_per_second": 9485.471
	},
	{
	"epoch": 1.8615384615384616,
	"grad_norm": 1.6135919094085693,
	"learning_rate": 1.522491349480969e-06,
	"loss": 0.4083,
	"num_input_tokens_seen": 10651486,
	"step": 273,
	"train_runtime": 1122.1591,
	"train_tokens_per_second": 9491.957
	},
	{
	"epoch": 1.8683760683760684,
	"grad_norm": 1.567018747329712,
	"learning_rate": 1.4532871972318341e-06,
	"loss": 0.4997,
	"num_input_tokens_seen": 10684862,
	"step": 274,
	"train_runtime": 1124.9704,
	"train_tokens_per_second": 9497.905
	},
	{
	"epoch": 1.8752136752136752,
	"grad_norm": 1.5761120319366455,
	"learning_rate": 1.3840830449826992e-06,
	"loss": 0.4252,
	"num_input_tokens_seen": 10718174,
	"step": 275,
	"train_runtime": 1127.7815,
	"train_tokens_per_second": 9503.768
	},
	{
	"epoch": 1.882051282051282,
	"grad_norm": 1.516058325767517,
	"learning_rate": 1.314878892733564e-06,
	"loss": 0.4248,
	"num_input_tokens_seen": 10751358,
	"step": 276,
	"train_runtime": 1130.5976,
	"train_tokens_per_second": 9509.447
	},
	{
	"epoch": 1.8888888888888888,
	"grad_norm": 1.4651241302490234,
	"learning_rate": 1.2456747404844292e-06,
	"loss": 0.4218,
	"num_input_tokens_seen": 10785278,
	"step": 277,
	"train_runtime": 1133.4807,
	"train_tokens_per_second": 9515.185
	},
	{
	"epoch": 1.8957264957264957,
	"grad_norm": 1.470118761062622,
	"learning_rate": 1.1764705882352942e-06,
	"loss": 0.4166,
	"num_input_tokens_seen": 10829086,
	"step": 278,
	"train_runtime": 1137.1675,
	"train_tokens_per_second": 9522.859
	},
	{
	"epoch": 1.9025641025641025,
	"grad_norm": 1.4548068046569824,
	"learning_rate": 1.1072664359861592e-06,
	"loss": 0.4197,
	"num_input_tokens_seen": 10869246,
	"step": 279,
	"train_runtime": 1140.5065,
	"train_tokens_per_second": 9530.192
	},
	{
	"epoch": 1.9094017094017093,
	"grad_norm": 1.4155503511428833,
	"learning_rate": 1.0380622837370243e-06,
	"loss": 0.3839,
	"num_input_tokens_seen": 10908350,
	"step": 280,
	"train_runtime": 1143.7836,
	"train_tokens_per_second": 9537.075
	},
	{
	"epoch": 1.916239316239316,
	"grad_norm": 1.5686092376708984,
	"learning_rate": 9.688581314878893e-07,
	"loss": 0.4483,
	"num_input_tokens_seen": 10936542,
	"step": 281,
	"train_runtime": 1146.1984,
	"train_tokens_per_second": 9541.578
	},
	{
	"epoch": 1.9230769230769231,
	"grad_norm": 1.4835528135299683,
	"learning_rate": 8.996539792387544e-07,
	"loss": 0.4117,
	"num_input_tokens_seen": 10970366,
	"step": 282,
	"train_runtime": 1149.0912,
	"train_tokens_per_second": 9546.994
	},
	{
	"epoch": 1.92991452991453,
	"grad_norm": 1.5472915172576904,
	"learning_rate": 8.304498269896194e-07,
	"loss": 0.4556,
	"num_input_tokens_seen": 11005310,
	"step": 283,
	"train_runtime": 1152.0483,
	"train_tokens_per_second": 9552.82
	},
	{
	"epoch": 1.9367521367521368,
	"grad_norm": 1.345395565032959,
	"learning_rate": 7.612456747404845e-07,
	"loss": 0.392,
	"num_input_tokens_seen": 11039710,
	"step": 284,
	"train_runtime": 1154.9982,
	"train_tokens_per_second": 9558.205
	},
	{
	"epoch": 1.9435897435897436,
	"grad_norm": 1.640942931175232,
	"learning_rate": 6.920415224913496e-07,
	"loss": 0.4268,
	"num_input_tokens_seen": 11068126,
	"step": 285,
	"train_runtime": 1157.4718,
	"train_tokens_per_second": 9562.329
	},
	{
	"epoch": 1.9504273504273504,
	"grad_norm": 1.4688515663146973,
	"learning_rate": 6.228373702422146e-07,
	"loss": 0.4585,
	"num_input_tokens_seen": 11100222,
	"step": 286,
	"train_runtime": 1160.1879,
	"train_tokens_per_second": 9567.607
	},
	{
	"epoch": 1.9572649572649574,
	"grad_norm": 1.353981375694275,
	"learning_rate": 5.536332179930796e-07,
	"loss": 0.4352,
	"num_input_tokens_seen": 11142846,
	"step": 287,
	"train_runtime": 1163.7686,
	"train_tokens_per_second": 9574.795
	},
	{
	"epoch": 1.9641025641025642,
	"grad_norm": 1.4443548917770386,
	"learning_rate": 4.844290657439446e-07,
	"loss": 0.4597,
	"num_input_tokens_seen": 11179038,
	"step": 288,
	"train_runtime": 1166.8565,
	"train_tokens_per_second": 9580.474
	},
	{
	"epoch": 1.970940170940171,
	"grad_norm": 1.463306188583374,
	"learning_rate": 4.152249134948097e-07,
	"loss": 0.3708,
	"num_input_tokens_seen": 11216158,
	"step": 289,
	"train_runtime": 1169.997,
	"train_tokens_per_second": 9586.485
	},
	{
	"epoch": 1.9777777777777779,
	"grad_norm": 1.3856064081192017,
	"learning_rate": 3.460207612456748e-07,
	"loss": 0.397,
	"num_input_tokens_seen": 11297406,
	"step": 290,
	"train_runtime": 1177.6388,
	"train_tokens_per_second": 9593.269
	},
	{
	"epoch": 1.9846153846153847,
	"grad_norm": 1.567358136177063,
	"learning_rate": 2.768166089965398e-07,
	"loss": 0.4152,
	"num_input_tokens_seen": 11332094,
	"step": 291,
	"train_runtime": 1180.5843,
	"train_tokens_per_second": 9598.717
	},
	{
	"epoch": 1.9914529914529915,
	"grad_norm": 1.3738164901733398,
	"learning_rate": 2.0761245674740486e-07,
	"loss": 0.3608,
	"num_input_tokens_seen": 11394046,
	"step": 292,
	"train_runtime": 1185.9037,
	"train_tokens_per_second": 9607.902
	},
	{
	"epoch": 1.9982905982905983,
	"grad_norm": 1.5582184791564941,
	"learning_rate": 1.384083044982699e-07,
	"loss": 0.4101,
	"num_input_tokens_seen": 11428286,
	"step": 293,
	"train_runtime": 1188.8125,
	"train_tokens_per_second": 9613.194
	},
	{
	"epoch": 2.0,
	"grad_norm": 2.8240842819213867,
	"learning_rate": 6.920415224913495e-08,
	"loss": 0.3817,
	"num_input_tokens_seen": 11435966,
	"step": 294,
	"train_runtime": 1189.5451,
	"train_tokens_per_second": 9613.731
	}
	],
	"logging_steps": 1,
	"max_steps": 294,
	"num_input_tokens_seen": 11435966,
	"num_train_epochs": 2,
	"save_steps": 30,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 1.9039331026231296e+17,
	"train_batch_size": 32,
	"trial_name": null,
	"trial_params": null
	}