Instructions to use baban/QwenTranslate_English_Russian with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use baban/QwenTranslate_English_Russian with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="baban/QwenTranslate_English_Russian")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("baban/QwenTranslate_English_Russian")
model = AutoModelForCausalLM.from_pretrained("baban/QwenTranslate_English_Russian", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use baban/QwenTranslate_English_Russian with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "baban/QwenTranslate_English_Russian"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baban/QwenTranslate_English_Russian",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/baban/QwenTranslate_English_Russian

SGLang

How to use baban/QwenTranslate_English_Russian with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "baban/QwenTranslate_English_Russian" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baban/QwenTranslate_English_Russian",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "baban/QwenTranslate_English_Russian" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "baban/QwenTranslate_English_Russian",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use baban/QwenTranslate_English_Russian with Docker Model Runner:
```
docker model run hf.co/baban/QwenTranslate_English_Russian
```

QwenTranslate_English_Russian / trainer_state.json

baban

Upload top-level Russian model files only

c1b5321 verified about 1 month ago

Raw

History Blame Contribute Delete

86.3 kB

	{
	"best_global_step": 2931,
	"best_metric": 1.456993818283081,
	"best_model_checkpoint": "/workspace/MT_En_Russian/checkpoint-2931",
	"epoch": 3.0,
	"eval_steps": 5000,
	"global_step": 2931,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"epoch": 0.01024,
	"grad_norm": 1.3203125,
	"learning_rate": 4.9977515176118345e-05,
	"loss": 0.9742774963378906,
	"num_input_tokens_seen": 2055872,
	"step": 10,
	"train_runtime": 80.1213,
	"train_tokens_per_second": 25659.479
	},
	{
	"epoch": 0.02048,
	"grad_norm": 0.82421875,
	"learning_rate": 4.9952567580506e-05,
	"loss": 0.7722540855407715,
	"num_input_tokens_seen": 4033984,
	"step": 20,
	"train_runtime": 149.3376,
	"train_tokens_per_second": 27012.507
	},
	{
	"epoch": 0.03072,
	"grad_norm": 0.83203125,
	"learning_rate": 4.992765730738634e-05,
	"loss": 0.7429269313812256,
	"num_input_tokens_seen": 6010880,
	"step": 30,
	"train_runtime": 217.0261,
	"train_tokens_per_second": 27696.574
	},
	{
	"epoch": 0.04096,
	"grad_norm": 0.7578125,
	"learning_rate": 4.9902784263792476e-05,
	"loss": 0.7194486618041992,
	"num_input_tokens_seen": 7999872,
	"step": 40,
	"train_runtime": 285.3869,
	"train_tokens_per_second": 28031.677
	},
	{
	"epoch": 0.0512,
	"grad_norm": 0.8046875,
	"learning_rate": 4.987794835708133e-05,
	"loss": 0.7071797370910644,
	"num_input_tokens_seen": 10049984,
	"step": 50,
	"train_runtime": 362.3108,
	"train_tokens_per_second": 27738.575
	},
	{
	"epoch": 0.06144,
	"grad_norm": 0.7109375,
	"learning_rate": 4.985314949493234e-05,
	"loss": 0.6854294776916504,
	"num_input_tokens_seen": 12056320,
	"step": 60,
	"train_runtime": 434.4414,
	"train_tokens_per_second": 27751.317
	},
	{
	"epoch": 0.07168,
	"grad_norm": 0.78125,
	"learning_rate": 4.982838758534584e-05,
	"loss": 0.6797103881835938,
	"num_input_tokens_seen": 14026240,
	"step": 70,
	"train_runtime": 503.3773,
	"train_tokens_per_second": 27864.269
	},
	{
	"epoch": 0.08192,
	"grad_norm": 0.74609375,
	"learning_rate": 4.980366253664179e-05,
	"loss": 0.6712788581848145,
	"num_input_tokens_seen": 16040128,
	"step": 80,
	"train_runtime": 573.4812,
	"train_tokens_per_second": 27969.755
	},
	{
	"epoch": 0.09216,
	"grad_norm": 0.7265625,
	"learning_rate": 4.977897425745825e-05,
	"loss": 0.6637749671936035,
	"num_input_tokens_seen": 18062848,
	"step": 90,
	"train_runtime": 645.8153,
	"train_tokens_per_second": 27969.061
	},
	{
	"epoch": 0.1024,
	"grad_norm": 0.703125,
	"learning_rate": 4.975432265674997e-05,
	"loss": 0.6546947479248046,
	"num_input_tokens_seen": 20042944,
	"step": 100,
	"train_runtime": 712.4646,
	"train_tokens_per_second": 28131.846
	},
	{
	"epoch": 0.11264,
	"grad_norm": 0.80859375,
	"learning_rate": 4.972970764378705e-05,
	"loss": 0.6477886199951172,
	"num_input_tokens_seen": 22080512,
	"step": 110,
	"train_runtime": 786.5413,
	"train_tokens_per_second": 28072.921
	},
	{
	"epoch": 0.12288,
	"grad_norm": 0.71875,
	"learning_rate": 4.970512912815344e-05,
	"loss": 0.6376790046691895,
	"num_input_tokens_seen": 24069056,
	"step": 120,
	"train_runtime": 856.9353,
	"train_tokens_per_second": 28087.366
	},
	{
	"epoch": 0.13312,
	"grad_norm": 0.6953125,
	"learning_rate": 4.968058701974564e-05,
	"loss": 0.6336091518402099,
	"num_input_tokens_seen": 26065472,
	"step": 130,
	"train_runtime": 928.6902,
	"train_tokens_per_second": 28066.918
	},
	{
	"epoch": 0.14336,
	"grad_norm": 0.7578125,
	"learning_rate": 4.96560812287712e-05,
	"loss": 0.6247911930084229,
	"num_input_tokens_seen": 28056704,
	"step": 140,
	"train_runtime": 998.2602,
	"train_tokens_per_second": 28105.601
	},
	{
	"epoch": 0.1536,
	"grad_norm": 0.71875,
	"learning_rate": 4.963161166574748e-05,
	"loss": 0.628913402557373,
	"num_input_tokens_seen": 30080064,
	"step": 150,
	"train_runtime": 1072.5324,
	"train_tokens_per_second": 28045.833
	},
	{
	"epoch": 0.16384,
	"grad_norm": 0.68359375,
	"learning_rate": 4.960717824150013e-05,
	"loss": 0.6165059089660645,
	"num_input_tokens_seen": 32061312,
	"step": 160,
	"train_runtime": 1144.3017,
	"train_tokens_per_second": 28018.234
	},
	{
	"epoch": 0.17408,
	"grad_norm": 0.7109375,
	"learning_rate": 4.9582780867161893e-05,
	"loss": 0.6091556549072266,
	"num_input_tokens_seen": 34045312,
	"step": 170,
	"train_runtime": 1213.0324,
	"train_tokens_per_second": 28066.285
	},
	{
	"epoch": 0.18432,
	"grad_norm": 0.78515625,
	"learning_rate": 4.955841945417105e-05,
	"loss": 0.6014857292175293,
	"num_input_tokens_seen": 36041984,
	"step": 180,
	"train_runtime": 1284.4625,
	"train_tokens_per_second": 28059.973
	},
	{
	"epoch": 0.19456,
	"grad_norm": 0.796875,
	"learning_rate": 4.953409391427024e-05,
	"loss": 0.6027495384216308,
	"num_input_tokens_seen": 38059520,
	"step": 190,
	"train_runtime": 1357.0611,
	"train_tokens_per_second": 28045.547
	},
	{
	"epoch": 0.2048,
	"grad_norm": 0.6953125,
	"learning_rate": 4.950980415950502e-05,
	"loss": 0.5953609943389893,
	"num_input_tokens_seen": 40065728,
	"step": 200,
	"train_runtime": 1427.2344,
	"train_tokens_per_second": 28072.282
	},
	{
	"epoch": 0.21504,
	"grad_norm": 0.69921875,
	"learning_rate": 4.9485550102222575e-05,
	"loss": 0.591309928894043,
	"num_input_tokens_seen": 42083968,
	"step": 210,
	"train_runtime": 1501.0913,
	"train_tokens_per_second": 28035.582
	},
	{
	"epoch": 0.22528,
	"grad_norm": 0.73046875,
	"learning_rate": 4.946133165507037e-05,
	"loss": 0.5896960258483886,
	"num_input_tokens_seen": 44073920,
	"step": 220,
	"train_runtime": 1569.5078,
	"train_tokens_per_second": 28081.364
	},
	{
	"epoch": 0.23552,
	"grad_norm": 0.72265625,
	"learning_rate": 4.943714873099483e-05,
	"loss": 0.5793526649475098,
	"num_input_tokens_seen": 46068544,
	"step": 230,
	"train_runtime": 1641.5574,
	"train_tokens_per_second": 28063.925
	},
	{
	"epoch": 0.24576,
	"grad_norm": 0.69921875,
	"learning_rate": 4.9413001243240024e-05,
	"loss": 0.5720966339111329,
	"num_input_tokens_seen": 48050304,
	"step": 240,
	"train_runtime": 1709.9005,
	"train_tokens_per_second": 28101.228
	},
	{
	"epoch": 0.256,
	"grad_norm": 0.765625,
	"learning_rate": 4.938888910534637e-05,
	"loss": 0.5646713256835938,
	"num_input_tokens_seen": 50064064,
	"step": 250,
	"train_runtime": 1781.3025,
	"train_tokens_per_second": 28105.313
	},
	{
	"epoch": 0.26624,
	"grad_norm": 0.7265625,
	"learning_rate": 4.936481223114932e-05,
	"loss": 0.5561185359954834,
	"num_input_tokens_seen": 52031040,
	"step": 260,
	"train_runtime": 1851.0666,
	"train_tokens_per_second": 28108.681
	},
	{
	"epoch": 0.27648,
	"grad_norm": 0.703125,
	"learning_rate": 4.934077053477808e-05,
	"loss": 0.5607205390930176,
	"num_input_tokens_seen": 54084992,
	"step": 270,
	"train_runtime": 1927.1749,
	"train_tokens_per_second": 28064.392
	},
	{
	"epoch": 0.28672,
	"grad_norm": 0.7734375,
	"learning_rate": 4.931676393065431e-05,
	"loss": 0.5556824207305908,
	"num_input_tokens_seen": 56127680,
	"step": 280,
	"train_runtime": 2002.628,
	"train_tokens_per_second": 28027.013
	},
	{
	"epoch": 0.29696,
	"grad_norm": 0.83984375,
	"learning_rate": 4.929279233349088e-05,
	"loss": 0.5448642253875733,
	"num_input_tokens_seen": 58121600,
	"step": 290,
	"train_runtime": 2075.5184,
	"train_tokens_per_second": 28003.414
	},
	{
	"epoch": 0.3072,
	"grad_norm": 0.75390625,
	"learning_rate": 4.926885565829051e-05,
	"loss": 0.5425375461578369,
	"num_input_tokens_seen": 60109120,
	"step": 300,
	"train_runtime": 2144.3013,
	"train_tokens_per_second": 28032.031
	},
	{
	"epoch": 0.31744,
	"grad_norm": 0.77734375,
	"learning_rate": 4.924495382034461e-05,
	"loss": 0.5364805221557617,
	"num_input_tokens_seen": 62133824,
	"step": 310,
	"train_runtime": 2217.0864,
	"train_tokens_per_second": 28024.989
	},
	{
	"epoch": 0.32768,
	"grad_norm": 0.7578125,
	"learning_rate": 4.9221086735231975e-05,
	"loss": 0.5378639221191406,
	"num_input_tokens_seen": 64119488,
	"step": 320,
	"train_runtime": 2287.4996,
	"train_tokens_per_second": 28030.382
	},
	{
	"epoch": 0.33792,
	"grad_norm": 0.73046875,
	"learning_rate": 4.919725431881751e-05,
	"loss": 0.5275018692016602,
	"num_input_tokens_seen": 66128448,
	"step": 330,
	"train_runtime": 2358.2459,
	"train_tokens_per_second": 28041.371
	},
	{
	"epoch": 0.34816,
	"grad_norm": 0.8125,
	"learning_rate": 4.917345648725101e-05,
	"loss": 0.5153440475463867,
	"num_input_tokens_seen": 68116544,
	"step": 340,
	"train_runtime": 2428.0793,
	"train_tokens_per_second": 28053.674
	},
	{
	"epoch": 0.3584,
	"grad_norm": 0.76171875,
	"learning_rate": 4.914969315696596e-05,
	"loss": 0.5185441493988037,
	"num_input_tokens_seen": 70149184,
	"step": 350,
	"train_runtime": 2501.5845,
	"train_tokens_per_second": 28041.901
	},
	{
	"epoch": 0.36864,
	"grad_norm": 0.77734375,
	"learning_rate": 4.912596424467818e-05,
	"loss": 0.5040504455566406,
	"num_input_tokens_seen": 72165696,
	"step": 360,
	"train_runtime": 2574.6205,
	"train_tokens_per_second": 28029.644
	},
	{
	"epoch": 0.37888,
	"grad_norm": 0.80078125,
	"learning_rate": 4.910226966738475e-05,
	"loss": 0.5027976989746094,
	"num_input_tokens_seen": 74176064,
	"step": 370,
	"train_runtime": 2644.8094,
	"train_tokens_per_second": 28045.902
	},
	{
	"epoch": 0.38912,
	"grad_norm": 0.796875,
	"learning_rate": 4.9078609342362666e-05,
	"loss": 0.5017495155334473,
	"num_input_tokens_seen": 76208256,
	"step": 380,
	"train_runtime": 2720.5861,
	"train_tokens_per_second": 28011.705
	},
	{
	"epoch": 0.39936,
	"grad_norm": 0.796875,
	"learning_rate": 4.905498318716775e-05,
	"loss": 0.49234929084777834,
	"num_input_tokens_seen": 78227520,
	"step": 390,
	"train_runtime": 2795.2918,
	"train_tokens_per_second": 27985.458
	},
	{
	"epoch": 0.4096,
	"grad_norm": 0.7578125,
	"learning_rate": 4.9031391119633295e-05,
	"loss": 0.484727144241333,
	"num_input_tokens_seen": 80247232,
	"step": 400,
	"train_runtime": 2869.3909,
	"train_tokens_per_second": 27966.644
	},
	{
	"epoch": 0.41984,
	"grad_norm": 0.828125,
	"learning_rate": 4.9007833057869e-05,
	"loss": 0.4728262901306152,
	"num_input_tokens_seen": 82212032,
	"step": 410,
	"train_runtime": 2936.8857,
	"train_tokens_per_second": 27992.928
	},
	{
	"epoch": 0.43008,
	"grad_norm": 0.8125,
	"learning_rate": 4.898430892025967e-05,
	"loss": 0.46816487312316896,
	"num_input_tokens_seen": 84214528,
	"step": 420,
	"train_runtime": 3010.5759,
	"train_tokens_per_second": 27972.897
	},
	{
	"epoch": 0.44032,
	"grad_norm": 0.796875,
	"learning_rate": 4.896081862546415e-05,
	"loss": 0.4631038665771484,
	"num_input_tokens_seen": 86207424,
	"step": 430,
	"train_runtime": 3081.7088,
	"train_tokens_per_second": 27973.904
	},
	{
	"epoch": 0.45056,
	"grad_norm": 0.828125,
	"learning_rate": 4.8937362092414e-05,
	"loss": 0.461610221862793,
	"num_input_tokens_seen": 88220096,
	"step": 440,
	"train_runtime": 3153.0068,
	"train_tokens_per_second": 27979.672
	},
	{
	"epoch": 0.4608,
	"grad_norm": 0.828125,
	"learning_rate": 4.891393924031244e-05,
	"loss": 0.4538113594055176,
	"num_input_tokens_seen": 90255296,
	"step": 450,
	"train_runtime": 3226.7084,
	"train_tokens_per_second": 27971.321
	},
	{
	"epoch": 0.47104,
	"grad_norm": 0.8671875,
	"learning_rate": 4.8890549988633095e-05,
	"loss": 0.44371371269226073,
	"num_input_tokens_seen": 92284992,
	"step": 460,
	"train_runtime": 3302.3683,
	"train_tokens_per_second": 27945.094
	},
	{
	"epoch": 0.48128,
	"grad_norm": 0.828125,
	"learning_rate": 4.8867194257118907e-05,
	"loss": 0.43736696243286133,
	"num_input_tokens_seen": 94290304,
	"step": 470,
	"train_runtime": 3372.562,
	"train_tokens_per_second": 27958.064
	},
	{
	"epoch": 0.49152,
	"grad_norm": 0.8828125,
	"learning_rate": 4.884387196578093e-05,
	"loss": 0.4319791793823242,
	"num_input_tokens_seen": 96299392,
	"step": 480,
	"train_runtime": 3444.2699,
	"train_tokens_per_second": 27959.305
	},
	{
	"epoch": 0.50176,
	"grad_norm": 0.828125,
	"learning_rate": 4.882058303489718e-05,
	"loss": 0.42551512718200685,
	"num_input_tokens_seen": 98281856,
	"step": 490,
	"train_runtime": 3512.8617,
	"train_tokens_per_second": 27977.719
	},
	{
	"epoch": 0.512,
	"grad_norm": 0.93359375,
	"learning_rate": 4.8797327385011496e-05,
	"loss": 0.4181091785430908,
	"num_input_tokens_seen": 100274880,
	"step": 500,
	"train_runtime": 3583.7895,
	"train_tokens_per_second": 27980.125
	},
	{
	"epoch": 0.52224,
	"grad_norm": 0.93359375,
	"learning_rate": 4.8774104936932425e-05,
	"loss": 0.41173620223999025,
	"num_input_tokens_seen": 102286784,
	"step": 510,
	"train_runtime": 3656.7659,
	"train_tokens_per_second": 27971.926
	},
	{
	"epoch": 0.53248,
	"grad_norm": 0.90234375,
	"learning_rate": 4.8750915611732076e-05,
	"loss": 0.4068614959716797,
	"num_input_tokens_seen": 104322752,
	"step": 520,
	"train_runtime": 3731.2961,
	"train_tokens_per_second": 27958.851
	},
	{
	"epoch": 0.54272,
	"grad_norm": 0.89453125,
	"learning_rate": 4.8727759330744986e-05,
	"loss": 0.39957451820373535,
	"num_input_tokens_seen": 106331264,
	"step": 530,
	"train_runtime": 3804.5982,
	"train_tokens_per_second": 27948.093
	},
	{
	"epoch": 0.55296,
	"grad_norm": 0.90234375,
	"learning_rate": 4.870463601556696e-05,
	"loss": 0.39169912338256835,
	"num_input_tokens_seen": 108363392,
	"step": 540,
	"train_runtime": 3878.5043,
	"train_tokens_per_second": 27939.48
	},
	{
	"epoch": 0.5632,
	"grad_norm": 0.96875,
	"learning_rate": 4.8681545588054075e-05,
	"loss": 0.39029181003570557,
	"num_input_tokens_seen": 110375744,
	"step": 550,
	"train_runtime": 3951.3561,
	"train_tokens_per_second": 27933.636
	},
	{
	"epoch": 0.57344,
	"grad_norm": 0.88671875,
	"learning_rate": 4.8658487970321404e-05,
	"loss": 0.37695889472961425,
	"num_input_tokens_seen": 112386624,
	"step": 560,
	"train_runtime": 4021.0522,
	"train_tokens_per_second": 27949.556
	},
	{
	"epoch": 0.58368,
	"grad_norm": 0.9296875,
	"learning_rate": 4.863546308474209e-05,
	"loss": 0.3717223644256592,
	"num_input_tokens_seen": 114487616,
	"step": 570,
	"train_runtime": 4102.2077,
	"train_tokens_per_second": 27908.781
	},
	{
	"epoch": 0.59392,
	"grad_norm": 1.0078125,
	"learning_rate": 4.86124708539461e-05,
	"loss": 0.36314547061920166,
	"num_input_tokens_seen": 116502464,
	"step": 580,
	"train_runtime": 4172.9104,
	"train_tokens_per_second": 27918.755
	},
	{
	"epoch": 0.60416,
	"grad_norm": 0.96484375,
	"learning_rate": 4.8589511200819216e-05,
	"loss": 0.35808553695678713,
	"num_input_tokens_seen": 118472896,
	"step": 590,
	"train_runtime": 4242.4324,
	"train_tokens_per_second": 27925.7
	},
	{
	"epoch": 0.6144,
	"grad_norm": 1.0234375,
	"learning_rate": 4.8566584048501926e-05,
	"loss": 0.354917311668396,
	"num_input_tokens_seen": 120475328,
	"step": 600,
	"train_runtime": 4312.8827,
	"train_tokens_per_second": 27933.829
	},
	{
	"epoch": 0.62464,
	"grad_norm": 1.140625,
	"learning_rate": 4.854368932038835e-05,
	"loss": 0.3452127456665039,
	"num_input_tokens_seen": 122458496,
	"step": 610,
	"train_runtime": 4383.9692,
	"train_tokens_per_second": 27933.247
	},
	{
	"epoch": 0.63488,
	"grad_norm": 0.9765625,
	"learning_rate": 4.8520826940125144e-05,
	"loss": 0.3381240129470825,
	"num_input_tokens_seen": 124439360,
	"step": 620,
	"train_runtime": 4453.0594,
	"train_tokens_per_second": 27944.689
	},
	{
	"epoch": 0.64512,
	"grad_norm": 1.0234375,
	"learning_rate": 4.849799683161046e-05,
	"loss": 0.3313805818557739,
	"num_input_tokens_seen": 126467840,
	"step": 630,
	"train_runtime": 4528.0794,
	"train_tokens_per_second": 27929.687
	},
	{
	"epoch": 0.65536,
	"grad_norm": 0.9921875,
	"learning_rate": 4.8475198918992835e-05,
	"loss": 0.3252664566040039,
	"num_input_tokens_seen": 128480448,
	"step": 640,
	"train_runtime": 4598.842,
	"train_tokens_per_second": 27937.565
	},
	{
	"epoch": 0.6656,
	"grad_norm": 1.03125,
	"learning_rate": 4.845243312667023e-05,
	"loss": 0.3170381784439087,
	"num_input_tokens_seen": 130479232,
	"step": 650,
	"train_runtime": 4670.9911,
	"train_tokens_per_second": 27933.95
	},
	{
	"epoch": 0.67584,
	"grad_norm": 1.0390625,
	"learning_rate": 4.842969937928884e-05,
	"loss": 0.3079851150512695,
	"num_input_tokens_seen": 132478912,
	"step": 660,
	"train_runtime": 4743.4655,
	"train_tokens_per_second": 27928.719
	},
	{
	"epoch": 0.68608,
	"grad_norm": 0.95703125,
	"learning_rate": 4.840699760174217e-05,
	"loss": 0.3024315357208252,
	"num_input_tokens_seen": 134473408,
	"step": 670,
	"train_runtime": 4814.1152,
	"train_tokens_per_second": 27933.151
	},
	{
	"epoch": 0.69632,
	"grad_norm": 0.9921875,
	"learning_rate": 4.8384327719169906e-05,
	"loss": 0.29927806854248046,
	"num_input_tokens_seen": 136482944,
	"step": 680,
	"train_runtime": 4885.2798,
	"train_tokens_per_second": 27937.59
	},
	{
	"epoch": 0.70656,
	"grad_norm": 1.03125,
	"learning_rate": 4.836168965695694e-05,
	"loss": 0.2894315242767334,
	"num_input_tokens_seen": 138476544,
	"step": 690,
	"train_runtime": 4954.8336,
	"train_tokens_per_second": 27947.769
	},
	{
	"epoch": 0.7168,
	"grad_norm": 1.0546875,
	"learning_rate": 4.8339083340732304e-05,
	"loss": 0.2819934129714966,
	"num_input_tokens_seen": 140478784,
	"step": 700,
	"train_runtime": 5026.8737,
	"train_tokens_per_second": 27945.557
	},
	{
	"epoch": 0.72704,
	"grad_norm": 1.0078125,
	"learning_rate": 4.8316508696368154e-05,
	"loss": 0.2754687309265137,
	"num_input_tokens_seen": 142543936,
	"step": 710,
	"train_runtime": 5103.5664,
	"train_tokens_per_second": 27930.26
	},
	{
	"epoch": 0.73728,
	"grad_norm": 1.03125,
	"learning_rate": 4.8293965649978714e-05,
	"loss": 0.2691352367401123,
	"num_input_tokens_seen": 144565184,
	"step": 720,
	"train_runtime": 5175.4661,
	"train_tokens_per_second": 27932.785
	},
	{
	"epoch": 0.74752,
	"grad_norm": 0.98046875,
	"learning_rate": 4.8271454127919364e-05,
	"loss": 0.2596245765686035,
	"num_input_tokens_seen": 146591872,
	"step": 730,
	"train_runtime": 5247.7226,
	"train_tokens_per_second": 27934.379
	},
	{
	"epoch": 0.75776,
	"grad_norm": 1.0390625,
	"learning_rate": 4.824897405678549e-05,
	"loss": 0.253094482421875,
	"num_input_tokens_seen": 148609728,
	"step": 740,
	"train_runtime": 5319.8801,
	"train_tokens_per_second": 27934.789
	},
	{
	"epoch": 0.768,
	"grad_norm": 1.0546875,
	"learning_rate": 4.8226525363411576e-05,
	"loss": 0.24751272201538085,
	"num_input_tokens_seen": 150619520,
	"step": 750,
	"train_runtime": 5392.3341,
	"train_tokens_per_second": 27932.156
	},
	{
	"epoch": 0.77824,
	"grad_norm": 1.09375,
	"learning_rate": 4.820410797487017e-05,
	"loss": 0.2425351619720459,
	"num_input_tokens_seen": 152618560,
	"step": 760,
	"train_runtime": 5463.9119,
	"train_tokens_per_second": 27932.105
	},
	{
	"epoch": 0.78848,
	"grad_norm": 1.046875,
	"learning_rate": 4.818172181847091e-05,
	"loss": 0.2338550090789795,
	"num_input_tokens_seen": 154616256,
	"step": 770,
	"train_runtime": 5536.4143,
	"train_tokens_per_second": 27927.147
	},
	{
	"epoch": 0.79872,
	"grad_norm": 1.0625,
	"learning_rate": 4.81593668217595e-05,
	"loss": 0.23002958297729492,
	"num_input_tokens_seen": 156632640,
	"step": 780,
	"train_runtime": 5609.7259,
	"train_tokens_per_second": 27921.621
	},
	{
	"epoch": 0.80896,
	"grad_norm": 1.0859375,
	"learning_rate": 4.813704291251675e-05,
	"loss": 0.22488293647766114,
	"num_input_tokens_seen": 158627584,
	"step": 790,
	"train_runtime": 5679.5407,
	"train_tokens_per_second": 27929.65
	},
	{
	"epoch": 0.8192,
	"grad_norm": 1.0859375,
	"learning_rate": 4.811475001875759e-05,
	"loss": 0.21674442291259766,
	"num_input_tokens_seen": 160628608,
	"step": 800,
	"train_runtime": 5751.3102,
	"train_tokens_per_second": 27929.046
	},
	{
	"epoch": 0.82944,
	"grad_norm": 1.046875,
	"learning_rate": 4.8092488068730105e-05,
	"loss": 0.21201133728027344,
	"num_input_tokens_seen": 162631040,
	"step": 810,
	"train_runtime": 5823.6015,
	"train_tokens_per_second": 27926.197
	},
	{
	"epoch": 0.83968,
	"grad_norm": 1.0703125,
	"learning_rate": 4.807025699091452e-05,
	"loss": 0.20578887462615966,
	"num_input_tokens_seen": 164655936,
	"step": 820,
	"train_runtime": 5896.4283,
	"train_tokens_per_second": 27924.691
	},
	{
	"epoch": 0.84992,
	"grad_norm": 1.0546875,
	"learning_rate": 4.8048056714022325e-05,
	"loss": 0.20376951694488527,
	"num_input_tokens_seen": 166635648,
	"step": 830,
	"train_runtime": 5965.7043,
	"train_tokens_per_second": 27932.267
	},
	{
	"epoch": 0.86016,
	"grad_norm": 1.015625,
	"learning_rate": 4.802588716699519e-05,
	"loss": 0.19258487224578857,
	"num_input_tokens_seen": 168677504,
	"step": 840,
	"train_runtime": 6039.5258,
	"train_tokens_per_second": 27928.932
	},
	{
	"epoch": 0.8704,
	"grad_norm": 1.0390625,
	"learning_rate": 4.8003748279004156e-05,
	"loss": 0.18773103952407838,
	"num_input_tokens_seen": 170712320,
	"step": 850,
	"train_runtime": 6113.2217,
	"train_tokens_per_second": 27925.099
	},
	{
	"epoch": 0.88064,
	"grad_norm": 1.0546875,
	"learning_rate": 4.798163997944854e-05,
	"loss": 0.1815708875656128,
	"num_input_tokens_seen": 172726592,
	"step": 860,
	"train_runtime": 6185.6798,
	"train_tokens_per_second": 27923.623
	},
	{
	"epoch": 0.89088,
	"grad_norm": 1.0234375,
	"learning_rate": 4.79595621979551e-05,
	"loss": 0.17781240940093995,
	"num_input_tokens_seen": 174723904,
	"step": 870,
	"train_runtime": 6257.3588,
	"train_tokens_per_second": 27922.948
	},
	{
	"epoch": 0.90112,
	"grad_norm": 1.0546875,
	"learning_rate": 4.793751486437702e-05,
	"loss": 0.1705024003982544,
	"num_input_tokens_seen": 176724608,
	"step": 880,
	"train_runtime": 6327.7475,
	"train_tokens_per_second": 27928.518
	},
	{
	"epoch": 0.91136,
	"grad_norm": 1.0703125,
	"learning_rate": 4.7915497908793064e-05,
	"loss": 0.1674124240875244,
	"num_input_tokens_seen": 178766720,
	"step": 890,
	"train_runtime": 6403.1314,
	"train_tokens_per_second": 27918.64
	},
	{
	"epoch": 0.9216,
	"grad_norm": 1.0234375,
	"learning_rate": 4.7893511261506516e-05,
	"loss": 0.1599474549293518,
	"num_input_tokens_seen": 180780864,
	"step": 900,
	"train_runtime": 6474.1823,
	"train_tokens_per_second": 27923.351
	},
	{
	"epoch": 0.93184,
	"grad_norm": 1.015625,
	"learning_rate": 4.787155485304435e-05,
	"loss": 0.1556488037109375,
	"num_input_tokens_seen": 182756544,
	"step": 910,
	"train_runtime": 6543.9244,
	"train_tokens_per_second": 27927.667
	},
	{
	"epoch": 0.94208,
	"grad_norm": 1.0078125,
	"learning_rate": 4.784962861415629e-05,
	"loss": 0.14749314785003662,
	"num_input_tokens_seen": 184760448,
	"step": 920,
	"train_runtime": 6614.61,
	"train_tokens_per_second": 27932.175
	},
	{
	"epoch": 0.95232,
	"grad_norm": 1.0078125,
	"learning_rate": 4.7827732475813884e-05,
	"loss": 0.14295361042022706,
	"num_input_tokens_seen": 186755072,
	"step": 930,
	"train_runtime": 6683.9789,
	"train_tokens_per_second": 27940.703
	},
	{
	"epoch": 0.96256,
	"grad_norm": 1.0625,
	"learning_rate": 4.7805866369209576e-05,
	"loss": 0.13959715366363526,
	"num_input_tokens_seen": 188760896,
	"step": 940,
	"train_runtime": 6755.7296,
	"train_tokens_per_second": 27940.86
	},
	{
	"epoch": 0.9728,
	"grad_norm": 0.94921875,
	"learning_rate": 4.778403022575583e-05,
	"loss": 0.13509231805801392,
	"num_input_tokens_seen": 190778560,
	"step": 950,
	"train_runtime": 6828.6224,
	"train_tokens_per_second": 27938.074
	},
	{
	"epoch": 0.98304,
	"grad_norm": 1.03125,
	"learning_rate": 4.7762223977084195e-05,
	"loss": 0.12972679138183593,
	"num_input_tokens_seen": 192799232,
	"step": 960,
	"train_runtime": 6901.063,
	"train_tokens_per_second": 27937.614
	},
	{
	"epoch": 0.99328,
	"grad_norm": 1.0234375,
	"learning_rate": 4.774044755504444e-05,
	"loss": 0.12503955364227295,
	"num_input_tokens_seen": 194792768,
	"step": 970,
	"train_runtime": 6970.347,
	"train_tokens_per_second": 27945.921
	},
	{
	"epoch": 1.003072,
	"grad_norm": 0.81640625,
	"learning_rate": 4.7718700891703616e-05,
	"loss": 0.10986135005950928,
	"num_input_tokens_seen": 196668032,
	"step": 980,
	"train_runtime": 7035.0779,
	"train_tokens_per_second": 27955.345
	},
	{
	"epoch": 1.013312,
	"grad_norm": 0.8671875,
	"learning_rate": 4.7696983919345215e-05,
	"loss": 0.08115079402923583,
	"num_input_tokens_seen": 198703552,
	"step": 990,
	"train_runtime": 7109.6942,
	"train_tokens_per_second": 27948.256
	},
	{
	"epoch": 1.023552,
	"grad_norm": 0.7890625,
	"learning_rate": 4.7675296570468216e-05,
	"loss": 0.07811311483383179,
	"num_input_tokens_seen": 200721088,
	"step": 1000,
	"train_runtime": 7182.0942,
	"train_tokens_per_second": 27947.432
	},
	{
	"epoch": 1.033792,
	"grad_norm": 0.83203125,
	"learning_rate": 4.76536387777863e-05,
	"loss": 0.07646113634109497,
	"num_input_tokens_seen": 202724032,
	"step": 1010,
	"train_runtime": 7255.6602,
	"train_tokens_per_second": 27940.122
	},
	{
	"epoch": 1.044032,
	"grad_norm": 0.8203125,
	"learning_rate": 4.7632010474226915e-05,
	"loss": 0.07352162599563598,
	"num_input_tokens_seen": 204720448,
	"step": 1020,
	"train_runtime": 7325.7825,
	"train_tokens_per_second": 27945.199
	},
	{
	"epoch": 1.054272,
	"grad_norm": 0.80078125,
	"learning_rate": 4.761041159293035e-05,
	"loss": 0.07193953990936279,
	"num_input_tokens_seen": 206747072,
	"step": 1030,
	"train_runtime": 7398.5275,
	"train_tokens_per_second": 27944.354
	},
	{
	"epoch": 1.064512,
	"grad_norm": 0.78515625,
	"learning_rate": 4.7588842067249e-05,
	"loss": 0.07109384536743164,
	"num_input_tokens_seen": 208767168,
	"step": 1040,
	"train_runtime": 7471.9759,
	"train_tokens_per_second": 27940.022
	},
	{
	"epoch": 1.074752,
	"grad_norm": 0.8125,
	"learning_rate": 4.756730183074637e-05,
	"loss": 0.06862571239471435,
	"num_input_tokens_seen": 210755648,
	"step": 1050,
	"train_runtime": 7542.5772,
	"train_tokens_per_second": 27942.127
	},
	{
	"epoch": 1.084992,
	"grad_norm": 0.75390625,
	"learning_rate": 4.7545790817196314e-05,
	"loss": 0.06509301662445069,
	"num_input_tokens_seen": 212746688,
	"step": 1060,
	"train_runtime": 7611.2774,
	"train_tokens_per_second": 27951.509
	},
	{
	"epoch": 1.095232,
	"grad_norm": 0.7421875,
	"learning_rate": 4.752430896058212e-05,
	"loss": 0.06395751237869263,
	"num_input_tokens_seen": 214772352,
	"step": 1070,
	"train_runtime": 7684.2744,
	"train_tokens_per_second": 27949.594
	},
	{
	"epoch": 1.105472,
	"grad_norm": 0.73046875,
	"learning_rate": 4.750285619509567e-05,
	"loss": 0.06260917186737061,
	"num_input_tokens_seen": 216773568,
	"step": 1080,
	"train_runtime": 7754.914,
	"train_tokens_per_second": 27953.059
	},
	{
	"epoch": 1.115712,
	"grad_norm": 0.73828125,
	"learning_rate": 4.7481432455136644e-05,
	"loss": 0.062073934078216556,
	"num_input_tokens_seen": 218764928,
	"step": 1090,
	"train_runtime": 7826.8462,
	"train_tokens_per_second": 27950.585
	},
	{
	"epoch": 1.125952,
	"grad_norm": 0.83984375,
	"learning_rate": 4.7460037675311584e-05,
	"loss": 0.057993775606155394,
	"num_input_tokens_seen": 220755776,
	"step": 1100,
	"train_runtime": 7897.56,
	"train_tokens_per_second": 27952.403
	},
	{
	"epoch": 1.136192,
	"grad_norm": 0.69921875,
	"learning_rate": 4.7438671790433126e-05,
	"loss": 0.05776026248931885,
	"num_input_tokens_seen": 222748608,
	"step": 1110,
	"train_runtime": 7967.3823,
	"train_tokens_per_second": 27957.565
	},
	{
	"epoch": 1.146432,
	"grad_norm": 0.703125,
	"learning_rate": 4.741733473551915e-05,
	"loss": 0.05773916840553284,
	"num_input_tokens_seen": 224784512,
	"step": 1120,
	"train_runtime": 8041.8087,
	"train_tokens_per_second": 27951.984
	},
	{
	"epoch": 1.156672,
	"grad_norm": 0.734375,
	"learning_rate": 4.7396026445791966e-05,
	"loss": 0.05507153272628784,
	"num_input_tokens_seen": 226813120,
	"step": 1130,
	"train_runtime": 8115.4978,
	"train_tokens_per_second": 27948.146
	},
	{
	"epoch": 1.166912,
	"grad_norm": 0.75390625,
	"learning_rate": 4.737474685667742e-05,
	"loss": 0.05341644287109375,
	"num_input_tokens_seen": 228815232,
	"step": 1140,
	"train_runtime": 8187.3704,
	"train_tokens_per_second": 27947.341
	},
	{
	"epoch": 1.177152,
	"grad_norm": 0.73828125,
	"learning_rate": 4.7353495903804165e-05,
	"loss": 0.05063482522964478,
	"num_input_tokens_seen": 230880320,
	"step": 1150,
	"train_runtime": 8264.2254,
	"train_tokens_per_second": 27937.321
	},
	{
	"epoch": 1.187392,
	"grad_norm": 0.69140625,
	"learning_rate": 4.733227352300277e-05,
	"loss": 0.050589507818222045,
	"num_input_tokens_seen": 232907904,
	"step": 1160,
	"train_runtime": 8338.7653,
	"train_tokens_per_second": 27930.742
	},
	{
	"epoch": 1.197632,
	"grad_norm": 0.671875,
	"learning_rate": 4.731107965030496e-05,
	"loss": 0.04946887493133545,
	"num_input_tokens_seen": 234922176,
	"step": 1170,
	"train_runtime": 8410.7105,
	"train_tokens_per_second": 27931.312
	},
	{
	"epoch": 1.207872,
	"grad_norm": 0.70703125,
	"learning_rate": 4.728991422194278e-05,
	"loss": 0.04885604977607727,
	"num_input_tokens_seen": 236912128,
	"step": 1180,
	"train_runtime": 8481.0624,
	"train_tokens_per_second": 27934.251
	},
	{
	"epoch": 1.218112,
	"grad_norm": 0.671875,
	"learning_rate": 4.726877717434773e-05,
	"loss": 0.048174849152565,
	"num_input_tokens_seen": 238900864,
	"step": 1190,
	"train_runtime": 8551.1679,
	"train_tokens_per_second": 27937.805
	},
	{
	"epoch": 1.228352,
	"grad_norm": 0.6328125,
	"learning_rate": 4.724766844415013e-05,
	"loss": 0.04597228169441223,
	"num_input_tokens_seen": 240960448,
	"step": 1200,
	"train_runtime": 8627.7254,
	"train_tokens_per_second": 27928.618
	},
	{
	"epoch": 1.238592,
	"grad_norm": 0.67578125,
	"learning_rate": 4.722658796817813e-05,
	"loss": 0.04480882287025452,
	"num_input_tokens_seen": 242998848,
	"step": 1210,
	"train_runtime": 8702.6979,
	"train_tokens_per_second": 27922.243
	},
	{
	"epoch": 1.248832,
	"grad_norm": 0.6328125,
	"learning_rate": 4.7205535683457044e-05,
	"loss": 0.04354588389396667,
	"num_input_tokens_seen": 244947776,
	"step": 1220,
	"train_runtime": 8767.6922,
	"train_tokens_per_second": 27937.543
	},
	{
	"epoch": 1.259072,
	"grad_norm": 0.68359375,
	"learning_rate": 4.7184511527208484e-05,
	"loss": 0.041252422332763675,
	"num_input_tokens_seen": 246951744,
	"step": 1230,
	"train_runtime": 8838.7543,
	"train_tokens_per_second": 27939.655
	},
	{
	"epoch": 1.269312,
	"grad_norm": 0.58984375,
	"learning_rate": 4.7163515436849644e-05,
	"loss": 0.03930726945400238,
	"num_input_tokens_seen": 248987840,
	"step": 1240,
	"train_runtime": 8912.5272,
	"train_tokens_per_second": 27936.839
	},
	{
	"epoch": 1.279552,
	"grad_norm": 0.6171875,
	"learning_rate": 4.714254734999245e-05,
	"loss": 0.03749307096004486,
	"num_input_tokens_seen": 250972928,
	"step": 1250,
	"train_runtime": 8981.968,
	"train_tokens_per_second": 27941.864
	},
	{
	"epoch": 1.289792,
	"grad_norm": 0.63671875,
	"learning_rate": 4.712160720444284e-05,
	"loss": 0.03726911842823029,
	"num_input_tokens_seen": 252991744,
	"step": 1260,
	"train_runtime": 9053.9267,
	"train_tokens_per_second": 27942.765
	},
	{
	"epoch": 1.300032,
	"grad_norm": 0.59765625,
	"learning_rate": 4.710069493819992e-05,
	"loss": 0.03688657283782959,
	"num_input_tokens_seen": 254978432,
	"step": 1270,
	"train_runtime": 9124.0045,
	"train_tokens_per_second": 27945.891
	},
	{
	"epoch": 1.3102719999999999,
	"grad_norm": 0.6484375,
	"learning_rate": 4.70798104894553e-05,
	"loss": 0.03622893989086151,
	"num_input_tokens_seen": 256958912,
	"step": 1280,
	"train_runtime": 9192.9137,
	"train_tokens_per_second": 27951.847
	},
	{
	"epoch": 1.320512,
	"grad_norm": 0.640625,
	"learning_rate": 4.705895379659219e-05,
	"loss": 0.03448793888092041,
	"num_input_tokens_seen": 258927104,
	"step": 1290,
	"train_runtime": 9261.8009,
	"train_tokens_per_second": 27956.453
	},
	{
	"epoch": 1.330752,
	"grad_norm": 0.62890625,
	"learning_rate": 4.7038124798184766e-05,
	"loss": 0.03333508670330047,
	"num_input_tokens_seen": 260902016,
	"step": 1300,
	"train_runtime": 9331.4024,
	"train_tokens_per_second": 27959.572
	},
	{
	"epoch": 1.340992,
	"grad_norm": 0.5625,
	"learning_rate": 4.7017323432997304e-05,
	"loss": 0.032725405693054196,
	"num_input_tokens_seen": 262909696,
	"step": 1310,
	"train_runtime": 9403.9992,
	"train_tokens_per_second": 27957.222
	},
	{
	"epoch": 1.351232,
	"grad_norm": 0.56640625,
	"learning_rate": 4.6996549639983506e-05,
	"loss": 0.03168002963066101,
	"num_input_tokens_seen": 264930176,
	"step": 1320,
	"train_runtime": 9476.4205,
	"train_tokens_per_second": 27956.777
	},
	{
	"epoch": 1.361472,
	"grad_norm": 0.55078125,
	"learning_rate": 4.697580335828569e-05,
	"loss": 0.02995384335517883,
	"num_input_tokens_seen": 266964480,
	"step": 1330,
	"train_runtime": 9550.7511,
	"train_tokens_per_second": 27952.197
	},
	{
	"epoch": 1.371712,
	"grad_norm": 0.6015625,
	"learning_rate": 4.6955084527234076e-05,
	"loss": 0.030216827988624573,
	"num_input_tokens_seen": 268993664,
	"step": 1340,
	"train_runtime": 9623.3266,
	"train_tokens_per_second": 27952.254
	},
	{
	"epoch": 1.381952,
	"grad_norm": 0.56640625,
	"learning_rate": 4.6934393086346034e-05,
	"loss": 0.028735750913619997,
	"num_input_tokens_seen": 270981248,
	"step": 1350,
	"train_runtime": 9691.2828,
	"train_tokens_per_second": 27961.339
	},
	{
	"epoch": 1.392192,
	"grad_norm": 0.55859375,
	"learning_rate": 4.6913728975325324e-05,
	"loss": 0.026437461376190186,
	"num_input_tokens_seen": 272987712,
	"step": 1360,
	"train_runtime": 9762.0951,
	"train_tokens_per_second": 27964.05
	},
	{
	"epoch": 1.4024320000000001,
	"grad_norm": 0.58984375,
	"learning_rate": 4.6893092134061393e-05,
	"loss": 0.02681639790534973,
	"num_input_tokens_seen": 275016192,
	"step": 1370,
	"train_runtime": 9836.7175,
	"train_tokens_per_second": 27958.127
	},
	{
	"epoch": 1.412672,
	"grad_norm": 0.46875,
	"learning_rate": 4.687248250262859e-05,
	"loss": 0.02621593475341797,
	"num_input_tokens_seen": 277001984,
	"step": 1380,
	"train_runtime": 9907.0035,
	"train_tokens_per_second": 27960.219
	},
	{
	"epoch": 1.422912,
	"grad_norm": 0.59375,
	"learning_rate": 4.685190002128548e-05,
	"loss": 0.025581035017967223,
	"num_input_tokens_seen": 279033856,
	"step": 1390,
	"train_runtime": 9982.0945,
	"train_tokens_per_second": 27953.438
	},
	{
	"epoch": 1.433152,
	"grad_norm": 0.55078125,
	"learning_rate": 4.6831344630474114e-05,
	"loss": 0.02472420036792755,
	"num_input_tokens_seen": 281106624,
	"step": 1400,
	"train_runtime": 10061.1573,
	"train_tokens_per_second": 27939.79
	},
	{
	"epoch": 1.443392,
	"grad_norm": 0.51953125,
	"learning_rate": 4.6810816270819276e-05,
	"loss": 0.023309352993965148,
	"num_input_tokens_seen": 283099072,
	"step": 1410,
	"train_runtime": 10129.2372,
	"train_tokens_per_second": 27948.706
	},
	{
	"epoch": 1.453632,
	"grad_norm": 0.494140625,
	"learning_rate": 4.679031488312777e-05,
	"loss": 0.0234044149518013,
	"num_input_tokens_seen": 285122560,
	"step": 1420,
	"train_runtime": 10204.0696,
	"train_tokens_per_second": 27942.044
	},
	{
	"epoch": 1.463872,
	"grad_norm": 0.64453125,
	"learning_rate": 4.6769840408387717e-05,
	"loss": 0.021864794194698334,
	"num_input_tokens_seen": 287115584,
	"step": 1430,
	"train_runtime": 10275.7948,
	"train_tokens_per_second": 27940.961
	},
	{
	"epoch": 1.4741119999999999,
	"grad_norm": 0.40234375,
	"learning_rate": 4.674939278776787e-05,
	"loss": 0.022062216699123383,
	"num_input_tokens_seen": 289142976,
	"step": 1440,
	"train_runtime": 10348.4017,
	"train_tokens_per_second": 27940.834
	},
	{
	"epoch": 1.484352,
	"grad_norm": 0.421875,
	"learning_rate": 4.672897196261683e-05,
	"loss": 0.020946532487869263,
	"num_input_tokens_seen": 291151552,
	"step": 1450,
	"train_runtime": 10420.2833,
	"train_tokens_per_second": 27940.848
	},
	{
	"epoch": 1.494592,
	"grad_norm": 0.4921875,
	"learning_rate": 4.670857787446238e-05,
	"loss": 0.021855458617210388,
	"num_input_tokens_seen": 293175936,
	"step": 1460,
	"train_runtime": 10494.1522,
	"train_tokens_per_second": 27937.077
	},
	{
	"epoch": 1.504832,
	"grad_norm": 0.427734375,
	"learning_rate": 4.668821046501082e-05,
	"loss": 0.019446633756160736,
	"num_input_tokens_seen": 295201984,
	"step": 1470,
	"train_runtime": 10566.8511,
	"train_tokens_per_second": 27936.609
	},
	{
	"epoch": 1.515072,
	"grad_norm": 0.447265625,
	"learning_rate": 4.6667869676146194e-05,
	"loss": 0.018910986185073853,
	"num_input_tokens_seen": 297239808,
	"step": 1480,
	"train_runtime": 10643.302,
	"train_tokens_per_second": 27927.405
	},
	{
	"epoch": 1.525312,
	"grad_norm": 0.416015625,
	"learning_rate": 4.6647555449929645e-05,
	"loss": 0.0188526451587677,
	"num_input_tokens_seen": 299265024,
	"step": 1490,
	"train_runtime": 10718.5795,
	"train_tokens_per_second": 27920.213
	},
	{
	"epoch": 1.535552,
	"grad_norm": 0.55859375,
	"learning_rate": 4.662726772859869e-05,
	"loss": 0.0179020956158638,
	"num_input_tokens_seen": 301289600,
	"step": 1500,
	"train_runtime": 10791.2591,
	"train_tokens_per_second": 27919.782
	},
	{
	"epoch": 1.545792,
	"grad_norm": 0.455078125,
	"learning_rate": 4.660700645456655e-05,
	"loss": 0.017698875069618224,
	"num_input_tokens_seen": 303298944,
	"step": 1510,
	"train_runtime": 10861.9474,
	"train_tokens_per_second": 27923.072
	},
	{
	"epoch": 1.556032,
	"grad_norm": 0.423828125,
	"learning_rate": 4.658677157042149e-05,
	"loss": 0.016229704022407532,
	"num_input_tokens_seen": 305301120,
	"step": 1520,
	"train_runtime": 10932.2024,
	"train_tokens_per_second": 27926.772
	},
	{
	"epoch": 1.566272,
	"grad_norm": 0.490234375,
	"learning_rate": 4.656656301892605e-05,
	"loss": 0.015268620848655701,
	"num_input_tokens_seen": 307290560,
	"step": 1530,
	"train_runtime": 11005.3403,
	"train_tokens_per_second": 27921.95
	},
	{
	"epoch": 1.5765120000000001,
	"grad_norm": 0.4453125,
	"learning_rate": 4.6546380743016465e-05,
	"loss": 0.016469526290893554,
	"num_input_tokens_seen": 309270848,
	"step": 1540,
	"train_runtime": 11074.3719,
	"train_tokens_per_second": 27926.717
	},
	{
	"epoch": 1.5867520000000002,
	"grad_norm": 0.4296875,
	"learning_rate": 4.652622468580193e-05,
	"loss": 0.015096321702003479,
	"num_input_tokens_seen": 311299328,
	"step": 1550,
	"train_runtime": 11146.7209,
	"train_tokens_per_second": 27927.435
	},
	{
	"epoch": 1.596992,
	"grad_norm": 0.349609375,
	"learning_rate": 4.650609479056392e-05,
	"loss": 0.015387380123138427,
	"num_input_tokens_seen": 313267840,
	"step": 1560,
	"train_runtime": 11214.5505,
	"train_tokens_per_second": 27934.052
	},
	{
	"epoch": 1.607232,
	"grad_norm": 0.365234375,
	"learning_rate": 4.648599100075556e-05,
	"loss": 0.014029040932655334,
	"num_input_tokens_seen": 315284992,
	"step": 1570,
	"train_runtime": 11285.1706,
	"train_tokens_per_second": 27937.991
	},
	{
	"epoch": 1.617472,
	"grad_norm": 0.35546875,
	"learning_rate": 4.6465913260000945e-05,
	"loss": 0.014028981328010559,
	"num_input_tokens_seen": 317306816,
	"step": 1580,
	"train_runtime": 11358.4592,
	"train_tokens_per_second": 27935.727
	},
	{
	"epoch": 1.627712,
	"grad_norm": 0.458984375,
	"learning_rate": 4.644586151209444e-05,
	"loss": 0.013350155949592591,
	"num_input_tokens_seen": 319310464,
	"step": 1590,
	"train_runtime": 11429.0808,
	"train_tokens_per_second": 27938.42
	},
	{
	"epoch": 1.6379519999999999,
	"grad_norm": 0.3515625,
	"learning_rate": 4.6425835701000084e-05,
	"loss": 0.013065680861473083,
	"num_input_tokens_seen": 321328896,
	"step": 1600,
	"train_runtime": 11500.7908,
	"train_tokens_per_second": 27939.722
	},
	{
	"epoch": 1.6481919999999999,
	"grad_norm": 0.31640625,
	"learning_rate": 4.640583577085084e-05,
	"loss": 0.012181369960308075,
	"num_input_tokens_seen": 323339008,
	"step": 1610,
	"train_runtime": 11573.5451,
	"train_tokens_per_second": 27937.767
	},
	{
	"epoch": 1.658432,
	"grad_norm": 0.33203125,
	"learning_rate": 4.638586166594806e-05,
	"loss": 0.012439670413732529,
	"num_input_tokens_seen": 325311936,
	"step": 1620,
	"train_runtime": 11642.5518,
	"train_tokens_per_second": 27941.635
	},
	{
	"epoch": 1.668672,
	"grad_norm": 0.322265625,
	"learning_rate": 4.6365913330760726e-05,
	"loss": 0.01156621277332306,
	"num_input_tokens_seen": 327330944,
	"step": 1630,
	"train_runtime": 11714.6579,
	"train_tokens_per_second": 27941.998
	},
	{
	"epoch": 1.678912,
	"grad_norm": 0.28515625,
	"learning_rate": 4.6345990709924855e-05,
	"loss": 0.011320900171995163,
	"num_input_tokens_seen": 329349504,
	"step": 1640,
	"train_runtime": 11785.6649,
	"train_tokens_per_second": 27944.924
	},
	{
	"epoch": 1.689152,
	"grad_norm": 0.298828125,
	"learning_rate": 4.632609374824284e-05,
	"loss": 0.011535357683897018,
	"num_input_tokens_seen": 331350144,
	"step": 1650,
	"train_runtime": 11854.516,
	"train_tokens_per_second": 27951.385
	},
	{
	"epoch": 1.699392,
	"grad_norm": 0.30859375,
	"learning_rate": 4.630622239068285e-05,
	"loss": 0.010813712328672408,
	"num_input_tokens_seen": 333357824,
	"step": 1660,
	"train_runtime": 11927.1145,
	"train_tokens_per_second": 27949.579
	},
	{
	"epoch": 1.709632,
	"grad_norm": 0.28515625,
	"learning_rate": 4.628637658237808e-05,
	"loss": 0.010460171103477477,
	"num_input_tokens_seen": 335369856,
	"step": 1670,
	"train_runtime": 11998.7216,
	"train_tokens_per_second": 27950.466
	},
	{
	"epoch": 1.719872,
	"grad_norm": 0.298828125,
	"learning_rate": 4.626655626862625e-05,
	"loss": 0.0098984993994236,
	"num_input_tokens_seen": 337365952,
	"step": 1680,
	"train_runtime": 12069.3316,
	"train_tokens_per_second": 27952.331
	},
	{
	"epoch": 1.730112,
	"grad_norm": 0.302734375,
	"learning_rate": 4.624676139488888e-05,
	"loss": 0.009945446252822876,
	"num_input_tokens_seen": 339410240,
	"step": 1690,
	"train_runtime": 12145.3379,
	"train_tokens_per_second": 27945.722
	},
	{
	"epoch": 1.7403520000000001,
	"grad_norm": 0.267578125,
	"learning_rate": 4.6226991906790686e-05,
	"loss": 0.009306684136390686,
	"num_input_tokens_seen": 341428992,
	"step": 1700,
	"train_runtime": 12215.6261,
	"train_tokens_per_second": 27950.184
	},
	{
	"epoch": 1.7505920000000001,
	"grad_norm": 0.2734375,
	"learning_rate": 4.620724775011897e-05,
	"loss": 0.009691517055034637,
	"num_input_tokens_seen": 343457216,
	"step": 1710,
	"train_runtime": 12288.8723,
	"train_tokens_per_second": 27948.636
	},
	{
	"epoch": 1.760832,
	"grad_norm": 0.255859375,
	"learning_rate": 4.618752887082297e-05,
	"loss": 0.008967689424753188,
	"num_input_tokens_seen": 345431232,
	"step": 1720,
	"train_runtime": 12357.6624,
	"train_tokens_per_second": 27952.797
	},
	{
	"epoch": 1.771072,
	"grad_norm": 0.2119140625,
	"learning_rate": 4.616783521501325e-05,
	"loss": 0.008772896230220794,
	"num_input_tokens_seen": 347450176,
	"step": 1730,
	"train_runtime": 12432.9551,
	"train_tokens_per_second": 27945.904
	},
	{
	"epoch": 1.781312,
	"grad_norm": 0.21484375,
	"learning_rate": 4.614816672896108e-05,
	"loss": 0.008689258992671967,
	"num_input_tokens_seen": 349421504,
	"step": 1740,
	"train_runtime": 12502.1965,
	"train_tokens_per_second": 27948.809
	},
	{
	"epoch": 1.791552,
	"grad_norm": 0.31640625,
	"learning_rate": 4.612852335909782e-05,
	"loss": 0.008518567681312561,
	"num_input_tokens_seen": 351366656,
	"step": 1750,
	"train_runtime": 12567.8285,
	"train_tokens_per_second": 27957.626
	},
	{
	"epoch": 1.8017919999999998,
	"grad_norm": 0.201171875,
	"learning_rate": 4.6108905052014323e-05,
	"loss": 0.008236590027809142,
	"num_input_tokens_seen": 353376960,
	"step": 1760,
	"train_runtime": 12639.0225,
	"train_tokens_per_second": 27959.2
	},
	{
	"epoch": 1.8120319999999999,
	"grad_norm": 0.205078125,
	"learning_rate": 4.608931175446027e-05,
	"loss": 0.007975803315639496,
	"num_input_tokens_seen": 355372096,
	"step": 1770,
	"train_runtime": 12707.5835,
	"train_tokens_per_second": 27965.356
	},
	{
	"epoch": 1.822272,
	"grad_norm": 0.2392578125,
	"learning_rate": 4.606974341334367e-05,
	"loss": 0.008116719126701356,
	"num_input_tokens_seen": 357352000,
	"step": 1780,
	"train_runtime": 12776.464,
	"train_tokens_per_second": 27969.554
	},
	{
	"epoch": 1.832512,
	"grad_norm": 0.2080078125,
	"learning_rate": 4.605019997573011e-05,
	"loss": 0.007819350808858871,
	"num_input_tokens_seen": 359375232,
	"step": 1790,
	"train_runtime": 12849.3423,
	"train_tokens_per_second": 27968.376
	},
	{
	"epoch": 1.842752,
	"grad_norm": 0.19140625,
	"learning_rate": 4.603068138884229e-05,
	"loss": 0.008013889193534851,
	"num_input_tokens_seen": 361425216,
	"step": 1800,
	"train_runtime": 12924.9298,
	"train_tokens_per_second": 27963.418
	},
	{
	"epoch": 1.852992,
	"grad_norm": 0.1669921875,
	"learning_rate": 4.6011187600059345e-05,
	"loss": 0.007500405609607697,
	"num_input_tokens_seen": 363422336,
	"step": 1810,
	"train_runtime": 12995.3355,
	"train_tokens_per_second": 27965.598
	},
	{
	"epoch": 1.863232,
	"grad_norm": 0.1669921875,
	"learning_rate": 4.599171855691629e-05,
	"loss": 0.007371760904788971,
	"num_input_tokens_seen": 365459840,
	"step": 1820,
	"train_runtime": 13073.2818,
	"train_tokens_per_second": 27954.713
	},
	{
	"epoch": 1.873472,
	"grad_norm": 0.14453125,
	"learning_rate": 4.597227420710335e-05,
	"loss": 0.007434654235839844,
	"num_input_tokens_seen": 367456320,
	"step": 1830,
	"train_runtime": 13143.2432,
	"train_tokens_per_second": 27957.812
	},
	{
	"epoch": 1.883712,
	"grad_norm": 0.154296875,
	"learning_rate": 4.595285449846551e-05,
	"loss": 0.007234874367713928,
	"num_input_tokens_seen": 369417920,
	"step": 1840,
	"train_runtime": 13211.8674,
	"train_tokens_per_second": 27961.068
	},
	{
	"epoch": 1.893952,
	"grad_norm": 0.1298828125,
	"learning_rate": 4.593345937900178e-05,
	"loss": 0.007048602402210236,
	"num_input_tokens_seen": 371435072,
	"step": 1850,
	"train_runtime": 13282.1544,
	"train_tokens_per_second": 27964.972
	},
	{
	"epoch": 1.904192,
	"grad_norm": 0.1708984375,
	"learning_rate": 4.591408879686472e-05,
	"loss": 0.007115562260150909,
	"num_input_tokens_seen": 373413504,
	"step": 1860,
	"train_runtime": 13349.1486,
	"train_tokens_per_second": 27972.833
	},
	{
	"epoch": 1.9144320000000001,
	"grad_norm": 0.12158203125,
	"learning_rate": 4.5894742700359775e-05,
	"loss": 0.0069166868925094604,
	"num_input_tokens_seen": 375452096,
	"step": 1870,
	"train_runtime": 13423.4693,
	"train_tokens_per_second": 27969.826
	},
	{
	"epoch": 1.9246720000000002,
	"grad_norm": 0.1474609375,
	"learning_rate": 4.587542103794477e-05,
	"loss": 0.006946581602096558,
	"num_input_tokens_seen": 377457408,
	"step": 1880,
	"train_runtime": 13494.4,
	"train_tokens_per_second": 27971.411
	},
	{
	"epoch": 1.934912,
	"grad_norm": 0.17578125,
	"learning_rate": 4.5856123758229247e-05,
	"loss": 0.006722895056009292,
	"num_input_tokens_seen": 379465216,
	"step": 1890,
	"train_runtime": 13564.8206,
	"train_tokens_per_second": 27974.216
	},
	{
	"epoch": 1.945152,
	"grad_norm": 0.134765625,
	"learning_rate": 4.5836850809973993e-05,
	"loss": 0.006712291389703751,
	"num_input_tokens_seen": 381443840,
	"step": 1900,
	"train_runtime": 13633.4473,
	"train_tokens_per_second": 27978.532
	},
	{
	"epoch": 1.955392,
	"grad_norm": 0.1435546875,
	"learning_rate": 4.5817602142090385e-05,
	"loss": 0.006593970954418183,
	"num_input_tokens_seen": 383492032,
	"step": 1910,
	"train_runtime": 13708.6021,
	"train_tokens_per_second": 27974.554
	},
	{
	"epoch": 1.965632,
	"grad_norm": 0.11181640625,
	"learning_rate": 4.579837770363989e-05,
	"loss": 0.006644654273986817,
	"num_input_tokens_seen": 385522624,
	"step": 1920,
	"train_runtime": 13781.8681,
	"train_tokens_per_second": 27973.176
	},
	{
	"epoch": 1.9758719999999999,
	"grad_norm": 0.146484375,
	"learning_rate": 4.57791774438334e-05,
	"loss": 0.006544043123722076,
	"num_input_tokens_seen": 387554240,
	"step": 1930,
	"train_runtime": 13856.7855,
	"train_tokens_per_second": 27968.553
	},
	{
	"epoch": 1.9861119999999999,
	"grad_norm": 0.12255859375,
	"learning_rate": 4.576000131203078e-05,
	"loss": 0.006355230510234833,
	"num_input_tokens_seen": 389552960,
	"step": 1940,
	"train_runtime": 13928.6465,
	"train_tokens_per_second": 27967.754
	},
	{
	"epoch": 1.996352,
	"grad_norm": 0.2099609375,
	"learning_rate": 4.574084925774023e-05,
	"loss": 0.006281337141990662,
	"num_input_tokens_seen": 391574656,
	"step": 1950,
	"train_runtime": 14002.5979,
	"train_tokens_per_second": 27964.429
	},
	{
	"epoch": 2.006144,
	"grad_norm": 0.10498046875,
	"learning_rate": 4.5721721230617795e-05,
	"loss": 0.005665350705385208,
	"num_input_tokens_seen": 393515200,
	"step": 1960,
	"train_runtime": 14072.0648,
	"train_tokens_per_second": 27964.283
	},
	{
	"epoch": 2.016384,
	"grad_norm": 0.1064453125,
	"learning_rate": 4.57026171804667e-05,
	"loss": 0.00525745153427124,
	"num_input_tokens_seen": 395529664,
	"step": 1970,
	"train_runtime": 14144.06,
	"train_tokens_per_second": 27964.366
	},
	{
	"epoch": 2.026624,
	"grad_norm": 0.07958984375,
	"learning_rate": 4.568353705723692e-05,
	"loss": 0.005201469361782074,
	"num_input_tokens_seen": 397487424,
	"step": 1980,
	"train_runtime": 14210.1189,
	"train_tokens_per_second": 27972.139
	},
	{
	"epoch": 2.036864,
	"grad_norm": 0.07373046875,
	"learning_rate": 4.566448081102455e-05,
	"loss": 0.005276227742433548,
	"num_input_tokens_seen": 399499776,
	"step": 1990,
	"train_runtime": 14282.3392,
	"train_tokens_per_second": 27971.593
	},
	{
	"epoch": 2.047104,
	"grad_norm": 0.1162109375,
	"learning_rate": 4.564544839207128e-05,
	"loss": 0.005213438719511032,
	"num_input_tokens_seen": 401505216,
	"step": 2000,
	"train_runtime": 14353.452,
	"train_tokens_per_second": 27972.729
	},
	{
	"epoch": 2.057344,
	"grad_norm": 0.1259765625,
	"learning_rate": 4.562643975076387e-05,
	"loss": 0.005236967653036118,
	"num_input_tokens_seen": 403493888,
	"step": 2010,
	"train_runtime": 14424.0273,
	"train_tokens_per_second": 27973.733
	},
	{
	"epoch": 2.067584,
	"grad_norm": 0.080078125,
	"learning_rate": 4.560745483763357e-05,
	"loss": 0.0052146721631288525,
	"num_input_tokens_seen": 405488704,
	"step": 2020,
	"train_runtime": 14494.4078,
	"train_tokens_per_second": 27975.527
	},
	{
	"epoch": 2.077824,
	"grad_norm": 0.0859375,
	"learning_rate": 4.5588493603355595e-05,
	"loss": 0.005139049887657165,
	"num_input_tokens_seen": 407464640,
	"step": 2030,
	"train_runtime": 14564.8182,
	"train_tokens_per_second": 27975.951
	},
	{
	"epoch": 2.088064,
	"grad_norm": 0.123046875,
	"learning_rate": 4.556955599874859e-05,
	"loss": 0.005121592432260513,
	"num_input_tokens_seen": 409450432,
	"step": 2040,
	"train_runtime": 14633.3791,
	"train_tokens_per_second": 27980.58
	},
	{
	"epoch": 2.098304,
	"grad_norm": 0.07861328125,
	"learning_rate": 4.555064197477409e-05,
	"loss": 0.0051218770444393154,
	"num_input_tokens_seen": 411460480,
	"step": 2050,
	"train_runtime": 14706.8322,
	"train_tokens_per_second": 27977.506
	},
	{
	"epoch": 2.108544,
	"grad_norm": 0.08642578125,
	"learning_rate": 4.5531751482536e-05,
	"loss": 0.005105789378285408,
	"num_input_tokens_seen": 413451776,
	"step": 2060,
	"train_runtime": 14777.0731,
	"train_tokens_per_second": 27979.274
	},
	{
	"epoch": 2.118784,
	"grad_norm": 0.07568359375,
	"learning_rate": 4.5512884473280024e-05,
	"loss": 0.005103696137666702,
	"num_input_tokens_seen": 415470592,
	"step": 2070,
	"train_runtime": 14849.8325,
	"train_tokens_per_second": 27978.133
	},
	{
	"epoch": 2.129024,
	"grad_norm": 0.0771484375,
	"learning_rate": 4.549404089839322e-05,
	"loss": 0.005021055787801742,
	"num_input_tokens_seen": 417460160,
	"step": 2080,
	"train_runtime": 14921.786,
	"train_tokens_per_second": 27976.555
	},
	{
	"epoch": 2.139264,
	"grad_norm": 0.076171875,
	"learning_rate": 4.547522070940335e-05,
	"loss": 0.005071662366390228,
	"num_input_tokens_seen": 419456640,
	"step": 2090,
	"train_runtime": 14992.2778,
	"train_tokens_per_second": 27978.18
	},
	{
	"epoch": 2.149504,
	"grad_norm": 0.08056640625,
	"learning_rate": 4.545642385797848e-05,
	"loss": 0.005032730847597122,
	"num_input_tokens_seen": 421486912,
	"step": 2100,
	"train_runtime": 15067.0008,
	"train_tokens_per_second": 27974.175
	},
	{
	"epoch": 2.159744,
	"grad_norm": 0.07421875,
	"learning_rate": 4.543765029592637e-05,
	"loss": 0.00504358783364296,
	"num_input_tokens_seen": 423541056,
	"step": 2110,
	"train_runtime": 15145.0393,
	"train_tokens_per_second": 27965.662
	},
	{
	"epoch": 2.169984,
	"grad_norm": 0.09033203125,
	"learning_rate": 4.541889997519403e-05,
	"loss": 0.0049100361764431,
	"num_input_tokens_seen": 425501760,
	"step": 2120,
	"train_runtime": 15212.0789,
	"train_tokens_per_second": 27971.309
	},
	{
	"epoch": 2.180224,
	"grad_norm": 0.11767578125,
	"learning_rate": 4.5400172847867095e-05,
	"loss": 0.005002960935235024,
	"num_input_tokens_seen": 427472320,
	"step": 2130,
	"train_runtime": 15279.4866,
	"train_tokens_per_second": 27976.877
	},
	{
	"epoch": 2.190464,
	"grad_norm": 0.08544921875,
	"learning_rate": 4.5381468866169466e-05,
	"loss": 0.005049411952495575,
	"num_input_tokens_seen": 429492544,
	"step": 2140,
	"train_runtime": 15350.0001,
	"train_tokens_per_second": 27979.97
	},
	{
	"epoch": 2.200704,
	"grad_norm": 0.08251953125,
	"learning_rate": 4.5362787982462616e-05,
	"loss": 0.004954206943511963,
	"num_input_tokens_seen": 431474560,
	"step": 2150,
	"train_runtime": 15417.1753,
	"train_tokens_per_second": 27986.616
	},
	{
	"epoch": 2.210944,
	"grad_norm": 0.080078125,
	"learning_rate": 4.5344130149245275e-05,
	"loss": 0.004945812746882439,
	"num_input_tokens_seen": 433476224,
	"step": 2160,
	"train_runtime": 15487.2137,
	"train_tokens_per_second": 27989.297
	},
	{
	"epoch": 2.221184,
	"grad_norm": 0.091796875,
	"learning_rate": 4.5325495319152715e-05,
	"loss": 0.004998266696929932,
	"num_input_tokens_seen": 435507776,
	"step": 2170,
	"train_runtime": 15558.3021,
	"train_tokens_per_second": 27991.986
	},
	{
	"epoch": 2.231424,
	"grad_norm": 0.0703125,
	"learning_rate": 4.530688344495644e-05,
	"loss": 0.00497533455491066,
	"num_input_tokens_seen": 437550336,
	"step": 2180,
	"train_runtime": 15634.0068,
	"train_tokens_per_second": 27987.089
	},
	{
	"epoch": 2.241664,
	"grad_norm": 0.07470703125,
	"learning_rate": 4.528829447956357e-05,
	"loss": 0.004857704415917397,
	"num_input_tokens_seen": 439513280,
	"step": 2190,
	"train_runtime": 15700.438,
	"train_tokens_per_second": 27993.696
	},
	{
	"epoch": 2.251904,
	"grad_norm": 0.078125,
	"learning_rate": 4.526972837601633e-05,
	"loss": 0.004866150766611099,
	"num_input_tokens_seen": 441508032,
	"step": 2200,
	"train_runtime": 15771.6893,
	"train_tokens_per_second": 27993.706
	},
	{
	"epoch": 2.262144,
	"grad_norm": 0.0771484375,
	"learning_rate": 4.525118508749165e-05,
	"loss": 0.004855437949299812,
	"num_input_tokens_seen": 443485504,
	"step": 2210,
	"train_runtime": 15840.4155,
	"train_tokens_per_second": 27997.088
	},
	{
	"epoch": 2.272384,
	"grad_norm": 0.07373046875,
	"learning_rate": 4.5232664567300546e-05,
	"loss": 0.0049121581017971035,
	"num_input_tokens_seen": 445490048,
	"step": 2220,
	"train_runtime": 15912.3409,
	"train_tokens_per_second": 27996.512
	},
	{
	"epoch": 2.282624,
	"grad_norm": 0.07177734375,
	"learning_rate": 4.521416676888773e-05,
	"loss": 0.004935600981116295,
	"num_input_tokens_seen": 447501248,
	"step": 2230,
	"train_runtime": 15984.4843,
	"train_tokens_per_second": 27995.977
	},
	{
	"epoch": 2.292864,
	"grad_norm": 0.06787109375,
	"learning_rate": 4.519569164583107e-05,
	"loss": 0.004881329089403153,
	"num_input_tokens_seen": 449501312,
	"step": 2240,
	"train_runtime": 16056.2463,
	"train_tokens_per_second": 27995.417
	},
	{
	"epoch": 2.303104,
	"grad_norm": 0.07763671875,
	"learning_rate": 4.517723915184109e-05,
	"loss": 0.004859179258346558,
	"num_input_tokens_seen": 451525888,
	"step": 2250,
	"train_runtime": 16130.4768,
	"train_tokens_per_second": 27992.098
	},
	{
	"epoch": 2.313344,
	"grad_norm": 0.0732421875,
	"learning_rate": 4.5158809240760506e-05,
	"loss": 0.0048702418804168705,
	"num_input_tokens_seen": 453539456,
	"step": 2260,
	"train_runtime": 16203.083,
	"train_tokens_per_second": 27990.936
	},
	{
	"epoch": 2.323584,
	"grad_norm": 0.06591796875,
	"learning_rate": 4.514040186656375e-05,
	"loss": 0.004770452529191971,
	"num_input_tokens_seen": 455540800,
	"step": 2270,
	"train_runtime": 16275.7088,
	"train_tokens_per_second": 27988.999
	},
	{
	"epoch": 2.333824,
	"grad_norm": 0.07080078125,
	"learning_rate": 4.512201698335644e-05,
	"loss": 0.004840082675218582,
	"num_input_tokens_seen": 457537984,
	"step": 2280,
	"train_runtime": 16346.6271,
	"train_tokens_per_second": 27989.749
	},
	{
	"epoch": 2.344064,
	"grad_norm": 0.0869140625,
	"learning_rate": 4.510365454537496e-05,
	"loss": 0.004802238196134567,
	"num_input_tokens_seen": 459519040,
	"step": 2290,
	"train_runtime": 16414.8718,
	"train_tokens_per_second": 27994.068
	},
	{
	"epoch": 2.354304,
	"grad_norm": 0.072265625,
	"learning_rate": 4.5085314506985945e-05,
	"loss": 0.0047688383609056475,
	"num_input_tokens_seen": 461504320,
	"step": 2300,
	"train_runtime": 16484.446,
	"train_tokens_per_second": 27996.35
	},
	{
	"epoch": 2.364544,
	"grad_norm": 0.0732421875,
	"learning_rate": 4.50669968226858e-05,
	"loss": 0.004731994122266769,
	"num_input_tokens_seen": 463484608,
	"step": 2310,
	"train_runtime": 16552.5783,
	"train_tokens_per_second": 28000.75
	},
	{
	"epoch": 2.374784,
	"grad_norm": 0.06884765625,
	"learning_rate": 4.504870144710027e-05,
	"loss": 0.004760279133915901,
	"num_input_tokens_seen": 465478912,
	"step": 2320,
	"train_runtime": 16623.4024,
	"train_tokens_per_second": 28001.422
	},
	{
	"epoch": 2.385024,
	"grad_norm": 0.076171875,
	"learning_rate": 4.5030428334983884e-05,
	"loss": 0.004723610356450081,
	"num_input_tokens_seen": 467551232,
	"step": 2330,
	"train_runtime": 16699.1593,
	"train_tokens_per_second": 27998.489
	},
	{
	"epoch": 2.395264,
	"grad_norm": 0.0654296875,
	"learning_rate": 4.501217744121959e-05,
	"loss": 0.004661402851343155,
	"num_input_tokens_seen": 469581568,
	"step": 2340,
	"train_runtime": 16773.939,
	"train_tokens_per_second": 27994.711
	},
	{
	"epoch": 2.405504,
	"grad_norm": 0.0712890625,
	"learning_rate": 4.499394872081821e-05,
	"loss": 0.004748685657978058,
	"num_input_tokens_seen": 471585152,
	"step": 2350,
	"train_runtime": 16845.2785,
	"train_tokens_per_second": 27995.094
	},
	{
	"epoch": 2.415744,
	"grad_norm": 0.0732421875,
	"learning_rate": 4.4975742128918e-05,
	"loss": 0.004757498577237129,
	"num_input_tokens_seen": 473578176,
	"step": 2360,
	"train_runtime": 16915.4571,
	"train_tokens_per_second": 27996.771
	},
	{
	"epoch": 2.425984,
	"grad_norm": 0.076171875,
	"learning_rate": 4.495755762078418e-05,
	"loss": 0.004699341207742691,
	"num_input_tokens_seen": 475608960,
	"step": 2370,
	"train_runtime": 16989.857,
	"train_tokens_per_second": 27993.7
	},
	{
	"epoch": 2.436224,
	"grad_norm": 0.06982421875,
	"learning_rate": 4.49393951518085e-05,
	"loss": 0.004687727242708206,
	"num_input_tokens_seen": 477675968,
	"step": 2380,
	"train_runtime": 17066.5009,
	"train_tokens_per_second": 27989.098
	},
	{
	"epoch": 2.446464,
	"grad_norm": 0.0693359375,
	"learning_rate": 4.4921254677508716e-05,
	"loss": 0.004727355390787125,
	"num_input_tokens_seen": 479688000,
	"step": 2390,
	"train_runtime": 17137.3138,
	"train_tokens_per_second": 27990.851
	},
	{
	"epoch": 2.456704,
	"grad_norm": 0.0712890625,
	"learning_rate": 4.490313615352821e-05,
	"loss": 0.004683735221624375,
	"num_input_tokens_seen": 481709440,
	"step": 2400,
	"train_runtime": 17210.3486,
	"train_tokens_per_second": 27989.523
	},
	{
	"epoch": 2.466944,
	"grad_norm": 0.083984375,
	"learning_rate": 4.48850395356355e-05,
	"loss": 0.0046593818813562395,
	"num_input_tokens_seen": 483715648,
	"step": 2410,
	"train_runtime": 17282.5834,
	"train_tokens_per_second": 27988.619
	},
	{
	"epoch": 2.477184,
	"grad_norm": 0.0654296875,
	"learning_rate": 4.486696477972375e-05,
	"loss": 0.004705347865819931,
	"num_input_tokens_seen": 485743040,
	"step": 2420,
	"train_runtime": 17356.521,
	"train_tokens_per_second": 27986.198
	},
	{
	"epoch": 2.487424,
	"grad_norm": 0.2197265625,
	"learning_rate": 4.484891184181041e-05,
	"loss": 0.004584659263491631,
	"num_input_tokens_seen": 487766208,
	"step": 2430,
	"train_runtime": 17429.6709,
	"train_tokens_per_second": 27984.82
	},
	{
	"epoch": 2.497664,
	"grad_norm": 0.08984375,
	"learning_rate": 4.483088067803662e-05,
	"loss": 0.0046070806682109834,
	"num_input_tokens_seen": 489803136,
	"step": 2440,
	"train_runtime": 17505.2822,
	"train_tokens_per_second": 27980.305
	},
	{
	"epoch": 2.507904,
	"grad_norm": 0.06689453125,
	"learning_rate": 4.481287124466697e-05,
	"loss": 0.004666749015450477,
	"num_input_tokens_seen": 491769280,
	"step": 2450,
	"train_runtime": 17572.5264,
	"train_tokens_per_second": 27985.121
	},
	{
	"epoch": 2.518144,
	"grad_norm": 0.0654296875,
	"learning_rate": 4.479488349808885e-05,
	"loss": 0.0045741736888885495,
	"num_input_tokens_seen": 493764288,
	"step": 2460,
	"train_runtime": 17640.503,
	"train_tokens_per_second": 27990.375
	},
	{
	"epoch": 2.528384,
	"grad_norm": 0.06787109375,
	"learning_rate": 4.4776917394812114e-05,
	"loss": 0.004661215096712112,
	"num_input_tokens_seen": 495765184,
	"step": 2470,
	"train_runtime": 17712.7955,
	"train_tokens_per_second": 27989.099
	},
	{
	"epoch": 2.538624,
	"grad_norm": 0.08154296875,
	"learning_rate": 4.475897289146862e-05,
	"loss": 0.004575810208916664,
	"num_input_tokens_seen": 497788736,
	"step": 2480,
	"train_runtime": 17786.4235,
	"train_tokens_per_second": 27987.006
	},
	{
	"epoch": 2.548864,
	"grad_norm": 0.0654296875,
	"learning_rate": 4.4741049944811806e-05,
	"loss": 0.0045924406498670575,
	"num_input_tokens_seen": 499810304,
	"step": 2490,
	"train_runtime": 17859.0861,
	"train_tokens_per_second": 27986.332
	},
	{
	"epoch": 2.559104,
	"grad_norm": 0.0703125,
	"learning_rate": 4.472314851171621e-05,
	"loss": 0.004592006653547287,
	"num_input_tokens_seen": 501800576,
	"step": 2500,
	"train_runtime": 17928.9445,
	"train_tokens_per_second": 27988.294
	},
	{
	"epoch": 2.569344,
	"grad_norm": 0.08251953125,
	"learning_rate": 4.4705268549177084e-05,
	"loss": 0.004537731781601906,
	"num_input_tokens_seen": 503798656,
	"step": 2510,
	"train_runtime": 18000.9507,
	"train_tokens_per_second": 27987.336
	},
	{
	"epoch": 2.579584,
	"grad_norm": 0.06591796875,
	"learning_rate": 4.468741001430989e-05,
	"loss": 0.004587111622095108,
	"num_input_tokens_seen": 505787584,
	"step": 2520,
	"train_runtime": 18070.7008,
	"train_tokens_per_second": 27989.373
	},
	{
	"epoch": 2.589824,
	"grad_norm": 0.06787109375,
	"learning_rate": 4.466957286434997e-05,
	"loss": 0.004565178602933884,
	"num_input_tokens_seen": 507778304,
	"step": 2530,
	"train_runtime": 18139.8689,
	"train_tokens_per_second": 27992.391
	},
	{
	"epoch": 2.600064,
	"grad_norm": 0.0732421875,
	"learning_rate": 4.4651757056652e-05,
	"loss": 0.004532522708177567,
	"num_input_tokens_seen": 509777024,
	"step": 2540,
	"train_runtime": 18210.7065,
	"train_tokens_per_second": 27993.259
	},
	{
	"epoch": 2.610304,
	"grad_norm": 0.06884765625,
	"learning_rate": 4.463396254868968e-05,
	"loss": 0.004580499976873398,
	"num_input_tokens_seen": 511806464,
	"step": 2550,
	"train_runtime": 18285.3626,
	"train_tokens_per_second": 27989.954
	},
	{
	"epoch": 2.6205439999999998,
	"grad_norm": 0.09619140625,
	"learning_rate": 4.461618929805519e-05,
	"loss": 0.0044912703335285185,
	"num_input_tokens_seen": 513789760,
	"step": 2560,
	"train_runtime": 18355.1328,
	"train_tokens_per_second": 27991.612
	},
	{
	"epoch": 2.6307840000000002,
	"grad_norm": 0.06884765625,
	"learning_rate": 4.459843726245888e-05,
	"loss": 0.0045277226716279985,
	"num_input_tokens_seen": 515820928,
	"step": 2570,
	"train_runtime": 18429.0204,
	"train_tokens_per_second": 27989.601
	},
	{
	"epoch": 2.641024,
	"grad_norm": 0.08056640625,
	"learning_rate": 4.458070639972875e-05,
	"loss": 0.004519717395305633,
	"num_input_tokens_seen": 517852160,
	"step": 2580,
	"train_runtime": 18502.377,
	"train_tokens_per_second": 27988.413
	},
	{
	"epoch": 2.651264,
	"grad_norm": 0.0703125,
	"learning_rate": 4.456299666781007e-05,
	"loss": 0.004499278962612152,
	"num_input_tokens_seen": 519837056,
	"step": 2590,
	"train_runtime": 18571.4439,
	"train_tokens_per_second": 27991.203
	},
	{
	"epoch": 2.661504,
	"grad_norm": 0.07177734375,
	"learning_rate": 4.4545308024764984e-05,
	"loss": 0.004471401870250702,
	"num_input_tokens_seen": 521829760,
	"step": 2600,
	"train_runtime": 18641.848,
	"train_tokens_per_second": 27992.384
	},
	{
	"epoch": 2.671744,
	"grad_norm": 0.076171875,
	"learning_rate": 4.452764042877207e-05,
	"loss": 0.004468469694256782,
	"num_input_tokens_seen": 523852928,
	"step": 2610,
	"train_runtime": 18714.0275,
	"train_tokens_per_second": 27992.527
	},
	{
	"epoch": 2.681984,
	"grad_norm": 0.087890625,
	"learning_rate": 4.45099938381259e-05,
	"loss": 0.0044468618929386135,
	"num_input_tokens_seen": 525863616,
	"step": 2620,
	"train_runtime": 18785.1898,
	"train_tokens_per_second": 27993.522
	},
	{
	"epoch": 2.692224,
	"grad_norm": 0.06884765625,
	"learning_rate": 4.449236821123667e-05,
	"loss": 0.004445591568946838,
	"num_input_tokens_seen": 527876672,
	"step": 2630,
	"train_runtime": 18856.6021,
	"train_tokens_per_second": 27994.263
	},
	{
	"epoch": 2.702464,
	"grad_norm": 0.06201171875,
	"learning_rate": 4.447476350662976e-05,
	"loss": 0.004489725083112716,
	"num_input_tokens_seen": 529852736,
	"step": 2640,
	"train_runtime": 18924.4151,
	"train_tokens_per_second": 27998.368
	},
	{
	"epoch": 2.712704,
	"grad_norm": 0.0654296875,
	"learning_rate": 4.4457179682945346e-05,
	"loss": 0.004479191452264786,
	"num_input_tokens_seen": 531877632,
	"step": 2650,
	"train_runtime": 18996.9669,
	"train_tokens_per_second": 27998.029
	},
	{
	"epoch": 2.722944,
	"grad_norm": 0.0654296875,
	"learning_rate": 4.443961669893798e-05,
	"loss": 0.004402218014001846,
	"num_input_tokens_seen": 533900416,
	"step": 2660,
	"train_runtime": 19070.1423,
	"train_tokens_per_second": 27996.667
	},
	{
	"epoch": 2.733184,
	"grad_norm": 0.06103515625,
	"learning_rate": 4.4422074513476155e-05,
	"loss": 0.004445427656173706,
	"num_input_tokens_seen": 535930112,
	"step": 2670,
	"train_runtime": 19141.6215,
	"train_tokens_per_second": 27998.156
	},
	{
	"epoch": 2.743424,
	"grad_norm": 0.07080078125,
	"learning_rate": 4.4404553085541955e-05,
	"loss": 0.004417391866445542,
	"num_input_tokens_seen": 537917952,
	"step": 2680,
	"train_runtime": 19212.531,
	"train_tokens_per_second": 27998.287
	},
	{
	"epoch": 2.753664,
	"grad_norm": 0.068359375,
	"learning_rate": 4.438705237423063e-05,
	"loss": 0.004418341070413589,
	"num_input_tokens_seen": 539928000,
	"step": 2690,
	"train_runtime": 19284.2139,
	"train_tokens_per_second": 27998.445
	},
	{
	"epoch": 2.763904,
	"grad_norm": 0.072265625,
	"learning_rate": 4.436957233875017e-05,
	"loss": 0.00445760264992714,
	"num_input_tokens_seen": 541990336,
	"step": 2700,
	"train_runtime": 19361.2012,
	"train_tokens_per_second": 27993.632
	},
	{
	"epoch": 2.774144,
	"grad_norm": 0.058349609375,
	"learning_rate": 4.4352112938420956e-05,
	"loss": 0.00442219078540802,
	"num_input_tokens_seen": 543993664,
	"step": 2710,
	"train_runtime": 19434.2897,
	"train_tokens_per_second": 27991.435
	},
	{
	"epoch": 2.784384,
	"grad_norm": 0.06689453125,
	"learning_rate": 4.433467413267529e-05,
	"loss": 0.004379033669829368,
	"num_input_tokens_seen": 545968128,
	"step": 2720,
	"train_runtime": 19501.9205,
	"train_tokens_per_second": 27995.608
	},
	{
	"epoch": 2.7946239999999998,
	"grad_norm": 0.07373046875,
	"learning_rate": 4.431725588105708e-05,
	"loss": 0.00442984439432621,
	"num_input_tokens_seen": 547993536,
	"step": 2730,
	"train_runtime": 19575.6782,
	"train_tokens_per_second": 27993.591
	},
	{
	"epoch": 2.8048640000000002,
	"grad_norm": 0.0712890625,
	"learning_rate": 4.4299858143221377e-05,
	"loss": 0.004416907578706742,
	"num_input_tokens_seen": 549978176,
	"step": 2740,
	"train_runtime": 19646.1077,
	"train_tokens_per_second": 27994.256
	},
	{
	"epoch": 2.815104,
	"grad_norm": 0.06787109375,
	"learning_rate": 4.4282480878934065e-05,
	"loss": 0.004360169917345047,
	"num_input_tokens_seen": 551979968,
	"step": 2750,
	"train_runtime": 19715.949,
	"train_tokens_per_second": 27996.622
	},
	{
	"epoch": 2.825344,
	"grad_norm": 0.06298828125,
	"learning_rate": 4.4265124048071346e-05,
	"loss": 0.00443723276257515,
	"num_input_tokens_seen": 554008768,
	"step": 2760,
	"train_runtime": 19790.2131,
	"train_tokens_per_second": 27994.078
	},
	{
	"epoch": 2.835584,
	"grad_norm": 0.0634765625,
	"learning_rate": 4.4247787610619477e-05,
	"loss": 0.004331726580858231,
	"num_input_tokens_seen": 556008704,
	"step": 2770,
	"train_runtime": 19861.2979,
	"train_tokens_per_second": 27994.581
	},
	{
	"epoch": 2.845824,
	"grad_norm": 0.06494140625,
	"learning_rate": 4.42304715266743e-05,
	"loss": 0.004372353851795197,
	"num_input_tokens_seen": 557996544,
	"step": 2780,
	"train_runtime": 19933.1706,
	"train_tokens_per_second": 27993.366
	},
	{
	"epoch": 2.856064,
	"grad_norm": 0.07177734375,
	"learning_rate": 4.421317575644092e-05,
	"loss": 0.004349645972251892,
	"num_input_tokens_seen": 559980992,
	"step": 2790,
	"train_runtime": 20003.345,
	"train_tokens_per_second": 27994.367
	},
	{
	"epoch": 2.866304,
	"grad_norm": 0.08251953125,
	"learning_rate": 4.419590026023325e-05,
	"loss": 0.004384344071149826,
	"num_input_tokens_seen": 561990848,
	"step": 2800,
	"train_runtime": 20074.5573,
	"train_tokens_per_second": 27995.18
	},
	{
	"epoch": 2.876544,
	"grad_norm": 0.06298828125,
	"learning_rate": 4.417864499847368e-05,
	"loss": 0.004311569407582283,
	"num_input_tokens_seen": 564012800,
	"step": 2810,
	"train_runtime": 20146.4854,
	"train_tokens_per_second": 27995.593
	},
	{
	"epoch": 2.886784,
	"grad_norm": 0.06982421875,
	"learning_rate": 4.4161409931692676e-05,
	"loss": 0.004370152205228806,
	"num_input_tokens_seen": 566014592,
	"step": 2820,
	"train_runtime": 20215.9782,
	"train_tokens_per_second": 27998.378
	},
	{
	"epoch": 2.897024,
	"grad_norm": 0.06787109375,
	"learning_rate": 4.414419502052841e-05,
	"loss": 0.004308675229549408,
	"num_input_tokens_seen": 568045312,
	"step": 2830,
	"train_runtime": 20291.0419,
	"train_tokens_per_second": 27994.881
	},
	{
	"epoch": 2.907264,
	"grad_norm": 0.08203125,
	"learning_rate": 4.412700022572637e-05,
	"loss": 0.0044147070497274395,
	"num_input_tokens_seen": 570100864,
	"step": 2840,
	"train_runtime": 20365.9712,
	"train_tokens_per_second": 27992.815
	},
	{
	"epoch": 2.917504,
	"grad_norm": 0.06884765625,
	"learning_rate": 4.410982550813902e-05,
	"loss": 0.004334438592195511,
	"num_input_tokens_seen": 572089472,
	"step": 2850,
	"train_runtime": 20434.147,
	"train_tokens_per_second": 27996.739
	},
	{
	"epoch": 2.927744,
	"grad_norm": 0.05908203125,
	"learning_rate": 4.409267082872535e-05,
	"loss": 0.004324203729629517,
	"num_input_tokens_seen": 574117952,
	"step": 2860,
	"train_runtime": 20507.6918,
	"train_tokens_per_second": 27995.25
	},
	{
	"epoch": 2.937984,
	"grad_norm": 0.06201171875,
	"learning_rate": 4.407553614855059e-05,
	"loss": 0.0042998895049095156,
	"num_input_tokens_seen": 576123392,
	"step": 2870,
	"train_runtime": 20581.5585,
	"train_tokens_per_second": 27992.214
	},
	{
	"epoch": 2.9482239999999997,
	"grad_norm": 0.064453125,
	"learning_rate": 4.405842142878579e-05,
	"loss": 0.004281196743249893,
	"num_input_tokens_seen": 578142144,
	"step": 2880,
	"train_runtime": 20653.4078,
	"train_tokens_per_second": 27992.579
	},
	{
	"epoch": 2.958464,
	"grad_norm": 0.0634765625,
	"learning_rate": 4.404132663070745e-05,
	"loss": 0.004318735748529434,
	"num_input_tokens_seen": 580136512,
	"step": 2890,
	"train_runtime": 20724.2011,
	"train_tokens_per_second": 27993.191
	},
	{
	"epoch": 2.968704,
	"grad_norm": 0.07958984375,
	"learning_rate": 4.402425171569716e-05,
	"loss": 0.004320795089006424,
	"num_input_tokens_seen": 582175680,
	"step": 2900,
	"train_runtime": 20799.0882,
	"train_tokens_per_second": 27990.442
	},
	{
	"epoch": 2.9789440000000003,
	"grad_norm": 0.0595703125,
	"learning_rate": 4.400719664524127e-05,
	"loss": 0.004287149757146835,
	"num_input_tokens_seen": 584189120,
	"step": 2910,
	"train_runtime": 20871.3369,
	"train_tokens_per_second": 27990.019
	},
	{
	"epoch": 2.989184,
	"grad_norm": 0.064453125,
	"learning_rate": 4.399016138093044e-05,
	"loss": 0.0043055802583694455,
	"num_input_tokens_seen": 586222400,
	"step": 2920,
	"train_runtime": 20945.292,
	"train_tokens_per_second": 27988.266
	},
	{
	"epoch": 2.999424,
	"grad_norm": 0.078125,
	"learning_rate": 4.397314588445937e-05,
	"loss": 0.004327042400836945,
	"num_input_tokens_seen": 588266880,
	"step": 2930,
	"train_runtime": 21020.7033,
	"train_tokens_per_second": 27985.119
	},
	{
	"epoch": 3.0,
	"eval_loss": 1.456993818283081,
	"eval_runtime": 1.1117,
	"eval_samples_per_second": 896.787,
	"eval_steps_per_second": 7.196,
	"num_input_tokens_seen": 588389376,
	"step": 2931
	},
	{
	"epoch": 3.0,
	"num_input_tokens_seen": 588389376,
	"step": 2931,
	"total_flos": 9.795884421293801e+18,
	"train_loss": 0.15086554328385768,
	"train_runtime": 21047.8725,
	"train_samples_per_second": 142.532,
	"train_steps_per_second": 0.139
	}
	],
	"logging_steps": 10,
	"max_steps": 2931,
	"num_input_tokens_seen": 588389376,
	"num_train_epochs": 3,
	"save_steps": 5000,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": true
	},
	"attributes": {}
	}
	},
	"total_flos": 9.795884421293801e+18,
	"train_batch_size": 8,
	"trial_name": null,
	"trial_params": null
	}