Instructions to use roonbug/v9xvk468 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use roonbug/v9xvk468 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="roonbug/v9xvk468")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("roonbug/v9xvk468")
model = AutoModelForImageTextToText.from_pretrained("roonbug/v9xvk468")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use roonbug/v9xvk468 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "roonbug/v9xvk468"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/v9xvk468",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/roonbug/v9xvk468

SGLang

How to use roonbug/v9xvk468 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "roonbug/v9xvk468" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/v9xvk468",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "roonbug/v9xvk468" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/v9xvk468",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use roonbug/v9xvk468 with Docker Model Runner:
```
docker model run hf.co/roonbug/v9xvk468
```

v9xvk468 / checkpoint-2000 /trainer_state.json

roonbug

Upload folder using huggingface_hub

5cd8a44 verified about 1 month ago

raw

history blame contribute delete

87.9 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 0.890224225226729,
	"eval_steps": 100,
	"global_step": 2000,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"entropy": 0.7626848379150033,
	"epoch": 0.004451121126133645,
	"grad_norm": 1888.0,
	"learning_rate": 1.8e-07,
	"loss": 36.6657,
	"mean_token_accuracy": 0.6676603280007839,
	"num_tokens": 63133.0,
	"step": 10
	},
	{
	"entropy": 0.81443009134382,
	"epoch": 0.00890224225226729,
	"grad_norm": 1792.0,
	"learning_rate": 3.8e-07,
	"loss": 36.4468,
	"mean_token_accuracy": 0.6505163595080375,
	"num_tokens": 128968.0,
	"step": 20
	},
	{
	"entropy": 0.8291782144457102,
	"epoch": 0.013353363378400934,
	"grad_norm": 968.0,
	"learning_rate": 5.800000000000001e-07,
	"loss": 34.7077,
	"mean_token_accuracy": 0.6580854222178459,
	"num_tokens": 193314.0,
	"step": 30
	},
	{
	"entropy": 0.9378003980964422,
	"epoch": 0.01780448450453458,
	"grad_norm": 656.0,
	"learning_rate": 7.8e-07,
	"loss": 33.9052,
	"mean_token_accuracy": 0.6638370925560594,
	"num_tokens": 257189.0,
	"step": 40
	},
	{
	"entropy": 0.9725235970690846,
	"epoch": 0.022255605630668224,
	"grad_norm": 724.0,
	"learning_rate": 9.800000000000001e-07,
	"loss": 29.8492,
	"mean_token_accuracy": 0.673487476631999,
	"num_tokens": 324224.0,
	"step": 50
	},
	{
	"entropy": 1.0304172853007914,
	"epoch": 0.026706726756801868,
	"grad_norm": 488.0,
	"learning_rate": 1.1800000000000001e-06,
	"loss": 28.3082,
	"mean_token_accuracy": 0.6809290068224072,
	"num_tokens": 394654.0,
	"step": 60
	},
	{
	"entropy": 1.000378421880305,
	"epoch": 0.031157847882935515,
	"grad_norm": 420.0,
	"learning_rate": 1.3800000000000001e-06,
	"loss": 26.217,
	"mean_token_accuracy": 0.6916999943554402,
	"num_tokens": 461658.0,
	"step": 70
	},
	{
	"entropy": 1.0228427596390248,
	"epoch": 0.03560896900906916,
	"grad_norm": 258.0,
	"learning_rate": 1.5800000000000001e-06,
	"loss": 23.6386,
	"mean_token_accuracy": 0.7020359886810184,
	"num_tokens": 528285.0,
	"step": 80
	},
	{
	"entropy": 1.0894161500036716,
	"epoch": 0.04006009013520281,
	"grad_norm": 218.0,
	"learning_rate": 1.7800000000000001e-06,
	"loss": 23.2798,
	"mean_token_accuracy": 0.6986296704038978,
	"num_tokens": 595336.0,
	"step": 90
	},
	{
	"entropy": 1.0631676118820905,
	"epoch": 0.04451121126133645,
	"grad_norm": 246.0,
	"learning_rate": 1.98e-06,
	"loss": 21.4229,
	"mean_token_accuracy": 0.7073360819369554,
	"num_tokens": 663126.0,
	"step": 100
	},
	{
	"epoch": 0.04451121126133645,
	"eval_biology_entropy": 1.48638627243042,
	"eval_biology_loss": 1.9182748794555664,
	"eval_biology_mean_token_accuracy": 0.6060935441255569,
	"eval_biology_num_tokens": 663126.0,
	"eval_biology_runtime": 49.1778,
	"eval_biology_samples_per_second": 10.167,
	"eval_biology_steps_per_second": 10.167,
	"step": 100
	},
	{
	"epoch": 0.04451121126133645,
	"eval_chemistry_entropy": 1.0556317766308785,
	"eval_chemistry_loss": 1.2898850440979004,
	"eval_chemistry_mean_token_accuracy": 0.7124251598715782,
	"eval_chemistry_num_tokens": 663126.0,
	"eval_chemistry_runtime": 57.0461,
	"eval_chemistry_samples_per_second": 8.765,
	"eval_chemistry_steps_per_second": 8.765,
	"step": 100
	},
	{
	"epoch": 0.04451121126133645,
	"eval_math_entropy": 0.8456157124042512,
	"eval_math_loss": 1.1942517757415771,
	"eval_math_mean_token_accuracy": 0.7334114202260971,
	"eval_math_num_tokens": 663126.0,
	"eval_math_runtime": 59.0482,
	"eval_math_samples_per_second": 8.468,
	"eval_math_steps_per_second": 8.468,
	"step": 100
	},
	{
	"epoch": 0.04451121126133645,
	"eval_physics_entropy": 0.9410946148037911,
	"eval_physics_loss": 1.1514631509780884,
	"eval_physics_mean_token_accuracy": 0.7352130191326142,
	"eval_physics_num_tokens": 663126.0,
	"eval_physics_runtime": 65.895,
	"eval_physics_samples_per_second": 7.588,
	"eval_physics_steps_per_second": 7.588,
	"step": 100
	},
	{
	"entropy": 1.0484314041212202,
	"epoch": 0.048962332387470095,
	"grad_norm": 197.0,
	"learning_rate": 2.1800000000000003e-06,
	"loss": 19.9524,
	"mean_token_accuracy": 0.7201540337875485,
	"num_tokens": 731060.0,
	"step": 110
	},
	{
	"entropy": 1.012692185677588,
	"epoch": 0.053413453513603736,
	"grad_norm": 178.0,
	"learning_rate": 2.38e-06,
	"loss": 18.8663,
	"mean_token_accuracy": 0.7315327662974596,
	"num_tokens": 801635.0,
	"step": 120
	},
	{
	"entropy": 1.0016614170745015,
	"epoch": 0.05786457463973738,
	"grad_norm": 150.0,
	"learning_rate": 2.5800000000000003e-06,
	"loss": 19.0949,
	"mean_token_accuracy": 0.7297663483768702,
	"num_tokens": 867260.0,
	"step": 130
	},
	{
	"entropy": 1.007587356492877,
	"epoch": 0.06231569576587103,
	"grad_norm": 131.0,
	"learning_rate": 2.7800000000000005e-06,
	"loss": 18.9083,
	"mean_token_accuracy": 0.7291399639099836,
	"num_tokens": 931344.0,
	"step": 140
	},
	{
	"entropy": 1.0287242146208881,
	"epoch": 0.06676681689200467,
	"grad_norm": 179.0,
	"learning_rate": 2.9800000000000003e-06,
	"loss": 17.5655,
	"mean_token_accuracy": 0.7393418021500111,
	"num_tokens": 993586.0,
	"step": 150
	},
	{
	"entropy": 1.0563909145072103,
	"epoch": 0.07121793801813832,
	"grad_norm": 126.0,
	"learning_rate": 3.1800000000000005e-06,
	"loss": 16.9203,
	"mean_token_accuracy": 0.7381794668734074,
	"num_tokens": 1060812.0,
	"step": 160
	},
	{
	"entropy": 1.002509175427258,
	"epoch": 0.07566905914427197,
	"grad_norm": 127.0,
	"learning_rate": 3.3800000000000007e-06,
	"loss": 15.5704,
	"mean_token_accuracy": 0.7520427949726581,
	"num_tokens": 1131832.0,
	"step": 170
	},
	{
	"entropy": 0.9855028538033366,
	"epoch": 0.08012018027040561,
	"grad_norm": 101.5,
	"learning_rate": 3.58e-06,
	"loss": 15.2418,
	"mean_token_accuracy": 0.7483469128608704,
	"num_tokens": 1200650.0,
	"step": 180
	},
	{
	"entropy": 0.953622367605567,
	"epoch": 0.08457130139653925,
	"grad_norm": 117.0,
	"learning_rate": 3.7800000000000002e-06,
	"loss": 14.9841,
	"mean_token_accuracy": 0.7532309770584107,
	"num_tokens": 1265107.0,
	"step": 190
	},
	{
	"entropy": 0.9146013628691435,
	"epoch": 0.0890224225226729,
	"grad_norm": 123.0,
	"learning_rate": 3.980000000000001e-06,
	"loss": 14.4834,
	"mean_token_accuracy": 0.7538760716095567,
	"num_tokens": 1327380.0,
	"step": 200
	},
	{
	"epoch": 0.0890224225226729,
	"eval_biology_entropy": 1.2566380687952041,
	"eval_biology_loss": 1.4376745223999023,
	"eval_biology_mean_token_accuracy": 0.6541494448781013,
	"eval_biology_num_tokens": 1327380.0,
	"eval_biology_runtime": 49.1875,
	"eval_biology_samples_per_second": 10.165,
	"eval_biology_steps_per_second": 10.165,
	"step": 200
	},
	{
	"epoch": 0.0890224225226729,
	"eval_chemistry_entropy": 0.8981590569019318,
	"eval_chemistry_loss": 0.9085330963134766,
	"eval_chemistry_mean_token_accuracy": 0.7548674891591072,
	"eval_chemistry_num_tokens": 1327380.0,
	"eval_chemistry_runtime": 57.0026,
	"eval_chemistry_samples_per_second": 8.772,
	"eval_chemistry_steps_per_second": 8.772,
	"step": 200
	},
	{
	"epoch": 0.0890224225226729,
	"eval_math_entropy": 0.7623042597174644,
	"eval_math_loss": 0.8974745869636536,
	"eval_math_mean_token_accuracy": 0.7647860664129257,
	"eval_math_num_tokens": 1327380.0,
	"eval_math_runtime": 58.8532,
	"eval_math_samples_per_second": 8.496,
	"eval_math_steps_per_second": 8.496,
	"step": 200
	},
	{
	"epoch": 0.0890224225226729,
	"eval_physics_entropy": 0.8175762248635292,
	"eval_physics_loss": 0.8456214666366577,
	"eval_physics_mean_token_accuracy": 0.771473237156868,
	"eval_physics_num_tokens": 1327380.0,
	"eval_physics_runtime": 65.7934,
	"eval_physics_samples_per_second": 7.6,
	"eval_physics_steps_per_second": 7.6,
	"step": 200
	},
	{
	"entropy": 0.9297651303932071,
	"epoch": 0.09347354364880654,
	"grad_norm": 121.0,
	"learning_rate": 4.18e-06,
	"loss": 15.1938,
	"mean_token_accuracy": 0.7461945816874505,
	"num_tokens": 1393379.0,
	"step": 210
	},
	{
	"entropy": 0.8862789057195186,
	"epoch": 0.09792466477494019,
	"grad_norm": 98.0,
	"learning_rate": 4.38e-06,
	"loss": 14.1582,
	"mean_token_accuracy": 0.759770842641592,
	"num_tokens": 1460130.0,
	"step": 220
	},
	{
	"entropy": 0.8502754125744104,
	"epoch": 0.10237578590107384,
	"grad_norm": 93.5,
	"learning_rate": 4.58e-06,
	"loss": 13.6798,
	"mean_token_accuracy": 0.7648626536130905,
	"num_tokens": 1529183.0,
	"step": 230
	},
	{
	"entropy": 0.8967906672507524,
	"epoch": 0.10682690702720747,
	"grad_norm": 117.0,
	"learning_rate": 4.78e-06,
	"loss": 14.5446,
	"mean_token_accuracy": 0.7548742420971394,
	"num_tokens": 1597405.0,
	"step": 240
	},
	{
	"entropy": 0.8926160346716643,
	"epoch": 0.11127802815334112,
	"grad_norm": 97.0,
	"learning_rate": 4.980000000000001e-06,
	"loss": 14.4754,
	"mean_token_accuracy": 0.7543840739876032,
	"num_tokens": 1661767.0,
	"step": 250
	},
	{
	"entropy": 0.906158303655684,
	"epoch": 0.11572914927947477,
	"grad_norm": 81.5,
	"learning_rate": 5.18e-06,
	"loss": 14.5882,
	"mean_token_accuracy": 0.7560893476009369,
	"num_tokens": 1728207.0,
	"step": 260
	},
	{
	"entropy": 0.8557273065671325,
	"epoch": 0.12018027040560841,
	"grad_norm": 98.5,
	"learning_rate": 5.380000000000001e-06,
	"loss": 13.7256,
	"mean_token_accuracy": 0.7653402671217918,
	"num_tokens": 1796055.0,
	"step": 270
	},
	{
	"entropy": 0.824279030226171,
	"epoch": 0.12463139153174206,
	"grad_norm": 97.0,
	"learning_rate": 5.580000000000001e-06,
	"loss": 13.2294,
	"mean_token_accuracy": 0.7715799886733293,
	"num_tokens": 1859684.0,
	"step": 280
	},
	{
	"entropy": 0.885771127231419,
	"epoch": 0.1290825126578757,
	"grad_norm": 102.5,
	"learning_rate": 5.78e-06,
	"loss": 14.3879,
	"mean_token_accuracy": 0.7568217847496271,
	"num_tokens": 1921704.0,
	"step": 290
	},
	{
	"entropy": 0.868380057439208,
	"epoch": 0.13353363378400934,
	"grad_norm": 87.5,
	"learning_rate": 5.98e-06,
	"loss": 13.9363,
	"mean_token_accuracy": 0.7626302808523178,
	"num_tokens": 1985766.0,
	"step": 300
	},
	{
	"epoch": 0.13353363378400934,
	"eval_biology_entropy": 1.2238210166096688,
	"eval_biology_loss": 1.366097331047058,
	"eval_biology_mean_token_accuracy": 0.6672504586577416,
	"eval_biology_num_tokens": 1985766.0,
	"eval_biology_runtime": 49.1235,
	"eval_biology_samples_per_second": 10.178,
	"eval_biology_steps_per_second": 10.178,
	"step": 300
	},
	{
	"epoch": 0.13353363378400934,
	"eval_chemistry_entropy": 0.8314956972002983,
	"eval_chemistry_loss": 0.8379236459732056,
	"eval_chemistry_mean_token_accuracy": 0.7699049721360207,
	"eval_chemistry_num_tokens": 1985766.0,
	"eval_chemistry_runtime": 57.0049,
	"eval_chemistry_samples_per_second": 8.771,
	"eval_chemistry_steps_per_second": 8.771,
	"step": 300
	},
	{
	"epoch": 0.13353363378400934,
	"eval_math_entropy": 0.714144405901432,
	"eval_math_loss": 0.874640703201294,
	"eval_math_mean_token_accuracy": 0.7714004491567612,
	"eval_math_num_tokens": 1985766.0,
	"eval_math_runtime": 58.9383,
	"eval_math_samples_per_second": 8.483,
	"eval_math_steps_per_second": 8.483,
	"step": 300
	},
	{
	"epoch": 0.13353363378400934,
	"eval_physics_entropy": 0.7712625939249992,
	"eval_physics_loss": 0.787721574306488,
	"eval_physics_mean_token_accuracy": 0.7829854927659035,
	"eval_physics_num_tokens": 1985766.0,
	"eval_physics_runtime": 65.8905,
	"eval_physics_samples_per_second": 7.588,
	"eval_physics_steps_per_second": 7.588,
	"step": 300
	},
	{
	"entropy": 0.8396226148121059,
	"epoch": 0.137984754910143,
	"grad_norm": 96.5,
	"learning_rate": 6.18e-06,
	"loss": 13.612,
	"mean_token_accuracy": 0.7638351432979107,
	"num_tokens": 2049041.0,
	"step": 310
	},
	{
	"entropy": 0.8020199347287417,
	"epoch": 0.14243587603627664,
	"grad_norm": 81.0,
	"learning_rate": 6.380000000000001e-06,
	"loss": 12.8945,
	"mean_token_accuracy": 0.776324712112546,
	"num_tokens": 2118068.0,
	"step": 320
	},
	{
	"entropy": 0.8072531137615442,
	"epoch": 0.14688699716241027,
	"grad_norm": 107.5,
	"learning_rate": 6.5800000000000005e-06,
	"loss": 12.776,
	"mean_token_accuracy": 0.7789770424365997,
	"num_tokens": 2186387.0,
	"step": 330
	},
	{
	"entropy": 0.8215415397658944,
	"epoch": 0.15133811828854393,
	"grad_norm": 84.5,
	"learning_rate": 6.780000000000001e-06,
	"loss": 13.3957,
	"mean_token_accuracy": 0.7686271335929632,
	"num_tokens": 2252650.0,
	"step": 340
	},
	{
	"entropy": 0.7919685224071145,
	"epoch": 0.15578923941467757,
	"grad_norm": 87.0,
	"learning_rate": 6.98e-06,
	"loss": 12.7862,
	"mean_token_accuracy": 0.7796416375786066,
	"num_tokens": 2320358.0,
	"step": 350
	},
	{
	"entropy": 0.7888439664617181,
	"epoch": 0.16024036054081123,
	"grad_norm": 90.5,
	"learning_rate": 7.180000000000001e-06,
	"loss": 12.6082,
	"mean_token_accuracy": 0.7811494890600443,
	"num_tokens": 2388824.0,
	"step": 360
	},
	{
	"entropy": 0.7693003987893462,
	"epoch": 0.16469148166694486,
	"grad_norm": 82.0,
	"learning_rate": 7.3800000000000005e-06,
	"loss": 12.3808,
	"mean_token_accuracy": 0.7856796395033598,
	"num_tokens": 2456144.0,
	"step": 370
	},
	{
	"entropy": 0.8541094154119492,
	"epoch": 0.1691426027930785,
	"grad_norm": 80.0,
	"learning_rate": 7.58e-06,
	"loss": 13.8583,
	"mean_token_accuracy": 0.7654584795236588,
	"num_tokens": 2515325.0,
	"step": 380
	},
	{
	"entropy": 0.7904249761253596,
	"epoch": 0.17359372391921216,
	"grad_norm": 68.5,
	"learning_rate": 7.78e-06,
	"loss": 12.659,
	"mean_token_accuracy": 0.7788146499544382,
	"num_tokens": 2580490.0,
	"step": 390
	},
	{
	"entropy": 0.7941560545936227,
	"epoch": 0.1780448450453458,
	"grad_norm": 72.5,
	"learning_rate": 7.980000000000002e-06,
	"loss": 12.8107,
	"mean_token_accuracy": 0.7767765510827302,
	"num_tokens": 2644330.0,
	"step": 400
	},
	{
	"epoch": 0.1780448450453458,
	"eval_biology_entropy": 1.2128318169116974,
	"eval_biology_loss": 1.3349602222442627,
	"eval_biology_mean_token_accuracy": 0.6734810082316398,
	"eval_biology_num_tokens": 2644330.0,
	"eval_biology_runtime": 49.0571,
	"eval_biology_samples_per_second": 10.192,
	"eval_biology_steps_per_second": 10.192,
	"step": 400
	},
	{
	"epoch": 0.1780448450453458,
	"eval_chemistry_entropy": 0.7846001248955726,
	"eval_chemistry_loss": 0.798675537109375,
	"eval_chemistry_mean_token_accuracy": 0.7786763918995857,
	"eval_chemistry_num_tokens": 2644330.0,
	"eval_chemistry_runtime": 56.9232,
	"eval_chemistry_samples_per_second": 8.784,
	"eval_chemistry_steps_per_second": 8.784,
	"step": 400
	},
	{
	"epoch": 0.1780448450453458,
	"eval_math_entropy": 0.6919120717644691,
	"eval_math_loss": 0.8597798943519592,
	"eval_math_mean_token_accuracy": 0.7751281379461289,
	"eval_math_num_tokens": 2644330.0,
	"eval_math_runtime": 58.7801,
	"eval_math_samples_per_second": 8.506,
	"eval_math_steps_per_second": 8.506,
	"step": 400
	},
	{
	"epoch": 0.1780448450453458,
	"eval_physics_entropy": 0.7369731608629226,
	"eval_physics_loss": 0.7621525526046753,
	"eval_physics_mean_token_accuracy": 0.7884504457712174,
	"eval_physics_num_tokens": 2644330.0,
	"eval_physics_runtime": 65.7104,
	"eval_physics_samples_per_second": 7.609,
	"eval_physics_steps_per_second": 7.609,
	"step": 400
	},
	{
	"entropy": 0.7377481091767549,
	"epoch": 0.18249596617147945,
	"grad_norm": 73.0,
	"learning_rate": 8.18e-06,
	"loss": 11.7926,
	"mean_token_accuracy": 0.7937501568347216,
	"num_tokens": 2713234.0,
	"step": 410
	},
	{
	"entropy": 0.7830979386344552,
	"epoch": 0.18694708729761308,
	"grad_norm": 76.5,
	"learning_rate": 8.380000000000001e-06,
	"loss": 12.9301,
	"mean_token_accuracy": 0.7784280318766832,
	"num_tokens": 2783261.0,
	"step": 420
	},
	{
	"entropy": 0.7945666501298547,
	"epoch": 0.19139820842374672,
	"grad_norm": 61.5,
	"learning_rate": 8.580000000000001e-06,
	"loss": 12.6515,
	"mean_token_accuracy": 0.7819751728326082,
	"num_tokens": 2850170.0,
	"step": 430
	},
	{
	"entropy": 0.7770357062108815,
	"epoch": 0.19584932954988038,
	"grad_norm": 75.0,
	"learning_rate": 8.78e-06,
	"loss": 12.5072,
	"mean_token_accuracy": 0.7829508323222398,
	"num_tokens": 2913700.0,
	"step": 440
	},
	{
	"entropy": 0.7575128111988306,
	"epoch": 0.200300450676014,
	"grad_norm": 81.0,
	"learning_rate": 8.98e-06,
	"loss": 12.4714,
	"mean_token_accuracy": 0.7839657858014106,
	"num_tokens": 2981212.0,
	"step": 450
	},
	{
	"entropy": 0.7656491708010436,
	"epoch": 0.20475157180214767,
	"grad_norm": 85.0,
	"learning_rate": 9.180000000000002e-06,
	"loss": 12.0974,
	"mean_token_accuracy": 0.7915447618812322,
	"num_tokens": 3048973.0,
	"step": 460
	},
	{
	"entropy": 0.7621381992474199,
	"epoch": 0.2092026929282813,
	"grad_norm": 61.75,
	"learning_rate": 9.38e-06,
	"loss": 12.5808,
	"mean_token_accuracy": 0.7818816823884844,
	"num_tokens": 3117033.0,
	"step": 470
	},
	{
	"entropy": 0.7816813634708524,
	"epoch": 0.21365381405441494,
	"grad_norm": 73.5,
	"learning_rate": 9.58e-06,
	"loss": 12.3819,
	"mean_token_accuracy": 0.7846902538090944,
	"num_tokens": 3185255.0,
	"step": 480
	},
	{
	"entropy": 0.7647983279079199,
	"epoch": 0.2181049351805486,
	"grad_norm": 76.5,
	"learning_rate": 9.780000000000001e-06,
	"loss": 12.4249,
	"mean_token_accuracy": 0.7847896091639995,
	"num_tokens": 3253121.0,
	"step": 490
	},
	{
	"entropy": 0.7214844634756445,
	"epoch": 0.22255605630668224,
	"grad_norm": 75.5,
	"learning_rate": 9.980000000000001e-06,
	"loss": 11.4214,
	"mean_token_accuracy": 0.7984224483370781,
	"num_tokens": 3322823.0,
	"step": 500
	},
	{
	"epoch": 0.22255605630668224,
	"eval_biology_entropy": 1.1607659912109376,
	"eval_biology_loss": 1.328081488609314,
	"eval_biology_mean_token_accuracy": 0.6758214622139931,
	"eval_biology_num_tokens": 3322823.0,
	"eval_biology_runtime": 49.1342,
	"eval_biology_samples_per_second": 10.176,
	"eval_biology_steps_per_second": 10.176,
	"step": 500
	},
	{
	"epoch": 0.22255605630668224,
	"eval_chemistry_entropy": 0.7407792928814888,
	"eval_chemistry_loss": 0.775718629360199,
	"eval_chemistry_mean_token_accuracy": 0.7834744991064072,
	"eval_chemistry_num_tokens": 3322823.0,
	"eval_chemistry_runtime": 57.1618,
	"eval_chemistry_samples_per_second": 8.747,
	"eval_chemistry_steps_per_second": 8.747,
	"step": 500
	},
	{
	"epoch": 0.22255605630668224,
	"eval_math_entropy": 0.6776470097601414,
	"eval_math_loss": 0.8585710525512695,
	"eval_math_mean_token_accuracy": 0.7761784045696258,
	"eval_math_num_tokens": 3322823.0,
	"eval_math_runtime": 58.8725,
	"eval_math_samples_per_second": 8.493,
	"eval_math_steps_per_second": 8.493,
	"step": 500
	},
	{
	"epoch": 0.22255605630668224,
	"eval_physics_entropy": 0.7148050211071968,
	"eval_physics_loss": 0.7503296732902527,
	"eval_physics_mean_token_accuracy": 0.7919680835604668,
	"eval_physics_num_tokens": 3322823.0,
	"eval_physics_runtime": 65.8508,
	"eval_physics_samples_per_second": 7.593,
	"eval_physics_steps_per_second": 7.593,
	"step": 500
	},
	{
	"entropy": 0.7285958690568805,
	"epoch": 0.2270071774328159,
	"grad_norm": 72.5,
	"learning_rate": 1.018e-05,
	"loss": 11.8973,
	"mean_token_accuracy": 0.7914360746741295,
	"num_tokens": 3393393.0,
	"step": 510
	},
	{
	"entropy": 0.8255035754293203,
	"epoch": 0.23145829855894953,
	"grad_norm": 80.0,
	"learning_rate": 1.038e-05,
	"loss": 13.0946,
	"mean_token_accuracy": 0.774183202162385,
	"num_tokens": 3454750.0,
	"step": 520
	},
	{
	"entropy": 0.708361669909209,
	"epoch": 0.2359094196850832,
	"grad_norm": 71.0,
	"learning_rate": 1.0580000000000002e-05,
	"loss": 11.7446,
	"mean_token_accuracy": 0.7961884457617998,
	"num_tokens": 3526914.0,
	"step": 530
	},
	{
	"entropy": 0.8215967276133597,
	"epoch": 0.24036054081121683,
	"grad_norm": 79.0,
	"learning_rate": 1.0780000000000002e-05,
	"loss": 13.0207,
	"mean_token_accuracy": 0.7765944451093674,
	"num_tokens": 3589449.0,
	"step": 540
	},
	{
	"entropy": 0.7564663768745958,
	"epoch": 0.24481166193735046,
	"grad_norm": 82.0,
	"learning_rate": 1.0980000000000002e-05,
	"loss": 12.2873,
	"mean_token_accuracy": 0.7854228101670742,
	"num_tokens": 3655092.0,
	"step": 550
	},
	{
	"entropy": 0.7720718337222934,
	"epoch": 0.24926278306348412,
	"grad_norm": 90.5,
	"learning_rate": 1.1180000000000001e-05,
	"loss": 12.4637,
	"mean_token_accuracy": 0.7846879370510578,
	"num_tokens": 3719113.0,
	"step": 560
	},
	{
	"entropy": 0.7583298075944185,
	"epoch": 0.25371390418961776,
	"grad_norm": 66.5,
	"learning_rate": 1.138e-05,
	"loss": 12.4348,
	"mean_token_accuracy": 0.781843975931406,
	"num_tokens": 3785282.0,
	"step": 570
	},
	{
	"entropy": 0.7643377220258116,
	"epoch": 0.2581650253157514,
	"grad_norm": 63.75,
	"learning_rate": 1.1580000000000001e-05,
	"loss": 12.4092,
	"mean_token_accuracy": 0.7843703482300043,
	"num_tokens": 3852276.0,
	"step": 580
	},
	{
	"entropy": 0.7012410337105394,
	"epoch": 0.262616146441885,
	"grad_norm": 67.0,
	"learning_rate": 1.178e-05,
	"loss": 11.3623,
	"mean_token_accuracy": 0.7991111058741808,
	"num_tokens": 3925649.0,
	"step": 590
	},
	{
	"entropy": 0.7401338685303926,
	"epoch": 0.2670672675680187,
	"grad_norm": 71.5,
	"learning_rate": 1.198e-05,
	"loss": 11.9064,
	"mean_token_accuracy": 0.7897865924984216,
	"num_tokens": 3990505.0,
	"step": 600
	},
	{
	"epoch": 0.2670672675680187,
	"eval_biology_entropy": 1.216272303700447,
	"eval_biology_loss": 1.3142751455307007,
	"eval_biology_mean_token_accuracy": 0.6766086940765381,
	"eval_biology_num_tokens": 3990505.0,
	"eval_biology_runtime": 49.3541,
	"eval_biology_samples_per_second": 10.131,
	"eval_biology_steps_per_second": 10.131,
	"step": 600
	},
	{
	"epoch": 0.2670672675680187,
	"eval_chemistry_entropy": 0.7538464320898056,
	"eval_chemistry_loss": 0.7621961832046509,
	"eval_chemistry_mean_token_accuracy": 0.7865861940383911,
	"eval_chemistry_num_tokens": 3990505.0,
	"eval_chemistry_runtime": 57.3005,
	"eval_chemistry_samples_per_second": 8.726,
	"eval_chemistry_steps_per_second": 8.726,
	"step": 600
	},
	{
	"epoch": 0.2670672675680187,
	"eval_math_entropy": 0.6806064413189888,
	"eval_math_loss": 0.8638136982917786,
	"eval_math_mean_token_accuracy": 0.7749927558898926,
	"eval_math_num_tokens": 3990505.0,
	"eval_math_runtime": 59.1764,
	"eval_math_samples_per_second": 8.449,
	"eval_math_steps_per_second": 8.449,
	"step": 600
	},
	{
	"epoch": 0.2670672675680187,
	"eval_physics_entropy": 0.7255966661572456,
	"eval_physics_loss": 0.7440791726112366,
	"eval_physics_mean_token_accuracy": 0.7921827912330628,
	"eval_physics_num_tokens": 3990505.0,
	"eval_physics_runtime": 66.0704,
	"eval_physics_samples_per_second": 7.568,
	"eval_physics_steps_per_second": 7.568,
	"step": 600
	},
	{
	"entropy": 0.738010767288506,
	"epoch": 0.27151838869415235,
	"grad_norm": 64.0,
	"learning_rate": 1.218e-05,
	"loss": 11.9409,
	"mean_token_accuracy": 0.7896174006164074,
	"num_tokens": 4060657.0,
	"step": 610
	},
	{
	"entropy": 0.7203703055158257,
	"epoch": 0.275969509820286,
	"grad_norm": 67.0,
	"learning_rate": 1.2380000000000002e-05,
	"loss": 11.5655,
	"mean_token_accuracy": 0.7945824809372425,
	"num_tokens": 4133119.0,
	"step": 620
	},
	{
	"entropy": 0.743073652498424,
	"epoch": 0.2804206309464196,
	"grad_norm": 69.5,
	"learning_rate": 1.2580000000000002e-05,
	"loss": 12.0073,
	"mean_token_accuracy": 0.7911369562149048,
	"num_tokens": 4200051.0,
	"step": 630
	},
	{
	"entropy": 0.7967112486250698,
	"epoch": 0.2848717520725533,
	"grad_norm": 72.5,
	"learning_rate": 1.2780000000000001e-05,
	"loss": 12.7974,
	"mean_token_accuracy": 0.7836818125098943,
	"num_tokens": 4267403.0,
	"step": 640
	},
	{
	"entropy": 0.7334000766277313,
	"epoch": 0.28932287319868694,
	"grad_norm": 71.5,
	"learning_rate": 1.2980000000000001e-05,
	"loss": 11.8952,
	"mean_token_accuracy": 0.7923904918134212,
	"num_tokens": 4333268.0,
	"step": 650
	},
	{
	"entropy": 0.7248436002992094,
	"epoch": 0.29377399432482054,
	"grad_norm": 79.5,
	"learning_rate": 1.3180000000000001e-05,
	"loss": 11.7473,
	"mean_token_accuracy": 0.7948706120252609,
	"num_tokens": 4400242.0,
	"step": 660
	},
	{
	"entropy": 0.7555086806416511,
	"epoch": 0.2982251154509542,
	"grad_norm": 77.0,
	"learning_rate": 1.3380000000000002e-05,
	"loss": 12.0493,
	"mean_token_accuracy": 0.7891406118869781,
	"num_tokens": 4464714.0,
	"step": 670
	},
	{
	"entropy": 0.7099162889644504,
	"epoch": 0.30267623657708786,
	"grad_norm": 62.75,
	"learning_rate": 1.3580000000000002e-05,
	"loss": 11.5969,
	"mean_token_accuracy": 0.7962878335267305,
	"num_tokens": 4535458.0,
	"step": 680
	},
	{
	"entropy": 0.7450342487543822,
	"epoch": 0.3071273577032215,
	"grad_norm": 66.5,
	"learning_rate": 1.378e-05,
	"loss": 11.9777,
	"mean_token_accuracy": 0.7921688258647919,
	"num_tokens": 4599099.0,
	"step": 690
	},
	{
	"entropy": 0.7137214630842209,
	"epoch": 0.31157847882935513,
	"grad_norm": 56.5,
	"learning_rate": 1.398e-05,
	"loss": 11.5022,
	"mean_token_accuracy": 0.7959922354668378,
	"num_tokens": 4662903.0,
	"step": 700
	},
	{
	"epoch": 0.31157847882935513,
	"eval_biology_entropy": 1.21318653935194,
	"eval_biology_loss": 1.3126111030578613,
	"eval_biology_mean_token_accuracy": 0.6781674832701683,
	"eval_biology_num_tokens": 4662903.0,
	"eval_biology_runtime": 49.2021,
	"eval_biology_samples_per_second": 10.162,
	"eval_biology_steps_per_second": 10.162,
	"step": 700
	},
	{
	"epoch": 0.31157847882935513,
	"eval_chemistry_entropy": 0.7448744902908802,
	"eval_chemistry_loss": 0.7531416416168213,
	"eval_chemistry_mean_token_accuracy": 0.788379067659378,
	"eval_chemistry_num_tokens": 4662903.0,
	"eval_chemistry_runtime": 57.0266,
	"eval_chemistry_samples_per_second": 8.768,
	"eval_chemistry_steps_per_second": 8.768,
	"step": 700
	},
	{
	"epoch": 0.31157847882935513,
	"eval_math_entropy": 0.6775659238696098,
	"eval_math_loss": 0.8588793277740479,
	"eval_math_mean_token_accuracy": 0.7769290524721145,
	"eval_math_num_tokens": 4662903.0,
	"eval_math_runtime": 58.9903,
	"eval_math_samples_per_second": 8.476,
	"eval_math_steps_per_second": 8.476,
	"step": 700
	},
	{
	"epoch": 0.31157847882935513,
	"eval_physics_entropy": 0.7208585093617439,
	"eval_physics_loss": 0.7438266277313232,
	"eval_physics_mean_token_accuracy": 0.792436719596386,
	"eval_physics_num_tokens": 4662903.0,
	"eval_physics_runtime": 65.9428,
	"eval_physics_samples_per_second": 7.582,
	"eval_physics_steps_per_second": 7.582,
	"step": 700
	},
	{
	"entropy": 0.7532984343357384,
	"epoch": 0.3160295999554888,
	"grad_norm": 82.5,
	"learning_rate": 1.418e-05,
	"loss": 12.3646,
	"mean_token_accuracy": 0.7847790695726872,
	"num_tokens": 4729166.0,
	"step": 710
	},
	{
	"entropy": 0.7946592880412936,
	"epoch": 0.32048072108162245,
	"grad_norm": 79.0,
	"learning_rate": 1.4380000000000001e-05,
	"loss": 12.7491,
	"mean_token_accuracy": 0.7800281438976526,
	"num_tokens": 4796815.0,
	"step": 720
	},
	{
	"entropy": 0.6913739880546927,
	"epoch": 0.32493184220775606,
	"grad_norm": 68.0,
	"learning_rate": 1.4580000000000001e-05,
	"loss": 11.0843,
	"mean_token_accuracy": 0.803905238583684,
	"num_tokens": 4865172.0,
	"step": 730
	},
	{
	"entropy": 0.7447662945836783,
	"epoch": 0.3293829633338897,
	"grad_norm": 65.5,
	"learning_rate": 1.478e-05,
	"loss": 12.1577,
	"mean_token_accuracy": 0.7854402553290128,
	"num_tokens": 4932671.0,
	"step": 740
	},
	{
	"entropy": 0.7749863206408918,
	"epoch": 0.3338340844600234,
	"grad_norm": 64.0,
	"learning_rate": 1.498e-05,
	"loss": 12.4497,
	"mean_token_accuracy": 0.7847358033061027,
	"num_tokens": 4998710.0,
	"step": 750
	},
	{
	"entropy": 0.7126937594264746,
	"epoch": 0.338285205586157,
	"grad_norm": 65.5,
	"learning_rate": 1.5180000000000002e-05,
	"loss": 11.549,
	"mean_token_accuracy": 0.7943205136805773,
	"num_tokens": 5066948.0,
	"step": 760
	},
	{
	"entropy": 0.7463799838908016,
	"epoch": 0.34273632671229065,
	"grad_norm": 66.5,
	"learning_rate": 1.5380000000000002e-05,
	"loss": 12.052,
	"mean_token_accuracy": 0.7893575474619865,
	"num_tokens": 5129950.0,
	"step": 770
	},
	{
	"entropy": 0.7669129572808743,
	"epoch": 0.3471874478384243,
	"grad_norm": 64.5,
	"learning_rate": 1.5580000000000003e-05,
	"loss": 12.4587,
	"mean_token_accuracy": 0.7811328556388617,
	"num_tokens": 5193519.0,
	"step": 780
	},
	{
	"entropy": 0.7404427804984153,
	"epoch": 0.351638568964558,
	"grad_norm": 87.0,
	"learning_rate": 1.578e-05,
	"loss": 11.9383,
	"mean_token_accuracy": 0.7870475023984909,
	"num_tokens": 5256503.0,
	"step": 790
	},
	{
	"entropy": 0.8177574999630451,
	"epoch": 0.3560896900906916,
	"grad_norm": 69.0,
	"learning_rate": 1.5980000000000003e-05,
	"loss": 13.1167,
	"mean_token_accuracy": 0.7771423246711493,
	"num_tokens": 5324751.0,
	"step": 800
	},
	{
	"epoch": 0.3560896900906916,
	"eval_biology_entropy": 1.2992450638413429,
	"eval_biology_loss": 1.3091635704040527,
	"eval_biology_mean_token_accuracy": 0.6765963944792748,
	"eval_biology_num_tokens": 5324751.0,
	"eval_biology_runtime": 49.4045,
	"eval_biology_samples_per_second": 10.121,
	"eval_biology_steps_per_second": 10.121,
	"step": 800
	},
	{
	"epoch": 0.3560896900906916,
	"eval_chemistry_entropy": 0.7858416083455085,
	"eval_chemistry_loss": 0.755605936050415,
	"eval_chemistry_mean_token_accuracy": 0.7878280827403068,
	"eval_chemistry_num_tokens": 5324751.0,
	"eval_chemistry_runtime": 57.2096,
	"eval_chemistry_samples_per_second": 8.74,
	"eval_chemistry_steps_per_second": 8.74,
	"step": 800
	},
	{
	"epoch": 0.3560896900906916,
	"eval_math_entropy": 0.7085552672445774,
	"eval_math_loss": 0.8628776669502258,
	"eval_math_mean_token_accuracy": 0.7751141096353531,
	"eval_math_num_tokens": 5324751.0,
	"eval_math_runtime": 58.9925,
	"eval_math_samples_per_second": 8.476,
	"eval_math_steps_per_second": 8.476,
	"step": 800
	},
	{
	"epoch": 0.3560896900906916,
	"eval_physics_entropy": 0.7641420513391495,
	"eval_physics_loss": 0.7484508156776428,
	"eval_physics_mean_token_accuracy": 0.7914790090918541,
	"eval_physics_num_tokens": 5324751.0,
	"eval_physics_runtime": 65.8581,
	"eval_physics_samples_per_second": 7.592,
	"eval_physics_steps_per_second": 7.592,
	"step": 800
	},
	{
	"entropy": 0.6714803985320031,
	"epoch": 0.36054081121682524,
	"grad_norm": 65.5,
	"learning_rate": 1.618e-05,
	"loss": 11.0959,
	"mean_token_accuracy": 0.8037038501352072,
	"num_tokens": 5391664.0,
	"step": 810
	},
	{
	"entropy": 0.7128949659876526,
	"epoch": 0.3649919323429589,
	"grad_norm": 71.5,
	"learning_rate": 1.638e-05,
	"loss": 11.223,
	"mean_token_accuracy": 0.7998184654861689,
	"num_tokens": 5458519.0,
	"step": 820
	},
	{
	"entropy": 0.7470979234203696,
	"epoch": 0.3694430534690925,
	"grad_norm": 64.5,
	"learning_rate": 1.658e-05,
	"loss": 12.1501,
	"mean_token_accuracy": 0.7894150543957948,
	"num_tokens": 5523391.0,
	"step": 830
	},
	{
	"entropy": 0.7144336312077939,
	"epoch": 0.37389417459522617,
	"grad_norm": 61.25,
	"learning_rate": 1.6780000000000002e-05,
	"loss": 11.736,
	"mean_token_accuracy": 0.7935146864503622,
	"num_tokens": 5585508.0,
	"step": 840
	},
	{
	"entropy": 0.7430889600887894,
	"epoch": 0.37834529572135983,
	"grad_norm": 72.0,
	"learning_rate": 1.698e-05,
	"loss": 11.8763,
	"mean_token_accuracy": 0.7954710725694895,
	"num_tokens": 5650755.0,
	"step": 850
	},
	{
	"entropy": 0.7431833678856492,
	"epoch": 0.38279641684749344,
	"grad_norm": 62.25,
	"learning_rate": 1.718e-05,
	"loss": 11.9703,
	"mean_token_accuracy": 0.7893174722790718,
	"num_tokens": 5716321.0,
	"step": 860
	},
	{
	"entropy": 0.7719974908977747,
	"epoch": 0.3872475379736271,
	"grad_norm": 66.5,
	"learning_rate": 1.7380000000000003e-05,
	"loss": 12.4773,
	"mean_token_accuracy": 0.7817879624664783,
	"num_tokens": 5782698.0,
	"step": 870
	},
	{
	"entropy": 0.7235654108226299,
	"epoch": 0.39169865909976076,
	"grad_norm": 53.75,
	"learning_rate": 1.758e-05,
	"loss": 11.6881,
	"mean_token_accuracy": 0.7943595319986343,
	"num_tokens": 5848889.0,
	"step": 880
	},
	{
	"entropy": 0.7637507017701864,
	"epoch": 0.3961497802258944,
	"grad_norm": 64.0,
	"learning_rate": 1.7780000000000003e-05,
	"loss": 12.5431,
	"mean_token_accuracy": 0.7863847129046917,
	"num_tokens": 5912919.0,
	"step": 890
	},
	{
	"entropy": 0.742395118996501,
	"epoch": 0.400600901352028,
	"grad_norm": 57.5,
	"learning_rate": 1.798e-05,
	"loss": 11.9098,
	"mean_token_accuracy": 0.79401213824749,
	"num_tokens": 5980229.0,
	"step": 900
	},
	{
	"epoch": 0.400600901352028,
	"eval_biology_entropy": 1.2397134986519813,
	"eval_biology_loss": 1.3142763376235962,
	"eval_biology_mean_token_accuracy": 0.6771629434227944,
	"eval_biology_num_tokens": 5980229.0,
	"eval_biology_runtime": 49.4582,
	"eval_biology_samples_per_second": 10.11,
	"eval_biology_steps_per_second": 10.11,
	"step": 900
	},
	{
	"epoch": 0.400600901352028,
	"eval_chemistry_entropy": 0.748387647986412,
	"eval_chemistry_loss": 0.7568163275718689,
	"eval_chemistry_mean_token_accuracy": 0.7869879400134087,
	"eval_chemistry_num_tokens": 5980229.0,
	"eval_chemistry_runtime": 57.0636,
	"eval_chemistry_samples_per_second": 8.762,
	"eval_chemistry_steps_per_second": 8.762,
	"step": 900
	},
	{
	"epoch": 0.400600901352028,
	"eval_math_entropy": 0.6898269492387772,
	"eval_math_loss": 0.866584062576294,
	"eval_math_mean_token_accuracy": 0.7752813557386399,
	"eval_math_num_tokens": 5980229.0,
	"eval_math_runtime": 59.1998,
	"eval_math_samples_per_second": 8.446,
	"eval_math_steps_per_second": 8.446,
	"step": 900
	},
	{
	"epoch": 0.400600901352028,
	"eval_physics_entropy": 0.7360889357328415,
	"eval_physics_loss": 0.7502807378768921,
	"eval_physics_mean_token_accuracy": 0.7913857514262199,
	"eval_physics_num_tokens": 5980229.0,
	"eval_physics_runtime": 66.2752,
	"eval_physics_samples_per_second": 7.544,
	"eval_physics_steps_per_second": 7.544,
	"step": 900
	},
	{
	"entropy": 0.6976769424974918,
	"epoch": 0.4050520224781617,
	"grad_norm": 57.75,
	"learning_rate": 1.8180000000000002e-05,
	"loss": 11.373,
	"mean_token_accuracy": 0.7986949652433395,
	"num_tokens": 6048508.0,
	"step": 910
	},
	{
	"entropy": 0.7293819307349623,
	"epoch": 0.40950314360429535,
	"grad_norm": 59.75,
	"learning_rate": 1.8380000000000004e-05,
	"loss": 11.7158,
	"mean_token_accuracy": 0.7940746705979109,
	"num_tokens": 6114855.0,
	"step": 920
	},
	{
	"entropy": 0.7529061837121844,
	"epoch": 0.41395426473042896,
	"grad_norm": 58.5,
	"learning_rate": 1.858e-05,
	"loss": 12.1233,
	"mean_token_accuracy": 0.7879564348608256,
	"num_tokens": 6179050.0,
	"step": 930
	},
	{
	"entropy": 0.7420403499156236,
	"epoch": 0.4184053858565626,
	"grad_norm": 60.25,
	"learning_rate": 1.878e-05,
	"loss": 12.11,
	"mean_token_accuracy": 0.7902424175292253,
	"num_tokens": 6244987.0,
	"step": 940
	},
	{
	"entropy": 0.7153761054389178,
	"epoch": 0.4228565069826963,
	"grad_norm": 65.0,
	"learning_rate": 1.898e-05,
	"loss": 11.5745,
	"mean_token_accuracy": 0.7952297408133745,
	"num_tokens": 6310799.0,
	"step": 950
	},
	{
	"entropy": 0.7087640034034848,
	"epoch": 0.4273076281088299,
	"grad_norm": 62.5,
	"learning_rate": 1.918e-05,
	"loss": 11.5358,
	"mean_token_accuracy": 0.7965508222579956,
	"num_tokens": 6378152.0,
	"step": 960
	},
	{
	"entropy": 0.7380615974776447,
	"epoch": 0.43175874923496355,
	"grad_norm": 52.25,
	"learning_rate": 1.938e-05,
	"loss": 11.7537,
	"mean_token_accuracy": 0.7940621051937342,
	"num_tokens": 6442491.0,
	"step": 970
	},
	{
	"entropy": 0.7700119714252651,
	"epoch": 0.4362098703610972,
	"grad_norm": 55.75,
	"learning_rate": 1.9580000000000002e-05,
	"loss": 12.5586,
	"mean_token_accuracy": 0.7843215424567461,
	"num_tokens": 6510231.0,
	"step": 980
	},
	{
	"entropy": 0.7417776238173246,
	"epoch": 0.44066099148723087,
	"grad_norm": 57.25,
	"learning_rate": 1.978e-05,
	"loss": 12.0337,
	"mean_token_accuracy": 0.7908143695443869,
	"num_tokens": 6572858.0,
	"step": 990
	},
	{
	"entropy": 0.7601414297707378,
	"epoch": 0.4451121126133645,
	"grad_norm": 49.75,
	"learning_rate": 1.9980000000000002e-05,
	"loss": 12.3686,
	"mean_token_accuracy": 0.7845167253166437,
	"num_tokens": 6637273.0,
	"step": 1000
	},
	{
	"epoch": 0.4451121126133645,
	"eval_biology_entropy": 1.2479438411593438,
	"eval_biology_loss": 1.3289152383804321,
	"eval_biology_mean_token_accuracy": 0.67418065315485,
	"eval_biology_num_tokens": 6637273.0,
	"eval_biology_runtime": 49.5469,
	"eval_biology_samples_per_second": 10.091,
	"eval_biology_steps_per_second": 10.091,
	"step": 1000
	},
	{
	"epoch": 0.4451121126133645,
	"eval_chemistry_entropy": 0.7556402538418769,
	"eval_chemistry_loss": 0.7549638748168945,
	"eval_chemistry_mean_token_accuracy": 0.7876601304411888,
	"eval_chemistry_num_tokens": 6637273.0,
	"eval_chemistry_runtime": 57.6175,
	"eval_chemistry_samples_per_second": 8.678,
	"eval_chemistry_steps_per_second": 8.678,
	"step": 1000
	},
	{
	"epoch": 0.4451121126133645,
	"eval_math_entropy": 0.7018868820667267,
	"eval_math_loss": 0.8697984218597412,
	"eval_math_mean_token_accuracy": 0.7734663159847259,
	"eval_math_num_tokens": 6637273.0,
	"eval_math_runtime": 59.4977,
	"eval_math_samples_per_second": 8.404,
	"eval_math_steps_per_second": 8.404,
	"step": 1000
	},
	{
	"epoch": 0.4451121126133645,
	"eval_physics_entropy": 0.7586731649637223,
	"eval_physics_loss": 0.7596898078918457,
	"eval_physics_mean_token_accuracy": 0.7887503932714462,
	"eval_physics_num_tokens": 6637273.0,
	"eval_physics_runtime": 66.4638,
	"eval_physics_samples_per_second": 7.523,
	"eval_physics_steps_per_second": 7.523,
	"step": 1000
	},
	{
	"entropy": 0.7277510199695826,
	"epoch": 0.44956323373949814,
	"grad_norm": 60.75,
	"learning_rate": 1.9980000000000002e-05,
	"loss": 11.7227,
	"mean_token_accuracy": 0.7909994043409825,
	"num_tokens": 6700799.0,
	"step": 1010
	},
	{
	"entropy": 0.7427934074774385,
	"epoch": 0.4540143548656318,
	"grad_norm": 58.25,
	"learning_rate": 1.995777777777778e-05,
	"loss": 12.1597,
	"mean_token_accuracy": 0.7892370050773024,
	"num_tokens": 6765798.0,
	"step": 1020
	},
	{
	"entropy": 0.6983263311907649,
	"epoch": 0.4584654759917654,
	"grad_norm": 57.25,
	"learning_rate": 1.9935555555555557e-05,
	"loss": 11.3167,
	"mean_token_accuracy": 0.7985999789088964,
	"num_tokens": 6832774.0,
	"step": 1030
	},
	{
	"entropy": 0.7731405256316066,
	"epoch": 0.46291659711789906,
	"grad_norm": 57.5,
	"learning_rate": 1.9913333333333335e-05,
	"loss": 12.4454,
	"mean_token_accuracy": 0.7837167579680682,
	"num_tokens": 6896684.0,
	"step": 1040
	},
	{
	"entropy": 0.7266025519929826,
	"epoch": 0.4673677182440327,
	"grad_norm": 58.0,
	"learning_rate": 1.9891111111111112e-05,
	"loss": 11.7153,
	"mean_token_accuracy": 0.7996914375573396,
	"num_tokens": 6964423.0,
	"step": 1050
	},
	{
	"entropy": 0.7238340148702264,
	"epoch": 0.4718188393701664,
	"grad_norm": 53.75,
	"learning_rate": 1.986888888888889e-05,
	"loss": 11.8798,
	"mean_token_accuracy": 0.7931064553558826,
	"num_tokens": 7033417.0,
	"step": 1060
	},
	{
	"entropy": 0.7728124929592013,
	"epoch": 0.4762699604963,
	"grad_norm": 55.75,
	"learning_rate": 1.9846666666666668e-05,
	"loss": 12.3099,
	"mean_token_accuracy": 0.7855284217745065,
	"num_tokens": 7101274.0,
	"step": 1070
	},
	{
	"entropy": 0.7387059103697539,
	"epoch": 0.48072108162243365,
	"grad_norm": 50.0,
	"learning_rate": 1.9824444444444445e-05,
	"loss": 12.1706,
	"mean_token_accuracy": 0.7893157167360186,
	"num_tokens": 7166608.0,
	"step": 1080
	},
	{
	"entropy": 0.7172827901318669,
	"epoch": 0.4851722027485673,
	"grad_norm": 71.0,
	"learning_rate": 1.9802222222222226e-05,
	"loss": 11.5067,
	"mean_token_accuracy": 0.801088172197342,
	"num_tokens": 7240958.0,
	"step": 1090
	},
	{
	"entropy": 0.7408099344000221,
	"epoch": 0.4896233238747009,
	"grad_norm": 57.5,
	"learning_rate": 1.978e-05,
	"loss": 11.9898,
	"mean_token_accuracy": 0.789643631502986,
	"num_tokens": 7311968.0,
	"step": 1100
	},
	{
	"epoch": 0.4896233238747009,
	"eval_biology_entropy": 1.2769847421050071,
	"eval_biology_loss": 1.3426203727722168,
	"eval_biology_mean_token_accuracy": 0.6723933680057526,
	"eval_biology_num_tokens": 7311968.0,
	"eval_biology_runtime": 49.9477,
	"eval_biology_samples_per_second": 10.01,
	"eval_biology_steps_per_second": 10.01,
	"step": 1100
	},
	{
	"epoch": 0.4896233238747009,
	"eval_chemistry_entropy": 0.7427506719231606,
	"eval_chemistry_loss": 0.7505869269371033,
	"eval_chemistry_mean_token_accuracy": 0.78900011998415,
	"eval_chemistry_num_tokens": 7311968.0,
	"eval_chemistry_runtime": 57.6815,
	"eval_chemistry_samples_per_second": 8.668,
	"eval_chemistry_steps_per_second": 8.668,
	"step": 1100
	},
	{
	"epoch": 0.4896233238747009,
	"eval_math_entropy": 0.6985864902734756,
	"eval_math_loss": 0.873950183391571,
	"eval_math_mean_token_accuracy": 0.7733350949287414,
	"eval_math_num_tokens": 7311968.0,
	"eval_math_runtime": 59.3201,
	"eval_math_samples_per_second": 8.429,
	"eval_math_steps_per_second": 8.429,
	"step": 1100
	},
	{
	"epoch": 0.4896233238747009,
	"eval_physics_entropy": 0.7379600911140441,
	"eval_physics_loss": 0.7551804780960083,
	"eval_physics_mean_token_accuracy": 0.7902662333846092,
	"eval_physics_num_tokens": 7311968.0,
	"eval_physics_runtime": 66.0636,
	"eval_physics_samples_per_second": 7.568,
	"eval_physics_steps_per_second": 7.568,
	"step": 1100
	},
	{
	"entropy": 0.7308115581981838,
	"epoch": 0.4940744450008346,
	"grad_norm": 57.5,
	"learning_rate": 1.975777777777778e-05,
	"loss": 11.7632,
	"mean_token_accuracy": 0.7919334191828966,
	"num_tokens": 7378961.0,
	"step": 1110
	},
	{
	"entropy": 0.7051068279892206,
	"epoch": 0.49852556612696824,
	"grad_norm": 62.5,
	"learning_rate": 1.9735555555555556e-05,
	"loss": 11.4412,
	"mean_token_accuracy": 0.7976060153916478,
	"num_tokens": 7444923.0,
	"step": 1120
	},
	{
	"entropy": 0.7438309504650533,
	"epoch": 0.5029766872531019,
	"grad_norm": 55.5,
	"learning_rate": 1.9713333333333337e-05,
	"loss": 11.8086,
	"mean_token_accuracy": 0.7935861334204674,
	"num_tokens": 7509885.0,
	"step": 1130
	},
	{
	"entropy": 0.7525124587118626,
	"epoch": 0.5074278083792355,
	"grad_norm": 55.0,
	"learning_rate": 1.969111111111111e-05,
	"loss": 12.4112,
	"mean_token_accuracy": 0.7843238320201635,
	"num_tokens": 7576456.0,
	"step": 1140
	},
	{
	"entropy": 0.7765682274475694,
	"epoch": 0.5118789295053692,
	"grad_norm": 56.5,
	"learning_rate": 1.9668888888888892e-05,
	"loss": 12.3668,
	"mean_token_accuracy": 0.7860391337424517,
	"num_tokens": 7641821.0,
	"step": 1150
	},
	{
	"entropy": 0.7054132025688886,
	"epoch": 0.5163300506315028,
	"grad_norm": 54.75,
	"learning_rate": 1.9646666666666666e-05,
	"loss": 11.5281,
	"mean_token_accuracy": 0.7944953251630068,
	"num_tokens": 7706502.0,
	"step": 1160
	},
	{
	"entropy": 0.7991299530491233,
	"epoch": 0.5207811717576365,
	"grad_norm": 61.5,
	"learning_rate": 1.9624444444444447e-05,
	"loss": 12.8341,
	"mean_token_accuracy": 0.782982062548399,
	"num_tokens": 7772353.0,
	"step": 1170
	},
	{
	"entropy": 0.7132468383759261,
	"epoch": 0.52523229288377,
	"grad_norm": 49.75,
	"learning_rate": 1.9602222222222225e-05,
	"loss": 11.7127,
	"mean_token_accuracy": 0.796727203950286,
	"num_tokens": 7838516.0,
	"step": 1180
	},
	{
	"entropy": 0.7221027395687998,
	"epoch": 0.5296834140099037,
	"grad_norm": 61.5,
	"learning_rate": 1.9580000000000002e-05,
	"loss": 11.4862,
	"mean_token_accuracy": 0.7964579328894615,
	"num_tokens": 7902568.0,
	"step": 1190
	},
	{
	"entropy": 0.7554050230421125,
	"epoch": 0.5341345351360374,
	"grad_norm": 55.0,
	"learning_rate": 1.955777777777778e-05,
	"loss": 12.018,
	"mean_token_accuracy": 0.7877018984407187,
	"num_tokens": 7969704.0,
	"step": 1200
	},
	{
	"epoch": 0.5341345351360374,
	"eval_biology_entropy": 1.2183623771071435,
	"eval_biology_loss": 1.3340805768966675,
	"eval_biology_mean_token_accuracy": 0.6738465265035629,
	"eval_biology_num_tokens": 7969704.0,
	"eval_biology_runtime": 49.7,
	"eval_biology_samples_per_second": 10.06,
	"eval_biology_steps_per_second": 10.06,
	"step": 1200
	},
	{
	"epoch": 0.5341345351360374,
	"eval_chemistry_entropy": 0.7287724848091602,
	"eval_chemistry_loss": 0.7458793520927429,
	"eval_chemistry_mean_token_accuracy": 0.7902151345610618,
	"eval_chemistry_num_tokens": 7969704.0,
	"eval_chemistry_runtime": 57.2957,
	"eval_chemistry_samples_per_second": 8.727,
	"eval_chemistry_steps_per_second": 8.727,
	"step": 1200
	},
	{
	"epoch": 0.5341345351360374,
	"eval_math_entropy": 0.6740069650709629,
	"eval_math_loss": 0.8847513794898987,
	"eval_math_mean_token_accuracy": 0.7725249216556549,
	"eval_math_num_tokens": 7969704.0,
	"eval_math_runtime": 59.1059,
	"eval_math_samples_per_second": 8.459,
	"eval_math_steps_per_second": 8.459,
	"step": 1200
	},
	{
	"epoch": 0.5341345351360374,
	"eval_physics_entropy": 0.7253319441080094,
	"eval_physics_loss": 0.7566389441490173,
	"eval_physics_mean_token_accuracy": 0.790060887992382,
	"eval_physics_num_tokens": 7969704.0,
	"eval_physics_runtime": 66.1619,
	"eval_physics_samples_per_second": 7.557,
	"eval_physics_steps_per_second": 7.557,
	"step": 1200
	},
	{
	"entropy": 0.692601692955941,
	"epoch": 0.538585656262171,
	"grad_norm": 47.25,
	"learning_rate": 1.9535555555555557e-05,
	"loss": 11.4509,
	"mean_token_accuracy": 0.7953360680490732,
	"num_tokens": 8034765.0,
	"step": 1210
	},
	{
	"entropy": 0.7510794518515468,
	"epoch": 0.5430367773883047,
	"grad_norm": 45.5,
	"learning_rate": 1.9513333333333335e-05,
	"loss": 11.9891,
	"mean_token_accuracy": 0.7911988774314522,
	"num_tokens": 8103869.0,
	"step": 1220
	},
	{
	"entropy": 0.6859279408119618,
	"epoch": 0.5474878985144384,
	"grad_norm": 44.25,
	"learning_rate": 1.9491111111111113e-05,
	"loss": 11.092,
	"mean_token_accuracy": 0.8032916888594628,
	"num_tokens": 8173386.0,
	"step": 1230
	},
	{
	"entropy": 0.7368258788250387,
	"epoch": 0.551939019640572,
	"grad_norm": 51.75,
	"learning_rate": 1.946888888888889e-05,
	"loss": 11.8628,
	"mean_token_accuracy": 0.7892861522734165,
	"num_tokens": 8242162.0,
	"step": 1240
	},
	{
	"entropy": 0.7227346986532212,
	"epoch": 0.5563901407667056,
	"grad_norm": 58.5,
	"learning_rate": 1.9446666666666668e-05,
	"loss": 11.6799,
	"mean_token_accuracy": 0.7937116343528032,
	"num_tokens": 8308919.0,
	"step": 1250
	},
	{
	"entropy": 0.7348461709916592,
	"epoch": 0.5608412618928392,
	"grad_norm": 65.5,
	"learning_rate": 1.9424444444444446e-05,
	"loss": 11.985,
	"mean_token_accuracy": 0.7889482360333204,
	"num_tokens": 8365617.0,
	"step": 1260
	},
	{
	"entropy": 0.7478212734684349,
	"epoch": 0.5652923830189729,
	"grad_norm": 49.0,
	"learning_rate": 1.9402222222222223e-05,
	"loss": 11.9453,
	"mean_token_accuracy": 0.7913831114768982,
	"num_tokens": 8430728.0,
	"step": 1270
	},
	{
	"entropy": 0.7140952883288264,
	"epoch": 0.5697435041451065,
	"grad_norm": 49.0,
	"learning_rate": 1.938e-05,
	"loss": 11.5919,
	"mean_token_accuracy": 0.7973519455641508,
	"num_tokens": 8497852.0,
	"step": 1280
	},
	{
	"entropy": 0.7445681392215192,
	"epoch": 0.5741946252712402,
	"grad_norm": 49.0,
	"learning_rate": 1.935777777777778e-05,
	"loss": 12.1553,
	"mean_token_accuracy": 0.78705313205719,
	"num_tokens": 8558952.0,
	"step": 1290
	},
	{
	"entropy": 0.739489266090095,
	"epoch": 0.5786457463973739,
	"grad_norm": 58.25,
	"learning_rate": 1.9335555555555556e-05,
	"loss": 11.8581,
	"mean_token_accuracy": 0.796079633757472,
	"num_tokens": 8622868.0,
	"step": 1300
	},
	{
	"epoch": 0.5786457463973739,
	"eval_biology_entropy": 1.2557496595382691,
	"eval_biology_loss": 1.3392294645309448,
	"eval_biology_mean_token_accuracy": 0.673343878209591,
	"eval_biology_num_tokens": 8622868.0,
	"eval_biology_runtime": 49.5632,
	"eval_biology_samples_per_second": 10.088,
	"eval_biology_steps_per_second": 10.088,
	"step": 1300
	},
	{
	"epoch": 0.5786457463973739,
	"eval_chemistry_entropy": 0.720954676181078,
	"eval_chemistry_loss": 0.7364406585693359,
	"eval_chemistry_mean_token_accuracy": 0.7934513345956803,
	"eval_chemistry_num_tokens": 8622868.0,
	"eval_chemistry_runtime": 57.2523,
	"eval_chemistry_samples_per_second": 8.733,
	"eval_chemistry_steps_per_second": 8.733,
	"step": 1300
	},
	{
	"epoch": 0.5786457463973739,
	"eval_math_entropy": 0.6704416258633137,
	"eval_math_loss": 0.892277717590332,
	"eval_math_mean_token_accuracy": 0.7730021580457688,
	"eval_math_num_tokens": 8622868.0,
	"eval_math_runtime": 59.0108,
	"eval_math_samples_per_second": 8.473,
	"eval_math_steps_per_second": 8.473,
	"step": 1300
	},
	{
	"epoch": 0.5786457463973739,
	"eval_physics_entropy": 0.7133055707812309,
	"eval_physics_loss": 0.7568183541297913,
	"eval_physics_mean_token_accuracy": 0.7910581494569778,
	"eval_physics_num_tokens": 8622868.0,
	"eval_physics_runtime": 66.0807,
	"eval_physics_samples_per_second": 7.567,
	"eval_physics_steps_per_second": 7.567,
	"step": 1300
	},
	{
	"entropy": 0.6994675314053893,
	"epoch": 0.5830968675235075,
	"grad_norm": 56.0,
	"learning_rate": 1.9313333333333334e-05,
	"loss": 11.4251,
	"mean_token_accuracy": 0.7967592880129815,
	"num_tokens": 8694510.0,
	"step": 1310
	},
	{
	"entropy": 0.78238091673702,
	"epoch": 0.5875479886496411,
	"grad_norm": 65.0,
	"learning_rate": 1.9291111111111115e-05,
	"loss": 12.6434,
	"mean_token_accuracy": 0.7801247961819172,
	"num_tokens": 8757753.0,
	"step": 1320
	},
	{
	"entropy": 0.7391151077114045,
	"epoch": 0.5919991097757747,
	"grad_norm": 50.75,
	"learning_rate": 1.926888888888889e-05,
	"loss": 11.9013,
	"mean_token_accuracy": 0.7954815953969956,
	"num_tokens": 8823153.0,
	"step": 1330
	},
	{
	"entropy": 0.7482655656524002,
	"epoch": 0.5964502309019084,
	"grad_norm": 53.0,
	"learning_rate": 1.924666666666667e-05,
	"loss": 12.1372,
	"mean_token_accuracy": 0.7891814015805722,
	"num_tokens": 8886401.0,
	"step": 1340
	},
	{
	"entropy": 0.7140636395663023,
	"epoch": 0.6009013520280421,
	"grad_norm": 46.75,
	"learning_rate": 1.9224444444444444e-05,
	"loss": 11.4324,
	"mean_token_accuracy": 0.7953898053616285,
	"num_tokens": 8958530.0,
	"step": 1350
	},
	{
	"entropy": 0.7349427663721144,
	"epoch": 0.6053524731541757,
	"grad_norm": 55.0,
	"learning_rate": 1.9202222222222225e-05,
	"loss": 11.8847,
	"mean_token_accuracy": 0.7926598964259028,
	"num_tokens": 9024677.0,
	"step": 1360
	},
	{
	"entropy": 0.7534153172746301,
	"epoch": 0.6098035942803094,
	"grad_norm": 52.0,
	"learning_rate": 1.918e-05,
	"loss": 12.0276,
	"mean_token_accuracy": 0.7890890792012215,
	"num_tokens": 9090280.0,
	"step": 1370
	},
	{
	"entropy": 0.7084095226600766,
	"epoch": 0.614254715406443,
	"grad_norm": 52.75,
	"learning_rate": 1.915777777777778e-05,
	"loss": 11.5771,
	"mean_token_accuracy": 0.7964484915137291,
	"num_tokens": 9153771.0,
	"step": 1380
	},
	{
	"entropy": 0.6750353675335645,
	"epoch": 0.6187058365325766,
	"grad_norm": 48.25,
	"learning_rate": 1.9135555555555555e-05,
	"loss": 10.7449,
	"mean_token_accuracy": 0.8097869891673326,
	"num_tokens": 9227117.0,
	"step": 1390
	},
	{
	"entropy": 0.7293720198795199,
	"epoch": 0.6231569576587103,
	"grad_norm": 49.0,
	"learning_rate": 1.9113333333333336e-05,
	"loss": 11.9594,
	"mean_token_accuracy": 0.7891727603971959,
	"num_tokens": 9291760.0,
	"step": 1400
	},
	{
	"epoch": 0.6231569576587103,
	"eval_biology_entropy": 1.2729821098446845,
	"eval_biology_loss": 1.3309696912765503,
	"eval_biology_mean_token_accuracy": 0.6723712916374207,
	"eval_biology_num_tokens": 9291760.0,
	"eval_biology_runtime": 49.2684,
	"eval_biology_samples_per_second": 10.148,
	"eval_biology_steps_per_second": 10.148,
	"step": 1400
	},
	{
	"epoch": 0.6231569576587103,
	"eval_chemistry_entropy": 0.7357699049711227,
	"eval_chemistry_loss": 0.7299637794494629,
	"eval_chemistry_mean_token_accuracy": 0.793709151148796,
	"eval_chemistry_num_tokens": 9291760.0,
	"eval_chemistry_runtime": 57.0749,
	"eval_chemistry_samples_per_second": 8.76,
	"eval_chemistry_steps_per_second": 8.76,
	"step": 1400
	},
	{
	"epoch": 0.6231569576587103,
	"eval_math_entropy": 0.713969606757164,
	"eval_math_loss": 0.8790645599365234,
	"eval_math_mean_token_accuracy": 0.7724634169340133,
	"eval_math_num_tokens": 9291760.0,
	"eval_math_runtime": 58.9543,
	"eval_math_samples_per_second": 8.481,
	"eval_math_steps_per_second": 8.481,
	"step": 1400
	},
	{
	"epoch": 0.6231569576587103,
	"eval_physics_entropy": 0.738197058737278,
	"eval_physics_loss": 0.7536621689796448,
	"eval_physics_mean_token_accuracy": 0.7910983339548111,
	"eval_physics_num_tokens": 9291760.0,
	"eval_physics_runtime": 65.823,
	"eval_physics_samples_per_second": 7.596,
	"eval_physics_steps_per_second": 7.596,
	"step": 1400
	},
	{
	"entropy": 0.7626627525314689,
	"epoch": 0.6276080787848439,
	"grad_norm": 51.25,
	"learning_rate": 1.9091111111111113e-05,
	"loss": 12.1176,
	"mean_token_accuracy": 0.7862323332577944,
	"num_tokens": 9357561.0,
	"step": 1410
	},
	{
	"entropy": 0.7020683256909251,
	"epoch": 0.6320591999109776,
	"grad_norm": 55.0,
	"learning_rate": 1.906888888888889e-05,
	"loss": 11.5299,
	"mean_token_accuracy": 0.7983165990561247,
	"num_tokens": 9425022.0,
	"step": 1420
	},
	{
	"entropy": 0.7129457925446332,
	"epoch": 0.6365103210371112,
	"grad_norm": 55.5,
	"learning_rate": 1.904666666666667e-05,
	"loss": 11.363,
	"mean_token_accuracy": 0.8003849845379591,
	"num_tokens": 9492028.0,
	"step": 1430
	},
	{
	"entropy": 0.682979597337544,
	"epoch": 0.6409614421632449,
	"grad_norm": 53.5,
	"learning_rate": 1.9024444444444446e-05,
	"loss": 11.0672,
	"mean_token_accuracy": 0.8002727191895247,
	"num_tokens": 9561091.0,
	"step": 1440
	},
	{
	"entropy": 0.7097643257118762,
	"epoch": 0.6454125632893785,
	"grad_norm": 52.5,
	"learning_rate": 1.9002222222222224e-05,
	"loss": 11.4105,
	"mean_token_accuracy": 0.798115537315607,
	"num_tokens": 9628383.0,
	"step": 1450
	},
	{
	"entropy": 0.6520757641643286,
	"epoch": 0.6498636844155121,
	"grad_norm": 45.0,
	"learning_rate": 1.898e-05,
	"loss": 10.6294,
	"mean_token_accuracy": 0.8115652769804,
	"num_tokens": 9695723.0,
	"step": 1460
	},
	{
	"entropy": 0.6995091023854911,
	"epoch": 0.6543148055416458,
	"grad_norm": 48.0,
	"learning_rate": 1.895777777777778e-05,
	"loss": 11.2764,
	"mean_token_accuracy": 0.8028371874243021,
	"num_tokens": 9763393.0,
	"step": 1470
	},
	{
	"entropy": 0.736479183845222,
	"epoch": 0.6587659266677794,
	"grad_norm": 46.75,
	"learning_rate": 1.8935555555555556e-05,
	"loss": 11.7277,
	"mean_token_accuracy": 0.7955611757934093,
	"num_tokens": 9827272.0,
	"step": 1480
	},
	{
	"entropy": 0.6883244249969721,
	"epoch": 0.6632170477939131,
	"grad_norm": 48.75,
	"learning_rate": 1.8913333333333334e-05,
	"loss": 11.1978,
	"mean_token_accuracy": 0.8004064936190843,
	"num_tokens": 9892418.0,
	"step": 1490
	},
	{
	"entropy": 0.6904169335030019,
	"epoch": 0.6676681689200468,
	"grad_norm": 48.25,
	"learning_rate": 1.8891111111111115e-05,
	"loss": 11.1417,
	"mean_token_accuracy": 0.8022403538227081,
	"num_tokens": 9955369.0,
	"step": 1500
	},
	{
	"epoch": 0.6676681689200468,
	"eval_biology_entropy": 1.1994027597904204,
	"eval_biology_loss": 1.3322917222976685,
	"eval_biology_mean_token_accuracy": 0.6736352816224098,
	"eval_biology_num_tokens": 9955369.0,
	"eval_biology_runtime": 49.2971,
	"eval_biology_samples_per_second": 10.143,
	"eval_biology_steps_per_second": 10.143,
	"step": 1500
	},
	{
	"epoch": 0.6676681689200468,
	"eval_chemistry_entropy": 0.6897619203329086,
	"eval_chemistry_loss": 0.7175793051719666,
	"eval_chemistry_mean_token_accuracy": 0.7974075863361358,
	"eval_chemistry_num_tokens": 9955369.0,
	"eval_chemistry_runtime": 57.473,
	"eval_chemistry_samples_per_second": 8.7,
	"eval_chemistry_steps_per_second": 8.7,
	"step": 1500
	},
	{
	"epoch": 0.6676681689200468,
	"eval_math_entropy": 0.659498262912035,
	"eval_math_loss": 0.8881840705871582,
	"eval_math_mean_token_accuracy": 0.773587989807129,
	"eval_math_num_tokens": 9955369.0,
	"eval_math_runtime": 59.1515,
	"eval_math_samples_per_second": 8.453,
	"eval_math_steps_per_second": 8.453,
	"step": 1500
	},
	{
	"epoch": 0.6676681689200468,
	"eval_physics_entropy": 0.6934876502454281,
	"eval_physics_loss": 0.747431755065918,
	"eval_physics_mean_token_accuracy": 0.7924305193424225,
	"eval_physics_num_tokens": 9955369.0,
	"eval_physics_runtime": 66.2089,
	"eval_physics_samples_per_second": 7.552,
	"eval_physics_steps_per_second": 7.552,
	"step": 1500
	},
	{
	"entropy": 0.7013076378963887,
	"epoch": 0.6721192900461804,
	"grad_norm": 60.75,
	"learning_rate": 1.886888888888889e-05,
	"loss": 11.3597,
	"mean_token_accuracy": 0.7976187769323587,
	"num_tokens": 10026298.0,
	"step": 1510
	},
	{
	"entropy": 0.66996502391994,
	"epoch": 0.676570411172314,
	"grad_norm": 48.25,
	"learning_rate": 1.884666666666667e-05,
	"loss": 10.6869,
	"mean_token_accuracy": 0.809822041168809,
	"num_tokens": 10096065.0,
	"step": 1520
	},
	{
	"entropy": 0.6915224540047348,
	"epoch": 0.6810215322984476,
	"grad_norm": 52.25,
	"learning_rate": 1.8824444444444445e-05,
	"loss": 11.3488,
	"mean_token_accuracy": 0.8012049272656441,
	"num_tokens": 10160745.0,
	"step": 1530
	},
	{
	"entropy": 0.6686725497245789,
	"epoch": 0.6854726534245813,
	"grad_norm": 46.75,
	"learning_rate": 1.8802222222222226e-05,
	"loss": 10.6806,
	"mean_token_accuracy": 0.8083007030189038,
	"num_tokens": 10233806.0,
	"step": 1540
	},
	{
	"entropy": 0.6853563466109336,
	"epoch": 0.689923774550715,
	"grad_norm": 56.25,
	"learning_rate": 1.878e-05,
	"loss": 11.2147,
	"mean_token_accuracy": 0.802401140332222,
	"num_tokens": 10298845.0,
	"step": 1550
	},
	{
	"entropy": 0.6936365003697574,
	"epoch": 0.6943748956768486,
	"grad_norm": 49.25,
	"learning_rate": 1.875777777777778e-05,
	"loss": 11.0701,
	"mean_token_accuracy": 0.8034875515848399,
	"num_tokens": 10364601.0,
	"step": 1560
	},
	{
	"entropy": 0.6916884422302246,
	"epoch": 0.6988260168029823,
	"grad_norm": 53.0,
	"learning_rate": 1.873555555555556e-05,
	"loss": 11.2846,
	"mean_token_accuracy": 0.7983762890100479,
	"num_tokens": 10434433.0,
	"step": 1570
	},
	{
	"entropy": 0.7143267430365086,
	"epoch": 0.703277137929116,
	"grad_norm": 56.5,
	"learning_rate": 1.8713333333333336e-05,
	"loss": 11.4464,
	"mean_token_accuracy": 0.7968503847718239,
	"num_tokens": 10500891.0,
	"step": 1580
	},
	{
	"entropy": 0.7026416527107358,
	"epoch": 0.7077282590552495,
	"grad_norm": 48.5,
	"learning_rate": 1.8691111111111114e-05,
	"loss": 11.4511,
	"mean_token_accuracy": 0.8001922458410263,
	"num_tokens": 10567784.0,
	"step": 1590
	},
	{
	"entropy": 0.7117853639647365,
	"epoch": 0.7121793801813832,
	"grad_norm": 47.25,
	"learning_rate": 1.866888888888889e-05,
	"loss": 11.2339,
	"mean_token_accuracy": 0.8018446419388056,
	"num_tokens": 10633325.0,
	"step": 1600
	},
	{
	"epoch": 0.7121793801813832,
	"eval_biology_entropy": 1.2444143397212029,
	"eval_biology_loss": 1.3356362581253052,
	"eval_biology_mean_token_accuracy": 0.6730564608573913,
	"eval_biology_num_tokens": 10633325.0,
	"eval_biology_runtime": 72.0334,
	"eval_biology_samples_per_second": 6.941,
	"eval_biology_steps_per_second": 6.941,
	"step": 1600
	},
	{
	"epoch": 0.7121793801813832,
	"eval_chemistry_entropy": 0.7002458871603012,
	"eval_chemistry_loss": 0.7100893259048462,
	"eval_chemistry_mean_token_accuracy": 0.7978058751821518,
	"eval_chemistry_num_tokens": 10633325.0,
	"eval_chemistry_runtime": 80.5635,
	"eval_chemistry_samples_per_second": 6.206,
	"eval_chemistry_steps_per_second": 6.206,
	"step": 1600
	},
	{
	"epoch": 0.7121793801813832,
	"eval_math_entropy": 0.6884930176436901,
	"eval_math_loss": 0.8825717568397522,
	"eval_math_mean_token_accuracy": 0.7724320909976959,
	"eval_math_num_tokens": 10633325.0,
	"eval_math_runtime": 62.848,
	"eval_math_samples_per_second": 7.956,
	"eval_math_steps_per_second": 7.956,
	"step": 1600
	},
	{
	"epoch": 0.7121793801813832,
	"eval_physics_entropy": 0.703965005338192,
	"eval_physics_loss": 0.7453010082244873,
	"eval_physics_mean_token_accuracy": 0.7937779233455659,
	"eval_physics_num_tokens": 10633325.0,
	"eval_physics_runtime": 65.8361,
	"eval_physics_samples_per_second": 7.595,
	"eval_physics_steps_per_second": 7.595,
	"step": 1600
	},
	{
	"entropy": 0.653254197910428,
	"epoch": 0.7166305013075168,
	"grad_norm": 47.75,
	"learning_rate": 1.864666666666667e-05,
	"loss": 10.7312,
	"mean_token_accuracy": 0.8095974924042821,
	"num_tokens": 10699737.0,
	"step": 1610
	},
	{
	"entropy": 0.6189540217630565,
	"epoch": 0.7210816224336505,
	"grad_norm": 52.25,
	"learning_rate": 1.8624444444444446e-05,
	"loss": 9.9286,
	"mean_token_accuracy": 0.8196552351117135,
	"num_tokens": 10764319.0,
	"step": 1620
	},
	{
	"entropy": 0.6991719406098127,
	"epoch": 0.7255327435597841,
	"grad_norm": 48.75,
	"learning_rate": 1.8602222222222224e-05,
	"loss": 11.3232,
	"mean_token_accuracy": 0.7975562997162342,
	"num_tokens": 10831617.0,
	"step": 1630
	},
	{
	"entropy": 0.7137979302555323,
	"epoch": 0.7299838646859178,
	"grad_norm": 58.5,
	"learning_rate": 1.858e-05,
	"loss": 11.3908,
	"mean_token_accuracy": 0.8014465879648924,
	"num_tokens": 10897916.0,
	"step": 1640
	},
	{
	"entropy": 0.6850169265642763,
	"epoch": 0.7344349858120514,
	"grad_norm": 50.75,
	"learning_rate": 1.855777777777778e-05,
	"loss": 11.1551,
	"mean_token_accuracy": 0.8014529883861542,
	"num_tokens": 10965483.0,
	"step": 1650
	},
	{
	"entropy": 0.7198417543433606,
	"epoch": 0.738886106938185,
	"grad_norm": 47.0,
	"learning_rate": 1.8535555555555557e-05,
	"loss": 11.6312,
	"mean_token_accuracy": 0.7964452419430017,
	"num_tokens": 11030485.0,
	"step": 1660
	},
	{
	"entropy": 0.6864631313830614,
	"epoch": 0.7433372280643187,
	"grad_norm": 50.0,
	"learning_rate": 1.8513333333333335e-05,
	"loss": 11.0248,
	"mean_token_accuracy": 0.804034722596407,
	"num_tokens": 11097108.0,
	"step": 1670
	},
	{
	"entropy": 0.6339416711591184,
	"epoch": 0.7477883491904523,
	"grad_norm": 48.75,
	"learning_rate": 1.8491111111111112e-05,
	"loss": 10.4331,
	"mean_token_accuracy": 0.8120405212044716,
	"num_tokens": 11165356.0,
	"step": 1680
	},
	{
	"entropy": 0.7042925103567541,
	"epoch": 0.752239470316586,
	"grad_norm": 46.25,
	"learning_rate": 1.846888888888889e-05,
	"loss": 11.2078,
	"mean_token_accuracy": 0.8007234908640385,
	"num_tokens": 11234429.0,
	"step": 1690
	},
	{
	"entropy": 0.6936139498837293,
	"epoch": 0.7566905914427197,
	"grad_norm": 50.0,
	"learning_rate": 1.8446666666666667e-05,
	"loss": 11.1976,
	"mean_token_accuracy": 0.8022552561014891,
	"num_tokens": 11299223.0,
	"step": 1700
	},
	{
	"epoch": 0.7566905914427197,
	"eval_biology_entropy": 1.2853857824206352,
	"eval_biology_loss": 1.333956003189087,
	"eval_biology_mean_token_accuracy": 0.6733258009552956,
	"eval_biology_num_tokens": 11299223.0,
	"eval_biology_runtime": 49.3568,
	"eval_biology_samples_per_second": 10.13,
	"eval_biology_steps_per_second": 10.13,
	"step": 1700
	},
	{
	"epoch": 0.7566905914427197,
	"eval_chemistry_entropy": 0.7026637124717235,
	"eval_chemistry_loss": 0.7078750133514404,
	"eval_chemistry_mean_token_accuracy": 0.7986145396232605,
	"eval_chemistry_num_tokens": 11299223.0,
	"eval_chemistry_runtime": 57.1452,
	"eval_chemistry_samples_per_second": 8.75,
	"eval_chemistry_steps_per_second": 8.75,
	"step": 1700
	},
	{
	"epoch": 0.7566905914427197,
	"eval_math_entropy": 0.7062955890595913,
	"eval_math_loss": 0.8821802735328674,
	"eval_math_mean_token_accuracy": 0.7737128890752792,
	"eval_math_num_tokens": 11299223.0,
	"eval_math_runtime": 61.1933,
	"eval_math_samples_per_second": 8.171,
	"eval_math_steps_per_second": 8.171,
	"step": 1700
	},
	{
	"epoch": 0.7566905914427197,
	"eval_physics_entropy": 0.7158967233300209,
	"eval_physics_loss": 0.7414941191673279,
	"eval_physics_mean_token_accuracy": 0.7944492185711861,
	"eval_physics_num_tokens": 11299223.0,
	"eval_physics_runtime": 66.3539,
	"eval_physics_samples_per_second": 7.535,
	"eval_physics_steps_per_second": 7.535,
	"step": 1700
	},
	{
	"entropy": 0.6807508809491992,
	"epoch": 0.7611417125688533,
	"grad_norm": 44.0,
	"learning_rate": 1.842444444444445e-05,
	"loss": 10.8976,
	"mean_token_accuracy": 0.8057308234274387,
	"num_tokens": 11371200.0,
	"step": 1710
	},
	{
	"entropy": 0.683816609159112,
	"epoch": 0.7655928336949869,
	"grad_norm": 48.75,
	"learning_rate": 1.8402222222222223e-05,
	"loss": 11.2848,
	"mean_token_accuracy": 0.7998971730470658,
	"num_tokens": 11436799.0,
	"step": 1720
	},
	{
	"entropy": 0.682188394293189,
	"epoch": 0.7700439548211205,
	"grad_norm": 57.25,
	"learning_rate": 1.8380000000000004e-05,
	"loss": 10.8721,
	"mean_token_accuracy": 0.8079916540533304,
	"num_tokens": 11502965.0,
	"step": 1730
	},
	{
	"entropy": 0.7102242357097566,
	"epoch": 0.7744950759472542,
	"grad_norm": 46.25,
	"learning_rate": 1.8357777777777778e-05,
	"loss": 11.5854,
	"mean_token_accuracy": 0.7963233698159456,
	"num_tokens": 11567422.0,
	"step": 1740
	},
	{
	"entropy": 0.6699508598074317,
	"epoch": 0.7789461970733879,
	"grad_norm": 38.75,
	"learning_rate": 1.833555555555556e-05,
	"loss": 10.8756,
	"mean_token_accuracy": 0.8071661192923785,
	"num_tokens": 11637076.0,
	"step": 1750
	},
	{
	"entropy": 0.7298749316483736,
	"epoch": 0.7833973181995215,
	"grad_norm": 48.25,
	"learning_rate": 1.8313333333333333e-05,
	"loss": 11.7861,
	"mean_token_accuracy": 0.7899679750204086,
	"num_tokens": 11703496.0,
	"step": 1760
	},
	{
	"entropy": 0.6433588310144842,
	"epoch": 0.7878484393256552,
	"grad_norm": 36.25,
	"learning_rate": 1.8291111111111114e-05,
	"loss": 10.2179,
	"mean_token_accuracy": 0.8177038058638573,
	"num_tokens": 11772773.0,
	"step": 1770
	},
	{
	"entropy": 0.6560773782432079,
	"epoch": 0.7922995604517888,
	"grad_norm": 47.0,
	"learning_rate": 1.8268888888888888e-05,
	"loss": 10.7873,
	"mean_token_accuracy": 0.8082022242248058,
	"num_tokens": 11840317.0,
	"step": 1780
	},
	{
	"entropy": 0.7407286252826453,
	"epoch": 0.7967506815779224,
	"grad_norm": 55.0,
	"learning_rate": 1.824666666666667e-05,
	"loss": 11.9908,
	"mean_token_accuracy": 0.793482593819499,
	"num_tokens": 11903376.0,
	"step": 1790
	},
	{
	"entropy": 0.6884403178468347,
	"epoch": 0.801201802704056,
	"grad_norm": 52.5,
	"learning_rate": 1.8224444444444447e-05,
	"loss": 10.9702,
	"mean_token_accuracy": 0.8052472107112407,
	"num_tokens": 11965530.0,
	"step": 1800
	},
	{
	"epoch": 0.801201802704056,
	"eval_biology_entropy": 1.2536135312318801,
	"eval_biology_loss": 1.3246384859085083,
	"eval_biology_mean_token_accuracy": 0.6739821583032608,
	"eval_biology_num_tokens": 11965530.0,
	"eval_biology_runtime": 49.5291,
	"eval_biology_samples_per_second": 10.095,
	"eval_biology_steps_per_second": 10.095,
	"step": 1800
	},
	{
	"epoch": 0.801201802704056,
	"eval_chemistry_entropy": 0.6877463189661502,
	"eval_chemistry_loss": 0.699435293674469,
	"eval_chemistry_mean_token_accuracy": 0.8004006836414337,
	"eval_chemistry_num_tokens": 11965530.0,
	"eval_chemistry_runtime": 57.2961,
	"eval_chemistry_samples_per_second": 8.727,
	"eval_chemistry_steps_per_second": 8.727,
	"step": 1800
	},
	{
	"epoch": 0.801201802704056,
	"eval_math_entropy": 0.6771485269367695,
	"eval_math_loss": 0.8832710385322571,
	"eval_math_mean_token_accuracy": 0.7728882786035538,
	"eval_math_num_tokens": 11965530.0,
	"eval_math_runtime": 59.0619,
	"eval_math_samples_per_second": 8.466,
	"eval_math_steps_per_second": 8.466,
	"step": 1800
	},
	{
	"epoch": 0.801201802704056,
	"eval_physics_entropy": 0.6993200024366378,
	"eval_physics_loss": 0.7383862733840942,
	"eval_physics_mean_token_accuracy": 0.794426214993,
	"eval_physics_num_tokens": 11965530.0,
	"eval_physics_runtime": 66.0,
	"eval_physics_samples_per_second": 7.576,
	"eval_physics_steps_per_second": 7.576,
	"step": 1800
	},
	{
	"entropy": 0.6854008860886097,
	"epoch": 0.8056529238301897,
	"grad_norm": 54.0,
	"learning_rate": 1.8202222222222225e-05,
	"loss": 11.1961,
	"mean_token_accuracy": 0.80209940392524,
	"num_tokens": 12031493.0,
	"step": 1810
	},
	{
	"entropy": 0.6895728969946504,
	"epoch": 0.8101040449563234,
	"grad_norm": 46.25,
	"learning_rate": 1.8180000000000002e-05,
	"loss": 11.1827,
	"mean_token_accuracy": 0.8035332810133695,
	"num_tokens": 12097917.0,
	"step": 1820
	},
	{
	"entropy": 0.6885084038600325,
	"epoch": 0.814555166082457,
	"grad_norm": 46.5,
	"learning_rate": 1.815777777777778e-05,
	"loss": 11.0952,
	"mean_token_accuracy": 0.8038906361907721,
	"num_tokens": 12163029.0,
	"step": 1830
	},
	{
	"entropy": 0.6959913271479309,
	"epoch": 0.8190062872085907,
	"grad_norm": 58.5,
	"learning_rate": 1.8135555555555557e-05,
	"loss": 11.3646,
	"mean_token_accuracy": 0.7995086185634136,
	"num_tokens": 12224427.0,
	"step": 1840
	},
	{
	"entropy": 0.7170273121446371,
	"epoch": 0.8234574083347244,
	"grad_norm": 43.0,
	"learning_rate": 1.8113333333333335e-05,
	"loss": 11.4108,
	"mean_token_accuracy": 0.80226712487638,
	"num_tokens": 12291917.0,
	"step": 1850
	},
	{
	"entropy": 0.6658903407864273,
	"epoch": 0.8279085294608579,
	"grad_norm": 46.25,
	"learning_rate": 1.8091111111111113e-05,
	"loss": 10.7704,
	"mean_token_accuracy": 0.8066752411425113,
	"num_tokens": 12365332.0,
	"step": 1860
	},
	{
	"entropy": 0.6656836275942624,
	"epoch": 0.8323596505869916,
	"grad_norm": 40.25,
	"learning_rate": 1.806888888888889e-05,
	"loss": 10.6917,
	"mean_token_accuracy": 0.8068058449774981,
	"num_tokens": 12441423.0,
	"step": 1870
	},
	{
	"entropy": 0.6506184342317283,
	"epoch": 0.8368107717131252,
	"grad_norm": 50.25,
	"learning_rate": 1.8046666666666668e-05,
	"loss": 10.4962,
	"mean_token_accuracy": 0.8128324717283248,
	"num_tokens": 12509124.0,
	"step": 1880
	},
	{
	"entropy": 0.626772437337786,
	"epoch": 0.8412618928392589,
	"grad_norm": 43.25,
	"learning_rate": 1.8024444444444445e-05,
	"loss": 10.2576,
	"mean_token_accuracy": 0.8113849848508835,
	"num_tokens": 12575355.0,
	"step": 1890
	},
	{
	"entropy": 0.6942961663939059,
	"epoch": 0.8457130139653926,
	"grad_norm": 57.25,
	"learning_rate": 1.8002222222222223e-05,
	"loss": 11.3533,
	"mean_token_accuracy": 0.7994141079485416,
	"num_tokens": 12642651.0,
	"step": 1900
	},
	{
	"epoch": 0.8457130139653926,
	"eval_biology_entropy": 1.2530685477256776,
	"eval_biology_loss": 1.3307456970214844,
	"eval_biology_mean_token_accuracy": 0.6736879838109017,
	"eval_biology_num_tokens": 12642651.0,
	"eval_biology_runtime": 49.3602,
	"eval_biology_samples_per_second": 10.13,
	"eval_biology_steps_per_second": 10.13,
	"step": 1900
	},
	{
	"epoch": 0.8457130139653926,
	"eval_chemistry_entropy": 0.6917615284323693,
	"eval_chemistry_loss": 0.6964050531387329,
	"eval_chemistry_mean_token_accuracy": 0.8021229248046875,
	"eval_chemistry_num_tokens": 12642651.0,
	"eval_chemistry_runtime": 57.0518,
	"eval_chemistry_samples_per_second": 8.764,
	"eval_chemistry_steps_per_second": 8.764,
	"step": 1900
	},
	{
	"epoch": 0.8457130139653926,
	"eval_math_entropy": 0.675072967261076,
	"eval_math_loss": 0.8877173066139221,
	"eval_math_mean_token_accuracy": 0.7728915640115738,
	"eval_math_num_tokens": 12642651.0,
	"eval_math_runtime": 58.9152,
	"eval_math_samples_per_second": 8.487,
	"eval_math_steps_per_second": 8.487,
	"step": 1900
	},
	{
	"epoch": 0.8457130139653926,
	"eval_physics_entropy": 0.7041290953755379,
	"eval_physics_loss": 0.7368381023406982,
	"eval_physics_mean_token_accuracy": 0.7941438639163971,
	"eval_physics_num_tokens": 12642651.0,
	"eval_physics_runtime": 65.8671,
	"eval_physics_samples_per_second": 7.591,
	"eval_physics_steps_per_second": 7.591,
	"step": 1900
	},
	{
	"entropy": 0.651003266312182,
	"epoch": 0.8501641350915262,
	"grad_norm": 42.75,
	"learning_rate": 1.798e-05,
	"loss": 10.3017,
	"mean_token_accuracy": 0.8126345485448837,
	"num_tokens": 12710065.0,
	"step": 1910
	},
	{
	"entropy": 0.6822556433267891,
	"epoch": 0.8546152562176598,
	"grad_norm": 49.5,
	"learning_rate": 1.7957777777777778e-05,
	"loss": 11.0203,
	"mean_token_accuracy": 0.8035612374544143,
	"num_tokens": 12778408.0,
	"step": 1920
	},
	{
	"entropy": 0.6388307875953615,
	"epoch": 0.8590663773437934,
	"grad_norm": 54.25,
	"learning_rate": 1.7935555555555556e-05,
	"loss": 10.413,
	"mean_token_accuracy": 0.8124310929328203,
	"num_tokens": 12845949.0,
	"step": 1930
	},
	{
	"entropy": 0.6870338559150696,
	"epoch": 0.8635174984699271,
	"grad_norm": 52.75,
	"learning_rate": 1.7913333333333337e-05,
	"loss": 10.9443,
	"mean_token_accuracy": 0.8076328281313181,
	"num_tokens": 12910630.0,
	"step": 1940
	},
	{
	"entropy": 0.6719606230035424,
	"epoch": 0.8679686195960608,
	"grad_norm": 46.5,
	"learning_rate": 1.789111111111111e-05,
	"loss": 11.1521,
	"mean_token_accuracy": 0.8018901061266661,
	"num_tokens": 12975379.0,
	"step": 1950
	},
	{
	"entropy": 0.6635585248470306,
	"epoch": 0.8724197407221944,
	"grad_norm": 40.75,
	"learning_rate": 1.7868888888888892e-05,
	"loss": 10.5294,
	"mean_token_accuracy": 0.8090925574302673,
	"num_tokens": 13046473.0,
	"step": 1960
	},
	{
	"entropy": 0.7126090485602617,
	"epoch": 0.8768708618483281,
	"grad_norm": 44.5,
	"learning_rate": 1.7846666666666666e-05,
	"loss": 11.7059,
	"mean_token_accuracy": 0.794832269102335,
	"num_tokens": 13105019.0,
	"step": 1970
	},
	{
	"entropy": 0.7243582944385707,
	"epoch": 0.8813219829744617,
	"grad_norm": 47.75,
	"learning_rate": 1.7824444444444447e-05,
	"loss": 11.4291,
	"mean_token_accuracy": 0.8015620414167642,
	"num_tokens": 13167768.0,
	"step": 1980
	},
	{
	"entropy": 0.6495856091380119,
	"epoch": 0.8857731041005953,
	"grad_norm": 45.25,
	"learning_rate": 1.780222222222222e-05,
	"loss": 10.6768,
	"mean_token_accuracy": 0.8102217994630336,
	"num_tokens": 13235899.0,
	"step": 1990
	},
	{
	"entropy": 0.6717848453670741,
	"epoch": 0.890224225226729,
	"grad_norm": 41.25,
	"learning_rate": 1.7780000000000003e-05,
	"loss": 10.7358,
	"mean_token_accuracy": 0.8080738458782435,
	"num_tokens": 13301659.0,
	"step": 2000
	},
	{
	"epoch": 0.890224225226729,
	"eval_biology_entropy": 1.2462715034484864,
	"eval_biology_loss": 1.323424220085144,
	"eval_biology_mean_token_accuracy": 0.6743068833947181,
	"eval_biology_num_tokens": 13301659.0,
	"eval_biology_runtime": 49.4062,
	"eval_biology_samples_per_second": 10.12,
	"eval_biology_steps_per_second": 10.12,
	"step": 2000
	},
	{
	"epoch": 0.890224225226729,
	"eval_chemistry_entropy": 0.6809069735705853,
	"eval_chemistry_loss": 0.6882286667823792,
	"eval_chemistry_mean_token_accuracy": 0.8031812709569931,
	"eval_chemistry_num_tokens": 13301659.0,
	"eval_chemistry_runtime": 58.7929,
	"eval_chemistry_samples_per_second": 8.504,
	"eval_chemistry_steps_per_second": 8.504,
	"step": 2000
	},
	{
	"epoch": 0.890224225226729,
	"eval_math_entropy": 0.6781798366010189,
	"eval_math_loss": 0.8778423070907593,
	"eval_math_mean_token_accuracy": 0.7743719705343246,
	"eval_math_num_tokens": 13301659.0,
	"eval_math_runtime": 58.9039,
	"eval_math_samples_per_second": 8.488,
	"eval_math_steps_per_second": 8.488,
	"step": 2000
	},
	{
	"epoch": 0.890224225226729,
	"eval_physics_entropy": 0.6972870816588401,
	"eval_physics_loss": 0.7357947826385498,
	"eval_physics_mean_token_accuracy": 0.7952804889678955,
	"eval_physics_num_tokens": 13301659.0,
	"eval_physics_runtime": 65.8896,
	"eval_physics_samples_per_second": 7.588,
	"eval_physics_steps_per_second": 7.588,
	"step": 2000
	}
	],
	"logging_steps": 10,
	"max_steps": 10000,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 5,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": false
	},
	"attributes": {}
	}
	},
	"total_flos": 8.919343158882048e+17,
	"train_batch_size": 1,
	"trial_name": null,
	"trial_params": null
	}