Instructions to use roonbug/afsck72p with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use roonbug/afsck72p with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="roonbug/afsck72p")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("roonbug/afsck72p")
model = AutoModelForImageTextToText.from_pretrained("roonbug/afsck72p")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use roonbug/afsck72p with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "roonbug/afsck72p"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/afsck72p",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/roonbug/afsck72p

SGLang

How to use roonbug/afsck72p with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "roonbug/afsck72p" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/afsck72p",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "roonbug/afsck72p" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/afsck72p",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use roonbug/afsck72p with Docker Model Runner:
```
docker model run hf.co/roonbug/afsck72p
```

roonbug commited on May 3

Commit

b37645e

verified ·

1 Parent(s): ab30cdd

Upload folder using huggingface_hub

Browse files

Files changed (20) hide show

.gitattributes +1 -0
checkpoint-1000/added_tokens.json +3 -0
checkpoint-1000/chat_template.jinja +47 -0
checkpoint-1000/config.json +110 -0
checkpoint-1000/generation_config.json +13 -0
checkpoint-1000/model-00001-of-00005.safetensors +3 -0
checkpoint-1000/model-00002-of-00005.safetensors +3 -0
checkpoint-1000/model-00003-of-00005.safetensors +3 -0
checkpoint-1000/model-00004-of-00005.safetensors +3 -0
checkpoint-1000/model-00005-of-00005.safetensors +3 -0
checkpoint-1000/model.safetensors.index.json +0 -0
checkpoint-1000/optimizer.pt +3 -0
checkpoint-1000/rng_state.pth +3 -0
checkpoint-1000/scheduler.pt +3 -0
checkpoint-1000/special_tokens_map.json +33 -0
checkpoint-1000/tokenizer.json +3 -0
checkpoint-1000/tokenizer.model +3 -0
checkpoint-1000/tokenizer_config.json +0 -0
checkpoint-1000/trainer_state.json +1254 -0
checkpoint-1000/training_args.bin +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-1000/added_tokens.json ADDED Viewed

	@@ -0,0 +1,3 @@

+{
+  "<image_soft_token>": 262144
+}

checkpoint-1000/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,47 @@

+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

checkpoint-1000/config.json ADDED Viewed

	@@ -0,0 +1,110 @@

+{
+  "architectures": [
+    "Gemma3ForConditionalGeneration"
+  ],
+  "boi_token_index": 255999,
+  "bos_token_id": 2,
+  "dtype": "bfloat16",
+  "eoi_token_index": 256000,
+  "eos_token_id": 1,
+  "image_token_index": 262144,
+  "initializer_range": 0.02,
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "pad_token_id": 0,
+  "text_config": {
+    "_sliding_window_pattern": 6,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "dtype": "bfloat16",
+    "final_logit_softcapping": null,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 3840,
+    "initializer_range": 0.02,
+    "intermediate_size": 15360,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 48,
+    "num_key_value_heads": 8,
+    "query_pre_attn_scalar": 256,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000.0,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": 1024,
+    "use_cache": true,
+    "vocab_size": 262208
+  },
+  "transformers_version": "4.56.1",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "vision_use_head": false
+  }
+}

checkpoint-1000/generation_config.json ADDED Viewed

	@@ -0,0 +1,13 @@

+{
+  "bos_token_id": 2,
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.56.1"
+}

checkpoint-1000/model-00001-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4847447e92599833e8dbaa3067cd201c3bb5c052efa91f11ba891e43234f7832
+size 4979902192

checkpoint-1000/model-00002-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:891bd54eed03cba9ee1e705533a02a8217fcc29f356e4a1f53e5fd0d178883ad
+size 4931296592

checkpoint-1000/model-00003-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f41386b12a2ddadd21196b653d964e5b9b534ae001260a687adff1ae6edce7f7
+size 4931296656

checkpoint-1000/model-00004-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:05e981269342c9b437ca55612c3377b9e4fe1a9b4d06f64f83335ce32366e90f
+size 4931296656

checkpoint-1000/model-00005-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:da1f19190ae4301dc7f272b5a1c2c14aac1937b6bff1dac09868955d0478002a
+size 4601000928

checkpoint-1000/model.safetensors.index.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-1000/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3421d0fdd1873d87c80c61950a5135269c3c53594564f538f9047eb98f3dc4f5
+size 14573116445

checkpoint-1000/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:acd74ad34e3c5060506b53a8a3fafbfb35d44065ee496b2abf3a0b9a3b93ba09
+size 14645

checkpoint-1000/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6078e52773a379f026b8fbbbc5546ab7ed9418d05b713f8aff2a4f3c7e12f108
+size 1465

checkpoint-1000/special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,33 @@

+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1000/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

checkpoint-1000/tokenizer.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

checkpoint-1000/tokenizer_config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-1000/trainer_state.json ADDED Viewed

	@@ -0,0 +1,1254 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.4,
+  "eval_steps": 100,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 0.3589675037190318,
+      "epoch": 0.004,
+      "grad_norm": 221.0,
+      "learning_rate": 1.8e-07,
+      "loss": 14.235,
+      "mean_token_accuracy": 0.8023731399327516,
+      "num_tokens": 89176.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.381451623653993,
+      "epoch": 0.008,
+      "grad_norm": 251.0,
+      "learning_rate": 3.8e-07,
+      "loss": 16.438,
+      "mean_token_accuracy": 0.7818928115069866,
+      "num_tokens": 173886.0,
+      "step": 20
+    },
+    {
+      "entropy": 0.36998924349900336,
+      "epoch": 0.012,
+      "grad_norm": 284.0,
+      "learning_rate": 5.800000000000001e-07,
+      "loss": 15.3069,
+      "mean_token_accuracy": 0.7961655277758837,
+      "num_tokens": 258970.0,
+      "step": 30
+    },
+    {
+      "entropy": 0.3889644297771156,
+      "epoch": 0.016,
+      "grad_norm": 296.0,
+      "learning_rate": 7.8e-07,
+      "loss": 15.6587,
+      "mean_token_accuracy": 0.7874974697828293,
+      "num_tokens": 338352.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.3830348197836429,
+      "epoch": 0.02,
+      "grad_norm": 330.0,
+      "learning_rate": 9.800000000000001e-07,
+      "loss": 15.0754,
+      "mean_token_accuracy": 0.7973605334758759,
+      "num_tokens": 424243.0,
+      "step": 50
+    },
+    {
+      "entropy": 0.40132593517191706,
+      "epoch": 0.024,
+      "grad_norm": 284.0,
+      "learning_rate": 1.1800000000000001e-06,
+      "loss": 15.9374,
+      "mean_token_accuracy": 0.7853362146764994,
+      "num_tokens": 505613.0,
+      "step": 60
+    },
+    {
+      "entropy": 0.41173397554084656,
+      "epoch": 0.028,
+      "grad_norm": 201.0,
+      "learning_rate": 1.3800000000000001e-06,
+      "loss": 15.1473,
+      "mean_token_accuracy": 0.7896888509392739,
+      "num_tokens": 584157.0,
+      "step": 70
+    },
+    {
+      "entropy": 0.4223371436353773,
+      "epoch": 0.032,
+      "grad_norm": 164.0,
+      "learning_rate": 1.5800000000000001e-06,
+      "loss": 13.7069,
+      "mean_token_accuracy": 0.8018626874312759,
+      "num_tokens": 671193.0,
+      "step": 80
+    },
+    {
+      "entropy": 0.4982604907825589,
+      "epoch": 0.036,
+      "grad_norm": 240.0,
+      "learning_rate": 1.7800000000000001e-06,
+      "loss": 14.6572,
+      "mean_token_accuracy": 0.7824514407664538,
+      "num_tokens": 753947.0,
+      "step": 90
+    },
+    {
+      "entropy": 0.4913600580766797,
+      "epoch": 0.04,
+      "grad_norm": 155.0,
+      "learning_rate": 1.98e-06,
+      "loss": 13.0149,
+      "mean_token_accuracy": 0.8003234412521124,
+      "num_tokens": 835868.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.04,
+      "eval_chemistry_entropy": 0.41473443768918516,
+      "eval_chemistry_loss": 0.7713431119918823,
+      "eval_chemistry_mean_token_accuracy": 0.8103395719528198,
+      "eval_chemistry_num_tokens": 835868.0,
+      "eval_chemistry_runtime": 53.7203,
+      "eval_chemistry_samples_per_second": 9.307,
+      "eval_chemistry_steps_per_second": 9.307,
+      "step": 100
+    },
+    {
+      "epoch": 0.04,
+      "eval_physics_entropy": 0.5291771245449781,
+      "eval_physics_loss": 0.8209348917007446,
+      "eval_physics_mean_token_accuracy": 0.7944456633925437,
+      "eval_physics_num_tokens": 835868.0,
+      "eval_physics_runtime": 62.4292,
+      "eval_physics_samples_per_second": 8.009,
+      "eval_physics_steps_per_second": 8.009,
+      "step": 100
+    },
+    {
+      "entropy": 0.5052730706520379,
+      "epoch": 0.044,
+      "grad_norm": 132.0,
+      "learning_rate": 2.1800000000000003e-06,
+      "loss": 11.9999,
+      "mean_token_accuracy": 0.807476469874382,
+      "num_tokens": 917534.0,
+      "step": 110
+    },
+    {
+      "entropy": 0.5510593980550766,
+      "epoch": 0.048,
+      "grad_norm": 100.5,
+      "learning_rate": 2.38e-06,
+      "loss": 11.8438,
+      "mean_token_accuracy": 0.8063667319715023,
+      "num_tokens": 1001008.0,
+      "step": 120
+    },
+    {
+      "entropy": 0.59730707956478,
+      "epoch": 0.052,
+      "grad_norm": 87.5,
+      "learning_rate": 2.5800000000000003e-06,
+      "loss": 11.8941,
+      "mean_token_accuracy": 0.8023553561419249,
+      "num_tokens": 1085098.0,
+      "step": 130
+    },
+    {
+      "entropy": 0.6581894762814045,
+      "epoch": 0.056,
+      "grad_norm": 102.0,
+      "learning_rate": 2.7800000000000005e-06,
+      "loss": 12.6245,
+      "mean_token_accuracy": 0.7945734079927206,
+      "num_tokens": 1166615.0,
+      "step": 140
+    },
+    {
+      "entropy": 0.6519844191148877,
+      "epoch": 0.06,
+      "grad_norm": 70.5,
+      "learning_rate": 2.9800000000000003e-06,
+      "loss": 11.5194,
+      "mean_token_accuracy": 0.8026637583971024,
+      "num_tokens": 1248387.0,
+      "step": 150
+    },
+    {
+      "entropy": 0.7085968372412026,
+      "epoch": 0.064,
+      "grad_norm": 72.0,
+      "learning_rate": 3.1800000000000005e-06,
+      "loss": 12.4073,
+      "mean_token_accuracy": 0.7906260795891284,
+      "num_tokens": 1331208.0,
+      "step": 160
+    },
+    {
+      "entropy": 0.6624301395379006,
+      "epoch": 0.068,
+      "grad_norm": 66.5,
+      "learning_rate": 3.3800000000000007e-06,
+      "loss": 11.3744,
+      "mean_token_accuracy": 0.8067036394029856,
+      "num_tokens": 1416841.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.664617495983839,
+      "epoch": 0.072,
+      "grad_norm": 70.5,
+      "learning_rate": 3.58e-06,
+      "loss": 11.3264,
+      "mean_token_accuracy": 0.8068108163774014,
+      "num_tokens": 1499449.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.6721668383106589,
+      "epoch": 0.076,
+      "grad_norm": 70.5,
+      "learning_rate": 3.7800000000000002e-06,
+      "loss": 11.2874,
+      "mean_token_accuracy": 0.8029039859771728,
+      "num_tokens": 1585982.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.6800463449209928,
+      "epoch": 0.08,
+      "grad_norm": 58.5,
+      "learning_rate": 3.980000000000001e-06,
+      "loss": 11.4959,
+      "mean_token_accuracy": 0.8001787267625332,
+      "num_tokens": 1667532.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.08,
+      "eval_chemistry_entropy": 0.579608644425869,
+      "eval_chemistry_loss": 0.6572756171226501,
+      "eval_chemistry_mean_token_accuracy": 0.8160659775733948,
+      "eval_chemistry_num_tokens": 1667532.0,
+      "eval_chemistry_runtime": 53.9736,
+      "eval_chemistry_samples_per_second": 9.264,
+      "eval_chemistry_steps_per_second": 9.264,
+      "step": 200
+    },
+    {
+      "epoch": 0.08,
+      "eval_physics_entropy": 0.6811966061294079,
+      "eval_physics_loss": 0.714818000793457,
+      "eval_physics_mean_token_accuracy": 0.8037424722313881,
+      "eval_physics_num_tokens": 1667532.0,
+      "eval_physics_runtime": 62.6745,
+      "eval_physics_samples_per_second": 7.978,
+      "eval_physics_steps_per_second": 7.978,
+      "step": 200
+    },
+    {
+      "entropy": 0.6441303874365986,
+      "epoch": 0.084,
+      "grad_norm": 65.5,
+      "learning_rate": 4.18e-06,
+      "loss": 10.9679,
+      "mean_token_accuracy": 0.8105842903256416,
+      "num_tokens": 1750813.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.7001293174922466,
+      "epoch": 0.088,
+      "grad_norm": 63.5,
+      "learning_rate": 4.38e-06,
+      "loss": 11.3739,
+      "mean_token_accuracy": 0.8039855100214481,
+      "num_tokens": 1835489.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.7212686208076775,
+      "epoch": 0.092,
+      "grad_norm": 68.5,
+      "learning_rate": 4.58e-06,
+      "loss": 11.9056,
+      "mean_token_accuracy": 0.7951413784176111,
+      "num_tokens": 1915115.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.6619318895973265,
+      "epoch": 0.096,
+      "grad_norm": 66.5,
+      "learning_rate": 4.78e-06,
+      "loss": 10.9428,
+      "mean_token_accuracy": 0.8108693141490221,
+      "num_tokens": 2007176.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.6880421325564384,
+      "epoch": 0.1,
+      "grad_norm": 55.5,
+      "learning_rate": 4.980000000000001e-06,
+      "loss": 11.3231,
+      "mean_token_accuracy": 0.8026650555431842,
+      "num_tokens": 2088696.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.7066576741635799,
+      "epoch": 0.104,
+      "grad_norm": 58.25,
+      "learning_rate": 5.18e-06,
+      "loss": 11.6232,
+      "mean_token_accuracy": 0.799198641628027,
+      "num_tokens": 2171335.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.6949395652860403,
+      "epoch": 0.108,
+      "grad_norm": 69.5,
+      "learning_rate": 5.380000000000001e-06,
+      "loss": 11.2549,
+      "mean_token_accuracy": 0.8055713165551424,
+      "num_tokens": 2254168.0,
+      "step": 270
+    },
+    {
+      "entropy": 0.666729858610779,
+      "epoch": 0.112,
+      "grad_norm": 67.0,
+      "learning_rate": 5.580000000000001e-06,
+      "loss": 10.8193,
+      "mean_token_accuracy": 0.8086711473762989,
+      "num_tokens": 2342259.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.6924487132579088,
+      "epoch": 0.116,
+      "grad_norm": 60.5,
+      "learning_rate": 5.78e-06,
+      "loss": 11.1656,
+      "mean_token_accuracy": 0.8066085658967495,
+      "num_tokens": 2426289.0,
+      "step": 290
+    },
+    {
+      "entropy": 0.7209857527166605,
+      "epoch": 0.12,
+      "grad_norm": 53.75,
+      "learning_rate": 5.98e-06,
+      "loss": 11.7716,
+      "mean_token_accuracy": 0.7946148321032525,
+      "num_tokens": 2504279.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.12,
+      "eval_chemistry_entropy": 0.6136138401329517,
+      "eval_chemistry_loss": 0.6401469707489014,
+      "eval_chemistry_mean_token_accuracy": 0.8167599263191223,
+      "eval_chemistry_num_tokens": 2504279.0,
+      "eval_chemistry_runtime": 53.6939,
+      "eval_chemistry_samples_per_second": 9.312,
+      "eval_chemistry_steps_per_second": 9.312,
+      "step": 300
+    },
+    {
+      "epoch": 0.12,
+      "eval_physics_entropy": 0.678692765057087,
+      "eval_physics_loss": 0.6855186820030212,
+      "eval_physics_mean_token_accuracy": 0.8083885474801064,
+      "eval_physics_num_tokens": 2504279.0,
+      "eval_physics_runtime": 62.4413,
+      "eval_physics_samples_per_second": 8.008,
+      "eval_physics_steps_per_second": 8.008,
+      "step": 300
+    },
+    {
+      "entropy": 0.7016886103898287,
+      "epoch": 0.124,
+      "grad_norm": 51.0,
+      "learning_rate": 6.18e-06,
+      "loss": 11.5109,
+      "mean_token_accuracy": 0.79740143828094,
+      "num_tokens": 2583599.0,
+      "step": 310
+    },
+    {
+      "entropy": 0.6659886735491455,
+      "epoch": 0.128,
+      "grad_norm": 50.5,
+      "learning_rate": 6.380000000000001e-06,
+      "loss": 10.6687,
+      "mean_token_accuracy": 0.8114107441157102,
+      "num_tokens": 2663987.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.6984131418168544,
+      "epoch": 0.132,
+      "grad_norm": 55.5,
+      "learning_rate": 6.5800000000000005e-06,
+      "loss": 11.5084,
+      "mean_token_accuracy": 0.8018826052546502,
+      "num_tokens": 2743933.0,
+      "step": 330
+    },
+    {
+      "entropy": 0.685006461199373,
+      "epoch": 0.136,
+      "grad_norm": 43.25,
+      "learning_rate": 6.780000000000001e-06,
+      "loss": 11.0885,
+      "mean_token_accuracy": 0.8083073671907186,
+      "num_tokens": 2829720.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.6919135926291347,
+      "epoch": 0.14,
+      "grad_norm": 44.25,
+      "learning_rate": 6.98e-06,
+      "loss": 11.1629,
+      "mean_token_accuracy": 0.8035723548382521,
+      "num_tokens": 2915814.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.6693393810652196,
+      "epoch": 0.144,
+      "grad_norm": 68.5,
+      "learning_rate": 7.180000000000001e-06,
+      "loss": 10.8935,
+      "mean_token_accuracy": 0.8098476737737655,
+      "num_tokens": 2997824.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.6817218182608485,
+      "epoch": 0.148,
+      "grad_norm": 55.0,
+      "learning_rate": 7.3800000000000005e-06,
+      "loss": 10.9932,
+      "mean_token_accuracy": 0.8062656305730342,
+      "num_tokens": 3076422.0,
+      "step": 370
+    },
+    {
+      "entropy": 0.6472045814618468,
+      "epoch": 0.152,
+      "grad_norm": 48.5,
+      "learning_rate": 7.58e-06,
+      "loss": 10.5274,
+      "mean_token_accuracy": 0.8134200550615788,
+      "num_tokens": 3159946.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.639185450039804,
+      "epoch": 0.156,
+      "grad_norm": 54.25,
+      "learning_rate": 7.78e-06,
+      "loss": 10.3706,
+      "mean_token_accuracy": 0.8159802399575711,
+      "num_tokens": 3247218.0,
+      "step": 390
+    },
+    {
+      "entropy": 0.6741150500252843,
+      "epoch": 0.16,
+      "grad_norm": 53.25,
+      "learning_rate": 7.980000000000002e-06,
+      "loss": 10.9071,
+      "mean_token_accuracy": 0.8088992539793253,
+      "num_tokens": 3330597.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.16,
+      "eval_chemistry_entropy": 0.6244144983887673,
+      "eval_chemistry_loss": 0.6363974809646606,
+      "eval_chemistry_mean_token_accuracy": 0.8181359567642212,
+      "eval_chemistry_num_tokens": 3330597.0,
+      "eval_chemistry_runtime": 53.6338,
+      "eval_chemistry_samples_per_second": 9.322,
+      "eval_chemistry_steps_per_second": 9.322,
+      "step": 400
+    },
+    {
+      "epoch": 0.16,
+      "eval_physics_entropy": 0.6705876287817955,
+      "eval_physics_loss": 0.6704522371292114,
+      "eval_physics_mean_token_accuracy": 0.8111195236444473,
+      "eval_physics_num_tokens": 3330597.0,
+      "eval_physics_runtime": 62.3611,
+      "eval_physics_samples_per_second": 8.018,
+      "eval_physics_steps_per_second": 8.018,
+      "step": 400
+    },
+    {
+      "entropy": 0.6919050188735127,
+      "epoch": 0.164,
+      "grad_norm": 57.25,
+      "learning_rate": 8.18e-06,
+      "loss": 11.1447,
+      "mean_token_accuracy": 0.8047709062695503,
+      "num_tokens": 3414445.0,
+      "step": 410
+    },
+    {
+      "entropy": 0.6699288227595389,
+      "epoch": 0.168,
+      "grad_norm": 59.75,
+      "learning_rate": 8.380000000000001e-06,
+      "loss": 10.7791,
+      "mean_token_accuracy": 0.8070078935474158,
+      "num_tokens": 3496104.0,
+      "step": 420
+    },
+    {
+      "entropy": 0.6792626342736184,
+      "epoch": 0.172,
+      "grad_norm": 48.75,
+      "learning_rate": 8.580000000000001e-06,
+      "loss": 11.0757,
+      "mean_token_accuracy": 0.8060608543455601,
+      "num_tokens": 3576979.0,
+      "step": 430
+    },
+    {
+      "entropy": 0.6533824939280748,
+      "epoch": 0.176,
+      "grad_norm": 61.0,
+      "learning_rate": 8.78e-06,
+      "loss": 10.5681,
+      "mean_token_accuracy": 0.8110670737922192,
+      "num_tokens": 3658264.0,
+      "step": 440
+    },
+    {
+      "entropy": 0.6711471493355929,
+      "epoch": 0.18,
+      "grad_norm": 57.25,
+      "learning_rate": 8.98e-06,
+      "loss": 10.9005,
+      "mean_token_accuracy": 0.8076804723590613,
+      "num_tokens": 3739142.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.6721100191585719,
+      "epoch": 0.184,
+      "grad_norm": 55.0,
+      "learning_rate": 9.180000000000002e-06,
+      "loss": 10.792,
+      "mean_token_accuracy": 0.8070170730352402,
+      "num_tokens": 3821352.0,
+      "step": 460
+    },
+    {
+      "entropy": 0.6371375021524728,
+      "epoch": 0.188,
+      "grad_norm": 45.75,
+      "learning_rate": 9.38e-06,
+      "loss": 10.3376,
+      "mean_token_accuracy": 0.8134377617388964,
+      "num_tokens": 3905291.0,
+      "step": 470
+    },
+    {
+      "entropy": 0.6436822958290577,
+      "epoch": 0.192,
+      "grad_norm": 45.75,
+      "learning_rate": 9.58e-06,
+      "loss": 10.4275,
+      "mean_token_accuracy": 0.8145703230053186,
+      "num_tokens": 3995568.0,
+      "step": 480
+    },
+    {
+      "entropy": 0.6615747599862516,
+      "epoch": 0.196,
+      "grad_norm": 55.25,
+      "learning_rate": 9.780000000000001e-06,
+      "loss": 10.75,
+      "mean_token_accuracy": 0.8106145299971104,
+      "num_tokens": 4075670.0,
+      "step": 490
+    },
+    {
+      "entropy": 0.6414908402599394,
+      "epoch": 0.2,
+      "grad_norm": 55.0,
+      "learning_rate": 9.980000000000001e-06,
+      "loss": 10.318,
+      "mean_token_accuracy": 0.8157790113240481,
+      "num_tokens": 4160918.0,
+      "step": 500
+    },
+    {
+      "epoch": 0.2,
+      "eval_chemistry_entropy": 0.6180844354629517,
+      "eval_chemistry_loss": 0.6358755230903625,
+      "eval_chemistry_mean_token_accuracy": 0.8190241982936859,
+      "eval_chemistry_num_tokens": 4160918.0,
+      "eval_chemistry_runtime": 53.6,
+      "eval_chemistry_samples_per_second": 9.328,
+      "eval_chemistry_steps_per_second": 9.328,
+      "step": 500
+    },
+    {
+      "epoch": 0.2,
+      "eval_physics_entropy": 0.6517913154959679,
+      "eval_physics_loss": 0.6597976088523865,
+      "eval_physics_mean_token_accuracy": 0.8130922448635102,
+      "eval_physics_num_tokens": 4160918.0,
+      "eval_physics_runtime": 62.5405,
+      "eval_physics_samples_per_second": 7.995,
+      "eval_physics_steps_per_second": 7.995,
+      "step": 500
+    },
+    {
+      "entropy": 0.6417077989317477,
+      "epoch": 0.204,
+      "grad_norm": 48.75,
+      "learning_rate": 1.018e-05,
+      "loss": 10.4263,
+      "mean_token_accuracy": 0.8166976355016231,
+      "num_tokens": 4238810.0,
+      "step": 510
+    },
+    {
+      "entropy": 0.635231535602361,
+      "epoch": 0.208,
+      "grad_norm": 48.75,
+      "learning_rate": 1.038e-05,
+      "loss": 10.327,
+      "mean_token_accuracy": 0.8165579732507467,
+      "num_tokens": 4321436.0,
+      "step": 520
+    },
+    {
+      "entropy": 0.6600574821233749,
+      "epoch": 0.212,
+      "grad_norm": 49.5,
+      "learning_rate": 1.0580000000000002e-05,
+      "loss": 10.4772,
+      "mean_token_accuracy": 0.811732816696167,
+      "num_tokens": 4401710.0,
+      "step": 530
+    },
+    {
+      "entropy": 0.6544907880946994,
+      "epoch": 0.216,
+      "grad_norm": 46.75,
+      "learning_rate": 1.0780000000000002e-05,
+      "loss": 10.7566,
+      "mean_token_accuracy": 0.808320876955986,
+      "num_tokens": 4482225.0,
+      "step": 540
+    },
+    {
+      "entropy": 0.6429057026281952,
+      "epoch": 0.22,
+      "grad_norm": 51.25,
+      "learning_rate": 1.0980000000000002e-05,
+      "loss": 10.4272,
+      "mean_token_accuracy": 0.8134562011808157,
+      "num_tokens": 4564662.0,
+      "step": 550
+    },
+    {
+      "entropy": 0.6186996471136809,
+      "epoch": 0.224,
+      "grad_norm": 42.0,
+      "learning_rate": 1.1180000000000001e-05,
+      "loss": 9.9666,
+      "mean_token_accuracy": 0.8213985275477171,
+      "num_tokens": 4648491.0,
+      "step": 560
+    },
+    {
+      "entropy": 0.6025784744881093,
+      "epoch": 0.228,
+      "grad_norm": 40.75,
+      "learning_rate": 1.138e-05,
+      "loss": 9.8638,
+      "mean_token_accuracy": 0.8230083137750626,
+      "num_tokens": 4731596.0,
+      "step": 570
+    },
+    {
+      "entropy": 0.6584511337801814,
+      "epoch": 0.232,
+      "grad_norm": 45.0,
+      "learning_rate": 1.1580000000000001e-05,
+      "loss": 10.6688,
+      "mean_token_accuracy": 0.8104485847055912,
+      "num_tokens": 4813449.0,
+      "step": 580
+    },
+    {
+      "entropy": 0.6477856112644076,
+      "epoch": 0.236,
+      "grad_norm": 40.25,
+      "learning_rate": 1.178e-05,
+      "loss": 10.3036,
+      "mean_token_accuracy": 0.8149273872375489,
+      "num_tokens": 4899144.0,
+      "step": 590
+    },
+    {
+      "entropy": 0.632176025211811,
+      "epoch": 0.24,
+      "grad_norm": 44.5,
+      "learning_rate": 1.198e-05,
+      "loss": 10.4072,
+      "mean_token_accuracy": 0.8141517870128154,
+      "num_tokens": 4986175.0,
+      "step": 600
+    },
+    {
+      "epoch": 0.24,
+      "eval_chemistry_entropy": 0.6246057401299476,
+      "eval_chemistry_loss": 0.6355127096176147,
+      "eval_chemistry_mean_token_accuracy": 0.8170186321735382,
+      "eval_chemistry_num_tokens": 4986175.0,
+      "eval_chemistry_runtime": 53.7353,
+      "eval_chemistry_samples_per_second": 9.305,
+      "eval_chemistry_steps_per_second": 9.305,
+      "step": 600
+    },
+    {
+      "epoch": 0.24,
+      "eval_physics_entropy": 0.6429958860874176,
+      "eval_physics_loss": 0.6528828740119934,
+      "eval_physics_mean_token_accuracy": 0.8143444744944572,
+      "eval_physics_num_tokens": 4986175.0,
+      "eval_physics_runtime": 62.4695,
+      "eval_physics_samples_per_second": 8.004,
+      "eval_physics_steps_per_second": 8.004,
+      "step": 600
+    },
+    {
+      "entropy": 0.6520758749917149,
+      "epoch": 0.244,
+      "grad_norm": 50.0,
+      "learning_rate": 1.218e-05,
+      "loss": 10.5332,
+      "mean_token_accuracy": 0.8124375600367785,
+      "num_tokens": 5071994.0,
+      "step": 610
+    },
+    {
+      "entropy": 0.6255051350221038,
+      "epoch": 0.248,
+      "grad_norm": 43.5,
+      "learning_rate": 1.2380000000000002e-05,
+      "loss": 10.0529,
+      "mean_token_accuracy": 0.8187242690473795,
+      "num_tokens": 5164741.0,
+      "step": 620
+    },
+    {
+      "entropy": 0.6554224069230259,
+      "epoch": 0.252,
+      "grad_norm": 47.5,
+      "learning_rate": 1.2580000000000002e-05,
+      "loss": 10.7383,
+      "mean_token_accuracy": 0.8093429666012526,
+      "num_tokens": 5245878.0,
+      "step": 630
+    },
+    {
+      "entropy": 0.635463903658092,
+      "epoch": 0.256,
+      "grad_norm": 51.0,
+      "learning_rate": 1.2780000000000001e-05,
+      "loss": 10.3241,
+      "mean_token_accuracy": 0.8143212880939246,
+      "num_tokens": 5329320.0,
+      "step": 640
+    },
+    {
+      "entropy": 0.6407582541927695,
+      "epoch": 0.26,
+      "grad_norm": 44.75,
+      "learning_rate": 1.2980000000000001e-05,
+      "loss": 10.3061,
+      "mean_token_accuracy": 0.8153956741094589,
+      "num_tokens": 5412557.0,
+      "step": 650
+    },
+    {
+      "entropy": 0.6218360926955938,
+      "epoch": 0.264,
+      "grad_norm": 55.25,
+      "learning_rate": 1.3180000000000001e-05,
+      "loss": 10.1715,
+      "mean_token_accuracy": 0.8187983844429254,
+      "num_tokens": 5497259.0,
+      "step": 660
+    },
+    {
+      "entropy": 0.6525244776159524,
+      "epoch": 0.268,
+      "grad_norm": 48.5,
+      "learning_rate": 1.3380000000000002e-05,
+      "loss": 10.5148,
+      "mean_token_accuracy": 0.8133967652916908,
+      "num_tokens": 5576721.0,
+      "step": 670
+    },
+    {
+      "entropy": 0.6617097955197095,
+      "epoch": 0.272,
+      "grad_norm": 50.0,
+      "learning_rate": 1.3580000000000002e-05,
+      "loss": 10.7486,
+      "mean_token_accuracy": 0.8067130610346794,
+      "num_tokens": 5658796.0,
+      "step": 680
+    },
+    {
+      "entropy": 0.6757121763192118,
+      "epoch": 0.276,
+      "grad_norm": 47.25,
+      "learning_rate": 1.378e-05,
+      "loss": 10.8755,
+      "mean_token_accuracy": 0.8065906465053558,
+      "num_tokens": 5737499.0,
+      "step": 690
+    },
+    {
+      "entropy": 0.6453007774427533,
+      "epoch": 0.28,
+      "grad_norm": 62.5,
+      "learning_rate": 1.398e-05,
+      "loss": 10.4792,
+      "mean_token_accuracy": 0.8154859133064747,
+      "num_tokens": 5819027.0,
+      "step": 700
+    },
+    {
+      "epoch": 0.28,
+      "eval_chemistry_entropy": 0.6300962825417519,
+      "eval_chemistry_loss": 0.6389291882514954,
+      "eval_chemistry_mean_token_accuracy": 0.8176450840234757,
+      "eval_chemistry_num_tokens": 5819027.0,
+      "eval_chemistry_runtime": 53.7695,
+      "eval_chemistry_samples_per_second": 9.299,
+      "eval_chemistry_steps_per_second": 9.299,
+      "step": 700
+    },
+    {
+      "epoch": 0.28,
+      "eval_physics_entropy": 0.6429837597608566,
+      "eval_physics_loss": 0.6483125686645508,
+      "eval_physics_mean_token_accuracy": 0.8155634801387787,
+      "eval_physics_num_tokens": 5819027.0,
+      "eval_physics_runtime": 62.5294,
+      "eval_physics_samples_per_second": 7.996,
+      "eval_physics_steps_per_second": 7.996,
+      "step": 700
+    },
+    {
+      "entropy": 0.6762596594169736,
+      "epoch": 0.284,
+      "grad_norm": 59.0,
+      "learning_rate": 1.418e-05,
+      "loss": 10.8583,
+      "mean_token_accuracy": 0.8072559602558613,
+      "num_tokens": 5898693.0,
+      "step": 710
+    },
+    {
+      "entropy": 0.6340524691157043,
+      "epoch": 0.288,
+      "grad_norm": 47.75,
+      "learning_rate": 1.4380000000000001e-05,
+      "loss": 10.3892,
+      "mean_token_accuracy": 0.8159883081912994,
+      "num_tokens": 5980201.0,
+      "step": 720
+    },
+    {
+      "entropy": 0.6257706139236688,
+      "epoch": 0.292,
+      "grad_norm": 46.25,
+      "learning_rate": 1.4580000000000001e-05,
+      "loss": 10.052,
+      "mean_token_accuracy": 0.8204058665782213,
+      "num_tokens": 6059507.0,
+      "step": 730
+    },
+    {
+      "entropy": 0.6355020292103291,
+      "epoch": 0.296,
+      "grad_norm": 47.0,
+      "learning_rate": 1.478e-05,
+      "loss": 10.3665,
+      "mean_token_accuracy": 0.8142751138657331,
+      "num_tokens": 6139830.0,
+      "step": 740
+    },
+    {
+      "entropy": 0.6532576438970864,
+      "epoch": 0.3,
+      "grad_norm": 47.25,
+      "learning_rate": 1.498e-05,
+      "loss": 10.3728,
+      "mean_token_accuracy": 0.8153154000639915,
+      "num_tokens": 6218134.0,
+      "step": 750
+    },
+    {
+      "entropy": 0.6463365102186799,
+      "epoch": 0.304,
+      "grad_norm": 47.0,
+      "learning_rate": 1.5180000000000002e-05,
+      "loss": 10.4965,
+      "mean_token_accuracy": 0.8147804290056229,
+      "num_tokens": 6298858.0,
+      "step": 760
+    },
+    {
+      "entropy": 0.6870367551222444,
+      "epoch": 0.308,
+      "grad_norm": 55.0,
+      "learning_rate": 1.5380000000000002e-05,
+      "loss": 11.2005,
+      "mean_token_accuracy": 0.8020890522748232,
+      "num_tokens": 6375761.0,
+      "step": 770
+    },
+    {
+      "entropy": 0.6256772884167731,
+      "epoch": 0.312,
+      "grad_norm": 41.75,
+      "learning_rate": 1.5580000000000003e-05,
+      "loss": 10.0914,
+      "mean_token_accuracy": 0.8196329109370708,
+      "num_tokens": 6456502.0,
+      "step": 780
+    },
+    {
+      "entropy": 0.6423514087684452,
+      "epoch": 0.316,
+      "grad_norm": 49.75,
+      "learning_rate": 1.578e-05,
+      "loss": 10.492,
+      "mean_token_accuracy": 0.8146003533154726,
+      "num_tokens": 6537521.0,
+      "step": 790
+    },
+    {
+      "entropy": 0.6631265950389207,
+      "epoch": 0.32,
+      "grad_norm": 43.0,
+      "learning_rate": 1.5980000000000003e-05,
+      "loss": 10.7489,
+      "mean_token_accuracy": 0.8101503010839224,
+      "num_tokens": 6622798.0,
+      "step": 800
+    },
+    {
+      "epoch": 0.32,
+      "eval_chemistry_entropy": 0.6421979127824307,
+      "eval_chemistry_loss": 0.6415462493896484,
+      "eval_chemistry_mean_token_accuracy": 0.817062867641449,
+      "eval_chemistry_num_tokens": 6622798.0,
+      "eval_chemistry_runtime": 53.6883,
+      "eval_chemistry_samples_per_second": 9.313,
+      "eval_chemistry_steps_per_second": 9.313,
+      "step": 800
+    },
+    {
+      "epoch": 0.32,
+      "eval_physics_entropy": 0.6440860625505448,
+      "eval_physics_loss": 0.6467884182929993,
+      "eval_physics_mean_token_accuracy": 0.8157004522681236,
+      "eval_physics_num_tokens": 6622798.0,
+      "eval_physics_runtime": 62.4499,
+      "eval_physics_samples_per_second": 8.006,
+      "eval_physics_steps_per_second": 8.006,
+      "step": 800
+    },
+    {
+      "entropy": 0.6417643938213586,
+      "epoch": 0.324,
+      "grad_norm": 54.0,
+      "learning_rate": 1.618e-05,
+      "loss": 10.54,
+      "mean_token_accuracy": 0.8144381437450647,
+      "num_tokens": 6699373.0,
+      "step": 810
+    },
+    {
+      "entropy": 0.6635292621329427,
+      "epoch": 0.328,
+      "grad_norm": 43.25,
+      "learning_rate": 1.638e-05,
+      "loss": 10.6337,
+      "mean_token_accuracy": 0.8078649930655957,
+      "num_tokens": 6782985.0,
+      "step": 820
+    },
+    {
+      "entropy": 0.6515420666895807,
+      "epoch": 0.332,
+      "grad_norm": 51.5,
+      "learning_rate": 1.658e-05,
+      "loss": 10.4447,
+      "mean_token_accuracy": 0.8119669426232576,
+      "num_tokens": 6868578.0,
+      "step": 830
+    },
+    {
+      "entropy": 0.6081873361952603,
+      "epoch": 0.336,
+      "grad_norm": 40.0,
+      "learning_rate": 1.6780000000000002e-05,
+      "loss": 9.8522,
+      "mean_token_accuracy": 0.8244299061596394,
+      "num_tokens": 6953611.0,
+      "step": 840
+    },
+    {
+      "entropy": 0.6503613390959799,
+      "epoch": 0.34,
+      "grad_norm": 52.0,
+      "learning_rate": 1.698e-05,
+      "loss": 10.6425,
+      "mean_token_accuracy": 0.8126828409731388,
+      "num_tokens": 7037831.0,
+      "step": 850
+    },
+    {
+      "entropy": 0.611198160611093,
+      "epoch": 0.344,
+      "grad_norm": 56.75,
+      "learning_rate": 1.718e-05,
+      "loss": 9.9189,
+      "mean_token_accuracy": 0.8207242820411921,
+      "num_tokens": 7121949.0,
+      "step": 860
+    },
+    {
+      "entropy": 0.6327245706692338,
+      "epoch": 0.348,
+      "grad_norm": 44.0,
+      "learning_rate": 1.7380000000000003e-05,
+      "loss": 10.2368,
+      "mean_token_accuracy": 0.8166516173630953,
+      "num_tokens": 7206056.0,
+      "step": 870
+    },
+    {
+      "entropy": 0.6341161539778113,
+      "epoch": 0.352,
+      "grad_norm": 51.5,
+      "learning_rate": 1.758e-05,
+      "loss": 10.2941,
+      "mean_token_accuracy": 0.816456102579832,
+      "num_tokens": 7290378.0,
+      "step": 880
+    },
+    {
+      "entropy": 0.6554373924620449,
+      "epoch": 0.356,
+      "grad_norm": 42.5,
+      "learning_rate": 1.7780000000000003e-05,
+      "loss": 10.5667,
+      "mean_token_accuracy": 0.8104542337357998,
+      "num_tokens": 7367952.0,
+      "step": 890
+    },
+    {
+      "entropy": 0.6374449659138918,
+      "epoch": 0.36,
+      "grad_norm": 59.75,
+      "learning_rate": 1.798e-05,
+      "loss": 10.3565,
+      "mean_token_accuracy": 0.816227027028799,
+      "num_tokens": 7452920.0,
+      "step": 900
+    },
+    {
+      "epoch": 0.36,
+      "eval_chemistry_entropy": 0.6640482069849968,
+      "eval_chemistry_loss": 0.6512866020202637,
+      "eval_chemistry_mean_token_accuracy": 0.8145369750261306,
+      "eval_chemistry_num_tokens": 7452920.0,
+      "eval_chemistry_runtime": 53.5846,
+      "eval_chemistry_samples_per_second": 9.331,
+      "eval_chemistry_steps_per_second": 9.331,
+      "step": 900
+    },
+    {
+      "epoch": 0.36,
+      "eval_physics_entropy": 0.65689039093256,
+      "eval_physics_loss": 0.64650559425354,
+      "eval_physics_mean_token_accuracy": 0.8153353578448296,
+      "eval_physics_num_tokens": 7452920.0,
+      "eval_physics_runtime": 62.3633,
+      "eval_physics_samples_per_second": 8.018,
+      "eval_physics_steps_per_second": 8.018,
+      "step": 900
+    },
+    {
+      "entropy": 0.6258705526590347,
+      "epoch": 0.364,
+      "grad_norm": 43.75,
+      "learning_rate": 1.8180000000000002e-05,
+      "loss": 10.013,
+      "mean_token_accuracy": 0.8192542195320129,
+      "num_tokens": 7535961.0,
+      "step": 910
+    },
+    {
+      "entropy": 0.6131137845106422,
+      "epoch": 0.368,
+      "grad_norm": 43.5,
+      "learning_rate": 1.8380000000000004e-05,
+      "loss": 9.9858,
+      "mean_token_accuracy": 0.8190763648599386,
+      "num_tokens": 7621082.0,
+      "step": 920
+    },
+    {
+      "entropy": 0.6138401796109975,
+      "epoch": 0.372,
+      "grad_norm": 46.75,
+      "learning_rate": 1.858e-05,
+      "loss": 10.0396,
+      "mean_token_accuracy": 0.8174721483141184,
+      "num_tokens": 7705651.0,
+      "step": 930
+    },
+    {
+      "entropy": 0.7146476589143276,
+      "epoch": 0.376,
+      "grad_norm": 46.25,
+      "learning_rate": 1.878e-05,
+      "loss": 11.4452,
+      "mean_token_accuracy": 0.7953129142522812,
+      "num_tokens": 7787036.0,
+      "step": 940
+    },
+    {
+      "entropy": 0.6485187427140773,
+      "epoch": 0.38,
+      "grad_norm": 56.5,
+      "learning_rate": 1.898e-05,
+      "loss": 10.5487,
+      "mean_token_accuracy": 0.8087487578392029,
+      "num_tokens": 7869437.0,
+      "step": 950
+    },
+    {
+      "entropy": 0.633660759497434,
+      "epoch": 0.384,
+      "grad_norm": 42.0,
+      "learning_rate": 1.918e-05,
+      "loss": 10.1676,
+      "mean_token_accuracy": 0.8168184392154216,
+      "num_tokens": 7955570.0,
+      "step": 960
+    },
+    {
+      "entropy": 0.5985365292057395,
+      "epoch": 0.388,
+      "grad_norm": 38.25,
+      "learning_rate": 1.938e-05,
+      "loss": 9.8157,
+      "mean_token_accuracy": 0.8218576721847057,
+      "num_tokens": 8038818.0,
+      "step": 970
+    },
+    {
+      "entropy": 0.6344430455937982,
+      "epoch": 0.392,
+      "grad_norm": 39.75,
+      "learning_rate": 1.9580000000000002e-05,
+      "loss": 10.1821,
+      "mean_token_accuracy": 0.8146390464156866,
+      "num_tokens": 8123527.0,
+      "step": 980
+    },
+    {
+      "entropy": 0.6378526957705617,
+      "epoch": 0.396,
+      "grad_norm": 43.75,
+      "learning_rate": 1.978e-05,
+      "loss": 10.2889,
+      "mean_token_accuracy": 0.8144868839532137,
+      "num_tokens": 8206501.0,
+      "step": 990
+    },
+    {
+      "entropy": 0.6167035531252623,
+      "epoch": 0.4,
+      "grad_norm": 36.25,
+      "learning_rate": 1.9980000000000002e-05,
+      "loss": 10.0591,
+      "mean_token_accuracy": 0.819990050420165,
+      "num_tokens": 8291049.0,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4,
+      "eval_chemistry_entropy": 0.6586475971937179,
+      "eval_chemistry_loss": 0.6543877124786377,
+      "eval_chemistry_mean_token_accuracy": 0.8140276271104813,
+      "eval_chemistry_num_tokens": 8291049.0,
+      "eval_chemistry_runtime": 53.801,
+      "eval_chemistry_samples_per_second": 9.294,
+      "eval_chemistry_steps_per_second": 9.294,
+      "step": 1000
+    },
+    {
+      "epoch": 0.4,
+      "eval_physics_entropy": 0.645909442961216,
+      "eval_physics_loss": 0.6488538980484009,
+      "eval_physics_mean_token_accuracy": 0.8152267854809762,
+      "eval_physics_num_tokens": 8291049.0,
+      "eval_physics_runtime": 63.4318,
+      "eval_physics_samples_per_second": 7.882,
+      "eval_physics_steps_per_second": 7.882,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 10000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 4,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 5.559510372210398e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4d00e94416d3bac3bf32729591fc3844592549d405ab9c5d76f890a1b4bd0d15
+size 6481