Instructions to use arunasank/qahjdtxg with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use arunasank/qahjdtxg with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="arunasank/qahjdtxg")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("arunasank/qahjdtxg")
model = AutoModelForImageTextToText.from_pretrained("arunasank/qahjdtxg")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use arunasank/qahjdtxg with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "arunasank/qahjdtxg"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arunasank/qahjdtxg",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/arunasank/qahjdtxg

SGLang

How to use arunasank/qahjdtxg with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "arunasank/qahjdtxg" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arunasank/qahjdtxg",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "arunasank/qahjdtxg" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "arunasank/qahjdtxg",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use arunasank/qahjdtxg with Docker Model Runner:
```
docker model run hf.co/arunasank/qahjdtxg
```

arunasank commited on Apr 22

Commit

45d4046

verified ·

1 Parent(s): 7c0c12b

Upload folder using huggingface_hub

Browse files

Files changed (20) hide show

.gitattributes +1 -0
checkpoint-1000/added_tokens.json +3 -0
checkpoint-1000/chat_template.jinja +47 -0
checkpoint-1000/config.json +110 -0
checkpoint-1000/generation_config.json +13 -0
checkpoint-1000/model-00001-of-00005.safetensors +3 -0
checkpoint-1000/model-00002-of-00005.safetensors +3 -0
checkpoint-1000/model-00003-of-00005.safetensors +3 -0
checkpoint-1000/model-00004-of-00005.safetensors +3 -0
checkpoint-1000/model-00005-of-00005.safetensors +3 -0
checkpoint-1000/model.safetensors.index.json +0 -0
checkpoint-1000/optimizer.pt +3 -0
checkpoint-1000/rng_state.pth +3 -0
checkpoint-1000/scheduler.pt +3 -0
checkpoint-1000/special_tokens_map.json +33 -0
checkpoint-1000/tokenizer.json +3 -0
checkpoint-1000/tokenizer.model +3 -0
checkpoint-1000/tokenizer_config.json +0 -0
checkpoint-1000/trainer_state.json +1474 -0
checkpoint-1000/training_args.bin +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text
+checkpoint-1000/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-1000/added_tokens.json ADDED Viewed

	@@ -0,0 +1,3 @@

+{
+  "<image_soft_token>": 262144
+}

checkpoint-1000/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,47 @@

+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

checkpoint-1000/config.json ADDED Viewed

	@@ -0,0 +1,110 @@

+{
+  "architectures": [
+    "Gemma3ForConditionalGeneration"
+  ],
+  "boi_token_index": 255999,
+  "bos_token_id": 2,
+  "dtype": "bfloat16",
+  "eoi_token_index": 256000,
+  "eos_token_id": 1,
+  "image_token_index": 262144,
+  "initializer_range": 0.02,
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "pad_token_id": 0,
+  "text_config": {
+    "_sliding_window_pattern": 6,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "dtype": "bfloat16",
+    "final_logit_softcapping": null,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 3840,
+    "initializer_range": 0.02,
+    "intermediate_size": 15360,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 48,
+    "num_key_value_heads": 8,
+    "query_pre_attn_scalar": 256,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000.0,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": 1024,
+    "use_cache": true,
+    "vocab_size": 262208
+  },
+  "transformers_version": "4.56.1",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "vision_use_head": false
+  }
+}

checkpoint-1000/generation_config.json ADDED Viewed

	@@ -0,0 +1,13 @@

+{
+  "bos_token_id": 2,
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.56.1"
+}

checkpoint-1000/model-00001-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4847447e92599833e8dbaa3067cd201c3bb5c052efa91f11ba891e43234f7832
+size 4979902192

checkpoint-1000/model-00002-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:891bd54eed03cba9ee1e705533a02a8217fcc29f356e4a1f53e5fd0d178883ad
+size 4931296592

checkpoint-1000/model-00003-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7cee411d9d57324e50ce064a192cc5a858276d508611b12fc599e0c9767112e0
+size 4931296656

checkpoint-1000/model-00004-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ea3ee9d6dac79952a851527bb64d6a142d02cd6dc1b182eaae599eff21352d2c
+size 4931296656

checkpoint-1000/model-00005-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:10d499da845841a77c8871ad3df1cff8c68ff794706b4514402c432fc35386a8
+size 4601000928

checkpoint-1000/model.safetensors.index.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-1000/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7a1730b4556d5ab8c4d715d03d23833dbd6093dc483e28c2bb532fae33f21bbd
+size 1793214689

checkpoint-1000/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:471b1c653aabe47d1dda07893496c6ec8f21353cd348bef0e1f9b8b6dd76552c
+size 14645

checkpoint-1000/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1047a716c066f9f4e78c42060834e4ca319f06029bfedda77528f4fcc6970b7e
+size 1465

checkpoint-1000/special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,33 @@

+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-1000/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

checkpoint-1000/tokenizer.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

checkpoint-1000/tokenizer_config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-1000/trainer_state.json ADDED Viewed

	@@ -0,0 +1,1474 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 1.6,
+  "eval_steps": 100,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 1.2676397945731879,
+      "epoch": 0.016,
+      "grad_norm": 25.25,
+      "learning_rate": 4.5000000000000003e-07,
+      "loss": 20.5205,
+      "mean_token_accuracy": 0.6834932763129473,
+      "num_tokens": 195524.0,
+      "step": 10
+    },
+    {
+      "entropy": 1.261214752867818,
+      "epoch": 0.032,
+      "grad_norm": 22.75,
+      "learning_rate": 9.500000000000001e-07,
+      "loss": 20.4527,
+      "mean_token_accuracy": 0.68165775090456,
+      "num_tokens": 390903.0,
+      "step": 20
+    },
+    {
+      "entropy": 1.2643471464514733,
+      "epoch": 0.048,
+      "grad_norm": 25.375,
+      "learning_rate": 1.45e-06,
+      "loss": 20.5098,
+      "mean_token_accuracy": 0.6823193147778511,
+      "num_tokens": 589868.0,
+      "step": 30
+    },
+    {
+      "entropy": 1.2646946143358946,
+      "epoch": 0.064,
+      "grad_norm": 20.625,
+      "learning_rate": 1.9500000000000004e-06,
+      "loss": 20.5081,
+      "mean_token_accuracy": 0.6804493211209774,
+      "num_tokens": 791190.0,
+      "step": 40
+    },
+    {
+      "entropy": 1.2883822418749333,
+      "epoch": 0.08,
+      "grad_norm": 22.875,
+      "learning_rate": 2.4500000000000003e-06,
+      "loss": 20.7929,
+      "mean_token_accuracy": 0.6806381791830063,
+      "num_tokens": 989860.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.302551382407546,
+      "epoch": 0.096,
+      "grad_norm": 23.625,
+      "learning_rate": 2.95e-06,
+      "loss": 21.0666,
+      "mean_token_accuracy": 0.6782213494181633,
+      "num_tokens": 1181777.0,
+      "step": 60
+    },
+    {
+      "entropy": 1.281064134463668,
+      "epoch": 0.112,
+      "grad_norm": 22.5,
+      "learning_rate": 3.45e-06,
+      "loss": 20.7617,
+      "mean_token_accuracy": 0.6805306363850832,
+      "num_tokens": 1385513.0,
+      "step": 70
+    },
+    {
+      "entropy": 1.2786549139767884,
+      "epoch": 0.128,
+      "grad_norm": 24.375,
+      "learning_rate": 3.95e-06,
+      "loss": 20.6078,
+      "mean_token_accuracy": 0.680861197039485,
+      "num_tokens": 1582368.0,
+      "step": 80
+    },
+    {
+      "entropy": 1.2977753818035125,
+      "epoch": 0.144,
+      "grad_norm": 22.5,
+      "learning_rate": 4.450000000000001e-06,
+      "loss": 20.7713,
+      "mean_token_accuracy": 0.6802186716347933,
+      "num_tokens": 1773764.0,
+      "step": 90
+    },
+    {
+      "entropy": 1.2986781910061835,
+      "epoch": 0.16,
+      "grad_norm": 26.0,
+      "learning_rate": 4.95e-06,
+      "loss": 20.9638,
+      "mean_token_accuracy": 0.6783340185880661,
+      "num_tokens": 1970077.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.16,
+      "eval_biology_entropy": 1.2467972478866578,
+      "eval_biology_loss": 1.3016669750213623,
+      "eval_biology_mean_token_accuracy": 0.6796614747047425,
+      "eval_biology_num_tokens": 1970077.0,
+      "eval_biology_runtime": 47.4105,
+      "eval_biology_samples_per_second": 10.546,
+      "eval_biology_steps_per_second": 2.637,
+      "step": 100
+    },
+    {
+      "epoch": 0.16,
+      "eval_chemistry_entropy": 1.0001942477226258,
+      "eval_chemistry_loss": 1.0766135454177856,
+      "eval_chemistry_mean_token_accuracy": 0.7288195652961731,
+      "eval_chemistry_num_tokens": 1970077.0,
+      "eval_chemistry_runtime": 58.7789,
+      "eval_chemistry_samples_per_second": 8.506,
+      "eval_chemistry_steps_per_second": 2.127,
+      "step": 100
+    },
+    {
+      "epoch": 0.16,
+      "eval_math_entropy": 0.9485790143013001,
+      "eval_math_loss": 1.2958632707595825,
+      "eval_math_mean_token_accuracy": 0.7046140608787537,
+      "eval_math_num_tokens": 1970077.0,
+      "eval_math_runtime": 60.2784,
+      "eval_math_samples_per_second": 8.295,
+      "eval_math_steps_per_second": 2.074,
+      "step": 100
+    },
+    {
+      "epoch": 0.16,
+      "eval_physics_entropy": 1.0591417260169982,
+      "eval_physics_loss": 1.182342529296875,
+      "eval_physics_mean_token_accuracy": 0.7221122503280639,
+      "eval_physics_num_tokens": 1970077.0,
+      "eval_physics_runtime": 68.6963,
+      "eval_physics_samples_per_second": 7.278,
+      "eval_physics_steps_per_second": 1.82,
+      "step": 100
+    },
+    {
+      "entropy": 1.2789885632693767,
+      "epoch": 0.176,
+      "grad_norm": 22.5,
+      "learning_rate": 5.450000000000001e-06,
+      "loss": 20.6234,
+      "mean_token_accuracy": 0.6804416611790657,
+      "num_tokens": 2168354.0,
+      "step": 110
+    },
+    {
+      "entropy": 1.2759516820311547,
+      "epoch": 0.192,
+      "grad_norm": 24.75,
+      "learning_rate": 5.950000000000001e-06,
+      "loss": 20.5679,
+      "mean_token_accuracy": 0.681303645670414,
+      "num_tokens": 2365822.0,
+      "step": 120
+    },
+    {
+      "entropy": 1.2930898044258357,
+      "epoch": 0.208,
+      "grad_norm": 25.125,
+      "learning_rate": 6.450000000000001e-06,
+      "loss": 20.8391,
+      "mean_token_accuracy": 0.6775826022028923,
+      "num_tokens": 2558762.0,
+      "step": 130
+    },
+    {
+      "entropy": 1.2815773852169514,
+      "epoch": 0.224,
+      "grad_norm": 24.0,
+      "learning_rate": 6.95e-06,
+      "loss": 20.7118,
+      "mean_token_accuracy": 0.6805033922195435,
+      "num_tokens": 2755347.0,
+      "step": 140
+    },
+    {
+      "entropy": 1.271333383768797,
+      "epoch": 0.24,
+      "grad_norm": 23.375,
+      "learning_rate": 7.450000000000001e-06,
+      "loss": 20.4896,
+      "mean_token_accuracy": 0.6833334766328335,
+      "num_tokens": 2947346.0,
+      "step": 150
+    },
+    {
+      "entropy": 1.2992272667586804,
+      "epoch": 0.256,
+      "grad_norm": 21.75,
+      "learning_rate": 7.950000000000002e-06,
+      "loss": 20.7815,
+      "mean_token_accuracy": 0.6796726986765862,
+      "num_tokens": 3139957.0,
+      "step": 160
+    },
+    {
+      "entropy": 1.2592430911958217,
+      "epoch": 0.272,
+      "grad_norm": 24.25,
+      "learning_rate": 8.45e-06,
+      "loss": 20.4802,
+      "mean_token_accuracy": 0.682121679186821,
+      "num_tokens": 3335951.0,
+      "step": 170
+    },
+    {
+      "entropy": 1.2648221261799335,
+      "epoch": 0.288,
+      "grad_norm": 25.25,
+      "learning_rate": 8.95e-06,
+      "loss": 20.3318,
+      "mean_token_accuracy": 0.6829198688268662,
+      "num_tokens": 3539731.0,
+      "step": 180
+    },
+    {
+      "entropy": 1.2702809911221267,
+      "epoch": 0.304,
+      "grad_norm": 22.375,
+      "learning_rate": 9.450000000000001e-06,
+      "loss": 20.4024,
+      "mean_token_accuracy": 0.6853499349206686,
+      "num_tokens": 3733488.0,
+      "step": 190
+    },
+    {
+      "entropy": 1.2876859605312347,
+      "epoch": 0.32,
+      "grad_norm": 25.875,
+      "learning_rate": 9.950000000000001e-06,
+      "loss": 20.7513,
+      "mean_token_accuracy": 0.677771307528019,
+      "num_tokens": 3920545.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.32,
+      "eval_biology_entropy": 1.2387963032722473,
+      "eval_biology_loss": 1.3000890016555786,
+      "eval_biology_mean_token_accuracy": 0.6799756650924682,
+      "eval_biology_num_tokens": 3920545.0,
+      "eval_biology_runtime": 47.4397,
+      "eval_biology_samples_per_second": 10.54,
+      "eval_biology_steps_per_second": 2.635,
+      "step": 200
+    },
+    {
+      "epoch": 0.32,
+      "eval_chemistry_entropy": 0.9924480357170105,
+      "eval_chemistry_loss": 1.0764909982681274,
+      "eval_chemistry_mean_token_accuracy": 0.7288783774375915,
+      "eval_chemistry_num_tokens": 3920545.0,
+      "eval_chemistry_runtime": 58.9282,
+      "eval_chemistry_samples_per_second": 8.485,
+      "eval_chemistry_steps_per_second": 2.121,
+      "step": 200
+    },
+    {
+      "epoch": 0.32,
+      "eval_math_entropy": 0.9413906736373902,
+      "eval_math_loss": 1.2954370975494385,
+      "eval_math_mean_token_accuracy": 0.7050176553726196,
+      "eval_math_num_tokens": 3920545.0,
+      "eval_math_runtime": 60.3526,
+      "eval_math_samples_per_second": 8.285,
+      "eval_math_steps_per_second": 2.071,
+      "step": 200
+    },
+    {
+      "epoch": 0.32,
+      "eval_physics_entropy": 1.0506655836105347,
+      "eval_physics_loss": 1.1805146932601929,
+      "eval_physics_mean_token_accuracy": 0.7223957962989808,
+      "eval_physics_num_tokens": 3920545.0,
+      "eval_physics_runtime": 68.7486,
+      "eval_physics_samples_per_second": 7.273,
+      "eval_physics_steps_per_second": 1.818,
+      "step": 200
+    },
+    {
+      "entropy": 1.252883468195796,
+      "epoch": 0.336,
+      "grad_norm": 23.5,
+      "learning_rate": 1.045e-05,
+      "loss": 20.089,
+      "mean_token_accuracy": 0.6867416497319937,
+      "num_tokens": 4114077.0,
+      "step": 210
+    },
+    {
+      "entropy": 1.2703593511134386,
+      "epoch": 0.352,
+      "grad_norm": 24.625,
+      "learning_rate": 1.095e-05,
+      "loss": 20.5308,
+      "mean_token_accuracy": 0.6833599351346493,
+      "num_tokens": 4306949.0,
+      "step": 220
+    },
+    {
+      "entropy": 1.260924168303609,
+      "epoch": 0.368,
+      "grad_norm": 22.875,
+      "learning_rate": 1.145e-05,
+      "loss": 20.3988,
+      "mean_token_accuracy": 0.6848072897642851,
+      "num_tokens": 4504001.0,
+      "step": 230
+    },
+    {
+      "entropy": 1.272016329318285,
+      "epoch": 0.384,
+      "grad_norm": 25.625,
+      "learning_rate": 1.195e-05,
+      "loss": 20.4684,
+      "mean_token_accuracy": 0.6823147349059582,
+      "num_tokens": 4693812.0,
+      "step": 240
+    },
+    {
+      "entropy": 1.281133072450757,
+      "epoch": 0.4,
+      "grad_norm": 22.5,
+      "learning_rate": 1.2450000000000003e-05,
+      "loss": 20.6448,
+      "mean_token_accuracy": 0.6809127051383257,
+      "num_tokens": 4887094.0,
+      "step": 250
+    },
+    {
+      "entropy": 1.2905787032097578,
+      "epoch": 0.416,
+      "grad_norm": 24.0,
+      "learning_rate": 1.295e-05,
+      "loss": 20.8418,
+      "mean_token_accuracy": 0.6782213684171439,
+      "num_tokens": 5085369.0,
+      "step": 260
+    },
+    {
+      "entropy": 1.2866624522954226,
+      "epoch": 0.432,
+      "grad_norm": 25.625,
+      "learning_rate": 1.3450000000000002e-05,
+      "loss": 20.6614,
+      "mean_token_accuracy": 0.6810611065477132,
+      "num_tokens": 5271275.0,
+      "step": 270
+    },
+    {
+      "entropy": 1.2861373092979194,
+      "epoch": 0.448,
+      "grad_norm": 26.5,
+      "learning_rate": 1.3950000000000002e-05,
+      "loss": 20.6931,
+      "mean_token_accuracy": 0.6811038155108691,
+      "num_tokens": 5460559.0,
+      "step": 280
+    },
+    {
+      "entropy": 1.2432711333036424,
+      "epoch": 0.464,
+      "grad_norm": 24.75,
+      "learning_rate": 1.4450000000000002e-05,
+      "loss": 20.0188,
+      "mean_token_accuracy": 0.6878789726644754,
+      "num_tokens": 5653809.0,
+      "step": 290
+    },
+    {
+      "entropy": 1.2743295758962632,
+      "epoch": 0.48,
+      "grad_norm": 21.75,
+      "learning_rate": 1.4950000000000003e-05,
+      "loss": 20.5069,
+      "mean_token_accuracy": 0.6826238337904215,
+      "num_tokens": 5850176.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.48,
+      "eval_biology_entropy": 1.2377551465034484,
+      "eval_biology_loss": 1.3000712394714355,
+      "eval_biology_mean_token_accuracy": 0.6797983174324036,
+      "eval_biology_num_tokens": 5850176.0,
+      "eval_biology_runtime": 47.4305,
+      "eval_biology_samples_per_second": 10.542,
+      "eval_biology_steps_per_second": 2.635,
+      "step": 300
+    },
+    {
+      "epoch": 0.48,
+      "eval_chemistry_entropy": 0.9973236765861512,
+      "eval_chemistry_loss": 1.0776281356811523,
+      "eval_chemistry_mean_token_accuracy": 0.7284700326919555,
+      "eval_chemistry_num_tokens": 5850176.0,
+      "eval_chemistry_runtime": 58.8579,
+      "eval_chemistry_samples_per_second": 8.495,
+      "eval_chemistry_steps_per_second": 2.124,
+      "step": 300
+    },
+    {
+      "epoch": 0.48,
+      "eval_math_entropy": 0.9509003200531005,
+      "eval_math_loss": 1.2968242168426514,
+      "eval_math_mean_token_accuracy": 0.7043156070709229,
+      "eval_math_num_tokens": 5850176.0,
+      "eval_math_runtime": 60.3466,
+      "eval_math_samples_per_second": 8.285,
+      "eval_math_steps_per_second": 2.071,
+      "step": 300
+    },
+    {
+      "epoch": 0.48,
+      "eval_physics_entropy": 1.059091604232788,
+      "eval_physics_loss": 1.1835830211639404,
+      "eval_physics_mean_token_accuracy": 0.722058045387268,
+      "eval_physics_num_tokens": 5850176.0,
+      "eval_physics_runtime": 68.7784,
+      "eval_physics_samples_per_second": 7.27,
+      "eval_physics_steps_per_second": 1.817,
+      "step": 300
+    },
+    {
+      "entropy": 1.2032247751951217,
+      "epoch": 0.496,
+      "grad_norm": 22.375,
+      "learning_rate": 1.545e-05,
+      "loss": 19.3781,
+      "mean_token_accuracy": 0.6952671084553004,
+      "num_tokens": 6046503.0,
+      "step": 310
+    },
+    {
+      "entropy": 1.2634436808526517,
+      "epoch": 0.512,
+      "grad_norm": 23.5,
+      "learning_rate": 1.595e-05,
+      "loss": 20.349,
+      "mean_token_accuracy": 0.6837061777710914,
+      "num_tokens": 6240456.0,
+      "step": 320
+    },
+    {
+      "entropy": 1.2657200925052166,
+      "epoch": 0.528,
+      "grad_norm": 24.125,
+      "learning_rate": 1.645e-05,
+      "loss": 20.3697,
+      "mean_token_accuracy": 0.6848184332251549,
+      "num_tokens": 6430555.0,
+      "step": 330
+    },
+    {
+      "entropy": 1.26240081936121,
+      "epoch": 0.544,
+      "grad_norm": 21.5,
+      "learning_rate": 1.6950000000000002e-05,
+      "loss": 20.3089,
+      "mean_token_accuracy": 0.6857475839555264,
+      "num_tokens": 6626006.0,
+      "step": 340
+    },
+    {
+      "entropy": 1.2498273212462663,
+      "epoch": 0.56,
+      "grad_norm": 22.0,
+      "learning_rate": 1.7450000000000004e-05,
+      "loss": 20.2072,
+      "mean_token_accuracy": 0.6853332217782736,
+      "num_tokens": 6820754.0,
+      "step": 350
+    },
+    {
+      "entropy": 1.241705995053053,
+      "epoch": 0.576,
+      "grad_norm": 23.125,
+      "learning_rate": 1.795e-05,
+      "loss": 19.9881,
+      "mean_token_accuracy": 0.6875007815659047,
+      "num_tokens": 7021844.0,
+      "step": 360
+    },
+    {
+      "entropy": 1.2584628861397504,
+      "epoch": 0.592,
+      "grad_norm": 23.375,
+      "learning_rate": 1.845e-05,
+      "loss": 20.224,
+      "mean_token_accuracy": 0.685963337123394,
+      "num_tokens": 7213951.0,
+      "step": 370
+    },
+    {
+      "entropy": 1.2345197916030883,
+      "epoch": 0.608,
+      "grad_norm": 23.875,
+      "learning_rate": 1.8950000000000003e-05,
+      "loss": 19.9716,
+      "mean_token_accuracy": 0.6883670844137668,
+      "num_tokens": 7416773.0,
+      "step": 380
+    },
+    {
+      "entropy": 1.256280617043376,
+      "epoch": 0.624,
+      "grad_norm": 25.0,
+      "learning_rate": 1.9450000000000002e-05,
+      "loss": 20.1361,
+      "mean_token_accuracy": 0.6866784512996673,
+      "num_tokens": 7612843.0,
+      "step": 390
+    },
+    {
+      "entropy": 1.2612476922571658,
+      "epoch": 0.64,
+      "grad_norm": 24.875,
+      "learning_rate": 1.9950000000000004e-05,
+      "loss": 20.2559,
+      "mean_token_accuracy": 0.6857616238296032,
+      "num_tokens": 7801633.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.64,
+      "eval_biology_entropy": 1.2140547132492066,
+      "eval_biology_loss": 1.2990955114364624,
+      "eval_biology_mean_token_accuracy": 0.6800553450584411,
+      "eval_biology_num_tokens": 7801633.0,
+      "eval_biology_runtime": 47.3483,
+      "eval_biology_samples_per_second": 10.56,
+      "eval_biology_steps_per_second": 2.64,
+      "step": 400
+    },
+    {
+      "epoch": 0.64,
+      "eval_chemistry_entropy": 0.976757568359375,
+      "eval_chemistry_loss": 1.0796345472335815,
+      "eval_chemistry_mean_token_accuracy": 0.7284749069213867,
+      "eval_chemistry_num_tokens": 7801633.0,
+      "eval_chemistry_runtime": 58.7233,
+      "eval_chemistry_samples_per_second": 8.515,
+      "eval_chemistry_steps_per_second": 2.129,
+      "step": 400
+    },
+    {
+      "epoch": 0.64,
+      "eval_math_entropy": 0.9269939508438111,
+      "eval_math_loss": 1.307834267616272,
+      "eval_math_mean_token_accuracy": 0.7034133396148682,
+      "eval_math_num_tokens": 7801633.0,
+      "eval_math_runtime": 60.2192,
+      "eval_math_samples_per_second": 8.303,
+      "eval_math_steps_per_second": 2.076,
+      "step": 400
+    },
+    {
+      "epoch": 0.64,
+      "eval_physics_entropy": 1.0324158954620362,
+      "eval_physics_loss": 1.1809009313583374,
+      "eval_physics_mean_token_accuracy": 0.7219376945495606,
+      "eval_physics_num_tokens": 7801633.0,
+      "eval_physics_runtime": 68.6721,
+      "eval_physics_samples_per_second": 7.281,
+      "eval_physics_steps_per_second": 1.82,
+      "step": 400
+    },
+    {
+      "entropy": 1.2535340417176486,
+      "epoch": 0.656,
+      "grad_norm": 25.0,
+      "learning_rate": 1.9950000000000004e-05,
+      "loss": 20.2012,
+      "mean_token_accuracy": 0.6859036896377801,
+      "num_tokens": 7995843.0,
+      "step": 410
+    },
+    {
+      "entropy": 1.2642133347690105,
+      "epoch": 0.672,
+      "grad_norm": 24.125,
+      "learning_rate": 1.9894444444444446e-05,
+      "loss": 20.3439,
+      "mean_token_accuracy": 0.6839185860008001,
+      "num_tokens": 8183103.0,
+      "step": 420
+    },
+    {
+      "entropy": 1.2550158943980931,
+      "epoch": 0.688,
+      "grad_norm": 23.125,
+      "learning_rate": 1.983888888888889e-05,
+      "loss": 20.2518,
+      "mean_token_accuracy": 0.6866896580904722,
+      "num_tokens": 8385976.0,
+      "step": 430
+    },
+    {
+      "entropy": 1.2480014562606812,
+      "epoch": 0.704,
+      "grad_norm": 22.875,
+      "learning_rate": 1.9783333333333334e-05,
+      "loss": 20.08,
+      "mean_token_accuracy": 0.6888119786977768,
+      "num_tokens": 8578431.0,
+      "step": 440
+    },
+    {
+      "entropy": 1.197030144929886,
+      "epoch": 0.72,
+      "grad_norm": 22.5,
+      "learning_rate": 1.972777777777778e-05,
+      "loss": 19.2953,
+      "mean_token_accuracy": 0.697203055024147,
+      "num_tokens": 8781342.0,
+      "step": 450
+    },
+    {
+      "entropy": 1.2185503005981446,
+      "epoch": 0.736,
+      "grad_norm": 22.625,
+      "learning_rate": 1.9672222222222222e-05,
+      "loss": 19.6133,
+      "mean_token_accuracy": 0.6910007946193218,
+      "num_tokens": 8977918.0,
+      "step": 460
+    },
+    {
+      "entropy": 1.2448308434337378,
+      "epoch": 0.752,
+      "grad_norm": 22.875,
+      "learning_rate": 1.9616666666666668e-05,
+      "loss": 20.0892,
+      "mean_token_accuracy": 0.688303318619728,
+      "num_tokens": 9169322.0,
+      "step": 470
+    },
+    {
+      "entropy": 1.2101924177259207,
+      "epoch": 0.768,
+      "grad_norm": 25.0,
+      "learning_rate": 1.9561111111111114e-05,
+      "loss": 19.4501,
+      "mean_token_accuracy": 0.6938675630837678,
+      "num_tokens": 9368141.0,
+      "step": 480
+    },
+    {
+      "entropy": 1.2336789827793837,
+      "epoch": 0.784,
+      "grad_norm": 25.125,
+      "learning_rate": 1.9505555555555556e-05,
+      "loss": 19.9195,
+      "mean_token_accuracy": 0.6904073975980282,
+      "num_tokens": 9565236.0,
+      "step": 490
+    },
+    {
+      "entropy": 1.2555153664201497,
+      "epoch": 0.8,
+      "grad_norm": 25.625,
+      "learning_rate": 1.9450000000000002e-05,
+      "loss": 20.3573,
+      "mean_token_accuracy": 0.6843966770917177,
+      "num_tokens": 9761227.0,
+      "step": 500
+    },
+    {
+      "epoch": 0.8,
+      "eval_biology_entropy": 1.2237661199569703,
+      "eval_biology_loss": 1.2929902076721191,
+      "eval_biology_mean_token_accuracy": 0.6812257404327393,
+      "eval_biology_num_tokens": 9761227.0,
+      "eval_biology_runtime": 47.3474,
+      "eval_biology_samples_per_second": 10.56,
+      "eval_biology_steps_per_second": 2.64,
+      "step": 500
+    },
+    {
+      "epoch": 0.8,
+      "eval_chemistry_entropy": 0.9801491451263428,
+      "eval_chemistry_loss": 1.0738970041275024,
+      "eval_chemistry_mean_token_accuracy": 0.7297394571304321,
+      "eval_chemistry_num_tokens": 9761227.0,
+      "eval_chemistry_runtime": 59.0653,
+      "eval_chemistry_samples_per_second": 8.465,
+      "eval_chemistry_steps_per_second": 2.116,
+      "step": 500
+    },
+    {
+      "epoch": 0.8,
+      "eval_math_entropy": 0.9275087294578552,
+      "eval_math_loss": 1.2924582958221436,
+      "eval_math_mean_token_accuracy": 0.7069363675117493,
+      "eval_math_num_tokens": 9761227.0,
+      "eval_math_runtime": 60.5026,
+      "eval_math_samples_per_second": 8.264,
+      "eval_math_steps_per_second": 2.066,
+      "step": 500
+    },
+    {
+      "epoch": 0.8,
+      "eval_physics_entropy": 1.0356942238807678,
+      "eval_physics_loss": 1.1801023483276367,
+      "eval_physics_mean_token_accuracy": 0.723124885559082,
+      "eval_physics_num_tokens": 9761227.0,
+      "eval_physics_runtime": 69.2781,
+      "eval_physics_samples_per_second": 7.217,
+      "eval_physics_steps_per_second": 1.804,
+      "step": 500
+    },
+    {
+      "entropy": 1.222337120398879,
+      "epoch": 0.816,
+      "grad_norm": 26.625,
+      "learning_rate": 1.9394444444444448e-05,
+      "loss": 19.6111,
+      "mean_token_accuracy": 0.691252876073122,
+      "num_tokens": 9958727.0,
+      "step": 510
+    },
+    {
+      "entropy": 1.2134733859449625,
+      "epoch": 0.832,
+      "grad_norm": 22.5,
+      "learning_rate": 1.933888888888889e-05,
+      "loss": 19.6687,
+      "mean_token_accuracy": 0.690800066664815,
+      "num_tokens": 10155771.0,
+      "step": 520
+    },
+    {
+      "entropy": 1.2252056252211332,
+      "epoch": 0.848,
+      "grad_norm": 24.75,
+      "learning_rate": 1.9283333333333332e-05,
+      "loss": 19.6088,
+      "mean_token_accuracy": 0.692672573402524,
+      "num_tokens": 10357721.0,
+      "step": 530
+    },
+    {
+      "entropy": 1.2455427911132575,
+      "epoch": 0.864,
+      "grad_norm": 23.75,
+      "learning_rate": 1.9227777777777778e-05,
+      "loss": 20.028,
+      "mean_token_accuracy": 0.6874771993607283,
+      "num_tokens": 10552495.0,
+      "step": 540
+    },
+    {
+      "entropy": 1.2235082875937224,
+      "epoch": 0.88,
+      "grad_norm": 25.125,
+      "learning_rate": 1.9172222222222224e-05,
+      "loss": 19.7291,
+      "mean_token_accuracy": 0.6916893098503352,
+      "num_tokens": 10748749.0,
+      "step": 550
+    },
+    {
+      "entropy": 1.2471147190779448,
+      "epoch": 0.896,
+      "grad_norm": 24.5,
+      "learning_rate": 1.911666666666667e-05,
+      "loss": 20.1356,
+      "mean_token_accuracy": 0.6882499437779188,
+      "num_tokens": 10943319.0,
+      "step": 560
+    },
+    {
+      "entropy": 1.2306789450347424,
+      "epoch": 0.912,
+      "grad_norm": 25.375,
+      "learning_rate": 1.9061111111111112e-05,
+      "loss": 19.9074,
+      "mean_token_accuracy": 0.6895338080823421,
+      "num_tokens": 11136935.0,
+      "step": 570
+    },
+    {
+      "entropy": 1.2452971942722797,
+      "epoch": 0.928,
+      "grad_norm": 23.875,
+      "learning_rate": 1.9005555555555557e-05,
+      "loss": 20.0341,
+      "mean_token_accuracy": 0.6869214996695518,
+      "num_tokens": 11331098.0,
+      "step": 580
+    },
+    {
+      "entropy": 1.1922047719359399,
+      "epoch": 0.944,
+      "grad_norm": 21.625,
+      "learning_rate": 1.8950000000000003e-05,
+      "loss": 19.322,
+      "mean_token_accuracy": 0.6946932781487704,
+      "num_tokens": 11530550.0,
+      "step": 590
+    },
+    {
+      "entropy": 1.226338443905115,
+      "epoch": 0.96,
+      "grad_norm": 26.125,
+      "learning_rate": 1.8894444444444446e-05,
+      "loss": 19.8277,
+      "mean_token_accuracy": 0.68995376303792,
+      "num_tokens": 11729645.0,
+      "step": 600
+    },
+    {
+      "epoch": 0.96,
+      "eval_biology_entropy": 1.2189953713417052,
+      "eval_biology_loss": 1.2870540618896484,
+      "eval_biology_mean_token_accuracy": 0.683146975517273,
+      "eval_biology_num_tokens": 11729645.0,
+      "eval_biology_runtime": 47.8218,
+      "eval_biology_samples_per_second": 10.455,
+      "eval_biology_steps_per_second": 2.614,
+      "step": 600
+    },
+    {
+      "epoch": 0.96,
+      "eval_chemistry_entropy": 0.976020311832428,
+      "eval_chemistry_loss": 1.0717357397079468,
+      "eval_chemistry_mean_token_accuracy": 0.7292985210418701,
+      "eval_chemistry_num_tokens": 11729645.0,
+      "eval_chemistry_runtime": 59.3311,
+      "eval_chemistry_samples_per_second": 8.427,
+      "eval_chemistry_steps_per_second": 2.107,
+      "step": 600
+    },
+    {
+      "epoch": 0.96,
+      "eval_math_entropy": 0.9220390543937683,
+      "eval_math_loss": 1.2949386835098267,
+      "eval_math_mean_token_accuracy": 0.7068283891677857,
+      "eval_math_num_tokens": 11729645.0,
+      "eval_math_runtime": 60.8254,
+      "eval_math_samples_per_second": 8.22,
+      "eval_math_steps_per_second": 2.055,
+      "step": 600
+    },
+    {
+      "epoch": 0.96,
+      "eval_physics_entropy": 1.0351580657958985,
+      "eval_physics_loss": 1.1791428327560425,
+      "eval_physics_mean_token_accuracy": 0.7229779815673828,
+      "eval_physics_num_tokens": 11729645.0,
+      "eval_physics_runtime": 69.3211,
+      "eval_physics_samples_per_second": 7.213,
+      "eval_physics_steps_per_second": 1.803,
+      "step": 600
+    },
+    {
+      "entropy": 1.2091780349612236,
+      "epoch": 0.976,
+      "grad_norm": 24.75,
+      "learning_rate": 1.883888888888889e-05,
+      "loss": 19.4529,
+      "mean_token_accuracy": 0.6921297512948513,
+      "num_tokens": 11924644.0,
+      "step": 610
+    },
+    {
+      "entropy": 1.2263813648372888,
+      "epoch": 0.992,
+      "grad_norm": 23.0,
+      "learning_rate": 1.8783333333333337e-05,
+      "loss": 19.9078,
+      "mean_token_accuracy": 0.6878966901451349,
+      "num_tokens": 12123059.0,
+      "step": 620
+    },
+    {
+      "entropy": 1.2347719121724368,
+      "epoch": 1.008,
+      "grad_norm": 21.75,
+      "learning_rate": 1.872777777777778e-05,
+      "loss": 19.7848,
+      "mean_token_accuracy": 0.6914879336953164,
+      "num_tokens": 12319366.0,
+      "step": 630
+    },
+    {
+      "entropy": 1.200057501345873,
+      "epoch": 1.024,
+      "grad_norm": 24.75,
+      "learning_rate": 1.8672222222222222e-05,
+      "loss": 19.2512,
+      "mean_token_accuracy": 0.6962312083691359,
+      "num_tokens": 12524183.0,
+      "step": 640
+    },
+    {
+      "entropy": 1.2318632639944553,
+      "epoch": 1.04,
+      "grad_norm": 23.125,
+      "learning_rate": 1.8616666666666667e-05,
+      "loss": 19.8582,
+      "mean_token_accuracy": 0.6885584607720375,
+      "num_tokens": 12718593.0,
+      "step": 650
+    },
+    {
+      "entropy": 1.1976032231003046,
+      "epoch": 1.056,
+      "grad_norm": 24.875,
+      "learning_rate": 1.8561111111111113e-05,
+      "loss": 19.2924,
+      "mean_token_accuracy": 0.697322390973568,
+      "num_tokens": 12917803.0,
+      "step": 660
+    },
+    {
+      "entropy": 1.223268049582839,
+      "epoch": 1.072,
+      "grad_norm": 26.125,
+      "learning_rate": 1.8505555555555556e-05,
+      "loss": 19.6292,
+      "mean_token_accuracy": 0.6911944530904293,
+      "num_tokens": 13105826.0,
+      "step": 670
+    },
+    {
+      "entropy": 1.2007815334945917,
+      "epoch": 1.088,
+      "grad_norm": 24.375,
+      "learning_rate": 1.845e-05,
+      "loss": 19.4022,
+      "mean_token_accuracy": 0.6958803411573171,
+      "num_tokens": 13298619.0,
+      "step": 680
+    },
+    {
+      "entropy": 1.2102480567991734,
+      "epoch": 1.104,
+      "grad_norm": 23.75,
+      "learning_rate": 1.8394444444444447e-05,
+      "loss": 19.551,
+      "mean_token_accuracy": 0.6936313558369875,
+      "num_tokens": 13491486.0,
+      "step": 690
+    },
+    {
+      "entropy": 1.2272425662726163,
+      "epoch": 1.12,
+      "grad_norm": 27.625,
+      "learning_rate": 1.833888888888889e-05,
+      "loss": 19.7097,
+      "mean_token_accuracy": 0.6926408246159553,
+      "num_tokens": 13674663.0,
+      "step": 700
+    },
+    {
+      "epoch": 1.12,
+      "eval_biology_entropy": 1.1717474513053894,
+      "eval_biology_loss": 1.2855128049850464,
+      "eval_biology_mean_token_accuracy": 0.683808819770813,
+      "eval_biology_num_tokens": 13674663.0,
+      "eval_biology_runtime": 47.8391,
+      "eval_biology_samples_per_second": 10.452,
+      "eval_biology_steps_per_second": 2.613,
+      "step": 700
+    },
+    {
+      "epoch": 1.12,
+      "eval_chemistry_entropy": 0.9375607762336731,
+      "eval_chemistry_loss": 1.0741384029388428,
+      "eval_chemistry_mean_token_accuracy": 0.7299527173042297,
+      "eval_chemistry_num_tokens": 13674663.0,
+      "eval_chemistry_runtime": 59.3574,
+      "eval_chemistry_samples_per_second": 8.424,
+      "eval_chemistry_steps_per_second": 2.106,
+      "step": 700
+    },
+    {
+      "epoch": 1.12,
+      "eval_math_entropy": 0.9116253685951233,
+      "eval_math_loss": 1.3001103401184082,
+      "eval_math_mean_token_accuracy": 0.7062767610549927,
+      "eval_math_num_tokens": 13674663.0,
+      "eval_math_runtime": 60.8743,
+      "eval_math_samples_per_second": 8.214,
+      "eval_math_steps_per_second": 2.053,
+      "step": 700
+    },
+    {
+      "epoch": 1.12,
+      "eval_physics_entropy": 0.997166962146759,
+      "eval_physics_loss": 1.1788647174835205,
+      "eval_physics_mean_token_accuracy": 0.7233795509338379,
+      "eval_physics_num_tokens": 13674663.0,
+      "eval_physics_runtime": 69.3947,
+      "eval_physics_samples_per_second": 7.205,
+      "eval_physics_steps_per_second": 1.801,
+      "step": 700
+    },
+    {
+      "entropy": 1.2040644731372594,
+      "epoch": 1.1360000000000001,
+      "grad_norm": 25.75,
+      "learning_rate": 1.8283333333333335e-05,
+      "loss": 19.5211,
+      "mean_token_accuracy": 0.6950324695557356,
+      "num_tokens": 13869134.0,
+      "step": 710
+    },
+    {
+      "entropy": 1.1908101249486207,
+      "epoch": 1.152,
+      "grad_norm": 22.75,
+      "learning_rate": 1.822777777777778e-05,
+      "loss": 19.1582,
+      "mean_token_accuracy": 0.6968125693500042,
+      "num_tokens": 14078365.0,
+      "step": 720
+    },
+    {
+      "entropy": 1.2142638489603996,
+      "epoch": 1.168,
+      "grad_norm": 22.875,
+      "learning_rate": 1.8172222222222223e-05,
+      "loss": 19.4965,
+      "mean_token_accuracy": 0.6936378438025713,
+      "num_tokens": 14266831.0,
+      "step": 730
+    },
+    {
+      "entropy": 1.2091089431196451,
+      "epoch": 1.184,
+      "grad_norm": 24.875,
+      "learning_rate": 1.811666666666667e-05,
+      "loss": 19.6041,
+      "mean_token_accuracy": 0.6928943891078234,
+      "num_tokens": 14465660.0,
+      "step": 740
+    },
+    {
+      "entropy": 1.2094362247735262,
+      "epoch": 1.2,
+      "grad_norm": 25.375,
+      "learning_rate": 1.806111111111111e-05,
+      "loss": 19.3313,
+      "mean_token_accuracy": 0.6956093374639749,
+      "num_tokens": 14653228.0,
+      "step": 750
+    },
+    {
+      "entropy": 1.172793022543192,
+      "epoch": 1.216,
+      "grad_norm": 24.125,
+      "learning_rate": 1.8005555555555557e-05,
+      "loss": 18.9709,
+      "mean_token_accuracy": 0.698998348787427,
+      "num_tokens": 14857782.0,
+      "step": 760
+    },
+    {
+      "entropy": 1.1956025019288063,
+      "epoch": 1.232,
+      "grad_norm": 22.25,
+      "learning_rate": 1.795e-05,
+      "loss": 19.2039,
+      "mean_token_accuracy": 0.6992277778685093,
+      "num_tokens": 15047356.0,
+      "step": 770
+    },
+    {
+      "entropy": 1.1972672935575246,
+      "epoch": 1.248,
+      "grad_norm": 24.25,
+      "learning_rate": 1.7894444444444445e-05,
+      "loss": 19.4124,
+      "mean_token_accuracy": 0.6940966844558716,
+      "num_tokens": 15241098.0,
+      "step": 780
+    },
+    {
+      "entropy": 1.2250222776085138,
+      "epoch": 1.264,
+      "grad_norm": 23.0,
+      "learning_rate": 1.783888888888889e-05,
+      "loss": 19.6582,
+      "mean_token_accuracy": 0.6926792357116938,
+      "num_tokens": 15437657.0,
+      "step": 790
+    },
+    {
+      "entropy": 1.2299235615879298,
+      "epoch": 1.28,
+      "grad_norm": 25.125,
+      "learning_rate": 1.7783333333333333e-05,
+      "loss": 19.9054,
+      "mean_token_accuracy": 0.6879028372466565,
+      "num_tokens": 15630795.0,
+      "step": 800
+    },
+    {
+      "epoch": 1.28,
+      "eval_biology_entropy": 1.1756790585517882,
+      "eval_biology_loss": 1.2825103998184204,
+      "eval_biology_mean_token_accuracy": 0.6839104061126708,
+      "eval_biology_num_tokens": 15630795.0,
+      "eval_biology_runtime": 47.8419,
+      "eval_biology_samples_per_second": 10.451,
+      "eval_biology_steps_per_second": 2.613,
+      "step": 800
+    },
+    {
+      "epoch": 1.28,
+      "eval_chemistry_entropy": 0.9409222221374511,
+      "eval_chemistry_loss": 1.0712758302688599,
+      "eval_chemistry_mean_token_accuracy": 0.7309756746292114,
+      "eval_chemistry_num_tokens": 15630795.0,
+      "eval_chemistry_runtime": 59.3395,
+      "eval_chemistry_samples_per_second": 8.426,
+      "eval_chemistry_steps_per_second": 2.107,
+      "step": 800
+    },
+    {
+      "epoch": 1.28,
+      "eval_math_entropy": 0.9131695604324341,
+      "eval_math_loss": 1.2970302104949951,
+      "eval_math_mean_token_accuracy": 0.7068103790283203,
+      "eval_math_num_tokens": 15630795.0,
+      "eval_math_runtime": 60.8564,
+      "eval_math_samples_per_second": 8.216,
+      "eval_math_steps_per_second": 2.054,
+      "step": 800
+    },
+    {
+      "epoch": 1.28,
+      "eval_physics_entropy": 1.0009545674324036,
+      "eval_physics_loss": 1.1740305423736572,
+      "eval_physics_mean_token_accuracy": 0.7239470205307007,
+      "eval_physics_num_tokens": 15630795.0,
+      "eval_physics_runtime": 69.3435,
+      "eval_physics_samples_per_second": 7.21,
+      "eval_physics_steps_per_second": 1.803,
+      "step": 800
+    },
+    {
+      "entropy": 1.2318371035158635,
+      "epoch": 1.296,
+      "grad_norm": 23.625,
+      "learning_rate": 1.772777777777778e-05,
+      "loss": 19.812,
+      "mean_token_accuracy": 0.6904978167265654,
+      "num_tokens": 15827105.0,
+      "step": 810
+    },
+    {
+      "entropy": 1.2156631674617528,
+      "epoch": 1.312,
+      "grad_norm": 24.25,
+      "learning_rate": 1.7672222222222224e-05,
+      "loss": 19.7293,
+      "mean_token_accuracy": 0.6915876884013414,
+      "num_tokens": 16019645.0,
+      "step": 820
+    },
+    {
+      "entropy": 1.2210196305066348,
+      "epoch": 1.328,
+      "grad_norm": 23.875,
+      "learning_rate": 1.761666666666667e-05,
+      "loss": 19.7658,
+      "mean_token_accuracy": 0.6908771209418774,
+      "num_tokens": 16221726.0,
+      "step": 830
+    },
+    {
+      "entropy": 1.1998186014592647,
+      "epoch": 1.3439999999999999,
+      "grad_norm": 22.875,
+      "learning_rate": 1.7561111111111113e-05,
+      "loss": 19.2177,
+      "mean_token_accuracy": 0.6973779436200858,
+      "num_tokens": 16427594.0,
+      "step": 840
+    },
+    {
+      "entropy": 1.206191548332572,
+      "epoch": 1.3599999999999999,
+      "grad_norm": 25.25,
+      "learning_rate": 1.7505555555555558e-05,
+      "loss": 19.5695,
+      "mean_token_accuracy": 0.6934758294373751,
+      "num_tokens": 16621605.0,
+      "step": 850
+    },
+    {
+      "entropy": 1.2538085140287876,
+      "epoch": 1.376,
+      "grad_norm": 26.25,
+      "learning_rate": 1.7450000000000004e-05,
+      "loss": 20.1019,
+      "mean_token_accuracy": 0.6873533941805363,
+      "num_tokens": 16813444.0,
+      "step": 860
+    },
+    {
+      "entropy": 1.2129262063652277,
+      "epoch": 1.392,
+      "grad_norm": 23.75,
+      "learning_rate": 1.7394444444444446e-05,
+      "loss": 19.7008,
+      "mean_token_accuracy": 0.6932430669665337,
+      "num_tokens": 17006509.0,
+      "step": 870
+    },
+    {
+      "entropy": 1.2133032407611608,
+      "epoch": 1.408,
+      "grad_norm": 21.125,
+      "learning_rate": 1.733888888888889e-05,
+      "loss": 19.4459,
+      "mean_token_accuracy": 0.6937982514500618,
+      "num_tokens": 17197870.0,
+      "step": 880
+    },
+    {
+      "entropy": 1.2197621572762727,
+      "epoch": 1.424,
+      "grad_norm": 24.375,
+      "learning_rate": 1.7283333333333334e-05,
+      "loss": 19.6331,
+      "mean_token_accuracy": 0.692104072868824,
+      "num_tokens": 17394390.0,
+      "step": 890
+    },
+    {
+      "entropy": 1.219193383306265,
+      "epoch": 1.44,
+      "grad_norm": 28.75,
+      "learning_rate": 1.722777777777778e-05,
+      "loss": 19.6606,
+      "mean_token_accuracy": 0.6919787086546421,
+      "num_tokens": 17587777.0,
+      "step": 900
+    },
+    {
+      "epoch": 1.44,
+      "eval_biology_entropy": 1.2083894481658937,
+      "eval_biology_loss": 1.2752834558486938,
+      "eval_biology_mean_token_accuracy": 0.6854755258560181,
+      "eval_biology_num_tokens": 17587777.0,
+      "eval_biology_runtime": 47.7503,
+      "eval_biology_samples_per_second": 10.471,
+      "eval_biology_steps_per_second": 2.618,
+      "step": 900
+    },
+    {
+      "epoch": 1.44,
+      "eval_chemistry_entropy": 0.9671227583885192,
+      "eval_chemistry_loss": 1.0685635805130005,
+      "eval_chemistry_mean_token_accuracy": 0.7308610730171203,
+      "eval_chemistry_num_tokens": 17587777.0,
+      "eval_chemistry_runtime": 59.2536,
+      "eval_chemistry_samples_per_second": 8.438,
+      "eval_chemistry_steps_per_second": 2.11,
+      "step": 900
+    },
+    {
+      "epoch": 1.44,
+      "eval_math_entropy": 0.9232685470581055,
+      "eval_math_loss": 1.2879754304885864,
+      "eval_math_mean_token_accuracy": 0.7072285838127136,
+      "eval_math_num_tokens": 17587777.0,
+      "eval_math_runtime": 60.7715,
+      "eval_math_samples_per_second": 8.228,
+      "eval_math_steps_per_second": 2.057,
+      "step": 900
+    },
+    {
+      "epoch": 1.44,
+      "eval_physics_entropy": 1.0261503925323487,
+      "eval_physics_loss": 1.1721413135528564,
+      "eval_physics_mean_token_accuracy": 0.7246381421089172,
+      "eval_physics_num_tokens": 17587777.0,
+      "eval_physics_runtime": 69.2559,
+      "eval_physics_samples_per_second": 7.22,
+      "eval_physics_steps_per_second": 1.805,
+      "step": 900
+    },
+    {
+      "entropy": 1.2059034764766694,
+      "epoch": 1.456,
+      "grad_norm": 25.125,
+      "learning_rate": 1.7172222222222223e-05,
+      "loss": 19.5801,
+      "mean_token_accuracy": 0.6930746156722307,
+      "num_tokens": 17788456.0,
+      "step": 910
+    },
+    {
+      "entropy": 1.2149245291948318,
+      "epoch": 1.472,
+      "grad_norm": 22.375,
+      "learning_rate": 1.7116666666666668e-05,
+      "loss": 19.4506,
+      "mean_token_accuracy": 0.6947890192270278,
+      "num_tokens": 17984063.0,
+      "step": 920
+    },
+    {
+      "entropy": 1.2022255394607781,
+      "epoch": 1.488,
+      "grad_norm": 23.5,
+      "learning_rate": 1.7061111111111114e-05,
+      "loss": 19.508,
+      "mean_token_accuracy": 0.695648743584752,
+      "num_tokens": 18175640.0,
+      "step": 930
+    },
+    {
+      "entropy": 1.2195066373795271,
+      "epoch": 1.504,
+      "grad_norm": 22.875,
+      "learning_rate": 1.7005555555555556e-05,
+      "loss": 19.7359,
+      "mean_token_accuracy": 0.6924235183745623,
+      "num_tokens": 18367857.0,
+      "step": 940
+    },
+    {
+      "entropy": 1.2132650747895242,
+      "epoch": 1.52,
+      "grad_norm": 22.125,
+      "learning_rate": 1.6950000000000002e-05,
+      "loss": 19.5773,
+      "mean_token_accuracy": 0.6921011231839657,
+      "num_tokens": 18569146.0,
+      "step": 950
+    },
+    {
+      "entropy": 1.233898864686489,
+      "epoch": 1.536,
+      "grad_norm": 25.25,
+      "learning_rate": 1.6894444444444448e-05,
+      "loss": 19.8446,
+      "mean_token_accuracy": 0.6897176470607519,
+      "num_tokens": 18755079.0,
+      "step": 960
+    },
+    {
+      "entropy": 1.2005723245441913,
+      "epoch": 1.552,
+      "grad_norm": 25.5,
+      "learning_rate": 1.683888888888889e-05,
+      "loss": 19.4576,
+      "mean_token_accuracy": 0.6952799737453461,
+      "num_tokens": 18956248.0,
+      "step": 970
+    },
+    {
+      "entropy": 1.1982890646904707,
+      "epoch": 1.568,
+      "grad_norm": 24.625,
+      "learning_rate": 1.6783333333333332e-05,
+      "loss": 19.3351,
+      "mean_token_accuracy": 0.696817371249199,
+      "num_tokens": 19150315.0,
+      "step": 980
+    },
+    {
+      "entropy": 1.212613246589899,
+      "epoch": 1.584,
+      "grad_norm": 24.375,
+      "learning_rate": 1.6727777777777778e-05,
+      "loss": 19.4591,
+      "mean_token_accuracy": 0.6943255145102739,
+      "num_tokens": 19344260.0,
+      "step": 990
+    },
+    {
+      "entropy": 1.237872240692377,
+      "epoch": 1.6,
+      "grad_norm": 23.5,
+      "learning_rate": 1.6672222222222224e-05,
+      "loss": 19.9617,
+      "mean_token_accuracy": 0.6886349719017744,
+      "num_tokens": 19532552.0,
+      "step": 1000
+    },
+    {
+      "epoch": 1.6,
+      "eval_biology_entropy": 1.1855721821784972,
+      "eval_biology_loss": 1.2722135782241821,
+      "eval_biology_mean_token_accuracy": 0.6856484136581421,
+      "eval_biology_num_tokens": 19532552.0,
+      "eval_biology_runtime": 47.7667,
+      "eval_biology_samples_per_second": 10.468,
+      "eval_biology_steps_per_second": 2.617,
+      "step": 1000
+    },
+    {
+      "epoch": 1.6,
+      "eval_chemistry_entropy": 0.9456389331817627,
+      "eval_chemistry_loss": 1.0643842220306396,
+      "eval_chemistry_mean_token_accuracy": 0.7323586492538452,
+      "eval_chemistry_num_tokens": 19532552.0,
+      "eval_chemistry_runtime": 59.0935,
+      "eval_chemistry_samples_per_second": 8.461,
+      "eval_chemistry_steps_per_second": 2.115,
+      "step": 1000
+    },
+    {
+      "epoch": 1.6,
+      "eval_math_entropy": 0.9028137063980103,
+      "eval_math_loss": 1.284372091293335,
+      "eval_math_mean_token_accuracy": 0.7095083913803101,
+      "eval_math_num_tokens": 19532552.0,
+      "eval_math_runtime": 60.7834,
+      "eval_math_samples_per_second": 8.226,
+      "eval_math_steps_per_second": 2.056,
+      "step": 1000
+    },
+    {
+      "epoch": 1.6,
+      "eval_physics_entropy": 0.9996754899024963,
+      "eval_physics_loss": 1.1675664186477661,
+      "eval_physics_mean_token_accuracy": 0.7260684304237366,
+      "eval_physics_num_tokens": 19532552.0,
+      "eval_physics_runtime": 69.2773,
+      "eval_physics_samples_per_second": 7.217,
+      "eval_physics_steps_per_second": 1.804,
+      "step": 1000
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 4000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 7,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.2020838985302374e+18,
+  "train_batch_size": 4,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-1000/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0d39621da3b5989e7ba65c3a57882f45d803db1b84d218aa54b7e14a9a65a314
+size 6545