Instructions to use roonbug/b5e92jps with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use roonbug/b5e92jps with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="roonbug/b5e92jps")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("roonbug/b5e92jps")
model = AutoModelForImageTextToText.from_pretrained("roonbug/b5e92jps")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use roonbug/b5e92jps with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "roonbug/b5e92jps"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/b5e92jps",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/roonbug/b5e92jps

SGLang

How to use roonbug/b5e92jps with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "roonbug/b5e92jps" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/b5e92jps",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "roonbug/b5e92jps" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "roonbug/b5e92jps",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use roonbug/b5e92jps with Docker Model Runner:
```
docker model run hf.co/roonbug/b5e92jps
```

roonbug commited on May 2

Commit

059f48b

verified ·

1 Parent(s): 720e695

Upload folder using huggingface_hub

Browse files

Files changed (20) hide show

.gitattributes +1 -0
checkpoint-500/added_tokens.json +3 -0
checkpoint-500/chat_template.jinja +47 -0
checkpoint-500/config.json +110 -0
checkpoint-500/generation_config.json +13 -0
checkpoint-500/model-00001-of-00005.safetensors +3 -0
checkpoint-500/model-00002-of-00005.safetensors +3 -0
checkpoint-500/model-00003-of-00005.safetensors +3 -0
checkpoint-500/model-00004-of-00005.safetensors +3 -0
checkpoint-500/model-00005-of-00005.safetensors +3 -0
checkpoint-500/model.safetensors.index.json +0 -0
checkpoint-500/optimizer.pt +3 -0
checkpoint-500/rng_state.pth +3 -0
checkpoint-500/scheduler.pt +3 -0
checkpoint-500/special_tokens_map.json +33 -0
checkpoint-500/tokenizer.json +3 -0
checkpoint-500/tokenizer.model +3 -0
checkpoint-500/tokenizer_config.json +0 -0
checkpoint-500/trainer_state.json +754 -0
checkpoint-500/training_args.bin +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+checkpoint-500/tokenizer.json filter=lfs diff=lfs merge=lfs -text

checkpoint-500/added_tokens.json ADDED Viewed

	@@ -0,0 +1,3 @@

+{
+  "<image_soft_token>": 262144
+}

checkpoint-500/chat_template.jinja ADDED Viewed

	@@ -0,0 +1,47 @@

+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

checkpoint-500/config.json ADDED Viewed

	@@ -0,0 +1,110 @@

+{
+  "architectures": [
+    "Gemma3ForConditionalGeneration"
+  ],
+  "boi_token_index": 255999,
+  "bos_token_id": 2,
+  "dtype": "bfloat16",
+  "eoi_token_index": 256000,
+  "eos_token_id": 1,
+  "image_token_index": 262144,
+  "initializer_range": 0.02,
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "pad_token_id": 0,
+  "text_config": {
+    "_sliding_window_pattern": 6,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "dtype": "bfloat16",
+    "final_logit_softcapping": null,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 3840,
+    "initializer_range": 0.02,
+    "intermediate_size": 15360,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 48,
+    "num_key_value_heads": 8,
+    "query_pre_attn_scalar": 256,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000.0,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": 1024,
+    "use_cache": true,
+    "vocab_size": 262208
+  },
+  "transformers_version": "4.56.1",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "bfloat16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "vision_use_head": false
+  }
+}

checkpoint-500/generation_config.json ADDED Viewed

	@@ -0,0 +1,13 @@

+{
+  "bos_token_id": 2,
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.56.1"
+}

checkpoint-500/model-00001-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4847447e92599833e8dbaa3067cd201c3bb5c052efa91f11ba891e43234f7832
+size 4979902192

checkpoint-500/model-00002-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:891bd54eed03cba9ee1e705533a02a8217fcc29f356e4a1f53e5fd0d178883ad
+size 4931296592

checkpoint-500/model-00003-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:797be1d5febe5f157baa576743fc6a2974c55982ac5eb7354909963d33647035
+size 4931296656

checkpoint-500/model-00004-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d37900b71893ef07dd64964a0b6ac649bc069922d0033fd748f249a6c55acf21
+size 4931296656

checkpoint-500/model-00005-of-00005.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f050c074704794833ffdf6ebe051067a08c909a2acca599f4f80aae20ce2d8ab
+size 4601000928

checkpoint-500/model.safetensors.index.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-500/optimizer.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:68a6b1dcbacbec428e191b64ec06d582e83a4256a18144b192d9e8196d4a5741
+size 14573116445

checkpoint-500/rng_state.pth ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fde986f97ecc270a34049538e5bcb495edbc4e7f5914a76b3cad083de287dc73
+size 14645

checkpoint-500/scheduler.pt ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:63481e620bd287eefb582d20daf306c54c7268088fea6f19af8100945c7e3e7e
+size 1465

checkpoint-500/special_tokens_map.json ADDED Viewed

	@@ -0,0 +1,33 @@

+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": {
+    "content": "<pad>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-500/tokenizer.json ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:4667f2089529e8e7657cfb6d1c19910ae71ff5f28aa7ab2ff2763330affad795
+size 33384568

checkpoint-500/tokenizer.model ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074

checkpoint-500/tokenizer_config.json ADDED Viewed

The diff for this file is too large to render. See raw diff

checkpoint-500/trainer_state.json ADDED Viewed

	@@ -0,0 +1,754 @@

+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.22255605630668224,
+  "eval_steps": 100,
+  "global_step": 500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "entropy": 0.7622424783185124,
+      "epoch": 0.004451121126133645,
+      "grad_norm": 1888.0,
+      "learning_rate": 1.8e-07,
+      "loss": 36.8185,
+      "mean_token_accuracy": 0.6663544788956642,
+      "num_tokens": 63133.0,
+      "step": 10
+    },
+    {
+      "entropy": 0.8160548411309719,
+      "epoch": 0.00890224225226729,
+      "grad_norm": 1728.0,
+      "learning_rate": 3.8e-07,
+      "loss": 36.6381,
+      "mean_token_accuracy": 0.649208546616137,
+      "num_tokens": 128968.0,
+      "step": 20
+    },
+    {
+      "entropy": 0.8332255953922868,
+      "epoch": 0.013353363378400934,
+      "grad_norm": 948.0,
+      "learning_rate": 5.800000000000001e-07,
+      "loss": 34.7797,
+      "mean_token_accuracy": 0.6578839531168341,
+      "num_tokens": 193314.0,
+      "step": 30
+    },
+    {
+      "entropy": 0.9456521481275558,
+      "epoch": 0.01780448450453458,
+      "grad_norm": 716.0,
+      "learning_rate": 7.8e-07,
+      "loss": 34.1076,
+      "mean_token_accuracy": 0.6613732609897852,
+      "num_tokens": 257189.0,
+      "step": 40
+    },
+    {
+      "entropy": 0.9749895730987191,
+      "epoch": 0.022255605630668224,
+      "grad_norm": 660.0,
+      "learning_rate": 9.800000000000001e-07,
+      "loss": 29.937,
+      "mean_token_accuracy": 0.6729008939117194,
+      "num_tokens": 324224.0,
+      "step": 50
+    },
+    {
+      "entropy": 1.0305172327905894,
+      "epoch": 0.026706726756801868,
+      "grad_norm": 492.0,
+      "learning_rate": 1.1800000000000001e-06,
+      "loss": 28.3561,
+      "mean_token_accuracy": 0.680633837915957,
+      "num_tokens": 394654.0,
+      "step": 60
+    },
+    {
+      "entropy": 1.0123192116618156,
+      "epoch": 0.031157847882935515,
+      "grad_norm": 404.0,
+      "learning_rate": 1.3800000000000001e-06,
+      "loss": 26.0449,
+      "mean_token_accuracy": 0.6903208963572979,
+      "num_tokens": 461658.0,
+      "step": 70
+    },
+    {
+      "entropy": 1.024911842495203,
+      "epoch": 0.03560896900906916,
+      "grad_norm": 250.0,
+      "learning_rate": 1.5800000000000001e-06,
+      "loss": 23.6986,
+      "mean_token_accuracy": 0.6996766375377774,
+      "num_tokens": 528285.0,
+      "step": 80
+    },
+    {
+      "entropy": 1.0939506959170102,
+      "epoch": 0.04006009013520281,
+      "grad_norm": 227.0,
+      "learning_rate": 1.7800000000000001e-06,
+      "loss": 23.2949,
+      "mean_token_accuracy": 0.6969541085883975,
+      "num_tokens": 595336.0,
+      "step": 90
+    },
+    {
+      "entropy": 1.0676697574555873,
+      "epoch": 0.04451121126133645,
+      "grad_norm": 278.0,
+      "learning_rate": 1.98e-06,
+      "loss": 21.5533,
+      "mean_token_accuracy": 0.7063678905367852,
+      "num_tokens": 663126.0,
+      "step": 100
+    },
+    {
+      "epoch": 0.04451121126133645,
+      "eval_biology_entropy": 1.5063953498601914,
+      "eval_biology_loss": 1.9373619556427002,
+      "eval_biology_mean_token_accuracy": 0.6030384682416916,
+      "eval_biology_num_tokens": 663126.0,
+      "eval_biology_runtime": 37.2317,
+      "eval_biology_samples_per_second": 13.429,
+      "eval_biology_steps_per_second": 13.429,
+      "step": 100
+    },
+    {
+      "epoch": 0.04451121126133645,
+      "eval_chemistry_entropy": 1.0538504299521447,
+      "eval_chemistry_loss": 1.2949438095092773,
+      "eval_chemistry_mean_token_accuracy": 0.7114702309370041,
+      "eval_chemistry_num_tokens": 663126.0,
+      "eval_chemistry_runtime": 41.1049,
+      "eval_chemistry_samples_per_second": 12.164,
+      "eval_chemistry_steps_per_second": 12.164,
+      "step": 100
+    },
+    {
+      "epoch": 0.04451121126133645,
+      "eval_math_entropy": 0.8524410739541054,
+      "eval_math_loss": 1.2029013633728027,
+      "eval_math_mean_token_accuracy": 0.7322385207414627,
+      "eval_math_num_tokens": 663126.0,
+      "eval_math_runtime": 41.5152,
+      "eval_math_samples_per_second": 12.044,
+      "eval_math_steps_per_second": 12.044,
+      "step": 100
+    },
+    {
+      "epoch": 0.04451121126133645,
+      "eval_physics_entropy": 0.9442882173061371,
+      "eval_physics_loss": 1.1572656631469727,
+      "eval_physics_mean_token_accuracy": 0.7343679416179657,
+      "eval_physics_num_tokens": 663126.0,
+      "eval_physics_runtime": 45.8024,
+      "eval_physics_samples_per_second": 10.916,
+      "eval_physics_steps_per_second": 10.916,
+      "step": 100
+    },
+    {
+      "entropy": 1.0567705381661654,
+      "epoch": 0.048962332387470095,
+      "grad_norm": 202.0,
+      "learning_rate": 2.1800000000000003e-06,
+      "loss": 20.0858,
+      "mean_token_accuracy": 0.7180810730904341,
+      "num_tokens": 731060.0,
+      "step": 110
+    },
+    {
+      "entropy": 1.0101547280326486,
+      "epoch": 0.053413453513603736,
+      "grad_norm": 202.0,
+      "learning_rate": 2.38e-06,
+      "loss": 18.8763,
+      "mean_token_accuracy": 0.7310481067746878,
+      "num_tokens": 801635.0,
+      "step": 120
+    },
+    {
+      "entropy": 1.0049639470875262,
+      "epoch": 0.05786457463973738,
+      "grad_norm": 154.0,
+      "learning_rate": 2.5800000000000003e-06,
+      "loss": 19.1729,
+      "mean_token_accuracy": 0.7293503576889634,
+      "num_tokens": 867260.0,
+      "step": 130
+    },
+    {
+      "entropy": 1.0176106113940477,
+      "epoch": 0.06231569576587103,
+      "grad_norm": 144.0,
+      "learning_rate": 2.7800000000000005e-06,
+      "loss": 18.97,
+      "mean_token_accuracy": 0.7298020845279097,
+      "num_tokens": 931344.0,
+      "step": 140
+    },
+    {
+      "entropy": 1.0698495015501976,
+      "epoch": 0.06676681689200467,
+      "grad_norm": 169.0,
+      "learning_rate": 2.9800000000000003e-06,
+      "loss": 17.4943,
+      "mean_token_accuracy": 0.7370716020464897,
+      "num_tokens": 993586.0,
+      "step": 150
+    },
+    {
+      "entropy": 1.0634983357042074,
+      "epoch": 0.07121793801813832,
+      "grad_norm": 123.0,
+      "learning_rate": 3.1800000000000005e-06,
+      "loss": 16.9373,
+      "mean_token_accuracy": 0.7367568526417017,
+      "num_tokens": 1060812.0,
+      "step": 160
+    },
+    {
+      "entropy": 1.0108168195933103,
+      "epoch": 0.07566905914427197,
+      "grad_norm": 125.0,
+      "learning_rate": 3.3800000000000007e-06,
+      "loss": 15.569,
+      "mean_token_accuracy": 0.7519758876413107,
+      "num_tokens": 1131832.0,
+      "step": 170
+    },
+    {
+      "entropy": 0.9829567475244403,
+      "epoch": 0.08012018027040561,
+      "grad_norm": 99.0,
+      "learning_rate": 3.58e-06,
+      "loss": 15.2082,
+      "mean_token_accuracy": 0.7475447814911604,
+      "num_tokens": 1200650.0,
+      "step": 180
+    },
+    {
+      "entropy": 0.9538513226434588,
+      "epoch": 0.08457130139653925,
+      "grad_norm": 119.0,
+      "learning_rate": 3.7800000000000002e-06,
+      "loss": 15.0014,
+      "mean_token_accuracy": 0.7515964422374963,
+      "num_tokens": 1265107.0,
+      "step": 190
+    },
+    {
+      "entropy": 0.9125228714197874,
+      "epoch": 0.0890224225226729,
+      "grad_norm": 117.0,
+      "learning_rate": 3.980000000000001e-06,
+      "loss": 14.4833,
+      "mean_token_accuracy": 0.7538308618590236,
+      "num_tokens": 1327380.0,
+      "step": 200
+    },
+    {
+      "epoch": 0.0890224225226729,
+      "eval_biology_entropy": 1.280277446448803,
+      "eval_biology_loss": 1.453066349029541,
+      "eval_biology_mean_token_accuracy": 0.6518876985907555,
+      "eval_biology_num_tokens": 1327380.0,
+      "eval_biology_runtime": 36.821,
+      "eval_biology_samples_per_second": 13.579,
+      "eval_biology_steps_per_second": 13.579,
+      "step": 200
+    },
+    {
+      "epoch": 0.0890224225226729,
+      "eval_chemistry_entropy": 0.8954760466217995,
+      "eval_chemistry_loss": 0.9106767177581787,
+      "eval_chemistry_mean_token_accuracy": 0.7551076735854149,
+      "eval_chemistry_num_tokens": 1327380.0,
+      "eval_chemistry_runtime": 40.4689,
+      "eval_chemistry_samples_per_second": 12.355,
+      "eval_chemistry_steps_per_second": 12.355,
+      "step": 200
+    },
+    {
+      "epoch": 0.0890224225226729,
+      "eval_math_entropy": 0.7658050524294376,
+      "eval_math_loss": 0.9040815234184265,
+      "eval_math_mean_token_accuracy": 0.7642563463449478,
+      "eval_math_num_tokens": 1327380.0,
+      "eval_math_runtime": 41.4556,
+      "eval_math_samples_per_second": 12.061,
+      "eval_math_steps_per_second": 12.061,
+      "step": 200
+    },
+    {
+      "epoch": 0.0890224225226729,
+      "eval_physics_entropy": 0.8174435417056084,
+      "eval_physics_loss": 0.8484614491462708,
+      "eval_physics_mean_token_accuracy": 0.7708570960760116,
+      "eval_physics_num_tokens": 1327380.0,
+      "eval_physics_runtime": 45.875,
+      "eval_physics_samples_per_second": 10.899,
+      "eval_physics_steps_per_second": 10.899,
+      "step": 200
+    },
+    {
+      "entropy": 0.9408239619806409,
+      "epoch": 0.09347354364880654,
+      "grad_norm": 126.5,
+      "learning_rate": 4.18e-06,
+      "loss": 15.2387,
+      "mean_token_accuracy": 0.7457781180739402,
+      "num_tokens": 1393379.0,
+      "step": 210
+    },
+    {
+      "entropy": 0.8950148992240429,
+      "epoch": 0.09792466477494019,
+      "grad_norm": 95.0,
+      "learning_rate": 4.38e-06,
+      "loss": 14.2337,
+      "mean_token_accuracy": 0.7587446294724941,
+      "num_tokens": 1460130.0,
+      "step": 220
+    },
+    {
+      "entropy": 0.859116705879569,
+      "epoch": 0.10237578590107384,
+      "grad_norm": 94.0,
+      "learning_rate": 4.58e-06,
+      "loss": 13.7082,
+      "mean_token_accuracy": 0.76347241923213,
+      "num_tokens": 1529183.0,
+      "step": 230
+    },
+    {
+      "entropy": 0.8980951353907585,
+      "epoch": 0.10682690702720747,
+      "grad_norm": 118.5,
+      "learning_rate": 4.78e-06,
+      "loss": 14.5897,
+      "mean_token_accuracy": 0.7553087200969457,
+      "num_tokens": 1597405.0,
+      "step": 240
+    },
+    {
+      "entropy": 0.9019628280773759,
+      "epoch": 0.11127802815334112,
+      "grad_norm": 98.5,
+      "learning_rate": 4.980000000000001e-06,
+      "loss": 14.5049,
+      "mean_token_accuracy": 0.7539074700325727,
+      "num_tokens": 1661767.0,
+      "step": 250
+    },
+    {
+      "entropy": 0.9095736019313335,
+      "epoch": 0.11572914927947477,
+      "grad_norm": 80.0,
+      "learning_rate": 5.18e-06,
+      "loss": 14.553,
+      "mean_token_accuracy": 0.7567780796438456,
+      "num_tokens": 1728207.0,
+      "step": 260
+    },
+    {
+      "entropy": 0.855735239200294,
+      "epoch": 0.12018027040560841,
+      "grad_norm": 99.0,
+      "learning_rate": 5.380000000000001e-06,
+      "loss": 13.731,
+      "mean_token_accuracy": 0.7652106903493404,
+      "num_tokens": 1796055.0,
+      "step": 270
+    },
+    {
+      "entropy": 0.830021676234901,
+      "epoch": 0.12463139153174206,
+      "grad_norm": 97.5,
+      "learning_rate": 5.580000000000001e-06,
+      "loss": 13.2554,
+      "mean_token_accuracy": 0.7721017152070999,
+      "num_tokens": 1859684.0,
+      "step": 280
+    },
+    {
+      "entropy": 0.8921441739425063,
+      "epoch": 0.1290825126578757,
+      "grad_norm": 106.5,
+      "learning_rate": 5.78e-06,
+      "loss": 14.4012,
+      "mean_token_accuracy": 0.7561684042215348,
+      "num_tokens": 1921704.0,
+      "step": 290
+    },
+    {
+      "entropy": 0.8731818461790681,
+      "epoch": 0.13353363378400934,
+      "grad_norm": 87.5,
+      "learning_rate": 5.98e-06,
+      "loss": 13.9502,
+      "mean_token_accuracy": 0.7628704030066729,
+      "num_tokens": 1985766.0,
+      "step": 300
+    },
+    {
+      "epoch": 0.13353363378400934,
+      "eval_biology_entropy": 1.2637426958680154,
+      "eval_biology_loss": 1.3796980381011963,
+      "eval_biology_mean_token_accuracy": 0.6648430502414704,
+      "eval_biology_num_tokens": 1985766.0,
+      "eval_biology_runtime": 37.0397,
+      "eval_biology_samples_per_second": 13.499,
+      "eval_biology_steps_per_second": 13.499,
+      "step": 300
+    },
+    {
+      "epoch": 0.13353363378400934,
+      "eval_chemistry_entropy": 0.8489444836974144,
+      "eval_chemistry_loss": 0.8385273814201355,
+      "eval_chemistry_mean_token_accuracy": 0.7699487572312355,
+      "eval_chemistry_num_tokens": 1985766.0,
+      "eval_chemistry_runtime": 40.8665,
+      "eval_chemistry_samples_per_second": 12.235,
+      "eval_chemistry_steps_per_second": 12.235,
+      "step": 300
+    },
+    {
+      "epoch": 0.13353363378400934,
+      "eval_math_entropy": 0.7357360773980618,
+      "eval_math_loss": 0.8767910599708557,
+      "eval_math_mean_token_accuracy": 0.7708948514461518,
+      "eval_math_num_tokens": 1985766.0,
+      "eval_math_runtime": 41.8192,
+      "eval_math_samples_per_second": 11.956,
+      "eval_math_steps_per_second": 11.956,
+      "step": 300
+    },
+    {
+      "epoch": 0.13353363378400934,
+      "eval_physics_entropy": 0.7905060887336731,
+      "eval_physics_loss": 0.7890318632125854,
+      "eval_physics_mean_token_accuracy": 0.7825853993296623,
+      "eval_physics_num_tokens": 1985766.0,
+      "eval_physics_runtime": 46.1418,
+      "eval_physics_samples_per_second": 10.836,
+      "eval_physics_steps_per_second": 10.836,
+      "step": 300
+    },
+    {
+      "entropy": 0.8452363438904286,
+      "epoch": 0.137984754910143,
+      "grad_norm": 96.0,
+      "learning_rate": 6.18e-06,
+      "loss": 13.634,
+      "mean_token_accuracy": 0.7645608998835087,
+      "num_tokens": 2049041.0,
+      "step": 310
+    },
+    {
+      "entropy": 0.8076417898759246,
+      "epoch": 0.14243587603627664,
+      "grad_norm": 80.5,
+      "learning_rate": 6.380000000000001e-06,
+      "loss": 12.8943,
+      "mean_token_accuracy": 0.7760006580501795,
+      "num_tokens": 2118068.0,
+      "step": 320
+    },
+    {
+      "entropy": 0.8085115062072873,
+      "epoch": 0.14688699716241027,
+      "grad_norm": 106.5,
+      "learning_rate": 6.5800000000000005e-06,
+      "loss": 12.7928,
+      "mean_token_accuracy": 0.7790838234126568,
+      "num_tokens": 2186387.0,
+      "step": 330
+    },
+    {
+      "entropy": 0.824213713593781,
+      "epoch": 0.15133811828854393,
+      "grad_norm": 82.5,
+      "learning_rate": 6.780000000000001e-06,
+      "loss": 13.389,
+      "mean_token_accuracy": 0.7689453203231096,
+      "num_tokens": 2252650.0,
+      "step": 340
+    },
+    {
+      "entropy": 0.797634300775826,
+      "epoch": 0.15578923941467757,
+      "grad_norm": 85.5,
+      "learning_rate": 6.98e-06,
+      "loss": 12.8088,
+      "mean_token_accuracy": 0.7797435287386179,
+      "num_tokens": 2320358.0,
+      "step": 350
+    },
+    {
+      "entropy": 0.7920836627483367,
+      "epoch": 0.16024036054081123,
+      "grad_norm": 88.0,
+      "learning_rate": 7.180000000000001e-06,
+      "loss": 12.6336,
+      "mean_token_accuracy": 0.7806293956935406,
+      "num_tokens": 2388824.0,
+      "step": 360
+    },
+    {
+      "entropy": 0.7749077206477523,
+      "epoch": 0.16469148166694486,
+      "grad_norm": 80.5,
+      "learning_rate": 7.3800000000000005e-06,
+      "loss": 12.3995,
+      "mean_token_accuracy": 0.7864189695566892,
+      "num_tokens": 2456144.0,
+      "step": 370
+    },
+    {
+      "entropy": 0.8578483302146196,
+      "epoch": 0.1691426027930785,
+      "grad_norm": 81.0,
+      "learning_rate": 7.58e-06,
+      "loss": 13.8496,
+      "mean_token_accuracy": 0.7662820883095265,
+      "num_tokens": 2515325.0,
+      "step": 380
+    },
+    {
+      "entropy": 0.7971056224778295,
+      "epoch": 0.17359372391921216,
+      "grad_norm": 68.0,
+      "learning_rate": 7.78e-06,
+      "loss": 12.6922,
+      "mean_token_accuracy": 0.7778837148100137,
+      "num_tokens": 2580490.0,
+      "step": 390
+    },
+    {
+      "entropy": 0.7984419580549001,
+      "epoch": 0.1780448450453458,
+      "grad_norm": 70.5,
+      "learning_rate": 7.980000000000002e-06,
+      "loss": 12.8255,
+      "mean_token_accuracy": 0.7773921977728605,
+      "num_tokens": 2644330.0,
+      "step": 400
+    },
+    {
+      "epoch": 0.1780448450453458,
+      "eval_biology_entropy": 1.2305693982243537,
+      "eval_biology_loss": 1.3490314483642578,
+      "eval_biology_mean_token_accuracy": 0.6716532117724419,
+      "eval_biology_num_tokens": 2644330.0,
+      "eval_biology_runtime": 36.6001,
+      "eval_biology_samples_per_second": 13.661,
+      "eval_biology_steps_per_second": 13.661,
+      "step": 400
+    },
+    {
+      "epoch": 0.1780448450453458,
+      "eval_chemistry_entropy": 0.7882854393720626,
+      "eval_chemistry_loss": 0.7991188764572144,
+      "eval_chemistry_mean_token_accuracy": 0.778610565006733,
+      "eval_chemistry_num_tokens": 2644330.0,
+      "eval_chemistry_runtime": 40.3843,
+      "eval_chemistry_samples_per_second": 12.381,
+      "eval_chemistry_steps_per_second": 12.381,
+      "step": 400
+    },
+    {
+      "epoch": 0.1780448450453458,
+      "eval_math_entropy": 0.6974474709928036,
+      "eval_math_loss": 0.8647986054420471,
+      "eval_math_mean_token_accuracy": 0.7742947788238526,
+      "eval_math_num_tokens": 2644330.0,
+      "eval_math_runtime": 41.8502,
+      "eval_math_samples_per_second": 11.947,
+      "eval_math_steps_per_second": 11.947,
+      "step": 400
+    },
+    {
+      "epoch": 0.1780448450453458,
+      "eval_physics_entropy": 0.740912606716156,
+      "eval_physics_loss": 0.7630434632301331,
+      "eval_physics_mean_token_accuracy": 0.7882855938673019,
+      "eval_physics_num_tokens": 2644330.0,
+      "eval_physics_runtime": 46.1492,
+      "eval_physics_samples_per_second": 10.834,
+      "eval_physics_steps_per_second": 10.834,
+      "step": 400
+    },
+    {
+      "entropy": 0.7443430663086474,
+      "epoch": 0.18249596617147945,
+      "grad_norm": 72.5,
+      "learning_rate": 8.18e-06,
+      "loss": 11.7994,
+      "mean_token_accuracy": 0.7938185524195432,
+      "num_tokens": 2713234.0,
+      "step": 410
+    },
+    {
+      "entropy": 0.7856309018097818,
+      "epoch": 0.18694708729761308,
+      "grad_norm": 78.0,
+      "learning_rate": 8.380000000000001e-06,
+      "loss": 12.9567,
+      "mean_token_accuracy": 0.7781453486531973,
+      "num_tokens": 2783261.0,
+      "step": 420
+    },
+    {
+      "entropy": 0.799421458132565,
+      "epoch": 0.19139820842374672,
+      "grad_norm": 61.25,
+      "learning_rate": 8.580000000000001e-06,
+      "loss": 12.6392,
+      "mean_token_accuracy": 0.7806750666350126,
+      "num_tokens": 2850170.0,
+      "step": 430
+    },
+    {
+      "entropy": 0.7786322007887065,
+      "epoch": 0.19584932954988038,
+      "grad_norm": 80.5,
+      "learning_rate": 8.78e-06,
+      "loss": 12.4988,
+      "mean_token_accuracy": 0.7827117882668972,
+      "num_tokens": 2913700.0,
+      "step": 440
+    },
+    {
+      "entropy": 0.763565490860492,
+      "epoch": 0.200300450676014,
+      "grad_norm": 81.0,
+      "learning_rate": 8.98e-06,
+      "loss": 12.4779,
+      "mean_token_accuracy": 0.7831043265759945,
+      "num_tokens": 2981212.0,
+      "step": 450
+    },
+    {
+      "entropy": 0.7708647299557925,
+      "epoch": 0.20475157180214767,
+      "grad_norm": 85.0,
+      "learning_rate": 9.180000000000002e-06,
+      "loss": 12.1503,
+      "mean_token_accuracy": 0.7912418570369482,
+      "num_tokens": 3048973.0,
+      "step": 460
+    },
+    {
+      "entropy": 0.769937212113291,
+      "epoch": 0.2092026929282813,
+      "grad_norm": 62.0,
+      "learning_rate": 9.38e-06,
+      "loss": 12.6137,
+      "mean_token_accuracy": 0.7829986553639173,
+      "num_tokens": 3117033.0,
+      "step": 470
+    },
+    {
+      "entropy": 0.7863288868218661,
+      "epoch": 0.21365381405441494,
+      "grad_norm": 74.5,
+      "learning_rate": 9.58e-06,
+      "loss": 12.3857,
+      "mean_token_accuracy": 0.7842021461576223,
+      "num_tokens": 3185255.0,
+      "step": 480
+    },
+    {
+      "entropy": 0.7699232887476682,
+      "epoch": 0.2181049351805486,
+      "grad_norm": 74.5,
+      "learning_rate": 9.780000000000001e-06,
+      "loss": 12.4166,
+      "mean_token_accuracy": 0.7846909243613481,
+      "num_tokens": 3253121.0,
+      "step": 490
+    },
+    {
+      "entropy": 0.7251296367496252,
+      "epoch": 0.22255605630668224,
+      "grad_norm": 75.0,
+      "learning_rate": 9.980000000000001e-06,
+      "loss": 11.4498,
+      "mean_token_accuracy": 0.7972834445536137,
+      "num_tokens": 3322823.0,
+      "step": 500
+    },
+    {
+      "epoch": 0.22255605630668224,
+      "eval_biology_entropy": 1.1898637573719024,
+      "eval_biology_loss": 1.338150978088379,
+      "eval_biology_mean_token_accuracy": 0.6746688461303711,
+      "eval_biology_num_tokens": 3322823.0,
+      "eval_biology_runtime": 37.4484,
+      "eval_biology_samples_per_second": 13.352,
+      "eval_biology_steps_per_second": 13.352,
+      "step": 500
+    },
+    {
+      "epoch": 0.22255605630668224,
+      "eval_chemistry_entropy": 0.7506220785379409,
+      "eval_chemistry_loss": 0.7758934497833252,
+      "eval_chemistry_mean_token_accuracy": 0.7828538768291473,
+      "eval_chemistry_num_tokens": 3322823.0,
+      "eval_chemistry_runtime": 40.7375,
+      "eval_chemistry_samples_per_second": 12.274,
+      "eval_chemistry_steps_per_second": 12.274,
+      "step": 500
+    },
+    {
+      "epoch": 0.22255605630668224,
+      "eval_math_entropy": 0.6889803909361363,
+      "eval_math_loss": 0.8622512817382812,
+      "eval_math_mean_token_accuracy": 0.7754237843751908,
+      "eval_math_num_tokens": 3322823.0,
+      "eval_math_runtime": 41.9238,
+      "eval_math_samples_per_second": 11.926,
+      "eval_math_steps_per_second": 11.926,
+      "step": 500
+    },
+    {
+      "epoch": 0.22255605630668224,
+      "eval_physics_entropy": 0.7242196025848389,
+      "eval_physics_loss": 0.7506964802742004,
+      "eval_physics_mean_token_accuracy": 0.7921671935915947,
+      "eval_physics_num_tokens": 3322823.0,
+      "eval_physics_runtime": 45.9692,
+      "eval_physics_samples_per_second": 10.877,
+      "eval_physics_steps_per_second": 10.877,
+      "step": 500
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 10000,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 5,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 2.2280979081801686e+17,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-500/training_args.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d5c309713d43008ffebf2b6192655dbdf8042a7ab75047241cf165dbf24ad3b0
+size 6481