Instructions to use 42e/mathcompose-verifier-1.5b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 42e/mathcompose-verifier-1.5b with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-Math-1.5B-Instruct")
model = PeftModel.from_pretrained(base_model, "42e/mathcompose-verifier-1.5b")

Transformers

How to use 42e/mathcompose-verifier-1.5b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="42e/mathcompose-verifier-1.5b")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("42e/mathcompose-verifier-1.5b", dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use 42e/mathcompose-verifier-1.5b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "42e/mathcompose-verifier-1.5b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "42e/mathcompose-verifier-1.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/42e/mathcompose-verifier-1.5b

SGLang

How to use 42e/mathcompose-verifier-1.5b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "42e/mathcompose-verifier-1.5b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "42e/mathcompose-verifier-1.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "42e/mathcompose-verifier-1.5b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "42e/mathcompose-verifier-1.5b",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 42e/mathcompose-verifier-1.5b with Docker Model Runner:
```
docker model run hf.co/42e/mathcompose-verifier-1.5b
```

42e commited on 3 days ago

Commit

41ed4b4

verified ·

1 Parent(s): 0eccff1

Upload folder using huggingface_hub

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

adapter_config.json +5 -5
adapter_model.safetensors +1 -1
checkpoint-100/adapter_config.json +5 -5
checkpoint-100/adapter_model.safetensors +1 -1
checkpoint-100/optimizer.pt +1 -1
checkpoint-100/trainer_state.json +51 -51
checkpoint-1000/adapter_config.json +5 -5
checkpoint-1000/adapter_model.safetensors +1 -1
checkpoint-1000/optimizer.pt +1 -1
checkpoint-1000/trainer_state.json +501 -501
checkpoint-1050/adapter_config.json +5 -5
checkpoint-1050/adapter_model.safetensors +1 -1
checkpoint-1050/optimizer.pt +1 -1
checkpoint-1050/trainer_state.json +526 -526
checkpoint-1100/adapter_config.json +5 -5
checkpoint-1100/adapter_model.safetensors +1 -1
checkpoint-1100/optimizer.pt +1 -1
checkpoint-1100/trainer_state.json +551 -551
checkpoint-1150/adapter_config.json +5 -5
checkpoint-1150/adapter_model.safetensors +1 -1
checkpoint-1150/optimizer.pt +1 -1
checkpoint-1150/trainer_state.json +575 -575
checkpoint-1200/adapter_config.json +5 -5
checkpoint-1200/adapter_model.safetensors +1 -1
checkpoint-1200/optimizer.pt +1 -1
checkpoint-1200/trainer_state.json +601 -601
checkpoint-1250/adapter_config.json +5 -5
checkpoint-1250/adapter_model.safetensors +1 -1
checkpoint-1250/optimizer.pt +1 -1
checkpoint-1250/trainer_state.json +625 -625
checkpoint-1300/adapter_config.json +5 -5
checkpoint-1300/adapter_model.safetensors +1 -1
checkpoint-1300/optimizer.pt +1 -1
checkpoint-1300/trainer_state.json +651 -651
checkpoint-1350/adapter_config.json +5 -5
checkpoint-1350/adapter_model.safetensors +1 -1
checkpoint-1350/optimizer.pt +1 -1
checkpoint-1350/trainer_state.json +677 -677
checkpoint-1400/adapter_config.json +5 -5
checkpoint-1400/adapter_model.safetensors +1 -1
checkpoint-1400/optimizer.pt +1 -1
checkpoint-1400/trainer_state.json +701 -701
checkpoint-1450/adapter_config.json +5 -5
checkpoint-1450/adapter_model.safetensors +1 -1
checkpoint-1450/optimizer.pt +1 -1
checkpoint-1450/trainer_state.json +725 -725
checkpoint-150/adapter_config.json +5 -5
checkpoint-150/adapter_model.safetensors +1 -1
checkpoint-150/optimizer.pt +1 -1
checkpoint-150/trainer_state.json +77 -77

adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:daa327a08e34b24d1fb838778e344666e08dc9be064f4729fa28e8d9aa75a75f
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:96c11c16cf625da7f238ab9661101cc53d37dd5761b4b049ee0f47e96605e63c
 size 73911504

checkpoint-100/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-100/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:41a352eac3525221c7e6492006ccf1a168de968d1936bfc484ec82d8516c1e7d
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:56c62c8eaccdaeaf161a17fdee058a3a13e143732b8e39ab9fe7a8b9866de859
 size 73911504

checkpoint-100/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1da9240a6a4f53d033d1c7ccb9219a8db25cf2054b464320188be62afbbc7005
 size 75480837

 version https://git-lfs.github.com/spec/v1
+oid sha256:8b0d9beaec302d5ca9064fd7c9246e8b645e53f8881b6a166771c6f02a284c58
 size 75480837

checkpoint-100/trainer_state.json CHANGED Viewed

@@ -10,125 +10,125 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     }
   ],

checkpoint-1000/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1000/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ee24119072f11c0af081f59c58bed9bbb7997ac37fcd76fd929cf8d51da4f518
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:f6f3f73bc571f5d20cb55bfe3e822b3bb5c393a9bda7c11c7f1b3c9896203bea
 size 73911504

checkpoint-1000/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6c64565b335948906b563dadefa773d27a12ffc17b35dc282189f2c04802e7ff
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:9c938080173fc567439cb26cf5ee059d9eb537081c40ef09eedf2131b2f0b81b
 size 75481221

checkpoint-1000/trainer_state.json CHANGED Viewed

@@ -10,1223 +10,1223 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     }
   ],

checkpoint-1050/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1050/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0adfe83ac12e39dc651a070343bc45b759ef58974ef14941c46f12971fa902bc
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:5119f7b056427769933e9af72f2b2ab1ecb540acbec60e42c3d5cfbaed2cd20f
 size 73911504

checkpoint-1050/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:07b45d83bdad1b2e9f30ea5c8a59789f0e9b89dfad37650c70f0f666e2de072b
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:e342a981c0ea88651fd2a20ed14353241876e826b2ae236d8ae9e3edd960323c
 size 75481221

checkpoint-1050/trainer_state.json CHANGED Viewed

@@ -10,1284 +10,1284 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     }
   ],

checkpoint-1100/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1100/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:638afaae366c6bb200e202b47dbf15d3d4ec51cee6afd45a9c47b13e49648b37
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:beaa3c764a30c1f749f206f2ae4a487c89b0f102c744deafe126dddcd5b3e7d3
 size 73911504

checkpoint-1100/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1b57ac7c918ec515e39245ff3eff83139cc8c503c0a78904ba2de8f7f8f37916
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:8425fbe85577c54b159a3c6499746e5e45872a68406b421b591e918fb9503d59
 size 75481221

checkpoint-1100/trainer_state.json CHANGED Viewed

@@ -10,1345 +10,1345 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     }
   ],

checkpoint-1150/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1150/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:34912f66e3f8ef3a006a793ea9a4a5fa140325760b401db19f0db3d684510ade
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:68c8c6aeebfb040464fe55d464bf297067d51009fcc22c4bf43d37e0906326a2
 size 73911504

checkpoint-1150/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f4b1aebff74b8ad7a098b69f0c01185ec1315c45fc61beab31d9cda304c20b3e
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:597816713da26501302e363195cfdb2d04ee79de8cd3db0546a144de28e078bd
 size 75481221

checkpoint-1150/trainer_state.json CHANGED Viewed

@@ -10,1405 +10,1405 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     },
     {
-      "entropy": 0.5992730811238289,
       "epoch": 2.1850393700787403,
-      "grad_norm": 0.474609375,
       "learning_rate": 3.644777706542845e-05,
-      "loss": 0.5308849811553955,
-      "mean_token_accuracy": 0.8519498467445373,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
-      "entropy": 0.5141457825899124,
       "epoch": 2.204724409448819,
-      "grad_norm": 0.49609375,
       "learning_rate": 3.4821141049373804e-05,
-      "loss": 0.4474374294281006,
-      "mean_token_accuracy": 0.8720535308122634,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
-      "entropy": 0.5664690762758255,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
-      "loss": 0.5310683250427246,
-      "mean_token_accuracy": 0.8515143841505051,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
-      "entropy": 0.5990463197231293,
       "epoch": 2.2440944881889764,
-      "grad_norm": 0.58984375,
       "learning_rate": 3.165693165312395e-05,
-      "loss": 0.5457014083862305,
-      "mean_token_accuracy": 0.8466474860906601,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
-      "entropy": 0.5571582794189454,
       "epoch": 2.263779527559055,
-      "grad_norm": 0.55078125,
       "learning_rate": 3.0120787824218554e-05,
-      "loss": 0.5153813362121582,
-      "mean_token_accuracy": 0.8554807037115097,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
-      "eval_entropy": 0.65144182741642,
-      "eval_loss": 0.8418021202087402,
-      "eval_mean_token_accuracy": 0.7838731220474949,
       "eval_num_tokens": 12763743.0,
-      "eval_runtime": 15.9352,
-      "eval_samples_per_second": 26.796,
       "eval_steps_per_second": 3.389,
       "step": 1150
     }

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     },
     {
+      "entropy": 0.5992338545620441,
       "epoch": 2.1850393700787403,
+      "grad_norm": 0.47265625,
       "learning_rate": 3.644777706542845e-05,
+      "loss": 0.5292766571044922,
+      "mean_token_accuracy": 0.8512318909168244,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
+      "entropy": 0.514266274124384,
       "epoch": 2.204724409448819,
+      "grad_norm": 0.5,
       "learning_rate": 3.4821141049373804e-05,
+      "loss": 0.4458413600921631,
+      "mean_token_accuracy": 0.8724471956491471,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
+      "entropy": 0.5649555295705795,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
+      "loss": 0.5298549652099609,
+      "mean_token_accuracy": 0.8531913965940475,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
+      "entropy": 0.5983025059103966,
       "epoch": 2.2440944881889764,
+      "grad_norm": 0.58203125,
       "learning_rate": 3.165693165312395e-05,
+      "loss": 0.5455004215240479,
+      "mean_token_accuracy": 0.8468506157398223,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
+      "entropy": 0.5571755915880203,
       "epoch": 2.263779527559055,
+      "grad_norm": 0.55859375,
       "learning_rate": 3.0120787824218554e-05,
+      "loss": 0.5133816719055175,
+      "mean_token_accuracy": 0.8542670160531998,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
+      "eval_entropy": 0.647307668571119,
+      "eval_loss": 0.844680666923523,
+      "eval_mean_token_accuracy": 0.7836638799420109,
       "eval_num_tokens": 12763743.0,
+      "eval_runtime": 15.9342,
+      "eval_samples_per_second": 26.798,
       "eval_steps_per_second": 3.389,
       "step": 1150
     }

checkpoint-1200/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1200/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0dc36da92bf715c19615a75482a8a7f061293f7f05346a05f607193fc4cfcab5
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:8544374ca00de21adfda105073871c0b76019d5cd8952701b8fb9dd77173386d
 size 73911504

checkpoint-1200/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fae56aff7b58a09bc3a0e5934978afab1d4a70f0d63c86aa81f9bc0bb308e2dd
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:ff6fcb76105b234506e8e6ad86e038259820f271efaac0226b0c8bdf2eceaf05
 size 75481221

checkpoint-1200/trainer_state.json CHANGED Viewed

@@ -10,1467 +10,1467 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     },
     {
-      "entropy": 0.5992730811238289,
       "epoch": 2.1850393700787403,
-      "grad_norm": 0.474609375,
       "learning_rate": 3.644777706542845e-05,
-      "loss": 0.5308849811553955,
-      "mean_token_accuracy": 0.8519498467445373,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
-      "entropy": 0.5141457825899124,
       "epoch": 2.204724409448819,
-      "grad_norm": 0.49609375,
       "learning_rate": 3.4821141049373804e-05,
-      "loss": 0.4474374294281006,
-      "mean_token_accuracy": 0.8720535308122634,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
-      "entropy": 0.5664690762758255,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
-      "loss": 0.5310683250427246,
-      "mean_token_accuracy": 0.8515143841505051,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
-      "entropy": 0.5990463197231293,
       "epoch": 2.2440944881889764,
-      "grad_norm": 0.58984375,
       "learning_rate": 3.165693165312395e-05,
-      "loss": 0.5457014083862305,
-      "mean_token_accuracy": 0.8466474860906601,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
-      "entropy": 0.5571582794189454,
       "epoch": 2.263779527559055,
-      "grad_norm": 0.55078125,
       "learning_rate": 3.0120787824218554e-05,
-      "loss": 0.5153813362121582,
-      "mean_token_accuracy": 0.8554807037115097,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
-      "eval_entropy": 0.65144182741642,
-      "eval_loss": 0.8418021202087402,
-      "eval_mean_token_accuracy": 0.7838731220474949,
       "eval_num_tokens": 12763743.0,
-      "eval_runtime": 15.9352,
-      "eval_samples_per_second": 26.796,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
-      "entropy": 0.6080173872411251,
       "epoch": 2.283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 2.8616214572789968e-05,
-      "loss": 0.5399338245391846,
-      "mean_token_accuracy": 0.8469231188297272,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
-      "entropy": 0.49714402854442596,
       "epoch": 2.3031496062992125,
-      "grad_norm": 0.578125,
       "learning_rate": 2.7143891646719265e-05,
-      "loss": 0.46820721626281736,
-      "mean_token_accuracy": 0.8644533902406693,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
-      "entropy": 0.5437434703111649,
       "epoch": 2.322834645669291,
-      "grad_norm": 0.6015625,
       "learning_rate": 2.570448422358297e-05,
-      "loss": 0.5030812740325927,
-      "mean_token_accuracy": 0.8589521288871765,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
-      "entropy": 0.5845290660858155,
       "epoch": 2.34251968503937,
-      "grad_norm": 0.60546875,
       "learning_rate": 2.4298642610134105e-05,
-      "loss": 0.5023557186126709,
-      "mean_token_accuracy": 0.8512013405561447,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
-      "entropy": 0.6722325071692467,
       "epoch": 2.362204724409449,
-      "grad_norm": 0.66015625,
       "learning_rate": 2.292700194850147e-05,
-      "loss": 0.5894911289215088,
-      "mean_token_accuracy": 0.8302695482969285,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
-      "eval_entropy": 0.6533891520014515,
-      "eval_loss": 0.8373947739601135,
-      "eval_mean_token_accuracy": 0.7844164106580946,
       "eval_num_tokens": 13314862.0,
-      "eval_runtime": 15.9338,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 1200
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     },
     {
+      "entropy": 0.5992338545620441,
       "epoch": 2.1850393700787403,
+      "grad_norm": 0.47265625,
       "learning_rate": 3.644777706542845e-05,
+      "loss": 0.5292766571044922,
+      "mean_token_accuracy": 0.8512318909168244,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
+      "entropy": 0.514266274124384,
       "epoch": 2.204724409448819,
+      "grad_norm": 0.5,
       "learning_rate": 3.4821141049373804e-05,
+      "loss": 0.4458413600921631,
+      "mean_token_accuracy": 0.8724471956491471,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
+      "entropy": 0.5649555295705795,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
+      "loss": 0.5298549652099609,
+      "mean_token_accuracy": 0.8531913965940475,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
+      "entropy": 0.5983025059103966,
       "epoch": 2.2440944881889764,
+      "grad_norm": 0.58203125,
       "learning_rate": 3.165693165312395e-05,
+      "loss": 0.5455004215240479,
+      "mean_token_accuracy": 0.8468506157398223,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
+      "entropy": 0.5571755915880203,
       "epoch": 2.263779527559055,
+      "grad_norm": 0.55859375,
       "learning_rate": 3.0120787824218554e-05,
+      "loss": 0.5133816719055175,
+      "mean_token_accuracy": 0.8542670160531998,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
+      "eval_entropy": 0.647307668571119,
+      "eval_loss": 0.844680666923523,
+      "eval_mean_token_accuracy": 0.7836638799420109,
       "eval_num_tokens": 12763743.0,
+      "eval_runtime": 15.9342,
+      "eval_samples_per_second": 26.798,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
+      "entropy": 0.6052810281515122,
       "epoch": 2.283464566929134,
+      "grad_norm": 0.482421875,
       "learning_rate": 2.8616214572789968e-05,
+      "loss": 0.5399055004119873,
+      "mean_token_accuracy": 0.8467662870883942,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
+      "entropy": 0.495437890291214,
       "epoch": 2.3031496062992125,
+      "grad_norm": 0.57421875,
       "learning_rate": 2.7143891646719265e-05,
+      "loss": 0.4656820297241211,
+      "mean_token_accuracy": 0.8652756780385971,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
+      "entropy": 0.5415738135576248,
       "epoch": 2.322834645669291,
+      "grad_norm": 0.59765625,
       "learning_rate": 2.570448422358297e-05,
+      "loss": 0.5029004573822021,
+      "mean_token_accuracy": 0.8594674915075302,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
+      "entropy": 0.5835595086216927,
       "epoch": 2.34251968503937,
+      "grad_norm": 0.6484375,
       "learning_rate": 2.4298642610134105e-05,
+      "loss": 0.502112865447998,
+      "mean_token_accuracy": 0.8509185254573822,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
+      "entropy": 0.6709924265742302,
       "epoch": 2.362204724409449,
+      "grad_norm": 0.6640625,
       "learning_rate": 2.292700194850147e-05,
+      "loss": 0.5888309001922607,
+      "mean_token_accuracy": 0.829518511891365,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
+      "eval_entropy": 0.6516638414727317,
+      "eval_loss": 0.8389837741851807,
+      "eval_mean_token_accuracy": 0.7852442959944407,
       "eval_num_tokens": 13314862.0,
+      "eval_runtime": 15.9159,
+      "eval_samples_per_second": 26.829,
+      "eval_steps_per_second": 3.393,
       "step": 1200
     }
   ],

checkpoint-1250/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1250/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2132bdf68bde8ad7a08239ed8c3f7e1619904c727f89ce3c230e290f6abcbb5f
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:1f87f8f66d9866585a33e9b50bca82915c6d27d4b7cc539b9c475eb251dd8082
 size 73911504

checkpoint-1250/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:71c5a527a74d77ed824bce248392fab6878be695f423c7133281852c5136b1ab
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:5978194de4505eb37eccc05595bb20ab74dd739e4fe0e05c1e704ddf0baf3e49
 size 75481221

checkpoint-1250/trainer_state.json CHANGED Viewed

@@ -10,1527 +10,1527 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     },
     {
-      "entropy": 0.5992730811238289,
       "epoch": 2.1850393700787403,
-      "grad_norm": 0.474609375,
       "learning_rate": 3.644777706542845e-05,
-      "loss": 0.5308849811553955,
-      "mean_token_accuracy": 0.8519498467445373,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
-      "entropy": 0.5141457825899124,
       "epoch": 2.204724409448819,
-      "grad_norm": 0.49609375,
       "learning_rate": 3.4821141049373804e-05,
-      "loss": 0.4474374294281006,
-      "mean_token_accuracy": 0.8720535308122634,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
-      "entropy": 0.5664690762758255,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
-      "loss": 0.5310683250427246,
-      "mean_token_accuracy": 0.8515143841505051,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
-      "entropy": 0.5990463197231293,
       "epoch": 2.2440944881889764,
-      "grad_norm": 0.58984375,
       "learning_rate": 3.165693165312395e-05,
-      "loss": 0.5457014083862305,
-      "mean_token_accuracy": 0.8466474860906601,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
-      "entropy": 0.5571582794189454,
       "epoch": 2.263779527559055,
-      "grad_norm": 0.55078125,
       "learning_rate": 3.0120787824218554e-05,
-      "loss": 0.5153813362121582,
-      "mean_token_accuracy": 0.8554807037115097,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
-      "eval_entropy": 0.65144182741642,
-      "eval_loss": 0.8418021202087402,
-      "eval_mean_token_accuracy": 0.7838731220474949,
       "eval_num_tokens": 12763743.0,
-      "eval_runtime": 15.9352,
-      "eval_samples_per_second": 26.796,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
-      "entropy": 0.6080173872411251,
       "epoch": 2.283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 2.8616214572789968e-05,
-      "loss": 0.5399338245391846,
-      "mean_token_accuracy": 0.8469231188297272,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
-      "entropy": 0.49714402854442596,
       "epoch": 2.3031496062992125,
-      "grad_norm": 0.578125,
       "learning_rate": 2.7143891646719265e-05,
-      "loss": 0.46820721626281736,
-      "mean_token_accuracy": 0.8644533902406693,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
-      "entropy": 0.5437434703111649,
       "epoch": 2.322834645669291,
-      "grad_norm": 0.6015625,
       "learning_rate": 2.570448422358297e-05,
-      "loss": 0.5030812740325927,
-      "mean_token_accuracy": 0.8589521288871765,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
-      "entropy": 0.5845290660858155,
       "epoch": 2.34251968503937,
-      "grad_norm": 0.60546875,
       "learning_rate": 2.4298642610134105e-05,
-      "loss": 0.5023557186126709,
-      "mean_token_accuracy": 0.8512013405561447,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
-      "entropy": 0.6722325071692467,
       "epoch": 2.362204724409449,
-      "grad_norm": 0.66015625,
       "learning_rate": 2.292700194850147e-05,
-      "loss": 0.5894911289215088,
-      "mean_token_accuracy": 0.8302695482969285,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
-      "eval_entropy": 0.6533891520014515,
-      "eval_loss": 0.8373947739601135,
-      "eval_mean_token_accuracy": 0.7844164106580946,
       "eval_num_tokens": 13314862.0,
-      "eval_runtime": 15.9338,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 1200
     },
     {
-      "entropy": 0.5659365668892861,
       "epoch": 2.3818897637795278,
-      "grad_norm": 0.5078125,
       "learning_rate": 2.159018192924004e-05,
-      "loss": 0.5257106304168702,
-      "mean_token_accuracy": 0.8536411643028259,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
-      "entropy": 0.5879238858819008,
       "epoch": 2.4015748031496065,
-      "grad_norm": 0.44921875,
       "learning_rate": 2.0288786511362525e-05,
-      "loss": 0.5346655368804931,
-      "mean_token_accuracy": 0.8476213365793228,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
-      "entropy": 0.5665464416146279,
       "epoch": 2.421259842519685,
-      "grad_norm": 0.4921875,
       "learning_rate": 1.9023403649477813e-05,
-      "loss": 0.5051110267639161,
-      "mean_token_accuracy": 0.8583743631839752,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
-      "entropy": 0.5041692733764649,
       "epoch": 2.440944881889764,
-      "grad_norm": 0.453125,
       "learning_rate": 1.7794605028160462e-05,
-      "loss": 0.46024274826049805,
-      "mean_token_accuracy": 0.8660459965467453,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
-      "entropy": 0.5898029208183289,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
-      "loss": 0.5336055755615234,
-      "mean_token_accuracy": 0.8486702740192413,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
-      "eval_entropy": 0.6570240441295836,
-      "eval_loss": 0.8387707471847534,
-      "eval_mean_token_accuracy": 0.7845993064067982,
       "eval_num_tokens": 13872536.0,
-      "eval_runtime": 15.9231,
-      "eval_samples_per_second": 26.816,
       "eval_steps_per_second": 3.391,
       "step": 1250
     }

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     },
     {
+      "entropy": 0.5992338545620441,
       "epoch": 2.1850393700787403,
+      "grad_norm": 0.47265625,
       "learning_rate": 3.644777706542845e-05,
+      "loss": 0.5292766571044922,
+      "mean_token_accuracy": 0.8512318909168244,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
+      "entropy": 0.514266274124384,
       "epoch": 2.204724409448819,
+      "grad_norm": 0.5,
       "learning_rate": 3.4821141049373804e-05,
+      "loss": 0.4458413600921631,
+      "mean_token_accuracy": 0.8724471956491471,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
+      "entropy": 0.5649555295705795,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
+      "loss": 0.5298549652099609,
+      "mean_token_accuracy": 0.8531913965940475,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
+      "entropy": 0.5983025059103966,
       "epoch": 2.2440944881889764,
+      "grad_norm": 0.58203125,
       "learning_rate": 3.165693165312395e-05,
+      "loss": 0.5455004215240479,
+      "mean_token_accuracy": 0.8468506157398223,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
+      "entropy": 0.5571755915880203,
       "epoch": 2.263779527559055,
+      "grad_norm": 0.55859375,
       "learning_rate": 3.0120787824218554e-05,
+      "loss": 0.5133816719055175,
+      "mean_token_accuracy": 0.8542670160531998,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
+      "eval_entropy": 0.647307668571119,
+      "eval_loss": 0.844680666923523,
+      "eval_mean_token_accuracy": 0.7836638799420109,
       "eval_num_tokens": 12763743.0,
+      "eval_runtime": 15.9342,
+      "eval_samples_per_second": 26.798,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
+      "entropy": 0.6052810281515122,
       "epoch": 2.283464566929134,
+      "grad_norm": 0.482421875,
       "learning_rate": 2.8616214572789968e-05,
+      "loss": 0.5399055004119873,
+      "mean_token_accuracy": 0.8467662870883942,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
+      "entropy": 0.495437890291214,
       "epoch": 2.3031496062992125,
+      "grad_norm": 0.57421875,
       "learning_rate": 2.7143891646719265e-05,
+      "loss": 0.4656820297241211,
+      "mean_token_accuracy": 0.8652756780385971,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
+      "entropy": 0.5415738135576248,
       "epoch": 2.322834645669291,
+      "grad_norm": 0.59765625,
       "learning_rate": 2.570448422358297e-05,
+      "loss": 0.5029004573822021,
+      "mean_token_accuracy": 0.8594674915075302,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
+      "entropy": 0.5835595086216927,
       "epoch": 2.34251968503937,
+      "grad_norm": 0.6484375,
       "learning_rate": 2.4298642610134105e-05,
+      "loss": 0.502112865447998,
+      "mean_token_accuracy": 0.8509185254573822,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
+      "entropy": 0.6709924265742302,
       "epoch": 2.362204724409449,
+      "grad_norm": 0.6640625,
       "learning_rate": 2.292700194850147e-05,
+      "loss": 0.5888309001922607,
+      "mean_token_accuracy": 0.829518511891365,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
+      "eval_entropy": 0.6516638414727317,
+      "eval_loss": 0.8389837741851807,
+      "eval_mean_token_accuracy": 0.7852442959944407,
       "eval_num_tokens": 13314862.0,
+      "eval_runtime": 15.9159,
+      "eval_samples_per_second": 26.829,
+      "eval_steps_per_second": 3.393,
       "step": 1200
     },
     {
+      "entropy": 0.5645383015275002,
       "epoch": 2.3818897637795278,
+      "grad_norm": 0.5390625,
       "learning_rate": 2.159018192924004e-05,
+      "loss": 0.5259816169738769,
+      "mean_token_accuracy": 0.8538117051124573,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
+      "entropy": 0.5880276575684548,
       "epoch": 2.4015748031496065,
+      "grad_norm": 0.439453125,
       "learning_rate": 2.0288786511362525e-05,
+      "loss": 0.5333644866943359,
+      "mean_token_accuracy": 0.8482611387968063,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
+      "entropy": 0.5676271229982376,
       "epoch": 2.421259842519685,
+      "grad_norm": 0.494140625,
       "learning_rate": 1.9023403649477813e-05,
+      "loss": 0.5034989833831787,
+      "mean_token_accuracy": 0.8573888152837753,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
+      "entropy": 0.5038574181497097,
       "epoch": 2.440944881889764,
+      "grad_norm": 0.435546875,
       "learning_rate": 1.7794605028160462e-05,
+      "loss": 0.45959224700927737,
+      "mean_token_accuracy": 0.866667166352272,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
+      "entropy": 0.5891189277172089,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
+      "loss": 0.5333945274353027,
+      "mean_token_accuracy": 0.8494920641183853,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
+      "eval_entropy": 0.6562480860286288,
+      "eval_loss": 0.8404139876365662,
+      "eval_mean_token_accuracy": 0.7847034048151087,
       "eval_num_tokens": 13872536.0,
+      "eval_runtime": 15.9225,
+      "eval_samples_per_second": 26.817,
       "eval_steps_per_second": 3.391,
       "step": 1250
     }

checkpoint-1300/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1300/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:78a9d535a6fca9c1c054812825ff56fce9d1ba1c7aa19a5587e825ed3eba0a58
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:09245116a23fe4cde24ebdc9b26e2cd56419ba683d2f8f25147a02ad731b4253
 size 73911504

checkpoint-1300/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:708e0e010b38b3c064b7fe2ad442f77d426973ff5e96bc75174338e9e12a82d5
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:146990b90464c9b97b2b9b5e5a9e7658056da02e4398b061d8fd876423ffe92c
 size 75481221

checkpoint-1300/trainer_state.json CHANGED Viewed

@@ -10,1589 +10,1589 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     },
     {
-      "entropy": 0.5992730811238289,
       "epoch": 2.1850393700787403,
-      "grad_norm": 0.474609375,
       "learning_rate": 3.644777706542845e-05,
-      "loss": 0.5308849811553955,
-      "mean_token_accuracy": 0.8519498467445373,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
-      "entropy": 0.5141457825899124,
       "epoch": 2.204724409448819,
-      "grad_norm": 0.49609375,
       "learning_rate": 3.4821141049373804e-05,
-      "loss": 0.4474374294281006,
-      "mean_token_accuracy": 0.8720535308122634,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
-      "entropy": 0.5664690762758255,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
-      "loss": 0.5310683250427246,
-      "mean_token_accuracy": 0.8515143841505051,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
-      "entropy": 0.5990463197231293,
       "epoch": 2.2440944881889764,
-      "grad_norm": 0.58984375,
       "learning_rate": 3.165693165312395e-05,
-      "loss": 0.5457014083862305,
-      "mean_token_accuracy": 0.8466474860906601,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
-      "entropy": 0.5571582794189454,
       "epoch": 2.263779527559055,
-      "grad_norm": 0.55078125,
       "learning_rate": 3.0120787824218554e-05,
-      "loss": 0.5153813362121582,
-      "mean_token_accuracy": 0.8554807037115097,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
-      "eval_entropy": 0.65144182741642,
-      "eval_loss": 0.8418021202087402,
-      "eval_mean_token_accuracy": 0.7838731220474949,
       "eval_num_tokens": 12763743.0,
-      "eval_runtime": 15.9352,
-      "eval_samples_per_second": 26.796,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
-      "entropy": 0.6080173872411251,
       "epoch": 2.283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 2.8616214572789968e-05,
-      "loss": 0.5399338245391846,
-      "mean_token_accuracy": 0.8469231188297272,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
-      "entropy": 0.49714402854442596,
       "epoch": 2.3031496062992125,
-      "grad_norm": 0.578125,
       "learning_rate": 2.7143891646719265e-05,
-      "loss": 0.46820721626281736,
-      "mean_token_accuracy": 0.8644533902406693,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
-      "entropy": 0.5437434703111649,
       "epoch": 2.322834645669291,
-      "grad_norm": 0.6015625,
       "learning_rate": 2.570448422358297e-05,
-      "loss": 0.5030812740325927,
-      "mean_token_accuracy": 0.8589521288871765,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
-      "entropy": 0.5845290660858155,
       "epoch": 2.34251968503937,
-      "grad_norm": 0.60546875,
       "learning_rate": 2.4298642610134105e-05,
-      "loss": 0.5023557186126709,
-      "mean_token_accuracy": 0.8512013405561447,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
-      "entropy": 0.6722325071692467,
       "epoch": 2.362204724409449,
-      "grad_norm": 0.66015625,
       "learning_rate": 2.292700194850147e-05,
-      "loss": 0.5894911289215088,
-      "mean_token_accuracy": 0.8302695482969285,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
-      "eval_entropy": 0.6533891520014515,
-      "eval_loss": 0.8373947739601135,
-      "eval_mean_token_accuracy": 0.7844164106580946,
       "eval_num_tokens": 13314862.0,
-      "eval_runtime": 15.9338,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 1200
     },
     {
-      "entropy": 0.5659365668892861,
       "epoch": 2.3818897637795278,
-      "grad_norm": 0.5078125,
       "learning_rate": 2.159018192924004e-05,
-      "loss": 0.5257106304168702,
-      "mean_token_accuracy": 0.8536411643028259,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
-      "entropy": 0.5879238858819008,
       "epoch": 2.4015748031496065,
-      "grad_norm": 0.44921875,
       "learning_rate": 2.0288786511362525e-05,
-      "loss": 0.5346655368804931,
-      "mean_token_accuracy": 0.8476213365793228,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
-      "entropy": 0.5665464416146279,
       "epoch": 2.421259842519685,
-      "grad_norm": 0.4921875,
       "learning_rate": 1.9023403649477813e-05,
-      "loss": 0.5051110267639161,
-      "mean_token_accuracy": 0.8583743631839752,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
-      "entropy": 0.5041692733764649,
       "epoch": 2.440944881889764,
-      "grad_norm": 0.453125,
       "learning_rate": 1.7794605028160462e-05,
-      "loss": 0.46024274826049805,
-      "mean_token_accuracy": 0.8660459965467453,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
-      "entropy": 0.5898029208183289,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
-      "loss": 0.5336055755615234,
-      "mean_token_accuracy": 0.8486702740192413,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
-      "eval_entropy": 0.6570240441295836,
-      "eval_loss": 0.8387707471847534,
-      "eval_mean_token_accuracy": 0.7845993064067982,
       "eval_num_tokens": 13872536.0,
-      "eval_runtime": 15.9231,
-      "eval_samples_per_second": 26.816,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
-      "entropy": 0.5317887738347054,
       "epoch": 2.4803149606299213,
-      "grad_norm": 0.6796875,
       "learning_rate": 1.544896435313997e-05,
-      "loss": 0.49010343551635743,
-      "mean_token_accuracy": 0.859997171163559,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
-      "entropy": 0.5348539352416992,
       "epoch": 2.5,
-      "grad_norm": 0.5390625,
       "learning_rate": 1.4333182031342185e-05,
-      "loss": 0.50527982711792,
-      "mean_token_accuracy": 0.8579596102237701,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
-      "entropy": 0.6260602816939353,
       "epoch": 2.5196850393700787,
-      "grad_norm": 0.49609375,
       "learning_rate": 1.3256102935148496e-05,
-      "loss": 0.5644484043121338,
-      "mean_token_accuracy": 0.8425815969705581,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
-      "entropy": 0.624223592877388,
       "epoch": 2.5393700787401574,
-      "grad_norm": 0.59375,
       "learning_rate": 1.2218213675784063e-05,
-      "loss": 0.5586697101593018,
-      "mean_token_accuracy": 0.8420278400182724,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
-      "entropy": 0.547905944287777,
       "epoch": 2.559055118110236,
-      "grad_norm": 0.43359375,
       "learning_rate": 1.1219983158982917e-05,
-      "loss": 0.49605836868286135,
-      "mean_token_accuracy": 0.8598576009273529,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
-      "eval_entropy": 0.6620006765480395,
-      "eval_loss": 0.8368703126907349,
-      "eval_mean_token_accuracy": 0.7841785737761745,
       "eval_num_tokens": 14431579.0,
-      "eval_runtime": 15.9696,
-      "eval_samples_per_second": 26.738,
-      "eval_steps_per_second": 3.381,
       "step": 1300
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     },
     {
+      "entropy": 0.5992338545620441,
       "epoch": 2.1850393700787403,
+      "grad_norm": 0.47265625,
       "learning_rate": 3.644777706542845e-05,
+      "loss": 0.5292766571044922,
+      "mean_token_accuracy": 0.8512318909168244,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
+      "entropy": 0.514266274124384,
       "epoch": 2.204724409448819,
+      "grad_norm": 0.5,
       "learning_rate": 3.4821141049373804e-05,
+      "loss": 0.4458413600921631,
+      "mean_token_accuracy": 0.8724471956491471,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
+      "entropy": 0.5649555295705795,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
+      "loss": 0.5298549652099609,
+      "mean_token_accuracy": 0.8531913965940475,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
+      "entropy": 0.5983025059103966,
       "epoch": 2.2440944881889764,
+      "grad_norm": 0.58203125,
       "learning_rate": 3.165693165312395e-05,
+      "loss": 0.5455004215240479,
+      "mean_token_accuracy": 0.8468506157398223,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
+      "entropy": 0.5571755915880203,
       "epoch": 2.263779527559055,
+      "grad_norm": 0.55859375,
       "learning_rate": 3.0120787824218554e-05,
+      "loss": 0.5133816719055175,
+      "mean_token_accuracy": 0.8542670160531998,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
+      "eval_entropy": 0.647307668571119,
+      "eval_loss": 0.844680666923523,
+      "eval_mean_token_accuracy": 0.7836638799420109,
       "eval_num_tokens": 12763743.0,
+      "eval_runtime": 15.9342,
+      "eval_samples_per_second": 26.798,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
+      "entropy": 0.6052810281515122,
       "epoch": 2.283464566929134,
+      "grad_norm": 0.482421875,
       "learning_rate": 2.8616214572789968e-05,
+      "loss": 0.5399055004119873,
+      "mean_token_accuracy": 0.8467662870883942,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
+      "entropy": 0.495437890291214,
       "epoch": 2.3031496062992125,
+      "grad_norm": 0.57421875,
       "learning_rate": 2.7143891646719265e-05,
+      "loss": 0.4656820297241211,
+      "mean_token_accuracy": 0.8652756780385971,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
+      "entropy": 0.5415738135576248,
       "epoch": 2.322834645669291,
+      "grad_norm": 0.59765625,
       "learning_rate": 2.570448422358297e-05,
+      "loss": 0.5029004573822021,
+      "mean_token_accuracy": 0.8594674915075302,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
+      "entropy": 0.5835595086216927,
       "epoch": 2.34251968503937,
+      "grad_norm": 0.6484375,
       "learning_rate": 2.4298642610134105e-05,
+      "loss": 0.502112865447998,
+      "mean_token_accuracy": 0.8509185254573822,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
+      "entropy": 0.6709924265742302,
       "epoch": 2.362204724409449,
+      "grad_norm": 0.6640625,
       "learning_rate": 2.292700194850147e-05,
+      "loss": 0.5888309001922607,
+      "mean_token_accuracy": 0.829518511891365,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
+      "eval_entropy": 0.6516638414727317,
+      "eval_loss": 0.8389837741851807,
+      "eval_mean_token_accuracy": 0.7852442959944407,
       "eval_num_tokens": 13314862.0,
+      "eval_runtime": 15.9159,
+      "eval_samples_per_second": 26.829,
+      "eval_steps_per_second": 3.393,
       "step": 1200
     },
     {
+      "entropy": 0.5645383015275002,
       "epoch": 2.3818897637795278,
+      "grad_norm": 0.5390625,
       "learning_rate": 2.159018192924004e-05,
+      "loss": 0.5259816169738769,
+      "mean_token_accuracy": 0.8538117051124573,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
+      "entropy": 0.5880276575684548,
       "epoch": 2.4015748031496065,
+      "grad_norm": 0.439453125,
       "learning_rate": 2.0288786511362525e-05,
+      "loss": 0.5333644866943359,
+      "mean_token_accuracy": 0.8482611387968063,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
+      "entropy": 0.5676271229982376,
       "epoch": 2.421259842519685,
+      "grad_norm": 0.494140625,
       "learning_rate": 1.9023403649477813e-05,
+      "loss": 0.5034989833831787,
+      "mean_token_accuracy": 0.8573888152837753,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
+      "entropy": 0.5038574181497097,
       "epoch": 2.440944881889764,
+      "grad_norm": 0.435546875,
       "learning_rate": 1.7794605028160462e-05,
+      "loss": 0.45959224700927737,
+      "mean_token_accuracy": 0.866667166352272,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
+      "entropy": 0.5891189277172089,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
+      "loss": 0.5333945274353027,
+      "mean_token_accuracy": 0.8494920641183853,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
+      "eval_entropy": 0.6562480860286288,
+      "eval_loss": 0.8404139876365662,
+      "eval_mean_token_accuracy": 0.7847034048151087,
       "eval_num_tokens": 13872536.0,
+      "eval_runtime": 15.9225,
+      "eval_samples_per_second": 26.817,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
+      "entropy": 0.5312069803476334,
       "epoch": 2.4803149606299213,
+      "grad_norm": 0.67578125,
       "learning_rate": 1.544896435313997e-05,
+      "loss": 0.48858180046081545,
+      "mean_token_accuracy": 0.860382902622223,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
+      "entropy": 0.5351930364966393,
       "epoch": 2.5,
+      "grad_norm": 0.53515625,
       "learning_rate": 1.4333182031342185e-05,
+      "loss": 0.5056577682495117,
+      "mean_token_accuracy": 0.857965275645256,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
+      "entropy": 0.6251816973090172,
       "epoch": 2.5196850393700787,
+      "grad_norm": 0.4921875,
       "learning_rate": 1.3256102935148496e-05,
+      "loss": 0.5645822048187256,
+      "mean_token_accuracy": 0.8422666400671005,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
+      "entropy": 0.6227434769272804,
       "epoch": 2.5393700787401574,
+      "grad_norm": 0.58984375,
       "learning_rate": 1.2218213675784063e-05,
+      "loss": 0.5586906909942627,
+      "mean_token_accuracy": 0.8418498873710633,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
+      "entropy": 0.5474544659256935,
       "epoch": 2.559055118110236,
+      "grad_norm": 0.4375,
       "learning_rate": 1.1219983158982917e-05,
+      "loss": 0.49657626152038575,
+      "mean_token_accuracy": 0.8581330984830856,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
+      "eval_entropy": 0.6617519998991931,
+      "eval_loss": 0.8383262157440186,
+      "eval_mean_token_accuracy": 0.7846880113637006,
       "eval_num_tokens": 14431579.0,
+      "eval_runtime": 15.9113,
+      "eval_samples_per_second": 26.836,
+      "eval_steps_per_second": 3.394,
       "step": 1300
     }
   ],

checkpoint-1350/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1350/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:07b0d07b88e66cdd8fa30340a33441a8398708f91c90c91b2c753866341ee5dd
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:1ca0df5225532d2897ffda7e8d0e408c3e7dc21e0b5623cb353cbca1a05def5d
 size 73911504

checkpoint-1350/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4071ba9f974d33b72f4f393e096939e433a41730a324d5f41514d7fc132028cc
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:57d0b80af7e84321cb339ba0282d030725674aeb8873c4fcce10a9858d2def28
 size 75481221

checkpoint-1350/trainer_state.json CHANGED Viewed

@@ -10,1650 +10,1650 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     },
     {
-      "entropy": 0.5992730811238289,
       "epoch": 2.1850393700787403,
-      "grad_norm": 0.474609375,
       "learning_rate": 3.644777706542845e-05,
-      "loss": 0.5308849811553955,
-      "mean_token_accuracy": 0.8519498467445373,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
-      "entropy": 0.5141457825899124,
       "epoch": 2.204724409448819,
-      "grad_norm": 0.49609375,
       "learning_rate": 3.4821141049373804e-05,
-      "loss": 0.4474374294281006,
-      "mean_token_accuracy": 0.8720535308122634,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
-      "entropy": 0.5664690762758255,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
-      "loss": 0.5310683250427246,
-      "mean_token_accuracy": 0.8515143841505051,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
-      "entropy": 0.5990463197231293,
       "epoch": 2.2440944881889764,
-      "grad_norm": 0.58984375,
       "learning_rate": 3.165693165312395e-05,
-      "loss": 0.5457014083862305,
-      "mean_token_accuracy": 0.8466474860906601,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
-      "entropy": 0.5571582794189454,
       "epoch": 2.263779527559055,
-      "grad_norm": 0.55078125,
       "learning_rate": 3.0120787824218554e-05,
-      "loss": 0.5153813362121582,
-      "mean_token_accuracy": 0.8554807037115097,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
-      "eval_entropy": 0.65144182741642,
-      "eval_loss": 0.8418021202087402,
-      "eval_mean_token_accuracy": 0.7838731220474949,
       "eval_num_tokens": 12763743.0,
-      "eval_runtime": 15.9352,
-      "eval_samples_per_second": 26.796,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
-      "entropy": 0.6080173872411251,
       "epoch": 2.283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 2.8616214572789968e-05,
-      "loss": 0.5399338245391846,
-      "mean_token_accuracy": 0.8469231188297272,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
-      "entropy": 0.49714402854442596,
       "epoch": 2.3031496062992125,
-      "grad_norm": 0.578125,
       "learning_rate": 2.7143891646719265e-05,
-      "loss": 0.46820721626281736,
-      "mean_token_accuracy": 0.8644533902406693,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
-      "entropy": 0.5437434703111649,
       "epoch": 2.322834645669291,
-      "grad_norm": 0.6015625,
       "learning_rate": 2.570448422358297e-05,
-      "loss": 0.5030812740325927,
-      "mean_token_accuracy": 0.8589521288871765,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
-      "entropy": 0.5845290660858155,
       "epoch": 2.34251968503937,
-      "grad_norm": 0.60546875,
       "learning_rate": 2.4298642610134105e-05,
-      "loss": 0.5023557186126709,
-      "mean_token_accuracy": 0.8512013405561447,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
-      "entropy": 0.6722325071692467,
       "epoch": 2.362204724409449,
-      "grad_norm": 0.66015625,
       "learning_rate": 2.292700194850147e-05,
-      "loss": 0.5894911289215088,
-      "mean_token_accuracy": 0.8302695482969285,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
-      "eval_entropy": 0.6533891520014515,
-      "eval_loss": 0.8373947739601135,
-      "eval_mean_token_accuracy": 0.7844164106580946,
       "eval_num_tokens": 13314862.0,
-      "eval_runtime": 15.9338,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 1200
     },
     {
-      "entropy": 0.5659365668892861,
       "epoch": 2.3818897637795278,
-      "grad_norm": 0.5078125,
       "learning_rate": 2.159018192924004e-05,
-      "loss": 0.5257106304168702,
-      "mean_token_accuracy": 0.8536411643028259,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
-      "entropy": 0.5879238858819008,
       "epoch": 2.4015748031496065,
-      "grad_norm": 0.44921875,
       "learning_rate": 2.0288786511362525e-05,
-      "loss": 0.5346655368804931,
-      "mean_token_accuracy": 0.8476213365793228,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
-      "entropy": 0.5665464416146279,
       "epoch": 2.421259842519685,
-      "grad_norm": 0.4921875,
       "learning_rate": 1.9023403649477813e-05,
-      "loss": 0.5051110267639161,
-      "mean_token_accuracy": 0.8583743631839752,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
-      "entropy": 0.5041692733764649,
       "epoch": 2.440944881889764,
-      "grad_norm": 0.453125,
       "learning_rate": 1.7794605028160462e-05,
-      "loss": 0.46024274826049805,
-      "mean_token_accuracy": 0.8660459965467453,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
-      "entropy": 0.5898029208183289,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
-      "loss": 0.5336055755615234,
-      "mean_token_accuracy": 0.8486702740192413,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
-      "eval_entropy": 0.6570240441295836,
-      "eval_loss": 0.8387707471847534,
-      "eval_mean_token_accuracy": 0.7845993064067982,
       "eval_num_tokens": 13872536.0,
-      "eval_runtime": 15.9231,
-      "eval_samples_per_second": 26.816,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
-      "entropy": 0.5317887738347054,
       "epoch": 2.4803149606299213,
-      "grad_norm": 0.6796875,
       "learning_rate": 1.544896435313997e-05,
-      "loss": 0.49010343551635743,
-      "mean_token_accuracy": 0.859997171163559,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
-      "entropy": 0.5348539352416992,
       "epoch": 2.5,
-      "grad_norm": 0.5390625,
       "learning_rate": 1.4333182031342185e-05,
-      "loss": 0.50527982711792,
-      "mean_token_accuracy": 0.8579596102237701,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
-      "entropy": 0.6260602816939353,
       "epoch": 2.5196850393700787,
-      "grad_norm": 0.49609375,
       "learning_rate": 1.3256102935148496e-05,
-      "loss": 0.5644484043121338,
-      "mean_token_accuracy": 0.8425815969705581,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
-      "entropy": 0.624223592877388,
       "epoch": 2.5393700787401574,
-      "grad_norm": 0.59375,
       "learning_rate": 1.2218213675784063e-05,
-      "loss": 0.5586697101593018,
-      "mean_token_accuracy": 0.8420278400182724,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
-      "entropy": 0.547905944287777,
       "epoch": 2.559055118110236,
-      "grad_norm": 0.43359375,
       "learning_rate": 1.1219983158982917e-05,
-      "loss": 0.49605836868286135,
-      "mean_token_accuracy": 0.8598576009273529,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
-      "eval_entropy": 0.6620006765480395,
-      "eval_loss": 0.8368703126907349,
-      "eval_mean_token_accuracy": 0.7841785737761745,
       "eval_num_tokens": 14431579.0,
-      "eval_runtime": 15.9696,
-      "eval_samples_per_second": 26.738,
-      "eval_steps_per_second": 3.381,
       "step": 1300
     },
     {
-      "entropy": 0.5208907574415207,
       "epoch": 2.578740157480315,
-      "grad_norm": 0.59765625,
       "learning_rate": 1.0261862373141839e-05,
-      "loss": 0.46456570625305177,
-      "mean_token_accuracy": 0.8702636033296585,
       "num_tokens": 14536887.0,
       "step": 1310
     },
     {
-      "entropy": 0.5214867204427719,
       "epoch": 2.5984251968503935,
-      "grad_norm": 0.54296875,
       "learning_rate": 9.344284185569463e-06,
-      "loss": 0.4646267890930176,
-      "mean_token_accuracy": 0.8666720151901245,
       "num_tokens": 14649040.0,
       "step": 1320
     },
     {
-      "entropy": 0.5516841277480126,
       "epoch": 2.6181102362204722,
-      "grad_norm": 0.4921875,
       "learning_rate": 8.467663146921979e-06,
-      "loss": 0.49184260368347166,
-      "mean_token_accuracy": 0.8584365308284759,
       "num_tokens": 14757590.0,
       "step": 1330
     },
     {
-      "entropy": 0.5462728574872017,
       "epoch": 2.637795275590551,
-      "grad_norm": 0.4765625,
       "learning_rate": 7.632395303914208e-06,
-      "loss": 0.47838892936706545,
-      "mean_token_accuracy": 0.8621183454990387,
       "num_tokens": 14870411.0,
       "step": 1340
     },
     {
-      "entropy": 0.5727479830384254,
       "epoch": 2.65748031496063,
-      "grad_norm": 0.5078125,
       "learning_rate": 6.838858020390615e-06,
-      "loss": 0.5193718433380127,
-      "mean_token_accuracy": 0.8569418281316757,
       "num_tokens": 14986705.0,
       "step": 1350
     },
     {
       "epoch": 2.65748031496063,
-      "eval_entropy": 0.6583189417918524,
-      "eval_loss": 0.8370347619056702,
-      "eval_mean_token_accuracy": 0.7841276548526905,
       "eval_num_tokens": 14986705.0,
-      "eval_runtime": 15.9498,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 1350
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     },
     {
+      "entropy": 0.5992338545620441,
       "epoch": 2.1850393700787403,
+      "grad_norm": 0.47265625,
       "learning_rate": 3.644777706542845e-05,
+      "loss": 0.5292766571044922,
+      "mean_token_accuracy": 0.8512318909168244,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
+      "entropy": 0.514266274124384,
       "epoch": 2.204724409448819,
+      "grad_norm": 0.5,
       "learning_rate": 3.4821141049373804e-05,
+      "loss": 0.4458413600921631,
+      "mean_token_accuracy": 0.8724471956491471,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
+      "entropy": 0.5649555295705795,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
+      "loss": 0.5298549652099609,
+      "mean_token_accuracy": 0.8531913965940475,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
+      "entropy": 0.5983025059103966,
       "epoch": 2.2440944881889764,
+      "grad_norm": 0.58203125,
       "learning_rate": 3.165693165312395e-05,
+      "loss": 0.5455004215240479,
+      "mean_token_accuracy": 0.8468506157398223,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
+      "entropy": 0.5571755915880203,
       "epoch": 2.263779527559055,
+      "grad_norm": 0.55859375,
       "learning_rate": 3.0120787824218554e-05,
+      "loss": 0.5133816719055175,
+      "mean_token_accuracy": 0.8542670160531998,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
+      "eval_entropy": 0.647307668571119,
+      "eval_loss": 0.844680666923523,
+      "eval_mean_token_accuracy": 0.7836638799420109,
       "eval_num_tokens": 12763743.0,
+      "eval_runtime": 15.9342,
+      "eval_samples_per_second": 26.798,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
+      "entropy": 0.6052810281515122,
       "epoch": 2.283464566929134,
+      "grad_norm": 0.482421875,
       "learning_rate": 2.8616214572789968e-05,
+      "loss": 0.5399055004119873,
+      "mean_token_accuracy": 0.8467662870883942,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
+      "entropy": 0.495437890291214,
       "epoch": 2.3031496062992125,
+      "grad_norm": 0.57421875,
       "learning_rate": 2.7143891646719265e-05,
+      "loss": 0.4656820297241211,
+      "mean_token_accuracy": 0.8652756780385971,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
+      "entropy": 0.5415738135576248,
       "epoch": 2.322834645669291,
+      "grad_norm": 0.59765625,
       "learning_rate": 2.570448422358297e-05,
+      "loss": 0.5029004573822021,
+      "mean_token_accuracy": 0.8594674915075302,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
+      "entropy": 0.5835595086216927,
       "epoch": 2.34251968503937,
+      "grad_norm": 0.6484375,
       "learning_rate": 2.4298642610134105e-05,
+      "loss": 0.502112865447998,
+      "mean_token_accuracy": 0.8509185254573822,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
+      "entropy": 0.6709924265742302,
       "epoch": 2.362204724409449,
+      "grad_norm": 0.6640625,
       "learning_rate": 2.292700194850147e-05,
+      "loss": 0.5888309001922607,
+      "mean_token_accuracy": 0.829518511891365,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
+      "eval_entropy": 0.6516638414727317,
+      "eval_loss": 0.8389837741851807,
+      "eval_mean_token_accuracy": 0.7852442959944407,
       "eval_num_tokens": 13314862.0,
+      "eval_runtime": 15.9159,
+      "eval_samples_per_second": 26.829,
+      "eval_steps_per_second": 3.393,
       "step": 1200
     },
     {
+      "entropy": 0.5645383015275002,
       "epoch": 2.3818897637795278,
+      "grad_norm": 0.5390625,
       "learning_rate": 2.159018192924004e-05,
+      "loss": 0.5259816169738769,
+      "mean_token_accuracy": 0.8538117051124573,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
+      "entropy": 0.5880276575684548,
       "epoch": 2.4015748031496065,
+      "grad_norm": 0.439453125,
       "learning_rate": 2.0288786511362525e-05,
+      "loss": 0.5333644866943359,
+      "mean_token_accuracy": 0.8482611387968063,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
+      "entropy": 0.5676271229982376,
       "epoch": 2.421259842519685,
+      "grad_norm": 0.494140625,
       "learning_rate": 1.9023403649477813e-05,
+      "loss": 0.5034989833831787,
+      "mean_token_accuracy": 0.8573888152837753,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
+      "entropy": 0.5038574181497097,
       "epoch": 2.440944881889764,
+      "grad_norm": 0.435546875,
       "learning_rate": 1.7794605028160462e-05,
+      "loss": 0.45959224700927737,
+      "mean_token_accuracy": 0.866667166352272,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
+      "entropy": 0.5891189277172089,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
+      "loss": 0.5333945274353027,
+      "mean_token_accuracy": 0.8494920641183853,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
+      "eval_entropy": 0.6562480860286288,
+      "eval_loss": 0.8404139876365662,
+      "eval_mean_token_accuracy": 0.7847034048151087,
       "eval_num_tokens": 13872536.0,
+      "eval_runtime": 15.9225,
+      "eval_samples_per_second": 26.817,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
+      "entropy": 0.5312069803476334,
       "epoch": 2.4803149606299213,
+      "grad_norm": 0.67578125,
       "learning_rate": 1.544896435313997e-05,
+      "loss": 0.48858180046081545,
+      "mean_token_accuracy": 0.860382902622223,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
+      "entropy": 0.5351930364966393,
       "epoch": 2.5,
+      "grad_norm": 0.53515625,
       "learning_rate": 1.4333182031342185e-05,
+      "loss": 0.5056577682495117,
+      "mean_token_accuracy": 0.857965275645256,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
+      "entropy": 0.6251816973090172,
       "epoch": 2.5196850393700787,
+      "grad_norm": 0.4921875,
       "learning_rate": 1.3256102935148496e-05,
+      "loss": 0.5645822048187256,
+      "mean_token_accuracy": 0.8422666400671005,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
+      "entropy": 0.6227434769272804,
       "epoch": 2.5393700787401574,
+      "grad_norm": 0.58984375,
       "learning_rate": 1.2218213675784063e-05,
+      "loss": 0.5586906909942627,
+      "mean_token_accuracy": 0.8418498873710633,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
+      "entropy": 0.5474544659256935,
       "epoch": 2.559055118110236,
+      "grad_norm": 0.4375,
       "learning_rate": 1.1219983158982917e-05,
+      "loss": 0.49657626152038575,
+      "mean_token_accuracy": 0.8581330984830856,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
+      "eval_entropy": 0.6617519998991931,
+      "eval_loss": 0.8383262157440186,
+      "eval_mean_token_accuracy": 0.7846880113637006,
       "eval_num_tokens": 14431579.0,
+      "eval_runtime": 15.9113,
+      "eval_samples_per_second": 26.836,
+      "eval_steps_per_second": 3.394,
       "step": 1300
     },
     {
+      "entropy": 0.5216698408126831,
       "epoch": 2.578740157480315,
+      "grad_norm": 0.6015625,
       "learning_rate": 1.0261862373141839e-05,
+      "loss": 0.46427440643310547,
+      "mean_token_accuracy": 0.8688691437244416,
       "num_tokens": 14536887.0,
       "step": 1310
     },
     {
+      "entropy": 0.5210306510329247,
       "epoch": 2.5984251968503935,
+      "grad_norm": 0.5625,
       "learning_rate": 9.344284185569463e-06,
+      "loss": 0.4667550563812256,
+      "mean_token_accuracy": 0.8664857149124146,
       "num_tokens": 14649040.0,
       "step": 1320
     },
     {
+      "entropy": 0.5522045940160751,
       "epoch": 2.6181102362204722,
+      "grad_norm": 0.494140625,
       "learning_rate": 8.467663146921979e-06,
+      "loss": 0.4914377689361572,
+      "mean_token_accuracy": 0.8588613420724869,
       "num_tokens": 14757590.0,
       "step": 1330
     },
     {
+      "entropy": 0.5440365478396416,
       "epoch": 2.637795275590551,
+      "grad_norm": 0.470703125,
       "learning_rate": 7.632395303914208e-06,
+      "loss": 0.4774484634399414,
+      "mean_token_accuracy": 0.8620693862438202,
       "num_tokens": 14870411.0,
       "step": 1340
     },
     {
+      "entropy": 0.5705034703016281,
       "epoch": 2.65748031496063,
+      "grad_norm": 0.515625,
       "learning_rate": 6.838858020390615e-06,
+      "loss": 0.5196453094482422,
+      "mean_token_accuracy": 0.8570811003446579,
       "num_tokens": 14986705.0,
       "step": 1350
     },
     {
       "epoch": 2.65748031496063,
+      "eval_entropy": 0.6574608446271332,
+      "eval_loss": 0.8387162685394287,
+      "eval_mean_token_accuracy": 0.7850221616250498,
       "eval_num_tokens": 14986705.0,
+      "eval_runtime": 15.9224,
+      "eval_samples_per_second": 26.818,
+      "eval_steps_per_second": 3.391,
       "step": 1350
     }
   ],

checkpoint-1400/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1400/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fc3de7b7546415dac4b46bf0957d5c0cf0b422cb501174f3bf003b1cdc36ca74
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:ed7e12e4889b4de282be654380676aa38c8c92f07579c104f4d0bc217ca3be41
 size 73911504

checkpoint-1400/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:fafff5754437ffd1e5c3295da973f6a7dbd72e4b17c8de501e814109a8ba19d9
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:ad6bcfbde460dbeb220efa120f5aa570d0f7af47423026672549f86e0d183347
 size 75481221

checkpoint-1400/trainer_state.json CHANGED Viewed

@@ -10,1711 +10,1711 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     },
     {
-      "entropy": 0.5992730811238289,
       "epoch": 2.1850393700787403,
-      "grad_norm": 0.474609375,
       "learning_rate": 3.644777706542845e-05,
-      "loss": 0.5308849811553955,
-      "mean_token_accuracy": 0.8519498467445373,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
-      "entropy": 0.5141457825899124,
       "epoch": 2.204724409448819,
-      "grad_norm": 0.49609375,
       "learning_rate": 3.4821141049373804e-05,
-      "loss": 0.4474374294281006,
-      "mean_token_accuracy": 0.8720535308122634,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
-      "entropy": 0.5664690762758255,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
-      "loss": 0.5310683250427246,
-      "mean_token_accuracy": 0.8515143841505051,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
-      "entropy": 0.5990463197231293,
       "epoch": 2.2440944881889764,
-      "grad_norm": 0.58984375,
       "learning_rate": 3.165693165312395e-05,
-      "loss": 0.5457014083862305,
-      "mean_token_accuracy": 0.8466474860906601,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
-      "entropy": 0.5571582794189454,
       "epoch": 2.263779527559055,
-      "grad_norm": 0.55078125,
       "learning_rate": 3.0120787824218554e-05,
-      "loss": 0.5153813362121582,
-      "mean_token_accuracy": 0.8554807037115097,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
-      "eval_entropy": 0.65144182741642,
-      "eval_loss": 0.8418021202087402,
-      "eval_mean_token_accuracy": 0.7838731220474949,
       "eval_num_tokens": 12763743.0,
-      "eval_runtime": 15.9352,
-      "eval_samples_per_second": 26.796,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
-      "entropy": 0.6080173872411251,
       "epoch": 2.283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 2.8616214572789968e-05,
-      "loss": 0.5399338245391846,
-      "mean_token_accuracy": 0.8469231188297272,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
-      "entropy": 0.49714402854442596,
       "epoch": 2.3031496062992125,
-      "grad_norm": 0.578125,
       "learning_rate": 2.7143891646719265e-05,
-      "loss": 0.46820721626281736,
-      "mean_token_accuracy": 0.8644533902406693,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
-      "entropy": 0.5437434703111649,
       "epoch": 2.322834645669291,
-      "grad_norm": 0.6015625,
       "learning_rate": 2.570448422358297e-05,
-      "loss": 0.5030812740325927,
-      "mean_token_accuracy": 0.8589521288871765,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
-      "entropy": 0.5845290660858155,
       "epoch": 2.34251968503937,
-      "grad_norm": 0.60546875,
       "learning_rate": 2.4298642610134105e-05,
-      "loss": 0.5023557186126709,
-      "mean_token_accuracy": 0.8512013405561447,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
-      "entropy": 0.6722325071692467,
       "epoch": 2.362204724409449,
-      "grad_norm": 0.66015625,
       "learning_rate": 2.292700194850147e-05,
-      "loss": 0.5894911289215088,
-      "mean_token_accuracy": 0.8302695482969285,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
-      "eval_entropy": 0.6533891520014515,
-      "eval_loss": 0.8373947739601135,
-      "eval_mean_token_accuracy": 0.7844164106580946,
       "eval_num_tokens": 13314862.0,
-      "eval_runtime": 15.9338,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 1200
     },
     {
-      "entropy": 0.5659365668892861,
       "epoch": 2.3818897637795278,
-      "grad_norm": 0.5078125,
       "learning_rate": 2.159018192924004e-05,
-      "loss": 0.5257106304168702,
-      "mean_token_accuracy": 0.8536411643028259,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
-      "entropy": 0.5879238858819008,
       "epoch": 2.4015748031496065,
-      "grad_norm": 0.44921875,
       "learning_rate": 2.0288786511362525e-05,
-      "loss": 0.5346655368804931,
-      "mean_token_accuracy": 0.8476213365793228,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
-      "entropy": 0.5665464416146279,
       "epoch": 2.421259842519685,
-      "grad_norm": 0.4921875,
       "learning_rate": 1.9023403649477813e-05,
-      "loss": 0.5051110267639161,
-      "mean_token_accuracy": 0.8583743631839752,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
-      "entropy": 0.5041692733764649,
       "epoch": 2.440944881889764,
-      "grad_norm": 0.453125,
       "learning_rate": 1.7794605028160462e-05,
-      "loss": 0.46024274826049805,
-      "mean_token_accuracy": 0.8660459965467453,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
-      "entropy": 0.5898029208183289,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
-      "loss": 0.5336055755615234,
-      "mean_token_accuracy": 0.8486702740192413,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
-      "eval_entropy": 0.6570240441295836,
-      "eval_loss": 0.8387707471847534,
-      "eval_mean_token_accuracy": 0.7845993064067982,
       "eval_num_tokens": 13872536.0,
-      "eval_runtime": 15.9231,
-      "eval_samples_per_second": 26.816,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
-      "entropy": 0.5317887738347054,
       "epoch": 2.4803149606299213,
-      "grad_norm": 0.6796875,
       "learning_rate": 1.544896435313997e-05,
-      "loss": 0.49010343551635743,
-      "mean_token_accuracy": 0.859997171163559,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
-      "entropy": 0.5348539352416992,
       "epoch": 2.5,
-      "grad_norm": 0.5390625,
       "learning_rate": 1.4333182031342185e-05,
-      "loss": 0.50527982711792,
-      "mean_token_accuracy": 0.8579596102237701,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
-      "entropy": 0.6260602816939353,
       "epoch": 2.5196850393700787,
-      "grad_norm": 0.49609375,
       "learning_rate": 1.3256102935148496e-05,
-      "loss": 0.5644484043121338,
-      "mean_token_accuracy": 0.8425815969705581,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
-      "entropy": 0.624223592877388,
       "epoch": 2.5393700787401574,
-      "grad_norm": 0.59375,
       "learning_rate": 1.2218213675784063e-05,
-      "loss": 0.5586697101593018,
-      "mean_token_accuracy": 0.8420278400182724,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
-      "entropy": 0.547905944287777,
       "epoch": 2.559055118110236,
-      "grad_norm": 0.43359375,
       "learning_rate": 1.1219983158982917e-05,
-      "loss": 0.49605836868286135,
-      "mean_token_accuracy": 0.8598576009273529,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
-      "eval_entropy": 0.6620006765480395,
-      "eval_loss": 0.8368703126907349,
-      "eval_mean_token_accuracy": 0.7841785737761745,
       "eval_num_tokens": 14431579.0,
-      "eval_runtime": 15.9696,
-      "eval_samples_per_second": 26.738,
-      "eval_steps_per_second": 3.381,
       "step": 1300
     },
     {
-      "entropy": 0.5208907574415207,
       "epoch": 2.578740157480315,
-      "grad_norm": 0.59765625,
       "learning_rate": 1.0261862373141839e-05,
-      "loss": 0.46456570625305177,
-      "mean_token_accuracy": 0.8702636033296585,
       "num_tokens": 14536887.0,
       "step": 1310
     },
     {
-      "entropy": 0.5214867204427719,
       "epoch": 2.5984251968503935,
-      "grad_norm": 0.54296875,
       "learning_rate": 9.344284185569463e-06,
-      "loss": 0.4646267890930176,
-      "mean_token_accuracy": 0.8666720151901245,
       "num_tokens": 14649040.0,
       "step": 1320
     },
     {
-      "entropy": 0.5516841277480126,
       "epoch": 2.6181102362204722,
-      "grad_norm": 0.4921875,
       "learning_rate": 8.467663146921979e-06,
-      "loss": 0.49184260368347166,
-      "mean_token_accuracy": 0.8584365308284759,
       "num_tokens": 14757590.0,
       "step": 1330
     },
     {
-      "entropy": 0.5462728574872017,
       "epoch": 2.637795275590551,
-      "grad_norm": 0.4765625,
       "learning_rate": 7.632395303914208e-06,
-      "loss": 0.47838892936706545,
-      "mean_token_accuracy": 0.8621183454990387,
       "num_tokens": 14870411.0,
       "step": 1340
     },
     {
-      "entropy": 0.5727479830384254,
       "epoch": 2.65748031496063,
-      "grad_norm": 0.5078125,
       "learning_rate": 6.838858020390615e-06,
-      "loss": 0.5193718433380127,
-      "mean_token_accuracy": 0.8569418281316757,
       "num_tokens": 14986705.0,
       "step": 1350
     },
     {
       "epoch": 2.65748031496063,
-      "eval_entropy": 0.6583189417918524,
-      "eval_loss": 0.8370347619056702,
-      "eval_mean_token_accuracy": 0.7841276548526905,
       "eval_num_tokens": 14986705.0,
-      "eval_runtime": 15.9498,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 1350
     },
     {
-      "entropy": 0.5645975567400455,
       "epoch": 2.677165354330709,
       "grad_norm": 0.640625,
       "learning_rate": 6.087409806836752e-06,
-      "loss": 0.5261539936065673,
-      "mean_token_accuracy": 0.8559167087078094,
       "num_tokens": 15095469.0,
       "step": 1360
     },
     {
-      "entropy": 0.5451949246227741,
       "epoch": 2.6968503937007875,
       "grad_norm": 0.5390625,
       "learning_rate": 5.378390158408731e-06,
-      "loss": 0.5070470333099365,
-      "mean_token_accuracy": 0.8579351454973221,
       "num_tokens": 15202248.0,
       "step": 1370
     },
     {
-      "entropy": 0.5048372894525528,
       "epoch": 2.716535433070866,
-      "grad_norm": 0.48828125,
       "learning_rate": 4.712119401553394e-06,
-      "loss": 0.45394439697265626,
-      "mean_token_accuracy": 0.8699903905391693,
       "num_tokens": 15312525.0,
       "step": 1380
     },
     {
-      "entropy": 0.5720473691821099,
       "epoch": 2.736220472440945,
-      "grad_norm": 0.439453125,
       "learning_rate": 4.088898549288689e-06,
-      "loss": 0.5240915298461915,
-      "mean_token_accuracy": 0.8507357180118561,
       "num_tokens": 15424901.0,
       "step": 1390
     },
     {
-      "entropy": 0.5502833247184753,
       "epoch": 2.7559055118110236,
-      "grad_norm": 0.51953125,
       "learning_rate": 3.5090091652096977e-06,
-      "loss": 0.49132513999938965,
-      "mean_token_accuracy": 0.861817353963852,
       "num_tokens": 15534177.0,
       "step": 1400
     },
     {
       "epoch": 2.7559055118110236,
-      "eval_entropy": 0.6594855669471953,
-      "eval_loss": 0.8369887471199036,
-      "eval_mean_token_accuracy": 0.7841665005242383,
       "eval_num_tokens": 15534177.0,
-      "eval_runtime": 15.9489,
-      "eval_samples_per_second": 26.773,
-      "eval_steps_per_second": 3.386,
       "step": 1400
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     },
     {
+      "entropy": 0.5992338545620441,
       "epoch": 2.1850393700787403,
+      "grad_norm": 0.47265625,
       "learning_rate": 3.644777706542845e-05,
+      "loss": 0.5292766571044922,
+      "mean_token_accuracy": 0.8512318909168244,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
+      "entropy": 0.514266274124384,
       "epoch": 2.204724409448819,
+      "grad_norm": 0.5,
       "learning_rate": 3.4821141049373804e-05,
+      "loss": 0.4458413600921631,
+      "mean_token_accuracy": 0.8724471956491471,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
+      "entropy": 0.5649555295705795,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
+      "loss": 0.5298549652099609,
+      "mean_token_accuracy": 0.8531913965940475,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
+      "entropy": 0.5983025059103966,
       "epoch": 2.2440944881889764,
+      "grad_norm": 0.58203125,
       "learning_rate": 3.165693165312395e-05,
+      "loss": 0.5455004215240479,
+      "mean_token_accuracy": 0.8468506157398223,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
+      "entropy": 0.5571755915880203,
       "epoch": 2.263779527559055,
+      "grad_norm": 0.55859375,
       "learning_rate": 3.0120787824218554e-05,
+      "loss": 0.5133816719055175,
+      "mean_token_accuracy": 0.8542670160531998,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
+      "eval_entropy": 0.647307668571119,
+      "eval_loss": 0.844680666923523,
+      "eval_mean_token_accuracy": 0.7836638799420109,
       "eval_num_tokens": 12763743.0,
+      "eval_runtime": 15.9342,
+      "eval_samples_per_second": 26.798,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
+      "entropy": 0.6052810281515122,
       "epoch": 2.283464566929134,
+      "grad_norm": 0.482421875,
       "learning_rate": 2.8616214572789968e-05,
+      "loss": 0.5399055004119873,
+      "mean_token_accuracy": 0.8467662870883942,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
+      "entropy": 0.495437890291214,
       "epoch": 2.3031496062992125,
+      "grad_norm": 0.57421875,
       "learning_rate": 2.7143891646719265e-05,
+      "loss": 0.4656820297241211,
+      "mean_token_accuracy": 0.8652756780385971,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
+      "entropy": 0.5415738135576248,
       "epoch": 2.322834645669291,
+      "grad_norm": 0.59765625,
       "learning_rate": 2.570448422358297e-05,
+      "loss": 0.5029004573822021,
+      "mean_token_accuracy": 0.8594674915075302,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
+      "entropy": 0.5835595086216927,
       "epoch": 2.34251968503937,
+      "grad_norm": 0.6484375,
       "learning_rate": 2.4298642610134105e-05,
+      "loss": 0.502112865447998,
+      "mean_token_accuracy": 0.8509185254573822,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
+      "entropy": 0.6709924265742302,
       "epoch": 2.362204724409449,
+      "grad_norm": 0.6640625,
       "learning_rate": 2.292700194850147e-05,
+      "loss": 0.5888309001922607,
+      "mean_token_accuracy": 0.829518511891365,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
+      "eval_entropy": 0.6516638414727317,
+      "eval_loss": 0.8389837741851807,
+      "eval_mean_token_accuracy": 0.7852442959944407,
       "eval_num_tokens": 13314862.0,
+      "eval_runtime": 15.9159,
+      "eval_samples_per_second": 26.829,
+      "eval_steps_per_second": 3.393,
       "step": 1200
     },
     {
+      "entropy": 0.5645383015275002,
       "epoch": 2.3818897637795278,
+      "grad_norm": 0.5390625,
       "learning_rate": 2.159018192924004e-05,
+      "loss": 0.5259816169738769,
+      "mean_token_accuracy": 0.8538117051124573,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
+      "entropy": 0.5880276575684548,
       "epoch": 2.4015748031496065,
+      "grad_norm": 0.439453125,
       "learning_rate": 2.0288786511362525e-05,
+      "loss": 0.5333644866943359,
+      "mean_token_accuracy": 0.8482611387968063,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
+      "entropy": 0.5676271229982376,
       "epoch": 2.421259842519685,
+      "grad_norm": 0.494140625,
       "learning_rate": 1.9023403649477813e-05,
+      "loss": 0.5034989833831787,
+      "mean_token_accuracy": 0.8573888152837753,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
+      "entropy": 0.5038574181497097,
       "epoch": 2.440944881889764,
+      "grad_norm": 0.435546875,
       "learning_rate": 1.7794605028160462e-05,
+      "loss": 0.45959224700927737,
+      "mean_token_accuracy": 0.866667166352272,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
+      "entropy": 0.5891189277172089,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
+      "loss": 0.5333945274353027,
+      "mean_token_accuracy": 0.8494920641183853,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
+      "eval_entropy": 0.6562480860286288,
+      "eval_loss": 0.8404139876365662,
+      "eval_mean_token_accuracy": 0.7847034048151087,
       "eval_num_tokens": 13872536.0,
+      "eval_runtime": 15.9225,
+      "eval_samples_per_second": 26.817,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
+      "entropy": 0.5312069803476334,
       "epoch": 2.4803149606299213,
+      "grad_norm": 0.67578125,
       "learning_rate": 1.544896435313997e-05,
+      "loss": 0.48858180046081545,
+      "mean_token_accuracy": 0.860382902622223,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
+      "entropy": 0.5351930364966393,
       "epoch": 2.5,
+      "grad_norm": 0.53515625,
       "learning_rate": 1.4333182031342185e-05,
+      "loss": 0.5056577682495117,
+      "mean_token_accuracy": 0.857965275645256,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
+      "entropy": 0.6251816973090172,
       "epoch": 2.5196850393700787,
+      "grad_norm": 0.4921875,
       "learning_rate": 1.3256102935148496e-05,
+      "loss": 0.5645822048187256,
+      "mean_token_accuracy": 0.8422666400671005,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
+      "entropy": 0.6227434769272804,
       "epoch": 2.5393700787401574,
+      "grad_norm": 0.58984375,
       "learning_rate": 1.2218213675784063e-05,
+      "loss": 0.5586906909942627,
+      "mean_token_accuracy": 0.8418498873710633,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
+      "entropy": 0.5474544659256935,
       "epoch": 2.559055118110236,
+      "grad_norm": 0.4375,
       "learning_rate": 1.1219983158982917e-05,
+      "loss": 0.49657626152038575,
+      "mean_token_accuracy": 0.8581330984830856,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
+      "eval_entropy": 0.6617519998991931,
+      "eval_loss": 0.8383262157440186,
+      "eval_mean_token_accuracy": 0.7846880113637006,
       "eval_num_tokens": 14431579.0,
+      "eval_runtime": 15.9113,
+      "eval_samples_per_second": 26.836,
+      "eval_steps_per_second": 3.394,
       "step": 1300
     },
     {
+      "entropy": 0.5216698408126831,
       "epoch": 2.578740157480315,
+      "grad_norm": 0.6015625,
       "learning_rate": 1.0261862373141839e-05,
+      "loss": 0.46427440643310547,
+      "mean_token_accuracy": 0.8688691437244416,
       "num_tokens": 14536887.0,
       "step": 1310
     },
     {
+      "entropy": 0.5210306510329247,
       "epoch": 2.5984251968503935,
+      "grad_norm": 0.5625,
       "learning_rate": 9.344284185569463e-06,
+      "loss": 0.4667550563812256,
+      "mean_token_accuracy": 0.8664857149124146,
       "num_tokens": 14649040.0,
       "step": 1320
     },
     {
+      "entropy": 0.5522045940160751,
       "epoch": 2.6181102362204722,
+      "grad_norm": 0.494140625,
       "learning_rate": 8.467663146921979e-06,
+      "loss": 0.4914377689361572,
+      "mean_token_accuracy": 0.8588613420724869,
       "num_tokens": 14757590.0,
       "step": 1330
     },
     {
+      "entropy": 0.5440365478396416,
       "epoch": 2.637795275590551,
+      "grad_norm": 0.470703125,
       "learning_rate": 7.632395303914208e-06,
+      "loss": 0.4774484634399414,
+      "mean_token_accuracy": 0.8620693862438202,
       "num_tokens": 14870411.0,
       "step": 1340
     },
     {
+      "entropy": 0.5705034703016281,
       "epoch": 2.65748031496063,
+      "grad_norm": 0.515625,
       "learning_rate": 6.838858020390615e-06,
+      "loss": 0.5196453094482422,
+      "mean_token_accuracy": 0.8570811003446579,
       "num_tokens": 14986705.0,
       "step": 1350
     },
     {
       "epoch": 2.65748031496063,
+      "eval_entropy": 0.6574608446271332,
+      "eval_loss": 0.8387162685394287,
+      "eval_mean_token_accuracy": 0.7850221616250498,
       "eval_num_tokens": 14986705.0,
+      "eval_runtime": 15.9224,
+      "eval_samples_per_second": 26.818,
+      "eval_steps_per_second": 3.391,
       "step": 1350
     },
     {
+      "entropy": 0.563168890029192,
       "epoch": 2.677165354330709,
       "grad_norm": 0.640625,
       "learning_rate": 6.087409806836752e-06,
+      "loss": 0.5263133525848389,
+      "mean_token_accuracy": 0.8563782393932342,
       "num_tokens": 15095469.0,
       "step": 1360
     },
     {
+      "entropy": 0.5424789242446423,
       "epoch": 2.6968503937007875,
       "grad_norm": 0.5390625,
       "learning_rate": 5.378390158408731e-06,
+      "loss": 0.5066217422485352,
+      "mean_token_accuracy": 0.8590426325798035,
       "num_tokens": 15202248.0,
       "step": 1370
     },
     {
+      "entropy": 0.5028943344950676,
       "epoch": 2.716535433070866,
+      "grad_norm": 0.486328125,
       "learning_rate": 4.712119401553394e-06,
+      "loss": 0.4527781009674072,
+      "mean_token_accuracy": 0.8694937527179718,
       "num_tokens": 15312525.0,
       "step": 1380
     },
     {
+      "entropy": 0.5716224625706673,
       "epoch": 2.736220472440945,
+      "grad_norm": 0.427734375,
       "learning_rate": 4.088898549288689e-06,
+      "loss": 0.5240719795227051,
+      "mean_token_accuracy": 0.8516256272792816,
       "num_tokens": 15424901.0,
       "step": 1390
     },
     {
+      "entropy": 0.5507901981472969,
       "epoch": 2.7559055118110236,
+      "grad_norm": 0.5234375,
       "learning_rate": 3.5090091652096977e-06,
+      "loss": 0.490921688079834,
+      "mean_token_accuracy": 0.8620281994342804,
       "num_tokens": 15534177.0,
       "step": 1400
     },
     {
       "epoch": 2.7559055118110236,
+      "eval_entropy": 0.659479867528986,
+      "eval_loss": 0.8384642601013184,
+      "eval_mean_token_accuracy": 0.7847910379921949,
       "eval_num_tokens": 15534177.0,
+      "eval_runtime": 15.9298,
+      "eval_samples_per_second": 26.805,
+      "eval_steps_per_second": 3.39,
       "step": 1400
     }
   ],

checkpoint-1450/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-1450/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:462d63bf9d09dc1538e0eaa5f3510e061f56f34dcdf0c146621f7a6e5d7e9935
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:32cb82f5859918f33f3abaadf148d9df8540f2f5f5b1b99142f4c97819ccecaf
 size 73911504

checkpoint-1450/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:9670371916bad55dd2eb2576d7a5ed989ee8e5fbdcdfdd0ad30fa499c0e90a4e
 size 75481221

 version https://git-lfs.github.com/spec/v1
+oid sha256:859b80f66a1be88fcb1dd6d00da1fbf673d850b6af4fe6612cf7edcdd2fa7827
 size 75481221

checkpoint-1450/trainer_state.json CHANGED Viewed

@@ -10,1772 +10,1772 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     },
     {
-      "entropy": 0.769407157599926,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
-      "loss": 0.7655523777008056,
-      "mean_token_accuracy": 0.8053037762641907,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
-      "entropy": 0.8818381905555726,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
-      "loss": 0.8408863067626953,
-      "mean_token_accuracy": 0.7815578579902649,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
-      "entropy": 0.8344995141029358,
       "epoch": 0.3543307086614173,
-      "grad_norm": 0.4765625,
       "learning_rate": 0.0001960305545112445,
-      "loss": 0.8205550193786622,
-      "mean_token_accuracy": 0.7871369570493698,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
-      "entropy": 0.8058241099119187,
       "epoch": 0.37401574803149606,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.0001954159780522056,
-      "loss": 0.8002825736999511,
-      "mean_token_accuracy": 0.7913093209266663,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
-      "entropy": 0.8938606142997741,
       "epoch": 0.3937007874015748,
-      "grad_norm": 0.61328125,
       "learning_rate": 0.00019475829381570566,
-      "loss": 0.8305434226989746,
-      "mean_token_accuracy": 0.7836446255445481,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
-      "eval_entropy": 0.9502577009024443,
-      "eval_loss": 0.8978050351142883,
-      "eval_mean_token_accuracy": 0.7696934717672842,
       "eval_num_tokens": 2195494.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 200
     },
     {
-      "entropy": 0.7890492826700211,
       "epoch": 0.41338582677165353,
-      "grad_norm": 0.40625,
       "learning_rate": 0.0001940577989354773,
-      "loss": 0.7953387260437011,
-      "mean_token_accuracy": 0.7982505321502685,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
-      "entropy": 0.8576261162757873,
       "epoch": 0.4330708661417323,
-      "grad_norm": 0.66015625,
       "learning_rate": 0.00019331480988658083,
-      "loss": 0.8120486259460449,
-      "mean_token_accuracy": 0.7829198718070984,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
-      "entropy": 0.8514308303594589,
       "epoch": 0.452755905511811,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00019252966234242468,
-      "loss": 0.8203784942626953,
-      "mean_token_accuracy": 0.7819343030452728,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
-      "entropy": 0.8994888842105866,
       "epoch": 0.47244094488188976,
-      "grad_norm": 0.59765625,
       "learning_rate": 0.00019170271102311238,
-      "loss": 0.9019557952880859,
-      "mean_token_accuracy": 0.7712395042181015,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
-      "entropy": 0.7910601049661636,
       "epoch": 0.4921259842519685,
-      "grad_norm": 0.494140625,
       "learning_rate": 0.00019083432953518418,
-      "loss": 0.8107678413391113,
-      "mean_token_accuracy": 0.7996705889701843,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
-      "eval_entropy": 0.9037868281205496,
-      "eval_loss": 0.8843846917152405,
-      "eval_mean_token_accuracy": 0.7717091352851303,
       "eval_num_tokens": 2741031.0,
-      "eval_runtime": 15.93,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 250
     },
     {
-      "entropy": 0.7740790158510208,
       "epoch": 0.5118110236220472,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.00018992491020282618,
-      "loss": 0.7579652309417725,
-      "mean_token_accuracy": 0.7955388814210892,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
-      "entropy": 0.8572690159082412,
       "epoch": 0.531496062992126,
-      "grad_norm": 1.171875,
       "learning_rate": 0.0001889748638906231,
-      "loss": 0.8072081565856933,
-      "mean_token_accuracy": 0.7910655379295349,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
-      "entropy": 0.8885004252195359,
       "epoch": 0.5511811023622047,
-      "grad_norm": 0.451171875,
       "learning_rate": 0.00018798461981793462,
-      "loss": 0.8460770606994629,
-      "mean_token_accuracy": 0.7776093751192092,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
-      "entropy": 0.829613895714283,
       "epoch": 0.5708661417322834,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00018695462536497934,
-      "loss": 0.823792839050293,
-      "mean_token_accuracy": 0.7848509013652801,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
-      "entropy": 0.8341330245137215,
       "epoch": 0.5905511811023622,
-      "grad_norm": 0.490234375,
       "learning_rate": 0.00018588534587071385,
-      "loss": 0.810818099975586,
-      "mean_token_accuracy": 0.7906568497419357,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
-      "eval_entropy": 0.8445847459413387,
-      "eval_loss": 0.8718247413635254,
-      "eval_mean_token_accuracy": 0.774838043583764,
       "eval_num_tokens": 3297319.0,
-      "eval_runtime": 15.9561,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 300
     },
     {
-      "entropy": 0.8317201763391495,
       "epoch": 0.610236220472441,
-      "grad_norm": 0.396484375,
       "learning_rate": 0.00018477726442259806,
-      "loss": 0.80075101852417,
-      "mean_token_accuracy": 0.7886052161455155,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
-      "entropy": 0.8597566410899162,
       "epoch": 0.6299212598425197,
-      "grad_norm": 0.404296875,
       "learning_rate": 0.00018363088163834252,
-      "loss": 0.8123328208923339,
-      "mean_token_accuracy": 0.7854531407356262,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
-      "entropy": 0.8519140958786011,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
-      "loss": 0.8214571952819825,
-      "mean_token_accuracy": 0.7828344345092774,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
-      "entropy": 0.7781533747911453,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
-      "loss": 0.753899097442627,
-      "mean_token_accuracy": 0.8011120349168778,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
-      "entropy": 0.8384901583194733,
       "epoch": 0.6889763779527559,
-      "grad_norm": 0.51171875,
       "learning_rate": 0.0001799671895953471,
-      "loss": 0.8112489700317382,
-      "mean_token_accuracy": 0.7898134678602219,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
-      "eval_entropy": 0.8795759567507991,
-      "eval_loss": 0.8568941354751587,
-      "eval_mean_token_accuracy": 0.7754804182935644,
       "eval_num_tokens": 3870377.0,
-      "eval_runtime": 15.9337,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 350
     },
     {
-      "entropy": 0.8008699864149094,
       "epoch": 0.7086614173228346,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.00017867295016916167,
-      "loss": 0.7585730075836181,
-      "mean_token_accuracy": 0.7942175149917603,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
-      "entropy": 0.9035080641508102,
       "epoch": 0.7283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 0.0001773431672617132,
-      "loss": 0.8653995513916015,
-      "mean_token_accuracy": 0.7730168104171753,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
-      "entropy": 0.760843014717102,
       "epoch": 0.7480314960629921,
-      "grad_norm": 0.49609375,
       "learning_rate": 0.00017597844165273286,
-      "loss": 0.7377382278442383,
-      "mean_token_accuracy": 0.7972156703472137,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
-      "entropy": 0.8075762838125229,
       "epoch": 0.7677165354330708,
-      "grad_norm": 0.447265625,
       "learning_rate": 0.00017457938990863908,
-      "loss": 0.7615089893341065,
-      "mean_token_accuracy": 0.8005614727735519,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
-      "entropy": 0.8942124575376511,
       "epoch": 0.7874015748031497,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.00017314664410398033,
-      "loss": 0.8365015983581543,
-      "mean_token_accuracy": 0.7790710270404816,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
-      "eval_entropy": 0.8613758815659417,
-      "eval_loss": 0.8455358743667603,
-      "eval_mean_token_accuracy": 0.7789811304321995,
       "eval_num_tokens": 4414002.0,
-      "eval_runtime": 16.2401,
-      "eval_samples_per_second": 26.293,
-      "eval_steps_per_second": 3.325,
       "step": 400
     },
     {
-      "entropy": 0.7841676041483879,
       "epoch": 0.8070866141732284,
-      "grad_norm": 0.57421875,
       "learning_rate": 0.0001716808515358714,
-      "loss": 0.8027224540710449,
-      "mean_token_accuracy": 0.79732064306736,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
-      "entropy": 0.7710117936134339,
       "epoch": 0.8267716535433071,
-      "grad_norm": 0.4453125,
       "learning_rate": 0.00017018267443155268,
-      "loss": 0.7185665130615234,
-      "mean_token_accuracy": 0.807221668958664,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
-      "entropy": 0.8494665056467057,
       "epoch": 0.8464566929133859,
-      "grad_norm": 0.453125,
       "learning_rate": 0.00016865278964920378,
-      "loss": 0.8273701667785645,
-      "mean_token_accuracy": 0.7870708376169204,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
-      "entropy": 0.7938957527279854,
       "epoch": 0.8661417322834646,
-      "grad_norm": 0.345703125,
       "learning_rate": 0.00016709188837214732,
-      "loss": 0.7776453971862793,
-      "mean_token_accuracy": 0.7911081612110138,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
-      "entropy": 0.7948901996016502,
       "epoch": 0.8858267716535433,
-      "grad_norm": 0.376953125,
       "learning_rate": 0.0001655006757965806,
-      "loss": 0.7554664611816406,
-      "mean_token_accuracy": 0.7939331024885178,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
-      "eval_entropy": 0.8405309479545664,
-      "eval_loss": 0.8371886610984802,
-      "eval_mean_token_accuracy": 0.7799682672376986,
       "eval_num_tokens": 4984410.0,
-      "eval_runtime": 15.9419,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 450
     },
     {
-      "entropy": 0.8764406844973565,
       "epoch": 0.905511811023622,
-      "grad_norm": 0.5625,
       "learning_rate": 0.0001638798708129768,
-      "loss": 0.8325652122497559,
-      "mean_token_accuracy": 0.7821455717086792,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
-      "entropy": 0.8582975655794144,
       "epoch": 0.9251968503937008,
-      "grad_norm": 0.3828125,
       "learning_rate": 0.00016223020568129854,
-      "loss": 0.8309861183166504,
-      "mean_token_accuracy": 0.7791573971509933,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
-      "entropy": 0.8273941457271576,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
-      "loss": 0.7878972053527832,
-      "mean_token_accuracy": 0.7900911033153534,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
-      "entropy": 0.8146103203296662,
       "epoch": 0.9645669291338582,
-      "grad_norm": 0.42578125,
       "learning_rate": 0.00015884728887017162,
-      "loss": 0.7865399837493896,
-      "mean_token_accuracy": 0.7947310745716095,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
-      "entropy": 0.6613125756382943,
       "epoch": 0.984251968503937,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.0001571155655513606,
-      "loss": 0.6725417137145996,
-      "mean_token_accuracy": 0.816859421133995,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
-      "eval_entropy": 0.8118620448642306,
-      "eval_loss": 0.8308539986610413,
-      "eval_mean_token_accuracy": 0.781522473803273,
       "eval_num_tokens": 5544086.0,
-      "eval_runtime": 15.9383,
-      "eval_samples_per_second": 26.791,
-      "eval_steps_per_second": 3.388,
       "step": 500
     },
     {
-      "entropy": 0.7347942754626274,
       "epoch": 1.0039370078740157,
-      "grad_norm": 0.39453125,
       "learning_rate": 0.0001553580381152559,
-      "loss": 0.6915054798126221,
-      "mean_token_accuracy": 0.8176700383424759,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
-      "entropy": 0.7550317242741584,
       "epoch": 1.0236220472440944,
-      "grad_norm": 0.4921875,
       "learning_rate": 0.0001535755005913602,
-      "loss": 0.6865068912506104,
-      "mean_token_accuracy": 0.8140248268842697,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
-      "entropy": 0.7479684263467788,
       "epoch": 1.0433070866141732,
-      "grad_norm": 0.60546875,
       "learning_rate": 0.00015176875830842938,
-      "loss": 0.7037695407867431,
-      "mean_token_accuracy": 0.8133333414793015,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
-      "entropy": 0.6792907789349556,
       "epoch": 1.0629921259842519,
-      "grad_norm": 0.439453125,
       "learning_rate": 0.0001499386275306348,
-      "loss": 0.640275239944458,
-      "mean_token_accuracy": 0.8249545454978943,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
-      "entropy": 0.6819084867835045,
       "epoch": 1.0826771653543308,
-      "grad_norm": 0.41015625,
       "learning_rate": 0.0001480859350887852,
-      "loss": 0.6469451904296875,
-      "mean_token_accuracy": 0.8235166579484939,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
-      "eval_entropy": 0.8042021377219094,
-      "eval_loss": 0.8352769017219543,
-      "eval_mean_token_accuracy": 0.7806271998970596,
       "eval_num_tokens": 6112085.0,
-      "eval_runtime": 15.9298,
-      "eval_samples_per_second": 26.805,
-      "eval_steps_per_second": 3.39,
       "step": 550
     },
     {
-      "entropy": 0.7026952266693115,
       "epoch": 1.1023622047244095,
-      "grad_norm": 0.47265625,
       "learning_rate": 0.00014621151800677434,
-      "loss": 0.667764139175415,
-      "mean_token_accuracy": 0.8164290696382522,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
-      "entropy": 0.6083555489778518,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
-      "loss": 0.5703843116760254,
-      "mean_token_accuracy": 0.8396952480077744,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
-      "entropy": 0.6823922500014306,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
-      "loss": 0.6488718032836914,
-      "mean_token_accuracy": 0.8208951652050018,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
-      "entropy": 0.6331134095788002,
       "epoch": 1.1614173228346456,
-      "grad_norm": 0.482421875,
       "learning_rate": 0.00014046643408282305,
-      "loss": 0.5916334629058838,
-      "mean_token_accuracy": 0.8340157747268677,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
-      "entropy": 0.6203767985105515,
       "epoch": 1.1811023622047245,
-      "grad_norm": 0.45703125,
       "learning_rate": 0.00013851367921340017,
-      "loss": 0.5833216667175293,
-      "mean_token_accuracy": 0.8345949351787567,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
-      "eval_entropy": 0.7756788493306549,
-      "eval_loss": 0.8351659178733826,
-      "eval_mean_token_accuracy": 0.7810865651678156,
       "eval_num_tokens": 6676922.0,
-      "eval_runtime": 15.9481,
-      "eval_samples_per_second": 26.774,
-      "eval_steps_per_second": 3.386,
       "step": 600
     },
     {
-      "entropy": 0.645186324417591,
       "epoch": 1.2007874015748032,
-      "grad_norm": 0.498046875,
       "learning_rate": 0.0001365435243325051,
-      "loss": 0.6237170219421386,
-      "mean_token_accuracy": 0.8277532249689102,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
-      "entropy": 0.6759833052754403,
       "epoch": 1.220472440944882,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00013455685953213194,
-      "loss": 0.6462036609649658,
-      "mean_token_accuracy": 0.8247299671173096,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
-      "entropy": 0.6737162366509437,
       "epoch": 1.2401574803149606,
-      "grad_norm": 0.48828125,
       "learning_rate": 0.0001325545823632554,
-      "loss": 0.6107028484344482,
-      "mean_token_accuracy": 0.8260864973068237,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
-      "entropy": 0.6702337503433228,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
-      "loss": 0.6504123210906982,
-      "mean_token_accuracy": 0.818900191783905,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
-      "entropy": 0.7739008516073227,
       "epoch": 1.279527559055118,
-      "grad_norm": 0.56640625,
       "learning_rate": 0.00012850681598259195,
-      "loss": 0.7173595905303956,
-      "mean_token_accuracy": 0.7997365325689316,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
-      "eval_entropy": 0.7500080190323017,
-      "eval_loss": 0.8297256231307983,
-      "eval_mean_token_accuracy": 0.7822851461392862,
       "eval_num_tokens": 7218739.0,
-      "eval_runtime": 15.9417,
-      "eval_samples_per_second": 26.785,
-      "eval_steps_per_second": 3.387,
       "step": 650
     },
     {
-      "entropy": 0.7080179423093795,
       "epoch": 1.2992125984251968,
-      "grad_norm": 0.53125,
       "learning_rate": 0.0001264631555023856,
-      "loss": 0.661363172531128,
-      "mean_token_accuracy": 0.8172380715608597,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
-      "entropy": 0.6184108316898346,
       "epoch": 1.3188976377952755,
-      "grad_norm": 0.78515625,
       "learning_rate": 0.00012440753929063826,
-      "loss": 0.6013145923614502,
-      "mean_token_accuracy": 0.8273262619972229,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
-      "entropy": 0.71297377794981,
       "epoch": 1.3385826771653544,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.00012234089604973303,
-      "loss": 0.6702734470367432,
-      "mean_token_accuracy": 0.8151450455188751,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
-      "entropy": 0.6389088705182076,
       "epoch": 1.358267716535433,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.0001202641594639305,
-      "loss": 0.6133567810058593,
-      "mean_token_accuracy": 0.8298202604055405,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
-      "entropy": 0.7520714432001114,
       "epoch": 1.3779527559055118,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00011817826777754171,
-      "loss": 0.6847642421722412,
-      "mean_token_accuracy": 0.8055585384368896,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
-      "eval_entropy": 0.7791200037355777,
-      "eval_loss": 0.8222253322601318,
-      "eval_mean_token_accuracy": 0.7843283326537521,
       "eval_num_tokens": 7782831.0,
-      "eval_runtime": 15.9494,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 700
     },
     {
-      "entropy": 0.7161616936326027,
       "epoch": 1.3976377952755905,
-      "grad_norm": 0.421875,
       "learning_rate": 0.00011608416337104065,
-      "loss": 0.6723262786865234,
-      "mean_token_accuracy": 0.8196182817220687,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
-      "entropy": 0.6407245740294456,
       "epoch": 1.4173228346456692,
-      "grad_norm": 0.443359375,
       "learning_rate": 0.00011398279233530829,
-      "loss": 0.6370048522949219,
-      "mean_token_accuracy": 0.8265632688999176,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
-      "entropy": 0.704089181125164,
       "epoch": 1.4370078740157481,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00011187510404420028,
-      "loss": 0.645330286026001,
-      "mean_token_accuracy": 0.8142861813306809,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
-      "entropy": 0.7673430755734444,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
-      "loss": 0.7298968315124512,
-      "mean_token_accuracy": 0.8080587953329086,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
-      "entropy": 0.6395664170384407,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
-      "loss": 0.6174263477325439,
-      "mean_token_accuracy": 0.8285290688276291,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
-      "eval_entropy": 0.7358508281133793,
-      "eval_loss": 0.8263605833053589,
-      "eval_mean_token_accuracy": 0.7838875507866895,
       "eval_num_tokens": 8327961.0,
-      "eval_runtime": 15.9403,
-      "eval_samples_per_second": 26.787,
-      "eval_steps_per_second": 3.388,
       "step": 750
     },
     {
-      "entropy": 0.701068015396595,
       "epoch": 1.4960629921259843,
-      "grad_norm": 0.65625,
       "learning_rate": 0.00010552366960573985,
-      "loss": 0.6718067646026611,
-      "mean_token_accuracy": 0.8166692137718201,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
-      "entropy": 0.6750249415636063,
       "epoch": 1.515748031496063,
-      "grad_norm": 0.41796875,
       "learning_rate": 0.00010340025665341684,
-      "loss": 0.6471256256103516,
-      "mean_token_accuracy": 0.8236999034881591,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
-      "entropy": 0.6500637173652649,
       "epoch": 1.5354330708661417,
-      "grad_norm": 0.470703125,
       "learning_rate": 0.00010127530750652642,
-      "loss": 0.5989052295684815,
-      "mean_token_accuracy": 0.8357917547225953,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
-      "entropy": 0.6429416075348854,
       "epoch": 1.5551181102362204,
-      "grad_norm": 0.69140625,
       "learning_rate": 9.914978219122791e-05,
-      "loss": 0.6052910327911377,
-      "mean_token_accuracy": 0.8295481771230697,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
-      "entropy": 0.6374648019671441,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
-      "loss": 0.6044469833374023,
-      "mean_token_accuracy": 0.8315474331378937,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
-      "eval_entropy": 0.7627564834223853,
-      "eval_loss": 0.8170398473739624,
-      "eval_mean_token_accuracy": 0.7843498874593664,
       "eval_num_tokens": 8886361.0,
-      "eval_runtime": 15.9518,
-      "eval_samples_per_second": 26.768,
-      "eval_steps_per_second": 3.385,
       "step": 800
     },
     {
-      "entropy": 0.6787230268120765,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
-      "loss": 0.6268706321716309,
-      "mean_token_accuracy": 0.8275486946105957,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
-      "entropy": 0.7012800723314285,
       "epoch": 1.6141732283464567,
-      "grad_norm": 0.5390625,
       "learning_rate": 9.277935079806795e-05,
-      "loss": 0.652205228805542,
-      "mean_token_accuracy": 0.8216399610042572,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
-      "entropy": 0.6063105046749115,
       "epoch": 1.6338582677165354,
-      "grad_norm": 0.8125,
       "learning_rate": 9.066111976982712e-05,
-      "loss": 0.597124719619751,
-      "mean_token_accuracy": 0.836309888958931,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
-      "entropy": 0.6953654140233994,
       "epoch": 1.6535433070866141,
-      "grad_norm": 0.5,
       "learning_rate": 8.854710793399947e-05,
-      "loss": 0.6501974105834961,
-      "mean_token_accuracy": 0.8178625762462616,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
-      "entropy": 0.6784902378916741,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
-      "loss": 0.636347484588623,
-      "mean_token_accuracy": 0.8207234263420105,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
-      "eval_entropy": 0.7619620506410245,
-      "eval_loss": 0.8146092295646667,
-      "eval_mean_token_accuracy": 0.7850696554890385,
       "eval_num_tokens": 9429496.0,
-      "eval_runtime": 15.937,
-      "eval_samples_per_second": 26.793,
-      "eval_steps_per_second": 3.388,
       "step": 850
     },
     {
-      "entropy": 0.6777935117483139,
       "epoch": 1.6929133858267718,
-      "grad_norm": 0.478515625,
       "learning_rate": 8.433555984116937e-05,
-      "loss": 0.6113362312316895,
-      "mean_token_accuracy": 0.8263391226530075,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
-      "entropy": 0.6014138415455819,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
-      "loss": 0.6068051338195801,
-      "mean_token_accuracy": 0.8325248181819915,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
-      "entropy": 0.6414325833320618,
       "epoch": 1.7322834645669292,
-      "grad_norm": 0.69140625,
       "learning_rate": 8.015231656460712e-05,
-      "loss": 0.6324831008911133,
-      "mean_token_accuracy": 0.834089708328247,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
-      "entropy": 0.6839869737625122,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
-      "loss": 0.6296841144561768,
-      "mean_token_accuracy": 0.8278504878282547,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
-      "entropy": 0.6922678858041763,
       "epoch": 1.7716535433070866,
-      "grad_norm": 0.48046875,
       "learning_rate": 7.60049370040338e-05,
-      "loss": 0.6325759887695312,
-      "mean_token_accuracy": 0.8210694164037704,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
-      "eval_entropy": 0.7464290934580343,
-      "eval_loss": 0.8118129372596741,
-      "eval_mean_token_accuracy": 0.7863925894101461,
       "eval_num_tokens": 9991758.0,
-      "eval_runtime": 15.9375,
-      "eval_samples_per_second": 26.792,
-      "eval_steps_per_second": 3.388,
       "step": 900
     },
     {
-      "entropy": 0.7232977166771889,
       "epoch": 1.7913385826771653,
-      "grad_norm": 0.50390625,
       "learning_rate": 7.394704092478742e-05,
-      "loss": 0.6871041774749755,
-      "mean_token_accuracy": 0.8109384387731552,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
-      "entropy": 0.639159494638443,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
-      "loss": 0.5977220058441162,
-      "mean_token_accuracy": 0.8346921026706695,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
-      "entropy": 0.6970785558223724,
       "epoch": 1.8307086614173227,
-      "grad_norm": 0.625,
       "learning_rate": 6.98674844103308e-05,
-      "loss": 0.6659546852111816,
-      "mean_token_accuracy": 0.8177712321281433,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
-      "entropy": 0.705052088201046,
       "epoch": 1.8503937007874016,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.784766706909662e-05,
-      "loss": 0.6586481094360351,
-      "mean_token_accuracy": 0.8198566019535065,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
-      "entropy": 0.6612584158778191,
       "epoch": 1.8700787401574803,
-      "grad_norm": 0.65625,
       "learning_rate": 6.58423757605158e-05,
-      "loss": 0.6631102561950684,
-      "mean_token_accuracy": 0.8250136137008667,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
-      "eval_entropy": 0.7582799510823356,
-      "eval_loss": 0.8087909817695618,
-      "eval_mean_token_accuracy": 0.7858658830324808,
       "eval_num_tokens": 10541702.0,
-      "eval_runtime": 15.9386,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 950
     },
     {
-      "entropy": 0.7153135001659393,
       "epoch": 1.889763779527559,
-      "grad_norm": 0.52734375,
       "learning_rate": 6.38525164507927e-05,
-      "loss": 0.6485848426818848,
-      "mean_token_accuracy": 0.8191622495651245,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
-      "entropy": 0.6808911249041557,
       "epoch": 1.909448818897638,
-      "grad_norm": 0.578125,
       "learning_rate": 6.187898813414251e-05,
-      "loss": 0.6528217315673828,
-      "mean_token_accuracy": 0.8256954222917556,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
-      "entropy": 0.6457046732306481,
       "epoch": 1.9291338582677167,
-      "grad_norm": 0.65625,
       "learning_rate": 5.99226824266365e-05,
-      "loss": 0.6104453086853028,
-      "mean_token_accuracy": 0.8297607868909835,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
-      "entropy": 0.6919459193944931,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
-      "loss": 0.6459432601928711,
-      "mean_token_accuracy": 0.8180149614810943,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
-      "entropy": 0.6444254025816918,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
-      "loss": 0.6290760517120362,
-      "mean_token_accuracy": 0.8319308578968048,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
-      "eval_entropy": 0.7388938731617398,
-      "eval_loss": 0.8063948154449463,
-      "eval_mean_token_accuracy": 0.786332004600101,
       "eval_num_tokens": 11090966.0,
-      "eval_runtime": 15.9242,
-      "eval_samples_per_second": 26.814,
-      "eval_steps_per_second": 3.391,
       "step": 1000
     },
     {
-      "entropy": 0.716805949807167,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
-      "loss": 0.6740987300872803,
-      "mean_token_accuracy": 0.8161014556884766,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
-      "entropy": 0.6367674544453621,
       "epoch": 2.0078740157480315,
-      "grad_norm": 0.38671875,
       "learning_rate": 5.2287237316322676e-05,
-      "loss": 0.5608503341674804,
-      "mean_token_accuracy": 0.843386459350586,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
-      "entropy": 0.5910033389925957,
       "epoch": 2.02755905511811,
-      "grad_norm": 0.515625,
       "learning_rate": 5.043013266497751e-05,
-      "loss": 0.5252383708953857,
-      "mean_token_accuracy": 0.8537230730056763,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
-      "entropy": 0.5933422192931175,
       "epoch": 2.047244094488189,
-      "grad_norm": 0.45703125,
       "learning_rate": 4.85954230763197e-05,
-      "loss": 0.5412720680236817,
-      "mean_token_accuracy": 0.8524353206157684,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
-      "entropy": 0.5587056159973145,
       "epoch": 2.0669291338582676,
-      "grad_norm": 0.482421875,
       "learning_rate": 4.6783937449808856e-05,
-      "loss": 0.4972053050994873,
-      "mean_token_accuracy": 0.8567238092422486,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
-      "eval_entropy": 0.6584087091463583,
-      "eval_loss": 0.8380542993545532,
-      "eval_mean_token_accuracy": 0.7845212673699414,
       "eval_num_tokens": 11645829.0,
-      "eval_runtime": 15.9414,
-      "eval_samples_per_second": 26.786,
-      "eval_steps_per_second": 3.387,
       "step": 1050
     },
     {
-      "entropy": 0.5155777037143707,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
-      "loss": 0.46482577323913576,
-      "mean_token_accuracy": 0.8648948520421982,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
-      "entropy": 0.5826122924685478,
       "epoch": 2.106299212598425,
-      "grad_norm": 0.390625,
       "learning_rate": 4.323390084980353e-05,
-      "loss": 0.5252526760101318,
-      "mean_token_accuracy": 0.8463617414236069,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
-      "entropy": 0.569524523615837,
       "epoch": 2.1259842519685037,
-      "grad_norm": 0.578125,
       "learning_rate": 4.1496953739633124e-05,
-      "loss": 0.4987020969390869,
-      "mean_token_accuracy": 0.8567680090665817,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
-      "entropy": 0.5740523621439934,
       "epoch": 2.145669291338583,
-      "grad_norm": 0.6015625,
       "learning_rate": 3.978643759365202e-05,
-      "loss": 0.5137018680572509,
-      "mean_token_accuracy": 0.8519427388906479,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
-      "entropy": 0.5408509276807308,
       "epoch": 2.1653543307086616,
-      "grad_norm": 0.54296875,
       "learning_rate": 3.810312520223419e-05,
-      "loss": 0.5046960830688476,
-      "mean_token_accuracy": 0.8586939305067063,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
-      "eval_entropy": 0.6570906854338117,
-      "eval_loss": 0.8393341898918152,
-      "eval_mean_token_accuracy": 0.7835258808400896,
       "eval_num_tokens": 12207307.0,
-      "eval_runtime": 15.9395,
-      "eval_samples_per_second": 26.789,
-      "eval_steps_per_second": 3.388,
       "step": 1100
     },
     {
-      "entropy": 0.5992730811238289,
       "epoch": 2.1850393700787403,
-      "grad_norm": 0.474609375,
       "learning_rate": 3.644777706542845e-05,
-      "loss": 0.5308849811553955,
-      "mean_token_accuracy": 0.8519498467445373,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
-      "entropy": 0.5141457825899124,
       "epoch": 2.204724409448819,
-      "grad_norm": 0.49609375,
       "learning_rate": 3.4821141049373804e-05,
-      "loss": 0.4474374294281006,
-      "mean_token_accuracy": 0.8720535308122634,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
-      "entropy": 0.5664690762758255,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
-      "loss": 0.5310683250427246,
-      "mean_token_accuracy": 0.8515143841505051,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
-      "entropy": 0.5990463197231293,
       "epoch": 2.2440944881889764,
-      "grad_norm": 0.58984375,
       "learning_rate": 3.165693165312395e-05,
-      "loss": 0.5457014083862305,
-      "mean_token_accuracy": 0.8466474860906601,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
-      "entropy": 0.5571582794189454,
       "epoch": 2.263779527559055,
-      "grad_norm": 0.55078125,
       "learning_rate": 3.0120787824218554e-05,
-      "loss": 0.5153813362121582,
-      "mean_token_accuracy": 0.8554807037115097,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
-      "eval_entropy": 0.65144182741642,
-      "eval_loss": 0.8418021202087402,
-      "eval_mean_token_accuracy": 0.7838731220474949,
       "eval_num_tokens": 12763743.0,
-      "eval_runtime": 15.9352,
-      "eval_samples_per_second": 26.796,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
-      "entropy": 0.6080173872411251,
       "epoch": 2.283464566929134,
-      "grad_norm": 0.48046875,
       "learning_rate": 2.8616214572789968e-05,
-      "loss": 0.5399338245391846,
-      "mean_token_accuracy": 0.8469231188297272,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
-      "entropy": 0.49714402854442596,
       "epoch": 2.3031496062992125,
-      "grad_norm": 0.578125,
       "learning_rate": 2.7143891646719265e-05,
-      "loss": 0.46820721626281736,
-      "mean_token_accuracy": 0.8644533902406693,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
-      "entropy": 0.5437434703111649,
       "epoch": 2.322834645669291,
-      "grad_norm": 0.6015625,
       "learning_rate": 2.570448422358297e-05,
-      "loss": 0.5030812740325927,
-      "mean_token_accuracy": 0.8589521288871765,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
-      "entropy": 0.5845290660858155,
       "epoch": 2.34251968503937,
-      "grad_norm": 0.60546875,
       "learning_rate": 2.4298642610134105e-05,
-      "loss": 0.5023557186126709,
-      "mean_token_accuracy": 0.8512013405561447,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
-      "entropy": 0.6722325071692467,
       "epoch": 2.362204724409449,
-      "grad_norm": 0.66015625,
       "learning_rate": 2.292700194850147e-05,
-      "loss": 0.5894911289215088,
-      "mean_token_accuracy": 0.8302695482969285,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
-      "eval_entropy": 0.6533891520014515,
-      "eval_loss": 0.8373947739601135,
-      "eval_mean_token_accuracy": 0.7844164106580946,
       "eval_num_tokens": 13314862.0,
-      "eval_runtime": 15.9338,
-      "eval_samples_per_second": 26.798,
-      "eval_steps_per_second": 3.389,
       "step": 1200
     },
     {
-      "entropy": 0.5659365668892861,
       "epoch": 2.3818897637795278,
-      "grad_norm": 0.5078125,
       "learning_rate": 2.159018192924004e-05,
-      "loss": 0.5257106304168702,
-      "mean_token_accuracy": 0.8536411643028259,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
-      "entropy": 0.5879238858819008,
       "epoch": 2.4015748031496065,
-      "grad_norm": 0.44921875,
       "learning_rate": 2.0288786511362525e-05,
-      "loss": 0.5346655368804931,
-      "mean_token_accuracy": 0.8476213365793228,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
-      "entropy": 0.5665464416146279,
       "epoch": 2.421259842519685,
-      "grad_norm": 0.4921875,
       "learning_rate": 1.9023403649477813e-05,
-      "loss": 0.5051110267639161,
-      "mean_token_accuracy": 0.8583743631839752,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
-      "entropy": 0.5041692733764649,
       "epoch": 2.440944881889764,
-      "grad_norm": 0.453125,
       "learning_rate": 1.7794605028160462e-05,
-      "loss": 0.46024274826049805,
-      "mean_token_accuracy": 0.8660459965467453,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
-      "entropy": 0.5898029208183289,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
-      "loss": 0.5336055755615234,
-      "mean_token_accuracy": 0.8486702740192413,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
-      "eval_entropy": 0.6570240441295836,
-      "eval_loss": 0.8387707471847534,
-      "eval_mean_token_accuracy": 0.7845993064067982,
       "eval_num_tokens": 13872536.0,
-      "eval_runtime": 15.9231,
-      "eval_samples_per_second": 26.816,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
-      "entropy": 0.5317887738347054,
       "epoch": 2.4803149606299213,
-      "grad_norm": 0.6796875,
       "learning_rate": 1.544896435313997e-05,
-      "loss": 0.49010343551635743,
-      "mean_token_accuracy": 0.859997171163559,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
-      "entropy": 0.5348539352416992,
       "epoch": 2.5,
-      "grad_norm": 0.5390625,
       "learning_rate": 1.4333182031342185e-05,
-      "loss": 0.50527982711792,
-      "mean_token_accuracy": 0.8579596102237701,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
-      "entropy": 0.6260602816939353,
       "epoch": 2.5196850393700787,
-      "grad_norm": 0.49609375,
       "learning_rate": 1.3256102935148496e-05,
-      "loss": 0.5644484043121338,
-      "mean_token_accuracy": 0.8425815969705581,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
-      "entropy": 0.624223592877388,
       "epoch": 2.5393700787401574,
-      "grad_norm": 0.59375,
       "learning_rate": 1.2218213675784063e-05,
-      "loss": 0.5586697101593018,
-      "mean_token_accuracy": 0.8420278400182724,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
-      "entropy": 0.547905944287777,
       "epoch": 2.559055118110236,
-      "grad_norm": 0.43359375,
       "learning_rate": 1.1219983158982917e-05,
-      "loss": 0.49605836868286135,
-      "mean_token_accuracy": 0.8598576009273529,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
-      "eval_entropy": 0.6620006765480395,
-      "eval_loss": 0.8368703126907349,
-      "eval_mean_token_accuracy": 0.7841785737761745,
       "eval_num_tokens": 14431579.0,
-      "eval_runtime": 15.9696,
-      "eval_samples_per_second": 26.738,
-      "eval_steps_per_second": 3.381,
       "step": 1300
     },
     {
-      "entropy": 0.5208907574415207,
       "epoch": 2.578740157480315,
-      "grad_norm": 0.59765625,
       "learning_rate": 1.0261862373141839e-05,
-      "loss": 0.46456570625305177,
-      "mean_token_accuracy": 0.8702636033296585,
       "num_tokens": 14536887.0,
       "step": 1310
     },
     {
-      "entropy": 0.5214867204427719,
       "epoch": 2.5984251968503935,
-      "grad_norm": 0.54296875,
       "learning_rate": 9.344284185569463e-06,
-      "loss": 0.4646267890930176,
-      "mean_token_accuracy": 0.8666720151901245,
       "num_tokens": 14649040.0,
       "step": 1320
     },
     {
-      "entropy": 0.5516841277480126,
       "epoch": 2.6181102362204722,
-      "grad_norm": 0.4921875,
       "learning_rate": 8.467663146921979e-06,
-      "loss": 0.49184260368347166,
-      "mean_token_accuracy": 0.8584365308284759,
       "num_tokens": 14757590.0,
       "step": 1330
     },
     {
-      "entropy": 0.5462728574872017,
       "epoch": 2.637795275590551,
-      "grad_norm": 0.4765625,
       "learning_rate": 7.632395303914208e-06,
-      "loss": 0.47838892936706545,
-      "mean_token_accuracy": 0.8621183454990387,
       "num_tokens": 14870411.0,
       "step": 1340
     },
     {
-      "entropy": 0.5727479830384254,
       "epoch": 2.65748031496063,
-      "grad_norm": 0.5078125,
       "learning_rate": 6.838858020390615e-06,
-      "loss": 0.5193718433380127,
-      "mean_token_accuracy": 0.8569418281316757,
       "num_tokens": 14986705.0,
       "step": 1350
     },
     {
       "epoch": 2.65748031496063,
-      "eval_entropy": 0.6583189417918524,
-      "eval_loss": 0.8370347619056702,
-      "eval_mean_token_accuracy": 0.7841276548526905,
       "eval_num_tokens": 14986705.0,
-      "eval_runtime": 15.9498,
-      "eval_samples_per_second": 26.772,
-      "eval_steps_per_second": 3.386,
       "step": 1350
     },
     {
-      "entropy": 0.5645975567400455,
       "epoch": 2.677165354330709,
       "grad_norm": 0.640625,
       "learning_rate": 6.087409806836752e-06,
-      "loss": 0.5261539936065673,
-      "mean_token_accuracy": 0.8559167087078094,
       "num_tokens": 15095469.0,
       "step": 1360
     },
     {
-      "entropy": 0.5451949246227741,
       "epoch": 2.6968503937007875,
       "grad_norm": 0.5390625,
       "learning_rate": 5.378390158408731e-06,
-      "loss": 0.5070470333099365,
-      "mean_token_accuracy": 0.8579351454973221,
       "num_tokens": 15202248.0,
       "step": 1370
     },
     {
-      "entropy": 0.5048372894525528,
       "epoch": 2.716535433070866,
-      "grad_norm": 0.48828125,
       "learning_rate": 4.712119401553394e-06,
-      "loss": 0.45394439697265626,
-      "mean_token_accuracy": 0.8699903905391693,
       "num_tokens": 15312525.0,
       "step": 1380
     },
     {
-      "entropy": 0.5720473691821099,
       "epoch": 2.736220472440945,
-      "grad_norm": 0.439453125,
       "learning_rate": 4.088898549288689e-06,
-      "loss": 0.5240915298461915,
-      "mean_token_accuracy": 0.8507357180118561,
       "num_tokens": 15424901.0,
       "step": 1390
     },
     {
-      "entropy": 0.5502833247184753,
       "epoch": 2.7559055118110236,
-      "grad_norm": 0.51953125,
       "learning_rate": 3.5090091652096977e-06,
-      "loss": 0.49132513999938965,
-      "mean_token_accuracy": 0.861817353963852,
       "num_tokens": 15534177.0,
       "step": 1400
     },
     {
       "epoch": 2.7559055118110236,
-      "eval_entropy": 0.6594855669471953,
-      "eval_loss": 0.8369887471199036,
-      "eval_mean_token_accuracy": 0.7841665005242383,
       "eval_num_tokens": 15534177.0,
-      "eval_runtime": 15.9489,
-      "eval_samples_per_second": 26.773,
-      "eval_steps_per_second": 3.386,
       "step": 1400
     },
     {
-      "entropy": 0.6329467430710792,
       "epoch": 2.7755905511811023,
-      "grad_norm": 0.625,
       "learning_rate": 2.972713236281632e-06,
-      "loss": 0.5616164684295655,
-      "mean_token_accuracy": 0.839983269572258,
       "num_tokens": 15642725.0,
       "step": 1410
     },
     {
-      "entropy": 0.5542490720748902,
       "epoch": 2.795275590551181,
-      "grad_norm": 0.478515625,
       "learning_rate": 2.480253054477244e-06,
-      "loss": 0.5034173011779786,
-      "mean_token_accuracy": 0.8591462254524231,
       "num_tokens": 15749590.0,
       "step": 1420
     },
     {
-      "entropy": 0.5890479996800423,
       "epoch": 2.8149606299212597,
-      "grad_norm": 0.515625,
       "learning_rate": 2.031851107312366e-06,
-      "loss": 0.5488580703735352,
-      "mean_token_accuracy": 0.8496778458356857,
       "num_tokens": 15865143.0,
       "step": 1430
     },
     {
-      "entropy": 0.5142885044217109,
       "epoch": 2.8346456692913384,
       "grad_norm": 0.59765625,
       "learning_rate": 1.6277099773286951e-06,
-      "loss": 0.4575981616973877,
-      "mean_token_accuracy": 0.8715613305568695,
       "num_tokens": 15967667.0,
       "step": 1440
     },
     {
-      "entropy": 0.5492492720484734,
       "epoch": 2.8543307086614176,
       "grad_norm": 0.5546875,
       "learning_rate": 1.2680122505694502e-06,
-      "loss": 0.49847822189331054,
-      "mean_token_accuracy": 0.85660480260849,
       "num_tokens": 16077301.0,
       "step": 1450
     },
     {
       "epoch": 2.8543307086614176,
-      "eval_entropy": 0.659164630152561,
-      "eval_loss": 0.8369385004043579,
-      "eval_mean_token_accuracy": 0.7844643968122976,
       "eval_num_tokens": 16077301.0,
-      "eval_runtime": 15.9387,
-      "eval_samples_per_second": 26.79,
-      "eval_steps_per_second": 3.388,
       "step": 1450
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     },
     {
+      "entropy": 0.771521957218647,
       "epoch": 0.31496062992125984,
       "grad_norm": 0.470703125,
       "learning_rate": 0.00019712929306529723,
+      "loss": 0.7662525177001953,
+      "mean_token_accuracy": 0.8057839542627334,
       "num_tokens": 1760181.0,
       "step": 160
     },
     {
+      "entropy": 0.8810683578252793,
       "epoch": 0.3346456692913386,
       "grad_norm": 0.625,
       "learning_rate": 0.000196601745534659,
+      "loss": 0.8402512550354004,
+      "mean_token_accuracy": 0.7819562494754791,
       "num_tokens": 1864683.0,
       "step": 170
     },
     {
+      "entropy": 0.8320274949073792,
       "epoch": 0.3543307086614173,
+      "grad_norm": 0.474609375,
       "learning_rate": 0.0001960305545112445,
+      "loss": 0.8194713592529297,
+      "mean_token_accuracy": 0.7880559414625168,
       "num_tokens": 1978641.0,
       "step": 180
     },
     {
+      "entropy": 0.8050856739282608,
       "epoch": 0.37401574803149606,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001954159780522056,
+      "loss": 0.8002090454101562,
+      "mean_token_accuracy": 0.791279673576355,
       "num_tokens": 2087989.0,
       "step": 190
     },
     {
+      "entropy": 0.8930813580751419,
       "epoch": 0.3937007874015748,
+      "grad_norm": 0.6171875,
       "learning_rate": 0.00019475829381570566,
+      "loss": 0.8300093650817871,
+      "mean_token_accuracy": 0.7832900315523148,
       "num_tokens": 2195494.0,
       "step": 200
     },
     {
       "epoch": 0.3937007874015748,
+      "eval_entropy": 0.9506367754053187,
+      "eval_loss": 0.8978024125099182,
+      "eval_mean_token_accuracy": 0.7694783839914534,
       "eval_num_tokens": 2195494.0,
+      "eval_runtime": 15.9025,
+      "eval_samples_per_second": 26.851,
+      "eval_steps_per_second": 3.396,
       "step": 200
     },
     {
+      "entropy": 0.7895667225122451,
       "epoch": 0.41338582677165353,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.0001940577989354773,
+      "loss": 0.7946149826049804,
+      "mean_token_accuracy": 0.7988882720470428,
       "num_tokens": 2314228.0,
       "step": 210
     },
     {
+      "entropy": 0.8575397372245789,
       "epoch": 0.4330708661417323,
+      "grad_norm": 0.6640625,
       "learning_rate": 0.00019331480988658083,
+      "loss": 0.812718391418457,
+      "mean_token_accuracy": 0.7836957842111587,
       "num_tokens": 2419122.0,
       "step": 220
     },
     {
+      "entropy": 0.8544128388166428,
       "epoch": 0.452755905511811,
+      "grad_norm": 0.462890625,
       "learning_rate": 0.00019252966234242468,
+      "loss": 0.8198953628540039,
+      "mean_token_accuracy": 0.7827243804931641,
       "num_tokens": 2524623.0,
       "step": 230
     },
     {
+      "entropy": 0.9001998215913772,
       "epoch": 0.47244094488188976,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019170271102311238,
+      "loss": 0.9015755653381348,
+      "mean_token_accuracy": 0.7709000498056412,
       "num_tokens": 2631031.0,
       "step": 240
     },
     {
+      "entropy": 0.7926925078034401,
       "epoch": 0.4921259842519685,
+      "grad_norm": 0.49609375,
       "learning_rate": 0.00019083432953518418,
+      "loss": 0.8107254028320312,
+      "mean_token_accuracy": 0.7990583658218384,
       "num_tokens": 2741031.0,
       "step": 250
     },
     {
       "epoch": 0.4921259842519685,
+      "eval_entropy": 0.9023421671655443,
+      "eval_loss": 0.884621262550354,
+      "eval_mean_token_accuracy": 0.7719593346118927,
       "eval_num_tokens": 2741031.0,
+      "eval_runtime": 15.8929,
+      "eval_samples_per_second": 26.867,
+      "eval_steps_per_second": 3.398,
       "step": 250
     },
     {
+      "entropy": 0.7738933145999909,
       "epoch": 0.5118110236220472,
+      "grad_norm": 0.58203125,
       "learning_rate": 0.00018992491020282618,
+      "loss": 0.7572973251342774,
+      "mean_token_accuracy": 0.7949966192245483,
       "num_tokens": 2853547.0,
       "step": 260
     },
     {
+      "entropy": 0.8569424450397491,
       "epoch": 0.531496062992126,
+      "grad_norm": 1.375,
       "learning_rate": 0.0001889748638906231,
+      "loss": 0.8083616256713867,
+      "mean_token_accuracy": 0.791676989197731,
       "num_tokens": 2968367.0,
       "step": 270
     },
     {
+      "entropy": 0.889449554681778,
       "epoch": 0.5511811023622047,
+      "grad_norm": 0.44921875,
       "learning_rate": 0.00018798461981793462,
+      "loss": 0.8472497940063477,
+      "mean_token_accuracy": 0.7779456943273544,
       "num_tokens": 3075963.0,
       "step": 280
     },
     {
+      "entropy": 0.8320091620087624,
       "epoch": 0.5708661417322834,
+      "grad_norm": 0.453125,
       "learning_rate": 0.00018695462536497934,
+      "loss": 0.8240900993347168,
+      "mean_token_accuracy": 0.7846117675304413,
       "num_tokens": 3185899.0,
       "step": 290
     },
     {
+      "entropy": 0.8333972141146659,
       "epoch": 0.5905511811023622,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00018588534587071385,
+      "loss": 0.8117186546325683,
+      "mean_token_accuracy": 0.7908233135938645,
       "num_tokens": 3297319.0,
       "step": 300
     },
     {
       "epoch": 0.5905511811023622,
+      "eval_entropy": 0.8452967874429844,
+      "eval_loss": 0.8709425926208496,
+      "eval_mean_token_accuracy": 0.7752477537702631,
       "eval_num_tokens": 3297319.0,
+      "eval_runtime": 15.8962,
+      "eval_samples_per_second": 26.862,
+      "eval_steps_per_second": 3.397,
       "step": 300
     },
     {
+      "entropy": 0.8324314504861832,
       "epoch": 0.610236220472441,
+      "grad_norm": 0.39453125,
       "learning_rate": 0.00018477726442259806,
+      "loss": 0.801210880279541,
+      "mean_token_accuracy": 0.7885541886091232,
       "num_tokens": 3415477.0,
       "step": 310
     },
     {
+      "entropy": 0.8624329552054405,
       "epoch": 0.6299212598425197,
+      "grad_norm": 0.400390625,
       "learning_rate": 0.00018363088163834252,
+      "loss": 0.8126995086669921,
+      "mean_token_accuracy": 0.785911500453949,
       "num_tokens": 3524759.0,
       "step": 320
     },
     {
+      "entropy": 0.8507865130901336,
       "epoch": 0.6496062992125984,
       "grad_norm": 0.458984375,
       "learning_rate": 0.00018244671543973513,
+      "loss": 0.8206624031066895,
+      "mean_token_accuracy": 0.7833890795707703,
       "num_tokens": 3640838.0,
       "step": 330
     },
     {
+      "entropy": 0.7785286724567413,
       "epoch": 0.6692913385826772,
       "grad_norm": 0.51953125,
       "learning_rate": 0.0001812253008186505,
+      "loss": 0.753394889831543,
+      "mean_token_accuracy": 0.8014622688293457,
       "num_tokens": 3756735.0,
       "step": 340
     },
     {
+      "entropy": 0.8389557063579559,
       "epoch": 0.6889763779527559,
+      "grad_norm": 0.50390625,
       "learning_rate": 0.0001799671895953471,
+      "loss": 0.8116916656494141,
+      "mean_token_accuracy": 0.7909021496772766,
       "num_tokens": 3870377.0,
       "step": 350
     },
     {
       "epoch": 0.6889763779527559,
+      "eval_entropy": 0.8793082656683745,
+      "eval_loss": 0.8570566177368164,
+      "eval_mean_token_accuracy": 0.7760916120476193,
       "eval_num_tokens": 3870377.0,
+      "eval_runtime": 15.9217,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 350
     },
     {
+      "entropy": 0.7998805820941925,
       "epoch": 0.7086614173228346,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017867295016916167,
+      "loss": 0.7578703880310058,
+      "mean_token_accuracy": 0.7953435629606247,
       "num_tokens": 3975692.0,
       "step": 360
     },
     {
+      "entropy": 0.9054980486631393,
       "epoch": 0.7283464566929134,
+      "grad_norm": 0.4765625,
       "learning_rate": 0.0001773431672617132,
+      "loss": 0.8652525901794433,
+      "mean_token_accuracy": 0.7738145262002945,
       "num_tokens": 4087256.0,
       "step": 370
     },
     {
+      "entropy": 0.7607795268297195,
       "epoch": 0.7480314960629921,
+      "grad_norm": 0.498046875,
       "learning_rate": 0.00017597844165273286,
+      "loss": 0.7365350723266602,
+      "mean_token_accuracy": 0.7966376423835755,
       "num_tokens": 4192629.0,
       "step": 380
     },
     {
+      "entropy": 0.8066679537296295,
       "epoch": 0.7677165354330708,
+      "grad_norm": 0.443359375,
       "learning_rate": 0.00017457938990863908,
+      "loss": 0.7610409736633301,
+      "mean_token_accuracy": 0.8000116467475891,
       "num_tokens": 4303838.0,
       "step": 390
     },
     {
+      "entropy": 0.8927708208560944,
       "epoch": 0.7874015748031497,
+      "grad_norm": 0.494140625,
       "learning_rate": 0.00017314664410398033,
+      "loss": 0.8361794471740722,
+      "mean_token_accuracy": 0.7784365385770797,
       "num_tokens": 4414002.0,
       "step": 400
     },
     {
       "epoch": 0.7874015748031497,
+      "eval_entropy": 0.8607326823252218,
+      "eval_loss": 0.8459087610244751,
+      "eval_mean_token_accuracy": 0.7791097947844753,
       "eval_num_tokens": 4414002.0,
+      "eval_runtime": 16.176,
+      "eval_samples_per_second": 26.397,
+      "eval_steps_per_second": 3.338,
       "step": 400
     },
     {
+      "entropy": 0.7851713225245476,
       "epoch": 0.8070866141732284,
+      "grad_norm": 0.56640625,
       "learning_rate": 0.0001716808515358714,
+      "loss": 0.8022769927978516,
+      "mean_token_accuracy": 0.7959282606840133,
       "num_tokens": 4525758.0,
       "step": 410
     },
     {
+      "entropy": 0.7688719958066941,
       "epoch": 0.8267716535433071,
+      "grad_norm": 0.447265625,
       "learning_rate": 0.00017018267443155268,
+      "loss": 0.7182153224945068,
+      "mean_token_accuracy": 0.8076630532741547,
       "num_tokens": 4640742.0,
       "step": 420
     },
     {
+      "entropy": 0.8512372523546219,
       "epoch": 0.8464566929133859,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00016865278964920378,
+      "loss": 0.8275189399719238,
+      "mean_token_accuracy": 0.7865870356559753,
       "num_tokens": 4754244.0,
       "step": 430
     },
     {
+      "entropy": 0.7933165490627289,
       "epoch": 0.8661417322834646,
+      "grad_norm": 0.34765625,
       "learning_rate": 0.00016709188837214732,
+      "loss": 0.7769423484802246,
+      "mean_token_accuracy": 0.7905291795730591,
       "num_tokens": 4862576.0,
       "step": 440
     },
     {
+      "entropy": 0.7952004507184028,
       "epoch": 0.8858267716535433,
+      "grad_norm": 0.375,
       "learning_rate": 0.0001655006757965806,
+      "loss": 0.755205774307251,
+      "mean_token_accuracy": 0.7940250873565674,
       "num_tokens": 4984410.0,
       "step": 450
     },
     {
       "epoch": 0.8858267716535433,
+      "eval_entropy": 0.8390876281040686,
+      "eval_loss": 0.8377042412757874,
+      "eval_mean_token_accuracy": 0.7800426229282662,
       "eval_num_tokens": 4984410.0,
+      "eval_runtime": 15.9314,
+      "eval_samples_per_second": 26.802,
+      "eval_steps_per_second": 3.39,
       "step": 450
     },
     {
+      "entropy": 0.8776025518774986,
       "epoch": 0.905511811023622,
+      "grad_norm": 0.5703125,
       "learning_rate": 0.0001638798708129768,
+      "loss": 0.8327735900878906,
+      "mean_token_accuracy": 0.7833347082138061,
       "num_tokens": 5094029.0,
       "step": 460
     },
     {
+      "entropy": 0.8570419639348984,
       "epoch": 0.9251968503937008,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.00016223020568129854,
+      "loss": 0.830419921875,
+      "mean_token_accuracy": 0.7789777547121048,
       "num_tokens": 5210613.0,
       "step": 470
     },
     {
+      "entropy": 0.8276598662137985,
       "epoch": 0.9448818897637795,
       "grad_norm": 0.59375,
       "learning_rate": 0.00016055242570017244,
+      "loss": 0.787315034866333,
+      "mean_token_accuracy": 0.7914238631725311,
       "num_tokens": 5322546.0,
       "step": 480
     },
     {
+      "entropy": 0.8109192818403244,
       "epoch": 0.9645669291338582,
+      "grad_norm": 0.4375,
       "learning_rate": 0.00015884728887017162,
+      "loss": 0.7857238292694092,
+      "mean_token_accuracy": 0.7958390027284622,
       "num_tokens": 5434008.0,
       "step": 490
     },
     {
+      "entropy": 0.6643441513180732,
       "epoch": 0.984251968503937,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.0001571155655513606,
+      "loss": 0.6719898700714111,
+      "mean_token_accuracy": 0.8172835201025009,
       "num_tokens": 5544086.0,
       "step": 500
     },
     {
       "epoch": 0.984251968503937,
+      "eval_entropy": 0.8121603627999624,
+      "eval_loss": 0.8310334086418152,
+      "eval_mean_token_accuracy": 0.7817157308260599,
       "eval_num_tokens": 5544086.0,
+      "eval_runtime": 15.9111,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 500
     },
     {
+      "entropy": 0.7330836996436119,
       "epoch": 1.0039370078740157,
+      "grad_norm": 0.38671875,
       "learning_rate": 0.0001553580381152559,
+      "loss": 0.6914227962493896,
+      "mean_token_accuracy": 0.8179270803928376,
       "num_tokens": 5657246.0,
       "step": 510
     },
     {
+      "entropy": 0.7551421344280242,
       "epoch": 1.0236220472440944,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.0001535755005913602,
+      "loss": 0.6854944705963135,
+      "mean_token_accuracy": 0.8148082345724106,
       "num_tokens": 5768184.0,
       "step": 520
     },
     {
+      "entropy": 0.7464401632547378,
       "epoch": 1.0433070866141732,
+      "grad_norm": 0.6015625,
       "learning_rate": 0.00015176875830842938,
+      "loss": 0.7031197071075439,
+      "mean_token_accuracy": 0.814039769768715,
       "num_tokens": 5885598.0,
       "step": 530
     },
     {
+      "entropy": 0.6804196655750274,
       "epoch": 1.0629921259842519,
+      "grad_norm": 0.435546875,
       "learning_rate": 0.0001499386275306348,
+      "loss": 0.6404387950897217,
+      "mean_token_accuracy": 0.8235772669315338,
       "num_tokens": 6002120.0,
       "step": 540
     },
     {
+      "entropy": 0.6835484370589257,
       "epoch": 1.0826771653543308,
+      "grad_norm": 0.416015625,
       "learning_rate": 0.0001480859350887852,
+      "loss": 0.6457780361175537,
+      "mean_token_accuracy": 0.8242624312639236,
       "num_tokens": 6112085.0,
       "step": 550
     },
     {
       "epoch": 1.0826771653543308,
+      "eval_entropy": 0.7967574977212482,
+      "eval_loss": 0.835203230381012,
+      "eval_mean_token_accuracy": 0.7809422369356509,
       "eval_num_tokens": 6112085.0,
+      "eval_runtime": 15.9214,
+      "eval_samples_per_second": 26.819,
+      "eval_steps_per_second": 3.392,
       "step": 550
     },
     {
+      "entropy": 0.7008466422557831,
       "epoch": 1.1023622047244095,
+      "grad_norm": 0.48046875,
       "learning_rate": 0.00014621151800677434,
+      "loss": 0.6664916515350342,
+      "mean_token_accuracy": 0.8154505103826523,
       "num_tokens": 6224645.0,
       "step": 560
     },
     {
+      "entropy": 0.6077279314398766,
       "epoch": 1.1220472440944882,
       "grad_norm": 0.37890625,
       "learning_rate": 0.0001443162231234242,
+      "loss": 0.5706528663635254,
+      "mean_token_accuracy": 0.8392371147871017,
       "num_tokens": 6336790.0,
       "step": 570
     },
     {
+      "entropy": 0.683767618238926,
       "epoch": 1.141732283464567,
       "grad_norm": 0.55078125,
       "learning_rate": 0.0001424009067098931,
+      "loss": 0.6489217758178711,
+      "mean_token_accuracy": 0.8194159388542175,
       "num_tokens": 6450761.0,
       "step": 580
     },
     {
+      "entropy": 0.6302935466170311,
       "epoch": 1.1614173228346456,
+      "grad_norm": 0.486328125,
       "learning_rate": 0.00014046643408282305,
+      "loss": 0.5900890350341796,
+      "mean_token_accuracy": 0.8340148836374283,
       "num_tokens": 6562060.0,
       "step": 590
     },
     {
+      "entropy": 0.6183161422610283,
       "epoch": 1.1811023622047245,
+      "grad_norm": 0.455078125,
       "learning_rate": 0.00013851367921340017,
+      "loss": 0.5828392982482911,
+      "mean_token_accuracy": 0.835411223769188,
       "num_tokens": 6676922.0,
       "step": 600
     },
     {
       "epoch": 1.1811023622047245,
+      "eval_entropy": 0.7808757076660792,
+      "eval_loss": 0.835366427898407,
+      "eval_mean_token_accuracy": 0.7813083474282865,
       "eval_num_tokens": 6676922.0,
+      "eval_runtime": 15.8899,
+      "eval_samples_per_second": 26.872,
+      "eval_steps_per_second": 3.398,
       "step": 600
     },
     {
+      "entropy": 0.6484389528632164,
       "epoch": 1.2007874015748032,
+      "grad_norm": 0.4921875,
       "learning_rate": 0.0001365435243325051,
+      "loss": 0.6237234592437744,
+      "mean_token_accuracy": 0.8273959398269654,
       "num_tokens": 6783497.0,
       "step": 610
     },
     {
+      "entropy": 0.6731782004237175,
       "epoch": 1.220472440944882,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00013455685953213194,
+      "loss": 0.6462783336639404,
+      "mean_token_accuracy": 0.8242785036563873,
       "num_tokens": 6894014.0,
       "step": 620
     },
     {
+      "entropy": 0.6746662899851799,
       "epoch": 1.2401574803149606,
+      "grad_norm": 0.484375,
       "learning_rate": 0.0001325545823632554,
+      "loss": 0.6090981006622315,
+      "mean_token_accuracy": 0.8258850336074829,
       "num_tokens": 6997350.0,
       "step": 630
     },
     {
+      "entropy": 0.6707195937633514,
       "epoch": 1.2598425196850394,
       "grad_norm": 0.451171875,
       "learning_rate": 0.00013053759743032823,
+      "loss": 0.6500792026519775,
+      "mean_token_accuracy": 0.8198623776435852,
       "num_tokens": 7109067.0,
       "step": 640
     },
     {
+      "entropy": 0.7742280542850495,
       "epoch": 1.279527559055118,
+      "grad_norm": 0.55859375,
       "learning_rate": 0.00012850681598259195,
+      "loss": 0.717244005203247,
+      "mean_token_accuracy": 0.7994646310806275,
       "num_tokens": 7218739.0,
       "step": 650
     },
     {
       "epoch": 1.279527559055118,
+      "eval_entropy": 0.7477047746932065,
+      "eval_loss": 0.8298369646072388,
+      "eval_mean_token_accuracy": 0.7822379061469326,
       "eval_num_tokens": 7218739.0,
+      "eval_runtime": 15.8838,
+      "eval_samples_per_second": 26.883,
+      "eval_steps_per_second": 3.4,
       "step": 650
     },
     {
+      "entropy": 0.7066994801163673,
       "epoch": 1.2992125984251968,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.0001264631555023856,
+      "loss": 0.6610036373138428,
+      "mean_token_accuracy": 0.8158259451389313,
       "num_tokens": 7334561.0,
       "step": 660
     },
     {
+      "entropy": 0.6165548652410507,
       "epoch": 1.3188976377952755,
+      "grad_norm": 0.76953125,
       "learning_rate": 0.00012440753929063826,
+      "loss": 0.6015829563140869,
+      "mean_token_accuracy": 0.8280171245336533,
       "num_tokens": 7443871.0,
       "step": 670
     },
     {
+      "entropy": 0.7171120464801788,
       "epoch": 1.3385826771653544,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.00012234089604973303,
+      "loss": 0.6697704792022705,
+      "mean_token_accuracy": 0.816604706645012,
       "num_tokens": 7555559.0,
       "step": 680
     },
     {
+      "entropy": 0.6375228509306907,
       "epoch": 1.358267716535433,
+      "grad_norm": 0.51171875,
       "learning_rate": 0.0001202641594639305,
+      "loss": 0.6138493061065674,
+      "mean_token_accuracy": 0.8299814134836196,
       "num_tokens": 7669861.0,
       "step": 690
     },
     {
+      "entropy": 0.7527845561504364,
       "epoch": 1.3779527559055118,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00011817826777754171,
+      "loss": 0.6848691940307617,
+      "mean_token_accuracy": 0.8056917071342469,
       "num_tokens": 7782831.0,
       "step": 700
     },
     {
       "epoch": 1.3779527559055118,
+      "eval_entropy": 0.7785312417480681,
+      "eval_loss": 0.822371244430542,
+      "eval_mean_token_accuracy": 0.7844836800186722,
       "eval_num_tokens": 7782831.0,
+      "eval_runtime": 15.9232,
+      "eval_samples_per_second": 26.816,
+      "eval_steps_per_second": 3.391,
       "step": 700
     },
     {
+      "entropy": 0.7150849819183349,
       "epoch": 1.3976377952755905,
+      "grad_norm": 0.419921875,
       "learning_rate": 0.00011608416337104065,
+      "loss": 0.6717889785766602,
+      "mean_token_accuracy": 0.8194838553667069,
       "num_tokens": 7895511.0,
       "step": 710
     },
     {
+      "entropy": 0.6417959302663803,
       "epoch": 1.4173228346456692,
+      "grad_norm": 0.44140625,
       "learning_rate": 0.00011398279233530829,
+      "loss": 0.6370071411132813,
+      "mean_token_accuracy": 0.8262952893972397,
       "num_tokens": 8006186.0,
       "step": 720
     },
     {
+      "entropy": 0.7074607491493226,
       "epoch": 1.4370078740157481,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00011187510404420028,
+      "loss": 0.6439510822296143,
+      "mean_token_accuracy": 0.8158842891454696,
       "num_tokens": 8110444.0,
       "step": 730
     },
     {
+      "entropy": 0.7658031359314919,
       "epoch": 1.4566929133858268,
       "grad_norm": 0.7109375,
       "learning_rate": 0.00010976205072563132,
+      "loss": 0.7316959857940674,
+      "mean_token_accuracy": 0.8061666846275329,
       "num_tokens": 8220128.0,
       "step": 740
     },
     {
+      "entropy": 0.6396550685167313,
       "epoch": 1.4763779527559056,
       "grad_norm": 0.5078125,
       "learning_rate": 0.00010764458703137024,
+      "loss": 0.6169118881225586,
+      "mean_token_accuracy": 0.8309739202260971,
       "num_tokens": 8327961.0,
       "step": 750
     },
     {
       "epoch": 1.4763779527559056,
+      "eval_entropy": 0.7342568177867819,
+      "eval_loss": 0.8272306323051453,
+      "eval_mean_token_accuracy": 0.7827356656392416,
       "eval_num_tokens": 8327961.0,
+      "eval_runtime": 15.911,
+      "eval_samples_per_second": 26.837,
+      "eval_steps_per_second": 3.394,
       "step": 750
     },
     {
+      "entropy": 0.6999662488698959,
       "epoch": 1.4960629921259843,
+      "grad_norm": 0.671875,
       "learning_rate": 0.00010552366960573985,
+      "loss": 0.6717596054077148,
+      "mean_token_accuracy": 0.8167526543140411,
       "num_tokens": 8432725.0,
       "step": 760
     },
     {
+      "entropy": 0.6736484602093696,
       "epoch": 1.515748031496063,
+      "grad_norm": 0.408203125,
       "learning_rate": 0.00010340025665341684,
+      "loss": 0.6454797744750976,
+      "mean_token_accuracy": 0.8241631388664246,
       "num_tokens": 8550042.0,
       "step": 770
     },
     {
+      "entropy": 0.6505535006523132,
       "epoch": 1.5354330708661417,
+      "grad_norm": 0.45703125,
       "learning_rate": 0.00010127530750652642,
+      "loss": 0.6001406669616699,
+      "mean_token_accuracy": 0.8359915494918824,
       "num_tokens": 8664499.0,
       "step": 780
     },
     {
+      "entropy": 0.643044576048851,
       "epoch": 1.5551181102362204,
+      "grad_norm": 0.6875,
       "learning_rate": 9.914978219122791e-05,
+      "loss": 0.6050069808959961,
+      "mean_token_accuracy": 0.8284690707921982,
       "num_tokens": 8778459.0,
       "step": 790
     },
     {
+      "entropy": 0.6357371553778648,
       "epoch": 1.574803149606299,
       "grad_norm": 0.6484375,
       "learning_rate": 9.702464099398632e-05,
+      "loss": 0.6026933193206787,
+      "mean_token_accuracy": 0.8313924312591553,
       "num_tokens": 8886361.0,
       "step": 800
     },
     {
       "epoch": 1.574803149606299,
+      "eval_entropy": 0.7611764465217237,
+      "eval_loss": 0.8170427083969116,
+      "eval_mean_token_accuracy": 0.7845815287695991,
       "eval_num_tokens": 8886361.0,
+      "eval_runtime": 15.9048,
+      "eval_samples_per_second": 26.847,
+      "eval_steps_per_second": 3.395,
       "step": 800
     },
     {
+      "entropy": 0.6775295034050941,
       "epoch": 1.594488188976378,
       "grad_norm": 0.515625,
       "learning_rate": 9.4900844027727e-05,
+      "loss": 0.6273195743560791,
+      "mean_token_accuracy": 0.8267990559339523,
       "num_tokens": 8995558.0,
       "step": 810
     },
     {
+      "entropy": 0.7035147309303283,
       "epoch": 1.6141732283464567,
+      "grad_norm": 0.53515625,
       "learning_rate": 9.277935079806795e-05,
+      "loss": 0.6532655715942383,
+      "mean_token_accuracy": 0.8209419757127762,
       "num_tokens": 9096245.0,
       "step": 820
     },
     {
+      "entropy": 0.6083378851413727,
       "epoch": 1.6338582677165354,
+      "grad_norm": 0.796875,
       "learning_rate": 9.066111976982712e-05,
+      "loss": 0.5965664386749268,
+      "mean_token_accuracy": 0.8344648003578186,
       "num_tokens": 9204264.0,
       "step": 830
     },
     {
+      "entropy": 0.6942630469799042,
       "epoch": 1.6535433070866141,
+      "grad_norm": 0.50390625,
       "learning_rate": 8.854710793399947e-05,
+      "loss": 0.6506452560424805,
+      "mean_token_accuracy": 0.8177429378032685,
       "num_tokens": 9314940.0,
       "step": 840
     },
     {
+      "entropy": 0.6778671249747277,
       "epoch": 1.673228346456693,
       "grad_norm": 0.546875,
       "learning_rate": 8.64382703754003e-05,
+      "loss": 0.6361007213592529,
+      "mean_token_accuracy": 0.8213603168725967,
       "num_tokens": 9429496.0,
       "step": 850
     },
     {
       "epoch": 1.673228346456693,
+      "eval_entropy": 0.7621777179064574,
+      "eval_loss": 0.8149666786193848,
+      "eval_mean_token_accuracy": 0.7850644290447235,
       "eval_num_tokens": 9429496.0,
+      "eval_runtime": 15.9323,
+      "eval_samples_per_second": 26.801,
+      "eval_steps_per_second": 3.389,
       "step": 850
     },
     {
+      "entropy": 0.6798793867230415,
       "epoch": 1.6929133858267718,
+      "grad_norm": 0.46875,
       "learning_rate": 8.433555984116937e-05,
+      "loss": 0.613386869430542,
+      "mean_token_accuracy": 0.8275126874446869,
       "num_tokens": 9544270.0,
       "step": 860
     },
     {
+      "entropy": 0.6023602589964867,
       "epoch": 1.7125984251968505,
       "grad_norm": 0.53125,
       "learning_rate": 8.22399263103314e-05,
+      "loss": 0.6046566963195801,
+      "mean_token_accuracy": 0.832113066315651,
       "num_tokens": 9654198.0,
       "step": 870
     },
     {
+      "entropy": 0.640959857404232,
       "epoch": 1.7322834645669292,
+      "grad_norm": 0.6875,
       "learning_rate": 8.015231656460712e-05,
+      "loss": 0.6335668563842773,
+      "mean_token_accuracy": 0.8338181763887406,
       "num_tokens": 9769457.0,
       "step": 880
     },
     {
+      "entropy": 0.6831659495830535,
       "epoch": 1.7519685039370079,
       "grad_norm": 0.5390625,
       "learning_rate": 7.807367376066889e-05,
+      "loss": 0.6274240970611572,
+      "mean_token_accuracy": 0.8270208179950714,
       "num_tokens": 9875831.0,
       "step": 890
     },
     {
+      "entropy": 0.691785454750061,
       "epoch": 1.7716535433070866,
+      "grad_norm": 0.48828125,
       "learning_rate": 7.60049370040338e-05,
+      "loss": 0.631044578552246,
+      "mean_token_accuracy": 0.8213751494884491,
       "num_tokens": 9991758.0,
       "step": 900
     },
     {
       "epoch": 1.7716535433070866,
+      "eval_entropy": 0.741917089731605,
+      "eval_loss": 0.8126867413520813,
+      "eval_mean_token_accuracy": 0.7860870074342798,
       "eval_num_tokens": 9991758.0,
+      "eval_runtime": 15.9328,
+      "eval_samples_per_second": 26.8,
+      "eval_steps_per_second": 3.389,
       "step": 900
     },
     {
+      "entropy": 0.7206486001610756,
       "epoch": 1.7913385826771653,
+      "grad_norm": 0.51171875,
       "learning_rate": 7.394704092478742e-05,
+      "loss": 0.685962438583374,
+      "mean_token_accuracy": 0.8114332675933837,
       "num_tokens": 10106255.0,
       "step": 910
     },
     {
+      "entropy": 0.6408105552196502,
       "epoch": 1.811023622047244,
       "grad_norm": 0.51953125,
       "learning_rate": 7.190091525532921e-05,
+      "loss": 0.5981018543243408,
+      "mean_token_accuracy": 0.8346310049295426,
       "num_tokens": 10218066.0,
       "step": 920
     },
     {
+      "entropy": 0.6974071592092514,
       "epoch": 1.8307086614173227,
+      "grad_norm": 0.62109375,
       "learning_rate": 6.98674844103308e-05,
+      "loss": 0.6660090446472168,
+      "mean_token_accuracy": 0.8166637867689133,
       "num_tokens": 10326991.0,
       "step": 930
     },
     {
+      "entropy": 0.7043072924017906,
       "epoch": 1.8503937007874016,
+      "grad_norm": 0.515625,
       "learning_rate": 6.784766706909662e-05,
+      "loss": 0.6577100276947021,
+      "mean_token_accuracy": 0.8193566143512726,
       "num_tokens": 10432558.0,
       "step": 940
     },
     {
+      "entropy": 0.6612386792898178,
       "epoch": 1.8700787401574803,
+      "grad_norm": 0.66015625,
       "learning_rate": 6.58423757605158e-05,
+      "loss": 0.663103723526001,
+      "mean_token_accuracy": 0.8252974808216095,
       "num_tokens": 10541702.0,
       "step": 950
     },
     {
       "epoch": 1.8700787401574803,
+      "eval_entropy": 0.7594050356635341,
+      "eval_loss": 0.809001624584198,
+      "eval_mean_token_accuracy": 0.7858329779571958,
       "eval_num_tokens": 10541702.0,
+      "eval_runtime": 15.9266,
+      "eval_samples_per_second": 26.811,
+      "eval_steps_per_second": 3.391,
       "step": 950
     },
     {
+      "entropy": 0.7143705666065217,
       "epoch": 1.889763779527559,
+      "grad_norm": 0.5234375,
       "learning_rate": 6.38525164507927e-05,
+      "loss": 0.6485679149627686,
+      "mean_token_accuracy": 0.8175264537334442,
       "num_tokens": 10652431.0,
       "step": 960
     },
     {
+      "entropy": 0.681491743028164,
       "epoch": 1.909448818897638,
+      "grad_norm": 0.5625,
       "learning_rate": 6.187898813414251e-05,
+      "loss": 0.6528646945953369,
+      "mean_token_accuracy": 0.82598916888237,
       "num_tokens": 10766036.0,
       "step": 970
     },
     {
+      "entropy": 0.6429278805851937,
       "epoch": 1.9291338582677167,
+      "grad_norm": 0.6484375,
       "learning_rate": 5.99226824266365e-05,
+      "loss": 0.6115736961364746,
+      "mean_token_accuracy": 0.8285941183567047,
       "num_tokens": 10869598.0,
       "step": 980
     },
     {
+      "entropy": 0.6953536570072174,
       "epoch": 1.9488188976377954,
       "grad_norm": 0.640625,
       "learning_rate": 5.7984483163380764e-05,
+      "loss": 0.6476809501647949,
+      "mean_token_accuracy": 0.8186965942382812,
       "num_tokens": 10983085.0,
       "step": 990
     },
     {
+      "entropy": 0.645970419049263,
       "epoch": 1.968503937007874,
       "grad_norm": 0.578125,
       "learning_rate": 5.606526599921026e-05,
+      "loss": 0.6282810211181641,
+      "mean_token_accuracy": 0.8330216020345688,
       "num_tokens": 11090966.0,
       "step": 1000
     },
     {
       "epoch": 1.968503937007874,
+      "eval_entropy": 0.7374509738551246,
+      "eval_loss": 0.80621737241745,
+      "eval_mean_token_accuracy": 0.7872726199803529,
       "eval_num_tokens": 11090966.0,
+      "eval_runtime": 15.9085,
+      "eval_samples_per_second": 26.841,
+      "eval_steps_per_second": 3.394,
       "step": 1000
     },
     {
+      "entropy": 0.7159051194787025,
       "epoch": 1.9881889763779528,
       "grad_norm": 0.51953125,
       "learning_rate": 5.416589801307874e-05,
+      "loss": 0.6746394634246826,
+      "mean_token_accuracy": 0.816139143705368,
       "num_tokens": 11204584.0,
       "step": 1010
     },
     {
+      "entropy": 0.6366253510117531,
       "epoch": 2.0078740157480315,
+      "grad_norm": 0.380859375,
       "learning_rate": 5.2287237316322676e-05,
+      "loss": 0.5599459171295166,
+      "mean_token_accuracy": 0.8438727766275406,
       "num_tokens": 11314232.0,
       "step": 1020
     },
     {
+      "entropy": 0.591021591424942,
       "epoch": 2.02755905511811,
+      "grad_norm": 0.5234375,
       "learning_rate": 5.043013266497751e-05,
+      "loss": 0.5247670173645019,
+      "mean_token_accuracy": 0.8531373143196106,
       "num_tokens": 11422130.0,
       "step": 1030
     },
     {
+      "entropy": 0.593072983622551,
       "epoch": 2.047244094488189,
+      "grad_norm": 0.4609375,
       "learning_rate": 4.85954230763197e-05,
+      "loss": 0.5415357112884521,
+      "mean_token_accuracy": 0.8525791645050049,
       "num_tokens": 11538400.0,
       "step": 1040
     },
     {
+      "entropy": 0.5583488404750824,
       "epoch": 2.0669291338582676,
+      "grad_norm": 0.46875,
       "learning_rate": 4.6783937449808856e-05,
+      "loss": 0.4957115173339844,
+      "mean_token_accuracy": 0.8567668944597244,
       "num_tokens": 11645829.0,
       "step": 1050
     },
     {
       "epoch": 2.0669291338582676,
+      "eval_entropy": 0.6570017006662157,
+      "eval_loss": 0.8396118879318237,
+      "eval_mean_token_accuracy": 0.7848087637512772,
       "eval_num_tokens": 11645829.0,
+      "eval_runtime": 15.9273,
+      "eval_samples_per_second": 26.809,
+      "eval_steps_per_second": 3.39,
       "step": 1050
     },
     {
+      "entropy": 0.5133202388882637,
       "epoch": 2.0866141732283463,
       "grad_norm": 0.5546875,
       "learning_rate": 4.499649419260133e-05,
+      "loss": 0.46275715827941893,
+      "mean_token_accuracy": 0.8645182639360428,
       "num_tokens": 11760076.0,
       "step": 1060
     },
     {
+      "entropy": 0.5825602993369102,
       "epoch": 2.106299212598425,
+      "grad_norm": 0.388671875,
       "learning_rate": 4.323390084980353e-05,
+      "loss": 0.5249548435211182,
+      "mean_token_accuracy": 0.8451484978199005,
       "num_tokens": 11875367.0,
       "step": 1070
     },
     {
+      "entropy": 0.5683134436607361,
       "epoch": 2.1259842519685037,
+      "grad_norm": 0.58203125,
       "learning_rate": 4.1496953739633124e-05,
+      "loss": 0.49717249870300295,
+      "mean_token_accuracy": 0.857735213637352,
       "num_tokens": 11983766.0,
       "step": 1080
     },
     {
+      "entropy": 0.5722413748502732,
       "epoch": 2.145669291338583,
+      "grad_norm": 0.61328125,
       "learning_rate": 3.978643759365202e-05,
+      "loss": 0.5132757186889648,
+      "mean_token_accuracy": 0.8525079637765884,
       "num_tokens": 12092559.0,
       "step": 1090
     },
     {
+      "entropy": 0.5411732234060764,
       "epoch": 2.1653543307086616,
+      "grad_norm": 0.52734375,
       "learning_rate": 3.810312520223419e-05,
+      "loss": 0.5036151885986329,
+      "mean_token_accuracy": 0.8593157321214676,
       "num_tokens": 12207307.0,
       "step": 1100
     },
     {
       "epoch": 2.1653543307086616,
+      "eval_entropy": 0.6568675924230505,
+      "eval_loss": 0.8406071662902832,
+      "eval_mean_token_accuracy": 0.7840766156161273,
       "eval_num_tokens": 12207307.0,
+      "eval_runtime": 15.9308,
+      "eval_samples_per_second": 26.803,
+      "eval_steps_per_second": 3.39,
       "step": 1100
     },
     {
+      "entropy": 0.5992338545620441,
       "epoch": 2.1850393700787403,
+      "grad_norm": 0.47265625,
       "learning_rate": 3.644777706542845e-05,
+      "loss": 0.5292766571044922,
+      "mean_token_accuracy": 0.8512318909168244,
       "num_tokens": 12319969.0,
       "step": 1110
     },
     {
+      "entropy": 0.514266274124384,
       "epoch": 2.204724409448819,
+      "grad_norm": 0.5,
       "learning_rate": 3.4821141049373804e-05,
+      "loss": 0.4458413600921631,
+      "mean_token_accuracy": 0.8724471956491471,
       "num_tokens": 12431027.0,
       "step": 1120
     },
     {
+      "entropy": 0.5649555295705795,
       "epoch": 2.2244094488188977,
       "grad_norm": 0.52734375,
       "learning_rate": 3.3223952048422544e-05,
+      "loss": 0.5298549652099609,
+      "mean_token_accuracy": 0.8531913965940475,
       "num_tokens": 12546037.0,
       "step": 1130
     },
     {
+      "entropy": 0.5983025059103966,
       "epoch": 2.2440944881889764,
+      "grad_norm": 0.58203125,
       "learning_rate": 3.165693165312395e-05,
+      "loss": 0.5455004215240479,
+      "mean_token_accuracy": 0.8468506157398223,
       "num_tokens": 12656297.0,
       "step": 1140
     },
     {
+      "entropy": 0.5571755915880203,
       "epoch": 2.263779527559055,
+      "grad_norm": 0.55859375,
       "learning_rate": 3.0120787824218554e-05,
+      "loss": 0.5133816719055175,
+      "mean_token_accuracy": 0.8542670160531998,
       "num_tokens": 12763743.0,
       "step": 1150
     },
     {
       "epoch": 2.263779527559055,
+      "eval_entropy": 0.647307668571119,
+      "eval_loss": 0.844680666923523,
+      "eval_mean_token_accuracy": 0.7836638799420109,
       "eval_num_tokens": 12763743.0,
+      "eval_runtime": 15.9342,
+      "eval_samples_per_second": 26.798,
       "eval_steps_per_second": 3.389,
       "step": 1150
     },
     {
+      "entropy": 0.6052810281515122,
       "epoch": 2.283464566929134,
+      "grad_norm": 0.482421875,
       "learning_rate": 2.8616214572789968e-05,
+      "loss": 0.5399055004119873,
+      "mean_token_accuracy": 0.8467662870883942,
       "num_tokens": 12874080.0,
       "step": 1160
     },
     {
+      "entropy": 0.495437890291214,
       "epoch": 2.3031496062992125,
+      "grad_norm": 0.57421875,
       "learning_rate": 2.7143891646719265e-05,
+      "loss": 0.4656820297241211,
+      "mean_token_accuracy": 0.8652756780385971,
       "num_tokens": 12986029.0,
       "step": 1170
     },
     {
+      "entropy": 0.5415738135576248,
       "epoch": 2.322834645669291,
+      "grad_norm": 0.59765625,
       "learning_rate": 2.570448422358297e-05,
+      "loss": 0.5029004573822021,
+      "mean_token_accuracy": 0.8594674915075302,
       "num_tokens": 13092818.0,
       "step": 1180
     },
     {
+      "entropy": 0.5835595086216927,
       "epoch": 2.34251968503937,
+      "grad_norm": 0.6484375,
       "learning_rate": 2.4298642610134105e-05,
+      "loss": 0.502112865447998,
+      "mean_token_accuracy": 0.8509185254573822,
       "num_tokens": 13206493.0,
       "step": 1190
     },
     {
+      "entropy": 0.6709924265742302,
       "epoch": 2.362204724409449,
+      "grad_norm": 0.6640625,
       "learning_rate": 2.292700194850147e-05,
+      "loss": 0.5888309001922607,
+      "mean_token_accuracy": 0.829518511891365,
       "num_tokens": 13314862.0,
       "step": 1200
     },
     {
       "epoch": 2.362204724409449,
+      "eval_entropy": 0.6516638414727317,
+      "eval_loss": 0.8389837741851807,
+      "eval_mean_token_accuracy": 0.7852442959944407,
       "eval_num_tokens": 13314862.0,
+      "eval_runtime": 15.9159,
+      "eval_samples_per_second": 26.829,
+      "eval_steps_per_second": 3.393,
       "step": 1200
     },
     {
+      "entropy": 0.5645383015275002,
       "epoch": 2.3818897637795278,
+      "grad_norm": 0.5390625,
       "learning_rate": 2.159018192924004e-05,
+      "loss": 0.5259816169738769,
+      "mean_token_accuracy": 0.8538117051124573,
       "num_tokens": 13423520.0,
       "step": 1210
     },
     {
+      "entropy": 0.5880276575684548,
       "epoch": 2.4015748031496065,
+      "grad_norm": 0.439453125,
       "learning_rate": 2.0288786511362525e-05,
+      "loss": 0.5333644866943359,
+      "mean_token_accuracy": 0.8482611387968063,
       "num_tokens": 13536197.0,
       "step": 1220
     },
     {
+      "entropy": 0.5676271229982376,
       "epoch": 2.421259842519685,
+      "grad_norm": 0.494140625,
       "learning_rate": 1.9023403649477813e-05,
+      "loss": 0.5034989833831787,
+      "mean_token_accuracy": 0.8573888152837753,
       "num_tokens": 13649116.0,
       "step": 1230
     },
     {
+      "entropy": 0.5038574181497097,
       "epoch": 2.440944881889764,
+      "grad_norm": 0.435546875,
       "learning_rate": 1.7794605028160462e-05,
+      "loss": 0.45959224700927737,
+      "mean_token_accuracy": 0.866667166352272,
       "num_tokens": 13759803.0,
       "step": 1240
     },
     {
+      "entropy": 0.5891189277172089,
       "epoch": 2.4606299212598426,
       "grad_norm": 0.6875,
       "learning_rate": 1.6602945803670345e-05,
+      "loss": 0.5333945274353027,
+      "mean_token_accuracy": 0.8494920641183853,
       "num_tokens": 13872536.0,
       "step": 1250
     },
     {
       "epoch": 2.4606299212598426,
+      "eval_entropy": 0.6562480860286288,
+      "eval_loss": 0.8404139876365662,
+      "eval_mean_token_accuracy": 0.7847034048151087,
       "eval_num_tokens": 13872536.0,
+      "eval_runtime": 15.9225,
+      "eval_samples_per_second": 26.817,
       "eval_steps_per_second": 3.391,
       "step": 1250
     },
     {
+      "entropy": 0.5312069803476334,
       "epoch": 2.4803149606299213,
+      "grad_norm": 0.67578125,
       "learning_rate": 1.544896435313997e-05,
+      "loss": 0.48858180046081545,
+      "mean_token_accuracy": 0.860382902622223,
       "num_tokens": 13982332.0,
       "step": 1260
     },
     {
+      "entropy": 0.5351930364966393,
       "epoch": 2.5,
+      "grad_norm": 0.53515625,
       "learning_rate": 1.4333182031342185e-05,
+      "loss": 0.5056577682495117,
+      "mean_token_accuracy": 0.857965275645256,
       "num_tokens": 14096535.0,
       "step": 1270
     },
     {
+      "entropy": 0.6251816973090172,
       "epoch": 2.5196850393700787,
+      "grad_norm": 0.4921875,
       "learning_rate": 1.3256102935148496e-05,
+      "loss": 0.5645822048187256,
+      "mean_token_accuracy": 0.8422666400671005,
       "num_tokens": 14214551.0,
       "step": 1280
     },
     {
+      "entropy": 0.6227434769272804,
       "epoch": 2.5393700787401574,
+      "grad_norm": 0.58984375,
       "learning_rate": 1.2218213675784063e-05,
+      "loss": 0.5586906909942627,
+      "mean_token_accuracy": 0.8418498873710633,
       "num_tokens": 14322085.0,
       "step": 1290
     },
     {
+      "entropy": 0.5474544659256935,
       "epoch": 2.559055118110236,
+      "grad_norm": 0.4375,
       "learning_rate": 1.1219983158982917e-05,
+      "loss": 0.49657626152038575,
+      "mean_token_accuracy": 0.8581330984830856,
       "num_tokens": 14431579.0,
       "step": 1300
     },
     {
       "epoch": 2.559055118110236,
+      "eval_entropy": 0.6617519998991931,
+      "eval_loss": 0.8383262157440186,
+      "eval_mean_token_accuracy": 0.7846880113637006,
       "eval_num_tokens": 14431579.0,
+      "eval_runtime": 15.9113,
+      "eval_samples_per_second": 26.836,
+      "eval_steps_per_second": 3.394,
       "step": 1300
     },
     {
+      "entropy": 0.5216698408126831,
       "epoch": 2.578740157480315,
+      "grad_norm": 0.6015625,
       "learning_rate": 1.0261862373141839e-05,
+      "loss": 0.46427440643310547,
+      "mean_token_accuracy": 0.8688691437244416,
       "num_tokens": 14536887.0,
       "step": 1310
     },
     {
+      "entropy": 0.5210306510329247,
       "epoch": 2.5984251968503935,
+      "grad_norm": 0.5625,
       "learning_rate": 9.344284185569463e-06,
+      "loss": 0.4667550563812256,
+      "mean_token_accuracy": 0.8664857149124146,
       "num_tokens": 14649040.0,
       "step": 1320
     },
     {
+      "entropy": 0.5522045940160751,
       "epoch": 2.6181102362204722,
+      "grad_norm": 0.494140625,
       "learning_rate": 8.467663146921979e-06,
+      "loss": 0.4914377689361572,
+      "mean_token_accuracy": 0.8588613420724869,
       "num_tokens": 14757590.0,
       "step": 1330
     },
     {
+      "entropy": 0.5440365478396416,
       "epoch": 2.637795275590551,
+      "grad_norm": 0.470703125,
       "learning_rate": 7.632395303914208e-06,
+      "loss": 0.4774484634399414,
+      "mean_token_accuracy": 0.8620693862438202,
       "num_tokens": 14870411.0,
       "step": 1340
     },
     {
+      "entropy": 0.5705034703016281,
       "epoch": 2.65748031496063,
+      "grad_norm": 0.515625,
       "learning_rate": 6.838858020390615e-06,
+      "loss": 0.5196453094482422,
+      "mean_token_accuracy": 0.8570811003446579,
       "num_tokens": 14986705.0,
       "step": 1350
     },
     {
       "epoch": 2.65748031496063,
+      "eval_entropy": 0.6574608446271332,
+      "eval_loss": 0.8387162685394287,
+      "eval_mean_token_accuracy": 0.7850221616250498,
       "eval_num_tokens": 14986705.0,
+      "eval_runtime": 15.9224,
+      "eval_samples_per_second": 26.818,
+      "eval_steps_per_second": 3.391,
       "step": 1350
     },
     {
+      "entropy": 0.563168890029192,
       "epoch": 2.677165354330709,
       "grad_norm": 0.640625,
       "learning_rate": 6.087409806836752e-06,
+      "loss": 0.5263133525848389,
+      "mean_token_accuracy": 0.8563782393932342,
       "num_tokens": 15095469.0,
       "step": 1360
     },
     {
+      "entropy": 0.5424789242446423,
       "epoch": 2.6968503937007875,
       "grad_norm": 0.5390625,
       "learning_rate": 5.378390158408731e-06,
+      "loss": 0.5066217422485352,
+      "mean_token_accuracy": 0.8590426325798035,
       "num_tokens": 15202248.0,
       "step": 1370
     },
     {
+      "entropy": 0.5028943344950676,
       "epoch": 2.716535433070866,
+      "grad_norm": 0.486328125,
       "learning_rate": 4.712119401553394e-06,
+      "loss": 0.4527781009674072,
+      "mean_token_accuracy": 0.8694937527179718,
       "num_tokens": 15312525.0,
       "step": 1380
     },
     {
+      "entropy": 0.5716224625706673,
       "epoch": 2.736220472440945,
+      "grad_norm": 0.427734375,
       "learning_rate": 4.088898549288689e-06,
+      "loss": 0.5240719795227051,
+      "mean_token_accuracy": 0.8516256272792816,
       "num_tokens": 15424901.0,
       "step": 1390
     },
     {
+      "entropy": 0.5507901981472969,
       "epoch": 2.7559055118110236,
+      "grad_norm": 0.5234375,
       "learning_rate": 3.5090091652096977e-06,
+      "loss": 0.490921688079834,
+      "mean_token_accuracy": 0.8620281994342804,
       "num_tokens": 15534177.0,
       "step": 1400
     },
     {
       "epoch": 2.7559055118110236,
+      "eval_entropy": 0.659479867528986,
+      "eval_loss": 0.8384642601013184,
+      "eval_mean_token_accuracy": 0.7847910379921949,
       "eval_num_tokens": 15534177.0,
+      "eval_runtime": 15.9298,
+      "eval_samples_per_second": 26.805,
+      "eval_steps_per_second": 3.39,
       "step": 1400
     },
     {
+      "entropy": 0.6328877091407776,
       "epoch": 2.7755905511811023,
+      "grad_norm": 0.62109375,
       "learning_rate": 2.972713236281632e-06,
+      "loss": 0.5606705188751221,
+      "mean_token_accuracy": 0.8396612524986267,
       "num_tokens": 15642725.0,
       "step": 1410
     },
     {
+      "entropy": 0.5535789176821708,
       "epoch": 2.795275590551181,
+      "grad_norm": 0.48046875,
       "learning_rate": 2.480253054477244e-06,
+      "loss": 0.5025914669036865,
+      "mean_token_accuracy": 0.8595594942569733,
       "num_tokens": 15749590.0,
       "step": 1420
     },
     {
+      "entropy": 0.5887665987014771,
       "epoch": 2.8149606299212597,
+      "grad_norm": 0.51171875,
       "learning_rate": 2.031851107312366e-06,
+      "loss": 0.5465099334716796,
+      "mean_token_accuracy": 0.8492792785167694,
       "num_tokens": 15865143.0,
       "step": 1430
     },
     {
+      "entropy": 0.5139057531952858,
       "epoch": 2.8346456692913384,
       "grad_norm": 0.59765625,
       "learning_rate": 1.6277099773286951e-06,
+      "loss": 0.4561887264251709,
+      "mean_token_accuracy": 0.8726439267396927,
       "num_tokens": 15967667.0,
       "step": 1440
     },
     {
+      "entropy": 0.5473496794700623,
       "epoch": 2.8543307086614176,
       "grad_norm": 0.5546875,
       "learning_rate": 1.2680122505694502e-06,
+      "loss": 0.4959095001220703,
+      "mean_token_accuracy": 0.858627387881279,
       "num_tokens": 16077301.0,
       "step": 1450
     },
     {
       "epoch": 2.8543307086614176,
+      "eval_entropy": 0.6591992874940237,
+      "eval_loss": 0.8384377956390381,
+      "eval_mean_token_accuracy": 0.7848462451387335,
       "eval_num_tokens": 16077301.0,
+      "eval_runtime": 15.8973,
+      "eval_samples_per_second": 26.86,
+      "eval_steps_per_second": 3.397,
       "step": 1450
     }
   ],

checkpoint-150/adapter_config.json CHANGED Viewed

@@ -30,13 +30,13 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
     "down_proj",
-    "v_proj",
-    "o_proj",
-    "q_proj",
     "up_proj",
-    "k_proj",
-    "gate_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "k_proj",
     "down_proj",
+    "gate_proj",
     "up_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
   ],
   "target_parameters": null,
   "task_type": "CAUSAL_LM",

checkpoint-150/adapter_model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a2aeed2f417956480c97cbb0cc8df20670ab940ec3a45efff201ae0b05426793
 size 73911504

 version https://git-lfs.github.com/spec/v1
+oid sha256:834bc43bac21ccbd7fe42ee9039e27282b5aeed141b4384557ffec88ae7824b7
 size 73911504

checkpoint-150/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ffe2d199357cd2a6acc787b962a7335bfa2bf4c195c2efe0e80fe87bbc1892ea
 size 75480837

 version https://git-lfs.github.com/spec/v1
+oid sha256:0fdb571e6a3b59b0daec38ef783275627fc526b1823b2a6c5d6da1919b40c4b1
 size 75480837

checkpoint-150/trainer_state.json CHANGED Viewed

@@ -10,186 +10,186 @@
   "is_world_process_zero": true,
   "log_history": [
     {
-      "entropy": 1.4378354728221894,
       "epoch": 0.01968503937007874,
-      "grad_norm": 0.62109375,
       "learning_rate": 3.91304347826087e-05,
-      "loss": 1.657682228088379,
-      "mean_token_accuracy": 0.6598833322525024,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
-      "entropy": 1.5141301095485686,
       "epoch": 0.03937007874015748,
-      "grad_norm": 0.44140625,
       "learning_rate": 8.260869565217392e-05,
-      "loss": 1.4646138191223144,
-      "mean_token_accuracy": 0.6834615051746369,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
-      "entropy": 1.2295305937528611,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
-      "loss": 1.1134957313537597,
-      "mean_token_accuracy": 0.7453642427921295,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
-      "entropy": 1.0573274463415145,
       "epoch": 0.07874015748031496,
-      "grad_norm": 0.474609375,
       "learning_rate": 0.00016956521739130436,
-      "loss": 1.077296829223633,
-      "mean_token_accuracy": 0.7446383863687516,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
-      "entropy": 0.952904561161995,
       "epoch": 0.0984251968503937,
-      "grad_norm": 0.50390625,
       "learning_rate": 0.00019999796688511753,
-      "loss": 0.9590332984924317,
-      "mean_token_accuracy": 0.768164387345314,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
-      "eval_entropy": 1.0900797645250957,
-      "eval_loss": 1.0484484434127808,
-      "eval_mean_token_accuracy": 0.7461776600943671,
       "eval_num_tokens": 551175.0,
-      "eval_runtime": 15.9866,
-      "eval_samples_per_second": 26.71,
-      "eval_steps_per_second": 3.378,
       "step": 50
     },
     {
-      "entropy": 0.9487997680902481,
       "epoch": 0.11811023622047244,
-      "grad_norm": 0.59375,
       "learning_rate": 0.0001999618249203163,
-      "loss": 0.9577752113342285,
-      "mean_token_accuracy": 0.7689193069934845,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
-      "entropy": 1.0506175458431244,
       "epoch": 0.1377952755905512,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019988052141977201,
-      "loss": 0.9976933479309082,
-      "mean_token_accuracy": 0.756487911939621,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
-      "entropy": 0.8528705358505249,
       "epoch": 0.15748031496062992,
-      "grad_norm": 0.5078125,
       "learning_rate": 0.0001997540931154166,
-      "loss": 0.8407608032226562,
-      "mean_token_accuracy": 0.7868376106023789,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
-      "entropy": 0.9220433413982392,
       "epoch": 0.17716535433070865,
-      "grad_norm": 0.484375,
       "learning_rate": 0.00019958259712601903,
-      "loss": 0.8958203315734863,
-      "mean_token_accuracy": 0.7736223608255386,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
-      "entropy": 0.8399994730949402,
       "epoch": 0.1968503937007874,
-      "grad_norm": 0.51953125,
       "learning_rate": 0.00019936611093137983,
-      "loss": 0.8394058227539063,
-      "mean_token_accuracy": 0.7876519203186035,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
-      "eval_entropy": 0.9908565746413337,
-      "eval_loss": 0.949418842792511,
-      "eval_mean_token_accuracy": 0.7609190278583102,
       "eval_num_tokens": 1103999.0,
-      "eval_runtime": 15.9475,
-      "eval_samples_per_second": 26.775,
-      "eval_steps_per_second": 3.386,
       "step": 100
     },
     {
-      "entropy": 0.8772778362035751,
       "epoch": 0.21653543307086615,
-      "grad_norm": 0.40625,
       "learning_rate": 0.00019910473233732668,
-      "loss": 0.8347702026367188,
-      "mean_token_accuracy": 0.7877911299467086,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
-      "entropy": 0.8745568007230758,
       "epoch": 0.23622047244094488,
-      "grad_norm": 0.609375,
       "learning_rate": 0.00019879857943152674,
-      "loss": 0.8359953880310058,
-      "mean_token_accuracy": 0.7790884494781494,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
-      "entropy": 0.878146781027317,
       "epoch": 0.2559055118110236,
-      "grad_norm": 0.5390625,
       "learning_rate": 0.00019844779053013637,
-      "loss": 0.8335110664367675,
-      "mean_token_accuracy": 0.7791220843791962,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
-      "entropy": 0.788033875823021,
       "epoch": 0.2755905511811024,
-      "grad_norm": 0.546875,
       "learning_rate": 0.00019805252411531126,
-      "loss": 0.7967269420623779,
-      "mean_token_accuracy": 0.7931749552488327,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
-      "entropy": 0.8667207360267639,
       "epoch": 0.2952755905511811,
-      "grad_norm": 0.5546875,
       "learning_rate": 0.0001976129587636064,
-      "loss": 0.8101954460144043,
-      "mean_token_accuracy": 0.7908060193061829,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
-      "eval_entropy": 0.9391163366812246,
-      "eval_loss": 0.9146928191184998,
-      "eval_mean_token_accuracy": 0.7670937886944523,
       "eval_num_tokens": 1654462.0,
-      "eval_runtime": 15.956,
-      "eval_samples_per_second": 26.761,
-      "eval_steps_per_second": 3.384,
       "step": 150
     }
   ],

   "is_world_process_zero": true,
   "log_history": [
     {
+      "entropy": 1.437505340576172,
       "epoch": 0.01968503937007874,
+      "grad_norm": 0.63671875,
       "learning_rate": 3.91304347826087e-05,
+      "loss": 1.657536506652832,
+      "mean_token_accuracy": 0.6597658038139343,
       "num_tokens": 115439.0,
       "step": 10
     },
     {
+      "entropy": 1.5124324679374694,
       "epoch": 0.03937007874015748,
+      "grad_norm": 0.4453125,
       "learning_rate": 8.260869565217392e-05,
+      "loss": 1.4648273468017579,
+      "mean_token_accuracy": 0.683241218328476,
       "num_tokens": 225264.0,
       "step": 20
     },
     {
+      "entropy": 1.2266616582870484,
       "epoch": 0.05905511811023622,
       "grad_norm": 0.6796875,
       "learning_rate": 0.00012608695652173915,
+      "loss": 1.113858699798584,
+      "mean_token_accuracy": 0.7449506461620331,
       "num_tokens": 330923.0,
       "step": 30
     },
     {
+      "entropy": 1.05804543197155,
       "epoch": 0.07874015748031496,
+      "grad_norm": 0.47265625,
       "learning_rate": 0.00016956521739130436,
+      "loss": 1.076363468170166,
+      "mean_token_accuracy": 0.7455369174480438,
       "num_tokens": 437622.0,
       "step": 40
     },
     {
+      "entropy": 0.9502384155988693,
       "epoch": 0.0984251968503937,
+      "grad_norm": 0.5,
       "learning_rate": 0.00019999796688511753,
+      "loss": 0.9587571144104003,
+      "mean_token_accuracy": 0.7683711260557174,
       "num_tokens": 551175.0,
       "step": 50
     },
     {
       "epoch": 0.0984251968503937,
+      "eval_entropy": 1.0894114132280703,
+      "eval_loss": 1.0480613708496094,
+      "eval_mean_token_accuracy": 0.745736007337217,
       "eval_num_tokens": 551175.0,
+      "eval_runtime": 15.9708,
+      "eval_samples_per_second": 26.736,
+      "eval_steps_per_second": 3.381,
       "step": 50
     },
     {
+      "entropy": 0.9499408066272735,
       "epoch": 0.11811023622047244,
+      "grad_norm": 0.5859375,
       "learning_rate": 0.0001999618249203163,
+      "loss": 0.9569917678833008,
+      "mean_token_accuracy": 0.7686831414699554,
       "num_tokens": 658006.0,
       "step": 60
     },
     {
+      "entropy": 1.0497896373271942,
       "epoch": 0.1377952755905512,
+      "grad_norm": 0.52734375,
       "learning_rate": 0.00019988052141977201,
+      "loss": 0.9973426818847656,
+      "mean_token_accuracy": 0.7564341962337494,
       "num_tokens": 771278.0,
       "step": 70
     },
     {
+      "entropy": 0.8503248453140259,
       "epoch": 0.15748031496062992,
+      "grad_norm": 0.5,
       "learning_rate": 0.0001997540931154166,
+      "loss": 0.839632225036621,
+      "mean_token_accuracy": 0.7869657546281814,
       "num_tokens": 886071.0,
       "step": 80
     },
     {
+      "entropy": 0.9232195496559144,
       "epoch": 0.17716535433070865,
+      "grad_norm": 0.482421875,
       "learning_rate": 0.00019958259712601903,
+      "loss": 0.8961009025573731,
+      "mean_token_accuracy": 0.7736886650323868,
       "num_tokens": 996361.0,
       "step": 90
     },
     {
+      "entropy": 0.8409020006656647,
       "epoch": 0.1968503937007874,
+      "grad_norm": 0.515625,
       "learning_rate": 0.00019936611093137983,
+      "loss": 0.839316463470459,
+      "mean_token_accuracy": 0.787284591794014,
       "num_tokens": 1103999.0,
       "step": 100
     },
     {
       "epoch": 0.1968503937007874,
+      "eval_entropy": 0.988924115896225,
+      "eval_loss": 0.9491098523139954,
+      "eval_mean_token_accuracy": 0.7609603978969433,
       "eval_num_tokens": 1103999.0,
+      "eval_runtime": 15.8986,
+      "eval_samples_per_second": 26.858,
+      "eval_steps_per_second": 3.397,
       "step": 100
     },
     {
+      "entropy": 0.8768177807331086,
       "epoch": 0.21653543307086615,
+      "grad_norm": 0.41015625,
       "learning_rate": 0.00019910473233732668,
+      "loss": 0.8348051071166992,
+      "mean_token_accuracy": 0.7883853554725647,
       "num_tokens": 1216586.0,
       "step": 110
     },
     {
+      "entropy": 0.8739962369203568,
       "epoch": 0.23622047244094488,
+      "grad_norm": 0.59375,
       "learning_rate": 0.00019879857943152674,
+      "loss": 0.8352558135986328,
+      "mean_token_accuracy": 0.7789304822683334,
       "num_tokens": 1324767.0,
       "step": 120
     },
     {
+      "entropy": 0.8777758985757828,
       "epoch": 0.2559055118110236,
+      "grad_norm": 0.54296875,
       "learning_rate": 0.00019844779053013637,
+      "loss": 0.8331022262573242,
+      "mean_token_accuracy": 0.7784219890832901,
       "num_tokens": 1432356.0,
       "step": 130
     },
     {
+      "entropy": 0.7877890735864639,
       "epoch": 0.2755905511811024,
+      "grad_norm": 0.55078125,
       "learning_rate": 0.00019805252411531126,
+      "loss": 0.7960677146911621,
+      "mean_token_accuracy": 0.7926767230033874,
       "num_tokens": 1542545.0,
       "step": 140
     },
     {
+      "entropy": 0.8658760845661163,
       "epoch": 0.2952755905511811,
+      "grad_norm": 0.5625,
       "learning_rate": 0.0001976129587636064,
+      "loss": 0.8095477104187012,
+      "mean_token_accuracy": 0.7904855847358704,
       "num_tokens": 1654462.0,
       "step": 150
     },
     {
       "epoch": 0.2952755905511811,
+      "eval_entropy": 0.9384135085123556,
+      "eval_loss": 0.9150573015213013,
+      "eval_mean_token_accuracy": 0.7675120333830515,
       "eval_num_tokens": 1654462.0,
+      "eval_runtime": 15.91,
+      "eval_samples_per_second": 26.839,
+      "eval_steps_per_second": 3.394,
       "step": 150
     }
   ],