Instructions to use codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT")
model = AutoModelForCausalLM.from_pretrained("codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT

SGLang

How to use codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT with Docker Model Runner:
```
docker model run hf.co/codingmonster1234/Llama-3.1-8B-Instruct-Chess-Reasoning-SFT
```

Llama-3.1-8B-Instruct-Chess-Reasoning-SFT / trainer_state.json

codingmonster1234

Upload folder using huggingface_hub

3b24b3e verified about 2 months ago

Raw

History Blame Contribute Delete

50.4 kB

	{
	"best_global_step": null,
	"best_metric": null,
	"best_model_checkpoint": null,
	"epoch": 1.0,
	"eval_steps": 500,
	"global_step": 168,
	"is_hyper_param_search": false,
	"is_local_process_zero": true,
	"is_world_process_zero": true,
	"log_history": [
	{
	"entropy": 1.8196021169424057,
	"epoch": 0.005956813104988831,
	"grad_norm": 1.28125,
	"learning_rate": 2e-05,
	"loss": 1.7708154916763306,
	"mean_token_accuracy": 0.599120743572712,
	"num_tokens": 14922.0,
	"step": 1
	},
	{
	"entropy": 1.2802767902612686,
	"epoch": 0.011913626209977662,
	"grad_norm": 3.953125,
	"learning_rate": 1.9999805729315383e-05,
	"loss": 2.1650872230529785,
	"mean_token_accuracy": 0.5873465985059738,
	"num_tokens": 30082.0,
	"step": 2
	},
	{
	"entropy": 1.7827413529157639,
	"epoch": 0.017870439314966492,
	"grad_norm": 1.3203125,
	"learning_rate": 1.999922292480975e-05,
	"loss": 1.5451161861419678,
	"mean_token_accuracy": 0.6535470560193062,
	"num_tokens": 44458.0,
	"step": 3
	},
	{
	"entropy": 1.881892368197441,
	"epoch": 0.023827252419955324,
	"grad_norm": 0.65625,
	"learning_rate": 1.9998251609127465e-05,
	"loss": 1.400605320930481,
	"mean_token_accuracy": 0.6719935461878777,
	"num_tokens": 59381.0,
	"step": 4
	},
	{
	"entropy": 1.9049408286809921,
	"epoch": 0.029784065524944156,
	"grad_norm": 0.54296875,
	"learning_rate": 1.9996891820008165e-05,
	"loss": 1.3754955530166626,
	"mean_token_accuracy": 0.6717504411935806,
	"num_tokens": 74182.0,
	"step": 5
	},
	{
	"entropy": 1.79595048725605,
	"epoch": 0.035740878629932984,
	"grad_norm": 0.55859375,
	"learning_rate": 1.9995143610285275e-05,
	"loss": 1.2846546173095703,
	"mean_token_accuracy": 0.6872891932725906,
	"num_tokens": 88869.0,
	"step": 6
	},
	{
	"entropy": 1.6670185774564743,
	"epoch": 0.04169769173492182,
	"grad_norm": 0.5390625,
	"learning_rate": 1.9993007047883988e-05,
	"loss": 1.1514159440994263,
	"mean_token_accuracy": 0.7146859243512154,
	"num_tokens": 103614.0,
	"step": 7
	},
	{
	"entropy": 1.6106412559747696,
	"epoch": 0.04765450483991065,
	"grad_norm": 0.50390625,
	"learning_rate": 1.999048221581858e-05,
	"loss": 1.1496959924697876,
	"mean_token_accuracy": 0.7096454501152039,
	"num_tokens": 118065.0,
	"step": 8
	},
	{
	"entropy": 1.5453107059001923,
	"epoch": 0.05361131794489948,
	"grad_norm": 0.5078125,
	"learning_rate": 1.9987569212189224e-05,
	"loss": 1.142591118812561,
	"mean_token_accuracy": 0.7142270430922508,
	"num_tokens": 133029.0,
	"step": 9
	},
	{
	"entropy": 1.5210696011781693,
	"epoch": 0.05956813104988831,
	"grad_norm": 0.47265625,
	"learning_rate": 1.998426815017817e-05,
	"loss": 1.0974477529525757,
	"mean_token_accuracy": 0.7146398201584816,
	"num_tokens": 147810.0,
	"step": 10
	},
	{
	"entropy": 1.5126763880252838,
	"epoch": 0.06552494415487714,
	"grad_norm": 0.46875,
	"learning_rate": 1.9980579158045322e-05,
	"loss": 1.1492289304733276,
	"mean_token_accuracy": 0.7059202417731285,
	"num_tokens": 162781.0,
	"step": 11
	},
	{
	"entropy": 1.5258464515209198,
	"epoch": 0.07148175725986597,
	"grad_norm": 0.4375,
	"learning_rate": 1.997650237912329e-05,
	"loss": 1.068825125694275,
	"mean_token_accuracy": 0.7209103479981422,
	"num_tokens": 176562.0,
	"step": 12
	},
	{
	"entropy": 1.5423607379198074,
	"epoch": 0.0774385703648548,
	"grad_norm": 0.462890625,
	"learning_rate": 1.9972037971811802e-05,
	"loss": 1.1400907039642334,
	"mean_token_accuracy": 0.7075617015361786,
	"num_tokens": 190938.0,
	"step": 13
	},
	{
	"entropy": 1.5203707218170166,
	"epoch": 0.08339538346984364,
	"grad_norm": 0.42578125,
	"learning_rate": 1.996718610957155e-05,
	"loss": 1.0239043235778809,
	"mean_token_accuracy": 0.7295805141329765,
	"num_tokens": 205420.0,
	"step": 14
	},
	{
	"entropy": 1.5007787346839905,
	"epoch": 0.08935219657483247,
	"grad_norm": 0.40234375,
	"learning_rate": 1.9961946980917457e-05,
	"loss": 0.9978266358375549,
	"mean_token_accuracy": 0.7387106791138649,
	"num_tokens": 219949.0,
	"step": 15
	},
	{
	"entropy": 1.4767527133226395,
	"epoch": 0.0953090096798213,
	"grad_norm": 0.375,
	"learning_rate": 1.9956320789411338e-05,
	"loss": 1.0312634706497192,
	"mean_token_accuracy": 0.7306794673204422,
	"num_tokens": 235321.0,
	"step": 16
	},
	{
	"entropy": 1.4968346804380417,
	"epoch": 0.10126582278481013,
	"grad_norm": 0.41796875,
	"learning_rate": 1.9950307753654016e-05,
	"loss": 1.0707520246505737,
	"mean_token_accuracy": 0.7274535074830055,
	"num_tokens": 249201.0,
	"step": 17
	},
	{
	"entropy": 1.476852685213089,
	"epoch": 0.10722263588979895,
	"grad_norm": 0.4140625,
	"learning_rate": 1.99439081072768e-05,
	"loss": 1.0673410892486572,
	"mean_token_accuracy": 0.7224985063076019,
	"num_tokens": 263940.0,
	"step": 18
	},
	{
	"entropy": 1.4178977608680725,
	"epoch": 0.11317944899478778,
	"grad_norm": 0.404296875,
	"learning_rate": 1.9937122098932428e-05,
	"loss": 0.9890223741531372,
	"mean_token_accuracy": 0.7390217557549477,
	"num_tokens": 279156.0,
	"step": 19
	},
	{
	"entropy": 1.4455726444721222,
	"epoch": 0.11913626209977662,
	"grad_norm": 0.404296875,
	"learning_rate": 1.9929949992285397e-05,
	"loss": 1.0276429653167725,
	"mean_token_accuracy": 0.7296848446130753,
	"num_tokens": 293490.0,
	"step": 20
	},
	{
	"entropy": 1.4110448211431503,
	"epoch": 0.12509307520476545,
	"grad_norm": 0.380859375,
	"learning_rate": 1.9922392066001724e-05,
	"loss": 0.9719092845916748,
	"mean_token_accuracy": 0.7374792844057083,
	"num_tokens": 308809.0,
	"step": 21
	},
	{
	"entropy": 1.405819684267044,
	"epoch": 0.13104988830975428,
	"grad_norm": 0.392578125,
	"learning_rate": 1.9914448613738107e-05,
	"loss": 0.9487384557723999,
	"mean_token_accuracy": 0.7479015067219734,
	"num_tokens": 323801.0,
	"step": 22
	},
	{
	"entropy": 1.3968916982412338,
	"epoch": 0.1370067014147431,
	"grad_norm": 0.3828125,
	"learning_rate": 1.9906119944130527e-05,
	"loss": 0.9097103476524353,
	"mean_token_accuracy": 0.7529696971178055,
	"num_tokens": 338809.0,
	"step": 23
	},
	{
	"entropy": 1.4245737493038177,
	"epoch": 0.14296351451973194,
	"grad_norm": 0.36328125,
	"learning_rate": 1.9897406380782262e-05,
	"loss": 0.9830621480941772,
	"mean_token_accuracy": 0.7378853410482407,
	"num_tokens": 353678.0,
	"step": 24
	},
	{
	"entropy": 1.3906199932098389,
	"epoch": 0.14892032762472077,
	"grad_norm": 0.365234375,
	"learning_rate": 1.9888308262251286e-05,
	"loss": 0.9530032277107239,
	"mean_token_accuracy": 0.7401829957962036,
	"num_tokens": 368974.0,
	"step": 25
	},
	{
	"entropy": 1.4205086827278137,
	"epoch": 0.1548771407297096,
	"grad_norm": 0.400390625,
	"learning_rate": 1.9878825942037147e-05,
	"loss": 0.9490904211997986,
	"mean_token_accuracy": 0.7428939715027809,
	"num_tokens": 383442.0,
	"step": 26
	},
	{
	"entropy": 1.4382268190383911,
	"epoch": 0.16083395383469842,
	"grad_norm": 0.408203125,
	"learning_rate": 1.9868959788567213e-05,
	"loss": 0.9662237763404846,
	"mean_token_accuracy": 0.7409680560231209,
	"num_tokens": 397144.0,
	"step": 27
	},
	{
	"entropy": 1.4032137542963028,
	"epoch": 0.16679076693968728,
	"grad_norm": 0.392578125,
	"learning_rate": 1.985871018518236e-05,
	"loss": 0.9321739673614502,
	"mean_token_accuracy": 0.7473694160580635,
	"num_tokens": 411072.0,
	"step": 28
	},
	{
	"entropy": 1.4221246838569641,
	"epoch": 0.1727475800446761,
	"grad_norm": 0.3828125,
	"learning_rate": 1.9848077530122083e-05,
	"loss": 0.9609735012054443,
	"mean_token_accuracy": 0.7417895272374153,
	"num_tokens": 424984.0,
	"step": 29
	},
	{
	"entropy": 1.3734628409147263,
	"epoch": 0.17870439314966494,
	"grad_norm": 0.3671875,
	"learning_rate": 1.9837062236509013e-05,
	"loss": 0.8709937930107117,
	"mean_token_accuracy": 0.7573041170835495,
	"num_tokens": 439653.0,
	"step": 30
	},
	{
	"entropy": 1.4108192771673203,
	"epoch": 0.18466120625465376,
	"grad_norm": 0.423828125,
	"learning_rate": 1.9825664732332886e-05,
	"loss": 0.9877131581306458,
	"mean_token_accuracy": 0.7332894876599312,
	"num_tokens": 453859.0,
	"step": 31
	},
	{
	"entropy": 1.4060807824134827,
	"epoch": 0.1906180193596426,
	"grad_norm": 0.376953125,
	"learning_rate": 1.981388546043388e-05,
	"loss": 1.0199761390686035,
	"mean_token_accuracy": 0.7262433990836143,
	"num_tokens": 469134.0,
	"step": 32
	},
	{
	"entropy": 1.395486667752266,
	"epoch": 0.19657483246463142,
	"grad_norm": 0.3984375,
	"learning_rate": 1.9801724878485438e-05,
	"loss": 0.9763211011886597,
	"mean_token_accuracy": 0.7416960969567299,
	"num_tokens": 483181.0,
	"step": 33
	},
	{
	"entropy": 1.371009811758995,
	"epoch": 0.20253164556962025,
	"grad_norm": 0.3671875,
	"learning_rate": 1.9789183458976485e-05,
	"loss": 0.8935137391090393,
	"mean_token_accuracy": 0.7588988393545151,
	"num_tokens": 497611.0,
	"step": 34
	},
	{
	"entropy": 1.3474263399839401,
	"epoch": 0.20848845867460908,
	"grad_norm": 0.359375,
	"learning_rate": 1.977626168919305e-05,
	"loss": 0.8481594324111938,
	"mean_token_accuracy": 0.7649582549929619,
	"num_tokens": 511742.0,
	"step": 35
	},
	{
	"entropy": 1.3111611157655716,
	"epoch": 0.2144452717795979,
	"grad_norm": 0.3359375,
	"learning_rate": 1.9762960071199334e-05,
	"loss": 0.8989245891571045,
	"mean_token_accuracy": 0.7640852630138397,
	"num_tokens": 527336.0,
	"step": 36
	},
	{
	"entropy": 1.3298669755458832,
	"epoch": 0.22040208488458674,
	"grad_norm": 0.3671875,
	"learning_rate": 1.9749279121818235e-05,
	"loss": 0.8918904066085815,
	"mean_token_accuracy": 0.7584179416298866,
	"num_tokens": 541854.0,
	"step": 37
	},
	{
	"entropy": 1.3399434834718704,
	"epoch": 0.22635889798957556,
	"grad_norm": 0.365234375,
	"learning_rate": 1.9735219372611232e-05,
	"loss": 0.9522192478179932,
	"mean_token_accuracy": 0.7403313592076302,
	"num_tokens": 556835.0,
	"step": 38
	},
	{
	"entropy": 1.3322739899158478,
	"epoch": 0.23231571109456442,
	"grad_norm": 0.34765625,
	"learning_rate": 1.9720781369857747e-05,
	"loss": 0.8859533071517944,
	"mean_token_accuracy": 0.7525262087583542,
	"num_tokens": 571583.0,
	"step": 39
	},
	{
	"entropy": 1.3456282019615173,
	"epoch": 0.23827252419955325,
	"grad_norm": 0.35546875,
	"learning_rate": 1.970596567453391e-05,
	"loss": 0.9205423593521118,
	"mean_token_accuracy": 0.7516355887055397,
	"num_tokens": 585756.0,
	"step": 40
	},
	{
	"entropy": 1.34938944876194,
	"epoch": 0.24422933730454208,
	"grad_norm": 0.3671875,
	"learning_rate": 1.969077286229078e-05,
	"loss": 0.9173880219459534,
	"mean_token_accuracy": 0.7490571588277817,
	"num_tokens": 600414.0,
	"step": 41
	},
	{
	"entropy": 1.342022642493248,
	"epoch": 0.2501861504095309,
	"grad_norm": 0.3671875,
	"learning_rate": 1.9675203523431964e-05,
	"loss": 0.9510252475738525,
	"mean_token_accuracy": 0.7395995110273361,
	"num_tokens": 615312.0,
	"step": 42
	},
	{
	"entropy": 1.3256245851516724,
	"epoch": 0.2561429635145197,
	"grad_norm": 0.359375,
	"learning_rate": 1.9659258262890683e-05,
	"loss": 0.8812260031700134,
	"mean_token_accuracy": 0.7551488727331161,
	"num_tokens": 630044.0,
	"step": 43
	},
	{
	"entropy": 1.300643116235733,
	"epoch": 0.26209977661950856,
	"grad_norm": 0.34765625,
	"learning_rate": 1.964293770020628e-05,
	"loss": 0.8403468728065491,
	"mean_token_accuracy": 0.7659397423267365,
	"num_tokens": 644892.0,
	"step": 44
	},
	{
	"entropy": 1.3294509053230286,
	"epoch": 0.2680565897244974,
	"grad_norm": 0.388671875,
	"learning_rate": 1.962624246950012e-05,
	"loss": 0.8782874345779419,
	"mean_token_accuracy": 0.7567607760429382,
	"num_tokens": 659231.0,
	"step": 45
	},
	{
	"entropy": 1.3091595023870468,
	"epoch": 0.2740134028294862,
	"grad_norm": 0.376953125,
	"learning_rate": 1.9609173219450998e-05,
	"loss": 0.9257646799087524,
	"mean_token_accuracy": 0.7511701583862305,
	"num_tokens": 673897.0,
	"step": 46
	},
	{
	"entropy": 1.3130007237195969,
	"epoch": 0.2799702159344751,
	"grad_norm": 0.3671875,
	"learning_rate": 1.9591730613269878e-05,
	"loss": 0.9010749459266663,
	"mean_token_accuracy": 0.7568229958415031,
	"num_tokens": 689100.0,
	"step": 47
	},
	{
	"entropy": 1.329409897327423,
	"epoch": 0.2859270290394639,
	"grad_norm": 0.404296875,
	"learning_rate": 1.957391532867418e-05,
	"loss": 0.889941930770874,
	"mean_token_accuracy": 0.7545234337449074,
	"num_tokens": 703112.0,
	"step": 48
	},
	{
	"entropy": 1.2916735112667084,
	"epoch": 0.29188384214445273,
	"grad_norm": 0.375,
	"learning_rate": 1.955572805786141e-05,
	"loss": 0.8604971170425415,
	"mean_token_accuracy": 0.7610657885670662,
	"num_tokens": 718088.0,
	"step": 49
	},
	{
	"entropy": 1.3215313851833344,
	"epoch": 0.29784065524944153,
	"grad_norm": 0.373046875,
	"learning_rate": 1.953716950748227e-05,
	"loss": 0.8792937994003296,
	"mean_token_accuracy": 0.7575221061706543,
	"num_tokens": 732625.0,
	"step": 50
	},
	{
	"entropy": 1.3200262635946274,
	"epoch": 0.3037974683544304,
	"grad_norm": 0.369140625,
	"learning_rate": 1.9518240398613226e-05,
	"loss": 0.891180157661438,
	"mean_token_accuracy": 0.7595923095941544,
	"num_tokens": 747023.0,
	"step": 51
	},
	{
	"entropy": 1.3076974749565125,
	"epoch": 0.3097542814594192,
	"grad_norm": 0.365234375,
	"learning_rate": 1.9498941466728462e-05,
	"loss": 0.8625985383987427,
	"mean_token_accuracy": 0.7610806971788406,
	"num_tokens": 762573.0,
	"step": 52
	},
	{
	"entropy": 1.3317556381225586,
	"epoch": 0.31571109456440805,
	"grad_norm": 0.365234375,
	"learning_rate": 1.947927346167132e-05,
	"loss": 0.8708853721618652,
	"mean_token_accuracy": 0.7519596815109253,
	"num_tokens": 777221.0,
	"step": 53
	},
	{
	"entropy": 1.354643628001213,
	"epoch": 0.32166790766939685,
	"grad_norm": 0.36328125,
	"learning_rate": 1.945923714762516e-05,
	"loss": 0.9412155151367188,
	"mean_token_accuracy": 0.7419139668345451,
	"num_tokens": 792127.0,
	"step": 54
	},
	{
	"entropy": 1.3347897082567215,
	"epoch": 0.3276247207743857,
	"grad_norm": 0.365234375,
	"learning_rate": 1.9438833303083677e-05,
	"loss": 0.8442082405090332,
	"mean_token_accuracy": 0.7618155255913734,
	"num_tokens": 806760.0,
	"step": 55
	},
	{
	"entropy": 1.3102798759937286,
	"epoch": 0.33358153387937456,
	"grad_norm": 0.359375,
	"learning_rate": 1.9418062720820636e-05,
	"loss": 0.8157958984375,
	"mean_token_accuracy": 0.7714768648147583,
	"num_tokens": 821430.0,
	"step": 56
	},
	{
	"entropy": 1.3602834939956665,
	"epoch": 0.33953834698436336,
	"grad_norm": 0.373046875,
	"learning_rate": 1.9396926207859085e-05,
	"loss": 0.9252448678016663,
	"mean_token_accuracy": 0.7447722256183624,
	"num_tokens": 835919.0,
	"step": 57
	},
	{
	"entropy": 1.3342349529266357,
	"epoch": 0.3454951600893522,
	"grad_norm": 0.369140625,
	"learning_rate": 1.9375424585439994e-05,
	"loss": 0.9243099689483643,
	"mean_token_accuracy": 0.7476465478539467,
	"num_tokens": 851231.0,
	"step": 58
	},
	{
	"entropy": 1.3102659434080124,
	"epoch": 0.351451973194341,
	"grad_norm": 0.3515625,
	"learning_rate": 1.935355868899034e-05,
	"loss": 0.7817438840866089,
	"mean_token_accuracy": 0.7773127183318138,
	"num_tokens": 865741.0,
	"step": 59
	},
	{
	"entropy": 1.3435720950365067,
	"epoch": 0.3574087862993299,
	"grad_norm": 0.404296875,
	"learning_rate": 1.9331329368090664e-05,
	"loss": 0.897109866142273,
	"mean_token_accuracy": 0.7542032599449158,
	"num_tokens": 879839.0,
	"step": 60
	},
	{
	"entropy": 1.3139615505933762,
	"epoch": 0.3633655994043187,
	"grad_norm": 0.357421875,
	"learning_rate": 1.9308737486442045e-05,
	"loss": 0.8411209583282471,
	"mean_token_accuracy": 0.768218956887722,
	"num_tokens": 894957.0,
	"step": 61
	},
	{
	"entropy": 1.354519784450531,
	"epoch": 0.36932241250930753,
	"grad_norm": 0.388671875,
	"learning_rate": 1.9285783921832537e-05,
	"loss": 0.9316556453704834,
	"mean_token_accuracy": 0.7457900270819664,
	"num_tokens": 909035.0,
	"step": 62
	},
	{
	"entropy": 1.2865931391716003,
	"epoch": 0.37527922561429633,
	"grad_norm": 0.365234375,
	"learning_rate": 1.926246956610309e-05,
	"loss": 0.8276340961456299,
	"mean_token_accuracy": 0.7709004059433937,
	"num_tokens": 923699.0,
	"step": 63
	},
	{
	"entropy": 1.2839124202728271,
	"epoch": 0.3812360387192852,
	"grad_norm": 0.357421875,
	"learning_rate": 1.9238795325112867e-05,
	"loss": 0.792696475982666,
	"mean_token_accuracy": 0.7811732813715935,
	"num_tokens": 937978.0,
	"step": 64
	},
	{
	"entropy": 1.2945790588855743,
	"epoch": 0.387192851824274,
	"grad_norm": 0.359375,
	"learning_rate": 1.921476211870408e-05,
	"loss": 0.874768853187561,
	"mean_token_accuracy": 0.7566314563155174,
	"num_tokens": 953189.0,
	"step": 65
	},
	{
	"entropy": 1.3046975582838058,
	"epoch": 0.39314966492926284,
	"grad_norm": 0.36328125,
	"learning_rate": 1.9190370880666206e-05,
	"loss": 0.8540111780166626,
	"mean_token_accuracy": 0.7612261921167374,
	"num_tokens": 967694.0,
	"step": 66
	},
	{
	"entropy": 1.3215627670288086,
	"epoch": 0.3991064780342517,
	"grad_norm": 0.384765625,
	"learning_rate": 1.9165622558699763e-05,
	"loss": 0.9076806306838989,
	"mean_token_accuracy": 0.7566501572728157,
	"num_tokens": 981909.0,
	"step": 67
	},
	{
	"entropy": 1.2821516543626785,
	"epoch": 0.4050632911392405,
	"grad_norm": 0.35546875,
	"learning_rate": 1.9140518114379433e-05,
	"loss": 0.8209891319274902,
	"mean_token_accuracy": 0.7717649862170219,
	"num_tokens": 996738.0,
	"step": 68
	},
	{
	"entropy": 1.3115236312150955,
	"epoch": 0.41102010424422936,
	"grad_norm": 0.341796875,
	"learning_rate": 1.9115058523116734e-05,
	"loss": 0.8256929516792297,
	"mean_token_accuracy": 0.7650889083743095,
	"num_tokens": 1011216.0,
	"step": 69
	},
	{
	"entropy": 1.3014082163572311,
	"epoch": 0.41697691734921816,
	"grad_norm": 0.33984375,
	"learning_rate": 1.908924477412211e-05,
	"loss": 0.8159216046333313,
	"mean_token_accuracy": 0.7695104703307152,
	"num_tokens": 1026284.0,
	"step": 70
	},
	{
	"entropy": 1.3315211981534958,
	"epoch": 0.422933730454207,
	"grad_norm": 0.349609375,
	"learning_rate": 1.9063077870366504e-05,
	"loss": 0.8810704350471497,
	"mean_token_accuracy": 0.7593883574008942,
	"num_tokens": 1041027.0,
	"step": 71
	},
	{
	"entropy": 1.3138697743415833,
	"epoch": 0.4288905435591958,
	"grad_norm": 0.330078125,
	"learning_rate": 1.903655882854237e-05,
	"loss": 0.8666151762008667,
	"mean_token_accuracy": 0.7636753916740417,
	"num_tokens": 1056203.0,
	"step": 72
	},
	{
	"entropy": 1.3296868354082108,
	"epoch": 0.43484735666418467,
	"grad_norm": 0.38671875,
	"learning_rate": 1.900968867902419e-05,
	"loss": 0.8226944804191589,
	"mean_token_accuracy": 0.7660787329077721,
	"num_tokens": 1069994.0,
	"step": 73
	},
	{
	"entropy": 1.327721655368805,
	"epoch": 0.44080416976917347,
	"grad_norm": 0.384765625,
	"learning_rate": 1.898246846582844e-05,
	"loss": 0.8633409738540649,
	"mean_token_accuracy": 0.7588323205709457,
	"num_tokens": 1084280.0,
	"step": 74
	},
	{
	"entropy": 1.2736104279756546,
	"epoch": 0.4467609828741623,
	"grad_norm": 0.345703125,
	"learning_rate": 1.895489924657301e-05,
	"loss": 0.7951964139938354,
	"mean_token_accuracy": 0.7775014489889145,
	"num_tokens": 1099332.0,
	"step": 75
	},
	{
	"entropy": 1.2568956762552261,
	"epoch": 0.4527177959791511,
	"grad_norm": 0.390625,
	"learning_rate": 1.8926982092436117e-05,
	"loss": 0.7958716750144958,
	"mean_token_accuracy": 0.7729126885533333,
	"num_tokens": 1113758.0,
	"step": 76
	},
	{
	"entropy": 1.2585568279027939,
	"epoch": 0.45867460908414,
	"grad_norm": 0.353515625,
	"learning_rate": 1.8898718088114688e-05,
	"loss": 0.8418722152709961,
	"mean_token_accuracy": 0.7698204517364502,
	"num_tokens": 1128814.0,
	"step": 77
	},
	{
	"entropy": 1.2552629113197327,
	"epoch": 0.46463142218912884,
	"grad_norm": 0.353515625,
	"learning_rate": 1.887010833178222e-05,
	"loss": 0.7894448041915894,
	"mean_token_accuracy": 0.7702115252614021,
	"num_tokens": 1143044.0,
	"step": 78
	},
	{
	"entropy": 1.237639144062996,
	"epoch": 0.47058823529411764,
	"grad_norm": 0.357421875,
	"learning_rate": 1.8841153935046098e-05,
	"loss": 0.7948979735374451,
	"mean_token_accuracy": 0.7740144208073616,
	"num_tokens": 1158074.0,
	"step": 79
	},
	{
	"entropy": 1.2586966007947922,
	"epoch": 0.4765450483991065,
	"grad_norm": 0.375,
	"learning_rate": 1.8811856022904423e-05,
	"loss": 0.8225394487380981,
	"mean_token_accuracy": 0.7743681222200394,
	"num_tokens": 1172247.0,
	"step": 80
	},
	{
	"entropy": 1.2005833238363266,
	"epoch": 0.4825018615040953,
	"grad_norm": 0.341796875,
	"learning_rate": 1.8782215733702286e-05,
	"loss": 0.7371789216995239,
	"mean_token_accuracy": 0.7894127815961838,
	"num_tokens": 1187054.0,
	"step": 81
	},
	{
	"entropy": 1.2627707123756409,
	"epoch": 0.48845867460908415,
	"grad_norm": 0.365234375,
	"learning_rate": 1.8752234219087538e-05,
	"loss": 0.8591660857200623,
	"mean_token_accuracy": 0.7659735381603241,
	"num_tokens": 1201655.0,
	"step": 82
	},
	{
	"entropy": 1.2909194976091385,
	"epoch": 0.49441548771407295,
	"grad_norm": 0.376953125,
	"learning_rate": 1.8721912643966055e-05,
	"loss": 0.8661763072013855,
	"mean_token_accuracy": 0.7588073536753654,
	"num_tokens": 1215941.0,
	"step": 83
	},
	{
	"entropy": 1.2870348989963531,
	"epoch": 0.5003723008190618,
	"grad_norm": 0.359375,
	"learning_rate": 1.8691252186456465e-05,
	"loss": 0.830467939376831,
	"mean_token_accuracy": 0.7688242942094803,
	"num_tokens": 1230066.0,
	"step": 84
	},
	{
	"entropy": 1.2635858803987503,
	"epoch": 0.5063291139240507,
	"grad_norm": 0.34765625,
	"learning_rate": 1.866025403784439e-05,
	"loss": 0.8112859725952148,
	"mean_token_accuracy": 0.7699639350175858,
	"num_tokens": 1245411.0,
	"step": 85
	},
	{
	"entropy": 1.2944375872612,
	"epoch": 0.5122859270290394,
	"grad_norm": 0.330078125,
	"learning_rate": 1.862891940253613e-05,
	"loss": 0.8428732752799988,
	"mean_token_accuracy": 0.7625590562820435,
	"num_tokens": 1260081.0,
	"step": 86
	},
	{
	"entropy": 1.2768390029668808,
	"epoch": 0.5182427401340283,
	"grad_norm": 0.333984375,
	"learning_rate": 1.8597249498011906e-05,
	"loss": 0.7818429470062256,
	"mean_token_accuracy": 0.7796064242720604,
	"num_tokens": 1274169.0,
	"step": 87
	},
	{
	"entropy": 1.2663612216711044,
	"epoch": 0.5241995532390171,
	"grad_norm": 0.34765625,
	"learning_rate": 1.8565245554778516e-05,
	"loss": 0.7885245084762573,
	"mean_token_accuracy": 0.7747808918356895,
	"num_tokens": 1288810.0,
	"step": 88
	},
	{
	"entropy": 1.299068808555603,
	"epoch": 0.530156366344006,
	"grad_norm": 0.36328125,
	"learning_rate": 1.8532908816321557e-05,
	"loss": 0.8310455083847046,
	"mean_token_accuracy": 0.7649297416210175,
	"num_tokens": 1303352.0,
	"step": 89
	},
	{
	"entropy": 1.2907497137784958,
	"epoch": 0.5361131794489948,
	"grad_norm": 0.359375,
	"learning_rate": 1.8500240539057093e-05,
	"loss": 0.8416677713394165,
	"mean_token_accuracy": 0.7609433010220528,
	"num_tokens": 1317948.0,
	"step": 90
	},
	{
	"entropy": 1.3166047036647797,
	"epoch": 0.5420699925539836,
	"grad_norm": 0.35546875,
	"learning_rate": 1.8467241992282842e-05,
	"loss": 0.8960973024368286,
	"mean_token_accuracy": 0.7464632913470268,
	"num_tokens": 1332383.0,
	"step": 91
	},
	{
	"entropy": 1.2651972472667694,
	"epoch": 0.5480268056589724,
	"grad_norm": 0.341796875,
	"learning_rate": 1.843391445812886e-05,
	"loss": 0.7947843074798584,
	"mean_token_accuracy": 0.7730053663253784,
	"num_tokens": 1346797.0,
	"step": 92
	},
	{
	"entropy": 1.2707823663949966,
	"epoch": 0.5539836187639613,
	"grad_norm": 0.359375,
	"learning_rate": 1.8400259231507716e-05,
	"loss": 0.7787963151931763,
	"mean_token_accuracy": 0.7789562344551086,
	"num_tokens": 1360935.0,
	"step": 93
	},
	{
	"entropy": 1.2486981898546219,
	"epoch": 0.5599404318689502,
	"grad_norm": 0.326171875,
	"learning_rate": 1.83662776200642e-05,
	"loss": 0.7839233875274658,
	"mean_token_accuracy": 0.7764812558889389,
	"num_tokens": 1375392.0,
	"step": 94
	},
	{
	"entropy": 1.2385842353105545,
	"epoch": 0.5658972449739389,
	"grad_norm": 0.3515625,
	"learning_rate": 1.833197094412449e-05,
	"loss": 0.7750946283340454,
	"mean_token_accuracy": 0.7754370123147964,
	"num_tokens": 1389642.0,
	"step": 95
	},
	{
	"entropy": 1.2582454234361649,
	"epoch": 0.5718540580789278,
	"grad_norm": 0.349609375,
	"learning_rate": 1.8297340536644877e-05,
	"loss": 0.8087575435638428,
	"mean_token_accuracy": 0.7721517831087112,
	"num_tokens": 1404072.0,
	"step": 96
	},
	{
	"entropy": 1.2536737322807312,
	"epoch": 0.5778108711839166,
	"grad_norm": 0.337890625,
	"learning_rate": 1.826238774315995e-05,
	"loss": 0.7741163372993469,
	"mean_token_accuracy": 0.7778245881199837,
	"num_tokens": 1418644.0,
	"step": 97
	},
	{
	"entropy": 1.2571983486413956,
	"epoch": 0.5837676842889055,
	"grad_norm": 0.357421875,
	"learning_rate": 1.8227113921730336e-05,
	"loss": 0.7704818844795227,
	"mean_token_accuracy": 0.7764224410057068,
	"num_tokens": 1432848.0,
	"step": 98
	},
	{
	"entropy": 1.2861786782741547,
	"epoch": 0.5897244973938943,
	"grad_norm": 0.404296875,
	"learning_rate": 1.819152044288992e-05,
	"loss": 0.840486466884613,
	"mean_token_accuracy": 0.7590076252818108,
	"num_tokens": 1446026.0,
	"step": 99
	},
	{
	"entropy": 1.259141594171524,
	"epoch": 0.5956813104988831,
	"grad_norm": 0.365234375,
	"learning_rate": 1.8155608689592604e-05,
	"loss": 0.8282920718193054,
	"mean_token_accuracy": 0.7636805027723312,
	"num_tokens": 1461039.0,
	"step": 100
	},
	{
	"entropy": 1.259206235408783,
	"epoch": 0.6016381236038719,
	"grad_norm": 0.345703125,
	"learning_rate": 1.811938005715857e-05,
	"loss": 0.7743256092071533,
	"mean_token_accuracy": 0.779683418571949,
	"num_tokens": 1475526.0,
	"step": 101
	},
	{
	"entropy": 1.2759947627782822,
	"epoch": 0.6075949367088608,
	"grad_norm": 0.34765625,
	"learning_rate": 1.8082835953220055e-05,
	"loss": 0.8088581562042236,
	"mean_token_accuracy": 0.7721528932452202,
	"num_tokens": 1489823.0,
	"step": 102
	},
	{
	"entropy": 1.293590635061264,
	"epoch": 0.6135517498138496,
	"grad_norm": 0.412109375,
	"learning_rate": 1.8045977797666685e-05,
	"loss": 0.8578399419784546,
	"mean_token_accuracy": 0.7592629492282867,
	"num_tokens": 1504383.0,
	"step": 103
	},
	{
	"entropy": 1.333486557006836,
	"epoch": 0.6195085629188384,
	"grad_norm": 0.396484375,
	"learning_rate": 1.8008807022590283e-05,
	"loss": 0.8810866475105286,
	"mean_token_accuracy": 0.7484598457813263,
	"num_tokens": 1517806.0,
	"step": 104
	},
	{
	"entropy": 1.2660276144742966,
	"epoch": 0.6254653760238272,
	"grad_norm": 0.357421875,
	"learning_rate": 1.7971325072229227e-05,
	"loss": 0.84173184633255,
	"mean_token_accuracy": 0.7654049769043922,
	"num_tokens": 1532497.0,
	"step": 105
	},
	{
	"entropy": 1.2698614448308945,
	"epoch": 0.6314221891288161,
	"grad_norm": 0.345703125,
	"learning_rate": 1.7933533402912354e-05,
	"loss": 0.8295232653617859,
	"mean_token_accuracy": 0.7681133523583412,
	"num_tokens": 1547634.0,
	"step": 106
	},
	{
	"entropy": 1.2812798023223877,
	"epoch": 0.637379002233805,
	"grad_norm": 0.33984375,
	"learning_rate": 1.7895433483002356e-05,
	"loss": 0.8399749398231506,
	"mean_token_accuracy": 0.7694036960601807,
	"num_tokens": 1562026.0,
	"step": 107
	},
	{
	"entropy": 1.3091949224472046,
	"epoch": 0.6433358153387937,
	"grad_norm": 0.404296875,
	"learning_rate": 1.785702679283874e-05,
	"loss": 0.8679311871528625,
	"mean_token_accuracy": 0.7622929587960243,
	"num_tokens": 1575785.0,
	"step": 108
	},
	{
	"entropy": 1.2573847621679306,
	"epoch": 0.6492926284437825,
	"grad_norm": 0.345703125,
	"learning_rate": 1.78183148246803e-05,
	"loss": 0.8026180863380432,
	"mean_token_accuracy": 0.7683332338929176,
	"num_tokens": 1591294.0,
	"step": 109
	},
	{
	"entropy": 1.2880352437496185,
	"epoch": 0.6552494415487714,
	"grad_norm": 0.361328125,
	"learning_rate": 1.777929908264715e-05,
	"loss": 0.8197344541549683,
	"mean_token_accuracy": 0.7702998444437981,
	"num_tokens": 1604958.0,
	"step": 110
	},
	{
	"entropy": 1.2673484981060028,
	"epoch": 0.6612062546537603,
	"grad_norm": 0.36328125,
	"learning_rate": 1.7739981082662275e-05,
	"loss": 0.8381748199462891,
	"mean_token_accuracy": 0.7616433724761009,
	"num_tokens": 1620204.0,
	"step": 111
	},
	{
	"entropy": 1.2442313134670258,
	"epoch": 0.6671630677587491,
	"grad_norm": 0.337890625,
	"learning_rate": 1.7700362352392632e-05,
	"loss": 0.7782060503959656,
	"mean_token_accuracy": 0.7786413952708244,
	"num_tokens": 1635257.0,
	"step": 112
	},
	{
	"entropy": 1.2735689878463745,
	"epoch": 0.6731198808637379,
	"grad_norm": 0.392578125,
	"learning_rate": 1.766044443118978e-05,
	"loss": 0.8409761786460876,
	"mean_token_accuracy": 0.7633680775761604,
	"num_tokens": 1649668.0,
	"step": 113
	},
	{
	"entropy": 1.280581310391426,
	"epoch": 0.6790766939687267,
	"grad_norm": 0.35546875,
	"learning_rate": 1.762022887003011e-05,
	"loss": 0.819488525390625,
	"mean_token_accuracy": 0.768619529902935,
	"num_tokens": 1663868.0,
	"step": 114
	},
	{
	"entropy": 1.2455193400382996,
	"epoch": 0.6850335070737156,
	"grad_norm": 0.361328125,
	"learning_rate": 1.757971723145453e-05,
	"loss": 0.7443034648895264,
	"mean_token_accuracy": 0.785475604236126,
	"num_tokens": 1678332.0,
	"step": 115
	},
	{
	"entropy": 1.2718180269002914,
	"epoch": 0.6909903201787044,
	"grad_norm": 0.349609375,
	"learning_rate": 1.75389110895078e-05,
	"loss": 0.8380476832389832,
	"mean_token_accuracy": 0.7637294083833694,
	"num_tokens": 1692798.0,
	"step": 116
	},
	{
	"entropy": 1.28805373609066,
	"epoch": 0.6969471332836932,
	"grad_norm": 0.373046875,
	"learning_rate": 1.7497812029677344e-05,
	"loss": 0.7940935492515564,
	"mean_token_accuracy": 0.7718535214662552,
	"num_tokens": 1706403.0,
	"step": 117
	},
	{
	"entropy": 1.2754413783550262,
	"epoch": 0.702903946388682,
	"grad_norm": 0.359375,
	"learning_rate": 1.7456421648831658e-05,
	"loss": 0.79500412940979,
	"mean_token_accuracy": 0.7691502496600151,
	"num_tokens": 1720637.0,
	"step": 118
	},
	{
	"entropy": 1.2628609538078308,
	"epoch": 0.7088607594936709,
	"grad_norm": 0.34375,
	"learning_rate": 1.741474155515827e-05,
	"loss": 0.8258920907974243,
	"mean_token_accuracy": 0.7655653059482574,
	"num_tokens": 1735839.0,
	"step": 119
	},
	{
	"entropy": 1.2739417552947998,
	"epoch": 0.7148175725986597,
	"grad_norm": 0.369140625,
	"learning_rate": 1.737277336810124e-05,
	"loss": 0.7685380578041077,
	"mean_token_accuracy": 0.7830494046211243,
	"num_tokens": 1750093.0,
	"step": 120
	},
	{
	"entropy": 1.2557980716228485,
	"epoch": 0.7207743857036486,
	"grad_norm": 0.36328125,
	"learning_rate": 1.7330518718298263e-05,
	"loss": 0.8203400373458862,
	"mean_token_accuracy": 0.7682551443576813,
	"num_tokens": 1765040.0,
	"step": 121
	},
	{
	"entropy": 1.2622184306383133,
	"epoch": 0.7267311988086373,
	"grad_norm": 0.359375,
	"learning_rate": 1.7287979247517285e-05,
	"loss": 0.8228541612625122,
	"mean_token_accuracy": 0.7695576474070549,
	"num_tokens": 1779610.0,
	"step": 122
	},
	{
	"entropy": 1.2455091774463654,
	"epoch": 0.7326880119136262,
	"grad_norm": 0.353515625,
	"learning_rate": 1.7245156608592727e-05,
	"loss": 0.754838228225708,
	"mean_token_accuracy": 0.7866516560316086,
	"num_tokens": 1794351.0,
	"step": 123
	},
	{
	"entropy": 1.2472401857376099,
	"epoch": 0.7386448250186151,
	"grad_norm": 0.361328125,
	"learning_rate": 1.7202052465361268e-05,
	"loss": 0.8274670243263245,
	"mean_token_accuracy": 0.7684604525566101,
	"num_tokens": 1809510.0,
	"step": 124
	},
	{
	"entropy": 1.2943197786808014,
	"epoch": 0.7446016381236039,
	"grad_norm": 0.373046875,
	"learning_rate": 1.7158668492597186e-05,
	"loss": 0.858670711517334,
	"mean_token_accuracy": 0.7580647841095924,
	"num_tokens": 1823089.0,
	"step": 125
	},
	{
	"entropy": 1.2716920524835587,
	"epoch": 0.7505584512285927,
	"grad_norm": 0.369140625,
	"learning_rate": 1.7115006375947304e-05,
	"loss": 0.8383840918540955,
	"mean_token_accuracy": 0.7634381875395775,
	"num_tokens": 1837660.0,
	"step": 126
	},
	{
	"entropy": 1.2710201442241669,
	"epoch": 0.7565152643335815,
	"grad_norm": 0.3671875,
	"learning_rate": 1.7071067811865477e-05,
	"loss": 0.8363803625106812,
	"mean_token_accuracy": 0.7592340558767319,
	"num_tokens": 1851869.0,
	"step": 127
	},
	{
	"entropy": 1.2555763572454453,
	"epoch": 0.7624720774385704,
	"grad_norm": 0.36328125,
	"learning_rate": 1.7026854507546694e-05,
	"loss": 0.7854242324829102,
	"mean_token_accuracy": 0.7739197090268135,
	"num_tokens": 1866454.0,
	"step": 128
	},
	{
	"entropy": 1.2632461488246918,
	"epoch": 0.7684288905435592,
	"grad_norm": 0.349609375,
	"learning_rate": 1.698236818086073e-05,
	"loss": 0.8233469724655151,
	"mean_token_accuracy": 0.7627609372138977,
	"num_tokens": 1880761.0,
	"step": 129
	},
	{
	"entropy": 1.252721443772316,
	"epoch": 0.774385703648548,
	"grad_norm": 0.34765625,
	"learning_rate": 1.693761056028542e-05,
	"loss": 0.8479862213134766,
	"mean_token_accuracy": 0.7652608901262283,
	"num_tokens": 1895812.0,
	"step": 130
	},
	{
	"entropy": 1.2346957474946976,
	"epoch": 0.7803425167535368,
	"grad_norm": 0.33203125,
	"learning_rate": 1.689258338483947e-05,
	"loss": 0.7700930237770081,
	"mean_token_accuracy": 0.7812119647860527,
	"num_tokens": 1910875.0,
	"step": 131
	},
	{
	"entropy": 1.2443837672472,
	"epoch": 0.7862993298585257,
	"grad_norm": 0.33984375,
	"learning_rate": 1.6847288404014937e-05,
	"loss": 0.7806112766265869,
	"mean_token_accuracy": 0.7737433910369873,
	"num_tokens": 1925210.0,
	"step": 132
	},
	{
	"entropy": 1.2574907839298248,
	"epoch": 0.7922561429635145,
	"grad_norm": 0.349609375,
	"learning_rate": 1.6801727377709195e-05,
	"loss": 0.7916486263275146,
	"mean_token_accuracy": 0.7723786979913712,
	"num_tokens": 1939706.0,
	"step": 133
	},
	{
	"entropy": 1.240403950214386,
	"epoch": 0.7982129560685034,
	"grad_norm": 0.361328125,
	"learning_rate": 1.6755902076156606e-05,
	"loss": 0.7764307260513306,
	"mean_token_accuracy": 0.7722450792789459,
	"num_tokens": 1953951.0,
	"step": 134
	},
	{
	"entropy": 1.2047400623559952,
	"epoch": 0.8041697691734921,
	"grad_norm": 0.341796875,
	"learning_rate": 1.67098142798597e-05,
	"loss": 0.7488443851470947,
	"mean_token_accuracy": 0.7864288538694382,
	"num_tokens": 1968478.0,
	"step": 135
	},
	{
	"entropy": 1.2280729860067368,
	"epoch": 0.810126582278481,
	"grad_norm": 0.35546875,
	"learning_rate": 1.6663465779520042e-05,
	"loss": 0.7793049216270447,
	"mean_token_accuracy": 0.7834365740418434,
	"num_tokens": 1982972.0,
	"step": 136
	},
	{
	"entropy": 1.2266123741865158,
	"epoch": 0.8160833953834699,
	"grad_norm": 0.34765625,
	"learning_rate": 1.6616858375968596e-05,
	"loss": 0.7857159376144409,
	"mean_token_accuracy": 0.7744308784604073,
	"num_tokens": 1997617.0,
	"step": 137
	},
	{
	"entropy": 1.229455679655075,
	"epoch": 0.8220402084884587,
	"grad_norm": 0.36328125,
	"learning_rate": 1.6569993880095807e-05,
	"loss": 0.8067857623100281,
	"mean_token_accuracy": 0.7750186920166016,
	"num_tokens": 2011907.0,
	"step": 138
	},
	{
	"entropy": 1.234218791127205,
	"epoch": 0.8279970215934475,
	"grad_norm": 0.35546875,
	"learning_rate": 1.6522874112781213e-05,
	"loss": 0.8134928941726685,
	"mean_token_accuracy": 0.7728847786784172,
	"num_tokens": 2026748.0,
	"step": 139
	},
	{
	"entropy": 1.2340240478515625,
	"epoch": 0.8339538346984363,
	"grad_norm": 0.345703125,
	"learning_rate": 1.6475500904822707e-05,
	"loss": 0.7837961912155151,
	"mean_token_accuracy": 0.7767920717597008,
	"num_tokens": 2041028.0,
	"step": 140
	},
	{
	"entropy": 1.2047637104988098,
	"epoch": 0.8399106478034252,
	"grad_norm": 0.337890625,
	"learning_rate": 1.6427876096865394e-05,
	"loss": 0.7514673471450806,
	"mean_token_accuracy": 0.7840846851468086,
	"num_tokens": 2056459.0,
	"step": 141
	},
	{
	"entropy": 1.2402971684932709,
	"epoch": 0.845867460908414,
	"grad_norm": 0.341796875,
	"learning_rate": 1.6380001539330088e-05,
	"loss": 0.8099113702774048,
	"mean_token_accuracy": 0.7708609700202942,
	"num_tokens": 2071572.0,
	"step": 142
	},
	{
	"entropy": 1.2505415827035904,
	"epoch": 0.8518242740134029,
	"grad_norm": 0.341796875,
	"learning_rate": 1.6331879092341402e-05,
	"loss": 0.8279533386230469,
	"mean_token_accuracy": 0.7612129226326942,
	"num_tokens": 2086270.0,
	"step": 143
	},
	{
	"entropy": 1.23782616853714,
	"epoch": 0.8577810871183916,
	"grad_norm": 0.33984375,
	"learning_rate": 1.6283510625655474e-05,
	"loss": 0.7820447683334351,
	"mean_token_accuracy": 0.7787378579378128,
	"num_tokens": 2100982.0,
	"step": 144
	},
	{
	"entropy": 1.2248000502586365,
	"epoch": 0.8637379002233805,
	"grad_norm": 0.318359375,
	"learning_rate": 1.6234898018587336e-05,
	"loss": 0.7593281269073486,
	"mean_token_accuracy": 0.778163269162178,
	"num_tokens": 2115803.0,
	"step": 145
	},
	{
	"entropy": 1.2075249254703522,
	"epoch": 0.8696947133283693,
	"grad_norm": 0.31640625,
	"learning_rate": 1.6186043159937884e-05,
	"loss": 0.7119603753089905,
	"mean_token_accuracy": 0.7936617583036423,
	"num_tokens": 2130971.0,
	"step": 146
	},
	{
	"entropy": 1.2165881991386414,
	"epoch": 0.8756515264333582,
	"grad_norm": 0.322265625,
	"learning_rate": 1.6136947947920477e-05,
	"loss": 0.740675687789917,
	"mean_token_accuracy": 0.7878302410244942,
	"num_tokens": 2145552.0,
	"step": 147
	},
	{
	"entropy": 1.2430049926042557,
	"epoch": 0.8816083395383469,
	"grad_norm": 0.361328125,
	"learning_rate": 1.608761429008721e-05,
	"loss": 0.8482354283332825,
	"mean_token_accuracy": 0.7628344818949699,
	"num_tokens": 2160069.0,
	"step": 148
	},
	{
	"entropy": 1.183465838432312,
	"epoch": 0.8875651526433358,
	"grad_norm": 0.34375,
	"learning_rate": 1.6038044103254775e-05,
	"loss": 0.7142577171325684,
	"mean_token_accuracy": 0.7939940765500069,
	"num_tokens": 2175381.0,
	"step": 149
	},
	{
	"entropy": 1.2008293867111206,
	"epoch": 0.8935219657483247,
	"grad_norm": 0.345703125,
	"learning_rate": 1.5988239313430004e-05,
	"loss": 0.760569155216217,
	"mean_token_accuracy": 0.7821842804551125,
	"num_tokens": 2190275.0,
	"step": 150
	},
	{
	"entropy": 1.2101170867681503,
	"epoch": 0.8994787788533135,
	"grad_norm": 0.337890625,
	"learning_rate": 1.5938201855735017e-05,
	"loss": 0.7829165458679199,
	"mean_token_accuracy": 0.7733455300331116,
	"num_tokens": 2204775.0,
	"step": 151
	},
	{
	"entropy": 1.232336938381195,
	"epoch": 0.9054355919583023,
	"grad_norm": 0.357421875,
	"learning_rate": 1.5887933674332048e-05,
	"loss": 0.8415957689285278,
	"mean_token_accuracy": 0.7633619755506516,
	"num_tokens": 2219016.0,
	"step": 152
	},
	{
	"entropy": 1.1960958242416382,
	"epoch": 0.9113924050632911,
	"grad_norm": 0.328125,
	"learning_rate": 1.5837436722347902e-05,
	"loss": 0.7200090885162354,
	"mean_token_accuracy": 0.7941614985466003,
	"num_tokens": 2233377.0,
	"step": 153
	},
	{
	"entropy": 1.1901395469903946,
	"epoch": 0.91734921816828,
	"grad_norm": 0.337890625,
	"learning_rate": 1.578671296179806e-05,
	"loss": 0.7320102453231812,
	"mean_token_accuracy": 0.7907885834574699,
	"num_tokens": 2248523.0,
	"step": 154
	},
	{
	"entropy": 1.205485075712204,
	"epoch": 0.9233060312732688,
	"grad_norm": 0.3359375,
	"learning_rate": 1.573576436351046e-05,
	"loss": 0.7730504870414734,
	"mean_token_accuracy": 0.7801787778735161,
	"num_tokens": 2263288.0,
	"step": 155
	},
	{
	"entropy": 1.2493800967931747,
	"epoch": 0.9292628443782577,
	"grad_norm": 0.3359375,
	"learning_rate": 1.5684592907048925e-05,
	"loss": 0.792695164680481,
	"mean_token_accuracy": 0.7768412679433823,
	"num_tokens": 2278085.0,
	"step": 156
	},
	{
	"entropy": 1.222463384270668,
	"epoch": 0.9352196574832464,
	"grad_norm": 0.341796875,
	"learning_rate": 1.563320058063622e-05,
	"loss": 0.7253803610801697,
	"mean_token_accuracy": 0.7881602421402931,
	"num_tokens": 2292673.0,
	"step": 157
	},
	{
	"entropy": 1.2182011902332306,
	"epoch": 0.9411764705882353,
	"grad_norm": 0.31640625,
	"learning_rate": 1.5581589381076843e-05,
	"loss": 0.7509200572967529,
	"mean_token_accuracy": 0.7809579819440842,
	"num_tokens": 2307646.0,
	"step": 158
	},
	{
	"entropy": 1.2366309016942978,
	"epoch": 0.9471332836932241,
	"grad_norm": 0.337890625,
	"learning_rate": 1.5529761313679396e-05,
	"loss": 0.7257631421089172,
	"mean_token_accuracy": 0.7827246561646461,
	"num_tokens": 2321559.0,
	"step": 159
	},
	{
	"entropy": 1.2403675764799118,
	"epoch": 0.953090096798213,
	"grad_norm": 0.337890625,
	"learning_rate": 1.5477718392178716e-05,
	"loss": 0.8105670809745789,
	"mean_token_accuracy": 0.7707022428512573,
	"num_tokens": 2336203.0,
	"step": 160
	},
	{
	"entropy": 1.2477657943964005,
	"epoch": 0.9590469099032017,
	"grad_norm": 0.36328125,
	"learning_rate": 1.5425462638657597e-05,
	"loss": 0.9120197296142578,
	"mean_token_accuracy": 0.7492518648505211,
	"num_tokens": 2351344.0,
	"step": 161
	},
	{
	"entropy": 1.1972443461418152,
	"epoch": 0.9650037230081906,
	"grad_norm": 0.32421875,
	"learning_rate": 1.5372996083468242e-05,
	"loss": 0.7211199402809143,
	"mean_token_accuracy": 0.7921843752264977,
	"num_tokens": 2366108.0,
	"step": 162
	},
	{
	"entropy": 1.1903489977121353,
	"epoch": 0.9709605361131795,
	"grad_norm": 0.337890625,
	"learning_rate": 1.5320320765153367e-05,
	"loss": 0.7322642207145691,
	"mean_token_accuracy": 0.7919035404920578,
	"num_tokens": 2381068.0,
	"step": 163
	},
	{
	"entropy": 1.206245943903923,
	"epoch": 0.9769173492181683,
	"grad_norm": 0.34375,
	"learning_rate": 1.526743873036701e-05,
	"loss": 0.7425603270530701,
	"mean_token_accuracy": 0.7824821397662163,
	"num_tokens": 2395295.0,
	"step": 164
	},
	{
	"entropy": 1.2014246433973312,
	"epoch": 0.9828741623231572,
	"grad_norm": 0.34375,
	"learning_rate": 1.5214352033794981e-05,
	"loss": 0.807823121547699,
	"mean_token_accuracy": 0.7702382802963257,
	"num_tokens": 2410456.0,
	"step": 165
	},
	{
	"entropy": 1.211999848484993,
	"epoch": 0.9888309754281459,
	"grad_norm": 0.369140625,
	"learning_rate": 1.5161062738075068e-05,
	"loss": 0.7507081627845764,
	"mean_token_accuracy": 0.7832650169730186,
	"num_tokens": 2424439.0,
	"step": 166
	},
	{
	"entropy": 1.2110302299261093,
	"epoch": 0.9947877885331348,
	"grad_norm": 0.36328125,
	"learning_rate": 1.5107572913716859e-05,
	"loss": 0.787371814250946,
	"mean_token_accuracy": 0.7756073325872421,
	"num_tokens": 2438585.0,
	"step": 167
	},
	{
	"entropy": 1.2203376974378313,
	"epoch": 1.0,
	"grad_norm": 0.373046875,
	"learning_rate": 1.505388463902131e-05,
	"loss": 0.8217554688453674,
	"mean_token_accuracy": 0.7624766315732684,
	"num_tokens": 2452101.0,
	"step": 168
	},
	{
	"epoch": 1.0,
	"eval_entropy": 1.2118239323298137,
	"eval_loss": 0.775408923625946,
	"eval_mean_token_accuracy": 0.7765555028120676,
	"eval_model_preparation_time": 0.0053,
	"eval_num_tokens": 2452101.0,
	"eval_runtime": 26.4936,
	"eval_samples_per_second": 5.662,
	"eval_steps_per_second": 5.662,
	"step": 168
	}
	],
	"logging_steps": 1,
	"max_steps": 504,
	"num_input_tokens_seen": 0,
	"num_train_epochs": 3,
	"save_steps": 500,
	"stateful_callbacks": {
	"TrainerControl": {
	"args": {
	"should_epoch_stop": false,
	"should_evaluate": false,
	"should_log": false,
	"should_save": true,
	"should_training_stop": false
	},
	"attributes": {}
	}
	},
	"total_flos": 1.1041699975881523e+17,
	"train_batch_size": 1,
	"trial_name": null,
	"trial_params": null
	}