Instructions to use Ba2han/TR_CPTs with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ba2han/TR_CPTs with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ba2han/TR_CPTs")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ba2han/TR_CPTs")
model = AutoModelForCausalLM.from_pretrained("Ba2han/TR_CPTs", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ba2han/TR_CPTs with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ba2han/TR_CPTs"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/TR_CPTs",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Ba2han/TR_CPTs

SGLang

How to use Ba2han/TR_CPTs with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ba2han/TR_CPTs" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/TR_CPTs",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ba2han/TR_CPTs" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/TR_CPTs",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Unsloth Studio

How to use Ba2han/TR_CPTs with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/TR_CPTs to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/TR_CPTs to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ba2han/TR_CPTs to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Ba2han/TR_CPTs",
    max_seq_length=2048,
)

Docker Model Runner
How to use Ba2han/TR_CPTs with Docker Model Runner:
```
docker model run hf.co/Ba2han/TR_CPTs
```

Ba2han commited on 21 days ago

Commit

bf86ed8

verified ·

1 Parent(s): 576f086

Training in progress, step 9449, checkpoint

Browse files

Files changed (6) hide show

last-checkpoint/model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/rng_state_0.pth +1 -1
last-checkpoint/rng_state_1.pth +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +3316 -4

last-checkpoint/model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c33fdef23442e2a1770461a8cda13ce741f9d9628623da7ae6e8a6d6c83e602
 size 2934389864

 version https://git-lfs.github.com/spec/v1
+oid sha256:35000bdf3954efc7ee30f01b98df087db334b08f2a22f665dff40f61200ba098
 size 2934389864

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:42748a8a4b0c233c53e98d8966864e20e4edc7621b077ccdbea4445eb34c023d
 size 1471203730

 version https://git-lfs.github.com/spec/v1
+oid sha256:1c9adc61bf25a736879c0336bd1614a2e0230e7d5f4a82be7e49909c14716386
 size 1471203730

last-checkpoint/rng_state_0.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:27978b0c0a2d70a6d9e24fba0f2ed928bba9b47aecb6cc92523c33dd4a9bc29f
 size 14917

 version https://git-lfs.github.com/spec/v1
+oid sha256:26592b1cd842c4ce361f446ba95c6207828468b148db942589de49f1b302404b
 size 14917

last-checkpoint/rng_state_1.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0d4883ea9e9e4170ef036b08f27772e01a500be9d13e3c9492d90b6357d7e8c6
 size 14917

 version https://git-lfs.github.com/spec/v1
+oid sha256:3bf408af229538a1e55063d321681aee8925a0f675b9de64e81e09aaf34869fc
 size 14917

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d47aea60248cde54f4a23f0a1ac4a5ef06d1748f861a52f5adf7cd29ddbca51b
 size 9785

 version https://git-lfs.github.com/spec/v1
+oid sha256:fed6a8c23b7b6bde187eeb9cf51deffe8a602d7a16dd7199c7b51d919654c30f
 size 9785

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,9 +2,9 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 1.1702024220937512,
   "eval_steps": 1890,
-  "global_step": 8505,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -29804,6 +29804,3318 @@
       "learning_rate": 0.0005763021626561072,
       "loss": 2.4784698486328125,
       "step": 8504
     }
   ],
   "logging_steps": 2,
@@ -29818,12 +33130,12 @@
         "should_evaluate": false,
         "should_log": false,
         "should_save": true,
-        "should_training_stop": false
       },
       "attributes": {}
     }
   },
-  "total_flos": 3.183537145144515e+19,
   "train_batch_size": 4,
   "trial_name": null,
   "trial_params": null

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 1.3000901233520383,
   "eval_steps": 1890,
+  "global_step": 9449,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "learning_rate": 0.0005763021626561072,
       "loss": 2.4784698486328125,
       "step": 8504
+    },
+    {
+      "epoch": 1.1703400149976266,
+      "grad_norm": 0.050933148711919785,
+      "learning_rate": 0.0005750837648492233,
+      "loss": 2.4722843170166016,
+      "step": 8506
+    },
+    {
+      "epoch": 1.1706152008053772,
+      "grad_norm": 0.09202922880649567,
+      "learning_rate": 0.0005738653670423393,
+      "loss": 2.474773645401001,
+      "step": 8508
+    },
+    {
+      "epoch": 1.1708903866131277,
+      "grad_norm": 0.062253188341856,
+      "learning_rate": 0.0005726469692354554,
+      "loss": 2.4313125610351562,
+      "step": 8510
+    },
+    {
+      "epoch": 1.1711655724208783,
+      "grad_norm": 0.055153340101242065,
+      "learning_rate": 0.0005714285714285714,
+      "loss": 2.431378126144409,
+      "step": 8512
+    },
+    {
+      "epoch": 1.171440758228629,
+      "grad_norm": 0.042116113007068634,
+      "learning_rate": 0.0005702101736216875,
+      "loss": 2.451446533203125,
+      "step": 8514
+    },
+    {
+      "epoch": 1.1717159440363796,
+      "grad_norm": 0.02820545621216297,
+      "learning_rate": 0.0005689917758148036,
+      "loss": 2.5025627613067627,
+      "step": 8516
+    },
+    {
+      "epoch": 1.1719911298441301,
+      "grad_norm": 0.026804449036717415,
+      "learning_rate": 0.0005677733780079196,
+      "loss": 2.5057525634765625,
+      "step": 8518
+    },
+    {
+      "epoch": 1.1722663156518807,
+      "grad_norm": 0.04188213124871254,
+      "learning_rate": 0.0005665549802010357,
+      "loss": 2.4838624000549316,
+      "step": 8520
+    },
+    {
+      "epoch": 1.1725415014596314,
+      "grad_norm": 0.026573574170470238,
+      "learning_rate": 0.0005653365823941517,
+      "loss": 2.4746599197387695,
+      "step": 8522
+    },
+    {
+      "epoch": 1.172816687267382,
+      "grad_norm": 0.03222622349858284,
+      "learning_rate": 0.0005641181845872677,
+      "loss": 2.4761481285095215,
+      "step": 8524
+    },
+    {
+      "epoch": 1.1730918730751325,
+      "grad_norm": 0.05096002295613289,
+      "learning_rate": 0.0005628997867803837,
+      "loss": 2.4424214363098145,
+      "step": 8526
+    },
+    {
+      "epoch": 1.1733670588828833,
+      "grad_norm": 0.027245238423347473,
+      "learning_rate": 0.0005616813889734999,
+      "loss": 2.47282075881958,
+      "step": 8528
+    },
+    {
+      "epoch": 1.1736422446906338,
+      "grad_norm": 0.02836611308157444,
+      "learning_rate": 0.000560462991166616,
+      "loss": 2.4825472831726074,
+      "step": 8530
+    },
+    {
+      "epoch": 1.1739174304983844,
+      "grad_norm": 0.0289299339056015,
+      "learning_rate": 0.000559244593359732,
+      "loss": 2.4726157188415527,
+      "step": 8532
+    },
+    {
+      "epoch": 1.1741926163061351,
+      "grad_norm": 0.026455475017428398,
+      "learning_rate": 0.0005580261955528481,
+      "loss": 2.4694809913635254,
+      "step": 8534
+    },
+    {
+      "epoch": 1.1744678021138857,
+      "grad_norm": 0.053854621946811676,
+      "learning_rate": 0.000556807797745964,
+      "loss": 2.4754741191864014,
+      "step": 8536
+    },
+    {
+      "epoch": 1.1747429879216362,
+      "grad_norm": 0.057867348194122314,
+      "learning_rate": 0.0005555893999390801,
+      "loss": 2.463995933532715,
+      "step": 8538
+    },
+    {
+      "epoch": 1.1750181737293868,
+      "grad_norm": 0.049233559519052505,
+      "learning_rate": 0.0005543710021321961,
+      "loss": 2.439931631088257,
+      "step": 8540
+    },
+    {
+      "epoch": 1.1752933595371375,
+      "grad_norm": 0.04286424070596695,
+      "learning_rate": 0.0005531526043253122,
+      "loss": 2.488924264907837,
+      "step": 8542
+    },
+    {
+      "epoch": 1.175568545344888,
+      "grad_norm": 0.03908887133002281,
+      "learning_rate": 0.0005519342065184282,
+      "loss": 2.4679527282714844,
+      "step": 8544
+    },
+    {
+      "epoch": 1.1758437311526386,
+      "grad_norm": 0.026763249188661575,
+      "learning_rate": 0.0005507158087115444,
+      "loss": 2.4716696739196777,
+      "step": 8546
+    },
+    {
+      "epoch": 1.1761189169603892,
+      "grad_norm": 0.057844407856464386,
+      "learning_rate": 0.0005494974109046605,
+      "loss": 2.450366973876953,
+      "step": 8548
+    },
+    {
+      "epoch": 1.17639410276814,
+      "grad_norm": 0.046296827495098114,
+      "learning_rate": 0.0005482790130977764,
+      "loss": 2.506364345550537,
+      "step": 8550
+    },
+    {
+      "epoch": 1.1766692885758905,
+      "grad_norm": 0.030109576880931854,
+      "learning_rate": 0.0005470606152908925,
+      "loss": 2.397833824157715,
+      "step": 8552
+    },
+    {
+      "epoch": 1.176944474383641,
+      "grad_norm": 0.03319597616791725,
+      "learning_rate": 0.0005458422174840085,
+      "loss": 2.4731743335723877,
+      "step": 8554
+    },
+    {
+      "epoch": 1.1772196601913918,
+      "grad_norm": 0.06661998480558395,
+      "learning_rate": 0.0005446238196771246,
+      "loss": 2.4965131282806396,
+      "step": 8556
+    },
+    {
+      "epoch": 1.1774948459991423,
+      "grad_norm": 0.03787920996546745,
+      "learning_rate": 0.0005434054218702406,
+      "loss": 2.436227798461914,
+      "step": 8558
+    },
+    {
+      "epoch": 1.1777700318068929,
+      "grad_norm": 0.04407722130417824,
+      "learning_rate": 0.0005421870240633567,
+      "loss": 2.5293312072753906,
+      "step": 8560
+    },
+    {
+      "epoch": 1.1780452176146436,
+      "grad_norm": 0.09424177557229996,
+      "learning_rate": 0.0005409686262564727,
+      "loss": 2.5078659057617188,
+      "step": 8562
+    },
+    {
+      "epoch": 1.1783204034223942,
+      "grad_norm": 0.0566101036965847,
+      "learning_rate": 0.0005397502284495888,
+      "loss": 2.46651029586792,
+      "step": 8564
+    },
+    {
+      "epoch": 1.1785955892301447,
+      "grad_norm": 0.03349711745977402,
+      "learning_rate": 0.0005385318306427049,
+      "loss": 2.462888240814209,
+      "step": 8566
+    },
+    {
+      "epoch": 1.1788707750378955,
+      "grad_norm": 0.055710434913635254,
+      "learning_rate": 0.0005373134328358209,
+      "loss": 2.485239028930664,
+      "step": 8568
+    },
+    {
+      "epoch": 1.179145960845646,
+      "grad_norm": 0.047695938497781754,
+      "learning_rate": 0.000536095035028937,
+      "loss": 2.4568920135498047,
+      "step": 8570
+    },
+    {
+      "epoch": 1.1794211466533966,
+      "grad_norm": 0.04080001637339592,
+      "learning_rate": 0.000534876637222053,
+      "loss": 2.442403793334961,
+      "step": 8572
+    },
+    {
+      "epoch": 1.1796963324611471,
+      "grad_norm": 0.07023676484823227,
+      "learning_rate": 0.0005336582394151691,
+      "loss": 2.455559253692627,
+      "step": 8574
+    },
+    {
+      "epoch": 1.179971518268898,
+      "grad_norm": 0.10817483067512512,
+      "learning_rate": 0.000532439841608285,
+      "loss": 2.430891990661621,
+      "step": 8576
+    },
+    {
+      "epoch": 1.1802467040766484,
+      "grad_norm": 0.11201369762420654,
+      "learning_rate": 0.0005312214438014011,
+      "loss": 2.4346795082092285,
+      "step": 8578
+    },
+    {
+      "epoch": 1.180521889884399,
+      "grad_norm": 0.04322679340839386,
+      "learning_rate": 0.0005300030459945173,
+      "loss": 2.483079433441162,
+      "step": 8580
+    },
+    {
+      "epoch": 1.1807970756921495,
+      "grad_norm": 0.05259304121136665,
+      "learning_rate": 0.0005287846481876333,
+      "loss": 2.5023889541625977,
+      "step": 8582
+    },
+    {
+      "epoch": 1.1810722614999003,
+      "grad_norm": 0.07048050314188004,
+      "learning_rate": 0.0005275662503807494,
+      "loss": 2.497157573699951,
+      "step": 8584
+    },
+    {
+      "epoch": 1.1813474473076508,
+      "grad_norm": 0.08057713508605957,
+      "learning_rate": 0.0005263478525738654,
+      "loss": 2.4843716621398926,
+      "step": 8586
+    },
+    {
+      "epoch": 1.1816226331154014,
+      "grad_norm": 0.04945176839828491,
+      "learning_rate": 0.0005251294547669815,
+      "loss": 2.4751336574554443,
+      "step": 8588
+    },
+    {
+      "epoch": 1.1818978189231522,
+      "grad_norm": 0.03627081215381622,
+      "learning_rate": 0.0005239110569600974,
+      "loss": 2.4722142219543457,
+      "step": 8590
+    },
+    {
+      "epoch": 1.1821730047309027,
+      "grad_norm": 0.026835620403289795,
+      "learning_rate": 0.0005226926591532135,
+      "loss": 2.4823660850524902,
+      "step": 8592
+    },
+    {
+      "epoch": 1.1824481905386532,
+      "grad_norm": 0.028032803907990456,
+      "learning_rate": 0.0005214742613463296,
+      "loss": 2.440993070602417,
+      "step": 8594
+    },
+    {
+      "epoch": 1.182723376346404,
+      "grad_norm": 0.02541583962738514,
+      "learning_rate": 0.0005202558635394456,
+      "loss": 2.5193333625793457,
+      "step": 8596
+    },
+    {
+      "epoch": 1.1829985621541546,
+      "grad_norm": 0.02785034477710724,
+      "learning_rate": 0.0005190374657325618,
+      "loss": 2.490316390991211,
+      "step": 8598
+    },
+    {
+      "epoch": 1.183273747961905,
+      "grad_norm": 0.02643394097685814,
+      "learning_rate": 0.0005178190679256778,
+      "loss": 2.4868125915527344,
+      "step": 8600
+    },
+    {
+      "epoch": 1.1835489337696556,
+      "grad_norm": 0.028258899226784706,
+      "learning_rate": 0.0005166006701187938,
+      "loss": 2.4677414894104004,
+      "step": 8602
+    },
+    {
+      "epoch": 1.1838241195774064,
+      "grad_norm": 0.04276259243488312,
+      "learning_rate": 0.0005153822723119098,
+      "loss": 2.4856784343719482,
+      "step": 8604
+    },
+    {
+      "epoch": 1.184099305385157,
+      "grad_norm": 0.029236052185297012,
+      "learning_rate": 0.0005141638745050259,
+      "loss": 2.4903740882873535,
+      "step": 8606
+    },
+    {
+      "epoch": 1.1843744911929075,
+      "grad_norm": 0.04992251098155975,
+      "learning_rate": 0.000512945476698142,
+      "loss": 2.4691739082336426,
+      "step": 8608
+    },
+    {
+      "epoch": 1.184649677000658,
+      "grad_norm": 0.026921182870864868,
+      "learning_rate": 0.000511727078891258,
+      "loss": 2.5242867469787598,
+      "step": 8610
+    },
+    {
+      "epoch": 1.1849248628084088,
+      "grad_norm": 0.03775021433830261,
+      "learning_rate": 0.0005105086810843741,
+      "loss": 2.4671568870544434,
+      "step": 8612
+    },
+    {
+      "epoch": 1.1852000486161594,
+      "grad_norm": 0.05926031619310379,
+      "learning_rate": 0.0005092902832774901,
+      "loss": 2.4535930156707764,
+      "step": 8614
+    },
+    {
+      "epoch": 1.18547523442391,
+      "grad_norm": 0.05345671996474266,
+      "learning_rate": 0.0005080718854706061,
+      "loss": 2.47186279296875,
+      "step": 8616
+    },
+    {
+      "epoch": 1.1857504202316607,
+      "grad_norm": 0.07402487844228745,
+      "learning_rate": 0.0005068534876637222,
+      "loss": 2.475618839263916,
+      "step": 8618
+    },
+    {
+      "epoch": 1.1860256060394112,
+      "grad_norm": 0.031279727816581726,
+      "learning_rate": 0.0005056350898568383,
+      "loss": 2.4523935317993164,
+      "step": 8620
+    },
+    {
+      "epoch": 1.1863007918471618,
+      "grad_norm": 0.029318010434508324,
+      "learning_rate": 0.0005044166920499543,
+      "loss": 2.4826366901397705,
+      "step": 8622
+    },
+    {
+      "epoch": 1.1865759776549125,
+      "grad_norm": 0.026097308844327927,
+      "learning_rate": 0.0005031982942430704,
+      "loss": 2.4601247310638428,
+      "step": 8624
+    },
+    {
+      "epoch": 1.186851163462663,
+      "grad_norm": 0.030194276943802834,
+      "learning_rate": 0.0005019798964361865,
+      "loss": 2.4754672050476074,
+      "step": 8626
+    },
+    {
+      "epoch": 1.1871263492704136,
+      "grad_norm": 0.03776143863797188,
+      "learning_rate": 0.0005007614986293024,
+      "loss": 2.4421730041503906,
+      "step": 8628
+    },
+    {
+      "epoch": 1.1874015350781641,
+      "grad_norm": 0.0535174161195755,
+      "learning_rate": 0.0004995431008224185,
+      "loss": 2.4875621795654297,
+      "step": 8630
+    },
+    {
+      "epoch": 1.187676720885915,
+      "grad_norm": 0.02867351658642292,
+      "learning_rate": 0.0004983247030155345,
+      "loss": 2.4918253421783447,
+      "step": 8632
+    },
+    {
+      "epoch": 1.1879519066936655,
+      "grad_norm": 0.025506816804409027,
+      "learning_rate": 0.0004971063052086507,
+      "loss": 2.4863393306732178,
+      "step": 8634
+    },
+    {
+      "epoch": 1.188227092501416,
+      "grad_norm": 0.04484231397509575,
+      "learning_rate": 0.0004958879074017667,
+      "loss": 2.493821382522583,
+      "step": 8636
+    },
+    {
+      "epoch": 1.1885022783091665,
+      "grad_norm": 0.030159184709191322,
+      "learning_rate": 0.0004946695095948827,
+      "loss": 2.4827747344970703,
+      "step": 8638
+    },
+    {
+      "epoch": 1.1887774641169173,
+      "grad_norm": 0.02866705134510994,
+      "learning_rate": 0.0004934511117879988,
+      "loss": 2.4644267559051514,
+      "step": 8640
+    },
+    {
+      "epoch": 1.1890526499246679,
+      "grad_norm": 0.05815074220299721,
+      "learning_rate": 0.0004922327139811149,
+      "loss": 2.4497568607330322,
+      "step": 8642
+    },
+    {
+      "epoch": 1.1893278357324184,
+      "grad_norm": 0.0282639991492033,
+      "learning_rate": 0.0004910143161742309,
+      "loss": 2.494298219680786,
+      "step": 8644
+    },
+    {
+      "epoch": 1.1896030215401692,
+      "grad_norm": 0.036480583250522614,
+      "learning_rate": 0.0004897959183673469,
+      "loss": 2.456871747970581,
+      "step": 8646
+    },
+    {
+      "epoch": 1.1898782073479197,
+      "grad_norm": 0.025883959606289864,
+      "learning_rate": 0.000488577520560463,
+      "loss": 2.5268614292144775,
+      "step": 8648
+    },
+    {
+      "epoch": 1.1901533931556703,
+      "grad_norm": 0.026932192966341972,
+      "learning_rate": 0.0004873591227535791,
+      "loss": 2.5119948387145996,
+      "step": 8650
+    },
+    {
+      "epoch": 1.190428578963421,
+      "grad_norm": 0.026563217863440514,
+      "learning_rate": 0.0004861407249466951,
+      "loss": 2.5146727561950684,
+      "step": 8652
+    },
+    {
+      "epoch": 1.1907037647711716,
+      "grad_norm": 0.053433675318956375,
+      "learning_rate": 0.00048492232713981116,
+      "loss": 2.4428560733795166,
+      "step": 8654
+    },
+    {
+      "epoch": 1.1909789505789221,
+      "grad_norm": 0.025516720488667488,
+      "learning_rate": 0.0004837039293329272,
+      "loss": 2.509239912033081,
+      "step": 8656
+    },
+    {
+      "epoch": 1.1912541363866727,
+      "grad_norm": 0.04371418058872223,
+      "learning_rate": 0.00048248553152604323,
+      "loss": 2.4663257598876953,
+      "step": 8658
+    },
+    {
+      "epoch": 1.1915293221944234,
+      "grad_norm": 0.030810056254267693,
+      "learning_rate": 0.0004812671337191593,
+      "loss": 2.4689602851867676,
+      "step": 8660
+    },
+    {
+      "epoch": 1.191804508002174,
+      "grad_norm": 0.03950062021613121,
+      "learning_rate": 0.00048004873591227535,
+      "loss": 2.4969935417175293,
+      "step": 8662
+    },
+    {
+      "epoch": 1.1920796938099245,
+      "grad_norm": 0.03437010198831558,
+      "learning_rate": 0.0004788303381053914,
+      "loss": 2.460188388824463,
+      "step": 8664
+    },
+    {
+      "epoch": 1.1923548796176753,
+      "grad_norm": 0.030644772574305534,
+      "learning_rate": 0.0004776119402985075,
+      "loss": 2.42600417137146,
+      "step": 8666
+    },
+    {
+      "epoch": 1.1926300654254258,
+      "grad_norm": 0.06939103454351425,
+      "learning_rate": 0.00047639354249162354,
+      "loss": 2.48158597946167,
+      "step": 8668
+    },
+    {
+      "epoch": 1.1929052512331764,
+      "grad_norm": 0.04514940455555916,
+      "learning_rate": 0.0004751751446847396,
+      "loss": 2.4459056854248047,
+      "step": 8670
+    },
+    {
+      "epoch": 1.193180437040927,
+      "grad_norm": 0.02769850753247738,
+      "learning_rate": 0.0004739567468778556,
+      "loss": 2.4718425273895264,
+      "step": 8672
+    },
+    {
+      "epoch": 1.1934556228486777,
+      "grad_norm": 0.025819890201091766,
+      "learning_rate": 0.00047273834907097167,
+      "loss": 2.4717519283294678,
+      "step": 8674
+    },
+    {
+      "epoch": 1.1937308086564282,
+      "grad_norm": 0.027808645740151405,
+      "learning_rate": 0.00047151995126408773,
+      "loss": 2.502664804458618,
+      "step": 8676
+    },
+    {
+      "epoch": 1.1940059944641788,
+      "grad_norm": 0.026463108137249947,
+      "learning_rate": 0.00047030155345720374,
+      "loss": 2.459801197052002,
+      "step": 8678
+    },
+    {
+      "epoch": 1.1942811802719295,
+      "grad_norm": 0.025830909609794617,
+      "learning_rate": 0.00046908315565031986,
+      "loss": 2.4705429077148438,
+      "step": 8680
+    },
+    {
+      "epoch": 1.19455636607968,
+      "grad_norm": 0.04405985400080681,
+      "learning_rate": 0.0004678647578434359,
+      "loss": 2.477264881134033,
+      "step": 8682
+    },
+    {
+      "epoch": 1.1948315518874306,
+      "grad_norm": 0.026802146807312965,
+      "learning_rate": 0.0004666463600365519,
+      "loss": 2.439289093017578,
+      "step": 8684
+    },
+    {
+      "epoch": 1.1951067376951814,
+      "grad_norm": 0.05216054245829582,
+      "learning_rate": 0.000465427962229668,
+      "loss": 2.4781992435455322,
+      "step": 8686
+    },
+    {
+      "epoch": 1.195381923502932,
+      "grad_norm": 0.031276822090148926,
+      "learning_rate": 0.00046420956442278405,
+      "loss": 2.469414472579956,
+      "step": 8688
+    },
+    {
+      "epoch": 1.1956571093106825,
+      "grad_norm": 0.036080725491046906,
+      "learning_rate": 0.0004629911666159001,
+      "loss": 2.489471435546875,
+      "step": 8690
+    },
+    {
+      "epoch": 1.195932295118433,
+      "grad_norm": 0.050610702484846115,
+      "learning_rate": 0.0004617727688090161,
+      "loss": 2.468400239944458,
+      "step": 8692
+    },
+    {
+      "epoch": 1.1962074809261838,
+      "grad_norm": 0.029053451493382454,
+      "learning_rate": 0.00046055437100213224,
+      "loss": 2.4778554439544678,
+      "step": 8694
+    },
+    {
+      "epoch": 1.1964826667339343,
+      "grad_norm": 0.026429925113916397,
+      "learning_rate": 0.0004593359731952483,
+      "loss": 2.4543070793151855,
+      "step": 8696
+    },
+    {
+      "epoch": 1.1967578525416849,
+      "grad_norm": 0.02676769159734249,
+      "learning_rate": 0.0004581175753883643,
+      "loss": 2.4431557655334473,
+      "step": 8698
+    },
+    {
+      "epoch": 1.1970330383494354,
+      "grad_norm": 0.027419958263635635,
+      "learning_rate": 0.00045689917758148037,
+      "loss": 2.4383792877197266,
+      "step": 8700
+    },
+    {
+      "epoch": 1.1973082241571862,
+      "grad_norm": 0.03429315984249115,
+      "learning_rate": 0.00045568077977459643,
+      "loss": 2.507478713989258,
+      "step": 8702
+    },
+    {
+      "epoch": 1.1975834099649367,
+      "grad_norm": 0.04442322626709938,
+      "learning_rate": 0.00045446238196771244,
+      "loss": 2.504960060119629,
+      "step": 8704
+    },
+    {
+      "epoch": 1.1978585957726873,
+      "grad_norm": 0.05759274214506149,
+      "learning_rate": 0.0004532439841608285,
+      "loss": 2.504065990447998,
+      "step": 8706
+    },
+    {
+      "epoch": 1.198133781580438,
+      "grad_norm": 0.034870464354753494,
+      "learning_rate": 0.00045202558635394456,
+      "loss": 2.481231927871704,
+      "step": 8708
+    },
+    {
+      "epoch": 1.1984089673881886,
+      "grad_norm": 0.056764017790555954,
+      "learning_rate": 0.0004508071885470606,
+      "loss": 2.4213624000549316,
+      "step": 8710
+    },
+    {
+      "epoch": 1.1986841531959391,
+      "grad_norm": 0.06754005700349808,
+      "learning_rate": 0.0004495887907401767,
+      "loss": 2.489760637283325,
+      "step": 8712
+    },
+    {
+      "epoch": 1.19895933900369,
+      "grad_norm": 0.06115381047129631,
+      "learning_rate": 0.00044837039293329275,
+      "loss": 2.458911418914795,
+      "step": 8714
+    },
+    {
+      "epoch": 1.1992345248114404,
+      "grad_norm": 0.054524607956409454,
+      "learning_rate": 0.0004471519951264088,
+      "loss": 2.442030191421509,
+      "step": 8716
+    },
+    {
+      "epoch": 1.199509710619191,
+      "grad_norm": 0.02639007940888405,
+      "learning_rate": 0.0004459335973195248,
+      "loss": 2.504216432571411,
+      "step": 8718
+    },
+    {
+      "epoch": 1.1997848964269415,
+      "grad_norm": 0.06522145122289658,
+      "learning_rate": 0.0004447151995126409,
+      "loss": 2.47813081741333,
+      "step": 8720
+    },
+    {
+      "epoch": 1.2000600822346923,
+      "grad_norm": 0.05460658296942711,
+      "learning_rate": 0.00044349680170575694,
+      "loss": 2.4694604873657227,
+      "step": 8722
+    },
+    {
+      "epoch": 1.2003352680424428,
+      "grad_norm": 0.04274258762598038,
+      "learning_rate": 0.00044227840389887295,
+      "loss": 2.504851818084717,
+      "step": 8724
+    },
+    {
+      "epoch": 1.2006104538501934,
+      "grad_norm": 0.02569100260734558,
+      "learning_rate": 0.00044106000609198906,
+      "loss": 2.478699207305908,
+      "step": 8726
+    },
+    {
+      "epoch": 1.200885639657944,
+      "grad_norm": 0.02615550346672535,
+      "learning_rate": 0.0004398416082851051,
+      "loss": 2.4744057655334473,
+      "step": 8728
+    },
+    {
+      "epoch": 1.2011608254656947,
+      "grad_norm": 0.02948834002017975,
+      "learning_rate": 0.00043862321047822113,
+      "loss": 2.459836483001709,
+      "step": 8730
+    },
+    {
+      "epoch": 1.2014360112734452,
+      "grad_norm": 0.026878049597144127,
+      "learning_rate": 0.0004374048126713372,
+      "loss": 2.470933198928833,
+      "step": 8732
+    },
+    {
+      "epoch": 1.2017111970811958,
+      "grad_norm": 0.030868232250213623,
+      "learning_rate": 0.00043618641486445326,
+      "loss": 2.453636646270752,
+      "step": 8734
+    },
+    {
+      "epoch": 1.2019863828889465,
+      "grad_norm": 0.04337636008858681,
+      "learning_rate": 0.0004349680170575693,
+      "loss": 2.4556937217712402,
+      "step": 8736
+    },
+    {
+      "epoch": 1.202261568696697,
+      "grad_norm": 0.030545899644494057,
+      "learning_rate": 0.0004337496192506853,
+      "loss": 2.4677176475524902,
+      "step": 8738
+    },
+    {
+      "epoch": 1.2025367545044476,
+      "grad_norm": 0.03017844632267952,
+      "learning_rate": 0.0004325312214438014,
+      "loss": 2.465578079223633,
+      "step": 8740
+    },
+    {
+      "epoch": 1.2028119403121984,
+      "grad_norm": 0.024908188730478287,
+      "learning_rate": 0.0004313128236369175,
+      "loss": 2.473461389541626,
+      "step": 8742
+    },
+    {
+      "epoch": 1.203087126119949,
+      "grad_norm": 0.0398617647588253,
+      "learning_rate": 0.0004300944258300335,
+      "loss": 2.4231667518615723,
+      "step": 8744
+    },
+    {
+      "epoch": 1.2033623119276995,
+      "grad_norm": 0.027448071166872978,
+      "learning_rate": 0.0004288760280231496,
+      "loss": 2.4859297275543213,
+      "step": 8746
+    },
+    {
+      "epoch": 1.20363749773545,
+      "grad_norm": 0.04996515065431595,
+      "learning_rate": 0.00042765763021626564,
+      "loss": 2.4736390113830566,
+      "step": 8748
+    },
+    {
+      "epoch": 1.2039126835432008,
+      "grad_norm": 0.1139926165342331,
+      "learning_rate": 0.00042643923240938164,
+      "loss": 2.4843201637268066,
+      "step": 8750
+    },
+    {
+      "epoch": 1.2041878693509513,
+      "grad_norm": 0.06015070527791977,
+      "learning_rate": 0.0004252208346024977,
+      "loss": 2.456665515899658,
+      "step": 8752
+    },
+    {
+      "epoch": 1.204463055158702,
+      "grad_norm": 0.04736275225877762,
+      "learning_rate": 0.00042400243679561377,
+      "loss": 2.463442325592041,
+      "step": 8754
+    },
+    {
+      "epoch": 1.2047382409664524,
+      "grad_norm": 0.042718224227428436,
+      "learning_rate": 0.0004227840389887298,
+      "loss": 2.456305503845215,
+      "step": 8756
+    },
+    {
+      "epoch": 1.2050134267742032,
+      "grad_norm": 0.03286021947860718,
+      "learning_rate": 0.0004215656411818459,
+      "loss": 2.4624977111816406,
+      "step": 8758
+    },
+    {
+      "epoch": 1.2052886125819537,
+      "grad_norm": 0.07304903864860535,
+      "learning_rate": 0.00042034724337496195,
+      "loss": 2.4338793754577637,
+      "step": 8760
+    },
+    {
+      "epoch": 1.2055637983897043,
+      "grad_norm": 0.06499272584915161,
+      "learning_rate": 0.000419128845568078,
+      "loss": 2.4429211616516113,
+      "step": 8762
+    },
+    {
+      "epoch": 1.205838984197455,
+      "grad_norm": 0.07357119023799896,
+      "learning_rate": 0.000417910447761194,
+      "loss": 2.4988369941711426,
+      "step": 8764
+    },
+    {
+      "epoch": 1.2061141700052056,
+      "grad_norm": 0.043625105172395706,
+      "learning_rate": 0.0004166920499543101,
+      "loss": 2.4689550399780273,
+      "step": 8766
+    },
+    {
+      "epoch": 1.2063893558129561,
+      "grad_norm": 0.04893765598535538,
+      "learning_rate": 0.00041547365214742615,
+      "loss": 2.456655979156494,
+      "step": 8768
+    },
+    {
+      "epoch": 1.206664541620707,
+      "grad_norm": 0.04677034541964531,
+      "learning_rate": 0.00041425525434054215,
+      "loss": 2.486523151397705,
+      "step": 8770
+    },
+    {
+      "epoch": 1.2069397274284575,
+      "grad_norm": 0.043685182929039,
+      "learning_rate": 0.00041303685653365827,
+      "loss": 2.4847545623779297,
+      "step": 8772
+    },
+    {
+      "epoch": 1.207214913236208,
+      "grad_norm": 0.03671583905816078,
+      "learning_rate": 0.00041181845872677433,
+      "loss": 2.491459369659424,
+      "step": 8774
+    },
+    {
+      "epoch": 1.2074900990439588,
+      "grad_norm": 0.042950164526700974,
+      "learning_rate": 0.00041060006091989034,
+      "loss": 2.488377571105957,
+      "step": 8776
+    },
+    {
+      "epoch": 1.2077652848517093,
+      "grad_norm": 0.05177051201462746,
+      "learning_rate": 0.0004093816631130064,
+      "loss": 2.4498181343078613,
+      "step": 8778
+    },
+    {
+      "epoch": 1.2080404706594599,
+      "grad_norm": 0.07465200126171112,
+      "learning_rate": 0.00040816326530612246,
+      "loss": 2.4598259925842285,
+      "step": 8780
+    },
+    {
+      "epoch": 1.2083156564672104,
+      "grad_norm": 0.025465643033385277,
+      "learning_rate": 0.0004069448674992385,
+      "loss": 2.4219954013824463,
+      "step": 8782
+    },
+    {
+      "epoch": 1.2085908422749612,
+      "grad_norm": 0.027311531826853752,
+      "learning_rate": 0.00040572646969235453,
+      "loss": 2.4905481338500977,
+      "step": 8784
+    },
+    {
+      "epoch": 1.2088660280827117,
+      "grad_norm": 0.029975229874253273,
+      "learning_rate": 0.0004045080718854706,
+      "loss": 2.4464774131774902,
+      "step": 8786
+    },
+    {
+      "epoch": 1.2091412138904623,
+      "grad_norm": 0.03452488034963608,
+      "learning_rate": 0.0004032896740785867,
+      "loss": 2.491997718811035,
+      "step": 8788
+    },
+    {
+      "epoch": 1.2094163996982128,
+      "grad_norm": 0.03516984358429909,
+      "learning_rate": 0.0004020712762717027,
+      "loss": 2.4754719734191895,
+      "step": 8790
+    },
+    {
+      "epoch": 1.2096915855059636,
+      "grad_norm": 0.039657458662986755,
+      "learning_rate": 0.0004008528784648188,
+      "loss": 2.452157974243164,
+      "step": 8792
+    },
+    {
+      "epoch": 1.209966771313714,
+      "grad_norm": 0.04248063638806343,
+      "learning_rate": 0.00039963448065793484,
+      "loss": 2.450876235961914,
+      "step": 8794
+    },
+    {
+      "epoch": 1.2102419571214647,
+      "grad_norm": 0.035585444420576096,
+      "learning_rate": 0.00039841608285105085,
+      "loss": 2.470597267150879,
+      "step": 8796
+    },
+    {
+      "epoch": 1.2105171429292154,
+      "grad_norm": 0.04110002517700195,
+      "learning_rate": 0.0003971976850441669,
+      "loss": 2.4914891719818115,
+      "step": 8798
+    },
+    {
+      "epoch": 1.210792328736966,
+      "grad_norm": 0.032566286623477936,
+      "learning_rate": 0.000395979287237283,
+      "loss": 2.4725921154022217,
+      "step": 8800
+    },
+    {
+      "epoch": 1.2110675145447165,
+      "grad_norm": 0.026849210262298584,
+      "learning_rate": 0.000394760889430399,
+      "loss": 2.4793097972869873,
+      "step": 8802
+    },
+    {
+      "epoch": 1.2113427003524673,
+      "grad_norm": 0.028937242925167084,
+      "learning_rate": 0.0003935424916235151,
+      "loss": 2.462602138519287,
+      "step": 8804
+    },
+    {
+      "epoch": 1.2116178861602178,
+      "grad_norm": 0.026845673099160194,
+      "learning_rate": 0.00039232409381663116,
+      "loss": 2.4440174102783203,
+      "step": 8806
+    },
+    {
+      "epoch": 1.2118930719679684,
+      "grad_norm": 0.041239842772483826,
+      "learning_rate": 0.0003911056960097472,
+      "loss": 2.4532017707824707,
+      "step": 8808
+    },
+    {
+      "epoch": 1.212168257775719,
+      "grad_norm": 0.027225656434893608,
+      "learning_rate": 0.00038988729820286323,
+      "loss": 2.4820194244384766,
+      "step": 8810
+    },
+    {
+      "epoch": 1.2124434435834697,
+      "grad_norm": 0.029069174081087112,
+      "learning_rate": 0.0003886689003959793,
+      "loss": 2.4859776496887207,
+      "step": 8812
+    },
+    {
+      "epoch": 1.2127186293912202,
+      "grad_norm": 0.024674084037542343,
+      "learning_rate": 0.00038745050258909535,
+      "loss": 2.4597177505493164,
+      "step": 8814
+    },
+    {
+      "epoch": 1.2129938151989708,
+      "grad_norm": 0.0267214123159647,
+      "learning_rate": 0.00038623210478221136,
+      "loss": 2.4306435585021973,
+      "step": 8816
+    },
+    {
+      "epoch": 1.2132690010067213,
+      "grad_norm": 0.0279080793261528,
+      "learning_rate": 0.0003850137069753274,
+      "loss": 2.4768738746643066,
+      "step": 8818
+    },
+    {
+      "epoch": 1.213544186814472,
+      "grad_norm": 0.05523066967725754,
+      "learning_rate": 0.00038379530916844354,
+      "loss": 2.4986910820007324,
+      "step": 8820
+    },
+    {
+      "epoch": 1.2138193726222226,
+      "grad_norm": 0.10398225486278534,
+      "learning_rate": 0.00038257691136155955,
+      "loss": 2.453376054763794,
+      "step": 8822
+    },
+    {
+      "epoch": 1.2140945584299732,
+      "grad_norm": 0.081839419901371,
+      "learning_rate": 0.0003813585135546756,
+      "loss": 2.464280128479004,
+      "step": 8824
+    },
+    {
+      "epoch": 1.214369744237724,
+      "grad_norm": 0.02556564472615719,
+      "learning_rate": 0.00038014011574779167,
+      "loss": 2.440875768661499,
+      "step": 8826
+    },
+    {
+      "epoch": 1.2146449300454745,
+      "grad_norm": 0.09514444321393967,
+      "learning_rate": 0.00037892171794090773,
+      "loss": 2.4620862007141113,
+      "step": 8828
+    },
+    {
+      "epoch": 1.214920115853225,
+      "grad_norm": 0.1271274834871292,
+      "learning_rate": 0.00037770332013402374,
+      "loss": 2.457763671875,
+      "step": 8830
+    },
+    {
+      "epoch": 1.2151953016609758,
+      "grad_norm": 0.032811474055051804,
+      "learning_rate": 0.0003764849223271398,
+      "loss": 2.4793639183044434,
+      "step": 8832
+    },
+    {
+      "epoch": 1.2154704874687263,
+      "grad_norm": 0.11348410695791245,
+      "learning_rate": 0.0003752665245202559,
+      "loss": 2.4599878787994385,
+      "step": 8834
+    },
+    {
+      "epoch": 1.2157456732764769,
+      "grad_norm": 0.07678667455911636,
+      "learning_rate": 0.0003740481267133719,
+      "loss": 2.486891746520996,
+      "step": 8836
+    },
+    {
+      "epoch": 1.2160208590842274,
+      "grad_norm": 0.04440055787563324,
+      "learning_rate": 0.000372829728906488,
+      "loss": 2.456447124481201,
+      "step": 8838
+    },
+    {
+      "epoch": 1.2162960448919782,
+      "grad_norm": 0.09812157601118088,
+      "learning_rate": 0.00037161133109960405,
+      "loss": 2.5057811737060547,
+      "step": 8840
+    },
+    {
+      "epoch": 1.2165712306997287,
+      "grad_norm": 0.031434498727321625,
+      "learning_rate": 0.00037039293329272006,
+      "loss": 2.4847702980041504,
+      "step": 8842
+    },
+    {
+      "epoch": 1.2168464165074793,
+      "grad_norm": 0.10320954024791718,
+      "learning_rate": 0.0003691745354858361,
+      "loss": 2.4739620685577393,
+      "step": 8844
+    },
+    {
+      "epoch": 1.2171216023152298,
+      "grad_norm": 0.11398742347955704,
+      "learning_rate": 0.0003679561376789522,
+      "loss": 2.4958982467651367,
+      "step": 8846
+    },
+    {
+      "epoch": 1.2173967881229806,
+      "grad_norm": 0.03367593139410019,
+      "learning_rate": 0.00036673773987206824,
+      "loss": 2.462498664855957,
+      "step": 8848
+    },
+    {
+      "epoch": 1.2176719739307311,
+      "grad_norm": 0.06830449402332306,
+      "learning_rate": 0.0003655193420651843,
+      "loss": 2.4631996154785156,
+      "step": 8850
+    },
+    {
+      "epoch": 1.2179471597384817,
+      "grad_norm": 0.0444931797683239,
+      "learning_rate": 0.00036430094425830037,
+      "loss": 2.439612865447998,
+      "step": 8852
+    },
+    {
+      "epoch": 1.2182223455462324,
+      "grad_norm": 0.03447304293513298,
+      "learning_rate": 0.00036308254645141643,
+      "loss": 2.5316848754882812,
+      "step": 8854
+    },
+    {
+      "epoch": 1.218497531353983,
+      "grad_norm": 0.06705034524202347,
+      "learning_rate": 0.00036186414864453244,
+      "loss": 2.4550440311431885,
+      "step": 8856
+    },
+    {
+      "epoch": 1.2187727171617335,
+      "grad_norm": 0.03081127442419529,
+      "learning_rate": 0.0003606457508376485,
+      "loss": 2.4654078483581543,
+      "step": 8858
+    },
+    {
+      "epoch": 1.2190479029694843,
+      "grad_norm": 0.04607037454843521,
+      "learning_rate": 0.00035942735303076456,
+      "loss": 2.4596080780029297,
+      "step": 8860
+    },
+    {
+      "epoch": 1.2193230887772348,
+      "grad_norm": 0.04579803720116615,
+      "learning_rate": 0.00035820895522388057,
+      "loss": 2.4844906330108643,
+      "step": 8862
+    },
+    {
+      "epoch": 1.2195982745849854,
+      "grad_norm": 0.026145927608013153,
+      "learning_rate": 0.00035699055741699663,
+      "loss": 2.4469752311706543,
+      "step": 8864
+    },
+    {
+      "epoch": 1.219873460392736,
+      "grad_norm": 0.03595560044050217,
+      "learning_rate": 0.00035577215961011275,
+      "loss": 2.461869239807129,
+      "step": 8866
+    },
+    {
+      "epoch": 1.2201486462004867,
+      "grad_norm": 0.03903980925679207,
+      "learning_rate": 0.00035455376180322875,
+      "loss": 2.483745574951172,
+      "step": 8868
+    },
+    {
+      "epoch": 1.2204238320082372,
+      "grad_norm": 0.026750175282359123,
+      "learning_rate": 0.0003533353639963448,
+      "loss": 2.455982208251953,
+      "step": 8870
+    },
+    {
+      "epoch": 1.2206990178159878,
+      "grad_norm": 0.06166665628552437,
+      "learning_rate": 0.0003521169661894609,
+      "loss": 2.4688618183135986,
+      "step": 8872
+    },
+    {
+      "epoch": 1.2209742036237385,
+      "grad_norm": 0.030036283656954765,
+      "learning_rate": 0.00035089856838257694,
+      "loss": 2.4493227005004883,
+      "step": 8874
+    },
+    {
+      "epoch": 1.221249389431489,
+      "grad_norm": 0.02775690332055092,
+      "learning_rate": 0.00034968017057569295,
+      "loss": 2.4693961143493652,
+      "step": 8876
+    },
+    {
+      "epoch": 1.2215245752392396,
+      "grad_norm": 0.029315469786524773,
+      "learning_rate": 0.000348461772768809,
+      "loss": 2.473182201385498,
+      "step": 8878
+    },
+    {
+      "epoch": 1.2217997610469902,
+      "grad_norm": 0.04584071785211563,
+      "learning_rate": 0.00034724337496192507,
+      "loss": 2.4239397048950195,
+      "step": 8880
+    },
+    {
+      "epoch": 1.222074946854741,
+      "grad_norm": 0.05503488704562187,
+      "learning_rate": 0.00034602497715504113,
+      "loss": 2.449183940887451,
+      "step": 8882
+    },
+    {
+      "epoch": 1.2223501326624915,
+      "grad_norm": 0.07854048162698746,
+      "learning_rate": 0.0003448065793481572,
+      "loss": 2.506598711013794,
+      "step": 8884
+    },
+    {
+      "epoch": 1.222625318470242,
+      "grad_norm": 0.024540036916732788,
+      "learning_rate": 0.00034358818154127326,
+      "loss": 2.4966657161712646,
+      "step": 8886
+    },
+    {
+      "epoch": 1.2229005042779928,
+      "grad_norm": 0.05817464739084244,
+      "learning_rate": 0.00034236978373438926,
+      "loss": 2.468033790588379,
+      "step": 8888
+    },
+    {
+      "epoch": 1.2231756900857433,
+      "grad_norm": 0.06302707642316818,
+      "learning_rate": 0.0003411513859275053,
+      "loss": 2.5060083866119385,
+      "step": 8890
+    },
+    {
+      "epoch": 1.2234508758934939,
+      "grad_norm": 0.0263026412576437,
+      "learning_rate": 0.0003399329881206214,
+      "loss": 2.454346179962158,
+      "step": 8892
+    },
+    {
+      "epoch": 1.2237260617012446,
+      "grad_norm": 0.02917100489139557,
+      "learning_rate": 0.00033871459031373745,
+      "loss": 2.47318172454834,
+      "step": 8894
+    },
+    {
+      "epoch": 1.2240012475089952,
+      "grad_norm": 0.041920337826013565,
+      "learning_rate": 0.00033749619250685346,
+      "loss": 2.4973304271698,
+      "step": 8896
+    },
+    {
+      "epoch": 1.2242764333167457,
+      "grad_norm": 0.059054918587207794,
+      "learning_rate": 0.0003362777946999696,
+      "loss": 2.4806647300720215,
+      "step": 8898
+    },
+    {
+      "epoch": 1.2245516191244963,
+      "grad_norm": 0.03674178570508957,
+      "learning_rate": 0.00033505939689308564,
+      "loss": 2.461876392364502,
+      "step": 8900
+    },
+    {
+      "epoch": 1.224826804932247,
+      "grad_norm": 0.0263822004199028,
+      "learning_rate": 0.00033384099908620164,
+      "loss": 2.496554136276245,
+      "step": 8902
+    },
+    {
+      "epoch": 1.2251019907399976,
+      "grad_norm": 0.05070227384567261,
+      "learning_rate": 0.0003326226012793177,
+      "loss": 2.4399871826171875,
+      "step": 8904
+    },
+    {
+      "epoch": 1.2253771765477481,
+      "grad_norm": 0.04350428655743599,
+      "learning_rate": 0.00033140420347243377,
+      "loss": 2.4651107788085938,
+      "step": 8906
+    },
+    {
+      "epoch": 1.2256523623554987,
+      "grad_norm": 0.030962686985731125,
+      "learning_rate": 0.0003301858056655498,
+      "loss": 2.4717912673950195,
+      "step": 8908
+    },
+    {
+      "epoch": 1.2259275481632494,
+      "grad_norm": 0.03303976729512215,
+      "learning_rate": 0.00032896740785866584,
+      "loss": 2.4325356483459473,
+      "step": 8910
+    },
+    {
+      "epoch": 1.226202733971,
+      "grad_norm": 0.029044678434729576,
+      "learning_rate": 0.00032774901005178195,
+      "loss": 2.4655752182006836,
+      "step": 8912
+    },
+    {
+      "epoch": 1.2264779197787505,
+      "grad_norm": 0.026125498116016388,
+      "learning_rate": 0.00032653061224489796,
+      "loss": 2.4559173583984375,
+      "step": 8914
+    },
+    {
+      "epoch": 1.2267531055865013,
+      "grad_norm": 0.028719931840896606,
+      "learning_rate": 0.000325312214438014,
+      "loss": 2.4583353996276855,
+      "step": 8916
+    },
+    {
+      "epoch": 1.2270282913942518,
+      "grad_norm": 0.0288032665848732,
+      "learning_rate": 0.0003240938166311301,
+      "loss": 2.45771861076355,
+      "step": 8918
+    },
+    {
+      "epoch": 1.2273034772020024,
+      "grad_norm": 0.02581830322742462,
+      "learning_rate": 0.00032287541882424615,
+      "loss": 2.4553146362304688,
+      "step": 8920
+    },
+    {
+      "epoch": 1.2275786630097532,
+      "grad_norm": 0.025615032762289047,
+      "learning_rate": 0.00032165702101736215,
+      "loss": 2.5127549171447754,
+      "step": 8922
+    },
+    {
+      "epoch": 1.2278538488175037,
+      "grad_norm": 0.031048277392983437,
+      "learning_rate": 0.0003204386232104782,
+      "loss": 2.4910147190093994,
+      "step": 8924
+    },
+    {
+      "epoch": 1.2281290346252542,
+      "grad_norm": 0.04934019595384598,
+      "learning_rate": 0.0003192202254035943,
+      "loss": 2.4799563884735107,
+      "step": 8926
+    },
+    {
+      "epoch": 1.2284042204330048,
+      "grad_norm": 0.06664245575666428,
+      "learning_rate": 0.00031800182759671034,
+      "loss": 2.4664297103881836,
+      "step": 8928
+    },
+    {
+      "epoch": 1.2286794062407556,
+      "grad_norm": 0.03732762858271599,
+      "learning_rate": 0.0003167834297898264,
+      "loss": 2.407378673553467,
+      "step": 8930
+    },
+    {
+      "epoch": 1.228954592048506,
+      "grad_norm": 0.02626616135239601,
+      "learning_rate": 0.00031556503198294246,
+      "loss": 2.4968457221984863,
+      "step": 8932
+    },
+    {
+      "epoch": 1.2292297778562566,
+      "grad_norm": 0.02694445662200451,
+      "learning_rate": 0.00031434663417605847,
+      "loss": 2.4582109451293945,
+      "step": 8934
+    },
+    {
+      "epoch": 1.2295049636640072,
+      "grad_norm": 0.02504737116396427,
+      "learning_rate": 0.00031312823636917453,
+      "loss": 2.4719767570495605,
+      "step": 8936
+    },
+    {
+      "epoch": 1.229780149471758,
+      "grad_norm": 0.026110438629984856,
+      "learning_rate": 0.0003119098385622906,
+      "loss": 2.494311571121216,
+      "step": 8938
+    },
+    {
+      "epoch": 1.2300553352795085,
+      "grad_norm": 0.026535863056778908,
+      "learning_rate": 0.00031069144075540666,
+      "loss": 2.4506266117095947,
+      "step": 8940
+    },
+    {
+      "epoch": 1.230330521087259,
+      "grad_norm": 0.027939949184656143,
+      "learning_rate": 0.00030947304294852266,
+      "loss": 2.4675183296203613,
+      "step": 8942
+    },
+    {
+      "epoch": 1.2306057068950098,
+      "grad_norm": 0.046726975589990616,
+      "learning_rate": 0.0003082546451416388,
+      "loss": 2.471871852874756,
+      "step": 8944
+    },
+    {
+      "epoch": 1.2308808927027604,
+      "grad_norm": 0.025117380544543266,
+      "learning_rate": 0.00030703624733475484,
+      "loss": 2.445155143737793,
+      "step": 8946
+    },
+    {
+      "epoch": 1.231156078510511,
+      "grad_norm": 0.0278630368411541,
+      "learning_rate": 0.00030581784952787085,
+      "loss": 2.4678473472595215,
+      "step": 8948
+    },
+    {
+      "epoch": 1.2314312643182617,
+      "grad_norm": 0.024387259036302567,
+      "learning_rate": 0.0003045994517209869,
+      "loss": 2.422560214996338,
+      "step": 8950
+    },
+    {
+      "epoch": 1.2317064501260122,
+      "grad_norm": 0.05300230532884598,
+      "learning_rate": 0.000303381053914103,
+      "loss": 2.4682836532592773,
+      "step": 8952
+    },
+    {
+      "epoch": 1.2319816359337628,
+      "grad_norm": 0.051963262259960175,
+      "learning_rate": 0.000302162656107219,
+      "loss": 2.4685332775115967,
+      "step": 8954
+    },
+    {
+      "epoch": 1.2322568217415133,
+      "grad_norm": 0.08862706273794174,
+      "learning_rate": 0.00030094425830033504,
+      "loss": 2.463040351867676,
+      "step": 8956
+    },
+    {
+      "epoch": 1.232532007549264,
+      "grad_norm": 0.04717070609331131,
+      "learning_rate": 0.0002997258604934511,
+      "loss": 2.488790988922119,
+      "step": 8958
+    },
+    {
+      "epoch": 1.2328071933570146,
+      "grad_norm": 0.02617405168712139,
+      "learning_rate": 0.00029850746268656717,
+      "loss": 2.4692492485046387,
+      "step": 8960
+    },
+    {
+      "epoch": 1.2330823791647652,
+      "grad_norm": 0.03451040759682655,
+      "learning_rate": 0.00029728906487968323,
+      "loss": 2.460096597671509,
+      "step": 8962
+    },
+    {
+      "epoch": 1.2333575649725157,
+      "grad_norm": 0.08512626588344574,
+      "learning_rate": 0.0002960706670727993,
+      "loss": 2.431614398956299,
+      "step": 8964
+    },
+    {
+      "epoch": 1.2336327507802665,
+      "grad_norm": 0.04286133497953415,
+      "learning_rate": 0.00029485226926591535,
+      "loss": 2.4373245239257812,
+      "step": 8966
+    },
+    {
+      "epoch": 1.233907936588017,
+      "grad_norm": 0.035752519965171814,
+      "learning_rate": 0.00029363387145903136,
+      "loss": 2.481111526489258,
+      "step": 8968
+    },
+    {
+      "epoch": 1.2341831223957676,
+      "grad_norm": 0.09488311409950256,
+      "learning_rate": 0.0002924154736521474,
+      "loss": 2.484152317047119,
+      "step": 8970
+    },
+    {
+      "epoch": 1.2344583082035183,
+      "grad_norm": 0.05059340223670006,
+      "learning_rate": 0.0002911970758452635,
+      "loss": 2.4567389488220215,
+      "step": 8972
+    },
+    {
+      "epoch": 1.2347334940112689,
+      "grad_norm": 0.031188562512397766,
+      "learning_rate": 0.0002899786780383795,
+      "loss": 2.473804473876953,
+      "step": 8974
+    },
+    {
+      "epoch": 1.2350086798190194,
+      "grad_norm": 0.062142908573150635,
+      "learning_rate": 0.0002887602802314956,
+      "loss": 2.452099084854126,
+      "step": 8976
+    },
+    {
+      "epoch": 1.2352838656267702,
+      "grad_norm": 0.041879184544086456,
+      "learning_rate": 0.00028754188242461167,
+      "loss": 2.4886016845703125,
+      "step": 8978
+    },
+    {
+      "epoch": 1.2355590514345207,
+      "grad_norm": 0.03959394246339798,
+      "learning_rate": 0.0002863234846177277,
+      "loss": 2.43761944770813,
+      "step": 8980
+    },
+    {
+      "epoch": 1.2358342372422713,
+      "grad_norm": 0.07003889232873917,
+      "learning_rate": 0.00028510508681084374,
+      "loss": 2.4809274673461914,
+      "step": 8982
+    },
+    {
+      "epoch": 1.2361094230500218,
+      "grad_norm": 0.02477721869945526,
+      "learning_rate": 0.0002838866890039598,
+      "loss": 2.4572484493255615,
+      "step": 8984
+    },
+    {
+      "epoch": 1.2363846088577726,
+      "grad_norm": 0.03895578160881996,
+      "learning_rate": 0.00028266829119707586,
+      "loss": 2.492746114730835,
+      "step": 8986
+    },
+    {
+      "epoch": 1.2366597946655231,
+      "grad_norm": 0.02589019387960434,
+      "learning_rate": 0.00028144989339019187,
+      "loss": 2.463683605194092,
+      "step": 8988
+    },
+    {
+      "epoch": 1.2369349804732737,
+      "grad_norm": 0.024336805567145348,
+      "learning_rate": 0.000280231495583308,
+      "loss": 2.462240219116211,
+      "step": 8990
+    },
+    {
+      "epoch": 1.2372101662810244,
+      "grad_norm": 0.02807413786649704,
+      "learning_rate": 0.00027901309777642405,
+      "loss": 2.4488985538482666,
+      "step": 8992
+    },
+    {
+      "epoch": 1.237485352088775,
+      "grad_norm": 0.045868195593357086,
+      "learning_rate": 0.00027779469996954006,
+      "loss": 2.4374823570251465,
+      "step": 8994
+    },
+    {
+      "epoch": 1.2377605378965255,
+      "grad_norm": 0.055717382580041885,
+      "learning_rate": 0.0002765763021626561,
+      "loss": 2.4435203075408936,
+      "step": 8996
+    },
+    {
+      "epoch": 1.238035723704276,
+      "grad_norm": 0.0261841993778944,
+      "learning_rate": 0.0002753579043557722,
+      "loss": 2.413534641265869,
+      "step": 8998
+    },
+    {
+      "epoch": 1.2383109095120268,
+      "grad_norm": 0.10789460688829422,
+      "learning_rate": 0.0002741395065488882,
+      "loss": 2.4650251865386963,
+      "step": 9000
+    },
+    {
+      "epoch": 1.2385860953197774,
+      "grad_norm": 0.06245919689536095,
+      "learning_rate": 0.00027292110874200425,
+      "loss": 2.457040309906006,
+      "step": 9002
+    },
+    {
+      "epoch": 1.238861281127528,
+      "grad_norm": 0.05550862103700638,
+      "learning_rate": 0.0002717027109351203,
+      "loss": 2.4677391052246094,
+      "step": 9004
+    },
+    {
+      "epoch": 1.2391364669352787,
+      "grad_norm": 0.08197030425071716,
+      "learning_rate": 0.0002704843131282364,
+      "loss": 2.44098162651062,
+      "step": 9006
+    },
+    {
+      "epoch": 1.2394116527430292,
+      "grad_norm": 0.04194927215576172,
+      "learning_rate": 0.00026926591532135244,
+      "loss": 2.474851608276367,
+      "step": 9008
+    },
+    {
+      "epoch": 1.2396868385507798,
+      "grad_norm": 0.06186150759458542,
+      "learning_rate": 0.0002680475175144685,
+      "loss": 2.4432456493377686,
+      "step": 9010
+    },
+    {
+      "epoch": 1.2399620243585305,
+      "grad_norm": 0.03969191387295723,
+      "learning_rate": 0.00026682911970758456,
+      "loss": 2.454441547393799,
+      "step": 9012
+    },
+    {
+      "epoch": 1.240237210166281,
+      "grad_norm": 0.05759728327393532,
+      "learning_rate": 0.00026561072190070057,
+      "loss": 2.4498791694641113,
+      "step": 9014
+    },
+    {
+      "epoch": 1.2405123959740316,
+      "grad_norm": 0.053908172994852066,
+      "learning_rate": 0.00026439232409381663,
+      "loss": 2.458129405975342,
+      "step": 9016
+    },
+    {
+      "epoch": 1.2407875817817822,
+      "grad_norm": 0.02970687486231327,
+      "learning_rate": 0.0002631739262869327,
+      "loss": 2.470946788787842,
+      "step": 9018
+    },
+    {
+      "epoch": 1.241062767589533,
+      "grad_norm": 0.0539206899702549,
+      "learning_rate": 0.0002619555284800487,
+      "loss": 2.4942121505737305,
+      "step": 9020
+    },
+    {
+      "epoch": 1.2413379533972835,
+      "grad_norm": 0.037933267652988434,
+      "learning_rate": 0.0002607371306731648,
+      "loss": 2.4189348220825195,
+      "step": 9022
+    },
+    {
+      "epoch": 1.241613139205034,
+      "grad_norm": 0.057484593242406845,
+      "learning_rate": 0.0002595187328662809,
+      "loss": 2.4525227546691895,
+      "step": 9024
+    },
+    {
+      "epoch": 1.2418883250127846,
+      "grad_norm": 0.07437066733837128,
+      "learning_rate": 0.0002583003350593969,
+      "loss": 2.5115623474121094,
+      "step": 9026
+    },
+    {
+      "epoch": 1.2421635108205353,
+      "grad_norm": 0.056315451860427856,
+      "learning_rate": 0.00025708193725251295,
+      "loss": 2.483595848083496,
+      "step": 9028
+    },
+    {
+      "epoch": 1.2424386966282859,
+      "grad_norm": 0.07366979122161865,
+      "learning_rate": 0.000255863539445629,
+      "loss": 2.471569538116455,
+      "step": 9030
+    },
+    {
+      "epoch": 1.2427138824360364,
+      "grad_norm": 0.026633277535438538,
+      "learning_rate": 0.00025464514163874507,
+      "loss": 2.462831974029541,
+      "step": 9032
+    },
+    {
+      "epoch": 1.2429890682437872,
+      "grad_norm": 0.09097911417484283,
+      "learning_rate": 0.0002534267438318611,
+      "loss": 2.491210460662842,
+      "step": 9034
+    },
+    {
+      "epoch": 1.2432642540515377,
+      "grad_norm": 0.04499977454543114,
+      "learning_rate": 0.00025220834602497714,
+      "loss": 2.476870059967041,
+      "step": 9036
+    },
+    {
+      "epoch": 1.2435394398592883,
+      "grad_norm": 0.048204243183135986,
+      "learning_rate": 0.00025098994821809326,
+      "loss": 2.446970224380493,
+      "step": 9038
+    },
+    {
+      "epoch": 1.243814625667039,
+      "grad_norm": 0.053671423345804214,
+      "learning_rate": 0.00024977155041120926,
+      "loss": 2.446554183959961,
+      "step": 9040
+    },
+    {
+      "epoch": 1.2440898114747896,
+      "grad_norm": 0.028479335829615593,
+      "learning_rate": 0.0002485531526043253,
+      "loss": 2.4327917098999023,
+      "step": 9042
+    },
+    {
+      "epoch": 1.2443649972825401,
+      "grad_norm": 0.04893416911363602,
+      "learning_rate": 0.00024733475479744133,
+      "loss": 2.496900796890259,
+      "step": 9044
+    },
+    {
+      "epoch": 1.2446401830902907,
+      "grad_norm": 0.025767788290977478,
+      "learning_rate": 0.00024611635699055745,
+      "loss": 2.4836649894714355,
+      "step": 9046
+    },
+    {
+      "epoch": 1.2449153688980414,
+      "grad_norm": 0.043032169342041016,
+      "learning_rate": 0.00024489795918367346,
+      "loss": 2.500868558883667,
+      "step": 9048
+    },
+    {
+      "epoch": 1.245190554705792,
+      "grad_norm": 0.026030398905277252,
+      "learning_rate": 0.00024367956137678955,
+      "loss": 2.488234519958496,
+      "step": 9050
+    },
+    {
+      "epoch": 1.2454657405135425,
+      "grad_norm": 0.03727688267827034,
+      "learning_rate": 0.00024246116356990558,
+      "loss": 2.4308507442474365,
+      "step": 9052
+    },
+    {
+      "epoch": 1.245740926321293,
+      "grad_norm": 0.02399192936718464,
+      "learning_rate": 0.00024124276576302162,
+      "loss": 2.466092109680176,
+      "step": 9054
+    },
+    {
+      "epoch": 1.2460161121290438,
+      "grad_norm": 0.04762476682662964,
+      "learning_rate": 0.00024002436795613768,
+      "loss": 2.4530653953552246,
+      "step": 9056
+    },
+    {
+      "epoch": 1.2462912979367944,
+      "grad_norm": 0.03271552547812462,
+      "learning_rate": 0.00023880597014925374,
+      "loss": 2.4567389488220215,
+      "step": 9058
+    },
+    {
+      "epoch": 1.246566483744545,
+      "grad_norm": 0.03467216715216637,
+      "learning_rate": 0.0002375875723423698,
+      "loss": 2.456214427947998,
+      "step": 9060
+    },
+    {
+      "epoch": 1.2468416695522957,
+      "grad_norm": 0.030006764456629753,
+      "learning_rate": 0.00023636917453548584,
+      "loss": 2.4589505195617676,
+      "step": 9062
+    },
+    {
+      "epoch": 1.2471168553600462,
+      "grad_norm": 0.02347988449037075,
+      "learning_rate": 0.00023515077672860187,
+      "loss": 2.4620018005371094,
+      "step": 9064
+    },
+    {
+      "epoch": 1.2473920411677968,
+      "grad_norm": 0.024628883227705956,
+      "learning_rate": 0.00023393237892171796,
+      "loss": 2.435481548309326,
+      "step": 9066
+    },
+    {
+      "epoch": 1.2476672269755475,
+      "grad_norm": 0.026944318786263466,
+      "learning_rate": 0.000232713981114834,
+      "loss": 2.4822263717651367,
+      "step": 9068
+    },
+    {
+      "epoch": 1.247942412783298,
+      "grad_norm": 0.024742722511291504,
+      "learning_rate": 0.00023149558330795006,
+      "loss": 2.440927505493164,
+      "step": 9070
+    },
+    {
+      "epoch": 1.2482175985910486,
+      "grad_norm": 0.025591425597667694,
+      "learning_rate": 0.00023027718550106612,
+      "loss": 2.4935126304626465,
+      "step": 9072
+    },
+    {
+      "epoch": 1.2484927843987992,
+      "grad_norm": 0.030644886195659637,
+      "learning_rate": 0.00022905878769418215,
+      "loss": 2.4689183235168457,
+      "step": 9074
+    },
+    {
+      "epoch": 1.24876797020655,
+      "grad_norm": 0.04450540617108345,
+      "learning_rate": 0.00022784038988729821,
+      "loss": 2.4379994869232178,
+      "step": 9076
+    },
+    {
+      "epoch": 1.2490431560143005,
+      "grad_norm": 0.024654757231473923,
+      "learning_rate": 0.00022662199208041425,
+      "loss": 2.464127540588379,
+      "step": 9078
+    },
+    {
+      "epoch": 1.249318341822051,
+      "grad_norm": 0.027313780039548874,
+      "learning_rate": 0.0002254035942735303,
+      "loss": 2.4595882892608643,
+      "step": 9080
+    },
+    {
+      "epoch": 1.2495935276298016,
+      "grad_norm": 0.02862522378563881,
+      "learning_rate": 0.00022418519646664637,
+      "loss": 2.47308087348938,
+      "step": 9082
+    },
+    {
+      "epoch": 1.2498687134375523,
+      "grad_norm": 0.03277771174907684,
+      "learning_rate": 0.0002229667986597624,
+      "loss": 2.4726290702819824,
+      "step": 9084
+    },
+    {
+      "epoch": 1.250143899245303,
+      "grad_norm": 0.06404554843902588,
+      "learning_rate": 0.00022174840085287847,
+      "loss": 2.444108009338379,
+      "step": 9086
+    },
+    {
+      "epoch": 1.2504190850530534,
+      "grad_norm": 0.034670136868953705,
+      "learning_rate": 0.00022053000304599453,
+      "loss": 2.4304516315460205,
+      "step": 9088
+    },
+    {
+      "epoch": 1.2506942708608042,
+      "grad_norm": 0.03571346402168274,
+      "learning_rate": 0.00021931160523911057,
+      "loss": 2.4753894805908203,
+      "step": 9090
+    },
+    {
+      "epoch": 1.2509694566685547,
+      "grad_norm": 0.024204323068261147,
+      "learning_rate": 0.00021809320743222663,
+      "loss": 2.479771137237549,
+      "step": 9092
+    },
+    {
+      "epoch": 1.2512446424763053,
+      "grad_norm": 0.03631271794438362,
+      "learning_rate": 0.00021687480962534266,
+      "loss": 2.416874408721924,
+      "step": 9094
+    },
+    {
+      "epoch": 1.251519828284056,
+      "grad_norm": 0.025105614215135574,
+      "learning_rate": 0.00021565641181845875,
+      "loss": 2.46626615524292,
+      "step": 9096
+    },
+    {
+      "epoch": 1.2517950140918066,
+      "grad_norm": 0.043047938495874405,
+      "learning_rate": 0.0002144380140115748,
+      "loss": 2.462360382080078,
+      "step": 9098
+    },
+    {
+      "epoch": 1.2520701998995571,
+      "grad_norm": 0.04422559216618538,
+      "learning_rate": 0.00021321961620469082,
+      "loss": 2.4511795043945312,
+      "step": 9100
+    },
+    {
+      "epoch": 1.252345385707308,
+      "grad_norm": 0.02456490322947502,
+      "learning_rate": 0.00021200121839780688,
+      "loss": 2.463484764099121,
+      "step": 9102
+    },
+    {
+      "epoch": 1.2526205715150585,
+      "grad_norm": 0.06656299531459808,
+      "learning_rate": 0.00021078282059092295,
+      "loss": 2.4093174934387207,
+      "step": 9104
+    },
+    {
+      "epoch": 1.252895757322809,
+      "grad_norm": 0.07564622163772583,
+      "learning_rate": 0.000209564422784039,
+      "loss": 2.4552369117736816,
+      "step": 9106
+    },
+    {
+      "epoch": 1.2531709431305595,
+      "grad_norm": 0.030012695118784904,
+      "learning_rate": 0.00020834602497715504,
+      "loss": 2.485116958618164,
+      "step": 9108
+    },
+    {
+      "epoch": 1.25344612893831,
+      "grad_norm": 0.08046410232782364,
+      "learning_rate": 0.00020712762717027108,
+      "loss": 2.4423084259033203,
+      "step": 9110
+    },
+    {
+      "epoch": 1.2537213147460609,
+      "grad_norm": 0.03732582926750183,
+      "learning_rate": 0.00020590922936338717,
+      "loss": 2.4297032356262207,
+      "step": 9112
+    },
+    {
+      "epoch": 1.2539965005538114,
+      "grad_norm": 0.05407774820923805,
+      "learning_rate": 0.0002046908315565032,
+      "loss": 2.470935344696045,
+      "step": 9114
+    },
+    {
+      "epoch": 1.254271686361562,
+      "grad_norm": 0.03944677114486694,
+      "learning_rate": 0.00020347243374961926,
+      "loss": 2.434739351272583,
+      "step": 9116
+    },
+    {
+      "epoch": 1.2545468721693127,
+      "grad_norm": 0.05136119946837425,
+      "learning_rate": 0.0002022540359427353,
+      "loss": 2.4616265296936035,
+      "step": 9118
+    },
+    {
+      "epoch": 1.2548220579770633,
+      "grad_norm": 0.04568967595696449,
+      "learning_rate": 0.00020103563813585136,
+      "loss": 2.468766689300537,
+      "step": 9120
+    },
+    {
+      "epoch": 1.2550972437848138,
+      "grad_norm": 0.04253152385354042,
+      "learning_rate": 0.00019981724032896742,
+      "loss": 2.4550366401672363,
+      "step": 9122
+    },
+    {
+      "epoch": 1.2553724295925646,
+      "grad_norm": 0.06559917330741882,
+      "learning_rate": 0.00019859884252208346,
+      "loss": 2.4733376502990723,
+      "step": 9124
+    },
+    {
+      "epoch": 1.255647615400315,
+      "grad_norm": 0.024422714486718178,
+      "learning_rate": 0.0001973804447151995,
+      "loss": 2.4262566566467285,
+      "step": 9126
+    },
+    {
+      "epoch": 1.2559228012080657,
+      "grad_norm": 0.024974917992949486,
+      "learning_rate": 0.00019616204690831558,
+      "loss": 2.4665987491607666,
+      "step": 9128
+    },
+    {
+      "epoch": 1.2561979870158164,
+      "grad_norm": 0.037340566515922546,
+      "learning_rate": 0.00019494364910143161,
+      "loss": 2.4573168754577637,
+      "step": 9130
+    },
+    {
+      "epoch": 1.256473172823567,
+      "grad_norm": 0.02619260549545288,
+      "learning_rate": 0.00019372525129454768,
+      "loss": 2.4704086780548096,
+      "step": 9132
+    },
+    {
+      "epoch": 1.2567483586313175,
+      "grad_norm": 0.045244187116622925,
+      "learning_rate": 0.0001925068534876637,
+      "loss": 2.4865617752075195,
+      "step": 9134
+    },
+    {
+      "epoch": 1.257023544439068,
+      "grad_norm": 0.022733744233846664,
+      "learning_rate": 0.00019128845568077977,
+      "loss": 2.471360921859741,
+      "step": 9136
+    },
+    {
+      "epoch": 1.2572987302468188,
+      "grad_norm": 0.04515485838055611,
+      "learning_rate": 0.00019007005787389584,
+      "loss": 2.4244136810302734,
+      "step": 9138
+    },
+    {
+      "epoch": 1.2575739160545694,
+      "grad_norm": 0.03306630998849869,
+      "learning_rate": 0.00018885166006701187,
+      "loss": 2.480607748031616,
+      "step": 9140
+    },
+    {
+      "epoch": 1.25784910186232,
+      "grad_norm": 0.025314409285783768,
+      "learning_rate": 0.00018763326226012796,
+      "loss": 2.4583816528320312,
+      "step": 9142
+    },
+    {
+      "epoch": 1.2581242876700705,
+      "grad_norm": 0.03562302887439728,
+      "learning_rate": 0.000186414864453244,
+      "loss": 2.4455204010009766,
+      "step": 9144
+    },
+    {
+      "epoch": 1.2583994734778212,
+      "grad_norm": 0.02388559654355049,
+      "learning_rate": 0.00018519646664636003,
+      "loss": 2.4644341468811035,
+      "step": 9146
+    },
+    {
+      "epoch": 1.2586746592855718,
+      "grad_norm": 0.02913445234298706,
+      "learning_rate": 0.0001839780688394761,
+      "loss": 2.4225234985351562,
+      "step": 9148
+    },
+    {
+      "epoch": 1.2589498450933223,
+      "grad_norm": 0.030540654435753822,
+      "learning_rate": 0.00018275967103259215,
+      "loss": 2.4465856552124023,
+      "step": 9150
+    },
+    {
+      "epoch": 1.259225030901073,
+      "grad_norm": 0.06099849194288254,
+      "learning_rate": 0.00018154127322570821,
+      "loss": 2.486354351043701,
+      "step": 9152
+    },
+    {
+      "epoch": 1.2595002167088236,
+      "grad_norm": 0.04487714543938637,
+      "learning_rate": 0.00018032287541882425,
+      "loss": 2.4405717849731445,
+      "step": 9154
+    },
+    {
+      "epoch": 1.2597754025165742,
+      "grad_norm": 0.026889760047197342,
+      "learning_rate": 0.00017910447761194028,
+      "loss": 2.429837226867676,
+      "step": 9156
+    },
+    {
+      "epoch": 1.260050588324325,
+      "grad_norm": 0.023671148344874382,
+      "learning_rate": 0.00017788607980505637,
+      "loss": 2.514474391937256,
+      "step": 9158
+    },
+    {
+      "epoch": 1.2603257741320755,
+      "grad_norm": 0.025050818920135498,
+      "learning_rate": 0.0001766676819981724,
+      "loss": 2.42932391166687,
+      "step": 9160
+    },
+    {
+      "epoch": 1.260600959939826,
+      "grad_norm": 0.02603321336209774,
+      "learning_rate": 0.00017544928419128847,
+      "loss": 2.4727368354797363,
+      "step": 9162
+    },
+    {
+      "epoch": 1.2608761457475768,
+      "grad_norm": 0.024021776393055916,
+      "learning_rate": 0.0001742308863844045,
+      "loss": 2.498426914215088,
+      "step": 9164
+    },
+    {
+      "epoch": 1.2611513315553273,
+      "grad_norm": 0.026128318160772324,
+      "learning_rate": 0.00017301248857752057,
+      "loss": 2.4605154991149902,
+      "step": 9166
+    },
+    {
+      "epoch": 1.2614265173630779,
+      "grad_norm": 0.03261302784085274,
+      "learning_rate": 0.00017179409077063663,
+      "loss": 2.476871967315674,
+      "step": 9168
+    },
+    {
+      "epoch": 1.2617017031708284,
+      "grad_norm": 0.030772998929023743,
+      "learning_rate": 0.00017057569296375266,
+      "loss": 2.4540953636169434,
+      "step": 9170
+    },
+    {
+      "epoch": 1.261976888978579,
+      "grad_norm": 0.06958162039518356,
+      "learning_rate": 0.00016935729515686872,
+      "loss": 2.463876962661743,
+      "step": 9172
+    },
+    {
+      "epoch": 1.2622520747863297,
+      "grad_norm": 0.02849862352013588,
+      "learning_rate": 0.0001681388973499848,
+      "loss": 2.458806037902832,
+      "step": 9174
+    },
+    {
+      "epoch": 1.2625272605940803,
+      "grad_norm": 0.08491968363523483,
+      "learning_rate": 0.00016692049954310082,
+      "loss": 2.4631829261779785,
+      "step": 9176
+    },
+    {
+      "epoch": 1.2628024464018308,
+      "grad_norm": 0.03385651856660843,
+      "learning_rate": 0.00016570210173621688,
+      "loss": 2.426635980606079,
+      "step": 9178
+    },
+    {
+      "epoch": 1.2630776322095816,
+      "grad_norm": 0.07140480726957321,
+      "learning_rate": 0.00016448370392933292,
+      "loss": 2.4901235103607178,
+      "step": 9180
+    },
+    {
+      "epoch": 1.2633528180173321,
+      "grad_norm": 0.047666873782873154,
+      "learning_rate": 0.00016326530612244898,
+      "loss": 2.446725845336914,
+      "step": 9182
+    },
+    {
+      "epoch": 1.2636280038250827,
+      "grad_norm": 0.03261895850300789,
+      "learning_rate": 0.00016204690831556504,
+      "loss": 2.4951424598693848,
+      "step": 9184
+    },
+    {
+      "epoch": 1.2639031896328334,
+      "grad_norm": 0.030796566978096962,
+      "learning_rate": 0.00016082851050868108,
+      "loss": 2.44705867767334,
+      "step": 9186
+    },
+    {
+      "epoch": 1.264178375440584,
+      "grad_norm": 0.0378551259636879,
+      "learning_rate": 0.00015961011270179714,
+      "loss": 2.4421229362487793,
+      "step": 9188
+    },
+    {
+      "epoch": 1.2644535612483345,
+      "grad_norm": 0.023401297628879547,
+      "learning_rate": 0.0001583917148949132,
+      "loss": 2.484905242919922,
+      "step": 9190
+    },
+    {
+      "epoch": 1.2647287470560853,
+      "grad_norm": 0.045987844467163086,
+      "learning_rate": 0.00015717331708802924,
+      "loss": 2.46445894241333,
+      "step": 9192
+    },
+    {
+      "epoch": 1.2650039328638358,
+      "grad_norm": 0.037334635853767395,
+      "learning_rate": 0.0001559549192811453,
+      "loss": 2.473769187927246,
+      "step": 9194
+    },
+    {
+      "epoch": 1.2652791186715864,
+      "grad_norm": 0.023572152480483055,
+      "learning_rate": 0.00015473652147426133,
+      "loss": 2.4687976837158203,
+      "step": 9196
+    },
+    {
+      "epoch": 1.265554304479337,
+      "grad_norm": 0.025871429592370987,
+      "learning_rate": 0.00015351812366737742,
+      "loss": 2.4440529346466064,
+      "step": 9198
+    },
+    {
+      "epoch": 1.2658294902870875,
+      "grad_norm": 0.02478305622935295,
+      "learning_rate": 0.00015229972586049346,
+      "loss": 2.4661784172058105,
+      "step": 9200
+    },
+    {
+      "epoch": 1.2661046760948382,
+      "grad_norm": 0.03008429892361164,
+      "learning_rate": 0.0001510813280536095,
+      "loss": 2.4998819828033447,
+      "step": 9202
+    },
+    {
+      "epoch": 1.2663798619025888,
+      "grad_norm": 0.02485612966120243,
+      "learning_rate": 0.00014986293024672555,
+      "loss": 2.4726576805114746,
+      "step": 9204
+    },
+    {
+      "epoch": 1.2666550477103393,
+      "grad_norm": 0.03864492103457451,
+      "learning_rate": 0.00014864453243984161,
+      "loss": 2.4271836280822754,
+      "step": 9206
+    },
+    {
+      "epoch": 1.26693023351809,
+      "grad_norm": 0.04841318726539612,
+      "learning_rate": 0.00014742613463295768,
+      "loss": 2.4692211151123047,
+      "step": 9208
+    },
+    {
+      "epoch": 1.2672054193258406,
+      "grad_norm": 0.025785375386476517,
+      "learning_rate": 0.0001462077368260737,
+      "loss": 2.42588472366333,
+      "step": 9210
+    },
+    {
+      "epoch": 1.2674806051335912,
+      "grad_norm": 0.026831820607185364,
+      "learning_rate": 0.00014498933901918975,
+      "loss": 2.4767918586730957,
+      "step": 9212
+    },
+    {
+      "epoch": 1.267755790941342,
+      "grad_norm": 0.024084411561489105,
+      "learning_rate": 0.00014377094121230583,
+      "loss": 2.467566967010498,
+      "step": 9214
+    },
+    {
+      "epoch": 1.2680309767490925,
+      "grad_norm": 0.024515077471733093,
+      "learning_rate": 0.00014255254340542187,
+      "loss": 2.482212543487549,
+      "step": 9216
+    },
+    {
+      "epoch": 1.268306162556843,
+      "grad_norm": 0.03350868821144104,
+      "learning_rate": 0.00014133414559853793,
+      "loss": 2.5129265785217285,
+      "step": 9218
+    },
+    {
+      "epoch": 1.2685813483645938,
+      "grad_norm": 0.0381435826420784,
+      "learning_rate": 0.000140115747791654,
+      "loss": 2.484919548034668,
+      "step": 9220
+    },
+    {
+      "epoch": 1.2688565341723443,
+      "grad_norm": 0.028340229764580727,
+      "learning_rate": 0.00013889734998477003,
+      "loss": 2.500598430633545,
+      "step": 9222
+    },
+    {
+      "epoch": 1.2691317199800949,
+      "grad_norm": 0.03274098411202431,
+      "learning_rate": 0.0001376789521778861,
+      "loss": 2.5282845497131348,
+      "step": 9224
+    },
+    {
+      "epoch": 1.2694069057878454,
+      "grad_norm": 0.03800109773874283,
+      "learning_rate": 0.00013646055437100213,
+      "loss": 2.450237274169922,
+      "step": 9226
+    },
+    {
+      "epoch": 1.269682091595596,
+      "grad_norm": 0.02317206934094429,
+      "learning_rate": 0.0001352421565641182,
+      "loss": 2.4409570693969727,
+      "step": 9228
+    },
+    {
+      "epoch": 1.2699572774033467,
+      "grad_norm": 0.04322657361626625,
+      "learning_rate": 0.00013402375875723425,
+      "loss": 2.4901251792907715,
+      "step": 9230
+    },
+    {
+      "epoch": 1.2702324632110973,
+      "grad_norm": 0.026108358055353165,
+      "learning_rate": 0.00013280536095035028,
+      "loss": 2.4575915336608887,
+      "step": 9232
+    },
+    {
+      "epoch": 1.2705076490188478,
+      "grad_norm": 0.023000361397862434,
+      "learning_rate": 0.00013158696314346635,
+      "loss": 2.4791340827941895,
+      "step": 9234
+    },
+    {
+      "epoch": 1.2707828348265986,
+      "grad_norm": 0.024312103167176247,
+      "learning_rate": 0.0001303685653365824,
+      "loss": 2.484363079071045,
+      "step": 9236
+    },
+    {
+      "epoch": 1.2710580206343491,
+      "grad_norm": 0.04383526369929314,
+      "learning_rate": 0.00012915016752969844,
+      "loss": 2.4615278244018555,
+      "step": 9238
+    },
+    {
+      "epoch": 1.2713332064420997,
+      "grad_norm": 0.05114690586924553,
+      "learning_rate": 0.0001279317697228145,
+      "loss": 2.4673848152160645,
+      "step": 9240
+    },
+    {
+      "epoch": 1.2716083922498504,
+      "grad_norm": 0.023235918954014778,
+      "learning_rate": 0.00012671337191593054,
+      "loss": 2.4590530395507812,
+      "step": 9242
+    },
+    {
+      "epoch": 1.271883578057601,
+      "grad_norm": 0.0435694120824337,
+      "learning_rate": 0.00012549497410904663,
+      "loss": 2.466273307800293,
+      "step": 9244
+    },
+    {
+      "epoch": 1.2721587638653515,
+      "grad_norm": 0.024943910539150238,
+      "learning_rate": 0.00012427657630216266,
+      "loss": 2.4613115787506104,
+      "step": 9246
+    },
+    {
+      "epoch": 1.2724339496731023,
+      "grad_norm": 0.032235994935035706,
+      "learning_rate": 0.00012305817849527872,
+      "loss": 2.4269895553588867,
+      "step": 9248
+    },
+    {
+      "epoch": 1.2727091354808528,
+      "grad_norm": 0.03350569307804108,
+      "learning_rate": 0.00012183978068839477,
+      "loss": 2.476099967956543,
+      "step": 9250
+    },
+    {
+      "epoch": 1.2729843212886034,
+      "grad_norm": 0.032456789165735245,
+      "learning_rate": 0.00012062138288151081,
+      "loss": 2.4684367179870605,
+      "step": 9252
+    },
+    {
+      "epoch": 1.273259507096354,
+      "grad_norm": 0.03041711077094078,
+      "learning_rate": 0.00011940298507462687,
+      "loss": 2.422231912612915,
+      "step": 9254
+    },
+    {
+      "epoch": 1.2735346929041047,
+      "grad_norm": 0.04057612270116806,
+      "learning_rate": 0.00011818458726774292,
+      "loss": 2.4237263202667236,
+      "step": 9256
+    },
+    {
+      "epoch": 1.2738098787118552,
+      "grad_norm": 0.02729097753763199,
+      "learning_rate": 0.00011696618946085898,
+      "loss": 2.469296932220459,
+      "step": 9258
+    },
+    {
+      "epoch": 1.2740850645196058,
+      "grad_norm": 0.025067254900932312,
+      "learning_rate": 0.00011574779165397503,
+      "loss": 2.4658350944519043,
+      "step": 9260
+    },
+    {
+      "epoch": 1.2743602503273563,
+      "grad_norm": 0.023453684523701668,
+      "learning_rate": 0.00011452939384709108,
+      "loss": 2.473017692565918,
+      "step": 9262
+    },
+    {
+      "epoch": 1.274635436135107,
+      "grad_norm": 0.02786240167915821,
+      "learning_rate": 0.00011331099604020712,
+      "loss": 2.438194751739502,
+      "step": 9264
+    },
+    {
+      "epoch": 1.2749106219428576,
+      "grad_norm": 0.025899238884449005,
+      "learning_rate": 0.00011209259823332319,
+      "loss": 2.4736762046813965,
+      "step": 9266
+    },
+    {
+      "epoch": 1.2751858077506082,
+      "grad_norm": 0.04977986589074135,
+      "learning_rate": 0.00011087420042643924,
+      "loss": 2.4508111476898193,
+      "step": 9268
+    },
+    {
+      "epoch": 1.275460993558359,
+      "grad_norm": 0.043027158826589584,
+      "learning_rate": 0.00010965580261955528,
+      "loss": 2.4447317123413086,
+      "step": 9270
+    },
+    {
+      "epoch": 1.2757361793661095,
+      "grad_norm": 0.025275692343711853,
+      "learning_rate": 0.00010843740481267133,
+      "loss": 2.4406440258026123,
+      "step": 9272
+    },
+    {
+      "epoch": 1.27601136517386,
+      "grad_norm": 0.02418424002826214,
+      "learning_rate": 0.0001072190070057874,
+      "loss": 2.4242236614227295,
+      "step": 9274
+    },
+    {
+      "epoch": 1.2762865509816108,
+      "grad_norm": 0.04770015925168991,
+      "learning_rate": 0.00010600060919890344,
+      "loss": 2.4857683181762695,
+      "step": 9276
+    },
+    {
+      "epoch": 1.2765617367893614,
+      "grad_norm": 0.026742637157440186,
+      "learning_rate": 0.0001047822113920195,
+      "loss": 2.469913959503174,
+      "step": 9278
+    },
+    {
+      "epoch": 1.276836922597112,
+      "grad_norm": 0.041335515677928925,
+      "learning_rate": 0.00010356381358513554,
+      "loss": 2.449284553527832,
+      "step": 9280
+    },
+    {
+      "epoch": 1.2771121084048627,
+      "grad_norm": 0.023468248546123505,
+      "learning_rate": 0.0001023454157782516,
+      "loss": 2.4724225997924805,
+      "step": 9282
+    },
+    {
+      "epoch": 1.2773872942126132,
+      "grad_norm": 0.04042794927954674,
+      "learning_rate": 0.00010112701797136765,
+      "loss": 2.5269744396209717,
+      "step": 9284
+    },
+    {
+      "epoch": 1.2776624800203638,
+      "grad_norm": 0.028787851333618164,
+      "learning_rate": 9.990862016448371e-05,
+      "loss": 2.4344067573547363,
+      "step": 9286
+    },
+    {
+      "epoch": 1.2779376658281143,
+      "grad_norm": 0.023643752560019493,
+      "learning_rate": 9.869022235759975e-05,
+      "loss": 2.445364475250244,
+      "step": 9288
+    },
+    {
+      "epoch": 1.2782128516358648,
+      "grad_norm": 0.024720333516597748,
+      "learning_rate": 9.747182455071581e-05,
+      "loss": 2.4627742767333984,
+      "step": 9290
+    },
+    {
+      "epoch": 1.2784880374436156,
+      "grad_norm": 0.02406024932861328,
+      "learning_rate": 9.625342674383186e-05,
+      "loss": 2.4599733352661133,
+      "step": 9292
+    },
+    {
+      "epoch": 1.2787632232513662,
+      "grad_norm": 0.035563528537750244,
+      "learning_rate": 9.503502893694792e-05,
+      "loss": 2.466968536376953,
+      "step": 9294
+    },
+    {
+      "epoch": 1.2790384090591167,
+      "grad_norm": 0.0645768791437149,
+      "learning_rate": 9.381663113006398e-05,
+      "loss": 2.5136094093322754,
+      "step": 9296
+    },
+    {
+      "epoch": 1.2793135948668675,
+      "grad_norm": 0.02829975076019764,
+      "learning_rate": 9.259823332318001e-05,
+      "loss": 2.456143856048584,
+      "step": 9298
+    },
+    {
+      "epoch": 1.279588780674618,
+      "grad_norm": 0.0350259393453598,
+      "learning_rate": 9.137983551629608e-05,
+      "loss": 2.4724559783935547,
+      "step": 9300
+    },
+    {
+      "epoch": 1.2798639664823686,
+      "grad_norm": 0.02330256812274456,
+      "learning_rate": 9.016143770941212e-05,
+      "loss": 2.4723682403564453,
+      "step": 9302
+    },
+    {
+      "epoch": 1.2801391522901193,
+      "grad_norm": 0.06514086574316025,
+      "learning_rate": 8.894303990252819e-05,
+      "loss": 2.441443920135498,
+      "step": 9304
+    },
+    {
+      "epoch": 1.2804143380978699,
+      "grad_norm": 0.02913660928606987,
+      "learning_rate": 8.772464209564423e-05,
+      "loss": 2.4984564781188965,
+      "step": 9306
+    },
+    {
+      "epoch": 1.2806895239056204,
+      "grad_norm": 0.04519667103886604,
+      "learning_rate": 8.650624428876028e-05,
+      "loss": 2.4247121810913086,
+      "step": 9308
+    },
+    {
+      "epoch": 1.2809647097133712,
+      "grad_norm": 0.030100347474217415,
+      "learning_rate": 8.528784648187633e-05,
+      "loss": 2.4700722694396973,
+      "step": 9310
+    },
+    {
+      "epoch": 1.2812398955211217,
+      "grad_norm": 0.04937539994716644,
+      "learning_rate": 8.40694486749924e-05,
+      "loss": 2.442837715148926,
+      "step": 9312
+    },
+    {
+      "epoch": 1.2815150813288723,
+      "grad_norm": 0.03726276010274887,
+      "learning_rate": 8.285105086810844e-05,
+      "loss": 2.4769034385681152,
+      "step": 9314
+    },
+    {
+      "epoch": 1.2817902671366228,
+      "grad_norm": 0.025645513087511063,
+      "learning_rate": 8.163265306122449e-05,
+      "loss": 2.41886043548584,
+      "step": 9316
+    },
+    {
+      "epoch": 1.2820654529443734,
+      "grad_norm": 0.059149716049432755,
+      "learning_rate": 8.041425525434054e-05,
+      "loss": 2.510690212249756,
+      "step": 9318
+    },
+    {
+      "epoch": 1.2823406387521241,
+      "grad_norm": 0.024304525926709175,
+      "learning_rate": 7.91958574474566e-05,
+      "loss": 2.474242687225342,
+      "step": 9320
+    },
+    {
+      "epoch": 1.2826158245598747,
+      "grad_norm": 0.02650417573750019,
+      "learning_rate": 7.797745964057265e-05,
+      "loss": 2.4739956855773926,
+      "step": 9322
+    },
+    {
+      "epoch": 1.2828910103676252,
+      "grad_norm": 0.024143308401107788,
+      "learning_rate": 7.675906183368871e-05,
+      "loss": 2.464813709259033,
+      "step": 9324
+    },
+    {
+      "epoch": 1.283166196175376,
+      "grad_norm": 0.02908219024538994,
+      "learning_rate": 7.554066402680475e-05,
+      "loss": 2.4559426307678223,
+      "step": 9326
+    },
+    {
+      "epoch": 1.2834413819831265,
+      "grad_norm": 0.028742872178554535,
+      "learning_rate": 7.432226621992081e-05,
+      "loss": 2.483405828475952,
+      "step": 9328
+    },
+    {
+      "epoch": 1.283716567790877,
+      "grad_norm": 0.025400971993803978,
+      "learning_rate": 7.310386841303686e-05,
+      "loss": 2.494812488555908,
+      "step": 9330
+    },
+    {
+      "epoch": 1.2839917535986278,
+      "grad_norm": 0.03265144303441048,
+      "learning_rate": 7.188547060615292e-05,
+      "loss": 2.4260623455047607,
+      "step": 9332
+    },
+    {
+      "epoch": 1.2842669394063784,
+      "grad_norm": 0.02289043366909027,
+      "learning_rate": 7.066707279926897e-05,
+      "loss": 2.4652228355407715,
+      "step": 9334
+    },
+    {
+      "epoch": 1.284542125214129,
+      "grad_norm": 0.024191470816731453,
+      "learning_rate": 6.944867499238501e-05,
+      "loss": 2.534149646759033,
+      "step": 9336
+    },
+    {
+      "epoch": 1.2848173110218797,
+      "grad_norm": 0.02270405925810337,
+      "learning_rate": 6.823027718550106e-05,
+      "loss": 2.442457675933838,
+      "step": 9338
+    },
+    {
+      "epoch": 1.2850924968296302,
+      "grad_norm": 0.02323989011347294,
+      "learning_rate": 6.701187937861712e-05,
+      "loss": 2.4669671058654785,
+      "step": 9340
+    },
+    {
+      "epoch": 1.2853676826373808,
+      "grad_norm": 0.026859449222683907,
+      "learning_rate": 6.579348157173317e-05,
+      "loss": 2.480384349822998,
+      "step": 9342
+    },
+    {
+      "epoch": 1.2856428684451313,
+      "grad_norm": 0.029326442629098892,
+      "learning_rate": 6.457508376484922e-05,
+      "loss": 2.460057020187378,
+      "step": 9344
+    },
+    {
+      "epoch": 1.285918054252882,
+      "grad_norm": 0.022145941853523254,
+      "learning_rate": 6.335668595796527e-05,
+      "loss": 2.431990146636963,
+      "step": 9346
+    },
+    {
+      "epoch": 1.2861932400606326,
+      "grad_norm": 0.026378050446510315,
+      "learning_rate": 6.213828815108133e-05,
+      "loss": 2.4867806434631348,
+      "step": 9348
+    },
+    {
+      "epoch": 1.2864684258683832,
+      "grad_norm": 0.03834126144647598,
+      "learning_rate": 6.0919890344197386e-05,
+      "loss": 2.4674935340881348,
+      "step": 9350
+    },
+    {
+      "epoch": 1.2867436116761337,
+      "grad_norm": 0.027719242498278618,
+      "learning_rate": 5.9701492537313435e-05,
+      "loss": 2.4642841815948486,
+      "step": 9352
+    },
+    {
+      "epoch": 1.2870187974838845,
+      "grad_norm": 0.031444255262613297,
+      "learning_rate": 5.848309473042949e-05,
+      "loss": 2.471680164337158,
+      "step": 9354
+    },
+    {
+      "epoch": 1.287293983291635,
+      "grad_norm": 0.03111363761126995,
+      "learning_rate": 5.726469692354554e-05,
+      "loss": 2.4536776542663574,
+      "step": 9356
+    },
+    {
+      "epoch": 1.2875691690993856,
+      "grad_norm": 0.02274118922650814,
+      "learning_rate": 5.604629911666159e-05,
+      "loss": 2.437633991241455,
+      "step": 9358
+    },
+    {
+      "epoch": 1.2878443549071363,
+      "grad_norm": 0.032064806669950485,
+      "learning_rate": 5.482790130977764e-05,
+      "loss": 2.4505927562713623,
+      "step": 9360
+    },
+    {
+      "epoch": 1.2881195407148869,
+      "grad_norm": 0.022799208760261536,
+      "learning_rate": 5.36095035028937e-05,
+      "loss": 2.4661970138549805,
+      "step": 9362
+    },
+    {
+      "epoch": 1.2883947265226374,
+      "grad_norm": 0.03112195059657097,
+      "learning_rate": 5.239110569600975e-05,
+      "loss": 2.4658312797546387,
+      "step": 9364
+    },
+    {
+      "epoch": 1.2886699123303882,
+      "grad_norm": 0.057028234004974365,
+      "learning_rate": 5.11727078891258e-05,
+      "loss": 2.4279375076293945,
+      "step": 9366
+    },
+    {
+      "epoch": 1.2889450981381387,
+      "grad_norm": 0.051933858543634415,
+      "learning_rate": 4.9954310082241855e-05,
+      "loss": 2.464329242706299,
+      "step": 9368
+    },
+    {
+      "epoch": 1.2892202839458893,
+      "grad_norm": 0.02488640509545803,
+      "learning_rate": 4.8735912275357904e-05,
+      "loss": 2.4464011192321777,
+      "step": 9370
+    },
+    {
+      "epoch": 1.2894954697536398,
+      "grad_norm": 0.033568259328603745,
+      "learning_rate": 4.751751446847396e-05,
+      "loss": 2.4866280555725098,
+      "step": 9372
+    },
+    {
+      "epoch": 1.2897706555613906,
+      "grad_norm": 0.03608524054288864,
+      "learning_rate": 4.629911666159001e-05,
+      "loss": 2.47916316986084,
+      "step": 9374
+    },
+    {
+      "epoch": 1.2900458413691411,
+      "grad_norm": 0.029297254979610443,
+      "learning_rate": 4.508071885470606e-05,
+      "loss": 2.450876235961914,
+      "step": 9376
+    },
+    {
+      "epoch": 1.2903210271768917,
+      "grad_norm": 0.02411983162164688,
+      "learning_rate": 4.386232104782212e-05,
+      "loss": 2.4418458938598633,
+      "step": 9378
+    },
+    {
+      "epoch": 1.2905962129846422,
+      "grad_norm": 0.028337502852082253,
+      "learning_rate": 4.2643923240938166e-05,
+      "loss": 2.466611623764038,
+      "step": 9380
+    },
+    {
+      "epoch": 1.290871398792393,
+      "grad_norm": 0.023473840206861496,
+      "learning_rate": 4.142552543405422e-05,
+      "loss": 2.4249253273010254,
+      "step": 9382
+    },
+    {
+      "epoch": 1.2911465846001435,
+      "grad_norm": 0.026947252452373505,
+      "learning_rate": 4.020712762717027e-05,
+      "loss": 2.5074667930603027,
+      "step": 9384
+    },
+    {
+      "epoch": 1.291421770407894,
+      "grad_norm": 0.024754062294960022,
+      "learning_rate": 3.8988729820286324e-05,
+      "loss": 2.4222748279571533,
+      "step": 9386
+    },
+    {
+      "epoch": 1.2916969562156448,
+      "grad_norm": 0.023064929991960526,
+      "learning_rate": 3.777033201340237e-05,
+      "loss": 2.486654758453369,
+      "step": 9388
+    },
+    {
+      "epoch": 1.2919721420233954,
+      "grad_norm": 0.02911621704697609,
+      "learning_rate": 3.655193420651843e-05,
+      "loss": 2.4243130683898926,
+      "step": 9390
+    },
+    {
+      "epoch": 1.292247327831146,
+      "grad_norm": 0.02474200911819935,
+      "learning_rate": 3.533353639963448e-05,
+      "loss": 2.4485673904418945,
+      "step": 9392
+    },
+    {
+      "epoch": 1.2925225136388967,
+      "grad_norm": 0.02235531620681286,
+      "learning_rate": 3.411513859275053e-05,
+      "loss": 2.4671630859375,
+      "step": 9394
+    },
+    {
+      "epoch": 1.2927976994466472,
+      "grad_norm": 0.025915909558534622,
+      "learning_rate": 3.2896740785866586e-05,
+      "loss": 2.4600918292999268,
+      "step": 9396
+    },
+    {
+      "epoch": 1.2930728852543978,
+      "grad_norm": 0.024452533572912216,
+      "learning_rate": 3.1678342978982635e-05,
+      "loss": 2.438499927520752,
+      "step": 9398
+    },
+    {
+      "epoch": 1.2933480710621486,
+      "grad_norm": 0.022966155782341957,
+      "learning_rate": 3.0459945172098693e-05,
+      "loss": 2.444155216217041,
+      "step": 9400
+    },
+    {
+      "epoch": 1.293623256869899,
+      "grad_norm": 0.023288989439606667,
+      "learning_rate": 2.9241547365214745e-05,
+      "loss": 2.4615085124969482,
+      "step": 9402
+    },
+    {
+      "epoch": 1.2938984426776496,
+      "grad_norm": 0.02599608525633812,
+      "learning_rate": 2.8023149558330797e-05,
+      "loss": 2.471071481704712,
+      "step": 9404
+    },
+    {
+      "epoch": 1.2941736284854002,
+      "grad_norm": 0.023484863340854645,
+      "learning_rate": 2.680475175144685e-05,
+      "loss": 2.492124557495117,
+      "step": 9406
+    },
+    {
+      "epoch": 1.2944488142931507,
+      "grad_norm": 0.02862633764743805,
+      "learning_rate": 2.55863539445629e-05,
+      "loss": 2.4857335090637207,
+      "step": 9408
+    },
+    {
+      "epoch": 1.2947240001009015,
+      "grad_norm": 0.023178761824965477,
+      "learning_rate": 2.4367956137678952e-05,
+      "loss": 2.4644484519958496,
+      "step": 9410
+    },
+    {
+      "epoch": 1.294999185908652,
+      "grad_norm": 0.024128887802362442,
+      "learning_rate": 2.3149558330795004e-05,
+      "loss": 2.4657139778137207,
+      "step": 9412
+    },
+    {
+      "epoch": 1.2952743717164026,
+      "grad_norm": 0.02320338971912861,
+      "learning_rate": 2.193116052391106e-05,
+      "loss": 2.43255877494812,
+      "step": 9414
+    },
+    {
+      "epoch": 1.2955495575241534,
+      "grad_norm": 0.027649955824017525,
+      "learning_rate": 2.071276271702711e-05,
+      "loss": 2.4375805854797363,
+      "step": 9416
+    },
+    {
+      "epoch": 1.295824743331904,
+      "grad_norm": 0.028232116252183914,
+      "learning_rate": 1.9494364910143162e-05,
+      "loss": 2.473940849304199,
+      "step": 9418
+    },
+    {
+      "epoch": 1.2960999291396544,
+      "grad_norm": 0.028577309101819992,
+      "learning_rate": 1.8275967103259214e-05,
+      "loss": 2.453368663787842,
+      "step": 9420
+    },
+    {
+      "epoch": 1.2963751149474052,
+      "grad_norm": 0.027163874357938766,
+      "learning_rate": 1.7057569296375266e-05,
+      "loss": 2.521793842315674,
+      "step": 9422
+    },
+    {
+      "epoch": 1.2966503007551557,
+      "grad_norm": 0.02428358420729637,
+      "learning_rate": 1.5839171489491317e-05,
+      "loss": 2.457825183868408,
+      "step": 9424
+    },
+    {
+      "epoch": 1.2969254865629063,
+      "grad_norm": 0.030103236436843872,
+      "learning_rate": 1.4620773682607372e-05,
+      "loss": 2.4823217391967773,
+      "step": 9426
+    },
+    {
+      "epoch": 1.297200672370657,
+      "grad_norm": 0.023016395047307014,
+      "learning_rate": 1.3402375875723424e-05,
+      "loss": 2.475515842437744,
+      "step": 9428
+    },
+    {
+      "epoch": 1.2974758581784076,
+      "grad_norm": 0.02249881997704506,
+      "learning_rate": 1.2183978068839476e-05,
+      "loss": 2.429299831390381,
+      "step": 9430
+    },
+    {
+      "epoch": 1.2977510439861581,
+      "grad_norm": 0.02431398257613182,
+      "learning_rate": 1.096558026195553e-05,
+      "loss": 2.4118399620056152,
+      "step": 9432
+    },
+    {
+      "epoch": 1.2980262297939087,
+      "grad_norm": 0.023900052532553673,
+      "learning_rate": 9.747182455071581e-06,
+      "loss": 2.4317069053649902,
+      "step": 9434
+    },
+    {
+      "epoch": 1.2983014156016592,
+      "grad_norm": 0.022661179304122925,
+      "learning_rate": 8.528784648187633e-06,
+      "loss": 2.464376926422119,
+      "step": 9436
+    },
+    {
+      "epoch": 1.29857660140941,
+      "grad_norm": 0.02387865260243416,
+      "learning_rate": 7.310386841303686e-06,
+      "loss": 2.4512503147125244,
+      "step": 9438
+    },
+    {
+      "epoch": 1.2988517872171605,
+      "grad_norm": 0.025546187534928322,
+      "learning_rate": 6.091989034419738e-06,
+      "loss": 2.4238765239715576,
+      "step": 9440
+    },
+    {
+      "epoch": 1.299126973024911,
+      "grad_norm": 0.025863472372293472,
+      "learning_rate": 4.8735912275357905e-06,
+      "loss": 2.452146053314209,
+      "step": 9442
+    },
+    {
+      "epoch": 1.2994021588326619,
+      "grad_norm": 0.03757040947675705,
+      "learning_rate": 3.655193420651843e-06,
+      "loss": 2.4905474185943604,
+      "step": 9444
+    },
+    {
+      "epoch": 1.2996773446404124,
+      "grad_norm": 0.023091528564691544,
+      "learning_rate": 2.4367956137678953e-06,
+      "loss": 2.4607601165771484,
+      "step": 9446
+    },
+    {
+      "epoch": 1.299952530448163,
+      "grad_norm": 0.026085583493113518,
+      "learning_rate": 1.2183978068839476e-06,
+      "loss": 2.480759620666504,
+      "step": 9448
+    },
+    {
+      "epoch": 1.3000901233520383,
+      "eval_loss": 2.497500419616699,
+      "eval_runtime": 379.6263,
+      "eval_samples_per_second": 92.636,
+      "eval_steps_per_second": 11.58,
+      "step": 9449
     }
   ],
   "logging_steps": 2,
         "should_evaluate": false,
         "should_log": false,
         "should_save": true,
+        "should_training_stop": true
       },
       "attributes": {}
     }
   },
+  "total_flos": 3.536902454968425e+19,
   "train_batch_size": 4,
   "trial_name": null,
   "trial_params": null