Instructions to use Ba2han/outputs with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ba2han/outputs with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ba2han/outputs")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ba2han/outputs")
model = AutoModelForCausalLM.from_pretrained("Ba2han/outputs")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ba2han/outputs with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ba2han/outputs"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/outputs",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Ba2han/outputs

SGLang

How to use Ba2han/outputs with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ba2han/outputs" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/outputs",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ba2han/outputs" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/outputs",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Unsloth Studio

How to use Ba2han/outputs with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/outputs to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/outputs to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ba2han/outputs to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Ba2han/outputs",
    max_seq_length=2048,
)

Docker Model Runner
How to use Ba2han/outputs with Docker Model Runner:
```
docker model run hf.co/Ba2han/outputs
```

Ba2han commited on 7 days ago

Commit

6aba132

verified ·

1 Parent(s): de9b82a

Training in progress, step 13200, checkpoint

Browse files

Files changed (5) hide show

last-checkpoint/model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/rng_state.pth +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +2321 -3

last-checkpoint/model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c3ebb0f35c41f687e9a55faebf52240856b40e8eb420901e4280d18e704a4e0
 size 2519929824

 version https://git-lfs.github.com/spec/v1
+oid sha256:9b0a561555fc3172580adceb809072bc2846a44d5e59981c233b4c0d395b7c2f
 size 2519929824

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:550e376976b9a10175ee0ec9ef0304aee2d3be924e37e1db77cf85ba662b301b
 size 1263881876

 version https://git-lfs.github.com/spec/v1
+oid sha256:5dfb13af2e1d5e3db594d283c34158f6db98d73878f28ba1428103efa7bde6e2
 size 1263881876

last-checkpoint/rng_state.pth CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:3d6d8fafcd1ee268414be5acf0366296af5b03d60871978712eac1979cb42d65
 size 14645

 version https://git-lfs.github.com/spec/v1
+oid sha256:89645f9f6a2a8823b334e0748e4f5ea683a27b3d02632f0a241397db55d82fad
 size 14645

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5d2c351b1644587f4ceb0777c6b12c5e2f72cef708c9c6805a96b66f20f29e08
 size 10233

 version https://git-lfs.github.com/spec/v1
+oid sha256:7f17bcfdee39118440e0416fa26db9754f1eab2d3bbc8cc308f52f610c157679
 size 10233

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,9 +2,9 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.76,
   "eval_steps": 3300,
-  "global_step": 12540,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -43922,6 +43922,2324 @@
       "learning_rate": 0.0008,
       "loss": 1.9308580160140991,
       "step": 12540
     }
   ],
   "logging_steps": 2,
@@ -43941,7 +46259,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 3.001708175932253e+19,
   "train_batch_size": 20,
   "trial_name": null,
   "trial_params": null

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 0.8,
   "eval_steps": 3300,
+  "global_step": 13200,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "learning_rate": 0.0008,
       "loss": 1.9308580160140991,
       "step": 12540
+    },
+    {
+      "epoch": 0.7601212121212121,
+      "grad_norm": 0.04483119770884514,
+      "learning_rate": 0.0007997979797979798,
+      "loss": 2.0346462726593018,
+      "step": 12542
+    },
+    {
+      "epoch": 0.7602424242424243,
+      "grad_norm": 0.040438830852508545,
+      "learning_rate": 0.0007993939393939395,
+      "loss": 1.979292392730713,
+      "step": 12544
+    },
+    {
+      "epoch": 0.7603636363636364,
+      "grad_norm": 0.04930206760764122,
+      "learning_rate": 0.000798989898989899,
+      "loss": 2.0374951362609863,
+      "step": 12546
+    },
+    {
+      "epoch": 0.7604848484848484,
+      "grad_norm": 0.04073861986398697,
+      "learning_rate": 0.0007985858585858586,
+      "loss": 1.9666820764541626,
+      "step": 12548
+    },
+    {
+      "epoch": 0.7606060606060606,
+      "grad_norm": 0.04052956774830818,
+      "learning_rate": 0.0007981818181818182,
+      "loss": 1.9964886903762817,
+      "step": 12550
+    },
+    {
+      "epoch": 0.7607272727272727,
+      "grad_norm": 0.038861118257045746,
+      "learning_rate": 0.0007977777777777778,
+      "loss": 2.013064384460449,
+      "step": 12552
+    },
+    {
+      "epoch": 0.7608484848484849,
+      "grad_norm": 0.04100293666124344,
+      "learning_rate": 0.0007973737373737374,
+      "loss": 2.008998155593872,
+      "step": 12554
+    },
+    {
+      "epoch": 0.760969696969697,
+      "grad_norm": 0.042416807264089584,
+      "learning_rate": 0.000796969696969697,
+      "loss": 1.9424998760223389,
+      "step": 12556
+    },
+    {
+      "epoch": 0.761090909090909,
+      "grad_norm": 0.040263108909130096,
+      "learning_rate": 0.0007965656565656566,
+      "loss": 2.018378973007202,
+      "step": 12558
+    },
+    {
+      "epoch": 0.7612121212121212,
+      "grad_norm": 0.04443824291229248,
+      "learning_rate": 0.0007961616161616162,
+      "loss": 2.00608491897583,
+      "step": 12560
+    },
+    {
+      "epoch": 0.7613333333333333,
+      "grad_norm": 0.0416722372174263,
+      "learning_rate": 0.0007957575757575759,
+      "loss": 2.014683246612549,
+      "step": 12562
+    },
+    {
+      "epoch": 0.7614545454545455,
+      "grad_norm": 0.042002227157354355,
+      "learning_rate": 0.0007953535353535354,
+      "loss": 1.984579086303711,
+      "step": 12564
+    },
+    {
+      "epoch": 0.7615757575757576,
+      "grad_norm": 0.0397065132856369,
+      "learning_rate": 0.000794949494949495,
+      "loss": 1.9998563528060913,
+      "step": 12566
+    },
+    {
+      "epoch": 0.7616969696969697,
+      "grad_norm": 0.04016793519258499,
+      "learning_rate": 0.0007945454545454546,
+      "loss": 2.0625665187835693,
+      "step": 12568
+    },
+    {
+      "epoch": 0.7618181818181818,
+      "grad_norm": 0.04011380672454834,
+      "learning_rate": 0.0007941414141414142,
+      "loss": 2.0061252117156982,
+      "step": 12570
+    },
+    {
+      "epoch": 0.7619393939393939,
+      "grad_norm": 0.05220978707075119,
+      "learning_rate": 0.0007937373737373737,
+      "loss": 2.0691137313842773,
+      "step": 12572
+    },
+    {
+      "epoch": 0.7620606060606061,
+      "grad_norm": 0.047436244785785675,
+      "learning_rate": 0.0007933333333333334,
+      "loss": 2.090104818344116,
+      "step": 12574
+    },
+    {
+      "epoch": 0.7621818181818182,
+      "grad_norm": 0.0454322025179863,
+      "learning_rate": 0.000792929292929293,
+      "loss": 2.0175178050994873,
+      "step": 12576
+    },
+    {
+      "epoch": 0.7623030303030303,
+      "grad_norm": 0.04178448021411896,
+      "learning_rate": 0.0007925252525252525,
+      "loss": 2.065824270248413,
+      "step": 12578
+    },
+    {
+      "epoch": 0.7624242424242424,
+      "grad_norm": 0.04005878046154976,
+      "learning_rate": 0.0007921212121212121,
+      "loss": 2.117156982421875,
+      "step": 12580
+    },
+    {
+      "epoch": 0.7625454545454545,
+      "grad_norm": 0.04271911829710007,
+      "learning_rate": 0.0007917171717171718,
+      "loss": 2.119072675704956,
+      "step": 12582
+    },
+    {
+      "epoch": 0.7626666666666667,
+      "grad_norm": 0.041054822504520416,
+      "learning_rate": 0.0007913131313131314,
+      "loss": 2.0002260208129883,
+      "step": 12584
+    },
+    {
+      "epoch": 0.7627878787878788,
+      "grad_norm": 0.04326825216412544,
+      "learning_rate": 0.0007909090909090909,
+      "loss": 2.062181234359741,
+      "step": 12586
+    },
+    {
+      "epoch": 0.7629090909090909,
+      "grad_norm": 0.04097198694944382,
+      "learning_rate": 0.0007905050505050505,
+      "loss": 2.1092638969421387,
+      "step": 12588
+    },
+    {
+      "epoch": 0.7630303030303031,
+      "grad_norm": 0.04077252373099327,
+      "learning_rate": 0.0007901010101010102,
+      "loss": 2.071500539779663,
+      "step": 12590
+    },
+    {
+      "epoch": 0.7631515151515151,
+      "grad_norm": 0.043130528181791306,
+      "learning_rate": 0.0007896969696969697,
+      "loss": 2.1335792541503906,
+      "step": 12592
+    },
+    {
+      "epoch": 0.7632727272727273,
+      "grad_norm": 0.043429844081401825,
+      "learning_rate": 0.0007892929292929293,
+      "loss": 2.0660293102264404,
+      "step": 12594
+    },
+    {
+      "epoch": 0.7633939393939394,
+      "grad_norm": 0.04609135910868645,
+      "learning_rate": 0.000788888888888889,
+      "loss": 2.182990550994873,
+      "step": 12596
+    },
+    {
+      "epoch": 0.7635151515151515,
+      "grad_norm": 0.04338185489177704,
+      "learning_rate": 0.0007884848484848486,
+      "loss": 2.155475378036499,
+      "step": 12598
+    },
+    {
+      "epoch": 0.7636363636363637,
+      "grad_norm": 0.042257003486156464,
+      "learning_rate": 0.0007880808080808081,
+      "loss": 2.0997626781463623,
+      "step": 12600
+    },
+    {
+      "epoch": 0.7637575757575757,
+      "grad_norm": 0.040354736149311066,
+      "learning_rate": 0.0007876767676767677,
+      "loss": 2.129809856414795,
+      "step": 12602
+    },
+    {
+      "epoch": 0.7638787878787878,
+      "grad_norm": 0.04482565447688103,
+      "learning_rate": 0.0007872727272727274,
+      "loss": 2.1358137130737305,
+      "step": 12604
+    },
+    {
+      "epoch": 0.764,
+      "grad_norm": 0.04288671910762787,
+      "learning_rate": 0.0007868686868686868,
+      "loss": 2.142606735229492,
+      "step": 12606
+    },
+    {
+      "epoch": 0.7641212121212121,
+      "grad_norm": 0.043324846774339676,
+      "learning_rate": 0.0007864646464646465,
+      "loss": 2.175297498703003,
+      "step": 12608
+    },
+    {
+      "epoch": 0.7642424242424243,
+      "grad_norm": 0.04380409047007561,
+      "learning_rate": 0.0007860606060606061,
+      "loss": 2.071028232574463,
+      "step": 12610
+    },
+    {
+      "epoch": 0.7643636363636364,
+      "grad_norm": 0.04719284176826477,
+      "learning_rate": 0.0007856565656565657,
+      "loss": 2.1706879138946533,
+      "step": 12612
+    },
+    {
+      "epoch": 0.7644848484848484,
+      "grad_norm": 0.052769482135772705,
+      "learning_rate": 0.0007852525252525252,
+      "loss": 2.2532317638397217,
+      "step": 12614
+    },
+    {
+      "epoch": 0.7646060606060606,
+      "grad_norm": 0.0526919886469841,
+      "learning_rate": 0.0007848484848484849,
+      "loss": 2.216707944869995,
+      "step": 12616
+    },
+    {
+      "epoch": 0.7647272727272727,
+      "grad_norm": 0.05870664864778519,
+      "learning_rate": 0.0007844444444444445,
+      "loss": 2.2171475887298584,
+      "step": 12618
+    },
+    {
+      "epoch": 0.7648484848484849,
+      "grad_norm": 0.04490828141570091,
+      "learning_rate": 0.0007840404040404041,
+      "loss": 2.140615940093994,
+      "step": 12620
+    },
+    {
+      "epoch": 0.764969696969697,
+      "grad_norm": 0.04558146744966507,
+      "learning_rate": 0.0007836363636363636,
+      "loss": 2.2159841060638428,
+      "step": 12622
+    },
+    {
+      "epoch": 0.765090909090909,
+      "grad_norm": 0.06300142407417297,
+      "learning_rate": 0.0007832323232323233,
+      "loss": 2.253589391708374,
+      "step": 12624
+    },
+    {
+      "epoch": 0.7652121212121212,
+      "grad_norm": 0.056389063596725464,
+      "learning_rate": 0.0007828282828282828,
+      "loss": 2.1611223220825195,
+      "step": 12626
+    },
+    {
+      "epoch": 0.7653333333333333,
+      "grad_norm": 0.05626522749662399,
+      "learning_rate": 0.0007824242424242425,
+      "loss": 2.1815221309661865,
+      "step": 12628
+    },
+    {
+      "epoch": 0.7654545454545455,
+      "grad_norm": 0.04235878214240074,
+      "learning_rate": 0.0007820202020202021,
+      "loss": 2.205601692199707,
+      "step": 12630
+    },
+    {
+      "epoch": 0.7655757575757576,
+      "grad_norm": 0.0455097034573555,
+      "learning_rate": 0.0007816161616161616,
+      "loss": 2.1855411529541016,
+      "step": 12632
+    },
+    {
+      "epoch": 0.7656969696969697,
+      "grad_norm": 0.042771559208631516,
+      "learning_rate": 0.0007812121212121213,
+      "loss": 2.126619577407837,
+      "step": 12634
+    },
+    {
+      "epoch": 0.7658181818181818,
+      "grad_norm": 0.047885436564683914,
+      "learning_rate": 0.0007808080808080808,
+      "loss": 2.224959373474121,
+      "step": 12636
+    },
+    {
+      "epoch": 0.7659393939393939,
+      "grad_norm": 0.044399600476026535,
+      "learning_rate": 0.0007804040404040405,
+      "loss": 2.187551498413086,
+      "step": 12638
+    },
+    {
+      "epoch": 0.7660606060606061,
+      "grad_norm": 0.04438615217804909,
+      "learning_rate": 0.00078,
+      "loss": 2.261439561843872,
+      "step": 12640
+    },
+    {
+      "epoch": 0.7661818181818182,
+      "grad_norm": 0.06503322720527649,
+      "learning_rate": 0.0007795959595959597,
+      "loss": 2.2749171257019043,
+      "step": 12642
+    },
+    {
+      "epoch": 0.7663030303030303,
+      "grad_norm": 0.0585797056555748,
+      "learning_rate": 0.0007791919191919192,
+      "loss": 2.209646701812744,
+      "step": 12644
+    },
+    {
+      "epoch": 0.7664242424242425,
+      "grad_norm": 0.04714217782020569,
+      "learning_rate": 0.0007787878787878788,
+      "loss": 2.196881055831909,
+      "step": 12646
+    },
+    {
+      "epoch": 0.7665454545454545,
+      "grad_norm": 0.041872140020132065,
+      "learning_rate": 0.0007783838383838384,
+      "loss": 2.188270330429077,
+      "step": 12648
+    },
+    {
+      "epoch": 0.7666666666666667,
+      "grad_norm": 0.04126197099685669,
+      "learning_rate": 0.0007779797979797981,
+      "loss": 2.2716634273529053,
+      "step": 12650
+    },
+    {
+      "epoch": 0.7667878787878788,
+      "grad_norm": 0.04184809327125549,
+      "learning_rate": 0.0007775757575757576,
+      "loss": 2.245332717895508,
+      "step": 12652
+    },
+    {
+      "epoch": 0.7669090909090909,
+      "grad_norm": 0.04115922003984451,
+      "learning_rate": 0.0007771717171717172,
+      "loss": 2.2641806602478027,
+      "step": 12654
+    },
+    {
+      "epoch": 0.7670303030303031,
+      "grad_norm": 0.046709779649972916,
+      "learning_rate": 0.0007767676767676768,
+      "loss": 2.31485652923584,
+      "step": 12656
+    },
+    {
+      "epoch": 0.7671515151515151,
+      "grad_norm": 0.04386667162179947,
+      "learning_rate": 0.0007763636363636364,
+      "loss": 2.274606227874756,
+      "step": 12658
+    },
+    {
+      "epoch": 0.7672727272727272,
+      "grad_norm": 0.04100430756807327,
+      "learning_rate": 0.0007759595959595959,
+      "loss": 2.2658705711364746,
+      "step": 12660
+    },
+    {
+      "epoch": 0.7673939393939394,
+      "grad_norm": 0.04348524287343025,
+      "learning_rate": 0.0007755555555555556,
+      "loss": 2.2559895515441895,
+      "step": 12662
+    },
+    {
+      "epoch": 0.7675151515151515,
+      "grad_norm": 0.044867586344480515,
+      "learning_rate": 0.0007751515151515153,
+      "loss": 2.3376922607421875,
+      "step": 12664
+    },
+    {
+      "epoch": 0.7676363636363637,
+      "grad_norm": 0.04329673945903778,
+      "learning_rate": 0.0007747474747474747,
+      "loss": 2.2448296546936035,
+      "step": 12666
+    },
+    {
+      "epoch": 0.7677575757575757,
+      "grad_norm": 0.0431828536093235,
+      "learning_rate": 0.0007743434343434344,
+      "loss": 2.2692770957946777,
+      "step": 12668
+    },
+    {
+      "epoch": 0.7678787878787878,
+      "grad_norm": 0.045329418033361435,
+      "learning_rate": 0.000773939393939394,
+      "loss": 2.297987222671509,
+      "step": 12670
+    },
+    {
+      "epoch": 0.768,
+      "grad_norm": 0.05611593648791313,
+      "learning_rate": 0.0007735353535353536,
+      "loss": 2.3240864276885986,
+      "step": 12672
+    },
+    {
+      "epoch": 0.7681212121212121,
+      "grad_norm": 0.04655918478965759,
+      "learning_rate": 0.0007731313131313131,
+      "loss": 2.343942642211914,
+      "step": 12674
+    },
+    {
+      "epoch": 0.7682424242424243,
+      "grad_norm": 0.04309716075658798,
+      "learning_rate": 0.0007727272727272728,
+      "loss": 2.3185503482818604,
+      "step": 12676
+    },
+    {
+      "epoch": 0.7683636363636364,
+      "grad_norm": 0.04363667964935303,
+      "learning_rate": 0.0007723232323232324,
+      "loss": 2.269561290740967,
+      "step": 12678
+    },
+    {
+      "epoch": 0.7684848484848484,
+      "grad_norm": 0.04190044850111008,
+      "learning_rate": 0.0007719191919191919,
+      "loss": 2.331587314605713,
+      "step": 12680
+    },
+    {
+      "epoch": 0.7686060606060606,
+      "grad_norm": 0.04826276749372482,
+      "learning_rate": 0.0007715151515151515,
+      "loss": 2.2813880443573,
+      "step": 12682
+    },
+    {
+      "epoch": 0.7687272727272727,
+      "grad_norm": 0.04747844859957695,
+      "learning_rate": 0.0007711111111111112,
+      "loss": 2.3475375175476074,
+      "step": 12684
+    },
+    {
+      "epoch": 0.7688484848484849,
+      "grad_norm": 0.052638374269008636,
+      "learning_rate": 0.0007707070707070708,
+      "loss": 2.321424961090088,
+      "step": 12686
+    },
+    {
+      "epoch": 0.768969696969697,
+      "grad_norm": 0.06004733592271805,
+      "learning_rate": 0.0007703030303030303,
+      "loss": 2.3726699352264404,
+      "step": 12688
+    },
+    {
+      "epoch": 0.769090909090909,
+      "grad_norm": 0.05712248384952545,
+      "learning_rate": 0.0007698989898989899,
+      "loss": 2.3676395416259766,
+      "step": 12690
+    },
+    {
+      "epoch": 0.7692121212121212,
+      "grad_norm": 0.0554472990334034,
+      "learning_rate": 0.0007694949494949496,
+      "loss": 2.3520894050598145,
+      "step": 12692
+    },
+    {
+      "epoch": 0.7693333333333333,
+      "grad_norm": 0.05357610434293747,
+      "learning_rate": 0.000769090909090909,
+      "loss": 2.3988537788391113,
+      "step": 12694
+    },
+    {
+      "epoch": 0.7694545454545455,
+      "grad_norm": 0.05547565594315529,
+      "learning_rate": 0.0007686868686868687,
+      "loss": 2.3897652626037598,
+      "step": 12696
+    },
+    {
+      "epoch": 0.7695757575757576,
+      "grad_norm": 0.04544910416007042,
+      "learning_rate": 0.0007682828282828284,
+      "loss": 2.364074230194092,
+      "step": 12698
+    },
+    {
+      "epoch": 0.7696969696969697,
+      "grad_norm": 0.045219916850328445,
+      "learning_rate": 0.0007678787878787879,
+      "loss": 2.371718406677246,
+      "step": 12700
+    },
+    {
+      "epoch": 0.7698181818181818,
+      "grad_norm": 0.04224534332752228,
+      "learning_rate": 0.0007674747474747475,
+      "loss": 2.4037485122680664,
+      "step": 12702
+    },
+    {
+      "epoch": 0.7699393939393939,
+      "grad_norm": 0.04393499717116356,
+      "learning_rate": 0.0007670707070707071,
+      "loss": 2.4616916179656982,
+      "step": 12704
+    },
+    {
+      "epoch": 0.7700606060606061,
+      "grad_norm": 0.04540430009365082,
+      "learning_rate": 0.0007666666666666667,
+      "loss": 2.45333194732666,
+      "step": 12706
+    },
+    {
+      "epoch": 0.7701818181818182,
+      "grad_norm": 0.043596651405096054,
+      "learning_rate": 0.0007662626262626263,
+      "loss": 2.4061942100524902,
+      "step": 12708
+    },
+    {
+      "epoch": 0.7703030303030303,
+      "grad_norm": 0.043312471359968185,
+      "learning_rate": 0.0007658585858585859,
+      "loss": 2.3886594772338867,
+      "step": 12710
+    },
+    {
+      "epoch": 0.7704242424242425,
+      "grad_norm": 0.04348092898726463,
+      "learning_rate": 0.0007654545454545455,
+      "loss": 2.3782169818878174,
+      "step": 12712
+    },
+    {
+      "epoch": 0.7705454545454545,
+      "grad_norm": 0.05074289068579674,
+      "learning_rate": 0.0007650505050505051,
+      "loss": 2.415584087371826,
+      "step": 12714
+    },
+    {
+      "epoch": 0.7706666666666667,
+      "grad_norm": 0.045656416565179825,
+      "learning_rate": 0.0007646464646464647,
+      "loss": 2.4495229721069336,
+      "step": 12716
+    },
+    {
+      "epoch": 0.7707878787878788,
+      "grad_norm": 0.06331637501716614,
+      "learning_rate": 0.0007642424242424243,
+      "loss": 2.4427340030670166,
+      "step": 12718
+    },
+    {
+      "epoch": 0.7709090909090909,
+      "grad_norm": 0.05128614977002144,
+      "learning_rate": 0.0007638383838383838,
+      "loss": 2.433253526687622,
+      "step": 12720
+    },
+    {
+      "epoch": 0.7710303030303031,
+      "grad_norm": 0.04832685738801956,
+      "learning_rate": 0.0007634343434343435,
+      "loss": 2.4685308933258057,
+      "step": 12722
+    },
+    {
+      "epoch": 0.7711515151515151,
+      "grad_norm": 0.04649219289422035,
+      "learning_rate": 0.000763030303030303,
+      "loss": 2.4227824211120605,
+      "step": 12724
+    },
+    {
+      "epoch": 0.7712727272727272,
+      "grad_norm": 0.04685407131910324,
+      "learning_rate": 0.0007626262626262626,
+      "loss": 2.3481926918029785,
+      "step": 12726
+    },
+    {
+      "epoch": 0.7713939393939394,
+      "grad_norm": 0.04710349813103676,
+      "learning_rate": 0.0007622222222222222,
+      "loss": 2.443406105041504,
+      "step": 12728
+    },
+    {
+      "epoch": 0.7715151515151515,
+      "grad_norm": 0.04605654254555702,
+      "learning_rate": 0.0007618181818181819,
+      "loss": 2.4351346492767334,
+      "step": 12730
+    },
+    {
+      "epoch": 0.7716363636363637,
+      "grad_norm": 0.04289453849196434,
+      "learning_rate": 0.0007614141414141415,
+      "loss": 2.4550678730010986,
+      "step": 12732
+    },
+    {
+      "epoch": 0.7717575757575758,
+      "grad_norm": 0.04204421117901802,
+      "learning_rate": 0.000761010101010101,
+      "loss": 2.4629085063934326,
+      "step": 12734
+    },
+    {
+      "epoch": 0.7718787878787878,
+      "grad_norm": 0.044543780386447906,
+      "learning_rate": 0.0007606060606060607,
+      "loss": 2.4982309341430664,
+      "step": 12736
+    },
+    {
+      "epoch": 0.772,
+      "grad_norm": 0.05362275242805481,
+      "learning_rate": 0.0007602020202020203,
+      "loss": 2.470094919204712,
+      "step": 12738
+    },
+    {
+      "epoch": 0.7721212121212121,
+      "grad_norm": 0.05270489677786827,
+      "learning_rate": 0.0007597979797979798,
+      "loss": 2.4776763916015625,
+      "step": 12740
+    },
+    {
+      "epoch": 0.7722424242424243,
+      "grad_norm": 0.04878697916865349,
+      "learning_rate": 0.0007593939393939394,
+      "loss": 2.4092442989349365,
+      "step": 12742
+    },
+    {
+      "epoch": 0.7723636363636364,
+      "grad_norm": 0.04614836722612381,
+      "learning_rate": 0.0007589898989898991,
+      "loss": 2.4621574878692627,
+      "step": 12744
+    },
+    {
+      "epoch": 0.7724848484848484,
+      "grad_norm": 0.04521850869059563,
+      "learning_rate": 0.0007585858585858586,
+      "loss": 2.499734401702881,
+      "step": 12746
+    },
+    {
+      "epoch": 0.7726060606060606,
+      "grad_norm": 0.04390499368309975,
+      "learning_rate": 0.0007581818181818182,
+      "loss": 2.501100540161133,
+      "step": 12748
+    },
+    {
+      "epoch": 0.7727272727272727,
+      "grad_norm": 0.045842792838811874,
+      "learning_rate": 0.0007577777777777778,
+      "loss": 2.475497007369995,
+      "step": 12750
+    },
+    {
+      "epoch": 0.7728484848484849,
+      "grad_norm": 0.04886357858777046,
+      "learning_rate": 0.0007573737373737375,
+      "loss": 2.445254325866699,
+      "step": 12752
+    },
+    {
+      "epoch": 0.772969696969697,
+      "grad_norm": 0.0511283203959465,
+      "learning_rate": 0.0007569696969696969,
+      "loss": 2.480238914489746,
+      "step": 12754
+    },
+    {
+      "epoch": 0.773090909090909,
+      "grad_norm": 0.04500778019428253,
+      "learning_rate": 0.0007565656565656566,
+      "loss": 2.4958903789520264,
+      "step": 12756
+    },
+    {
+      "epoch": 0.7732121212121212,
+      "grad_norm": 0.047281090170145035,
+      "learning_rate": 0.0007561616161616162,
+      "loss": 2.448146343231201,
+      "step": 12758
+    },
+    {
+      "epoch": 0.7733333333333333,
+      "grad_norm": 0.044455934315919876,
+      "learning_rate": 0.0007557575757575758,
+      "loss": 2.518587350845337,
+      "step": 12760
+    },
+    {
+      "epoch": 0.7734545454545455,
+      "grad_norm": 0.043076545000076294,
+      "learning_rate": 0.0007553535353535353,
+      "loss": 2.510380744934082,
+      "step": 12762
+    },
+    {
+      "epoch": 0.7735757575757576,
+      "grad_norm": 0.04863133653998375,
+      "learning_rate": 0.000754949494949495,
+      "loss": 2.552694320678711,
+      "step": 12764
+    },
+    {
+      "epoch": 0.7736969696969697,
+      "grad_norm": 0.04575743526220322,
+      "learning_rate": 0.0007545454545454546,
+      "loss": 2.482844352722168,
+      "step": 12766
+    },
+    {
+      "epoch": 0.7738181818181818,
+      "grad_norm": 0.04459463059902191,
+      "learning_rate": 0.0007541414141414141,
+      "loss": 2.4896492958068848,
+      "step": 12768
+    },
+    {
+      "epoch": 0.7739393939393939,
+      "grad_norm": 0.04890776425600052,
+      "learning_rate": 0.0007537373737373738,
+      "loss": 2.4572486877441406,
+      "step": 12770
+    },
+    {
+      "epoch": 0.7740606060606061,
+      "grad_norm": 0.04549407213926315,
+      "learning_rate": 0.0007533333333333334,
+      "loss": 2.491011381149292,
+      "step": 12772
+    },
+    {
+      "epoch": 0.7741818181818182,
+      "grad_norm": 0.04394189640879631,
+      "learning_rate": 0.000752929292929293,
+      "loss": 2.494438886642456,
+      "step": 12774
+    },
+    {
+      "epoch": 0.7743030303030303,
+      "grad_norm": 0.04909466952085495,
+      "learning_rate": 0.0007525252525252525,
+      "loss": 2.511922836303711,
+      "step": 12776
+    },
+    {
+      "epoch": 0.7744242424242425,
+      "grad_norm": 0.04470570385456085,
+      "learning_rate": 0.0007521212121212122,
+      "loss": 2.533024787902832,
+      "step": 12778
+    },
+    {
+      "epoch": 0.7745454545454545,
+      "grad_norm": 0.04376324266195297,
+      "learning_rate": 0.0007517171717171717,
+      "loss": 2.528724193572998,
+      "step": 12780
+    },
+    {
+      "epoch": 0.7746666666666666,
+      "grad_norm": 0.042851898819208145,
+      "learning_rate": 0.0007513131313131314,
+      "loss": 2.4977169036865234,
+      "step": 12782
+    },
+    {
+      "epoch": 0.7747878787878788,
+      "grad_norm": 0.047850750386714935,
+      "learning_rate": 0.0007509090909090909,
+      "loss": 2.643157482147217,
+      "step": 12784
+    },
+    {
+      "epoch": 0.7749090909090909,
+      "grad_norm": 0.0448644682765007,
+      "learning_rate": 0.0007505050505050506,
+      "loss": 2.5524837970733643,
+      "step": 12786
+    },
+    {
+      "epoch": 0.7750303030303031,
+      "grad_norm": 0.04567348212003708,
+      "learning_rate": 0.0007501010101010101,
+      "loss": 2.577564001083374,
+      "step": 12788
+    },
+    {
+      "epoch": 0.7751515151515151,
+      "grad_norm": 0.046540822833776474,
+      "learning_rate": 0.0007496969696969697,
+      "loss": 2.5711677074432373,
+      "step": 12790
+    },
+    {
+      "epoch": 0.7752727272727272,
+      "grad_norm": 0.047974999994039536,
+      "learning_rate": 0.0007492929292929293,
+      "loss": 2.4955170154571533,
+      "step": 12792
+    },
+    {
+      "epoch": 0.7753939393939394,
+      "grad_norm": 0.05598282068967819,
+      "learning_rate": 0.0007488888888888889,
+      "loss": 2.5107104778289795,
+      "step": 12794
+    },
+    {
+      "epoch": 0.7755151515151515,
+      "grad_norm": 0.06310189515352249,
+      "learning_rate": 0.0007484848484848485,
+      "loss": 2.542616844177246,
+      "step": 12796
+    },
+    {
+      "epoch": 0.7756363636363637,
+      "grad_norm": 0.07566085457801819,
+      "learning_rate": 0.0007480808080808081,
+      "loss": 2.652275800704956,
+      "step": 12798
+    },
+    {
+      "epoch": 0.7757575757575758,
+      "grad_norm": 0.07600195705890656,
+      "learning_rate": 0.0007476767676767677,
+      "loss": 2.6317834854125977,
+      "step": 12800
+    },
+    {
+      "epoch": 0.7758787878787878,
+      "grad_norm": 0.07576204091310501,
+      "learning_rate": 0.0007472727272727273,
+      "loss": 2.628864049911499,
+      "step": 12802
+    },
+    {
+      "epoch": 0.776,
+      "grad_norm": 0.05260759964585304,
+      "learning_rate": 0.000746868686868687,
+      "loss": 2.575460910797119,
+      "step": 12804
+    },
+    {
+      "epoch": 0.7761212121212121,
+      "grad_norm": 0.0445241704583168,
+      "learning_rate": 0.0007464646464646465,
+      "loss": 2.5369937419891357,
+      "step": 12806
+    },
+    {
+      "epoch": 0.7762424242424243,
+      "grad_norm": 0.04242302477359772,
+      "learning_rate": 0.0007460606060606061,
+      "loss": 2.58801531791687,
+      "step": 12808
+    },
+    {
+      "epoch": 0.7763636363636364,
+      "grad_norm": 0.042774222791194916,
+      "learning_rate": 0.0007456565656565657,
+      "loss": 2.5607545375823975,
+      "step": 12810
+    },
+    {
+      "epoch": 0.7764848484848484,
+      "grad_norm": 0.043507445603609085,
+      "learning_rate": 0.0007452525252525254,
+      "loss": 2.580716371536255,
+      "step": 12812
+    },
+    {
+      "epoch": 0.7766060606060606,
+      "grad_norm": 0.04378197714686394,
+      "learning_rate": 0.0007448484848484848,
+      "loss": 2.5691936016082764,
+      "step": 12814
+    },
+    {
+      "epoch": 0.7767272727272727,
+      "grad_norm": 0.04529546946287155,
+      "learning_rate": 0.0007444444444444445,
+      "loss": 2.5750393867492676,
+      "step": 12816
+    },
+    {
+      "epoch": 0.7768484848484849,
+      "grad_norm": 0.053495798259973526,
+      "learning_rate": 0.0007440404040404041,
+      "loss": 2.5900967121124268,
+      "step": 12818
+    },
+    {
+      "epoch": 0.776969696969697,
+      "grad_norm": 0.05101901665329933,
+      "learning_rate": 0.0007436363636363636,
+      "loss": 2.5482840538024902,
+      "step": 12820
+    },
+    {
+      "epoch": 0.777090909090909,
+      "grad_norm": 0.060583245009183884,
+      "learning_rate": 0.0007432323232323232,
+      "loss": 2.5953850746154785,
+      "step": 12822
+    },
+    {
+      "epoch": 0.7772121212121212,
+      "grad_norm": 0.052583303302526474,
+      "learning_rate": 0.0007428282828282829,
+      "loss": 2.6629562377929688,
+      "step": 12824
+    },
+    {
+      "epoch": 0.7773333333333333,
+      "grad_norm": 0.04834415391087532,
+      "learning_rate": 0.0007424242424242425,
+      "loss": 2.5758848190307617,
+      "step": 12826
+    },
+    {
+      "epoch": 0.7774545454545455,
+      "grad_norm": 0.04479902610182762,
+      "learning_rate": 0.000742020202020202,
+      "loss": 2.547616720199585,
+      "step": 12828
+    },
+    {
+      "epoch": 0.7775757575757576,
+      "grad_norm": 0.05078183859586716,
+      "learning_rate": 0.0007416161616161616,
+      "loss": 2.612529993057251,
+      "step": 12830
+    },
+    {
+      "epoch": 0.7776969696969697,
+      "grad_norm": 0.05115080997347832,
+      "learning_rate": 0.0007412121212121213,
+      "loss": 2.6378161907196045,
+      "step": 12832
+    },
+    {
+      "epoch": 0.7778181818181819,
+      "grad_norm": 0.052601709961891174,
+      "learning_rate": 0.0007408080808080808,
+      "loss": 2.6748812198638916,
+      "step": 12834
+    },
+    {
+      "epoch": 0.7779393939393939,
+      "grad_norm": 0.04368458688259125,
+      "learning_rate": 0.0007404040404040404,
+      "loss": 2.5678372383117676,
+      "step": 12836
+    },
+    {
+      "epoch": 0.778060606060606,
+      "grad_norm": 0.05059303715825081,
+      "learning_rate": 0.0007400000000000001,
+      "loss": 2.6174116134643555,
+      "step": 12838
+    },
+    {
+      "epoch": 0.7781818181818182,
+      "grad_norm": 0.04612511396408081,
+      "learning_rate": 0.0007395959595959597,
+      "loss": 2.6149260997772217,
+      "step": 12840
+    },
+    {
+      "epoch": 0.7783030303030303,
+      "grad_norm": 0.049865275621414185,
+      "learning_rate": 0.0007391919191919192,
+      "loss": 2.6045048236846924,
+      "step": 12842
+    },
+    {
+      "epoch": 0.7784242424242425,
+      "grad_norm": 0.05206015333533287,
+      "learning_rate": 0.0007387878787878788,
+      "loss": 2.6025266647338867,
+      "step": 12844
+    },
+    {
+      "epoch": 0.7785454545454545,
+      "grad_norm": 0.05972357839345932,
+      "learning_rate": 0.0007383838383838385,
+      "loss": 2.615468978881836,
+      "step": 12846
+    },
+    {
+      "epoch": 0.7786666666666666,
+      "grad_norm": 0.05425636097788811,
+      "learning_rate": 0.000737979797979798,
+      "loss": 2.635451078414917,
+      "step": 12848
+    },
+    {
+      "epoch": 0.7787878787878788,
+      "grad_norm": 0.04588595777750015,
+      "learning_rate": 0.0007375757575757576,
+      "loss": 2.616811990737915,
+      "step": 12850
+    },
+    {
+      "epoch": 0.7789090909090909,
+      "grad_norm": 0.05082259699702263,
+      "learning_rate": 0.0007371717171717172,
+      "loss": 2.6473841667175293,
+      "step": 12852
+    },
+    {
+      "epoch": 0.7790303030303031,
+      "grad_norm": 0.04821845144033432,
+      "learning_rate": 0.0007367676767676768,
+      "loss": 2.7158071994781494,
+      "step": 12854
+    },
+    {
+      "epoch": 0.7791515151515152,
+      "grad_norm": 0.04695912078022957,
+      "learning_rate": 0.0007363636363636363,
+      "loss": 2.6932218074798584,
+      "step": 12856
+    },
+    {
+      "epoch": 0.7792727272727272,
+      "grad_norm": 0.04561370238661766,
+      "learning_rate": 0.000735959595959596,
+      "loss": 2.6690220832824707,
+      "step": 12858
+    },
+    {
+      "epoch": 0.7793939393939394,
+      "grad_norm": 0.05558645725250244,
+      "learning_rate": 0.0007355555555555556,
+      "loss": 2.628401279449463,
+      "step": 12860
+    },
+    {
+      "epoch": 0.7795151515151515,
+      "grad_norm": 0.04310929402709007,
+      "learning_rate": 0.0007351515151515152,
+      "loss": 2.7098588943481445,
+      "step": 12862
+    },
+    {
+      "epoch": 0.7796363636363637,
+      "grad_norm": 0.04445216432213783,
+      "learning_rate": 0.0007347474747474747,
+      "loss": 2.6680805683135986,
+      "step": 12864
+    },
+    {
+      "epoch": 0.7797575757575758,
+      "grad_norm": 0.04591483995318413,
+      "learning_rate": 0.0007343434343434344,
+      "loss": 2.673837184906006,
+      "step": 12866
+    },
+    {
+      "epoch": 0.7798787878787878,
+      "grad_norm": 0.046467218548059464,
+      "learning_rate": 0.000733939393939394,
+      "loss": 2.6872410774230957,
+      "step": 12868
+    },
+    {
+      "epoch": 0.78,
+      "grad_norm": 0.044584546238183975,
+      "learning_rate": 0.0007335353535353536,
+      "loss": 2.7282626628875732,
+      "step": 12870
+    },
+    {
+      "epoch": 0.7801212121212121,
+      "grad_norm": 0.04937903583049774,
+      "learning_rate": 0.0007331313131313132,
+      "loss": 2.7435553073883057,
+      "step": 12872
+    },
+    {
+      "epoch": 0.7802424242424243,
+      "grad_norm": 0.05095362663269043,
+      "learning_rate": 0.0007327272727272727,
+      "loss": 2.765939474105835,
+      "step": 12874
+    },
+    {
+      "epoch": 0.7803636363636364,
+      "grad_norm": 0.04567648470401764,
+      "learning_rate": 0.0007323232323232324,
+      "loss": 2.699469566345215,
+      "step": 12876
+    },
+    {
+      "epoch": 0.7804848484848484,
+      "grad_norm": 0.05411459878087044,
+      "learning_rate": 0.000731919191919192,
+      "loss": 2.704542875289917,
+      "step": 12878
+    },
+    {
+      "epoch": 0.7806060606060606,
+      "grad_norm": 0.05779287591576576,
+      "learning_rate": 0.0007315151515151516,
+      "loss": 2.643942356109619,
+      "step": 12880
+    },
+    {
+      "epoch": 0.7807272727272727,
+      "grad_norm": 0.045056987553834915,
+      "learning_rate": 0.0007311111111111111,
+      "loss": 2.724759817123413,
+      "step": 12882
+    },
+    {
+      "epoch": 0.7808484848484849,
+      "grad_norm": 0.06000228971242905,
+      "learning_rate": 0.0007307070707070708,
+      "loss": 2.685663938522339,
+      "step": 12884
+    },
+    {
+      "epoch": 0.780969696969697,
+      "grad_norm": 0.04651818796992302,
+      "learning_rate": 0.0007303030303030303,
+      "loss": 2.6892802715301514,
+      "step": 12886
+    },
+    {
+      "epoch": 0.7810909090909091,
+      "grad_norm": 0.047202080488204956,
+      "learning_rate": 0.0007298989898989899,
+      "loss": 2.738321542739868,
+      "step": 12888
+    },
+    {
+      "epoch": 0.7812121212121212,
+      "grad_norm": 0.05054826661944389,
+      "learning_rate": 0.0007294949494949495,
+      "loss": 2.6726486682891846,
+      "step": 12890
+    },
+    {
+      "epoch": 0.7813333333333333,
+      "grad_norm": 0.044609908014535904,
+      "learning_rate": 0.0007290909090909092,
+      "loss": 2.6633214950561523,
+      "step": 12892
+    },
+    {
+      "epoch": 0.7814545454545454,
+      "grad_norm": 0.045361876487731934,
+      "learning_rate": 0.0007286868686868687,
+      "loss": 2.72220516204834,
+      "step": 12894
+    },
+    {
+      "epoch": 0.7815757575757576,
+      "grad_norm": 0.045131269842386246,
+      "learning_rate": 0.0007282828282828283,
+      "loss": 2.7291667461395264,
+      "step": 12896
+    },
+    {
+      "epoch": 0.7816969696969697,
+      "grad_norm": 0.047776106745004654,
+      "learning_rate": 0.0007278787878787879,
+      "loss": 2.70430326461792,
+      "step": 12898
+    },
+    {
+      "epoch": 0.7818181818181819,
+      "grad_norm": 0.04864230379462242,
+      "learning_rate": 0.0007274747474747476,
+      "loss": 2.7283806800842285,
+      "step": 12900
+    },
+    {
+      "epoch": 0.7819393939393939,
+      "grad_norm": 0.049562204629182816,
+      "learning_rate": 0.0007270707070707071,
+      "loss": 2.6835734844207764,
+      "step": 12902
+    },
+    {
+      "epoch": 0.782060606060606,
+      "grad_norm": 0.048225220292806625,
+      "learning_rate": 0.0007266666666666667,
+      "loss": 2.7117843627929688,
+      "step": 12904
+    },
+    {
+      "epoch": 0.7821818181818182,
+      "grad_norm": 0.06183773651719093,
+      "learning_rate": 0.0007262626262626264,
+      "loss": 2.7601511478424072,
+      "step": 12906
+    },
+    {
+      "epoch": 0.7823030303030303,
+      "grad_norm": 0.04668540507555008,
+      "learning_rate": 0.0007258585858585858,
+      "loss": 2.6988728046417236,
+      "step": 12908
+    },
+    {
+      "epoch": 0.7824242424242425,
+      "grad_norm": 0.04506373405456543,
+      "learning_rate": 0.0007254545454545455,
+      "loss": 2.7016801834106445,
+      "step": 12910
+    },
+    {
+      "epoch": 0.7825454545454545,
+      "grad_norm": 0.045195143669843674,
+      "learning_rate": 0.0007250505050505051,
+      "loss": 2.713531970977783,
+      "step": 12912
+    },
+    {
+      "epoch": 0.7826666666666666,
+      "grad_norm": 0.04459849372506142,
+      "learning_rate": 0.0007246464646464647,
+      "loss": 2.6549806594848633,
+      "step": 12914
+    },
+    {
+      "epoch": 0.7827878787878788,
+      "grad_norm": 0.0514596551656723,
+      "learning_rate": 0.0007242424242424242,
+      "loss": 2.721348762512207,
+      "step": 12916
+    },
+    {
+      "epoch": 0.7829090909090909,
+      "grad_norm": 0.06073107197880745,
+      "learning_rate": 0.0007238383838383839,
+      "loss": 2.7245302200317383,
+      "step": 12918
+    },
+    {
+      "epoch": 0.7830303030303031,
+      "grad_norm": 0.060770634561777115,
+      "learning_rate": 0.0007234343434343435,
+      "loss": 2.7223246097564697,
+      "step": 12920
+    },
+    {
+      "epoch": 0.7831515151515152,
+      "grad_norm": 0.054707907140254974,
+      "learning_rate": 0.000723030303030303,
+      "loss": 2.698256492614746,
+      "step": 12922
+    },
+    {
+      "epoch": 0.7832727272727272,
+      "grad_norm": 0.05258503556251526,
+      "learning_rate": 0.0007226262626262626,
+      "loss": 2.720781087875366,
+      "step": 12924
+    },
+    {
+      "epoch": 0.7833939393939394,
+      "grad_norm": 0.04706183075904846,
+      "learning_rate": 0.0007222222222222223,
+      "loss": 2.7440528869628906,
+      "step": 12926
+    },
+    {
+      "epoch": 0.7835151515151515,
+      "grad_norm": 0.04530370607972145,
+      "learning_rate": 0.0007218181818181819,
+      "loss": 2.704131841659546,
+      "step": 12928
+    },
+    {
+      "epoch": 0.7836363636363637,
+      "grad_norm": 0.04579959437251091,
+      "learning_rate": 0.0007214141414141414,
+      "loss": 2.718264579772949,
+      "step": 12930
+    },
+    {
+      "epoch": 0.7837575757575758,
+      "grad_norm": 0.04477856308221817,
+      "learning_rate": 0.000721010101010101,
+      "loss": 2.698681592941284,
+      "step": 12932
+    },
+    {
+      "epoch": 0.7838787878787878,
+      "grad_norm": 0.05169568210840225,
+      "learning_rate": 0.0007206060606060607,
+      "loss": 2.6752841472625732,
+      "step": 12934
+    },
+    {
+      "epoch": 0.784,
+      "grad_norm": 0.047062329947948456,
+      "learning_rate": 0.0007202020202020203,
+      "loss": 2.7047150135040283,
+      "step": 12936
+    },
+    {
+      "epoch": 0.7841212121212121,
+      "grad_norm": 0.045735280960798264,
+      "learning_rate": 0.0007197979797979798,
+      "loss": 2.726062059402466,
+      "step": 12938
+    },
+    {
+      "epoch": 0.7842424242424243,
+      "grad_norm": 0.05146170035004616,
+      "learning_rate": 0.0007193939393939395,
+      "loss": 2.7242558002471924,
+      "step": 12940
+    },
+    {
+      "epoch": 0.7843636363636364,
+      "grad_norm": 0.04407952353358269,
+      "learning_rate": 0.000718989898989899,
+      "loss": 2.7610719203948975,
+      "step": 12942
+    },
+    {
+      "epoch": 0.7844848484848485,
+      "grad_norm": 0.052351050078868866,
+      "learning_rate": 0.0007185858585858587,
+      "loss": 2.705827236175537,
+      "step": 12944
+    },
+    {
+      "epoch": 0.7846060606060606,
+      "grad_norm": 0.05580949783325195,
+      "learning_rate": 0.0007181818181818182,
+      "loss": 2.7750582695007324,
+      "step": 12946
+    },
+    {
+      "epoch": 0.7847272727272727,
+      "grad_norm": 0.0536104217171669,
+      "learning_rate": 0.0007177777777777778,
+      "loss": 2.7427170276641846,
+      "step": 12948
+    },
+    {
+      "epoch": 0.7848484848484848,
+      "grad_norm": 0.05571214109659195,
+      "learning_rate": 0.0007173737373737374,
+      "loss": 2.7508697509765625,
+      "step": 12950
+    },
+    {
+      "epoch": 0.784969696969697,
+      "grad_norm": 0.05746801570057869,
+      "learning_rate": 0.000716969696969697,
+      "loss": 2.673309564590454,
+      "step": 12952
+    },
+    {
+      "epoch": 0.7850909090909091,
+      "grad_norm": 0.06027154251933098,
+      "learning_rate": 0.0007165656565656566,
+      "loss": 2.695446014404297,
+      "step": 12954
+    },
+    {
+      "epoch": 0.7852121212121213,
+      "grad_norm": 0.049843501299619675,
+      "learning_rate": 0.0007161616161616162,
+      "loss": 2.782951831817627,
+      "step": 12956
+    },
+    {
+      "epoch": 0.7853333333333333,
+      "grad_norm": 0.050677720457315445,
+      "learning_rate": 0.0007157575757575758,
+      "loss": 2.726875066757202,
+      "step": 12958
+    },
+    {
+      "epoch": 0.7854545454545454,
+      "grad_norm": 0.04752598702907562,
+      "learning_rate": 0.0007153535353535354,
+      "loss": 2.756880044937134,
+      "step": 12960
+    },
+    {
+      "epoch": 0.7855757575757576,
+      "grad_norm": 0.05594700574874878,
+      "learning_rate": 0.0007149494949494949,
+      "loss": 2.7204201221466064,
+      "step": 12962
+    },
+    {
+      "epoch": 0.7856969696969697,
+      "grad_norm": 0.046631257981061935,
+      "learning_rate": 0.0007145454545454546,
+      "loss": 2.7649831771850586,
+      "step": 12964
+    },
+    {
+      "epoch": 0.7858181818181819,
+      "grad_norm": 0.05859227851033211,
+      "learning_rate": 0.0007141414141414141,
+      "loss": 2.7316203117370605,
+      "step": 12966
+    },
+    {
+      "epoch": 0.7859393939393939,
+      "grad_norm": 0.059217147529125214,
+      "learning_rate": 0.0007137373737373737,
+      "loss": 2.6934635639190674,
+      "step": 12968
+    },
+    {
+      "epoch": 0.786060606060606,
+      "grad_norm": 0.06383583694696426,
+      "learning_rate": 0.0007133333333333334,
+      "loss": 2.75429630279541,
+      "step": 12970
+    },
+    {
+      "epoch": 0.7861818181818182,
+      "grad_norm": 0.06373012810945511,
+      "learning_rate": 0.000712929292929293,
+      "loss": 2.7576754093170166,
+      "step": 12972
+    },
+    {
+      "epoch": 0.7863030303030303,
+      "grad_norm": 0.05767196789383888,
+      "learning_rate": 0.0007125252525252526,
+      "loss": 2.7992441654205322,
+      "step": 12974
+    },
+    {
+      "epoch": 0.7864242424242425,
+      "grad_norm": 0.050360869616270065,
+      "learning_rate": 0.0007121212121212121,
+      "loss": 2.73905086517334,
+      "step": 12976
+    },
+    {
+      "epoch": 0.7865454545454545,
+      "grad_norm": 0.04965699091553688,
+      "learning_rate": 0.0007117171717171718,
+      "loss": 2.7936692237854004,
+      "step": 12978
+    },
+    {
+      "epoch": 0.7866666666666666,
+      "grad_norm": 0.051724161952733994,
+      "learning_rate": 0.0007113131313131314,
+      "loss": 2.749941110610962,
+      "step": 12980
+    },
+    {
+      "epoch": 0.7867878787878788,
+      "grad_norm": 0.04734734445810318,
+      "learning_rate": 0.0007109090909090909,
+      "loss": 2.729030132293701,
+      "step": 12982
+    },
+    {
+      "epoch": 0.7869090909090909,
+      "grad_norm": 0.050445009022951126,
+      "learning_rate": 0.0007105050505050505,
+      "loss": 2.7496376037597656,
+      "step": 12984
+    },
+    {
+      "epoch": 0.7870303030303031,
+      "grad_norm": 0.06057169288396835,
+      "learning_rate": 0.0007101010101010102,
+      "loss": 2.76599383354187,
+      "step": 12986
+    },
+    {
+      "epoch": 0.7871515151515152,
+      "grad_norm": 0.055337272584438324,
+      "learning_rate": 0.0007096969696969698,
+      "loss": 2.7353909015655518,
+      "step": 12988
+    },
+    {
+      "epoch": 0.7872727272727272,
+      "grad_norm": 0.046721674501895905,
+      "learning_rate": 0.0007092929292929293,
+      "loss": 2.7717623710632324,
+      "step": 12990
+    },
+    {
+      "epoch": 0.7873939393939394,
+      "grad_norm": 0.0477215014398098,
+      "learning_rate": 0.0007088888888888889,
+      "loss": 2.747539520263672,
+      "step": 12992
+    },
+    {
+      "epoch": 0.7875151515151515,
+      "grad_norm": 0.05087020993232727,
+      "learning_rate": 0.0007084848484848486,
+      "loss": 2.7480525970458984,
+      "step": 12994
+    },
+    {
+      "epoch": 0.7876363636363637,
+      "grad_norm": 0.05453432723879814,
+      "learning_rate": 0.000708080808080808,
+      "loss": 2.738396644592285,
+      "step": 12996
+    },
+    {
+      "epoch": 0.7877575757575758,
+      "grad_norm": 0.055683087557554245,
+      "learning_rate": 0.0007076767676767677,
+      "loss": 2.6789801120758057,
+      "step": 12998
+    },
+    {
+      "epoch": 0.7878787878787878,
+      "grad_norm": 0.04935922846198082,
+      "learning_rate": 0.0007072727272727273,
+      "loss": 2.7556052207946777,
+      "step": 13000
+    },
+    {
+      "epoch": 0.788,
+      "grad_norm": 0.04959288239479065,
+      "learning_rate": 0.0007068686868686869,
+      "loss": 2.7819857597351074,
+      "step": 13002
+    },
+    {
+      "epoch": 0.7881212121212121,
+      "grad_norm": 0.05278705060482025,
+      "learning_rate": 0.0007064646464646465,
+      "loss": 2.7619662284851074,
+      "step": 13004
+    },
+    {
+      "epoch": 0.7882424242424242,
+      "grad_norm": 0.0550333634018898,
+      "learning_rate": 0.0007060606060606061,
+      "loss": 2.7777469158172607,
+      "step": 13006
+    },
+    {
+      "epoch": 0.7883636363636364,
+      "grad_norm": 0.06331103295087814,
+      "learning_rate": 0.0007056565656565657,
+      "loss": 2.776226043701172,
+      "step": 13008
+    },
+    {
+      "epoch": 0.7884848484848485,
+      "grad_norm": 0.05263159051537514,
+      "learning_rate": 0.0007052525252525252,
+      "loss": 2.773317813873291,
+      "step": 13010
+    },
+    {
+      "epoch": 0.7886060606060606,
+      "grad_norm": 0.04408787563443184,
+      "learning_rate": 0.0007048484848484849,
+      "loss": 2.7556824684143066,
+      "step": 13012
+    },
+    {
+      "epoch": 0.7887272727272727,
+      "grad_norm": 0.04878704622387886,
+      "learning_rate": 0.0007044444444444445,
+      "loss": 2.780618667602539,
+      "step": 13014
+    },
+    {
+      "epoch": 0.7888484848484848,
+      "grad_norm": 0.0501292422413826,
+      "learning_rate": 0.0007040404040404041,
+      "loss": 2.8232169151306152,
+      "step": 13016
+    },
+    {
+      "epoch": 0.788969696969697,
+      "grad_norm": 0.04706967994570732,
+      "learning_rate": 0.0007036363636363636,
+      "loss": 2.763132095336914,
+      "step": 13018
+    },
+    {
+      "epoch": 0.7890909090909091,
+      "grad_norm": 0.04455282539129257,
+      "learning_rate": 0.0007032323232323233,
+      "loss": 2.777132511138916,
+      "step": 13020
+    },
+    {
+      "epoch": 0.7892121212121213,
+      "grad_norm": 0.057565778493881226,
+      "learning_rate": 0.0007028282828282829,
+      "loss": 2.766697406768799,
+      "step": 13022
+    },
+    {
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.057665493339300156,
+      "learning_rate": 0.0007024242424242425,
+      "loss": 2.7381362915039062,
+      "step": 13024
+    },
+    {
+      "epoch": 0.7894545454545454,
+      "grad_norm": 0.054578278213739395,
+      "learning_rate": 0.000702020202020202,
+      "loss": 2.7788350582122803,
+      "step": 13026
+    },
+    {
+      "epoch": 0.7895757575757576,
+      "grad_norm": 0.053561292588710785,
+      "learning_rate": 0.0007016161616161617,
+      "loss": 2.8209919929504395,
+      "step": 13028
+    },
+    {
+      "epoch": 0.7896969696969697,
+      "grad_norm": 0.048120979219675064,
+      "learning_rate": 0.0007012121212121212,
+      "loss": 2.7841873168945312,
+      "step": 13030
+    },
+    {
+      "epoch": 0.7898181818181819,
+      "grad_norm": 0.045977674424648285,
+      "learning_rate": 0.0007008080808080809,
+      "loss": 2.7455508708953857,
+      "step": 13032
+    },
+    {
+      "epoch": 0.7899393939393939,
+      "grad_norm": 0.05141545459628105,
+      "learning_rate": 0.0007004040404040404,
+      "loss": 2.7507729530334473,
+      "step": 13034
+    },
+    {
+      "epoch": 0.790060606060606,
+      "grad_norm": 0.045574966818094254,
+      "learning_rate": 0.0007,
+      "loss": 2.7910284996032715,
+      "step": 13036
+    },
+    {
+      "epoch": 0.7901818181818182,
+      "grad_norm": 0.052448056638240814,
+      "learning_rate": 0.0006995959595959597,
+      "loss": 2.754842758178711,
+      "step": 13038
+    },
+    {
+      "epoch": 0.7903030303030303,
+      "grad_norm": 0.07158570736646652,
+      "learning_rate": 0.0006991919191919192,
+      "loss": 2.804783344268799,
+      "step": 13040
+    },
+    {
+      "epoch": 0.7904242424242425,
+      "grad_norm": 0.0506352074444294,
+      "learning_rate": 0.0006987878787878788,
+      "loss": 2.7546157836914062,
+      "step": 13042
+    },
+    {
+      "epoch": 0.7905454545454546,
+      "grad_norm": 0.05178079381585121,
+      "learning_rate": 0.0006983838383838384,
+      "loss": 2.7702338695526123,
+      "step": 13044
+    },
+    {
+      "epoch": 0.7906666666666666,
+      "grad_norm": 0.047850269824266434,
+      "learning_rate": 0.0006979797979797981,
+      "loss": 2.670844793319702,
+      "step": 13046
+    },
+    {
+      "epoch": 0.7907878787878788,
+      "grad_norm": 0.050300486385822296,
+      "learning_rate": 0.0006975757575757576,
+      "loss": 2.7778849601745605,
+      "step": 13048
+    },
+    {
+      "epoch": 0.7909090909090909,
+      "grad_norm": 0.0465228408575058,
+      "learning_rate": 0.0006971717171717172,
+      "loss": 2.771880865097046,
+      "step": 13050
+    },
+    {
+      "epoch": 0.7910303030303031,
+      "grad_norm": 0.04728807508945465,
+      "learning_rate": 0.0006967676767676768,
+      "loss": 2.7457480430603027,
+      "step": 13052
+    },
+    {
+      "epoch": 0.7911515151515152,
+      "grad_norm": 0.048885516822338104,
+      "learning_rate": 0.0006963636363636365,
+      "loss": 2.757521152496338,
+      "step": 13054
+    },
+    {
+      "epoch": 0.7912727272727272,
+      "grad_norm": 0.04819686338305473,
+      "learning_rate": 0.0006959595959595959,
+      "loss": 2.7572150230407715,
+      "step": 13056
+    },
+    {
+      "epoch": 0.7913939393939394,
+      "grad_norm": 0.04992295056581497,
+      "learning_rate": 0.0006955555555555556,
+      "loss": 2.8354079723358154,
+      "step": 13058
+    },
+    {
+      "epoch": 0.7915151515151515,
+      "grad_norm": 0.04702826961874962,
+      "learning_rate": 0.0006951515151515152,
+      "loss": 2.690727710723877,
+      "step": 13060
+    },
+    {
+      "epoch": 0.7916363636363636,
+      "grad_norm": 0.04663178324699402,
+      "learning_rate": 0.0006947474747474747,
+      "loss": 2.788396120071411,
+      "step": 13062
+    },
+    {
+      "epoch": 0.7917575757575758,
+      "grad_norm": 0.04557148367166519,
+      "learning_rate": 0.0006943434343434343,
+      "loss": 2.7726669311523438,
+      "step": 13064
+    },
+    {
+      "epoch": 0.7918787878787878,
+      "grad_norm": 0.0482805073261261,
+      "learning_rate": 0.000693939393939394,
+      "loss": 2.8176770210266113,
+      "step": 13066
+    },
+    {
+      "epoch": 0.792,
+      "grad_norm": 0.04674425721168518,
+      "learning_rate": 0.0006935353535353536,
+      "loss": 2.786742925643921,
+      "step": 13068
+    },
+    {
+      "epoch": 0.7921212121212121,
+      "grad_norm": 0.04674660041928291,
+      "learning_rate": 0.0006931313131313131,
+      "loss": 2.857231616973877,
+      "step": 13070
+    },
+    {
+      "epoch": 0.7922424242424242,
+      "grad_norm": 0.05158160254359245,
+      "learning_rate": 0.0006927272727272728,
+      "loss": 2.8024277687072754,
+      "step": 13072
+    },
+    {
+      "epoch": 0.7923636363636364,
+      "grad_norm": 0.05631386861205101,
+      "learning_rate": 0.0006923232323232324,
+      "loss": 2.7656500339508057,
+      "step": 13074
+    },
+    {
+      "epoch": 0.7924848484848485,
+      "grad_norm": 0.045749370008707047,
+      "learning_rate": 0.000691919191919192,
+      "loss": 2.819755792617798,
+      "step": 13076
+    },
+    {
+      "epoch": 0.7926060606060606,
+      "grad_norm": 0.04596208035945892,
+      "learning_rate": 0.0006915151515151515,
+      "loss": 2.8225109577178955,
+      "step": 13078
+    },
+    {
+      "epoch": 0.7927272727272727,
+      "grad_norm": 0.04624848812818527,
+      "learning_rate": 0.0006911111111111112,
+      "loss": 2.732177734375,
+      "step": 13080
+    },
+    {
+      "epoch": 0.7928484848484848,
+      "grad_norm": 0.047418806701898575,
+      "learning_rate": 0.0006907070707070708,
+      "loss": 2.793968915939331,
+      "step": 13082
+    },
+    {
+      "epoch": 0.792969696969697,
+      "grad_norm": 0.047092847526073456,
+      "learning_rate": 0.0006903030303030303,
+      "loss": 2.773519277572632,
+      "step": 13084
+    },
+    {
+      "epoch": 0.7930909090909091,
+      "grad_norm": 0.04617905616760254,
+      "learning_rate": 0.0006898989898989899,
+      "loss": 2.819995641708374,
+      "step": 13086
+    },
+    {
+      "epoch": 0.7932121212121213,
+      "grad_norm": 0.0469779446721077,
+      "learning_rate": 0.0006894949494949496,
+      "loss": 2.8254201412200928,
+      "step": 13088
+    },
+    {
+      "epoch": 0.7933333333333333,
+      "grad_norm": 0.06597650051116943,
+      "learning_rate": 0.0006890909090909091,
+      "loss": 2.7560386657714844,
+      "step": 13090
+    },
+    {
+      "epoch": 0.7934545454545454,
+      "grad_norm": 0.04744812101125717,
+      "learning_rate": 0.0006886868686868687,
+      "loss": 2.815908432006836,
+      "step": 13092
+    },
+    {
+      "epoch": 0.7935757575757576,
+      "grad_norm": 0.04491783678531647,
+      "learning_rate": 0.0006882828282828283,
+      "loss": 2.729473829269409,
+      "step": 13094
+    },
+    {
+      "epoch": 0.7936969696969697,
+      "grad_norm": 0.057662151753902435,
+      "learning_rate": 0.0006878787878787879,
+      "loss": 2.8109512329101562,
+      "step": 13096
+    },
+    {
+      "epoch": 0.7938181818181819,
+      "grad_norm": 0.05767659842967987,
+      "learning_rate": 0.0006874747474747474,
+      "loss": 2.8007898330688477,
+      "step": 13098
+    },
+    {
+      "epoch": 0.793939393939394,
+      "grad_norm": 0.04468579590320587,
+      "learning_rate": 0.0006870707070707071,
+      "loss": 2.799636125564575,
+      "step": 13100
+    },
+    {
+      "epoch": 0.794060606060606,
+      "grad_norm": 0.050391267985105515,
+      "learning_rate": 0.0006866666666666667,
+      "loss": 2.770794153213501,
+      "step": 13102
+    },
+    {
+      "epoch": 0.7941818181818182,
+      "grad_norm": 0.045197244733572006,
+      "learning_rate": 0.0006862626262626263,
+      "loss": 2.7634739875793457,
+      "step": 13104
+    },
+    {
+      "epoch": 0.7943030303030303,
+      "grad_norm": 0.04555029422044754,
+      "learning_rate": 0.000685858585858586,
+      "loss": 2.809053897857666,
+      "step": 13106
+    },
+    {
+      "epoch": 0.7944242424242425,
+      "grad_norm": 0.046583376824855804,
+      "learning_rate": 0.0006854545454545455,
+      "loss": 2.801868200302124,
+      "step": 13108
+    },
+    {
+      "epoch": 0.7945454545454546,
+      "grad_norm": 0.04825208708643913,
+      "learning_rate": 0.0006850505050505051,
+      "loss": 2.779825210571289,
+      "step": 13110
+    },
+    {
+      "epoch": 0.7946666666666666,
+      "grad_norm": 0.046051450073719025,
+      "learning_rate": 0.0006846464646464647,
+      "loss": 2.8168609142303467,
+      "step": 13112
+    },
+    {
+      "epoch": 0.7947878787878788,
+      "grad_norm": 0.04784306511282921,
+      "learning_rate": 0.0006842424242424243,
+      "loss": 2.7126646041870117,
+      "step": 13114
+    },
+    {
+      "epoch": 0.7949090909090909,
+      "grad_norm": 0.04582807421684265,
+      "learning_rate": 0.0006838383838383839,
+      "loss": 2.75772762298584,
+      "step": 13116
+    },
+    {
+      "epoch": 0.795030303030303,
+      "grad_norm": 0.04639701917767525,
+      "learning_rate": 0.0006834343434343435,
+      "loss": 2.79938006401062,
+      "step": 13118
+    },
+    {
+      "epoch": 0.7951515151515152,
+      "grad_norm": 0.047561246901750565,
+      "learning_rate": 0.000683030303030303,
+      "loss": 2.790745496749878,
+      "step": 13120
+    },
+    {
+      "epoch": 0.7952727272727272,
+      "grad_norm": 0.043681301176548004,
+      "learning_rate": 0.0006826262626262627,
+      "loss": 2.8214879035949707,
+      "step": 13122
+    },
+    {
+      "epoch": 0.7953939393939394,
+      "grad_norm": 0.04868364334106445,
+      "learning_rate": 0.0006822222222222222,
+      "loss": 2.8022592067718506,
+      "step": 13124
+    },
+    {
+      "epoch": 0.7955151515151515,
+      "grad_norm": 0.05575468763709068,
+      "learning_rate": 0.0006818181818181819,
+      "loss": 2.805053234100342,
+      "step": 13126
+    },
+    {
+      "epoch": 0.7956363636363636,
+      "grad_norm": 0.045058466494083405,
+      "learning_rate": 0.0006814141414141414,
+      "loss": 2.814293384552002,
+      "step": 13128
+    },
+    {
+      "epoch": 0.7957575757575758,
+      "grad_norm": 0.04958244040608406,
+      "learning_rate": 0.000681010101010101,
+      "loss": 2.8165037631988525,
+      "step": 13130
+    },
+    {
+      "epoch": 0.7958787878787879,
+      "grad_norm": 0.0470423586666584,
+      "learning_rate": 0.0006806060606060606,
+      "loss": 2.779778480529785,
+      "step": 13132
+    },
+    {
+      "epoch": 0.796,
+      "grad_norm": 0.04629089683294296,
+      "learning_rate": 0.0006802020202020203,
+      "loss": 2.8176965713500977,
+      "step": 13134
+    },
+    {
+      "epoch": 0.7961212121212121,
+      "grad_norm": 0.05449686944484711,
+      "learning_rate": 0.0006797979797979798,
+      "loss": 2.858067512512207,
+      "step": 13136
+    },
+    {
+      "epoch": 0.7962424242424242,
+      "grad_norm": 0.051575928926467896,
+      "learning_rate": 0.0006793939393939394,
+      "loss": 2.790954351425171,
+      "step": 13138
+    },
+    {
+      "epoch": 0.7963636363636364,
+      "grad_norm": 0.04488060995936394,
+      "learning_rate": 0.0006789898989898991,
+      "loss": 2.7668046951293945,
+      "step": 13140
+    },
+    {
+      "epoch": 0.7964848484848485,
+      "grad_norm": 0.053057342767715454,
+      "learning_rate": 0.0006785858585858587,
+      "loss": 2.8132989406585693,
+      "step": 13142
+    },
+    {
+      "epoch": 0.7966060606060607,
+      "grad_norm": 0.04662083461880684,
+      "learning_rate": 0.0006781818181818182,
+      "loss": 2.793585777282715,
+      "step": 13144
+    },
+    {
+      "epoch": 0.7967272727272727,
+      "grad_norm": 0.05255010351538658,
+      "learning_rate": 0.0006777777777777778,
+      "loss": 2.867196798324585,
+      "step": 13146
+    },
+    {
+      "epoch": 0.7968484848484848,
+      "grad_norm": 0.05707961320877075,
+      "learning_rate": 0.0006773737373737375,
+      "loss": 2.7568771839141846,
+      "step": 13148
+    },
+    {
+      "epoch": 0.796969696969697,
+      "grad_norm": 0.06333516538143158,
+      "learning_rate": 0.0006769696969696969,
+      "loss": 2.7879064083099365,
+      "step": 13150
+    },
+    {
+      "epoch": 0.7970909090909091,
+      "grad_norm": 0.060131702572107315,
+      "learning_rate": 0.0006765656565656566,
+      "loss": 2.7454588413238525,
+      "step": 13152
+    },
+    {
+      "epoch": 0.7972121212121213,
+      "grad_norm": 0.04718490689992905,
+      "learning_rate": 0.0006761616161616162,
+      "loss": 2.8699684143066406,
+      "step": 13154
+    },
+    {
+      "epoch": 0.7973333333333333,
+      "grad_norm": 0.04780982434749603,
+      "learning_rate": 0.0006757575757575758,
+      "loss": 2.8111112117767334,
+      "step": 13156
+    },
+    {
+      "epoch": 0.7974545454545454,
+      "grad_norm": 0.05249437689781189,
+      "learning_rate": 0.0006753535353535353,
+      "loss": 2.8280692100524902,
+      "step": 13158
+    },
+    {
+      "epoch": 0.7975757575757576,
+      "grad_norm": 0.052157800644636154,
+      "learning_rate": 0.000674949494949495,
+      "loss": 2.838442802429199,
+      "step": 13160
+    },
+    {
+      "epoch": 0.7976969696969697,
+      "grad_norm": 0.05081898719072342,
+      "learning_rate": 0.0006745454545454546,
+      "loss": 2.803860902786255,
+      "step": 13162
+    },
+    {
+      "epoch": 0.7978181818181819,
+      "grad_norm": 0.046473365277051926,
+      "learning_rate": 0.0006741414141414142,
+      "loss": 2.730713129043579,
+      "step": 13164
+    },
+    {
+      "epoch": 0.797939393939394,
+      "grad_norm": 0.0503675602376461,
+      "learning_rate": 0.0006737373737373737,
+      "loss": 2.8000776767730713,
+      "step": 13166
+    },
+    {
+      "epoch": 0.798060606060606,
+      "grad_norm": 0.047450922429561615,
+      "learning_rate": 0.0006733333333333334,
+      "loss": 2.81809401512146,
+      "step": 13168
+    },
+    {
+      "epoch": 0.7981818181818182,
+      "grad_norm": 0.04956042766571045,
+      "learning_rate": 0.000672929292929293,
+      "loss": 2.8566439151763916,
+      "step": 13170
+    },
+    {
+      "epoch": 0.7983030303030303,
+      "grad_norm": 0.04845551401376724,
+      "learning_rate": 0.0006725252525252525,
+      "loss": 2.835970401763916,
+      "step": 13172
+    },
+    {
+      "epoch": 0.7984242424242424,
+      "grad_norm": 0.05379324033856392,
+      "learning_rate": 0.0006721212121212122,
+      "loss": 2.786409854888916,
+      "step": 13174
+    },
+    {
+      "epoch": 0.7985454545454546,
+      "grad_norm": 0.05436549708247185,
+      "learning_rate": 0.0006717171717171718,
+      "loss": 2.796386480331421,
+      "step": 13176
+    },
+    {
+      "epoch": 0.7986666666666666,
+      "grad_norm": 0.046918828040361404,
+      "learning_rate": 0.0006713131313131314,
+      "loss": 2.7975411415100098,
+      "step": 13178
+    },
+    {
+      "epoch": 0.7987878787878788,
+      "grad_norm": 0.04895201325416565,
+      "learning_rate": 0.0006709090909090909,
+      "loss": 2.8221256732940674,
+      "step": 13180
+    },
+    {
+      "epoch": 0.7989090909090909,
+      "grad_norm": 0.048175204545259476,
+      "learning_rate": 0.0006705050505050506,
+      "loss": 2.7906148433685303,
+      "step": 13182
+    },
+    {
+      "epoch": 0.799030303030303,
+      "grad_norm": 0.046021219342947006,
+      "learning_rate": 0.0006701010101010101,
+      "loss": 2.7953836917877197,
+      "step": 13184
+    },
+    {
+      "epoch": 0.7991515151515152,
+      "grad_norm": 0.057057954370975494,
+      "learning_rate": 0.0006696969696969698,
+      "loss": 2.799076557159424,
+      "step": 13186
+    },
+    {
+      "epoch": 0.7992727272727272,
+      "grad_norm": 0.05215838551521301,
+      "learning_rate": 0.0006692929292929293,
+      "loss": 2.765577793121338,
+      "step": 13188
+    },
+    {
+      "epoch": 0.7993939393939394,
+      "grad_norm": 0.051222022622823715,
+      "learning_rate": 0.0006688888888888889,
+      "loss": 2.825721502304077,
+      "step": 13190
+    },
+    {
+      "epoch": 0.7995151515151515,
+      "grad_norm": 0.0453946590423584,
+      "learning_rate": 0.0006684848484848485,
+      "loss": 2.825640916824341,
+      "step": 13192
+    },
+    {
+      "epoch": 0.7996363636363636,
+      "grad_norm": 0.04847656935453415,
+      "learning_rate": 0.0006680808080808081,
+      "loss": 2.814901113510132,
+      "step": 13194
+    },
+    {
+      "epoch": 0.7997575757575758,
+      "grad_norm": 0.047098420560359955,
+      "learning_rate": 0.0006676767676767677,
+      "loss": 2.8131229877471924,
+      "step": 13196
+    },
+    {
+      "epoch": 0.7998787878787879,
+      "grad_norm": 0.04574795812368393,
+      "learning_rate": 0.0006672727272727273,
+      "loss": 2.8427302837371826,
+      "step": 13198
+    },
+    {
+      "epoch": 0.8,
+      "grad_norm": 0.05004130303859711,
+      "learning_rate": 0.0006668686868686869,
+      "loss": 2.796891212463379,
+      "step": 13200
+    },
+    {
+      "epoch": 0.8,
+      "eval_loss": 2.0081608295440674,
+      "eval_runtime": 148.9358,
+      "eval_samples_per_second": 1.679,
+      "eval_steps_per_second": 0.087,
+      "step": 13200
     }
   ],
   "logging_steps": 2,
       "attributes": {}
     }
   },
+  "total_flos": 3.1619067480666644e+19,
   "train_batch_size": 20,
   "trial_name": null,
   "trial_params": null