Instructions to use Ba2han/model-sft-q2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ba2han/model-sft-q2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ba2han/model-sft-q2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ba2han/model-sft-q2", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("Ba2han/model-sft-q2", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use Ba2han/model-sft-q2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ba2han/model-sft-q2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/model-sft-q2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ba2han/model-sft-q2

SGLang

How to use Ba2han/model-sft-q2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ba2han/model-sft-q2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/model-sft-q2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ba2han/model-sft-q2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/model-sft-q2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use Ba2han/model-sft-q2 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/model-sft-q2 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/model-sft-q2 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ba2han/model-sft-q2 to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Ba2han/model-sft-q2",
    max_seq_length=2048,
)

Docker Model Runner
How to use Ba2han/model-sft-q2 with Docker Model Runner:
```
docker model run hf.co/Ba2han/model-sft-q2
```

Ba2han commited on 2 days ago

Commit

dd3ba70

verified ·

1 Parent(s): fff8ec6

Training in progress, step 7043, checkpoint

Browse files

Files changed (4) hide show

last-checkpoint/model.safetensors +1 -1
last-checkpoint/optimizer.pt +1 -1
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +2455 -4

last-checkpoint/model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:ead1ffa66fefc73d009e2283dd80283791464d453a0775b2de6554a6a8802359
 size 1049614696

 version https://git-lfs.github.com/spec/v1
+oid sha256:830c6e82e873a112a37dd424f4f371fe20d8a3252d03b78bfd3e287c8de3d81a
 size 1049614696

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:73c5985ef28acb6a48f8d3edce929eedd508dc36b1b351ffbcb05df26aa0b67b
 size 1372902609

 version https://git-lfs.github.com/spec/v1
+oid sha256:4e32fd6dca63545237e61e1d298819978874abd9335a646c649a89010c736cef
 size 1372902609

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7ce28be2d31f2157a8e0925dd44a92e7c3e4ac83585d62aed83f60148925a122
 size 1465

 version https://git-lfs.github.com/spec/v1
+oid sha256:c10a0c9be6ff9bdac8694187029e0ec06b326fd90a60892aa3c63f5e1914a58f
 size 1465

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,9 +2,9 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.9009744581905962,
   "eval_steps": 705,
-  "global_step": 6345,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
@@ -22284,6 +22284,2457 @@
       "eval_samples_per_second": 55.127,
       "eval_steps_per_second": 6.892,
       "step": 6345
     }
   ],
   "logging_steps": 2,
@@ -22298,12 +24749,12 @@
         "should_evaluate": false,
         "should_log": false,
         "should_save": true,
-        "should_training_stop": false
       },
       "attributes": {}
     }
   },
-  "total_flos": 4.709826150593987e+18,
   "train_batch_size": 4,
   "trial_name": null,
   "trial_params": null

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 1.0,
   "eval_steps": 705,
+  "global_step": 7043,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
       "eval_samples_per_second": 55.127,
       "eval_steps_per_second": 6.892,
       "step": 6345
+    },
+    {
+      "epoch": 0.9011164557411384,
+      "grad_norm": 0.1337890625,
+      "learning_rate": 4.9663883488730087e-05,
+      "loss": 1.6326470375061035,
+      "step": 6346
+    },
+    {
+      "epoch": 0.901400450842223,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 4.9439813315402384e-05,
+      "loss": 1.6485044956207275,
+      "step": 6348
+    },
+    {
+      "epoch": 0.9016844459433074,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 4.9215754393194874e-05,
+      "loss": 1.6287813186645508,
+      "step": 6350
+    },
+    {
+      "epoch": 0.901968441044392,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 4.89917112222393e-05,
+      "loss": 1.6801517009735107,
+      "step": 6352
+    },
+    {
+      "epoch": 0.9022524361454765,
+      "grad_norm": 0.130859375,
+      "learning_rate": 4.8767688302350956e-05,
+      "loss": 1.6600592136383057,
+      "step": 6354
+    },
+    {
+      "epoch": 0.902536431246561,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 4.8543690132938494e-05,
+      "loss": 1.6464087963104248,
+      "step": 6356
+    },
+    {
+      "epoch": 0.9028204263476455,
+      "grad_norm": 0.12890625,
+      "learning_rate": 4.8319721212913436e-05,
+      "loss": 1.6564900875091553,
+      "step": 6358
+    },
+    {
+      "epoch": 0.90310442144873,
+      "grad_norm": 0.12890625,
+      "learning_rate": 4.8095786040599844e-05,
+      "loss": 1.6286182403564453,
+      "step": 6360
+    },
+    {
+      "epoch": 0.9033884165498145,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 4.787188911364393e-05,
+      "loss": 1.640902042388916,
+      "step": 6362
+    },
+    {
+      "epoch": 0.9036724116508991,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 4.764803492892382e-05,
+      "loss": 1.6655956506729126,
+      "step": 6364
+    },
+    {
+      "epoch": 0.9039564067519835,
+      "grad_norm": 0.1337890625,
+      "learning_rate": 4.742422798245916e-05,
+      "loss": 1.629305362701416,
+      "step": 6366
+    },
+    {
+      "epoch": 0.9042404018530681,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 4.720047276932084e-05,
+      "loss": 1.6638224124908447,
+      "step": 6368
+    },
+    {
+      "epoch": 0.9045243969541525,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 4.697677378354069e-05,
+      "loss": 1.654644250869751,
+      "step": 6370
+    },
+    {
+      "epoch": 0.904808392055237,
+      "grad_norm": 0.12109375,
+      "learning_rate": 4.675313551802123e-05,
+      "loss": 1.679664969444275,
+      "step": 6372
+    },
+    {
+      "epoch": 0.9050923871563216,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 4.6529562464445484e-05,
+      "loss": 1.6713364124298096,
+      "step": 6374
+    },
+    {
+      "epoch": 0.905376382257406,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 4.630605911318667e-05,
+      "loss": 1.6539726257324219,
+      "step": 6376
+    },
+    {
+      "epoch": 0.9056603773584906,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 4.6082629953218086e-05,
+      "loss": 1.682896375656128,
+      "step": 6378
+    },
+    {
+      "epoch": 0.905944372459575,
+      "grad_norm": 0.12890625,
+      "learning_rate": 4.585927947202294e-05,
+      "loss": 1.6160575151443481,
+      "step": 6380
+    },
+    {
+      "epoch": 0.9062283675606596,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 4.563601215550422e-05,
+      "loss": 1.6658324003219604,
+      "step": 6382
+    },
+    {
+      "epoch": 0.9065123626617441,
+      "grad_norm": 0.130859375,
+      "learning_rate": 4.5412832487894495e-05,
+      "loss": 1.6549348831176758,
+      "step": 6384
+    },
+    {
+      "epoch": 0.9067963577628286,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 4.518974495166606e-05,
+      "loss": 1.6667382717132568,
+      "step": 6386
+    },
+    {
+      "epoch": 0.9070803528639131,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 4.496675402744072e-05,
+      "loss": 1.6492877006530762,
+      "step": 6388
+    },
+    {
+      "epoch": 0.9073643479649977,
+      "grad_norm": 0.140625,
+      "learning_rate": 4.4743864193899914e-05,
+      "loss": 1.688408374786377,
+      "step": 6390
+    },
+    {
+      "epoch": 0.9076483430660821,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 4.452107992769463e-05,
+      "loss": 1.6715737581253052,
+      "step": 6392
+    },
+    {
+      "epoch": 0.9079323381671666,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 4.4298405703355685e-05,
+      "loss": 1.6229228973388672,
+      "step": 6394
+    },
+    {
+      "epoch": 0.9082163332682511,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 4.407584599320368e-05,
+      "loss": 1.635927677154541,
+      "step": 6396
+    },
+    {
+      "epoch": 0.9085003283693356,
+      "grad_norm": 0.126953125,
+      "learning_rate": 4.385340526725928e-05,
+      "loss": 1.6395500898361206,
+      "step": 6398
+    },
+    {
+      "epoch": 0.9087843234704202,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 4.3631087993153375e-05,
+      "loss": 1.68593168258667,
+      "step": 6400
+    },
+    {
+      "epoch": 0.9090683185715046,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 4.3408898636037384e-05,
+      "loss": 1.6664785146713257,
+      "step": 6402
+    },
+    {
+      "epoch": 0.9093523136725892,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 4.318684165849359e-05,
+      "loss": 1.6475862264633179,
+      "step": 6404
+    },
+    {
+      "epoch": 0.9096363087736736,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 4.2964921520445466e-05,
+      "loss": 1.6552307605743408,
+      "step": 6406
+    },
+    {
+      "epoch": 0.9099203038747582,
+      "grad_norm": 0.130859375,
+      "learning_rate": 4.274314267906809e-05,
+      "loss": 1.6667717695236206,
+      "step": 6408
+    },
+    {
+      "epoch": 0.9102042989758427,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 4.252150958869871e-05,
+      "loss": 1.635591745376587,
+      "step": 6410
+    },
+    {
+      "epoch": 0.9104882940769272,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 4.230002670074719e-05,
+      "loss": 1.6460670232772827,
+      "step": 6412
+    },
+    {
+      "epoch": 0.9107722891780117,
+      "grad_norm": 0.126953125,
+      "learning_rate": 4.2078698463606674e-05,
+      "loss": 1.6729345321655273,
+      "step": 6414
+    },
+    {
+      "epoch": 0.9110562842790962,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 4.1857529322564136e-05,
+      "loss": 1.683729648590088,
+      "step": 6416
+    },
+    {
+      "epoch": 0.9113402793801807,
+      "grad_norm": 0.12109375,
+      "learning_rate": 4.163652371971125e-05,
+      "loss": 1.66416597366333,
+      "step": 6418
+    },
+    {
+      "epoch": 0.9116242744812652,
+      "grad_norm": 0.125,
+      "learning_rate": 4.141568609385504e-05,
+      "loss": 1.673246145248413,
+      "step": 6420
+    },
+    {
+      "epoch": 0.9119082695823497,
+      "grad_norm": 0.35546875,
+      "learning_rate": 4.119502088042882e-05,
+      "loss": 1.6377911567687988,
+      "step": 6422
+    },
+    {
+      "epoch": 0.9121922646834342,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 4.097453251140305e-05,
+      "loss": 1.6818692684173584,
+      "step": 6424
+    },
+    {
+      "epoch": 0.9124762597845187,
+      "grad_norm": 0.123046875,
+      "learning_rate": 4.0754225415196325e-05,
+      "loss": 1.664668083190918,
+      "step": 6426
+    },
+    {
+      "epoch": 0.9127602548856032,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 4.053410401658646e-05,
+      "loss": 1.648174524307251,
+      "step": 6428
+    },
+    {
+      "epoch": 0.9130442499866878,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 4.031417273662167e-05,
+      "loss": 1.6480052471160889,
+      "step": 6430
+    },
+    {
+      "epoch": 0.9133282450877722,
+      "grad_norm": 0.11767578125,
+      "learning_rate": 4.00944359925316e-05,
+      "loss": 1.61641263961792,
+      "step": 6432
+    },
+    {
+      "epoch": 0.9136122401888568,
+      "grad_norm": 0.125,
+      "learning_rate": 3.987489819763882e-05,
+      "loss": 1.6644868850708008,
+      "step": 6434
+    },
+    {
+      "epoch": 0.9138962352899412,
+      "grad_norm": 0.12890625,
+      "learning_rate": 3.965556376127007e-05,
+      "loss": 1.653461217880249,
+      "step": 6436
+    },
+    {
+      "epoch": 0.9141802303910258,
+      "grad_norm": 0.126953125,
+      "learning_rate": 3.943643708866764e-05,
+      "loss": 1.662238359451294,
+      "step": 6438
+    },
+    {
+      "epoch": 0.9144642254921103,
+      "grad_norm": 0.126953125,
+      "learning_rate": 3.921752258090108e-05,
+      "loss": 1.6478242874145508,
+      "step": 6440
+    },
+    {
+      "epoch": 0.9147482205931947,
+      "grad_norm": 0.126953125,
+      "learning_rate": 3.899882463477861e-05,
+      "loss": 1.7027912139892578,
+      "step": 6442
+    },
+    {
+      "epoch": 0.9150322156942793,
+      "grad_norm": 0.119140625,
+      "learning_rate": 3.878034764275897e-05,
+      "loss": 1.6240694522857666,
+      "step": 6444
+    },
+    {
+      "epoch": 0.9153162107953637,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 3.856209599286305e-05,
+      "loss": 1.63136887550354,
+      "step": 6446
+    },
+    {
+      "epoch": 0.9156002058964483,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 3.83440740685859e-05,
+      "loss": 1.6573362350463867,
+      "step": 6448
+    },
+    {
+      "epoch": 0.9158842009975328,
+      "grad_norm": 0.11962890625,
+      "learning_rate": 3.8126286248808586e-05,
+      "loss": 1.67368745803833,
+      "step": 6450
+    },
+    {
+      "epoch": 0.9161681960986173,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 3.790873690771032e-05,
+      "loss": 1.6893939971923828,
+      "step": 6452
+    },
+    {
+      "epoch": 0.9164521911997018,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 3.769143041468051e-05,
+      "loss": 1.641585111618042,
+      "step": 6454
+    },
+    {
+      "epoch": 0.9167361863007863,
+      "grad_norm": 0.123046875,
+      "learning_rate": 3.747437113423111e-05,
+      "loss": 1.647886872291565,
+      "step": 6456
+    },
+    {
+      "epoch": 0.9170201814018708,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 3.72575634259089e-05,
+      "loss": 1.667672872543335,
+      "step": 6458
+    },
+    {
+      "epoch": 0.9173041765029554,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 3.7041011644207943e-05,
+      "loss": 1.637958288192749,
+      "step": 6460
+    },
+    {
+      "epoch": 0.9175881716040398,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 3.682472013848207e-05,
+      "loss": 1.6648609638214111,
+      "step": 6462
+    },
+    {
+      "epoch": 0.9178721667051243,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 3.6608693252857625e-05,
+      "loss": 1.6248102188110352,
+      "step": 6464
+    },
+    {
+      "epoch": 0.9181561618062088,
+      "grad_norm": 0.11865234375,
+      "learning_rate": 3.639293532614616e-05,
+      "loss": 1.664893627166748,
+      "step": 6466
+    },
+    {
+      "epoch": 0.9184401569072933,
+      "grad_norm": 0.1357421875,
+      "learning_rate": 3.617745069175732e-05,
+      "loss": 1.6504709720611572,
+      "step": 6468
+    },
+    {
+      "epoch": 0.9187241520083779,
+      "grad_norm": 0.125,
+      "learning_rate": 3.5962243677611704e-05,
+      "loss": 1.666931390762329,
+      "step": 6470
+    },
+    {
+      "epoch": 0.9190081471094623,
+      "grad_norm": 0.1357421875,
+      "learning_rate": 3.5747318606054105e-05,
+      "loss": 1.6407499313354492,
+      "step": 6472
+    },
+    {
+      "epoch": 0.9192921422105469,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 3.5532679793766584e-05,
+      "loss": 1.66179621219635,
+      "step": 6474
+    },
+    {
+      "epoch": 0.9195761373116313,
+      "grad_norm": 0.166015625,
+      "learning_rate": 3.531833155168182e-05,
+      "loss": 1.6688923835754395,
+      "step": 6476
+    },
+    {
+      "epoch": 0.9198601324127159,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 3.510427818489643e-05,
+      "loss": 1.6631419658660889,
+      "step": 6478
+    },
+    {
+      "epoch": 0.9201441275138004,
+      "grad_norm": 0.130859375,
+      "learning_rate": 3.48905239925847e-05,
+      "loss": 1.6476298570632935,
+      "step": 6480
+    },
+    {
+      "epoch": 0.9204281226148849,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 3.467707326791204e-05,
+      "loss": 1.7067539691925049,
+      "step": 6482
+    },
+    {
+      "epoch": 0.9207121177159694,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 3.4463930297948877e-05,
+      "loss": 1.5974493026733398,
+      "step": 6484
+    },
+    {
+      "epoch": 0.920996112817054,
+      "grad_norm": 0.123046875,
+      "learning_rate": 3.4251099363584466e-05,
+      "loss": 1.6740283966064453,
+      "step": 6486
+    },
+    {
+      "epoch": 0.9212801079181384,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 3.4038584739440985e-05,
+      "loss": 1.6813969612121582,
+      "step": 6488
+    },
+    {
+      "epoch": 0.9215641030192229,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 3.3826390693787656e-05,
+      "loss": 1.6607600450515747,
+      "step": 6490
+    },
+    {
+      "epoch": 0.9218480981203074,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 3.3614521488455026e-05,
+      "loss": 1.6704461574554443,
+      "step": 6492
+    },
+    {
+      "epoch": 0.9221320932213919,
+      "grad_norm": 0.125,
+      "learning_rate": 3.3402981378749285e-05,
+      "loss": 1.6542103290557861,
+      "step": 6494
+    },
+    {
+      "epoch": 0.9224160883224765,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 3.319177461336695e-05,
+      "loss": 1.6618162393569946,
+      "step": 6496
+    },
+    {
+      "epoch": 0.9227000834235609,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 3.298090543430944e-05,
+      "loss": 1.670125961303711,
+      "step": 6498
+    },
+    {
+      "epoch": 0.9229840785246455,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 3.277037807679783e-05,
+      "loss": 1.6545251607894897,
+      "step": 6500
+    },
+    {
+      "epoch": 0.9232680736257299,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 3.256019676918793e-05,
+      "loss": 1.6476445198059082,
+      "step": 6502
+    },
+    {
+      "epoch": 0.9235520687268145,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 3.235036573288525e-05,
+      "loss": 1.679513931274414,
+      "step": 6504
+    },
+    {
+      "epoch": 0.923836063827899,
+      "grad_norm": 0.12109375,
+      "learning_rate": 3.2140889182260225e-05,
+      "loss": 1.630507230758667,
+      "step": 6506
+    },
+    {
+      "epoch": 0.9241200589289835,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 3.19317713245636e-05,
+      "loss": 1.6423308849334717,
+      "step": 6508
+    },
+    {
+      "epoch": 0.924404054030068,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 3.172301635984194e-05,
+      "loss": 1.6178326606750488,
+      "step": 6510
+    },
+    {
+      "epoch": 0.9246880491311524,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 3.1514628480853226e-05,
+      "loss": 1.6421864032745361,
+      "step": 6512
+    },
+    {
+      "epoch": 0.924972044232237,
+      "grad_norm": 0.130859375,
+      "learning_rate": 3.13066118729827e-05,
+      "loss": 1.6278722286224365,
+      "step": 6514
+    },
+    {
+      "epoch": 0.9252560393333215,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 3.1098970714158746e-05,
+      "loss": 1.6514774560928345,
+      "step": 6516
+    },
+    {
+      "epoch": 0.925540034434406,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 3.0891709174769e-05,
+      "loss": 1.678621768951416,
+      "step": 6518
+    },
+    {
+      "epoch": 0.9258240295354905,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 3.068483141757665e-05,
+      "loss": 1.6730953454971313,
+      "step": 6520
+    },
+    {
+      "epoch": 0.926108024636575,
+      "grad_norm": 0.126953125,
+      "learning_rate": 3.0478341597636762e-05,
+      "loss": 1.6621979475021362,
+      "step": 6522
+    },
+    {
+      "epoch": 0.9263920197376595,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 3.0272243862212756e-05,
+      "loss": 1.6461114883422852,
+      "step": 6524
+    },
+    {
+      "epoch": 0.9266760148387441,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 3.0066542350693317e-05,
+      "loss": 1.6254409551620483,
+      "step": 6526
+    },
+    {
+      "epoch": 0.9269600099398285,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.9861241194509048e-05,
+      "loss": 1.6794788837432861,
+      "step": 6528
+    },
+    {
+      "epoch": 0.9272440050409131,
+      "grad_norm": 0.123046875,
+      "learning_rate": 2.9656344517049656e-05,
+      "loss": 1.6605265140533447,
+      "step": 6530
+    },
+    {
+      "epoch": 0.9275280001419975,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 2.945185643358096e-05,
+      "loss": 1.6932629346847534,
+      "step": 6532
+    },
+    {
+      "epoch": 0.9278119952430821,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.924778105116241e-05,
+      "loss": 1.6501400470733643,
+      "step": 6534
+    },
+    {
+      "epoch": 0.9280959903441666,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 2.90441224685645e-05,
+      "loss": 1.6564494371414185,
+      "step": 6536
+    },
+    {
+      "epoch": 0.928379985445251,
+      "grad_norm": 0.125,
+      "learning_rate": 2.8840884776186515e-05,
+      "loss": 1.657090425491333,
+      "step": 6538
+    },
+    {
+      "epoch": 0.9286639805463356,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 2.863807205597422e-05,
+      "loss": 1.6438353061676025,
+      "step": 6540
+    },
+    {
+      "epoch": 0.92894797564742,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.8435688381338116e-05,
+      "loss": 1.6713764667510986,
+      "step": 6542
+    },
+    {
+      "epoch": 0.9292319707485046,
+      "grad_norm": 0.125,
+      "learning_rate": 2.8233737817071405e-05,
+      "loss": 1.639617681503296,
+      "step": 6544
+    },
+    {
+      "epoch": 0.9295159658495891,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 2.8032224419268517e-05,
+      "loss": 1.6151936054229736,
+      "step": 6546
+    },
+    {
+      "epoch": 0.9297999609506736,
+      "grad_norm": 0.123046875,
+      "learning_rate": 2.7831152235243517e-05,
+      "loss": 1.6277703046798706,
+      "step": 6548
+    },
+    {
+      "epoch": 0.9300839560517581,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 2.7630525303448897e-05,
+      "loss": 1.6260011196136475,
+      "step": 6550
+    },
+    {
+      "epoch": 0.9303679511528427,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 2.7430347653394418e-05,
+      "loss": 1.6701017618179321,
+      "step": 6552
+    },
+    {
+      "epoch": 0.9306519462539271,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.7230623305566254e-05,
+      "loss": 1.6378331184387207,
+      "step": 6554
+    },
+    {
+      "epoch": 0.9309359413550117,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 2.7031356271346144e-05,
+      "loss": 1.6505334377288818,
+      "step": 6556
+    },
+    {
+      "epoch": 0.9312199364560961,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 2.683255055293084e-05,
+      "loss": 1.6599491834640503,
+      "step": 6558
+    },
+    {
+      "epoch": 0.9315039315571806,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 2.663421014325188e-05,
+      "loss": 1.6618008613586426,
+      "step": 6560
+    },
+    {
+      "epoch": 0.9317879266582652,
+      "grad_norm": 0.12109375,
+      "learning_rate": 2.6436339025895084e-05,
+      "loss": 1.6160664558410645,
+      "step": 6562
+    },
+    {
+      "epoch": 0.9320719217593496,
+      "grad_norm": 0.12109375,
+      "learning_rate": 2.6238941175020903e-05,
+      "loss": 1.624788761138916,
+      "step": 6564
+    },
+    {
+      "epoch": 0.9323559168604342,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 2.604202055528429e-05,
+      "loss": 1.6408872604370117,
+      "step": 6566
+    },
+    {
+      "epoch": 0.9326399119615186,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.5845581121755335e-05,
+      "loss": 1.667233943939209,
+      "step": 6568
+    },
+    {
+      "epoch": 0.9329239070626032,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 2.5649626819839555e-05,
+      "loss": 1.6937549114227295,
+      "step": 6570
+    },
+    {
+      "epoch": 0.9332079021636877,
+      "grad_norm": 0.125,
+      "learning_rate": 2.545416158519891e-05,
+      "loss": 1.657670497894287,
+      "step": 6572
+    },
+    {
+      "epoch": 0.9334918972647722,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 2.5259189343672583e-05,
+      "loss": 1.6195838451385498,
+      "step": 6574
+    },
+    {
+      "epoch": 0.9337758923658567,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.5064714011198252e-05,
+      "loss": 1.6312012672424316,
+      "step": 6576
+    },
+    {
+      "epoch": 0.9340598874669412,
+      "grad_norm": 0.1328125,
+      "learning_rate": 2.487073949373332e-05,
+      "loss": 1.6684744358062744,
+      "step": 6578
+    },
+    {
+      "epoch": 0.9343438825680257,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.4677269687176526e-05,
+      "loss": 1.6751328706741333,
+      "step": 6580
+    },
+    {
+      "epoch": 0.9346278776691103,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 2.4484308477289758e-05,
+      "loss": 1.638372540473938,
+      "step": 6582
+    },
+    {
+      "epoch": 0.9349118727701947,
+      "grad_norm": 0.1181640625,
+      "learning_rate": 2.4291859739619888e-05,
+      "loss": 1.662428379058838,
+      "step": 6584
+    },
+    {
+      "epoch": 0.9351958678712792,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 2.409992733942102e-05,
+      "loss": 1.6858851909637451,
+      "step": 6586
+    },
+    {
+      "epoch": 0.9354798629723637,
+      "grad_norm": 0.12890625,
+      "learning_rate": 2.3908515131576805e-05,
+      "loss": 1.6546742916107178,
+      "step": 6588
+    },
+    {
+      "epoch": 0.9357638580734482,
+      "grad_norm": 0.11962890625,
+      "learning_rate": 2.3717626960523103e-05,
+      "loss": 1.6390483379364014,
+      "step": 6590
+    },
+    {
+      "epoch": 0.9360478531745328,
+      "grad_norm": 0.1328125,
+      "learning_rate": 2.352726666017065e-05,
+      "loss": 1.6461637020111084,
+      "step": 6592
+    },
+    {
+      "epoch": 0.9363318482756172,
+      "grad_norm": 0.1162109375,
+      "learning_rate": 2.3337438053828136e-05,
+      "loss": 1.619539737701416,
+      "step": 6594
+    },
+    {
+      "epoch": 0.9366158433767018,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.3148144954125372e-05,
+      "loss": 1.6508175134658813,
+      "step": 6596
+    },
+    {
+      "epoch": 0.9368998384777862,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.295939116293681e-05,
+      "loss": 1.6342027187347412,
+      "step": 6598
+    },
+    {
+      "epoch": 0.9371838335788708,
+      "grad_norm": 0.12109375,
+      "learning_rate": 2.277118047130503e-05,
+      "loss": 1.6193323135375977,
+      "step": 6600
+    },
+    {
+      "epoch": 0.9374678286799553,
+      "grad_norm": 0.12109375,
+      "learning_rate": 2.2583516659364716e-05,
+      "loss": 1.6812199354171753,
+      "step": 6602
+    },
+    {
+      "epoch": 0.9377518237810398,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.2396403496266676e-05,
+      "loss": 1.6526546478271484,
+      "step": 6604
+    },
+    {
+      "epoch": 0.9380358188821243,
+      "grad_norm": 0.1328125,
+      "learning_rate": 2.220984474010221e-05,
+      "loss": 1.6458194255828857,
+      "step": 6606
+    },
+    {
+      "epoch": 0.9383198139832087,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 2.202384413782754e-05,
+      "loss": 1.6680066585540771,
+      "step": 6608
+    },
+    {
+      "epoch": 0.9386038090842933,
+      "grad_norm": 0.11962890625,
+      "learning_rate": 2.183840542518857e-05,
+      "loss": 1.6331427097320557,
+      "step": 6610
+    },
+    {
+      "epoch": 0.9388878041853778,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.1653532326645963e-05,
+      "loss": 1.6482030153274536,
+      "step": 6612
+    },
+    {
+      "epoch": 0.9391717992864623,
+      "grad_norm": 0.12890625,
+      "learning_rate": 2.146922855530018e-05,
+      "loss": 1.628589153289795,
+      "step": 6614
+    },
+    {
+      "epoch": 0.9394557943875468,
+      "grad_norm": 0.125,
+      "learning_rate": 2.1285497812817044e-05,
+      "loss": 1.631730556488037,
+      "step": 6616
+    },
+    {
+      "epoch": 0.9397397894886313,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 2.110234378935323e-05,
+      "loss": 1.6600490808486938,
+      "step": 6618
+    },
+    {
+      "epoch": 0.9400237845897158,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.091977016348235e-05,
+      "loss": 1.7200943231582642,
+      "step": 6620
+    },
+    {
+      "epoch": 0.9403077796908004,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 2.0737780602120914e-05,
+      "loss": 1.6427900791168213,
+      "step": 6622
+    },
+    {
+      "epoch": 0.9405917747918848,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 2.0556378760454747e-05,
+      "loss": 1.6485931873321533,
+      "step": 6624
+    },
+    {
+      "epoch": 0.9408757698929694,
+      "grad_norm": 0.138671875,
+      "learning_rate": 2.037556828186555e-05,
+      "loss": 1.6425774097442627,
+      "step": 6626
+    },
+    {
+      "epoch": 0.9411597649940538,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 2.019535279785778e-05,
+      "loss": 1.6571080684661865,
+      "step": 6628
+    },
+    {
+      "epoch": 0.9414437600951383,
+      "grad_norm": 0.130859375,
+      "learning_rate": 2.001573592798563e-05,
+      "loss": 1.6546900272369385,
+      "step": 6630
+    },
+    {
+      "epoch": 0.9417277551962229,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.9836721279780373e-05,
+      "loss": 1.6568119525909424,
+      "step": 6632
+    },
+    {
+      "epoch": 0.9420117502973073,
+      "grad_norm": 0.1181640625,
+      "learning_rate": 1.965831244867795e-05,
+      "loss": 1.6468884944915771,
+      "step": 6634
+    },
+    {
+      "epoch": 0.9422957453983919,
+      "grad_norm": 0.12890625,
+      "learning_rate": 1.948051301794666e-05,
+      "loss": 1.6835911273956299,
+      "step": 6636
+    },
+    {
+      "epoch": 0.9425797404994763,
+      "grad_norm": 0.123046875,
+      "learning_rate": 1.9303326558615315e-05,
+      "loss": 1.6839208602905273,
+      "step": 6638
+    },
+    {
+      "epoch": 0.9428637356005609,
+      "grad_norm": 0.123046875,
+      "learning_rate": 1.912675662940133e-05,
+      "loss": 1.6442358493804932,
+      "step": 6640
+    },
+    {
+      "epoch": 0.9431477307016454,
+      "grad_norm": 0.12109375,
+      "learning_rate": 1.8950806776639503e-05,
+      "loss": 1.6288275718688965,
+      "step": 6642
+    },
+    {
+      "epoch": 0.9434317258027299,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.8775480534210526e-05,
+      "loss": 1.6193206310272217,
+      "step": 6644
+    },
+    {
+      "epoch": 0.9437157209038144,
+      "grad_norm": 0.125,
+      "learning_rate": 1.8600781423470264e-05,
+      "loss": 1.675943374633789,
+      "step": 6646
+    },
+    {
+      "epoch": 0.943999716004899,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.842671295317875e-05,
+      "loss": 1.6317131519317627,
+      "step": 6648
+    },
+    {
+      "epoch": 0.9442837111059834,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 1.8253278619429994e-05,
+      "loss": 1.642336368560791,
+      "step": 6650
+    },
+    {
+      "epoch": 0.944567706207068,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.808048190558153e-05,
+      "loss": 1.634922742843628,
+      "step": 6652
+    },
+    {
+      "epoch": 0.9448517013081524,
+      "grad_norm": 0.13671875,
+      "learning_rate": 1.7908326282184666e-05,
+      "loss": 1.6575052738189697,
+      "step": 6654
+    },
+    {
+      "epoch": 0.9451356964092369,
+      "grad_norm": 0.11962890625,
+      "learning_rate": 1.7736815206914544e-05,
+      "loss": 1.6284030675888062,
+      "step": 6656
+    },
+    {
+      "epoch": 0.9454196915103215,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 1.7565952124500927e-05,
+      "loss": 1.6644654273986816,
+      "step": 6658
+    },
+    {
+      "epoch": 0.9457036866114059,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.739574046665884e-05,
+      "loss": 1.625715732574463,
+      "step": 6660
+    },
+    {
+      "epoch": 0.9459876817124905,
+      "grad_norm": 0.126953125,
+      "learning_rate": 1.7226183652019767e-05,
+      "loss": 1.6447319984436035,
+      "step": 6662
+    },
+    {
+      "epoch": 0.9462716768135749,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.7057285086062897e-05,
+      "loss": 1.6117002964019775,
+      "step": 6664
+    },
+    {
+      "epoch": 0.9465556719146595,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 1.688904816104674e-05,
+      "loss": 1.632535696029663,
+      "step": 6666
+    },
+    {
+      "epoch": 0.946839667015744,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 1.67214762559411e-05,
+      "loss": 1.6630303859710693,
+      "step": 6668
+    },
+    {
+      "epoch": 0.9471236621168285,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 1.6554572736359052e-05,
+      "loss": 1.6721594333648682,
+      "step": 6670
+    },
+    {
+      "epoch": 0.947407657217913,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 1.6388340954489458e-05,
+      "loss": 1.6387388706207275,
+      "step": 6672
+    },
+    {
+      "epoch": 0.9476916523189975,
+      "grad_norm": 0.12890625,
+      "learning_rate": 1.6222784249029566e-05,
+      "loss": 1.6531705856323242,
+      "step": 6674
+    },
+    {
+      "epoch": 0.947975647420082,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.6057905945118056e-05,
+      "loss": 1.6726493835449219,
+      "step": 6676
+    },
+    {
+      "epoch": 0.9482596425211665,
+      "grad_norm": 0.123046875,
+      "learning_rate": 1.5893709354268143e-05,
+      "loss": 1.6192290782928467,
+      "step": 6678
+    },
+    {
+      "epoch": 0.948543637622251,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.5730197774301114e-05,
+      "loss": 1.6476869583129883,
+      "step": 6680
+    },
+    {
+      "epoch": 0.9488276327233355,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 1.5567374489280096e-05,
+      "loss": 1.692678451538086,
+      "step": 6682
+    },
+    {
+      "epoch": 0.94911162782442,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 1.5405242769444127e-05,
+      "loss": 1.650191307067871,
+      "step": 6684
+    },
+    {
+      "epoch": 0.9493956229255045,
+      "grad_norm": 0.125,
+      "learning_rate": 1.5243805871142402e-05,
+      "loss": 1.6335558891296387,
+      "step": 6686
+    },
+    {
+      "epoch": 0.9496796180265891,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 1.5083067036768905e-05,
+      "loss": 1.6644365787506104,
+      "step": 6688
+    },
+    {
+      "epoch": 0.9499636131276735,
+      "grad_norm": 0.125,
+      "learning_rate": 1.4923029494697348e-05,
+      "loss": 1.640918493270874,
+      "step": 6690
+    },
+    {
+      "epoch": 0.9502476082287581,
+      "grad_norm": 0.1328125,
+      "learning_rate": 1.476369645921622e-05,
+      "loss": 1.639884352684021,
+      "step": 6692
+    },
+    {
+      "epoch": 0.9505316033298425,
+      "grad_norm": 0.126953125,
+      "learning_rate": 1.4605071130464293e-05,
+      "loss": 1.7039388418197632,
+      "step": 6694
+    },
+    {
+      "epoch": 0.9508155984309271,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 1.4447156694366332e-05,
+      "loss": 1.6638362407684326,
+      "step": 6696
+    },
+    {
+      "epoch": 0.9510995935320116,
+      "grad_norm": 0.125,
+      "learning_rate": 1.428995632256917e-05,
+      "loss": 1.6702481508255005,
+      "step": 6698
+    },
+    {
+      "epoch": 0.9513835886330961,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.4133473172377882e-05,
+      "loss": 1.6522693634033203,
+      "step": 6700
+    },
+    {
+      "epoch": 0.9516675837341806,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 1.3977710386692473e-05,
+      "loss": 1.6674026250839233,
+      "step": 6702
+    },
+    {
+      "epoch": 0.951951578835265,
+      "grad_norm": 0.125,
+      "learning_rate": 1.38226710939447e-05,
+      "loss": 1.6179451942443848,
+      "step": 6704
+    },
+    {
+      "epoch": 0.9522355739363496,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 1.366835840803531e-05,
+      "loss": 1.654510498046875,
+      "step": 6706
+    },
+    {
+      "epoch": 0.9525195690374341,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 1.3514775428271393e-05,
+      "loss": 1.673440933227539,
+      "step": 6708
+    },
+    {
+      "epoch": 0.9528035641385186,
+      "grad_norm": 0.1171875,
+      "learning_rate": 1.336192523930423e-05,
+      "loss": 1.621158480644226,
+      "step": 6710
+    },
+    {
+      "epoch": 0.9530875592396031,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 1.320981091106725e-05,
+      "loss": 1.6313719749450684,
+      "step": 6712
+    },
+    {
+      "epoch": 0.9533715543406877,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.3058435498714494e-05,
+      "loss": 1.6824302673339844,
+      "step": 6714
+    },
+    {
+      "epoch": 0.9536555494417721,
+      "grad_norm": 0.1318359375,
+      "learning_rate": 1.2907802042559119e-05,
+      "loss": 1.6434886455535889,
+      "step": 6716
+    },
+    {
+      "epoch": 0.9539395445428567,
+      "grad_norm": 0.125,
+      "learning_rate": 1.2757913568012408e-05,
+      "loss": 1.6681139469146729,
+      "step": 6718
+    },
+    {
+      "epoch": 0.9542235396439411,
+      "grad_norm": 0.123046875,
+      "learning_rate": 1.2608773085523017e-05,
+      "loss": 1.676089882850647,
+      "step": 6720
+    },
+    {
+      "epoch": 0.9545075347450257,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 1.2460383590516456e-05,
+      "loss": 1.653357744216919,
+      "step": 6722
+    },
+    {
+      "epoch": 0.9547915298461102,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.2312748063335006e-05,
+      "loss": 1.6455838680267334,
+      "step": 6724
+    },
+    {
+      "epoch": 0.9550755249471946,
+      "grad_norm": 0.11669921875,
+      "learning_rate": 1.2165869469177731e-05,
+      "loss": 1.6417770385742188,
+      "step": 6726
+    },
+    {
+      "epoch": 0.9553595200482792,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 1.2019750758041099e-05,
+      "loss": 1.6262093782424927,
+      "step": 6728
+    },
+    {
+      "epoch": 0.9556435151493636,
+      "grad_norm": 0.1357421875,
+      "learning_rate": 1.187439486465956e-05,
+      "loss": 1.6558501720428467,
+      "step": 6730
+    },
+    {
+      "epoch": 0.9559275102504482,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 1.1729804708446768e-05,
+      "loss": 1.6913039684295654,
+      "step": 6732
+    },
+    {
+      "epoch": 0.9562115053515327,
+      "grad_norm": 0.126953125,
+      "learning_rate": 1.158598319343675e-05,
+      "loss": 1.6481530666351318,
+      "step": 6734
+    },
+    {
+      "epoch": 0.9564955004526172,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 1.1442933208225786e-05,
+      "loss": 1.654104471206665,
+      "step": 6736
+    },
+    {
+      "epoch": 0.9567794955537017,
+      "grad_norm": 0.125,
+      "learning_rate": 1.130065762591423e-05,
+      "loss": 1.642432689666748,
+      "step": 6738
+    },
+    {
+      "epoch": 0.9570634906547862,
+      "grad_norm": 0.125,
+      "learning_rate": 1.1159159304048934e-05,
+      "loss": 1.6536848545074463,
+      "step": 6740
+    },
+    {
+      "epoch": 0.9573474857558707,
+      "grad_norm": 0.12109375,
+      "learning_rate": 1.1018441084565728e-05,
+      "loss": 1.6880589723587036,
+      "step": 6742
+    },
+    {
+      "epoch": 0.9576314808569553,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 1.0878505793732452e-05,
+      "loss": 1.6856951713562012,
+      "step": 6744
+    },
+    {
+      "epoch": 0.9579154759580397,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 1.0739356242092108e-05,
+      "loss": 1.6466474533081055,
+      "step": 6746
+    },
+    {
+      "epoch": 0.9581994710591242,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.0600995224406462e-05,
+      "loss": 1.6551868915557861,
+      "step": 6748
+    },
+    {
+      "epoch": 0.9584834661602087,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 1.0463425519599934e-05,
+      "loss": 1.6469042301177979,
+      "step": 6750
+    },
+    {
+      "epoch": 0.9587674612612932,
+      "grad_norm": 0.12890625,
+      "learning_rate": 1.0326649890703682e-05,
+      "loss": 1.6663037538528442,
+      "step": 6752
+    },
+    {
+      "epoch": 0.9590514563623778,
+      "grad_norm": 0.125,
+      "learning_rate": 1.0190671084800257e-05,
+      "loss": 1.6584124565124512,
+      "step": 6754
+    },
+    {
+      "epoch": 0.9593354514634622,
+      "grad_norm": 0.11767578125,
+      "learning_rate": 1.0055491832968239e-05,
+      "loss": 1.6256663799285889,
+      "step": 6756
+    },
+    {
+      "epoch": 0.9596194465645468,
+      "grad_norm": 0.12890625,
+      "learning_rate": 9.921114850227597e-06,
+      "loss": 1.6238288879394531,
+      "step": 6758
+    },
+    {
+      "epoch": 0.9599034416656312,
+      "grad_norm": 0.125,
+      "learning_rate": 9.78754283548498e-06,
+      "loss": 1.6390256881713867,
+      "step": 6760
+    },
+    {
+      "epoch": 0.9601874367667158,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 9.65477847147966e-06,
+      "loss": 1.6369152069091797,
+      "step": 6762
+    },
+    {
+      "epoch": 0.9604714318678003,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 9.522824424729459e-06,
+      "loss": 1.6385613679885864,
+      "step": 6764
+    },
+    {
+      "epoch": 0.9607554269688848,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 9.391683345477436e-06,
+      "loss": 1.6186283826828003,
+      "step": 6766
+    },
+    {
+      "epoch": 0.9610394220699693,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 9.261357867638436e-06,
+      "loss": 1.6255123615264893,
+      "step": 6768
+    },
+    {
+      "epoch": 0.9613234171710539,
+      "grad_norm": 0.119140625,
+      "learning_rate": 9.131850608746351e-06,
+      "loss": 1.6574430465698242,
+      "step": 6770
+    },
+    {
+      "epoch": 0.9616074122721383,
+      "grad_norm": 0.12109375,
+      "learning_rate": 9.00316416990143e-06,
+      "loss": 1.6040034294128418,
+      "step": 6772
+    },
+    {
+      "epoch": 0.9618914073732228,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 8.875301135718094e-06,
+      "loss": 1.6074063777923584,
+      "step": 6774
+    },
+    {
+      "epoch": 0.9621754024743073,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 8.748264074273065e-06,
+      "loss": 1.6380449533462524,
+      "step": 6776
+    },
+    {
+      "epoch": 0.9624593975753918,
+      "grad_norm": 0.123046875,
+      "learning_rate": 8.622055537053692e-06,
+      "loss": 1.6326104402542114,
+      "step": 6778
+    },
+    {
+      "epoch": 0.9627433926764764,
+      "grad_norm": 0.125,
+      "learning_rate": 8.496678058906765e-06,
+      "loss": 1.6591897010803223,
+      "step": 6780
+    },
+    {
+      "epoch": 0.9630273877775608,
+      "grad_norm": 0.126953125,
+      "learning_rate": 8.372134157987588e-06,
+      "loss": 1.6626744270324707,
+      "step": 6782
+    },
+    {
+      "epoch": 0.9633113828786454,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 8.248426335709442e-06,
+      "loss": 1.6613175868988037,
+      "step": 6784
+    },
+    {
+      "epoch": 0.9635953779797298,
+      "grad_norm": 0.130859375,
+      "learning_rate": 8.125557076693268e-06,
+      "loss": 1.6723227500915527,
+      "step": 6786
+    },
+    {
+      "epoch": 0.9638793730808144,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 8.003528848717828e-06,
+      "loss": 1.6429615020751953,
+      "step": 6788
+    },
+    {
+      "epoch": 0.9641633681818988,
+      "grad_norm": 0.12109375,
+      "learning_rate": 7.882344102670108e-06,
+      "loss": 1.6666431427001953,
+      "step": 6790
+    },
+    {
+      "epoch": 0.9644473632829834,
+      "grad_norm": 0.125,
+      "learning_rate": 7.762005272496137e-06,
+      "loss": 1.6811373233795166,
+      "step": 6792
+    },
+    {
+      "epoch": 0.9647313583840679,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 7.642514775152027e-06,
+      "loss": 1.6308224201202393,
+      "step": 6794
+    },
+    {
+      "epoch": 0.9650153534851523,
+      "grad_norm": 0.1162109375,
+      "learning_rate": 7.5238750105555035e-06,
+      "loss": 1.6672980785369873,
+      "step": 6796
+    },
+    {
+      "epoch": 0.9652993485862369,
+      "grad_norm": 0.126953125,
+      "learning_rate": 7.406088361537639e-06,
+      "loss": 1.6616121530532837,
+      "step": 6798
+    },
+    {
+      "epoch": 0.9655833436873213,
+      "grad_norm": 0.126953125,
+      "learning_rate": 7.289157193795082e-06,
+      "loss": 1.6759233474731445,
+      "step": 6800
+    },
+    {
+      "epoch": 0.9658673387884059,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 7.173083855842444e-06,
+      "loss": 1.6492140293121338,
+      "step": 6802
+    },
+    {
+      "epoch": 0.9661513338894904,
+      "grad_norm": 0.1328125,
+      "learning_rate": 7.057870678965189e-06,
+      "loss": 1.6711063385009766,
+      "step": 6804
+    },
+    {
+      "epoch": 0.9664353289905749,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 6.943519977172808e-06,
+      "loss": 1.638947606086731,
+      "step": 6806
+    },
+    {
+      "epoch": 0.9667193240916594,
+      "grad_norm": 0.12890625,
+      "learning_rate": 6.830034047152328e-06,
+      "loss": 1.679686188697815,
+      "step": 6808
+    },
+    {
+      "epoch": 0.967003319192744,
+      "grad_norm": 0.125,
+      "learning_rate": 6.7174151682221854e-06,
+      "loss": 1.6446295976638794,
+      "step": 6810
+    },
+    {
+      "epoch": 0.9672873142938284,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 6.6056656022864315e-06,
+      "loss": 1.6103341579437256,
+      "step": 6812
+    },
+    {
+      "epoch": 0.967571309394913,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 6.494787593789376e-06,
+      "loss": 1.6539026498794556,
+      "step": 6814
+    },
+    {
+      "epoch": 0.9678553044959974,
+      "grad_norm": 0.12890625,
+      "learning_rate": 6.384783369670405e-06,
+      "loss": 1.6685879230499268,
+      "step": 6816
+    },
+    {
+      "epoch": 0.968139299597082,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 6.2756551393193195e-06,
+      "loss": 1.661516785621643,
+      "step": 6818
+    },
+    {
+      "epoch": 0.9684232946981665,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 6.16740509453193e-06,
+      "loss": 1.7060528993606567,
+      "step": 6820
+    },
+    {
+      "epoch": 0.9687072897992509,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 6.060035409466086e-06,
+      "loss": 1.6504364013671875,
+      "step": 6822
+    },
+    {
+      "epoch": 0.9689912849003355,
+      "grad_norm": 0.126953125,
+      "learning_rate": 5.953548240597945e-06,
+      "loss": 1.6967990398406982,
+      "step": 6824
+    },
+    {
+      "epoch": 0.9692752800014199,
+      "grad_norm": 0.126953125,
+      "learning_rate": 5.847945726678672e-06,
+      "loss": 1.659487009048462,
+      "step": 6826
+    },
+    {
+      "epoch": 0.9695592751025045,
+      "grad_norm": 0.126953125,
+      "learning_rate": 5.743229988691551e-06,
+      "loss": 1.6479309797286987,
+      "step": 6828
+    },
+    {
+      "epoch": 0.969843270203589,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 5.6394031298092785e-06,
+      "loss": 1.6425352096557617,
+      "step": 6830
+    },
+    {
+      "epoch": 0.9701272653046735,
+      "grad_norm": 0.125,
+      "learning_rate": 5.536467235351839e-06,
+      "loss": 1.670881986618042,
+      "step": 6832
+    },
+    {
+      "epoch": 0.970411260405758,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 5.434424372744473e-06,
+      "loss": 1.6419832706451416,
+      "step": 6834
+    },
+    {
+      "epoch": 0.9706952555068425,
+      "grad_norm": 0.123046875,
+      "learning_rate": 5.333276591476338e-06,
+      "loss": 1.6520084142684937,
+      "step": 6836
+    },
+    {
+      "epoch": 0.970979250607927,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 5.233025923059182e-06,
+      "loss": 1.6769835948944092,
+      "step": 6838
+    },
+    {
+      "epoch": 0.9712632457090116,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 5.133674380986664e-06,
+      "loss": 1.6453429460525513,
+      "step": 6840
+    },
+    {
+      "epoch": 0.971547240810096,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 5.03522396069378e-06,
+      "loss": 1.644869327545166,
+      "step": 6842
+    },
+    {
+      "epoch": 0.9718312359111805,
+      "grad_norm": 0.125,
+      "learning_rate": 4.9376766395169525e-06,
+      "loss": 1.6352219581604004,
+      "step": 6844
+    },
+    {
+      "epoch": 0.972115231012265,
+      "grad_norm": 0.125,
+      "learning_rate": 4.841034376654158e-06,
+      "loss": 1.6524848937988281,
+      "step": 6846
+    },
+    {
+      "epoch": 0.9723992261133495,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 4.745299113125701e-06,
+      "loss": 1.6345412731170654,
+      "step": 6848
+    },
+    {
+      "epoch": 0.9726832212144341,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 4.650472771735137e-06,
+      "loss": 1.6433501243591309,
+      "step": 6850
+    },
+    {
+      "epoch": 0.9729672163155185,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 4.556557257030708e-06,
+      "loss": 1.6776695251464844,
+      "step": 6852
+    },
+    {
+      "epoch": 0.9732512114166031,
+      "grad_norm": 0.123046875,
+      "learning_rate": 4.463554455267066e-06,
+      "loss": 1.624751329421997,
+      "step": 6854
+    },
+    {
+      "epoch": 0.9735352065176875,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 4.371466234367421e-06,
+      "loss": 1.686977744102478,
+      "step": 6856
+    },
+    {
+      "epoch": 0.9738192016187721,
+      "grad_norm": 0.11962890625,
+      "learning_rate": 4.280294443885973e-06,
+      "loss": 1.6380048990249634,
+      "step": 6858
+    },
+    {
+      "epoch": 0.9741031967198566,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 4.190040914970805e-06,
+      "loss": 1.6679741144180298,
+      "step": 6860
+    },
+    {
+      "epoch": 0.9743871918209411,
+      "grad_norm": 0.12109375,
+      "learning_rate": 4.100707460327108e-06,
+      "loss": 1.6518080234527588,
+      "step": 6862
+    },
+    {
+      "epoch": 0.9746711869220256,
+      "grad_norm": 0.123046875,
+      "learning_rate": 4.01229587418071e-06,
+      "loss": 1.6278109550476074,
+      "step": 6864
+    },
+    {
+      "epoch": 0.9749551820231102,
+      "grad_norm": 0.12109375,
+      "learning_rate": 3.924807932242142e-06,
+      "loss": 1.6852149963378906,
+      "step": 6866
+    },
+    {
+      "epoch": 0.9752391771241946,
+      "grad_norm": 0.1328125,
+      "learning_rate": 3.8382453916708805e-06,
+      "loss": 1.6273053884506226,
+      "step": 6868
+    },
+    {
+      "epoch": 0.9755231722252791,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 3.7526099910401348e-06,
+      "loss": 1.649459958076477,
+      "step": 6870
+    },
+    {
+      "epoch": 0.9758071673263636,
+      "grad_norm": 0.125,
+      "learning_rate": 3.6679034503018293e-06,
+      "loss": 1.6260278224945068,
+      "step": 6872
+    },
+    {
+      "epoch": 0.9760911624274481,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 3.5841274707521856e-06,
+      "loss": 1.7036383152008057,
+      "step": 6874
+    },
+    {
+      "epoch": 0.9763751575285327,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 3.501283734997418e-06,
+      "loss": 1.6645114421844482,
+      "step": 6876
+    },
+    {
+      "epoch": 0.9766591526296171,
+      "grad_norm": 0.1181640625,
+      "learning_rate": 3.4193739069200703e-06,
+      "loss": 1.6695365905761719,
+      "step": 6878
+    },
+    {
+      "epoch": 0.9769431477307017,
+      "grad_norm": 0.125,
+      "learning_rate": 3.338399631645489e-06,
+      "loss": 1.66145920753479,
+      "step": 6880
+    },
+    {
+      "epoch": 0.9772271428317861,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 3.2583625355088243e-06,
+      "loss": 1.6386982202529907,
+      "step": 6882
+    },
+    {
+      "epoch": 0.9775111379328707,
+      "grad_norm": 0.130859375,
+      "learning_rate": 3.1792642260224047e-06,
+      "loss": 1.6711777448654175,
+      "step": 6884
+    },
+    {
+      "epoch": 0.9777951330339552,
+      "grad_norm": 0.123046875,
+      "learning_rate": 3.101106291843381e-06,
+      "loss": 1.6668962240219116,
+      "step": 6886
+    },
+    {
+      "epoch": 0.9780791281350397,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 3.0238903027418553e-06,
+      "loss": 1.6597087383270264,
+      "step": 6888
+    },
+    {
+      "epoch": 0.9783631232361242,
+      "grad_norm": 0.12890625,
+      "learning_rate": 2.9476178095693472e-06,
+      "loss": 1.6730546951293945,
+      "step": 6890
+    },
+    {
+      "epoch": 0.9786471183372086,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 2.872290344227668e-06,
+      "loss": 1.6554884910583496,
+      "step": 6892
+    },
+    {
+      "epoch": 0.9789311134382932,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.7979094196381117e-06,
+      "loss": 1.6670081615447998,
+      "step": 6894
+    },
+    {
+      "epoch": 0.9792151085393777,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.724476529711084e-06,
+      "loss": 1.6630990505218506,
+      "step": 6896
+    },
+    {
+      "epoch": 0.9794991036404622,
+      "grad_norm": 0.125,
+      "learning_rate": 2.6519931493161176e-06,
+      "loss": 1.676001787185669,
+      "step": 6898
+    },
+    {
+      "epoch": 0.9797830987415467,
+      "grad_norm": 0.126953125,
+      "learning_rate": 2.580460734252238e-06,
+      "loss": 1.660456895828247,
+      "step": 6900
+    },
+    {
+      "epoch": 0.9800670938426312,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.509880721218705e-06,
+      "loss": 1.6949119567871094,
+      "step": 6902
+    },
+    {
+      "epoch": 0.9803510889437157,
+      "grad_norm": 0.12109375,
+      "learning_rate": 2.4402545277861788e-06,
+      "loss": 1.6519129276275635,
+      "step": 6904
+    },
+    {
+      "epoch": 0.9806350840448003,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 2.371583552368245e-06,
+      "loss": 1.6996619701385498,
+      "step": 6906
+    },
+    {
+      "epoch": 0.9809190791458847,
+      "grad_norm": 0.125,
+      "learning_rate": 2.3038691741933406e-06,
+      "loss": 1.6510899066925049,
+      "step": 6908
+    },
+    {
+      "epoch": 0.9812030742469693,
+      "grad_norm": 0.130859375,
+      "learning_rate": 2.2371127532770155e-06,
+      "loss": 1.666084885597229,
+      "step": 6910
+    },
+    {
+      "epoch": 0.9814870693480537,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 2.171315630394649e-06,
+      "loss": 1.6329777240753174,
+      "step": 6912
+    },
+    {
+      "epoch": 0.9817710644491382,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 2.1064791270545225e-06,
+      "loss": 1.6736146211624146,
+      "step": 6914
+    },
+    {
+      "epoch": 0.9820550595502228,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 2.0426045454712493e-06,
+      "loss": 1.6562269926071167,
+      "step": 6916
+    },
+    {
+      "epoch": 0.9823390546513072,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 1.9796931685396647e-06,
+      "loss": 1.6335300207138062,
+      "step": 6918
+    },
+    {
+      "epoch": 0.9826230497523918,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.917746259808989e-06,
+      "loss": 1.6502379179000854,
+      "step": 6920
+    },
+    {
+      "epoch": 0.9829070448534762,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 1.8567650634575384e-06,
+      "loss": 1.6750049591064453,
+      "step": 6922
+    },
+    {
+      "epoch": 0.9831910399545608,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 1.7967508042676662e-06,
+      "loss": 1.6218786239624023,
+      "step": 6924
+    },
+    {
+      "epoch": 0.9834750350556453,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 1.7377046876011993e-06,
+      "loss": 1.6702368259429932,
+      "step": 6926
+    },
+    {
+      "epoch": 0.9837590301567298,
+      "grad_norm": 0.123046875,
+      "learning_rate": 1.6796278993752024e-06,
+      "loss": 1.6584724187850952,
+      "step": 6928
+    },
+    {
+      "epoch": 0.9840430252578143,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 1.6225216060382075e-06,
+      "loss": 1.6555862426757812,
+      "step": 6930
+    },
+    {
+      "epoch": 0.9843270203588989,
+      "grad_norm": 0.12109375,
+      "learning_rate": 1.566386954546728e-06,
+      "loss": 1.6616029739379883,
+      "step": 6932
+    },
+    {
+      "epoch": 0.9846110154599833,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 1.5112250723422427e-06,
+      "loss": 1.6865534782409668,
+      "step": 6934
+    },
+    {
+      "epoch": 0.9848950105610679,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 1.4570370673285815e-06,
+      "loss": 1.656646728515625,
+      "step": 6936
+    },
+    {
+      "epoch": 0.9851790056621523,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 1.4038240278496374e-06,
+      "loss": 1.6809053421020508,
+      "step": 6938
+    },
+    {
+      "epoch": 0.9854630007632368,
+      "grad_norm": 0.11962890625,
+      "learning_rate": 1.351587022667522e-06,
+      "loss": 1.6200101375579834,
+      "step": 6940
+    },
+    {
+      "epoch": 0.9857469958643214,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.3003271009410844e-06,
+      "loss": 1.6474997997283936,
+      "step": 6942
+    },
+    {
+      "epoch": 0.9860309909654058,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 1.2500452922048766e-06,
+      "loss": 1.6671321392059326,
+      "step": 6944
+    },
+    {
+      "epoch": 0.9863149860664904,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.2007426063484261e-06,
+      "loss": 1.6690073013305664,
+      "step": 6946
+    },
+    {
+      "epoch": 0.9865989811675748,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 1.1524200335960134e-06,
+      "loss": 1.6505852937698364,
+      "step": 6948
+    },
+    {
+      "epoch": 0.9868829762686594,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 1.105078544486704e-06,
+      "loss": 1.6218998432159424,
+      "step": 6950
+    },
+    {
+      "epoch": 0.9871669713697439,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 1.058719089854937e-06,
+      "loss": 1.7010353803634644,
+      "step": 6952
+    },
+    {
+      "epoch": 0.9874509664708284,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 1.0133426008113678e-06,
+      "loss": 1.6759577989578247,
+      "step": 6954
+    },
+    {
+      "epoch": 0.9877349615719129,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 9.689499887242104e-07,
+      "loss": 1.6455817222595215,
+      "step": 6956
+    },
+    {
+      "epoch": 0.9880189566729974,
+      "grad_norm": 0.11865234375,
+      "learning_rate": 9.255421452008916e-07,
+      "loss": 1.6357256174087524,
+      "step": 6958
+    },
+    {
+      "epoch": 0.9883029517740819,
+      "grad_norm": 0.12109375,
+      "learning_rate": 8.831199420701763e-07,
+      "loss": 1.6579055786132812,
+      "step": 6960
+    },
+    {
+      "epoch": 0.9885869468751664,
+      "grad_norm": 0.130859375,
+      "learning_rate": 8.416842313646478e-07,
+      "loss": 1.6640031337738037,
+      "step": 6962
+    },
+    {
+      "epoch": 0.9888709419762509,
+      "grad_norm": 0.123046875,
+      "learning_rate": 8.012358453035939e-07,
+      "loss": 1.6428769826889038,
+      "step": 6964
+    },
+    {
+      "epoch": 0.9891549370773354,
+      "grad_norm": 0.1328125,
+      "learning_rate": 7.617755962762874e-07,
+      "loss": 1.6737720966339111,
+      "step": 6966
+    },
+    {
+      "epoch": 0.9894389321784199,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 7.233042768256593e-07,
+      "loss": 1.667173147201538,
+      "step": 6968
+    },
+    {
+      "epoch": 0.9897229272795044,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 6.858226596324235e-07,
+      "loss": 1.684898853302002,
+      "step": 6970
+    },
+    {
+      "epoch": 0.990006922380589,
+      "grad_norm": 0.126953125,
+      "learning_rate": 6.493314974995113e-07,
+      "loss": 1.6259828805923462,
+      "step": 6972
+    },
+    {
+      "epoch": 0.9902909174816734,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 6.138315233369663e-07,
+      "loss": 1.6429195404052734,
+      "step": 6974
+    },
+    {
+      "epoch": 0.990574912582758,
+      "grad_norm": 0.12060546875,
+      "learning_rate": 5.7932345014724e-07,
+      "loss": 1.6119306087493896,
+      "step": 6976
+    },
+    {
+      "epoch": 0.9908589076838424,
+      "grad_norm": 0.125,
+      "learning_rate": 5.458079710108533e-07,
+      "loss": 1.6442238092422485,
+      "step": 6978
+    },
+    {
+      "epoch": 0.991142902784927,
+      "grad_norm": 0.11572265625,
+      "learning_rate": 5.132857590724794e-07,
+      "loss": 1.6374033689498901,
+      "step": 6980
+    },
+    {
+      "epoch": 0.9914268978860115,
+      "grad_norm": 0.1240234375,
+      "learning_rate": 4.817574675274328e-07,
+      "loss": 1.6524770259857178,
+      "step": 6982
+    },
+    {
+      "epoch": 0.991710892987096,
+      "grad_norm": 0.123046875,
+      "learning_rate": 4.512237296085353e-07,
+      "loss": 1.6336588859558105,
+      "step": 6984
+    },
+    {
+      "epoch": 0.9919948880881805,
+      "grad_norm": 0.12109375,
+      "learning_rate": 4.2168515857341497e-07,
+      "loss": 1.6137917041778564,
+      "step": 6986
+    },
+    {
+      "epoch": 0.9922788831892649,
+      "grad_norm": 0.12158203125,
+      "learning_rate": 3.9314234769217137e-07,
+      "loss": 1.6510038375854492,
+      "step": 6988
+    },
+    {
+      "epoch": 0.9925628782903495,
+      "grad_norm": 0.130859375,
+      "learning_rate": 3.6559587023548e-07,
+      "loss": 1.6569230556488037,
+      "step": 6990
+    },
+    {
+      "epoch": 0.992846873391434,
+      "grad_norm": 0.12890625,
+      "learning_rate": 3.3904627946305667e-07,
+      "loss": 1.6170315742492676,
+      "step": 6992
+    },
+    {
+      "epoch": 0.9931308684925185,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 3.1349410861256645e-07,
+      "loss": 1.652906060218811,
+      "step": 6994
+    },
+    {
+      "epoch": 0.993414863593603,
+      "grad_norm": 0.12890625,
+      "learning_rate": 2.889398708888991e-07,
+      "loss": 1.646234393119812,
+      "step": 6996
+    },
+    {
+      "epoch": 0.9936988586946875,
+      "grad_norm": 0.1181640625,
+      "learning_rate": 2.6538405945385483e-07,
+      "loss": 1.6336615085601807,
+      "step": 6998
+    },
+    {
+      "epoch": 0.993982853795772,
+      "grad_norm": 0.125,
+      "learning_rate": 2.4282714741627464e-07,
+      "loss": 1.643019199371338,
+      "step": 7000
+    },
+    {
+      "epoch": 0.9942668488968566,
+      "grad_norm": 0.130859375,
+      "learning_rate": 2.2126958782249796e-07,
+      "loss": 1.679926872253418,
+      "step": 7002
+    },
+    {
+      "epoch": 0.994550843997941,
+      "grad_norm": 0.1259765625,
+      "learning_rate": 2.007118136472752e-07,
+      "loss": 1.6487417221069336,
+      "step": 7004
+    },
+    {
+      "epoch": 0.9948348390990256,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 1.8115423778509167e-07,
+      "loss": 1.6569416522979736,
+      "step": 7006
+    },
+    {
+      "epoch": 0.99511883420011,
+      "grad_norm": 0.125,
+      "learning_rate": 1.6259725304184626e-07,
+      "loss": 1.680594563484192,
+      "step": 7008
+    },
+    {
+      "epoch": 0.9954028293011945,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 1.4504123212698006e-07,
+      "loss": 1.6156156063079834,
+      "step": 7010
+    },
+    {
+      "epoch": 0.9956868244022791,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 1.2848652764598789e-07,
+      "loss": 1.6274839639663696,
+      "step": 7012
+    },
+    {
+      "epoch": 0.9959708195033635,
+      "grad_norm": 0.12451171875,
+      "learning_rate": 1.1293347209332395e-07,
+      "loss": 1.6665003299713135,
+      "step": 7014
+    },
+    {
+      "epoch": 0.9962548146044481,
+      "grad_norm": 0.130859375,
+      "learning_rate": 9.838237784575155e-08,
+      "loss": 1.6789369583129883,
+      "step": 7016
+    },
+    {
+      "epoch": 0.9965388097055325,
+      "grad_norm": 0.1220703125,
+      "learning_rate": 8.48335371560316e-08,
+      "loss": 1.6686030626296997,
+      "step": 7018
+    },
+    {
+      "epoch": 0.9968228048066171,
+      "grad_norm": 0.1201171875,
+      "learning_rate": 7.228722214708272e-08,
+      "loss": 1.6648080348968506,
+      "step": 7020
+    },
+    {
+      "epoch": 0.9971067999077016,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 6.074368480650794e-08,
+      "loss": 1.6471000909805298,
+      "step": 7022
+    },
+    {
+      "epoch": 0.9973907950087861,
+      "grad_norm": 0.126953125,
+      "learning_rate": 5.0203156981526446e-08,
+      "loss": 1.6676064729690552,
+      "step": 7024
+    },
+    {
+      "epoch": 0.9976747901098706,
+      "grad_norm": 0.126953125,
+      "learning_rate": 4.066585037432735e-08,
+      "loss": 1.6278235912322998,
+      "step": 7026
+    },
+    {
+      "epoch": 0.9979587852109552,
+      "grad_norm": 0.125,
+      "learning_rate": 3.213195653780643e-08,
+      "loss": 1.662877082824707,
+      "step": 7028
+    },
+    {
+      "epoch": 0.9982427803120396,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 2.4601646871724726e-08,
+      "loss": 1.6766605377197266,
+      "step": 7030
+    },
+    {
+      "epoch": 0.9985267754131242,
+      "grad_norm": 0.12109375,
+      "learning_rate": 1.8075072619272436e-08,
+      "loss": 1.6600494384765625,
+      "step": 7032
+    },
+    {
+      "epoch": 0.9988107705142086,
+      "grad_norm": 0.1279296875,
+      "learning_rate": 1.2552364864021337e-08,
+      "loss": 1.6512842178344727,
+      "step": 7034
+    },
+    {
+      "epoch": 0.9990947656152931,
+      "grad_norm": 0.12353515625,
+      "learning_rate": 8.033634527288004e-09,
+      "loss": 1.6618376970291138,
+      "step": 7036
+    },
+    {
+      "epoch": 0.9993787607163777,
+      "grad_norm": 0.12255859375,
+      "learning_rate": 4.518972365930019e-09,
+      "loss": 1.6308016777038574,
+      "step": 7038
+    },
+    {
+      "epoch": 0.9996627558174621,
+      "grad_norm": 0.1298828125,
+      "learning_rate": 2.0084489704974563e-09,
+      "loss": 1.6697489023208618,
+      "step": 7040
+    },
+    {
+      "epoch": 0.9999467509185467,
+      "grad_norm": 0.126953125,
+      "learning_rate": 5.021147638173407e-10,
+      "loss": 1.6276886463165283,
+      "step": 7042
+    },
+    {
+      "epoch": 1.0,
+      "eval_loss": 1.6529316902160645,
+      "eval_runtime": 267.921,
+      "eval_samples_per_second": 54.901,
+      "eval_steps_per_second": 6.864,
+      "step": 7043
     }
   ],
   "logging_steps": 2,
         "should_evaluate": false,
         "should_log": false,
         "should_save": true,
+        "should_training_stop": true
       },
       "attributes": {}
     }
   },
+  "total_flos": 5.227389362626363e+18,
   "train_batch_size": 4,
   "trial_name": null,
   "trial_params": null