Instructions to use Ba2han/test_CPTlsft2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Ba2han/test_CPTlsft2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Ba2han/test_CPTlsft2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Ba2han/test_CPTlsft2")
model = AutoModelForCausalLM.from_pretrained("Ba2han/test_CPTlsft2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Ba2han/test_CPTlsft2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Ba2han/test_CPTlsft2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/test_CPTlsft2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Ba2han/test_CPTlsft2

SGLang

How to use Ba2han/test_CPTlsft2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Ba2han/test_CPTlsft2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/test_CPTlsft2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Ba2han/test_CPTlsft2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Ba2han/test_CPTlsft2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Unsloth Studio

How to use Ba2han/test_CPTlsft2 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/test_CPTlsft2 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for Ba2han/test_CPTlsft2 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for Ba2han/test_CPTlsft2 to start chatting

Load model with FastModel

pip install unsloth
from unsloth import FastModel
model, tokenizer = FastModel.from_pretrained(
    model_name="Ba2han/test_CPTlsft2",
    max_seq_length=2048,
)

Docker Model Runner
How to use Ba2han/test_CPTlsft2 with Docker Model Runner:
```
docker model run hf.co/Ba2han/test_CPTlsft2
```

Ba2han commited on 11 days ago

Commit

24c20f0

verified ·

1 Parent(s): 5f749f1

Training in progress, step 478, checkpoint

Browse files

Files changed (5) hide show

last-checkpoint/model.safetensors +1 -1
last-checkpoint/optimizer.pt +2 -2
last-checkpoint/scheduler.pt +1 -1
last-checkpoint/trainer_state.json +1033 -613
last-checkpoint/training_args.bin +1 -1

last-checkpoint/model.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1cf89c40bf12b0ae068ccd3c77499f256c0bb8be5d17010124d7185885f58ff8
 size 2934397032

 version https://git-lfs.github.com/spec/v1
+oid sha256:78ab92f17e79148c28205a6292a4e9129615c540263dfd5010c1796329172af1
 size 2934397032

last-checkpoint/optimizer.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:68f585ebc96f25b1202640a2aa0898d84adc82670b03813bab892643b156c624
-size 1474576440

 version https://git-lfs.github.com/spec/v1
+oid sha256:d408a4cd55a990fe7e36dba0c2e404245487f51ba882e497f61db3c3ee0cf0d8
+size 1474334008

last-checkpoint/scheduler.pt CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:104c3fbf3c321994c619d8c4771921dc4f8440dbfb44f110518116de11f7cc63
 size 9785

 version https://git-lfs.github.com/spec/v1
+oid sha256:9e847d2ffd52f94ae0de60045530898f95b82ce7734cc0dbaf4457fb5810ec2a
 size 9785

last-checkpoint/trainer_state.json CHANGED Viewed

@@ -2,1272 +2,1692 @@
   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
-  "epoch": 0.20028941541572956,
   "eval_steps": 500,
-  "global_step": 359,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
-      "epoch": 0.001115818470282616,
-      "grad_norm": 0.35119014978408813,
-      "learning_rate": 9.259259259259259e-06,
-      "loss": 2.7863893508911133,
       "step": 2
     },
     {
-      "epoch": 0.002231636940565232,
-      "grad_norm": 0.3843597173690796,
-      "learning_rate": 2.7777777777777776e-05,
-      "loss": 2.8073158264160156,
       "step": 4
     },
     {
-      "epoch": 0.0033474554108478475,
-      "grad_norm": 0.3032338321208954,
-      "learning_rate": 4.6296296296296294e-05,
-      "loss": 2.897202968597412,
       "step": 6
     },
     {
-      "epoch": 0.004463273881130464,
-      "grad_norm": 0.40724217891693115,
-      "learning_rate": 6.481481481481482e-05,
-      "loss": 2.869870662689209,
       "step": 8
     },
     {
-      "epoch": 0.005579092351413079,
-      "grad_norm": 0.34800443053245544,
-      "learning_rate": 8.333333333333333e-05,
-      "loss": 2.7431254386901855,
       "step": 10
     },
     {
-      "epoch": 0.006694910821695695,
-      "grad_norm": 0.2605589032173157,
-      "learning_rate": 0.00010185185185185185,
-      "loss": 2.7429285049438477,
       "step": 12
     },
     {
-      "epoch": 0.007810729291978311,
-      "grad_norm": 0.4218830466270447,
-      "learning_rate": 0.00012037037037037036,
-      "loss": 2.8209400177001953,
       "step": 14
     },
     {
-      "epoch": 0.008926547762260928,
-      "grad_norm": 0.32858774065971375,
-      "learning_rate": 0.0001388888888888889,
-      "loss": 2.8327112197875977,
       "step": 16
     },
     {
-      "epoch": 0.010042366232543544,
-      "grad_norm": 0.3018340766429901,
-      "learning_rate": 0.0001574074074074074,
-      "loss": 2.8165016174316406,
       "step": 18
     },
     {
-      "epoch": 0.011158184702826158,
-      "grad_norm": 0.3222309350967407,
-      "learning_rate": 0.00017592592592592595,
-      "loss": 2.6792030334472656,
       "step": 20
     },
     {
-      "epoch": 0.012274003173108774,
-      "grad_norm": 0.2766020596027374,
-      "learning_rate": 0.00019444444444444446,
-      "loss": 2.7933080196380615,
       "step": 22
     },
     {
-      "epoch": 0.01338982164339139,
-      "grad_norm": 0.32303234934806824,
-      "learning_rate": 0.00021296296296296298,
-      "loss": 2.845729351043701,
       "step": 24
     },
     {
-      "epoch": 0.014505640113674006,
-      "grad_norm": 0.4417007267475128,
-      "learning_rate": 0.0002314814814814815,
-      "loss": 2.765629529953003,
       "step": 26
     },
     {
-      "epoch": 0.015621458583956622,
-      "grad_norm": 0.21694634854793549,
-      "learning_rate": 0.00025,
-      "loss": 2.68699312210083,
       "step": 28
     },
     {
-      "epoch": 0.01673727705423924,
-      "grad_norm": 0.30644547939300537,
-      "learning_rate": 0.0002685185185185186,
-      "loss": 2.8008785247802734,
       "step": 30
     },
     {
-      "epoch": 0.017853095524521856,
-      "grad_norm": 0.24403627216815948,
-      "learning_rate": 0.00028703703703703703,
-      "loss": 2.8291001319885254,
       "step": 32
     },
     {
-      "epoch": 0.018968913994804472,
-      "grad_norm": 0.27576687932014465,
-      "learning_rate": 0.0003055555555555556,
-      "loss": 2.941161632537842,
       "step": 34
     },
     {
-      "epoch": 0.020084732465087088,
-      "grad_norm": 0.2644183933734894,
-      "learning_rate": 0.00032407407407407406,
-      "loss": 3.044625759124756,
       "step": 36
     },
     {
-      "epoch": 0.0212005509353697,
-      "grad_norm": 0.2418198585510254,
-      "learning_rate": 0.00034259259259259263,
-      "loss": 2.6974642276763916,
       "step": 38
     },
     {
-      "epoch": 0.022316369405652316,
-      "grad_norm": 0.24539807438850403,
-      "learning_rate": 0.0003611111111111111,
-      "loss": 2.889753818511963,
       "step": 40
     },
     {
-      "epoch": 0.023432187875934932,
-      "grad_norm": 0.3314460217952728,
-      "learning_rate": 0.00037962962962962966,
-      "loss": 2.9927704334259033,
       "step": 42
     },
     {
-      "epoch": 0.02454800634621755,
-      "grad_norm": 0.2458345592021942,
-      "learning_rate": 0.0003981481481481481,
-      "loss": 2.73818302154541,
       "step": 44
     },
     {
-      "epoch": 0.025663824816500164,
-      "grad_norm": 0.24675895273685455,
-      "learning_rate": 0.0004166666666666667,
-      "loss": 2.9188570976257324,
       "step": 46
     },
     {
-      "epoch": 0.02677964328678278,
-      "grad_norm": 0.28446805477142334,
-      "learning_rate": 0.0004351851851851852,
-      "loss": 2.619231700897217,
       "step": 48
     },
     {
-      "epoch": 0.027895461757065396,
-      "grad_norm": 0.242530956864357,
-      "learning_rate": 0.0004537037037037037,
-      "loss": 2.8235979080200195,
       "step": 50
     },
     {
-      "epoch": 0.029011280227348012,
-      "grad_norm": 0.24973484873771667,
-      "learning_rate": 0.00047222222222222224,
-      "loss": 2.7562460899353027,
       "step": 52
     },
     {
-      "epoch": 0.03012709869763063,
-      "grad_norm": 0.20091792941093445,
-      "learning_rate": 0.0004907407407407408,
-      "loss": 2.693488597869873,
       "step": 54
     },
     {
-      "epoch": 0.031242917167913244,
-      "grad_norm": 0.16492688655853271,
-      "learning_rate": 0.0005092592592592593,
-      "loss": 2.697176933288574,
       "step": 56
     },
     {
-      "epoch": 0.032358735638195864,
-      "grad_norm": 0.2837952971458435,
-      "learning_rate": 0.0005277777777777778,
-      "loss": 2.742260694503784,
       "step": 58
     },
     {
-      "epoch": 0.03347455410847848,
-      "grad_norm": 0.17709504067897797,
-      "learning_rate": 0.0005462962962962962,
-      "loss": 2.77925181388855,
       "step": 60
     },
     {
-      "epoch": 0.034590372578761096,
-      "grad_norm": 0.21625208854675293,
-      "learning_rate": 0.0005648148148148148,
-      "loss": 2.5686936378479004,
       "step": 62
     },
     {
-      "epoch": 0.03570619104904371,
-      "grad_norm": 0.21876399219036102,
-      "learning_rate": 0.0005833333333333334,
-      "loss": 2.559140682220459,
       "step": 64
     },
     {
-      "epoch": 0.03682200951932633,
-      "grad_norm": 0.2158040851354599,
-      "learning_rate": 0.0006018518518518519,
-      "loss": 2.856241464614868,
       "step": 66
     },
     {
-      "epoch": 0.037937827989608944,
-      "grad_norm": 0.2351156324148178,
-      "learning_rate": 0.0006203703703703704,
-      "loss": 2.8853726387023926,
       "step": 68
     },
     {
-      "epoch": 0.03905364645989156,
-      "grad_norm": 0.21792782843112946,
-      "learning_rate": 0.0006388888888888888,
-      "loss": 2.8599424362182617,
       "step": 70
     },
     {
-      "epoch": 0.040169464930174176,
-      "grad_norm": 0.22209179401397705,
-      "learning_rate": 0.0006574074074074074,
-      "loss": 2.8283114433288574,
       "step": 72
     },
     {
-      "epoch": 0.041285283400456785,
-      "grad_norm": 0.19672609865665436,
-      "learning_rate": 0.000675925925925926,
-      "loss": 2.766714572906494,
       "step": 74
     },
     {
-      "epoch": 0.0424011018707394,
-      "grad_norm": 0.1954922378063202,
-      "learning_rate": 0.0006944444444444445,
-      "loss": 2.6967718601226807,
       "step": 76
     },
     {
-      "epoch": 0.04351692034102202,
-      "grad_norm": 0.23223549127578735,
-      "learning_rate": 0.0007129629629629629,
-      "loss": 2.763343334197998,
       "step": 78
     },
     {
-      "epoch": 0.04463273881130463,
-      "grad_norm": 0.16624204814434052,
-      "learning_rate": 0.0007314814814814815,
-      "loss": 2.6691741943359375,
       "step": 80
     },
     {
-      "epoch": 0.04574855728158725,
-      "grad_norm": 0.16215577721595764,
-      "learning_rate": 0.00075,
-      "loss": 2.7990283966064453,
       "step": 82
     },
     {
-      "epoch": 0.046864375751869865,
-      "grad_norm": 0.20904570817947388,
-      "learning_rate": 0.0007685185185185185,
-      "loss": 2.940199375152588,
       "step": 84
     },
     {
-      "epoch": 0.04798019422215248,
-      "grad_norm": 0.20124569535255432,
-      "learning_rate": 0.0007870370370370372,
-      "loss": 2.810028314590454,
       "step": 86
     },
     {
-      "epoch": 0.0490960126924351,
-      "grad_norm": 0.2001955211162567,
-      "learning_rate": 0.0008055555555555556,
-      "loss": 2.6326546669006348,
       "step": 88
     },
     {
-      "epoch": 0.05021183116271771,
-      "grad_norm": 0.17109206318855286,
-      "learning_rate": 0.0008240740740740741,
-      "loss": 2.6173388957977295,
       "step": 90
     },
     {
-      "epoch": 0.05132764963300033,
-      "grad_norm": 0.18255700170993805,
-      "learning_rate": 0.0008425925925925926,
-      "loss": 2.7689356803894043,
       "step": 92
     },
     {
-      "epoch": 0.052443468103282945,
-      "grad_norm": 0.22842863202095032,
-      "learning_rate": 0.0008611111111111112,
-      "loss": 2.831810474395752,
       "step": 94
     },
     {
-      "epoch": 0.05355928657356556,
-      "grad_norm": 0.17924705147743225,
-      "learning_rate": 0.0008796296296296296,
-      "loss": 2.563502550125122,
       "step": 96
     },
     {
-      "epoch": 0.05467510504384818,
-      "grad_norm": 0.1710575968027115,
-      "learning_rate": 0.0008981481481481481,
-      "loss": 2.5608701705932617,
       "step": 98
     },
     {
-      "epoch": 0.05579092351413079,
-      "grad_norm": 0.1925666779279709,
-      "learning_rate": 0.0009166666666666666,
-      "loss": 2.7008936405181885,
       "step": 100
     },
     {
-      "epoch": 0.05690674198441341,
-      "grad_norm": 0.1998261958360672,
-      "learning_rate": 0.0009351851851851853,
-      "loss": 2.6525344848632812,
       "step": 102
     },
     {
-      "epoch": 0.058022560454696025,
-      "grad_norm": 0.21880614757537842,
-      "learning_rate": 0.0009537037037037038,
-      "loss": 2.644655227661133,
       "step": 104
     },
     {
-      "epoch": 0.05913837892497864,
-      "grad_norm": 0.23569966852664948,
-      "learning_rate": 0.0009722222222222222,
-      "loss": 2.778639078140259,
       "step": 106
     },
     {
-      "epoch": 0.06025419739526126,
-      "grad_norm": 0.20484603941440582,
-      "learning_rate": 0.0009907407407407408,
-      "loss": 2.8949196338653564,
       "step": 108
     },
     {
-      "epoch": 0.06137001586554387,
-      "grad_norm": 0.1707783192396164,
-      "learning_rate": 0.001,
-      "loss": 2.5926766395568848,
       "step": 110
     },
     {
-      "epoch": 0.06248583433582649,
-      "grad_norm": 0.251994788646698,
-      "learning_rate": 0.001,
-      "loss": 2.7643930912017822,
       "step": 112
     },
     {
-      "epoch": 0.0636016528061091,
-      "grad_norm": 0.20418298244476318,
-      "learning_rate": 0.001,
-      "loss": 2.7868242263793945,
       "step": 114
     },
     {
-      "epoch": 0.06471747127639173,
-      "grad_norm": 0.21040396392345428,
-      "learning_rate": 0.001,
-      "loss": 2.6057658195495605,
       "step": 116
     },
     {
-      "epoch": 0.06583328974667434,
-      "grad_norm": 0.17298687994480133,
-      "learning_rate": 0.001,
-      "loss": 2.655816078186035,
       "step": 118
     },
     {
-      "epoch": 0.06694910821695696,
-      "grad_norm": 0.27325186133384705,
-      "learning_rate": 0.001,
-      "loss": 2.6814215183258057,
       "step": 120
     },
     {
-      "epoch": 0.06806492668723957,
-      "grad_norm": 0.22967654466629028,
-      "learning_rate": 0.001,
-      "loss": 2.605759859085083,
       "step": 122
     },
     {
-      "epoch": 0.06918074515752219,
-      "grad_norm": 0.2560531198978424,
-      "learning_rate": 0.001,
-      "loss": 2.6205925941467285,
       "step": 124
     },
     {
-      "epoch": 0.0702965636278048,
-      "grad_norm": 0.24636557698249817,
-      "learning_rate": 0.001,
-      "loss": 2.731029987335205,
       "step": 126
     },
     {
-      "epoch": 0.07141238209808742,
-      "grad_norm": 0.1971369832754135,
-      "learning_rate": 0.001,
-      "loss": 2.670501708984375,
       "step": 128
     },
     {
-      "epoch": 0.07252820056837003,
-      "grad_norm": 0.1941949874162674,
-      "learning_rate": 0.001,
-      "loss": 2.6893820762634277,
       "step": 130
     },
     {
-      "epoch": 0.07364401903865266,
-      "grad_norm": 0.2338646650314331,
-      "learning_rate": 0.001,
-      "loss": 2.587491512298584,
       "step": 132
     },
     {
-      "epoch": 0.07475983750893526,
-      "grad_norm": 0.15633897483348846,
-      "learning_rate": 0.001,
-      "loss": 2.4439656734466553,
       "step": 134
     },
     {
-      "epoch": 0.07587565597921789,
-      "grad_norm": 0.1933259218931198,
-      "learning_rate": 0.001,
-      "loss": 2.6602632999420166,
       "step": 136
     },
     {
-      "epoch": 0.0769914744495005,
-      "grad_norm": 0.21623004972934723,
-      "learning_rate": 0.001,
-      "loss": 2.6813745498657227,
       "step": 138
     },
     {
-      "epoch": 0.07810729291978312,
-      "grad_norm": 0.14753678441047668,
-      "learning_rate": 0.001,
-      "loss": 2.504720687866211,
       "step": 140
     },
     {
-      "epoch": 0.07922311139006573,
-      "grad_norm": 0.1963011920452118,
-      "learning_rate": 0.001,
-      "loss": 2.622713327407837,
       "step": 142
     },
     {
-      "epoch": 0.08033892986034835,
-      "grad_norm": 0.22170260548591614,
       "learning_rate": 0.001,
-      "loss": 2.7199618816375732,
       "step": 144
     },
     {
-      "epoch": 0.08145474833063096,
-      "grad_norm": 0.2357204407453537,
       "learning_rate": 0.001,
-      "loss": 2.7126598358154297,
       "step": 146
     },
     {
-      "epoch": 0.08257056680091357,
-      "grad_norm": 0.2318502962589264,
       "learning_rate": 0.001,
-      "loss": 2.5498247146606445,
       "step": 148
     },
     {
-      "epoch": 0.08368638527119619,
-      "grad_norm": 0.17542071640491486,
       "learning_rate": 0.001,
-      "loss": 2.5129213333129883,
       "step": 150
     },
     {
-      "epoch": 0.0848022037414788,
-      "grad_norm": 0.20685723423957825,
       "learning_rate": 0.001,
-      "loss": 2.541411876678467,
       "step": 152
     },
     {
-      "epoch": 0.08591802221176142,
-      "grad_norm": 0.16904857754707336,
       "learning_rate": 0.001,
-      "loss": 2.577758312225342,
       "step": 154
     },
     {
-      "epoch": 0.08703384068204403,
-      "grad_norm": 0.16783322393894196,
       "learning_rate": 0.001,
-      "loss": 2.5067102909088135,
       "step": 156
     },
     {
-      "epoch": 0.08814965915232666,
-      "grad_norm": 0.19620636105537415,
       "learning_rate": 0.001,
-      "loss": 2.558516502380371,
       "step": 158
     },
     {
-      "epoch": 0.08926547762260927,
-      "grad_norm": 0.2538161873817444,
       "learning_rate": 0.001,
-      "loss": 2.7212512493133545,
       "step": 160
     },
     {
-      "epoch": 0.09038129609289189,
-      "grad_norm": 0.15611612796783447,
       "learning_rate": 0.001,
-      "loss": 2.458073616027832,
       "step": 162
     },
     {
-      "epoch": 0.0914971145631745,
-      "grad_norm": 0.2112778127193451,
       "learning_rate": 0.001,
-      "loss": 2.7351760864257812,
       "step": 164
     },
     {
-      "epoch": 0.09261293303345712,
-      "grad_norm": 0.21397119760513306,
       "learning_rate": 0.001,
-      "loss": 2.77217435836792,
       "step": 166
     },
     {
-      "epoch": 0.09372875150373973,
-      "grad_norm": 0.17090432345867157,
       "learning_rate": 0.001,
-      "loss": 2.5256545543670654,
       "step": 168
     },
     {
-      "epoch": 0.09484456997402235,
-      "grad_norm": 0.2283623218536377,
       "learning_rate": 0.001,
-      "loss": 2.570855140686035,
       "step": 170
     },
     {
-      "epoch": 0.09596038844430496,
-      "grad_norm": 0.26056423783302307,
       "learning_rate": 0.001,
-      "loss": 2.528101921081543,
       "step": 172
     },
     {
-      "epoch": 0.09707620691458758,
-      "grad_norm": 0.19249887764453888,
       "learning_rate": 0.001,
-      "loss": 2.5205156803131104,
       "step": 174
     },
     {
-      "epoch": 0.0981920253848702,
-      "grad_norm": 0.1602402627468109,
       "learning_rate": 0.001,
-      "loss": 2.5078482627868652,
       "step": 176
     },
     {
-      "epoch": 0.09930784385515282,
-      "grad_norm": 0.16434869170188904,
       "learning_rate": 0.001,
-      "loss": 2.352069854736328,
       "step": 178
     },
     {
-      "epoch": 0.10042366232543543,
-      "grad_norm": 0.2256188690662384,
       "learning_rate": 0.001,
-      "loss": 2.661741256713867,
       "step": 180
     },
     {
-      "epoch": 0.10153948079571805,
-      "grad_norm": 0.19142936170101166,
       "learning_rate": 0.001,
-      "loss": 2.5546834468841553,
       "step": 182
     },
     {
-      "epoch": 0.10265529926600066,
-      "grad_norm": 0.14948126673698425,
       "learning_rate": 0.001,
-      "loss": 2.5786380767822266,
       "step": 184
     },
     {
-      "epoch": 0.10377111773628328,
-      "grad_norm": 0.1690729260444641,
       "learning_rate": 0.001,
-      "loss": 2.531883478164673,
       "step": 186
     },
     {
-      "epoch": 0.10488693620656589,
-      "grad_norm": 0.18182902038097382,
       "learning_rate": 0.001,
-      "loss": 2.6663694381713867,
       "step": 188
     },
     {
-      "epoch": 0.10600275467684851,
-      "grad_norm": 0.17782893776893616,
       "learning_rate": 0.001,
-      "loss": 2.4051547050476074,
       "step": 190
     },
     {
-      "epoch": 0.10711857314713112,
-      "grad_norm": 0.1466338336467743,
       "learning_rate": 0.001,
-      "loss": 2.478933811187744,
       "step": 192
     },
     {
-      "epoch": 0.10823439161741374,
-      "grad_norm": 0.17003557085990906,
       "learning_rate": 0.001,
-      "loss": 2.451418876647949,
       "step": 194
     },
     {
-      "epoch": 0.10935021008769635,
-      "grad_norm": 0.20216721296310425,
       "learning_rate": 0.001,
-      "loss": 2.6341495513916016,
       "step": 196
     },
     {
-      "epoch": 0.11046602855797898,
-      "grad_norm": 0.16193339228630066,
       "learning_rate": 0.001,
-      "loss": 2.6115782260894775,
       "step": 198
     },
     {
-      "epoch": 0.11158184702826159,
-      "grad_norm": 0.15258647501468658,
       "learning_rate": 0.001,
-      "loss": 2.4345383644104004,
       "step": 200
     },
     {
-      "epoch": 0.11269766549854421,
-      "grad_norm": 0.17912527918815613,
       "learning_rate": 0.001,
-      "loss": 2.466099262237549,
       "step": 202
     },
     {
-      "epoch": 0.11381348396882682,
-      "grad_norm": 0.1584368199110031,
       "learning_rate": 0.001,
-      "loss": 2.3626391887664795,
       "step": 204
     },
     {
-      "epoch": 0.11492930243910944,
-      "grad_norm": 0.1478145271539688,
       "learning_rate": 0.001,
-      "loss": 2.503824234008789,
       "step": 206
     },
     {
-      "epoch": 0.11604512090939205,
-      "grad_norm": 0.27247354388237,
       "learning_rate": 0.001,
-      "loss": 2.5153417587280273,
       "step": 208
     },
     {
-      "epoch": 0.11716093937967467,
-      "grad_norm": 0.14682932198047638,
       "learning_rate": 0.001,
-      "loss": 2.35432767868042,
       "step": 210
     },
     {
-      "epoch": 0.11827675784995728,
-      "grad_norm": 0.1576104313135147,
       "learning_rate": 0.001,
-      "loss": 2.3110432624816895,
       "step": 212
     },
     {
-      "epoch": 0.1193925763202399,
-      "grad_norm": 0.3291161060333252,
       "learning_rate": 0.001,
-      "loss": 2.55582332611084,
       "step": 214
     },
     {
-      "epoch": 0.12050839479052251,
-      "grad_norm": 0.19020876288414001,
       "learning_rate": 0.001,
-      "loss": 2.5640926361083984,
       "step": 216
     },
     {
-      "epoch": 0.12162421326080514,
-      "grad_norm": 0.19632890820503235,
       "learning_rate": 0.001,
-      "loss": 2.3768601417541504,
       "step": 218
     },
     {
-      "epoch": 0.12274003173108775,
-      "grad_norm": 0.17404712736606598,
       "learning_rate": 0.001,
-      "loss": 2.5435166358947754,
       "step": 220
     },
     {
-      "epoch": 0.12385585020137037,
-      "grad_norm": 0.1816665232181549,
       "learning_rate": 0.001,
-      "loss": 2.5662050247192383,
       "step": 222
     },
     {
-      "epoch": 0.12497166867165298,
-      "grad_norm": 0.23474931716918945,
       "learning_rate": 0.001,
-      "loss": 2.4437813758850098,
       "step": 224
     },
     {
-      "epoch": 0.1260874871419356,
-      "grad_norm": 0.1601305454969406,
       "learning_rate": 0.001,
-      "loss": 2.5338168144226074,
       "step": 226
     },
     {
-      "epoch": 0.1272033056122182,
-      "grad_norm": 0.157420814037323,
       "learning_rate": 0.001,
-      "loss": 2.325195550918579,
       "step": 228
     },
     {
-      "epoch": 0.12831912408250082,
-      "grad_norm": 0.18968823552131653,
       "learning_rate": 0.001,
-      "loss": 2.466125726699829,
       "step": 230
     },
     {
-      "epoch": 0.12943494255278346,
-      "grad_norm": 0.16184599697589874,
       "learning_rate": 0.001,
-      "loss": 2.368532180786133,
       "step": 232
     },
     {
-      "epoch": 0.13055076102306606,
-      "grad_norm": 0.1610761433839798,
       "learning_rate": 0.001,
-      "loss": 2.308600425720215,
       "step": 234
     },
     {
-      "epoch": 0.13166657949334867,
-      "grad_norm": 0.16984932124614716,
       "learning_rate": 0.001,
-      "loss": 2.301692247390747,
       "step": 236
     },
     {
-      "epoch": 0.13278239796363128,
-      "grad_norm": 0.1499878168106079,
       "learning_rate": 0.001,
-      "loss": 2.3857920169830322,
       "step": 238
     },
     {
-      "epoch": 0.13389821643391392,
-      "grad_norm": 0.15753424167633057,
       "learning_rate": 0.001,
-      "loss": 2.3552494049072266,
       "step": 240
     },
     {
-      "epoch": 0.13501403490419653,
-      "grad_norm": 0.17578540742397308,
       "learning_rate": 0.001,
-      "loss": 2.3947925567626953,
       "step": 242
     },
     {
-      "epoch": 0.13612985337447914,
-      "grad_norm": 0.16845861077308655,
       "learning_rate": 0.001,
-      "loss": 2.4716451168060303,
       "step": 244
     },
     {
-      "epoch": 0.13724567184476175,
-      "grad_norm": 0.16819730401039124,
       "learning_rate": 0.001,
-      "loss": 2.428820848464966,
       "step": 246
     },
     {
-      "epoch": 0.13836149031504438,
-      "grad_norm": 0.16237416863441467,
       "learning_rate": 0.001,
-      "loss": 2.381443500518799,
       "step": 248
     },
     {
-      "epoch": 0.139477308785327,
-      "grad_norm": 0.14720472693443298,
       "learning_rate": 0.001,
-      "loss": 2.3820714950561523,
       "step": 250
     },
     {
-      "epoch": 0.1405931272556096,
-      "grad_norm": 0.2367306798696518,
       "learning_rate": 0.001,
-      "loss": 2.49891996383667,
       "step": 252
     },
     {
-      "epoch": 0.1417089457258922,
-      "grad_norm": 0.17773057520389557,
       "learning_rate": 0.001,
-      "loss": 2.278524398803711,
       "step": 254
     },
     {
-      "epoch": 0.14282476419617485,
-      "grad_norm": 0.1689394861459732,
       "learning_rate": 0.001,
-      "loss": 2.388369560241699,
       "step": 256
     },
     {
-      "epoch": 0.14394058266645746,
-      "grad_norm": 0.1583975851535797,
       "learning_rate": 0.001,
-      "loss": 2.527921438217163,
       "step": 258
     },
     {
-      "epoch": 0.14505640113674007,
-      "grad_norm": 0.14717847108840942,
       "learning_rate": 0.001,
-      "loss": 2.3611836433410645,
       "step": 260
     },
     {
-      "epoch": 0.14617221960702267,
-      "grad_norm": 0.1563814878463745,
       "learning_rate": 0.001,
-      "loss": 2.3533551692962646,
       "step": 262
     },
     {
-      "epoch": 0.1472880380773053,
-      "grad_norm": 0.23029778897762299,
       "learning_rate": 0.001,
-      "loss": 2.24751615524292,
       "step": 264
     },
     {
-      "epoch": 0.14840385654758792,
-      "grad_norm": 0.19167381525039673,
       "learning_rate": 0.001,
-      "loss": 2.3520631790161133,
       "step": 266
     },
     {
-      "epoch": 0.14951967501787053,
-      "grad_norm": 0.18839257955551147,
       "learning_rate": 0.001,
-      "loss": 2.5304384231567383,
       "step": 268
     },
     {
-      "epoch": 0.15063549348815314,
-      "grad_norm": 0.17384597659111023,
       "learning_rate": 0.001,
-      "loss": 2.3341143131256104,
       "step": 270
     },
     {
-      "epoch": 0.15175131195843578,
-      "grad_norm": 0.18357686698436737,
       "learning_rate": 0.001,
-      "loss": 2.308727979660034,
       "step": 272
     },
     {
-      "epoch": 0.15286713042871838,
-      "grad_norm": 0.19404777884483337,
       "learning_rate": 0.001,
-      "loss": 2.3632216453552246,
       "step": 274
     },
     {
-      "epoch": 0.153982948899001,
-      "grad_norm": 0.16565217077732086,
       "learning_rate": 0.001,
-      "loss": 2.347688674926758,
       "step": 276
     },
     {
-      "epoch": 0.1550987673692836,
-      "grad_norm": 0.1784813106060028,
       "learning_rate": 0.001,
-      "loss": 2.4546000957489014,
       "step": 278
     },
     {
-      "epoch": 0.15621458583956624,
-      "grad_norm": 0.1565283238887787,
       "learning_rate": 0.001,
-      "loss": 2.176791191101074,
       "step": 280
     },
     {
-      "epoch": 0.15733040430984885,
-      "grad_norm": 0.14966997504234314,
       "learning_rate": 0.001,
-      "loss": 2.41264009475708,
       "step": 282
     },
     {
-      "epoch": 0.15844622278013146,
-      "grad_norm": 0.20107389986515045,
       "learning_rate": 0.001,
-      "loss": 2.2635366916656494,
       "step": 284
     },
     {
-      "epoch": 0.15956204125041407,
-      "grad_norm": 0.21727515757083893,
       "learning_rate": 0.001,
-      "loss": 2.3660287857055664,
       "step": 286
     },
     {
-      "epoch": 0.1606778597206967,
-      "grad_norm": 0.24364490807056427,
       "learning_rate": 0.001,
-      "loss": 2.3493285179138184,
       "step": 288
     },
     {
-      "epoch": 0.1617936781909793,
-      "grad_norm": 0.16181355714797974,
       "learning_rate": 0.001,
-      "loss": 2.308107852935791,
       "step": 290
     },
     {
-      "epoch": 0.16290949666126192,
-      "grad_norm": 0.1615009605884552,
       "learning_rate": 0.001,
-      "loss": 2.3350491523742676,
       "step": 292
     },
     {
-      "epoch": 0.16402531513154453,
-      "grad_norm": 0.2130037397146225,
       "learning_rate": 0.001,
-      "loss": 2.3082756996154785,
       "step": 294
     },
     {
-      "epoch": 0.16514113360182714,
-      "grad_norm": 0.20168843865394592,
       "learning_rate": 0.001,
-      "loss": 2.2046236991882324,
       "step": 296
     },
     {
-      "epoch": 0.16625695207210978,
-      "grad_norm": 0.15809662640094757,
       "learning_rate": 0.001,
-      "loss": 2.330395221710205,
       "step": 298
     },
     {
-      "epoch": 0.16737277054239239,
-      "grad_norm": 0.15686053037643433,
       "learning_rate": 0.001,
-      "loss": 2.292219638824463,
       "step": 300
     },
     {
-      "epoch": 0.168488589012675,
-      "grad_norm": 0.15200775861740112,
       "learning_rate": 0.001,
-      "loss": 2.2800769805908203,
       "step": 302
     },
     {
-      "epoch": 0.1696044074829576,
-      "grad_norm": 0.15325236320495605,
       "learning_rate": 0.001,
-      "loss": 2.299412727355957,
       "step": 304
     },
     {
-      "epoch": 0.17072022595324024,
-      "grad_norm": 0.21173831820487976,
       "learning_rate": 0.001,
-      "loss": 2.264983654022217,
       "step": 306
     },
     {
-      "epoch": 0.17183604442352285,
-      "grad_norm": 0.13201819360256195,
       "learning_rate": 0.001,
-      "loss": 2.2686514854431152,
       "step": 308
     },
     {
-      "epoch": 0.17295186289380546,
-      "grad_norm": 0.18356923758983612,
       "learning_rate": 0.001,
-      "loss": 2.212728261947632,
       "step": 310
     },
     {
-      "epoch": 0.17406768136408807,
-      "grad_norm": 0.1575055569410324,
       "learning_rate": 0.001,
-      "loss": 2.205423593521118,
       "step": 312
     },
     {
-      "epoch": 0.1751834998343707,
-      "grad_norm": 0.15859168767929077,
       "learning_rate": 0.001,
-      "loss": 2.1404268741607666,
       "step": 314
     },
     {
-      "epoch": 0.1762993183046533,
-      "grad_norm": 0.15114940702915192,
       "learning_rate": 0.001,
-      "loss": 2.2634060382843018,
       "step": 316
     },
     {
-      "epoch": 0.17741513677493592,
-      "grad_norm": 0.19858042895793915,
       "learning_rate": 0.001,
-      "loss": 2.3260059356689453,
       "step": 318
     },
     {
-      "epoch": 0.17853095524521853,
-      "grad_norm": 0.1441042721271515,
       "learning_rate": 0.001,
-      "loss": 2.4158811569213867,
       "step": 320
     },
     {
-      "epoch": 0.17964677371550117,
-      "grad_norm": 0.18635860085487366,
       "learning_rate": 0.001,
-      "loss": 2.2518999576568604,
       "step": 322
     },
     {
-      "epoch": 0.18076259218578378,
-      "grad_norm": 0.17651648819446564,
       "learning_rate": 0.001,
-      "loss": 2.4583635330200195,
       "step": 324
     },
     {
-      "epoch": 0.18187841065606639,
-      "grad_norm": 0.1565348356962204,
       "learning_rate": 0.001,
-      "loss": 2.1475563049316406,
       "step": 326
     },
     {
-      "epoch": 0.182994229126349,
-      "grad_norm": 0.16634659469127655,
       "learning_rate": 0.001,
-      "loss": 2.3245418071746826,
       "step": 328
     },
     {
-      "epoch": 0.18411004759663163,
-      "grad_norm": 0.1804242581129074,
       "learning_rate": 0.001,
-      "loss": 2.273958206176758,
       "step": 330
     },
     {
-      "epoch": 0.18522586606691424,
-      "grad_norm": 0.16945241391658783,
       "learning_rate": 0.001,
-      "loss": 2.300755500793457,
       "step": 332
     },
     {
-      "epoch": 0.18634168453719685,
-      "grad_norm": 0.14885590970516205,
       "learning_rate": 0.001,
-      "loss": 2.2134432792663574,
       "step": 334
     },
     {
-      "epoch": 0.18745750300747946,
-      "grad_norm": 0.15701663494110107,
       "learning_rate": 0.001,
-      "loss": 2.1696512699127197,
       "step": 336
     },
     {
-      "epoch": 0.1885733214777621,
-      "grad_norm": 0.1239931732416153,
       "learning_rate": 0.001,
-      "loss": 2.4542789459228516,
       "step": 338
     },
     {
-      "epoch": 0.1896891399480447,
-      "grad_norm": 0.14475102722644806,
       "learning_rate": 0.001,
-      "loss": 2.1178598403930664,
       "step": 340
     },
     {
-      "epoch": 0.19080495841832731,
-      "grad_norm": 0.1316973865032196,
       "learning_rate": 0.001,
-      "loss": 2.1780142784118652,
       "step": 342
     },
     {
-      "epoch": 0.19192077688860992,
-      "grad_norm": 0.24160316586494446,
       "learning_rate": 0.001,
-      "loss": 2.304316282272339,
       "step": 344
     },
     {
-      "epoch": 0.19303659535889256,
-      "grad_norm": 0.2638860046863556,
       "learning_rate": 0.001,
-      "loss": 2.007706642150879,
       "step": 346
     },
     {
-      "epoch": 0.19415241382917517,
-      "grad_norm": 0.1564919799566269,
       "learning_rate": 0.001,
-      "loss": 2.293002128601074,
       "step": 348
     },
     {
-      "epoch": 0.19526823229945778,
-      "grad_norm": 0.15639542043209076,
       "learning_rate": 0.001,
-      "loss": 2.196688652038574,
       "step": 350
     },
     {
-      "epoch": 0.1963840507697404,
-      "grad_norm": 0.17722801864147186,
       "learning_rate": 0.001,
-      "loss": 2.3652334213256836,
       "step": 352
     },
     {
-      "epoch": 0.19749986924002302,
-      "grad_norm": 0.13440386950969696,
       "learning_rate": 0.001,
-      "loss": 2.3144712448120117,
       "step": 354
     },
     {
-      "epoch": 0.19861568771030563,
-      "grad_norm": 0.155509814620018,
       "learning_rate": 0.001,
-      "loss": 2.225184679031372,
       "step": 356
     },
     {
-      "epoch": 0.19973150618058824,
-      "grad_norm": 0.1406576782464981,
       "learning_rate": 0.001,
-      "loss": 2.120429515838623,
       "step": 358
     }
   ],
   "logging_steps": 2,
-  "max_steps": 3586,
   "num_input_tokens_seen": 0,
   "num_train_epochs": 2,
-  "save_steps": 359,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
@@ -1280,7 +1700,7 @@
       "attributes": {}
     }
   },
-  "total_flos": 1.023882922379182e+17,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null

   "best_global_step": null,
   "best_metric": null,
   "best_model_checkpoint": null,
+  "epoch": 0.2000104607981589,
   "eval_steps": 500,
+  "global_step": 478,
   "is_hyper_param_search": false,
   "is_local_process_zero": true,
   "is_world_process_zero": true,
   "log_history": [
     {
+      "epoch": 0.0008368638527119619,
+      "grad_norm": 0.3337200880050659,
+      "learning_rate": 6.993006993006993e-06,
+      "loss": 2.762211322784424,
       "step": 2
     },
     {
+      "epoch": 0.0016737277054239238,
+      "grad_norm": 0.34152543544769287,
+      "learning_rate": 2.097902097902098e-05,
+      "loss": 2.7241592407226562,
       "step": 4
     },
     {
+      "epoch": 0.002510591558135886,
+      "grad_norm": 0.4462524950504303,
+      "learning_rate": 3.4965034965034965e-05,
+      "loss": 2.980071544647217,
       "step": 6
     },
     {
+      "epoch": 0.0033474554108478475,
+      "grad_norm": 0.3854326009750366,
+      "learning_rate": 4.895104895104895e-05,
+      "loss": 2.878772258758545,
       "step": 8
     },
     {
+      "epoch": 0.00418431926355981,
+      "grad_norm": 0.3850240409374237,
+      "learning_rate": 6.293706293706295e-05,
+      "loss": 2.9001553058624268,
       "step": 10
     },
     {
+      "epoch": 0.005021183116271772,
+      "grad_norm": 0.3354458808898926,
+      "learning_rate": 7.692307692307693e-05,
+      "loss": 2.647189140319824,
       "step": 12
     },
     {
+      "epoch": 0.005858046968983733,
+      "grad_norm": 0.3446701765060425,
+      "learning_rate": 9.090909090909092e-05,
+      "loss": 2.8157906532287598,
       "step": 14
     },
     {
+      "epoch": 0.006694910821695695,
+      "grad_norm": 0.23931172490119934,
+      "learning_rate": 0.0001048951048951049,
+      "loss": 2.746607780456543,
       "step": 16
     },
     {
+      "epoch": 0.007531774674407657,
+      "grad_norm": 0.37482818961143494,
+      "learning_rate": 0.00011888111888111889,
+      "loss": 2.8264577388763428,
       "step": 18
     },
     {
+      "epoch": 0.00836863852711962,
+      "grad_norm": 0.4178614020347595,
+      "learning_rate": 0.00013286713286713288,
+      "loss": 3.0178070068359375,
       "step": 20
     },
     {
+      "epoch": 0.009205502379831582,
+      "grad_norm": 0.3486288785934448,
+      "learning_rate": 0.00014685314685314684,
+      "loss": 2.670069694519043,
       "step": 22
     },
     {
+      "epoch": 0.010042366232543544,
+      "grad_norm": 0.27930015325546265,
+      "learning_rate": 0.00016083916083916083,
+      "loss": 2.8119349479675293,
       "step": 24
     },
     {
+      "epoch": 0.010879230085255504,
+      "grad_norm": 0.3605128824710846,
+      "learning_rate": 0.00017482517482517483,
+      "loss": 2.65777587890625,
       "step": 26
     },
     {
+      "epoch": 0.011716093937967466,
+      "grad_norm": 0.23365077376365662,
+      "learning_rate": 0.00018881118881118882,
+      "loss": 2.6923704147338867,
       "step": 28
     },
     {
+      "epoch": 0.012552957790679428,
+      "grad_norm": 0.24191462993621826,
+      "learning_rate": 0.00020279720279720278,
+      "loss": 2.791667938232422,
       "step": 30
     },
     {
+      "epoch": 0.01338982164339139,
+      "grad_norm": 0.3497094511985779,
+      "learning_rate": 0.00021678321678321677,
+      "loss": 2.957498073577881,
       "step": 32
     },
     {
+      "epoch": 0.014226685496103352,
+      "grad_norm": 0.26852497458457947,
+      "learning_rate": 0.0002307692307692308,
+      "loss": 2.6886894702911377,
       "step": 34
     },
     {
+      "epoch": 0.015063549348815314,
+      "grad_norm": 0.23093470931053162,
+      "learning_rate": 0.00024475524475524476,
+      "loss": 2.6747422218322754,
       "step": 36
     },
     {
+      "epoch": 0.015900413201527276,
+      "grad_norm": 0.30593279004096985,
+      "learning_rate": 0.0002587412587412588,
+      "loss": 2.6460132598876953,
       "step": 38
     },
     {
+      "epoch": 0.01673727705423924,
+      "grad_norm": 0.29761403799057007,
+      "learning_rate": 0.00027272727272727274,
+      "loss": 2.8844642639160156,
       "step": 40
     },
     {
+      "epoch": 0.0175741409069512,
+      "grad_norm": 0.2961265444755554,
+      "learning_rate": 0.0002867132867132867,
+      "loss": 2.834332227706909,
       "step": 42
     },
     {
+      "epoch": 0.018411004759663164,
+      "grad_norm": 0.23883932828903198,
+      "learning_rate": 0.00030069930069930067,
+      "loss": 2.7588770389556885,
       "step": 44
     },
     {
+      "epoch": 0.019247868612375124,
+      "grad_norm": 0.33696091175079346,
+      "learning_rate": 0.0003146853146853147,
+      "loss": 3.117276191711426,
       "step": 46
     },
     {
+      "epoch": 0.020084732465087088,
+      "grad_norm": 0.26080676913261414,
+      "learning_rate": 0.00032867132867132865,
+      "loss": 2.974804401397705,
       "step": 48
     },
     {
+      "epoch": 0.020921596317799048,
+      "grad_norm": 0.2795754373073578,
+      "learning_rate": 0.00034265734265734267,
+      "loss": 2.7198033332824707,
       "step": 50
     },
     {
+      "epoch": 0.02175846017051101,
+      "grad_norm": 0.3086928129196167,
+      "learning_rate": 0.0003566433566433567,
+      "loss": 2.9028894901275635,
       "step": 52
     },
     {
+      "epoch": 0.022595324023222972,
+      "grad_norm": 0.23168659210205078,
+      "learning_rate": 0.00037062937062937065,
+      "loss": 2.7301340103149414,
       "step": 54
     },
     {
+      "epoch": 0.023432187875934932,
+      "grad_norm": 0.32262903451919556,
+      "learning_rate": 0.00038461538461538467,
+      "loss": 3.091489791870117,
       "step": 56
     },
     {
+      "epoch": 0.024269051728646896,
+      "grad_norm": 0.2182423323392868,
+      "learning_rate": 0.0003986013986013986,
+      "loss": 2.7238540649414062,
       "step": 58
     },
     {
+      "epoch": 0.025105915581358856,
+      "grad_norm": 0.39541855454444885,
+      "learning_rate": 0.0004125874125874126,
+      "loss": 2.951951026916504,
       "step": 60
     },
     {
+      "epoch": 0.02594277943407082,
+      "grad_norm": 0.27646228671073914,
+      "learning_rate": 0.00042657342657342656,
+      "loss": 2.7110178470611572,
       "step": 62
     },
     {
+      "epoch": 0.02677964328678278,
+      "grad_norm": 0.29226240515708923,
+      "learning_rate": 0.0004405594405594406,
+      "loss": 2.6049153804779053,
       "step": 64
     },
     {
+      "epoch": 0.027616507139494744,
+      "grad_norm": 0.24897636473178864,
+      "learning_rate": 0.00045454545454545455,
+      "loss": 2.796565055847168,
       "step": 66
     },
     {
+      "epoch": 0.028453370992206704,
+      "grad_norm": 0.2893967926502228,
+      "learning_rate": 0.00046853146853146856,
+      "loss": 2.8579421043395996,
       "step": 68
     },
     {
+      "epoch": 0.029290234844918668,
+      "grad_norm": 0.21815066039562225,
+      "learning_rate": 0.00048251748251748253,
+      "loss": 2.6854171752929688,
       "step": 70
     },
     {
+      "epoch": 0.03012709869763063,
+      "grad_norm": 0.20148544013500214,
+      "learning_rate": 0.0004965034965034965,
+      "loss": 2.7209699153900146,
       "step": 72
     },
     {
+      "epoch": 0.030963962550342592,
+      "grad_norm": 0.24300239980220795,
+      "learning_rate": 0.0005104895104895105,
+      "loss": 2.743356704711914,
       "step": 74
     },
     {
+      "epoch": 0.03180082640305455,
+      "grad_norm": 0.18438707292079926,
+      "learning_rate": 0.0005244755244755245,
+      "loss": 2.5426692962646484,
       "step": 76
     },
     {
+      "epoch": 0.032637690255766516,
+      "grad_norm": 0.1793614774942398,
+      "learning_rate": 0.0005384615384615384,
+      "loss": 2.8510994911193848,
       "step": 78
     },
     {
+      "epoch": 0.03347455410847848,
+      "grad_norm": 0.1852932721376419,
+      "learning_rate": 0.0005524475524475524,
+      "loss": 2.814250946044922,
       "step": 80
     },
     {
+      "epoch": 0.03431141796119044,
+      "grad_norm": 0.21407772600650787,
+      "learning_rate": 0.0005664335664335665,
+      "loss": 2.5141940116882324,
       "step": 82
     },
     {
+      "epoch": 0.0351482818139024,
+      "grad_norm": 0.19092167913913727,
+      "learning_rate": 0.0005804195804195804,
+      "loss": 2.5931949615478516,
       "step": 84
     },
     {
+      "epoch": 0.035985145666614364,
+      "grad_norm": 0.21834400296211243,
+      "learning_rate": 0.0005944055944055944,
+      "loss": 2.6622567176818848,
       "step": 86
     },
     {
+      "epoch": 0.03682200951932633,
+      "grad_norm": 0.21294254064559937,
+      "learning_rate": 0.0006083916083916085,
+      "loss": 2.83119535446167,
       "step": 88
     },
     {
+      "epoch": 0.037658873372038285,
+      "grad_norm": 0.24639266729354858,
+      "learning_rate": 0.0006223776223776224,
+      "loss": 2.848186492919922,
       "step": 90
     },
     {
+      "epoch": 0.03849573722475025,
+      "grad_norm": 0.25334954261779785,
+      "learning_rate": 0.0006363636363636364,
+      "loss": 2.902247190475464,
       "step": 92
     },
     {
+      "epoch": 0.03933260107746221,
+      "grad_norm": 0.2275724560022354,
+      "learning_rate": 0.0006503496503496504,
+      "loss": 2.900848388671875,
       "step": 94
     },
     {
+      "epoch": 0.040169464930174176,
+      "grad_norm": 0.21515345573425293,
+      "learning_rate": 0.0006643356643356644,
+      "loss": 2.75372314453125,
       "step": 96
     },
     {
+      "epoch": 0.04100632878288613,
+      "grad_norm": 0.3527977466583252,
+      "learning_rate": 0.0006783216783216783,
+      "loss": 2.8876872062683105,
       "step": 98
     },
     {
+      "epoch": 0.041843192635598096,
+      "grad_norm": 0.20772546529769897,
+      "learning_rate": 0.0006923076923076923,
+      "loss": 2.5789003372192383,
       "step": 100
     },
     {
+      "epoch": 0.04268005648831006,
+      "grad_norm": 0.20183342695236206,
+      "learning_rate": 0.0007062937062937062,
+      "loss": 2.765045404434204,
       "step": 102
     },
     {
+      "epoch": 0.04351692034102202,
+      "grad_norm": 0.24883008003234863,
+      "learning_rate": 0.0007202797202797203,
+      "loss": 2.7212958335876465,
       "step": 104
     },
     {
+      "epoch": 0.04435378419373398,
+      "grad_norm": 0.18041826784610748,
+      "learning_rate": 0.0007342657342657343,
+      "loss": 2.6955039501190186,
       "step": 106
     },
     {
+      "epoch": 0.045190648046445944,
+      "grad_norm": 0.26103323698043823,
+      "learning_rate": 0.0007482517482517482,
+      "loss": 2.747805118560791,
       "step": 108
     },
     {
+      "epoch": 0.04602751189915791,
+      "grad_norm": 0.2088606357574463,
+      "learning_rate": 0.0007622377622377622,
+      "loss": 2.8328471183776855,
       "step": 110
     },
     {
+      "epoch": 0.046864375751869865,
+      "grad_norm": 0.25358667969703674,
+      "learning_rate": 0.0007762237762237763,
+      "loss": 2.838820457458496,
       "step": 112
     },
     {
+      "epoch": 0.04770123960458183,
+      "grad_norm": 0.2747185230255127,
+      "learning_rate": 0.0007902097902097902,
+      "loss": 2.862504482269287,
       "step": 114
     },
     {
+      "epoch": 0.04853810345729379,
+      "grad_norm": 0.2087952196598053,
+      "learning_rate": 0.0008041958041958042,
+      "loss": 2.549870014190674,
       "step": 116
     },
     {
+      "epoch": 0.049374967310005756,
+      "grad_norm": 0.19894322752952576,
+      "learning_rate": 0.0008181818181818183,
+      "loss": 2.5750961303710938,
       "step": 118
     },
     {
+      "epoch": 0.05021183116271771,
+      "grad_norm": 0.1930788904428482,
+      "learning_rate": 0.0008321678321678322,
+      "loss": 2.6718544960021973,
       "step": 120
     },
     {
+      "epoch": 0.051048695015429676,
+      "grad_norm": 0.24285174906253815,
+      "learning_rate": 0.0008461538461538462,
+      "loss": 2.6672987937927246,
       "step": 122
     },
     {
+      "epoch": 0.05188555886814164,
+      "grad_norm": 0.20339679718017578,
+      "learning_rate": 0.0008601398601398602,
+      "loss": 2.789313793182373,
       "step": 124
     },
     {
+      "epoch": 0.052722422720853604,
+      "grad_norm": 0.24407540261745453,
+      "learning_rate": 0.0008741258741258742,
+      "loss": 2.8427278995513916,
       "step": 126
     },
     {
+      "epoch": 0.05355928657356556,
+      "grad_norm": 0.18503564596176147,
+      "learning_rate": 0.0008881118881118882,
+      "loss": 2.490847587585449,
       "step": 128
     },
     {
+      "epoch": 0.054396150426277524,
+      "grad_norm": 0.18528853356838226,
+      "learning_rate": 0.000902097902097902,
+      "loss": 2.5757269859313965,
       "step": 130
     },
     {
+      "epoch": 0.05523301427898949,
+      "grad_norm": 0.19775666296482086,
+      "learning_rate": 0.000916083916083916,
+      "loss": 2.6315667629241943,
       "step": 132
     },
     {
+      "epoch": 0.05606987813170145,
+      "grad_norm": 0.17242901027202606,
+      "learning_rate": 0.0009300699300699301,
+      "loss": 2.6123533248901367,
       "step": 134
     },
     {
+      "epoch": 0.05690674198441341,
+      "grad_norm": 0.23534800112247467,
+      "learning_rate": 0.0009440559440559441,
+      "loss": 2.6102347373962402,
       "step": 136
     },
     {
+      "epoch": 0.05774360583712537,
+      "grad_norm": 0.24209259450435638,
+      "learning_rate": 0.000958041958041958,
+      "loss": 2.613840103149414,
       "step": 138
     },
     {
+      "epoch": 0.058580469689837336,
+      "grad_norm": 0.3087214529514313,
+      "learning_rate": 0.000972027972027972,
+      "loss": 2.6876606941223145,
       "step": 140
     },
     {
+      "epoch": 0.0594173335425493,
+      "grad_norm": 0.22155095636844635,
+      "learning_rate": 0.0009860139860139862,
+      "loss": 2.8234755992889404,
       "step": 142
     },
     {
+      "epoch": 0.06025419739526126,
+      "grad_norm": 0.21793203055858612,
       "learning_rate": 0.001,
+      "loss": 2.7899413108825684,
       "step": 144
     },
     {
+      "epoch": 0.06109106124797322,
+      "grad_norm": 0.17000611126422882,
       "learning_rate": 0.001,
+      "loss": 2.5153496265411377,
       "step": 146
     },
     {
+      "epoch": 0.061927925100685184,
+      "grad_norm": 0.21351996064186096,
       "learning_rate": 0.001,
+      "loss": 2.664409637451172,
       "step": 148
     },
     {
+      "epoch": 0.06276478895339714,
+      "grad_norm": 0.3195689916610718,
       "learning_rate": 0.001,
+      "loss": 2.7900476455688477,
       "step": 150
     },
     {
+      "epoch": 0.0636016528061091,
+      "grad_norm": 0.26381903886795044,
       "learning_rate": 0.001,
+      "loss": 2.7683029174804688,
       "step": 152
     },
     {
+      "epoch": 0.06443851665882107,
+      "grad_norm": 0.2637115716934204,
       "learning_rate": 0.001,
+      "loss": 2.655928373336792,
       "step": 154
     },
     {
+      "epoch": 0.06527538051153303,
+      "grad_norm": 0.18573474884033203,
       "learning_rate": 0.001,
+      "loss": 2.5204102993011475,
       "step": 156
     },
     {
+      "epoch": 0.066112244364245,
+      "grad_norm": 0.1925807148218155,
       "learning_rate": 0.001,
+      "loss": 2.63466215133667,
       "step": 158
     },
     {
+      "epoch": 0.06694910821695696,
+      "grad_norm": 0.30184367299079895,
       "learning_rate": 0.001,
+      "loss": 2.704226016998291,
       "step": 160
     },
     {
+      "epoch": 0.06778597206966891,
+      "grad_norm": 0.22436286509037018,
       "learning_rate": 0.001,
+      "loss": 2.5878210067749023,
       "step": 162
     },
     {
+      "epoch": 0.06862283592238087,
+      "grad_norm": 0.16771288216114044,
       "learning_rate": 0.001,
+      "loss": 2.5527443885803223,
       "step": 164
     },
     {
+      "epoch": 0.06945969977509284,
+      "grad_norm": 0.2011052370071411,
       "learning_rate": 0.001,
+      "loss": 2.544546127319336,
       "step": 166
     },
     {
+      "epoch": 0.0702965636278048,
+      "grad_norm": 0.2775769531726837,
       "learning_rate": 0.001,
+      "loss": 2.7540626525878906,
       "step": 168
     },
     {
+      "epoch": 0.07113342748051676,
+      "grad_norm": 0.2225073277950287,
       "learning_rate": 0.001,
+      "loss": 2.64829421043396,
       "step": 170
     },
     {
+      "epoch": 0.07197029133322873,
+      "grad_norm": 0.21853874623775482,
       "learning_rate": 0.001,
+      "loss": 2.6177806854248047,
       "step": 172
     },
     {
+      "epoch": 0.07280715518594069,
+      "grad_norm": 0.2083626687526703,
       "learning_rate": 0.001,
+      "loss": 2.607222557067871,
       "step": 174
     },
     {
+      "epoch": 0.07364401903865266,
+      "grad_norm": 0.2196309119462967,
       "learning_rate": 0.001,
+      "loss": 2.5272817611694336,
       "step": 176
     },
     {
+      "epoch": 0.0744808828913646,
+      "grad_norm": 0.16887669265270233,
       "learning_rate": 0.001,
+      "loss": 2.3910434246063232,
       "step": 178
     },
     {
+      "epoch": 0.07531774674407657,
+      "grad_norm": 0.30048754811286926,
       "learning_rate": 0.001,
+      "loss": 2.5647621154785156,
       "step": 180
     },
     {
+      "epoch": 0.07615461059678853,
+      "grad_norm": 0.15770305693149567,
       "learning_rate": 0.001,
+      "loss": 2.614769697189331,
       "step": 182
     },
     {
+      "epoch": 0.0769914744495005,
+      "grad_norm": 0.1820078045129776,
       "learning_rate": 0.001,
+      "loss": 2.6347270011901855,
       "step": 184
     },
     {
+      "epoch": 0.07782833830221246,
+      "grad_norm": 0.1556544154882431,
       "learning_rate": 0.001,
+      "loss": 2.465865135192871,
       "step": 186
     },
     {
+      "epoch": 0.07866520215492442,
+      "grad_norm": 0.20094197988510132,
       "learning_rate": 0.001,
+      "loss": 2.5143823623657227,
       "step": 188
     },
     {
+      "epoch": 0.07950206600763639,
+      "grad_norm": 0.1762073040008545,
       "learning_rate": 0.001,
+      "loss": 2.694760322570801,
       "step": 190
     },
     {
+      "epoch": 0.08033892986034835,
+      "grad_norm": 0.20501920580863953,
       "learning_rate": 0.001,
+      "loss": 2.632472515106201,
       "step": 192
     },
     {
+      "epoch": 0.0811757937130603,
+      "grad_norm": 0.21597976982593536,
       "learning_rate": 0.001,
+      "loss": 2.707061767578125,
       "step": 194
     },
     {
+      "epoch": 0.08201265756577227,
+      "grad_norm": 0.2293614000082016,
       "learning_rate": 0.001,
+      "loss": 2.540835380554199,
       "step": 196
     },
     {
+      "epoch": 0.08284952141848423,
+      "grad_norm": 0.167021706700325,
       "learning_rate": 0.001,
+      "loss": 2.4219717979431152,
       "step": 198
     },
     {
+      "epoch": 0.08368638527119619,
+      "grad_norm": 0.16806422173976898,
       "learning_rate": 0.001,
+      "loss": 2.5145626068115234,
       "step": 200
     },
     {
+      "epoch": 0.08452324912390816,
+      "grad_norm": 0.257610559463501,
       "learning_rate": 0.001,
+      "loss": 2.4911866188049316,
       "step": 202
     },
     {
+      "epoch": 0.08536011297662012,
+      "grad_norm": 0.17307472229003906,
       "learning_rate": 0.001,
+      "loss": 2.5154426097869873,
       "step": 204
     },
     {
+      "epoch": 0.08619697682933208,
+      "grad_norm": 0.17001985013484955,
       "learning_rate": 0.001,
+      "loss": 2.4847593307495117,
       "step": 206
     },
     {
+      "epoch": 0.08703384068204403,
+      "grad_norm": 0.18028728663921356,
       "learning_rate": 0.001,
+      "loss": 2.4794325828552246,
       "step": 208
     },
     {
+      "epoch": 0.087870704534756,
+      "grad_norm": 0.15724672377109528,
       "learning_rate": 0.001,
+      "loss": 2.4882359504699707,
       "step": 210
     },
     {
+      "epoch": 0.08870756838746796,
+      "grad_norm": 0.15854695439338684,
       "learning_rate": 0.001,
+      "loss": 2.526923894882202,
       "step": 212
     },
     {
+      "epoch": 0.08954443224017992,
+      "grad_norm": 0.1744718700647354,
       "learning_rate": 0.001,
+      "loss": 2.7622461318969727,
       "step": 214
     },
     {
+      "epoch": 0.09038129609289189,
+      "grad_norm": 0.20886735618114471,
       "learning_rate": 0.001,
+      "loss": 2.3967981338500977,
       "step": 216
     },
     {
+      "epoch": 0.09121815994560385,
+      "grad_norm": 0.22155894339084625,
       "learning_rate": 0.001,
+      "loss": 2.7355470657348633,
       "step": 218
     },
     {
+      "epoch": 0.09205502379831582,
+      "grad_norm": 0.22641800343990326,
       "learning_rate": 0.001,
+      "loss": 2.64965558052063,
       "step": 220
     },
     {
+      "epoch": 0.09289188765102778,
+      "grad_norm": 0.21956373751163483,
       "learning_rate": 0.001,
+      "loss": 2.612518787384033,
       "step": 222
     },
     {
+      "epoch": 0.09372875150373973,
+      "grad_norm": 0.15686312317848206,
       "learning_rate": 0.001,
+      "loss": 2.4648630619049072,
       "step": 224
     },
     {
+      "epoch": 0.0945656153564517,
+      "grad_norm": 0.19716182351112366,
       "learning_rate": 0.001,
+      "loss": 2.503875255584717,
       "step": 226
     },
     {
+      "epoch": 0.09540247920916366,
+      "grad_norm": 0.19988487660884857,
       "learning_rate": 0.001,
+      "loss": 2.503325939178467,
       "step": 228
     },
     {
+      "epoch": 0.09623934306187562,
+      "grad_norm": 0.30279281735420227,
       "learning_rate": 0.001,
+      "loss": 2.512791156768799,
       "step": 230
     },
     {
+      "epoch": 0.09707620691458758,
+      "grad_norm": 0.23493289947509766,
       "learning_rate": 0.001,
+      "loss": 2.343080997467041,
       "step": 232
     },
     {
+      "epoch": 0.09791307076729955,
+      "grad_norm": 0.16931183636188507,
       "learning_rate": 0.001,
+      "loss": 2.421757698059082,
       "step": 234
     },
     {
+      "epoch": 0.09874993462001151,
+      "grad_norm": 0.22772321105003357,
       "learning_rate": 0.001,
+      "loss": 2.5251705646514893,
       "step": 236
     },
     {
+      "epoch": 0.09958679847272348,
+      "grad_norm": 0.2512679100036621,
       "learning_rate": 0.001,
+      "loss": 2.380053758621216,
       "step": 238
     },
     {
+      "epoch": 0.10042366232543543,
+      "grad_norm": 0.21434853971004486,
       "learning_rate": 0.001,
+      "loss": 2.513920783996582,
       "step": 240
     },
     {
+      "epoch": 0.10126052617814739,
+      "grad_norm": 0.18697810173034668,
       "learning_rate": 0.001,
+      "loss": 2.4869494438171387,
       "step": 242
     },
     {
+      "epoch": 0.10209739003085935,
+      "grad_norm": 0.193964421749115,
       "learning_rate": 0.001,
+      "loss": 2.5449650287628174,
       "step": 244
     },
     {
+      "epoch": 0.10293425388357132,
+      "grad_norm": 0.17594163119792938,
       "learning_rate": 0.001,
+      "loss": 2.410844326019287,
       "step": 246
     },
     {
+      "epoch": 0.10377111773628328,
+      "grad_norm": 0.18385350704193115,
       "learning_rate": 0.001,
+      "loss": 2.4730141162872314,
       "step": 248
     },
     {
+      "epoch": 0.10460798158899524,
+      "grad_norm": 0.20136837661266327,
       "learning_rate": 0.001,
+      "loss": 2.6027231216430664,
       "step": 250
     },
     {
+      "epoch": 0.10544484544170721,
+      "grad_norm": 0.2197246551513672,
       "learning_rate": 0.001,
+      "loss": 2.5501255989074707,
       "step": 252
     },
     {
+      "epoch": 0.10628170929441916,
+      "grad_norm": 0.19168448448181152,
       "learning_rate": 0.001,
+      "loss": 2.1543636322021484,
       "step": 254
     },
     {
+      "epoch": 0.10711857314713112,
+      "grad_norm": 0.17592091858386993,
       "learning_rate": 0.001,
+      "loss": 2.464280605316162,
       "step": 256
     },
     {
+      "epoch": 0.10795543699984309,
+      "grad_norm": 0.19394183158874512,
       "learning_rate": 0.001,
+      "loss": 2.4328200817108154,
       "step": 258
     },
     {
+      "epoch": 0.10879230085255505,
+      "grad_norm": 0.18573930859565735,
       "learning_rate": 0.001,
+      "loss": 2.4670631885528564,
       "step": 260
     },
     {
+      "epoch": 0.10962916470526701,
+      "grad_norm": 0.1931317299604416,
       "learning_rate": 0.001,
+      "loss": 2.5041933059692383,
       "step": 262
     },
     {
+      "epoch": 0.11046602855797898,
+      "grad_norm": 0.19977472722530365,
       "learning_rate": 0.001,
+      "loss": 2.5579640865325928,
       "step": 264
     },
     {
+      "epoch": 0.11130289241069094,
+      "grad_norm": 0.1664435863494873,
       "learning_rate": 0.001,
+      "loss": 2.426025390625,
       "step": 266
     },
     {
+      "epoch": 0.1121397562634029,
+      "grad_norm": 0.24706660211086273,
       "learning_rate": 0.001,
+      "loss": 2.2006964683532715,
       "step": 268
     },
     {
+      "epoch": 0.11297662011611485,
+      "grad_norm": 0.1717725545167923,
       "learning_rate": 0.001,
+      "loss": 2.4420886039733887,
       "step": 270
     },
     {
+      "epoch": 0.11381348396882682,
+      "grad_norm": 0.18404383957386017,
       "learning_rate": 0.001,
+      "loss": 2.245258092880249,
       "step": 272
     },
     {
+      "epoch": 0.11465034782153878,
+      "grad_norm": 0.16331890225410461,
       "learning_rate": 0.001,
+      "loss": 2.4557533264160156,
       "step": 274
     },
     {
+      "epoch": 0.11548721167425074,
+      "grad_norm": 0.17149455845355988,
       "learning_rate": 0.001,
+      "loss": 2.4081532955169678,
       "step": 276
     },
     {
+      "epoch": 0.11632407552696271,
+      "grad_norm": 0.19532416760921478,
       "learning_rate": 0.001,
+      "loss": 2.3092994689941406,
       "step": 278
     },
     {
+      "epoch": 0.11716093937967467,
+      "grad_norm": 0.1597677618265152,
       "learning_rate": 0.001,
+      "loss": 2.3227827548980713,
       "step": 280
     },
     {
+      "epoch": 0.11799780323238664,
+      "grad_norm": 0.22333088517189026,
       "learning_rate": 0.001,
+      "loss": 2.1312952041625977,
       "step": 282
     },
     {
+      "epoch": 0.1188346670850986,
+      "grad_norm": 0.15165381133556366,
       "learning_rate": 0.001,
+      "loss": 2.4377574920654297,
       "step": 284
     },
     {
+      "epoch": 0.11967153093781055,
+      "grad_norm": 0.18507729470729828,
       "learning_rate": 0.001,
+      "loss": 2.4601995944976807,
       "step": 286
     },
     {
+      "epoch": 0.12050839479052251,
+      "grad_norm": 0.2087109535932541,
       "learning_rate": 0.001,
+      "loss": 2.4728264808654785,
       "step": 288
     },
     {
+      "epoch": 0.12134525864323448,
+      "grad_norm": 0.18060186505317688,
       "learning_rate": 0.001,
+      "loss": 2.356816291809082,
       "step": 290
     },
     {
+      "epoch": 0.12218212249594644,
+      "grad_norm": 0.19519281387329102,
       "learning_rate": 0.001,
+      "loss": 2.4263811111450195,
       "step": 292
     },
     {
+      "epoch": 0.1230189863486584,
+      "grad_norm": 0.19043858349323273,
       "learning_rate": 0.001,
+      "loss": 2.4366135597229004,
       "step": 294
     },
     {
+      "epoch": 0.12385585020137037,
+      "grad_norm": 0.1955493986606598,
       "learning_rate": 0.001,
+      "loss": 2.46212100982666,
       "step": 296
     },
     {
+      "epoch": 0.12469271405408233,
+      "grad_norm": 0.2401607781648636,
       "learning_rate": 0.001,
+      "loss": 2.3406548500061035,
       "step": 298
     },
     {
+      "epoch": 0.12552957790679428,
+      "grad_norm": 0.18235540390014648,
       "learning_rate": 0.001,
+      "loss": 2.3392832279205322,
       "step": 300
     },
     {
+      "epoch": 0.12636644175950626,
+      "grad_norm": 0.18857377767562866,
       "learning_rate": 0.001,
+      "loss": 2.4723331928253174,
       "step": 302
     },
     {
+      "epoch": 0.1272033056122182,
+      "grad_norm": 0.17727996408939362,
       "learning_rate": 0.001,
+      "loss": 2.2125983238220215,
       "step": 304
     },
     {
+      "epoch": 0.1280401694649302,
+      "grad_norm": 0.21876183152198792,
       "learning_rate": 0.001,
+      "loss": 2.3917431831359863,
       "step": 306
     },
     {
+      "epoch": 0.12887703331764214,
+      "grad_norm": 0.2529425621032715,
       "learning_rate": 0.001,
+      "loss": 2.261897325515747,
       "step": 308
     },
     {
+      "epoch": 0.1297138971703541,
+      "grad_norm": 0.16678006947040558,
       "learning_rate": 0.001,
+      "loss": 2.3113372325897217,
       "step": 310
     },
     {
+      "epoch": 0.13055076102306606,
+      "grad_norm": 0.17776496708393097,
       "learning_rate": 0.001,
+      "loss": 2.124931573867798,
       "step": 312
     },
     {
+      "epoch": 0.13138762487577801,
+      "grad_norm": 0.16098302602767944,
       "learning_rate": 0.001,
+      "loss": 2.259241819381714,
       "step": 314
     },
     {
+      "epoch": 0.13222448872849,
+      "grad_norm": 0.17063255608081818,
       "learning_rate": 0.001,
+      "loss": 2.383655309677124,
       "step": 316
     },
     {
+      "epoch": 0.13306135258120194,
+      "grad_norm": 0.21245349943637848,
       "learning_rate": 0.001,
+      "loss": 2.134192705154419,
       "step": 318
     },
     {
+      "epoch": 0.13389821643391392,
+      "grad_norm": 0.15914179384708405,
       "learning_rate": 0.001,
+      "loss": 2.264336347579956,
       "step": 320
     },
     {
+      "epoch": 0.13473508028662587,
+      "grad_norm": 0.16097018122673035,
       "learning_rate": 0.001,
+      "loss": 2.3899030685424805,
       "step": 322
     },
     {
+      "epoch": 0.13557194413933782,
+      "grad_norm": 0.3283947706222534,
       "learning_rate": 0.001,
+      "loss": 2.3028290271759033,
       "step": 324
     },
     {
+      "epoch": 0.1364088079920498,
+      "grad_norm": 0.16399675607681274,
       "learning_rate": 0.001,
+      "loss": 2.232480525970459,
       "step": 326
     },
     {
+      "epoch": 0.13724567184476175,
+      "grad_norm": 0.20280639827251434,
       "learning_rate": 0.001,
+      "loss": 2.4445927143096924,
       "step": 328
     },
     {
+      "epoch": 0.13808253569747372,
+      "grad_norm": 0.14698295295238495,
       "learning_rate": 0.001,
+      "loss": 2.3281705379486084,
       "step": 330
     },
     {
+      "epoch": 0.13891939955018567,
+      "grad_norm": 0.22716903686523438,
       "learning_rate": 0.001,
+      "loss": 2.3673253059387207,
       "step": 332
     },
     {
+      "epoch": 0.13975626340289765,
+      "grad_norm": 0.17115481197834015,
       "learning_rate": 0.001,
+      "loss": 2.244957208633423,
       "step": 334
     },
     {
+      "epoch": 0.1405931272556096,
+      "grad_norm": 0.23570868372917175,
       "learning_rate": 0.001,
+      "loss": 2.3726439476013184,
       "step": 336
     },
     {
+      "epoch": 0.14142999110832155,
+      "grad_norm": 0.19731327891349792,
       "learning_rate": 0.001,
+      "loss": 2.2067930698394775,
       "step": 338
     },
     {
+      "epoch": 0.14226685496103353,
+      "grad_norm": 0.21654491126537323,
       "learning_rate": 0.001,
+      "loss": 2.2029366493225098,
       "step": 340
     },
     {
+      "epoch": 0.14310371881374548,
+      "grad_norm": 0.20219019055366516,
       "learning_rate": 0.001,
+      "loss": 2.4190821647644043,
       "step": 342
     },
     {
+      "epoch": 0.14394058266645746,
+      "grad_norm": 0.19677171111106873,
       "learning_rate": 0.001,
+      "loss": 2.3690743446350098,
       "step": 344
     },
     {
+      "epoch": 0.1447774465191694,
+      "grad_norm": 0.16813816130161285,
       "learning_rate": 0.001,
+      "loss": 2.247950315475464,
       "step": 346
     },
     {
+      "epoch": 0.14561431037188138,
+      "grad_norm": 0.1347476840019226,
       "learning_rate": 0.001,
+      "loss": 2.246828079223633,
       "step": 348
     },
     {
+      "epoch": 0.14645117422459333,
+      "grad_norm": 0.20790256559848785,
       "learning_rate": 0.001,
+      "loss": 2.355773448944092,
       "step": 350
     },
     {
+      "epoch": 0.1472880380773053,
+      "grad_norm": 0.28660014271736145,
       "learning_rate": 0.001,
+      "loss": 2.0762479305267334,
       "step": 352
     },
     {
+      "epoch": 0.14812490193001726,
+      "grad_norm": 0.22139360010623932,
       "learning_rate": 0.001,
+      "loss": 2.250910758972168,
       "step": 354
     },
     {
+      "epoch": 0.1489617657827292,
+      "grad_norm": 0.2779833972454071,
       "learning_rate": 0.001,
+      "loss": 2.3055973052978516,
       "step": 356
     },
     {
+      "epoch": 0.1497986296354412,
+      "grad_norm": 0.20132610201835632,
       "learning_rate": 0.001,
+      "loss": 2.35807728767395,
       "step": 358
+    },
+    {
+      "epoch": 0.15063549348815314,
+      "grad_norm": 0.19047747552394867,
+      "learning_rate": 0.001,
+      "loss": 2.2559621334075928,
+      "step": 360
+    },
+    {
+      "epoch": 0.15147235734086512,
+      "grad_norm": 0.24329423904418945,
+      "learning_rate": 0.001,
+      "loss": 2.2966115474700928,
+      "step": 362
+    },
+    {
+      "epoch": 0.15230922119357707,
+      "grad_norm": 0.16524982452392578,
+      "learning_rate": 0.001,
+      "loss": 2.2288427352905273,
+      "step": 364
+    },
+    {
+      "epoch": 0.15314608504628904,
+      "grad_norm": 0.18306642770767212,
+      "learning_rate": 0.001,
+      "loss": 2.1365907192230225,
+      "step": 366
+    },
+    {
+      "epoch": 0.153982948899001,
+      "grad_norm": 0.16839206218719482,
+      "learning_rate": 0.001,
+      "loss": 2.313831329345703,
+      "step": 368
+    },
+    {
+      "epoch": 0.15481981275171294,
+      "grad_norm": 0.18757441639900208,
+      "learning_rate": 0.001,
+      "loss": 2.3335580825805664,
+      "step": 370
+    },
+    {
+      "epoch": 0.15565667660442492,
+      "grad_norm": 0.1820153146982193,
+      "learning_rate": 0.001,
+      "loss": 2.1603517532348633,
+      "step": 372
+    },
+    {
+      "epoch": 0.15649354045713687,
+      "grad_norm": 0.1970907747745514,
+      "learning_rate": 0.001,
+      "loss": 2.1379265785217285,
+      "step": 374
+    },
+    {
+      "epoch": 0.15733040430984885,
+      "grad_norm": 0.15956442058086395,
+      "learning_rate": 0.001,
+      "loss": 2.3535470962524414,
+      "step": 376
+    },
+    {
+      "epoch": 0.1581672681625608,
+      "grad_norm": 0.23435263335704803,
+      "learning_rate": 0.001,
+      "loss": 2.1850802898406982,
+      "step": 378
+    },
+    {
+      "epoch": 0.15900413201527278,
+      "grad_norm": 0.22058139741420746,
+      "learning_rate": 0.001,
+      "loss": 2.2454843521118164,
+      "step": 380
+    },
+    {
+      "epoch": 0.15984099586798473,
+      "grad_norm": 0.1469336450099945,
+      "learning_rate": 0.001,
+      "loss": 2.144195795059204,
+      "step": 382
+    },
+    {
+      "epoch": 0.1606778597206967,
+      "grad_norm": 0.2044474184513092,
+      "learning_rate": 0.001,
+      "loss": 2.287320375442505,
+      "step": 384
+    },
+    {
+      "epoch": 0.16151472357340865,
+      "grad_norm": 0.17819838225841522,
+      "learning_rate": 0.001,
+      "loss": 2.24429988861084,
+      "step": 386
+    },
+    {
+      "epoch": 0.1623515874261206,
+      "grad_norm": 0.19163808226585388,
+      "learning_rate": 0.001,
+      "loss": 2.2899177074432373,
+      "step": 388
+    },
+    {
+      "epoch": 0.16318845127883258,
+      "grad_norm": 0.1723131686449051,
+      "learning_rate": 0.001,
+      "loss": 2.270850896835327,
+      "step": 390
+    },
+    {
+      "epoch": 0.16402531513154453,
+      "grad_norm": 0.27207961678504944,
+      "learning_rate": 0.001,
+      "loss": 2.0873520374298096,
+      "step": 392
+    },
+    {
+      "epoch": 0.1648621789842565,
+      "grad_norm": 0.21700863540172577,
+      "learning_rate": 0.001,
+      "loss": 2.1603569984436035,
+      "step": 394
+    },
+    {
+      "epoch": 0.16569904283696846,
+      "grad_norm": 0.1941995620727539,
+      "learning_rate": 0.001,
+      "loss": 2.0528125762939453,
+      "step": 396
+    },
+    {
+      "epoch": 0.16653590668968044,
+      "grad_norm": 0.21802550554275513,
+      "learning_rate": 0.001,
+      "loss": 2.2607946395874023,
+      "step": 398
+    },
+    {
+      "epoch": 0.16737277054239239,
+      "grad_norm": 0.16812366247177124,
+      "learning_rate": 0.001,
+      "loss": 2.233855962753296,
+      "step": 400
+    },
+    {
+      "epoch": 0.16820963439510433,
+      "grad_norm": 0.14934280514717102,
+      "learning_rate": 0.001,
+      "loss": 2.185300827026367,
+      "step": 402
+    },
+    {
+      "epoch": 0.1690464982478163,
+      "grad_norm": 0.20246241986751556,
+      "learning_rate": 0.001,
+      "loss": 2.357269287109375,
+      "step": 404
+    },
+    {
+      "epoch": 0.16988336210052826,
+      "grad_norm": 0.16528300940990448,
+      "learning_rate": 0.001,
+      "loss": 2.1169815063476562,
+      "step": 406
+    },
+    {
+      "epoch": 0.17072022595324024,
+      "grad_norm": 0.1721237450838089,
+      "learning_rate": 0.001,
+      "loss": 2.1101884841918945,
+      "step": 408
+    },
+    {
+      "epoch": 0.1715570898059522,
+      "grad_norm": 0.13873209059238434,
+      "learning_rate": 0.001,
+      "loss": 2.2253384590148926,
+      "step": 410
+    },
+    {
+      "epoch": 0.17239395365866417,
+      "grad_norm": 0.16752997040748596,
+      "learning_rate": 0.001,
+      "loss": 2.0188238620758057,
+      "step": 412
+    },
+    {
+      "epoch": 0.17323081751137612,
+      "grad_norm": 0.18625566363334656,
+      "learning_rate": 0.001,
+      "loss": 2.2110636234283447,
+      "step": 414
+    },
+    {
+      "epoch": 0.17406768136408807,
+      "grad_norm": 0.1517583131790161,
+      "learning_rate": 0.001,
+      "loss": 2.0919532775878906,
+      "step": 416
+    },
+    {
+      "epoch": 0.17490454521680004,
+      "grad_norm": 0.2596180737018585,
+      "learning_rate": 0.001,
+      "loss": 2.106278896331787,
+      "step": 418
+    },
+    {
+      "epoch": 0.175741409069512,
+      "grad_norm": 0.16954955458641052,
+      "learning_rate": 0.001,
+      "loss": 2.096888780593872,
+      "step": 420
+    },
+    {
+      "epoch": 0.17657827292222397,
+      "grad_norm": 0.17552365362644196,
+      "learning_rate": 0.001,
+      "loss": 2.125303030014038,
+      "step": 422
+    },
+    {
+      "epoch": 0.17741513677493592,
+      "grad_norm": 0.17826397716999054,
+      "learning_rate": 0.001,
+      "loss": 2.27309250831604,
+      "step": 424
+    },
+    {
+      "epoch": 0.1782520006276479,
+      "grad_norm": 0.16280092298984528,
+      "learning_rate": 0.001,
+      "loss": 2.281273365020752,
+      "step": 426
+    },
+    {
+      "epoch": 0.17908886448035985,
+      "grad_norm": 0.16074128448963165,
+      "learning_rate": 0.001,
+      "loss": 2.273179292678833,
+      "step": 428
+    },
+    {
+      "epoch": 0.17992572833307183,
+      "grad_norm": 0.26484987139701843,
+      "learning_rate": 0.001,
+      "loss": 2.224212408065796,
+      "step": 430
+    },
+    {
+      "epoch": 0.18076259218578378,
+      "grad_norm": 0.1629486382007599,
+      "learning_rate": 0.001,
+      "loss": 2.289903163909912,
+      "step": 432
+    },
+    {
+      "epoch": 0.18159945603849573,
+      "grad_norm": 0.2637065052986145,
+      "learning_rate": 0.001,
+      "loss": 2.0451815128326416,
+      "step": 434
+    },
+    {
+      "epoch": 0.1824363198912077,
+      "grad_norm": 0.16380563378334045,
+      "learning_rate": 0.001,
+      "loss": 2.1637766361236572,
+      "step": 436
+    },
+    {
+      "epoch": 0.18327318374391965,
+      "grad_norm": 0.20121097564697266,
+      "learning_rate": 0.001,
+      "loss": 2.2749342918395996,
+      "step": 438
+    },
+    {
+      "epoch": 0.18411004759663163,
+      "grad_norm": 0.18869942426681519,
+      "learning_rate": 0.001,
+      "loss": 2.1522178649902344,
+      "step": 440
+    },
+    {
+      "epoch": 0.18494691144934358,
+      "grad_norm": 0.1619872748851776,
+      "learning_rate": 0.001,
+      "loss": 2.221017360687256,
+      "step": 442
+    },
+    {
+      "epoch": 0.18578377530205556,
+      "grad_norm": 0.19862474501132965,
+      "learning_rate": 0.001,
+      "loss": 2.2464003562927246,
+      "step": 444
+    },
+    {
+      "epoch": 0.1866206391547675,
+      "grad_norm": 0.15089546144008636,
+      "learning_rate": 0.001,
+      "loss": 2.1089327335357666,
+      "step": 446
+    },
+    {
+      "epoch": 0.18745750300747946,
+      "grad_norm": 0.1756240427494049,
+      "learning_rate": 0.001,
+      "loss": 1.980682611465454,
+      "step": 448
+    },
+    {
+      "epoch": 0.18829436686019144,
+      "grad_norm": 0.15822061896324158,
+      "learning_rate": 0.001,
+      "loss": 2.3368139266967773,
+      "step": 450
+    },
+    {
+      "epoch": 0.1891312307129034,
+      "grad_norm": 0.1786164939403534,
+      "learning_rate": 0.001,
+      "loss": 2.182668685913086,
+      "step": 452
+    },
+    {
+      "epoch": 0.18996809456561536,
+      "grad_norm": 0.13735046982765198,
+      "learning_rate": 0.001,
+      "loss": 2.0651893615722656,
+      "step": 454
+    },
+    {
+      "epoch": 0.19080495841832731,
+      "grad_norm": 0.17371626198291779,
+      "learning_rate": 0.001,
+      "loss": 2.1581287384033203,
+      "step": 456
+    },
+    {
+      "epoch": 0.1916418222710393,
+      "grad_norm": 0.13661286234855652,
+      "learning_rate": 0.001,
+      "loss": 2.1913771629333496,
+      "step": 458
+    },
+    {
+      "epoch": 0.19247868612375124,
+      "grad_norm": 0.16636984050273895,
+      "learning_rate": 0.001,
+      "loss": 2.0781407356262207,
+      "step": 460
+    },
+    {
+      "epoch": 0.1933155499764632,
+      "grad_norm": 0.17853944003582,
+      "learning_rate": 0.001,
+      "loss": 2.0326449871063232,
+      "step": 462
+    },
+    {
+      "epoch": 0.19415241382917517,
+      "grad_norm": 0.17257000505924225,
+      "learning_rate": 0.001,
+      "loss": 2.1686558723449707,
+      "step": 464
+    },
+    {
+      "epoch": 0.19498927768188712,
+      "grad_norm": 0.16631971299648285,
+      "learning_rate": 0.001,
+      "loss": 2.137887716293335,
+      "step": 466
+    },
+    {
+      "epoch": 0.1958261415345991,
+      "grad_norm": 0.15770383179187775,
+      "learning_rate": 0.001,
+      "loss": 2.2857449054718018,
+      "step": 468
+    },
+    {
+      "epoch": 0.19666300538731105,
+      "grad_norm": 0.15963241457939148,
+      "learning_rate": 0.001,
+      "loss": 2.2357115745544434,
+      "step": 470
+    },
+    {
+      "epoch": 0.19749986924002302,
+      "grad_norm": 0.13476617634296417,
+      "learning_rate": 0.001,
+      "loss": 2.2002687454223633,
+      "step": 472
+    },
+    {
+      "epoch": 0.19833673309273497,
+      "grad_norm": 0.17565374076366425,
+      "learning_rate": 0.001,
+      "loss": 2.0896546840667725,
+      "step": 474
+    },
+    {
+      "epoch": 0.19917359694544695,
+      "grad_norm": 0.15945622324943542,
+      "learning_rate": 0.001,
+      "loss": 2.210458755493164,
+      "step": 476
+    },
+    {
+      "epoch": 0.2000104607981589,
+      "grad_norm": 0.17885638773441315,
+      "learning_rate": 0.001,
+      "loss": 1.9332520961761475,
+      "step": 478
     }
   ],
   "logging_steps": 2,
+  "max_steps": 4780,
   "num_input_tokens_seen": 0,
   "num_train_epochs": 2,
+  "save_steps": 478,
   "stateful_callbacks": {
     "TrainerControl": {
       "args": {
       "attributes": {}
     }
   },
+  "total_flos": 1.0220987562223206e+17,
   "train_batch_size": 1,
   "trial_name": null,
   "trial_params": null

last-checkpoint/training_args.bin CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:a0f634d41cc4d3072ce614e087d770fe4270471a62f64f5458138c47f794068d
 size 5777

 version https://git-lfs.github.com/spec/v1
+oid sha256:72e90960c8da6cd72bbd83bc9bafc41378554c080eb9e206a6353e8db4b93f5d
 size 5777