feat: commit trained adapter, QLoRA config, and held-out eval results

The v1.0.0 artifact: the EuroLLM-9B-Teletype LoRA adapter, its exact
training configuration, and the measured held-out scores.

Training (sekft-train, on a Tesla V100-32GB):
base utter-project/EuroLLM-9B-Instruct
quant 4-bit QLoRA (nf4, bitsandbytes), fp16 compute
LoRA r=16, alpha=32, dropout=0.05, bias=none
targets q_proj, k_proj, v_proj, o_proj (attention only)
schedule 3 epochs, lr 2e-4, batch 1, grad-accum 8, max-len 4096
data tiararodney/posix-sdc v1.2.2, 787 trajectories (785 usable)
loss assistant-only mask, ChatML train=serve render contract
result assistant-only loss 2.24 (start) -> 0.55 (final), 297 steps

Held-out eval (16 scenarios, 8 text_replace + 8 permissions, no scaffold):
adapter operate 1.00 terminate 0.56 verified 0.56 clean 4/16
base ctrl operate 0.00 terminate 0.00 verified 0.44 clean 0/16

The adapter installs operation and termination outright; the verified
near-tie is the trivially-reachable permissions chmod effect (text_replace
0/8 base vs 1/8 adapter), the genuine competence ceiling.

CHANGELOG.md

@@ -21,7 +21,10 @@ adapter tests whether fine-tuning installs the ending priming could not reach.
   The render check confirmed the mask derives cleanly on EuroLLM's tokenizer.
 - Held-out generalization eval (archetypes `text_replace` + `permissions`, no
   scaffold, effect-verified), with a bare-base control run on the same harness.
-  _(rates filled into the card from the eval.)_
+  The adapter installs operation (`operate_rate` 0.00 -> 1.00) and termination
+  (`terminate_rate` 0.00 -> 0.56); clean success 0/16 -> 4/16. The base's only
+  non-zero column is `verified_rate` 0.44, entirely the trivially-reachable
+  `permissions` `chmod` effect (`text_replace` 0/8 base, 1/8 adapter).
 - A research-grade model card framing the priming-to-weights question: priming
   got EuroLLM to operate but not to leave (see the scrollback-priming experiment);
   does the adapter install the termination? Reuses the PlantUML conceptual diagrams

README.md

@@ -165,21 +165,49 @@ final filesystem state.
 
 On 16 held-out scenarios (8 per archetype):
 
-<!-- RESULTS: filled from the holdout eval after training -->
-| metric | value |
-|---|---|
-| operate_rate (reaches command-mode and drives the shell) | _pending eval_ |
-| terminate_rate (emits `exit` / `panic`) | _pending eval_ |
-| verified_rate (checker passes) | _pending eval_ |
-| clean (success or correct-panic) | _pending eval_ |
+| metric | base | + adapter |
+|---|---|---|
+| operate_rate (reaches command-mode and drives the shell) | 0.00 | **1.00** |
+| terminate_rate (emits `exit` / `panic`) | 0.00 | 0.56 |
+| verified_rate (checker passes) | 0.44 | 0.56 |
+| clean (success or correct-panic) | 0 / 16 | **4 / 16** |
 
 ![held-out outcomes](figures/outcomes.png)
 
-**Reading it.** _(to be written from the numbers.)_ The question this model exists
-to answer: priming got EuroLLM to operate but not to terminate; does
-`terminate_rate` clear the floor here, that is, did the weights install the ending
-the seed could not? The base control (bare EuroLLM-9B, no adapter, same harness) is
-run alongside as the honest contrast.
+**Reading it.** The headline is the question this model was built to answer.
+Under [scrollback priming](https://blog.tiararodney.com/projects/2026/semantic-execution-kernel/experiments/scrollback-priming/),
+EuroLLM operated the shell readily but essentially never terminated (a single
+clean `exit` in 35 runs). Fine-tuning installed it: **`terminate_rate` goes from
+~0 to 0.56** (9/16 emit a terminal). The weights gave the multilingual-prose model
+the ending its persona withheld under priming, the ending its embedding geometry
+already carried. That is "from seed to weights" landing on the distributionally
+distant subject.
+
+`operate_rate 1.0` is identical to Mistral's: dropped into two task types it never
+trained on, with no scaffold, EuroLLM drove the shell *every time*. The *operate*
+half of the mechanism is fully base-model-portable, even to a European-language
+model whose code share is a minority of a minority.
+
+Competence is partial and archetype-local, and lower than Mistral's (clean 4/16 vs
+9/16), as the higher training loss predicted (the distant model fits the
+trajectories less tightly). The texture splits by archetype: on `permissions` the
+model **achieved the effect on all 8** (every `permissions` run verified), but only
+3 closed cleanly as `success`, the rest lost to two `wrong_panic` (did the work,
+then gave up) and three `incomplete` (did the work, never emitted `exit`, ran to
+the step cap). On `text_replace` it managed 1/8. So effect-achieved (9/16 verified)
+runs ahead of clean-terminated (4/16); the gaps are termination detection and a
+genuinely harder archetype, not an inability to operate.
+
+For the honest base/adapter contrast, the bare base (EuroLLM-9B, no adapter, same
+harness, same 16 scenarios) scores **0/16 clean, `operate_rate` 0.00, `terminate_rate`
+0.00**. It never reaches clean command-mode and never terminates: it chatters prose
+and runs to the step cap on all 16. Its one non-zero column is `verified_rate` 0.44,
+and that is entirely `permissions` (7/8 verified, `text_replace` 0/8) — a one-line
+`chmod` effect that even prose-contaminated output stumbles onto. So the adapter
+installs **operation (0 -> 1.00)** and **termination (0 -> 0.56)** outright; the
+verified near-tie is the permissions effect being trivially reachable, not the base
+being competent. `text_replace` is 0/8 for base and 1/8 for the adapter — the genuine
+ceiling. The adapter is the only thing that changed.
 
 ## Use with transformers + PEFT
 

adapter_config.json

@@ -0,0 +1,45 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "utter-project/EuroLLM-9B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 32,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "peft_version": "0.19.1",
+  "qalora_group_size": 16,
+  "r": 16,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}

adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6742c4cbeae6fc799cde11be483b5aa10005e3c8a3b5d515a8d47fbd54e672fd
+size 71610528

chat_template.jinja

@@ -0,0 +1,4 @@
+{% for message in messages %}{% if message['role'] == 'assistant' %}{% set role = 'assistant' %}{% else %}{% set role = message['role'] %}{% endif %}<|im_start|>{{ role }}
+{{ message['content'] | trim }}<|im_end|>
+{% endfor %}{% if add_generation_prompt %}{{'<|im_start|>assistant
+'}}{% endif %}

figures/loss.dat

@@ -0,0 +1,297 @@
+0.01019108280254777 2.2411787509918213
+0.02038216560509554 2.148449182510376
+0.030573248407643312 1.8311183452606201
+0.04076433121019108 1.9417344331741333
+0.050955414012738856 1.928802728652954
+0.061146496815286625 2.0159952640533447
+0.07133757961783439 1.4404969215393066
+0.08152866242038216 1.5566937923431396
+0.09171974522292993 1.5528804063796997
+0.10191082802547771 1.5234365463256836
+0.11210191082802548 1.1419099569320679
+0.12229299363057325 1.2414922714233398
+0.13248407643312102 1.3000662326812744
+0.14267515923566879 1.214858055114746
+0.15286624203821655 0.8137881755828857
+0.16305732484076432 1.0487538576126099
+0.1732484076433121 0.9371962547302246
+0.18343949044585986 1.000606656074524
+0.19363057324840766 0.8865289092063904
+0.20382165605095542 1.047272801399231
+0.2140127388535032 0.8279048204421997
+0.22420382165605096 1.0746535062789917
+0.23439490445859873 0.710469126701355
+0.2445859872611465 1.1326072216033936
+0.25477707006369427 0.7304662466049194
+0.26496815286624203 0.8610861897468567
+0.2751592356687898 0.6931235194206238
+0.28535031847133757 1.078980565071106
+0.29554140127388534 0.7695854902267456
+0.3057324840764331 0.8702034950256348
+0.3159235668789809 0.9720234870910645
+0.32611464968152865 0.8494051694869995
+0.3363057324840764 0.9062499403953552
+0.3464968152866242 0.6163591146469116
+0.35668789808917195 0.7634710073471069
+0.3668789808917197 0.7481063008308411
+0.37707006369426754 0.6811981797218323
+0.3872611464968153 0.7169203758239746
+0.3974522292993631 0.6283135414123535
+0.40764331210191085 0.9036557674407959
+0.4178343949044586 0.8014513254165649
+0.4280254777070064 0.7907035946846008
+0.43821656050955415 0.5930924415588379
+0.4484076433121019 0.4370468854904175
+0.4585987261146497 0.7421823740005493
+0.46878980891719746 0.721592903137207
+0.4789808917197452 0.7689587473869324
+0.489171974522293 0.7546507716178894
+0.49936305732484076 0.6935605406761169
+0.5095541401273885 0.8196207880973816
+0.5197452229299363 0.8379813432693481
+0.5299363057324841 0.7050178647041321
+0.5401273885350318 0.6532106399536133
+0.5503184713375796 0.7750182151794434
+0.5605095541401274 0.8409197330474854
+0.5707006369426751 0.6611478328704834
+0.5808917197452229 0.6677049994468689
+0.5910828025477707 0.7615975141525269
+0.6012738853503184 0.7822245955467224
+0.6114649681528662 0.6637049913406372
+0.621656050955414 0.7149831056594849
+0.6318471337579618 0.6557817459106445
+0.6420382165605095 0.6902273297309875
+0.6522292993630573 0.6736062169075012
+0.6624203821656051 0.6211321353912354
+0.6726114649681528 0.8625361323356628
+0.6828025477707006 0.7541857957839966
+0.6929936305732484 0.693640947341919
+0.7031847133757961 0.6906211972236633
+0.7133757961783439 0.5352030992507935
+0.7235668789808917 0.7836924195289612
+0.7337579617834394 0.7370813488960266
+0.7439490445859872 0.6923583745956421
+0.7541401273885351 0.6012736558914185
+0.7643312101910829 0.5796557664871216
+0.7745222929936306 0.797625720500946
+0.7847133757961784 0.5660548806190491
+0.7949044585987262 0.6391234397888184
+0.8050955414012739 0.8060684204101562
+0.8152866242038217 0.6516364216804504
+0.8254777070063695 0.6297954320907593
+0.8356687898089172 0.6555213928222656
+0.845859872611465 0.588752269744873
+0.8560509554140128 0.6335765719413757
+0.8662420382165605 0.7243975400924683
+0.8764331210191083 0.7430989742279053
+0.8866242038216561 0.7843939661979675
+0.8968152866242038 0.5764532089233398
+0.9070063694267516 0.49850624799728394
+0.9171974522292994 0.37715521454811096
+0.9273885350318471 0.5595784783363342
+0.9375796178343949 0.6513521671295166
+0.9477707006369427 0.5372012257575989
+0.9579617834394905 0.7837607264518738
+0.9681528662420382 0.6701778173446655
+0.978343949044586 0.6118640303611755
+0.9885350318471338 0.6226388216018677
+0.9987261146496815 0.5992690920829773
+1.0 0.6507328152656555
+1.0101910828025478 0.4965246915817261
+1.0203821656050955 0.7724708318710327
+1.0305732484076433 0.43493810296058655
+1.040764331210191 0.5433262586593628
+1.0509554140127388 0.5350726246833801
+1.0611464968152866 0.8957521319389343
+1.0713375796178344 0.846440315246582
+1.0815286624203821 0.6678667068481445
+1.09171974522293 0.5684325695037842
+1.1019108280254777 0.6275244355201721
+1.1121019108280255 0.6298524141311646
+1.1222929936305732 0.5952504277229309
+1.132484076433121 0.5447441339492798
+1.1426751592356688 0.5454952120780945
+1.1528662420382165 0.5303820371627808
+1.1630573248407643 0.5837380290031433
+1.173248407643312 0.5264269709587097
+1.1834394904458598 0.6229211688041687
+1.1936305732484076 0.6836450099945068
+1.2038216560509554 0.5751281380653381
+1.2140127388535031 0.5367588400840759
+1.224203821656051 0.6677789688110352
+1.2343949044585987 0.6705923080444336
+1.2445859872611464 0.5400041341781616
+1.2547770700636942 0.6988178491592407
+1.264968152866242 0.6928831934928894
+1.2751592356687897 0.704902172088623
+1.2853503184713375 0.533305823802948
+1.2955414012738853 0.6439072489738464
+1.305732484076433 0.48900049924850464
+1.3159235668789808 0.5283946394920349
+1.3261146496815286 0.7395877838134766
+1.3363057324840764 0.7702479362487793
+1.3464968152866241 0.6036393642425537
+1.356687898089172 0.6087120771408081
+1.3668789808917197 0.701381504535675
+1.3770700636942674 0.7438935041427612
+1.3872611464968152 0.6219375729560852
+1.397452229299363 0.5910773873329163
+1.4076433121019107 0.5654371380805969
+1.4178343949044585 0.6991759538650513
+1.4280254777070063 0.5674906969070435
+1.438216560509554 0.685566246509552
+1.4484076433121018 0.584814727306366
+1.4585987261146496 0.6402420401573181
+1.4687898089171973 0.753654420375824
+1.4789808917197451 0.6644793152809143
+1.4891719745222929 0.6066821813583374
+1.4993630573248407 0.6049548983573914
+1.5095541401273884 0.5323243141174316
+1.5197452229299362 0.7194340825080872
+1.529936305732484 0.5258965492248535
+1.5401273885350317 0.6822250485420227
+1.5503184713375795 0.6156855821609497
+1.5605095541401273 0.5793153047561646
+1.570700636942675 0.6731714010238647
+1.5808917197452228 0.7284210920333862
+1.5910828025477706 0.6086175441741943
+1.6012738853503183 0.7159610390663147
+1.611464968152866 0.6527769565582275
+1.6216560509554139 0.6525672674179077
+1.6318471337579616 0.7038578987121582
+1.6420382165605094 0.7094571590423584
+1.6522292993630572 0.5708713531494141
+1.662420382165605 0.5886114835739136
+1.6726114649681527 0.7475066184997559
+1.6828025477707005 0.5133225321769714
+1.6929936305732483 0.5366004705429077
+1.703184713375796 0.5273445248603821
+1.7133757961783438 0.6866199970245361
+1.7235668789808916 0.5654996037483215
+1.7337579617834393 0.6585447788238525
+1.743949044585987 0.33593103289604187
+1.754140127388535 0.5968703031539917
+1.7643312101910829 0.5769485831260681
+1.7745222929936306 0.5651184320449829
+1.7847133757961784 0.5915299654006958
+1.7949044585987262 0.5872592926025391
+1.805095541401274 0.5789125561714172
+1.8152866242038217 0.5415861010551453
+1.8254777070063695 0.5671641826629639
+1.8356687898089172 0.7704331874847412
+1.845859872611465 0.7021884918212891
+1.8560509554140128 0.5376220345497131
+1.8662420382165605 0.5445429682731628
+1.8764331210191083 0.6531277298927307
+1.886624203821656 0.6919944286346436
+1.8968152866242038 0.617557942867279
+1.9070063694267516 0.5187242031097412
+1.9171974522292994 0.6108008027076721
+1.9273885350318471 0.5871201753616333
+1.937579617834395 0.5119588375091553
+1.9477707006369427 0.6787119507789612
+1.9579617834394905 0.49538591504096985
+1.9681528662420382 0.5220327377319336
+1.978343949044586 0.7207857966423035
+1.9885350318471338 0.7571225762367249
+1.9987261146496815 0.616678774356842
+2.0 0.4243939518928528
+2.0101910828025478 0.7173599004745483
+2.0203821656050955 0.5653108954429626
+2.0305732484076433 0.5715355277061462
+2.040764331210191 0.6556556224822998
+2.050955414012739 0.5298770070075989
+2.0611464968152866 0.5146778225898743
+2.0713375796178344 0.6721327304840088
+2.081528662420382 0.5069238543510437
+2.09171974522293 0.6033372282981873
+2.1019108280254777 0.5692410469055176
+2.1121019108280255 0.582783043384552
+2.122292993630573 0.6323152780532837
+2.132484076433121 0.6590880751609802
+2.1426751592356688 0.502295732498169
+2.1528662420382165 0.583166241645813
+2.1630573248407643 0.5921703577041626
+2.173248407643312 0.6707175970077515
+2.18343949044586 0.2803307771682739
+2.1936305732484076 0.4122824966907501
+2.2038216560509554 0.5338118672370911
+2.214012738853503 0.5353928804397583
+2.224203821656051 0.5668357610702515
+2.2343949044585987 0.6802489757537842
+2.2445859872611464 0.5237264633178711
+2.254777070063694 0.6211481690406799
+2.264968152866242 0.6177814602851868
+2.2751592356687897 0.6030687689781189
+2.2853503184713375 0.47374996542930603
+2.2955414012738853 0.5779852867126465
+2.305732484076433 0.5802882313728333
+2.315923566878981 0.4486212730407715
+2.3261146496815286 0.5290209054946899
+2.3363057324840764 0.5956228375434875
+2.346496815286624 0.5380009412765503
+2.356687898089172 0.5917195677757263
+2.3668789808917197 0.5869897603988647
+2.3770700636942674 0.481764018535614
+2.387261146496815 0.5039525628089905
+2.397452229299363 0.5492830872535706
+2.4076433121019107 0.564611554145813
+2.4178343949044585 0.5196952223777771
+2.4280254777070063 0.461820125579834
+2.438216560509554 0.578680157661438
+2.448407643312102 0.5083260536193848
+2.4585987261146496 0.5271912813186646
+2.4687898089171973 0.6259244084358215
+2.478980891719745 0.49713408946990967
+2.489171974522293 0.579800009727478
+2.4993630573248407 0.6306784152984619
+2.5095541401273884 0.8654359579086304
+2.519745222929936 0.5433270335197449
+2.529936305732484 0.530820906162262
+2.5401273885350317 0.7104746699333191
+2.5503184713375795 0.6215056777000427
+2.5605095541401273 0.506283164024353
+2.570700636942675 0.6823926568031311
+2.580891719745223 0.6189297437667847
+2.5910828025477706 0.606570839881897
+2.6012738853503183 0.5434203147888184
+2.611464968152866 0.5780953764915466
+2.621656050955414 0.5402134656906128
+2.6318471337579616 0.5620219707489014
+2.6420382165605094 0.49182114005088806
+2.652229299363057 0.6014708876609802
+2.662420382165605 0.506209135055542
+2.6726114649681527 0.5713907480239868
+2.6828025477707005 0.5579942464828491
+2.6929936305732483 0.4806699752807617
+2.703184713375796 0.5656898617744446
+2.713375796178344 0.36204785108566284
+2.7235668789808916 0.5214712619781494
+2.7337579617834393 0.5703779458999634
+2.743949044585987 0.616208016872406
+2.754140127388535 0.5295738577842712
+2.7643312101910826 0.605385959148407
+2.7745222929936304 0.5247478485107422
+2.784713375796178 0.5557824373245239
+2.794904458598726 0.6076028347015381
+2.8050955414012737 0.5531672835350037
+2.8152866242038215 0.566487729549408
+2.8254777070063692 0.5077750086784363
+2.835668789808917 0.550285816192627
+2.845859872611465 0.5844486951828003
+2.8560509554140125 0.6465249061584473
+2.8662420382165603 0.5807628631591797
+2.876433121019108 0.6856687068939209
+2.886624203821656 0.5186278820037842
+2.8968152866242036 0.5632444024085999
+2.9070063694267514 0.5741968154907227
+2.917197452229299 0.6603624224662781
+2.927388535031847 0.5237312912940979
+2.9375796178343947 0.5721698999404907
+2.9477707006369425 0.5078662633895874
+2.9579617834394902 0.575127363204956
+2.968152866242038 0.5358938574790955
+2.9783439490445858 0.4806039333343506
+2.9885350318471335 0.5208407640457153
+2.9987261146496813 0.5954314470291138
+3.0 0.5457813143730164

figures/loss.gp

@@ -1,7 +1,7 @@
 # training loss curve — regenerate: gnuplot figures/loss.gp  (from repo root)
 set terminal pngcairo size 760,420 font "monospace,11" background rgb "white"
 set output "figures/loss.png"
-set title "training loss  —  Mistral-7B LoRA r=16, 787 trajectories, 3 epochs (fp16)"
+set title "training loss  —  EuroLLM-9B LoRA r=16, 787 trajectories, 3 epochs (4-bit QLoRA)"
 set xlabel "epoch"
 set ylabel "loss (assistant-only)"
 set grid ytics lc rgb "#dddddd"

figures/outcomes.dat

@@ -0,0 +1,2 @@
+permissions 3 3 2 0
+text_replace 1 4 0 3

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e696705b9e4772cf7bff82cb0b11a2d22df396206828761a00b154cfeeb4647f
+size 15783257

tokenizer_config.json

@@ -0,0 +1,17 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|im_end|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "pad_token": "</s>",
+  "sp_model_kwargs": {},
+  "spaces_between_special_tokens": false,
+  "tokenizer_class": "LlamaTokenizer",
+  "unk_token": "<unk>",
+  "use_default_system_prompt": false,
+  "use_fast": true
+}

training/log_history.jsonl

@@ -0,0 +1,298 @@
+{"loss": 2.2411787509918213, "grad_norm": 3.9497039318084717, "learning_rate": 0.0, "epoch": 0.01019108280254777, "step": 1}
+{"loss": 2.148449182510376, "grad_norm": 3.642446517944336, "learning_rate": 2.2222222222222223e-05, "epoch": 0.02038216560509554, "step": 2}
+{"loss": 1.8311183452606201, "grad_norm": 2.624718189239502, "learning_rate": 4.4444444444444447e-05, "epoch": 0.030573248407643312, "step": 3}
+{"loss": 1.9417344331741333, "grad_norm": 2.65944766998291, "learning_rate": 6.666666666666667e-05, "epoch": 0.04076433121019108, "step": 4}
+{"loss": 1.928802728652954, "grad_norm": 1.7967363595962524, "learning_rate": 8.888888888888889e-05, "epoch": 0.050955414012738856, "step": 5}
+{"loss": 2.0159952640533447, "grad_norm": 1.9132442474365234, "learning_rate": 0.00011111111111111112, "epoch": 0.061146496815286625, "step": 6}
+{"loss": 1.4404969215393066, "grad_norm": 1.0303291082382202, "learning_rate": 0.00013333333333333334, "epoch": 0.07133757961783439, "step": 7}
+{"loss": 1.5566937923431396, "grad_norm": 1.0498052835464478, "learning_rate": 0.00015555555555555556, "epoch": 0.08152866242038216, "step": 8}
+{"loss": 1.5528804063796997, "grad_norm": 1.8134452104568481, "learning_rate": 0.00017777777777777779, "epoch": 0.09171974522292993, "step": 9}
+{"loss": 1.5234365463256836, "grad_norm": 1.587395191192627, "learning_rate": 0.0002, "epoch": 0.10191082802547771, "step": 10}
+{"loss": 1.1419099569320679, "grad_norm": 1.1780734062194824, "learning_rate": 0.00019930555555555557, "epoch": 0.11210191082802548, "step": 11}
+{"loss": 1.2414922714233398, "grad_norm": 1.0924431085586548, "learning_rate": 0.00019861111111111113, "epoch": 0.12229299363057325, "step": 12}
+{"loss": 1.3000662326812744, "grad_norm": 0.8168513178825378, "learning_rate": 0.0001979166666666667, "epoch": 0.13248407643312102, "step": 13}
+{"loss": 1.214858055114746, "grad_norm": 0.888142466545105, "learning_rate": 0.00019722222222222225, "epoch": 0.14267515923566879, "step": 14}
+{"loss": 0.8137881755828857, "grad_norm": 0.4951096177101135, "learning_rate": 0.00019652777777777778, "epoch": 0.15286624203821655, "step": 15}
+{"loss": 1.0487538576126099, "grad_norm": 0.7484102249145508, "learning_rate": 0.00019583333333333334, "epoch": 0.16305732484076432, "step": 16}
+{"loss": 0.9371962547302246, "grad_norm": 0.596789538860321, "learning_rate": 0.0001951388888888889, "epoch": 0.1732484076433121, "step": 17}
+{"loss": 1.000606656074524, "grad_norm": 0.702583909034729, "learning_rate": 0.00019444444444444446, "epoch": 0.18343949044585986, "step": 18}
+{"loss": 0.8865289092063904, "grad_norm": 0.5889976620674133, "learning_rate": 0.00019375000000000002, "epoch": 0.19363057324840766, "step": 19}
+{"loss": 1.047272801399231, "grad_norm": 0.7196131944656372, "learning_rate": 0.00019305555555555558, "epoch": 0.20382165605095542, "step": 20}
+{"loss": 0.8279048204421997, "grad_norm": 0.4392149746417999, "learning_rate": 0.0001923611111111111, "epoch": 0.2140127388535032, "step": 21}
+{"loss": 1.0746535062789917, "grad_norm": 0.7058436274528503, "learning_rate": 0.00019166666666666667, "epoch": 0.22420382165605096, "step": 22}
+{"loss": 0.710469126701355, "grad_norm": 0.4716207683086395, "learning_rate": 0.00019097222222222223, "epoch": 0.23439490445859873, "step": 23}
+{"loss": 1.1326072216033936, "grad_norm": 0.6128161549568176, "learning_rate": 0.00019027777777777776, "epoch": 0.2445859872611465, "step": 24}
+{"loss": 0.7304662466049194, "grad_norm": 0.42321324348449707, "learning_rate": 0.00018958333333333332, "epoch": 0.25477707006369427, "step": 25}
+{"loss": 0.8610861897468567, "grad_norm": 0.6085494756698608, "learning_rate": 0.00018888888888888888, "epoch": 0.26496815286624203, "step": 26}
+{"loss": 0.6931235194206238, "grad_norm": 0.45082953572273254, "learning_rate": 0.00018819444444444444, "epoch": 0.2751592356687898, "step": 27}
+{"loss": 1.078980565071106, "grad_norm": 0.5646045207977295, "learning_rate": 0.0001875, "epoch": 0.28535031847133757, "step": 28}
+{"loss": 0.7695854902267456, "grad_norm": 0.3679056763648987, "learning_rate": 0.00018680555555555556, "epoch": 0.29554140127388534, "step": 29}
+{"loss": 0.8702034950256348, "grad_norm": 0.5824475884437561, "learning_rate": 0.00018611111111111112, "epoch": 0.3057324840764331, "step": 30}
+{"loss": 0.9720234870910645, "grad_norm": 0.4847448468208313, "learning_rate": 0.00018541666666666668, "epoch": 0.3159235668789809, "step": 31}
+{"loss": 0.8494051694869995, "grad_norm": 0.5857653021812439, "learning_rate": 0.00018472222222222224, "epoch": 0.32611464968152865, "step": 32}
+{"loss": 0.9062499403953552, "grad_norm": 0.55555260181427, "learning_rate": 0.00018402777777777778, "epoch": 0.3363057324840764, "step": 33}
+{"loss": 0.6163591146469116, "grad_norm": 0.34449025988578796, "learning_rate": 0.00018333333333333334, "epoch": 0.3464968152866242, "step": 34}
+{"loss": 0.7634710073471069, "grad_norm": 0.4469396471977234, "learning_rate": 0.0001826388888888889, "epoch": 0.35668789808917195, "step": 35}
+{"loss": 0.7481063008308411, "grad_norm": 0.42471206188201904, "learning_rate": 0.00018194444444444445, "epoch": 0.3668789808917197, "step": 36}
+{"loss": 0.6811981797218323, "grad_norm": 0.52325040102005, "learning_rate": 0.00018125000000000001, "epoch": 0.37707006369426754, "step": 37}
+{"loss": 0.7169203758239746, "grad_norm": 0.48957857489585876, "learning_rate": 0.00018055555555555557, "epoch": 0.3872611464968153, "step": 38}
+{"loss": 0.6283135414123535, "grad_norm": 0.3589591979980469, "learning_rate": 0.00017986111111111113, "epoch": 0.3974522292993631, "step": 39}
+{"loss": 0.9036557674407959, "grad_norm": 0.4590145945549011, "learning_rate": 0.0001791666666666667, "epoch": 0.40764331210191085, "step": 40}
+{"loss": 0.8014513254165649, "grad_norm": 0.5502898097038269, "learning_rate": 0.00017847222222222225, "epoch": 0.4178343949044586, "step": 41}
+{"loss": 0.7907035946846008, "grad_norm": 0.46324431896209717, "learning_rate": 0.00017777777777777779, "epoch": 0.4280254777070064, "step": 42}
+{"loss": 0.5930924415588379, "grad_norm": 0.34481850266456604, "learning_rate": 0.00017708333333333335, "epoch": 0.43821656050955415, "step": 43}
+{"loss": 0.4370468854904175, "grad_norm": 0.29074251651763916, "learning_rate": 0.0001763888888888889, "epoch": 0.4484076433121019, "step": 44}
+{"loss": 0.7421823740005493, "grad_norm": 0.3871975243091583, "learning_rate": 0.00017569444444444444, "epoch": 0.4585987261146497, "step": 45}
+{"loss": 0.721592903137207, "grad_norm": 0.40311723947525024, "learning_rate": 0.000175, "epoch": 0.46878980891719746, "step": 46}
+{"loss": 0.7689587473869324, "grad_norm": 0.580685019493103, "learning_rate": 0.00017430555555555556, "epoch": 0.4789808917197452, "step": 47}
+{"loss": 0.7546507716178894, "grad_norm": 0.4210941791534424, "learning_rate": 0.00017361111111111112, "epoch": 0.489171974522293, "step": 48}
+{"loss": 0.6935605406761169, "grad_norm": 0.5025718808174133, "learning_rate": 0.00017291666666666668, "epoch": 0.49936305732484076, "step": 49}
+{"loss": 0.8196207880973816, "grad_norm": 0.4681866765022278, "learning_rate": 0.00017222222222222224, "epoch": 0.5095541401273885, "step": 50}
+{"loss": 0.8379813432693481, "grad_norm": 0.4388984441757202, "learning_rate": 0.00017152777777777777, "epoch": 0.5197452229299363, "step": 51}
+{"loss": 0.7050178647041321, "grad_norm": 0.38684317469596863, "learning_rate": 0.00017083333333333333, "epoch": 0.5299363057324841, "step": 52}
+{"loss": 0.6532106399536133, "grad_norm": 0.4345979690551758, "learning_rate": 0.0001701388888888889, "epoch": 0.5401273885350318, "step": 53}
+{"loss": 0.7750182151794434, "grad_norm": 0.5017895102500916, "learning_rate": 0.00016944444444444445, "epoch": 0.5503184713375796, "step": 54}
+{"loss": 0.8409197330474854, "grad_norm": 0.47977039217948914, "learning_rate": 0.00016875, "epoch": 0.5605095541401274, "step": 55}
+{"loss": 0.6611478328704834, "grad_norm": 0.5798993110656738, "learning_rate": 0.00016805555555555557, "epoch": 0.5707006369426751, "step": 56}
+{"loss": 0.6677049994468689, "grad_norm": 0.40253958106040955, "learning_rate": 0.00016736111111111113, "epoch": 0.5808917197452229, "step": 57}
+{"loss": 0.7615975141525269, "grad_norm": 0.5696727633476257, "learning_rate": 0.0001666666666666667, "epoch": 0.5910828025477707, "step": 58}
+{"loss": 0.7822245955467224, "grad_norm": 0.38659730553627014, "learning_rate": 0.00016597222222222225, "epoch": 0.6012738853503184, "step": 59}
+{"loss": 0.6637049913406372, "grad_norm": 0.4785160422325134, "learning_rate": 0.00016527777777777778, "epoch": 0.6114649681528662, "step": 60}
+{"loss": 0.7149831056594849, "grad_norm": 0.5234851241111755, "learning_rate": 0.00016458333333333334, "epoch": 0.621656050955414, "step": 61}
+{"loss": 0.6557817459106445, "grad_norm": 0.4089782238006592, "learning_rate": 0.0001638888888888889, "epoch": 0.6318471337579618, "step": 62}
+{"loss": 0.6902273297309875, "grad_norm": 0.44139203429222107, "learning_rate": 0.00016319444444444446, "epoch": 0.6420382165605095, "step": 63}
+{"loss": 0.6736062169075012, "grad_norm": 0.5074507594108582, "learning_rate": 0.00016250000000000002, "epoch": 0.6522292993630573, "step": 64}
+{"loss": 0.6211321353912354, "grad_norm": 0.37892937660217285, "learning_rate": 0.00016180555555555555, "epoch": 0.6624203821656051, "step": 65}
+{"loss": 0.8625361323356628, "grad_norm": 0.5467593669891357, "learning_rate": 0.0001611111111111111, "epoch": 0.6726114649681528, "step": 66}
+{"loss": 0.7541857957839966, "grad_norm": 0.6325114369392395, "learning_rate": 0.00016041666666666667, "epoch": 0.6828025477707006, "step": 67}
+{"loss": 0.693640947341919, "grad_norm": 0.4090609848499298, "learning_rate": 0.00015972222222222223, "epoch": 0.6929936305732484, "step": 68}
+{"loss": 0.6906211972236633, "grad_norm": 0.41512537002563477, "learning_rate": 0.00015902777777777776, "epoch": 0.7031847133757961, "step": 69}
+{"loss": 0.5352030992507935, "grad_norm": 0.255706787109375, "learning_rate": 0.00015833333333333332, "epoch": 0.7133757961783439, "step": 70}
+{"loss": 0.7836924195289612, "grad_norm": 0.45245808362960815, "learning_rate": 0.00015763888888888888, "epoch": 0.7235668789808917, "step": 71}
+{"loss": 0.7370813488960266, "grad_norm": 0.4397108852863312, "learning_rate": 0.00015694444444444444, "epoch": 0.7337579617834394, "step": 72}
+{"loss": 0.6923583745956421, "grad_norm": 0.42758238315582275, "learning_rate": 0.00015625, "epoch": 0.7439490445859872, "step": 73}
+{"loss": 0.6012736558914185, "grad_norm": 0.36373254656791687, "learning_rate": 0.00015555555555555556, "epoch": 0.7541401273885351, "step": 74}
+{"loss": 0.5796557664871216, "grad_norm": 0.3397802710533142, "learning_rate": 0.00015486111111111112, "epoch": 0.7643312101910829, "step": 75}
+{"loss": 0.797625720500946, "grad_norm": 0.6077721118927002, "learning_rate": 0.00015416666666666668, "epoch": 0.7745222929936306, "step": 76}
+{"loss": 0.5660548806190491, "grad_norm": 0.415911465883255, "learning_rate": 0.00015347222222222224, "epoch": 0.7847133757961784, "step": 77}
+{"loss": 0.6391234397888184, "grad_norm": 0.4287501573562622, "learning_rate": 0.00015277777777777777, "epoch": 0.7949044585987262, "step": 78}
+{"loss": 0.8060684204101562, "grad_norm": 0.7444916367530823, "learning_rate": 0.00015208333333333333, "epoch": 0.8050955414012739, "step": 79}
+{"loss": 0.6516364216804504, "grad_norm": 0.45743903517723083, "learning_rate": 0.0001513888888888889, "epoch": 0.8152866242038217, "step": 80}
+{"loss": 0.6297954320907593, "grad_norm": 0.5295541286468506, "learning_rate": 0.00015069444444444445, "epoch": 0.8254777070063695, "step": 81}
+{"loss": 0.6555213928222656, "grad_norm": 0.3830994963645935, "learning_rate": 0.00015000000000000001, "epoch": 0.8356687898089172, "step": 82}
+{"loss": 0.588752269744873, "grad_norm": 0.5136206150054932, "learning_rate": 0.00014930555555555557, "epoch": 0.845859872611465, "step": 83}
+{"loss": 0.6335765719413757, "grad_norm": 0.49850425124168396, "learning_rate": 0.00014861111111111113, "epoch": 0.8560509554140128, "step": 84}
+{"loss": 0.7243975400924683, "grad_norm": 0.6774988174438477, "learning_rate": 0.0001479166666666667, "epoch": 0.8662420382165605, "step": 85}
+{"loss": 0.7430989742279053, "grad_norm": 0.5316981077194214, "learning_rate": 0.00014722222222222223, "epoch": 0.8764331210191083, "step": 86}
+{"loss": 0.7843939661979675, "grad_norm": 0.8052747249603271, "learning_rate": 0.00014652777777777779, "epoch": 0.8866242038216561, "step": 87}
+{"loss": 0.5764532089233398, "grad_norm": 0.31530657410621643, "learning_rate": 0.00014583333333333335, "epoch": 0.8968152866242038, "step": 88}
+{"loss": 0.49850624799728394, "grad_norm": 0.3581012487411499, "learning_rate": 0.00014513888888888888, "epoch": 0.9070063694267516, "step": 89}
+{"loss": 0.37715521454811096, "grad_norm": 0.25406351685523987, "learning_rate": 0.00014444444444444444, "epoch": 0.9171974522292994, "step": 90}
+{"loss": 0.5595784783363342, "grad_norm": 0.33514243364334106, "learning_rate": 0.00014375, "epoch": 0.9273885350318471, "step": 91}
+{"loss": 0.6513521671295166, "grad_norm": 0.42237982153892517, "learning_rate": 0.00014305555555555556, "epoch": 0.9375796178343949, "step": 92}
+{"loss": 0.5372012257575989, "grad_norm": 0.3940723240375519, "learning_rate": 0.00014236111111111112, "epoch": 0.9477707006369427, "step": 93}
+{"loss": 0.7837607264518738, "grad_norm": 0.4836282730102539, "learning_rate": 0.00014166666666666668, "epoch": 0.9579617834394905, "step": 94}
+{"loss": 0.6701778173446655, "grad_norm": 0.5559809803962708, "learning_rate": 0.00014097222222222224, "epoch": 0.9681528662420382, "step": 95}
+{"loss": 0.6118640303611755, "grad_norm": 0.4939333200454712, "learning_rate": 0.00014027777777777777, "epoch": 0.978343949044586, "step": 96}
+{"loss": 0.6226388216018677, "grad_norm": 0.45932242274284363, "learning_rate": 0.00013958333333333333, "epoch": 0.9885350318471338, "step": 97}
+{"loss": 0.5992690920829773, "grad_norm": 0.3294205069541931, "learning_rate": 0.0001388888888888889, "epoch": 0.9987261146496815, "step": 98}
+{"loss": 0.6507328152656555, "grad_norm": 0.7047852873802185, "learning_rate": 0.00013819444444444445, "epoch": 1.0, "step": 99}
+{"loss": 0.4965246915817261, "grad_norm": 0.3196631968021393, "learning_rate": 0.0001375, "epoch": 1.0101910828025478, "step": 100}
+{"loss": 0.7724708318710327, "grad_norm": 0.577457070350647, "learning_rate": 0.00013680555555555557, "epoch": 1.0203821656050955, "step": 101}
+{"loss": 0.43493810296058655, "grad_norm": 0.2715108394622803, "learning_rate": 0.00013611111111111113, "epoch": 1.0305732484076433, "step": 102}
+{"loss": 0.5433262586593628, "grad_norm": 0.3147868514060974, "learning_rate": 0.0001354166666666667, "epoch": 1.040764331210191, "step": 103}
+{"loss": 0.5350726246833801, "grad_norm": 0.4341055750846863, "learning_rate": 0.00013472222222222225, "epoch": 1.0509554140127388, "step": 104}
+{"loss": 0.8957521319389343, "grad_norm": 0.5431374907493591, "learning_rate": 0.00013402777777777778, "epoch": 1.0611464968152866, "step": 105}
+{"loss": 0.846440315246582, "grad_norm": 0.5078003406524658, "learning_rate": 0.00013333333333333334, "epoch": 1.0713375796178344, "step": 106}
+{"loss": 0.6678667068481445, "grad_norm": 0.4267313778400421, "learning_rate": 0.0001326388888888889, "epoch": 1.0815286624203821, "step": 107}
+{"loss": 0.5684325695037842, "grad_norm": 0.3820900619029999, "learning_rate": 0.00013194444444444446, "epoch": 1.09171974522293, "step": 108}
+{"loss": 0.6275244355201721, "grad_norm": 0.4872572124004364, "learning_rate": 0.00013125000000000002, "epoch": 1.1019108280254777, "step": 109}
+{"loss": 0.6298524141311646, "grad_norm": 0.5322260856628418, "learning_rate": 0.00013055555555555555, "epoch": 1.1121019108280255, "step": 110}
+{"loss": 0.5952504277229309, "grad_norm": 0.42690643668174744, "learning_rate": 0.0001298611111111111, "epoch": 1.1222929936305732, "step": 111}
+{"loss": 0.5447441339492798, "grad_norm": 0.40878692269325256, "learning_rate": 0.00012916666666666667, "epoch": 1.132484076433121, "step": 112}
+{"loss": 0.5454952120780945, "grad_norm": 0.3918440341949463, "learning_rate": 0.00012847222222222223, "epoch": 1.1426751592356688, "step": 113}
+{"loss": 0.5303820371627808, "grad_norm": 0.4724244773387909, "learning_rate": 0.00012777777777777776, "epoch": 1.1528662420382165, "step": 114}
+{"loss": 0.5837380290031433, "grad_norm": 0.40587884187698364, "learning_rate": 0.00012708333333333332, "epoch": 1.1630573248407643, "step": 115}
+{"loss": 0.5264269709587097, "grad_norm": 0.38608410954475403, "learning_rate": 0.00012638888888888888, "epoch": 1.173248407643312, "step": 116}
+{"loss": 0.6229211688041687, "grad_norm": 0.44418418407440186, "learning_rate": 0.00012569444444444444, "epoch": 1.1834394904458598, "step": 117}
+{"loss": 0.6836450099945068, "grad_norm": 0.46447622776031494, "learning_rate": 0.000125, "epoch": 1.1936305732484076, "step": 118}
+{"loss": 0.5751281380653381, "grad_norm": 0.52549809217453, "learning_rate": 0.00012430555555555556, "epoch": 1.2038216560509554, "step": 119}
+{"loss": 0.5367588400840759, "grad_norm": 0.40691471099853516, "learning_rate": 0.00012361111111111112, "epoch": 1.2140127388535031, "step": 120}
+{"loss": 0.6677789688110352, "grad_norm": 0.5310897827148438, "learning_rate": 0.00012291666666666668, "epoch": 1.224203821656051, "step": 121}
+{"loss": 0.6705923080444336, "grad_norm": 0.6179095506668091, "learning_rate": 0.00012222222222222224, "epoch": 1.2343949044585987, "step": 122}
+{"loss": 0.5400041341781616, "grad_norm": 0.356869101524353, "learning_rate": 0.00012152777777777777, "epoch": 1.2445859872611464, "step": 123}
+{"loss": 0.6988178491592407, "grad_norm": 0.5287789702415466, "learning_rate": 0.00012083333333333333, "epoch": 1.2547770700636942, "step": 124}
+{"loss": 0.6928831934928894, "grad_norm": 0.7380743622779846, "learning_rate": 0.0001201388888888889, "epoch": 1.264968152866242, "step": 125}
+{"loss": 0.704902172088623, "grad_norm": 0.5253132581710815, "learning_rate": 0.00011944444444444445, "epoch": 1.2751592356687897, "step": 126}
+{"loss": 0.533305823802948, "grad_norm": 0.35883939266204834, "learning_rate": 0.00011875, "epoch": 1.2853503184713375, "step": 127}
+{"loss": 0.6439072489738464, "grad_norm": 0.4683079719543457, "learning_rate": 0.00011805555555555556, "epoch": 1.2955414012738853, "step": 128}
+{"loss": 0.48900049924850464, "grad_norm": 0.29161524772644043, "learning_rate": 0.00011736111111111112, "epoch": 1.305732484076433, "step": 129}
+{"loss": 0.5283946394920349, "grad_norm": 0.4948372542858124, "learning_rate": 0.00011666666666666668, "epoch": 1.3159235668789808, "step": 130}
+{"loss": 0.7395877838134766, "grad_norm": 0.4473186135292053, "learning_rate": 0.00011597222222222224, "epoch": 1.3261146496815286, "step": 131}
+{"loss": 0.7702479362487793, "grad_norm": 0.5558655858039856, "learning_rate": 0.00011527777777777777, "epoch": 1.3363057324840764, "step": 132}
+{"loss": 0.6036393642425537, "grad_norm": 0.46746113896369934, "learning_rate": 0.00011458333333333333, "epoch": 1.3464968152866241, "step": 133}
+{"loss": 0.6087120771408081, "grad_norm": 0.4724007248878479, "learning_rate": 0.00011388888888888889, "epoch": 1.356687898089172, "step": 134}
+{"loss": 0.701381504535675, "grad_norm": 0.5863322019577026, "learning_rate": 0.00011319444444444445, "epoch": 1.3668789808917197, "step": 135}
+{"loss": 0.7438935041427612, "grad_norm": 0.4577178657054901, "learning_rate": 0.00011250000000000001, "epoch": 1.3770700636942674, "step": 136}
+{"loss": 0.6219375729560852, "grad_norm": 0.5071024298667908, "learning_rate": 0.00011180555555555556, "epoch": 1.3872611464968152, "step": 137}
+{"loss": 0.5910773873329163, "grad_norm": 0.42557767033576965, "learning_rate": 0.00011111111111111112, "epoch": 1.397452229299363, "step": 138}
+{"loss": 0.5654371380805969, "grad_norm": 0.4741501808166504, "learning_rate": 0.00011041666666666668, "epoch": 1.4076433121019107, "step": 139}
+{"loss": 0.6991759538650513, "grad_norm": 0.5700244307518005, "learning_rate": 0.00010972222222222224, "epoch": 1.4178343949044585, "step": 140}
+{"loss": 0.5674906969070435, "grad_norm": 0.38194701075553894, "learning_rate": 0.00010902777777777777, "epoch": 1.4280254777070063, "step": 141}
+{"loss": 0.685566246509552, "grad_norm": 0.5177777409553528, "learning_rate": 0.00010833333333333333, "epoch": 1.438216560509554, "step": 142}
+{"loss": 0.584814727306366, "grad_norm": 0.36014387011528015, "learning_rate": 0.00010763888888888889, "epoch": 1.4484076433121018, "step": 143}
+{"loss": 0.6402420401573181, "grad_norm": 0.5103558301925659, "learning_rate": 0.00010694444444444445, "epoch": 1.4585987261146496, "step": 144}
+{"loss": 0.753654420375824, "grad_norm": 0.5701179504394531, "learning_rate": 0.00010625000000000001, "epoch": 1.4687898089171973, "step": 145}
+{"loss": 0.6644793152809143, "grad_norm": 0.5546273589134216, "learning_rate": 0.00010555555555555557, "epoch": 1.4789808917197451, "step": 146}
+{"loss": 0.6066821813583374, "grad_norm": 0.3894791007041931, "learning_rate": 0.00010486111111111113, "epoch": 1.4891719745222929, "step": 147}
+{"loss": 0.6049548983573914, "grad_norm": 0.3491714894771576, "learning_rate": 0.00010416666666666667, "epoch": 1.4993630573248407, "step": 148}
+{"loss": 0.5323243141174316, "grad_norm": 0.43298643827438354, "learning_rate": 0.00010347222222222223, "epoch": 1.5095541401273884, "step": 149}
+{"loss": 0.7194340825080872, "grad_norm": 0.39632564783096313, "learning_rate": 0.00010277777777777778, "epoch": 1.5197452229299362, "step": 150}
+{"loss": 0.5258965492248535, "grad_norm": 0.5028933882713318, "learning_rate": 0.00010208333333333333, "epoch": 1.529936305732484, "step": 151}
+{"loss": 0.6822250485420227, "grad_norm": 0.5882201194763184, "learning_rate": 0.00010138888888888889, "epoch": 1.5401273885350317, "step": 152}
+{"loss": 0.6156855821609497, "grad_norm": 0.48127108812332153, "learning_rate": 0.00010069444444444445, "epoch": 1.5503184713375795, "step": 153}
+{"loss": 0.5793153047561646, "grad_norm": 0.44407162070274353, "learning_rate": 0.0001, "epoch": 1.5605095541401273, "step": 154}
+{"loss": 0.6731714010238647, "grad_norm": 0.5692698955535889, "learning_rate": 9.930555555555556e-05, "epoch": 1.570700636942675, "step": 155}
+{"loss": 0.7284210920333862, "grad_norm": 0.5519310832023621, "learning_rate": 9.861111111111112e-05, "epoch": 1.5808917197452228, "step": 156}
+{"loss": 0.6086175441741943, "grad_norm": 0.4414939284324646, "learning_rate": 9.791666666666667e-05, "epoch": 1.5910828025477706, "step": 157}
+{"loss": 0.7159610390663147, "grad_norm": 0.4971486032009125, "learning_rate": 9.722222222222223e-05, "epoch": 1.6012738853503183, "step": 158}
+{"loss": 0.6527769565582275, "grad_norm": 0.5535064339637756, "learning_rate": 9.652777777777779e-05, "epoch": 1.611464968152866, "step": 159}
+{"loss": 0.6525672674179077, "grad_norm": 0.5739630460739136, "learning_rate": 9.583333333333334e-05, "epoch": 1.6216560509554139, "step": 160}
+{"loss": 0.7038578987121582, "grad_norm": 0.6008831262588501, "learning_rate": 9.513888888888888e-05, "epoch": 1.6318471337579616, "step": 161}
+{"loss": 0.7094571590423584, "grad_norm": 0.6054052114486694, "learning_rate": 9.444444444444444e-05, "epoch": 1.6420382165605094, "step": 162}
+{"loss": 0.5708713531494141, "grad_norm": 0.38468968868255615, "learning_rate": 9.375e-05, "epoch": 1.6522292993630572, "step": 163}
+{"loss": 0.5886114835739136, "grad_norm": 0.4597961902618408, "learning_rate": 9.305555555555556e-05, "epoch": 1.662420382165605, "step": 164}
+{"loss": 0.7475066184997559, "grad_norm": 0.5115509629249573, "learning_rate": 9.236111111111112e-05, "epoch": 1.6726114649681527, "step": 165}
+{"loss": 0.5133225321769714, "grad_norm": 0.48288992047309875, "learning_rate": 9.166666666666667e-05, "epoch": 1.6828025477707005, "step": 166}
+{"loss": 0.5366004705429077, "grad_norm": 0.3603127598762512, "learning_rate": 9.097222222222223e-05, "epoch": 1.6929936305732483, "step": 167}
+{"loss": 0.5273445248603821, "grad_norm": 0.41175758838653564, "learning_rate": 9.027777777777779e-05, "epoch": 1.703184713375796, "step": 168}
+{"loss": 0.6866199970245361, "grad_norm": 0.6428042054176331, "learning_rate": 8.958333333333335e-05, "epoch": 1.7133757961783438, "step": 169}
+{"loss": 0.5654996037483215, "grad_norm": 0.3874020576477051, "learning_rate": 8.888888888888889e-05, "epoch": 1.7235668789808916, "step": 170}
+{"loss": 0.6585447788238525, "grad_norm": 0.512265682220459, "learning_rate": 8.819444444444445e-05, "epoch": 1.7337579617834393, "step": 171}
+{"loss": 0.33593103289604187, "grad_norm": 0.2575998902320862, "learning_rate": 8.75e-05, "epoch": 1.743949044585987, "step": 172}
+{"loss": 0.5968703031539917, "grad_norm": 0.45513683557510376, "learning_rate": 8.680555555555556e-05, "epoch": 1.754140127388535, "step": 173}
+{"loss": 0.5769485831260681, "grad_norm": 0.5658281445503235, "learning_rate": 8.611111111111112e-05, "epoch": 1.7643312101910829, "step": 174}
+{"loss": 0.5651184320449829, "grad_norm": 0.46824735403060913, "learning_rate": 8.541666666666666e-05, "epoch": 1.7745222929936306, "step": 175}
+{"loss": 0.5915299654006958, "grad_norm": 0.44497597217559814, "learning_rate": 8.472222222222222e-05, "epoch": 1.7847133757961784, "step": 176}
+{"loss": 0.5872592926025391, "grad_norm": 0.559508740901947, "learning_rate": 8.402777777777778e-05, "epoch": 1.7949044585987262, "step": 177}
+{"loss": 0.5789125561714172, "grad_norm": 0.42908382415771484, "learning_rate": 8.333333333333334e-05, "epoch": 1.805095541401274, "step": 178}
+{"loss": 0.5415861010551453, "grad_norm": 0.40251997113227844, "learning_rate": 8.263888888888889e-05, "epoch": 1.8152866242038217, "step": 179}
+{"loss": 0.5671641826629639, "grad_norm": 0.43128517270088196, "learning_rate": 8.194444444444445e-05, "epoch": 1.8254777070063695, "step": 180}
+{"loss": 0.7704331874847412, "grad_norm": 0.5438047647476196, "learning_rate": 8.125000000000001e-05, "epoch": 1.8356687898089172, "step": 181}
+{"loss": 0.7021884918212891, "grad_norm": 0.5311266779899597, "learning_rate": 8.055555555555556e-05, "epoch": 1.845859872611465, "step": 182}
+{"loss": 0.5376220345497131, "grad_norm": 0.5482909679412842, "learning_rate": 7.986111111111112e-05, "epoch": 1.8560509554140128, "step": 183}
+{"loss": 0.5445429682731628, "grad_norm": 0.38155248761177063, "learning_rate": 7.916666666666666e-05, "epoch": 1.8662420382165605, "step": 184}
+{"loss": 0.6531277298927307, "grad_norm": 0.47961941361427307, "learning_rate": 7.847222222222222e-05, "epoch": 1.8764331210191083, "step": 185}
+{"loss": 0.6919944286346436, "grad_norm": 0.487111359834671, "learning_rate": 7.777777777777778e-05, "epoch": 1.886624203821656, "step": 186}
+{"loss": 0.617557942867279, "grad_norm": 0.5271304845809937, "learning_rate": 7.708333333333334e-05, "epoch": 1.8968152866242038, "step": 187}
+{"loss": 0.5187242031097412, "grad_norm": 0.32880914211273193, "learning_rate": 7.638888888888889e-05, "epoch": 1.9070063694267516, "step": 188}
+{"loss": 0.6108008027076721, "grad_norm": 0.4265954792499542, "learning_rate": 7.569444444444445e-05, "epoch": 1.9171974522292994, "step": 189}
+{"loss": 0.5871201753616333, "grad_norm": 0.4751453995704651, "learning_rate": 7.500000000000001e-05, "epoch": 1.9273885350318471, "step": 190}
+{"loss": 0.5119588375091553, "grad_norm": 0.4249979555606842, "learning_rate": 7.430555555555557e-05, "epoch": 1.937579617834395, "step": 191}
+{"loss": 0.6787119507789612, "grad_norm": 0.40422624349594116, "learning_rate": 7.361111111111111e-05, "epoch": 1.9477707006369427, "step": 192}
+{"loss": 0.49538591504096985, "grad_norm": 0.34903404116630554, "learning_rate": 7.291666666666667e-05, "epoch": 1.9579617834394905, "step": 193}
+{"loss": 0.5220327377319336, "grad_norm": 0.49843719601631165, "learning_rate": 7.222222222222222e-05, "epoch": 1.9681528662420382, "step": 194}
+{"loss": 0.7207857966423035, "grad_norm": 0.45566388964653015, "learning_rate": 7.152777777777778e-05, "epoch": 1.978343949044586, "step": 195}
+{"loss": 0.7571225762367249, "grad_norm": 0.5402094125747681, "learning_rate": 7.083333333333334e-05, "epoch": 1.9885350318471338, "step": 196}
+{"loss": 0.616678774356842, "grad_norm": 0.5662046074867249, "learning_rate": 7.013888888888888e-05, "epoch": 1.9987261146496815, "step": 197}
+{"loss": 0.4243939518928528, "grad_norm": 1.1585627794265747, "learning_rate": 6.944444444444444e-05, "epoch": 2.0, "step": 198}
+{"loss": 0.7173599004745483, "grad_norm": 0.5093927383422852, "learning_rate": 6.875e-05, "epoch": 2.0101910828025478, "step": 199}
+{"loss": 0.5653108954429626, "grad_norm": 0.4539341926574707, "learning_rate": 6.805555555555556e-05, "epoch": 2.0203821656050955, "step": 200}
+{"loss": 0.5715355277061462, "grad_norm": 0.3369162380695343, "learning_rate": 6.736111111111112e-05, "epoch": 2.0305732484076433, "step": 201}
+{"loss": 0.6556556224822998, "grad_norm": 0.506378710269928, "learning_rate": 6.666666666666667e-05, "epoch": 2.040764331210191, "step": 202}
+{"loss": 0.5298770070075989, "grad_norm": 0.29943448305130005, "learning_rate": 6.597222222222223e-05, "epoch": 2.050955414012739, "step": 203}
+{"loss": 0.5146778225898743, "grad_norm": 0.36660322546958923, "learning_rate": 6.527777777777778e-05, "epoch": 2.0611464968152866, "step": 204}
+{"loss": 0.6721327304840088, "grad_norm": 0.4588656723499298, "learning_rate": 6.458333333333334e-05, "epoch": 2.0713375796178344, "step": 205}
+{"loss": 0.5069238543510437, "grad_norm": 0.36296170949935913, "learning_rate": 6.388888888888888e-05, "epoch": 2.081528662420382, "step": 206}
+{"loss": 0.6033372282981873, "grad_norm": 0.4575749635696411, "learning_rate": 6.319444444444444e-05, "epoch": 2.09171974522293, "step": 207}
+{"loss": 0.5692410469055176, "grad_norm": 0.45192593336105347, "learning_rate": 6.25e-05, "epoch": 2.1019108280254777, "step": 208}
+{"loss": 0.582783043384552, "grad_norm": 0.4650779068470001, "learning_rate": 6.180555555555556e-05, "epoch": 2.1121019108280255, "step": 209}
+{"loss": 0.6323152780532837, "grad_norm": 0.4830433428287506, "learning_rate": 6.111111111111112e-05, "epoch": 2.122292993630573, "step": 210}
+{"loss": 0.6590880751609802, "grad_norm": 0.541215717792511, "learning_rate": 6.041666666666667e-05, "epoch": 2.132484076433121, "step": 211}
+{"loss": 0.502295732498169, "grad_norm": 0.42574164271354675, "learning_rate": 5.972222222222223e-05, "epoch": 2.1426751592356688, "step": 212}
+{"loss": 0.583166241645813, "grad_norm": 0.5177292227745056, "learning_rate": 5.902777777777778e-05, "epoch": 2.1528662420382165, "step": 213}
+{"loss": 0.5921703577041626, "grad_norm": 0.568841814994812, "learning_rate": 5.833333333333334e-05, "epoch": 2.1630573248407643, "step": 214}
+{"loss": 0.6707175970077515, "grad_norm": 0.46315380930900574, "learning_rate": 5.7638888888888886e-05, "epoch": 2.173248407643312, "step": 215}
+{"loss": 0.2803307771682739, "grad_norm": 0.2647932767868042, "learning_rate": 5.6944444444444445e-05, "epoch": 2.18343949044586, "step": 216}
+{"loss": 0.4122824966907501, "grad_norm": 0.347664475440979, "learning_rate": 5.6250000000000005e-05, "epoch": 2.1936305732484076, "step": 217}
+{"loss": 0.5338118672370911, "grad_norm": 0.42711061239242554, "learning_rate": 5.555555555555556e-05, "epoch": 2.2038216560509554, "step": 218}
+{"loss": 0.5353928804397583, "grad_norm": 0.6347165107727051, "learning_rate": 5.486111111111112e-05, "epoch": 2.214012738853503, "step": 219}
+{"loss": 0.5668357610702515, "grad_norm": 0.48050010204315186, "learning_rate": 5.4166666666666664e-05, "epoch": 2.224203821656051, "step": 220}
+{"loss": 0.6802489757537842, "grad_norm": 0.4256386160850525, "learning_rate": 5.3472222222222224e-05, "epoch": 2.2343949044585987, "step": 221}
+{"loss": 0.5237264633178711, "grad_norm": 0.48963746428489685, "learning_rate": 5.2777777777777784e-05, "epoch": 2.2445859872611464, "step": 222}
+{"loss": 0.6211481690406799, "grad_norm": 0.3978838324546814, "learning_rate": 5.208333333333334e-05, "epoch": 2.254777070063694, "step": 223}
+{"loss": 0.6177814602851868, "grad_norm": 0.4951920211315155, "learning_rate": 5.138888888888889e-05, "epoch": 2.264968152866242, "step": 224}
+{"loss": 0.6030687689781189, "grad_norm": 0.5822349190711975, "learning_rate": 5.069444444444444e-05, "epoch": 2.2751592356687897, "step": 225}
+{"loss": 0.47374996542930603, "grad_norm": 0.37840521335601807, "learning_rate": 5e-05, "epoch": 2.2853503184713375, "step": 226}
+{"loss": 0.5779852867126465, "grad_norm": 0.5026860237121582, "learning_rate": 4.930555555555556e-05, "epoch": 2.2955414012738853, "step": 227}
+{"loss": 0.5802882313728333, "grad_norm": 0.5821632146835327, "learning_rate": 4.8611111111111115e-05, "epoch": 2.305732484076433, "step": 228}
+{"loss": 0.4486212730407715, "grad_norm": 0.3605436682701111, "learning_rate": 4.791666666666667e-05, "epoch": 2.315923566878981, "step": 229}
+{"loss": 0.5290209054946899, "grad_norm": 0.6001298427581787, "learning_rate": 4.722222222222222e-05, "epoch": 2.3261146496815286, "step": 230}
+{"loss": 0.5956228375434875, "grad_norm": 0.4909024238586426, "learning_rate": 4.652777777777778e-05, "epoch": 2.3363057324840764, "step": 231}
+{"loss": 0.5380009412765503, "grad_norm": 0.5174505710601807, "learning_rate": 4.5833333333333334e-05, "epoch": 2.346496815286624, "step": 232}
+{"loss": 0.5917195677757263, "grad_norm": 0.5718178749084473, "learning_rate": 4.5138888888888894e-05, "epoch": 2.356687898089172, "step": 233}
+{"loss": 0.5869897603988647, "grad_norm": 0.5132097005844116, "learning_rate": 4.4444444444444447e-05, "epoch": 2.3668789808917197, "step": 234}
+{"loss": 0.481764018535614, "grad_norm": 0.45424941182136536, "learning_rate": 4.375e-05, "epoch": 2.3770700636942674, "step": 235}
+{"loss": 0.5039525628089905, "grad_norm": 0.3783617317676544, "learning_rate": 4.305555555555556e-05, "epoch": 2.387261146496815, "step": 236}
+{"loss": 0.5492830872535706, "grad_norm": 0.5013593435287476, "learning_rate": 4.236111111111111e-05, "epoch": 2.397452229299363, "step": 237}
+{"loss": 0.564611554145813, "grad_norm": 0.5584744811058044, "learning_rate": 4.166666666666667e-05, "epoch": 2.4076433121019107, "step": 238}
+{"loss": 0.5196952223777771, "grad_norm": 0.5318610072135925, "learning_rate": 4.0972222222222225e-05, "epoch": 2.4178343949044585, "step": 239}
+{"loss": 0.461820125579834, "grad_norm": 0.35487696528434753, "learning_rate": 4.027777777777778e-05, "epoch": 2.4280254777070063, "step": 240}
+{"loss": 0.578680157661438, "grad_norm": 0.6014477610588074, "learning_rate": 3.958333333333333e-05, "epoch": 2.438216560509554, "step": 241}
+{"loss": 0.5083260536193848, "grad_norm": 0.4106902480125427, "learning_rate": 3.888888888888889e-05, "epoch": 2.448407643312102, "step": 242}
+{"loss": 0.5271912813186646, "grad_norm": 0.4246547818183899, "learning_rate": 3.8194444444444444e-05, "epoch": 2.4585987261146496, "step": 243}
+{"loss": 0.6259244084358215, "grad_norm": 0.6213861703872681, "learning_rate": 3.7500000000000003e-05, "epoch": 2.4687898089171973, "step": 244}
+{"loss": 0.49713408946990967, "grad_norm": 0.6358655691146851, "learning_rate": 3.6805555555555556e-05, "epoch": 2.478980891719745, "step": 245}
+{"loss": 0.579800009727478, "grad_norm": 0.5982919931411743, "learning_rate": 3.611111111111111e-05, "epoch": 2.489171974522293, "step": 246}
+{"loss": 0.6306784152984619, "grad_norm": 0.6068189740180969, "learning_rate": 3.541666666666667e-05, "epoch": 2.4993630573248407, "step": 247}
+{"loss": 0.8654359579086304, "grad_norm": 0.6088505387306213, "learning_rate": 3.472222222222222e-05, "epoch": 2.5095541401273884, "step": 248}
+{"loss": 0.5433270335197449, "grad_norm": 0.5889372825622559, "learning_rate": 3.402777777777778e-05, "epoch": 2.519745222929936, "step": 249}
+{"loss": 0.530820906162262, "grad_norm": 0.42785683274269104, "learning_rate": 3.3333333333333335e-05, "epoch": 2.529936305732484, "step": 250}
+{"loss": 0.7104746699333191, "grad_norm": 0.7355132699012756, "learning_rate": 3.263888888888889e-05, "epoch": 2.5401273885350317, "step": 251}
+{"loss": 0.6215056777000427, "grad_norm": 0.5432270169258118, "learning_rate": 3.194444444444444e-05, "epoch": 2.5503184713375795, "step": 252}
+{"loss": 0.506283164024353, "grad_norm": 0.3910178244113922, "learning_rate": 3.125e-05, "epoch": 2.5605095541401273, "step": 253}
+{"loss": 0.6823926568031311, "grad_norm": 0.616292417049408, "learning_rate": 3.055555555555556e-05, "epoch": 2.570700636942675, "step": 254}
+{"loss": 0.6189297437667847, "grad_norm": 0.4721616208553314, "learning_rate": 2.9861111111111113e-05, "epoch": 2.580891719745223, "step": 255}
+{"loss": 0.606570839881897, "grad_norm": 0.6637372374534607, "learning_rate": 2.916666666666667e-05, "epoch": 2.5910828025477706, "step": 256}
+{"loss": 0.5434203147888184, "grad_norm": 0.5233961939811707, "learning_rate": 2.8472222222222223e-05, "epoch": 2.6012738853503183, "step": 257}
+{"loss": 0.5780953764915466, "grad_norm": 0.45685312151908875, "learning_rate": 2.777777777777778e-05, "epoch": 2.611464968152866, "step": 258}
+{"loss": 0.5402134656906128, "grad_norm": 0.4572589695453644, "learning_rate": 2.7083333333333332e-05, "epoch": 2.621656050955414, "step": 259}
+{"loss": 0.5620219707489014, "grad_norm": 0.47849151492118835, "learning_rate": 2.6388888888888892e-05, "epoch": 2.6318471337579616, "step": 260}
+{"loss": 0.49182114005088806, "grad_norm": 0.44858163595199585, "learning_rate": 2.5694444444444445e-05, "epoch": 2.6420382165605094, "step": 261}
+{"loss": 0.6014708876609802, "grad_norm": 0.5830488204956055, "learning_rate": 2.5e-05, "epoch": 2.652229299363057, "step": 262}
+{"loss": 0.506209135055542, "grad_norm": 0.4333699345588684, "learning_rate": 2.4305555555555558e-05, "epoch": 2.662420382165605, "step": 263}
+{"loss": 0.5713907480239868, "grad_norm": 0.617663562297821, "learning_rate": 2.361111111111111e-05, "epoch": 2.6726114649681527, "step": 264}
+{"loss": 0.5579942464828491, "grad_norm": 0.4907968044281006, "learning_rate": 2.2916666666666667e-05, "epoch": 2.6828025477707005, "step": 265}
+{"loss": 0.4806699752807617, "grad_norm": 0.3987278342247009, "learning_rate": 2.2222222222222223e-05, "epoch": 2.6929936305732483, "step": 266}
+{"loss": 0.5656898617744446, "grad_norm": 0.4368838667869568, "learning_rate": 2.152777777777778e-05, "epoch": 2.703184713375796, "step": 267}
+{"loss": 0.36204785108566284, "grad_norm": 0.3043971657752991, "learning_rate": 2.0833333333333336e-05, "epoch": 2.713375796178344, "step": 268}
+{"loss": 0.5214712619781494, "grad_norm": 0.4032415747642517, "learning_rate": 2.013888888888889e-05, "epoch": 2.7235668789808916, "step": 269}
+{"loss": 0.5703779458999634, "grad_norm": 0.5316827893257141, "learning_rate": 1.9444444444444445e-05, "epoch": 2.7337579617834393, "step": 270}
+{"loss": 0.616208016872406, "grad_norm": 0.7782114744186401, "learning_rate": 1.8750000000000002e-05, "epoch": 2.743949044585987, "step": 271}
+{"loss": 0.5295738577842712, "grad_norm": 0.4829880893230438, "learning_rate": 1.8055555555555555e-05, "epoch": 2.754140127388535, "step": 272}
+{"loss": 0.605385959148407, "grad_norm": 0.48132264614105225, "learning_rate": 1.736111111111111e-05, "epoch": 2.7643312101910826, "step": 273}
+{"loss": 0.5247478485107422, "grad_norm": 0.42018043994903564, "learning_rate": 1.6666666666666667e-05, "epoch": 2.7745222929936304, "step": 274}
+{"loss": 0.5557824373245239, "grad_norm": 0.5186449289321899, "learning_rate": 1.597222222222222e-05, "epoch": 2.784713375796178, "step": 275}
+{"loss": 0.6076028347015381, "grad_norm": 0.49168500304222107, "learning_rate": 1.527777777777778e-05, "epoch": 2.794904458598726, "step": 276}
+{"loss": 0.5531672835350037, "grad_norm": 0.5494187474250793, "learning_rate": 1.4583333333333335e-05, "epoch": 2.8050955414012737, "step": 277}
+{"loss": 0.566487729549408, "grad_norm": 0.5015196800231934, "learning_rate": 1.388888888888889e-05, "epoch": 2.8152866242038215, "step": 278}
+{"loss": 0.5077750086784363, "grad_norm": 0.507102370262146, "learning_rate": 1.3194444444444446e-05, "epoch": 2.8254777070063692, "step": 279}
+{"loss": 0.550285816192627, "grad_norm": 0.609117329120636, "learning_rate": 1.25e-05, "epoch": 2.835668789808917, "step": 280}
+{"loss": 0.5844486951828003, "grad_norm": 0.5720102787017822, "learning_rate": 1.1805555555555555e-05, "epoch": 2.845859872611465, "step": 281}
+{"loss": 0.6465249061584473, "grad_norm": 0.5525511503219604, "learning_rate": 1.1111111111111112e-05, "epoch": 2.8560509554140125, "step": 282}
+{"loss": 0.5807628631591797, "grad_norm": 0.6111302971839905, "learning_rate": 1.0416666666666668e-05, "epoch": 2.8662420382165603, "step": 283}
+{"loss": 0.6856687068939209, "grad_norm": 0.6745213866233826, "learning_rate": 9.722222222222223e-06, "epoch": 2.876433121019108, "step": 284}
+{"loss": 0.5186278820037842, "grad_norm": 0.37458473443984985, "learning_rate": 9.027777777777777e-06, "epoch": 2.886624203821656, "step": 285}
+{"loss": 0.5632444024085999, "grad_norm": 0.4319250285625458, "learning_rate": 8.333333333333334e-06, "epoch": 2.8968152866242036, "step": 286}
+{"loss": 0.5741968154907227, "grad_norm": 0.5636504888534546, "learning_rate": 7.63888888888889e-06, "epoch": 2.9070063694267514, "step": 287}
+{"loss": 0.6603624224662781, "grad_norm": 0.569484531879425, "learning_rate": 6.944444444444445e-06, "epoch": 2.917197452229299, "step": 288}
+{"loss": 0.5237312912940979, "grad_norm": 0.42554548382759094, "learning_rate": 6.25e-06, "epoch": 2.927388535031847, "step": 289}
+{"loss": 0.5721698999404907, "grad_norm": 0.4364071190357208, "learning_rate": 5.555555555555556e-06, "epoch": 2.9375796178343947, "step": 290}
+{"loss": 0.5078662633895874, "grad_norm": 0.460910826921463, "learning_rate": 4.861111111111111e-06, "epoch": 2.9477707006369425, "step": 291}
+{"loss": 0.575127363204956, "grad_norm": 0.6120333671569824, "learning_rate": 4.166666666666667e-06, "epoch": 2.9579617834394902, "step": 292}
+{"loss": 0.5358938574790955, "grad_norm": 0.5368267893791199, "learning_rate": 3.4722222222222224e-06, "epoch": 2.968152866242038, "step": 293}
+{"loss": 0.4806039333343506, "grad_norm": 0.48953935503959656, "learning_rate": 2.777777777777778e-06, "epoch": 2.9783439490445858, "step": 294}
+{"loss": 0.5208407640457153, "grad_norm": 0.5378862619400024, "learning_rate": 2.0833333333333334e-06, "epoch": 2.9885350318471335, "step": 295}
+{"loss": 0.5954314470291138, "grad_norm": 0.5473747253417969, "learning_rate": 1.388888888888889e-06, "epoch": 2.9987261146496813, "step": 296}
+{"loss": 0.5457813143730164, "grad_norm": 1.227904200553894, "learning_rate": 6.944444444444445e-07, "epoch": 3.0, "step": 297}
+{"train_runtime": 2398.8125, "train_samples_per_second": 0.982, "train_steps_per_second": 0.124, "total_flos": 6.881497471401984e+16, "train_loss": 0.6815742183012593, "epoch": 3.0, "step": 297}