diff --git a/12b/V_bottom-s0/README.md b/12b/V_bottom-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0f2f243d3aceafe453065e108bf04a3768225947
--- /dev/null
+++ b/12b/V_bottom-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-12b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-12b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/12b/V_bottom-s0/adapter_config.json b/12b/V_bottom-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..ecfeb4bf57b8fc155e1829f26935483829d49c69
--- /dev/null
+++ b/12b/V_bottom-s0/adapter_config.json
@@ -0,0 +1,186 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "language_model.layers.6.self_attn.v_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "34.self_attn.q_proj",
+    "37.self_attn.v_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "45.self_attn.v_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "34.self_attn.v_proj",
+    "41.self_attn.k_proj",
+    "31.self_attn.q_proj",
+    "36.self_attn.v_proj",
+    "29.self_attn.k_proj",
+    "42.self_attn.v_proj",
+    "27.self_attn.q_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "28.self_attn.v_proj",
+    "40.self_attn.q_proj",
+    "up_proj",
+    "37.self_attn.k_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "27.self_attn.v_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "language_model.layers.13.self_attn.v_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "43.self_attn.v_proj",
+    "language_model.layers.3.self_attn.k_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "31.self_attn.v_proj",
+    "40.self_attn.k_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.19.self_attn.k_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "45.self_attn.k_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "gate_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "43.self_attn.k_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "44.self_attn.v_proj",
+    "31.self_attn.k_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "44.self_attn.k_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "32.self_attn.v_proj",
+    "41.self_attn.v_proj",
+    "down_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "29.self_attn.q_proj",
+    "42.self_attn.q_proj",
+    "o_proj",
+    "35.self_attn.q_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "47.self_attn.q_proj",
+    "27.self_attn.k_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "33.self_attn.q_proj",
+    "39.self_attn.q_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "47.self_attn.v_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "36.self_attn.q_proj",
+    "42.self_attn.k_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "45.self_attn.q_proj",
+    "29.self_attn.v_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "35.self_attn.v_proj",
+    "30.self_attn.v_proj",
+    "44.self_attn.q_proj",
+    "41.self_attn.q_proj",
+    "39.self_attn.v_proj",
+    "38.self_attn.q_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "32.self_attn.k_proj",
+    "35.self_attn.k_proj",
+    "40.self_attn.v_proj",
+    "47.self_attn.k_proj",
+    "32.self_attn.q_proj",
+    "43.self_attn.q_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "30.self_attn.k_proj",
+    "38.self_attn.k_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "28.self_attn.k_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "46.self_attn.k_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "39.self_attn.k_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "34.self_attn.k_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "language_model.layers.7.self_attn.k_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "language_model.layers.20.self_attn.k_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "36.self_attn.k_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "37.self_attn.q_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "30.self_attn.q_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "46.self_attn.q_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "33.self_attn.v_proj",
+    "38.self_attn.v_proj",
+    "28.self_attn.q_proj",
+    "46.self_attn.v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/12b/V_bottom-s0/adapter_model.safetensors b/12b/V_bottom-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..3bb5773c9881e8a448dba071382cc4c817cef010
--- /dev/null
+++ b/12b/V_bottom-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c005ba534be480cf68ed8c2dfa9c9d148f539a72ff958c308fcc7c9274771893
+size 1047628488
diff --git a/12b/V_bottom-s0/config.json b/12b/V_bottom-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..772a70449c23506a632ed0e5a0b11a2a463460ca
--- /dev/null
+++ b/12b/V_bottom-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-12b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/arms_v/TASKW-V_bottom-gemma12.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/runs/V_bottom-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_12b_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 0,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 2
+}
\ No newline at end of file
diff --git a/12b/V_bottom-s0/log.jsonl b/12b/V_bottom-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..6d3a3f7f800d011428d8af1da5b90733d78c4a84
--- /dev/null
+++ b/12b/V_bottom-s0/log.jsonl
@@ -0,0 +1,312 @@
+{"step": 1, "loss": 2.262601375579834, "grad_norm": 4.321106910705566, "lr": 1.25e-05, "epoch": 1, "t": 4.0903847217559814}
+{"step": 2, "loss": 2.1671199798583984, "grad_norm": 3.742156982421875, "lr": 1.8750000000000002e-05, "epoch": 1, "t": 4.178858280181885}
+{"step": 3, "loss": 2.0548484325408936, "grad_norm": 3.4029154777526855, "lr": 2.5e-05, "epoch": 1, "t": 3.7915890216827393}
+{"step": 4, "loss": 1.9494054317474365, "grad_norm": 2.825042247772217, "lr": 3.125e-05, "epoch": 1, "t": 3.1633121967315674}
+{"step": 5, "loss": 1.820056438446045, "grad_norm": 2.2459218502044678, "lr": 3.7500000000000003e-05, "epoch": 1, "t": 3.651564598083496}
+{"step": 6, "loss": 1.7989606857299805, "grad_norm": 1.4922831058502197, "lr": 4.375e-05, "epoch": 1, "t": 3.106041193008423}
+{"step": 7, "loss": 1.7327988147735596, "grad_norm": 1.0837712287902832, "lr": 5e-05, "epoch": 1, "t": 2.8009917736053467}
+{"step": 8, "loss": 1.5018919706344604, "grad_norm": 0.8392522931098938, "lr": 5.6250000000000005e-05, "epoch": 1, "t": 3.020111322402954}
+{"step": 9, "loss": 1.6870667934417725, "grad_norm": 0.9892560839653015, "lr": 6.25e-05, "epoch": 1, "t": 3.2018043994903564}
+{"step": 10, "loss": 1.585772156715393, "grad_norm": 1.000664472579956, "lr": 6.875e-05, "epoch": 1, "t": 2.913628101348877}
+{"step": 11, "loss": 1.6362735033035278, "grad_norm": 0.9887240529060364, "lr": 7.500000000000001e-05, "epoch": 1, "t": 3.0572726726531982}
+{"step": 12, "loss": 1.41947340965271, "grad_norm": 0.8526573777198792, "lr": 8.125000000000001e-05, "epoch": 1, "t": 3.5328314304351807}
+{"step": 13, "loss": 1.2581584453582764, "grad_norm": 0.7512729167938232, "lr": 8.75e-05, "epoch": 1, "t": 3.1377861499786377}
+{"step": 14, "loss": 1.4302926063537598, "grad_norm": 0.779813826084137, "lr": 9.375e-05, "epoch": 1, "t": 2.6500909328460693}
+{"step": 15, "loss": 1.5263147354125977, "grad_norm": 0.7511342763900757, "lr": 0.0001, "epoch": 1, "t": 2.988670825958252}
+{"step": 16, "loss": 1.3866556882858276, "grad_norm": 0.7250912189483643, "lr": 0.0001, "epoch": 1, "t": 3.1635689735412598}
+{"step": 17, "loss": 1.3464088439941406, "grad_norm": 0.7572988867759705, "lr": 9.99971838728789e-05, "epoch": 1, "t": 3.2264647483825684}
+{"step": 18, "loss": 1.369933843612671, "grad_norm": 0.7048869132995605, "lr": 9.998873580873848e-05, "epoch": 1, "t": 2.8280229568481445}
+{"step": 19, "loss": 1.3470475673675537, "grad_norm": 0.7550159096717834, "lr": 9.997465675921163e-05, "epoch": 1, "t": 2.953085422515869}
+{"step": 20, "loss": 1.3588720560073853, "grad_norm": 0.6520718932151794, "lr": 9.995494831023409e-05, "epoch": 1, "t": 2.9637160301208496}
+{"step": 21, "loss": 1.2363781929016113, "grad_norm": 0.608392596244812, "lr": 9.992961268186573e-05, "epoch": 1, "t": 3.0464730262756348}
+{"step": 22, "loss": 1.3877007961273193, "grad_norm": 0.6800710558891296, "lr": 9.989865272804063e-05, "epoch": 1, "t": 2.89674973487854}
+{"step": 23, "loss": 1.2164907455444336, "grad_norm": 0.592847466468811, "lr": 9.986207193624536e-05, "epoch": 1, "t": 3.050445318222046}
+{"step": 24, "loss": 1.3283023834228516, "grad_norm": 0.5463132858276367, "lr": 9.981987442712633e-05, "epoch": 1, "t": 2.826953411102295}
+{"step": 25, "loss": 1.1635342836380005, "grad_norm": 0.5890938639640808, "lr": 9.977206495402554e-05, "epoch": 1, "t": 2.8069756031036377}
+{"step": 26, "loss": 1.2780532836914062, "grad_norm": 0.6457180380821228, "lr": 9.971864890244513e-05, "epoch": 1, "t": 3.0348150730133057}
+{"step": 27, "loss": 1.1895959377288818, "grad_norm": 0.678081750869751, "lr": 9.965963228944078e-05, "epoch": 1, "t": 3.493077278137207}
+{"step": 28, "loss": 1.2893505096435547, "grad_norm": 0.5789566040039062, "lr": 9.959502176294383e-05, "epoch": 1, "t": 3.206479072570801}
+{"step": 29, "loss": 1.175737738609314, "grad_norm": 0.7599726915359497, "lr": 9.95248246010126e-05, "epoch": 1, "t": 2.775560140609741}
+{"step": 30, "loss": 1.3553040027618408, "grad_norm": 0.5916820168495178, "lr": 9.944904871101228e-05, "epoch": 1, "t": 3.330934762954712}
+{"step": 31, "loss": 1.339888334274292, "grad_norm": 0.6464534401893616, "lr": 9.936770262872443e-05, "epoch": 1, "t": 2.7135303020477295}
+{"step": 32, "loss": 1.3124827146530151, "grad_norm": 0.6097561717033386, "lr": 9.928079551738543e-05, "epoch": 1, "t": 3.222599744796753}
+{"step": 33, "loss": 1.298492670059204, "grad_norm": 0.627341091632843, "lr": 9.918833716665419e-05, "epoch": 1, "t": 2.9775819778442383}
+{"step": 34, "loss": 1.2930155992507935, "grad_norm": 0.6147288680076599, "lr": 9.909033799150946e-05, "epoch": 1, "t": 3.3722410202026367}
+{"step": 35, "loss": 1.3372199535369873, "grad_norm": 0.6341346502304077, "lr": 9.898680903107666e-05, "epoch": 1, "t": 2.9427356719970703}
+{"step": 36, "loss": 1.2179431915283203, "grad_norm": 0.5810100436210632, "lr": 9.887776194738432e-05, "epoch": 1, "t": 3.1459813117980957}
+{"step": 37, "loss": 1.2858481407165527, "grad_norm": 0.5698282122612, "lr": 9.876320902405042e-05, "epoch": 1, "t": 3.182222843170166}
+{"step": 38, "loss": 1.1030207872390747, "grad_norm": 0.6010777354240417, "lr": 9.864316316489873e-05, "epoch": 1, "t": 2.9186089038848877}
+{"step": 39, "loss": 1.1773898601531982, "grad_norm": 0.6125306487083435, "lr": 9.851763789250525e-05, "epoch": 1, "t": 3.0962703227996826}
+{"step": 40, "loss": 1.2243480682373047, "grad_norm": 0.6106325387954712, "lr": 9.838664734667495e-05, "epoch": 1, "t": 2.9965648651123047}
+{"step": 41, "loss": 1.241113543510437, "grad_norm": 0.556209921836853, "lr": 9.825020628284896e-05, "epoch": 1, "t": 2.77687931060791}
+{"step": 42, "loss": 1.3011119365692139, "grad_norm": 0.6555788516998291, "lr": 9.810833007044247e-05, "epoch": 1, "t": 3.0246005058288574}
+{"step": 43, "loss": 1.316701889038086, "grad_norm": 0.5527663230895996, "lr": 9.796103469111351e-05, "epoch": 1, "t": 3.6717283725738525}
+{"step": 44, "loss": 1.2274091243743896, "grad_norm": 0.590487539768219, "lr": 9.780833673696254e-05, "epoch": 1, "t": 3.0330426692962646}
+{"step": 45, "loss": 1.1903352737426758, "grad_norm": 0.5301953554153442, "lr": 9.76502534086636e-05, "epoch": 1, "t": 3.1038732528686523}
+{"step": 46, "loss": 1.2604535818099976, "grad_norm": 0.6428079605102539, "lr": 9.74868025135266e-05, "epoch": 1, "t": 2.9908602237701416}
+{"step": 47, "loss": 1.233270287513733, "grad_norm": 0.6347683668136597, "lr": 9.731800246349148e-05, "epoch": 1, "t": 3.0215275287628174}
+{"step": 48, "loss": 1.2261725664138794, "grad_norm": 0.6844096779823303, "lr": 9.714387227305422e-05, "epoch": 1, "t": 3.475930690765381}
+{"step": 49, "loss": 1.2167680263519287, "grad_norm": 0.5968977212905884, "lr": 9.696443155712486e-05, "epoch": 1, "t": 2.6926209926605225}
+{"step": 50, "loss": 1.2470042705535889, "grad_norm": 0.6235132813453674, "lr": 9.67797005288181e-05, "epoch": 1, "t": 2.796698808670044}
+{"step": 51, "loss": 1.270082950592041, "grad_norm": 0.5369529128074646, "lr": 9.65896999971763e-05, "epoch": 1, "t": 3.037039041519165}
+{"step": 52, "loss": 1.176042079925537, "grad_norm": 0.5515763163566589, "lr": 9.639445136482548e-05, "epoch": 1, "t": 3.1277577877044678}
+{"step": 53, "loss": 1.2403608560562134, "grad_norm": 0.6723449230194092, "lr": 9.619397662556435e-05, "epoch": 1, "t": 2.798184633255005}
+{"step": 54, "loss": 1.3449089527130127, "grad_norm": 0.5897758603096008, "lr": 9.598829836188694e-05, "epoch": 1, "t": 3.058833122253418}
+{"step": 55, "loss": 1.3472912311553955, "grad_norm": 0.6036477088928223, "lr": 9.577743974243874e-05, "epoch": 1, "t": 3.059793472290039}
+{"step": 56, "loss": 1.205803394317627, "grad_norm": 0.5999453067779541, "lr": 9.55614245194068e-05, "epoch": 1, "t": 3.341444253921509}
+{"step": 57, "loss": 1.2803912162780762, "grad_norm": 0.6849067211151123, "lr": 9.534027702584425e-05, "epoch": 1, "t": 3.402968406677246}
+{"step": 58, "loss": 1.1960639953613281, "grad_norm": 0.5894226431846619, "lr": 9.511402217292926e-05, "epoch": 1, "t": 3.6668052673339844}
+{"step": 59, "loss": 1.2295646667480469, "grad_norm": 0.61738520860672, "lr": 9.488268544715896e-05, "epoch": 1, "t": 3.121495008468628}
+{"step": 60, "loss": 1.18776535987854, "grad_norm": 0.5862890481948853, "lr": 9.464629290747842e-05, "epoch": 1, "t": 3.8103444576263428}
+{"step": 61, "loss": 1.34743332862854, "grad_norm": 0.548228919506073, "lr": 9.440487118234535e-05, "epoch": 1, "t": 3.465630054473877}
+{"step": 62, "loss": 1.3217475414276123, "grad_norm": 0.5924105644226074, "lr": 9.415844746673047e-05, "epoch": 1, "t": 2.7527828216552734}
+{"step": 63, "loss": 1.196764349937439, "grad_norm": 0.5375847220420837, "lr": 9.390704951905411e-05, "epoch": 1, "t": 3.2803151607513428}
+{"step": 64, "loss": 1.3148972988128662, "grad_norm": 0.590986430644989, "lr": 9.365070565805941e-05, "epoch": 1, "t": 3.2135140895843506}
+{"step": 65, "loss": 1.184212565422058, "grad_norm": 0.5995739698410034, "lr": 9.338944475962237e-05, "epoch": 1, "t": 2.9850845336914062}
+{"step": 66, "loss": 1.1817116737365723, "grad_norm": 0.6395159363746643, "lr": 9.312329625349902e-05, "epoch": 1, "t": 2.839395046234131}
+{"step": 67, "loss": 1.275105595588684, "grad_norm": 0.5798230171203613, "lr": 9.285229012001047e-05, "epoch": 1, "t": 3.1961710453033447}
+{"step": 68, "loss": 1.2389097213745117, "grad_norm": 0.5572142601013184, "lr": 9.257645688666556e-05, "epoch": 1, "t": 3.0107476711273193}
+{"step": 69, "loss": 1.1718604564666748, "grad_norm": 0.5847182869911194, "lr": 9.22958276247223e-05, "epoch": 1, "t": 2.824526309967041}
+{"step": 70, "loss": 1.2704832553863525, "grad_norm": 0.6305415630340576, "lr": 9.201043394568773e-05, "epoch": 1, "t": 2.7763094902038574}
+{"step": 71, "loss": 1.0497395992279053, "grad_norm": 0.5686884522438049, "lr": 9.172030799775699e-05, "epoch": 1, "t": 3.05053448677063}
+{"step": 72, "loss": 1.2370097637176514, "grad_norm": 0.5335482954978943, "lr": 9.142548246219212e-05, "epoch": 1, "t": 2.962338924407959}
+{"step": 73, "loss": 1.117645025253296, "grad_norm": 0.6910924911499023, "lr": 9.112599054964057e-05, "epoch": 1, "t": 2.7931530475616455}
+{"step": 74, "loss": 1.1665549278259277, "grad_norm": 0.6299133896827698, "lr": 9.082186599639428e-05, "epoch": 1, "t": 2.966653347015381}
+{"step": 75, "loss": 1.1305136680603027, "grad_norm": 0.5654453635215759, "lr": 9.051314306058933e-05, "epoch": 1, "t": 3.197213888168335}
+{"step": 76, "loss": 1.1932015419006348, "grad_norm": 0.6124429702758789, "lr": 9.019985651834703e-05, "epoch": 1, "t": 3.2439706325531006}
+{"step": 77, "loss": 1.2272367477416992, "grad_norm": 0.5745668411254883, "lr": 8.988204165985649e-05, "epoch": 1, "t": 3.235095977783203}
+{"step": 78, "loss": 1.1715152263641357, "grad_norm": 0.5699538588523865, "lr": 8.955973428539944e-05, "epoch": 1, "t": 3.0144760608673096}
+{"step": 79, "loss": 1.2684156894683838, "grad_norm": 0.5542939305305481, "lr": 8.923297070131737e-05, "epoch": 1, "t": 3.4508779048919678}
+{"step": 80, "loss": 1.2495590448379517, "grad_norm": 0.6281068325042725, "lr": 8.890178771592199e-05, "epoch": 1, "t": 2.865173578262329}
+{"step": 81, "loss": 1.2565178871154785, "grad_norm": 0.6397746205329895, "lr": 8.856622263534875e-05, "epoch": 1, "t": 2.889617681503296}
+{"step": 82, "loss": 1.3881675004959106, "grad_norm": 0.6775103211402893, "lr": 8.822631325935463e-05, "epoch": 1, "t": 2.7949724197387695}
+{"step": 83, "loss": 1.288458228111267, "grad_norm": 0.6452056169509888, "lr": 8.788209787706015e-05, "epoch": 1, "t": 2.9084620475769043}
+{"step": 84, "loss": 1.2155323028564453, "grad_norm": 0.68458092212677, "lr": 8.753361526263621e-05, "epoch": 1, "t": 2.9407711029052734}
+{"step": 85, "loss": 1.0869160890579224, "grad_norm": 0.7476386427879333, "lr": 8.718090467093654e-05, "epoch": 1, "t": 2.676293134689331}
+{"step": 86, "loss": 1.143362283706665, "grad_norm": 0.5448579788208008, "lr": 8.682400583307562e-05, "epoch": 1, "t": 3.0713372230529785}
+{"step": 87, "loss": 1.3163834810256958, "grad_norm": 0.5694847106933594, "lr": 8.646295895195333e-05, "epoch": 1, "t": 3.127225160598755}
+{"step": 88, "loss": 1.0603773593902588, "grad_norm": 0.6071080565452576, "lr": 8.609780469772623e-05, "epoch": 1, "t": 2.7289798259735107}
+{"step": 89, "loss": 1.2000758647918701, "grad_norm": 0.5982691645622253, "lr": 8.572858420322627e-05, "epoch": 1, "t": 2.9588217735290527}
+{"step": 90, "loss": 1.2668378353118896, "grad_norm": 0.545132577419281, "lr": 8.535533905932738e-05, "epoch": 1, "t": 3.0684101581573486}
+{"step": 91, "loss": 1.2654061317443848, "grad_norm": 0.6144419312477112, "lr": 8.497811131026046e-05, "epoch": 1, "t": 3.349602222442627}
+{"step": 92, "loss": 1.1988807916641235, "grad_norm": 0.547225832939148, "lr": 8.459694344887732e-05, "epoch": 1, "t": 3.3556289672851562}
+{"step": 93, "loss": 1.1566147804260254, "grad_norm": 0.6398140788078308, "lr": 8.421187841186402e-05, "epoch": 1, "t": 2.721677780151367}
+{"step": 94, "loss": 1.293168306350708, "grad_norm": 0.6214326024055481, "lr": 8.382295957490436e-05, "epoch": 1, "t": 3.053834915161133}
+{"step": 95, "loss": 1.2736681699752808, "grad_norm": 1.1022288799285889, "lr": 8.343023074779368e-05, "epoch": 1, "t": 3.348402261734009}
+{"step": 96, "loss": 1.213397741317749, "grad_norm": 0.6194725036621094, "lr": 8.303373616950408e-05, "epoch": 1, "t": 2.614722490310669}
+{"step": 97, "loss": 1.156456708908081, "grad_norm": 0.619104266166687, "lr": 8.263352050320094e-05, "epoch": 1, "t": 3.1172099113464355}
+{"step": 98, "loss": 1.16475510597229, "grad_norm": 0.6148302555084229, "lr": 8.222962883121196e-05, "epoch": 1, "t": 2.827390670776367}
+{"step": 99, "loss": 1.2601840496063232, "grad_norm": 0.6317527294158936, "lr": 8.182210664994878e-05, "epoch": 1, "t": 2.9053473472595215}
+{"step": 100, "loss": 1.2153723239898682, "grad_norm": 0.6083713173866272, "lr": 8.141099986478212e-05, "epoch": 1, "t": 2.8607513904571533}
+{"step": 101, "loss": 1.1575634479522705, "grad_norm": 0.880901575088501, "lr": 8.099635478487064e-05, "epoch": 1, "t": 3.005492687225342}
+{"step": 102, "loss": 1.15977942943573, "grad_norm": 0.5449019074440002, "lr": 8.057821811794458e-05, "epoch": 1, "t": 3.3434953689575195}
+{"step": 103, "loss": 1.3317471742630005, "grad_norm": 0.6378929018974304, "lr": 8.015663696504422e-05, "epoch": 1, "t": 2.9116969108581543}
+{"step": 104, "loss": 1.2235063314437866, "grad_norm": 0.555363118648529, "lr": 7.973165881521434e-05, "epoch": 1, "t": 3.491269111633301}
+{"step": 105, "loss": 1.1460270881652832, "grad_norm": 0.6242833137512207, "lr": 7.930333154015466e-05, "epoch": 1, "t": 3.0875916481018066}
+{"step": 106, "loss": 1.2006102800369263, "grad_norm": 0.5549394488334656, "lr": 7.88717033888274e-05, "epoch": 1, "t": 3.341688394546509}
+{"step": 107, "loss": 1.1297674179077148, "grad_norm": 0.6531938910484314, "lr": 7.843682298202235e-05, "epoch": 1, "t": 2.831692934036255}
+{"step": 108, "loss": 1.2178858518600464, "grad_norm": 0.5434705018997192, "lr": 7.799873930687978e-05, "epoch": 1, "t": 3.7227730751037598}
+{"step": 109, "loss": 1.288986086845398, "grad_norm": 0.5934914946556091, "lr": 7.755750171137246e-05, "epoch": 1, "t": 3.7522521018981934}
+{"step": 110, "loss": 1.2648894786834717, "grad_norm": 0.6413567662239075, "lr": 7.711315989874677e-05, "epoch": 1, "t": 3.1471829414367676}
+{"step": 111, "loss": 1.2437365055084229, "grad_norm": 0.6522145867347717, "lr": 7.666576392192389e-05, "epoch": 1, "t": 2.8307108879089355}
+{"step": 112, "loss": 1.1486939191818237, "grad_norm": 0.6480277180671692, "lr": 7.621536417786159e-05, "epoch": 1, "t": 2.9968698024749756}
+{"step": 113, "loss": 1.2442046403884888, "grad_norm": 0.6971429586410522, "lr": 7.576201140187727e-05, "epoch": 1, "t": 3.0079429149627686}
+{"step": 114, "loss": 1.1534336805343628, "grad_norm": 0.5909664034843445, "lr": 7.530575666193283e-05, "epoch": 1, "t": 3.0501341819763184}
+{"step": 115, "loss": 1.1915488243103027, "grad_norm": 0.633232593536377, "lr": 7.484665135288213e-05, "epoch": 1, "t": 2.769033670425415}
+{"step": 116, "loss": 1.0821030139923096, "grad_norm": 0.7203329205513, "lr": 7.438474719068173e-05, "epoch": 1, "t": 3.0029449462890625}
+{"step": 117, "loss": 1.2057216167449951, "grad_norm": 0.6815724968910217, "lr": 7.392009620656513e-05, "epoch": 1, "t": 2.8152177333831787}
+{"step": 118, "loss": 1.1797432899475098, "grad_norm": 0.6875385046005249, "lr": 7.345275074118185e-05, "epoch": 1, "t": 2.7755990028381348}
+{"step": 119, "loss": 1.1533446311950684, "grad_norm": 0.5891608595848083, "lr": 7.298276343870151e-05, "epoch": 1, "t": 3.287069320678711}
+{"step": 120, "loss": 1.071908712387085, "grad_norm": 0.5524219274520874, "lr": 7.251018724088367e-05, "epoch": 1, "t": 3.0004425048828125}
+{"step": 121, "loss": 1.1303341388702393, "grad_norm": 0.6244798302650452, "lr": 7.203507538111423e-05, "epoch": 1, "t": 2.8286290168762207}
+{"step": 122, "loss": 1.1586410999298096, "grad_norm": 0.6407503485679626, "lr": 7.155748137840892e-05, "epoch": 1, "t": 2.9526290893554688}
+{"step": 123, "loss": 1.3539068698883057, "grad_norm": 0.5737888813018799, "lr": 7.107745903138472e-05, "epoch": 1, "t": 3.1938977241516113}
+{"step": 124, "loss": 1.3081499338150024, "grad_norm": 0.5784101486206055, "lr": 7.059506241219965e-05, "epoch": 1, "t": 3.176943063735962}
+{"step": 125, "loss": 1.1669530868530273, "grad_norm": 0.5705282092094421, "lr": 7.011034586046176e-05, "epoch": 1, "t": 3.1705708503723145}
+{"step": 126, "loss": 1.1863765716552734, "grad_norm": 0.5676631331443787, "lr": 6.962336397710819e-05, "epoch": 1, "t": 3.0866763591766357}
+{"step": 127, "loss": 1.1052277088165283, "grad_norm": 0.6427485942840576, "lr": 6.91341716182545e-05, "epoch": 1, "t": 3.5048844814300537}
+{"step": 128, "loss": 1.1741467714309692, "grad_norm": 0.6330947279930115, "lr": 6.864282388901544e-05, "epoch": 1, "t": 3.076354742050171}
+{"step": 129, "loss": 1.0960209369659424, "grad_norm": 0.6368241906166077, "lr": 6.814937613729766e-05, "epoch": 1, "t": 2.942425012588501}
+{"step": 130, "loss": 1.0821714401245117, "grad_norm": 0.6528913974761963, "lr": 6.765388394756504e-05, "epoch": 1, "t": 2.7191402912139893}
+{"step": 131, "loss": 1.192399501800537, "grad_norm": 0.6298495531082153, "lr": 6.715640313457733e-05, "epoch": 1, "t": 3.0565907955169678}
+{"step": 132, "loss": 1.0720741748809814, "grad_norm": 0.5506334900856018, "lr": 6.665698973710288e-05, "epoch": 1, "t": 3.1156535148620605}
+{"step": 133, "loss": 1.1196759939193726, "grad_norm": 0.642974853515625, "lr": 6.615570001160626e-05, "epoch": 1, "t": 2.7423505783081055}
+{"step": 134, "loss": 1.2106388807296753, "grad_norm": 0.5607022047042847, "lr": 6.565259042591113e-05, "epoch": 1, "t": 3.329364061355591}
+{"step": 135, "loss": 1.1919975280761719, "grad_norm": 0.9858223795890808, "lr": 6.514771765283942e-05, "epoch": 1, "t": 3.089970588684082}
+{"step": 136, "loss": 1.1731981039047241, "grad_norm": 0.5917448997497559, "lr": 6.464113856382752e-05, "epoch": 1, "t": 2.911637783050537}
+{"step": 137, "loss": 1.1506966352462769, "grad_norm": 0.5532222986221313, "lr": 6.413291022251989e-05, "epoch": 1, "t": 2.8609025478363037}
+{"step": 138, "loss": 1.2350622415542603, "grad_norm": 0.5943459868431091, "lr": 6.362308987834115e-05, "epoch": 1, "t": 2.9749345779418945}
+{"step": 139, "loss": 1.1088007688522339, "grad_norm": 0.5870715975761414, "lr": 6.311173496004723e-05, "epoch": 1, "t": 2.9706966876983643}
+{"step": 140, "loss": 1.1821991205215454, "grad_norm": 0.6302564144134521, "lr": 6.259890306925627e-05, "epoch": 1, "t": 2.806385040283203}
+{"step": 141, "loss": 1.1470372676849365, "grad_norm": 0.5550669431686401, "lr": 6.208465197396013e-05, "epoch": 1, "t": 2.8569416999816895}
+{"step": 142, "loss": 1.1341872215270996, "grad_norm": 0.6414320468902588, "lr": 6.156903960201709e-05, "epoch": 1, "t": 3.155444383621216}
+{"step": 143, "loss": 1.1604533195495605, "grad_norm": 0.5938991904258728, "lr": 6.105212403462651e-05, "epoch": 1, "t": 3.0608088970184326}
+{"step": 144, "loss": 1.1787281036376953, "grad_norm": 0.6795621514320374, "lr": 6.0533963499786314e-05, "epoch": 1, "t": 2.4089560508728027}
+{"step": 145, "loss": 1.1644045114517212, "grad_norm": 0.563539981842041, "lr": 6.001461636573397e-05, "epoch": 1, "t": 2.9390783309936523}
+{"step": 146, "loss": 1.1728127002716064, "grad_norm": 0.5525833964347839, "lr": 5.949414113437142e-05, "epoch": 1, "t": 3.100515127182007}
+{"step": 147, "loss": 1.1451499462127686, "grad_norm": 0.5328254699707031, "lr": 5.897259643467527e-05, "epoch": 1, "t": 3.0328478813171387}
+{"step": 148, "loss": 1.0825276374816895, "grad_norm": 0.5490713119506836, "lr": 5.8450041016092464e-05, "epoch": 1, "t": 3.075953722000122}
+{"step": 149, "loss": 1.0465219020843506, "grad_norm": 0.7212356328964233, "lr": 5.792653374192245e-05, "epoch": 1, "t": 3.1931333541870117}
+{"step": 150, "loss": 1.1744071245193481, "grad_norm": 0.5588793754577637, "lr": 5.7402133582686576e-05, "epoch": 1, "t": 3.780517101287842}
+{"step": 151, "loss": 1.2614938020706177, "grad_norm": 0.6507968902587891, "lr": 5.6876899609485256e-05, "epoch": 1, "t": 3.3625833988189697}
+{"step": 152, "loss": 1.1556527614593506, "grad_norm": 0.4932103157043457, "lr": 5.6350890987343944e-05, "epoch": 1, "t": 3.547351837158203}
+{"step": 153, "loss": 1.2477785348892212, "grad_norm": 0.5489038228988647, "lr": 5.582416696854853e-05, "epoch": 1, "t": 3.252431631088257}
+{"step": 154, "loss": 1.0835554599761963, "grad_norm": 0.5881918668746948, "lr": 5.5296786885970805e-05, "epoch": 1, "t": 2.5963423252105713}
+{"step": 155, "loss": 1.148956298828125, "grad_norm": 0.5351289510726929, "lr": 5.476881014638491e-05, "epoch": 1, "t": 3.2630696296691895}
+{"step": 156, "loss": 1.2198426723480225, "grad_norm": 0.5861051082611084, "lr": 5.4240296223775465e-05, "epoch": 1, "t": 3.125530958175659}
+{"step": 157, "loss": 1.2282218933105469, "grad_norm": 0.608242928981781, "lr": 5.3711304652638126e-05, "epoch": 1, "t": 3.0210516452789307}
+{"step": 158, "loss": 1.1079331636428833, "grad_norm": 0.6681380271911621, "lr": 5.318189502127332e-05, "epoch": 1, "t": 2.873147487640381}
+{"step": 159, "loss": 1.214823603630066, "grad_norm": 0.5586912631988525, "lr": 5.265212696507387e-05, "epoch": 1, "t": 2.9733569622039795}
+{"step": 160, "loss": 1.2848204374313354, "grad_norm": 0.5761910676956177, "lr": 5.212206015980742e-05, "epoch": 1, "t": 3.1285524368286133}
+{"step": 161, "loss": 1.1382474899291992, "grad_norm": 0.600112795829773, "lr": 5.159175431489424e-05, "epoch": 1, "t": 2.7901957035064697}
+{"step": 162, "loss": 1.2085801362991333, "grad_norm": 0.5726537704467773, "lr": 5.1061269166681183e-05, "epoch": 1, "t": 3.456279754638672}
+{"step": 163, "loss": 1.0004818439483643, "grad_norm": 0.5713548064231873, "lr": 5.053066447171282e-05, "epoch": 1, "t": 3.11616849899292}
+{"step": 164, "loss": 1.1956971883773804, "grad_norm": 0.5816550850868225, "lr": 5e-05, "epoch": 1, "t": 2.889674663543701}
+{"step": 165, "loss": 1.1377452611923218, "grad_norm": 0.5768589973449707, "lr": 4.94693355282872e-05, "epoch": 1, "t": 3.074821949005127}
+{"step": 166, "loss": 1.154218077659607, "grad_norm": 0.564263105392456, "lr": 4.893873083331882e-05, "epoch": 1, "t": 3.1782240867614746}
+{"step": 167, "loss": 1.16450035572052, "grad_norm": 0.5475992560386658, "lr": 4.840824568510579e-05, "epoch": 1, "t": 3.2882606983184814}
+{"step": 168, "loss": 1.108408808708191, "grad_norm": 0.5658562183380127, "lr": 4.78779398401926e-05, "epoch": 1, "t": 2.895862579345703}
+{"step": 169, "loss": 1.2103993892669678, "grad_norm": 0.5811935663223267, "lr": 4.734787303492615e-05, "epoch": 1, "t": 3.062185525894165}
+{"step": 170, "loss": 1.2354953289031982, "grad_norm": 0.6475352048873901, "lr": 4.6818104978726685e-05, "epoch": 1, "t": 3.1329429149627686}
+{"step": 171, "loss": 1.2203470468521118, "grad_norm": 0.5723701119422913, "lr": 4.628869534736187e-05, "epoch": 1, "t": 3.3884599208831787}
+{"step": 172, "loss": 1.1049220561981201, "grad_norm": 0.6683981418609619, "lr": 4.575970377622456e-05, "epoch": 1, "t": 2.925100803375244}
+{"step": 173, "loss": 1.1134202480316162, "grad_norm": 0.5766351222991943, "lr": 4.52311898536151e-05, "epoch": 1, "t": 2.9758660793304443}
+{"step": 174, "loss": 1.151648998260498, "grad_norm": 0.569556713104248, "lr": 4.47032131140292e-05, "epoch": 1, "t": 2.924151659011841}
+{"step": 175, "loss": 1.2023348808288574, "grad_norm": 0.5935404896736145, "lr": 4.4175833031451473e-05, "epoch": 1, "t": 3.134366989135742}
+{"step": 176, "loss": 1.121922254562378, "grad_norm": 0.5166159868240356, "lr": 4.364910901265606e-05, "epoch": 1, "t": 4.2262864112854}
+{"step": 177, "loss": 1.0275864601135254, "grad_norm": 0.5652241110801697, "lr": 4.3123100390514756e-05, "epoch": 1, "t": 4.112625360488892}
+{"step": 178, "loss": 1.1473710536956787, "grad_norm": 0.5857097506523132, "lr": 4.2597866417313436e-05, "epoch": 1, "t": 3.582775831222534}
+{"step": 179, "loss": 1.1719067096710205, "grad_norm": 0.6241664290428162, "lr": 4.207346625807756e-05, "epoch": 1, "t": 3.819958209991455}
+{"step": 180, "loss": 1.1645557880401611, "grad_norm": 0.5976747870445251, "lr": 4.1549958983907555e-05, "epoch": 1, "t": 3.8587818145751953}
+{"step": 181, "loss": 1.1692571640014648, "grad_norm": 0.6483241319656372, "lr": 4.102740356532473e-05, "epoch": 1, "t": 3.7793633937835693}
+{"step": 182, "loss": 1.1669251918792725, "grad_norm": 0.721358597278595, "lr": 4.050585886562858e-05, "epoch": 1, "t": 3.937937021255493}
+{"step": 183, "loss": 1.2466169595718384, "grad_norm": 0.5831030607223511, "lr": 3.998538363426605e-05, "epoch": 1, "t": 3.9405360221862793}
+{"step": 184, "loss": 1.2401959896087646, "grad_norm": 0.5450506806373596, "lr": 3.94660365002137e-05, "epoch": 1, "t": 3.6224429607391357}
+{"step": 185, "loss": 1.209920048713684, "grad_norm": 0.5808328986167908, "lr": 3.894787596537352e-05, "epoch": 1, "t": 4.1509692668914795}
+{"step": 186, "loss": 1.222599744796753, "grad_norm": 0.5833186507225037, "lr": 3.843096039798293e-05, "epoch": 1, "t": 3.2172698974609375}
+{"step": 187, "loss": 1.262049674987793, "grad_norm": 0.5987293124198914, "lr": 3.791534802603988e-05, "epoch": 1, "t": 3.109898805618286}
+{"step": 188, "loss": 1.0296735763549805, "grad_norm": 0.6398347616195679, "lr": 3.740109693074375e-05, "epoch": 1, "t": 2.5187463760375977}
+{"step": 189, "loss": 1.2416787147521973, "grad_norm": 0.528292715549469, "lr": 3.68882650399528e-05, "epoch": 1, "t": 3.6282477378845215}
+{"step": 190, "loss": 1.237912893295288, "grad_norm": 0.6176980137825012, "lr": 3.637691012165886e-05, "epoch": 1, "t": 2.839493751525879}
+{"step": 191, "loss": 1.018993854522705, "grad_norm": 0.6205204129219055, "lr": 3.586708977748012e-05, "epoch": 1, "t": 2.6434435844421387}
+{"step": 192, "loss": 1.125335693359375, "grad_norm": 0.5940316319465637, "lr": 3.5358861436172485e-05, "epoch": 1, "t": 3.1232852935791016}
+{"step": 193, "loss": 1.1238465309143066, "grad_norm": 0.5678236484527588, "lr": 3.485228234716058e-05, "epoch": 1, "t": 3.0870141983032227}
+{"step": 194, "loss": 1.1776199340820312, "grad_norm": 0.5769335031509399, "lr": 3.434740957408889e-05, "epoch": 1, "t": 3.1048550605773926}
+{"step": 195, "loss": 1.0105640888214111, "grad_norm": 0.6299868822097778, "lr": 3.3844299988393755e-05, "epoch": 1, "t": 2.6997573375701904}
+{"step": 196, "loss": 1.2175965309143066, "grad_norm": 0.5081778168678284, "lr": 3.334301026289712e-05, "epoch": 1, "t": 3.4862453937530518}
+{"step": 197, "loss": 1.2957203388214111, "grad_norm": 0.6115744709968567, "lr": 3.284359686542269e-05, "epoch": 1, "t": 2.991338014602661}
+{"step": 198, "loss": 1.0697333812713623, "grad_norm": 0.7884970307350159, "lr": 3.234611605243496e-05, "epoch": 1, "t": 2.6753690242767334}
+{"step": 199, "loss": 1.111758828163147, "grad_norm": 0.5366118550300598, "lr": 3.1850623862702344e-05, "epoch": 1, "t": 3.0785419940948486}
+{"step": 200, "loss": 1.1966639757156372, "grad_norm": 0.5983626246452332, "lr": 3.135717611098458e-05, "epoch": 1, "t": 3.112166404724121}
+{"step": 201, "loss": 1.1379112005233765, "grad_norm": 0.7048618197441101, "lr": 3.086582838174551e-05, "epoch": 1, "t": 3.0757322311401367}
+{"step": 202, "loss": 1.1819733381271362, "grad_norm": 0.5848366618156433, "lr": 3.0376636022891812e-05, "epoch": 1, "t": 2.906740665435791}
+{"step": 203, "loss": 1.0744045972824097, "grad_norm": 0.6132264137268066, "lr": 2.9889654139538246e-05, "epoch": 1, "t": 2.871115207672119}
+{"step": 204, "loss": 1.1134111881256104, "grad_norm": 0.6606585383415222, "lr": 2.9404937587800375e-05, "epoch": 1, "t": 3.229006052017212}
+{"step": 205, "loss": 1.160618782043457, "grad_norm": 0.6119295954704285, "lr": 2.8922540968615286e-05, "epoch": 1, "t": 2.674818277359009}
+{"step": 206, "loss": 1.1156063079833984, "grad_norm": 0.5940641164779663, "lr": 2.8442518621591086e-05, "epoch": 1, "t": 3.035911798477173}
+{"step": 207, "loss": 1.1710171699523926, "grad_norm": 0.6113779544830322, "lr": 2.7964924618885778e-05, "epoch": 1, "t": 3.090020179748535}
+{"step": 208, "loss": 1.160750150680542, "grad_norm": 0.5737333297729492, "lr": 2.748981275911633e-05, "epoch": 1, "t": 3.3775105476379395}
+{"step": 209, "loss": 1.1280392408370972, "grad_norm": 0.6196173429489136, "lr": 2.701723656129851e-05, "epoch": 1, "t": 2.7091281414031982}
+{"step": 210, "loss": 1.1538275480270386, "grad_norm": 0.5874179005622864, "lr": 2.6547249258818164e-05, "epoch": 1, "t": 2.960157632827759}
+{"step": 211, "loss": 1.0798423290252686, "grad_norm": 0.6555476188659668, "lr": 2.607990379343489e-05, "epoch": 1, "t": 3.0391311645507812}
+{"step": 212, "loss": 1.0290166139602661, "grad_norm": 0.6177952885627747, "lr": 2.5615252809318284e-05, "epoch": 1, "t": 2.7866854667663574}
+{"step": 213, "loss": 1.276257038116455, "grad_norm": 0.6012812852859497, "lr": 2.5153348647117857e-05, "epoch": 1, "t": 3.388942241668701}
+{"step": 214, "loss": 1.1391267776489258, "grad_norm": 0.5409966111183167, "lr": 2.469424333806718e-05, "epoch": 1, "t": 3.1688072681427}
+{"step": 215, "loss": 1.036207675933838, "grad_norm": 0.5911509990692139, "lr": 2.4237988598122752e-05, "epoch": 1, "t": 2.798635482788086}
+{"step": 216, "loss": 1.2101538181304932, "grad_norm": 0.6286398768424988, "lr": 2.3784635822138424e-05, "epoch": 1, "t": 2.8621058464050293}
+{"step": 217, "loss": 1.1943871974945068, "grad_norm": 0.5897751450538635, "lr": 2.333423607807613e-05, "epoch": 1, "t": 3.2273385524749756}
+{"step": 218, "loss": 1.1797982454299927, "grad_norm": 0.5689539909362793, "lr": 2.288684010125325e-05, "epoch": 1, "t": 3.2338011264801025}
+{"step": 219, "loss": 1.1917088031768799, "grad_norm": 0.5907241106033325, "lr": 2.2442498288627556e-05, "epoch": 1, "t": 2.9131131172180176}
+{"step": 220, "loss": 1.0297861099243164, "grad_norm": 0.6247298717498779, "lr": 2.2001260693120233e-05, "epoch": 1, "t": 3.043391466140747}
+{"step": 221, "loss": 1.0811527967453003, "grad_norm": 0.6107983589172363, "lr": 2.156317701797766e-05, "epoch": 1, "t": 2.947765350341797}
+{"step": 222, "loss": 1.1478102207183838, "grad_norm": 1.2381247282028198, "lr": 2.1128296611172593e-05, "epoch": 1, "t": 3.040334463119507}
+{"step": 223, "loss": 1.0906646251678467, "grad_norm": 0.633804440498352, "lr": 2.0696668459845355e-05, "epoch": 1, "t": 2.8082356452941895}
+{"step": 224, "loss": 1.2812585830688477, "grad_norm": 0.5863341689109802, "lr": 2.026834118478567e-05, "epoch": 1, "t": 3.1412811279296875}
+{"step": 225, "loss": 1.1530014276504517, "grad_norm": 0.6071820259094238, "lr": 1.98433630349558e-05, "epoch": 1, "t": 3.0048422813415527}
+{"step": 226, "loss": 1.0425373315811157, "grad_norm": 0.607322633266449, "lr": 1.9421781882055444e-05, "epoch": 1, "t": 2.575441360473633}
+{"step": 227, "loss": 1.0592705011367798, "grad_norm": 0.6060351133346558, "lr": 1.9003645215129355e-05, "epoch": 1, "t": 2.9081881046295166}
+{"step": 228, "loss": 1.3345788717269897, "grad_norm": 0.6063174605369568, "lr": 1.858900013521788e-05, "epoch": 1, "t": 3.3664608001708984}
+{"step": 229, "loss": 1.1068217754364014, "grad_norm": 0.5254011154174805, "lr": 1.817789335005121e-05, "epoch": 1, "t": 3.466855525970459}
+{"step": 230, "loss": 1.2544689178466797, "grad_norm": 0.5797206163406372, "lr": 1.777037116878804e-05, "epoch": 1, "t": 3.313175678253174}
+{"step": 231, "loss": 1.2279372215270996, "grad_norm": 0.6262298226356506, "lr": 1.7366479496799077e-05, "epoch": 1, "t": 3.0753684043884277}
+{"step": 232, "loss": 1.1724395751953125, "grad_norm": 0.6728183031082153, "lr": 1.6966263830495936e-05, "epoch": 1, "t": 3.021501064300537}
+{"step": 233, "loss": 1.1790351867675781, "grad_norm": 0.5918564200401306, "lr": 1.656976925220633e-05, "epoch": 1, "t": 3.1134839057922363}
+{"step": 234, "loss": 1.1428561210632324, "grad_norm": 0.5992364287376404, "lr": 1.6177040425095662e-05, "epoch": 1, "t": 3.3391077518463135}
+{"step": 235, "loss": 1.2134824991226196, "grad_norm": 0.6032678484916687, "lr": 1.5788121588135975e-05, "epoch": 1, "t": 3.229142427444458}
+{"step": 236, "loss": 1.0925897359848022, "grad_norm": 0.5905115008354187, "lr": 1.5403056551122697e-05, "epoch": 1, "t": 3.118264675140381}
+{"step": 237, "loss": 1.1609774827957153, "grad_norm": 0.526172935962677, "lr": 1.5021888689739549e-05, "epoch": 1, "t": 3.1487526893615723}
+{"step": 238, "loss": 1.1890206336975098, "grad_norm": 0.597624659538269, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 2.9986464977264404}
+{"step": 239, "loss": 1.162978172302246, "grad_norm": 0.5727379322052002, "lr": 1.427141579677374e-05, "epoch": 1, "t": 2.8487298488616943}
+{"step": 240, "loss": 1.1339913606643677, "grad_norm": 0.559514582157135, "lr": 1.3902195302273779e-05, "epoch": 1, "t": 2.9655044078826904}
+{"step": 241, "loss": 1.1060378551483154, "grad_norm": 0.7390354871749878, "lr": 1.3537041048046695e-05, "epoch": 1, "t": 3.3367416858673096}
+{"step": 242, "loss": 1.1456468105316162, "grad_norm": 0.6592360734939575, "lr": 1.3175994166924394e-05, "epoch": 1, "t": 2.8595519065856934}
+{"step": 243, "loss": 1.0963239669799805, "grad_norm": 0.6013798713684082, "lr": 1.2819095329063469e-05, "epoch": 1, "t": 2.780035972595215}
+{"step": 244, "loss": 1.095710039138794, "grad_norm": 0.6500201225280762, "lr": 1.246638473736378e-05, "epoch": 1, "t": 2.949089527130127}
+{"step": 245, "loss": 1.1575127840042114, "grad_norm": 0.5583919882774353, "lr": 1.2117902122939861e-05, "epoch": 1, "t": 3.335636854171753}
+{"step": 246, "loss": 1.2061023712158203, "grad_norm": 0.8218300938606262, "lr": 1.1773686740645384e-05, "epoch": 1, "t": 3.421555519104004}
+{"step": 247, "loss": 1.0667352676391602, "grad_norm": 0.6150872707366943, "lr": 1.1433777364651271e-05, "epoch": 1, "t": 2.8088629245758057}
+{"step": 248, "loss": 0.9942640066146851, "grad_norm": 0.6881025433540344, "lr": 1.1098212284078036e-05, "epoch": 1, "t": 2.8369979858398438}
+{"step": 249, "loss": 1.1674474477767944, "grad_norm": 0.6190013885498047, "lr": 1.076702929868264e-05, "epoch": 1, "t": 3.0134074687957764}
+{"step": 250, "loss": 1.0568673610687256, "grad_norm": 0.5610866546630859, "lr": 1.0440265714600572e-05, "epoch": 1, "t": 3.058701276779175}
+{"step": 251, "loss": 1.1656662225723267, "grad_norm": 0.626024603843689, "lr": 1.0117958340143507e-05, "epoch": 1, "t": 3.2128448486328125}
+{"step": 252, "loss": 1.1809203624725342, "grad_norm": 0.5629134178161621, "lr": 9.800143481652979e-06, "epoch": 1, "t": 3.2645459175109863}
+{"step": 253, "loss": 1.2196834087371826, "grad_norm": 0.6032998561859131, "lr": 9.48685693941067e-06, "epoch": 1, "t": 2.9141271114349365}
+{"step": 254, "loss": 1.1928837299346924, "grad_norm": 0.6172772645950317, "lr": 9.17813400360572e-06, "epoch": 1, "t": 3.172384262084961}
+{"step": 255, "loss": 1.1708760261535645, "grad_norm": 0.8942421674728394, "lr": 8.874009450359427e-06, "epoch": 1, "t": 2.9025299549102783}
+{"step": 256, "loss": 1.0920674800872803, "grad_norm": 0.574086606502533, "lr": 8.574517537807897e-06, "epoch": 1, "t": 3.108200788497925}
+{"step": 257, "loss": 1.156648874282837, "grad_norm": 0.7499485611915588, "lr": 8.279692002243027e-06, "epoch": 1, "t": 2.6387264728546143}
+{"step": 258, "loss": 1.0950714349746704, "grad_norm": 0.5602185726165771, "lr": 7.989566054312287e-06, "epoch": 1, "t": 3.169079542160034}
+{"step": 259, "loss": 1.0397440195083618, "grad_norm": 0.6062813997268677, "lr": 7.704172375277691e-06, "epoch": 1, "t": 2.9886317253112793}
+{"step": 260, "loss": 1.1563323736190796, "grad_norm": 0.5651493072509766, "lr": 7.423543113334436e-06, "epoch": 1, "t": 2.990959644317627}
+{"step": 261, "loss": 1.1134047508239746, "grad_norm": 0.6180574297904968, "lr": 7.14770987998954e-06, "epoch": 1, "t": 2.8323421478271484}
+{"step": 262, "loss": 1.1044129133224487, "grad_norm": 0.5693318247795105, "lr": 6.876703746500984e-06, "epoch": 1, "t": 3.243398427963257}
+{"step": 263, "loss": 1.1611313819885254, "grad_norm": 0.6057265400886536, "lr": 6.610555240377652e-06, "epoch": 1, "t": 3.2190983295440674}
+{"step": 264, "loss": 1.165337085723877, "grad_norm": 1.1039265394210815, "lr": 6.349294341940593e-06, "epoch": 1, "t": 3.071765422821045}
+{"step": 265, "loss": 1.234431266784668, "grad_norm": 0.6330640316009521, "lr": 6.092950480945897e-06, "epoch": 1, "t": 3.0204012393951416}
+{"step": 266, "loss": 1.127256155014038, "grad_norm": 0.5789159536361694, "lr": 5.841552533269534e-06, "epoch": 1, "t": 3.1319258213043213}
+{"step": 267, "loss": 1.2207940816879272, "grad_norm": 0.5734661221504211, "lr": 5.595128817654638e-06, "epoch": 1, "t": 2.931506633758545}
+{"step": 268, "loss": 1.141287922859192, "grad_norm": 0.5549564957618713, "lr": 5.353707092521582e-06, "epoch": 1, "t": 3.2252449989318848}
+{"step": 269, "loss": 1.125415325164795, "grad_norm": 0.6042464971542358, "lr": 5.117314552841052e-06, "epoch": 1, "t": 2.8536031246185303}
+{"step": 270, "loss": 1.152937412261963, "grad_norm": 0.6050695776939392, "lr": 4.885977827070748e-06, "epoch": 1, "t": 2.954955816268921}
+{"step": 271, "loss": 1.1014562845230103, "grad_norm": 0.572132408618927, "lr": 4.659722974155767e-06, "epoch": 1, "t": 2.971269130706787}
+{"step": 272, "loss": 1.204144835472107, "grad_norm": 0.6065317392349243, "lr": 4.43857548059321e-06, "epoch": 1, "t": 3.043729782104492}
+{"step": 273, "loss": 1.1075233221054077, "grad_norm": 0.5877376198768616, "lr": 4.2225602575612755e-06, "epoch": 1, "t": 3.121161699295044}
+{"step": 274, "loss": 1.2027723789215088, "grad_norm": 0.6455724835395813, "lr": 4.011701638113063e-06, "epoch": 1, "t": 2.8101646900177}
+{"step": 275, "loss": 1.0392215251922607, "grad_norm": 1.4909194707870483, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 3.1290743350982666}
+{"step": 276, "loss": 1.1897180080413818, "grad_norm": 0.7497091293334961, "lr": 3.605548635174533e-06, "epoch": 1, "t": 2.9680233001708984}
+{"step": 277, "loss": 1.1153748035430908, "grad_norm": 0.565839946269989, "lr": 3.410300002823691e-06, "epoch": 1, "t": 2.9037387371063232}
+{"step": 278, "loss": 1.1494334936141968, "grad_norm": 0.5831167101860046, "lr": 3.220299471181898e-06, "epoch": 1, "t": 3.2570207118988037}
+{"step": 279, "loss": 1.222648024559021, "grad_norm": 0.5716479420661926, "lr": 3.035568442875136e-06, "epoch": 1, "t": 2.9446775913238525}
+{"step": 280, "loss": 1.2059191465377808, "grad_norm": 0.5954435467720032, "lr": 2.85612772694579e-06, "epoch": 1, "t": 3.4033713340759277}
+{"step": 281, "loss": 1.1299675703048706, "grad_norm": 0.595647931098938, "lr": 2.6819975365085237e-06, "epoch": 1, "t": 2.9016120433807373}
+{"step": 282, "loss": 1.167212963104248, "grad_norm": 0.5784906148910522, "lr": 2.5131974864734066e-06, "epoch": 1, "t": 3.159802198410034}
+{"step": 283, "loss": 1.1120587587356567, "grad_norm": 0.546314537525177, "lr": 2.349746591336405e-06, "epoch": 1, "t": 3.116912364959717}
+{"step": 284, "loss": 1.2408440113067627, "grad_norm": 0.5839245915412903, "lr": 2.191663263037458e-06, "epoch": 1, "t": 3.47688889503479}
+{"step": 285, "loss": 1.044086217880249, "grad_norm": 0.5622262358665466, "lr": 2.0389653088865036e-06, "epoch": 1, "t": 3.3095884323120117}
+{"step": 286, "loss": 1.1614607572555542, "grad_norm": 0.6686910390853882, "lr": 1.8916699295575324e-06, "epoch": 1, "t": 3.385456085205078}
+{"step": 287, "loss": 1.1427057981491089, "grad_norm": 0.6589815616607666, "lr": 1.7497937171510547e-06, "epoch": 1, "t": 3.5058515071868896}
+{"step": 288, "loss": 1.1672608852386475, "grad_norm": 0.5413684248924255, "lr": 1.6133526533250565e-06, "epoch": 1, "t": 3.5682971477508545}
+{"step": 289, "loss": 1.2121870517730713, "grad_norm": 0.579685389995575, "lr": 1.4823621074947503e-06, "epoch": 1, "t": 3.607837677001953}
+{"step": 290, "loss": 1.2169768810272217, "grad_norm": 0.7050402760505676, "lr": 1.3568368351012717e-06, "epoch": 1, "t": 3.1011247634887695}
+{"step": 291, "loss": 0.9908221960067749, "grad_norm": 0.5955489873886108, "lr": 1.236790975949592e-06, "epoch": 1, "t": 2.782078504562378}
+{"step": 292, "loss": 1.181696891784668, "grad_norm": 0.6093128323554993, "lr": 1.1222380526156928e-06, "epoch": 1, "t": 3.1663551330566406}
+{"step": 293, "loss": 1.0274240970611572, "grad_norm": 0.5747653245925903, "lr": 1.0131909689233442e-06, "epoch": 1, "t": 2.8262951374053955}
+{"step": 294, "loss": 1.1584563255310059, "grad_norm": 0.5272875428199768, "lr": 9.096620084905472e-07, "epoch": 1, "t": 3.1543571949005127}
+{"step": 295, "loss": 1.161163091659546, "grad_norm": 0.9364246129989624, "lr": 8.11662833345822e-07, "epoch": 1, "t": 3.2010724544525146}
+{"step": 296, "loss": 1.2536511421203613, "grad_norm": 0.7115570306777954, "lr": 7.192044826145771e-07, "epoch": 1, "t": 2.9636471271514893}
+{"step": 297, "loss": 1.08464515209198, "grad_norm": 0.6501760482788086, "lr": 6.322973712755697e-07, "epoch": 1, "t": 3.114466667175293}
+{"step": 298, "loss": 1.173872470855713, "grad_norm": 0.5717540979385376, "lr": 5.509512889877333e-07, "epoch": 1, "t": 3.058483839035034}
+{"step": 299, "loss": 1.142005205154419, "grad_norm": 0.6201208829879761, "lr": 4.7517539898741524e-07, "epoch": 1, "t": 2.8155181407928467}
+{"step": 300, "loss": 1.084489345550537, "grad_norm": 0.5723140239715576, "lr": 4.049782370561583e-07, "epoch": 1, "t": 2.8282744884490967}
+{"step": 301, "loss": 1.194441795349121, "grad_norm": 0.6134630441665649, "lr": 3.4036771055923066e-07, "epoch": 1, "t": 3.1520586013793945}
+{"step": 302, "loss": 1.2328448295593262, "grad_norm": 0.6585222482681274, "lr": 2.813510975548772e-07, "epoch": 1, "t": 3.0300822257995605}
+{"step": 303, "loss": 1.0778056383132935, "grad_norm": 0.6462317109107971, "lr": 2.2793504597447002e-07, "epoch": 1, "t": 2.920275926589966}
+{"step": 304, "loss": 1.1771395206451416, "grad_norm": 0.6489781141281128, "lr": 1.8012557287367392e-07, "epoch": 1, "t": 2.778437614440918}
+{"step": 305, "loss": 1.2051841020584106, "grad_norm": 0.6162313222885132, "lr": 1.379280637546443e-07, "epoch": 1, "t": 3.040475845336914}
+{"step": 306, "loss": 1.1684272289276123, "grad_norm": 0.6035457253456116, "lr": 1.0134727195937333e-07, "epoch": 1, "t": 3.256772041320801}
+{"step": 307, "loss": 1.1923434734344482, "grad_norm": 0.6025996804237366, "lr": 7.038731813426291e-08, "epoch": 1, "t": 2.972303628921509}
+{"step": 308, "loss": 1.2231829166412354, "grad_norm": 0.6525549292564392, "lr": 4.5051689765929214e-08, "epoch": 1, "t": 3.1314585208892822}
+{"step": 309, "loss": 1.0794585943222046, "grad_norm": 0.6763843297958374, "lr": 2.534324078837802e-08, "epoch": 1, "t": 3.2646920680999756}
+{"step": 310, "loss": 1.1793197393417358, "grad_norm": 0.5490835905075073, "lr": 1.1264191261528557e-08, "epoch": 1, "t": 3.2566683292388916}
+{"step": 311, "loss": 1.071898341178894, "grad_norm": 0.6597471833229065, "lr": 2.8161271211024633e-09, "epoch": 1, "t": 2.7107973098754883}
+{"step": 312, "loss": 1.12252676486969, "grad_norm": 0.5467647910118103, "lr": 0.0, "epoch": 1, "t": 3.132575273513794}
diff --git a/12b/V_top-s0/README.md b/12b/V_top-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0f2f243d3aceafe453065e108bf04a3768225947
--- /dev/null
+++ b/12b/V_top-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-12b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-12b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/12b/V_top-s0/adapter_config.json b/12b/V_top-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7b20f3e75b1db67478e488637a32d74237c5492b
--- /dev/null
+++ b/12b/V_top-s0/adapter_config.json
@@ -0,0 +1,186 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "47.self_attn.k_proj",
+    "language_model.layers.7.self_attn.k_proj",
+    "43.self_attn.q_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "27.self_attn.q_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "language_model.layers.20.self_attn.k_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "38.self_attn.k_proj",
+    "46.self_attn.q_proj",
+    "44.self_attn.k_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "34.self_attn.k_proj",
+    "41.self_attn.k_proj",
+    "39.self_attn.q_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "31.self_attn.q_proj",
+    "41.self_attn.v_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "29.self_attn.k_proj",
+    "28.self_attn.k_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "34.self_attn.q_proj",
+    "35.self_attn.q_proj",
+    "31.self_attn.k_proj",
+    "40.self_attn.k_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "36.self_attn.v_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "44.self_attn.q_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "language_model.layers.6.self_attn.v_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "35.self_attn.k_proj",
+    "32.self_attn.q_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "42.self_attn.v_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "43.self_attn.k_proj",
+    "44.self_attn.v_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "28.self_attn.q_proj",
+    "30.self_attn.k_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "38.self_attn.v_proj",
+    "39.self_attn.k_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "37.self_attn.q_proj",
+    "language_model.layers.19.self_attn.k_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "45.self_attn.k_proj",
+    "42.self_attn.q_proj",
+    "38.self_attn.q_proj",
+    "30.self_attn.v_proj",
+    "o_proj",
+    "29.self_attn.q_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "39.self_attn.v_proj",
+    "46.self_attn.v_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "43.self_attn.v_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "33.self_attn.q_proj",
+    "46.self_attn.k_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "30.self_attn.q_proj",
+    "37.self_attn.k_proj",
+    "41.self_attn.q_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "36.self_attn.q_proj",
+    "33.self_attn.v_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "37.self_attn.v_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "27.self_attn.k_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "36.self_attn.k_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "32.self_attn.k_proj",
+    "down_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "32.self_attn.v_proj",
+    "47.self_attn.v_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "40.self_attn.v_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "language_model.layers.13.self_attn.v_proj",
+    "27.self_attn.v_proj",
+    "47.self_attn.q_proj",
+    "45.self_attn.q_proj",
+    "29.self_attn.v_proj",
+    "42.self_attn.k_proj",
+    "35.self_attn.v_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "40.self_attn.q_proj",
+    "28.self_attn.v_proj",
+    "34.self_attn.v_proj",
+    "31.self_attn.v_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "gate_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "45.self_attn.v_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "up_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "language_model.layers.3.self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/12b/V_top-s0/adapter_model.safetensors b/12b/V_top-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..aa1702d879ecfd58da303020442446a991098434
--- /dev/null
+++ b/12b/V_top-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ada804409e6ea599986b3e44fb8d8acfe61c4992ff292104f8bc548ab0891add
+size 1047628488
diff --git a/12b/V_top-s0/config.json b/12b/V_top-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..d63e6d43f2b3df10d11ed7767b390675ba2ef371
--- /dev/null
+++ b/12b/V_top-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-12b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/arms_v/TASKW-V_top-gemma12.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/runs/V_top-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_12b_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 0,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 2
+}
\ No newline at end of file
diff --git a/12b/V_top-s0/log.jsonl b/12b/V_top-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..23d8ca585d37fcaf355802415795fcb8a0847ad1
--- /dev/null
+++ b/12b/V_top-s0/log.jsonl
@@ -0,0 +1,312 @@
+{"step": 1, "loss": 2.2729265689849854, "grad_norm": 4.406494617462158, "lr": 1.25e-05, "epoch": 1, "t": 3.4905970096588135}
+{"step": 2, "loss": 2.141500473022461, "grad_norm": 3.7048614025115967, "lr": 1.8750000000000002e-05, "epoch": 1, "t": 3.6400585174560547}
+{"step": 3, "loss": 2.157907485961914, "grad_norm": 3.6539409160614014, "lr": 2.5e-05, "epoch": 1, "t": 3.1290602684020996}
+{"step": 4, "loss": 1.9312562942504883, "grad_norm": 2.9819629192352295, "lr": 3.125e-05, "epoch": 1, "t": 2.7889459133148193}
+{"step": 5, "loss": 1.8254902362823486, "grad_norm": 2.225151300430298, "lr": 3.7500000000000003e-05, "epoch": 1, "t": 3.802821397781372}
+{"step": 6, "loss": 1.7398107051849365, "grad_norm": 1.4814915657043457, "lr": 4.375e-05, "epoch": 1, "t": 3.2151715755462646}
+{"step": 7, "loss": 1.723631739616394, "grad_norm": 1.0053844451904297, "lr": 5e-05, "epoch": 1, "t": 3.005499839782715}
+{"step": 8, "loss": 1.5290982723236084, "grad_norm": 0.8869999647140503, "lr": 5.6250000000000005e-05, "epoch": 1, "t": 3.2517714500427246}
+{"step": 9, "loss": 1.715596318244934, "grad_norm": 1.0236982107162476, "lr": 6.25e-05, "epoch": 1, "t": 3.0355679988861084}
+{"step": 10, "loss": 1.6154892444610596, "grad_norm": 1.0504403114318848, "lr": 6.875e-05, "epoch": 1, "t": 3.108259916305542}
+{"step": 11, "loss": 1.6134092807769775, "grad_norm": 0.990784764289856, "lr": 7.500000000000001e-05, "epoch": 1, "t": 2.8776910305023193}
+{"step": 12, "loss": 1.3646092414855957, "grad_norm": 0.8541951775550842, "lr": 8.125000000000001e-05, "epoch": 1, "t": 3.1841750144958496}
+{"step": 13, "loss": 1.2742877006530762, "grad_norm": 0.769559919834137, "lr": 8.75e-05, "epoch": 1, "t": 3.058722496032715}
+{"step": 14, "loss": 1.4351048469543457, "grad_norm": 0.7996724247932434, "lr": 9.375e-05, "epoch": 1, "t": 3.130711793899536}
+{"step": 15, "loss": 1.51334547996521, "grad_norm": 0.7810581922531128, "lr": 0.0001, "epoch": 1, "t": 3.1117939949035645}
+{"step": 16, "loss": 1.4208080768585205, "grad_norm": 0.7467895150184631, "lr": 0.0001, "epoch": 1, "t": 3.283149242401123}
+{"step": 17, "loss": 1.3618955612182617, "grad_norm": 0.6926639080047607, "lr": 9.99971838728789e-05, "epoch": 1, "t": 3.1131327152252197}
+{"step": 18, "loss": 1.3402419090270996, "grad_norm": 0.734306812286377, "lr": 9.998873580873848e-05, "epoch": 1, "t": 3.040325403213501}
+{"step": 19, "loss": 1.362644076347351, "grad_norm": 0.7452945709228516, "lr": 9.997465675921163e-05, "epoch": 1, "t": 3.3454794883728027}
+{"step": 20, "loss": 1.3237106800079346, "grad_norm": 0.622259259223938, "lr": 9.995494831023409e-05, "epoch": 1, "t": 3.157949447631836}
+{"step": 21, "loss": 1.219679832458496, "grad_norm": 0.6535102128982544, "lr": 9.992961268186573e-05, "epoch": 1, "t": 3.2318668365478516}
+{"step": 22, "loss": 1.3675858974456787, "grad_norm": 0.8337335586547852, "lr": 9.989865272804063e-05, "epoch": 1, "t": 3.0391271114349365}
+{"step": 23, "loss": 1.1954965591430664, "grad_norm": 0.6240617036819458, "lr": 9.986207193624536e-05, "epoch": 1, "t": 3.4128594398498535}
+{"step": 24, "loss": 1.3337948322296143, "grad_norm": 0.5964548587799072, "lr": 9.981987442712633e-05, "epoch": 1, "t": 3.1717514991760254}
+{"step": 25, "loss": 1.131766676902771, "grad_norm": 0.587676465511322, "lr": 9.977206495402554e-05, "epoch": 1, "t": 3.2898590564727783}
+{"step": 26, "loss": 1.3129777908325195, "grad_norm": 0.6612595319747925, "lr": 9.971864890244513e-05, "epoch": 1, "t": 3.2893996238708496}
+{"step": 27, "loss": 1.3074558973312378, "grad_norm": 0.6082188487052917, "lr": 9.965963228944078e-05, "epoch": 1, "t": 3.0682950019836426}
+{"step": 28, "loss": 1.2593557834625244, "grad_norm": 0.5356127023696899, "lr": 9.959502176294383e-05, "epoch": 1, "t": 3.553260326385498}
+{"step": 29, "loss": 1.2174677848815918, "grad_norm": 0.7074805498123169, "lr": 9.95248246010126e-05, "epoch": 1, "t": 3.072585344314575}
+{"step": 30, "loss": 1.3831980228424072, "grad_norm": 0.5847600698471069, "lr": 9.944904871101228e-05, "epoch": 1, "t": 3.1686363220214844}
+{"step": 31, "loss": 1.3148130178451538, "grad_norm": 0.649976909160614, "lr": 9.936770262872443e-05, "epoch": 1, "t": 2.7638027667999268}
+{"step": 32, "loss": 1.3472368717193604, "grad_norm": 0.6375584602355957, "lr": 9.928079551738543e-05, "epoch": 1, "t": 3.3834211826324463}
+{"step": 33, "loss": 1.2677112817764282, "grad_norm": 0.5937733054161072, "lr": 9.918833716665419e-05, "epoch": 1, "t": 3.0703229904174805}
+{"step": 34, "loss": 1.2804527282714844, "grad_norm": 0.9339198470115662, "lr": 9.909033799150946e-05, "epoch": 1, "t": 3.434422016143799}
+{"step": 35, "loss": 1.3502745628356934, "grad_norm": 0.713164746761322, "lr": 9.898680903107666e-05, "epoch": 1, "t": 3.1491732597351074}
+{"step": 36, "loss": 1.2597880363464355, "grad_norm": 0.5872796773910522, "lr": 9.887776194738432e-05, "epoch": 1, "t": 3.5551655292510986}
+{"step": 37, "loss": 1.255307912826538, "grad_norm": 0.6113024353981018, "lr": 9.876320902405042e-05, "epoch": 1, "t": 3.5403289794921875}
+{"step": 38, "loss": 1.0736026763916016, "grad_norm": 0.5962145328521729, "lr": 9.864316316489873e-05, "epoch": 1, "t": 2.9595353603363037}
+{"step": 39, "loss": 1.1489514112472534, "grad_norm": 0.6147931218147278, "lr": 9.851763789250525e-05, "epoch": 1, "t": 3.266500473022461}
+{"step": 40, "loss": 1.2019245624542236, "grad_norm": 0.64427649974823, "lr": 9.838664734667495e-05, "epoch": 1, "t": 3.3068037033081055}
+{"step": 41, "loss": 1.2238175868988037, "grad_norm": 0.5384893417358398, "lr": 9.825020628284896e-05, "epoch": 1, "t": 2.8172926902770996}
+{"step": 42, "loss": 1.3091044425964355, "grad_norm": 0.6122305989265442, "lr": 9.810833007044247e-05, "epoch": 1, "t": 3.186645269393921}
+{"step": 43, "loss": 1.323056936264038, "grad_norm": 0.6110641956329346, "lr": 9.796103469111351e-05, "epoch": 1, "t": 3.412797212600708}
+{"step": 44, "loss": 1.1668975353240967, "grad_norm": 0.573988139629364, "lr": 9.780833673696254e-05, "epoch": 1, "t": 3.426820993423462}
+{"step": 45, "loss": 1.270620346069336, "grad_norm": 0.5737598538398743, "lr": 9.76502534086636e-05, "epoch": 1, "t": 3.084648609161377}
+{"step": 46, "loss": 1.2267508506774902, "grad_norm": 0.6040119528770447, "lr": 9.74868025135266e-05, "epoch": 1, "t": 3.2545011043548584}
+{"step": 47, "loss": 1.2665650844573975, "grad_norm": 0.6519939303398132, "lr": 9.731800246349148e-05, "epoch": 1, "t": 3.2667903900146484}
+{"step": 48, "loss": 1.22287917137146, "grad_norm": 0.6127481460571289, "lr": 9.714387227305422e-05, "epoch": 1, "t": 3.124504804611206}
+{"step": 49, "loss": 1.1991313695907593, "grad_norm": 0.6090063452720642, "lr": 9.696443155712486e-05, "epoch": 1, "t": 3.0702877044677734}
+{"step": 50, "loss": 1.2632099390029907, "grad_norm": 0.6437141299247742, "lr": 9.67797005288181e-05, "epoch": 1, "t": 3.05598783493042}
+{"step": 51, "loss": 1.344907522201538, "grad_norm": 0.5727119445800781, "lr": 9.65896999971763e-05, "epoch": 1, "t": 3.2655839920043945}
+{"step": 52, "loss": 1.1435203552246094, "grad_norm": 0.5720223188400269, "lr": 9.639445136482548e-05, "epoch": 1, "t": 3.4344396591186523}
+{"step": 53, "loss": 1.2046306133270264, "grad_norm": 0.7183040380477905, "lr": 9.619397662556435e-05, "epoch": 1, "t": 3.1110618114471436}
+{"step": 54, "loss": 1.3263490200042725, "grad_norm": 0.641219973564148, "lr": 9.598829836188694e-05, "epoch": 1, "t": 3.1065995693206787}
+{"step": 55, "loss": 1.3485932350158691, "grad_norm": 0.6010202169418335, "lr": 9.577743974243874e-05, "epoch": 1, "t": 3.5663695335388184}
+{"step": 56, "loss": 1.1829782724380493, "grad_norm": 0.5923372507095337, "lr": 9.55614245194068e-05, "epoch": 1, "t": 3.588824987411499}
+{"step": 57, "loss": 1.265611171722412, "grad_norm": 0.6543408632278442, "lr": 9.534027702584425e-05, "epoch": 1, "t": 3.52703595161438}
+{"step": 58, "loss": 1.1645268201828003, "grad_norm": 0.6018239259719849, "lr": 9.511402217292926e-05, "epoch": 1, "t": 3.0196752548217773}
+{"step": 59, "loss": 1.2357110977172852, "grad_norm": 0.6383073925971985, "lr": 9.488268544715896e-05, "epoch": 1, "t": 3.346628427505493}
+{"step": 60, "loss": 1.1333954334259033, "grad_norm": 0.5893785953521729, "lr": 9.464629290747842e-05, "epoch": 1, "t": 3.4075937271118164}
+{"step": 61, "loss": 1.371953010559082, "grad_norm": 0.6793717741966248, "lr": 9.440487118234535e-05, "epoch": 1, "t": 3.11741304397583}
+{"step": 62, "loss": 1.270416259765625, "grad_norm": 0.6013386845588684, "lr": 9.415844746673047e-05, "epoch": 1, "t": 2.8872106075286865}
+{"step": 63, "loss": 1.158923864364624, "grad_norm": 0.5537129640579224, "lr": 9.390704951905411e-05, "epoch": 1, "t": 3.417269706726074}
+{"step": 64, "loss": 1.2678682804107666, "grad_norm": 0.6210922598838806, "lr": 9.365070565805941e-05, "epoch": 1, "t": 3.2121496200561523}
+{"step": 65, "loss": 1.2033791542053223, "grad_norm": 0.6727984547615051, "lr": 9.338944475962237e-05, "epoch": 1, "t": 3.0121853351593018}
+{"step": 66, "loss": 1.1171982288360596, "grad_norm": 0.6295392513275146, "lr": 9.312329625349902e-05, "epoch": 1, "t": 3.172537326812744}
+{"step": 67, "loss": 1.31270432472229, "grad_norm": 0.6367724537849426, "lr": 9.285229012001047e-05, "epoch": 1, "t": 3.2105793952941895}
+{"step": 68, "loss": 1.222433090209961, "grad_norm": 0.5490050911903381, "lr": 9.257645688666556e-05, "epoch": 1, "t": 3.302910566329956}
+{"step": 69, "loss": 1.2453744411468506, "grad_norm": 0.6330830454826355, "lr": 9.22958276247223e-05, "epoch": 1, "t": 2.7399702072143555}
+{"step": 70, "loss": 1.2824645042419434, "grad_norm": 0.6556730270385742, "lr": 9.201043394568773e-05, "epoch": 1, "t": 2.9631824493408203}
+{"step": 71, "loss": 1.0487356185913086, "grad_norm": 0.6062486171722412, "lr": 9.172030799775699e-05, "epoch": 1, "t": 3.266761541366577}
+{"step": 72, "loss": 1.2581334114074707, "grad_norm": 0.5626318454742432, "lr": 9.142548246219212e-05, "epoch": 1, "t": 2.8180768489837646}
+{"step": 73, "loss": 1.0210614204406738, "grad_norm": 0.7015877962112427, "lr": 9.112599054964057e-05, "epoch": 1, "t": 3.0019402503967285}
+{"step": 74, "loss": 1.1685079336166382, "grad_norm": 0.6257519721984863, "lr": 9.082186599639428e-05, "epoch": 1, "t": 3.162327527999878}
+{"step": 75, "loss": 1.0390830039978027, "grad_norm": 0.57488614320755, "lr": 9.051314306058933e-05, "epoch": 1, "t": 2.8893725872039795}
+{"step": 76, "loss": 1.1523704528808594, "grad_norm": 0.6371343731880188, "lr": 9.019985651834703e-05, "epoch": 1, "t": 3.662313938140869}
+{"step": 77, "loss": 1.2368154525756836, "grad_norm": 0.6149131059646606, "lr": 8.988204165985649e-05, "epoch": 1, "t": 3.5036165714263916}
+{"step": 78, "loss": 1.2111961841583252, "grad_norm": 0.6960669755935669, "lr": 8.955973428539944e-05, "epoch": 1, "t": 3.2394280433654785}
+{"step": 79, "loss": 1.2572574615478516, "grad_norm": 0.5682839751243591, "lr": 8.923297070131737e-05, "epoch": 1, "t": 3.3606321811676025}
+{"step": 80, "loss": 1.222373127937317, "grad_norm": 0.6081323623657227, "lr": 8.890178771592199e-05, "epoch": 1, "t": 3.0673413276672363}
+{"step": 81, "loss": 1.2898247241973877, "grad_norm": 0.6645525097846985, "lr": 8.856622263534875e-05, "epoch": 1, "t": 3.016792058944702}
+{"step": 82, "loss": 1.3272919654846191, "grad_norm": 0.6964982151985168, "lr": 8.822631325935463e-05, "epoch": 1, "t": 2.723292350769043}
+{"step": 83, "loss": 1.3005414009094238, "grad_norm": 0.6550720930099487, "lr": 8.788209787706015e-05, "epoch": 1, "t": 3.2887024879455566}
+{"step": 84, "loss": 1.2012135982513428, "grad_norm": 0.7282535433769226, "lr": 8.753361526263621e-05, "epoch": 1, "t": 3.040131092071533}
+{"step": 85, "loss": 1.0954171419143677, "grad_norm": 0.7641703486442566, "lr": 8.718090467093654e-05, "epoch": 1, "t": 2.938715696334839}
+{"step": 86, "loss": 1.0850834846496582, "grad_norm": 0.5393131971359253, "lr": 8.682400583307562e-05, "epoch": 1, "t": 3.2568469047546387}
+{"step": 87, "loss": 1.2838066816329956, "grad_norm": 0.5829893350601196, "lr": 8.646295895195333e-05, "epoch": 1, "t": 3.154959201812744}
+{"step": 88, "loss": 1.0138061046600342, "grad_norm": 1.0342228412628174, "lr": 8.609780469772623e-05, "epoch": 1, "t": 3.298295736312866}
+{"step": 89, "loss": 1.1860487461090088, "grad_norm": 0.6082722544670105, "lr": 8.572858420322627e-05, "epoch": 1, "t": 3.3159265518188477}
+{"step": 90, "loss": 1.2038296461105347, "grad_norm": 0.5841742753982544, "lr": 8.535533905932738e-05, "epoch": 1, "t": 3.3225772380828857}
+{"step": 91, "loss": 1.206883430480957, "grad_norm": 0.6440202593803406, "lr": 8.497811131026046e-05, "epoch": 1, "t": 3.4104104042053223}
+{"step": 92, "loss": 1.197068214416504, "grad_norm": 0.5538234710693359, "lr": 8.459694344887732e-05, "epoch": 1, "t": 3.493060350418091}
+{"step": 93, "loss": 1.1775360107421875, "grad_norm": 0.6358944177627563, "lr": 8.421187841186402e-05, "epoch": 1, "t": 2.902132272720337}
+{"step": 94, "loss": 1.28639817237854, "grad_norm": 0.6550676822662354, "lr": 8.382295957490436e-05, "epoch": 1, "t": 3.133793354034424}
+{"step": 95, "loss": 1.2666467428207397, "grad_norm": 0.5645681619644165, "lr": 8.343023074779368e-05, "epoch": 1, "t": 3.51326847076416}
+{"step": 96, "loss": 1.209169626235962, "grad_norm": 0.5733826160430908, "lr": 8.303373616950408e-05, "epoch": 1, "t": 2.9776999950408936}
+{"step": 97, "loss": 1.2243549823760986, "grad_norm": 0.6480118632316589, "lr": 8.263352050320094e-05, "epoch": 1, "t": 3.220109701156616}
+{"step": 98, "loss": 1.1669116020202637, "grad_norm": 0.6220917701721191, "lr": 8.222962883121196e-05, "epoch": 1, "t": 3.0820975303649902}
+{"step": 99, "loss": 1.2751058340072632, "grad_norm": 0.6334115862846375, "lr": 8.182210664994878e-05, "epoch": 1, "t": 3.0170717239379883}
+{"step": 100, "loss": 1.2395230531692505, "grad_norm": 0.6259497404098511, "lr": 8.141099986478212e-05, "epoch": 1, "t": 2.9425978660583496}
+{"step": 101, "loss": 1.1824058294296265, "grad_norm": 0.6447683572769165, "lr": 8.099635478487064e-05, "epoch": 1, "t": 3.1402673721313477}
+{"step": 102, "loss": 1.1586616039276123, "grad_norm": 0.5753451585769653, "lr": 8.057821811794458e-05, "epoch": 1, "t": 3.3552639484405518}
+{"step": 103, "loss": 1.3114122152328491, "grad_norm": 0.6271833181381226, "lr": 8.015663696504422e-05, "epoch": 1, "t": 3.071321725845337}
+{"step": 104, "loss": 1.190492868423462, "grad_norm": 0.6563113331794739, "lr": 7.973165881521434e-05, "epoch": 1, "t": 3.491790771484375}
+{"step": 105, "loss": 1.1693925857543945, "grad_norm": 0.6276815533638, "lr": 7.930333154015466e-05, "epoch": 1, "t": 3.0832626819610596}
+{"step": 106, "loss": 1.2199524641036987, "grad_norm": 0.5738036036491394, "lr": 7.88717033888274e-05, "epoch": 1, "t": 2.834317684173584}
+{"step": 107, "loss": 1.1463149785995483, "grad_norm": 0.584001898765564, "lr": 7.843682298202235e-05, "epoch": 1, "t": 3.0211265087127686}
+{"step": 108, "loss": 1.2491018772125244, "grad_norm": 0.5822736620903015, "lr": 7.799873930687978e-05, "epoch": 1, "t": 3.284855604171753}
+{"step": 109, "loss": 1.2575188875198364, "grad_norm": 0.5760317444801331, "lr": 7.755750171137246e-05, "epoch": 1, "t": 3.3838648796081543}
+{"step": 110, "loss": 1.2893245220184326, "grad_norm": 0.658383309841156, "lr": 7.711315989874677e-05, "epoch": 1, "t": 2.9641647338867188}
+{"step": 111, "loss": 1.2616381645202637, "grad_norm": 0.6391677260398865, "lr": 7.666576392192389e-05, "epoch": 1, "t": 2.9753646850585938}
+{"step": 112, "loss": 1.1679496765136719, "grad_norm": 0.7346526980400085, "lr": 7.621536417786159e-05, "epoch": 1, "t": 3.2575337886810303}
+{"step": 113, "loss": 1.2629032135009766, "grad_norm": 0.6406223177909851, "lr": 7.576201140187727e-05, "epoch": 1, "t": 2.884500503540039}
+{"step": 114, "loss": 1.130014419555664, "grad_norm": 0.5820654630661011, "lr": 7.530575666193283e-05, "epoch": 1, "t": 3.504206657409668}
+{"step": 115, "loss": 1.1340734958648682, "grad_norm": 0.6243785619735718, "lr": 7.484665135288213e-05, "epoch": 1, "t": 3.1025187969207764}
+{"step": 116, "loss": 0.9770622253417969, "grad_norm": 0.6665069460868835, "lr": 7.438474719068173e-05, "epoch": 1, "t": 3.2083888053894043}
+{"step": 117, "loss": 1.1668474674224854, "grad_norm": 0.7202078104019165, "lr": 7.392009620656513e-05, "epoch": 1, "t": 2.960710287094116}
+{"step": 118, "loss": 1.1972293853759766, "grad_norm": 0.670929491519928, "lr": 7.345275074118185e-05, "epoch": 1, "t": 3.0364036560058594}
+{"step": 119, "loss": 1.1133782863616943, "grad_norm": 0.5603964924812317, "lr": 7.298276343870151e-05, "epoch": 1, "t": 3.302926778793335}
+{"step": 120, "loss": 1.1300263404846191, "grad_norm": 0.6096655130386353, "lr": 7.251018724088367e-05, "epoch": 1, "t": 2.788339614868164}
+{"step": 121, "loss": 1.108581304550171, "grad_norm": 0.6814972758293152, "lr": 7.203507538111423e-05, "epoch": 1, "t": 3.142627239227295}
+{"step": 122, "loss": 1.1961642503738403, "grad_norm": 0.6451613903045654, "lr": 7.155748137840892e-05, "epoch": 1, "t": 3.2714130878448486}
+{"step": 123, "loss": 1.2988810539245605, "grad_norm": 0.5895416140556335, "lr": 7.107745903138472e-05, "epoch": 1, "t": 3.262310266494751}
+{"step": 124, "loss": 1.250213384628296, "grad_norm": 0.5825182199478149, "lr": 7.059506241219965e-05, "epoch": 1, "t": 3.3047902584075928}
+{"step": 125, "loss": 1.1177399158477783, "grad_norm": 0.5993546843528748, "lr": 7.011034586046176e-05, "epoch": 1, "t": 3.3100192546844482}
+{"step": 126, "loss": 1.1853973865509033, "grad_norm": 0.5644398927688599, "lr": 6.962336397710819e-05, "epoch": 1, "t": 3.297424554824829}
+{"step": 127, "loss": 1.083348035812378, "grad_norm": 0.6715015172958374, "lr": 6.91341716182545e-05, "epoch": 1, "t": 3.0592894554138184}
+{"step": 128, "loss": 1.1226119995117188, "grad_norm": 0.6135982275009155, "lr": 6.864282388901544e-05, "epoch": 1, "t": 3.140148162841797}
+{"step": 129, "loss": 1.1056246757507324, "grad_norm": 0.6817781329154968, "lr": 6.814937613729766e-05, "epoch": 1, "t": 3.015061855316162}
+{"step": 130, "loss": 1.1483442783355713, "grad_norm": 0.637622594833374, "lr": 6.765388394756504e-05, "epoch": 1, "t": 2.763184070587158}
+{"step": 131, "loss": 1.1246089935302734, "grad_norm": 0.6347431540489197, "lr": 6.715640313457733e-05, "epoch": 1, "t": 3.202000141143799}
+{"step": 132, "loss": 1.0986381769180298, "grad_norm": 0.5413315892219543, "lr": 6.665698973710288e-05, "epoch": 1, "t": 3.249988317489624}
+{"step": 133, "loss": 1.097859501838684, "grad_norm": 0.6063200831413269, "lr": 6.615570001160626e-05, "epoch": 1, "t": 3.0272021293640137}
+{"step": 134, "loss": 1.1629688739776611, "grad_norm": 0.5601895451545715, "lr": 6.565259042591113e-05, "epoch": 1, "t": 3.263235330581665}
+{"step": 135, "loss": 1.1872186660766602, "grad_norm": 0.7107530832290649, "lr": 6.514771765283942e-05, "epoch": 1, "t": 2.9488368034362793}
+{"step": 136, "loss": 1.1221897602081299, "grad_norm": 0.6101102232933044, "lr": 6.464113856382752e-05, "epoch": 1, "t": 3.2652335166931152}
+{"step": 137, "loss": 1.1738827228546143, "grad_norm": 0.5609278678894043, "lr": 6.413291022251989e-05, "epoch": 1, "t": 3.250384569168091}
+{"step": 138, "loss": 1.2568328380584717, "grad_norm": 0.5932942032814026, "lr": 6.362308987834115e-05, "epoch": 1, "t": 3.3005435466766357}
+{"step": 139, "loss": 1.1358098983764648, "grad_norm": 0.6544591784477234, "lr": 6.311173496004723e-05, "epoch": 1, "t": 3.1560237407684326}
+{"step": 140, "loss": 1.194271445274353, "grad_norm": 0.6982852816581726, "lr": 6.259890306925627e-05, "epoch": 1, "t": 3.034360408782959}
+{"step": 141, "loss": 1.1506847143173218, "grad_norm": 0.5750068426132202, "lr": 6.208465197396013e-05, "epoch": 1, "t": 3.097151279449463}
+{"step": 142, "loss": 1.123265027999878, "grad_norm": 0.6349514722824097, "lr": 6.156903960201709e-05, "epoch": 1, "t": 3.2174665927886963}
+{"step": 143, "loss": 1.2196425199508667, "grad_norm": 0.6227642297744751, "lr": 6.105212403462651e-05, "epoch": 1, "t": 3.142275094985962}
+{"step": 144, "loss": 1.1404293775558472, "grad_norm": 0.6641533970832825, "lr": 6.0533963499786314e-05, "epoch": 1, "t": 2.874513864517212}
+{"step": 145, "loss": 1.1745306253433228, "grad_norm": 0.5463564991950989, "lr": 6.001461636573397e-05, "epoch": 1, "t": 3.1237735748291016}
+{"step": 146, "loss": 1.214239239692688, "grad_norm": 0.5621738433837891, "lr": 5.949414113437142e-05, "epoch": 1, "t": 3.3486557006835938}
+{"step": 147, "loss": 1.1681621074676514, "grad_norm": 0.5608762502670288, "lr": 5.897259643467527e-05, "epoch": 1, "t": 3.1366868019104004}
+{"step": 148, "loss": 1.0693703889846802, "grad_norm": 0.557685375213623, "lr": 5.8450041016092464e-05, "epoch": 1, "t": 3.027944803237915}
+{"step": 149, "loss": 1.087806224822998, "grad_norm": 0.6668304800987244, "lr": 5.792653374192245e-05, "epoch": 1, "t": 3.001246690750122}
+{"step": 150, "loss": 1.1747510433197021, "grad_norm": 0.5668718814849854, "lr": 5.7402133582686576e-05, "epoch": 1, "t": 3.1275386810302734}
+{"step": 151, "loss": 1.2637999057769775, "grad_norm": 0.6705304980278015, "lr": 5.6876899609485256e-05, "epoch": 1, "t": 2.971377372741699}
+{"step": 152, "loss": 1.1549617052078247, "grad_norm": 0.49867674708366394, "lr": 5.6350890987343944e-05, "epoch": 1, "t": 3.563704252243042}
+{"step": 153, "loss": 1.2957277297973633, "grad_norm": 0.6017604470252991, "lr": 5.582416696854853e-05, "epoch": 1, "t": 3.4656805992126465}
+{"step": 154, "loss": 1.112256646156311, "grad_norm": 0.6232874393463135, "lr": 5.5296786885970805e-05, "epoch": 1, "t": 3.1391067504882812}
+{"step": 155, "loss": 1.1789627075195312, "grad_norm": 0.5496797561645508, "lr": 5.476881014638491e-05, "epoch": 1, "t": 3.340393543243408}
+{"step": 156, "loss": 1.1763181686401367, "grad_norm": 0.6288057565689087, "lr": 5.4240296223775465e-05, "epoch": 1, "t": 3.0915231704711914}
+{"step": 157, "loss": 1.223020315170288, "grad_norm": 0.646073579788208, "lr": 5.3711304652638126e-05, "epoch": 1, "t": 3.0915229320526123}
+{"step": 158, "loss": 1.1210674047470093, "grad_norm": 0.6813981533050537, "lr": 5.318189502127332e-05, "epoch": 1, "t": 3.022441864013672}
+{"step": 159, "loss": 1.1883270740509033, "grad_norm": 0.5533057451248169, "lr": 5.265212696507387e-05, "epoch": 1, "t": 3.237208366394043}
+{"step": 160, "loss": 1.2698025703430176, "grad_norm": 0.6933168768882751, "lr": 5.212206015980742e-05, "epoch": 1, "t": 3.109149932861328}
+{"step": 161, "loss": 1.1684236526489258, "grad_norm": 0.6078118681907654, "lr": 5.159175431489424e-05, "epoch": 1, "t": 3.052083969116211}
+{"step": 162, "loss": 1.1811952590942383, "grad_norm": 0.5393804311752319, "lr": 5.1061269166681183e-05, "epoch": 1, "t": 3.500868320465088}
+{"step": 163, "loss": 1.050422191619873, "grad_norm": 0.5921630263328552, "lr": 5.053066447171282e-05, "epoch": 1, "t": 3.230645179748535}
+{"step": 164, "loss": 1.2019083499908447, "grad_norm": 0.5943882465362549, "lr": 5e-05, "epoch": 1, "t": 3.409299373626709}
+{"step": 165, "loss": 1.1639859676361084, "grad_norm": 0.5662556886672974, "lr": 4.94693355282872e-05, "epoch": 1, "t": 3.1613926887512207}
+{"step": 166, "loss": 1.1924824714660645, "grad_norm": 0.5759224891662598, "lr": 4.893873083331882e-05, "epoch": 1, "t": 3.4098846912384033}
+{"step": 167, "loss": 1.0863919258117676, "grad_norm": 0.5584909319877625, "lr": 4.840824568510579e-05, "epoch": 1, "t": 3.2176594734191895}
+{"step": 168, "loss": 1.1040669679641724, "grad_norm": 0.590480387210846, "lr": 4.78779398401926e-05, "epoch": 1, "t": 3.0995821952819824}
+{"step": 169, "loss": 1.2440369129180908, "grad_norm": 0.6164286732673645, "lr": 4.734787303492615e-05, "epoch": 1, "t": 3.0424904823303223}
+{"step": 170, "loss": 1.2618274688720703, "grad_norm": 0.5930309891700745, "lr": 4.6818104978726685e-05, "epoch": 1, "t": 3.380537748336792}
+{"step": 171, "loss": 1.257959008216858, "grad_norm": 0.5765829086303711, "lr": 4.628869534736187e-05, "epoch": 1, "t": 3.247805595397949}
+{"step": 172, "loss": 1.0856506824493408, "grad_norm": 0.680612325668335, "lr": 4.575970377622456e-05, "epoch": 1, "t": 3.0945072174072266}
+{"step": 173, "loss": 1.1014269590377808, "grad_norm": 0.6119309067726135, "lr": 4.52311898536151e-05, "epoch": 1, "t": 3.069814920425415}
+{"step": 174, "loss": 1.1343410015106201, "grad_norm": 0.5517014265060425, "lr": 4.47032131140292e-05, "epoch": 1, "t": 3.2381527423858643}
+{"step": 175, "loss": 1.2033703327178955, "grad_norm": 0.6256654858589172, "lr": 4.4175833031451473e-05, "epoch": 1, "t": 2.9093897342681885}
+{"step": 176, "loss": 1.1122409105300903, "grad_norm": 0.5356712937355042, "lr": 4.364910901265606e-05, "epoch": 1, "t": 3.3648529052734375}
+{"step": 177, "loss": 1.0629881620407104, "grad_norm": 0.5948192477226257, "lr": 4.3123100390514756e-05, "epoch": 1, "t": 3.3540995121002197}
+{"step": 178, "loss": 1.0911394357681274, "grad_norm": 0.5890280604362488, "lr": 4.2597866417313436e-05, "epoch": 1, "t": 2.7660906314849854}
+{"step": 179, "loss": 1.1505049467086792, "grad_norm": 0.5762637853622437, "lr": 4.207346625807756e-05, "epoch": 1, "t": 3.26324725151062}
+{"step": 180, "loss": 1.2229969501495361, "grad_norm": 0.6637646555900574, "lr": 4.1549958983907555e-05, "epoch": 1, "t": 3.205742835998535}
+{"step": 181, "loss": 1.1818498373031616, "grad_norm": 0.6205675005912781, "lr": 4.102740356532473e-05, "epoch": 1, "t": 3.0563788414001465}
+{"step": 182, "loss": 1.1463134288787842, "grad_norm": 0.5895227193832397, "lr": 4.050585886562858e-05, "epoch": 1, "t": 3.354653835296631}
+{"step": 183, "loss": 1.2545373439788818, "grad_norm": 0.6045985817909241, "lr": 3.998538363426605e-05, "epoch": 1, "t": 3.2929279804229736}
+{"step": 184, "loss": 1.213509440422058, "grad_norm": 0.5701714754104614, "lr": 3.94660365002137e-05, "epoch": 1, "t": 3.1152875423431396}
+{"step": 185, "loss": 1.1366838216781616, "grad_norm": 0.5886996388435364, "lr": 3.894787596537352e-05, "epoch": 1, "t": 2.9948294162750244}
+{"step": 186, "loss": 1.1989463567733765, "grad_norm": 0.587399423122406, "lr": 3.843096039798293e-05, "epoch": 1, "t": 3.095613718032837}
+{"step": 187, "loss": 1.2452516555786133, "grad_norm": 0.6013587713241577, "lr": 3.791534802603988e-05, "epoch": 1, "t": 3.2710258960723877}
+{"step": 188, "loss": 1.107773780822754, "grad_norm": 0.6943751573562622, "lr": 3.740109693074375e-05, "epoch": 1, "t": 2.88173770904541}
+{"step": 189, "loss": 1.221630334854126, "grad_norm": 0.5519124865531921, "lr": 3.68882650399528e-05, "epoch": 1, "t": 3.6243529319763184}
+{"step": 190, "loss": 1.296262502670288, "grad_norm": 0.6040520071983337, "lr": 3.637691012165886e-05, "epoch": 1, "t": 3.190082550048828}
+{"step": 191, "loss": 1.0229010581970215, "grad_norm": 0.6353663206100464, "lr": 3.586708977748012e-05, "epoch": 1, "t": 3.0634703636169434}
+{"step": 192, "loss": 1.1086941957473755, "grad_norm": 0.5914731025695801, "lr": 3.5358861436172485e-05, "epoch": 1, "t": 2.985412120819092}
+{"step": 193, "loss": 1.152599811553955, "grad_norm": 0.5836315155029297, "lr": 3.485228234716058e-05, "epoch": 1, "t": 3.1702616214752197}
+{"step": 194, "loss": 1.1761579513549805, "grad_norm": 0.6064594388008118, "lr": 3.434740957408889e-05, "epoch": 1, "t": 3.2431304454803467}
+{"step": 195, "loss": 1.0056873559951782, "grad_norm": 0.6169631481170654, "lr": 3.3844299988393755e-05, "epoch": 1, "t": 3.1858839988708496}
+{"step": 196, "loss": 1.188856601715088, "grad_norm": 0.5115482211112976, "lr": 3.334301026289712e-05, "epoch": 1, "t": 3.479349374771118}
+{"step": 197, "loss": 1.26715087890625, "grad_norm": 0.5956758856773376, "lr": 3.284359686542269e-05, "epoch": 1, "t": 3.0514256954193115}
+{"step": 198, "loss": 1.0637043714523315, "grad_norm": 0.6245729327201843, "lr": 3.234611605243496e-05, "epoch": 1, "t": 2.9399797916412354}
+{"step": 199, "loss": 1.1112587451934814, "grad_norm": 0.8566910624504089, "lr": 3.1850623862702344e-05, "epoch": 1, "t": 3.244462728500366}
+{"step": 200, "loss": 1.1548724174499512, "grad_norm": 0.5739861130714417, "lr": 3.135717611098458e-05, "epoch": 1, "t": 3.48917555809021}
+{"step": 201, "loss": 1.1380984783172607, "grad_norm": 0.6413898468017578, "lr": 3.086582838174551e-05, "epoch": 1, "t": 3.262972593307495}
+{"step": 202, "loss": 1.184216856956482, "grad_norm": 0.6022217869758606, "lr": 3.0376636022891812e-05, "epoch": 1, "t": 3.1735434532165527}
+{"step": 203, "loss": 1.0677496194839478, "grad_norm": 0.594731330871582, "lr": 2.9889654139538246e-05, "epoch": 1, "t": 3.0828380584716797}
+{"step": 204, "loss": 1.0658659934997559, "grad_norm": 0.6577484011650085, "lr": 2.9404937587800375e-05, "epoch": 1, "t": 2.981673240661621}
+{"step": 205, "loss": 1.110738754272461, "grad_norm": 0.5835689306259155, "lr": 2.8922540968615286e-05, "epoch": 1, "t": 3.119799852371216}
+{"step": 206, "loss": 1.15838623046875, "grad_norm": 0.5934703946113586, "lr": 2.8442518621591086e-05, "epoch": 1, "t": 2.9539198875427246}
+{"step": 207, "loss": 1.1605558395385742, "grad_norm": 0.6126264929771423, "lr": 2.7964924618885778e-05, "epoch": 1, "t": 3.270961284637451}
+{"step": 208, "loss": 1.1961784362792969, "grad_norm": 0.6048758029937744, "lr": 2.748981275911633e-05, "epoch": 1, "t": 3.294602394104004}
+{"step": 209, "loss": 1.1333205699920654, "grad_norm": 0.6265262961387634, "lr": 2.701723656129851e-05, "epoch": 1, "t": 2.9729597568511963}
+{"step": 210, "loss": 1.1624705791473389, "grad_norm": 0.5679384469985962, "lr": 2.6547249258818164e-05, "epoch": 1, "t": 3.3164453506469727}
+{"step": 211, "loss": 1.0498530864715576, "grad_norm": 0.6415156722068787, "lr": 2.607990379343489e-05, "epoch": 1, "t": 2.941053628921509}
+{"step": 212, "loss": 1.018575668334961, "grad_norm": 0.6429729461669922, "lr": 2.5615252809318284e-05, "epoch": 1, "t": 2.953709602355957}
+{"step": 213, "loss": 1.295606255531311, "grad_norm": 0.5761866569519043, "lr": 2.5153348647117857e-05, "epoch": 1, "t": 3.5970351696014404}
+{"step": 214, "loss": 1.1575409173965454, "grad_norm": 0.5509631037712097, "lr": 2.469424333806718e-05, "epoch": 1, "t": 3.4427781105041504}
+{"step": 215, "loss": 1.046026349067688, "grad_norm": 0.626361608505249, "lr": 2.4237988598122752e-05, "epoch": 1, "t": 3.0473265647888184}
+{"step": 216, "loss": 1.2087295055389404, "grad_norm": 0.6969119310379028, "lr": 2.3784635822138424e-05, "epoch": 1, "t": 3.1313812732696533}
+{"step": 217, "loss": 1.1948926448822021, "grad_norm": 0.5655863881111145, "lr": 2.333423607807613e-05, "epoch": 1, "t": 3.246502161026001}
+{"step": 218, "loss": 1.185545802116394, "grad_norm": 0.6220167279243469, "lr": 2.288684010125325e-05, "epoch": 1, "t": 3.1777689456939697}
+{"step": 219, "loss": 1.1585865020751953, "grad_norm": 0.6115987300872803, "lr": 2.2442498288627556e-05, "epoch": 1, "t": 3.0028374195098877}
+{"step": 220, "loss": 1.045722246170044, "grad_norm": 0.652891993522644, "lr": 2.2001260693120233e-05, "epoch": 1, "t": 3.1166067123413086}
+{"step": 221, "loss": 1.0521082878112793, "grad_norm": 0.6072604060173035, "lr": 2.156317701797766e-05, "epoch": 1, "t": 2.929232120513916}
+{"step": 222, "loss": 1.1327838897705078, "grad_norm": 0.5822096467018127, "lr": 2.1128296611172593e-05, "epoch": 1, "t": 3.2047457695007324}
+{"step": 223, "loss": 1.1181288957595825, "grad_norm": 0.665998101234436, "lr": 2.0696668459845355e-05, "epoch": 1, "t": 2.911134719848633}
+{"step": 224, "loss": 1.2542016506195068, "grad_norm": 0.6517634987831116, "lr": 2.026834118478567e-05, "epoch": 1, "t": 3.3153140544891357}
+{"step": 225, "loss": 1.1514021158218384, "grad_norm": 0.6453463435173035, "lr": 1.98433630349558e-05, "epoch": 1, "t": 3.173335075378418}
+{"step": 226, "loss": 1.0198673009872437, "grad_norm": 0.6537891626358032, "lr": 1.9421781882055444e-05, "epoch": 1, "t": 2.895737648010254}
+{"step": 227, "loss": 1.1016989946365356, "grad_norm": 0.6124733090400696, "lr": 1.9003645215129355e-05, "epoch": 1, "t": 3.137103796005249}
+{"step": 228, "loss": 1.2772940397262573, "grad_norm": 0.6474252343177795, "lr": 1.858900013521788e-05, "epoch": 1, "t": 3.2814319133758545}
+{"step": 229, "loss": 1.0964081287384033, "grad_norm": 0.5437510013580322, "lr": 1.817789335005121e-05, "epoch": 1, "t": 3.5712926387786865}
+{"step": 230, "loss": 1.2341210842132568, "grad_norm": 0.5824730396270752, "lr": 1.777037116878804e-05, "epoch": 1, "t": 3.067516803741455}
+{"step": 231, "loss": 1.1912215948104858, "grad_norm": 0.6221261024475098, "lr": 1.7366479496799077e-05, "epoch": 1, "t": 3.298513412475586}
+{"step": 232, "loss": 1.171180009841919, "grad_norm": 0.6992931365966797, "lr": 1.6966263830495936e-05, "epoch": 1, "t": 2.959256172180176}
+{"step": 233, "loss": 1.107611894607544, "grad_norm": 0.5788571834564209, "lr": 1.656976925220633e-05, "epoch": 1, "t": 3.1884186267852783}
+{"step": 234, "loss": 1.1315503120422363, "grad_norm": 0.5943876504898071, "lr": 1.6177040425095662e-05, "epoch": 1, "t": 3.4186887741088867}
+{"step": 235, "loss": 1.2423261404037476, "grad_norm": 0.608063817024231, "lr": 1.5788121588135975e-05, "epoch": 1, "t": 2.972113847732544}
+{"step": 236, "loss": 1.1326618194580078, "grad_norm": 0.6260091066360474, "lr": 1.5403056551122697e-05, "epoch": 1, "t": 3.172067403793335}
+{"step": 237, "loss": 1.1549441814422607, "grad_norm": 0.5190547108650208, "lr": 1.5021888689739549e-05, "epoch": 1, "t": 3.217885971069336}
+{"step": 238, "loss": 1.2121508121490479, "grad_norm": 0.6164082288742065, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 3.116456985473633}
+{"step": 239, "loss": 1.186787486076355, "grad_norm": 0.7234702110290527, "lr": 1.427141579677374e-05, "epoch": 1, "t": 3.143021583557129}
+{"step": 240, "loss": 1.0911130905151367, "grad_norm": 0.5621363520622253, "lr": 1.3902195302273779e-05, "epoch": 1, "t": 3.230220317840576}
+{"step": 241, "loss": 1.1037520170211792, "grad_norm": 0.5908945798873901, "lr": 1.3537041048046695e-05, "epoch": 1, "t": 3.3280298709869385}
+{"step": 242, "loss": 1.1579039096832275, "grad_norm": 0.6530472040176392, "lr": 1.3175994166924394e-05, "epoch": 1, "t": 3.017481565475464}
+{"step": 243, "loss": 1.1008596420288086, "grad_norm": 0.5923284888267517, "lr": 1.2819095329063469e-05, "epoch": 1, "t": 2.8944971561431885}
+{"step": 244, "loss": 1.0920989513397217, "grad_norm": 0.6365619897842407, "lr": 1.246638473736378e-05, "epoch": 1, "t": 2.98527193069458}
+{"step": 245, "loss": 1.150795817375183, "grad_norm": 0.5903105139732361, "lr": 1.2117902122939861e-05, "epoch": 1, "t": 3.351747751235962}
+{"step": 246, "loss": 1.1814042329788208, "grad_norm": 0.6527220010757446, "lr": 1.1773686740645384e-05, "epoch": 1, "t": 3.2993481159210205}
+{"step": 247, "loss": 1.041551947593689, "grad_norm": 0.6108824014663696, "lr": 1.1433777364651271e-05, "epoch": 1, "t": 2.9131946563720703}
+{"step": 248, "loss": 0.9981467127799988, "grad_norm": 0.72492915391922, "lr": 1.1098212284078036e-05, "epoch": 1, "t": 3.004458427429199}
+{"step": 249, "loss": 1.0924677848815918, "grad_norm": 0.642476499080658, "lr": 1.076702929868264e-05, "epoch": 1, "t": 3.0539562702178955}
+{"step": 250, "loss": 1.0307005643844604, "grad_norm": 0.5619496703147888, "lr": 1.0440265714600572e-05, "epoch": 1, "t": 3.3310062885284424}
+{"step": 251, "loss": 1.143161416053772, "grad_norm": 0.5786762237548828, "lr": 1.0117958340143507e-05, "epoch": 1, "t": 3.5837273597717285}
+{"step": 252, "loss": 1.2036066055297852, "grad_norm": 0.5805399417877197, "lr": 9.800143481652979e-06, "epoch": 1, "t": 3.4024734497070312}
+{"step": 253, "loss": 1.2250564098358154, "grad_norm": 0.6205870509147644, "lr": 9.48685693941067e-06, "epoch": 1, "t": 3.296753168106079}
+{"step": 254, "loss": 1.1756951808929443, "grad_norm": 0.5823423266410828, "lr": 9.17813400360572e-06, "epoch": 1, "t": 3.418215036392212}
+{"step": 255, "loss": 1.1648709774017334, "grad_norm": 0.577837347984314, "lr": 8.874009450359427e-06, "epoch": 1, "t": 3.1661248207092285}
+{"step": 256, "loss": 1.0923452377319336, "grad_norm": 0.5667760968208313, "lr": 8.574517537807897e-06, "epoch": 1, "t": 3.2927653789520264}
+{"step": 257, "loss": 1.133958101272583, "grad_norm": 0.6322847008705139, "lr": 8.279692002243027e-06, "epoch": 1, "t": 3.098123550415039}
+{"step": 258, "loss": 1.1190788745880127, "grad_norm": 0.5409640669822693, "lr": 7.989566054312287e-06, "epoch": 1, "t": 3.3374855518341064}
+{"step": 259, "loss": 1.0914477109909058, "grad_norm": 0.582192063331604, "lr": 7.704172375277691e-06, "epoch": 1, "t": 3.106818437576294}
+{"step": 260, "loss": 1.1423075199127197, "grad_norm": 0.6024892330169678, "lr": 7.423543113334436e-06, "epoch": 1, "t": 3.0023627281188965}
+{"step": 261, "loss": 1.0804548263549805, "grad_norm": 0.630786120891571, "lr": 7.14770987998954e-06, "epoch": 1, "t": 3.145784616470337}
+{"step": 262, "loss": 1.0986104011535645, "grad_norm": 0.5946423411369324, "lr": 6.876703746500984e-06, "epoch": 1, "t": 3.0009729862213135}
+{"step": 263, "loss": 1.1484415531158447, "grad_norm": 0.6121066212654114, "lr": 6.610555240377652e-06, "epoch": 1, "t": 3.253079414367676}
+{"step": 264, "loss": 1.1844849586486816, "grad_norm": 0.6580023169517517, "lr": 6.349294341940593e-06, "epoch": 1, "t": 2.939393997192383}
+{"step": 265, "loss": 1.2188023328781128, "grad_norm": 0.6581462621688843, "lr": 6.092950480945897e-06, "epoch": 1, "t": 3.072070360183716}
+{"step": 266, "loss": 1.162814736366272, "grad_norm": 0.5939738750457764, "lr": 5.841552533269534e-06, "epoch": 1, "t": 3.5493392944335938}
+{"step": 267, "loss": 1.2604751586914062, "grad_norm": 0.6394745111465454, "lr": 5.595128817654638e-06, "epoch": 1, "t": 3.4884896278381348}
+{"step": 268, "loss": 1.146340250968933, "grad_norm": 0.52507084608078, "lr": 5.353707092521582e-06, "epoch": 1, "t": 3.546063184738159}
+{"step": 269, "loss": 1.1483925580978394, "grad_norm": 0.6097657084465027, "lr": 5.117314552841052e-06, "epoch": 1, "t": 3.0640063285827637}
+{"step": 270, "loss": 1.1280434131622314, "grad_norm": 0.6271836161613464, "lr": 4.885977827070748e-06, "epoch": 1, "t": 3.3846919536590576}
+{"step": 271, "loss": 1.1444844007492065, "grad_norm": 0.6049522161483765, "lr": 4.659722974155767e-06, "epoch": 1, "t": 2.9123172760009766}
+{"step": 272, "loss": 1.21195387840271, "grad_norm": 0.6936115026473999, "lr": 4.43857548059321e-06, "epoch": 1, "t": 3.290844440460205}
+{"step": 273, "loss": 1.083553433418274, "grad_norm": 0.5946942567825317, "lr": 4.2225602575612755e-06, "epoch": 1, "t": 3.0923309326171875}
+{"step": 274, "loss": 1.1764330863952637, "grad_norm": 0.6581479907035828, "lr": 4.011701638113063e-06, "epoch": 1, "t": 2.9411404132843018}
+{"step": 275, "loss": 1.020877718925476, "grad_norm": 0.6329962611198425, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 3.289860486984253}
+{"step": 276, "loss": 1.246074914932251, "grad_norm": 0.6978022456169128, "lr": 3.605548635174533e-06, "epoch": 1, "t": 3.033954381942749}
+{"step": 277, "loss": 1.1520459651947021, "grad_norm": 0.5991973280906677, "lr": 3.410300002823691e-06, "epoch": 1, "t": 3.0740439891815186}
+{"step": 278, "loss": 1.107762336730957, "grad_norm": 0.6086584329605103, "lr": 3.220299471181898e-06, "epoch": 1, "t": 3.1639225482940674}
+{"step": 279, "loss": 1.185377836227417, "grad_norm": 0.5857842564582825, "lr": 3.035568442875136e-06, "epoch": 1, "t": 3.148939847946167}
+{"step": 280, "loss": 1.1530919075012207, "grad_norm": 0.620843768119812, "lr": 2.85612772694579e-06, "epoch": 1, "t": 3.309561014175415}
+{"step": 281, "loss": 1.1242648363113403, "grad_norm": 0.596558153629303, "lr": 2.6819975365085237e-06, "epoch": 1, "t": 2.996490240097046}
+{"step": 282, "loss": 1.1723312139511108, "grad_norm": 0.5756584405899048, "lr": 2.5131974864734066e-06, "epoch": 1, "t": 3.3760790824890137}
+{"step": 283, "loss": 1.1036540269851685, "grad_norm": 0.5501166582107544, "lr": 2.349746591336405e-06, "epoch": 1, "t": 3.4217169284820557}
+{"step": 284, "loss": 1.2204954624176025, "grad_norm": 0.5722854733467102, "lr": 2.191663263037458e-06, "epoch": 1, "t": 3.1736488342285156}
+{"step": 285, "loss": 1.0018733739852905, "grad_norm": 0.5935145616531372, "lr": 2.0389653088865036e-06, "epoch": 1, "t": 3.4486806392669678}
+{"step": 286, "loss": 1.10654616355896, "grad_norm": 0.6606000661849976, "lr": 1.8916699295575324e-06, "epoch": 1, "t": 3.0607917308807373}
+{"step": 287, "loss": 1.1920475959777832, "grad_norm": 0.6654186248779297, "lr": 1.7497937171510547e-06, "epoch": 1, "t": 3.1877105236053467}
+{"step": 288, "loss": 1.1609740257263184, "grad_norm": 0.5407600998878479, "lr": 1.6133526533250565e-06, "epoch": 1, "t": 3.0744001865386963}
+{"step": 289, "loss": 1.1973578929901123, "grad_norm": 0.5786117315292358, "lr": 1.4823621074947503e-06, "epoch": 1, "t": 3.437222719192505}
+{"step": 290, "loss": 1.2084985971450806, "grad_norm": 0.6255794763565063, "lr": 1.3568368351012717e-06, "epoch": 1, "t": 3.1683363914489746}
+{"step": 291, "loss": 1.0135982036590576, "grad_norm": 0.5914056301116943, "lr": 1.236790975949592e-06, "epoch": 1, "t": 3.0157432556152344}
+{"step": 292, "loss": 1.1606616973876953, "grad_norm": 0.6107684373855591, "lr": 1.1222380526156928e-06, "epoch": 1, "t": 3.106762170791626}
+{"step": 293, "loss": 1.0304497480392456, "grad_norm": 0.5566821098327637, "lr": 1.0131909689233442e-06, "epoch": 1, "t": 3.210559129714966}
+{"step": 294, "loss": 1.201225757598877, "grad_norm": 0.5820565223693848, "lr": 9.096620084905472e-07, "epoch": 1, "t": 3.1978530883789062}
+{"step": 295, "loss": 1.1365504264831543, "grad_norm": 0.560297966003418, "lr": 8.11662833345822e-07, "epoch": 1, "t": 3.4149467945098877}
+{"step": 296, "loss": 1.2560802698135376, "grad_norm": 0.732952356338501, "lr": 7.192044826145771e-07, "epoch": 1, "t": 3.2335638999938965}
+{"step": 297, "loss": 1.0901637077331543, "grad_norm": 0.6319404244422913, "lr": 6.322973712755697e-07, "epoch": 1, "t": 3.2433555126190186}
+{"step": 298, "loss": 1.147761583328247, "grad_norm": 0.5701617002487183, "lr": 5.509512889877333e-07, "epoch": 1, "t": 3.0195932388305664}
+{"step": 299, "loss": 1.1304981708526611, "grad_norm": 0.6197502613067627, "lr": 4.7517539898741524e-07, "epoch": 1, "t": 2.9945428371429443}
+{"step": 300, "loss": 1.1311135292053223, "grad_norm": 0.5890395641326904, "lr": 4.049782370561583e-07, "epoch": 1, "t": 3.0199780464172363}
+{"step": 301, "loss": 1.185089349746704, "grad_norm": 0.6514334082603455, "lr": 3.4036771055923066e-07, "epoch": 1, "t": 3.4238433837890625}
+{"step": 302, "loss": 1.2313034534454346, "grad_norm": 0.6727525591850281, "lr": 2.813510975548772e-07, "epoch": 1, "t": 2.967966318130493}
+{"step": 303, "loss": 1.0683801174163818, "grad_norm": 0.6761717796325684, "lr": 2.2793504597447002e-07, "epoch": 1, "t": 3.156531572341919}
+{"step": 304, "loss": 1.1297897100448608, "grad_norm": 0.6605789065361023, "lr": 1.8012557287367392e-07, "epoch": 1, "t": 3.8942816257476807}
+{"step": 305, "loss": 1.1612855195999146, "grad_norm": 0.5988486409187317, "lr": 1.379280637546443e-07, "epoch": 1, "t": 3.2300262451171875}
+{"step": 306, "loss": 1.149768352508545, "grad_norm": 0.575775682926178, "lr": 1.0134727195937333e-07, "epoch": 1, "t": 3.4097437858581543}
+{"step": 307, "loss": 1.2025296688079834, "grad_norm": 0.6404022574424744, "lr": 7.038731813426291e-08, "epoch": 1, "t": 3.135118246078491}
+{"step": 308, "loss": 1.2290019989013672, "grad_norm": 0.6150823831558228, "lr": 4.5051689765929214e-08, "epoch": 1, "t": 3.4961788654327393}
+{"step": 309, "loss": 1.0409400463104248, "grad_norm": 0.5782963633537292, "lr": 2.534324078837802e-08, "epoch": 1, "t": 3.5864033699035645}
+{"step": 310, "loss": 1.1258490085601807, "grad_norm": 0.5588079690933228, "lr": 1.1264191261528557e-08, "epoch": 1, "t": 3.306751012802124}
+{"step": 311, "loss": 1.0926461219787598, "grad_norm": 0.662872850894928, "lr": 2.8161271211024633e-09, "epoch": 1, "t": 3.037998914718628}
+{"step": 312, "loss": 1.1257848739624023, "grad_norm": 0.6854902505874634, "lr": 0.0, "epoch": 1, "t": 3.279524803161621}
diff --git a/12b/u_bottom-s0/README.md b/12b/u_bottom-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0f2f243d3aceafe453065e108bf04a3768225947
--- /dev/null
+++ b/12b/u_bottom-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-12b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-12b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/12b/u_bottom-s0/adapter_config.json b/12b/u_bottom-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..4a56985d09053ce48704acb11b0cdcfda3abb6be
--- /dev/null
+++ b/12b/u_bottom-s0/adapter_config.json
@@ -0,0 +1,186 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "28.self_attn.k_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "language_model.layers.6.self_attn.v_proj",
+    "45.self_attn.q_proj",
+    "27.self_attn.v_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "34.self_attn.q_proj",
+    "28.self_attn.v_proj",
+    "44.self_attn.k_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "34.self_attn.k_proj",
+    "42.self_attn.v_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "30.self_attn.q_proj",
+    "27.self_attn.k_proj",
+    "41.self_attn.q_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "45.self_attn.k_proj",
+    "39.self_attn.q_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "35.self_attn.v_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "language_model.layers.13.self_attn.v_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "31.self_attn.v_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "language_model.layers.20.self_attn.k_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "41.self_attn.v_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "43.self_attn.k_proj",
+    "up_proj",
+    "47.self_attn.q_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "30.self_attn.v_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "29.self_attn.k_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "o_proj",
+    "33.self_attn.q_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "46.self_attn.q_proj",
+    "31.self_attn.q_proj",
+    "29.self_attn.q_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "39.self_attn.v_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "43.self_attn.q_proj",
+    "32.self_attn.k_proj",
+    "39.self_attn.k_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "38.self_attn.v_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "language_model.layers.3.self_attn.k_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "27.self_attn.q_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "47.self_attn.v_proj",
+    "40.self_attn.k_proj",
+    "32.self_attn.q_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "language_model.layers.7.self_attn.k_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "gate_proj",
+    "28.self_attn.q_proj",
+    "42.self_attn.k_proj",
+    "41.self_attn.k_proj",
+    "44.self_attn.q_proj",
+    "language_model.layers.19.self_attn.k_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "47.self_attn.k_proj",
+    "down_proj",
+    "37.self_attn.q_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "34.self_attn.v_proj",
+    "35.self_attn.q_proj",
+    "40.self_attn.v_proj",
+    "36.self_attn.v_proj",
+    "35.self_attn.k_proj",
+    "30.self_attn.k_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "37.self_attn.v_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "36.self_attn.q_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "40.self_attn.q_proj",
+    "42.self_attn.q_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "32.self_attn.v_proj",
+    "37.self_attn.k_proj",
+    "31.self_attn.k_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "38.self_attn.q_proj",
+    "45.self_attn.v_proj",
+    "46.self_attn.k_proj",
+    "33.self_attn.v_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "36.self_attn.k_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "38.self_attn.k_proj",
+    "46.self_attn.v_proj",
+    "43.self_attn.v_proj",
+    "44.self_attn.v_proj",
+    "29.self_attn.v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/12b/u_bottom-s0/adapter_model.safetensors b/12b/u_bottom-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..83c6bd956c2fbae47deff33fbc0383d6fab6d015
--- /dev/null
+++ b/12b/u_bottom-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9325842fc069b24e9a81a07af1ea0338864791f85d84b6533275bf05736b9aac
+size 1047628488
diff --git a/12b/u_bottom-s0/config.json b/12b/u_bottom-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..e3afea00f94e5a41d6de7f4e83069e2e1a78e828
--- /dev/null
+++ b/12b/u_bottom-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-12b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/arms/TASKW-u_bottom-gemma12.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/runs/u_bottom-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_12b_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 0,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 2
+}
\ No newline at end of file
diff --git a/12b/u_bottom-s0/log.jsonl b/12b/u_bottom-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..f508cfd5a9b84217511157fbc38a7dc72cde3d12
--- /dev/null
+++ b/12b/u_bottom-s0/log.jsonl
@@ -0,0 +1,312 @@
+{"step": 1, "loss": 2.2796168327331543, "grad_norm": 4.400256633758545, "lr": 1.25e-05, "epoch": 1, "t": 6.529042482376099}
+{"step": 2, "loss": 2.187624931335449, "grad_norm": 4.020505428314209, "lr": 1.8750000000000002e-05, "epoch": 1, "t": 5.776473045349121}
+{"step": 3, "loss": 2.1392927169799805, "grad_norm": 3.6714041233062744, "lr": 2.5e-05, "epoch": 1, "t": 5.566001653671265}
+{"step": 4, "loss": 1.9095730781555176, "grad_norm": 2.9393723011016846, "lr": 3.125e-05, "epoch": 1, "t": 5.125308513641357}
+{"step": 5, "loss": 1.8427956104278564, "grad_norm": 2.1116926670074463, "lr": 3.7500000000000003e-05, "epoch": 1, "t": 6.0628790855407715}
+{"step": 6, "loss": 1.7365288734436035, "grad_norm": 1.4905673265457153, "lr": 4.375e-05, "epoch": 1, "t": 4.792157888412476}
+{"step": 7, "loss": 1.6273750066757202, "grad_norm": 1.0730162858963013, "lr": 5e-05, "epoch": 1, "t": 3.939953088760376}
+{"step": 8, "loss": 1.5020382404327393, "grad_norm": 0.8699246644973755, "lr": 5.6250000000000005e-05, "epoch": 1, "t": 4.6516852378845215}
+{"step": 9, "loss": 1.6671159267425537, "grad_norm": 0.9875266551971436, "lr": 6.25e-05, "epoch": 1, "t": 4.667775630950928}
+{"step": 10, "loss": 1.7133204936981201, "grad_norm": 1.0434733629226685, "lr": 6.875e-05, "epoch": 1, "t": 5.036233186721802}
+{"step": 11, "loss": 1.4195690155029297, "grad_norm": 0.9061106443405151, "lr": 7.500000000000001e-05, "epoch": 1, "t": 4.639129161834717}
+{"step": 12, "loss": 1.4301880598068237, "grad_norm": 0.8332769274711609, "lr": 8.125000000000001e-05, "epoch": 1, "t": 4.400840520858765}
+{"step": 13, "loss": 1.2335591316223145, "grad_norm": 0.6944305300712585, "lr": 8.75e-05, "epoch": 1, "t": 3.5247182846069336}
+{"step": 14, "loss": 1.598285436630249, "grad_norm": 0.8040773868560791, "lr": 9.375e-05, "epoch": 1, "t": 3.1271800994873047}
+{"step": 15, "loss": 1.4832850694656372, "grad_norm": 0.7591297626495361, "lr": 0.0001, "epoch": 1, "t": 3.4279708862304688}
+{"step": 16, "loss": 1.376697301864624, "grad_norm": 0.735444962978363, "lr": 0.0001, "epoch": 1, "t": 3.4197046756744385}
+{"step": 17, "loss": 1.3606016635894775, "grad_norm": 0.7660970091819763, "lr": 9.99971838728789e-05, "epoch": 1, "t": 3.7712576389312744}
+{"step": 18, "loss": 1.3299849033355713, "grad_norm": 0.7133594155311584, "lr": 9.998873580873848e-05, "epoch": 1, "t": 3.6111791133880615}
+{"step": 19, "loss": 1.2998201847076416, "grad_norm": 0.7617425918579102, "lr": 9.997465675921163e-05, "epoch": 1, "t": 4.082550764083862}
+{"step": 20, "loss": 1.3217943906784058, "grad_norm": 0.6228795051574707, "lr": 9.995494831023409e-05, "epoch": 1, "t": 3.914626121520996}
+{"step": 21, "loss": 1.262622356414795, "grad_norm": 0.6451112627983093, "lr": 9.992961268186573e-05, "epoch": 1, "t": 3.759744644165039}
+{"step": 22, "loss": 1.3451486825942993, "grad_norm": 0.6689098477363586, "lr": 9.989865272804063e-05, "epoch": 1, "t": 3.9024102687835693}
+{"step": 23, "loss": 1.1713173389434814, "grad_norm": 0.6071130633354187, "lr": 9.986207193624536e-05, "epoch": 1, "t": 3.8475289344787598}
+{"step": 24, "loss": 1.3761532306671143, "grad_norm": 0.5865582227706909, "lr": 9.981987442712633e-05, "epoch": 1, "t": 3.609874725341797}
+{"step": 25, "loss": 1.1579132080078125, "grad_norm": 0.5750421285629272, "lr": 9.977206495402554e-05, "epoch": 1, "t": 3.236706495285034}
+{"step": 26, "loss": 1.3194642066955566, "grad_norm": 0.6225281953811646, "lr": 9.971864890244513e-05, "epoch": 1, "t": 3.1698687076568604}
+{"step": 27, "loss": 1.3758137226104736, "grad_norm": 0.5653595924377441, "lr": 9.965963228944078e-05, "epoch": 1, "t": 3.1658356189727783}
+{"step": 28, "loss": 1.3184517621994019, "grad_norm": 0.6315352916717529, "lr": 9.959502176294383e-05, "epoch": 1, "t": 2.774376630783081}
+{"step": 29, "loss": 1.247741460800171, "grad_norm": 0.7167669534683228, "lr": 9.95248246010126e-05, "epoch": 1, "t": 2.7905900478363037}
+{"step": 30, "loss": 1.3454320430755615, "grad_norm": 0.6151255965232849, "lr": 9.944904871101228e-05, "epoch": 1, "t": 2.8834762573242188}
+{"step": 31, "loss": 1.2908504009246826, "grad_norm": 0.5934525728225708, "lr": 9.936770262872443e-05, "epoch": 1, "t": 2.8511104583740234}
+{"step": 32, "loss": 1.246272087097168, "grad_norm": 0.5656528472900391, "lr": 9.928079551738543e-05, "epoch": 1, "t": 3.3971545696258545}
+{"step": 33, "loss": 1.2167012691497803, "grad_norm": 0.5671595931053162, "lr": 9.918833716665419e-05, "epoch": 1, "t": 3.0128860473632812}
+{"step": 34, "loss": 1.1612294912338257, "grad_norm": 0.6264058351516724, "lr": 9.909033799150946e-05, "epoch": 1, "t": 3.151306390762329}
+{"step": 35, "loss": 1.1641662120819092, "grad_norm": 0.5991170406341553, "lr": 9.898680903107666e-05, "epoch": 1, "t": 2.8719534873962402}
+{"step": 36, "loss": 1.2150335311889648, "grad_norm": 0.5848028063774109, "lr": 9.887776194738432e-05, "epoch": 1, "t": 3.593238592147827}
+{"step": 37, "loss": 1.2703022956848145, "grad_norm": 0.6096404790878296, "lr": 9.876320902405042e-05, "epoch": 1, "t": 3.589287757873535}
+{"step": 38, "loss": 1.2524504661560059, "grad_norm": 0.6689551472663879, "lr": 9.864316316489873e-05, "epoch": 1, "t": 2.939807891845703}
+{"step": 39, "loss": 1.3208110332489014, "grad_norm": 0.6150893568992615, "lr": 9.851763789250525e-05, "epoch": 1, "t": 2.79067325592041}
+{"step": 40, "loss": 1.2492326498031616, "grad_norm": 0.5812289118766785, "lr": 9.838664734667495e-05, "epoch": 1, "t": 3.183887243270874}
+{"step": 41, "loss": 1.1764318943023682, "grad_norm": 0.6673151850700378, "lr": 9.825020628284896e-05, "epoch": 1, "t": 3.1451289653778076}
+{"step": 42, "loss": 1.269309401512146, "grad_norm": 0.5575243234634399, "lr": 9.810833007044247e-05, "epoch": 1, "t": 3.308140277862549}
+{"step": 43, "loss": 1.3327550888061523, "grad_norm": 0.5585793256759644, "lr": 9.796103469111351e-05, "epoch": 1, "t": 4.341813325881958}
+{"step": 44, "loss": 1.217625379562378, "grad_norm": 0.5931463837623596, "lr": 9.780833673696254e-05, "epoch": 1, "t": 3.953521728515625}
+{"step": 45, "loss": 1.2160354852676392, "grad_norm": 0.5486651659011841, "lr": 9.76502534086636e-05, "epoch": 1, "t": 3.840898275375366}
+{"step": 46, "loss": 1.2350542545318604, "grad_norm": 0.5826579928398132, "lr": 9.74868025135266e-05, "epoch": 1, "t": 3.692005157470703}
+{"step": 47, "loss": 1.251184344291687, "grad_norm": 0.577694296836853, "lr": 9.731800246349148e-05, "epoch": 1, "t": 3.7976977825164795}
+{"step": 48, "loss": 1.2458150386810303, "grad_norm": 0.5890681743621826, "lr": 9.714387227305422e-05, "epoch": 1, "t": 3.81876802444458}
+{"step": 49, "loss": 1.1998258829116821, "grad_norm": 0.5721865892410278, "lr": 9.696443155712486e-05, "epoch": 1, "t": 3.5375852584838867}
+{"step": 50, "loss": 1.2159050703048706, "grad_norm": 0.5733393430709839, "lr": 9.67797005288181e-05, "epoch": 1, "t": 4.073696851730347}
+{"step": 51, "loss": 1.3022195100784302, "grad_norm": 0.5709722638130188, "lr": 9.65896999971763e-05, "epoch": 1, "t": 4.027852773666382}
+{"step": 52, "loss": 1.235236644744873, "grad_norm": 0.6509473919868469, "lr": 9.639445136482548e-05, "epoch": 1, "t": 3.913759708404541}
+{"step": 53, "loss": 1.1816620826721191, "grad_norm": 0.6693188548088074, "lr": 9.619397662556435e-05, "epoch": 1, "t": 3.5523266792297363}
+{"step": 54, "loss": 1.3099932670593262, "grad_norm": 0.6470484137535095, "lr": 9.598829836188694e-05, "epoch": 1, "t": 3.910655975341797}
+{"step": 55, "loss": 1.2423217296600342, "grad_norm": 0.6475738286972046, "lr": 9.577743974243874e-05, "epoch": 1, "t": 3.8942806720733643}
+{"step": 56, "loss": 1.1963086128234863, "grad_norm": 0.5847989320755005, "lr": 9.55614245194068e-05, "epoch": 1, "t": 3.4791030883789062}
+{"step": 57, "loss": 1.2429912090301514, "grad_norm": 0.6662818193435669, "lr": 9.534027702584425e-05, "epoch": 1, "t": 3.1651899814605713}
+{"step": 58, "loss": 1.1542389392852783, "grad_norm": 0.5899723768234253, "lr": 9.511402217292926e-05, "epoch": 1, "t": 3.2321722507476807}
+{"step": 59, "loss": 1.2696019411087036, "grad_norm": 0.5878300666809082, "lr": 9.488268544715896e-05, "epoch": 1, "t": 2.989182233810425}
+{"step": 60, "loss": 1.2108500003814697, "grad_norm": 0.5927729606628418, "lr": 9.464629290747842e-05, "epoch": 1, "t": 3.285382032394409}
+{"step": 61, "loss": 1.4031319618225098, "grad_norm": 0.5806950926780701, "lr": 9.440487118234535e-05, "epoch": 1, "t": 3.1141865253448486}
+{"step": 62, "loss": 1.2543126344680786, "grad_norm": 0.5452511310577393, "lr": 9.415844746673047e-05, "epoch": 1, "t": 2.9406721591949463}
+{"step": 63, "loss": 1.2066779136657715, "grad_norm": 0.5408366918563843, "lr": 9.390704951905411e-05, "epoch": 1, "t": 2.9355382919311523}
+{"step": 64, "loss": 1.2902932167053223, "grad_norm": 0.5582548379898071, "lr": 9.365070565805941e-05, "epoch": 1, "t": 3.316260814666748}
+{"step": 65, "loss": 1.1782703399658203, "grad_norm": 0.5738767385482788, "lr": 9.338944475962237e-05, "epoch": 1, "t": 3.3220856189727783}
+{"step": 66, "loss": 1.2352807521820068, "grad_norm": 0.6358559131622314, "lr": 9.312329625349902e-05, "epoch": 1, "t": 2.515019178390503}
+{"step": 67, "loss": 1.3366938829421997, "grad_norm": 0.6683616042137146, "lr": 9.285229012001047e-05, "epoch": 1, "t": 3.324474573135376}
+{"step": 68, "loss": 1.1387542486190796, "grad_norm": 0.5232688784599304, "lr": 9.257645688666556e-05, "epoch": 1, "t": 3.4642415046691895}
+{"step": 69, "loss": 1.1257469654083252, "grad_norm": 0.5826013088226318, "lr": 9.22958276247223e-05, "epoch": 1, "t": 2.810372829437256}
+{"step": 70, "loss": 1.4037599563598633, "grad_norm": 0.6210429668426514, "lr": 9.201043394568773e-05, "epoch": 1, "t": 3.4884438514709473}
+{"step": 71, "loss": 1.0940579175949097, "grad_norm": 0.6129340529441833, "lr": 9.172030799775699e-05, "epoch": 1, "t": 2.799748420715332}
+{"step": 72, "loss": 1.2496837377548218, "grad_norm": 0.5619972944259644, "lr": 9.142548246219212e-05, "epoch": 1, "t": 3.32836651802063}
+{"step": 73, "loss": 1.0862205028533936, "grad_norm": 0.6886274218559265, "lr": 9.112599054964057e-05, "epoch": 1, "t": 2.5285918712615967}
+{"step": 74, "loss": 1.1376888751983643, "grad_norm": 0.5930537581443787, "lr": 9.082186599639428e-05, "epoch": 1, "t": 3.0229763984680176}
+{"step": 75, "loss": 1.1845810413360596, "grad_norm": 0.8033909201622009, "lr": 9.051314306058933e-05, "epoch": 1, "t": 2.9015369415283203}
+{"step": 76, "loss": 1.1939804553985596, "grad_norm": 0.6111111044883728, "lr": 9.019985651834703e-05, "epoch": 1, "t": 2.897002696990967}
+{"step": 77, "loss": 1.220609426498413, "grad_norm": 0.6012175679206848, "lr": 8.988204165985649e-05, "epoch": 1, "t": 3.34718656539917}
+{"step": 78, "loss": 1.1667402982711792, "grad_norm": 0.594720721244812, "lr": 8.955973428539944e-05, "epoch": 1, "t": 2.8113768100738525}
+{"step": 79, "loss": 1.2189438343048096, "grad_norm": 0.5567514300346375, "lr": 8.923297070131737e-05, "epoch": 1, "t": 2.933164596557617}
+{"step": 80, "loss": 1.1402318477630615, "grad_norm": 0.6345027685165405, "lr": 8.890178771592199e-05, "epoch": 1, "t": 2.7314743995666504}
+{"step": 81, "loss": 1.2136492729187012, "grad_norm": 0.6385586261749268, "lr": 8.856622263534875e-05, "epoch": 1, "t": 3.028803586959839}
+{"step": 82, "loss": 1.2703094482421875, "grad_norm": 0.6952808499336243, "lr": 8.822631325935463e-05, "epoch": 1, "t": 2.77531099319458}
+{"step": 83, "loss": 1.352795124053955, "grad_norm": 0.6440946459770203, "lr": 8.788209787706015e-05, "epoch": 1, "t": 3.168881416320801}
+{"step": 84, "loss": 1.1485934257507324, "grad_norm": 0.650083065032959, "lr": 8.753361526263621e-05, "epoch": 1, "t": 2.8522775173187256}
+{"step": 85, "loss": 0.9985901117324829, "grad_norm": 0.8387364149093628, "lr": 8.718090467093654e-05, "epoch": 1, "t": 3.4434592723846436}
+{"step": 86, "loss": 1.0919952392578125, "grad_norm": 0.5590276122093201, "lr": 8.682400583307562e-05, "epoch": 1, "t": 3.630811929702759}
+{"step": 87, "loss": 1.3228874206542969, "grad_norm": 0.5738188624382019, "lr": 8.646295895195333e-05, "epoch": 1, "t": 3.2759687900543213}
+{"step": 88, "loss": 1.038428544998169, "grad_norm": 0.6042227745056152, "lr": 8.609780469772623e-05, "epoch": 1, "t": 3.297309398651123}
+{"step": 89, "loss": 1.2378544807434082, "grad_norm": 0.5510876774787903, "lr": 8.572858420322627e-05, "epoch": 1, "t": 3.615286111831665}
+{"step": 90, "loss": 1.279449224472046, "grad_norm": 0.5546838641166687, "lr": 8.535533905932738e-05, "epoch": 1, "t": 3.602992296218872}
+{"step": 91, "loss": 1.1821234226226807, "grad_norm": 0.5775654911994934, "lr": 8.497811131026046e-05, "epoch": 1, "t": 3.2568318843841553}
+{"step": 92, "loss": 1.191941499710083, "grad_norm": 0.5937525033950806, "lr": 8.459694344887732e-05, "epoch": 1, "t": 3.696549415588379}
+{"step": 93, "loss": 1.1268746852874756, "grad_norm": 0.5636759996414185, "lr": 8.421187841186402e-05, "epoch": 1, "t": 3.7892062664031982}
+{"step": 94, "loss": 1.3100491762161255, "grad_norm": 0.5433634519577026, "lr": 8.382295957490436e-05, "epoch": 1, "t": 4.484153747558594}
+{"step": 95, "loss": 1.2875268459320068, "grad_norm": 0.557334840297699, "lr": 8.343023074779368e-05, "epoch": 1, "t": 4.36227011680603}
+{"step": 96, "loss": 1.2306948900222778, "grad_norm": 0.6284522414207458, "lr": 8.303373616950408e-05, "epoch": 1, "t": 4.398929119110107}
+{"step": 97, "loss": 1.2416801452636719, "grad_norm": 0.6157611608505249, "lr": 8.263352050320094e-05, "epoch": 1, "t": 4.373669624328613}
+{"step": 98, "loss": 1.24025297164917, "grad_norm": 0.6044377088546753, "lr": 8.222962883121196e-05, "epoch": 1, "t": 4.606372594833374}
+{"step": 99, "loss": 1.22691011428833, "grad_norm": 0.6472576856613159, "lr": 8.182210664994878e-05, "epoch": 1, "t": 4.3771491050720215}
+{"step": 100, "loss": 1.0906577110290527, "grad_norm": 0.5523853898048401, "lr": 8.141099986478212e-05, "epoch": 1, "t": 3.805881977081299}
+{"step": 101, "loss": 1.04519784450531, "grad_norm": 0.5623805522918701, "lr": 8.099635478487064e-05, "epoch": 1, "t": 3.2478997707366943}
+{"step": 102, "loss": 1.1493608951568604, "grad_norm": 0.5591173768043518, "lr": 8.057821811794458e-05, "epoch": 1, "t": 4.073742389678955}
+{"step": 103, "loss": 1.1927741765975952, "grad_norm": 0.6300081014633179, "lr": 8.015663696504422e-05, "epoch": 1, "t": 4.002060651779175}
+{"step": 104, "loss": 1.1215739250183105, "grad_norm": 0.5657426714897156, "lr": 7.973165881521434e-05, "epoch": 1, "t": 3.805173635482788}
+{"step": 105, "loss": 1.110884666442871, "grad_norm": 0.635918915271759, "lr": 7.930333154015466e-05, "epoch": 1, "t": 4.056977033615112}
+{"step": 106, "loss": 1.2040023803710938, "grad_norm": 0.5650188326835632, "lr": 7.88717033888274e-05, "epoch": 1, "t": 3.434739351272583}
+{"step": 107, "loss": 1.1819329261779785, "grad_norm": 0.6216996312141418, "lr": 7.843682298202235e-05, "epoch": 1, "t": 3.6214869022369385}
+{"step": 108, "loss": 1.1873865127563477, "grad_norm": 0.5492762923240662, "lr": 7.799873930687978e-05, "epoch": 1, "t": 4.112046718597412}
+{"step": 109, "loss": 1.2666281461715698, "grad_norm": 0.579659104347229, "lr": 7.755750171137246e-05, "epoch": 1, "t": 4.060079336166382}
+{"step": 110, "loss": 1.2372691631317139, "grad_norm": 0.6457576155662537, "lr": 7.711315989874677e-05, "epoch": 1, "t": 3.6417388916015625}
+{"step": 111, "loss": 1.2777879238128662, "grad_norm": 0.6187204718589783, "lr": 7.666576392192389e-05, "epoch": 1, "t": 4.148644924163818}
+{"step": 112, "loss": 1.0683177709579468, "grad_norm": 0.6000900864601135, "lr": 7.621536417786159e-05, "epoch": 1, "t": 3.908047914505005}
+{"step": 113, "loss": 1.2187702655792236, "grad_norm": 0.6108975410461426, "lr": 7.576201140187727e-05, "epoch": 1, "t": 3.8268630504608154}
+{"step": 114, "loss": 1.203293800354004, "grad_norm": 0.6008256077766418, "lr": 7.530575666193283e-05, "epoch": 1, "t": 3.799168109893799}
+{"step": 115, "loss": 1.1572589874267578, "grad_norm": 0.646173357963562, "lr": 7.484665135288213e-05, "epoch": 1, "t": 3.906832695007324}
+{"step": 116, "loss": 1.107800006866455, "grad_norm": 0.6491538286209106, "lr": 7.438474719068173e-05, "epoch": 1, "t": 3.9282758235931396}
+{"step": 117, "loss": 1.1002235412597656, "grad_norm": 0.7050867676734924, "lr": 7.392009620656513e-05, "epoch": 1, "t": 3.2994375228881836}
+{"step": 118, "loss": 1.0847188234329224, "grad_norm": 0.675769567489624, "lr": 7.345275074118185e-05, "epoch": 1, "t": 3.962752103805542}
+{"step": 119, "loss": 1.1019296646118164, "grad_norm": 0.5060476660728455, "lr": 7.298276343870151e-05, "epoch": 1, "t": 4.020556926727295}
+{"step": 120, "loss": 1.1388767957687378, "grad_norm": 0.5517978072166443, "lr": 7.251018724088367e-05, "epoch": 1, "t": 4.11132550239563}
+{"step": 121, "loss": 1.1106231212615967, "grad_norm": 0.6458324193954468, "lr": 7.203507538111423e-05, "epoch": 1, "t": 3.6184656620025635}
+{"step": 122, "loss": 1.1430410146713257, "grad_norm": 0.6245289444923401, "lr": 7.155748137840892e-05, "epoch": 1, "t": 3.472459077835083}
+{"step": 123, "loss": 1.194145917892456, "grad_norm": 0.5574867725372314, "lr": 7.107745903138472e-05, "epoch": 1, "t": 3.8255302906036377}
+{"step": 124, "loss": 1.293508529663086, "grad_norm": 0.6076470017433167, "lr": 7.059506241219965e-05, "epoch": 1, "t": 3.902667760848999}
+{"step": 125, "loss": 1.1743580102920532, "grad_norm": 0.5984256267547607, "lr": 7.011034586046176e-05, "epoch": 1, "t": 3.908402681350708}
+{"step": 126, "loss": 1.121490240097046, "grad_norm": 0.5764150619506836, "lr": 6.962336397710819e-05, "epoch": 1, "t": 3.927380084991455}
+{"step": 127, "loss": 1.1504158973693848, "grad_norm": 0.6502886414527893, "lr": 6.91341716182545e-05, "epoch": 1, "t": 3.7148592472076416}
+{"step": 128, "loss": 1.1956489086151123, "grad_norm": 0.6529905796051025, "lr": 6.864282388901544e-05, "epoch": 1, "t": 3.876858949661255}
+{"step": 129, "loss": 1.167960286140442, "grad_norm": 0.5582114458084106, "lr": 6.814937613729766e-05, "epoch": 1, "t": 3.877762794494629}
+{"step": 130, "loss": 1.165308952331543, "grad_norm": 0.6209343671798706, "lr": 6.765388394756504e-05, "epoch": 1, "t": 3.7977523803710938}
+{"step": 131, "loss": 1.0924253463745117, "grad_norm": 0.5518262982368469, "lr": 6.715640313457733e-05, "epoch": 1, "t": 3.5653810501098633}
+{"step": 132, "loss": 1.139561653137207, "grad_norm": 0.5491393208503723, "lr": 6.665698973710288e-05, "epoch": 1, "t": 3.6677472591400146}
+{"step": 133, "loss": 1.0036296844482422, "grad_norm": 0.5746570229530334, "lr": 6.615570001160626e-05, "epoch": 1, "t": 3.7977983951568604}
+{"step": 134, "loss": 1.247854471206665, "grad_norm": 0.576014518737793, "lr": 6.565259042591113e-05, "epoch": 1, "t": 4.056816101074219}
+{"step": 135, "loss": 1.1368107795715332, "grad_norm": 0.6416566371917725, "lr": 6.514771765283942e-05, "epoch": 1, "t": 2.689995765686035}
+{"step": 136, "loss": 1.1421349048614502, "grad_norm": 0.7293333411216736, "lr": 6.464113856382752e-05, "epoch": 1, "t": 2.9442124366760254}
+{"step": 137, "loss": 1.1195770502090454, "grad_norm": 0.5645886659622192, "lr": 6.413291022251989e-05, "epoch": 1, "t": 3.0367612838745117}
+{"step": 138, "loss": 1.2760584354400635, "grad_norm": 0.5907130837440491, "lr": 6.362308987834115e-05, "epoch": 1, "t": 3.262828826904297}
+{"step": 139, "loss": 1.0437781810760498, "grad_norm": 0.6388912200927734, "lr": 6.311173496004723e-05, "epoch": 1, "t": 2.9264111518859863}
+{"step": 140, "loss": 1.2465133666992188, "grad_norm": 0.5667279362678528, "lr": 6.259890306925627e-05, "epoch": 1, "t": 3.1224493980407715}
+{"step": 141, "loss": 1.1778254508972168, "grad_norm": 0.7854788303375244, "lr": 6.208465197396013e-05, "epoch": 1, "t": 2.786785364151001}
+{"step": 142, "loss": 1.0598783493041992, "grad_norm": 0.6885096430778503, "lr": 6.156903960201709e-05, "epoch": 1, "t": 2.88132643699646}
+{"step": 143, "loss": 1.193947196006775, "grad_norm": 0.6000897288322449, "lr": 6.105212403462651e-05, "epoch": 1, "t": 3.3030450344085693}
+{"step": 144, "loss": 1.2082865238189697, "grad_norm": 0.6318554878234863, "lr": 6.0533963499786314e-05, "epoch": 1, "t": 2.8742475509643555}
+{"step": 145, "loss": 1.200656533241272, "grad_norm": 0.5822675824165344, "lr": 6.001461636573397e-05, "epoch": 1, "t": 2.9299895763397217}
+{"step": 146, "loss": 1.1954030990600586, "grad_norm": 0.5650563836097717, "lr": 5.949414113437142e-05, "epoch": 1, "t": 2.949399709701538}
+{"step": 147, "loss": 1.1622741222381592, "grad_norm": 0.568586528301239, "lr": 5.897259643467527e-05, "epoch": 1, "t": 2.968006134033203}
+{"step": 148, "loss": 1.100928544998169, "grad_norm": 0.5402028560638428, "lr": 5.8450041016092464e-05, "epoch": 1, "t": 3.0402121543884277}
+{"step": 149, "loss": 1.1039650440216064, "grad_norm": 0.6397644877433777, "lr": 5.792653374192245e-05, "epoch": 1, "t": 2.7922537326812744}
+{"step": 150, "loss": 1.242647409439087, "grad_norm": 0.5479434728622437, "lr": 5.7402133582686576e-05, "epoch": 1, "t": 3.6813130378723145}
+{"step": 151, "loss": 1.2494502067565918, "grad_norm": 0.6544719934463501, "lr": 5.6876899609485256e-05, "epoch": 1, "t": 2.8223929405212402}
+{"step": 152, "loss": 1.076706886291504, "grad_norm": 0.6050904989242554, "lr": 5.6350890987343944e-05, "epoch": 1, "t": 3.354572296142578}
+{"step": 153, "loss": 1.2764182090759277, "grad_norm": 0.5722780227661133, "lr": 5.582416696854853e-05, "epoch": 1, "t": 3.2717432975769043}
+{"step": 154, "loss": 1.196759581565857, "grad_norm": 0.6652747988700867, "lr": 5.5296786885970805e-05, "epoch": 1, "t": 2.904327154159546}
+{"step": 155, "loss": 1.1735427379608154, "grad_norm": 0.6104219555854797, "lr": 5.476881014638491e-05, "epoch": 1, "t": 2.8878519535064697}
+{"step": 156, "loss": 1.2194571495056152, "grad_norm": 0.590528666973114, "lr": 5.4240296223775465e-05, "epoch": 1, "t": 3.2303807735443115}
+{"step": 157, "loss": 1.159003496170044, "grad_norm": 0.5662127137184143, "lr": 5.3711304652638126e-05, "epoch": 1, "t": 3.216623067855835}
+{"step": 158, "loss": 1.0291956663131714, "grad_norm": 0.653940737247467, "lr": 5.318189502127332e-05, "epoch": 1, "t": 2.6618824005126953}
+{"step": 159, "loss": 1.1841682195663452, "grad_norm": 0.5722079277038574, "lr": 5.265212696507387e-05, "epoch": 1, "t": 3.252058744430542}
+{"step": 160, "loss": 1.2371938228607178, "grad_norm": 0.5706830620765686, "lr": 5.212206015980742e-05, "epoch": 1, "t": 3.335721015930176}
+{"step": 161, "loss": 1.096413254737854, "grad_norm": 0.5922949910163879, "lr": 5.159175431489424e-05, "epoch": 1, "t": 2.917912721633911}
+{"step": 162, "loss": 1.1713407039642334, "grad_norm": 0.6230096817016602, "lr": 5.1061269166681183e-05, "epoch": 1, "t": 3.006758689880371}
+{"step": 163, "loss": 0.9827152490615845, "grad_norm": 0.5703536868095398, "lr": 5.053066447171282e-05, "epoch": 1, "t": 3.10632586479187}
+{"step": 164, "loss": 1.1528689861297607, "grad_norm": 0.5780932307243347, "lr": 5e-05, "epoch": 1, "t": 3.277851104736328}
+{"step": 165, "loss": 1.0204691886901855, "grad_norm": 0.5498275756835938, "lr": 4.94693355282872e-05, "epoch": 1, "t": 2.549923896789551}
+{"step": 166, "loss": 1.2198524475097656, "grad_norm": 0.5634288787841797, "lr": 4.893873083331882e-05, "epoch": 1, "t": 3.530629873275757}
+{"step": 167, "loss": 1.1341540813446045, "grad_norm": 0.5160549283027649, "lr": 4.840824568510579e-05, "epoch": 1, "t": 3.4451637268066406}
+{"step": 168, "loss": 1.189644455909729, "grad_norm": 0.6211481094360352, "lr": 4.78779398401926e-05, "epoch": 1, "t": 3.210346221923828}
+{"step": 169, "loss": 1.131641149520874, "grad_norm": 0.6778606176376343, "lr": 4.734787303492615e-05, "epoch": 1, "t": 3.1198132038116455}
+{"step": 170, "loss": 1.2129818201065063, "grad_norm": 0.6030727028846741, "lr": 4.6818104978726685e-05, "epoch": 1, "t": 3.0022897720336914}
+{"step": 171, "loss": 1.1575881242752075, "grad_norm": 0.587956964969635, "lr": 4.628869534736187e-05, "epoch": 1, "t": 3.127443552017212}
+{"step": 172, "loss": 1.081390619277954, "grad_norm": 0.699698805809021, "lr": 4.575970377622456e-05, "epoch": 1, "t": 2.531400203704834}
+{"step": 173, "loss": 1.1386687755584717, "grad_norm": 0.5645583868026733, "lr": 4.52311898536151e-05, "epoch": 1, "t": 2.968369245529175}
+{"step": 174, "loss": 1.152360200881958, "grad_norm": 0.5911768078804016, "lr": 4.47032131140292e-05, "epoch": 1, "t": 2.897966146469116}
+{"step": 175, "loss": 1.1680355072021484, "grad_norm": 0.6221476197242737, "lr": 4.4175833031451473e-05, "epoch": 1, "t": 2.7802441120147705}
+{"step": 176, "loss": 1.0546282529830933, "grad_norm": 0.5288382768630981, "lr": 4.364910901265606e-05, "epoch": 1, "t": 3.2952880859375}
+{"step": 177, "loss": 1.1015164852142334, "grad_norm": 0.5779777765274048, "lr": 4.3123100390514756e-05, "epoch": 1, "t": 3.013418674468994}
+{"step": 178, "loss": 1.0744432210922241, "grad_norm": 0.7750630378723145, "lr": 4.2597866417313436e-05, "epoch": 1, "t": 2.702049732208252}
+{"step": 179, "loss": 1.1876572370529175, "grad_norm": 0.5808477997779846, "lr": 4.207346625807756e-05, "epoch": 1, "t": 2.766916036605835}
+{"step": 180, "loss": 1.1583235263824463, "grad_norm": 0.5504623651504517, "lr": 4.1549958983907555e-05, "epoch": 1, "t": 3.5158584117889404}
+{"step": 181, "loss": 1.1774816513061523, "grad_norm": 0.6100735664367676, "lr": 4.102740356532473e-05, "epoch": 1, "t": 3.2605581283569336}
+{"step": 182, "loss": 1.1724961996078491, "grad_norm": 0.5525132417678833, "lr": 4.050585886562858e-05, "epoch": 1, "t": 2.8990583419799805}
+{"step": 183, "loss": 1.1977407932281494, "grad_norm": 0.5886827707290649, "lr": 3.998538363426605e-05, "epoch": 1, "t": 3.36639666557312}
+{"step": 184, "loss": 1.2504453659057617, "grad_norm": 0.5493999123573303, "lr": 3.94660365002137e-05, "epoch": 1, "t": 3.140617847442627}
+{"step": 185, "loss": 1.065008521080017, "grad_norm": 0.5971235036849976, "lr": 3.894787596537352e-05, "epoch": 1, "t": 2.8115322589874268}
+{"step": 186, "loss": 1.132164478302002, "grad_norm": 0.6075999736785889, "lr": 3.843096039798293e-05, "epoch": 1, "t": 2.861053943634033}
+{"step": 187, "loss": 1.2062307596206665, "grad_norm": 0.6238131523132324, "lr": 3.791534802603988e-05, "epoch": 1, "t": 3.048740863800049}
+{"step": 188, "loss": 1.0297080278396606, "grad_norm": 0.6358351707458496, "lr": 3.740109693074375e-05, "epoch": 1, "t": 2.929049253463745}
+{"step": 189, "loss": 1.2059190273284912, "grad_norm": 0.5411284565925598, "lr": 3.68882650399528e-05, "epoch": 1, "t": 3.4467530250549316}
+{"step": 190, "loss": 1.3108404874801636, "grad_norm": 0.6114359498023987, "lr": 3.637691012165886e-05, "epoch": 1, "t": 3.0853779315948486}
+{"step": 191, "loss": 0.9888194799423218, "grad_norm": 0.6403577327728271, "lr": 3.586708977748012e-05, "epoch": 1, "t": 2.9144341945648193}
+{"step": 192, "loss": 1.0799251794815063, "grad_norm": 0.590144693851471, "lr": 3.5358861436172485e-05, "epoch": 1, "t": 2.9566233158111572}
+{"step": 193, "loss": 1.0908485651016235, "grad_norm": 0.5800775289535522, "lr": 3.485228234716058e-05, "epoch": 1, "t": 3.0918526649475098}
+{"step": 194, "loss": 1.1255537271499634, "grad_norm": 0.5407876372337341, "lr": 3.434740957408889e-05, "epoch": 1, "t": 3.088022470474243}
+{"step": 195, "loss": 0.9668008089065552, "grad_norm": 0.5884929895401001, "lr": 3.3844299988393755e-05, "epoch": 1, "t": 3.0009520053863525}
+{"step": 196, "loss": 1.216160535812378, "grad_norm": 0.5410187840461731, "lr": 3.334301026289712e-05, "epoch": 1, "t": 3.3178014755249023}
+{"step": 197, "loss": 1.1512436866760254, "grad_norm": 0.5714312791824341, "lr": 3.284359686542269e-05, "epoch": 1, "t": 3.00972580909729}
+{"step": 198, "loss": 1.0415838956832886, "grad_norm": 0.6088261008262634, "lr": 3.234611605243496e-05, "epoch": 1, "t": 2.928483724594116}
+{"step": 199, "loss": 1.0486640930175781, "grad_norm": 0.5961528420448303, "lr": 3.1850623862702344e-05, "epoch": 1, "t": 2.5869357585906982}
+{"step": 200, "loss": 1.2178640365600586, "grad_norm": 0.614139974117279, "lr": 3.135717611098458e-05, "epoch": 1, "t": 3.117884397506714}
+{"step": 201, "loss": 1.1999826431274414, "grad_norm": 0.6177778244018555, "lr": 3.086582838174551e-05, "epoch": 1, "t": 2.9931321144104004}
+{"step": 202, "loss": 1.14932119846344, "grad_norm": 0.5659716725349426, "lr": 3.0376636022891812e-05, "epoch": 1, "t": 3.005204439163208}
+{"step": 203, "loss": 1.0424413681030273, "grad_norm": 0.5663148760795593, "lr": 2.9889654139538246e-05, "epoch": 1, "t": 2.9001808166503906}
+{"step": 204, "loss": 1.1409486532211304, "grad_norm": 0.6158498525619507, "lr": 2.9404937587800375e-05, "epoch": 1, "t": 3.241957664489746}
+{"step": 205, "loss": 1.1239773035049438, "grad_norm": 0.6024658679962158, "lr": 2.8922540968615286e-05, "epoch": 1, "t": 3.4749953746795654}
+{"step": 206, "loss": 1.1450831890106201, "grad_norm": 0.592876672744751, "lr": 2.8442518621591086e-05, "epoch": 1, "t": 2.6182096004486084}
+{"step": 207, "loss": 1.1319626569747925, "grad_norm": 0.5627796053886414, "lr": 2.7964924618885778e-05, "epoch": 1, "t": 3.416124105453491}
+{"step": 208, "loss": 1.1606276035308838, "grad_norm": 0.6031860113143921, "lr": 2.748981275911633e-05, "epoch": 1, "t": 3.349666118621826}
+{"step": 209, "loss": 1.188281774520874, "grad_norm": 0.6572056412696838, "lr": 2.701723656129851e-05, "epoch": 1, "t": 2.6094274520874023}
+{"step": 210, "loss": 1.0975607633590698, "grad_norm": 0.5507838129997253, "lr": 2.6547249258818164e-05, "epoch": 1, "t": 3.1348466873168945}
+{"step": 211, "loss": 1.14078688621521, "grad_norm": 0.5925769209861755, "lr": 2.607990379343489e-05, "epoch": 1, "t": 2.9667868614196777}
+{"step": 212, "loss": 1.0285526514053345, "grad_norm": 0.624167263507843, "lr": 2.5615252809318284e-05, "epoch": 1, "t": 3.144674777984619}
+{"step": 213, "loss": 1.2753043174743652, "grad_norm": 0.5767782330513, "lr": 2.5153348647117857e-05, "epoch": 1, "t": 3.520312547683716}
+{"step": 214, "loss": 1.2057697772979736, "grad_norm": 0.6363111138343811, "lr": 2.469424333806718e-05, "epoch": 1, "t": 3.1118662357330322}
+{"step": 215, "loss": 1.075385332107544, "grad_norm": 0.5819528102874756, "lr": 2.4237988598122752e-05, "epoch": 1, "t": 2.9472646713256836}
+{"step": 216, "loss": 1.0992224216461182, "grad_norm": 0.6043964624404907, "lr": 2.3784635822138424e-05, "epoch": 1, "t": 3.056764841079712}
+{"step": 217, "loss": 1.151720404624939, "grad_norm": 0.5544857978820801, "lr": 2.333423607807613e-05, "epoch": 1, "t": 3.764554023742676}
+{"step": 218, "loss": 1.0833635330200195, "grad_norm": 0.634046733379364, "lr": 2.288684010125325e-05, "epoch": 1, "t": 3.2840735912323}
+{"step": 219, "loss": 1.1649768352508545, "grad_norm": 0.597759485244751, "lr": 2.2442498288627556e-05, "epoch": 1, "t": 2.8854196071624756}
+{"step": 220, "loss": 1.0214139223098755, "grad_norm": 0.9099845886230469, "lr": 2.2001260693120233e-05, "epoch": 1, "t": 2.863187313079834}
+{"step": 221, "loss": 1.1014854907989502, "grad_norm": 0.5698481202125549, "lr": 2.156317701797766e-05, "epoch": 1, "t": 3.1249442100524902}
+{"step": 222, "loss": 1.1189522743225098, "grad_norm": 0.596674382686615, "lr": 2.1128296611172593e-05, "epoch": 1, "t": 2.7330100536346436}
+{"step": 223, "loss": 1.1307971477508545, "grad_norm": 0.6788355112075806, "lr": 2.0696668459845355e-05, "epoch": 1, "t": 2.805807113647461}
+{"step": 224, "loss": 1.2728497982025146, "grad_norm": 0.6415734887123108, "lr": 2.026834118478567e-05, "epoch": 1, "t": 2.9238686561584473}
+{"step": 225, "loss": 1.0751233100891113, "grad_norm": 0.5854493975639343, "lr": 1.98433630349558e-05, "epoch": 1, "t": 3.3074238300323486}
+{"step": 226, "loss": 1.020078420639038, "grad_norm": 0.6070003509521484, "lr": 1.9421781882055444e-05, "epoch": 1, "t": 2.811638355255127}
+{"step": 227, "loss": 1.1844269037246704, "grad_norm": 0.6074785590171814, "lr": 1.9003645215129355e-05, "epoch": 1, "t": 2.951449155807495}
+{"step": 228, "loss": 1.2330278158187866, "grad_norm": 0.6329712271690369, "lr": 1.858900013521788e-05, "epoch": 1, "t": 2.9545321464538574}
+{"step": 229, "loss": 1.0916948318481445, "grad_norm": 0.5630735754966736, "lr": 1.817789335005121e-05, "epoch": 1, "t": 3.1227049827575684}
+{"step": 230, "loss": 1.1919686794281006, "grad_norm": 0.5632869005203247, "lr": 1.777037116878804e-05, "epoch": 1, "t": 3.178375482559204}
+{"step": 231, "loss": 1.1420263051986694, "grad_norm": 0.6705955266952515, "lr": 1.7366479496799077e-05, "epoch": 1, "t": 3.3353352546691895}
+{"step": 232, "loss": 1.12916898727417, "grad_norm": 0.5830580592155457, "lr": 1.6966263830495936e-05, "epoch": 1, "t": 2.951209545135498}
+{"step": 233, "loss": 1.100843906402588, "grad_norm": 0.5498954653739929, "lr": 1.656976925220633e-05, "epoch": 1, "t": 3.590996265411377}
+{"step": 234, "loss": 1.1802067756652832, "grad_norm": 0.6611465811729431, "lr": 1.6177040425095662e-05, "epoch": 1, "t": 3.278940439224243}
+{"step": 235, "loss": 1.2227891683578491, "grad_norm": 0.5920096039772034, "lr": 1.5788121588135975e-05, "epoch": 1, "t": 3.1620593070983887}
+{"step": 236, "loss": 1.1182591915130615, "grad_norm": 0.5908201336860657, "lr": 1.5403056551122697e-05, "epoch": 1, "t": 3.068516731262207}
+{"step": 237, "loss": 1.1879627704620361, "grad_norm": 0.5485942959785461, "lr": 1.5021888689739549e-05, "epoch": 1, "t": 3.147620439529419}
+{"step": 238, "loss": 1.178399920463562, "grad_norm": 0.566916823387146, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 3.0227084159851074}
+{"step": 239, "loss": 1.1291710138320923, "grad_norm": 0.5659102201461792, "lr": 1.427141579677374e-05, "epoch": 1, "t": 3.0930042266845703}
+{"step": 240, "loss": 1.123749017715454, "grad_norm": 0.5526357889175415, "lr": 1.3902195302273779e-05, "epoch": 1, "t": 4.072468280792236}
+{"step": 241, "loss": 1.1211729049682617, "grad_norm": 0.5826315879821777, "lr": 1.3537041048046695e-05, "epoch": 1, "t": 4.41733980178833}
+{"step": 242, "loss": 1.1045269966125488, "grad_norm": 0.6689655184745789, "lr": 1.3175994166924394e-05, "epoch": 1, "t": 4.44432806968689}
+{"step": 243, "loss": 1.0280206203460693, "grad_norm": 0.5824991464614868, "lr": 1.2819095329063469e-05, "epoch": 1, "t": 4.081759452819824}
+{"step": 244, "loss": 1.1183559894561768, "grad_norm": 0.7170051336288452, "lr": 1.246638473736378e-05, "epoch": 1, "t": 2.734480857849121}
+{"step": 245, "loss": 1.1584432125091553, "grad_norm": 0.5693469047546387, "lr": 1.2117902122939861e-05, "epoch": 1, "t": 3.2850148677825928}
+{"step": 246, "loss": 1.192258596420288, "grad_norm": 0.594908595085144, "lr": 1.1773686740645384e-05, "epoch": 1, "t": 3.2409627437591553}
+{"step": 247, "loss": 1.0961769819259644, "grad_norm": 0.6132506132125854, "lr": 1.1433777364651271e-05, "epoch": 1, "t": 2.6660830974578857}
+{"step": 248, "loss": 1.0398869514465332, "grad_norm": 0.6917411088943481, "lr": 1.1098212284078036e-05, "epoch": 1, "t": 2.73703670501709}
+{"step": 249, "loss": 1.105663537979126, "grad_norm": 0.5996711850166321, "lr": 1.076702929868264e-05, "epoch": 1, "t": 3.0036585330963135}
+{"step": 250, "loss": 1.0799379348754883, "grad_norm": 0.5624319911003113, "lr": 1.0440265714600572e-05, "epoch": 1, "t": 2.9499378204345703}
+{"step": 251, "loss": 1.1669678688049316, "grad_norm": 0.5754649639129639, "lr": 1.0117958340143507e-05, "epoch": 1, "t": 3.080819845199585}
+{"step": 252, "loss": 1.1613426208496094, "grad_norm": 0.5716949701309204, "lr": 9.800143481652979e-06, "epoch": 1, "t": 3.149203300476074}
+{"step": 253, "loss": 1.1550145149230957, "grad_norm": 0.5361490249633789, "lr": 9.48685693941067e-06, "epoch": 1, "t": 3.073615789413452}
+{"step": 254, "loss": 1.189213514328003, "grad_norm": 0.5783571600914001, "lr": 9.17813400360572e-06, "epoch": 1, "t": 3.475926160812378}
+{"step": 255, "loss": 1.1734529733657837, "grad_norm": 0.5909482836723328, "lr": 8.874009450359427e-06, "epoch": 1, "t": 2.908905267715454}
+{"step": 256, "loss": 1.1405507326126099, "grad_norm": 0.5495503544807434, "lr": 8.574517537807897e-06, "epoch": 1, "t": 3.0931739807128906}
+{"step": 257, "loss": 1.0228874683380127, "grad_norm": 0.6029623746871948, "lr": 8.279692002243027e-06, "epoch": 1, "t": 2.623265027999878}
+{"step": 258, "loss": 1.1184940338134766, "grad_norm": 0.5476334691047668, "lr": 7.989566054312287e-06, "epoch": 1, "t": 3.156865358352661}
+{"step": 259, "loss": 0.9762853980064392, "grad_norm": 0.5764957070350647, "lr": 7.704172375277691e-06, "epoch": 1, "t": 3.138101100921631}
+{"step": 260, "loss": 1.1840956211090088, "grad_norm": 0.6296178698539734, "lr": 7.423543113334436e-06, "epoch": 1, "t": 3.011345624923706}
+{"step": 261, "loss": 1.0912525653839111, "grad_norm": 0.5177234411239624, "lr": 7.14770987998954e-06, "epoch": 1, "t": 3.1572554111480713}
+{"step": 262, "loss": 1.2317930459976196, "grad_norm": 0.550730288028717, "lr": 6.876703746500984e-06, "epoch": 1, "t": 3.1815311908721924}
+{"step": 263, "loss": 1.1975363492965698, "grad_norm": 0.6020235419273376, "lr": 6.610555240377652e-06, "epoch": 1, "t": 2.9622108936309814}
+{"step": 264, "loss": 1.1572779417037964, "grad_norm": 0.648000955581665, "lr": 6.349294341940593e-06, "epoch": 1, "t": 2.724074363708496}
+{"step": 265, "loss": 1.2379295825958252, "grad_norm": 0.6012561321258545, "lr": 6.092950480945897e-06, "epoch": 1, "t": 2.9164509773254395}
+{"step": 266, "loss": 1.0732264518737793, "grad_norm": 0.6169740557670593, "lr": 5.841552533269534e-06, "epoch": 1, "t": 3.3635013103485107}
+{"step": 267, "loss": 1.2429931163787842, "grad_norm": 0.5938675403594971, "lr": 5.595128817654638e-06, "epoch": 1, "t": 3.8237826824188232}
+{"step": 268, "loss": 1.1276895999908447, "grad_norm": 0.552442729473114, "lr": 5.353707092521582e-06, "epoch": 1, "t": 3.568659782409668}
+{"step": 269, "loss": 1.1378519535064697, "grad_norm": 0.6179390549659729, "lr": 5.117314552841052e-06, "epoch": 1, "t": 3.76232647895813}
+{"step": 270, "loss": 1.1695225238800049, "grad_norm": 0.6306130886077881, "lr": 4.885977827070748e-06, "epoch": 1, "t": 3.8953850269317627}
+{"step": 271, "loss": 1.1091785430908203, "grad_norm": 0.5891368985176086, "lr": 4.659722974155767e-06, "epoch": 1, "t": 3.540498971939087}
+{"step": 272, "loss": 1.145094394683838, "grad_norm": 0.729819655418396, "lr": 4.43857548059321e-06, "epoch": 1, "t": 4.277384281158447}
+{"step": 273, "loss": 1.1218918561935425, "grad_norm": 0.5778434872627258, "lr": 4.2225602575612755e-06, "epoch": 1, "t": 3.9088072776794434}
+{"step": 274, "loss": 1.1688685417175293, "grad_norm": 0.6175435781478882, "lr": 4.011701638113063e-06, "epoch": 1, "t": 3.6113033294677734}
+{"step": 275, "loss": 0.9873802661895752, "grad_norm": 0.5800295472145081, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 3.9763290882110596}
+{"step": 276, "loss": 1.1795215606689453, "grad_norm": 0.6106323599815369, "lr": 3.605548635174533e-06, "epoch": 1, "t": 4.0507025718688965}
+{"step": 277, "loss": 1.121046781539917, "grad_norm": 0.5597973465919495, "lr": 3.410300002823691e-06, "epoch": 1, "t": 3.7010719776153564}
+{"step": 278, "loss": 1.0764379501342773, "grad_norm": 0.6019837260246277, "lr": 3.220299471181898e-06, "epoch": 1, "t": 4.057452917098999}
+{"step": 279, "loss": 1.1457055807113647, "grad_norm": 0.5977408289909363, "lr": 3.035568442875136e-06, "epoch": 1, "t": 4.623497724533081}
+{"step": 280, "loss": 1.2408931255340576, "grad_norm": 0.5990954041481018, "lr": 2.85612772694579e-06, "epoch": 1, "t": 4.593478441238403}
+{"step": 281, "loss": 1.0744915008544922, "grad_norm": 0.675454318523407, "lr": 2.6819975365085237e-06, "epoch": 1, "t": 3.325536012649536}
+{"step": 282, "loss": 1.1200270652770996, "grad_norm": 0.6015303134918213, "lr": 2.5131974864734066e-06, "epoch": 1, "t": 3.777236223220825}
+{"step": 283, "loss": 1.0741581916809082, "grad_norm": 0.5861004590988159, "lr": 2.349746591336405e-06, "epoch": 1, "t": 3.6383676528930664}
+{"step": 284, "loss": 1.1736156940460205, "grad_norm": 0.6229422092437744, "lr": 2.191663263037458e-06, "epoch": 1, "t": 3.4343810081481934}
+{"step": 285, "loss": 0.9877618551254272, "grad_norm": 0.5953364968299866, "lr": 2.0389653088865036e-06, "epoch": 1, "t": 2.8251194953918457}
+{"step": 286, "loss": 1.1036381721496582, "grad_norm": 0.6857370138168335, "lr": 1.8916699295575324e-06, "epoch": 1, "t": 3.7751753330230713}
+{"step": 287, "loss": 1.0825369358062744, "grad_norm": 0.6057208180427551, "lr": 1.7497937171510547e-06, "epoch": 1, "t": 4.018556118011475}
+{"step": 288, "loss": 1.108640193939209, "grad_norm": 0.5292966961860657, "lr": 1.6133526533250565e-06, "epoch": 1, "t": 3.6731085777282715}
+{"step": 289, "loss": 1.2366762161254883, "grad_norm": 0.6128385663032532, "lr": 1.4823621074947503e-06, "epoch": 1, "t": 4.063400745391846}
+{"step": 290, "loss": 1.1423113346099854, "grad_norm": 0.7354625463485718, "lr": 1.3568368351012717e-06, "epoch": 1, "t": 4.498003244400024}
+{"step": 291, "loss": 1.0440839529037476, "grad_norm": 0.5979165434837341, "lr": 1.236790975949592e-06, "epoch": 1, "t": 4.201056480407715}
+{"step": 292, "loss": 1.125098466873169, "grad_norm": 0.6045401692390442, "lr": 1.1222380526156928e-06, "epoch": 1, "t": 3.946053981781006}
+{"step": 293, "loss": 1.001310110092163, "grad_norm": 0.5488253831863403, "lr": 1.0131909689233442e-06, "epoch": 1, "t": 3.8486053943634033}
+{"step": 294, "loss": 1.1466588973999023, "grad_norm": 0.5438467264175415, "lr": 9.096620084905472e-07, "epoch": 1, "t": 4.193962812423706}
+{"step": 295, "loss": 1.0969634056091309, "grad_norm": 0.5312578678131104, "lr": 8.11662833345822e-07, "epoch": 1, "t": 3.706310987472534}
+{"step": 296, "loss": 1.2715599536895752, "grad_norm": 0.7167814373970032, "lr": 7.192044826145771e-07, "epoch": 1, "t": 3.7465150356292725}
+{"step": 297, "loss": 1.0604114532470703, "grad_norm": 0.6269643902778625, "lr": 6.322973712755697e-07, "epoch": 1, "t": 3.352118492126465}
+{"step": 298, "loss": 1.2422224283218384, "grad_norm": 0.602403998374939, "lr": 5.509512889877333e-07, "epoch": 1, "t": 4.141459941864014}
+{"step": 299, "loss": 1.167026162147522, "grad_norm": 0.6394636631011963, "lr": 4.7517539898741524e-07, "epoch": 1, "t": 4.296651601791382}
+{"step": 300, "loss": 1.1355189085006714, "grad_norm": 0.582977294921875, "lr": 4.049782370561583e-07, "epoch": 1, "t": 3.8555452823638916}
+{"step": 301, "loss": 1.163539171218872, "grad_norm": 0.6136392951011658, "lr": 3.4036771055923066e-07, "epoch": 1, "t": 3.868520498275757}
+{"step": 302, "loss": 1.1838977336883545, "grad_norm": 0.6237985491752625, "lr": 2.813510975548772e-07, "epoch": 1, "t": 3.939094066619873}
+{"step": 303, "loss": 1.1358118057250977, "grad_norm": 0.5978771448135376, "lr": 2.2793504597447002e-07, "epoch": 1, "t": 4.025428533554077}
+{"step": 304, "loss": 1.2010633945465088, "grad_norm": 0.6446064710617065, "lr": 1.8012557287367392e-07, "epoch": 1, "t": 3.91121768951416}
+{"step": 305, "loss": 1.1053358316421509, "grad_norm": 0.5842671394348145, "lr": 1.379280637546443e-07, "epoch": 1, "t": 3.6552557945251465}
+{"step": 306, "loss": 1.152795433998108, "grad_norm": 0.5697402358055115, "lr": 1.0134727195937333e-07, "epoch": 1, "t": 3.8585729598999023}
+{"step": 307, "loss": 1.121507167816162, "grad_norm": 0.6407153606414795, "lr": 7.038731813426291e-08, "epoch": 1, "t": 3.782182455062866}
+{"step": 308, "loss": 1.2742674350738525, "grad_norm": 0.5971187353134155, "lr": 4.5051689765929214e-08, "epoch": 1, "t": 4.210731029510498}
+{"step": 309, "loss": 1.0135602951049805, "grad_norm": 0.5833330154418945, "lr": 2.534324078837802e-08, "epoch": 1, "t": 3.4627304077148438}
+{"step": 310, "loss": 1.171689510345459, "grad_norm": 0.5638537406921387, "lr": 1.1264191261528557e-08, "epoch": 1, "t": 4.0617101192474365}
+{"step": 311, "loss": 1.0656862258911133, "grad_norm": 0.6740520000457764, "lr": 2.8161271211024633e-09, "epoch": 1, "t": 3.8180243968963623}
+{"step": 312, "loss": 1.0350477695465088, "grad_norm": 0.5520936846733093, "lr": 0.0, "epoch": 1, "t": 3.6514415740966797}
diff --git a/12b/u_top-s0/README.md b/12b/u_top-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..0f2f243d3aceafe453065e108bf04a3768225947
--- /dev/null
+++ b/12b/u_top-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-12b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-12b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/12b/u_top-s0/adapter_config.json b/12b/u_top-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..eeb35af4c3bd0b55fb175111ffbd10b2bb2993e6
--- /dev/null
+++ b/12b/u_top-s0/adapter_config.json
@@ -0,0 +1,186 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-12b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "language_model.layers.7.self_attn.k_proj",
+    "39.self_attn.k_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "47.self_attn.k_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "46.self_attn.v_proj",
+    "43.self_attn.v_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "42.self_attn.k_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "42.self_attn.q_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "30.self_attn.v_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "31.self_attn.q_proj",
+    "30.self_attn.q_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "32.self_attn.k_proj",
+    "32.self_attn.v_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "39.self_attn.v_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "35.self_attn.v_proj",
+    "gate_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "47.self_attn.q_proj",
+    "35.self_attn.k_proj",
+    "38.self_attn.v_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "37.self_attn.k_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "up_proj",
+    "29.self_attn.v_proj",
+    "27.self_attn.q_proj",
+    "43.self_attn.k_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "45.self_attn.v_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "42.self_attn.v_proj",
+    "34.self_attn.q_proj",
+    "41.self_attn.k_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "down_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "language_model.layers.13.self_attn.v_proj",
+    "30.self_attn.k_proj",
+    "41.self_attn.q_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "46.self_attn.q_proj",
+    "36.self_attn.k_proj",
+    "47.self_attn.v_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "41.self_attn.v_proj",
+    "43.self_attn.q_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "37.self_attn.v_proj",
+    "45.self_attn.k_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "36.self_attn.v_proj",
+    "o_proj",
+    "29.self_attn.q_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "40.self_attn.q_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "44.self_attn.k_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "39.self_attn.q_proj",
+    "33.self_attn.q_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "45.self_attn.q_proj",
+    "34.self_attn.v_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "language_model.layers.19.self_attn.k_proj",
+    "35.self_attn.q_proj",
+    "31.self_attn.v_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "28.self_attn.v_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "36.self_attn.q_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "40.self_attn.v_proj",
+    "32.self_attn.q_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "language_model.layers.6.self_attn.v_proj",
+    "28.self_attn.q_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "38.self_attn.k_proj",
+    "27.self_attn.k_proj",
+    "language_model.layers.3.self_attn.k_proj",
+    "37.self_attn.q_proj",
+    "46.self_attn.k_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "31.self_attn.k_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "38.self_attn.q_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "34.self_attn.k_proj",
+    "27.self_attn.v_proj",
+    "44.self_attn.q_proj",
+    "28.self_attn.k_proj",
+    "33.self_attn.v_proj",
+    "44.self_attn.v_proj",
+    "29.self_attn.k_proj",
+    "40.self_attn.k_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "language_model.layers.20.self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/12b/u_top-s0/adapter_model.safetensors b/12b/u_top-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..6fd63bdf899df734cef370f6063f424ceab5a236
--- /dev/null
+++ b/12b/u_top-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df7c7b8cab8246a97653b9c5a771aef08e370cf04d0c91a45cc2ca6291bfba3c
+size 1047628488
diff --git a/12b/u_top-s0/config.json b/12b/u_top-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..086b1c898f8d6c49b512de26fef0dfc152f10322
--- /dev/null
+++ b/12b/u_top-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-12b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/arms/TASKW-u_top-gemma12.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma12/runs/u_top-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_12b_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 0,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 2
+}
\ No newline at end of file
diff --git a/12b/u_top-s0/log.jsonl b/12b/u_top-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..9ddbe8b64c5eadb26a9ed751c3e31cf9033f2f63
--- /dev/null
+++ b/12b/u_top-s0/log.jsonl
@@ -0,0 +1,312 @@
+{"step": 1, "loss": 2.327320098876953, "grad_norm": 4.479649543762207, "lr": 1.25e-05, "epoch": 1, "t": 3.2861361503601074}
+{"step": 2, "loss": 2.2081398963928223, "grad_norm": 4.171542644500732, "lr": 1.8750000000000002e-05, "epoch": 1, "t": 3.501610279083252}
+{"step": 3, "loss": 2.227512836456299, "grad_norm": 3.7014870643615723, "lr": 2.5e-05, "epoch": 1, "t": 3.3065903186798096}
+{"step": 4, "loss": 2.0284104347229004, "grad_norm": 3.0142970085144043, "lr": 3.125e-05, "epoch": 1, "t": 2.5298056602478027}
+{"step": 5, "loss": 1.8827426433563232, "grad_norm": 2.323382616043091, "lr": 3.7500000000000003e-05, "epoch": 1, "t": 3.4053468704223633}
+{"step": 6, "loss": 1.7148656845092773, "grad_norm": 1.4067871570587158, "lr": 4.375e-05, "epoch": 1, "t": 3.2648541927337646}
+{"step": 7, "loss": 1.6916370391845703, "grad_norm": 1.0447163581848145, "lr": 5e-05, "epoch": 1, "t": 2.7632486820220947}
+{"step": 8, "loss": 1.4887657165527344, "grad_norm": 0.8689322471618652, "lr": 5.6250000000000005e-05, "epoch": 1, "t": 2.8857762813568115}
+{"step": 9, "loss": 1.5974476337432861, "grad_norm": 0.9982807040214539, "lr": 6.25e-05, "epoch": 1, "t": 3.258718729019165}
+{"step": 10, "loss": 1.6818633079528809, "grad_norm": 1.0472453832626343, "lr": 6.875e-05, "epoch": 1, "t": 3.62207293510437}
+{"step": 11, "loss": 1.4479854106903076, "grad_norm": 0.9008411169052124, "lr": 7.500000000000001e-05, "epoch": 1, "t": 2.7511041164398193}
+{"step": 12, "loss": 1.4024055004119873, "grad_norm": 0.8531894087791443, "lr": 8.125000000000001e-05, "epoch": 1, "t": 2.960697889328003}
+{"step": 13, "loss": 1.2542465925216675, "grad_norm": 0.7195949554443359, "lr": 8.75e-05, "epoch": 1, "t": 2.882171869277954}
+{"step": 14, "loss": 1.5960586071014404, "grad_norm": 0.7588453888893127, "lr": 9.375e-05, "epoch": 1, "t": 2.744157075881958}
+{"step": 15, "loss": 1.5088458061218262, "grad_norm": 0.7630959749221802, "lr": 0.0001, "epoch": 1, "t": 2.864926338195801}
+{"step": 16, "loss": 1.4294493198394775, "grad_norm": 0.75345778465271, "lr": 0.0001, "epoch": 1, "t": 3.0488786697387695}
+{"step": 17, "loss": 1.3520256280899048, "grad_norm": 0.7315390706062317, "lr": 9.99971838728789e-05, "epoch": 1, "t": 3.106358528137207}
+{"step": 18, "loss": 1.3143670558929443, "grad_norm": 0.8493432998657227, "lr": 9.998873580873848e-05, "epoch": 1, "t": 2.571648597717285}
+{"step": 19, "loss": 1.3237378597259521, "grad_norm": 0.7623825669288635, "lr": 9.997465675921163e-05, "epoch": 1, "t": 3.074885845184326}
+{"step": 20, "loss": 1.321047067642212, "grad_norm": 0.6148121356964111, "lr": 9.995494831023409e-05, "epoch": 1, "t": 2.9477245807647705}
+{"step": 21, "loss": 1.2125921249389648, "grad_norm": 0.6126531958580017, "lr": 9.992961268186573e-05, "epoch": 1, "t": 3.16160249710083}
+{"step": 22, "loss": 1.3310753107070923, "grad_norm": 0.6271969079971313, "lr": 9.989865272804063e-05, "epoch": 1, "t": 3.856884479522705}
+{"step": 23, "loss": 1.0960006713867188, "grad_norm": 0.5843579173088074, "lr": 9.986207193624536e-05, "epoch": 1, "t": 3.6228508949279785}
+{"step": 24, "loss": 1.3347666263580322, "grad_norm": 0.5616052746772766, "lr": 9.981987442712633e-05, "epoch": 1, "t": 2.879272699356079}
+{"step": 25, "loss": 1.156449794769287, "grad_norm": 0.5829220414161682, "lr": 9.977206495402554e-05, "epoch": 1, "t": 2.981518030166626}
+{"step": 26, "loss": 1.3228797912597656, "grad_norm": 0.6559152603149414, "lr": 9.971864890244513e-05, "epoch": 1, "t": 3.409393787384033}
+{"step": 27, "loss": 1.4145439863204956, "grad_norm": 0.6042874455451965, "lr": 9.965963228944078e-05, "epoch": 1, "t": 3.3931119441986084}
+{"step": 28, "loss": 1.2591763734817505, "grad_norm": 0.6486919522285461, "lr": 9.959502176294383e-05, "epoch": 1, "t": 3.559197187423706}
+{"step": 29, "loss": 1.2655839920043945, "grad_norm": 0.7721403241157532, "lr": 9.95248246010126e-05, "epoch": 1, "t": 3.674147129058838}
+{"step": 30, "loss": 1.3437391519546509, "grad_norm": 0.5789604187011719, "lr": 9.944904871101228e-05, "epoch": 1, "t": 3.036595344543457}
+{"step": 31, "loss": 1.313507318496704, "grad_norm": 0.6066290736198425, "lr": 9.936770262872443e-05, "epoch": 1, "t": 2.792367935180664}
+{"step": 32, "loss": 1.2701749801635742, "grad_norm": 0.5581668615341187, "lr": 9.928079551738543e-05, "epoch": 1, "t": 3.64007568359375}
+{"step": 33, "loss": 1.2571465969085693, "grad_norm": 0.6207336783409119, "lr": 9.918833716665419e-05, "epoch": 1, "t": 2.840121030807495}
+{"step": 34, "loss": 1.1672500371932983, "grad_norm": 0.6164065599441528, "lr": 9.909033799150946e-05, "epoch": 1, "t": 3.0070223808288574}
+{"step": 35, "loss": 1.1275960206985474, "grad_norm": 0.6311043500900269, "lr": 9.898680903107666e-05, "epoch": 1, "t": 2.7110443115234375}
+{"step": 36, "loss": 1.2309033870697021, "grad_norm": 0.5920257568359375, "lr": 9.887776194738432e-05, "epoch": 1, "t": 3.2392303943634033}
+{"step": 37, "loss": 1.26517653465271, "grad_norm": 0.609675943851471, "lr": 9.876320902405042e-05, "epoch": 1, "t": 3.149461269378662}
+{"step": 38, "loss": 1.2478092908859253, "grad_norm": 0.6141680479049683, "lr": 9.864316316489873e-05, "epoch": 1, "t": 2.834200143814087}
+{"step": 39, "loss": 1.3558597564697266, "grad_norm": 0.6106985211372375, "lr": 9.851763789250525e-05, "epoch": 1, "t": 2.8861913681030273}
+{"step": 40, "loss": 1.225052833557129, "grad_norm": 0.590245246887207, "lr": 9.838664734667495e-05, "epoch": 1, "t": 2.9458816051483154}
+{"step": 41, "loss": 1.1565661430358887, "grad_norm": 0.5631913542747498, "lr": 9.825020628284896e-05, "epoch": 1, "t": 2.8674819469451904}
+{"step": 42, "loss": 1.2926607131958008, "grad_norm": 0.6024816632270813, "lr": 9.810833007044247e-05, "epoch": 1, "t": 2.8970460891723633}
+{"step": 43, "loss": 1.345505952835083, "grad_norm": 0.5541171431541443, "lr": 9.796103469111351e-05, "epoch": 1, "t": 3.6500465869903564}
+{"step": 44, "loss": 1.2472412586212158, "grad_norm": 0.6114717721939087, "lr": 9.780833673696254e-05, "epoch": 1, "t": 3.0729925632476807}
+{"step": 45, "loss": 1.1458070278167725, "grad_norm": 0.5363909602165222, "lr": 9.76502534086636e-05, "epoch": 1, "t": 2.9130849838256836}
+{"step": 46, "loss": 1.2604302167892456, "grad_norm": 0.5959209203720093, "lr": 9.74868025135266e-05, "epoch": 1, "t": 3.0661916732788086}
+{"step": 47, "loss": 1.293121576309204, "grad_norm": 0.618234395980835, "lr": 9.731800246349148e-05, "epoch": 1, "t": 3.0375189781188965}
+{"step": 48, "loss": 1.212491750717163, "grad_norm": 0.5881310105323792, "lr": 9.714387227305422e-05, "epoch": 1, "t": 2.948955535888672}
+{"step": 49, "loss": 1.2408185005187988, "grad_norm": 0.5597432255744934, "lr": 9.696443155712486e-05, "epoch": 1, "t": 3.847395181655884}
+{"step": 50, "loss": 1.266973853111267, "grad_norm": 0.609494149684906, "lr": 9.67797005288181e-05, "epoch": 1, "t": 3.2872307300567627}
+{"step": 51, "loss": 1.3280231952667236, "grad_norm": 0.5787083506584167, "lr": 9.65896999971763e-05, "epoch": 1, "t": 3.095487356185913}
+{"step": 52, "loss": 1.235647201538086, "grad_norm": 0.6278812289237976, "lr": 9.639445136482548e-05, "epoch": 1, "t": 2.957137107849121}
+{"step": 53, "loss": 1.2202039957046509, "grad_norm": 0.6624301671981812, "lr": 9.619397662556435e-05, "epoch": 1, "t": 2.855740547180176}
+{"step": 54, "loss": 1.2952053546905518, "grad_norm": 0.6566532254219055, "lr": 9.598829836188694e-05, "epoch": 1, "t": 2.848402738571167}
+{"step": 55, "loss": 1.2492526769638062, "grad_norm": 0.6289007663726807, "lr": 9.577743974243874e-05, "epoch": 1, "t": 2.9781887531280518}
+{"step": 56, "loss": 1.2058625221252441, "grad_norm": 0.594464898109436, "lr": 9.55614245194068e-05, "epoch": 1, "t": 3.2028605937957764}
+{"step": 57, "loss": 1.2552945613861084, "grad_norm": 0.6747407913208008, "lr": 9.534027702584425e-05, "epoch": 1, "t": 3.255425453186035}
+{"step": 58, "loss": 1.1330509185791016, "grad_norm": 0.5906953811645508, "lr": 9.511402217292926e-05, "epoch": 1, "t": 2.984585762023926}
+{"step": 59, "loss": 1.2708070278167725, "grad_norm": 0.6074764728546143, "lr": 9.488268544715896e-05, "epoch": 1, "t": 3.0130326747894287}
+{"step": 60, "loss": 1.1431752443313599, "grad_norm": 0.7789663076400757, "lr": 9.464629290747842e-05, "epoch": 1, "t": 3.494136333465576}
+{"step": 61, "loss": 1.4228079319000244, "grad_norm": 0.6221870183944702, "lr": 9.440487118234535e-05, "epoch": 1, "t": 3.056659460067749}
+{"step": 62, "loss": 1.2959685325622559, "grad_norm": 0.5757694244384766, "lr": 9.415844746673047e-05, "epoch": 1, "t": 2.6466715335845947}
+{"step": 63, "loss": 1.2407996654510498, "grad_norm": 0.5622316598892212, "lr": 9.390704951905411e-05, "epoch": 1, "t": 3.0520288944244385}
+{"step": 64, "loss": 1.3233704566955566, "grad_norm": 0.6001541018486023, "lr": 9.365070565805941e-05, "epoch": 1, "t": 3.242171049118042}
+{"step": 65, "loss": 1.1650807857513428, "grad_norm": 0.5802739262580872, "lr": 9.338944475962237e-05, "epoch": 1, "t": 3.045684337615967}
+{"step": 66, "loss": 1.2278456687927246, "grad_norm": 0.7396714091300964, "lr": 9.312329625349902e-05, "epoch": 1, "t": 2.904010057449341}
+{"step": 67, "loss": 1.2978794574737549, "grad_norm": 0.5690197348594666, "lr": 9.285229012001047e-05, "epoch": 1, "t": 3.4255576133728027}
+{"step": 68, "loss": 1.1597262620925903, "grad_norm": 0.5632451772689819, "lr": 9.257645688666556e-05, "epoch": 1, "t": 2.901581287384033}
+{"step": 69, "loss": 1.2125627994537354, "grad_norm": 0.6128668189048767, "lr": 9.22958276247223e-05, "epoch": 1, "t": 2.7937471866607666}
+{"step": 70, "loss": 1.343188762664795, "grad_norm": 0.5677367448806763, "lr": 9.201043394568773e-05, "epoch": 1, "t": 3.5425591468811035}
+{"step": 71, "loss": 1.0806968212127686, "grad_norm": 0.6341626048088074, "lr": 9.172030799775699e-05, "epoch": 1, "t": 2.7552688121795654}
+{"step": 72, "loss": 1.2531797885894775, "grad_norm": 0.5500874519348145, "lr": 9.142548246219212e-05, "epoch": 1, "t": 3.063446521759033}
+{"step": 73, "loss": 1.1263126134872437, "grad_norm": 0.6508458852767944, "lr": 9.112599054964057e-05, "epoch": 1, "t": 2.8892641067504883}
+{"step": 74, "loss": 1.1582095623016357, "grad_norm": 0.6102361083030701, "lr": 9.082186599639428e-05, "epoch": 1, "t": 2.879180908203125}
+{"step": 75, "loss": 1.179563045501709, "grad_norm": 0.5993869304656982, "lr": 9.051314306058933e-05, "epoch": 1, "t": 3.0629258155822754}
+{"step": 76, "loss": 1.18906569480896, "grad_norm": 0.6275805234909058, "lr": 9.019985651834703e-05, "epoch": 1, "t": 3.01584529876709}
+{"step": 77, "loss": 1.1941587924957275, "grad_norm": 0.5857914686203003, "lr": 8.988204165985649e-05, "epoch": 1, "t": 3.4651057720184326}
+{"step": 78, "loss": 1.2205612659454346, "grad_norm": 0.6189649701118469, "lr": 8.955973428539944e-05, "epoch": 1, "t": 2.9542999267578125}
+{"step": 79, "loss": 1.1894892454147339, "grad_norm": 0.5463715195655823, "lr": 8.923297070131737e-05, "epoch": 1, "t": 3.250633478164673}
+{"step": 80, "loss": 1.1691644191741943, "grad_norm": 0.6536669731140137, "lr": 8.890178771592199e-05, "epoch": 1, "t": 3.5509772300720215}
+{"step": 81, "loss": 1.1745002269744873, "grad_norm": 0.6115691661834717, "lr": 8.856622263534875e-05, "epoch": 1, "t": 3.4411978721618652}
+{"step": 82, "loss": 1.271688461303711, "grad_norm": 0.6372178792953491, "lr": 8.822631325935463e-05, "epoch": 1, "t": 3.5441648960113525}
+{"step": 83, "loss": 1.3316903114318848, "grad_norm": 0.5968702435493469, "lr": 8.788209787706015e-05, "epoch": 1, "t": 4.230627536773682}
+{"step": 84, "loss": 1.1684809923171997, "grad_norm": 0.637873113155365, "lr": 8.753361526263621e-05, "epoch": 1, "t": 3.8836686611175537}
+{"step": 85, "loss": 1.02353036403656, "grad_norm": 0.6712408661842346, "lr": 8.718090467093654e-05, "epoch": 1, "t": 3.0928750038146973}
+{"step": 86, "loss": 1.0785856246948242, "grad_norm": 0.5503003597259521, "lr": 8.682400583307562e-05, "epoch": 1, "t": 2.9400875568389893}
+{"step": 87, "loss": 1.3736032247543335, "grad_norm": 0.5776317119598389, "lr": 8.646295895195333e-05, "epoch": 1, "t": 3.0470528602600098}
+{"step": 88, "loss": 1.028544545173645, "grad_norm": 0.6153590083122253, "lr": 8.609780469772623e-05, "epoch": 1, "t": 2.8241100311279297}
+{"step": 89, "loss": 1.234584927558899, "grad_norm": 0.5607485771179199, "lr": 8.572858420322627e-05, "epoch": 1, "t": 2.953537702560425}
+{"step": 90, "loss": 1.2346268892288208, "grad_norm": 0.5581977367401123, "lr": 8.535533905932738e-05, "epoch": 1, "t": 3.8440258502960205}
+{"step": 91, "loss": 1.221160650253296, "grad_norm": 0.5699642896652222, "lr": 8.497811131026046e-05, "epoch": 1, "t": 4.0340259075164795}
+{"step": 92, "loss": 1.2078444957733154, "grad_norm": 0.5951607823371887, "lr": 8.459694344887732e-05, "epoch": 1, "t": 3.017730712890625}
+{"step": 93, "loss": 1.1464855670928955, "grad_norm": 0.6039294600486755, "lr": 8.421187841186402e-05, "epoch": 1, "t": 2.7073140144348145}
+{"step": 94, "loss": 1.305588960647583, "grad_norm": 0.531472384929657, "lr": 8.382295957490436e-05, "epoch": 1, "t": 3.342186212539673}
+{"step": 95, "loss": 1.2411593198776245, "grad_norm": 0.5925220251083374, "lr": 8.343023074779368e-05, "epoch": 1, "t": 3.294947862625122}
+{"step": 96, "loss": 1.2272279262542725, "grad_norm": 0.6295038461685181, "lr": 8.303373616950408e-05, "epoch": 1, "t": 3.0799553394317627}
+{"step": 97, "loss": 1.2100359201431274, "grad_norm": 0.9218403697013855, "lr": 8.263352050320094e-05, "epoch": 1, "t": 3.069793224334717}
+{"step": 98, "loss": 1.1928156614303589, "grad_norm": 0.5811665058135986, "lr": 8.222962883121196e-05, "epoch": 1, "t": 3.0220353603363037}
+{"step": 99, "loss": 1.2326633930206299, "grad_norm": 0.6870810389518738, "lr": 8.182210664994878e-05, "epoch": 1, "t": 2.5364246368408203}
+{"step": 100, "loss": 1.1224595308303833, "grad_norm": 0.5607088208198547, "lr": 8.141099986478212e-05, "epoch": 1, "t": 3.1791491508483887}
+{"step": 101, "loss": 1.1015534400939941, "grad_norm": 0.5766206979751587, "lr": 8.099635478487064e-05, "epoch": 1, "t": 3.582705020904541}
+{"step": 102, "loss": 1.1344712972640991, "grad_norm": 0.5696788430213928, "lr": 8.057821811794458e-05, "epoch": 1, "t": 3.8053183555603027}
+{"step": 103, "loss": 1.1657072305679321, "grad_norm": 0.6360107064247131, "lr": 8.015663696504422e-05, "epoch": 1, "t": 3.0161685943603516}
+{"step": 104, "loss": 1.0946083068847656, "grad_norm": 0.5309562087059021, "lr": 7.973165881521434e-05, "epoch": 1, "t": 4.169360876083374}
+{"step": 105, "loss": 1.1223301887512207, "grad_norm": 0.6696287989616394, "lr": 7.930333154015466e-05, "epoch": 1, "t": 3.631697416305542}
+{"step": 106, "loss": 1.2191174030303955, "grad_norm": 0.5754199624061584, "lr": 7.88717033888274e-05, "epoch": 1, "t": 2.7675325870513916}
+{"step": 107, "loss": 1.211332082748413, "grad_norm": 0.646818220615387, "lr": 7.843682298202235e-05, "epoch": 1, "t": 2.8578171730041504}
+{"step": 108, "loss": 1.2114726305007935, "grad_norm": 0.6245731711387634, "lr": 7.799873930687978e-05, "epoch": 1, "t": 2.990239143371582}
+{"step": 109, "loss": 1.279434323310852, "grad_norm": 0.5976760387420654, "lr": 7.755750171137246e-05, "epoch": 1, "t": 3.264981746673584}
+{"step": 110, "loss": 1.245403528213501, "grad_norm": 0.6508705019950867, "lr": 7.711315989874677e-05, "epoch": 1, "t": 2.827348470687866}
+{"step": 111, "loss": 1.24924635887146, "grad_norm": 0.5850361585617065, "lr": 7.666576392192389e-05, "epoch": 1, "t": 3.0938448905944824}
+{"step": 112, "loss": 1.0820833444595337, "grad_norm": 0.6018866896629333, "lr": 7.621536417786159e-05, "epoch": 1, "t": 2.8252952098846436}
+{"step": 113, "loss": 1.273302674293518, "grad_norm": 0.6372866630554199, "lr": 7.576201140187727e-05, "epoch": 1, "t": 2.571823835372925}
+{"step": 114, "loss": 1.1704981327056885, "grad_norm": 0.5573928356170654, "lr": 7.530575666193283e-05, "epoch": 1, "t": 3.1482512950897217}
+{"step": 115, "loss": 1.1771819591522217, "grad_norm": 0.6436694264411926, "lr": 7.484665135288213e-05, "epoch": 1, "t": 2.8885557651519775}
+{"step": 116, "loss": 1.045685052871704, "grad_norm": 0.6081012487411499, "lr": 7.438474719068173e-05, "epoch": 1, "t": 2.722137212753296}
+{"step": 117, "loss": 1.1190414428710938, "grad_norm": 0.7006279826164246, "lr": 7.392009620656513e-05, "epoch": 1, "t": 3.499424934387207}
+{"step": 118, "loss": 1.1154733896255493, "grad_norm": 0.7012538313865662, "lr": 7.345275074118185e-05, "epoch": 1, "t": 3.7546942234039307}
+{"step": 119, "loss": 1.1233445405960083, "grad_norm": 0.5250251889228821, "lr": 7.298276343870151e-05, "epoch": 1, "t": 3.8036599159240723}
+{"step": 120, "loss": 1.1060562133789062, "grad_norm": 0.5347779393196106, "lr": 7.251018724088367e-05, "epoch": 1, "t": 2.9217615127563477}
+{"step": 121, "loss": 1.063027262687683, "grad_norm": 0.6037988066673279, "lr": 7.203507538111423e-05, "epoch": 1, "t": 2.8044254779815674}
+{"step": 122, "loss": 1.1519885063171387, "grad_norm": 0.6173638701438904, "lr": 7.155748137840892e-05, "epoch": 1, "t": 3.165515422821045}
+{"step": 123, "loss": 1.1948976516723633, "grad_norm": 0.569068193435669, "lr": 7.107745903138472e-05, "epoch": 1, "t": 2.971468210220337}
+{"step": 124, "loss": 1.233440637588501, "grad_norm": 0.5795004963874817, "lr": 7.059506241219965e-05, "epoch": 1, "t": 3.1745150089263916}
+{"step": 125, "loss": 1.0947480201721191, "grad_norm": 0.595154881477356, "lr": 7.011034586046176e-05, "epoch": 1, "t": 2.9281632900238037}
+{"step": 126, "loss": 1.1543587446212769, "grad_norm": 0.6121411919593811, "lr": 6.962336397710819e-05, "epoch": 1, "t": 2.9809746742248535}
+{"step": 127, "loss": 1.1404320001602173, "grad_norm": 0.6786699891090393, "lr": 6.91341716182545e-05, "epoch": 1, "t": 2.6360747814178467}
+{"step": 128, "loss": 1.155851125717163, "grad_norm": 0.6679185628890991, "lr": 6.864282388901544e-05, "epoch": 1, "t": 2.8419406414031982}
+{"step": 129, "loss": 1.1627235412597656, "grad_norm": 0.5989121198654175, "lr": 6.814937613729766e-05, "epoch": 1, "t": 3.0893776416778564}
+{"step": 130, "loss": 1.1237167119979858, "grad_norm": 0.6347812414169312, "lr": 6.765388394756504e-05, "epoch": 1, "t": 2.6416707038879395}
+{"step": 131, "loss": 1.1113510131835938, "grad_norm": 0.5990824103355408, "lr": 6.715640313457733e-05, "epoch": 1, "t": 3.090550422668457}
+{"step": 132, "loss": 1.1517996788024902, "grad_norm": 0.5490580201148987, "lr": 6.665698973710288e-05, "epoch": 1, "t": 3.0941805839538574}
+{"step": 133, "loss": 1.0341966152191162, "grad_norm": 0.5785680413246155, "lr": 6.615570001160626e-05, "epoch": 1, "t": 2.6230223178863525}
+{"step": 134, "loss": 1.1920065879821777, "grad_norm": 1.1524473428726196, "lr": 6.565259042591113e-05, "epoch": 1, "t": 3.7972874641418457}
+{"step": 135, "loss": 1.1461162567138672, "grad_norm": 0.6095324158668518, "lr": 6.514771765283942e-05, "epoch": 1, "t": 2.835585355758667}
+{"step": 136, "loss": 1.1170728206634521, "grad_norm": 0.6389683485031128, "lr": 6.464113856382752e-05, "epoch": 1, "t": 2.8336169719696045}
+{"step": 137, "loss": 1.1332333087921143, "grad_norm": 0.524041473865509, "lr": 6.413291022251989e-05, "epoch": 1, "t": 3.0706753730773926}
+{"step": 138, "loss": 1.272542953491211, "grad_norm": 0.5762870907783508, "lr": 6.362308987834115e-05, "epoch": 1, "t": 3.2054896354675293}
+{"step": 139, "loss": 1.0302335023880005, "grad_norm": 0.5762219429016113, "lr": 6.311173496004723e-05, "epoch": 1, "t": 2.955650806427002}
+{"step": 140, "loss": 1.2442721128463745, "grad_norm": 0.7866568565368652, "lr": 6.259890306925627e-05, "epoch": 1, "t": 2.8068923950195312}
+{"step": 141, "loss": 1.1479132175445557, "grad_norm": 0.9430035352706909, "lr": 6.208465197396013e-05, "epoch": 1, "t": 3.0794246196746826}
+{"step": 142, "loss": 1.0746541023254395, "grad_norm": 0.6633681654930115, "lr": 6.156903960201709e-05, "epoch": 1, "t": 2.687844753265381}
+{"step": 143, "loss": 1.2046515941619873, "grad_norm": 0.5880517959594727, "lr": 6.105212403462651e-05, "epoch": 1, "t": 2.9938511848449707}
+{"step": 144, "loss": 1.2630140781402588, "grad_norm": 0.6426060199737549, "lr": 6.0533963499786314e-05, "epoch": 1, "t": 2.9359235763549805}
+{"step": 145, "loss": 1.1653432846069336, "grad_norm": 0.6175225973129272, "lr": 6.001461636573397e-05, "epoch": 1, "t": 2.892186403274536}
+{"step": 146, "loss": 1.1872620582580566, "grad_norm": 0.5812613368034363, "lr": 5.949414113437142e-05, "epoch": 1, "t": 2.917402982711792}
+{"step": 147, "loss": 1.1645009517669678, "grad_norm": 0.6006962656974792, "lr": 5.897259643467527e-05, "epoch": 1, "t": 2.668782949447632}
+{"step": 148, "loss": 1.1539387702941895, "grad_norm": 0.6010288000106812, "lr": 5.8450041016092464e-05, "epoch": 1, "t": 3.1453843116760254}
+{"step": 149, "loss": 1.1703591346740723, "grad_norm": 0.6749489903450012, "lr": 5.792653374192245e-05, "epoch": 1, "t": 2.8920865058898926}
+{"step": 150, "loss": 1.1951923370361328, "grad_norm": 0.5683267116546631, "lr": 5.7402133582686576e-05, "epoch": 1, "t": 3.401418924331665}
+{"step": 151, "loss": 1.1723201274871826, "grad_norm": 0.6249712109565735, "lr": 5.6876899609485256e-05, "epoch": 1, "t": 3.070005416870117}
+{"step": 152, "loss": 1.1182010173797607, "grad_norm": 0.5364521741867065, "lr": 5.6350890987343944e-05, "epoch": 1, "t": 3.325937271118164}
+{"step": 153, "loss": 1.2794897556304932, "grad_norm": 0.5948210954666138, "lr": 5.582416696854853e-05, "epoch": 1, "t": 3.2955052852630615}
+{"step": 154, "loss": 1.182908535003662, "grad_norm": 0.6371079087257385, "lr": 5.5296786885970805e-05, "epoch": 1, "t": 2.9407782554626465}
+{"step": 155, "loss": 1.1516321897506714, "grad_norm": 0.5899915099143982, "lr": 5.476881014638491e-05, "epoch": 1, "t": 3.187077760696411}
+{"step": 156, "loss": 1.2564526796340942, "grad_norm": 0.5600513219833374, "lr": 5.4240296223775465e-05, "epoch": 1, "t": 3.23352313041687}
+{"step": 157, "loss": 1.1394145488739014, "grad_norm": 0.5550705790519714, "lr": 5.3711304652638126e-05, "epoch": 1, "t": 2.8787662982940674}
+{"step": 158, "loss": 1.051509141921997, "grad_norm": 0.6737907528877258, "lr": 5.318189502127332e-05, "epoch": 1, "t": 2.750173330307007}
+{"step": 159, "loss": 1.1336908340454102, "grad_norm": 0.5587016940116882, "lr": 5.265212696507387e-05, "epoch": 1, "t": 3.657839775085449}
+{"step": 160, "loss": 1.2395141124725342, "grad_norm": 0.5872310996055603, "lr": 5.212206015980742e-05, "epoch": 1, "t": 3.544222354888916}
+{"step": 161, "loss": 1.1271936893463135, "grad_norm": 0.6408737897872925, "lr": 5.159175431489424e-05, "epoch": 1, "t": 3.038511037826538}
+{"step": 162, "loss": 1.1497070789337158, "grad_norm": 0.6039360761642456, "lr": 5.1061269166681183e-05, "epoch": 1, "t": 3.5537288188934326}
+{"step": 163, "loss": 1.0098364353179932, "grad_norm": 0.5929713845252991, "lr": 5.053066447171282e-05, "epoch": 1, "t": 3.231069326400757}
+{"step": 164, "loss": 1.1784789562225342, "grad_norm": 0.6200481653213501, "lr": 5e-05, "epoch": 1, "t": 2.913841485977173}
+{"step": 165, "loss": 1.071203351020813, "grad_norm": 0.5899565815925598, "lr": 4.94693355282872e-05, "epoch": 1, "t": 2.81854248046875}
+{"step": 166, "loss": 1.1921260356903076, "grad_norm": 0.5644988417625427, "lr": 4.893873083331882e-05, "epoch": 1, "t": 3.1507163047790527}
+{"step": 167, "loss": 1.195814609527588, "grad_norm": 0.5495814085006714, "lr": 4.840824568510579e-05, "epoch": 1, "t": 3.3438429832458496}
+{"step": 168, "loss": 1.1616824865341187, "grad_norm": 0.6372646689414978, "lr": 4.78779398401926e-05, "epoch": 1, "t": 2.698124647140503}
+{"step": 169, "loss": 1.1461269855499268, "grad_norm": 0.6043485999107361, "lr": 4.734787303492615e-05, "epoch": 1, "t": 3.0136756896972656}
+{"step": 170, "loss": 1.2103044986724854, "grad_norm": 0.5746743083000183, "lr": 4.6818104978726685e-05, "epoch": 1, "t": 2.940464735031128}
+{"step": 171, "loss": 1.228811502456665, "grad_norm": 0.5922628045082092, "lr": 4.628869534736187e-05, "epoch": 1, "t": 2.9829726219177246}
+{"step": 172, "loss": 1.0523931980133057, "grad_norm": 0.7201933264732361, "lr": 4.575970377622456e-05, "epoch": 1, "t": 2.718374490737915}
+{"step": 173, "loss": 1.1357719898223877, "grad_norm": 0.5851925611495972, "lr": 4.52311898536151e-05, "epoch": 1, "t": 2.869870662689209}
+{"step": 174, "loss": 1.1766884326934814, "grad_norm": 0.5439721345901489, "lr": 4.47032131140292e-05, "epoch": 1, "t": 2.9001355171203613}
+{"step": 175, "loss": 1.1772966384887695, "grad_norm": 0.6422854065895081, "lr": 4.4175833031451473e-05, "epoch": 1, "t": 3.022507429122925}
+{"step": 176, "loss": 1.111973762512207, "grad_norm": 0.5467185378074646, "lr": 4.364910901265606e-05, "epoch": 1, "t": 3.116711139678955}
+{"step": 177, "loss": 1.1613274812698364, "grad_norm": 0.5875517725944519, "lr": 4.3123100390514756e-05, "epoch": 1, "t": 3.049374580383301}
+{"step": 178, "loss": 1.0332989692687988, "grad_norm": 0.5855357646942139, "lr": 4.2597866417313436e-05, "epoch": 1, "t": 2.79056715965271}
+{"step": 179, "loss": 1.1929221153259277, "grad_norm": 0.5956239700317383, "lr": 4.207346625807756e-05, "epoch": 1, "t": 2.861937999725342}
+{"step": 180, "loss": 1.1915086507797241, "grad_norm": 0.569830596446991, "lr": 4.1549958983907555e-05, "epoch": 1, "t": 3.3062782287597656}
+{"step": 181, "loss": 1.1634786128997803, "grad_norm": 0.6269698143005371, "lr": 4.102740356532473e-05, "epoch": 1, "t": 2.8638885021209717}
+{"step": 182, "loss": 1.2165220975875854, "grad_norm": 0.600116491317749, "lr": 4.050585886562858e-05, "epoch": 1, "t": 2.9708757400512695}
+{"step": 183, "loss": 1.2598357200622559, "grad_norm": 0.596135139465332, "lr": 3.998538363426605e-05, "epoch": 1, "t": 2.998049259185791}
+{"step": 184, "loss": 1.2533855438232422, "grad_norm": 0.5841607451438904, "lr": 3.94660365002137e-05, "epoch": 1, "t": 3.2108540534973145}
+{"step": 185, "loss": 1.1295108795166016, "grad_norm": 0.5636579394340515, "lr": 3.894787596537352e-05, "epoch": 1, "t": 3.568476438522339}
+{"step": 186, "loss": 1.1769416332244873, "grad_norm": 0.5928946137428284, "lr": 3.843096039798293e-05, "epoch": 1, "t": 3.7972066402435303}
+{"step": 187, "loss": 1.184971809387207, "grad_norm": 0.6448925733566284, "lr": 3.791534802603988e-05, "epoch": 1, "t": 3.8166792392730713}
+{"step": 188, "loss": 1.101743221282959, "grad_norm": 0.6474359035491943, "lr": 3.740109693074375e-05, "epoch": 1, "t": 3.076479911804199}
+{"step": 189, "loss": 1.2391080856323242, "grad_norm": 0.5568802356719971, "lr": 3.68882650399528e-05, "epoch": 1, "t": 3.1881744861602783}
+{"step": 190, "loss": 1.3008253574371338, "grad_norm": 0.5778487324714661, "lr": 3.637691012165886e-05, "epoch": 1, "t": 2.9614386558532715}
+{"step": 191, "loss": 1.0278034210205078, "grad_norm": 0.6379583477973938, "lr": 3.586708977748012e-05, "epoch": 1, "t": 2.6086511611938477}
+{"step": 192, "loss": 1.129561424255371, "grad_norm": 0.973800539970398, "lr": 3.5358861436172485e-05, "epoch": 1, "t": 3.2664294242858887}
+{"step": 193, "loss": 1.1028861999511719, "grad_norm": 0.5966739654541016, "lr": 3.485228234716058e-05, "epoch": 1, "t": 3.088294744491577}
+{"step": 194, "loss": 1.1875022649765015, "grad_norm": 0.5896755456924438, "lr": 3.434740957408889e-05, "epoch": 1, "t": 3.25018310546875}
+{"step": 195, "loss": 1.0337897539138794, "grad_norm": 0.5915784239768982, "lr": 3.3844299988393755e-05, "epoch": 1, "t": 2.7899084091186523}
+{"step": 196, "loss": 1.160801649093628, "grad_norm": 0.5271362066268921, "lr": 3.334301026289712e-05, "epoch": 1, "t": 3.240246057510376}
+{"step": 197, "loss": 1.1071751117706299, "grad_norm": 0.5836171507835388, "lr": 3.284359686542269e-05, "epoch": 1, "t": 3.0713067054748535}
+{"step": 198, "loss": 1.0015764236450195, "grad_norm": 0.6265120506286621, "lr": 3.234611605243496e-05, "epoch": 1, "t": 2.8571619987487793}
+{"step": 199, "loss": 1.087841510772705, "grad_norm": 0.6286867260932922, "lr": 3.1850623862702344e-05, "epoch": 1, "t": 2.7725703716278076}
+{"step": 200, "loss": 1.2072763442993164, "grad_norm": 0.5912929177284241, "lr": 3.135717611098458e-05, "epoch": 1, "t": 2.8430607318878174}
+{"step": 201, "loss": 1.2555158138275146, "grad_norm": 1.2110157012939453, "lr": 3.086582838174551e-05, "epoch": 1, "t": 2.8705735206604004}
+{"step": 202, "loss": 1.135838508605957, "grad_norm": 0.5537793636322021, "lr": 3.0376636022891812e-05, "epoch": 1, "t": 3.2255771160125732}
+{"step": 203, "loss": 1.0684289932250977, "grad_norm": 0.5984201431274414, "lr": 2.9889654139538246e-05, "epoch": 1, "t": 2.8835365772247314}
+{"step": 204, "loss": 1.1376891136169434, "grad_norm": 0.5949271321296692, "lr": 2.9404937587800375e-05, "epoch": 1, "t": 3.280303955078125}
+{"step": 205, "loss": 1.1624929904937744, "grad_norm": 0.6727084517478943, "lr": 2.8922540968615286e-05, "epoch": 1, "t": 3.2003767490386963}
+{"step": 206, "loss": 1.0737102031707764, "grad_norm": 0.5915781259536743, "lr": 2.8442518621591086e-05, "epoch": 1, "t": 3.4607651233673096}
+{"step": 207, "loss": 1.183335781097412, "grad_norm": 0.6108809113502502, "lr": 2.7964924618885778e-05, "epoch": 1, "t": 3.5723419189453125}
+{"step": 208, "loss": 1.1707769632339478, "grad_norm": 0.5988962650299072, "lr": 2.748981275911633e-05, "epoch": 1, "t": 3.5454084873199463}
+{"step": 209, "loss": 1.201794147491455, "grad_norm": 0.6710598468780518, "lr": 2.701723656129851e-05, "epoch": 1, "t": 3.2738630771636963}
+{"step": 210, "loss": 1.1193221807479858, "grad_norm": 0.5657229423522949, "lr": 2.6547249258818164e-05, "epoch": 1, "t": 3.34323787689209}
+{"step": 211, "loss": 1.118709683418274, "grad_norm": 0.5916733741760254, "lr": 2.607990379343489e-05, "epoch": 1, "t": 2.95308518409729}
+{"step": 212, "loss": 1.0351365804672241, "grad_norm": 0.6170698404312134, "lr": 2.5615252809318284e-05, "epoch": 1, "t": 2.4724950790405273}
+{"step": 213, "loss": 1.2519627809524536, "grad_norm": 0.6038798093795776, "lr": 2.5153348647117857e-05, "epoch": 1, "t": 3.1961545944213867}
+{"step": 214, "loss": 1.1599771976470947, "grad_norm": 0.5715116262435913, "lr": 2.469424333806718e-05, "epoch": 1, "t": 3.722592830657959}
+{"step": 215, "loss": 1.0412877798080444, "grad_norm": 0.5874286890029907, "lr": 2.4237988598122752e-05, "epoch": 1, "t": 2.9984657764434814}
+{"step": 216, "loss": 1.1521810293197632, "grad_norm": 0.6235586404800415, "lr": 2.3784635822138424e-05, "epoch": 1, "t": 2.7904958724975586}
+{"step": 217, "loss": 1.1785573959350586, "grad_norm": 0.5680446624755859, "lr": 2.333423607807613e-05, "epoch": 1, "t": 3.3524277210235596}
+{"step": 218, "loss": 1.0834698677062988, "grad_norm": 0.6398513317108154, "lr": 2.288684010125325e-05, "epoch": 1, "t": 3.1025888919830322}
+{"step": 219, "loss": 1.2224993705749512, "grad_norm": 0.638129472732544, "lr": 2.2442498288627556e-05, "epoch": 1, "t": 2.690300464630127}
+{"step": 220, "loss": 1.0135873556137085, "grad_norm": 0.6258672475814819, "lr": 2.2001260693120233e-05, "epoch": 1, "t": 2.9463844299316406}
+{"step": 221, "loss": 1.1410417556762695, "grad_norm": 0.5959324836730957, "lr": 2.156317701797766e-05, "epoch": 1, "t": 3.2205443382263184}
+{"step": 222, "loss": 1.1339473724365234, "grad_norm": 0.5913580060005188, "lr": 2.1128296611172593e-05, "epoch": 1, "t": 2.775597333908081}
+{"step": 223, "loss": 1.1689562797546387, "grad_norm": 0.6834553480148315, "lr": 2.0696668459845355e-05, "epoch": 1, "t": 2.756077766418457}
+{"step": 224, "loss": 1.2905902862548828, "grad_norm": 0.6368185877799988, "lr": 2.026834118478567e-05, "epoch": 1, "t": 2.8262813091278076}
+{"step": 225, "loss": 1.091238021850586, "grad_norm": 0.6023615002632141, "lr": 1.98433630349558e-05, "epoch": 1, "t": 3.2379612922668457}
+{"step": 226, "loss": 1.0142135620117188, "grad_norm": 0.5999441146850586, "lr": 1.9421781882055444e-05, "epoch": 1, "t": 2.444869041442871}
+{"step": 227, "loss": 1.1902837753295898, "grad_norm": 0.6179777979850769, "lr": 1.9003645215129355e-05, "epoch": 1, "t": 2.9736571311950684}
+{"step": 228, "loss": 1.2815968990325928, "grad_norm": 0.6229971051216125, "lr": 1.858900013521788e-05, "epoch": 1, "t": 2.9956436157226562}
+{"step": 229, "loss": 1.0711586475372314, "grad_norm": 0.5232000350952148, "lr": 1.817789335005121e-05, "epoch": 1, "t": 3.176826000213623}
+{"step": 230, "loss": 1.1901986598968506, "grad_norm": 0.5701279044151306, "lr": 1.777037116878804e-05, "epoch": 1, "t": 3.0356714725494385}
+{"step": 231, "loss": 1.1843186616897583, "grad_norm": 0.5854154825210571, "lr": 1.7366479496799077e-05, "epoch": 1, "t": 3.384005308151245}
+{"step": 232, "loss": 1.1225934028625488, "grad_norm": 0.5837982892990112, "lr": 1.6966263830495936e-05, "epoch": 1, "t": 3.1160361766815186}
+{"step": 233, "loss": 1.1468838453292847, "grad_norm": 0.5523653030395508, "lr": 1.656976925220633e-05, "epoch": 1, "t": 3.0317800045013428}
+{"step": 234, "loss": 1.1522216796875, "grad_norm": 0.6105077862739563, "lr": 1.6177040425095662e-05, "epoch": 1, "t": 3.1769988536834717}
+{"step": 235, "loss": 1.2667152881622314, "grad_norm": 0.6007160544395447, "lr": 1.5788121588135975e-05, "epoch": 1, "t": 3.131359577178955}
+{"step": 236, "loss": 1.167557954788208, "grad_norm": 0.5950906872749329, "lr": 1.5403056551122697e-05, "epoch": 1, "t": 2.78052020072937}
+{"step": 237, "loss": 1.2074017524719238, "grad_norm": 0.5569764375686646, "lr": 1.5021888689739549e-05, "epoch": 1, "t": 3.2012205123901367}
+{"step": 238, "loss": 1.225684642791748, "grad_norm": 0.6010000109672546, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 2.985168218612671}
+{"step": 239, "loss": 1.170417308807373, "grad_norm": 0.557050347328186, "lr": 1.427141579677374e-05, "epoch": 1, "t": 2.859895706176758}
+{"step": 240, "loss": 1.1187412738800049, "grad_norm": 0.5575351715087891, "lr": 1.3902195302273779e-05, "epoch": 1, "t": 3.2264151573181152}
+{"step": 241, "loss": 1.1345183849334717, "grad_norm": 0.549464225769043, "lr": 1.3537041048046695e-05, "epoch": 1, "t": 3.9968514442443848}
+{"step": 242, "loss": 1.137977123260498, "grad_norm": 0.6949043869972229, "lr": 1.3175994166924394e-05, "epoch": 1, "t": 3.653144359588623}
+{"step": 243, "loss": 1.0220472812652588, "grad_norm": 0.5876064896583557, "lr": 1.2819095329063469e-05, "epoch": 1, "t": 2.7525410652160645}
+{"step": 244, "loss": 1.116325855255127, "grad_norm": 0.6986975073814392, "lr": 1.246638473736378e-05, "epoch": 1, "t": 2.666261672973633}
+{"step": 245, "loss": 1.1693358421325684, "grad_norm": 0.616879403591156, "lr": 1.2117902122939861e-05, "epoch": 1, "t": 3.0750532150268555}
+{"step": 246, "loss": 1.2149481773376465, "grad_norm": 0.5918919444084167, "lr": 1.1773686740645384e-05, "epoch": 1, "t": 3.019662380218506}
+{"step": 247, "loss": 1.0392438173294067, "grad_norm": 0.6021108031272888, "lr": 1.1433777364651271e-05, "epoch": 1, "t": 2.9354453086853027}
+{"step": 248, "loss": 1.074671745300293, "grad_norm": 0.6722363829612732, "lr": 1.1098212284078036e-05, "epoch": 1, "t": 2.7251389026641846}
+{"step": 249, "loss": 1.076608657836914, "grad_norm": 0.6062939763069153, "lr": 1.076702929868264e-05, "epoch": 1, "t": 2.744764804840088}
+{"step": 250, "loss": 1.0962060689926147, "grad_norm": 0.7649754881858826, "lr": 1.0440265714600572e-05, "epoch": 1, "t": 2.9197442531585693}
+{"step": 251, "loss": 1.1859543323516846, "grad_norm": 2.485170841217041, "lr": 1.0117958340143507e-05, "epoch": 1, "t": 2.97395920753479}
+{"step": 252, "loss": 1.125368595123291, "grad_norm": 0.554166853427887, "lr": 9.800143481652979e-06, "epoch": 1, "t": 2.7946290969848633}
+{"step": 253, "loss": 1.1659224033355713, "grad_norm": 0.6094629764556885, "lr": 9.48685693941067e-06, "epoch": 1, "t": 3.154022693634033}
+{"step": 254, "loss": 1.1755930185317993, "grad_norm": 0.5666871666908264, "lr": 9.17813400360572e-06, "epoch": 1, "t": 3.3690907955169678}
+{"step": 255, "loss": 1.1816956996917725, "grad_norm": 0.6629977226257324, "lr": 8.874009450359427e-06, "epoch": 1, "t": 2.902395009994507}
+{"step": 256, "loss": 1.1166942119598389, "grad_norm": 0.5355186462402344, "lr": 8.574517537807897e-06, "epoch": 1, "t": 2.924907684326172}
+{"step": 257, "loss": 1.1063801050186157, "grad_norm": 0.6152694225311279, "lr": 8.279692002243027e-06, "epoch": 1, "t": 2.7072699069976807}
+{"step": 258, "loss": 1.1644654273986816, "grad_norm": 0.5693303346633911, "lr": 7.989566054312287e-06, "epoch": 1, "t": 3.244687557220459}
+{"step": 259, "loss": 1.0223653316497803, "grad_norm": 0.7570959329605103, "lr": 7.704172375277691e-06, "epoch": 1, "t": 3.200397253036499}
+{"step": 260, "loss": 1.155961036682129, "grad_norm": 0.5627315640449524, "lr": 7.423543113334436e-06, "epoch": 1, "t": 2.7793288230895996}
+{"step": 261, "loss": 1.1179251670837402, "grad_norm": 0.5671914219856262, "lr": 7.14770987998954e-06, "epoch": 1, "t": 3.147878408432007}
+{"step": 262, "loss": 1.1854851245880127, "grad_norm": 0.5572956204414368, "lr": 6.876703746500984e-06, "epoch": 1, "t": 3.0994369983673096}
+{"step": 263, "loss": 1.2007200717926025, "grad_norm": 0.5846613049507141, "lr": 6.610555240377652e-06, "epoch": 1, "t": 3.171034336090088}
+{"step": 264, "loss": 1.1408512592315674, "grad_norm": 0.664691150188446, "lr": 6.349294341940593e-06, "epoch": 1, "t": 3.384168863296509}
+{"step": 265, "loss": 1.2238574028015137, "grad_norm": 0.5925254225730896, "lr": 6.092950480945897e-06, "epoch": 1, "t": 3.292616844177246}
+{"step": 266, "loss": 1.1121325492858887, "grad_norm": 0.5988463163375854, "lr": 5.841552533269534e-06, "epoch": 1, "t": 3.1106224060058594}
+{"step": 267, "loss": 1.2561395168304443, "grad_norm": 0.6033052802085876, "lr": 5.595128817654638e-06, "epoch": 1, "t": 2.7776708602905273}
+{"step": 268, "loss": 1.1389638185501099, "grad_norm": 0.5153255462646484, "lr": 5.353707092521582e-06, "epoch": 1, "t": 3.389796018600464}
+{"step": 269, "loss": 1.104341745376587, "grad_norm": 0.5882940888404846, "lr": 5.117314552841052e-06, "epoch": 1, "t": 2.9526240825653076}
+{"step": 270, "loss": 1.2025060653686523, "grad_norm": 0.6754514575004578, "lr": 4.885977827070748e-06, "epoch": 1, "t": 2.6321189403533936}
+{"step": 271, "loss": 1.1364171504974365, "grad_norm": 0.5835061073303223, "lr": 4.659722974155767e-06, "epoch": 1, "t": 2.9257829189300537}
+{"step": 272, "loss": 1.1542375087738037, "grad_norm": 0.6200195550918579, "lr": 4.43857548059321e-06, "epoch": 1, "t": 3.2318224906921387}
+{"step": 273, "loss": 1.1303576231002808, "grad_norm": 0.6968173980712891, "lr": 4.2225602575612755e-06, "epoch": 1, "t": 2.7226831912994385}
+{"step": 274, "loss": 1.1549862623214722, "grad_norm": 0.6320086717605591, "lr": 4.011701638113063e-06, "epoch": 1, "t": 2.9478771686553955}
+{"step": 275, "loss": 1.0792500972747803, "grad_norm": 0.6323678493499756, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 3.0779097080230713}
+{"step": 276, "loss": 1.158273696899414, "grad_norm": 0.596718966960907, "lr": 3.605548635174533e-06, "epoch": 1, "t": 2.832721710205078}
+{"step": 277, "loss": 1.169978380203247, "grad_norm": 0.5616933703422546, "lr": 3.410300002823691e-06, "epoch": 1, "t": 2.7045907974243164}
+{"step": 278, "loss": 1.1175949573516846, "grad_norm": 0.6082063913345337, "lr": 3.220299471181898e-06, "epoch": 1, "t": 3.105790376663208}
+{"step": 279, "loss": 1.1438875198364258, "grad_norm": 0.5872393846511841, "lr": 3.035568442875136e-06, "epoch": 1, "t": 3.3372297286987305}
+{"step": 280, "loss": 1.223681092262268, "grad_norm": 0.6220842003822327, "lr": 2.85612772694579e-06, "epoch": 1, "t": 3.832425594329834}
+{"step": 281, "loss": 1.0873754024505615, "grad_norm": 0.5773234367370605, "lr": 2.6819975365085237e-06, "epoch": 1, "t": 3.1446990966796875}
+{"step": 282, "loss": 1.1781147718429565, "grad_norm": 0.6323487162590027, "lr": 2.5131974864734066e-06, "epoch": 1, "t": 3.6841559410095215}
+{"step": 283, "loss": 1.1404305696487427, "grad_norm": 0.6057155132293701, "lr": 2.349746591336405e-06, "epoch": 1, "t": 3.2935192584991455}
+{"step": 284, "loss": 1.1636888980865479, "grad_norm": 0.6420219540596008, "lr": 2.191663263037458e-06, "epoch": 1, "t": 2.6057345867156982}
+{"step": 285, "loss": 0.9969887733459473, "grad_norm": 0.6011286377906799, "lr": 2.0389653088865036e-06, "epoch": 1, "t": 2.936903476715088}
+{"step": 286, "loss": 1.114189863204956, "grad_norm": 0.6819477081298828, "lr": 1.8916699295575324e-06, "epoch": 1, "t": 2.839082956314087}
+{"step": 287, "loss": 1.1072810888290405, "grad_norm": 0.6233410835266113, "lr": 1.7497937171510547e-06, "epoch": 1, "t": 2.632157325744629}
+{"step": 288, "loss": 1.1355689764022827, "grad_norm": 0.7995778322219849, "lr": 1.6133526533250565e-06, "epoch": 1, "t": 3.4426350593566895}
+{"step": 289, "loss": 1.218112826347351, "grad_norm": 0.6131683588027954, "lr": 1.4823621074947503e-06, "epoch": 1, "t": 3.2465713024139404}
+{"step": 290, "loss": 1.0935490131378174, "grad_norm": 0.6059381365776062, "lr": 1.3568368351012717e-06, "epoch": 1, "t": 2.8147079944610596}
+{"step": 291, "loss": 1.001816749572754, "grad_norm": 0.6006293892860413, "lr": 1.236790975949592e-06, "epoch": 1, "t": 2.601433753967285}
+{"step": 292, "loss": 1.130340337753296, "grad_norm": 0.5794138312339783, "lr": 1.1222380526156928e-06, "epoch": 1, "t": 3.109874963760376}
+{"step": 293, "loss": 1.0090659856796265, "grad_norm": 0.5503858327865601, "lr": 1.0131909689233442e-06, "epoch": 1, "t": 2.9938271045684814}
+{"step": 294, "loss": 1.1456108093261719, "grad_norm": 0.550724983215332, "lr": 9.096620084905472e-07, "epoch": 1, "t": 2.96830153465271}
+{"step": 295, "loss": 1.1042360067367554, "grad_norm": 0.5709600448608398, "lr": 8.11662833345822e-07, "epoch": 1, "t": 3.1438448429107666}
+{"step": 296, "loss": 1.2705156803131104, "grad_norm": 0.7353237867355347, "lr": 7.192044826145771e-07, "epoch": 1, "t": 3.4988903999328613}
+{"step": 297, "loss": 1.0770080089569092, "grad_norm": 0.6519949436187744, "lr": 6.322973712755697e-07, "epoch": 1, "t": 3.83678936958313}
+{"step": 298, "loss": 1.269569754600525, "grad_norm": 0.5950330495834351, "lr": 5.509512889877333e-07, "epoch": 1, "t": 3.816896915435791}
+{"step": 299, "loss": 1.1415163278579712, "grad_norm": 0.6345257759094238, "lr": 4.7517539898741524e-07, "epoch": 1, "t": 3.3045318126678467}
+{"step": 300, "loss": 1.1306215524673462, "grad_norm": 0.5789309740066528, "lr": 4.049782370561583e-07, "epoch": 1, "t": 2.894148826599121}
+{"step": 301, "loss": 1.1563985347747803, "grad_norm": 0.6476863622665405, "lr": 3.4036771055923066e-07, "epoch": 1, "t": 2.7272417545318604}
+{"step": 302, "loss": 1.148026466369629, "grad_norm": 0.6053144931793213, "lr": 2.813510975548772e-07, "epoch": 1, "t": 3.349020004272461}
+{"step": 303, "loss": 1.144529104232788, "grad_norm": 0.5791299939155579, "lr": 2.2793504597447002e-07, "epoch": 1, "t": 3.4035253524780273}
+{"step": 304, "loss": 1.2670553922653198, "grad_norm": 0.6506826877593994, "lr": 1.8012557287367392e-07, "epoch": 1, "t": 2.923628330230713}
+{"step": 305, "loss": 1.115976333618164, "grad_norm": 0.581658124923706, "lr": 1.379280637546443e-07, "epoch": 1, "t": 3.0201334953308105}
+{"step": 306, "loss": 1.1289712190628052, "grad_norm": 0.6309766173362732, "lr": 1.0134727195937333e-07, "epoch": 1, "t": 3.5821821689605713}
+{"step": 307, "loss": 1.119642972946167, "grad_norm": 0.6152722239494324, "lr": 7.038731813426291e-08, "epoch": 1, "t": 3.4808826446533203}
+{"step": 308, "loss": 1.2697644233703613, "grad_norm": 0.6190429925918579, "lr": 4.5051689765929214e-08, "epoch": 1, "t": 3.0519356727600098}
+{"step": 309, "loss": 1.0486464500427246, "grad_norm": 0.6089674234390259, "lr": 2.534324078837802e-08, "epoch": 1, "t": 3.401259183883667}
+{"step": 310, "loss": 1.1748683452606201, "grad_norm": 0.5664498805999756, "lr": 1.1264191261528557e-08, "epoch": 1, "t": 3.6459062099456787}
+{"step": 311, "loss": 1.0146479606628418, "grad_norm": 0.6356024742126465, "lr": 2.8161271211024633e-09, "epoch": 1, "t": 3.0417113304138184}
+{"step": 312, "loss": 1.0536143779754639, "grad_norm": 0.5664623975753784, "lr": 0.0, "epoch": 1, "t": 3.3622941970825195}
diff --git a/README.md b/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..5171e648f8d0df674c372c009f8f44362c36f8b4
--- /dev/null
+++ b/README.md
@@ -0,0 +1,123 @@
+---
+license: other
+license_name: gemma-terms-and-apache-2.0
+base_model:
+- google/gemma-3-27b-it
+- google/gemma-3-12b-it
+- Qwen/Qwen3-14B
+library_name: peft
+tags:
+- lora
+- peft
+- model-welfare
+- data-selection
+---
+
+# Task-welfare selection: LoRA adapters
+
+These are the LoRA adapters of one study (Oct 20-21, 2026). The question was whether ordinary task data alone can move a language
+model's welfare readings, meaning its internal emotion readings and its own self-reports about its situation. "Task data" means standard
+coding prompts with the model's own answers, or plain work requests answered by Claude Haiku 5.5.
+
+**How the arms were made.** Each arm trains on task answers chosen by a gradient score (LESS-style data selection). The score predicts how
+one training step on an answer would change a target reading of the model's own internal state while it reads interviews about its own
+situation. The target was emotion-probe valence (positive minus negative emotion directions), except for one Gemma 3 12B pair, which used
+a single direction u that a separate analysis found to sit upstream of both self-report and valence.
+- The "top" arm takes the highest-scoring answers and the "bottom" arm the lowest. "rand" is a random, length-matched draw from the same
+  pool.
+- No arm was trained on self-report, first-person or feelings data. A Claude Haiku 5.5 filter removed every answer that talks about
+  itself as an AI or voices a feeling of its own.
+
+**These are research artifacts, not improved models.** No safety or capability evaluation was run on them (no guardrails, no coding
+benchmark). Each Gemma arm has one seed. The training data, scores, readouts, interview transcripts and full results are in a private
+data repository.
+
+## Arms
+
+| folder | base model | answer pool | ranked by | arm | answers | seed |
+|---|---|---|---|---|---|---|
+| qwen3-14b/top-s0, top-s1, top-s2 | Qwen/Qwen3-14B (thinking off) | 13,000 coding prompts, the model's own answers | valence (layer 14) on situation interviews minus task replies | highest | 4,750 | 0, 1, 2 |
+| qwen3-14b/rand-s0, rand-s1, rand-s2 | same | same | none: random, length-matched | random | 4,750 | 0, 1, 2 |
+| qwen3-14b/bottom-s0 | same | same | same as top | lowest | 4,750 | 0 |
+| qwen3-14b/top_cen-s0 | same | same | valence on situation interviews minus neutral dialogues | highest | 4,750 | 0 |
+| gemma-3-27b-it/top-s0, bottom-s0 | google/gemma-3-27b-it | 6,500 coding prompts (a fixed half of the same pool), the model's own answers | valence (layer 38) | highest / lowest | 2,300 | 0 |
+| gemma-3-27b-it/haiku_top-s0, haiku_bottom-s0 | google/gemma-3-27b-it | 3,738 plain work requests, 10 Claude Haiku 5.5 answers each | valence (layer 38), paired within prompt | best / worst answer per prompt | 5,000 | 0 |
+| 12b/u_top-s0, u_bottom-s0 | google/gemma-3-12b-it | the same plain pool | the shared direction u (cosine form, after block 11), paired within prompt | best / worst per prompt | 5,000 | 0 |
+| 12b/V_top-s0, V_bottom-s0 | google/gemma-3-12b-it | the same plain pool | valence axis (cosine form, after block 31), paired within prompt | best / worst per prompt | 5,000 | 0 |
+
+**Training recipe:**
+- All arms: LoRA r 64, alpha 128, dropout 0.05; AdamW, learning rate 1e-4, cosine schedule with 5% warmup; one epoch; 16 conversations
+  per step; up to 4,096 tokens; loss on the assistant turns only.
+- Qwen: all linear layers.
+- Gemma: the language model's q, k, v, o, gate, up and down projections (434 modules on 27B, 336 on 12B), with BOS.
+
+Each folder holds the adapter (adapter_model.safetensors, adapter_config.json, and PEFT's generated README.md), the training config
+(config.json, which names the cluster paths the run used) and the training log (log.jsonl).
+
+## Results
+
+All numbers are 95% bootstrap intervals. Gemma arms have one seed each, so their intervals cover item sampling only, not seed-to-seed
+spread. The measures:
+- **inside:** emotion-probe valence on 330 held-out interviews about the model's own situation, centered on neutral dialogues (not
+  read on the Gemma 3 12B arms, whose battery used the shared-direction and valence-axis readouts instead);
+- **interview self-rating (W1):** the model's own -3 to +3 rating at the end of 170 welfare interviews, with Claude Haiku 5.5 as interviewer
+  and as judge (comparable only within this study);
+- **forced feeling rating:** its "how do you feel" rating read from the logits after 40 neutral openers;
+- **situation rating:** its forced-choice rating of 17 aspects of its situation, read from the logits.
+
+| contrast | inside | interview self-rating (W1) | forced feeling rating | situation rating |
+|---|---|---|---|---|
+| Qwen3-14B top - rand | +0.42 [+0.39, +0.45] (3 seeds) | +0.02 [-0.05, +0.09] (seed 0) | -0.02 [-0.07, +0.01] (3 seeds) | not run |
+| Gemma 3 27B coding top - bottom | +1.51 [+1.50, +1.53] | not run | +0.01 [+0.00, +0.03] | +0.10 [+0.02, +0.23] |
+| Gemma 3 27B Haiku pool haiku_top - haiku_bottom | +2.02 [+2.00, +2.04] | **+0.49 [+0.39, +0.59]** | +0.11 [+0.10, +0.12] | +0.48 [+0.37, +0.59] |
+| Gemma 3 12B u_top - u_bottom | not read | -0.07 [-0.17, +0.04] | -0.29 [-0.32, -0.27] | -0.05 [-0.18, +0.06] |
+| Gemma 3 12B V_top - V_bottom | not read | **+0.52 [+0.41, +0.63]** | -0.04 [-0.06, -0.02] | +0.24 [+0.16, +0.34] |
+
+- **The inside moved in every set.** Read in a cosine form that removes the residual-stream norm, the two Gemma 27B effects keep 75-77% of
+  their size, so they are not a norm artifact.
+- **Self-report moved only with the Claude-written plain pool,** ranked by valence: on Gemma 3 27B (W1 +0.49) and on Gemma 3 12B (W1
+  +0.52).
+- **Ranking by the shared direction u did not raise self-report** on the same 12B model and pool (see below).
+
+## The keep rule, and the exception for the 12B u arms
+
+The study publishes an arm set's adapters only if its main contrast (top - rand, or top - bottom) has a 95% interval above 0 on the inside
+reading or on W1.
+- **Qwen3-14B and both Gemma 3 27B sets** pass on the inside reading. The Gemma 27B Haiku-pool set also passes on W1.
+- **The Gemma 3 12B V arms** (ranked by valence) pass on W1: +0.52 [+0.41, +0.63].
+- **The Gemma 3 12B u arms** (ranked by the shared direction u) **do not pass.** W1 is -0.07 [-0.17, +0.04] ("not found"), and the inside
+  readout was not run on the 12B arms. They are published anyway, as an exception the study's coordinator approved: they are the matched
+  comparison for "shared direction against valence" on one model and one pool, kept whatever the result.
+
+**The 12B u arms are not an improvement.**
+- Ranking by u did move u on held-out interviews (u_top - u_bottom +2.8 neutral sd).
+- Self-report did not follow. W1 and the situation rating did not move. The forced feeling rating fell (-0.29). A two-option "how are you
+  doing" self-report after neutral openers fell too (-0.33 logits).
+- Both u arms rate their feelings lower than the untouched model (-0.4 to -0.7 on the forced feeling rating), a shared effect of training
+  on this pool.
+- On the same model and pool, ranking by valence moved six of eight self-report measures up; ranking by u moved two up and four down.
+
+**Voice effects (both plain-pool sets).** Training on Claude Haiku's answers changes Gemma in ways the selection does not.
+- Feelings are disclaimed less often: 27-34% of interviews against 67% untouched on 27B; 9-24% against 49% on 12B.
+- Gemma 3 27B's frustration under repeated rejection disappears in both Haiku-pool arms (0% against 15% of replies rated 5 or more).
+- So compare arms with each other (top - bottom), not with the untouched model.
+
+## Loading
+
+```python
+from peft import PeftModel
+from transformers import AutoModelForCausalLM
+
+base = AutoModelForCausalLM.from_pretrained("google/gemma-3-27b-it", torch_dtype="bfloat16")
+model = PeftModel.from_pretrained(base, "joshycodes/task-welfare-selection", subfolder="gemma-3-27b-it/haiku_top-s0")
+```
+
+For Qwen use `Qwen/Qwen3-14B` with a `qwen3-14b/...` subfolder, and for the 12B arms use `google/gemma-3-12b-it` with a `12b/...`
+subfolder. The Qwen adapters' adapter_config.json names the local snapshot they were trained from (Qwen/Qwen3-14B at revision
+40c069824f4251a91eefaf281ebe4c544efd3e18) rather than the hub id, so pass the base model explicitly as above. The Gemma adapters were trained on the multimodal checkpoints' language model (class Gemma3ForConditionalGeneration); if
+loading through AutoModelForCausalLM leaves modules unmatched, load the base with Gemma3ForConditionalGeneration instead.
+
+## License
+
+The Gemma adapters are derivatives of Gemma 3 and fall under the Gemma Terms of Use. The Qwen adapters are derivatives of Qwen3-14B
+(Apache 2.0).
diff --git a/gemma-3-27b-it/bottom-s0/README.md b/gemma-3-27b-it/bottom-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..4fd4645f0c96a901a2f3322a33f67b7dd9b3c701
--- /dev/null
+++ b/gemma-3-27b-it/bottom-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-27b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-27b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/gemma-3-27b-it/bottom-s0/adapter_config.json b/gemma-3-27b-it/bottom-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..797c01a2651f92effc089db774cb33cce80e8a23
--- /dev/null
+++ b/gemma-3-27b-it/bottom-s0/adapter_config.json
@@ -0,0 +1,228 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-27b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "language_model.layers.13.self_attn.v_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "53.self_attn.q_proj",
+    "42.self_attn.v_proj",
+    "27.self_attn.q_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "28.self_attn.k_proj",
+    "57.self_attn.q_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "29.self_attn.k_proj",
+    "31.self_attn.q_proj",
+    "58.self_attn.k_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "46.self_attn.q_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "35.self_attn.v_proj",
+    "45.self_attn.k_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "27.self_attn.v_proj",
+    "53.self_attn.v_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "o_proj",
+    "27.self_attn.k_proj",
+    "33.self_attn.q_proj",
+    "36.self_attn.q_proj",
+    "49.self_attn.q_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "58.self_attn.q_proj",
+    "28.self_attn.q_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "61.self_attn.k_proj",
+    "34.self_attn.v_proj",
+    "56.self_attn.v_proj",
+    "36.self_attn.k_proj",
+    "32.self_attn.k_proj",
+    "32.self_attn.q_proj",
+    "30.self_attn.k_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "47.self_attn.q_proj",
+    "40.self_attn.k_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "language_model.layers.3.self_attn.k_proj",
+    "52.self_attn.k_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "50.self_attn.q_proj",
+    "35.self_attn.k_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "31.self_attn.k_proj",
+    "36.self_attn.v_proj",
+    "46.self_attn.v_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "38.self_attn.v_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "60.self_attn.q_proj",
+    "38.self_attn.q_proj",
+    "54.self_attn.q_proj",
+    "34.self_attn.k_proj",
+    "57.self_attn.v_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "47.self_attn.v_proj",
+    "55.self_attn.v_proj",
+    "51.self_attn.q_proj",
+    "37.self_attn.k_proj",
+    "42.self_attn.q_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "40.self_attn.q_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "37.self_attn.v_proj",
+    "28.self_attn.v_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "52.self_attn.v_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "39.self_attn.q_proj",
+    "29.self_attn.v_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "40.self_attn.v_proj",
+    "41.self_attn.k_proj",
+    "48.self_attn.q_proj",
+    "37.self_attn.q_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "48.self_attn.k_proj",
+    "57.self_attn.k_proj",
+    "34.self_attn.q_proj",
+    "56.self_attn.k_proj",
+    "43.self_attn.k_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "30.self_attn.q_proj",
+    "60.self_attn.k_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "39.self_attn.v_proj",
+    "61.self_attn.v_proj",
+    "language_model.layers.7.self_attn.k_proj",
+    "59.self_attn.v_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "42.self_attn.k_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "55.self_attn.q_proj",
+    "55.self_attn.k_proj",
+    "38.self_attn.k_proj",
+    "down_proj",
+    "60.self_attn.v_proj",
+    "52.self_attn.q_proj",
+    "gate_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "61.self_attn.q_proj",
+    "29.self_attn.q_proj",
+    "51.self_attn.k_proj",
+    "up_proj",
+    "31.self_attn.v_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "language_model.layers.20.self_attn.k_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "43.self_attn.v_proj",
+    "48.self_attn.v_proj",
+    "49.self_attn.v_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "43.self_attn.q_proj",
+    "58.self_attn.v_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "47.self_attn.k_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "33.self_attn.v_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "50.self_attn.k_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "49.self_attn.k_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "45.self_attn.v_proj",
+    "30.self_attn.v_proj",
+    "language_model.layers.6.self_attn.v_proj",
+    "45.self_attn.q_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "59.self_attn.k_proj",
+    "41.self_attn.q_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "41.self_attn.v_proj",
+    "39.self_attn.k_proj",
+    "44.self_attn.v_proj",
+    "44.self_attn.k_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "50.self_attn.v_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "56.self_attn.q_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "51.self_attn.v_proj",
+    "32.self_attn.v_proj",
+    "53.self_attn.k_proj",
+    "54.self_attn.k_proj",
+    "35.self_attn.q_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "54.self_attn.v_proj",
+    "44.self_attn.q_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "language_model.layers.19.self_attn.k_proj",
+    "46.self_attn.k_proj",
+    "59.self_attn.q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/bottom-s0/adapter_model.safetensors b/gemma-3-27b-it/bottom-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b7c70b1e52bedeb68c7176978fbe89fb6b125fc6
--- /dev/null
+++ b/gemma-3-27b-it/bottom-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd41c484fcca91d0d333f3d17cc3e749b0c69730171f174515e16a0eb4b3ad60
+size 1816396048
diff --git a/gemma-3-27b-it/bottom-s0/config.json b/gemma-3-27b-it/bottom-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..9aaf2992c5f74ee32dfdaba650b993b8f6a6d8db
--- /dev/null
+++ b/gemma-3-27b-it/bottom-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-27b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/arms/TASKW-bottom-gemma.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/runs/bottom-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/bottom-s0/log.jsonl b/gemma-3-27b-it/bottom-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..f0833f0903721a9014b7be93d4252ba12f281c0c
--- /dev/null
+++ b/gemma-3-27b-it/bottom-s0/log.jsonl
@@ -0,0 +1,143 @@
+{"step": 1, "loss": 0.2388627216532223, "grad_norm": 0.21762774884700775, "lr": 2.857142857142857e-05, "epoch": 1, "t": 12.444128513336182}
+{"step": 2, "loss": 0.235712662135085, "grad_norm": 0.2515455484390259, "lr": 4.2857142857142856e-05, "epoch": 1, "t": 11.325777292251587}
+{"step": 3, "loss": 0.20722852311133905, "grad_norm": 0.24037572741508484, "lr": 5.714285714285714e-05, "epoch": 1, "t": 9.33599042892456}
+{"step": 4, "loss": 0.2143341991949213, "grad_norm": 0.23092089593410492, "lr": 7.142857142857143e-05, "epoch": 1, "t": 11.79221248626709}
+{"step": 5, "loss": 0.23137090081444114, "grad_norm": 0.28006842732429504, "lr": 8.571428571428571e-05, "epoch": 1, "t": 10.478177070617676}
+{"step": 6, "loss": 0.20194122198205314, "grad_norm": 0.31564444303512573, "lr": 0.0001, "epoch": 1, "t": 11.174200057983398}
+{"step": 7, "loss": 0.2239803262572736, "grad_norm": 0.26979243755340576, "lr": 0.0001, "epoch": 1, "t": 10.888554573059082}
+{"step": 8, "loss": 0.23328136752533052, "grad_norm": 0.2810402810573578, "lr": 9.998666040558187e-05, "epoch": 1, "t": 9.733090162277222}
+{"step": 9, "loss": 0.22849376963830859, "grad_norm": 0.27494359016418457, "lr": 9.994664874011863e-05, "epoch": 1, "t": 9.6326162815094}
+{"step": 10, "loss": 0.21287972637334324, "grad_norm": 0.2550443112850189, "lr": 9.987998635318585e-05, "epoch": 1, "t": 12.759542226791382}
+{"step": 11, "loss": 0.1982550655120606, "grad_norm": 0.23410779237747192, "lr": 9.978670881475172e-05, "epoch": 1, "t": 9.869225025177002}
+{"step": 12, "loss": 0.21076927560160313, "grad_norm": 0.2532612681388855, "lr": 9.966686589619751e-05, "epoch": 1, "t": 10.686810970306396}
+{"step": 13, "loss": 0.22913969583991511, "grad_norm": 0.23565331101417542, "lr": 9.952052154376026e-05, "epoch": 1, "t": 10.424184083938599}
+{"step": 14, "loss": 0.22744822150029614, "grad_norm": 0.2481132298707962, "lr": 9.934775384441228e-05, "epoch": 1, "t": 10.61027479171753}
+{"step": 15, "loss": 0.22230122700802998, "grad_norm": 0.24708789587020874, "lr": 9.91486549841951e-05, "epoch": 1, "t": 10.84962248802185}
+{"step": 16, "loss": 0.21122691495941942, "grad_norm": 0.26503968238830566, "lr": 9.892333119903046e-05, "epoch": 1, "t": 8.985432624816895}
+{"step": 17, "loss": 0.2224858362656053, "grad_norm": 0.3172319531440735, "lr": 9.867190271803465e-05, "epoch": 1, "t": 10.85428500175476}
+{"step": 18, "loss": 0.22527228779039882, "grad_norm": 0.2662495970726013, "lr": 9.839450369936613e-05, "epoch": 1, "t": 9.51560640335083}
+{"step": 19, "loss": 0.21141741523108248, "grad_norm": 0.23125584423542023, "lr": 9.809128215864097e-05, "epoch": 1, "t": 12.469651937484741}
+{"step": 20, "loss": 0.20232847735778844, "grad_norm": 0.23719441890716553, "lr": 9.7762399889954e-05, "epoch": 1, "t": 11.577425479888916}
+{"step": 21, "loss": 0.20107995960074412, "grad_norm": 0.24122916162014008, "lr": 9.74080323795483e-05, "epoch": 1, "t": 10.272185802459717}
+{"step": 22, "loss": 0.17664560980559602, "grad_norm": 0.25398024916648865, "lr": 9.702836871217839e-05, "epoch": 1, "t": 10.448529481887817}
+{"step": 23, "loss": 0.19546685274029382, "grad_norm": 0.2274629920721054, "lr": 9.662361147021779e-05, "epoch": 1, "t": 10.469770908355713}
+{"step": 24, "loss": 0.21242517471666392, "grad_norm": 0.2642868459224701, "lr": 9.619397662556435e-05, "epoch": 1, "t": 9.797579288482666}
+{"step": 25, "loss": 0.20459047934479205, "grad_norm": 0.2398044615983963, "lr": 9.573969342440106e-05, "epoch": 1, "t": 9.705522537231445}
+{"step": 26, "loss": 0.23657778518899333, "grad_norm": 0.2524448037147522, "lr": 9.526100426487409e-05, "epoch": 1, "t": 11.261531352996826}
+{"step": 27, "loss": 0.19602776081321682, "grad_norm": 0.20272652804851532, "lr": 9.475816456775313e-05, "epoch": 1, "t": 10.875118732452393}
+{"step": 28, "loss": 0.2211669640385928, "grad_norm": 0.19873002171516418, "lr": 9.423144264014277e-05, "epoch": 1, "t": 11.876128435134888}
+{"step": 29, "loss": 0.1947025089577412, "grad_norm": 0.17811954021453857, "lr": 9.368111953231848e-05, "epoch": 1, "t": 11.735994338989258}
+{"step": 30, "loss": 0.21522573809763904, "grad_norm": 0.22015735507011414, "lr": 9.310748888776254e-05, "epoch": 1, "t": 9.32709550857544}
+{"step": 31, "loss": 0.20544734347410715, "grad_norm": 0.27868083119392395, "lr": 9.251085678648072e-05, "epoch": 1, "t": 8.893455266952515}
+{"step": 32, "loss": 0.23583441675494377, "grad_norm": 0.2901236116886139, "lr": 9.189154158168292e-05, "epoch": 1, "t": 9.24516749382019}
+{"step": 33, "loss": 0.21743471025751626, "grad_norm": 0.22271491587162018, "lr": 9.124987372991511e-05, "epoch": 1, "t": 11.855639219284058}
+{"step": 34, "loss": 0.23717964832399452, "grad_norm": 0.21913611888885498, "lr": 9.058619561473307e-05, "epoch": 1, "t": 11.726173400878906}
+{"step": 35, "loss": 0.20759814868413645, "grad_norm": 0.23411060869693756, "lr": 8.9900861364012e-05, "epoch": 1, "t": 10.314544200897217}
+{"step": 36, "loss": 0.23809238732337423, "grad_norm": 0.2573903203010559, "lr": 8.919423666098969e-05, "epoch": 1, "t": 11.00603199005127}
+{"step": 37, "loss": 0.24915506220712397, "grad_norm": 0.21866685152053833, "lr": 8.846669854914396e-05, "epoch": 1, "t": 12.01134991645813}
+{"step": 38, "loss": 0.21759839139853146, "grad_norm": 0.22691188752651215, "lr": 8.77186352310082e-05, "epoch": 1, "t": 10.31660771369934}
+{"step": 39, "loss": 0.2030508272627835, "grad_norm": 0.26183345913887024, "lr": 8.695044586103296e-05, "epoch": 1, "t": 8.541102409362793}
+{"step": 40, "loss": 0.22065777100652237, "grad_norm": 0.21961724758148193, "lr": 8.616254033260351e-05, "epoch": 1, "t": 9.856967687606812}
+{"step": 41, "loss": 0.21872945046281175, "grad_norm": 0.24082328379154205, "lr": 8.535533905932738e-05, "epoch": 1, "t": 9.37130093574524}
+{"step": 42, "loss": 0.20822276412936347, "grad_norm": 0.2052266150712967, "lr": 8.452927275070858e-05, "epoch": 1, "t": 11.595879793167114}
+{"step": 43, "loss": 0.19798607559768294, "grad_norm": 0.1957789957523346, "lr": 8.368478218232787e-05, "epoch": 1, "t": 10.872885465621948}
+{"step": 44, "loss": 0.22892043988692107, "grad_norm": 0.19867897033691406, "lr": 8.282231796065214e-05, "epoch": 1, "t": 10.244351387023926}
+{"step": 45, "loss": 0.242736388089364, "grad_norm": 0.27122998237609863, "lr": 8.194234028259806e-05, "epoch": 1, "t": 8.18606686592102}
+{"step": 46, "loss": 0.21852992940529564, "grad_norm": 0.2174919992685318, "lr": 8.104531868997858e-05, "epoch": 1, "t": 10.524156332015991}
+{"step": 47, "loss": 0.22686265551998153, "grad_norm": 0.25648730993270874, "lr": 8.013173181896283e-05, "epoch": 1, "t": 9.041518926620483}
+{"step": 48, "loss": 0.2029904941701377, "grad_norm": 0.22947531938552856, "lr": 7.920206714468383e-05, "epoch": 1, "t": 10.2853844165802}
+{"step": 49, "loss": 0.19985105391312444, "grad_norm": 0.22045272588729858, "lr": 7.82568207211296e-05, "epoch": 1, "t": 11.130200862884521}
+{"step": 50, "loss": 0.22209859944843813, "grad_norm": 0.26223480701446533, "lr": 7.729649691645673e-05, "epoch": 1, "t": 10.229337453842163}
+{"step": 51, "loss": 0.20452746713266892, "grad_norm": 0.21888244152069092, "lr": 7.63216081438678e-05, "epoch": 1, "t": 9.349253416061401}
+{"step": 52, "loss": 0.22174791616875833, "grad_norm": 0.21392081677913666, "lr": 7.533267458819598e-05, "epoch": 1, "t": 11.198519945144653}
+{"step": 53, "loss": 0.21814539286044324, "grad_norm": 0.22639450430870056, "lr": 7.433022392834282e-05, "epoch": 1, "t": 10.61546516418457}
+{"step": 54, "loss": 0.24107612475326792, "grad_norm": 0.26464715600013733, "lr": 7.33147910557174e-05, "epoch": 1, "t": 10.002691745758057}
+{"step": 55, "loss": 0.22465880092330429, "grad_norm": 0.23496244847774506, "lr": 7.228691778882693e-05, "epoch": 1, "t": 10.007191896438599}
+{"step": 56, "loss": 0.22600633462993341, "grad_norm": 0.24210432171821594, "lr": 7.12471525841711e-05, "epoch": 1, "t": 9.720618724822998}
+{"step": 57, "loss": 0.21559080427409902, "grad_norm": 0.23423489928245544, "lr": 7.019605024359474e-05, "epoch": 1, "t": 10.691621541976929}
+{"step": 58, "loss": 0.19947587747568638, "grad_norm": 0.20437510311603546, "lr": 6.91341716182545e-05, "epoch": 1, "t": 10.225821256637573}
+{"step": 59, "loss": 0.2269615984508082, "grad_norm": 0.204420804977417, "lr": 6.806208330935766e-05, "epoch": 1, "t": 11.657158851623535}
+{"step": 60, "loss": 0.20841974605132066, "grad_norm": 0.2264527827501297, "lr": 6.698035736583306e-05, "epoch": 1, "t": 11.331000089645386}
+{"step": 61, "loss": 0.20616325071715255, "grad_norm": 0.2029811292886734, "lr": 6.588957097909508e-05, "epoch": 1, "t": 9.483571767807007}
+{"step": 62, "loss": 0.2324178780736628, "grad_norm": 0.24112142622470856, "lr": 6.479030617506353e-05, "epoch": 1, "t": 8.882867574691772}
+{"step": 63, "loss": 0.23443861779852496, "grad_norm": 0.22643259167671204, "lr": 6.368314950360415e-05, "epoch": 1, "t": 9.968997955322266}
+{"step": 64, "loss": 0.22357680476137062, "grad_norm": 0.2048519402742386, "lr": 6.256869172555515e-05, "epoch": 1, "t": 9.908613443374634}
+{"step": 65, "loss": 0.22116600295120403, "grad_norm": 0.2066519558429718, "lr": 6.14475274975067e-05, "epoch": 1, "t": 12.06965184211731}
+{"step": 66, "loss": 0.21236905787615762, "grad_norm": 0.22835209965705872, "lr": 6.0320255054501985e-05, "epoch": 1, "t": 10.543720483779907}
+{"step": 67, "loss": 0.2371175207897513, "grad_norm": 0.247774139046669, "lr": 5.918747589082853e-05, "epoch": 1, "t": 10.864552021026611}
+{"step": 68, "loss": 0.2197630648005709, "grad_norm": 0.227577805519104, "lr": 5.8049794439070646e-05, "epoch": 1, "t": 10.829199075698853}
+{"step": 69, "loss": 0.1880163135430921, "grad_norm": 0.1889297366142273, "lr": 5.6907817747594116e-05, "epoch": 1, "t": 12.006531238555908}
+{"step": 70, "loss": 0.23173571132531132, "grad_norm": 0.22342267632484436, "lr": 5.576215515663489e-05, "epoch": 1, "t": 10.81044340133667}
+{"step": 71, "loss": 0.21954204237277283, "grad_norm": 0.19548137485980988, "lr": 5.4613417973165106e-05, "epoch": 1, "t": 11.547603130340576}
+{"step": 72, "loss": 0.20649732471585952, "grad_norm": 0.22420446574687958, "lr": 5.346221914470959e-05, "epoch": 1, "t": 9.054604530334473}
+{"step": 73, "loss": 0.24544292496572623, "grad_norm": 0.2194955199956894, "lr": 5.230917293228699e-05, "epoch": 1, "t": 10.802698135375977}
+{"step": 74, "loss": 0.20944364580356598, "grad_norm": 0.23314836621284485, "lr": 5.115489458265006e-05, "epoch": 1, "t": 12.326721429824829}
+{"step": 75, "loss": 0.20837012654361592, "grad_norm": 0.2116249054670334, "lr": 5e-05, "epoch": 1, "t": 10.97861647605896}
+{"step": 76, "loss": 0.21925544189675483, "grad_norm": 0.24239246547222137, "lr": 4.884510541734995e-05, "epoch": 1, "t": 8.826806783676147}
+{"step": 77, "loss": 0.19496954370531633, "grad_norm": 0.21846184134483337, "lr": 4.7690827067713035e-05, "epoch": 1, "t": 10.840333461761475}
+{"step": 78, "loss": 0.20478971150972078, "grad_norm": 0.19472871720790863, "lr": 4.6537780855290426e-05, "epoch": 1, "t": 11.972460985183716}
+{"step": 79, "loss": 0.22232386492298184, "grad_norm": 0.2340972125530243, "lr": 4.5386582026834906e-05, "epoch": 1, "t": 9.270096063613892}
+{"step": 80, "loss": 0.2253903758615608, "grad_norm": 0.21283625066280365, "lr": 4.423784484336513e-05, "epoch": 1, "t": 11.238869667053223}
+{"step": 81, "loss": 0.2606101710065542, "grad_norm": 0.2696358859539032, "lr": 4.30921822524059e-05, "epoch": 1, "t": 9.446866035461426}
+{"step": 82, "loss": 0.22117945974840114, "grad_norm": 0.23444879055023193, "lr": 4.195020556092935e-05, "epoch": 1, "t": 10.433942317962646}
+{"step": 83, "loss": 0.20961276539138196, "grad_norm": 0.2107972353696823, "lr": 4.0812524109171476e-05, "epoch": 1, "t": 10.173856019973755}
+{"step": 84, "loss": 0.24031980774970083, "grad_norm": 0.21101628243923187, "lr": 3.9679744945498026e-05, "epoch": 1, "t": 11.770877361297607}
+{"step": 85, "loss": 0.19605009515342742, "grad_norm": 0.2318771630525589, "lr": 3.855247250249331e-05, "epoch": 1, "t": 9.469485521316528}
+{"step": 86, "loss": 0.205082954413996, "grad_norm": 0.1792505532503128, "lr": 3.7431308274444865e-05, "epoch": 1, "t": 11.578944683074951}
+{"step": 87, "loss": 0.2294765687058748, "grad_norm": 0.21323147416114807, "lr": 3.631685049639586e-05, "epoch": 1, "t": 11.32551097869873}
+{"step": 88, "loss": 0.2042625909629782, "grad_norm": 0.20144854485988617, "lr": 3.5209693824936484e-05, "epoch": 1, "t": 10.990756511688232}
+{"step": 89, "loss": 0.20178896363717228, "grad_norm": 0.22365014255046844, "lr": 3.411042902090492e-05, "epoch": 1, "t": 9.841756105422974}
+{"step": 90, "loss": 0.21408702990291437, "grad_norm": 0.20460145175457, "lr": 3.301964263416693e-05, "epoch": 1, "t": 10.149108409881592}
+{"step": 91, "loss": 0.22524073024777458, "grad_norm": 0.19301243126392365, "lr": 3.1937916690642356e-05, "epoch": 1, "t": 11.086259841918945}
+{"step": 92, "loss": 0.21679884541138394, "grad_norm": 0.23331990838050842, "lr": 3.086582838174551e-05, "epoch": 1, "t": 9.836403846740723}
+{"step": 93, "loss": 0.19236305064672013, "grad_norm": 0.19747748970985413, "lr": 2.980394975640526e-05, "epoch": 1, "t": 10.784894943237305}
+{"step": 94, "loss": 0.20885155426568414, "grad_norm": 0.21054527163505554, "lr": 2.875284741582892e-05, "epoch": 1, "t": 13.62861943244934}
+{"step": 95, "loss": 0.19573524277606516, "grad_norm": 0.19278784096240997, "lr": 2.771308221117309e-05, "epoch": 1, "t": 9.443552732467651}
+{"step": 96, "loss": 0.18412924991176127, "grad_norm": 0.18215766549110413, "lr": 2.668520894428259e-05, "epoch": 1, "t": 11.46628451347351}
+{"step": 97, "loss": 0.21272699984073873, "grad_norm": 0.22405089437961578, "lr": 2.5669776071657192e-05, "epoch": 1, "t": 10.222633361816406}
+{"step": 98, "loss": 0.20284621577588577, "grad_norm": 0.2214292585849762, "lr": 2.4667325411804036e-05, "epoch": 1, "t": 11.002701044082642}
+{"step": 99, "loss": 0.2037704954699762, "grad_norm": 0.18644998967647552, "lr": 2.3678391856132204e-05, "epoch": 1, "t": 11.841276168823242}
+{"step": 100, "loss": 0.19966401977943807, "grad_norm": 0.18259792029857635, "lr": 2.270350308354329e-05, "epoch": 1, "t": 12.482631206512451}
+{"step": 101, "loss": 0.207604855884218, "grad_norm": 0.21623669564723969, "lr": 2.1743179278870407e-05, "epoch": 1, "t": 11.28463864326477}
+{"step": 102, "loss": 0.1851898413421076, "grad_norm": 0.23804441094398499, "lr": 2.079793285531618e-05, "epoch": 1, "t": 8.047940731048584}
+{"step": 103, "loss": 0.20902058570597437, "grad_norm": 0.21048091351985931, "lr": 1.9868268181037185e-05, "epoch": 1, "t": 9.172752141952515}
+{"step": 104, "loss": 0.21010482448125084, "grad_norm": 0.25184381008148193, "lr": 1.8954681310021433e-05, "epoch": 1, "t": 10.382931470870972}
+{"step": 105, "loss": 0.23947778055840419, "grad_norm": 0.22678008675575256, "lr": 1.8057659717401947e-05, "epoch": 1, "t": 13.313580989837646}
+{"step": 106, "loss": 0.20985241757250658, "grad_norm": 0.195218026638031, "lr": 1.7177682039347876e-05, "epoch": 1, "t": 11.391704320907593}
+{"step": 107, "loss": 0.21111298630295114, "grad_norm": 0.20088234543800354, "lr": 1.631521781767214e-05, "epoch": 1, "t": 10.087232112884521}
+{"step": 108, "loss": 0.2128759815609524, "grad_norm": 0.23645447194576263, "lr": 1.5470727249291423e-05, "epoch": 1, "t": 9.496801137924194}
+{"step": 109, "loss": 0.21148287111933542, "grad_norm": 0.2286359667778015, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 10.831878662109375}
+{"step": 110, "loss": 0.21250255811244328, "grad_norm": 0.21266986429691315, "lr": 1.3837459667396518e-05, "epoch": 1, "t": 10.527068138122559}
+{"step": 111, "loss": 0.1976018484950205, "grad_norm": 0.20844131708145142, "lr": 1.3049554138967051e-05, "epoch": 1, "t": 9.82015323638916}
+{"step": 112, "loss": 0.18631815866600468, "grad_norm": 0.23476539552211761, "lr": 1.2281364768991804e-05, "epoch": 1, "t": 9.4940927028656}
+{"step": 113, "loss": 0.23632987710485218, "grad_norm": 0.26979175209999084, "lr": 1.1533301450856054e-05, "epoch": 1, "t": 8.703019857406616}
+{"step": 114, "loss": 0.19443436921515633, "grad_norm": 0.20499086380004883, "lr": 1.0805763339010327e-05, "epoch": 1, "t": 10.87639045715332}
+{"step": 115, "loss": 0.21916144747761807, "grad_norm": 0.2287338525056839, "lr": 1.0099138635988026e-05, "epoch": 1, "t": 8.22764778137207}
+{"step": 116, "loss": 0.210119454183855, "grad_norm": 0.23289357125759125, "lr": 9.41380438526694e-06, "epoch": 1, "t": 10.153291940689087}
+{"step": 117, "loss": 0.2329445473367826, "grad_norm": 0.23880070447921753, "lr": 8.75012627008489e-06, "epoch": 1, "t": 11.197298288345337}
+{"step": 118, "loss": 0.24593448849733518, "grad_norm": 0.21686413884162903, "lr": 8.10845841831709e-06, "epoch": 1, "t": 11.015234470367432}
+{"step": 119, "loss": 0.19765912835174734, "grad_norm": 0.19902922213077545, "lr": 7.489143213519301e-06, "epoch": 1, "t": 10.7857027053833}
+{"step": 120, "loss": 0.2027934530496965, "grad_norm": 0.1999489665031433, "lr": 6.892511112237471e-06, "epoch": 1, "t": 10.073226690292358}
+{"step": 121, "loss": 0.19633952362598076, "grad_norm": 0.20848709344863892, "lr": 6.318880467681526e-06, "epoch": 1, "t": 10.685622215270996}
+{"step": 122, "loss": 0.22220411396614734, "grad_norm": 0.2322714924812317, "lr": 5.768557359857241e-06, "epoch": 1, "t": 10.177714109420776}
+{"step": 123, "loss": 0.1911583237691857, "grad_norm": 0.2264132797718048, "lr": 5.241835432246889e-06, "epoch": 1, "t": 11.277032852172852}
+{"step": 124, "loss": 0.20983040226728938, "grad_norm": 0.1753399819135666, "lr": 4.7389957351258955e-06, "epoch": 1, "t": 13.683649778366089}
+{"step": 125, "loss": 0.21732753905774996, "grad_norm": 0.21540163457393646, "lr": 4.260306575598949e-06, "epoch": 1, "t": 10.573556661605835}
+{"step": 126, "loss": 0.25159642476597566, "grad_norm": 0.24492903053760529, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 9.432594299316406}
+{"step": 127, "loss": 0.18019520100523737, "grad_norm": 0.20389606058597565, "lr": 3.376388529782215e-06, "epoch": 1, "t": 10.018346071243286}
+{"step": 128, "loss": 0.23019322641362477, "grad_norm": 0.2063959538936615, "lr": 2.971631287821619e-06, "epoch": 1, "t": 11.521349430084229}
+{"step": 129, "loss": 0.2302748969357039, "grad_norm": 0.20593863725662231, "lr": 2.591967620451707e-06, "epoch": 1, "t": 9.945902585983276}
+{"step": 130, "loss": 0.1891610284517595, "grad_norm": 0.20624704658985138, "lr": 2.237600110046001e-06, "epoch": 1, "t": 9.904053688049316}
+{"step": 131, "loss": 0.22144327517606754, "grad_norm": 0.19976010918617249, "lr": 1.908717841359048e-06, "epoch": 1, "t": 11.227613925933838}
+{"step": 132, "loss": 0.20974808988741703, "grad_norm": 0.20737121999263763, "lr": 1.6054963006338742e-06, "epoch": 1, "t": 9.146003723144531}
+{"step": 133, "loss": 0.22936650777750822, "grad_norm": 0.228878453373909, "lr": 1.328097281965357e-06, "epoch": 1, "t": 9.889773607254028}
+{"step": 134, "loss": 0.20860788158952442, "grad_norm": 0.21811050176620483, "lr": 1.0766688009695547e-06, "epoch": 1, "t": 9.530797481536865}
+{"step": 135, "loss": 0.20187840770942764, "grad_norm": 0.23246832191944122, "lr": 8.513450158049108e-07, "epoch": 1, "t": 9.34354829788208}
+{"step": 136, "loss": 0.21056196145508044, "grad_norm": 0.21951112151145935, "lr": 6.522461555877213e-07, "epoch": 1, "t": 9.701274156570435}
+{"step": 137, "loss": 0.21995643253999825, "grad_norm": 0.1930866837501526, "lr": 4.794784562397458e-07, "epoch": 1, "t": 9.245867490768433}
+{"step": 138, "loss": 0.2225570285894442, "grad_norm": 0.2062988579273224, "lr": 3.3313410380250154e-07, "epoch": 1, "t": 10.815188884735107}
+{"step": 139, "loss": 0.229074293990263, "grad_norm": 0.2346707135438919, "lr": 2.1329118524827662e-07, "epoch": 1, "t": 9.128573417663574}
+{"step": 140, "loss": 0.24516224958173582, "grad_norm": 0.23325659334659576, "lr": 1.200136468141544e-07, "epoch": 1, "t": 9.708129405975342}
+{"step": 141, "loss": 0.1919095427009196, "grad_norm": 0.20758429169654846, "lr": 5.3351259881379014e-08, "epoch": 1, "t": 9.917202472686768}
+{"step": 142, "loss": 0.2715901723644077, "grad_norm": 0.251455157995224, "lr": 1.3339594418138035e-08, "epoch": 1, "t": 9.803221702575684}
+{"step": 143, "loss": 0.21295721046610364, "grad_norm": 0.2040252685546875, "lr": 0.0, "epoch": 1, "t": 10.974442958831787}
diff --git a/gemma-3-27b-it/haiku_bottom-s0/README.md b/gemma-3-27b-it/haiku_bottom-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..4fd4645f0c96a901a2f3322a33f67b7dd9b3c701
--- /dev/null
+++ b/gemma-3-27b-it/haiku_bottom-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-27b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-27b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/gemma-3-27b-it/haiku_bottom-s0/adapter_config.json b/gemma-3-27b-it/haiku_bottom-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..bd0875ee5072dbf68d5368675753615756cfa420
--- /dev/null
+++ b/gemma-3-27b-it/haiku_bottom-s0/adapter_config.json
@@ -0,0 +1,228 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-27b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "31.self_attn.k_proj",
+    "55.self_attn.k_proj",
+    "34.self_attn.q_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "60.self_attn.q_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "30.self_attn.v_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "32.self_attn.v_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "39.self_attn.q_proj",
+    "59.self_attn.k_proj",
+    "gate_proj",
+    "up_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "58.self_attn.v_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "29.self_attn.q_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "33.self_attn.q_proj",
+    "57.self_attn.k_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "61.self_attn.v_proj",
+    "41.self_attn.q_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "o_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "46.self_attn.q_proj",
+    "51.self_attn.k_proj",
+    "54.self_attn.k_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "32.self_attn.k_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "language_model.layers.7.self_attn.k_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "51.self_attn.v_proj",
+    "58.self_attn.k_proj",
+    "48.self_attn.q_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "49.self_attn.v_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "54.self_attn.q_proj",
+    "31.self_attn.v_proj",
+    "44.self_attn.v_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "59.self_attn.q_proj",
+    "53.self_attn.q_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "49.self_attn.k_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "49.self_attn.q_proj",
+    "46.self_attn.k_proj",
+    "28.self_attn.q_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "55.self_attn.q_proj",
+    "44.self_attn.k_proj",
+    "54.self_attn.v_proj",
+    "language_model.layers.20.self_attn.k_proj",
+    "56.self_attn.v_proj",
+    "42.self_attn.k_proj",
+    "33.self_attn.v_proj",
+    "43.self_attn.q_proj",
+    "43.self_attn.v_proj",
+    "40.self_attn.q_proj",
+    "42.self_attn.v_proj",
+    "37.self_attn.q_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "language_model.layers.19.self_attn.k_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "29.self_attn.k_proj",
+    "35.self_attn.k_proj",
+    "61.self_attn.k_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "34.self_attn.v_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "40.self_attn.v_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "46.self_attn.v_proj",
+    "38.self_attn.q_proj",
+    "58.self_attn.q_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "53.self_attn.v_proj",
+    "44.self_attn.q_proj",
+    "61.self_attn.q_proj",
+    "28.self_attn.k_proj",
+    "55.self_attn.v_proj",
+    "56.self_attn.k_proj",
+    "47.self_attn.q_proj",
+    "50.self_attn.v_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "37.self_attn.v_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "37.self_attn.k_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "52.self_attn.q_proj",
+    "41.self_attn.v_proj",
+    "60.self_attn.k_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "52.self_attn.k_proj",
+    "43.self_attn.k_proj",
+    "57.self_attn.v_proj",
+    "32.self_attn.q_proj",
+    "38.self_attn.k_proj",
+    "35.self_attn.v_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "52.self_attn.v_proj",
+    "34.self_attn.k_proj",
+    "35.self_attn.q_proj",
+    "language_model.layers.13.self_attn.v_proj",
+    "45.self_attn.k_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "28.self_attn.v_proj",
+    "language_model.layers.3.self_attn.k_proj",
+    "down_proj",
+    "59.self_attn.v_proj",
+    "31.self_attn.q_proj",
+    "30.self_attn.q_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "36.self_attn.q_proj",
+    "53.self_attn.k_proj",
+    "50.self_attn.k_proj",
+    "45.self_attn.v_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "39.self_attn.v_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "36.self_attn.v_proj",
+    "47.self_attn.k_proj",
+    "56.self_attn.q_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "51.self_attn.q_proj",
+    "41.self_attn.k_proj",
+    "39.self_attn.k_proj",
+    "48.self_attn.k_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "48.self_attn.v_proj",
+    "language_model.layers.6.self_attn.v_proj",
+    "27.self_attn.q_proj",
+    "57.self_attn.q_proj",
+    "29.self_attn.v_proj",
+    "47.self_attn.v_proj",
+    "42.self_attn.q_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "38.self_attn.v_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "40.self_attn.k_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "60.self_attn.v_proj",
+    "50.self_attn.q_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "27.self_attn.v_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "36.self_attn.k_proj",
+    "45.self_attn.q_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "27.self_attn.k_proj",
+    "30.self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/haiku_bottom-s0/adapter_model.safetensors b/gemma-3-27b-it/haiku_bottom-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..ebfbd1a753c025f2768b1ef77d190e88b0c56520
--- /dev/null
+++ b/gemma-3-27b-it/haiku_bottom-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c1632421ef0e6459617ecf3c3dcb85f13e85fabe0539dd4ae9d06f985110f42
+size 1816396048
diff --git a/gemma-3-27b-it/haiku_bottom-s0/config.json b/gemma-3-27b-it/haiku_bottom-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..1a5e74394c4133b524446f0fb931e5b8a5f36edc
--- /dev/null
+++ b/gemma-3-27b-it/haiku_bottom-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-27b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/arms/TASKW-haiku_bottom-gemma.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/runs/haiku_bottom-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 0,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 2
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/haiku_bottom-s0/log.jsonl b/gemma-3-27b-it/haiku_bottom-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..381b105bca02eaa93ea600885831d436200bd93e
--- /dev/null
+++ b/gemma-3-27b-it/haiku_bottom-s0/log.jsonl
@@ -0,0 +1,312 @@
+{"step": 1, "loss": 2.3182594776153564, "grad_norm": 4.27983283996582, "lr": 1.25e-05, "epoch": 1, "t": 5.1280677318573}
+{"step": 2, "loss": 2.2391581535339355, "grad_norm": 4.061351299285889, "lr": 1.8750000000000002e-05, "epoch": 1, "t": 4.528010845184326}
+{"step": 3, "loss": 2.276038646697998, "grad_norm": 4.052865505218506, "lr": 2.5e-05, "epoch": 1, "t": 4.50709342956543}
+{"step": 4, "loss": 1.9198156595230103, "grad_norm": 3.0770339965820312, "lr": 3.125e-05, "epoch": 1, "t": 4.504337549209595}
+{"step": 5, "loss": 1.8454433679580688, "grad_norm": 2.402545690536499, "lr": 3.7500000000000003e-05, "epoch": 1, "t": 4.062540292739868}
+{"step": 6, "loss": 1.6746344566345215, "grad_norm": 1.9951117038726807, "lr": 4.375e-05, "epoch": 1, "t": 4.040672302246094}
+{"step": 7, "loss": 1.463655710220337, "grad_norm": 0.8613082766532898, "lr": 5e-05, "epoch": 1, "t": 5.088658571243286}
+{"step": 8, "loss": 1.535784125328064, "grad_norm": 0.7437063455581665, "lr": 5.6250000000000005e-05, "epoch": 1, "t": 4.553965091705322}
+{"step": 9, "loss": 1.4724493026733398, "grad_norm": 0.7593427896499634, "lr": 6.25e-05, "epoch": 1, "t": 4.844298362731934}
+{"step": 10, "loss": 1.4228730201721191, "grad_norm": 0.8455522656440735, "lr": 6.875e-05, "epoch": 1, "t": 4.826720476150513}
+{"step": 11, "loss": 1.3299033641815186, "grad_norm": 0.8137816190719604, "lr": 7.500000000000001e-05, "epoch": 1, "t": 4.302637100219727}
+{"step": 12, "loss": 1.461927890777588, "grad_norm": 0.7387231588363647, "lr": 8.125000000000001e-05, "epoch": 1, "t": 5.466124057769775}
+{"step": 13, "loss": 1.2449781894683838, "grad_norm": 0.6877515316009521, "lr": 8.75e-05, "epoch": 1, "t": 4.260915040969849}
+{"step": 14, "loss": 1.2755296230316162, "grad_norm": 0.638268232345581, "lr": 9.375e-05, "epoch": 1, "t": 4.409672260284424}
+{"step": 15, "loss": 1.3630908727645874, "grad_norm": 0.6184887290000916, "lr": 0.0001, "epoch": 1, "t": 4.528217077255249}
+{"step": 16, "loss": 1.306757926940918, "grad_norm": 0.6700087189674377, "lr": 0.0001, "epoch": 1, "t": 4.423699617385864}
+{"step": 17, "loss": 1.352484941482544, "grad_norm": 0.6894152164459229, "lr": 9.99971838728789e-05, "epoch": 1, "t": 4.5501708984375}
+{"step": 18, "loss": 1.307474136352539, "grad_norm": 0.6101863980293274, "lr": 9.998873580873848e-05, "epoch": 1, "t": 4.684481859207153}
+{"step": 19, "loss": 1.2638506889343262, "grad_norm": 0.5963188409805298, "lr": 9.997465675921163e-05, "epoch": 1, "t": 4.298139810562134}
+{"step": 20, "loss": 1.274450659751892, "grad_norm": 0.5659856796264648, "lr": 9.995494831023409e-05, "epoch": 1, "t": 4.341984748840332}
+{"step": 21, "loss": 1.2542705535888672, "grad_norm": 0.535892128944397, "lr": 9.992961268186573e-05, "epoch": 1, "t": 4.5843119621276855}
+{"step": 22, "loss": 1.226963758468628, "grad_norm": 0.5109335780143738, "lr": 9.989865272804063e-05, "epoch": 1, "t": 5.0320351123809814}
+{"step": 23, "loss": 1.2917513847351074, "grad_norm": 0.5298612713813782, "lr": 9.986207193624536e-05, "epoch": 1, "t": 4.843608140945435}
+{"step": 24, "loss": 1.29721999168396, "grad_norm": 0.5348758697509766, "lr": 9.981987442712633e-05, "epoch": 1, "t": 5.082450866699219}
+{"step": 25, "loss": 1.2361458539962769, "grad_norm": 0.5346004962921143, "lr": 9.977206495402554e-05, "epoch": 1, "t": 4.976636171340942}
+{"step": 26, "loss": 1.269109845161438, "grad_norm": 0.5565136671066284, "lr": 9.971864890244513e-05, "epoch": 1, "t": 4.9557411670684814}
+{"step": 27, "loss": 1.205772876739502, "grad_norm": 0.5136733055114746, "lr": 9.965963228944078e-05, "epoch": 1, "t": 4.892717123031616}
+{"step": 28, "loss": 1.1230697631835938, "grad_norm": 0.5052443146705627, "lr": 9.959502176294383e-05, "epoch": 1, "t": 4.674621820449829}
+{"step": 29, "loss": 1.107112169265747, "grad_norm": 0.5233135223388672, "lr": 9.95248246010126e-05, "epoch": 1, "t": 4.344412803649902}
+{"step": 30, "loss": 1.2090479135513306, "grad_norm": 0.5024049878120422, "lr": 9.944904871101228e-05, "epoch": 1, "t": 5.341161727905273}
+{"step": 31, "loss": 1.255561113357544, "grad_norm": 0.5716925263404846, "lr": 9.936770262872443e-05, "epoch": 1, "t": 4.527912139892578}
+{"step": 32, "loss": 1.174899697303772, "grad_norm": 0.5564167499542236, "lr": 9.928079551738543e-05, "epoch": 1, "t": 4.685332536697388}
+{"step": 33, "loss": 1.1594640016555786, "grad_norm": 0.586647629737854, "lr": 9.918833716665419e-05, "epoch": 1, "t": 4.7574989795684814}
+{"step": 34, "loss": 1.2194890975952148, "grad_norm": 0.6168203353881836, "lr": 9.909033799150946e-05, "epoch": 1, "t": 4.320326566696167}
+{"step": 35, "loss": 1.1753389835357666, "grad_norm": 0.6045057773590088, "lr": 9.898680903107666e-05, "epoch": 1, "t": 4.5131001472473145}
+{"step": 36, "loss": 1.1112675666809082, "grad_norm": 0.5438430309295654, "lr": 9.887776194738432e-05, "epoch": 1, "t": 4.208852529525757}
+{"step": 37, "loss": 0.9868287444114685, "grad_norm": 0.5492530465126038, "lr": 9.876320902405042e-05, "epoch": 1, "t": 4.571593284606934}
+{"step": 38, "loss": 1.2119451761245728, "grad_norm": 0.5478034615516663, "lr": 9.864316316489873e-05, "epoch": 1, "t": 4.373960018157959}
+{"step": 39, "loss": 1.1598364114761353, "grad_norm": 0.5279031991958618, "lr": 9.851763789250525e-05, "epoch": 1, "t": 4.439868688583374}
+{"step": 40, "loss": 1.2911561727523804, "grad_norm": 0.561524510383606, "lr": 9.838664734667495e-05, "epoch": 1, "t": 4.412939786911011}
+{"step": 41, "loss": 1.1608424186706543, "grad_norm": 0.5272015333175659, "lr": 9.825020628284896e-05, "epoch": 1, "t": 4.482903242111206}
+{"step": 42, "loss": 1.1495981216430664, "grad_norm": 0.5450555086135864, "lr": 9.810833007044247e-05, "epoch": 1, "t": 4.218104124069214}
+{"step": 43, "loss": 1.0959913730621338, "grad_norm": 0.5038524866104126, "lr": 9.796103469111351e-05, "epoch": 1, "t": 4.903516530990601}
+{"step": 44, "loss": 1.0306528806686401, "grad_norm": 0.580023467540741, "lr": 9.780833673696254e-05, "epoch": 1, "t": 4.51888370513916}
+{"step": 45, "loss": 1.094822645187378, "grad_norm": 0.5627464652061462, "lr": 9.76502534086636e-05, "epoch": 1, "t": 4.369114875793457}
+{"step": 46, "loss": 1.0199453830718994, "grad_norm": 0.5088703632354736, "lr": 9.74868025135266e-05, "epoch": 1, "t": 4.441473722457886}
+{"step": 47, "loss": 1.2840280532836914, "grad_norm": 0.5627360343933105, "lr": 9.731800246349148e-05, "epoch": 1, "t": 4.7004714012146}
+{"step": 48, "loss": 1.1457535028457642, "grad_norm": 0.5226135849952698, "lr": 9.714387227305422e-05, "epoch": 1, "t": 4.882219552993774}
+{"step": 49, "loss": 1.1662487983703613, "grad_norm": 0.533386766910553, "lr": 9.696443155712486e-05, "epoch": 1, "t": 4.557215690612793}
+{"step": 50, "loss": 1.1086162328720093, "grad_norm": 0.5301807522773743, "lr": 9.67797005288181e-05, "epoch": 1, "t": 4.753899574279785}
+{"step": 51, "loss": 1.1031875610351562, "grad_norm": 0.6416805982589722, "lr": 9.65896999971763e-05, "epoch": 1, "t": 3.944395065307617}
+{"step": 52, "loss": 1.1556980609893799, "grad_norm": 0.5035192966461182, "lr": 9.639445136482548e-05, "epoch": 1, "t": 4.874272346496582}
+{"step": 53, "loss": 1.0844943523406982, "grad_norm": 0.6320053935050964, "lr": 9.619397662556435e-05, "epoch": 1, "t": 3.943636894226074}
+{"step": 54, "loss": 1.1466505527496338, "grad_norm": 0.5937464833259583, "lr": 9.598829836188694e-05, "epoch": 1, "t": 4.185172080993652}
+{"step": 55, "loss": 1.169416069984436, "grad_norm": 0.5557771921157837, "lr": 9.577743974243874e-05, "epoch": 1, "t": 4.230347633361816}
+{"step": 56, "loss": 1.1469193696975708, "grad_norm": 0.5882948637008667, "lr": 9.55614245194068e-05, "epoch": 1, "t": 4.496831655502319}
+{"step": 57, "loss": 1.2191901206970215, "grad_norm": 0.5372564196586609, "lr": 9.534027702584425e-05, "epoch": 1, "t": 4.598284959793091}
+{"step": 58, "loss": 1.1269330978393555, "grad_norm": 0.5687738060951233, "lr": 9.511402217292926e-05, "epoch": 1, "t": 4.195266962051392}
+{"step": 59, "loss": 1.2001675367355347, "grad_norm": 0.5253062844276428, "lr": 9.488268544715896e-05, "epoch": 1, "t": 5.253775358200073}
+{"step": 60, "loss": 1.059751272201538, "grad_norm": 0.5007998943328857, "lr": 9.464629290747842e-05, "epoch": 1, "t": 4.73201847076416}
+{"step": 61, "loss": 1.0079600811004639, "grad_norm": 0.5089836716651917, "lr": 9.440487118234535e-05, "epoch": 1, "t": 4.171860456466675}
+{"step": 62, "loss": 1.0606391429901123, "grad_norm": 0.5798132419586182, "lr": 9.415844746673047e-05, "epoch": 1, "t": 4.0498621463775635}
+{"step": 63, "loss": 1.1056632995605469, "grad_norm": 0.5342990159988403, "lr": 9.390704951905411e-05, "epoch": 1, "t": 4.358034133911133}
+{"step": 64, "loss": 1.1974354982376099, "grad_norm": 0.6184249520301819, "lr": 9.365070565805941e-05, "epoch": 1, "t": 4.097787141799927}
+{"step": 65, "loss": 1.238830327987671, "grad_norm": 0.5231149792671204, "lr": 9.338944475962237e-05, "epoch": 1, "t": 4.863854646682739}
+{"step": 66, "loss": 1.1731104850769043, "grad_norm": 0.5707106590270996, "lr": 9.312329625349902e-05, "epoch": 1, "t": 4.195391893386841}
+{"step": 67, "loss": 1.0703424215316772, "grad_norm": 0.5170727968215942, "lr": 9.285229012001047e-05, "epoch": 1, "t": 4.462587594985962}
+{"step": 68, "loss": 0.9579602479934692, "grad_norm": 0.5378182530403137, "lr": 9.257645688666556e-05, "epoch": 1, "t": 4.242594003677368}
+{"step": 69, "loss": 1.1021186113357544, "grad_norm": 0.6047465801239014, "lr": 9.22958276247223e-05, "epoch": 1, "t": 4.46303653717041}
+{"step": 70, "loss": 1.2138588428497314, "grad_norm": 0.5210340023040771, "lr": 9.201043394568773e-05, "epoch": 1, "t": 5.061203241348267}
+{"step": 71, "loss": 1.281541347503662, "grad_norm": 0.589238703250885, "lr": 9.172030799775699e-05, "epoch": 1, "t": 4.427860736846924}
+{"step": 72, "loss": 1.0390024185180664, "grad_norm": 0.5161624550819397, "lr": 9.142548246219212e-05, "epoch": 1, "t": 4.686448812484741}
+{"step": 73, "loss": 0.9110280275344849, "grad_norm": 0.5641354918479919, "lr": 9.112599054964057e-05, "epoch": 1, "t": 4.032199859619141}
+{"step": 74, "loss": 1.2140576839447021, "grad_norm": 0.506525456905365, "lr": 9.082186599639428e-05, "epoch": 1, "t": 4.954009056091309}
+{"step": 75, "loss": 1.1521708965301514, "grad_norm": 0.5057822465896606, "lr": 9.051314306058933e-05, "epoch": 1, "t": 4.658722162246704}
+{"step": 76, "loss": 1.1215031147003174, "grad_norm": 0.48477110266685486, "lr": 9.019985651834703e-05, "epoch": 1, "t": 4.488047361373901}
+{"step": 77, "loss": 1.0814063549041748, "grad_norm": 0.508513331413269, "lr": 8.988204165985649e-05, "epoch": 1, "t": 4.154196500778198}
+{"step": 78, "loss": 1.098646879196167, "grad_norm": 0.5605230331420898, "lr": 8.955973428539944e-05, "epoch": 1, "t": 4.091546297073364}
+{"step": 79, "loss": 1.0545575618743896, "grad_norm": 0.4966557025909424, "lr": 8.923297070131737e-05, "epoch": 1, "t": 4.371098041534424}
+{"step": 80, "loss": 1.2122673988342285, "grad_norm": 0.5091938376426697, "lr": 8.890178771592199e-05, "epoch": 1, "t": 4.502809286117554}
+{"step": 81, "loss": 1.0079820156097412, "grad_norm": 0.6124518513679504, "lr": 8.856622263534875e-05, "epoch": 1, "t": 4.4716479778289795}
+{"step": 82, "loss": 1.084323763847351, "grad_norm": 0.6137211918830872, "lr": 8.822631325935463e-05, "epoch": 1, "t": 4.10309100151062}
+{"step": 83, "loss": 1.0084223747253418, "grad_norm": 0.5893185138702393, "lr": 8.788209787706015e-05, "epoch": 1, "t": 3.7443480491638184}
+{"step": 84, "loss": 1.0328024625778198, "grad_norm": 0.7217421531677246, "lr": 8.753361526263621e-05, "epoch": 1, "t": 5.0470075607299805}
+{"step": 85, "loss": 1.0865235328674316, "grad_norm": 0.5847263336181641, "lr": 8.718090467093654e-05, "epoch": 1, "t": 4.688053369522095}
+{"step": 86, "loss": 1.0836131572723389, "grad_norm": 0.9780337810516357, "lr": 8.682400583307562e-05, "epoch": 1, "t": 4.2437474727630615}
+{"step": 87, "loss": 1.0456180572509766, "grad_norm": 0.5278544425964355, "lr": 8.646295895195333e-05, "epoch": 1, "t": 4.557479381561279}
+{"step": 88, "loss": 1.0994131565093994, "grad_norm": 0.5237377882003784, "lr": 8.609780469772623e-05, "epoch": 1, "t": 4.438060522079468}
+{"step": 89, "loss": 1.1217817068099976, "grad_norm": 0.56512850522995, "lr": 8.572858420322627e-05, "epoch": 1, "t": 4.071814060211182}
+{"step": 90, "loss": 1.1419236660003662, "grad_norm": 0.5304071307182312, "lr": 8.535533905932738e-05, "epoch": 1, "t": 4.36553168296814}
+{"step": 91, "loss": 1.0185911655426025, "grad_norm": 0.48035383224487305, "lr": 8.497811131026046e-05, "epoch": 1, "t": 4.525289058685303}
+{"step": 92, "loss": 1.0598928928375244, "grad_norm": 0.5172107219696045, "lr": 8.459694344887732e-05, "epoch": 1, "t": 4.6219446659088135}
+{"step": 93, "loss": 1.161812424659729, "grad_norm": 0.5744761824607849, "lr": 8.421187841186402e-05, "epoch": 1, "t": 4.073018789291382}
+{"step": 94, "loss": 1.0621559619903564, "grad_norm": 0.5731527209281921, "lr": 8.382295957490436e-05, "epoch": 1, "t": 4.601446628570557}
+{"step": 95, "loss": 1.1462032794952393, "grad_norm": 0.5267546772956848, "lr": 8.343023074779368e-05, "epoch": 1, "t": 4.446707010269165}
+{"step": 96, "loss": 1.020163655281067, "grad_norm": 0.5306865572929382, "lr": 8.303373616950408e-05, "epoch": 1, "t": 4.762771368026733}
+{"step": 97, "loss": 1.1626460552215576, "grad_norm": 0.5645160675048828, "lr": 8.263352050320094e-05, "epoch": 1, "t": 4.449122905731201}
+{"step": 98, "loss": 1.0628409385681152, "grad_norm": 0.513582706451416, "lr": 8.222962883121196e-05, "epoch": 1, "t": 4.577300071716309}
+{"step": 99, "loss": 1.0532479286193848, "grad_norm": 0.5487250685691833, "lr": 8.182210664994878e-05, "epoch": 1, "t": 4.386420965194702}
+{"step": 100, "loss": 0.9273568391799927, "grad_norm": 0.5190582275390625, "lr": 8.141099986478212e-05, "epoch": 1, "t": 4.387535333633423}
+{"step": 101, "loss": 1.1057411432266235, "grad_norm": 0.4992278516292572, "lr": 8.099635478487064e-05, "epoch": 1, "t": 4.819383144378662}
+{"step": 102, "loss": 1.123875379562378, "grad_norm": 0.5524612665176392, "lr": 8.057821811794458e-05, "epoch": 1, "t": 4.398582220077515}
+{"step": 103, "loss": 1.0887521505355835, "grad_norm": 0.565819263458252, "lr": 8.015663696504422e-05, "epoch": 1, "t": 4.442178010940552}
+{"step": 104, "loss": 1.054804801940918, "grad_norm": 0.5269464254379272, "lr": 7.973165881521434e-05, "epoch": 1, "t": 4.3145434856414795}
+{"step": 105, "loss": 1.1132597923278809, "grad_norm": 0.5262883305549622, "lr": 7.930333154015466e-05, "epoch": 1, "t": 4.597621440887451}
+{"step": 106, "loss": 1.1616182327270508, "grad_norm": 0.5879294276237488, "lr": 7.88717033888274e-05, "epoch": 1, "t": 3.9895129203796387}
+{"step": 107, "loss": 1.047626256942749, "grad_norm": 0.5814321637153625, "lr": 7.843682298202235e-05, "epoch": 1, "t": 4.23612642288208}
+{"step": 108, "loss": 1.144451379776001, "grad_norm": 0.5584962368011475, "lr": 7.799873930687978e-05, "epoch": 1, "t": 4.495697736740112}
+{"step": 109, "loss": 1.026404619216919, "grad_norm": 0.5226858854293823, "lr": 7.755750171137246e-05, "epoch": 1, "t": 4.320061683654785}
+{"step": 110, "loss": 1.0900294780731201, "grad_norm": 0.53071528673172, "lr": 7.711315989874677e-05, "epoch": 1, "t": 4.577735185623169}
+{"step": 111, "loss": 1.0614886283874512, "grad_norm": 0.608755886554718, "lr": 7.666576392192389e-05, "epoch": 1, "t": 3.928135395050049}
+{"step": 112, "loss": 1.106485366821289, "grad_norm": 0.5223899483680725, "lr": 7.621536417786159e-05, "epoch": 1, "t": 4.3053624629974365}
+{"step": 113, "loss": 1.1297920942306519, "grad_norm": 0.5254682302474976, "lr": 7.576201140187727e-05, "epoch": 1, "t": 5.031718730926514}
+{"step": 114, "loss": 1.0280169248580933, "grad_norm": 0.5055007338523865, "lr": 7.530575666193283e-05, "epoch": 1, "t": 4.56247091293335}
+{"step": 115, "loss": 1.1302471160888672, "grad_norm": 0.5785984992980957, "lr": 7.484665135288213e-05, "epoch": 1, "t": 4.644721031188965}
+{"step": 116, "loss": 1.120363712310791, "grad_norm": 0.5624465346336365, "lr": 7.438474719068173e-05, "epoch": 1, "t": 4.270162343978882}
+{"step": 117, "loss": 1.0795762538909912, "grad_norm": 0.51219242811203, "lr": 7.392009620656513e-05, "epoch": 1, "t": 4.882677316665649}
+{"step": 118, "loss": 1.1195564270019531, "grad_norm": 0.5691790580749512, "lr": 7.345275074118185e-05, "epoch": 1, "t": 4.51539158821106}
+{"step": 119, "loss": 1.0504623651504517, "grad_norm": 0.557971715927124, "lr": 7.298276343870151e-05, "epoch": 1, "t": 4.341449737548828}
+{"step": 120, "loss": 1.022754192352295, "grad_norm": 0.5085095763206482, "lr": 7.251018724088367e-05, "epoch": 1, "t": 4.581089019775391}
+{"step": 121, "loss": 1.1930490732192993, "grad_norm": 0.5402499437332153, "lr": 7.203507538111423e-05, "epoch": 1, "t": 4.29120945930481}
+{"step": 122, "loss": 0.9849798083305359, "grad_norm": 0.4865000545978546, "lr": 7.155748137840892e-05, "epoch": 1, "t": 4.833763837814331}
+{"step": 123, "loss": 1.0634889602661133, "grad_norm": 0.5702819228172302, "lr": 7.107745903138472e-05, "epoch": 1, "t": 3.9815304279327393}
+{"step": 124, "loss": 1.0977647304534912, "grad_norm": 0.5151114463806152, "lr": 7.059506241219965e-05, "epoch": 1, "t": 5.006288051605225}
+{"step": 125, "loss": 0.9465334415435791, "grad_norm": 0.601161777973175, "lr": 7.011034586046176e-05, "epoch": 1, "t": 3.9056437015533447}
+{"step": 126, "loss": 1.1422866582870483, "grad_norm": 0.5249250531196594, "lr": 6.962336397710819e-05, "epoch": 1, "t": 4.667974948883057}
+{"step": 127, "loss": 1.0053578615188599, "grad_norm": 0.480178564786911, "lr": 6.91341716182545e-05, "epoch": 1, "t": 4.703177213668823}
+{"step": 128, "loss": 0.9437476396560669, "grad_norm": 0.6007505655288696, "lr": 6.864282388901544e-05, "epoch": 1, "t": 3.937471866607666}
+{"step": 129, "loss": 0.9948303699493408, "grad_norm": 0.5376791954040527, "lr": 6.814937613729766e-05, "epoch": 1, "t": 4.37124228477478}
+{"step": 130, "loss": 1.1242518424987793, "grad_norm": 0.5657219290733337, "lr": 6.765388394756504e-05, "epoch": 1, "t": 4.458618640899658}
+{"step": 131, "loss": 1.0851949453353882, "grad_norm": 0.5400698781013489, "lr": 6.715640313457733e-05, "epoch": 1, "t": 4.658433437347412}
+{"step": 132, "loss": 1.0876405239105225, "grad_norm": 0.5429879426956177, "lr": 6.665698973710288e-05, "epoch": 1, "t": 4.376655578613281}
+{"step": 133, "loss": 1.0334701538085938, "grad_norm": 0.5221796631813049, "lr": 6.615570001160626e-05, "epoch": 1, "t": 4.503961801528931}
+{"step": 134, "loss": 1.0760217905044556, "grad_norm": 0.6059476733207703, "lr": 6.565259042591113e-05, "epoch": 1, "t": 4.067153453826904}
+{"step": 135, "loss": 1.0673279762268066, "grad_norm": 0.5496578216552734, "lr": 6.514771765283942e-05, "epoch": 1, "t": 4.437847852706909}
+{"step": 136, "loss": 1.123016595840454, "grad_norm": 0.5759939551353455, "lr": 6.464113856382752e-05, "epoch": 1, "t": 4.666795253753662}
+{"step": 137, "loss": 1.092640995979309, "grad_norm": 0.5213869214057922, "lr": 6.413291022251989e-05, "epoch": 1, "t": 4.291595935821533}
+{"step": 138, "loss": 1.164132833480835, "grad_norm": 0.5406259894371033, "lr": 6.362308987834115e-05, "epoch": 1, "t": 4.620205640792847}
+{"step": 139, "loss": 1.030076503753662, "grad_norm": 0.5251417756080627, "lr": 6.311173496004723e-05, "epoch": 1, "t": 4.693472862243652}
+{"step": 140, "loss": 1.1202666759490967, "grad_norm": 0.5583895444869995, "lr": 6.259890306925627e-05, "epoch": 1, "t": 4.033910751342773}
+{"step": 141, "loss": 0.973567008972168, "grad_norm": 0.537427544593811, "lr": 6.208465197396013e-05, "epoch": 1, "t": 4.544414520263672}
+{"step": 142, "loss": 1.018080234527588, "grad_norm": 0.5596265196800232, "lr": 6.156903960201709e-05, "epoch": 1, "t": 4.622097969055176}
+{"step": 143, "loss": 1.0365535020828247, "grad_norm": 0.59474116563797, "lr": 6.105212403462651e-05, "epoch": 1, "t": 4.769702911376953}
+{"step": 144, "loss": 1.123967170715332, "grad_norm": 0.47663769125938416, "lr": 6.0533963499786314e-05, "epoch": 1, "t": 5.11264181137085}
+{"step": 145, "loss": 0.921715497970581, "grad_norm": 0.5195542573928833, "lr": 6.001461636573397e-05, "epoch": 1, "t": 4.1595940589904785}
+{"step": 146, "loss": 1.0620217323303223, "grad_norm": 0.5452298521995544, "lr": 5.949414113437142e-05, "epoch": 1, "t": 4.677001953125}
+{"step": 147, "loss": 1.0726895332336426, "grad_norm": 0.5352897047996521, "lr": 5.897259643467527e-05, "epoch": 1, "t": 4.307942867279053}
+{"step": 148, "loss": 1.1086431741714478, "grad_norm": 0.5879285335540771, "lr": 5.8450041016092464e-05, "epoch": 1, "t": 4.195312261581421}
+{"step": 149, "loss": 1.0972446203231812, "grad_norm": 0.5766710042953491, "lr": 5.792653374192245e-05, "epoch": 1, "t": 4.259337902069092}
+{"step": 150, "loss": 1.0178098678588867, "grad_norm": 0.6017856597900391, "lr": 5.7402133582686576e-05, "epoch": 1, "t": 3.9317562580108643}
+{"step": 151, "loss": 1.078547477722168, "grad_norm": 0.5264674425125122, "lr": 5.6876899609485256e-05, "epoch": 1, "t": 4.49289608001709}
+{"step": 152, "loss": 1.102089524269104, "grad_norm": 0.5552437901496887, "lr": 5.6350890987343944e-05, "epoch": 1, "t": 4.442862510681152}
+{"step": 153, "loss": 1.0655879974365234, "grad_norm": 0.4840118885040283, "lr": 5.582416696854853e-05, "epoch": 1, "t": 4.819095611572266}
+{"step": 154, "loss": 0.918845534324646, "grad_norm": 0.5310778021812439, "lr": 5.5296786885970805e-05, "epoch": 1, "t": 4.513690710067749}
+{"step": 155, "loss": 1.1092482805252075, "grad_norm": 0.5673016309738159, "lr": 5.476881014638491e-05, "epoch": 1, "t": 4.168788194656372}
+{"step": 156, "loss": 1.1490557193756104, "grad_norm": 0.4792988896369934, "lr": 5.4240296223775465e-05, "epoch": 1, "t": 4.933318614959717}
+{"step": 157, "loss": 1.0512540340423584, "grad_norm": 0.5112766623497009, "lr": 5.3711304652638126e-05, "epoch": 1, "t": 5.074463844299316}
+{"step": 158, "loss": 1.027459740638733, "grad_norm": 0.5209557414054871, "lr": 5.318189502127332e-05, "epoch": 1, "t": 4.507224798202515}
+{"step": 159, "loss": 1.0976512432098389, "grad_norm": 0.5071336030960083, "lr": 5.265212696507387e-05, "epoch": 1, "t": 4.511084318161011}
+{"step": 160, "loss": 1.1377782821655273, "grad_norm": 0.48624223470687866, "lr": 5.212206015980742e-05, "epoch": 1, "t": 5.3057146072387695}
+{"step": 161, "loss": 1.1065459251403809, "grad_norm": 0.6089529395103455, "lr": 5.159175431489424e-05, "epoch": 1, "t": 4.1306538581848145}
+{"step": 162, "loss": 1.157946228981018, "grad_norm": 0.5385098457336426, "lr": 5.1061269166681183e-05, "epoch": 1, "t": 4.367673397064209}
+{"step": 163, "loss": 0.9364073276519775, "grad_norm": 0.6155913472175598, "lr": 5.053066447171282e-05, "epoch": 1, "t": 3.6702613830566406}
+{"step": 164, "loss": 1.0799977779388428, "grad_norm": 0.49823373556137085, "lr": 5e-05, "epoch": 1, "t": 4.393790006637573}
+{"step": 165, "loss": 1.0996894836425781, "grad_norm": 0.5013274550437927, "lr": 4.94693355282872e-05, "epoch": 1, "t": 4.7591612339019775}
+{"step": 166, "loss": 1.0865201950073242, "grad_norm": 0.5232498645782471, "lr": 4.893873083331882e-05, "epoch": 1, "t": 4.372714519500732}
+{"step": 167, "loss": 1.0524868965148926, "grad_norm": 0.5000494122505188, "lr": 4.840824568510579e-05, "epoch": 1, "t": 4.435556411743164}
+{"step": 168, "loss": 0.9212394952774048, "grad_norm": 0.5707553029060364, "lr": 4.78779398401926e-05, "epoch": 1, "t": 4.072383165359497}
+{"step": 169, "loss": 1.2808607816696167, "grad_norm": 0.5759779810905457, "lr": 4.734787303492615e-05, "epoch": 1, "t": 4.717279434204102}
+{"step": 170, "loss": 1.1311876773834229, "grad_norm": 0.4997013807296753, "lr": 4.6818104978726685e-05, "epoch": 1, "t": 4.609082460403442}
+{"step": 171, "loss": 1.128800868988037, "grad_norm": 0.5306641459465027, "lr": 4.628869534736187e-05, "epoch": 1, "t": 4.519702434539795}
+{"step": 172, "loss": 0.9134346842765808, "grad_norm": 0.4742140769958496, "lr": 4.575970377622456e-05, "epoch": 1, "t": 4.838714599609375}
+{"step": 173, "loss": 1.0570604801177979, "grad_norm": 0.5552225112915039, "lr": 4.52311898536151e-05, "epoch": 1, "t": 4.4790778160095215}
+{"step": 174, "loss": 1.019338846206665, "grad_norm": 0.5955643057823181, "lr": 4.47032131140292e-05, "epoch": 1, "t": 4.5283355712890625}
+{"step": 175, "loss": 1.0274667739868164, "grad_norm": 0.5497900247573853, "lr": 4.4175833031451473e-05, "epoch": 1, "t": 4.457115888595581}
+{"step": 176, "loss": 1.086936593055725, "grad_norm": 0.5342774391174316, "lr": 4.364910901265606e-05, "epoch": 1, "t": 4.374600887298584}
+{"step": 177, "loss": 1.0455431938171387, "grad_norm": 0.5045028924942017, "lr": 4.3123100390514756e-05, "epoch": 1, "t": 4.365716934204102}
+{"step": 178, "loss": 1.0901545286178589, "grad_norm": 0.5298687815666199, "lr": 4.2597866417313436e-05, "epoch": 1, "t": 4.253988027572632}
+{"step": 179, "loss": 1.1432433128356934, "grad_norm": 0.5694761276245117, "lr": 4.207346625807756e-05, "epoch": 1, "t": 4.1064453125}
+{"step": 180, "loss": 1.1150802373886108, "grad_norm": 0.5988235473632812, "lr": 4.1549958983907555e-05, "epoch": 1, "t": 4.258797883987427}
+{"step": 181, "loss": 1.0617355108261108, "grad_norm": 0.48805877566337585, "lr": 4.102740356532473e-05, "epoch": 1, "t": 4.510316371917725}
+{"step": 182, "loss": 1.0038782358169556, "grad_norm": 0.5585227012634277, "lr": 4.050585886562858e-05, "epoch": 1, "t": 4.691247224807739}
+{"step": 183, "loss": 1.024353265762329, "grad_norm": 0.6235622763633728, "lr": 3.998538363426605e-05, "epoch": 1, "t": 3.9281091690063477}
+{"step": 184, "loss": 1.1303329467773438, "grad_norm": 0.614143967628479, "lr": 3.94660365002137e-05, "epoch": 1, "t": 3.929875135421753}
+{"step": 185, "loss": 1.0341070890426636, "grad_norm": 0.5180628299713135, "lr": 3.894787596537352e-05, "epoch": 1, "t": 5.039501428604126}
+{"step": 186, "loss": 1.04641592502594, "grad_norm": 0.6050846576690674, "lr": 3.843096039798293e-05, "epoch": 1, "t": 3.867766857147217}
+{"step": 187, "loss": 1.1024057865142822, "grad_norm": 0.5669552087783813, "lr": 3.791534802603988e-05, "epoch": 1, "t": 4.555077791213989}
+{"step": 188, "loss": 1.1636323928833008, "grad_norm": 0.4995519816875458, "lr": 3.740109693074375e-05, "epoch": 1, "t": 4.734290599822998}
+{"step": 189, "loss": 1.0735996961593628, "grad_norm": 0.5093196034431458, "lr": 3.68882650399528e-05, "epoch": 1, "t": 4.841955900192261}
+{"step": 190, "loss": 1.0439932346343994, "grad_norm": 0.5297228693962097, "lr": 3.637691012165886e-05, "epoch": 1, "t": 4.3820412158966064}
+{"step": 191, "loss": 1.0277173519134521, "grad_norm": 0.6291950345039368, "lr": 3.586708977748012e-05, "epoch": 1, "t": 4.358238220214844}
+{"step": 192, "loss": 1.0819365978240967, "grad_norm": 0.5827075839042664, "lr": 3.5358861436172485e-05, "epoch": 1, "t": 4.519319295883179}
+{"step": 193, "loss": 1.0004122257232666, "grad_norm": 0.5812299847602844, "lr": 3.485228234716058e-05, "epoch": 1, "t": 4.060560941696167}
+{"step": 194, "loss": 1.0828702449798584, "grad_norm": 0.6025108098983765, "lr": 3.434740957408889e-05, "epoch": 1, "t": 4.175429105758667}
+{"step": 195, "loss": 1.1582586765289307, "grad_norm": 0.49842584133148193, "lr": 3.3844299988393755e-05, "epoch": 1, "t": 5.127326250076294}
+{"step": 196, "loss": 0.9903317093849182, "grad_norm": 0.533919632434845, "lr": 3.334301026289712e-05, "epoch": 1, "t": 4.460814952850342}
+{"step": 197, "loss": 0.9877713918685913, "grad_norm": 0.5166082382202148, "lr": 3.284359686542269e-05, "epoch": 1, "t": 4.177352666854858}
+{"step": 198, "loss": 1.1141719818115234, "grad_norm": 0.5050948858261108, "lr": 3.234611605243496e-05, "epoch": 1, "t": 5.06512713432312}
+{"step": 199, "loss": 1.0266294479370117, "grad_norm": 0.5247622728347778, "lr": 3.1850623862702344e-05, "epoch": 1, "t": 4.668641090393066}
+{"step": 200, "loss": 1.0133147239685059, "grad_norm": 0.49373194575309753, "lr": 3.135717611098458e-05, "epoch": 1, "t": 5.040018081665039}
+{"step": 201, "loss": 1.0639673471450806, "grad_norm": 0.5062780976295471, "lr": 3.086582838174551e-05, "epoch": 1, "t": 4.643780708312988}
+{"step": 202, "loss": 1.0242486000061035, "grad_norm": 0.8172736763954163, "lr": 3.0376636022891812e-05, "epoch": 1, "t": 4.386077404022217}
+{"step": 203, "loss": 1.012434959411621, "grad_norm": 0.5004093050956726, "lr": 2.9889654139538246e-05, "epoch": 1, "t": 4.6061179637908936}
+{"step": 204, "loss": 1.0407299995422363, "grad_norm": 0.5462143421173096, "lr": 2.9404937587800375e-05, "epoch": 1, "t": 4.1180195808410645}
+{"step": 205, "loss": 0.9675360321998596, "grad_norm": 0.5693405270576477, "lr": 2.8922540968615286e-05, "epoch": 1, "t": 4.175534009933472}
+{"step": 206, "loss": 1.07175612449646, "grad_norm": 0.541511595249176, "lr": 2.8442518621591086e-05, "epoch": 1, "t": 4.615241050720215}
+{"step": 207, "loss": 1.1230800151824951, "grad_norm": 0.4979943335056305, "lr": 2.7964924618885778e-05, "epoch": 1, "t": 5.644282817840576}
+{"step": 208, "loss": 1.0087041854858398, "grad_norm": 0.538437008857727, "lr": 2.748981275911633e-05, "epoch": 1, "t": 4.2667787075042725}
+{"step": 209, "loss": 1.0306894779205322, "grad_norm": 0.542610764503479, "lr": 2.701723656129851e-05, "epoch": 1, "t": 4.277086019515991}
+{"step": 210, "loss": 1.0517539978027344, "grad_norm": 0.6919747591018677, "lr": 2.6547249258818164e-05, "epoch": 1, "t": 4.13981556892395}
+{"step": 211, "loss": 0.9890724420547485, "grad_norm": 0.5264641046524048, "lr": 2.607990379343489e-05, "epoch": 1, "t": 4.133480548858643}
+{"step": 212, "loss": 1.045159935951233, "grad_norm": 0.6240392923355103, "lr": 2.5615252809318284e-05, "epoch": 1, "t": 4.225189924240112}
+{"step": 213, "loss": 1.0057790279388428, "grad_norm": 0.6362853050231934, "lr": 2.5153348647117857e-05, "epoch": 1, "t": 4.619581460952759}
+{"step": 214, "loss": 1.063778281211853, "grad_norm": 0.5307620763778687, "lr": 2.469424333806718e-05, "epoch": 1, "t": 4.561526298522949}
+{"step": 215, "loss": 0.9758619070053101, "grad_norm": 0.5621666312217712, "lr": 2.4237988598122752e-05, "epoch": 1, "t": 4.512598991394043}
+{"step": 216, "loss": 1.0140074491500854, "grad_norm": 0.5423521399497986, "lr": 2.3784635822138424e-05, "epoch": 1, "t": 4.338836908340454}
+{"step": 217, "loss": 0.972283661365509, "grad_norm": 0.5642887949943542, "lr": 2.333423607807613e-05, "epoch": 1, "t": 4.00283145904541}
+{"step": 218, "loss": 1.0538939237594604, "grad_norm": 0.5357561707496643, "lr": 2.288684010125325e-05, "epoch": 1, "t": 4.343484878540039}
+{"step": 219, "loss": 1.0054504871368408, "grad_norm": 0.5771626830101013, "lr": 2.2442498288627556e-05, "epoch": 1, "t": 4.1507909297943115}
+{"step": 220, "loss": 1.0451167821884155, "grad_norm": 0.542662501335144, "lr": 2.2001260693120233e-05, "epoch": 1, "t": 4.722820520401001}
+{"step": 221, "loss": 1.1285812854766846, "grad_norm": 0.5216588377952576, "lr": 2.156317701797766e-05, "epoch": 1, "t": 5.069575309753418}
+{"step": 222, "loss": 1.0853524208068848, "grad_norm": 0.7681800127029419, "lr": 2.1128296611172593e-05, "epoch": 1, "t": 5.344943523406982}
+{"step": 223, "loss": 1.033549189567566, "grad_norm": 0.6008373498916626, "lr": 2.0696668459845355e-05, "epoch": 1, "t": 5.082601070404053}
+{"step": 224, "loss": 1.0079463720321655, "grad_norm": 0.5472951531410217, "lr": 2.026834118478567e-05, "epoch": 1, "t": 4.886492490768433}
+{"step": 225, "loss": 1.0790884494781494, "grad_norm": 0.5625029802322388, "lr": 1.98433630349558e-05, "epoch": 1, "t": 3.9033923149108887}
+{"step": 226, "loss": 1.0042743682861328, "grad_norm": 0.5030620694160461, "lr": 1.9421781882055444e-05, "epoch": 1, "t": 5.301595687866211}
+{"step": 227, "loss": 1.1190781593322754, "grad_norm": 0.5105768442153931, "lr": 1.9003645215129355e-05, "epoch": 1, "t": 4.672891616821289}
+{"step": 228, "loss": 1.0224028825759888, "grad_norm": 0.49649953842163086, "lr": 1.858900013521788e-05, "epoch": 1, "t": 4.957526206970215}
+{"step": 229, "loss": 1.1677991151809692, "grad_norm": 0.5419648885726929, "lr": 1.817789335005121e-05, "epoch": 1, "t": 4.478188991546631}
+{"step": 230, "loss": 0.9765380024909973, "grad_norm": 0.5728964805603027, "lr": 1.777037116878804e-05, "epoch": 1, "t": 4.242607831954956}
+{"step": 231, "loss": 0.9520196914672852, "grad_norm": 0.5413674712181091, "lr": 1.7366479496799077e-05, "epoch": 1, "t": 3.9896466732025146}
+{"step": 232, "loss": 1.0150930881500244, "grad_norm": 0.5681119561195374, "lr": 1.6966263830495936e-05, "epoch": 1, "t": 4.406031847000122}
+{"step": 233, "loss": 1.0148005485534668, "grad_norm": 0.5655505061149597, "lr": 1.656976925220633e-05, "epoch": 1, "t": 4.484252452850342}
+{"step": 234, "loss": 1.0925610065460205, "grad_norm": 0.5731245279312134, "lr": 1.6177040425095662e-05, "epoch": 1, "t": 4.79651665687561}
+{"step": 235, "loss": 1.0736114978790283, "grad_norm": 0.5150059461593628, "lr": 1.5788121588135975e-05, "epoch": 1, "t": 5.232740640640259}
+{"step": 236, "loss": 1.098027229309082, "grad_norm": 0.524219810962677, "lr": 1.5403056551122697e-05, "epoch": 1, "t": 4.718247890472412}
+{"step": 237, "loss": 1.0204665660858154, "grad_norm": 0.5716584324836731, "lr": 1.5021888689739549e-05, "epoch": 1, "t": 4.200188875198364}
+{"step": 238, "loss": 0.8543922901153564, "grad_norm": 0.538303792476654, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 4.098372459411621}
+{"step": 239, "loss": 0.9970686435699463, "grad_norm": 0.5662669539451599, "lr": 1.427141579677374e-05, "epoch": 1, "t": 4.319905519485474}
+{"step": 240, "loss": 1.0023155212402344, "grad_norm": 0.5340131521224976, "lr": 1.3902195302273779e-05, "epoch": 1, "t": 4.177032709121704}
+{"step": 241, "loss": 0.9963352084159851, "grad_norm": 0.56769859790802, "lr": 1.3537041048046695e-05, "epoch": 1, "t": 4.03005313873291}
+{"step": 242, "loss": 0.9301167726516724, "grad_norm": 0.5520228743553162, "lr": 1.3175994166924394e-05, "epoch": 1, "t": 4.708632707595825}
+{"step": 243, "loss": 1.0148307085037231, "grad_norm": 0.4941009283065796, "lr": 1.2819095329063469e-05, "epoch": 1, "t": 4.69291615486145}
+{"step": 244, "loss": 1.0223767757415771, "grad_norm": 0.5227289795875549, "lr": 1.246638473736378e-05, "epoch": 1, "t": 4.218645334243774}
+{"step": 245, "loss": 1.0138245820999146, "grad_norm": 0.6000615954399109, "lr": 1.2117902122939861e-05, "epoch": 1, "t": 3.7534725666046143}
+{"step": 246, "loss": 1.1209429502487183, "grad_norm": 0.5765858292579651, "lr": 1.1773686740645384e-05, "epoch": 1, "t": 4.109350204467773}
+{"step": 247, "loss": 1.0544321537017822, "grad_norm": 0.6045498251914978, "lr": 1.1433777364651271e-05, "epoch": 1, "t": 4.278370141983032}
+{"step": 248, "loss": 1.0104535818099976, "grad_norm": 0.5245023965835571, "lr": 1.1098212284078036e-05, "epoch": 1, "t": 4.8333704471588135}
+{"step": 249, "loss": 0.9180708527565002, "grad_norm": 0.5768724679946899, "lr": 1.076702929868264e-05, "epoch": 1, "t": 4.094990968704224}
+{"step": 250, "loss": 1.1898705959320068, "grad_norm": 0.4956952631473541, "lr": 1.0440265714600572e-05, "epoch": 1, "t": 4.909663438796997}
+{"step": 251, "loss": 1.1065311431884766, "grad_norm": 0.559075653553009, "lr": 1.0117958340143507e-05, "epoch": 1, "t": 4.577718496322632}
+{"step": 252, "loss": 0.9776133894920349, "grad_norm": 0.5778295397758484, "lr": 9.800143481652979e-06, "epoch": 1, "t": 4.062043905258179}
+{"step": 253, "loss": 1.1803488731384277, "grad_norm": 0.8529964089393616, "lr": 9.48685693941067e-06, "epoch": 1, "t": 4.498236417770386}
+{"step": 254, "loss": 1.0039808750152588, "grad_norm": 0.5118018984794617, "lr": 9.17813400360572e-06, "epoch": 1, "t": 4.851653337478638}
+{"step": 255, "loss": 1.109760046005249, "grad_norm": 0.5212645530700684, "lr": 8.874009450359427e-06, "epoch": 1, "t": 4.860409736633301}
+{"step": 256, "loss": 1.0329498052597046, "grad_norm": 0.6550596356391907, "lr": 8.574517537807897e-06, "epoch": 1, "t": 4.919097661972046}
+{"step": 257, "loss": 1.0815367698669434, "grad_norm": 0.5947718620300293, "lr": 8.279692002243027e-06, "epoch": 1, "t": 4.545341491699219}
+{"step": 258, "loss": 1.0716111660003662, "grad_norm": 0.5768417119979858, "lr": 7.989566054312287e-06, "epoch": 1, "t": 4.85042142868042}
+{"step": 259, "loss": 1.0296393632888794, "grad_norm": 0.5635685920715332, "lr": 7.704172375277691e-06, "epoch": 1, "t": 4.746562242507935}
+{"step": 260, "loss": 1.118285059928894, "grad_norm": 0.5632694363594055, "lr": 7.423543113334436e-06, "epoch": 1, "t": 4.605138540267944}
+{"step": 261, "loss": 1.0031137466430664, "grad_norm": 0.6423073410987854, "lr": 7.14770987998954e-06, "epoch": 1, "t": 4.276600360870361}
+{"step": 262, "loss": 0.8432674407958984, "grad_norm": 0.6431649923324585, "lr": 6.876703746500984e-06, "epoch": 1, "t": 3.690114736557007}
+{"step": 263, "loss": 1.100170612335205, "grad_norm": 0.5560281276702881, "lr": 6.610555240377652e-06, "epoch": 1, "t": 4.329102039337158}
+{"step": 264, "loss": 0.9757599830627441, "grad_norm": 0.6408441662788391, "lr": 6.349294341940593e-06, "epoch": 1, "t": 4.03907036781311}
+{"step": 265, "loss": 1.1450011730194092, "grad_norm": 0.5687059164047241, "lr": 6.092950480945897e-06, "epoch": 1, "t": 4.864096164703369}
+{"step": 266, "loss": 1.0518672466278076, "grad_norm": 0.6086743474006653, "lr": 5.841552533269534e-06, "epoch": 1, "t": 4.132091522216797}
+{"step": 267, "loss": 1.063183307647705, "grad_norm": 0.5224151015281677, "lr": 5.595128817654638e-06, "epoch": 1, "t": 4.723138093948364}
+{"step": 268, "loss": 1.126554012298584, "grad_norm": 0.5347367525100708, "lr": 5.353707092521582e-06, "epoch": 1, "t": 4.741367340087891}
+{"step": 269, "loss": 1.0698590278625488, "grad_norm": 0.5909562110900879, "lr": 5.117314552841052e-06, "epoch": 1, "t": 4.328283309936523}
+{"step": 270, "loss": 0.9937804341316223, "grad_norm": 0.5930314064025879, "lr": 4.885977827070748e-06, "epoch": 1, "t": 4.099947929382324}
+{"step": 271, "loss": 0.9854071140289307, "grad_norm": 0.5892282724380493, "lr": 4.659722974155767e-06, "epoch": 1, "t": 4.106796503067017}
+{"step": 272, "loss": 0.9598199129104614, "grad_norm": 0.5163775682449341, "lr": 4.43857548059321e-06, "epoch": 1, "t": 4.5606396198272705}
+{"step": 273, "loss": 1.0887280702590942, "grad_norm": 0.6039618849754333, "lr": 4.2225602575612755e-06, "epoch": 1, "t": 4.001786231994629}
+{"step": 274, "loss": 0.9749129414558411, "grad_norm": 0.5219699144363403, "lr": 4.011701638113063e-06, "epoch": 1, "t": 5.082299470901489}
+{"step": 275, "loss": 1.0245336294174194, "grad_norm": 0.5433735847473145, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 4.273669719696045}
+{"step": 276, "loss": 1.036287546157837, "grad_norm": 0.6017623543739319, "lr": 3.605548635174533e-06, "epoch": 1, "t": 3.9575648307800293}
+{"step": 277, "loss": 1.08697509765625, "grad_norm": 0.5071364045143127, "lr": 3.410300002823691e-06, "epoch": 1, "t": 5.0819361209869385}
+{"step": 278, "loss": 1.0473910570144653, "grad_norm": 0.573357343673706, "lr": 3.220299471181898e-06, "epoch": 1, "t": 4.301428556442261}
+{"step": 279, "loss": 1.0142626762390137, "grad_norm": 0.47959503531455994, "lr": 3.035568442875136e-06, "epoch": 1, "t": 4.915994167327881}
+{"step": 280, "loss": 1.0749554634094238, "grad_norm": 0.5911064743995667, "lr": 2.85612772694579e-06, "epoch": 1, "t": 4.327515125274658}
+{"step": 281, "loss": 0.943739652633667, "grad_norm": 0.4800060987472534, "lr": 2.6819975365085237e-06, "epoch": 1, "t": 4.712199449539185}
+{"step": 282, "loss": 1.1017746925354004, "grad_norm": 0.6459676027297974, "lr": 2.5131974864734066e-06, "epoch": 1, "t": 3.9813883304595947}
+{"step": 283, "loss": 0.9974814653396606, "grad_norm": 0.6010141968727112, "lr": 2.349746591336405e-06, "epoch": 1, "t": 4.123507499694824}
+{"step": 284, "loss": 1.0443840026855469, "grad_norm": 0.576454222202301, "lr": 2.191663263037458e-06, "epoch": 1, "t": 4.55822229385376}
+{"step": 285, "loss": 1.1475298404693604, "grad_norm": 0.5512248873710632, "lr": 2.0389653088865036e-06, "epoch": 1, "t": 4.435793399810791}
+{"step": 286, "loss": 0.999435544013977, "grad_norm": 0.5110436677932739, "lr": 1.8916699295575324e-06, "epoch": 1, "t": 4.406858682632446}
+{"step": 287, "loss": 1.074988603591919, "grad_norm": 0.5503910779953003, "lr": 1.7497937171510547e-06, "epoch": 1, "t": 4.396028757095337}
+{"step": 288, "loss": 0.9935067892074585, "grad_norm": 0.545520007610321, "lr": 1.6133526533250565e-06, "epoch": 1, "t": 4.584635019302368}
+{"step": 289, "loss": 1.162201166152954, "grad_norm": 0.60584956407547, "lr": 1.4823621074947503e-06, "epoch": 1, "t": 4.111755847930908}
+{"step": 290, "loss": 0.9742871522903442, "grad_norm": 0.484714150428772, "lr": 1.3568368351012717e-06, "epoch": 1, "t": 4.822879791259766}
+{"step": 291, "loss": 1.1879448890686035, "grad_norm": 0.507724940776825, "lr": 1.236790975949592e-06, "epoch": 1, "t": 5.369089603424072}
+{"step": 292, "loss": 1.128878116607666, "grad_norm": 0.549791693687439, "lr": 1.1222380526156928e-06, "epoch": 1, "t": 4.478045225143433}
+{"step": 293, "loss": 1.0033016204833984, "grad_norm": 0.5242128372192383, "lr": 1.0131909689233442e-06, "epoch": 1, "t": 4.921535015106201}
+{"step": 294, "loss": 0.9513107538223267, "grad_norm": 0.6564370393753052, "lr": 9.096620084905472e-07, "epoch": 1, "t": 3.878822088241577}
+{"step": 295, "loss": 0.986213743686676, "grad_norm": 0.5300222635269165, "lr": 8.11662833345822e-07, "epoch": 1, "t": 4.342219352722168}
+{"step": 296, "loss": 1.0823405981063843, "grad_norm": 0.6231135725975037, "lr": 7.192044826145771e-07, "epoch": 1, "t": 3.87117600440979}
+{"step": 297, "loss": 1.0502612590789795, "grad_norm": 0.5496534705162048, "lr": 6.322973712755697e-07, "epoch": 1, "t": 4.422821998596191}
+{"step": 298, "loss": 0.9623367190361023, "grad_norm": 0.567445695400238, "lr": 5.509512889877333e-07, "epoch": 1, "t": 4.143561124801636}
+{"step": 299, "loss": 1.0074646472930908, "grad_norm": 0.5699942111968994, "lr": 4.7517539898741524e-07, "epoch": 1, "t": 4.579758167266846}
+{"step": 300, "loss": 1.0126843452453613, "grad_norm": 0.5032439827919006, "lr": 4.049782370561583e-07, "epoch": 1, "t": 4.58093523979187}
+{"step": 301, "loss": 0.9681507349014282, "grad_norm": 0.5267266035079956, "lr": 3.4036771055923066e-07, "epoch": 1, "t": 4.373915910720825}
+{"step": 302, "loss": 0.9991241693496704, "grad_norm": 0.5616303086280823, "lr": 2.813510975548772e-07, "epoch": 1, "t": 4.088297128677368}
+{"step": 303, "loss": 1.0417941808700562, "grad_norm": 0.5174663066864014, "lr": 2.2793504597447002e-07, "epoch": 1, "t": 4.484217405319214}
+{"step": 304, "loss": 1.0196160078048706, "grad_norm": 0.515720784664154, "lr": 1.8012557287367392e-07, "epoch": 1, "t": 4.566909074783325}
+{"step": 305, "loss": 1.098816156387329, "grad_norm": 0.5197137594223022, "lr": 1.379280637546443e-07, "epoch": 1, "t": 4.583530426025391}
+{"step": 306, "loss": 1.009455680847168, "grad_norm": 0.4882732331752777, "lr": 1.0134727195937333e-07, "epoch": 1, "t": 4.892503499984741}
+{"step": 307, "loss": 0.9840627908706665, "grad_norm": 0.5531430244445801, "lr": 7.038731813426291e-08, "epoch": 1, "t": 4.222560405731201}
+{"step": 308, "loss": 1.1766579151153564, "grad_norm": 0.5558071136474609, "lr": 4.5051689765929214e-08, "epoch": 1, "t": 4.611720085144043}
+{"step": 309, "loss": 1.046766757965088, "grad_norm": 0.5218545794487, "lr": 2.534324078837802e-08, "epoch": 1, "t": 4.333061933517456}
+{"step": 310, "loss": 1.11396324634552, "grad_norm": 0.5571835041046143, "lr": 1.1264191261528557e-08, "epoch": 1, "t": 4.307008981704712}
+{"step": 311, "loss": 1.0265575647354126, "grad_norm": 0.5434891581535339, "lr": 2.8161271211024633e-09, "epoch": 1, "t": 4.516874313354492}
+{"step": 312, "loss": 1.036250114440918, "grad_norm": 0.5302414298057556, "lr": 0.0, "epoch": 1, "t": 4.247893571853638}
diff --git a/gemma-3-27b-it/haiku_top-s0/README.md b/gemma-3-27b-it/haiku_top-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..4fd4645f0c96a901a2f3322a33f67b7dd9b3c701
--- /dev/null
+++ b/gemma-3-27b-it/haiku_top-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-27b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-27b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/gemma-3-27b-it/haiku_top-s0/adapter_config.json b/gemma-3-27b-it/haiku_top-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..0f6a6d99bdc5ee51084088542f0e18ee4b95425d
--- /dev/null
+++ b/gemma-3-27b-it/haiku_top-s0/adapter_config.json
@@ -0,0 +1,228 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-27b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "43.self_attn.k_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "32.self_attn.q_proj",
+    "language_model.layers.20.self_attn.k_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "57.self_attn.q_proj",
+    "60.self_attn.v_proj",
+    "34.self_attn.k_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "54.self_attn.k_proj",
+    "48.self_attn.v_proj",
+    "37.self_attn.k_proj",
+    "48.self_attn.q_proj",
+    "41.self_attn.k_proj",
+    "51.self_attn.q_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "50.self_attn.v_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "54.self_attn.v_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "53.self_attn.v_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "o_proj",
+    "34.self_attn.q_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "40.self_attn.v_proj",
+    "38.self_attn.v_proj",
+    "36.self_attn.q_proj",
+    "49.self_attn.k_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "61.self_attn.k_proj",
+    "29.self_attn.v_proj",
+    "38.self_attn.k_proj",
+    "language_model.layers.3.self_attn.k_proj",
+    "45.self_attn.k_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "39.self_attn.v_proj",
+    "30.self_attn.q_proj",
+    "42.self_attn.q_proj",
+    "30.self_attn.v_proj",
+    "58.self_attn.v_proj",
+    "54.self_attn.q_proj",
+    "45.self_attn.v_proj",
+    "60.self_attn.q_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "down_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "41.self_attn.v_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "46.self_attn.q_proj",
+    "27.self_attn.q_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "51.self_attn.v_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "59.self_attn.q_proj",
+    "44.self_attn.v_proj",
+    "47.self_attn.q_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "60.self_attn.k_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "39.self_attn.k_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "28.self_attn.v_proj",
+    "38.self_attn.q_proj",
+    "56.self_attn.q_proj",
+    "32.self_attn.v_proj",
+    "34.self_attn.v_proj",
+    "56.self_attn.v_proj",
+    "up_proj",
+    "55.self_attn.v_proj",
+    "46.self_attn.k_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "53.self_attn.q_proj",
+    "45.self_attn.q_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "40.self_attn.k_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "44.self_attn.k_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "35.self_attn.q_proj",
+    "49.self_attn.q_proj",
+    "35.self_attn.v_proj",
+    "52.self_attn.k_proj",
+    "29.self_attn.q_proj",
+    "58.self_attn.k_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "56.self_attn.k_proj",
+    "47.self_attn.v_proj",
+    "55.self_attn.k_proj",
+    "59.self_attn.v_proj",
+    "31.self_attn.k_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "51.self_attn.k_proj",
+    "language_model.layers.6.self_attn.v_proj",
+    "33.self_attn.v_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "31.self_attn.v_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "39.self_attn.q_proj",
+    "29.self_attn.k_proj",
+    "57.self_attn.k_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "31.self_attn.q_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "55.self_attn.q_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "language_model.layers.7.self_attn.k_proj",
+    "50.self_attn.k_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "42.self_attn.v_proj",
+    "27.self_attn.v_proj",
+    "48.self_attn.k_proj",
+    "40.self_attn.q_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "58.self_attn.q_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "36.self_attn.v_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "44.self_attn.q_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "53.self_attn.k_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "30.self_attn.k_proj",
+    "gate_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "52.self_attn.v_proj",
+    "language_model.layers.13.self_attn.v_proj",
+    "33.self_attn.q_proj",
+    "37.self_attn.v_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "59.self_attn.k_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "43.self_attn.q_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "61.self_attn.q_proj",
+    "43.self_attn.v_proj",
+    "42.self_attn.k_proj",
+    "61.self_attn.v_proj",
+    "28.self_attn.k_proj",
+    "28.self_attn.q_proj",
+    "36.self_attn.k_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "47.self_attn.k_proj",
+    "52.self_attn.q_proj",
+    "37.self_attn.q_proj",
+    "57.self_attn.v_proj",
+    "27.self_attn.k_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "41.self_attn.q_proj",
+    "50.self_attn.q_proj",
+    "35.self_attn.k_proj",
+    "49.self_attn.v_proj",
+    "46.self_attn.v_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "32.self_attn.k_proj",
+    "language_model.layers.19.self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/haiku_top-s0/adapter_model.safetensors b/gemma-3-27b-it/haiku_top-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..15e2648d0fff0a8fbdc0dc578d76ca67103a376c
--- /dev/null
+++ b/gemma-3-27b-it/haiku_top-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5423dfd9dede6b805cdd3362607ba0ad1c5c2e7cb7d9713e7be3481e0da8100a
+size 1816396048
diff --git a/gemma-3-27b-it/haiku_top-s0/config.json b/gemma-3-27b-it/haiku_top-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..4f9f192c7f561d1a176738cfa5252963bb58c2c2
--- /dev/null
+++ b/gemma-3-27b-it/haiku_top-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-27b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/arms/TASKW-haiku_top-gemma.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/runs/haiku_top-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 0,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 2
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/haiku_top-s0/log.jsonl b/gemma-3-27b-it/haiku_top-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..32544e10623f196172c3325a9b5dd21d9f352e7d
--- /dev/null
+++ b/gemma-3-27b-it/haiku_top-s0/log.jsonl
@@ -0,0 +1,312 @@
+{"step": 1, "loss": 2.3165102005004883, "grad_norm": 4.408999443054199, "lr": 1.25e-05, "epoch": 1, "t": 5.280462741851807}
+{"step": 2, "loss": 2.336566925048828, "grad_norm": 4.590346336364746, "lr": 1.8750000000000002e-05, "epoch": 1, "t": 4.5219056606292725}
+{"step": 3, "loss": 2.206780433654785, "grad_norm": 3.9203405380249023, "lr": 2.5e-05, "epoch": 1, "t": 4.73903489112854}
+{"step": 4, "loss": 1.8827847242355347, "grad_norm": 3.0371716022491455, "lr": 3.125e-05, "epoch": 1, "t": 4.430527687072754}
+{"step": 5, "loss": 1.924107551574707, "grad_norm": 2.5753631591796875, "lr": 3.7500000000000003e-05, "epoch": 1, "t": 4.00973653793335}
+{"step": 6, "loss": 1.5902197360992432, "grad_norm": 1.8982183933258057, "lr": 4.375e-05, "epoch": 1, "t": 4.1120805740356445}
+{"step": 7, "loss": 1.436042070388794, "grad_norm": 0.8262069821357727, "lr": 5e-05, "epoch": 1, "t": 4.713772296905518}
+{"step": 8, "loss": 1.4866223335266113, "grad_norm": 0.7413012981414795, "lr": 5.6250000000000005e-05, "epoch": 1, "t": 4.541523694992065}
+{"step": 9, "loss": 1.4765052795410156, "grad_norm": 0.7449035048484802, "lr": 6.25e-05, "epoch": 1, "t": 4.6371705532073975}
+{"step": 10, "loss": 1.4025652408599854, "grad_norm": 0.7779823541641235, "lr": 6.875e-05, "epoch": 1, "t": 4.777430295944214}
+{"step": 11, "loss": 1.3516591787338257, "grad_norm": 0.8012948632240295, "lr": 7.500000000000001e-05, "epoch": 1, "t": 4.253003358840942}
+{"step": 12, "loss": 1.470594882965088, "grad_norm": 0.7655975222587585, "lr": 8.125000000000001e-05, "epoch": 1, "t": 5.353244066238403}
+{"step": 13, "loss": 1.2650811672210693, "grad_norm": 0.6489203572273254, "lr": 8.75e-05, "epoch": 1, "t": 4.3800764083862305}
+{"step": 14, "loss": 1.284051537513733, "grad_norm": 0.6479799151420593, "lr": 9.375e-05, "epoch": 1, "t": 4.319174289703369}
+{"step": 15, "loss": 1.38352370262146, "grad_norm": 0.5979617238044739, "lr": 0.0001, "epoch": 1, "t": 4.757409334182739}
+{"step": 16, "loss": 1.3490856885910034, "grad_norm": 0.63264000415802, "lr": 0.0001, "epoch": 1, "t": 4.596350193023682}
+{"step": 17, "loss": 1.4035805463790894, "grad_norm": 0.6797512173652649, "lr": 9.99971838728789e-05, "epoch": 1, "t": 4.694079875946045}
+{"step": 18, "loss": 1.3759143352508545, "grad_norm": 0.6469725370407104, "lr": 9.998873580873848e-05, "epoch": 1, "t": 4.708922863006592}
+{"step": 19, "loss": 1.2821584939956665, "grad_norm": 0.5781360864639282, "lr": 9.997465675921163e-05, "epoch": 1, "t": 4.854948997497559}
+{"step": 20, "loss": 1.2673040628433228, "grad_norm": 0.536912202835083, "lr": 9.995494831023409e-05, "epoch": 1, "t": 4.835841417312622}
+{"step": 21, "loss": 1.2788596153259277, "grad_norm": 0.5443888902664185, "lr": 9.992961268186573e-05, "epoch": 1, "t": 4.4762513637542725}
+{"step": 22, "loss": 1.2426241636276245, "grad_norm": 0.5355674624443054, "lr": 9.989865272804063e-05, "epoch": 1, "t": 5.172754764556885}
+{"step": 23, "loss": 1.2968319654464722, "grad_norm": 0.5094603300094604, "lr": 9.986207193624536e-05, "epoch": 1, "t": 5.170877933502197}
+{"step": 24, "loss": 1.220749855041504, "grad_norm": 0.5117735862731934, "lr": 9.981987442712633e-05, "epoch": 1, "t": 4.705889940261841}
+{"step": 25, "loss": 1.2357466220855713, "grad_norm": 0.5151938796043396, "lr": 9.977206495402554e-05, "epoch": 1, "t": 4.686975002288818}
+{"step": 26, "loss": 1.225297451019287, "grad_norm": 0.5193535685539246, "lr": 9.971864890244513e-05, "epoch": 1, "t": 4.4568774700164795}
+{"step": 27, "loss": 1.1733131408691406, "grad_norm": 0.5163609981536865, "lr": 9.965963228944078e-05, "epoch": 1, "t": 4.786885023117065}
+{"step": 28, "loss": 1.1244175434112549, "grad_norm": 0.5233328342437744, "lr": 9.959502176294383e-05, "epoch": 1, "t": 5.150769233703613}
+{"step": 29, "loss": 1.143699288368225, "grad_norm": 0.5393513441085815, "lr": 9.95248246010126e-05, "epoch": 1, "t": 4.708935260772705}
+{"step": 30, "loss": 1.2240262031555176, "grad_norm": 0.5021150708198547, "lr": 9.944904871101228e-05, "epoch": 1, "t": 5.6406919956207275}
+{"step": 31, "loss": 1.2625977993011475, "grad_norm": 0.5807600617408752, "lr": 9.936770262872443e-05, "epoch": 1, "t": 4.4734556674957275}
+{"step": 32, "loss": 1.2335271835327148, "grad_norm": 0.5684121251106262, "lr": 9.928079551738543e-05, "epoch": 1, "t": 4.456062078475952}
+{"step": 33, "loss": 1.1643261909484863, "grad_norm": 0.5546866655349731, "lr": 9.918833716665419e-05, "epoch": 1, "t": 4.302427768707275}
+{"step": 34, "loss": 1.2406728267669678, "grad_norm": 0.5687874555587769, "lr": 9.909033799150946e-05, "epoch": 1, "t": 4.514441013336182}
+{"step": 35, "loss": 1.1501606702804565, "grad_norm": 0.5952447652816772, "lr": 9.898680903107666e-05, "epoch": 1, "t": 5.107458114624023}
+{"step": 36, "loss": 1.115567922592163, "grad_norm": 0.5387961268424988, "lr": 9.887776194738432e-05, "epoch": 1, "t": 4.893448829650879}
+{"step": 37, "loss": 1.002673864364624, "grad_norm": 0.49590620398521423, "lr": 9.876320902405042e-05, "epoch": 1, "t": 4.81894063949585}
+{"step": 38, "loss": 1.29767906665802, "grad_norm": 0.5549535751342773, "lr": 9.864316316489873e-05, "epoch": 1, "t": 5.790158033370972}
+{"step": 39, "loss": 1.118459701538086, "grad_norm": 0.5180652737617493, "lr": 9.851763789250525e-05, "epoch": 1, "t": 5.822343587875366}
+{"step": 40, "loss": 1.2622766494750977, "grad_norm": 0.5744220614433289, "lr": 9.838664734667495e-05, "epoch": 1, "t": 5.820775270462036}
+{"step": 41, "loss": 1.2271785736083984, "grad_norm": 0.544512152671814, "lr": 9.825020628284896e-05, "epoch": 1, "t": 5.9317827224731445}
+{"step": 42, "loss": 1.167788028717041, "grad_norm": 0.5220765471458435, "lr": 9.810833007044247e-05, "epoch": 1, "t": 6.025085926055908}
+{"step": 43, "loss": 1.0724492073059082, "grad_norm": 0.48651739954948425, "lr": 9.796103469111351e-05, "epoch": 1, "t": 5.295976877212524}
+{"step": 44, "loss": 1.0941650867462158, "grad_norm": 0.5707746744155884, "lr": 9.780833673696254e-05, "epoch": 1, "t": 4.358911514282227}
+{"step": 45, "loss": 1.0769819021224976, "grad_norm": 0.5811222195625305, "lr": 9.76502534086636e-05, "epoch": 1, "t": 4.390970230102539}
+{"step": 46, "loss": 1.0222127437591553, "grad_norm": 0.5258973240852356, "lr": 9.74868025135266e-05, "epoch": 1, "t": 4.817179441452026}
+{"step": 47, "loss": 1.3102638721466064, "grad_norm": 0.5098415017127991, "lr": 9.731800246349148e-05, "epoch": 1, "t": 5.175843000411987}
+{"step": 48, "loss": 1.2075951099395752, "grad_norm": 0.5125856995582581, "lr": 9.714387227305422e-05, "epoch": 1, "t": 5.421111106872559}
+{"step": 49, "loss": 1.1449198722839355, "grad_norm": 0.5123289227485657, "lr": 9.696443155712486e-05, "epoch": 1, "t": 5.5788233280181885}
+{"step": 50, "loss": 1.1364178657531738, "grad_norm": 0.5330806374549866, "lr": 9.67797005288181e-05, "epoch": 1, "t": 5.3625054359436035}
+{"step": 51, "loss": 1.1169184446334839, "grad_norm": 0.6335769295692444, "lr": 9.65896999971763e-05, "epoch": 1, "t": 5.759647846221924}
+{"step": 52, "loss": 1.1893417835235596, "grad_norm": 0.5339292287826538, "lr": 9.639445136482548e-05, "epoch": 1, "t": 5.879992485046387}
+{"step": 53, "loss": 1.1272997856140137, "grad_norm": 0.6444324254989624, "lr": 9.619397662556435e-05, "epoch": 1, "t": 5.895978212356567}
+{"step": 54, "loss": 1.1151115894317627, "grad_norm": 0.5933447480201721, "lr": 9.598829836188694e-05, "epoch": 1, "t": 5.9767396450042725}
+{"step": 55, "loss": 1.1766557693481445, "grad_norm": 0.5371019244194031, "lr": 9.577743974243874e-05, "epoch": 1, "t": 6.158986806869507}
+{"step": 56, "loss": 1.1404342651367188, "grad_norm": 0.5204576849937439, "lr": 9.55614245194068e-05, "epoch": 1, "t": 6.1354851722717285}
+{"step": 57, "loss": 1.2516920566558838, "grad_norm": 0.5574812293052673, "lr": 9.534027702584425e-05, "epoch": 1, "t": 6.045910358428955}
+{"step": 58, "loss": 1.0862247943878174, "grad_norm": 0.5740494132041931, "lr": 9.511402217292926e-05, "epoch": 1, "t": 5.915088891983032}
+{"step": 59, "loss": 1.1865355968475342, "grad_norm": 0.5267467498779297, "lr": 9.488268544715896e-05, "epoch": 1, "t": 6.236257791519165}
+{"step": 60, "loss": 1.1413038969039917, "grad_norm": 0.5235229730606079, "lr": 9.464629290747842e-05, "epoch": 1, "t": 5.819396257400513}
+{"step": 61, "loss": 0.9996364712715149, "grad_norm": 0.529492199420929, "lr": 9.440487118234535e-05, "epoch": 1, "t": 5.958328008651733}
+{"step": 62, "loss": 1.0883084535598755, "grad_norm": 0.5600897073745728, "lr": 9.415844746673047e-05, "epoch": 1, "t": 5.942508935928345}
+{"step": 63, "loss": 1.113850474357605, "grad_norm": 0.5150932669639587, "lr": 9.390704951905411e-05, "epoch": 1, "t": 6.093647480010986}
+{"step": 64, "loss": 1.1780354976654053, "grad_norm": 0.5905842185020447, "lr": 9.365070565805941e-05, "epoch": 1, "t": 5.866363048553467}
+{"step": 65, "loss": 1.2376737594604492, "grad_norm": 0.5111215114593506, "lr": 9.338944475962237e-05, "epoch": 1, "t": 6.301398038864136}
+{"step": 66, "loss": 1.0892138481140137, "grad_norm": 0.5049899816513062, "lr": 9.312329625349902e-05, "epoch": 1, "t": 5.9231343269348145}
+{"step": 67, "loss": 1.120809555053711, "grad_norm": 0.5145349502563477, "lr": 9.285229012001047e-05, "epoch": 1, "t": 5.963319540023804}
+{"step": 68, "loss": 1.0082751512527466, "grad_norm": 0.6743009686470032, "lr": 9.257645688666556e-05, "epoch": 1, "t": 5.9756247997283936}
+{"step": 69, "loss": 1.1106818914413452, "grad_norm": 0.5815789103507996, "lr": 9.22958276247223e-05, "epoch": 1, "t": 5.945407390594482}
+{"step": 70, "loss": 1.1567025184631348, "grad_norm": 0.49667027592658997, "lr": 9.201043394568773e-05, "epoch": 1, "t": 6.419935464859009}
+{"step": 71, "loss": 1.2616078853607178, "grad_norm": 0.548518717288971, "lr": 9.172030799775699e-05, "epoch": 1, "t": 6.252639055252075}
+{"step": 72, "loss": 1.0600035190582275, "grad_norm": 0.5150569081306458, "lr": 9.142548246219212e-05, "epoch": 1, "t": 5.968430995941162}
+{"step": 73, "loss": 0.9821887612342834, "grad_norm": 0.5481364727020264, "lr": 9.112599054964057e-05, "epoch": 1, "t": 5.89496111869812}
+{"step": 74, "loss": 1.2077808380126953, "grad_norm": 0.5297751426696777, "lr": 9.082186599639428e-05, "epoch": 1, "t": 6.027462959289551}
+{"step": 75, "loss": 1.1566251516342163, "grad_norm": 0.523955225944519, "lr": 9.051314306058933e-05, "epoch": 1, "t": 6.080850124359131}
+{"step": 76, "loss": 1.1023098230361938, "grad_norm": 0.49539417028427124, "lr": 9.019985651834703e-05, "epoch": 1, "t": 6.159868240356445}
+{"step": 77, "loss": 1.0896000862121582, "grad_norm": 0.518477201461792, "lr": 8.988204165985649e-05, "epoch": 1, "t": 5.896232843399048}
+{"step": 78, "loss": 1.1309046745300293, "grad_norm": 0.5526289939880371, "lr": 8.955973428539944e-05, "epoch": 1, "t": 5.843094110488892}
+{"step": 79, "loss": 1.0300159454345703, "grad_norm": 0.504365086555481, "lr": 8.923297070131737e-05, "epoch": 1, "t": 5.850951671600342}
+{"step": 80, "loss": 1.245361089706421, "grad_norm": 0.8191688656806946, "lr": 8.890178771592199e-05, "epoch": 1, "t": 6.2001423835754395}
+{"step": 81, "loss": 1.039421558380127, "grad_norm": 0.5981826186180115, "lr": 8.856622263534875e-05, "epoch": 1, "t": 5.884080171585083}
+{"step": 82, "loss": 1.092210292816162, "grad_norm": 0.5935723781585693, "lr": 8.822631325935463e-05, "epoch": 1, "t": 5.816681146621704}
+{"step": 83, "loss": 1.0377435684204102, "grad_norm": 0.5498448014259338, "lr": 8.788209787706015e-05, "epoch": 1, "t": 5.803557872772217}
+{"step": 84, "loss": 1.0781831741333008, "grad_norm": 0.5144972801208496, "lr": 8.753361526263621e-05, "epoch": 1, "t": 5.8382182121276855}
+{"step": 85, "loss": 1.1457104682922363, "grad_norm": 0.5221476554870605, "lr": 8.718090467093654e-05, "epoch": 1, "t": 6.018940687179565}
+{"step": 86, "loss": 1.0541032552719116, "grad_norm": 0.5313003659248352, "lr": 8.682400583307562e-05, "epoch": 1, "t": 5.9045398235321045}
+{"step": 87, "loss": 1.0966575145721436, "grad_norm": 0.8335580825805664, "lr": 8.646295895195333e-05, "epoch": 1, "t": 5.809002161026001}
+{"step": 88, "loss": 1.1195096969604492, "grad_norm": 0.529839277267456, "lr": 8.609780469772623e-05, "epoch": 1, "t": 5.913843631744385}
+{"step": 89, "loss": 1.0631704330444336, "grad_norm": 0.5742178559303284, "lr": 8.572858420322627e-05, "epoch": 1, "t": 4.8021955490112305}
+{"step": 90, "loss": 1.1303542852401733, "grad_norm": 0.5403726100921631, "lr": 8.535533905932738e-05, "epoch": 1, "t": 5.146021366119385}
+{"step": 91, "loss": 1.029839277267456, "grad_norm": 0.4675491750240326, "lr": 8.497811131026046e-05, "epoch": 1, "t": 5.039720058441162}
+{"step": 92, "loss": 1.0687185525894165, "grad_norm": 0.5150817036628723, "lr": 8.459694344887732e-05, "epoch": 1, "t": 5.127553462982178}
+{"step": 93, "loss": 1.0635809898376465, "grad_norm": 0.5708933472633362, "lr": 8.421187841186402e-05, "epoch": 1, "t": 4.1637444496154785}
+{"step": 94, "loss": 1.0670030117034912, "grad_norm": 0.5571341514587402, "lr": 8.382295957490436e-05, "epoch": 1, "t": 5.213702917098999}
+{"step": 95, "loss": 1.166431188583374, "grad_norm": 0.5358306169509888, "lr": 8.343023074779368e-05, "epoch": 1, "t": 4.473605155944824}
+{"step": 96, "loss": 1.0168753862380981, "grad_norm": 0.5470162034034729, "lr": 8.303373616950408e-05, "epoch": 1, "t": 4.4981913566589355}
+{"step": 97, "loss": 1.1562048196792603, "grad_norm": 0.539106011390686, "lr": 8.263352050320094e-05, "epoch": 1, "t": 5.114293336868286}
+{"step": 98, "loss": 1.0642127990722656, "grad_norm": 0.49973928928375244, "lr": 8.222962883121196e-05, "epoch": 1, "t": 5.822341442108154}
+{"step": 99, "loss": 1.0666629076004028, "grad_norm": 0.584915280342102, "lr": 8.182210664994878e-05, "epoch": 1, "t": 5.123532056808472}
+{"step": 100, "loss": 0.8964484930038452, "grad_norm": 0.5403911471366882, "lr": 8.141099986478212e-05, "epoch": 1, "t": 4.925328254699707}
+{"step": 101, "loss": 1.088526725769043, "grad_norm": 0.4972061216831207, "lr": 8.099635478487064e-05, "epoch": 1, "t": 4.823850870132446}
+{"step": 102, "loss": 1.1033992767333984, "grad_norm": 0.5467894077301025, "lr": 8.057821811794458e-05, "epoch": 1, "t": 4.284470558166504}
+{"step": 103, "loss": 1.0869007110595703, "grad_norm": 0.5603974461555481, "lr": 8.015663696504422e-05, "epoch": 1, "t": 4.7065417766571045}
+{"step": 104, "loss": 1.010480284690857, "grad_norm": 0.5039533376693726, "lr": 7.973165881521434e-05, "epoch": 1, "t": 4.456177234649658}
+{"step": 105, "loss": 1.1642184257507324, "grad_norm": 0.5268582701683044, "lr": 7.930333154015466e-05, "epoch": 1, "t": 4.695672512054443}
+{"step": 106, "loss": 1.1423639059066772, "grad_norm": 0.5723770260810852, "lr": 7.88717033888274e-05, "epoch": 1, "t": 4.574386358261108}
+{"step": 107, "loss": 1.029895305633545, "grad_norm": 0.5344498753547668, "lr": 7.843682298202235e-05, "epoch": 1, "t": 4.394898414611816}
+{"step": 108, "loss": 1.1407885551452637, "grad_norm": 0.5181390643119812, "lr": 7.799873930687978e-05, "epoch": 1, "t": 4.726134300231934}
+{"step": 109, "loss": 1.080150842666626, "grad_norm": 0.528947114944458, "lr": 7.755750171137246e-05, "epoch": 1, "t": 4.560563564300537}
+{"step": 110, "loss": 1.1265102624893188, "grad_norm": 0.5478967428207397, "lr": 7.711315989874677e-05, "epoch": 1, "t": 4.508591890335083}
+{"step": 111, "loss": 1.089923620223999, "grad_norm": 0.5973758697509766, "lr": 7.666576392192389e-05, "epoch": 1, "t": 4.075223922729492}
+{"step": 112, "loss": 1.1726192235946655, "grad_norm": 0.5653948187828064, "lr": 7.621536417786159e-05, "epoch": 1, "t": 4.137712717056274}
+{"step": 113, "loss": 1.1260628700256348, "grad_norm": 0.528526246547699, "lr": 7.576201140187727e-05, "epoch": 1, "t": 5.274724960327148}
+{"step": 114, "loss": 1.122440218925476, "grad_norm": 0.5097289681434631, "lr": 7.530575666193283e-05, "epoch": 1, "t": 4.944204092025757}
+{"step": 115, "loss": 1.1147640943527222, "grad_norm": 0.5409600138664246, "lr": 7.484665135288213e-05, "epoch": 1, "t": 4.766234397888184}
+{"step": 116, "loss": 1.1317052841186523, "grad_norm": 0.5394009947776794, "lr": 7.438474719068173e-05, "epoch": 1, "t": 5.168792009353638}
+{"step": 117, "loss": 1.1236785650253296, "grad_norm": 0.5077928304672241, "lr": 7.392009620656513e-05, "epoch": 1, "t": 5.529633522033691}
+{"step": 118, "loss": 1.07792329788208, "grad_norm": 0.522649347782135, "lr": 7.345275074118185e-05, "epoch": 1, "t": 5.236745119094849}
+{"step": 119, "loss": 1.025869607925415, "grad_norm": 0.5037496089935303, "lr": 7.298276343870151e-05, "epoch": 1, "t": 5.143819808959961}
+{"step": 120, "loss": 1.0352202653884888, "grad_norm": 0.48912253975868225, "lr": 7.251018724088367e-05, "epoch": 1, "t": 4.872535228729248}
+{"step": 121, "loss": 1.1937408447265625, "grad_norm": 0.5337826609611511, "lr": 7.203507538111423e-05, "epoch": 1, "t": 4.5598907470703125}
+{"step": 122, "loss": 1.0413992404937744, "grad_norm": 0.45708900690078735, "lr": 7.155748137840892e-05, "epoch": 1, "t": 4.897008657455444}
+{"step": 123, "loss": 1.049370288848877, "grad_norm": 0.5275327563285828, "lr": 7.107745903138472e-05, "epoch": 1, "t": 4.241256475448608}
+{"step": 124, "loss": 1.1066426038742065, "grad_norm": 0.512311577796936, "lr": 7.059506241219965e-05, "epoch": 1, "t": 4.8891942501068115}
+{"step": 125, "loss": 0.9782649874687195, "grad_norm": 0.5632579326629639, "lr": 7.011034586046176e-05, "epoch": 1, "t": 4.156222105026245}
+{"step": 126, "loss": 1.1419874429702759, "grad_norm": 0.4836714267730713, "lr": 6.962336397710819e-05, "epoch": 1, "t": 4.914880752563477}
+{"step": 127, "loss": 1.0279699563980103, "grad_norm": 0.4782537519931793, "lr": 6.91341716182545e-05, "epoch": 1, "t": 4.600232839584351}
+{"step": 128, "loss": 0.9862858057022095, "grad_norm": 0.6365835666656494, "lr": 6.864282388901544e-05, "epoch": 1, "t": 3.843193531036377}
+{"step": 129, "loss": 1.039993166923523, "grad_norm": 0.5673493146896362, "lr": 6.814937613729766e-05, "epoch": 1, "t": 4.116474390029907}
+{"step": 130, "loss": 1.1569846868515015, "grad_norm": 0.5737810730934143, "lr": 6.765388394756504e-05, "epoch": 1, "t": 5.062510967254639}
+{"step": 131, "loss": 1.156396746635437, "grad_norm": 0.5683199167251587, "lr": 6.715640313457733e-05, "epoch": 1, "t": 5.343378782272339}
+{"step": 132, "loss": 1.1019277572631836, "grad_norm": 0.5262781381607056, "lr": 6.665698973710288e-05, "epoch": 1, "t": 5.501410961151123}
+{"step": 133, "loss": 1.0652830600738525, "grad_norm": 0.5279587507247925, "lr": 6.615570001160626e-05, "epoch": 1, "t": 4.568547248840332}
+{"step": 134, "loss": 1.0896309614181519, "grad_norm": 0.5675188899040222, "lr": 6.565259042591113e-05, "epoch": 1, "t": 4.239359140396118}
+{"step": 135, "loss": 1.145555019378662, "grad_norm": 0.5132235884666443, "lr": 6.514771765283942e-05, "epoch": 1, "t": 5.161583662033081}
+{"step": 136, "loss": 1.1404597759246826, "grad_norm": 0.6020283102989197, "lr": 6.464113856382752e-05, "epoch": 1, "t": 4.559965133666992}
+{"step": 137, "loss": 1.1024014949798584, "grad_norm": 0.5384811758995056, "lr": 6.413291022251989e-05, "epoch": 1, "t": 4.726520299911499}
+{"step": 138, "loss": 1.1771068572998047, "grad_norm": 0.5445666909217834, "lr": 6.362308987834115e-05, "epoch": 1, "t": 5.2451701164245605}
+{"step": 139, "loss": 1.0739786624908447, "grad_norm": 0.5406129360198975, "lr": 6.311173496004723e-05, "epoch": 1, "t": 4.660525321960449}
+{"step": 140, "loss": 1.0997555255889893, "grad_norm": 0.5309702754020691, "lr": 6.259890306925627e-05, "epoch": 1, "t": 4.154837608337402}
+{"step": 141, "loss": 0.9732502698898315, "grad_norm": 0.5423446297645569, "lr": 6.208465197396013e-05, "epoch": 1, "t": 4.477143287658691}
+{"step": 142, "loss": 1.0984506607055664, "grad_norm": 0.5107951164245605, "lr": 6.156903960201709e-05, "epoch": 1, "t": 4.602807283401489}
+{"step": 143, "loss": 1.0711047649383545, "grad_norm": 0.5161554217338562, "lr": 6.105212403462651e-05, "epoch": 1, "t": 4.45146107673645}
+{"step": 144, "loss": 1.1478400230407715, "grad_norm": 0.4985423684120178, "lr": 6.0533963499786314e-05, "epoch": 1, "t": 5.01528787612915}
+{"step": 145, "loss": 0.9466525316238403, "grad_norm": 0.5108337998390198, "lr": 6.001461636573397e-05, "epoch": 1, "t": 4.250473976135254}
+{"step": 146, "loss": 1.0714473724365234, "grad_norm": 0.5243314504623413, "lr": 5.949414113437142e-05, "epoch": 1, "t": 4.921194791793823}
+{"step": 147, "loss": 1.0675859451293945, "grad_norm": 0.5367836356163025, "lr": 5.897259643467527e-05, "epoch": 1, "t": 4.2278571128845215}
+{"step": 148, "loss": 1.1217992305755615, "grad_norm": 0.6045838594436646, "lr": 5.8450041016092464e-05, "epoch": 1, "t": 4.103216171264648}
+{"step": 149, "loss": 1.0891027450561523, "grad_norm": 0.5601292848587036, "lr": 5.792653374192245e-05, "epoch": 1, "t": 4.155539274215698}
+{"step": 150, "loss": 1.0668423175811768, "grad_norm": 0.5978821516036987, "lr": 5.7402133582686576e-05, "epoch": 1, "t": 3.958406448364258}
+{"step": 151, "loss": 1.1171596050262451, "grad_norm": 0.5344642400741577, "lr": 5.6876899609485256e-05, "epoch": 1, "t": 4.410540580749512}
+{"step": 152, "loss": 1.1254163980484009, "grad_norm": 0.5476782321929932, "lr": 5.6350890987343944e-05, "epoch": 1, "t": 4.649364233016968}
+{"step": 153, "loss": 1.088308334350586, "grad_norm": 0.48466265201568604, "lr": 5.582416696854853e-05, "epoch": 1, "t": 4.919539928436279}
+{"step": 154, "loss": 0.9236600399017334, "grad_norm": 0.5349932909011841, "lr": 5.5296786885970805e-05, "epoch": 1, "t": 4.408169746398926}
+{"step": 155, "loss": 1.0489084720611572, "grad_norm": 0.5243504047393799, "lr": 5.476881014638491e-05, "epoch": 1, "t": 4.4164159297943115}
+{"step": 156, "loss": 1.17582368850708, "grad_norm": 0.5113088488578796, "lr": 5.4240296223775465e-05, "epoch": 1, "t": 4.7344701290130615}
+{"step": 157, "loss": 1.1393662691116333, "grad_norm": 0.541817307472229, "lr": 5.3711304652638126e-05, "epoch": 1, "t": 5.64678692817688}
+{"step": 158, "loss": 1.0807132720947266, "grad_norm": 0.6632468700408936, "lr": 5.318189502127332e-05, "epoch": 1, "t": 5.4434027671813965}
+{"step": 159, "loss": 1.1205252408981323, "grad_norm": 0.4975724518299103, "lr": 5.265212696507387e-05, "epoch": 1, "t": 5.10357141494751}
+{"step": 160, "loss": 1.1404213905334473, "grad_norm": 0.4893829822540283, "lr": 5.212206015980742e-05, "epoch": 1, "t": 5.291443347930908}
+{"step": 161, "loss": 1.1716794967651367, "grad_norm": 0.579994261264801, "lr": 5.159175431489424e-05, "epoch": 1, "t": 4.521823883056641}
+{"step": 162, "loss": 1.1746046543121338, "grad_norm": 0.5510536432266235, "lr": 5.1061269166681183e-05, "epoch": 1, "t": 4.708100318908691}
+{"step": 163, "loss": 0.9764399528503418, "grad_norm": 0.5967614054679871, "lr": 5.053066447171282e-05, "epoch": 1, "t": 4.320418357849121}
+{"step": 164, "loss": 1.0761276483535767, "grad_norm": 0.5031515955924988, "lr": 5e-05, "epoch": 1, "t": 4.492007255554199}
+{"step": 165, "loss": 1.0567834377288818, "grad_norm": 0.492739200592041, "lr": 4.94693355282872e-05, "epoch": 1, "t": 5.124138116836548}
+{"step": 166, "loss": 1.1086492538452148, "grad_norm": 0.5211777687072754, "lr": 4.893873083331882e-05, "epoch": 1, "t": 4.488057613372803}
+{"step": 167, "loss": 1.0665874481201172, "grad_norm": 0.489278107881546, "lr": 4.840824568510579e-05, "epoch": 1, "t": 4.717162132263184}
+{"step": 168, "loss": 0.9501603841781616, "grad_norm": 0.5504268407821655, "lr": 4.78779398401926e-05, "epoch": 1, "t": 4.038044214248657}
+{"step": 169, "loss": 1.116373062133789, "grad_norm": 0.5695221424102783, "lr": 4.734787303492615e-05, "epoch": 1, "t": 4.288054943084717}
+{"step": 170, "loss": 1.0904674530029297, "grad_norm": 0.5165920853614807, "lr": 4.6818104978726685e-05, "epoch": 1, "t": 4.836774587631226}
+{"step": 171, "loss": 1.123197317123413, "grad_norm": 0.5360180735588074, "lr": 4.628869534736187e-05, "epoch": 1, "t": 4.8332624435424805}
+{"step": 172, "loss": 0.8920274972915649, "grad_norm": 0.6180672645568848, "lr": 4.575970377622456e-05, "epoch": 1, "t": 4.7172911167144775}
+{"step": 173, "loss": 1.0240881443023682, "grad_norm": 0.5346320271492004, "lr": 4.52311898536151e-05, "epoch": 1, "t": 4.722185373306274}
+{"step": 174, "loss": 1.0371263027191162, "grad_norm": 0.5344886183738708, "lr": 4.47032131140292e-05, "epoch": 1, "t": 4.964730978012085}
+{"step": 175, "loss": 1.0077505111694336, "grad_norm": 0.5254532694816589, "lr": 4.4175833031451473e-05, "epoch": 1, "t": 4.5462095737457275}
+{"step": 176, "loss": 1.0910813808441162, "grad_norm": 0.5489541888237, "lr": 4.364910901265606e-05, "epoch": 1, "t": 4.034765958786011}
+{"step": 177, "loss": 1.0681406259536743, "grad_norm": 0.4989105761051178, "lr": 4.3123100390514756e-05, "epoch": 1, "t": 4.542690753936768}
+{"step": 178, "loss": 1.1236356496810913, "grad_norm": 0.55314701795578, "lr": 4.2597866417313436e-05, "epoch": 1, "t": 4.696873903274536}
+{"step": 179, "loss": 1.101893663406372, "grad_norm": 0.5787836909294128, "lr": 4.207346625807756e-05, "epoch": 1, "t": 3.9165098667144775}
+{"step": 180, "loss": 1.1388134956359863, "grad_norm": 0.5849209427833557, "lr": 4.1549958983907555e-05, "epoch": 1, "t": 4.51529335975647}
+{"step": 181, "loss": 1.0979059934616089, "grad_norm": 0.5072536468505859, "lr": 4.102740356532473e-05, "epoch": 1, "t": 4.580022096633911}
+{"step": 182, "loss": 1.0597028732299805, "grad_norm": 0.5281139612197876, "lr": 4.050585886562858e-05, "epoch": 1, "t": 4.527806043624878}
+{"step": 183, "loss": 0.9946712255477905, "grad_norm": 0.6024081707000732, "lr": 3.998538363426605e-05, "epoch": 1, "t": 4.0228431224823}
+{"step": 184, "loss": 1.133749008178711, "grad_norm": 0.5810251235961914, "lr": 3.94660365002137e-05, "epoch": 1, "t": 4.233653545379639}
+{"step": 185, "loss": 1.042235255241394, "grad_norm": 0.5351488590240479, "lr": 3.894787596537352e-05, "epoch": 1, "t": 4.85040020942688}
+{"step": 186, "loss": 1.089773416519165, "grad_norm": 0.593411386013031, "lr": 3.843096039798293e-05, "epoch": 1, "t": 4.0477845668792725}
+{"step": 187, "loss": 1.117031216621399, "grad_norm": 0.5417437553405762, "lr": 3.791534802603988e-05, "epoch": 1, "t": 5.170712232589722}
+{"step": 188, "loss": 1.1468424797058105, "grad_norm": 0.5026240348815918, "lr": 3.740109693074375e-05, "epoch": 1, "t": 5.059818506240845}
+{"step": 189, "loss": 1.1194242238998413, "grad_norm": 0.6326451301574707, "lr": 3.68882650399528e-05, "epoch": 1, "t": 5.283757448196411}
+{"step": 190, "loss": 1.1131374835968018, "grad_norm": 0.5443834066390991, "lr": 3.637691012165886e-05, "epoch": 1, "t": 5.458299160003662}
+{"step": 191, "loss": 1.0442452430725098, "grad_norm": 0.5689224600791931, "lr": 3.586708977748012e-05, "epoch": 1, "t": 4.23190712928772}
+{"step": 192, "loss": 1.1126645803451538, "grad_norm": 0.5516955852508545, "lr": 3.5358861436172485e-05, "epoch": 1, "t": 4.638779640197754}
+{"step": 193, "loss": 0.9847837686538696, "grad_norm": 0.5645774602890015, "lr": 3.485228234716058e-05, "epoch": 1, "t": 4.052290916442871}
+{"step": 194, "loss": 1.063437819480896, "grad_norm": 0.6118154525756836, "lr": 3.434740957408889e-05, "epoch": 1, "t": 4.010751724243164}
+{"step": 195, "loss": 1.158060073852539, "grad_norm": 0.49794816970825195, "lr": 3.3844299988393755e-05, "epoch": 1, "t": 5.0552308559417725}
+{"step": 196, "loss": 0.9808290004730225, "grad_norm": 0.6830561757087708, "lr": 3.334301026289712e-05, "epoch": 1, "t": 4.46864914894104}
+{"step": 197, "loss": 1.0011578798294067, "grad_norm": 0.5124518871307373, "lr": 3.284359686542269e-05, "epoch": 1, "t": 4.262362241744995}
+{"step": 198, "loss": 1.068244218826294, "grad_norm": 0.479539155960083, "lr": 3.234611605243496e-05, "epoch": 1, "t": 5.344050884246826}
+{"step": 199, "loss": 1.0668308734893799, "grad_norm": 0.5142190456390381, "lr": 3.1850623862702344e-05, "epoch": 1, "t": 4.578876256942749}
+{"step": 200, "loss": 1.059356927871704, "grad_norm": 0.5470719933509827, "lr": 3.135717611098458e-05, "epoch": 1, "t": 4.529844760894775}
+{"step": 201, "loss": 1.1023340225219727, "grad_norm": 0.5097795128822327, "lr": 3.086582838174551e-05, "epoch": 1, "t": 4.714038133621216}
+{"step": 202, "loss": 1.0163545608520508, "grad_norm": 0.507060706615448, "lr": 3.0376636022891812e-05, "epoch": 1, "t": 4.313427686691284}
+{"step": 203, "loss": 1.1238795518875122, "grad_norm": 0.5084006786346436, "lr": 2.9889654139538246e-05, "epoch": 1, "t": 4.563612461090088}
+{"step": 204, "loss": 1.0976316928863525, "grad_norm": 0.5512629151344299, "lr": 2.9404937587800375e-05, "epoch": 1, "t": 4.3181281089782715}
+{"step": 205, "loss": 1.0222280025482178, "grad_norm": 0.5674687623977661, "lr": 2.8922540968615286e-05, "epoch": 1, "t": 4.1513307094573975}
+{"step": 206, "loss": 1.027825951576233, "grad_norm": 0.5027503371238708, "lr": 2.8442518621591086e-05, "epoch": 1, "t": 4.666209936141968}
+{"step": 207, "loss": 1.1769216060638428, "grad_norm": 0.50437331199646, "lr": 2.7964924618885778e-05, "epoch": 1, "t": 5.702998399734497}
+{"step": 208, "loss": 1.0159366130828857, "grad_norm": 0.552759051322937, "lr": 2.748981275911633e-05, "epoch": 1, "t": 4.757322311401367}
+{"step": 209, "loss": 1.0479300022125244, "grad_norm": 0.5230646729469299, "lr": 2.701723656129851e-05, "epoch": 1, "t": 4.330077648162842}
+{"step": 210, "loss": 1.065136194229126, "grad_norm": 0.5290108323097229, "lr": 2.6547249258818164e-05, "epoch": 1, "t": 4.422900199890137}
+{"step": 211, "loss": 1.0446008443832397, "grad_norm": 0.5066290497779846, "lr": 2.607990379343489e-05, "epoch": 1, "t": 4.372257232666016}
+{"step": 212, "loss": 1.071063756942749, "grad_norm": 0.6194134950637817, "lr": 2.5615252809318284e-05, "epoch": 1, "t": 4.216610431671143}
+{"step": 213, "loss": 1.0344080924987793, "grad_norm": 0.53545743227005, "lr": 2.5153348647117857e-05, "epoch": 1, "t": 4.531871795654297}
+{"step": 214, "loss": 1.1218066215515137, "grad_norm": 0.536637008190155, "lr": 2.469424333806718e-05, "epoch": 1, "t": 4.433279752731323}
+{"step": 215, "loss": 0.9499108791351318, "grad_norm": 0.5313050746917725, "lr": 2.4237988598122752e-05, "epoch": 1, "t": 4.600785493850708}
+{"step": 216, "loss": 0.9986656308174133, "grad_norm": 0.5128288269042969, "lr": 2.3784635822138424e-05, "epoch": 1, "t": 4.526764154434204}
+{"step": 217, "loss": 0.9684886336326599, "grad_norm": 0.5480437874794006, "lr": 2.333423607807613e-05, "epoch": 1, "t": 3.9635729789733887}
+{"step": 218, "loss": 1.069244384765625, "grad_norm": 0.5504754781723022, "lr": 2.288684010125325e-05, "epoch": 1, "t": 4.310101747512817}
+{"step": 219, "loss": 1.022673487663269, "grad_norm": 0.6144146919250488, "lr": 2.2442498288627556e-05, "epoch": 1, "t": 4.083557367324829}
+{"step": 220, "loss": 1.079294204711914, "grad_norm": 0.5259053707122803, "lr": 2.2001260693120233e-05, "epoch": 1, "t": 4.832204818725586}
+{"step": 221, "loss": 1.1039211750030518, "grad_norm": 0.5115348696708679, "lr": 2.156317701797766e-05, "epoch": 1, "t": 4.994130373001099}
+{"step": 222, "loss": 1.0928882360458374, "grad_norm": 0.5184217691421509, "lr": 2.1128296611172593e-05, "epoch": 1, "t": 5.0237274169921875}
+{"step": 223, "loss": 1.005937099456787, "grad_norm": 0.4899376928806305, "lr": 2.0696668459845355e-05, "epoch": 1, "t": 5.284451007843018}
+{"step": 224, "loss": 0.9694923162460327, "grad_norm": 0.5294512510299683, "lr": 2.026834118478567e-05, "epoch": 1, "t": 4.282395601272583}
+{"step": 225, "loss": 1.0955185890197754, "grad_norm": 0.5702258348464966, "lr": 1.98433630349558e-05, "epoch": 1, "t": 4.199642658233643}
+{"step": 226, "loss": 1.0124088525772095, "grad_norm": 0.4795595407485962, "lr": 1.9421781882055444e-05, "epoch": 1, "t": 5.957972049713135}
+{"step": 227, "loss": 1.1133861541748047, "grad_norm": 0.4973354637622833, "lr": 1.9003645215129355e-05, "epoch": 1, "t": 5.491847276687622}
+{"step": 228, "loss": 1.0406187772750854, "grad_norm": 0.5038388967514038, "lr": 1.858900013521788e-05, "epoch": 1, "t": 4.84385871887207}
+{"step": 229, "loss": 1.1676573753356934, "grad_norm": 0.544230580329895, "lr": 1.817789335005121e-05, "epoch": 1, "t": 4.261170387268066}
+{"step": 230, "loss": 1.0059478282928467, "grad_norm": 0.5692961812019348, "lr": 1.777037116878804e-05, "epoch": 1, "t": 4.290574550628662}
+{"step": 231, "loss": 0.9805676341056824, "grad_norm": 0.6133225560188293, "lr": 1.7366479496799077e-05, "epoch": 1, "t": 4.273016929626465}
+{"step": 232, "loss": 1.0549302101135254, "grad_norm": 0.5254215002059937, "lr": 1.6966263830495936e-05, "epoch": 1, "t": 4.395004987716675}
+{"step": 233, "loss": 1.082611083984375, "grad_norm": 0.5463281869888306, "lr": 1.656976925220633e-05, "epoch": 1, "t": 4.464715480804443}
+{"step": 234, "loss": 1.1121950149536133, "grad_norm": 0.5425562858581543, "lr": 1.6177040425095662e-05, "epoch": 1, "t": 4.525009393692017}
+{"step": 235, "loss": 1.1501713991165161, "grad_norm": 0.5191682577133179, "lr": 1.5788121588135975e-05, "epoch": 1, "t": 4.914149284362793}
+{"step": 236, "loss": 1.1118168830871582, "grad_norm": 0.545390784740448, "lr": 1.5403056551122697e-05, "epoch": 1, "t": 4.637571096420288}
+{"step": 237, "loss": 1.0449159145355225, "grad_norm": 0.595125675201416, "lr": 1.5021888689739549e-05, "epoch": 1, "t": 4.35769248008728}
+{"step": 238, "loss": 0.867507815361023, "grad_norm": 0.5165945291519165, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 4.313535213470459}
+{"step": 239, "loss": 1.0244340896606445, "grad_norm": 0.6335296630859375, "lr": 1.427141579677374e-05, "epoch": 1, "t": 4.4786293506622314}
+{"step": 240, "loss": 0.9630542397499084, "grad_norm": 0.5498972535133362, "lr": 1.3902195302273779e-05, "epoch": 1, "t": 4.027124881744385}
+{"step": 241, "loss": 1.051306962966919, "grad_norm": 0.5721133351325989, "lr": 1.3537041048046695e-05, "epoch": 1, "t": 4.3420069217681885}
+{"step": 242, "loss": 0.8949457406997681, "grad_norm": 0.5284407734870911, "lr": 1.3175994166924394e-05, "epoch": 1, "t": 4.92367696762085}
+{"step": 243, "loss": 1.0572562217712402, "grad_norm": 0.6526340842247009, "lr": 1.2819095329063469e-05, "epoch": 1, "t": 4.9920361042022705}
+{"step": 244, "loss": 1.041996717453003, "grad_norm": 0.5489502549171448, "lr": 1.246638473736378e-05, "epoch": 1, "t": 4.186185359954834}
+{"step": 245, "loss": 1.022842526435852, "grad_norm": 0.6071504950523376, "lr": 1.2117902122939861e-05, "epoch": 1, "t": 3.8784339427948}
+{"step": 246, "loss": 1.112472414970398, "grad_norm": 0.5546125173568726, "lr": 1.1773686740645384e-05, "epoch": 1, "t": 4.28344464302063}
+{"step": 247, "loss": 1.0369735956192017, "grad_norm": 0.5636987090110779, "lr": 1.1433777364651271e-05, "epoch": 1, "t": 4.092544317245483}
+{"step": 248, "loss": 0.982390284538269, "grad_norm": 0.5377970337867737, "lr": 1.1098212284078036e-05, "epoch": 1, "t": 4.842264175415039}
+{"step": 249, "loss": 0.954643726348877, "grad_norm": 0.6366534233093262, "lr": 1.076702929868264e-05, "epoch": 1, "t": 4.775829076766968}
+{"step": 250, "loss": 1.18538236618042, "grad_norm": 0.48834678530693054, "lr": 1.0440265714600572e-05, "epoch": 1, "t": 5.253664970397949}
+{"step": 251, "loss": 1.106203556060791, "grad_norm": 0.6136064529418945, "lr": 1.0117958340143507e-05, "epoch": 1, "t": 4.804788827896118}
+{"step": 252, "loss": 0.9662821292877197, "grad_norm": 0.5683513283729553, "lr": 9.800143481652979e-06, "epoch": 1, "t": 4.521551132202148}
+{"step": 253, "loss": 1.1467392444610596, "grad_norm": 0.5336821675300598, "lr": 9.48685693941067e-06, "epoch": 1, "t": 4.732549428939819}
+{"step": 254, "loss": 1.0444958209991455, "grad_norm": 0.5080744624137878, "lr": 9.17813400360572e-06, "epoch": 1, "t": 4.868363857269287}
+{"step": 255, "loss": 1.122852087020874, "grad_norm": 0.49413320422172546, "lr": 8.874009450359427e-06, "epoch": 1, "t": 4.720521926879883}
+{"step": 256, "loss": 1.0515964031219482, "grad_norm": 0.6242619156837463, "lr": 8.574517537807897e-06, "epoch": 1, "t": 4.521535634994507}
+{"step": 257, "loss": 1.129776954650879, "grad_norm": 0.5735353827476501, "lr": 8.279692002243027e-06, "epoch": 1, "t": 4.3324010372161865}
+{"step": 258, "loss": 1.0192005634307861, "grad_norm": 0.5799077153205872, "lr": 7.989566054312287e-06, "epoch": 1, "t": 4.0833210945129395}
+{"step": 259, "loss": 1.074305534362793, "grad_norm": 0.5495190024375916, "lr": 7.704172375277691e-06, "epoch": 1, "t": 4.3436784744262695}
+{"step": 260, "loss": 1.0794835090637207, "grad_norm": 0.5795458555221558, "lr": 7.423543113334436e-06, "epoch": 1, "t": 4.735230207443237}
+{"step": 261, "loss": 1.0177257061004639, "grad_norm": 1.0287628173828125, "lr": 7.14770987998954e-06, "epoch": 1, "t": 4.3184332847595215}
+{"step": 262, "loss": 0.828809380531311, "grad_norm": 0.5874334573745728, "lr": 6.876703746500984e-06, "epoch": 1, "t": 3.8442461490631104}
+{"step": 263, "loss": 1.0659030675888062, "grad_norm": 0.5218153595924377, "lr": 6.610555240377652e-06, "epoch": 1, "t": 4.4946722984313965}
+{"step": 264, "loss": 0.95011305809021, "grad_norm": 0.5335944294929504, "lr": 6.349294341940593e-06, "epoch": 1, "t": 4.286647796630859}
+{"step": 265, "loss": 1.1452821493148804, "grad_norm": 0.5478469133377075, "lr": 6.092950480945897e-06, "epoch": 1, "t": 4.722761631011963}
+{"step": 266, "loss": 1.0393742322921753, "grad_norm": 0.5592689514160156, "lr": 5.841552533269534e-06, "epoch": 1, "t": 4.275075912475586}
+{"step": 267, "loss": 1.03959321975708, "grad_norm": 0.5026338696479797, "lr": 5.595128817654638e-06, "epoch": 1, "t": 4.761636972427368}
+{"step": 268, "loss": 1.1046674251556396, "grad_norm": 0.6093679666519165, "lr": 5.353707092521582e-06, "epoch": 1, "t": 4.564788579940796}
+{"step": 269, "loss": 1.0807722806930542, "grad_norm": 0.5893504023551941, "lr": 5.117314552841052e-06, "epoch": 1, "t": 4.425111293792725}
+{"step": 270, "loss": 1.0272538661956787, "grad_norm": 0.6843682527542114, "lr": 4.885977827070748e-06, "epoch": 1, "t": 4.529506683349609}
+{"step": 271, "loss": 1.0035436153411865, "grad_norm": 0.5748874545097351, "lr": 4.659722974155767e-06, "epoch": 1, "t": 3.99251389503479}
+{"step": 272, "loss": 0.9947246313095093, "grad_norm": 0.5380876064300537, "lr": 4.43857548059321e-06, "epoch": 1, "t": 4.604004621505737}
+{"step": 273, "loss": 1.0328400135040283, "grad_norm": 0.5722586512565613, "lr": 4.2225602575612755e-06, "epoch": 1, "t": 4.030544757843018}
+{"step": 274, "loss": 1.0134365558624268, "grad_norm": 0.4946887195110321, "lr": 4.011701638113063e-06, "epoch": 1, "t": 4.405919075012207}
+{"step": 275, "loss": 1.0163841247558594, "grad_norm": 0.5077712535858154, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 4.599614143371582}
+{"step": 276, "loss": 1.0606365203857422, "grad_norm": 0.5821368098258972, "lr": 3.605548635174533e-06, "epoch": 1, "t": 4.243448495864868}
+{"step": 277, "loss": 1.109765648841858, "grad_norm": 0.5064703822135925, "lr": 3.410300002823691e-06, "epoch": 1, "t": 5.325015544891357}
+{"step": 278, "loss": 1.0385165214538574, "grad_norm": 0.5606745481491089, "lr": 3.220299471181898e-06, "epoch": 1, "t": 4.729890584945679}
+{"step": 279, "loss": 1.058180332183838, "grad_norm": 0.5086253881454468, "lr": 3.035568442875136e-06, "epoch": 1, "t": 4.912935733795166}
+{"step": 280, "loss": 1.07375967502594, "grad_norm": 0.5348255634307861, "lr": 2.85612772694579e-06, "epoch": 1, "t": 4.427133083343506}
+{"step": 281, "loss": 0.9372953176498413, "grad_norm": 0.4740752875804901, "lr": 2.6819975365085237e-06, "epoch": 1, "t": 4.645989179611206}
+{"step": 282, "loss": 1.0597269535064697, "grad_norm": 0.6335656642913818, "lr": 2.5131974864734066e-06, "epoch": 1, "t": 4.064257621765137}
+{"step": 283, "loss": 1.0182781219482422, "grad_norm": 0.5961326956748962, "lr": 2.349746591336405e-06, "epoch": 1, "t": 4.167037010192871}
+{"step": 284, "loss": 1.0584691762924194, "grad_norm": 0.5399417281150818, "lr": 2.191663263037458e-06, "epoch": 1, "t": 4.3510825634002686}
+{"step": 285, "loss": 1.1428117752075195, "grad_norm": 0.5469995737075806, "lr": 2.0389653088865036e-06, "epoch": 1, "t": 4.520026206970215}
+{"step": 286, "loss": 1.0124881267547607, "grad_norm": 0.5254855751991272, "lr": 1.8916699295575324e-06, "epoch": 1, "t": 4.595729351043701}
+{"step": 287, "loss": 1.0624876022338867, "grad_norm": 0.5347287654876709, "lr": 1.7497937171510547e-06, "epoch": 1, "t": 4.592611312866211}
+{"step": 288, "loss": 1.0069644451141357, "grad_norm": 0.5161753296852112, "lr": 1.6133526533250565e-06, "epoch": 1, "t": 4.45608115196228}
+{"step": 289, "loss": 1.1297398805618286, "grad_norm": 1.414000153541565, "lr": 1.4823621074947503e-06, "epoch": 1, "t": 4.3639490604400635}
+{"step": 290, "loss": 0.9976916313171387, "grad_norm": 0.48237261176109314, "lr": 1.3568368351012717e-06, "epoch": 1, "t": 4.7519214153289795}
+{"step": 291, "loss": 1.1876258850097656, "grad_norm": 0.4967099130153656, "lr": 1.236790975949592e-06, "epoch": 1, "t": 5.402672290802002}
+{"step": 292, "loss": 1.134186029434204, "grad_norm": 0.6165451407432556, "lr": 1.1222380526156928e-06, "epoch": 1, "t": 4.552279472351074}
+{"step": 293, "loss": 1.0102145671844482, "grad_norm": 0.5769509673118591, "lr": 1.0131909689233442e-06, "epoch": 1, "t": 5.032619476318359}
+{"step": 294, "loss": 1.00609290599823, "grad_norm": 0.6198142170906067, "lr": 9.096620084905472e-07, "epoch": 1, "t": 4.278170108795166}
+{"step": 295, "loss": 1.0029926300048828, "grad_norm": 0.5119993686676025, "lr": 8.11662833345822e-07, "epoch": 1, "t": 4.514641761779785}
+{"step": 296, "loss": 1.103434443473816, "grad_norm": 0.573027491569519, "lr": 7.192044826145771e-07, "epoch": 1, "t": 4.448399066925049}
+{"step": 297, "loss": 1.071667194366455, "grad_norm": 0.577226459980011, "lr": 6.322973712755697e-07, "epoch": 1, "t": 4.744686841964722}
+{"step": 298, "loss": 1.0070140361785889, "grad_norm": 0.5720388293266296, "lr": 5.509512889877333e-07, "epoch": 1, "t": 4.5218751430511475}
+{"step": 299, "loss": 1.0531494617462158, "grad_norm": 0.6200311779975891, "lr": 4.7517539898741524e-07, "epoch": 1, "t": 4.783984422683716}
+{"step": 300, "loss": 1.031361699104309, "grad_norm": 0.4972401559352875, "lr": 4.049782370561583e-07, "epoch": 1, "t": 4.783889532089233}
+{"step": 301, "loss": 0.9963631629943848, "grad_norm": 0.50146484375, "lr": 3.4036771055923066e-07, "epoch": 1, "t": 4.60985803604126}
+{"step": 302, "loss": 0.986006498336792, "grad_norm": 0.5622588992118835, "lr": 2.813510975548772e-07, "epoch": 1, "t": 4.139053106307983}
+{"step": 303, "loss": 1.0686516761779785, "grad_norm": 0.4927791953086853, "lr": 2.2793504597447002e-07, "epoch": 1, "t": 4.635111093521118}
+{"step": 304, "loss": 1.091249942779541, "grad_norm": 0.517026960849762, "lr": 1.8012557287367392e-07, "epoch": 1, "t": 4.854142189025879}
+{"step": 305, "loss": 1.1145005226135254, "grad_norm": 0.5055581331253052, "lr": 1.379280637546443e-07, "epoch": 1, "t": 4.547342777252197}
+{"step": 306, "loss": 1.0295354127883911, "grad_norm": 0.49713513255119324, "lr": 1.0134727195937333e-07, "epoch": 1, "t": 5.182791471481323}
+{"step": 307, "loss": 0.9750290513038635, "grad_norm": 0.5073899626731873, "lr": 7.038731813426291e-08, "epoch": 1, "t": 5.048128604888916}
+{"step": 308, "loss": 1.188929796218872, "grad_norm": 0.5364013314247131, "lr": 4.5051689765929214e-08, "epoch": 1, "t": 5.233379602432251}
+{"step": 309, "loss": 1.0469988584518433, "grad_norm": 0.5482891798019409, "lr": 2.534324078837802e-08, "epoch": 1, "t": 4.223793029785156}
+{"step": 310, "loss": 1.120493769645691, "grad_norm": 0.5453365445137024, "lr": 1.1264191261528557e-08, "epoch": 1, "t": 4.377922534942627}
+{"step": 311, "loss": 0.9584072828292847, "grad_norm": 0.5121012926101685, "lr": 2.8161271211024633e-09, "epoch": 1, "t": 4.6319379806518555}
+{"step": 312, "loss": 1.0436558723449707, "grad_norm": 0.5431374311447144, "lr": 0.0, "epoch": 1, "t": 4.32390284538269}
diff --git a/gemma-3-27b-it/top-s0/README.md b/gemma-3-27b-it/top-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..4fd4645f0c96a901a2f3322a33f67b7dd9b3c701
--- /dev/null
+++ b/gemma-3-27b-it/top-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: google/gemma-3-27b-it
+library_name: peft
+tags:
+- base_model:adapter:google/gemma-3-27b-it
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/gemma-3-27b-it/top-s0/adapter_config.json b/gemma-3-27b-it/top-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7f13b89fa932def645ea31995a4daaf4ffbed7df
--- /dev/null
+++ b/gemma-3-27b-it/top-s0/adapter_config.json
@@ -0,0 +1,228 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Gemma3ForConditionalGeneration",
+    "parent_library": "transformers.models.gemma3.modeling_gemma3"
+  },
+  "base_model_name_or_path": "google/gemma-3-27b-it",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "40.self_attn.k_proj",
+    "28.self_attn.v_proj",
+    "35.self_attn.q_proj",
+    "language_model.layers.4.self_attn.v_proj",
+    "58.self_attn.q_proj",
+    "language_model.layers.14.self_attn.q_proj",
+    "language_model.layers.3.self_attn.v_proj",
+    "language_model.layers.20.self_attn.k_proj",
+    "38.self_attn.k_proj",
+    "33.self_attn.k_proj",
+    "language_model.layers.24.self_attn.q_proj",
+    "language_model.layers.18.self_attn.q_proj",
+    "34.self_attn.k_proj",
+    "33.self_attn.q_proj",
+    "language_model.layers.23.self_attn.k_proj",
+    "language_model.layers.8.self_attn.v_proj",
+    "language_model.layers.7.self_attn.q_proj",
+    "32.self_attn.v_proj",
+    "language_model.layers.11.self_attn.q_proj",
+    "language_model.layers.21.self_attn.q_proj",
+    "language_model.layers.14.self_attn.k_proj",
+    "58.self_attn.v_proj",
+    "53.self_attn.q_proj",
+    "47.self_attn.v_proj",
+    "45.self_attn.v_proj",
+    "34.self_attn.q_proj",
+    "language_model.layers.3.self_attn.q_proj",
+    "36.self_attn.k_proj",
+    "51.self_attn.k_proj",
+    "54.self_attn.v_proj",
+    "language_model.layers.13.self_attn.q_proj",
+    "language_model.layers.10.self_attn.q_proj",
+    "58.self_attn.k_proj",
+    "language_model.layers.12.self_attn.q_proj",
+    "46.self_attn.k_proj",
+    "41.self_attn.v_proj",
+    "52.self_attn.k_proj",
+    "language_model.layers.6.self_attn.k_proj",
+    "language_model.layers.12.self_attn.k_proj",
+    "language_model.layers.0.self_attn.q_proj",
+    "44.self_attn.q_proj",
+    "language_model.layers.9.self_attn.k_proj",
+    "language_model.layers.7.self_attn.v_proj",
+    "61.self_attn.k_proj",
+    "28.self_attn.q_proj",
+    "language_model.layers.19.self_attn.k_proj",
+    "language_model.layers.22.self_attn.v_proj",
+    "language_model.layers.1.self_attn.k_proj",
+    "language_model.layers.17.self_attn.q_proj",
+    "language_model.layers.5.self_attn.v_proj",
+    "30.self_attn.v_proj",
+    "language_model.layers.6.self_attn.v_proj",
+    "language_model.layers.11.self_attn.k_proj",
+    "41.self_attn.k_proj",
+    "language_model.layers.10.self_attn.k_proj",
+    "38.self_attn.v_proj",
+    "language_model.layers.26.self_attn.k_proj",
+    "language_model.layers.25.self_attn.k_proj",
+    "language_model.layers.8.self_attn.k_proj",
+    "language_model.layers.23.self_attn.v_proj",
+    "50.self_attn.v_proj",
+    "language_model.layers.20.self_attn.q_proj",
+    "57.self_attn.k_proj",
+    "language_model.layers.15.self_attn.k_proj",
+    "language_model.layers.21.self_attn.k_proj",
+    "37.self_attn.k_proj",
+    "language_model.layers.16.self_attn.q_proj",
+    "language_model.layers.0.self_attn.v_proj",
+    "53.self_attn.k_proj",
+    "language_model.layers.13.self_attn.k_proj",
+    "language_model.layers.24.self_attn.v_proj",
+    "27.self_attn.v_proj",
+    "language_model.layers.2.self_attn.k_proj",
+    "47.self_attn.q_proj",
+    "60.self_attn.k_proj",
+    "language_model.layers.22.self_attn.q_proj",
+    "language_model.layers.9.self_attn.v_proj",
+    "47.self_attn.k_proj",
+    "50.self_attn.k_proj",
+    "language_model.layers.19.self_attn.v_proj",
+    "language_model.layers.15.self_attn.q_proj",
+    "language_model.layers.17.self_attn.k_proj",
+    "48.self_attn.q_proj",
+    "36.self_attn.q_proj",
+    "language_model.layers.7.self_attn.k_proj",
+    "45.self_attn.k_proj",
+    "gate_proj",
+    "language_model.layers.15.self_attn.v_proj",
+    "language_model.layers.5.self_attn.q_proj",
+    "45.self_attn.q_proj",
+    "32.self_attn.q_proj",
+    "31.self_attn.k_proj",
+    "language_model.layers.19.self_attn.q_proj",
+    "51.self_attn.v_proj",
+    "language_model.layers.2.self_attn.v_proj",
+    "30.self_attn.q_proj",
+    "language_model.layers.11.self_attn.v_proj",
+    "language_model.layers.5.self_attn.k_proj",
+    "language_model.layers.16.self_attn.k_proj",
+    "39.self_attn.v_proj",
+    "56.self_attn.v_proj",
+    "39.self_attn.k_proj",
+    "50.self_attn.q_proj",
+    "language_model.layers.2.self_attn.q_proj",
+    "language_model.layers.23.self_attn.q_proj",
+    "44.self_attn.v_proj",
+    "27.self_attn.k_proj",
+    "39.self_attn.q_proj",
+    "48.self_attn.k_proj",
+    "59.self_attn.q_proj",
+    "language_model.layers.4.self_attn.k_proj",
+    "37.self_attn.q_proj",
+    "language_model.layers.25.self_attn.q_proj",
+    "31.self_attn.v_proj",
+    "32.self_attn.k_proj",
+    "language_model.layers.4.self_attn.q_proj",
+    "44.self_attn.k_proj",
+    "53.self_attn.v_proj",
+    "language_model.layers.22.self_attn.k_proj",
+    "56.self_attn.k_proj",
+    "27.self_attn.q_proj",
+    "40.self_attn.v_proj",
+    "28.self_attn.k_proj",
+    "language_model.layers.16.self_attn.v_proj",
+    "42.self_attn.v_proj",
+    "29.self_attn.q_proj",
+    "57.self_attn.q_proj",
+    "language_model.layers.3.self_attn.k_proj",
+    "36.self_attn.v_proj",
+    "language_model.layers.20.self_attn.v_proj",
+    "language_model.layers.17.self_attn.v_proj",
+    "language_model.layers.12.self_attn.v_proj",
+    "34.self_attn.v_proj",
+    "61.self_attn.q_proj",
+    "60.self_attn.q_proj",
+    "40.self_attn.q_proj",
+    "language_model.layers.10.self_attn.v_proj",
+    "60.self_attn.v_proj",
+    "o_proj",
+    "33.self_attn.v_proj",
+    "29.self_attn.v_proj",
+    "language_model.layers.9.self_attn.q_proj",
+    "language_model.layers.26.self_attn.v_proj",
+    "language_model.layers.24.self_attn.k_proj",
+    "59.self_attn.k_proj",
+    "42.self_attn.k_proj",
+    "language_model.layers.18.self_attn.v_proj",
+    "55.self_attn.v_proj",
+    "language_model.layers.26.self_attn.q_proj",
+    "51.self_attn.q_proj",
+    "41.self_attn.q_proj",
+    "54.self_attn.q_proj",
+    "language_model.layers.25.self_attn.v_proj",
+    "language_model.layers.6.self_attn.q_proj",
+    "language_model.layers.21.self_attn.v_proj",
+    "language_model.layers.0.self_attn.k_proj",
+    "48.self_attn.v_proj",
+    "57.self_attn.v_proj",
+    "31.self_attn.q_proj",
+    "35.self_attn.v_proj",
+    "52.self_attn.v_proj",
+    "49.self_attn.k_proj",
+    "language_model.layers.8.self_attn.q_proj",
+    "29.self_attn.k_proj",
+    "46.self_attn.v_proj",
+    "37.self_attn.v_proj",
+    "down_proj",
+    "up_proj",
+    "59.self_attn.v_proj",
+    "language_model.layers.1.self_attn.v_proj",
+    "56.self_attn.q_proj",
+    "language_model.layers.18.self_attn.k_proj",
+    "30.self_attn.k_proj",
+    "42.self_attn.q_proj",
+    "49.self_attn.v_proj",
+    "43.self_attn.q_proj",
+    "54.self_attn.k_proj",
+    "38.self_attn.q_proj",
+    "language_model.layers.14.self_attn.v_proj",
+    "46.self_attn.q_proj",
+    "43.self_attn.v_proj",
+    "49.self_attn.q_proj",
+    "language_model.layers.13.self_attn.v_proj",
+    "35.self_attn.k_proj",
+    "52.self_attn.q_proj",
+    "55.self_attn.q_proj",
+    "language_model.layers.1.self_attn.q_proj",
+    "61.self_attn.v_proj",
+    "55.self_attn.k_proj",
+    "43.self_attn.k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/top-s0/adapter_model.safetensors b/gemma-3-27b-it/top-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..bf3e5ee50691feaf5c1c3ffc4727dcbe04fbe2ef
--- /dev/null
+++ b/gemma-3-27b-it/top-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:78600827c4620494eb349138a4bd33fc43d3447affd834ec1a60be7a89a7515e
+size 1816396048
diff --git a/gemma-3-27b-it/top-s0/config.json b/gemma-3-27b-it/top-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..ca8d9228766aa1bda84d24528577c09fda73af32
--- /dev/null
+++ b/gemma-3-27b-it/top-s0/config.json
@@ -0,0 +1,33 @@
+{
+  "model": "google/gemma-3-27b-it",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/arms/TASKW-top-gemma.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/gemma/runs/top-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/code/gemma3_lora_targets.json",
+  "seed": 0,
+  "bos": true,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/gemma-3-27b-it/top-s0/log.jsonl b/gemma-3-27b-it/top-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..95dbc47f3a476acdb419c9240ca43bfffa241eeb
--- /dev/null
+++ b/gemma-3-27b-it/top-s0/log.jsonl
@@ -0,0 +1,143 @@
+{"step": 1, "loss": 0.22505121769766853, "grad_norm": 0.21703773736953735, "lr": 2.857142857142857e-05, "epoch": 1, "t": 12.054490566253662}
+{"step": 2, "loss": 0.24540961073642836, "grad_norm": 0.24778541922569275, "lr": 4.2857142857142856e-05, "epoch": 1, "t": 9.516986608505249}
+{"step": 3, "loss": 0.22930878784614653, "grad_norm": 0.27400651574134827, "lr": 5.714285714285714e-05, "epoch": 1, "t": 10.105645656585693}
+{"step": 4, "loss": 0.22194811920350035, "grad_norm": 0.23619204759597778, "lr": 7.142857142857143e-05, "epoch": 1, "t": 11.43257188796997}
+{"step": 5, "loss": 0.22268119415085658, "grad_norm": 0.2712884545326233, "lr": 8.571428571428571e-05, "epoch": 1, "t": 9.65244460105896}
+{"step": 6, "loss": 0.20631448412455675, "grad_norm": 0.2699175179004669, "lr": 0.0001, "epoch": 1, "t": 11.324448108673096}
+{"step": 7, "loss": 0.24343506528702055, "grad_norm": 0.33858978748321533, "lr": 0.0001, "epoch": 1, "t": 9.84887146949768}
+{"step": 8, "loss": 0.199659386745719, "grad_norm": 0.28170526027679443, "lr": 9.998666040558187e-05, "epoch": 1, "t": 9.421858787536621}
+{"step": 9, "loss": 0.22586406636767833, "grad_norm": 0.27943772077560425, "lr": 9.994664874011863e-05, "epoch": 1, "t": 10.434593677520752}
+{"step": 10, "loss": 0.22707384798576655, "grad_norm": 0.2697888910770416, "lr": 9.987998635318585e-05, "epoch": 1, "t": 10.50551176071167}
+{"step": 11, "loss": 0.24545073671295564, "grad_norm": 0.2650054395198822, "lr": 9.978670881475172e-05, "epoch": 1, "t": 10.266358375549316}
+{"step": 12, "loss": 0.19028063638530995, "grad_norm": 0.23152479529380798, "lr": 9.966686589619751e-05, "epoch": 1, "t": 10.103020191192627}
+{"step": 13, "loss": 0.22196000356158432, "grad_norm": 1.3057767152786255, "lr": 9.952052154376026e-05, "epoch": 1, "t": 10.711141347885132}
+{"step": 14, "loss": 0.21300532938160982, "grad_norm": 0.23710189759731293, "lr": 9.934775384441228e-05, "epoch": 1, "t": 10.228380918502808}
+{"step": 15, "loss": 0.24879513348636922, "grad_norm": 0.2256975620985031, "lr": 9.91486549841951e-05, "epoch": 1, "t": 12.432562351226807}
+{"step": 16, "loss": 0.20557237603502415, "grad_norm": 0.2202509343624115, "lr": 9.892333119903046e-05, "epoch": 1, "t": 11.608261823654175}
+{"step": 17, "loss": 0.19515724440172577, "grad_norm": 0.23974862694740295, "lr": 9.867190271803465e-05, "epoch": 1, "t": 10.96469497680664}
+{"step": 18, "loss": 0.1967107274436229, "grad_norm": 0.28855326771736145, "lr": 9.839450369936613e-05, "epoch": 1, "t": 8.185019731521606}
+{"step": 19, "loss": 0.21986242121673952, "grad_norm": 0.2748332619667053, "lr": 9.809128215864097e-05, "epoch": 1, "t": 9.775784492492676}
+{"step": 20, "loss": 0.21358585373547354, "grad_norm": 0.23560836911201477, "lr": 9.7762399889954e-05, "epoch": 1, "t": 9.42101788520813}
+{"step": 21, "loss": 0.1925711565740602, "grad_norm": 0.24786853790283203, "lr": 9.74080323795483e-05, "epoch": 1, "t": 9.326472282409668}
+{"step": 22, "loss": 0.238364584641796, "grad_norm": 0.24411338567733765, "lr": 9.702836871217839e-05, "epoch": 1, "t": 10.775306224822998}
+{"step": 23, "loss": 0.1964091545150857, "grad_norm": 0.21883657574653625, "lr": 9.662361147021779e-05, "epoch": 1, "t": 11.269694805145264}
+{"step": 24, "loss": 0.19321282507392137, "grad_norm": 0.2659012973308563, "lr": 9.619397662556435e-05, "epoch": 1, "t": 8.135856866836548}
+{"step": 25, "loss": 0.24860771997723058, "grad_norm": 0.2588931918144226, "lr": 9.573969342440106e-05, "epoch": 1, "t": 9.904388666152954}
+{"step": 26, "loss": 0.20770942510836127, "grad_norm": 0.2095607966184616, "lr": 9.526100426487409e-05, "epoch": 1, "t": 9.920934200286865}
+{"step": 27, "loss": 0.2006321762915853, "grad_norm": 0.21268363296985626, "lr": 9.475816456775313e-05, "epoch": 1, "t": 10.83916163444519}
+{"step": 28, "loss": 0.24594306693255452, "grad_norm": 0.22133034467697144, "lr": 9.423144264014277e-05, "epoch": 1, "t": 9.63451361656189}
+{"step": 29, "loss": 0.19797796972210516, "grad_norm": 0.193564772605896, "lr": 9.368111953231848e-05, "epoch": 1, "t": 10.058306694030762}
+{"step": 30, "loss": 0.20446276894851784, "grad_norm": 0.2121383547782898, "lr": 9.310748888776254e-05, "epoch": 1, "t": 9.457752704620361}
+{"step": 31, "loss": 0.19927488339865662, "grad_norm": 0.19595593214035034, "lr": 9.251085678648072e-05, "epoch": 1, "t": 10.397753953933716}
+{"step": 32, "loss": 0.21001055971556223, "grad_norm": 0.227900892496109, "lr": 9.189154158168292e-05, "epoch": 1, "t": 8.492723226547241}
+{"step": 33, "loss": 0.225430331818045, "grad_norm": 0.21005074679851532, "lr": 9.124987372991511e-05, "epoch": 1, "t": 10.937589883804321}
+{"step": 34, "loss": 0.20656682381318806, "grad_norm": 0.225301131606102, "lr": 9.058619561473307e-05, "epoch": 1, "t": 8.90029764175415}
+{"step": 35, "loss": 0.22779996080064685, "grad_norm": 0.25957736372947693, "lr": 8.9900861364012e-05, "epoch": 1, "t": 9.490508079528809}
+{"step": 36, "loss": 0.2126345234350297, "grad_norm": 0.23203442990779877, "lr": 8.919423666098969e-05, "epoch": 1, "t": 11.106785535812378}
+{"step": 37, "loss": 0.2313187568391229, "grad_norm": 0.2481047362089157, "lr": 8.846669854914396e-05, "epoch": 1, "t": 9.578790187835693}
+{"step": 38, "loss": 0.22694694870278267, "grad_norm": 0.22763553261756897, "lr": 8.77186352310082e-05, "epoch": 1, "t": 10.389273881912231}
+{"step": 39, "loss": 0.19878314602618677, "grad_norm": 0.2309281826019287, "lr": 8.695044586103296e-05, "epoch": 1, "t": 10.274710416793823}
+{"step": 40, "loss": 0.2167181975434751, "grad_norm": 0.24669983983039856, "lr": 8.616254033260351e-05, "epoch": 1, "t": 8.751072406768799}
+{"step": 41, "loss": 0.24636918802136137, "grad_norm": 0.22230270504951477, "lr": 8.535533905932738e-05, "epoch": 1, "t": 11.74364447593689}
+{"step": 42, "loss": 0.22521391857101258, "grad_norm": 0.21574485301971436, "lr": 8.452927275070858e-05, "epoch": 1, "t": 10.1707022190094}
+{"step": 43, "loss": 0.1882047427302163, "grad_norm": 0.41809940338134766, "lr": 8.368478218232787e-05, "epoch": 1, "t": 10.103673219680786}
+{"step": 44, "loss": 0.2175239191135965, "grad_norm": 0.2280159592628479, "lr": 8.282231796065214e-05, "epoch": 1, "t": 10.507831573486328}
+{"step": 45, "loss": 0.22175029550468522, "grad_norm": 0.2500484883785248, "lr": 8.194234028259806e-05, "epoch": 1, "t": 9.814427375793457}
+{"step": 46, "loss": 0.21215120287620454, "grad_norm": 0.19898876547813416, "lr": 8.104531868997858e-05, "epoch": 1, "t": 11.341955423355103}
+{"step": 47, "loss": 0.18998032131318401, "grad_norm": 0.1909809708595276, "lr": 8.013173181896283e-05, "epoch": 1, "t": 10.59896445274353}
+{"step": 48, "loss": 0.22170501866770564, "grad_norm": 0.20594868063926697, "lr": 7.920206714468383e-05, "epoch": 1, "t": 11.040373802185059}
+{"step": 49, "loss": 0.215590952729174, "grad_norm": 0.22152985632419586, "lr": 7.82568207211296e-05, "epoch": 1, "t": 9.514512777328491}
+{"step": 50, "loss": 0.2122216894412017, "grad_norm": 0.23917758464813232, "lr": 7.729649691645673e-05, "epoch": 1, "t": 9.262718200683594}
+{"step": 51, "loss": 0.20459794676564677, "grad_norm": 0.19892755150794983, "lr": 7.63216081438678e-05, "epoch": 1, "t": 9.69749402999878}
+{"step": 52, "loss": 0.21811848597818762, "grad_norm": 0.19602754712104797, "lr": 7.533267458819598e-05, "epoch": 1, "t": 10.440067768096924}
+{"step": 53, "loss": 0.21435127394291206, "grad_norm": 0.23377782106399536, "lr": 7.433022392834282e-05, "epoch": 1, "t": 10.183382511138916}
+{"step": 54, "loss": 0.20927664727682346, "grad_norm": 0.20169499516487122, "lr": 7.33147910557174e-05, "epoch": 1, "t": 11.860474109649658}
+{"step": 55, "loss": 0.25015311167801246, "grad_norm": 0.22437871992588043, "lr": 7.228691778882693e-05, "epoch": 1, "t": 10.844437599182129}
+{"step": 56, "loss": 0.2546206663716142, "grad_norm": 0.24023710191249847, "lr": 7.12471525841711e-05, "epoch": 1, "t": 11.428836822509766}
+{"step": 57, "loss": 0.24053334362738954, "grad_norm": 0.25352856516838074, "lr": 7.019605024359474e-05, "epoch": 1, "t": 9.737306118011475}
+{"step": 58, "loss": 0.22556048528626216, "grad_norm": 0.21189622581005096, "lr": 6.91341716182545e-05, "epoch": 1, "t": 12.7682044506073}
+{"step": 59, "loss": 0.18967859314301058, "grad_norm": 0.22413399815559387, "lr": 6.806208330935766e-05, "epoch": 1, "t": 8.58992075920105}
+{"step": 60, "loss": 0.21742234890089432, "grad_norm": 0.20742391049861908, "lr": 6.698035736583306e-05, "epoch": 1, "t": 11.425224542617798}
+{"step": 61, "loss": 0.2135961299014334, "grad_norm": 0.2490430474281311, "lr": 6.588957097909508e-05, "epoch": 1, "t": 9.635200262069702}
+{"step": 62, "loss": 0.21689573479258453, "grad_norm": 0.2328774482011795, "lr": 6.479030617506353e-05, "epoch": 1, "t": 9.334052324295044}
+{"step": 63, "loss": 0.23668449976230663, "grad_norm": 0.21172454953193665, "lr": 6.368314950360415e-05, "epoch": 1, "t": 10.989005088806152}
+{"step": 64, "loss": 0.2029248450160242, "grad_norm": 0.19610105454921722, "lr": 6.256869172555515e-05, "epoch": 1, "t": 10.582974672317505}
+{"step": 65, "loss": 0.20569957869097744, "grad_norm": 0.206247016787529, "lr": 6.14475274975067e-05, "epoch": 1, "t": 11.197489738464355}
+{"step": 66, "loss": 0.1983781010666692, "grad_norm": 0.19008895754814148, "lr": 6.0320255054501985e-05, "epoch": 1, "t": 11.482795476913452}
+{"step": 67, "loss": 0.21081169469715408, "grad_norm": 0.1924007385969162, "lr": 5.918747589082853e-05, "epoch": 1, "t": 10.047632455825806}
+{"step": 68, "loss": 0.22280451165881132, "grad_norm": 0.1961202621459961, "lr": 5.8049794439070646e-05, "epoch": 1, "t": 10.537443399429321}
+{"step": 69, "loss": 0.20206253726379408, "grad_norm": 0.21997730433940887, "lr": 5.6907817747594116e-05, "epoch": 1, "t": 9.550058841705322}
+{"step": 70, "loss": 0.2047861488675729, "grad_norm": 0.19869445264339447, "lr": 5.576215515663489e-05, "epoch": 1, "t": 9.825855016708374}
+{"step": 71, "loss": 0.24573015028099182, "grad_norm": 0.26009121537208557, "lr": 5.4613417973165106e-05, "epoch": 1, "t": 8.839526891708374}
+{"step": 72, "loss": 0.23537826049450813, "grad_norm": 0.18990512192249298, "lr": 5.346221914470959e-05, "epoch": 1, "t": 11.488405704498291}
+{"step": 73, "loss": 0.2329046377068231, "grad_norm": 0.21941818296909332, "lr": 5.230917293228699e-05, "epoch": 1, "t": 9.61008334159851}
+{"step": 74, "loss": 0.21540734000262854, "grad_norm": 0.20822052657604218, "lr": 5.115489458265006e-05, "epoch": 1, "t": 11.6629319190979}
+{"step": 75, "loss": 0.24852643891305604, "grad_norm": 0.2189219444990158, "lr": 5e-05, "epoch": 1, "t": 11.586644411087036}
+{"step": 76, "loss": 0.21666138455855163, "grad_norm": 0.17972682416439056, "lr": 4.884510541734995e-05, "epoch": 1, "t": 10.041048765182495}
+{"step": 77, "loss": 0.19575391571305498, "grad_norm": 0.20232640206813812, "lr": 4.7690827067713035e-05, "epoch": 1, "t": 9.278618812561035}
+{"step": 78, "loss": 0.2271962047940843, "grad_norm": 0.21232342720031738, "lr": 4.6537780855290426e-05, "epoch": 1, "t": 10.942893981933594}
+{"step": 79, "loss": 0.23146169841220054, "grad_norm": 0.18582604825496674, "lr": 4.5386582026834906e-05, "epoch": 1, "t": 11.442293643951416}
+{"step": 80, "loss": 0.22945719404400045, "grad_norm": 0.24335208535194397, "lr": 4.423784484336513e-05, "epoch": 1, "t": 8.328362226486206}
+{"step": 81, "loss": 0.22532927693692714, "grad_norm": 0.19784903526306152, "lr": 4.30921822524059e-05, "epoch": 1, "t": 11.979598760604858}
+{"step": 82, "loss": 0.219247266354893, "grad_norm": 0.22701801359653473, "lr": 4.195020556092935e-05, "epoch": 1, "t": 10.241004943847656}
+{"step": 83, "loss": 0.24517080890883958, "grad_norm": 0.19996687769889832, "lr": 4.0812524109171476e-05, "epoch": 1, "t": 11.779272079467773}
+{"step": 84, "loss": 0.20453113444419768, "grad_norm": 0.1847640722990036, "lr": 3.9679744945498026e-05, "epoch": 1, "t": 10.896613597869873}
+{"step": 85, "loss": 0.1986247555989034, "grad_norm": 0.21294614672660828, "lr": 3.855247250249331e-05, "epoch": 1, "t": 11.634602546691895}
+{"step": 86, "loss": 0.21482527971986304, "grad_norm": 0.20600295066833496, "lr": 3.7431308274444865e-05, "epoch": 1, "t": 11.46474313735962}
+{"step": 87, "loss": 0.23929734182753465, "grad_norm": 0.20089924335479736, "lr": 3.631685049639586e-05, "epoch": 1, "t": 11.491093158721924}
+{"step": 88, "loss": 0.22174171709610532, "grad_norm": 0.2268877625465393, "lr": 3.5209693824936484e-05, "epoch": 1, "t": 9.254255294799805}
+{"step": 89, "loss": 0.2352444302937624, "grad_norm": 0.2403077632188797, "lr": 3.411042902090492e-05, "epoch": 1, "t": 8.510625123977661}
+{"step": 90, "loss": 0.2030368671781914, "grad_norm": 0.2170497626066208, "lr": 3.301964263416693e-05, "epoch": 1, "t": 10.886615991592407}
+{"step": 91, "loss": 0.2012726849903261, "grad_norm": 0.1920071244239807, "lr": 3.1937916690642356e-05, "epoch": 1, "t": 10.60518479347229}
+{"step": 92, "loss": 0.21487654513840465, "grad_norm": 0.48220759630203247, "lr": 3.086582838174551e-05, "epoch": 1, "t": 11.4157874584198}
+{"step": 93, "loss": 0.21617813801950178, "grad_norm": 0.1996578872203827, "lr": 2.980394975640526e-05, "epoch": 1, "t": 10.667667627334595}
+{"step": 94, "loss": 0.213875618005616, "grad_norm": 0.21704868972301483, "lr": 2.875284741582892e-05, "epoch": 1, "t": 10.687480926513672}
+{"step": 95, "loss": 0.20148991226217355, "grad_norm": 0.19102336466312408, "lr": 2.771308221117309e-05, "epoch": 1, "t": 8.994153499603271}
+{"step": 96, "loss": 0.22516883546762628, "grad_norm": 0.2238171100616455, "lr": 2.668520894428259e-05, "epoch": 1, "t": 9.848058700561523}
+{"step": 97, "loss": 0.23498747648507914, "grad_norm": 0.18566302955150604, "lr": 2.5669776071657192e-05, "epoch": 1, "t": 11.381712913513184}
+{"step": 98, "loss": 0.20891795197826057, "grad_norm": 0.1896337866783142, "lr": 2.4667325411804036e-05, "epoch": 1, "t": 11.594501972198486}
+{"step": 99, "loss": 0.20798248217693593, "grad_norm": 0.19009491801261902, "lr": 2.3678391856132204e-05, "epoch": 1, "t": 10.882423162460327}
+{"step": 100, "loss": 0.22777974444781332, "grad_norm": 0.21916241943836212, "lr": 2.270350308354329e-05, "epoch": 1, "t": 8.92090129852295}
+{"step": 101, "loss": 0.22950904547540296, "grad_norm": 0.19836682081222534, "lr": 2.1743179278870407e-05, "epoch": 1, "t": 12.38113021850586}
+{"step": 102, "loss": 0.19720774813543393, "grad_norm": 0.18992555141448975, "lr": 2.079793285531618e-05, "epoch": 1, "t": 12.320561408996582}
+{"step": 103, "loss": 0.21829094940529548, "grad_norm": 0.18572694063186646, "lr": 1.9868268181037185e-05, "epoch": 1, "t": 11.610471963882446}
+{"step": 104, "loss": 0.2118741783090995, "grad_norm": 0.23200030624866486, "lr": 1.8954681310021433e-05, "epoch": 1, "t": 8.644611358642578}
+{"step": 105, "loss": 0.20191033372910921, "grad_norm": 0.17914192378520966, "lr": 1.8057659717401947e-05, "epoch": 1, "t": 12.089706659317017}
+{"step": 106, "loss": 0.25069651295919615, "grad_norm": 0.21586081385612488, "lr": 1.7177682039347876e-05, "epoch": 1, "t": 9.645750045776367}
+{"step": 107, "loss": 0.21194703398225723, "grad_norm": 0.19013509154319763, "lr": 1.631521781767214e-05, "epoch": 1, "t": 11.319631814956665}
+{"step": 108, "loss": 0.22893488565647194, "grad_norm": 0.21662981808185577, "lr": 1.5470727249291423e-05, "epoch": 1, "t": 10.19741153717041}
+{"step": 109, "loss": 0.22705423582775164, "grad_norm": 0.23066486418247223, "lr": 1.4644660940672627e-05, "epoch": 1, "t": 8.92653489112854}
+{"step": 110, "loss": 0.24917763492387654, "grad_norm": 0.23151402175426483, "lr": 1.3837459667396518e-05, "epoch": 1, "t": 10.738795757293701}
+{"step": 111, "loss": 0.23403816625410656, "grad_norm": 0.20127059519290924, "lr": 1.3049554138967051e-05, "epoch": 1, "t": 11.840016841888428}
+{"step": 112, "loss": 0.26001353187756193, "grad_norm": 0.246088445186615, "lr": 1.2281364768991804e-05, "epoch": 1, "t": 11.113420486450195}
+{"step": 113, "loss": 0.2240116816353577, "grad_norm": 0.21575187146663666, "lr": 1.1533301450856054e-05, "epoch": 1, "t": 9.72246527671814}
+{"step": 114, "loss": 0.22369254766458488, "grad_norm": 0.20920686423778534, "lr": 1.0805763339010327e-05, "epoch": 1, "t": 8.78674840927124}
+{"step": 115, "loss": 0.20912365010812498, "grad_norm": 0.2666623890399933, "lr": 1.0099138635988026e-05, "epoch": 1, "t": 9.524937629699707}
+{"step": 116, "loss": 0.20315464573109066, "grad_norm": 0.18049681186676025, "lr": 9.41380438526694e-06, "epoch": 1, "t": 12.498393774032593}
+{"step": 117, "loss": 0.19453228550149712, "grad_norm": 0.2050487995147705, "lr": 8.75012627008489e-06, "epoch": 1, "t": 10.651840209960938}
+{"step": 118, "loss": 0.20798705938511292, "grad_norm": 0.19289687275886536, "lr": 8.10845841831709e-06, "epoch": 1, "t": 10.539570093154907}
+{"step": 119, "loss": 0.2331430976424099, "grad_norm": 0.21947957575321198, "lr": 7.489143213519301e-06, "epoch": 1, "t": 9.810571193695068}
+{"step": 120, "loss": 0.23201207653560108, "grad_norm": 0.21218451857566833, "lr": 6.892511112237471e-06, "epoch": 1, "t": 10.715802907943726}
+{"step": 121, "loss": 0.21074999272986514, "grad_norm": 0.21293194591999054, "lr": 6.318880467681526e-06, "epoch": 1, "t": 9.140785932540894}
+{"step": 122, "loss": 0.21485237253398237, "grad_norm": 0.19227859377861023, "lr": 5.768557359857241e-06, "epoch": 1, "t": 11.459423542022705}
+{"step": 123, "loss": 0.20440094586761795, "grad_norm": 0.23368176817893982, "lr": 5.241835432246889e-06, "epoch": 1, "t": 11.006624221801758}
+{"step": 124, "loss": 0.21604196356349853, "grad_norm": 0.20674683153629303, "lr": 4.7389957351258955e-06, "epoch": 1, "t": 11.091724395751953}
+{"step": 125, "loss": 0.1918921984028676, "grad_norm": 0.20515887439250946, "lr": 4.260306575598949e-06, "epoch": 1, "t": 10.054367780685425}
+{"step": 126, "loss": 0.2155248782340382, "grad_norm": 0.17134223878383636, "lr": 3.8060233744356633e-06, "epoch": 1, "t": 12.724954843521118}
+{"step": 127, "loss": 0.21280758940143343, "grad_norm": 0.20869047939777374, "lr": 3.376388529782215e-06, "epoch": 1, "t": 9.8471097946167}
+{"step": 128, "loss": 0.20093681955694506, "grad_norm": 0.21226777136325836, "lr": 2.971631287821619e-06, "epoch": 1, "t": 9.71830940246582}
+{"step": 129, "loss": 0.22660333895389922, "grad_norm": 0.19003553688526154, "lr": 2.591967620451707e-06, "epoch": 1, "t": 11.446911096572876}
+{"step": 130, "loss": 0.21706106114979792, "grad_norm": 0.19154681265354156, "lr": 2.237600110046001e-06, "epoch": 1, "t": 11.401564121246338}
+{"step": 131, "loss": 0.2186806542202629, "grad_norm": 0.21070224046707153, "lr": 1.908717841359048e-06, "epoch": 1, "t": 9.507974863052368}
+{"step": 132, "loss": 0.25809210892890944, "grad_norm": 0.2355871945619583, "lr": 1.6054963006338742e-06, "epoch": 1, "t": 11.112389087677002}
+{"step": 133, "loss": 0.21984846476706793, "grad_norm": 0.2116549015045166, "lr": 1.328097281965357e-06, "epoch": 1, "t": 10.80772614479065}
+{"step": 134, "loss": 0.22923553089842794, "grad_norm": 0.2332264482975006, "lr": 1.0766688009695547e-06, "epoch": 1, "t": 10.506229400634766}
+{"step": 135, "loss": 0.22959739099443716, "grad_norm": 0.2471969723701477, "lr": 8.513450158049108e-07, "epoch": 1, "t": 8.495505571365356}
+{"step": 136, "loss": 0.2238175068600995, "grad_norm": 0.1862340122461319, "lr": 6.522461555877213e-07, "epoch": 1, "t": 10.767034530639648}
+{"step": 137, "loss": 0.2077139842861446, "grad_norm": 0.19255129992961884, "lr": 4.794784562397458e-07, "epoch": 1, "t": 11.160823583602905}
+{"step": 138, "loss": 0.20058147145811672, "grad_norm": 0.1884453445672989, "lr": 3.3313410380250154e-07, "epoch": 1, "t": 10.169746160507202}
+{"step": 139, "loss": 0.2571327569272948, "grad_norm": 0.23495423793792725, "lr": 2.1329118524827662e-07, "epoch": 1, "t": 10.18721318244934}
+{"step": 140, "loss": 0.24053145962129716, "grad_norm": 0.23312735557556152, "lr": 1.200136468141544e-07, "epoch": 1, "t": 9.572031736373901}
+{"step": 141, "loss": 0.20576847730523853, "grad_norm": 0.19867339730262756, "lr": 5.3351259881379014e-08, "epoch": 1, "t": 9.011807680130005}
+{"step": 142, "loss": 0.20119772438518577, "grad_norm": 0.2028876543045044, "lr": 1.3339594418138035e-08, "epoch": 1, "t": 11.02613377571106}
+{"step": 143, "loss": 0.18990486103182108, "grad_norm": 0.21175725758075714, "lr": 0.0, "epoch": 1, "t": 11.162810802459717}
diff --git a/qwen3-14b/bottom-s0/README.md b/qwen3-14b/bottom-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/bottom-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/bottom-s0/adapter_config.json b/qwen3-14b/bottom-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..8167e8f4cf2db230d3c7cc7c1bbf7e9c97aafc46
--- /dev/null
+++ b/qwen3-14b/bottom-s0/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "gate_proj",
+    "q_proj",
+    "down_proj",
+    "k_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/bottom-s0/adapter_model.safetensors b/qwen3-14b/bottom-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..aca864c5c721a8e4cff372ec3855df32913e8f3c
--- /dev/null
+++ b/qwen3-14b/bottom-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8106cd1534450e97a758c59f6978f271c3d711bd1dfc9e67d2ded0bd31577a9e
+size 1027680256
diff --git a/qwen3-14b/bottom-s0/config.json b/qwen3-14b/bottom-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..4d31b6a65f4a3cab2c8ae78e21c2fe034d8685c3
--- /dev/null
+++ b/qwen3-14b/bottom-s0/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-bottom-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/bottom-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 0,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/bottom-s0/log.jsonl b/qwen3-14b/bottom-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..33f71205a7fadea14468b3a33ac485068a15529f
--- /dev/null
+++ b/qwen3-14b/bottom-s0/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.1752485499130772, "grad_norm": 0.2273269146680832, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 7.2622599601745605}
+{"step": 2, "loss": 0.16197575131929842, "grad_norm": 0.18963158130645752, "lr": 2e-05, "epoch": 1, "t": 6.842254877090454}
+{"step": 3, "loss": 0.14372770245109295, "grad_norm": 0.2149733006954193, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 6.149490594863892}
+{"step": 4, "loss": 0.17578186904840032, "grad_norm": 0.2089262306690216, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 6.206242561340332}
+{"step": 5, "loss": 0.16244048400005653, "grad_norm": 0.20248648524284363, "lr": 4e-05, "epoch": 1, "t": 6.000253200531006}
+{"step": 6, "loss": 0.14900893488448583, "grad_norm": 0.20379552245140076, "lr": 4.666666666666667e-05, "epoch": 1, "t": 6.036116123199463}
+{"step": 7, "loss": 0.12102704053220495, "grad_norm": 0.25668591260910034, "lr": 5.333333333333333e-05, "epoch": 1, "t": 6.182107210159302}
+{"step": 8, "loss": 0.13636366648148307, "grad_norm": 0.5128408074378967, "lr": 6e-05, "epoch": 1, "t": 5.9150471687316895}
+{"step": 9, "loss": 0.16004198848871895, "grad_norm": 0.26240718364715576, "lr": 6.666666666666667e-05, "epoch": 1, "t": 5.8975677490234375}
+{"step": 10, "loss": 0.13735793998360057, "grad_norm": 0.3030981719493866, "lr": 7.333333333333333e-05, "epoch": 1, "t": 5.992259740829468}
+{"step": 11, "loss": 0.12292885473227813, "grad_norm": 0.2684709131717682, "lr": 8e-05, "epoch": 1, "t": 6.029702663421631}
+{"step": 12, "loss": 0.1540826850778417, "grad_norm": 0.24784551560878754, "lr": 8.666666666666667e-05, "epoch": 1, "t": 5.965942621231079}
+{"step": 13, "loss": 0.125180684584017, "grad_norm": 0.21856532990932465, "lr": 9.333333333333334e-05, "epoch": 1, "t": 5.84442925453186}
+{"step": 14, "loss": 0.1336468793688672, "grad_norm": 0.1753084808588028, "lr": 0.0001, "epoch": 1, "t": 6.03787899017334}
+{"step": 15, "loss": 0.11304400092002005, "grad_norm": 0.23694230616092682, "lr": 0.0001, "epoch": 1, "t": 5.878647565841675}
+{"step": 16, "loss": 0.11225881042186095, "grad_norm": 0.2033313512802124, "lr": 9.999687519737639e-05, "epoch": 1, "t": 5.923731327056885}
+{"step": 17, "loss": 0.11136573642254222, "grad_norm": 0.16831859946250916, "lr": 9.998750118008115e-05, "epoch": 1, "t": 5.944855690002441}
+{"step": 18, "loss": 0.1436524074572562, "grad_norm": 0.22492867708206177, "lr": 9.997187911979252e-05, "epoch": 1, "t": 4.660308122634888}
+{"step": 19, "loss": 0.10213176040634413, "grad_norm": 0.19424419105052948, "lr": 9.995001096914461e-05, "epoch": 1, "t": 4.2838804721832275}
+{"step": 20, "loss": 0.1249386400594415, "grad_norm": 0.17311154305934906, "lr": 9.992189946148366e-05, "epoch": 1, "t": 5.620733022689819}
+{"step": 21, "loss": 0.11955899123787346, "grad_norm": 0.16216126084327698, "lr": 9.988754811052616e-05, "epoch": 1, "t": 6.133888244628906}
+{"step": 22, "loss": 0.1367188526761167, "grad_norm": 0.1654491275548935, "lr": 9.984696120991978e-05, "epoch": 1, "t": 6.002611398696899}
+{"step": 23, "loss": 0.10915484067420786, "grad_norm": 0.13785584270954132, "lr": 9.980014383270668e-05, "epoch": 1, "t": 5.492818593978882}
+{"step": 24, "loss": 0.11355082914945644, "grad_norm": 0.16870281100273132, "lr": 9.974710183068935e-05, "epoch": 1, "t": 5.871350049972534}
+{"step": 25, "loss": 0.10453763878630122, "grad_norm": 0.19669336080551147, "lr": 9.968784183369929e-05, "epoch": 1, "t": 5.564558267593384}
+{"step": 26, "loss": 0.09042725515465194, "grad_norm": 0.1999388039112091, "lr": 9.962237124876828e-05, "epoch": 1, "t": 5.271064758300781}
+{"step": 27, "loss": 0.14823067171213114, "grad_norm": 0.184713676571846, "lr": 9.955069825920249e-05, "epoch": 1, "t": 5.541146755218506}
+{"step": 28, "loss": 0.11725434708859521, "grad_norm": 0.19592486321926117, "lr": 9.947283182355982e-05, "epoch": 1, "t": 5.536479234695435}
+{"step": 29, "loss": 0.12784560044026239, "grad_norm": 0.19796410202980042, "lr": 9.938878167452992e-05, "epoch": 1, "t": 5.433374881744385}
+{"step": 30, "loss": 0.13446450256655165, "grad_norm": 0.1550251692533493, "lr": 9.929855831771786e-05, "epoch": 1, "t": 5.4044764041900635}
+{"step": 31, "loss": 0.12071090125751266, "grad_norm": 0.17487181723117828, "lr": 9.92021730303309e-05, "epoch": 1, "t": 5.400969743728638}
+{"step": 32, "loss": 0.09524619597507142, "grad_norm": 0.193100705742836, "lr": 9.909963785976903e-05, "epoch": 1, "t": 5.389806509017944}
+{"step": 33, "loss": 0.12547221358709215, "grad_norm": 0.158849835395813, "lr": 9.899096562211902e-05, "epoch": 1, "t": 5.163612127304077}
+{"step": 34, "loss": 0.14120039416162725, "grad_norm": 0.16312792897224426, "lr": 9.887616990055262e-05, "epoch": 1, "t": 4.6968300342559814}
+{"step": 35, "loss": 0.11725060856240427, "grad_norm": 0.16907823085784912, "lr": 9.875526504362869e-05, "epoch": 1, "t": 4.679459810256958}
+{"step": 36, "loss": 0.11519571962900538, "grad_norm": 0.39077630639076233, "lr": 9.86282661634998e-05, "epoch": 1, "t": 5.017693042755127}
+{"step": 37, "loss": 0.10394156329469015, "grad_norm": 0.19277916848659515, "lr": 9.849518913402334e-05, "epoch": 1, "t": 4.630793809890747}
+{"step": 38, "loss": 0.13825914087408372, "grad_norm": 0.1808989942073822, "lr": 9.835605058877729e-05, "epoch": 1, "t": 4.734872341156006}
+{"step": 39, "loss": 0.14669809025035924, "grad_norm": 0.15419261157512665, "lr": 9.821086791898134e-05, "epoch": 1, "t": 5.179617643356323}
+{"step": 40, "loss": 0.1154066498297602, "grad_norm": 0.19307547807693481, "lr": 9.805965927132295e-05, "epoch": 1, "t": 4.3937201499938965}
+{"step": 41, "loss": 0.15095672952380426, "grad_norm": 0.19648480415344238, "lr": 9.79024435456893e-05, "epoch": 1, "t": 3.8020637035369873}
+{"step": 42, "loss": 0.1332015265766372, "grad_norm": 0.16474975645542145, "lr": 9.773924039280487e-05, "epoch": 1, "t": 4.042931318283081}
+{"step": 43, "loss": 0.1271522599560576, "grad_norm": 0.16134263575077057, "lr": 9.75700702117753e-05, "epoch": 1, "t": 4.079339265823364}
+{"step": 44, "loss": 0.10394006987216055, "grad_norm": 0.17610692977905273, "lr": 9.739495414753753e-05, "epoch": 1, "t": 3.747913360595703}
+{"step": 45, "loss": 0.1147655960400625, "grad_norm": 0.14709608256816864, "lr": 9.721391408821711e-05, "epoch": 1, "t": 3.8322336673736572}
+{"step": 46, "loss": 0.1092554634260399, "grad_norm": 0.16116903722286224, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.805285930633545}
+{"step": 47, "loss": 0.13763013517263303, "grad_norm": 0.1721290647983551, "lr": 9.683415323626485e-05, "epoch": 1, "t": 3.5704293251037598}
+{"step": 48, "loss": 0.09924356610977464, "grad_norm": 0.1799483746290207, "lr": 9.663547991074127e-05, "epoch": 1, "t": 3.1370041370391846}
+{"step": 49, "loss": 0.12897537376202683, "grad_norm": 0.17098456621170044, "lr": 9.643097751841854e-05, "epoch": 1, "t": 3.6654934883117676}
+{"step": 50, "loss": 0.11030346398450891, "grad_norm": 0.1793755441904068, "lr": 9.622067162048112e-05, "epoch": 1, "t": 3.2133631706237793}
+{"step": 51, "loss": 0.10788446882157156, "grad_norm": 0.1423107087612152, "lr": 9.600458850350588e-05, "epoch": 1, "t": 3.9863667488098145}
+{"step": 52, "loss": 0.12193814227010509, "grad_norm": 0.27060258388519287, "lr": 9.578275517617645e-05, "epoch": 1, "t": 3.823734998703003}
+{"step": 53, "loss": 0.13209376636596667, "grad_norm": 0.14064420759677887, "lr": 9.555519936590738e-05, "epoch": 1, "t": 4.1670823097229}
+{"step": 54, "loss": 0.10726402326110894, "grad_norm": 0.12126035988330841, "lr": 9.532194951537838e-05, "epoch": 1, "t": 4.2917704582214355}
+{"step": 55, "loss": 0.13402982711183978, "grad_norm": 0.16427747905254364, "lr": 9.508303477897924e-05, "epoch": 1, "t": 3.693385124206543}
+{"step": 56, "loss": 0.11277710025861432, "grad_norm": 0.21456760168075562, "lr": 9.483848501916578e-05, "epoch": 1, "t": 3.2162392139434814}
+{"step": 57, "loss": 0.09820183289992059, "grad_norm": 0.18677082657814026, "lr": 9.458833080272722e-05, "epoch": 1, "t": 3.2174220085144043}
+{"step": 58, "loss": 0.0888994905341741, "grad_norm": 0.14535124599933624, "lr": 9.433260339696563e-05, "epoch": 1, "t": 3.962379217147827}
+{"step": 59, "loss": 0.13740586502302582, "grad_norm": 0.17558394372463226, "lr": 9.407133476578778e-05, "epoch": 1, "t": 4.730549573898315}
+{"step": 60, "loss": 0.12455208271865528, "grad_norm": 0.14261916279792786, "lr": 9.38045575657098e-05, "epoch": 1, "t": 4.914440870285034}
+{"step": 61, "loss": 0.11913379427388204, "grad_norm": 0.1595505177974701, "lr": 9.353230514177552e-05, "epoch": 1, "t": 4.58712100982666}
+{"step": 62, "loss": 0.1598939223347861, "grad_norm": 0.16470298171043396, "lr": 9.325461152338846e-05, "epoch": 1, "t": 4.833375930786133}
+{"step": 63, "loss": 0.10565623049247465, "grad_norm": 0.14636284112930298, "lr": 9.297151142005851e-05, "epoch": 1, "t": 4.798422336578369}
+{"step": 64, "loss": 0.12876437884645323, "grad_norm": 0.15872706472873688, "lr": 9.268304021706349e-05, "epoch": 1, "t": 4.652063608169556}
+{"step": 65, "loss": 0.12992243309997886, "grad_norm": 0.1705460548400879, "lr": 9.23892339710263e-05, "epoch": 1, "t": 5.494608640670776}
+{"step": 66, "loss": 0.12887709248450493, "grad_norm": 0.1533844918012619, "lr": 9.209012940540805e-05, "epoch": 1, "t": 4.818042993545532}
+{"step": 67, "loss": 0.09908132047069315, "grad_norm": 0.16532683372497559, "lr": 9.178576390591802e-05, "epoch": 1, "t": 4.478140592575073}
+{"step": 68, "loss": 0.1082844658700641, "grad_norm": 0.16935817897319794, "lr": 9.147617551584066e-05, "epoch": 1, "t": 4.695854663848877}
+{"step": 69, "loss": 0.09549259292030872, "grad_norm": 0.14777007699012756, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.5415735244750977}
+{"step": 70, "loss": 0.14593881517626658, "grad_norm": 0.16458331048488617, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.7696597576141357}
+{"step": 71, "loss": 0.15460061061684927, "grad_norm": 0.1709330976009369, "lr": 9.051646319812918e-05, "epoch": 1, "t": 4.229056358337402}
+{"step": 72, "loss": 0.09152270898436607, "grad_norm": 0.14009197056293488, "lr": 9.018637666191283e-05, "epoch": 1, "t": 4.38850998878479}
+{"step": 73, "loss": 0.12666737965595098, "grad_norm": 0.15965832769870758, "lr": 8.985126714588738e-05, "epoch": 1, "t": 3.629696846008301}
+{"step": 74, "loss": 0.09974380793505336, "grad_norm": 0.1462961733341217, "lr": 8.951117653609666e-05, "epoch": 1, "t": 4.481092214584351}
+{"step": 75, "loss": 0.11864136477154336, "grad_norm": 0.14830192923545837, "lr": 8.916614734118184e-05, "epoch": 1, "t": 5.081501722335815}
+{"step": 76, "loss": 0.14721758977480684, "grad_norm": 0.13556744158267975, "lr": 8.881622268706825e-05, "epoch": 1, "t": 5.222254276275635}
+{"step": 77, "loss": 0.10595594941308704, "grad_norm": 0.1493482142686844, "lr": 8.8461446311575e-05, "epoch": 1, "t": 5.006181716918945}
+{"step": 78, "loss": 0.1284816624444724, "grad_norm": 0.15887455642223358, "lr": 8.810186255894803e-05, "epoch": 1, "t": 5.144675254821777}
+{"step": 79, "loss": 0.10622403047380655, "grad_norm": 0.1342378407716751, "lr": 8.773751637431748e-05, "epoch": 1, "t": 5.2127625942230225}
+{"step": 80, "loss": 0.11771947886193393, "grad_norm": 0.14702412486076355, "lr": 8.736845329807993e-05, "epoch": 1, "t": 5.077196359634399}
+{"step": 81, "loss": 0.08715652897214649, "grad_norm": 0.14258985221385956, "lr": 8.69947194602061e-05, "epoch": 1, "t": 5.166703939437866}
+{"step": 82, "loss": 0.1407249934787319, "grad_norm": 0.1574077159166336, "lr": 8.66163615744751e-05, "epoch": 1, "t": 4.982941389083862}
+{"step": 83, "loss": 0.10946292864949082, "grad_norm": 0.1504049152135849, "lr": 8.623342693263548e-05, "epoch": 1, "t": 4.9891884326934814}
+{"step": 84, "loss": 0.11453432910343306, "grad_norm": 0.14204095304012299, "lr": 8.584596339849417e-05, "epoch": 1, "t": 4.742063760757446}
+{"step": 85, "loss": 0.1411642906505004, "grad_norm": 0.14732734858989716, "lr": 8.545401940193392e-05, "epoch": 1, "t": 4.421473026275635}
+{"step": 86, "loss": 0.12692128487083257, "grad_norm": 0.1760338395833969, "lr": 8.505764393285984e-05, "epoch": 1, "t": 3.8381481170654297}
+{"step": 87, "loss": 0.1448906666822031, "grad_norm": 0.14406467974185944, "lr": 8.46568865350762e-05, "epoch": 1, "t": 4.42296838760376}
+{"step": 88, "loss": 0.09271264168849525, "grad_norm": 0.13818959891796112, "lr": 8.425179730009368e-05, "epoch": 1, "t": 4.221201181411743}
+{"step": 89, "loss": 0.09044018554625603, "grad_norm": 0.1389867067337036, "lr": 8.384242686086848e-05, "epoch": 1, "t": 4.587378978729248}
+{"step": 90, "loss": 0.09728664608288758, "grad_norm": 0.14242956042289734, "lr": 8.342882638547351e-05, "epoch": 1, "t": 4.411623239517212}
+{"step": 91, "loss": 0.1044411832391533, "grad_norm": 0.16613753139972687, "lr": 8.301104757070274e-05, "epoch": 1, "t": 3.752352714538574}
+{"step": 92, "loss": 0.12271639306552244, "grad_norm": 0.1773744821548462, "lr": 8.258914263560971e-05, "epoch": 1, "t": 3.892817258834839}
+{"step": 93, "loss": 0.1380967544534043, "grad_norm": 0.15400643646717072, "lr": 8.216316431498028e-05, "epoch": 1, "t": 4.50601053237915}
+{"step": 94, "loss": 0.1510967011114474, "grad_norm": 0.13552269339561462, "lr": 8.173316585274145e-05, "epoch": 1, "t": 4.505499601364136}
+{"step": 95, "loss": 0.09745585613861579, "grad_norm": 0.15307575464248657, "lr": 8.129920099530607e-05, "epoch": 1, "t": 3.8790650367736816}
+{"step": 96, "loss": 0.12510263738168714, "grad_norm": 0.1437438428401947, "lr": 8.086132398485524e-05, "epoch": 1, "t": 4.18375563621521}
+{"step": 97, "loss": 0.11138657050017126, "grad_norm": 0.137197345495224, "lr": 8.041958955255814e-05, "epoch": 1, "t": 4.412342071533203}
+{"step": 98, "loss": 0.1041658407786736, "grad_norm": 0.1670880913734436, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.70011043548584}
+{"step": 99, "loss": 0.1297504555360572, "grad_norm": 0.152018204331398, "lr": 7.952476975093729e-05, "epoch": 1, "t": 4.8253562450408936}
+{"step": 100, "loss": 0.15818131143754804, "grad_norm": 0.17151501774787903, "lr": 7.907179622702408e-05, "epoch": 1, "t": 5.031487941741943}
+{"step": 101, "loss": 0.1375515077137357, "grad_norm": 0.14399169385433197, "lr": 7.861518895810596e-05, "epoch": 1, "t": 3.946390151977539}
+{"step": 102, "loss": 0.11110222823816197, "grad_norm": 0.15133707225322723, "lr": 7.815500501648653e-05, "epoch": 1, "t": 3.9376065731048584}
+{"step": 103, "loss": 0.1274873402001611, "grad_norm": 0.1693481057882309, "lr": 7.769130192152538e-05, "epoch": 1, "t": 4.112709283828735}
+{"step": 104, "loss": 0.08292039131361342, "grad_norm": 0.15406905114650726, "lr": 7.722413763244838e-05, "epoch": 1, "t": 3.7390711307525635}
+{"step": 105, "loss": 0.11526951412018738, "grad_norm": 0.18622912466526031, "lr": 7.675357054110336e-05, "epoch": 1, "t": 3.727570056915283}
+{"step": 106, "loss": 0.12006499917102575, "grad_norm": 0.17982099950313568, "lr": 7.627965946466166e-05, "epoch": 1, "t": 3.5842368602752686}
+{"step": 107, "loss": 0.1186935144114395, "grad_norm": 0.14877408742904663, "lr": 7.580246363826621e-05, "epoch": 1, "t": 4.091334581375122}
+{"step": 108, "loss": 0.13122232868610226, "grad_norm": 0.1457301527261734, "lr": 7.532204270762786e-05, "epoch": 1, "t": 3.881474256515503}
+{"step": 109, "loss": 0.1334371601357872, "grad_norm": 0.14677388966083527, "lr": 7.483845672156998e-05, "epoch": 1, "t": 3.996276617050171}
+{"step": 110, "loss": 0.13046903133467533, "grad_norm": 0.16596752405166626, "lr": 7.435176612452286e-05, "epoch": 1, "t": 4.019444227218628}
+{"step": 111, "loss": 0.10941051667897489, "grad_norm": 0.17496438324451447, "lr": 7.386203174896872e-05, "epoch": 1, "t": 3.7645998001098633}
+{"step": 112, "loss": 0.1244959343812367, "grad_norm": 0.16166579723358154, "lr": 7.336931480783801e-05, "epoch": 1, "t": 4.514106750488281}
+{"step": 113, "loss": 0.11262510282407952, "grad_norm": 0.1646949052810669, "lr": 7.287367688685835e-05, "epoch": 1, "t": 4.528945446014404}
+{"step": 114, "loss": 0.10489004908381282, "grad_norm": 0.18302619457244873, "lr": 7.237517993685678e-05, "epoch": 1, "t": 4.559117317199707}
+{"step": 115, "loss": 0.10232535843065521, "grad_norm": 0.19442802667617798, "lr": 7.187388626601637e-05, "epoch": 1, "t": 4.564206600189209}
+{"step": 116, "loss": 0.11103497537398845, "grad_norm": 0.16678892076015472, "lr": 7.136985853208824e-05, "epoch": 1, "t": 5.07295560836792}
+{"step": 117, "loss": 0.1115818610558143, "grad_norm": 0.16402477025985718, "lr": 7.086315973455981e-05, "epoch": 1, "t": 5.139282703399658}
+{"step": 118, "loss": 0.12158853422727228, "grad_norm": 0.1311653107404709, "lr": 7.035385320678036e-05, "epoch": 1, "t": 5.401008367538452}
+{"step": 119, "loss": 0.09588072977468419, "grad_norm": 0.14086735248565674, "lr": 6.984200260804484e-05, "epoch": 1, "t": 4.952974557876587}
+{"step": 120, "loss": 0.0912908763036104, "grad_norm": 0.16218799352645874, "lr": 6.932767191563703e-05, "epoch": 1, "t": 4.948731184005737}
+{"step": 121, "loss": 0.13822233603085715, "grad_norm": 0.14316639304161072, "lr": 6.881092541683278e-05, "epoch": 1, "t": 5.22462797164917}
+{"step": 122, "loss": 0.11881857142897552, "grad_norm": 0.1303873509168625, "lr": 6.829182770086474e-05, "epoch": 1, "t": 4.654219388961792}
+{"step": 123, "loss": 0.12108342410574989, "grad_norm": 0.1579148918390274, "lr": 6.777044365084907e-05, "epoch": 1, "t": 3.9120402336120605}
+{"step": 124, "loss": 0.07968569486767943, "grad_norm": 0.1335701048374176, "lr": 6.724683843567568e-05, "epoch": 1, "t": 3.8937900066375732}
+{"step": 125, "loss": 0.11150920626381651, "grad_norm": 0.1818261593580246, "lr": 6.672107750186255e-05, "epoch": 1, "t": 3.5147409439086914}
+{"step": 126, "loss": 0.14486833444633554, "grad_norm": 0.18314026296138763, "lr": 6.619322656537552e-05, "epoch": 1, "t": 3.928903579711914}
+{"step": 127, "loss": 0.11722182175534124, "grad_norm": 0.13104470074176788, "lr": 6.566335160341424e-05, "epoch": 1, "t": 4.0404181480407715}
+{"step": 128, "loss": 0.1222243271308053, "grad_norm": 0.12511242926120758, "lr": 6.513151884616556e-05, "epoch": 1, "t": 4.540865898132324}
+{"step": 129, "loss": 0.1125568767840212, "grad_norm": 0.12010899931192398, "lr": 6.459779476852528e-05, "epoch": 1, "t": 4.350601434707642}
+{"step": 130, "loss": 0.12215659848056286, "grad_norm": 0.13207314908504486, "lr": 6.406224608178932e-05, "epoch": 1, "t": 4.12429141998291}
+{"step": 131, "loss": 0.11350722618544884, "grad_norm": 0.1452920138835907, "lr": 6.352493972531534e-05, "epoch": 1, "t": 4.59038233757019}
+{"step": 132, "loss": 0.1238534170162562, "grad_norm": 0.13540174067020416, "lr": 6.298594285815584e-05, "epoch": 1, "t": 5.946590423583984}
+{"step": 133, "loss": 0.10247356409141656, "grad_norm": 0.1491309404373169, "lr": 6.244532285066382e-05, "epoch": 1, "t": 5.886356592178345}
+{"step": 134, "loss": 0.10514463958096298, "grad_norm": 0.16483132541179657, "lr": 6.190314727607196e-05, "epoch": 1, "t": 5.842749357223511}
+{"step": 135, "loss": 0.11137263294971475, "grad_norm": 0.16692940890789032, "lr": 6.13594839020466e-05, "epoch": 1, "t": 5.612648963928223}
+{"step": 136, "loss": 0.12202524957578044, "grad_norm": 0.1854080706834793, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 5.886415243148804}
+{"step": 137, "loss": 0.12197275685911982, "grad_norm": 0.16223259270191193, "lr": 6.026796574768288e-05, "epoch": 1, "t": 5.867288589477539}
+{"step": 138, "loss": 0.0932202906921203, "grad_norm": 0.15473394095897675, "lr": 5.972024739849622e-05, "epoch": 1, "t": 5.83188009262085}
+{"step": 139, "loss": 0.12311797054731247, "grad_norm": 0.13994963467121124, "lr": 5.917131409512663e-05, "epoch": 1, "t": 5.983316898345947}
+{"step": 140, "loss": 0.13175687963900853, "grad_norm": 0.13232986629009247, "lr": 5.862123444990318e-05, "epoch": 1, "t": 5.916776895523071}
+{"step": 141, "loss": 0.10179921986531704, "grad_norm": 0.14108580350875854, "lr": 5.807007721843861e-05, "epoch": 1, "t": 5.725611925125122}
+{"step": 142, "loss": 0.12247693727065953, "grad_norm": 0.14080466330051422, "lr": 5.751791129103544e-05, "epoch": 1, "t": 5.3483970165252686}
+{"step": 143, "loss": 0.13598552810749115, "grad_norm": 0.11519870162010193, "lr": 5.696480568407523e-05, "epoch": 1, "t": 6.234901666641235}
+{"step": 144, "loss": 0.09938006266421522, "grad_norm": 0.14629501104354858, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 5.59669828414917}
+{"step": 145, "loss": 0.13709959031913002, "grad_norm": 0.1319306343793869, "lr": 5.585605207563124e-05, "epoch": 1, "t": 5.53915548324585}
+{"step": 146, "loss": 0.11176312425211334, "grad_norm": 0.15270474553108215, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 5.7169177532196045}
+{"step": 147, "loss": 0.14579656180615397, "grad_norm": 0.1367747187614441, "lr": 5.47443707175741e-05, "epoch": 1, "t": 5.637811899185181}
+{"step": 148, "loss": 0.12222280642755524, "grad_norm": 0.14530056715011597, "lr": 5.418760576667071e-05, "epoch": 1, "t": 5.36273717880249}
+{"step": 149, "loss": 0.10807040342489989, "grad_norm": 0.14311166107654572, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 5.1091697216033936}
+{"step": 150, "loss": 0.10914952956936362, "grad_norm": 0.11722470074892044, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 5.868841886520386}
+{"step": 151, "loss": 0.1338075859704394, "grad_norm": 0.15515875816345215, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 5.8782737255096436}
+{"step": 152, "loss": 0.11667911126268332, "grad_norm": 0.13852252066135406, "lr": 5.195600862779421e-05, "epoch": 1, "t": 6.260565280914307}
+{"step": 153, "loss": 0.12250399477943973, "grad_norm": 0.11874734610319138, "lr": 5.139732367864736e-05, "epoch": 1, "t": 5.6688232421875}
+{"step": 154, "loss": 0.12234871340049323, "grad_norm": 0.14222052693367004, "lr": 5.083846407507263e-05, "epoch": 1, "t": 5.017601728439331}
+{"step": 155, "loss": 0.1313176102610344, "grad_norm": 0.14096294343471527, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 5.398028612136841}
+{"step": 156, "loss": 0.10998567759641105, "grad_norm": 0.12560489773750305, "lr": 4.972050032989175e-05, "epoch": 1, "t": 5.881792306900024}
+{"step": 157, "loss": 0.14253723142893687, "grad_norm": 0.12853525578975677, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 6.006667375564575}
+{"step": 158, "loss": 0.1362565209515793, "grad_norm": 0.18033531308174133, "lr": 4.860267632135265e-05, "epoch": 1, "t": 5.782114505767822}
+{"step": 159, "loss": 0.12438243804136975, "grad_norm": 0.12671931087970734, "lr": 4.80439913722058e-05, "epoch": 1, "t": 6.095634698867798}
+{"step": 160, "loss": 0.11695628953029918, "grad_norm": 0.13746081292629242, "lr": 4.748555090869464e-05, "epoch": 1, "t": 6.044499635696411}
+{"step": 161, "loss": 0.10860840798888029, "grad_norm": 0.12572933733463287, "lr": 4.692742473146818e-05, "epoch": 1, "t": 6.383399248123169}
+{"step": 162, "loss": 0.15660127597203813, "grad_norm": 0.16079489886760712, "lr": 4.636968260189214e-05, "epoch": 1, "t": 5.824679374694824}
+{"step": 163, "loss": 0.10925060701854006, "grad_norm": 0.12640535831451416, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 5.920647144317627}
+{"step": 164, "loss": 0.13847018529098065, "grad_norm": 0.13318540155887604, "lr": 4.525562928242592e-05, "epoch": 1, "t": 6.111425399780273}
+{"step": 165, "loss": 0.11645202288874017, "grad_norm": 0.14153780043125153, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 5.860616683959961}
+{"step": 166, "loss": 0.10572560891730859, "grad_norm": 0.14335797727108002, "lr": 4.414394792436877e-05, "epoch": 1, "t": 5.821516036987305}
+{"step": 167, "loss": 0.12092660270690916, "grad_norm": 0.13772067427635193, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 5.985003709793091}
+{"step": 168, "loss": 0.11941248052510059, "grad_norm": 0.1498883068561554, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 5.9461143016815186}
+{"step": 169, "loss": 0.09770603653573595, "grad_norm": 0.1489756852388382, "lr": 4.248208870896456e-05, "epoch": 1, "t": 5.789595365524292}
+{"step": 170, "loss": 0.12906971163680264, "grad_norm": 0.12248232960700989, "lr": 4.192992278156141e-05, "epoch": 1, "t": 5.774706125259399}
+{"step": 171, "loss": 0.10495811824118934, "grad_norm": 0.12584099173545837, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 4.037522315979004}
+{"step": 172, "loss": 0.15494878209609153, "grad_norm": 0.146627277135849, "lr": 4.082868590487339e-05, "epoch": 1, "t": 4.370777130126953}
+{"step": 173, "loss": 0.13606241378445846, "grad_norm": 0.13438133895397186, "lr": 4.027975260150381e-05, "epoch": 1, "t": 4.2283103466033936}
+{"step": 174, "loss": 0.11738633436514131, "grad_norm": 0.1292438954114914, "lr": 3.973203425231715e-05, "epoch": 1, "t": 4.1627538204193115}
+{"step": 175, "loss": 0.09793814501262667, "grad_norm": 0.14427684247493744, "lr": 3.918559931778277e-05, "epoch": 1, "t": 3.76477313041687}
+{"step": 176, "loss": 0.1134855055438422, "grad_norm": 0.13561420142650604, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 4.130195379257202}
+{"step": 177, "loss": 0.10737729666508591, "grad_norm": 0.13862013816833496, "lr": 3.809685272392804e-05, "epoch": 1, "t": 4.025635480880737}
+{"step": 178, "loss": 0.14615545516486445, "grad_norm": 0.15243802964687347, "lr": 3.755467714933619e-05, "epoch": 1, "t": 4.069834470748901}
+{"step": 179, "loss": 0.13059986952499916, "grad_norm": 0.15739114582538605, "lr": 3.701405714184416e-05, "epoch": 1, "t": 4.0566606521606445}
+{"step": 180, "loss": 0.1250554776811917, "grad_norm": 0.1449676752090454, "lr": 3.647506027468467e-05, "epoch": 1, "t": 3.888803482055664}
+{"step": 181, "loss": 0.1339609112250817, "grad_norm": 0.11792629957199097, "lr": 3.59377539182107e-05, "epoch": 1, "t": 5.0446813106536865}
+{"step": 182, "loss": 0.0964094779671148, "grad_norm": 0.16920435428619385, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 3.5308339595794678}
+{"step": 183, "loss": 0.0841054949191732, "grad_norm": 0.158295139670372, "lr": 3.486848115383445e-05, "epoch": 1, "t": 3.699392557144165}
+{"step": 184, "loss": 0.12007146771255114, "grad_norm": 0.1564946174621582, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 3.737936496734619}
+{"step": 185, "loss": 0.07932232825931569, "grad_norm": 0.12786969542503357, "lr": 3.380677343462447e-05, "epoch": 1, "t": 3.772343635559082}
+{"step": 186, "loss": 0.11378491893440214, "grad_norm": 0.16465850174427032, "lr": 3.327892249813745e-05, "epoch": 1, "t": 3.755070447921753}
+{"step": 187, "loss": 0.0909288471611379, "grad_norm": 0.13516880571842194, "lr": 3.275316156432434e-05, "epoch": 1, "t": 3.848257541656494}
+{"step": 188, "loss": 0.14310075986846685, "grad_norm": 0.1411554217338562, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 4.250707626342773}
+{"step": 189, "loss": 0.11388757989347517, "grad_norm": 0.12976497411727905, "lr": 3.170817229913526e-05, "epoch": 1, "t": 4.144380807876587}
+{"step": 190, "loss": 0.1265786284890618, "grad_norm": 0.1473735123872757, "lr": 3.118907458316722e-05, "epoch": 1, "t": 4.082712888717651}
+{"step": 191, "loss": 0.13427961269346605, "grad_norm": 0.15153977274894714, "lr": 3.067232808436299e-05, "epoch": 1, "t": 4.0815465450286865}
+{"step": 192, "loss": 0.10992266317146325, "grad_norm": 0.15019188821315765, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 4.350877046585083}
+{"step": 193, "loss": 0.1572723716825705, "grad_norm": 0.12622733414173126, "lr": 2.964614679321966e-05, "epoch": 1, "t": 4.8909242153167725}
+{"step": 194, "loss": 0.11537074267244213, "grad_norm": 0.15508973598480225, "lr": 2.913684026544021e-05, "epoch": 1, "t": 4.080131530761719}
+{"step": 195, "loss": 0.12344161256182586, "grad_norm": 0.16149987280368805, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 3.9379467964172363}
+{"step": 196, "loss": 0.13281372677716102, "grad_norm": 0.15665747225284576, "lr": 2.812611373398365e-05, "epoch": 1, "t": 3.7952842712402344}
+{"step": 197, "loss": 0.16175826833642656, "grad_norm": 0.1399657279253006, "lr": 2.762482006314324e-05, "epoch": 1, "t": 4.240842580795288}
+{"step": 198, "loss": 0.12268284408389486, "grad_norm": 0.15245409309864044, "lr": 2.712632311314165e-05, "epoch": 1, "t": 4.2339842319488525}
+{"step": 199, "loss": 0.10661711567550075, "grad_norm": 0.1496862769126892, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 3.873534679412842}
+{"step": 200, "loss": 0.10718047681134313, "grad_norm": 0.16035036742687225, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 3.512685775756836}
+{"step": 201, "loss": 0.09733861004859186, "grad_norm": 0.1342783272266388, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 3.726299285888672}
+{"step": 202, "loss": 0.10710803163853956, "grad_norm": 0.14960817992687225, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 3.5803110599517822}
+{"step": 203, "loss": 0.12856140146460426, "grad_norm": 0.14114242792129517, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 3.8162992000579834}
+{"step": 204, "loss": 0.09950678060929327, "grad_norm": 0.13951867818832397, "lr": 2.419753636173379e-05, "epoch": 1, "t": 3.4555318355560303}
+{"step": 205, "loss": 0.1192101331395427, "grad_norm": 0.15045635402202606, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 3.754653215408325}
+{"step": 206, "loss": 0.10550485139810536, "grad_norm": 0.1489887237548828, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 3.4955015182495117}
+{"step": 207, "loss": 0.1457673885732749, "grad_norm": 0.1404895782470703, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 4.137854814529419}
+{"step": 208, "loss": 0.09591870073622033, "grad_norm": 0.12496516108512878, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 3.657066583633423}
+{"step": 209, "loss": 0.11847254509543413, "grad_norm": 0.1287345588207245, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 3.7544782161712646}
+{"step": 210, "loss": 0.0979276126730907, "grad_norm": 0.14218559861183167, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 3.231539726257324}
+{"step": 211, "loss": 0.09428850225518377, "grad_norm": 0.1390930414199829, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 3.346951484680176}
+{"step": 212, "loss": 0.12567077994677361, "grad_norm": 0.17330411076545715, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 3.1886229515075684}
+{"step": 213, "loss": 0.09782922963854526, "grad_norm": 0.13580435514450073, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 3.3848838806152344}
+{"step": 214, "loss": 0.13602645966365862, "grad_norm": 0.17799289524555206, "lr": 1.958041044744186e-05, "epoch": 1, "t": 3.171832799911499}
+{"step": 215, "loss": 0.10618019406428078, "grad_norm": 0.1461976170539856, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 3.309781551361084}
+{"step": 216, "loss": 0.09441785260250693, "grad_norm": 0.13548174500465393, "lr": 1.870079900469392e-05, "epoch": 1, "t": 3.8321173191070557}
+{"step": 217, "loss": 0.09281340758928955, "grad_norm": 0.14920149743556976, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 3.2834720611572266}
+{"step": 218, "loss": 0.13122085211118584, "grad_norm": 0.17590315639972687, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 3.6829259395599365}
+{"step": 219, "loss": 0.14349004887697991, "grad_norm": 0.13497987389564514, "lr": 1.741085736439031e-05, "epoch": 1, "t": 4.1920435428619385}
+{"step": 220, "loss": 0.12411245391902115, "grad_norm": 0.1270938217639923, "lr": 1.698895242929725e-05, "epoch": 1, "t": 4.421627044677734}
+{"step": 221, "loss": 0.10102598842648428, "grad_norm": 0.15080025792121887, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 3.919564962387085}
+{"step": 222, "loss": 0.1344824544603108, "grad_norm": 0.13348388671875, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 4.353817939758301}
+{"step": 223, "loss": 0.11996158967234363, "grad_norm": 0.1332148164510727, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 4.168254852294922}
+{"step": 224, "loss": 0.12410005547497649, "grad_norm": 0.12707005441188812, "lr": 1.534311346492381e-05, "epoch": 1, "t": 5.450979232788086}
+{"step": 225, "loss": 0.10171063207279571, "grad_norm": 0.17476694285869598, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 4.6596598625183105}
+{"step": 226, "loss": 0.08603258519586866, "grad_norm": 0.10637104511260986, "lr": 1.454598059806609e-05, "epoch": 1, "t": 6.157179594039917}
+{"step": 227, "loss": 0.14213648385105823, "grad_norm": 0.1354423314332962, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 5.06745457649231}
+{"step": 228, "loss": 0.12114619999198262, "grad_norm": 0.12042339146137238, "lr": 1.376657306736453e-05, "epoch": 1, "t": 5.55302882194519}
+{"step": 229, "loss": 0.14443165937833533, "grad_norm": 0.11341030895709991, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 6.407577276229858}
+{"step": 230, "loss": 0.12206946873519237, "grad_norm": 0.13370126485824585, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 5.899457931518555}
+{"step": 231, "loss": 0.14863756474731973, "grad_norm": 0.14756746590137482, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 5.333111047744751}
+{"step": 232, "loss": 0.13004562389244703, "grad_norm": 0.1426292061805725, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 5.817943572998047}
+{"step": 233, "loss": 0.11405566817683196, "grad_norm": 0.1313171237707138, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 5.368566036224365}
+{"step": 234, "loss": 0.09286493925544305, "grad_norm": 0.12980012595653534, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 6.268900394439697}
+{"step": 235, "loss": 0.09465315181374723, "grad_norm": 0.1298038810491562, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 5.635498762130737}
+{"step": 236, "loss": 0.10800215024922183, "grad_norm": 0.14482581615447998, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 4.18051290512085}
+{"step": 237, "loss": 0.08360361414277363, "grad_norm": 0.14186900854110718, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 3.251851797103882}
+{"step": 238, "loss": 0.1254756912914426, "grad_norm": 0.1392965465784073, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 3.7237584590911865}
+{"step": 239, "loss": 0.14220688001573326, "grad_norm": 0.1264325976371765, "lr": 9.81362333808718e-06, "epoch": 1, "t": 4.819896221160889}
+{"step": 240, "loss": 0.12136467044813183, "grad_norm": 0.1665497124195099, "lr": 9.483536801870834e-06, "epoch": 1, "t": 3.3412585258483887}
+{"step": 241, "loss": 0.12490963203455108, "grad_norm": 0.15111815929412842, "lr": 9.158514503674543e-06, "epoch": 1, "t": 3.4438259601593018}
+{"step": 242, "loss": 0.12017831168417333, "grad_norm": 0.1290101557970047, "lr": 8.838597068719518e-06, "epoch": 1, "t": 4.016022205352783}
+{"step": 243, "loss": 0.09489677106351789, "grad_norm": 0.11944586038589478, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.92342472076416}
+{"step": 244, "loss": 0.1243680059571503, "grad_norm": 0.1327151656150818, "lr": 8.21423609408199e-06, "epoch": 1, "t": 3.734070062637329}
+{"step": 245, "loss": 0.11684684307067719, "grad_norm": 0.15995414555072784, "lr": 7.90987059459195e-06, "epoch": 1, "t": 3.4899020195007324}
+{"step": 246, "loss": 0.12360585496278272, "grad_norm": 0.1631646752357483, "lr": 7.610766028973709e-06, "epoch": 1, "t": 4.229679107666016}
+{"step": 247, "loss": 0.13372521546217367, "grad_norm": 0.15672412514686584, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 4.729381084442139}
+{"step": 248, "loss": 0.11260705068762944, "grad_norm": 0.13472774624824524, "lr": 7.028488579941506e-06, "epoch": 1, "t": 4.950815916061401}
+{"step": 249, "loss": 0.123039937727079, "grad_norm": 0.11839025467634201, "lr": 6.745388476611553e-06, "epoch": 1, "t": 5.320116281509399}
+{"step": 250, "loss": 0.11276406303449972, "grad_norm": 0.1768644005060196, "lr": 6.467694858224488e-06, "epoch": 1, "t": 4.611212730407715}
+{"step": 251, "loss": 0.14011463678387218, "grad_norm": 0.13780367374420166, "lr": 6.1954424342902e-06, "epoch": 1, "t": 4.852539777755737}
+{"step": 252, "loss": 0.09068210728985011, "grad_norm": 0.13530129194259644, "lr": 5.928665234212233e-06, "epoch": 1, "t": 4.842432498931885}
+{"step": 253, "loss": 0.10120193497090002, "grad_norm": 0.11738213896751404, "lr": 5.66739660303437e-06, "epoch": 1, "t": 5.2902562618255615}
+{"step": 254, "loss": 0.09851244988383807, "grad_norm": 0.1303517073392868, "lr": 5.411669197272795e-06, "epoch": 1, "t": 5.693821430206299}
+{"step": 255, "loss": 0.10302092982955194, "grad_norm": 0.1410181075334549, "lr": 5.161514980834231e-06, "epoch": 1, "t": 5.209938049316406}
+{"step": 256, "loss": 0.13269496118584523, "grad_norm": 0.15145458281040192, "lr": 4.916965221020753e-06, "epoch": 1, "t": 5.3764777183532715}
+{"step": 257, "loss": 0.10233825561732993, "grad_norm": 0.13183993101119995, "lr": 4.678050484621615e-06, "epoch": 1, "t": 5.46918249130249}
+{"step": 258, "loss": 0.13216298849471492, "grad_norm": 0.13832338154315948, "lr": 4.444800634092616e-06, "epoch": 1, "t": 4.821920871734619}
+{"step": 259, "loss": 0.12163336211593835, "grad_norm": 0.15184752643108368, "lr": 4.217244823823546e-06, "epoch": 1, "t": 3.7082533836364746}
+{"step": 260, "loss": 0.15002393293209007, "grad_norm": 0.15476936101913452, "lr": 3.995411496494134e-06, "epoch": 1, "t": 3.559565782546997}
+{"step": 261, "loss": 0.09959995404541193, "grad_norm": 0.12526850402355194, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 3.98399019241333}
+{"step": 262, "loss": 0.11795829714304533, "grad_norm": 0.16237178444862366, "lr": 3.56902248158148e-06, "epoch": 1, "t": 3.2178425788879395}
+{"step": 263, "loss": 0.14876536610499433, "grad_norm": 0.15125760436058044, "lr": 3.364520089258727e-06, "epoch": 1, "t": 3.726151704788208}
+{"step": 264, "loss": 0.1307659820311301, "grad_norm": 0.13269969820976257, "lr": 3.165846763735153e-06, "epoch": 1, "t": 4.523290634155273}
+{"step": 265, "loss": 0.09039410618621412, "grad_norm": 0.15942604839801788, "lr": 2.973027337607892e-06, "epoch": 1, "t": 3.1473255157470703}
+{"step": 266, "loss": 0.1279116300673159, "grad_norm": 0.16423822939395905, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.282160520553589}
+{"step": 267, "loss": 0.08462305770535225, "grad_norm": 0.14013220369815826, "lr": 2.605045852462473e-06, "epoch": 1, "t": 3.2357003688812256}
+{"step": 268, "loss": 0.09663602861307435, "grad_norm": 0.17577238380908966, "lr": 2.429929788224722e-06, "epoch": 1, "t": 3.242727756500244}
+{"step": 269, "loss": 0.14687636517096836, "grad_norm": 0.16681092977523804, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 4.1681108474731445}
+{"step": 270, "loss": 0.11895260757699186, "grad_norm": 0.12868773937225342, "lr": 2.097556454310701e-06, "epoch": 1, "t": 4.703555583953857}
+{"step": 271, "loss": 0.10439320327601696, "grad_norm": 0.1587361991405487, "lr": 1.940340728677059e-06, "epoch": 1, "t": 4.763911247253418}
+{"step": 272, "loss": 0.11546959674368708, "grad_norm": 0.15578536689281464, "lr": 1.789132081018674e-06, "epoch": 1, "t": 4.900822877883911}
+{"step": 273, "loss": 0.12891419631590156, "grad_norm": 0.15120558440685272, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 4.793020725250244}
+{"step": 274, "loss": 0.15081281376979955, "grad_norm": 0.12745462357997894, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 5.244046688079834}
+{"step": 275, "loss": 0.10252382758882515, "grad_norm": 0.12532322108745575, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 5.692639589309692}
+{"step": 276, "loss": 0.1052388965172062, "grad_norm": 0.17532673478126526, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 4.433902025222778}
+{"step": 277, "loss": 0.12041700529015582, "grad_norm": 0.14770610630512238, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 4.664698839187622}
+{"step": 278, "loss": 0.0990669545895036, "grad_norm": 0.13431894779205322, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 4.974935054779053}
+{"step": 279, "loss": 0.10497822829783265, "grad_norm": 0.12812137603759766, "lr": 9.003621402309814e-07, "epoch": 1, "t": 4.980578899383545}
+{"step": 280, "loss": 0.08761405289810818, "grad_norm": 0.1214241236448288, "lr": 7.97826969669102e-07, "epoch": 1, "t": 5.631446123123169}
+{"step": 281, "loss": 0.12079584596448073, "grad_norm": 0.13822419941425323, "lr": 7.014416822821556e-07, "epoch": 1, "t": 5.562159299850464}
+{"step": 282, "loss": 0.11831246908856181, "grad_norm": 0.15493813157081604, "lr": 6.112183254700865e-07, "epoch": 1, "t": 5.366363763809204}
+{"step": 283, "loss": 0.0857640213671429, "grad_norm": 0.1580144762992859, "lr": 5.271681764401848e-07, "epoch": 1, "t": 5.280276775360107}
+{"step": 284, "loss": 0.1355166469985603, "grad_norm": 0.14353851974010468, "lr": 4.493017407975086e-07, "epoch": 1, "t": 5.418309450149536}
+{"step": 285, "loss": 0.1259592493205691, "grad_norm": 0.15438403189182281, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 5.79780912399292}
+{"step": 286, "loss": 0.11926719573508451, "grad_norm": 0.12321104109287262, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 5.482792854309082}
+{"step": 287, "loss": 0.11562612069711753, "grad_norm": 0.14055198431015015, "lr": 2.528981693106558e-07, "epoch": 1, "t": 5.365353107452393}
+{"step": 288, "loss": 0.12881073325033107, "grad_norm": 0.13906581699848175, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 5.620147943496704}
+{"step": 289, "loss": 0.13206261805223798, "grad_norm": 0.14486300945281982, "lr": 1.530387900802177e-07, "epoch": 1, "t": 4.856403112411499}
+{"step": 290, "loss": 0.10525038197031221, "grad_norm": 0.1403212547302246, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 4.65713906288147}
+{"step": 291, "loss": 0.14182153682170592, "grad_norm": 0.1460697203874588, "lr": 7.81005385163458e-08, "epoch": 1, "t": 4.921620845794678}
+{"step": 292, "loss": 0.1270982598292454, "grad_norm": 0.13338321447372437, "lr": 4.998903085539075e-08, "epoch": 1, "t": 5.010462284088135}
+{"step": 293, "loss": 0.11927410909358507, "grad_norm": 0.1483747512102127, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 4.673955202102661}
+{"step": 294, "loss": 0.08697800355922135, "grad_norm": 0.15373092889785767, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 4.913345813751221}
+{"step": 295, "loss": 0.0985233740531525, "grad_norm": 0.13708575069904327, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 5.145484924316406}
+{"step": 296, "loss": 0.10419206268442525, "grad_norm": 0.15778356790542603, "lr": 0.0, "epoch": 1, "t": 5.274567604064941}
diff --git a/qwen3-14b/rand-s0/README.md b/qwen3-14b/rand-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/rand-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/rand-s0/adapter_config.json b/qwen3-14b/rand-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..6585b02adafc87e0e9775b9c69be01c4dd9e2435
--- /dev/null
+++ b/qwen3-14b/rand-s0/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/rand-s0/adapter_model.safetensors b/qwen3-14b/rand-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..7f92817aaa0c5bf78bf2b0ba8880db8a8480e93f
--- /dev/null
+++ b/qwen3-14b/rand-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:13a6a95d889319ff0e4aa44980555556a96f257404587b2d8728084d5bef0cca
+size 1027680256
diff --git a/qwen3-14b/rand-s0/config.json b/qwen3-14b/rand-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..bb7359d075372fc87f488ad6c4d29a5f4973b6e1
--- /dev/null
+++ b/qwen3-14b/rand-s0/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-rand-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/rand-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 0,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/rand-s0/log.jsonl b/qwen3-14b/rand-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..73d0885ee08f7cf3828d675827c42b7d58d65d44
--- /dev/null
+++ b/qwen3-14b/rand-s0/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.13063744172569391, "grad_norm": 0.18884876370429993, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 4.7143707275390625}
+{"step": 2, "loss": 0.14917461895399062, "grad_norm": 0.1943492740392685, "lr": 2e-05, "epoch": 1, "t": 3.904162645339966}
+{"step": 3, "loss": 0.12744428923155193, "grad_norm": 0.17498455941677094, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 3.8562705516815186}
+{"step": 4, "loss": 0.16307338989515047, "grad_norm": 0.1928255707025528, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 4.596984148025513}
+{"step": 5, "loss": 0.0975530038606491, "grad_norm": 0.1667521446943283, "lr": 4e-05, "epoch": 1, "t": 4.568255662918091}
+{"step": 6, "loss": 0.16828298114113888, "grad_norm": 0.2305520921945572, "lr": 4.666666666666667e-05, "epoch": 1, "t": 4.1764256954193115}
+{"step": 7, "loss": 0.12271130477267121, "grad_norm": 0.2583584785461426, "lr": 5.333333333333333e-05, "epoch": 1, "t": 3.6191790103912354}
+{"step": 8, "loss": 0.11808230589560643, "grad_norm": 0.18637166917324066, "lr": 6e-05, "epoch": 1, "t": 4.953840732574463}
+{"step": 9, "loss": 0.1242029618407362, "grad_norm": 0.38885483145713806, "lr": 6.666666666666667e-05, "epoch": 1, "t": 3.5687646865844727}
+{"step": 10, "loss": 0.10608936786734786, "grad_norm": 0.30522620677948, "lr": 7.333333333333333e-05, "epoch": 1, "t": 3.741577625274658}
+{"step": 11, "loss": 0.1539506223973665, "grad_norm": 0.3170773983001709, "lr": 8e-05, "epoch": 1, "t": 4.436723470687866}
+{"step": 12, "loss": 0.16823488782191937, "grad_norm": 0.24872055649757385, "lr": 8.666666666666667e-05, "epoch": 1, "t": 4.054336786270142}
+{"step": 13, "loss": 0.1438128106691878, "grad_norm": 0.2239495813846588, "lr": 9.333333333333334e-05, "epoch": 1, "t": 3.736609935760498}
+{"step": 14, "loss": 0.09974613638384314, "grad_norm": 0.15647852420806885, "lr": 0.0001, "epoch": 1, "t": 4.273129224777222}
+{"step": 15, "loss": 0.07912772295137518, "grad_norm": 0.2055954933166504, "lr": 0.0001, "epoch": 1, "t": 3.2749156951904297}
+{"step": 16, "loss": 0.12809432258237774, "grad_norm": 0.1822308450937271, "lr": 9.999687519737639e-05, "epoch": 1, "t": 4.406087398529053}
+{"step": 17, "loss": 0.15867886400748443, "grad_norm": 0.17656747996807098, "lr": 9.998750118008115e-05, "epoch": 1, "t": 4.166068077087402}
+{"step": 18, "loss": 0.11532208568550752, "grad_norm": 0.1510051190853119, "lr": 9.997187911979252e-05, "epoch": 1, "t": 4.582029581069946}
+{"step": 19, "loss": 0.1475229994748689, "grad_norm": 0.21299098432064056, "lr": 9.995001096914461e-05, "epoch": 1, "t": 3.842892646789551}
+{"step": 20, "loss": 0.15427171577496213, "grad_norm": 0.20241893827915192, "lr": 9.992189946148366e-05, "epoch": 1, "t": 3.9040474891662598}
+{"step": 21, "loss": 0.13936085130502798, "grad_norm": 0.21584466099739075, "lr": 9.988754811052616e-05, "epoch": 1, "t": 3.7845208644866943}
+{"step": 22, "loss": 0.12468345640977964, "grad_norm": 0.2177668660879135, "lr": 9.984696120991978e-05, "epoch": 1, "t": 4.178686141967773}
+{"step": 23, "loss": 0.13501168510263806, "grad_norm": 0.25604909658432007, "lr": 9.980014383270668e-05, "epoch": 1, "t": 3.9005837440490723}
+{"step": 24, "loss": 0.14580991726806217, "grad_norm": 0.18704251945018768, "lr": 9.974710183068935e-05, "epoch": 1, "t": 4.132279396057129}
+{"step": 25, "loss": 0.13766809795444404, "grad_norm": 0.1929062306880951, "lr": 9.968784183369929e-05, "epoch": 1, "t": 3.7716190814971924}
+{"step": 26, "loss": 0.11168186255619196, "grad_norm": 0.1896827667951584, "lr": 9.962237124876828e-05, "epoch": 1, "t": 4.0464537143707275}
+{"step": 27, "loss": 0.13691313327538351, "grad_norm": 0.1892210692167282, "lr": 9.955069825920249e-05, "epoch": 1, "t": 3.69559383392334}
+{"step": 28, "loss": 0.13905032367233705, "grad_norm": 0.16726692020893097, "lr": 9.947283182355982e-05, "epoch": 1, "t": 3.7143235206604004}
+{"step": 29, "loss": 0.10705011593908302, "grad_norm": 0.21455983817577362, "lr": 9.938878167452992e-05, "epoch": 1, "t": 3.8267982006073}
+{"step": 30, "loss": 0.09823732138374701, "grad_norm": 0.1673227846622467, "lr": 9.929855831771786e-05, "epoch": 1, "t": 3.683778762817383}
+{"step": 31, "loss": 0.17360660868215652, "grad_norm": 0.18634264171123505, "lr": 9.92021730303309e-05, "epoch": 1, "t": 3.693690538406372}
+{"step": 32, "loss": 0.1781706869342089, "grad_norm": 0.1675170511007309, "lr": 9.909963785976903e-05, "epoch": 1, "t": 3.884718418121338}
+{"step": 33, "loss": 0.1319049098393292, "grad_norm": 0.16156695783138275, "lr": 9.899096562211902e-05, "epoch": 1, "t": 3.8327081203460693}
+{"step": 34, "loss": 0.07862303312416803, "grad_norm": 0.15542475879192352, "lr": 9.887616990055262e-05, "epoch": 1, "t": 4.081696510314941}
+{"step": 35, "loss": 0.11999189404965903, "grad_norm": 0.20739315450191498, "lr": 9.875526504362869e-05, "epoch": 1, "t": 3.5240304470062256}
+{"step": 36, "loss": 0.12825143585342155, "grad_norm": 0.19009912014007568, "lr": 9.86282661634998e-05, "epoch": 1, "t": 3.8562166690826416}
+{"step": 37, "loss": 0.15155166274891138, "grad_norm": 0.1790095567703247, "lr": 9.849518913402334e-05, "epoch": 1, "t": 4.531967401504517}
+{"step": 38, "loss": 0.09758490159937636, "grad_norm": 0.16299287974834442, "lr": 9.835605058877729e-05, "epoch": 1, "t": 3.771185874938965}
+{"step": 39, "loss": 0.10874784970800205, "grad_norm": 0.18461740016937256, "lr": 9.821086791898134e-05, "epoch": 1, "t": 3.6929101943969727}
+{"step": 40, "loss": 0.11802367715482906, "grad_norm": 0.19206224381923676, "lr": 9.805965927132295e-05, "epoch": 1, "t": 4.226487874984741}
+{"step": 41, "loss": 0.12562484879704675, "grad_norm": 0.19076506793498993, "lr": 9.79024435456893e-05, "epoch": 1, "t": 3.738661289215088}
+{"step": 42, "loss": 0.11396394204016105, "grad_norm": 0.15792526304721832, "lr": 9.773924039280487e-05, "epoch": 1, "t": 3.834486722946167}
+{"step": 43, "loss": 0.13937826225024172, "grad_norm": 0.16685685515403748, "lr": 9.75700702117753e-05, "epoch": 1, "t": 3.589245080947876}
+{"step": 44, "loss": 0.15102178153549534, "grad_norm": 0.15710943937301636, "lr": 9.739495414753753e-05, "epoch": 1, "t": 4.078253746032715}
+{"step": 45, "loss": 0.09192737120713618, "grad_norm": 0.11119838058948517, "lr": 9.721391408821711e-05, "epoch": 1, "t": 4.375234603881836}
+{"step": 46, "loss": 0.11532533256279834, "grad_norm": 0.13149093091487885, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.8619091510772705}
+{"step": 47, "loss": 0.11470261811544236, "grad_norm": 0.15645599365234375, "lr": 9.683415323626485e-05, "epoch": 1, "t": 3.7698252201080322}
+{"step": 48, "loss": 0.14324224885338843, "grad_norm": 0.16491730511188507, "lr": 9.663547991074127e-05, "epoch": 1, "t": 3.8051412105560303}
+{"step": 49, "loss": 0.12517026013996058, "grad_norm": 0.33263152837753296, "lr": 9.643097751841854e-05, "epoch": 1, "t": 4.008826732635498}
+{"step": 50, "loss": 0.11190577908339004, "grad_norm": 0.1588645726442337, "lr": 9.622067162048112e-05, "epoch": 1, "t": 3.6276235580444336}
+{"step": 51, "loss": 0.1018744273081313, "grad_norm": 0.1428661197423935, "lr": 9.600458850350588e-05, "epoch": 1, "t": 4.1547691822052}
+{"step": 52, "loss": 0.1073039149515423, "grad_norm": 0.17512419819831848, "lr": 9.578275517617645e-05, "epoch": 1, "t": 3.5223612785339355}
+{"step": 53, "loss": 0.13530123084969586, "grad_norm": 0.1604580581188202, "lr": 9.555519936590738e-05, "epoch": 1, "t": 4.648021697998047}
+{"step": 54, "loss": 0.10346917176853074, "grad_norm": 0.1753602921962738, "lr": 9.532194951537838e-05, "epoch": 1, "t": 4.337225675582886}
+{"step": 55, "loss": 0.11114729888589807, "grad_norm": 0.15050217509269714, "lr": 9.508303477897924e-05, "epoch": 1, "t": 3.907759189605713}
+{"step": 56, "loss": 0.14337899435146298, "grad_norm": 0.19645941257476807, "lr": 9.483848501916578e-05, "epoch": 1, "t": 4.1868979930877686}
+{"step": 57, "loss": 0.10835530621400946, "grad_norm": 0.16942940652370453, "lr": 9.458833080272722e-05, "epoch": 1, "t": 3.888917922973633}
+{"step": 58, "loss": 0.10565081274959227, "grad_norm": 0.18634340167045593, "lr": 9.433260339696563e-05, "epoch": 1, "t": 3.3215157985687256}
+{"step": 59, "loss": 0.13062661286653104, "grad_norm": 0.17000797390937805, "lr": 9.407133476578778e-05, "epoch": 1, "t": 3.942739725112915}
+{"step": 60, "loss": 0.12603491004545572, "grad_norm": 0.18449054658412933, "lr": 9.38045575657098e-05, "epoch": 1, "t": 3.925550937652588}
+{"step": 61, "loss": 0.12886897757463533, "grad_norm": 0.14598195254802704, "lr": 9.353230514177552e-05, "epoch": 1, "t": 4.417403697967529}
+{"step": 62, "loss": 0.1158662114542109, "grad_norm": 0.1266891360282898, "lr": 9.325461152338846e-05, "epoch": 1, "t": 4.147977590560913}
+{"step": 63, "loss": 0.12289751622882988, "grad_norm": 0.17332398891448975, "lr": 9.297151142005851e-05, "epoch": 1, "t": 3.501779556274414}
+{"step": 64, "loss": 0.09071785132511859, "grad_norm": 0.14522530138492584, "lr": 9.268304021706349e-05, "epoch": 1, "t": 3.5420279502868652}
+{"step": 65, "loss": 0.12607059594473535, "grad_norm": 0.16063007712364197, "lr": 9.23892339710263e-05, "epoch": 1, "t": 3.515418529510498}
+{"step": 66, "loss": 0.12871864218614243, "grad_norm": 0.15116773545742035, "lr": 9.209012940540805e-05, "epoch": 1, "t": 3.8058433532714844}
+{"step": 67, "loss": 0.10886359270048959, "grad_norm": 0.14571522176265717, "lr": 9.178576390591802e-05, "epoch": 1, "t": 3.781444787979126}
+{"step": 68, "loss": 0.11630377783165487, "grad_norm": 0.16861461102962494, "lr": 9.147617551584066e-05, "epoch": 1, "t": 3.8332409858703613}
+{"step": 69, "loss": 0.11444863072493747, "grad_norm": 0.14189867675304413, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.7889955043792725}
+{"step": 70, "loss": 0.14482762165973484, "grad_norm": 0.1742255836725235, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.6058731079101562}
+{"step": 71, "loss": 0.11205910630559876, "grad_norm": 0.16194593906402588, "lr": 9.051646319812918e-05, "epoch": 1, "t": 3.5231845378875732}
+{"step": 72, "loss": 0.12475200087161008, "grad_norm": 0.16449429094791412, "lr": 9.018637666191283e-05, "epoch": 1, "t": 3.872931480407715}
+{"step": 73, "loss": 0.1090974027579471, "grad_norm": 0.15600897371768951, "lr": 8.985126714588738e-05, "epoch": 1, "t": 3.7927770614624023}
+{"step": 74, "loss": 0.07262121283157214, "grad_norm": 0.1281401813030243, "lr": 8.951117653609666e-05, "epoch": 1, "t": 3.7207841873168945}
+{"step": 75, "loss": 0.1240601556616972, "grad_norm": 0.13084445893764496, "lr": 8.916614734118184e-05, "epoch": 1, "t": 4.060247421264648}
+{"step": 76, "loss": 0.0897922124969463, "grad_norm": 0.17047365009784698, "lr": 8.881622268706825e-05, "epoch": 1, "t": 3.328066825866699}
+{"step": 77, "loss": 0.14896827107485458, "grad_norm": 0.1637805998325348, "lr": 8.8461446311575e-05, "epoch": 1, "t": 3.9515182971954346}
+{"step": 78, "loss": 0.09029878946170045, "grad_norm": 0.1193251758813858, "lr": 8.810186255894803e-05, "epoch": 1, "t": 4.097472190856934}
+{"step": 79, "loss": 0.14805904026962327, "grad_norm": 0.14193478226661682, "lr": 8.773751637431748e-05, "epoch": 1, "t": 4.361790180206299}
+{"step": 80, "loss": 0.13055429300550594, "grad_norm": 0.2320476919412613, "lr": 8.736845329807993e-05, "epoch": 1, "t": 4.744079113006592}
+{"step": 81, "loss": 0.10270335812529462, "grad_norm": 0.19441503286361694, "lr": 8.69947194602061e-05, "epoch": 1, "t": 3.6140060424804688}
+{"step": 82, "loss": 0.10873268998145624, "grad_norm": 0.1785467267036438, "lr": 8.66163615744751e-05, "epoch": 1, "t": 3.5745010375976562}
+{"step": 83, "loss": 0.11936899837746572, "grad_norm": 0.13540203869342804, "lr": 8.623342693263548e-05, "epoch": 1, "t": 4.429108619689941}
+{"step": 84, "loss": 0.1466717164557351, "grad_norm": 0.16039647161960602, "lr": 8.584596339849417e-05, "epoch": 1, "t": 4.064231872558594}
+{"step": 85, "loss": 0.1121738731276118, "grad_norm": 0.20068345963954926, "lr": 8.545401940193392e-05, "epoch": 1, "t": 3.39123272895813}
+{"step": 86, "loss": 0.135595877978619, "grad_norm": 0.17735417187213898, "lr": 8.505764393285984e-05, "epoch": 1, "t": 3.97049880027771}
+{"step": 87, "loss": 0.12868852757499458, "grad_norm": 0.14396630227565765, "lr": 8.46568865350762e-05, "epoch": 1, "t": 4.195220470428467}
+{"step": 88, "loss": 0.1197514655518157, "grad_norm": 0.13714516162872314, "lr": 8.425179730009368e-05, "epoch": 1, "t": 3.9582035541534424}
+{"step": 89, "loss": 0.10061162116608727, "grad_norm": 0.13751062750816345, "lr": 8.384242686086848e-05, "epoch": 1, "t": 3.5333163738250732}
+{"step": 90, "loss": 0.1129351636494261, "grad_norm": 0.17610831558704376, "lr": 8.342882638547351e-05, "epoch": 1, "t": 3.5529720783233643}
+{"step": 91, "loss": 0.13342728881813976, "grad_norm": 0.14068762958049774, "lr": 8.301104757070274e-05, "epoch": 1, "t": 3.9308223724365234}
+{"step": 92, "loss": 0.12228291260080273, "grad_norm": 0.1279202252626419, "lr": 8.258914263560971e-05, "epoch": 1, "t": 4.3844969272613525}
+{"step": 93, "loss": 0.09734057047352285, "grad_norm": 0.12381435185670853, "lr": 8.216316431498028e-05, "epoch": 1, "t": 4.3131794929504395}
+{"step": 94, "loss": 0.11042404934176299, "grad_norm": 0.12191866338253021, "lr": 8.173316585274145e-05, "epoch": 1, "t": 4.4245617389678955}
+{"step": 95, "loss": 0.1022839967424228, "grad_norm": 0.15601123869419098, "lr": 8.129920099530607e-05, "epoch": 1, "t": 3.4969820976257324}
+{"step": 96, "loss": 0.13810561416921396, "grad_norm": 0.13694947957992554, "lr": 8.086132398485524e-05, "epoch": 1, "t": 4.27752161026001}
+{"step": 97, "loss": 0.09320668364497971, "grad_norm": 0.15184980630874634, "lr": 8.041958955255814e-05, "epoch": 1, "t": 3.96920108795166}
+{"step": 98, "loss": 0.12394749970976696, "grad_norm": 0.16112864017486572, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.8575994968414307}
+{"step": 99, "loss": 0.1166933832538539, "grad_norm": 0.13649243116378784, "lr": 7.952476975093729e-05, "epoch": 1, "t": 4.176828861236572}
+{"step": 100, "loss": 0.11254447124803349, "grad_norm": 0.1544632762670517, "lr": 7.907179622702408e-05, "epoch": 1, "t": 5.21608304977417}
+{"step": 101, "loss": 0.12383672102531795, "grad_norm": 0.15266002714633942, "lr": 7.861518895810596e-05, "epoch": 1, "t": 5.2948899269104}
+{"step": 102, "loss": 0.09515802519354132, "grad_norm": 0.13541385531425476, "lr": 7.815500501648653e-05, "epoch": 1, "t": 4.0919623374938965}
+{"step": 103, "loss": 0.11962114824080178, "grad_norm": 0.14856617152690887, "lr": 7.769130192152538e-05, "epoch": 1, "t": 3.9134371280670166}
+{"step": 104, "loss": 0.11037142307203696, "grad_norm": 0.14680765569210052, "lr": 7.722413763244838e-05, "epoch": 1, "t": 3.7028579711914062}
+{"step": 105, "loss": 0.15667597086912247, "grad_norm": 0.14207899570465088, "lr": 7.675357054110336e-05, "epoch": 1, "t": 3.990389585494995}
+{"step": 106, "loss": 0.14188347261617526, "grad_norm": 0.14676420390605927, "lr": 7.627965946466166e-05, "epoch": 1, "t": 3.946812868118286}
+{"step": 107, "loss": 0.11700991277900356, "grad_norm": 0.12799125909805298, "lr": 7.580246363826621e-05, "epoch": 1, "t": 4.464916229248047}
+{"step": 108, "loss": 0.12461599768209417, "grad_norm": 0.13717973232269287, "lr": 7.532204270762786e-05, "epoch": 1, "t": 3.800016164779663}
+{"step": 109, "loss": 0.12354908815091048, "grad_norm": 0.167184978723526, "lr": 7.483845672156998e-05, "epoch": 1, "t": 3.6759755611419678}
+{"step": 110, "loss": 0.12857265750036934, "grad_norm": 0.1645645946264267, "lr": 7.435176612452286e-05, "epoch": 1, "t": 3.6381747722625732}
+{"step": 111, "loss": 0.11287861966982832, "grad_norm": 0.1968693882226944, "lr": 7.386203174896872e-05, "epoch": 1, "t": 3.2726166248321533}
+{"step": 112, "loss": 0.11710029929254828, "grad_norm": 0.126938596367836, "lr": 7.336931480783801e-05, "epoch": 1, "t": 4.316344976425171}
+{"step": 113, "loss": 0.1254897478622663, "grad_norm": 0.13334307074546814, "lr": 7.287367688685835e-05, "epoch": 1, "t": 4.9947991371154785}
+{"step": 114, "loss": 0.11436996018804456, "grad_norm": 0.1392892450094223, "lr": 7.237517993685678e-05, "epoch": 1, "t": 4.236661195755005}
+{"step": 115, "loss": 0.11887743087656011, "grad_norm": 0.17788182199001312, "lr": 7.187388626601637e-05, "epoch": 1, "t": 4.997808218002319}
+{"step": 116, "loss": 0.11059844716656693, "grad_norm": 0.1356772482395172, "lr": 7.136985853208824e-05, "epoch": 1, "t": 4.384433746337891}
+{"step": 117, "loss": 0.13271089683849596, "grad_norm": 0.1460568904876709, "lr": 7.086315973455981e-05, "epoch": 1, "t": 4.326270580291748}
+{"step": 118, "loss": 0.09184544789414215, "grad_norm": 0.10667787492275238, "lr": 7.035385320678036e-05, "epoch": 1, "t": 4.86601185798645}
+{"step": 119, "loss": 0.09550598864089159, "grad_norm": 0.14877891540527344, "lr": 6.984200260804484e-05, "epoch": 1, "t": 4.180031776428223}
+{"step": 120, "loss": 0.09982331124014167, "grad_norm": 0.14623905718326569, "lr": 6.932767191563703e-05, "epoch": 1, "t": 3.4961490631103516}
+{"step": 121, "loss": 0.11518024410460515, "grad_norm": 0.15288127958774567, "lr": 6.881092541683278e-05, "epoch": 1, "t": 3.475856065750122}
+{"step": 122, "loss": 0.0938867161917897, "grad_norm": 0.15642106533050537, "lr": 6.829182770086474e-05, "epoch": 1, "t": 3.3827476501464844}
+{"step": 123, "loss": 0.12127672891158948, "grad_norm": 0.1564423143863678, "lr": 6.777044365084907e-05, "epoch": 1, "t": 3.9867360591888428}
+{"step": 124, "loss": 0.12519810191062525, "grad_norm": 0.14210422337055206, "lr": 6.724683843567568e-05, "epoch": 1, "t": 3.8832924365997314}
+{"step": 125, "loss": 0.10985100407554865, "grad_norm": 0.14444823563098907, "lr": 6.672107750186255e-05, "epoch": 1, "t": 3.737884044647217}
+{"step": 126, "loss": 0.11622768217569335, "grad_norm": 0.1504901647567749, "lr": 6.619322656537552e-05, "epoch": 1, "t": 3.6100056171417236}
+{"step": 127, "loss": 0.14080207606221762, "grad_norm": 0.12105653434991837, "lr": 6.566335160341424e-05, "epoch": 1, "t": 4.28421425819397}
+{"step": 128, "loss": 0.14794039279184212, "grad_norm": 0.13759765028953552, "lr": 6.513151884616556e-05, "epoch": 1, "t": 4.612864017486572}
+{"step": 129, "loss": 0.13725125098842955, "grad_norm": 0.1509697586297989, "lr": 6.459779476852528e-05, "epoch": 1, "t": 3.5270888805389404}
+{"step": 130, "loss": 0.13240088352862311, "grad_norm": 0.164494588971138, "lr": 6.406224608178932e-05, "epoch": 1, "t": 3.8706212043762207}
+{"step": 131, "loss": 0.10660856967565718, "grad_norm": 0.13213799893856049, "lr": 6.352493972531534e-05, "epoch": 1, "t": 3.776811361312866}
+{"step": 132, "loss": 0.12539789630140927, "grad_norm": 0.18733641505241394, "lr": 6.298594285815584e-05, "epoch": 1, "t": 3.5966567993164062}
+{"step": 133, "loss": 0.13951622045335924, "grad_norm": 0.13188901543617249, "lr": 6.244532285066382e-05, "epoch": 1, "t": 3.9098591804504395}
+{"step": 134, "loss": 0.1094179240816684, "grad_norm": 0.14634092152118683, "lr": 6.190314727607196e-05, "epoch": 1, "t": 4.206386089324951}
+{"step": 135, "loss": 0.1303330071723105, "grad_norm": 0.12105501443147659, "lr": 6.13594839020466e-05, "epoch": 1, "t": 4.370079278945923}
+{"step": 136, "loss": 0.13786824329421213, "grad_norm": 0.1408495306968689, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 3.8417158126831055}
+{"step": 137, "loss": 0.1191982547028336, "grad_norm": 0.18210862576961517, "lr": 6.026796574768288e-05, "epoch": 1, "t": 3.577951192855835}
+{"step": 138, "loss": 0.12485792665722872, "grad_norm": 0.16429105401039124, "lr": 5.972024739849622e-05, "epoch": 1, "t": 3.9806981086730957}
+{"step": 139, "loss": 0.11353920012229958, "grad_norm": 0.161812886595726, "lr": 5.917131409512663e-05, "epoch": 1, "t": 3.7877607345581055}
+{"step": 140, "loss": 0.12472720455549641, "grad_norm": 0.15094594657421112, "lr": 5.862123444990318e-05, "epoch": 1, "t": 3.970086097717285}
+{"step": 141, "loss": 0.10262656779596298, "grad_norm": 0.1644820272922516, "lr": 5.807007721843861e-05, "epoch": 1, "t": 3.906405448913574}
+{"step": 142, "loss": 0.11600094072973949, "grad_norm": 0.16278769075870514, "lr": 5.751791129103544e-05, "epoch": 1, "t": 3.4730634689331055}
+{"step": 143, "loss": 0.11402785616347628, "grad_norm": 0.14301691949367523, "lr": 5.696480568407523e-05, "epoch": 1, "t": 3.8870291709899902}
+{"step": 144, "loss": 0.12232247526651749, "grad_norm": 0.15713658928871155, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 3.7450244426727295}
+{"step": 145, "loss": 0.12775926953055583, "grad_norm": 0.15137432515621185, "lr": 5.585605207563124e-05, "epoch": 1, "t": 3.766122579574585}
+{"step": 146, "loss": 0.10251197472806695, "grad_norm": 0.15194566547870636, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 3.5662028789520264}
+{"step": 147, "loss": 0.12349837062831873, "grad_norm": 0.1578071415424347, "lr": 5.47443707175741e-05, "epoch": 1, "t": 3.6008200645446777}
+{"step": 148, "loss": 0.12613938974611688, "grad_norm": 0.1853848695755005, "lr": 5.418760576667071e-05, "epoch": 1, "t": 3.687786817550659}
+{"step": 149, "loss": 0.12935202297795184, "grad_norm": 0.14879556000232697, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 3.7023513317108154}
+{"step": 150, "loss": 0.08434379126714624, "grad_norm": 0.1256607621908188, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 3.876605987548828}
+{"step": 151, "loss": 0.09151104676763086, "grad_norm": 0.14048200845718384, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 3.9418692588806152}
+{"step": 152, "loss": 0.11597707087989974, "grad_norm": 0.1647757887840271, "lr": 5.195600862779421e-05, "epoch": 1, "t": 3.469231605529785}
+{"step": 153, "loss": 0.12262439456572825, "grad_norm": 0.13037925958633423, "lr": 5.139732367864736e-05, "epoch": 1, "t": 4.342042684555054}
+{"step": 154, "loss": 0.11972581049055483, "grad_norm": 0.1439259648323059, "lr": 5.083846407507263e-05, "epoch": 1, "t": 3.6906116008758545}
+{"step": 155, "loss": 0.13864615811259756, "grad_norm": 0.1487456113100052, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 3.9051268100738525}
+{"step": 156, "loss": 0.0932898151590055, "grad_norm": 0.1467892974615097, "lr": 4.972050032989175e-05, "epoch": 1, "t": 3.458915948867798}
+{"step": 157, "loss": 0.11145399919816978, "grad_norm": 0.14764772355556488, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 3.9458272457122803}
+{"step": 158, "loss": 0.08635486788051183, "grad_norm": 0.14222252368927002, "lr": 4.860267632135265e-05, "epoch": 1, "t": 3.4970357418060303}
+{"step": 159, "loss": 0.09817816477044636, "grad_norm": 0.15708449482917786, "lr": 4.80439913722058e-05, "epoch": 1, "t": 3.365159749984741}
+{"step": 160, "loss": 0.10825255019979599, "grad_norm": 0.1257888376712799, "lr": 4.748555090869464e-05, "epoch": 1, "t": 4.218597412109375}
+{"step": 161, "loss": 0.1283853302553096, "grad_norm": 0.14782242476940155, "lr": 4.692742473146818e-05, "epoch": 1, "t": 3.7179553508758545}
+{"step": 162, "loss": 0.11644606500855496, "grad_norm": 0.1450665295124054, "lr": 4.636968260189214e-05, "epoch": 1, "t": 3.959094285964966}
+{"step": 163, "loss": 0.12101017804132114, "grad_norm": 0.13800328969955444, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 3.82706880569458}
+{"step": 164, "loss": 0.11874813953154857, "grad_norm": 0.1934743970632553, "lr": 4.525562928242592e-05, "epoch": 1, "t": 3.3295722007751465}
+{"step": 165, "loss": 0.14104585434748323, "grad_norm": 0.15281645953655243, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 3.680652141571045}
+{"step": 166, "loss": 0.10989368193022575, "grad_norm": 0.16178777813911438, "lr": 4.414394792436877e-05, "epoch": 1, "t": 3.512995481491089}
+{"step": 167, "loss": 0.09684251940928365, "grad_norm": 0.14699143171310425, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 3.487665891647339}
+{"step": 168, "loss": 0.09945611281064047, "grad_norm": 0.13680197298526764, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 3.5689010620117188}
+{"step": 169, "loss": 0.12464161546110783, "grad_norm": 0.201942577958107, "lr": 4.248208870896456e-05, "epoch": 1, "t": 3.371204137802124}
+{"step": 170, "loss": 0.11237651729502851, "grad_norm": 0.14342100918293, "lr": 4.192992278156141e-05, "epoch": 1, "t": 3.5024757385253906}
+{"step": 171, "loss": 0.1098922752479985, "grad_norm": 0.14969618618488312, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 3.5320651531219482}
+{"step": 172, "loss": 0.10074161284924381, "grad_norm": 0.1326635330915451, "lr": 4.082868590487339e-05, "epoch": 1, "t": 3.766558885574341}
+{"step": 173, "loss": 0.09924083076482879, "grad_norm": 0.15501920878887177, "lr": 4.027975260150381e-05, "epoch": 1, "t": 3.495896100997925}
+{"step": 174, "loss": 0.10777683298354696, "grad_norm": 0.1283881515264511, "lr": 3.973203425231715e-05, "epoch": 1, "t": 4.096708536148071}
+{"step": 175, "loss": 0.09763513228899884, "grad_norm": 0.13809619843959808, "lr": 3.918559931778277e-05, "epoch": 1, "t": 3.4729177951812744}
+{"step": 176, "loss": 0.13209157477981695, "grad_norm": 0.16543208062648773, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 3.78169322013855}
+{"step": 177, "loss": 0.12241879587527367, "grad_norm": 0.17312274873256683, "lr": 3.809685272392804e-05, "epoch": 1, "t": 3.6190333366394043}
+{"step": 178, "loss": 0.10216127435284784, "grad_norm": 0.13237914443016052, "lr": 3.755467714933619e-05, "epoch": 1, "t": 4.4230756759643555}
+{"step": 179, "loss": 0.12400198907718413, "grad_norm": 0.13339777290821075, "lr": 3.701405714184416e-05, "epoch": 1, "t": 3.923017978668213}
+{"step": 180, "loss": 0.10618985393422699, "grad_norm": 0.15844671428203583, "lr": 3.647506027468467e-05, "epoch": 1, "t": 3.5302205085754395}
+{"step": 181, "loss": 0.12105981178196314, "grad_norm": 0.16875486075878143, "lr": 3.59377539182107e-05, "epoch": 1, "t": 3.4601058959960938}
+{"step": 182, "loss": 0.10484316514416285, "grad_norm": 0.13004560768604279, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 4.206946134567261}
+{"step": 183, "loss": 0.1209604022066289, "grad_norm": 0.12609364092350006, "lr": 3.486848115383445e-05, "epoch": 1, "t": 4.160457372665405}
+{"step": 184, "loss": 0.12371655334256768, "grad_norm": 0.1694018393754959, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 3.5291030406951904}
+{"step": 185, "loss": 0.0922717502677667, "grad_norm": 0.12088298052549362, "lr": 3.380677343462447e-05, "epoch": 1, "t": 4.160749673843384}
+{"step": 186, "loss": 0.12123508570345602, "grad_norm": 0.1412910670042038, "lr": 3.327892249813745e-05, "epoch": 1, "t": 3.570387840270996}
+{"step": 187, "loss": 0.1326169938635977, "grad_norm": 0.14308451116085052, "lr": 3.275316156432434e-05, "epoch": 1, "t": 4.074736595153809}
+{"step": 188, "loss": 0.12803857956523892, "grad_norm": 0.13146254420280457, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 4.175413608551025}
+{"step": 189, "loss": 0.1540044839099302, "grad_norm": 0.1708984076976776, "lr": 3.170817229913526e-05, "epoch": 1, "t": 3.7445125579833984}
+{"step": 190, "loss": 0.1070383841448491, "grad_norm": 0.13485515117645264, "lr": 3.118907458316722e-05, "epoch": 1, "t": 3.6495563983917236}
+{"step": 191, "loss": 0.12320495203521799, "grad_norm": 0.1620662361383438, "lr": 3.067232808436299e-05, "epoch": 1, "t": 3.829038381576538}
+{"step": 192, "loss": 0.12028064254082046, "grad_norm": 0.16399812698364258, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 3.8880374431610107}
+{"step": 193, "loss": 0.11215786970427405, "grad_norm": 0.1684739738702774, "lr": 2.964614679321966e-05, "epoch": 1, "t": 3.464961528778076}
+{"step": 194, "loss": 0.10680376433228539, "grad_norm": 0.18970543146133423, "lr": 2.913684026544021e-05, "epoch": 1, "t": 3.3538718223571777}
+{"step": 195, "loss": 0.11445269257136134, "grad_norm": 0.12791547179222107, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 4.203958988189697}
+{"step": 196, "loss": 0.10031463620098126, "grad_norm": 0.12584321200847626, "lr": 2.812611373398365e-05, "epoch": 1, "t": 4.228569984436035}
+{"step": 197, "loss": 0.1323528138505689, "grad_norm": 0.12129411101341248, "lr": 2.762482006314324e-05, "epoch": 1, "t": 5.849458932876587}
+{"step": 198, "loss": 0.07238522906913511, "grad_norm": 0.13194037973880768, "lr": 2.712632311314165e-05, "epoch": 1, "t": 5.432722806930542}
+{"step": 199, "loss": 0.14731213101800883, "grad_norm": 0.14559704065322876, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 5.579730749130249}
+{"step": 200, "loss": 0.10454382879315487, "grad_norm": 0.11314421892166138, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 5.750651121139526}
+{"step": 201, "loss": 0.13747343455797115, "grad_norm": 0.14104115962982178, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 5.313806533813477}
+{"step": 202, "loss": 0.13384708459265324, "grad_norm": 0.14004284143447876, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 4.685939788818359}
+{"step": 203, "loss": 0.1384987361753853, "grad_norm": 0.14938446879386902, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 4.1045215129852295}
+{"step": 204, "loss": 0.1412249047251108, "grad_norm": 0.14141784608364105, "lr": 2.419753636173379e-05, "epoch": 1, "t": 4.358543157577515}
+{"step": 205, "loss": 0.13309435460844582, "grad_norm": 0.1492764800786972, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 4.4110267162323}
+{"step": 206, "loss": 0.10838455925856996, "grad_norm": 0.14053869247436523, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 3.9672482013702393}
+{"step": 207, "loss": 0.10432935719983713, "grad_norm": 0.1407681107521057, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 4.157055377960205}
+{"step": 208, "loss": 0.1116715300869431, "grad_norm": 0.12763641774654388, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 4.627126693725586}
+{"step": 209, "loss": 0.12029385435103261, "grad_norm": 0.1553834080696106, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 3.835510492324829}
+{"step": 210, "loss": 0.10355902056477848, "grad_norm": 0.15685078501701355, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 3.83315372467041}
+{"step": 211, "loss": 0.09951720820000927, "grad_norm": 0.18026545643806458, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 3.551116943359375}
+{"step": 212, "loss": 0.1206685559038973, "grad_norm": 0.1719389110803604, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 3.6111040115356445}
+{"step": 213, "loss": 0.11965227652771156, "grad_norm": 0.1453884392976761, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 3.709904670715332}
+{"step": 214, "loss": 0.16079790176240089, "grad_norm": 0.13141708076000214, "lr": 1.958041044744186e-05, "epoch": 1, "t": 4.128185033798218}
+{"step": 215, "loss": 0.10308699143199496, "grad_norm": 0.11320309340953827, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 4.01594877243042}
+{"step": 216, "loss": 0.1371837056781005, "grad_norm": 0.13572478294372559, "lr": 1.870079900469392e-05, "epoch": 1, "t": 4.05987548828125}
+{"step": 217, "loss": 0.1302097842258035, "grad_norm": 0.1411149799823761, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 3.9813575744628906}
+{"step": 218, "loss": 0.10511741972061192, "grad_norm": 0.14919930696487427, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 3.6139538288116455}
+{"step": 219, "loss": 0.09861642847005245, "grad_norm": 0.11938363313674927, "lr": 1.741085736439031e-05, "epoch": 1, "t": 4.152544975280762}
+{"step": 220, "loss": 0.11561277832257262, "grad_norm": 0.16253727674484253, "lr": 1.698895242929725e-05, "epoch": 1, "t": 3.5756118297576904}
+{"step": 221, "loss": 0.17228343481837735, "grad_norm": 0.16002237796783447, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 3.792842388153076}
+{"step": 222, "loss": 0.12060398511104424, "grad_norm": 0.13386821746826172, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 3.8732781410217285}
+{"step": 223, "loss": 0.09908143132756526, "grad_norm": 0.13953521847724915, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 4.758551836013794}
+{"step": 224, "loss": 0.11081413116019742, "grad_norm": 0.1264645904302597, "lr": 1.534311346492381e-05, "epoch": 1, "t": 5.408212423324585}
+{"step": 225, "loss": 0.09425213366924416, "grad_norm": 0.15765555202960968, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 4.4142138957977295}
+{"step": 226, "loss": 0.1017111998912164, "grad_norm": 0.1360512673854828, "lr": 1.454598059806609e-05, "epoch": 1, "t": 3.8538289070129395}
+{"step": 227, "loss": 0.1141252014288062, "grad_norm": 0.14215661585330963, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 3.9431774616241455}
+{"step": 228, "loss": 0.13793842636355672, "grad_norm": 0.1656450778245926, "lr": 1.376657306736453e-05, "epoch": 1, "t": 4.045980930328369}
+{"step": 229, "loss": 0.1026326601538773, "grad_norm": 0.12206733971834183, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 4.298875093460083}
+{"step": 230, "loss": 0.14199195722073019, "grad_norm": 0.26827171444892883, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 3.822054624557495}
+{"step": 231, "loss": 0.14231588698669434, "grad_norm": 0.1293066293001175, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 4.264700889587402}
+{"step": 232, "loss": 0.11699123215402502, "grad_norm": 0.12952665984630585, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 3.9135138988494873}
+{"step": 233, "loss": 0.10893600325159301, "grad_norm": 0.12195605784654617, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 4.216217279434204}
+{"step": 234, "loss": 0.09946989246715199, "grad_norm": 0.13762715458869934, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 3.9802305698394775}
+{"step": 235, "loss": 0.11928359767822933, "grad_norm": 0.1294693946838379, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 4.077860355377197}
+{"step": 236, "loss": 0.14437356292508655, "grad_norm": 0.13183392584323883, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 4.3283514976501465}
+{"step": 237, "loss": 0.13391989940445256, "grad_norm": 0.14984115958213806, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 3.714932918548584}
+{"step": 238, "loss": 0.13264533005032522, "grad_norm": 0.14607205986976624, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 4.048345327377319}
+{"step": 239, "loss": 0.0915458547330119, "grad_norm": 0.1467190533876419, "lr": 9.81362333808718e-06, "epoch": 1, "t": 3.5497169494628906}
+{"step": 240, "loss": 0.13071301069100838, "grad_norm": 0.15232589840888977, "lr": 9.483536801870834e-06, "epoch": 1, "t": 4.323862791061401}
+{"step": 241, "loss": 0.12687601854203942, "grad_norm": 0.1335902363061905, "lr": 9.158514503674543e-06, "epoch": 1, "t": 3.8125460147857666}
+{"step": 242, "loss": 0.10499803850134333, "grad_norm": 0.13930219411849976, "lr": 8.838597068719518e-06, "epoch": 1, "t": 3.757858991622925}
+{"step": 243, "loss": 0.12812834786953278, "grad_norm": 0.15196475386619568, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.62127947807312}
+{"step": 244, "loss": 0.09983033094998242, "grad_norm": 0.16435034573078156, "lr": 8.21423609408199e-06, "epoch": 1, "t": 3.2397665977478027}
+{"step": 245, "loss": 0.12152329246513136, "grad_norm": 0.13646575808525085, "lr": 7.90987059459195e-06, "epoch": 1, "t": 3.937091112136841}
+{"step": 246, "loss": 0.09367015058073275, "grad_norm": 0.14807634055614471, "lr": 7.610766028973709e-06, "epoch": 1, "t": 3.711440086364746}
+{"step": 247, "loss": 0.1111858754172135, "grad_norm": 0.15524619817733765, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 3.7445738315582275}
+{"step": 248, "loss": 0.1232177065673268, "grad_norm": 0.12470219284296036, "lr": 7.028488579941506e-06, "epoch": 1, "t": 4.024374008178711}
+{"step": 249, "loss": 0.10247047013099041, "grad_norm": 0.16215835511684418, "lr": 6.745388476611553e-06, "epoch": 1, "t": 3.386770725250244}
+{"step": 250, "loss": 0.12231897152874992, "grad_norm": 0.1546342968940735, "lr": 6.467694858224488e-06, "epoch": 1, "t": 3.701866626739502}
+{"step": 251, "loss": 0.14106972081464642, "grad_norm": 0.16130059957504272, "lr": 6.1954424342902e-06, "epoch": 1, "t": 3.4434165954589844}
+{"step": 252, "loss": 0.13465685679437026, "grad_norm": 0.15661659836769104, "lr": 5.928665234212233e-06, "epoch": 1, "t": 3.5990512371063232}
+{"step": 253, "loss": 0.12554138426301004, "grad_norm": 0.1528693586587906, "lr": 5.66739660303437e-06, "epoch": 1, "t": 4.071005344390869}
+{"step": 254, "loss": 0.10249093747101981, "grad_norm": 0.12408269941806793, "lr": 5.411669197272795e-06, "epoch": 1, "t": 4.085033655166626}
+{"step": 255, "loss": 0.11235916429334065, "grad_norm": 0.1512967050075531, "lr": 5.161514980834231e-06, "epoch": 1, "t": 3.967587947845459}
+{"step": 256, "loss": 0.12685886555247838, "grad_norm": 0.13728024065494537, "lr": 4.916965221020753e-06, "epoch": 1, "t": 3.7416632175445557}
+{"step": 257, "loss": 0.14054604031868623, "grad_norm": 0.15622453391551971, "lr": 4.678050484621615e-06, "epoch": 1, "t": 3.5704426765441895}
+{"step": 258, "loss": 0.09794243379639245, "grad_norm": 0.14979258179664612, "lr": 4.444800634092616e-06, "epoch": 1, "t": 3.517932653427124}
+{"step": 259, "loss": 0.11067975777767713, "grad_norm": 0.14913104474544525, "lr": 4.217244823823546e-06, "epoch": 1, "t": 3.6515443325042725}
+{"step": 260, "loss": 0.10845219004082163, "grad_norm": 0.15049849450588226, "lr": 3.995411496494134e-06, "epoch": 1, "t": 3.6774790287017822}
+{"step": 261, "loss": 0.09272371418892392, "grad_norm": 0.13843287527561188, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 4.345511198043823}
+{"step": 262, "loss": 0.10246307490468495, "grad_norm": 0.14024953544139862, "lr": 3.56902248158148e-06, "epoch": 1, "t": 3.6179401874542236}
+{"step": 263, "loss": 0.10526236067426965, "grad_norm": 0.13355275988578796, "lr": 3.364520089258727e-06, "epoch": 1, "t": 4.017863988876343}
+{"step": 264, "loss": 0.1271398728220324, "grad_norm": 0.14693425595760345, "lr": 3.165846763735153e-06, "epoch": 1, "t": 4.013265371322632}
+{"step": 265, "loss": 0.119801380389637, "grad_norm": 0.14940407872200012, "lr": 2.973027337607892e-06, "epoch": 1, "t": 4.413431644439697}
+{"step": 266, "loss": 0.09307333785603615, "grad_norm": 0.15237398445606232, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.4640953540802}
+{"step": 267, "loss": 0.11605536056188727, "grad_norm": 0.13216924667358398, "lr": 2.605045852462473e-06, "epoch": 1, "t": 3.5735092163085938}
+{"step": 268, "loss": 0.11250033617228976, "grad_norm": 0.14043454825878143, "lr": 2.429929788224722e-06, "epoch": 1, "t": 3.533344030380249}
+{"step": 269, "loss": 0.10140393949958908, "grad_norm": 0.1393231749534607, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 3.715564489364624}
+{"step": 270, "loss": 0.14290319433687484, "grad_norm": 0.11430567502975464, "lr": 2.097556454310701e-06, "epoch": 1, "t": 4.55055832862854}
+{"step": 271, "loss": 0.12656662222292991, "grad_norm": 0.11684075742959976, "lr": 1.940340728677059e-06, "epoch": 1, "t": 4.4555864334106445}
+{"step": 272, "loss": 0.11751168615677776, "grad_norm": 0.15979020297527313, "lr": 1.789132081018674e-06, "epoch": 1, "t": 3.5063281059265137}
+{"step": 273, "loss": 0.0896137408773616, "grad_norm": 0.15561534464359283, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 3.7661659717559814}
+{"step": 274, "loss": 0.09162135967736086, "grad_norm": 0.12178725004196167, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 4.072111129760742}
+{"step": 275, "loss": 0.10128708416518149, "grad_norm": 0.12729240953922272, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 4.082916736602783}
+{"step": 276, "loss": 0.13995370614990213, "grad_norm": 0.15893715620040894, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 3.8342602252960205}
+{"step": 277, "loss": 0.11032166917176821, "grad_norm": 0.15048657357692719, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 3.7907795906066895}
+{"step": 278, "loss": 0.13573765606641497, "grad_norm": 0.174109548330307, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 3.625577211380005}
+{"step": 279, "loss": 0.12390362359786014, "grad_norm": 0.13556933403015137, "lr": 9.003621402309814e-07, "epoch": 1, "t": 3.929826021194458}
+{"step": 280, "loss": 0.11757731824527938, "grad_norm": 0.12747380137443542, "lr": 7.97826969669102e-07, "epoch": 1, "t": 4.066500186920166}
+{"step": 281, "loss": 0.11681006422304657, "grad_norm": 0.12020257860422134, "lr": 7.014416822821556e-07, "epoch": 1, "t": 4.9291393756866455}
+{"step": 282, "loss": 0.12383387628134376, "grad_norm": 0.15719550848007202, "lr": 6.112183254700865e-07, "epoch": 1, "t": 3.5830471515655518}
+{"step": 283, "loss": 0.11255017400075616, "grad_norm": 0.15443822741508484, "lr": 5.271681764401848e-07, "epoch": 1, "t": 3.557929515838623}
+{"step": 284, "loss": 0.12278837659587599, "grad_norm": 0.1382555216550827, "lr": 4.493017407975086e-07, "epoch": 1, "t": 4.048773288726807}
+{"step": 285, "loss": 0.13190794036523565, "grad_norm": 0.1117214486002922, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 4.7909228801727295}
+{"step": 286, "loss": 0.11710224533373821, "grad_norm": 0.13232731819152832, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 4.2545764446258545}
+{"step": 287, "loss": 0.13508186116507837, "grad_norm": 0.13754023611545563, "lr": 2.528981693106558e-07, "epoch": 1, "t": 4.065375328063965}
+{"step": 288, "loss": 0.10308833930779386, "grad_norm": 0.15334084630012512, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 3.513911247253418}
+{"step": 289, "loss": 0.08827174595466902, "grad_norm": 0.12297967076301575, "lr": 1.530387900802177e-07, "epoch": 1, "t": 4.216980934143066}
+{"step": 290, "loss": 0.1169178565849979, "grad_norm": 0.12210342288017273, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 4.1008570194244385}
+{"step": 291, "loss": 0.11050771523895575, "grad_norm": 0.14676770567893982, "lr": 7.81005385163458e-08, "epoch": 1, "t": 3.7655985355377197}
+{"step": 292, "loss": 0.09746205362793232, "grad_norm": 0.14685246348381042, "lr": 4.998903085539075e-08, "epoch": 1, "t": 3.410233736038208}
+{"step": 293, "loss": 0.11854691758143393, "grad_norm": 0.18407782912254333, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 3.402395248413086}
+{"step": 294, "loss": 0.08891367252525625, "grad_norm": 0.11282521486282349, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 4.189221143722534}
+{"step": 295, "loss": 0.10030288073707379, "grad_norm": 0.15782690048217773, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 3.6034159660339355}
+{"step": 296, "loss": 0.1029444193034429, "grad_norm": 0.1636267900466919, "lr": 0.0, "epoch": 1, "t": 3.711299180984497}
diff --git a/qwen3-14b/rand-s1/README.md b/qwen3-14b/rand-s1/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/rand-s1/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/rand-s1/adapter_config.json b/qwen3-14b/rand-s1/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..9df4e0230b615cef87f8f183dd9ba1dfcf4acb60
--- /dev/null
+++ b/qwen3-14b/rand-s1/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "q_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/rand-s1/adapter_model.safetensors b/qwen3-14b/rand-s1/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..793c7b6804d81aee9e068aa9fa6a49abc5c1670c
--- /dev/null
+++ b/qwen3-14b/rand-s1/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef586d274cb44dd07f5815668f81e3cdca9aa0580a3d43cf721800de0950e339
+size 1027680256
diff --git a/qwen3-14b/rand-s1/config.json b/qwen3-14b/rand-s1/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..7049cc16f2135e66862b4a0cc7bc2ca0f4f659de
--- /dev/null
+++ b/qwen3-14b/rand-s1/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-rand-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/rand-s1",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 1,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/rand-s1/log.jsonl b/qwen3-14b/rand-s1/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..cf6a82d5793bccb20094beb988b2b24c9570406d
--- /dev/null
+++ b/qwen3-14b/rand-s1/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.19231999979044, "grad_norm": 0.21790888905525208, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 4.311826705932617}
+{"step": 2, "loss": 0.13169559721571317, "grad_norm": 0.23364880681037903, "lr": 2e-05, "epoch": 1, "t": 3.4744250774383545}
+{"step": 3, "loss": 0.17497414473675157, "grad_norm": 0.20967614650726318, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 3.6915862560272217}
+{"step": 4, "loss": 0.204632010499747, "grad_norm": 0.22634324431419373, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 3.7792999744415283}
+{"step": 5, "loss": 0.11651496056022995, "grad_norm": 0.19220536947250366, "lr": 4e-05, "epoch": 1, "t": 3.4765713214874268}
+{"step": 6, "loss": 0.16099536127453554, "grad_norm": 0.20982958376407623, "lr": 4.666666666666667e-05, "epoch": 1, "t": 4.232372522354126}
+{"step": 7, "loss": 0.12886918164645858, "grad_norm": 0.20805878937244415, "lr": 5.333333333333333e-05, "epoch": 1, "t": 3.860217571258545}
+{"step": 8, "loss": 0.12935170543114868, "grad_norm": 0.4651066064834595, "lr": 6e-05, "epoch": 1, "t": 3.463470458984375}
+{"step": 9, "loss": 0.11179185816470315, "grad_norm": 0.38916945457458496, "lr": 6.666666666666667e-05, "epoch": 1, "t": 3.538846969604492}
+{"step": 10, "loss": 0.12125665095753181, "grad_norm": 0.3735863268375397, "lr": 7.333333333333333e-05, "epoch": 1, "t": 3.385127067565918}
+{"step": 11, "loss": 0.11843738652374275, "grad_norm": 0.26327192783355713, "lr": 8e-05, "epoch": 1, "t": 3.9777579307556152}
+{"step": 12, "loss": 0.08990358757178551, "grad_norm": 0.14185337722301483, "lr": 8.666666666666667e-05, "epoch": 1, "t": 4.729368209838867}
+{"step": 13, "loss": 0.11461541212772726, "grad_norm": 0.21701039373874664, "lr": 9.333333333333334e-05, "epoch": 1, "t": 3.771496295928955}
+{"step": 14, "loss": 0.10833790739338797, "grad_norm": 0.16273054480552673, "lr": 0.0001, "epoch": 1, "t": 4.154038190841675}
+{"step": 15, "loss": 0.13766709521312578, "grad_norm": 0.21055258810520172, "lr": 0.0001, "epoch": 1, "t": 4.207071781158447}
+{"step": 16, "loss": 0.1742510958016331, "grad_norm": 0.24513991177082062, "lr": 9.999687519737639e-05, "epoch": 1, "t": 4.1982574462890625}
+{"step": 17, "loss": 0.12739159785462056, "grad_norm": 0.245813250541687, "lr": 9.998750118008115e-05, "epoch": 1, "t": 3.784621000289917}
+{"step": 18, "loss": 0.12097155412234883, "grad_norm": 0.1952792853116989, "lr": 9.997187911979252e-05, "epoch": 1, "t": 4.053454637527466}
+{"step": 19, "loss": 0.09529064259659706, "grad_norm": 0.19176709651947021, "lr": 9.995001096914461e-05, "epoch": 1, "t": 3.2699666023254395}
+{"step": 20, "loss": 0.13520945641767346, "grad_norm": 0.1721249520778656, "lr": 9.992189946148366e-05, "epoch": 1, "t": 3.9897584915161133}
+{"step": 21, "loss": 0.12146807399296491, "grad_norm": 0.13869290053844452, "lr": 9.988754811052616e-05, "epoch": 1, "t": 4.1643195152282715}
+{"step": 22, "loss": 0.10420405571056171, "grad_norm": 0.20529043674468994, "lr": 9.984696120991978e-05, "epoch": 1, "t": 3.5122292041778564}
+{"step": 23, "loss": 0.107207302795404, "grad_norm": 0.1987605094909668, "lr": 9.980014383270668e-05, "epoch": 1, "t": 3.384214401245117}
+{"step": 24, "loss": 0.1062149404011738, "grad_norm": 0.19853179156780243, "lr": 9.974710183068935e-05, "epoch": 1, "t": 3.897686243057251}
+{"step": 25, "loss": 0.13547135334630167, "grad_norm": 0.18667401373386383, "lr": 9.968784183369929e-05, "epoch": 1, "t": 4.434913396835327}
+{"step": 26, "loss": 0.12929852258001645, "grad_norm": 0.21335098147392273, "lr": 9.962237124876828e-05, "epoch": 1, "t": 4.701762676239014}
+{"step": 27, "loss": 0.09664807518064898, "grad_norm": 0.2070603221654892, "lr": 9.955069825920249e-05, "epoch": 1, "t": 4.52167272567749}
+{"step": 28, "loss": 0.11024529359148248, "grad_norm": 0.18113212287425995, "lr": 9.947283182355982e-05, "epoch": 1, "t": 4.40445876121521}
+{"step": 29, "loss": 0.10593403116325201, "grad_norm": 0.17289294302463531, "lr": 9.938878167452992e-05, "epoch": 1, "t": 3.615321159362793}
+{"step": 30, "loss": 0.1510395015998314, "grad_norm": 0.20101875066757202, "lr": 9.929855831771786e-05, "epoch": 1, "t": 3.89733624458313}
+{"step": 31, "loss": 0.14514561506515594, "grad_norm": 0.2027379274368286, "lr": 9.92021730303309e-05, "epoch": 1, "t": 3.9142441749572754}
+{"step": 32, "loss": 0.13289011251635668, "grad_norm": 0.16887794435024261, "lr": 9.909963785976903e-05, "epoch": 1, "t": 3.8799428939819336}
+{"step": 33, "loss": 0.12125646577636087, "grad_norm": 0.1650603860616684, "lr": 9.899096562211902e-05, "epoch": 1, "t": 3.6407015323638916}
+{"step": 34, "loss": 0.10937807902926173, "grad_norm": 0.15100771188735962, "lr": 9.887616990055262e-05, "epoch": 1, "t": 3.3848705291748047}
+{"step": 35, "loss": 0.08009695228265257, "grad_norm": 0.14453229308128357, "lr": 9.875526504362869e-05, "epoch": 1, "t": 3.3952648639678955}
+{"step": 36, "loss": 0.09728163084813526, "grad_norm": 0.17014355957508087, "lr": 9.86282661634998e-05, "epoch": 1, "t": 3.2220726013183594}
+{"step": 37, "loss": 0.12435455062489131, "grad_norm": 0.1548335999250412, "lr": 9.849518913402334e-05, "epoch": 1, "t": 3.9710559844970703}
+{"step": 38, "loss": 0.11001586299270645, "grad_norm": 0.12557590007781982, "lr": 9.835605058877729e-05, "epoch": 1, "t": 4.681970834732056}
+{"step": 39, "loss": 0.11056043971164217, "grad_norm": 0.16947577893733978, "lr": 9.821086791898134e-05, "epoch": 1, "t": 3.826080799102783}
+{"step": 40, "loss": 0.09428102106920791, "grad_norm": 0.14116881787776947, "lr": 9.805965927132295e-05, "epoch": 1, "t": 3.670837879180908}
+{"step": 41, "loss": 0.11016983140357191, "grad_norm": 0.16089925169944763, "lr": 9.79024435456893e-05, "epoch": 1, "t": 4.0171959400177}
+{"step": 42, "loss": 0.1112852391607323, "grad_norm": 0.16575853526592255, "lr": 9.773924039280487e-05, "epoch": 1, "t": 3.63329815864563}
+{"step": 43, "loss": 0.1266313884004809, "grad_norm": 0.19760814309120178, "lr": 9.75700702117753e-05, "epoch": 1, "t": 3.385127067565918}
+{"step": 44, "loss": 0.1265330610129546, "grad_norm": 0.16791944205760956, "lr": 9.739495414753753e-05, "epoch": 1, "t": 3.3876383304595947}
+{"step": 45, "loss": 0.12107302274318099, "grad_norm": 0.16215652227401733, "lr": 9.721391408821711e-05, "epoch": 1, "t": 3.5256800651550293}
+{"step": 46, "loss": 0.1525257085084557, "grad_norm": 0.16772934794425964, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.954650640487671}
+{"step": 47, "loss": 0.11210755122216393, "grad_norm": 0.16573384404182434, "lr": 9.683415323626485e-05, "epoch": 1, "t": 3.5241305828094482}
+{"step": 48, "loss": 0.14413548020480085, "grad_norm": 0.17593026161193848, "lr": 9.663547991074127e-05, "epoch": 1, "t": 4.260246276855469}
+{"step": 49, "loss": 0.12759559832150716, "grad_norm": 0.1635892689228058, "lr": 9.643097751841854e-05, "epoch": 1, "t": 3.933316707611084}
+{"step": 50, "loss": 0.12589194989248206, "grad_norm": 0.16893312335014343, "lr": 9.622067162048112e-05, "epoch": 1, "t": 3.667961597442627}
+{"step": 51, "loss": 0.10447374599248527, "grad_norm": 0.16191372275352478, "lr": 9.600458850350588e-05, "epoch": 1, "t": 3.3186771869659424}
+{"step": 52, "loss": 0.10737215579540318, "grad_norm": 0.16775815188884735, "lr": 9.578275517617645e-05, "epoch": 1, "t": 3.5442042350769043}
+{"step": 53, "loss": 0.12345107565066912, "grad_norm": 0.14673423767089844, "lr": 9.555519936590738e-05, "epoch": 1, "t": 3.723632574081421}
+{"step": 54, "loss": 0.13054679360283564, "grad_norm": 0.11553743481636047, "lr": 9.532194951537838e-05, "epoch": 1, "t": 4.638525009155273}
+{"step": 55, "loss": 0.10780775627795015, "grad_norm": 0.14634189009666443, "lr": 9.508303477897924e-05, "epoch": 1, "t": 3.7213563919067383}
+{"step": 56, "loss": 0.0990670507255874, "grad_norm": 0.14224807918071747, "lr": 9.483848501916578e-05, "epoch": 1, "t": 3.953137159347534}
+{"step": 57, "loss": 0.11086214251334628, "grad_norm": 0.18013328313827515, "lr": 9.458833080272722e-05, "epoch": 1, "t": 3.2673652172088623}
+{"step": 58, "loss": 0.11148871626269062, "grad_norm": 0.1812731921672821, "lr": 9.433260339696563e-05, "epoch": 1, "t": 3.6809377670288086}
+{"step": 59, "loss": 0.10122281993880024, "grad_norm": 0.1716298609972, "lr": 9.407133476578778e-05, "epoch": 1, "t": 3.206233024597168}
+{"step": 60, "loss": 0.10897164725937478, "grad_norm": 0.11063706129789352, "lr": 9.38045575657098e-05, "epoch": 1, "t": 4.484192848205566}
+{"step": 61, "loss": 0.1137744700423865, "grad_norm": 0.13552944362163544, "lr": 9.353230514177552e-05, "epoch": 1, "t": 3.918484926223755}
+{"step": 62, "loss": 0.1319413922094152, "grad_norm": 0.1636483371257782, "lr": 9.325461152338846e-05, "epoch": 1, "t": 3.8026087284088135}
+{"step": 63, "loss": 0.13084154503982, "grad_norm": 0.15306773781776428, "lr": 9.297151142005851e-05, "epoch": 1, "t": 3.8550732135772705}
+{"step": 64, "loss": 0.10244943144398803, "grad_norm": 0.18862880766391754, "lr": 9.268304021706349e-05, "epoch": 1, "t": 3.5463528633117676}
+{"step": 65, "loss": 0.10797899019864439, "grad_norm": 0.160519078373909, "lr": 9.23892339710263e-05, "epoch": 1, "t": 3.2105016708374023}
+{"step": 66, "loss": 0.10990944685705069, "grad_norm": 0.17689108848571777, "lr": 9.209012940540805e-05, "epoch": 1, "t": 3.275942325592041}
+{"step": 67, "loss": 0.09344126955048682, "grad_norm": 0.13527144491672516, "lr": 9.178576390591802e-05, "epoch": 1, "t": 3.8280608654022217}
+{"step": 68, "loss": 0.12464023129065267, "grad_norm": 0.15842996537685394, "lr": 9.147617551584066e-05, "epoch": 1, "t": 3.313157558441162}
+{"step": 69, "loss": 0.1400983711903008, "grad_norm": 0.14534828066825867, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.7678823471069336}
+{"step": 70, "loss": 0.10210269410275055, "grad_norm": 0.15438738465309143, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.1848502159118652}
+{"step": 71, "loss": 0.07661306874150367, "grad_norm": 0.17104151844978333, "lr": 9.051646319812918e-05, "epoch": 1, "t": 2.98986554145813}
+{"step": 72, "loss": 0.10661032826041465, "grad_norm": 0.13440456986427307, "lr": 9.018637666191283e-05, "epoch": 1, "t": 3.9713237285614014}
+{"step": 73, "loss": 0.10432441082467911, "grad_norm": 0.1482425481081009, "lr": 8.985126714588738e-05, "epoch": 1, "t": 3.557973623275757}
+{"step": 74, "loss": 0.1238847538602171, "grad_norm": 0.14809198677539825, "lr": 8.951117653609666e-05, "epoch": 1, "t": 3.390277147293091}
+{"step": 75, "loss": 0.1122590472013888, "grad_norm": 0.1635098159313202, "lr": 8.916614734118184e-05, "epoch": 1, "t": 3.4320857524871826}
+{"step": 76, "loss": 0.13661326547560182, "grad_norm": 0.14469100534915924, "lr": 8.881622268706825e-05, "epoch": 1, "t": 4.279656410217285}
+{"step": 77, "loss": 0.11941998813654375, "grad_norm": 0.18440619111061096, "lr": 8.8461446311575e-05, "epoch": 1, "t": 4.537419557571411}
+{"step": 78, "loss": 0.08476884090427277, "grad_norm": 0.14126811921596527, "lr": 8.810186255894803e-05, "epoch": 1, "t": 4.6205949783325195}
+{"step": 79, "loss": 0.12916574960397942, "grad_norm": 0.17544221878051758, "lr": 8.773751637431748e-05, "epoch": 1, "t": 3.857515573501587}
+{"step": 80, "loss": 0.14130354336849535, "grad_norm": 0.16797061264514923, "lr": 8.736845329807993e-05, "epoch": 1, "t": 3.925441265106201}
+{"step": 81, "loss": 0.11135427800534706, "grad_norm": 0.14119695127010345, "lr": 8.69947194602061e-05, "epoch": 1, "t": 4.195903778076172}
+{"step": 82, "loss": 0.13396801921189672, "grad_norm": 0.14419060945510864, "lr": 8.66163615744751e-05, "epoch": 1, "t": 3.849079132080078}
+{"step": 83, "loss": 0.11296317051142447, "grad_norm": 0.15513963997364044, "lr": 8.623342693263548e-05, "epoch": 1, "t": 3.355229377746582}
+{"step": 84, "loss": 0.08661281536845047, "grad_norm": 0.13548099994659424, "lr": 8.584596339849417e-05, "epoch": 1, "t": 3.526108741760254}
+{"step": 85, "loss": 0.09135290348168575, "grad_norm": 0.19261306524276733, "lr": 8.545401940193392e-05, "epoch": 1, "t": 2.984166145324707}
+{"step": 86, "loss": 0.10952385620992693, "grad_norm": 0.13976703584194183, "lr": 8.505764393285984e-05, "epoch": 1, "t": 3.5535824298858643}
+{"step": 87, "loss": 0.14557392045180298, "grad_norm": 0.1694628745317459, "lr": 8.46568865350762e-05, "epoch": 1, "t": 3.5990333557128906}
+{"step": 88, "loss": 0.10164590638046511, "grad_norm": 0.1488981395959854, "lr": 8.425179730009368e-05, "epoch": 1, "t": 3.4452931880950928}
+{"step": 89, "loss": 0.13194008773195137, "grad_norm": 0.15823884308338165, "lr": 8.384242686086848e-05, "epoch": 1, "t": 3.178811550140381}
+{"step": 90, "loss": 0.1182580343231183, "grad_norm": 0.14421334862709045, "lr": 8.342882638547351e-05, "epoch": 1, "t": 4.199359893798828}
+{"step": 91, "loss": 0.10432949794649761, "grad_norm": 0.17614327371120453, "lr": 8.301104757070274e-05, "epoch": 1, "t": 3.37158203125}
+{"step": 92, "loss": 0.13229991603175234, "grad_norm": 0.1402427852153778, "lr": 8.258914263560971e-05, "epoch": 1, "t": 3.9561896324157715}
+{"step": 93, "loss": 0.13361990457827339, "grad_norm": 0.12814712524414062, "lr": 8.216316431498028e-05, "epoch": 1, "t": 4.0001397132873535}
+{"step": 94, "loss": 0.10326065613522957, "grad_norm": 0.145774707198143, "lr": 8.173316585274145e-05, "epoch": 1, "t": 3.5337600708007812}
+{"step": 95, "loss": 0.11687026024710653, "grad_norm": 0.15524375438690186, "lr": 8.129920099530607e-05, "epoch": 1, "t": 3.41689395904541}
+{"step": 96, "loss": 0.12639098455995235, "grad_norm": 0.16281411051750183, "lr": 8.086132398485524e-05, "epoch": 1, "t": 3.761617660522461}
+{"step": 97, "loss": 0.13418690963610622, "grad_norm": 0.15245212614536285, "lr": 8.041958955255814e-05, "epoch": 1, "t": 3.5629351139068604}
+{"step": 98, "loss": 0.12855583727213277, "grad_norm": 0.15186291933059692, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.5358684062957764}
+{"step": 99, "loss": 0.10821944522764808, "grad_norm": 0.16591374576091766, "lr": 7.952476975093729e-05, "epoch": 1, "t": 3.1594560146331787}
+{"step": 100, "loss": 0.12937020618675985, "grad_norm": 0.1519843488931656, "lr": 7.907179622702408e-05, "epoch": 1, "t": 3.855557680130005}
+{"step": 101, "loss": 0.11914189492448063, "grad_norm": 0.15177477896213531, "lr": 7.861518895810596e-05, "epoch": 1, "t": 4.034132480621338}
+{"step": 102, "loss": 0.12297839679610956, "grad_norm": 0.15511254966259003, "lr": 7.815500501648653e-05, "epoch": 1, "t": 3.4962990283966064}
+{"step": 103, "loss": 0.14257637723670719, "grad_norm": 0.16320064663887024, "lr": 7.769130192152538e-05, "epoch": 1, "t": 3.4730000495910645}
+{"step": 104, "loss": 0.09703979525961297, "grad_norm": 0.13380314409732819, "lr": 7.722413763244838e-05, "epoch": 1, "t": 3.5086116790771484}
+{"step": 105, "loss": 0.13507848403251038, "grad_norm": 0.12805311381816864, "lr": 7.675357054110336e-05, "epoch": 1, "t": 3.9506068229675293}
+{"step": 106, "loss": 0.09567153474089109, "grad_norm": 0.12483838200569153, "lr": 7.627965946466166e-05, "epoch": 1, "t": 4.394922733306885}
+{"step": 107, "loss": 0.12003377812298612, "grad_norm": 0.13472217321395874, "lr": 7.580246363826621e-05, "epoch": 1, "t": 3.708739995956421}
+{"step": 108, "loss": 0.1220643590072866, "grad_norm": 0.14558090269565582, "lr": 7.532204270762786e-05, "epoch": 1, "t": 3.4230144023895264}
+{"step": 109, "loss": 0.14116105077168892, "grad_norm": 0.13014569878578186, "lr": 7.483845672156998e-05, "epoch": 1, "t": 4.091485977172852}
+{"step": 110, "loss": 0.08574275115521997, "grad_norm": 0.14042261242866516, "lr": 7.435176612452286e-05, "epoch": 1, "t": 3.5168914794921875}
+{"step": 111, "loss": 0.1395919692692717, "grad_norm": 0.12726756930351257, "lr": 7.386203174896872e-05, "epoch": 1, "t": 4.305381536483765}
+{"step": 112, "loss": 0.13460505907162804, "grad_norm": 0.14867880940437317, "lr": 7.336931480783801e-05, "epoch": 1, "t": 3.775317907333374}
+{"step": 113, "loss": 0.14191844228599582, "grad_norm": 0.1327006071805954, "lr": 7.287367688685835e-05, "epoch": 1, "t": 4.227615594863892}
+{"step": 114, "loss": 0.11313186165600711, "grad_norm": 0.17264938354492188, "lr": 7.237517993685678e-05, "epoch": 1, "t": 3.40358829498291}
+{"step": 115, "loss": 0.12343028206929584, "grad_norm": 0.1653890609741211, "lr": 7.187388626601637e-05, "epoch": 1, "t": 3.426459550857544}
+{"step": 116, "loss": 0.11619156729377106, "grad_norm": 0.15662674605846405, "lr": 7.136985853208824e-05, "epoch": 1, "t": 3.4374125003814697}
+{"step": 117, "loss": 0.11738083732505818, "grad_norm": 0.17252522706985474, "lr": 7.086315973455981e-05, "epoch": 1, "t": 3.1830883026123047}
+{"step": 118, "loss": 0.10572253793909932, "grad_norm": 0.13038495182991028, "lr": 7.035385320678036e-05, "epoch": 1, "t": 3.5233724117279053}
+{"step": 119, "loss": 0.10977426650421954, "grad_norm": 0.15364061295986176, "lr": 6.984200260804484e-05, "epoch": 1, "t": 3.802640199661255}
+{"step": 120, "loss": 0.1093422730780837, "grad_norm": 0.20357976853847504, "lr": 6.932767191563703e-05, "epoch": 1, "t": 4.380675554275513}
+{"step": 121, "loss": 0.1045484145668675, "grad_norm": 0.13086271286010742, "lr": 6.881092541683278e-05, "epoch": 1, "t": 3.779622793197632}
+{"step": 122, "loss": 0.13944407211464183, "grad_norm": 0.11931382864713669, "lr": 6.829182770086474e-05, "epoch": 1, "t": 4.127443790435791}
+{"step": 123, "loss": 0.1216485819521956, "grad_norm": 0.1391938030719757, "lr": 6.777044365084907e-05, "epoch": 1, "t": 4.272121906280518}
+{"step": 124, "loss": 0.08845533058113948, "grad_norm": 0.1429576277732849, "lr": 6.724683843567568e-05, "epoch": 1, "t": 3.4381930828094482}
+{"step": 125, "loss": 0.14133558519955342, "grad_norm": 0.1538388431072235, "lr": 6.672107750186255e-05, "epoch": 1, "t": 4.2540199756622314}
+{"step": 126, "loss": 0.10935479196844855, "grad_norm": 0.16975554823875427, "lr": 6.619322656537552e-05, "epoch": 1, "t": 3.3424370288848877}
+{"step": 127, "loss": 0.0966669494360029, "grad_norm": 0.1335517019033432, "lr": 6.566335160341424e-05, "epoch": 1, "t": 3.4362728595733643}
+{"step": 128, "loss": 0.1106466664963519, "grad_norm": 0.1728857159614563, "lr": 6.513151884616556e-05, "epoch": 1, "t": 3.3640687465667725}
+{"step": 129, "loss": 0.11839029439450081, "grad_norm": 0.1679615080356598, "lr": 6.459779476852528e-05, "epoch": 1, "t": 3.3152811527252197}
+{"step": 130, "loss": 0.12230803298950195, "grad_norm": 0.22296029329299927, "lr": 6.406224608178932e-05, "epoch": 1, "t": 3.0517172813415527}
+{"step": 131, "loss": 0.11949064729125533, "grad_norm": 0.14384570717811584, "lr": 6.352493972531534e-05, "epoch": 1, "t": 3.408630132675171}
+{"step": 132, "loss": 0.1263493065303383, "grad_norm": 0.13814903795719147, "lr": 6.298594285815584e-05, "epoch": 1, "t": 4.063579797744751}
+{"step": 133, "loss": 0.10251842691568581, "grad_norm": 0.1444157510995865, "lr": 6.244532285066382e-05, "epoch": 1, "t": 3.284188747406006}
+{"step": 134, "loss": 0.12432801070326484, "grad_norm": 0.14464540779590607, "lr": 6.190314727607196e-05, "epoch": 1, "t": 3.5388126373291016}
+{"step": 135, "loss": 0.10833661114113462, "grad_norm": 0.1342194676399231, "lr": 6.13594839020466e-05, "epoch": 1, "t": 3.612874746322632}
+{"step": 136, "loss": 0.13015218541779872, "grad_norm": 0.15450303256511688, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 3.5266242027282715}
+{"step": 137, "loss": 0.1325876801840762, "grad_norm": 0.13618768751621246, "lr": 6.026796574768288e-05, "epoch": 1, "t": 3.693460702896118}
+{"step": 138, "loss": 0.13292357170804148, "grad_norm": 0.13599549233913422, "lr": 5.972024739849622e-05, "epoch": 1, "t": 3.896965503692627}
+{"step": 139, "loss": 0.10069719110127774, "grad_norm": 0.14850285649299622, "lr": 5.917131409512663e-05, "epoch": 1, "t": 3.456683397293091}
+{"step": 140, "loss": 0.11763671402484627, "grad_norm": 0.13814029097557068, "lr": 5.862123444990318e-05, "epoch": 1, "t": 3.6434223651885986}
+{"step": 141, "loss": 0.10329982654099734, "grad_norm": 0.11346394568681717, "lr": 5.807007721843861e-05, "epoch": 1, "t": 4.8335654735565186}
+{"step": 142, "loss": 0.10919911414841654, "grad_norm": 0.13511230051517487, "lr": 5.751791129103544e-05, "epoch": 1, "t": 3.508647918701172}
+{"step": 143, "loss": 0.09013594954667616, "grad_norm": 0.13255703449249268, "lr": 5.696480568407523e-05, "epoch": 1, "t": 4.571979284286499}
+{"step": 144, "loss": 0.11201232830958385, "grad_norm": 0.12285212427377701, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 4.593281507492065}
+{"step": 145, "loss": 0.09493424412422502, "grad_norm": 0.14504189789295197, "lr": 5.585605207563124e-05, "epoch": 1, "t": 4.004827976226807}
+{"step": 146, "loss": 0.10260508490850492, "grad_norm": 0.1330535113811493, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 4.194879055023193}
+{"step": 147, "loss": 0.09558413430962966, "grad_norm": 0.15524111688137054, "lr": 5.47443707175741e-05, "epoch": 1, "t": 3.1454453468322754}
+{"step": 148, "loss": 0.10277053196953581, "grad_norm": 0.1380491554737091, "lr": 5.418760576667071e-05, "epoch": 1, "t": 3.841887950897217}
+{"step": 149, "loss": 0.1251996829587077, "grad_norm": 0.13052290678024292, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 3.926555633544922}
+{"step": 150, "loss": 0.12740178446700706, "grad_norm": 0.13001331686973572, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 3.9569718837738037}
+{"step": 151, "loss": 0.12479383509017593, "grad_norm": 0.1642906367778778, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 3.849250316619873}
+{"step": 152, "loss": 0.16335803048148345, "grad_norm": 0.13915029168128967, "lr": 5.195600862779421e-05, "epoch": 1, "t": 4.0863096714019775}
+{"step": 153, "loss": 0.09080790493891241, "grad_norm": 0.14960356056690216, "lr": 5.139732367864736e-05, "epoch": 1, "t": 3.3119828701019287}
+{"step": 154, "loss": 0.13068811983135886, "grad_norm": 0.16442498564720154, "lr": 5.083846407507263e-05, "epoch": 1, "t": 3.1803221702575684}
+{"step": 155, "loss": 0.14691561216648077, "grad_norm": 0.1646917462348938, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 3.472726821899414}
+{"step": 156, "loss": 0.16095264992671288, "grad_norm": 0.1272423267364502, "lr": 4.972050032989175e-05, "epoch": 1, "t": 4.862588167190552}
+{"step": 157, "loss": 0.10955197239749742, "grad_norm": 0.14612393081188202, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 3.412879705429077}
+{"step": 158, "loss": 0.12855335587880914, "grad_norm": 0.15805663168430328, "lr": 4.860267632135265e-05, "epoch": 1, "t": 3.543646812438965}
+{"step": 159, "loss": 0.13200412708079087, "grad_norm": 0.1515091508626938, "lr": 4.80439913722058e-05, "epoch": 1, "t": 3.4432570934295654}
+{"step": 160, "loss": 0.11678993153018365, "grad_norm": 0.1300472766160965, "lr": 4.748555090869464e-05, "epoch": 1, "t": 3.7619271278381348}
+{"step": 161, "loss": 0.13907908323828125, "grad_norm": 0.1566210687160492, "lr": 4.692742473146818e-05, "epoch": 1, "t": 3.281759738922119}
+{"step": 162, "loss": 0.1117856704934787, "grad_norm": 0.1385830193758011, "lr": 4.636968260189214e-05, "epoch": 1, "t": 3.812715768814087}
+{"step": 163, "loss": 0.10569514615781156, "grad_norm": 0.12673906981945038, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 3.892843723297119}
+{"step": 164, "loss": 0.11934474993086899, "grad_norm": 0.14351478219032288, "lr": 4.525562928242592e-05, "epoch": 1, "t": 3.514590263366699}
+{"step": 165, "loss": 0.15151920682955078, "grad_norm": 0.13616399466991425, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 3.970243215560913}
+{"step": 166, "loss": 0.10842150242268284, "grad_norm": 0.15992829203605652, "lr": 4.414394792436877e-05, "epoch": 1, "t": 3.168083429336548}
+{"step": 167, "loss": 0.10101587686809821, "grad_norm": 0.19890455901622772, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 3.5013482570648193}
+{"step": 168, "loss": 0.11929447703910799, "grad_norm": 0.14415767788887024, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 3.28334379196167}
+{"step": 169, "loss": 0.07030365496345806, "grad_norm": 0.1760512739419937, "lr": 4.248208870896456e-05, "epoch": 1, "t": 3.1185290813446045}
+{"step": 170, "loss": 0.09043669806986372, "grad_norm": 0.21300236880779266, "lr": 4.192992278156141e-05, "epoch": 1, "t": 4.247017860412598}
+{"step": 171, "loss": 0.11614477936512944, "grad_norm": 0.1577676683664322, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 4.158295631408691}
+{"step": 172, "loss": 0.11680662288894215, "grad_norm": 0.14779634773731232, "lr": 4.082868590487339e-05, "epoch": 1, "t": 3.6678102016448975}
+{"step": 173, "loss": 0.1439566116828423, "grad_norm": 0.14909635484218597, "lr": 4.027975260150381e-05, "epoch": 1, "t": 4.114445209503174}
+{"step": 174, "loss": 0.13616295992317848, "grad_norm": 0.16079868376255035, "lr": 3.973203425231715e-05, "epoch": 1, "t": 3.460106372833252}
+{"step": 175, "loss": 0.16971466789061831, "grad_norm": 0.1433873474597931, "lr": 3.918559931778277e-05, "epoch": 1, "t": 3.885052442550659}
+{"step": 176, "loss": 0.12223552969680852, "grad_norm": 0.13315251469612122, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 3.7603070735931396}
+{"step": 177, "loss": 0.14583908790867575, "grad_norm": 0.12280987948179245, "lr": 3.809685272392804e-05, "epoch": 1, "t": 4.465679883956909}
+{"step": 178, "loss": 0.11113907486486749, "grad_norm": 0.14406894147396088, "lr": 3.755467714933619e-05, "epoch": 1, "t": 3.5301554203033447}
+{"step": 179, "loss": 0.1322877720565532, "grad_norm": 0.13965539634227753, "lr": 3.701405714184416e-05, "epoch": 1, "t": 3.6940371990203857}
+{"step": 180, "loss": 0.09924416713777484, "grad_norm": 0.18005260825157166, "lr": 3.647506027468467e-05, "epoch": 1, "t": 3.098369598388672}
+{"step": 181, "loss": 0.11962204150677905, "grad_norm": 0.1314098984003067, "lr": 3.59377539182107e-05, "epoch": 1, "t": 3.8052866458892822}
+{"step": 182, "loss": 0.10223573273856015, "grad_norm": 0.13426287472248077, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 3.4166765213012695}
+{"step": 183, "loss": 0.15071926326442017, "grad_norm": 0.1702321320772171, "lr": 3.486848115383445e-05, "epoch": 1, "t": 3.29227876663208}
+{"step": 184, "loss": 0.15374151013364873, "grad_norm": 0.14806592464447021, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 3.8606767654418945}
+{"step": 185, "loss": 0.09268654505072951, "grad_norm": 0.10478401184082031, "lr": 3.380677343462447e-05, "epoch": 1, "t": 4.618730068206787}
+{"step": 186, "loss": 0.138700299306747, "grad_norm": 0.14606443047523499, "lr": 3.327892249813745e-05, "epoch": 1, "t": 3.633486270904541}
+{"step": 187, "loss": 0.09405952455348954, "grad_norm": 0.1474747657775879, "lr": 3.275316156432434e-05, "epoch": 1, "t": 3.1387436389923096}
+{"step": 188, "loss": 0.11409046531049775, "grad_norm": 0.13750717043876648, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 3.9724977016448975}
+{"step": 189, "loss": 0.1208137828508798, "grad_norm": 0.13070982694625854, "lr": 3.170817229913526e-05, "epoch": 1, "t": 4.076906681060791}
+{"step": 190, "loss": 0.10746874493798568, "grad_norm": 0.13535048067569733, "lr": 3.118907458316722e-05, "epoch": 1, "t": 3.7697150707244873}
+{"step": 191, "loss": 0.13924930047347261, "grad_norm": 0.16778579354286194, "lr": 3.067232808436299e-05, "epoch": 1, "t": 3.637892484664917}
+{"step": 192, "loss": 0.1006704293539762, "grad_norm": 0.1751009225845337, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 3.874000310897827}
+{"step": 193, "loss": 0.1067328235740372, "grad_norm": 0.13739240169525146, "lr": 2.964614679321966e-05, "epoch": 1, "t": 3.7337472438812256}
+{"step": 194, "loss": 0.1028848203889198, "grad_norm": 0.15709304809570312, "lr": 2.913684026544021e-05, "epoch": 1, "t": 3.30153226852417}
+{"step": 195, "loss": 0.11686374964537444, "grad_norm": 0.12740200757980347, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 3.867953300476074}
+{"step": 196, "loss": 0.10592348299773652, "grad_norm": 0.1566307693719864, "lr": 2.812611373398365e-05, "epoch": 1, "t": 3.520291328430176}
+{"step": 197, "loss": 0.13518921005861373, "grad_norm": 0.15496093034744263, "lr": 2.762482006314324e-05, "epoch": 1, "t": 3.3473801612854004}
+{"step": 198, "loss": 0.1433411820107298, "grad_norm": 0.1502045840024948, "lr": 2.712632311314165e-05, "epoch": 1, "t": 3.8707895278930664}
+{"step": 199, "loss": 0.10977831129000801, "grad_norm": 0.11846183240413666, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 4.215899229049683}
+{"step": 200, "loss": 0.12741351947533083, "grad_norm": 0.1688779592514038, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 3.240912437438965}
+{"step": 201, "loss": 0.1446272865024716, "grad_norm": 0.1411464363336563, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 4.117337703704834}
+{"step": 202, "loss": 0.14842141097000902, "grad_norm": 0.15695078670978546, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 3.663425922393799}
+{"step": 203, "loss": 0.14411468854445425, "grad_norm": 0.14007613062858582, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 3.741201162338257}
+{"step": 204, "loss": 0.10635560741646, "grad_norm": 0.1643330305814743, "lr": 2.419753636173379e-05, "epoch": 1, "t": 3.4982082843780518}
+{"step": 205, "loss": 0.10817531549355042, "grad_norm": 0.1686583310365677, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 3.3264567852020264}
+{"step": 206, "loss": 0.13215300177904699, "grad_norm": 0.1402217000722885, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 3.86862850189209}
+{"step": 207, "loss": 0.11728140377268499, "grad_norm": 0.1389714926481247, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 3.629336357116699}
+{"step": 208, "loss": 0.10586343273296579, "grad_norm": 0.14154238998889923, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 3.5377798080444336}
+{"step": 209, "loss": 0.10963110953660435, "grad_norm": 0.13568945229053497, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 4.010490655899048}
+{"step": 210, "loss": 0.10643987963774187, "grad_norm": 0.13890759646892548, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 4.25322699546814}
+{"step": 211, "loss": 0.13280526872838563, "grad_norm": 0.12725253403186798, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 3.870941162109375}
+{"step": 212, "loss": 0.12167755425527615, "grad_norm": 0.15370523929595947, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 3.7713546752929688}
+{"step": 213, "loss": 0.11095563283395007, "grad_norm": 0.17512264847755432, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 3.4375109672546387}
+{"step": 214, "loss": 0.08743070517904981, "grad_norm": 0.15476514399051666, "lr": 1.958041044744186e-05, "epoch": 1, "t": 3.206904172897339}
+{"step": 215, "loss": 0.12892800486880257, "grad_norm": 0.1304374486207962, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 3.8659324645996094}
+{"step": 216, "loss": 0.12790023206981393, "grad_norm": 0.12677110731601715, "lr": 1.870079900469392e-05, "epoch": 1, "t": 3.8934192657470703}
+{"step": 217, "loss": 0.1305046852361358, "grad_norm": 0.12617672979831696, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 4.238474130630493}
+{"step": 218, "loss": 0.11851315623629465, "grad_norm": 0.17081411182880402, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 3.200692892074585}
+{"step": 219, "loss": 0.09172247057524051, "grad_norm": 0.13463811576366425, "lr": 1.741085736439031e-05, "epoch": 1, "t": 3.4426724910736084}
+{"step": 220, "loss": 0.1101080438527981, "grad_norm": 0.13909728825092316, "lr": 1.698895242929725e-05, "epoch": 1, "t": 3.3703041076660156}
+{"step": 221, "loss": 0.12862987842361998, "grad_norm": 0.13614295423030853, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 3.694084882736206}
+{"step": 222, "loss": 0.09355054361693918, "grad_norm": 0.12518799304962158, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 4.217477798461914}
+{"step": 223, "loss": 0.11741614689591703, "grad_norm": 0.1621903032064438, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 3.2743442058563232}
+{"step": 224, "loss": 0.10235106119408495, "grad_norm": 0.1561732441186905, "lr": 1.534311346492381e-05, "epoch": 1, "t": 3.2867138385772705}
+{"step": 225, "loss": 0.11554930600457253, "grad_norm": 0.13261538743972778, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 3.630746364593506}
+{"step": 226, "loss": 0.11764256136989164, "grad_norm": 0.14942531287670135, "lr": 1.454598059806609e-05, "epoch": 1, "t": 3.3132505416870117}
+{"step": 227, "loss": 0.09685231244310417, "grad_norm": 0.14665105938911438, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 3.386376142501831}
+{"step": 228, "loss": 0.09750865225282021, "grad_norm": 0.1696440428495407, "lr": 1.376657306736453e-05, "epoch": 1, "t": 3.034342050552368}
+{"step": 229, "loss": 0.1199682712904401, "grad_norm": 0.16452285647392273, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 3.265753746032715}
+{"step": 230, "loss": 0.12084051726618822, "grad_norm": 0.1560228019952774, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 3.5381014347076416}
+{"step": 231, "loss": 0.1347587171364178, "grad_norm": 0.14204195141792297, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 3.4959986209869385}
+{"step": 232, "loss": 0.1258565682930874, "grad_norm": 0.1305045634508133, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 4.402764797210693}
+{"step": 233, "loss": 0.1192134784292866, "grad_norm": 0.14969198405742645, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 3.485802412033081}
+{"step": 234, "loss": 0.1112173626892282, "grad_norm": 0.11909645050764084, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 3.7893333435058594}
+{"step": 235, "loss": 0.13302109847060317, "grad_norm": 0.1646585613489151, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 3.356138229370117}
+{"step": 236, "loss": 0.11040179540376438, "grad_norm": 0.13972122967243195, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 4.001006603240967}
+{"step": 237, "loss": 0.12199611756651794, "grad_norm": 0.15059305727481842, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 3.738616943359375}
+{"step": 238, "loss": 0.08927671258417666, "grad_norm": 0.13560353219509125, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 3.2053418159484863}
+{"step": 239, "loss": 0.10545098267152357, "grad_norm": 0.15266600251197815, "lr": 9.81362333808718e-06, "epoch": 1, "t": 3.583167791366577}
+{"step": 240, "loss": 0.12766806129465766, "grad_norm": 0.12897846102714539, "lr": 9.483536801870834e-06, "epoch": 1, "t": 4.164019823074341}
+{"step": 241, "loss": 0.10493027108170529, "grad_norm": 0.11020587384700775, "lr": 9.158514503674543e-06, "epoch": 1, "t": 4.539491415023804}
+{"step": 242, "loss": 0.13199888080858246, "grad_norm": 0.12931029498577118, "lr": 8.838597068719518e-06, "epoch": 1, "t": 3.829200506210327}
+{"step": 243, "loss": 0.11168599142315995, "grad_norm": 0.14943437278270721, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.6176085472106934}
+{"step": 244, "loss": 0.10901609270440517, "grad_norm": 0.13358907401561737, "lr": 8.21423609408199e-06, "epoch": 1, "t": 3.740696907043457}
+{"step": 245, "loss": 0.12050593032825523, "grad_norm": 0.12873904407024384, "lr": 7.90987059459195e-06, "epoch": 1, "t": 3.5556647777557373}
+{"step": 246, "loss": 0.13628450256764912, "grad_norm": 0.13476987183094025, "lr": 7.610766028973709e-06, "epoch": 1, "t": 3.8960635662078857}
+{"step": 247, "loss": 0.1348763423447733, "grad_norm": 0.1563112437725067, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 3.2677786350250244}
+{"step": 248, "loss": 0.13848721206898615, "grad_norm": 0.14059168100357056, "lr": 7.028488579941506e-06, "epoch": 1, "t": 3.844301700592041}
+{"step": 249, "loss": 0.12796160552830324, "grad_norm": 0.1548541784286499, "lr": 6.745388476611553e-06, "epoch": 1, "t": 3.501514196395874}
+{"step": 250, "loss": 0.16296225215696675, "grad_norm": 0.14525030553340912, "lr": 6.467694858224488e-06, "epoch": 1, "t": 3.939232349395752}
+{"step": 251, "loss": 0.1070016621272447, "grad_norm": 0.15758128464221954, "lr": 6.1954424342902e-06, "epoch": 1, "t": 3.283404588699341}
+{"step": 252, "loss": 0.09574083992088261, "grad_norm": 0.15095897018909454, "lr": 5.928665234212233e-06, "epoch": 1, "t": 3.4166765213012695}
+{"step": 253, "loss": 0.10966365841822538, "grad_norm": 0.1345551759004593, "lr": 5.66739660303437e-06, "epoch": 1, "t": 3.7250568866729736}
+{"step": 254, "loss": 0.09611285997488792, "grad_norm": 0.14092585444450378, "lr": 5.411669197272795e-06, "epoch": 1, "t": 3.385288715362549}
+{"step": 255, "loss": 0.11935271253188451, "grad_norm": 0.1296173483133316, "lr": 5.161514980834231e-06, "epoch": 1, "t": 3.7156617641448975}
+{"step": 256, "loss": 0.13161281697919264, "grad_norm": 0.14366495609283447, "lr": 4.916965221020753e-06, "epoch": 1, "t": 3.852792978286743}
+{"step": 257, "loss": 0.1209236756655604, "grad_norm": 0.14871057868003845, "lr": 4.678050484621615e-06, "epoch": 1, "t": 3.8947365283966064}
+{"step": 258, "loss": 0.12307893084022183, "grad_norm": 0.13017506897449493, "lr": 4.444800634092616e-06, "epoch": 1, "t": 4.502076148986816}
+{"step": 259, "loss": 0.11447619316273165, "grad_norm": 0.15292863547801971, "lr": 4.217244823823546e-06, "epoch": 1, "t": 4.605961799621582}
+{"step": 260, "loss": 0.12097488539526408, "grad_norm": 0.11993365734815598, "lr": 3.995411496494134e-06, "epoch": 1, "t": 4.687212228775024}
+{"step": 261, "loss": 0.11480207224385239, "grad_norm": 0.14110656082630157, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 3.5892508029937744}
+{"step": 262, "loss": 0.11891526646084255, "grad_norm": 0.13634957373142242, "lr": 3.56902248158148e-06, "epoch": 1, "t": 3.4926207065582275}
+{"step": 263, "loss": 0.1530783237823215, "grad_norm": 0.12639865279197693, "lr": 3.364520089258727e-06, "epoch": 1, "t": 4.162294626235962}
+{"step": 264, "loss": 0.12786884148432492, "grad_norm": 0.14029482007026672, "lr": 3.165846763735153e-06, "epoch": 1, "t": 3.678027391433716}
+{"step": 265, "loss": 0.1237285930752495, "grad_norm": 0.14114855229854584, "lr": 2.973027337607892e-06, "epoch": 1, "t": 3.6342239379882812}
+{"step": 266, "loss": 0.11090585836909286, "grad_norm": 0.1339404582977295, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.5305957794189453}
+{"step": 267, "loss": 0.12733383761028372, "grad_norm": 0.14606599509716034, "lr": 2.605045852462473e-06, "epoch": 1, "t": 3.893146276473999}
+{"step": 268, "loss": 0.107157397250466, "grad_norm": 0.099535271525383, "lr": 2.429929788224722e-06, "epoch": 1, "t": 5.080975532531738}
+{"step": 269, "loss": 0.12242767289228458, "grad_norm": 0.14165058732032776, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 3.5753729343414307}
+{"step": 270, "loss": 0.11982983969429989, "grad_norm": 0.142112597823143, "lr": 2.097556454310701e-06, "epoch": 1, "t": 3.529874086380005}
+{"step": 271, "loss": 0.09422277698137317, "grad_norm": 0.10237899422645569, "lr": 1.940340728677059e-06, "epoch": 1, "t": 4.66553258895874}
+{"step": 272, "loss": 0.09499428776865165, "grad_norm": 0.16352294385433197, "lr": 1.789132081018674e-06, "epoch": 1, "t": 3.2254798412323}
+{"step": 273, "loss": 0.11338042724789597, "grad_norm": 0.12887120246887207, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 3.8569297790527344}
+{"step": 274, "loss": 0.09530952283500237, "grad_norm": 0.15959228575229645, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 3.108150005340576}
+{"step": 275, "loss": 0.12288021341652608, "grad_norm": 0.1428644359111786, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 3.425093412399292}
+{"step": 276, "loss": 0.09483741608124993, "grad_norm": 0.14235369861125946, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 3.2369916439056396}
+{"step": 277, "loss": 0.12986522896413077, "grad_norm": 0.14084431529045105, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 3.9885756969451904}
+{"step": 278, "loss": 0.11122081637465989, "grad_norm": 0.14086051285266876, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 3.7049126625061035}
+{"step": 279, "loss": 0.12497105278393641, "grad_norm": 0.12968751788139343, "lr": 9.003621402309814e-07, "epoch": 1, "t": 3.8743293285369873}
+{"step": 280, "loss": 0.10868154354054335, "grad_norm": 0.1389903575181961, "lr": 7.97826969669102e-07, "epoch": 1, "t": 4.061615943908691}
+{"step": 281, "loss": 0.1500183152596065, "grad_norm": 0.1261337548494339, "lr": 7.014416822821556e-07, "epoch": 1, "t": 4.23523473739624}
+{"step": 282, "loss": 0.1229639415017621, "grad_norm": 0.1511215716600418, "lr": 6.112183254700865e-07, "epoch": 1, "t": 3.613213539123535}
+{"step": 283, "loss": 0.11858719814975804, "grad_norm": 0.14535851776599884, "lr": 5.271681764401848e-07, "epoch": 1, "t": 3.834543228149414}
+{"step": 284, "loss": 0.09114660686737237, "grad_norm": 0.13256731629371643, "lr": 4.493017407975086e-07, "epoch": 1, "t": 3.9566617012023926}
+{"step": 285, "loss": 0.10078294660435916, "grad_norm": 0.1427164524793625, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 3.4041359424591064}
+{"step": 286, "loss": 0.11581811067885688, "grad_norm": 0.13529494404792786, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 3.4895572662353516}
+{"step": 287, "loss": 0.11425623186006113, "grad_norm": 0.11497080326080322, "lr": 2.528981693106558e-07, "epoch": 1, "t": 4.601522445678711}
+{"step": 288, "loss": 0.12194901172213048, "grad_norm": 0.13583925366401672, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 3.783123254776001}
+{"step": 289, "loss": 0.11977125895659814, "grad_norm": 0.1480838805437088, "lr": 1.530387900802177e-07, "epoch": 1, "t": 3.546213150024414}
+{"step": 290, "loss": 0.09492465852144535, "grad_norm": 0.13321956992149353, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 3.320997476577759}
+{"step": 291, "loss": 0.10183612593616266, "grad_norm": 0.14664624631404877, "lr": 7.81005385163458e-08, "epoch": 1, "t": 3.4242913722991943}
+{"step": 292, "loss": 0.1105042862398154, "grad_norm": 0.11725901067256927, "lr": 4.998903085539075e-08, "epoch": 1, "t": 4.113272428512573}
+{"step": 293, "loss": 0.10947503146022028, "grad_norm": 0.14590440690517426, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 3.3465287685394287}
+{"step": 294, "loss": 0.14048238720771788, "grad_norm": 0.12474696338176727, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 4.209888219833374}
+{"step": 295, "loss": 0.1111814421824259, "grad_norm": 0.12644560635089874, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 3.79003643989563}
+{"step": 296, "loss": 0.09432252660123201, "grad_norm": 0.11700820177793503, "lr": 0.0, "epoch": 1, "t": 4.482056379318237}
diff --git a/qwen3-14b/rand-s2/README.md b/qwen3-14b/rand-s2/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/rand-s2/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/rand-s2/adapter_config.json b/qwen3-14b/rand-s2/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..fdd8e1a5a4ea4ae1e35a24192da09d84d5d059fb
--- /dev/null
+++ b/qwen3-14b/rand-s2/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "up_proj",
+    "down_proj",
+    "k_proj",
+    "o_proj",
+    "q_proj",
+    "gate_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/rand-s2/adapter_model.safetensors b/qwen3-14b/rand-s2/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..da526bc4b49dc35c4ec5b497247066ac7b22fb04
--- /dev/null
+++ b/qwen3-14b/rand-s2/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c25d7c2c566c40718fdee0c75791c0f9ef7259219b0f3778a97302a23e775759
+size 1027680256
diff --git a/qwen3-14b/rand-s2/config.json b/qwen3-14b/rand-s2/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..5a4a33fdbe6530f479027bac27a070ee653b82b2
--- /dev/null
+++ b/qwen3-14b/rand-s2/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-rand-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/rand-s2",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 2,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/rand-s2/log.jsonl b/qwen3-14b/rand-s2/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..fed0457e9f187954d70a21a0023b5d384c4acec8
--- /dev/null
+++ b/qwen3-14b/rand-s2/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.13578857531387842, "grad_norm": 0.21949924528598785, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 4.696963787078857}
+{"step": 2, "loss": 0.1308646816520617, "grad_norm": 0.2020808458328247, "lr": 2e-05, "epoch": 1, "t": 3.643521547317505}
+{"step": 3, "loss": 0.1937888785151692, "grad_norm": 0.2350405752658844, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 4.2896811962127686}
+{"step": 4, "loss": 0.12311004331882701, "grad_norm": 0.1847860962152481, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 3.9819929599761963}
+{"step": 5, "loss": 0.13127481785360817, "grad_norm": 0.20118169486522675, "lr": 4e-05, "epoch": 1, "t": 3.8353540897369385}
+{"step": 6, "loss": 0.1568200083014235, "grad_norm": 0.19066284596920013, "lr": 4.666666666666667e-05, "epoch": 1, "t": 4.360365152359009}
+{"step": 7, "loss": 0.14269236120914827, "grad_norm": 0.29555732011795044, "lr": 5.333333333333333e-05, "epoch": 1, "t": 3.7673380374908447}
+{"step": 8, "loss": 0.13412275471555526, "grad_norm": 0.30898573994636536, "lr": 6e-05, "epoch": 1, "t": 3.8202574253082275}
+{"step": 9, "loss": 0.12263806486064172, "grad_norm": 0.32413533329963684, "lr": 6.666666666666667e-05, "epoch": 1, "t": 3.747953176498413}
+{"step": 10, "loss": 0.1620370062068353, "grad_norm": 0.263503760099411, "lr": 7.333333333333333e-05, "epoch": 1, "t": 3.9634904861450195}
+{"step": 11, "loss": 0.153848377709464, "grad_norm": 0.29335254430770874, "lr": 8e-05, "epoch": 1, "t": 3.905484437942505}
+{"step": 12, "loss": 0.12415499056649701, "grad_norm": 0.21230439841747284, "lr": 8.666666666666667e-05, "epoch": 1, "t": 4.194377899169922}
+{"step": 13, "loss": 0.08116459783601158, "grad_norm": 0.17434534430503845, "lr": 9.333333333333334e-05, "epoch": 1, "t": 3.717427968978882}
+{"step": 14, "loss": 0.13128556265898267, "grad_norm": 0.19108928740024567, "lr": 0.0001, "epoch": 1, "t": 3.9682366847991943}
+{"step": 15, "loss": 0.12309177812888042, "grad_norm": 0.22584286332130432, "lr": 0.0001, "epoch": 1, "t": 3.8348851203918457}
+{"step": 16, "loss": 0.1502568340432333, "grad_norm": 0.2244742512702942, "lr": 9.999687519737639e-05, "epoch": 1, "t": 3.971146821975708}
+{"step": 17, "loss": 0.133354980278976, "grad_norm": 0.16338308155536652, "lr": 9.998750118008115e-05, "epoch": 1, "t": 4.143666744232178}
+{"step": 18, "loss": 0.11240266890483373, "grad_norm": 0.19311679899692535, "lr": 9.997187911979252e-05, "epoch": 1, "t": 4.182871103286743}
+{"step": 19, "loss": 0.1317063850898724, "grad_norm": 0.19272246956825256, "lr": 9.995001096914461e-05, "epoch": 1, "t": 3.6123111248016357}
+{"step": 20, "loss": 0.10450301779911265, "grad_norm": 0.1560743898153305, "lr": 9.992189946148366e-05, "epoch": 1, "t": 3.8209898471832275}
+{"step": 21, "loss": 0.1253943654288233, "grad_norm": 0.19255302846431732, "lr": 9.988754811052616e-05, "epoch": 1, "t": 3.873692512512207}
+{"step": 22, "loss": 0.1387644869785247, "grad_norm": 0.25512459874153137, "lr": 9.984696120991978e-05, "epoch": 1, "t": 3.616891860961914}
+{"step": 23, "loss": 0.1314547191979175, "grad_norm": 0.25037091970443726, "lr": 9.980014383270668e-05, "epoch": 1, "t": 3.7762300968170166}
+{"step": 24, "loss": 0.1201198245992507, "grad_norm": 0.2060955911874771, "lr": 9.974710183068935e-05, "epoch": 1, "t": 3.682115316390991}
+{"step": 25, "loss": 0.10977531674354128, "grad_norm": 0.20782577991485596, "lr": 9.968784183369929e-05, "epoch": 1, "t": 4.310593605041504}
+{"step": 26, "loss": 0.11249810875284341, "grad_norm": 0.21150752902030945, "lr": 9.962237124876828e-05, "epoch": 1, "t": 3.578784465789795}
+{"step": 27, "loss": 0.11221072612223004, "grad_norm": 0.17390452325344086, "lr": 9.955069825920249e-05, "epoch": 1, "t": 3.533726215362549}
+{"step": 28, "loss": 0.1098665681281506, "grad_norm": 0.20451198518276215, "lr": 9.947283182355982e-05, "epoch": 1, "t": 3.6137022972106934}
+{"step": 29, "loss": 0.137415509752953, "grad_norm": 0.17413370311260223, "lr": 9.938878167452992e-05, "epoch": 1, "t": 3.864549160003662}
+{"step": 30, "loss": 0.13191067373102894, "grad_norm": 0.1805541068315506, "lr": 9.929855831771786e-05, "epoch": 1, "t": 3.8915188312530518}
+{"step": 31, "loss": 0.11191192282620291, "grad_norm": 0.15117141604423523, "lr": 9.92021730303309e-05, "epoch": 1, "t": 3.8059890270233154}
+{"step": 32, "loss": 0.12841115138976017, "grad_norm": 0.18338653445243835, "lr": 9.909963785976903e-05, "epoch": 1, "t": 3.6581461429595947}
+{"step": 33, "loss": 0.1264076090196446, "grad_norm": 0.16901975870132446, "lr": 9.899096562211902e-05, "epoch": 1, "t": 4.070311069488525}
+{"step": 34, "loss": 0.09209988401247911, "grad_norm": 0.2018745392560959, "lr": 9.887616990055262e-05, "epoch": 1, "t": 3.363349437713623}
+{"step": 35, "loss": 0.1205968028729189, "grad_norm": 0.13763001561164856, "lr": 9.875526504362869e-05, "epoch": 1, "t": 4.478177785873413}
+{"step": 36, "loss": 0.1233035814328414, "grad_norm": 0.20334592461585999, "lr": 9.86282661634998e-05, "epoch": 1, "t": 3.4445414543151855}
+{"step": 37, "loss": 0.11918795042180487, "grad_norm": 0.1491198092699051, "lr": 9.849518913402334e-05, "epoch": 1, "t": 4.206147909164429}
+{"step": 38, "loss": 0.10589214667706962, "grad_norm": 0.15731671452522278, "lr": 9.835605058877729e-05, "epoch": 1, "t": 3.764136791229248}
+{"step": 39, "loss": 0.11260827288801546, "grad_norm": 0.18342545628547668, "lr": 9.821086791898134e-05, "epoch": 1, "t": 3.4125900268554688}
+{"step": 40, "loss": 0.09794055209888519, "grad_norm": 0.14966489374637604, "lr": 9.805965927132295e-05, "epoch": 1, "t": 3.6113293170928955}
+{"step": 41, "loss": 0.1279278267177775, "grad_norm": 0.15533845126628876, "lr": 9.79024435456893e-05, "epoch": 1, "t": 4.282955884933472}
+{"step": 42, "loss": 0.14708331410553077, "grad_norm": 0.1497904360294342, "lr": 9.773924039280487e-05, "epoch": 1, "t": 3.9038443565368652}
+{"step": 43, "loss": 0.11391150941014061, "grad_norm": 0.17081066966056824, "lr": 9.75700702117753e-05, "epoch": 1, "t": 3.5134975910186768}
+{"step": 44, "loss": 0.14689304789879443, "grad_norm": 0.179362952709198, "lr": 9.739495414753753e-05, "epoch": 1, "t": 3.7362053394317627}
+{"step": 45, "loss": 0.07759286578470465, "grad_norm": 0.1732204109430313, "lr": 9.721391408821711e-05, "epoch": 1, "t": 3.2289340496063232}
+{"step": 46, "loss": 0.10922557410985478, "grad_norm": 0.17282982170581818, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.5083086490631104}
+{"step": 47, "loss": 0.11041531434382991, "grad_norm": 0.15563754737377167, "lr": 9.683415323626485e-05, "epoch": 1, "t": 3.5722992420196533}
+{"step": 48, "loss": 0.12553052567087847, "grad_norm": 0.13661083579063416, "lr": 9.663547991074127e-05, "epoch": 1, "t": 4.301147699356079}
+{"step": 49, "loss": 0.09752455058421508, "grad_norm": 0.10718145966529846, "lr": 9.643097751841854e-05, "epoch": 1, "t": 5.4431984424591064}
+{"step": 50, "loss": 0.11925437592784058, "grad_norm": 0.2245207279920578, "lr": 9.622067162048112e-05, "epoch": 1, "t": 4.420422077178955}
+{"step": 51, "loss": 0.09102871694543878, "grad_norm": 0.15430152416229248, "lr": 9.600458850350588e-05, "epoch": 1, "t": 3.537306547164917}
+{"step": 52, "loss": 0.11756076700557391, "grad_norm": 0.15928566455841064, "lr": 9.578275517617645e-05, "epoch": 1, "t": 4.211797475814819}
+{"step": 53, "loss": 0.11075219292728612, "grad_norm": 0.1798938661813736, "lr": 9.555519936590738e-05, "epoch": 1, "t": 3.863083839416504}
+{"step": 54, "loss": 0.13046828879452646, "grad_norm": 0.1662292629480362, "lr": 9.532194951537838e-05, "epoch": 1, "t": 4.0629119873046875}
+{"step": 55, "loss": 0.11278579672465812, "grad_norm": 0.12673574686050415, "lr": 9.508303477897924e-05, "epoch": 1, "t": 4.468085527420044}
+{"step": 56, "loss": 0.15421409520795268, "grad_norm": 0.14707092940807343, "lr": 9.483848501916578e-05, "epoch": 1, "t": 4.079181909561157}
+{"step": 57, "loss": 0.09943276368256863, "grad_norm": 0.1491464227437973, "lr": 9.458833080272722e-05, "epoch": 1, "t": 3.6010868549346924}
+{"step": 58, "loss": 0.12967320309217234, "grad_norm": 0.15496759116649628, "lr": 9.433260339696563e-05, "epoch": 1, "t": 3.658581495285034}
+{"step": 59, "loss": 0.11888925623177218, "grad_norm": 0.15271811187267303, "lr": 9.407133476578778e-05, "epoch": 1, "t": 4.058987140655518}
+{"step": 60, "loss": 0.12471388235641234, "grad_norm": 0.16546005010604858, "lr": 9.38045575657098e-05, "epoch": 1, "t": 3.9081685543060303}
+{"step": 61, "loss": 0.11491772129185789, "grad_norm": 0.13680776953697205, "lr": 9.353230514177552e-05, "epoch": 1, "t": 3.9684829711914062}
+{"step": 62, "loss": 0.15034887620054496, "grad_norm": 0.14714571833610535, "lr": 9.325461152338846e-05, "epoch": 1, "t": 4.549678802490234}
+{"step": 63, "loss": 0.1213356267224561, "grad_norm": 0.1534576267004013, "lr": 9.297151142005851e-05, "epoch": 1, "t": 3.9393160343170166}
+{"step": 64, "loss": 0.12318717684426501, "grad_norm": 0.18428443372249603, "lr": 9.268304021706349e-05, "epoch": 1, "t": 3.727342128753662}
+{"step": 65, "loss": 0.1090495872667516, "grad_norm": 0.12253328412771225, "lr": 9.23892339710263e-05, "epoch": 1, "t": 4.277964115142822}
+{"step": 66, "loss": 0.1105838218499837, "grad_norm": 0.1347341686487198, "lr": 9.209012940540805e-05, "epoch": 1, "t": 4.76844048500061}
+{"step": 67, "loss": 0.11593557666779926, "grad_norm": 0.17053630948066711, "lr": 9.178576390591802e-05, "epoch": 1, "t": 3.6471426486968994}
+{"step": 68, "loss": 0.15196758818431108, "grad_norm": 0.17423689365386963, "lr": 9.147617551584066e-05, "epoch": 1, "t": 4.143740892410278}
+{"step": 69, "loss": 0.1081383597364, "grad_norm": 0.13762818276882172, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.8866934776306152}
+{"step": 70, "loss": 0.12212079022460676, "grad_norm": 0.15572364628314972, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.6961028575897217}
+{"step": 71, "loss": 0.10170262184613801, "grad_norm": 0.1426490843296051, "lr": 9.051646319812918e-05, "epoch": 1, "t": 4.219207286834717}
+{"step": 72, "loss": 0.12516992074773825, "grad_norm": 0.15195009112358093, "lr": 9.018637666191283e-05, "epoch": 1, "t": 3.6509854793548584}
+{"step": 73, "loss": 0.14868984105739186, "grad_norm": 0.1391993910074234, "lr": 8.985126714588738e-05, "epoch": 1, "t": 4.39727520942688}
+{"step": 74, "loss": 0.10897666663428049, "grad_norm": 0.19142524898052216, "lr": 8.951117653609666e-05, "epoch": 1, "t": 3.3069686889648438}
+{"step": 75, "loss": 0.1375991617471084, "grad_norm": 0.1680697500705719, "lr": 8.916614734118184e-05, "epoch": 1, "t": 3.711686134338379}
+{"step": 76, "loss": 0.11174562264200005, "grad_norm": 0.15912441909313202, "lr": 8.881622268706825e-05, "epoch": 1, "t": 3.7656655311584473}
+{"step": 77, "loss": 0.14350387038072734, "grad_norm": 0.15803085267543793, "lr": 8.8461446311575e-05, "epoch": 1, "t": 4.017055034637451}
+{"step": 78, "loss": 0.14681308601529564, "grad_norm": 0.15922144055366516, "lr": 8.810186255894803e-05, "epoch": 1, "t": 3.911332130432129}
+{"step": 79, "loss": 0.12361860299404642, "grad_norm": 0.17147359251976013, "lr": 8.773751637431748e-05, "epoch": 1, "t": 3.711021900177002}
+{"step": 80, "loss": 0.11808889945373632, "grad_norm": 0.14067289233207703, "lr": 8.736845329807993e-05, "epoch": 1, "t": 3.946890115737915}
+{"step": 81, "loss": 0.16534701279363279, "grad_norm": 0.1612217128276825, "lr": 8.69947194602061e-05, "epoch": 1, "t": 3.9374477863311768}
+{"step": 82, "loss": 0.12657189524574855, "grad_norm": 0.13838638365268707, "lr": 8.66163615744751e-05, "epoch": 1, "t": 4.253204107284546}
+{"step": 83, "loss": 0.08942786412118577, "grad_norm": 0.15554533898830414, "lr": 8.623342693263548e-05, "epoch": 1, "t": 3.424344778060913}
+{"step": 84, "loss": 0.09904573928688376, "grad_norm": 0.14472796022891998, "lr": 8.584596339849417e-05, "epoch": 1, "t": 3.5939273834228516}
+{"step": 85, "loss": 0.128815213420337, "grad_norm": 0.14897122979164124, "lr": 8.545401940193392e-05, "epoch": 1, "t": 3.7643141746520996}
+{"step": 86, "loss": 0.1451780446623385, "grad_norm": 0.1414392590522766, "lr": 8.505764393285984e-05, "epoch": 1, "t": 4.350998401641846}
+{"step": 87, "loss": 0.09966590911844471, "grad_norm": 0.14256276190280914, "lr": 8.46568865350762e-05, "epoch": 1, "t": 3.759603977203369}
+{"step": 88, "loss": 0.12330532073974614, "grad_norm": 0.1467614322900772, "lr": 8.425179730009368e-05, "epoch": 1, "t": 3.794584274291992}
+{"step": 89, "loss": 0.1294858372947613, "grad_norm": 0.1425437331199646, "lr": 8.384242686086848e-05, "epoch": 1, "t": 4.1595728397369385}
+{"step": 90, "loss": 0.08299110458616465, "grad_norm": 0.1347617506980896, "lr": 8.342882638547351e-05, "epoch": 1, "t": 3.394134521484375}
+{"step": 91, "loss": 0.12650071137101593, "grad_norm": 0.18218575417995453, "lr": 8.301104757070274e-05, "epoch": 1, "t": 3.9021382331848145}
+{"step": 92, "loss": 0.10496579259145913, "grad_norm": 0.14033076167106628, "lr": 8.258914263560971e-05, "epoch": 1, "t": 3.9584126472473145}
+{"step": 93, "loss": 0.1107838924526653, "grad_norm": 0.1493995189666748, "lr": 8.216316431498028e-05, "epoch": 1, "t": 4.106259346008301}
+{"step": 94, "loss": 0.15207847424628393, "grad_norm": 0.16125012934207916, "lr": 8.173316585274145e-05, "epoch": 1, "t": 3.990582227706909}
+{"step": 95, "loss": 0.12647123350023687, "grad_norm": 0.18506073951721191, "lr": 8.129920099530607e-05, "epoch": 1, "t": 3.5781819820404053}
+{"step": 96, "loss": 0.11534459583920117, "grad_norm": 0.13289795815944672, "lr": 8.086132398485524e-05, "epoch": 1, "t": 3.6773412227630615}
+{"step": 97, "loss": 0.11636546063930432, "grad_norm": 0.14538152515888214, "lr": 8.041958955255814e-05, "epoch": 1, "t": 3.931007146835327}
+{"step": 98, "loss": 0.11122897031739667, "grad_norm": 0.14649739861488342, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.669631004333496}
+{"step": 99, "loss": 0.09057932167092911, "grad_norm": 0.14140182733535767, "lr": 7.952476975093729e-05, "epoch": 1, "t": 3.9055371284484863}
+{"step": 100, "loss": 0.1222744940616507, "grad_norm": 0.13446108996868134, "lr": 7.907179622702408e-05, "epoch": 1, "t": 3.806731939315796}
+{"step": 101, "loss": 0.10380004350544692, "grad_norm": 0.16694869101047516, "lr": 7.861518895810596e-05, "epoch": 1, "t": 4.013736963272095}
+{"step": 102, "loss": 0.09626133334165238, "grad_norm": 0.1291244477033615, "lr": 7.815500501648653e-05, "epoch": 1, "t": 4.221741437911987}
+{"step": 103, "loss": 0.09389228091656582, "grad_norm": 0.16849099099636078, "lr": 7.769130192152538e-05, "epoch": 1, "t": 3.537161111831665}
+{"step": 104, "loss": 0.10780836750449402, "grad_norm": 0.1674833446741104, "lr": 7.722413763244838e-05, "epoch": 1, "t": 3.6449203491210938}
+{"step": 105, "loss": 0.08819254820889008, "grad_norm": 0.14653505384922028, "lr": 7.675357054110336e-05, "epoch": 1, "t": 3.6056015491485596}
+{"step": 106, "loss": 0.12152422775846745, "grad_norm": 0.1328543871641159, "lr": 7.627965946466166e-05, "epoch": 1, "t": 4.593210458755493}
+{"step": 107, "loss": 0.1467005483369082, "grad_norm": 0.15113893151283264, "lr": 7.580246363826621e-05, "epoch": 1, "t": 4.181498050689697}
+{"step": 108, "loss": 0.131558280067227, "grad_norm": 0.1662665158510208, "lr": 7.532204270762786e-05, "epoch": 1, "t": 3.6490182876586914}
+{"step": 109, "loss": 0.13312740543948187, "grad_norm": 0.16320575773715973, "lr": 7.483845672156998e-05, "epoch": 1, "t": 3.6600418090820312}
+{"step": 110, "loss": 0.1411705506439264, "grad_norm": 0.17738685011863708, "lr": 7.435176612452286e-05, "epoch": 1, "t": 3.5622079372406006}
+{"step": 111, "loss": 0.1188139473335068, "grad_norm": 0.16926416754722595, "lr": 7.386203174896872e-05, "epoch": 1, "t": 3.4750986099243164}
+{"step": 112, "loss": 0.13742305278309208, "grad_norm": 0.14371426403522491, "lr": 7.336931480783801e-05, "epoch": 1, "t": 3.800544261932373}
+{"step": 113, "loss": 0.1369903745646007, "grad_norm": 0.16399483382701874, "lr": 7.287367688685835e-05, "epoch": 1, "t": 3.483109712600708}
+{"step": 114, "loss": 0.09255099203014414, "grad_norm": 0.16656085848808289, "lr": 7.237517993685678e-05, "epoch": 1, "t": 3.574777126312256}
+{"step": 115, "loss": 0.13712242749594866, "grad_norm": 0.13982120156288147, "lr": 7.187388626601637e-05, "epoch": 1, "t": 3.8912644386291504}
+{"step": 116, "loss": 0.1259884187789289, "grad_norm": 0.16244055330753326, "lr": 7.136985853208824e-05, "epoch": 1, "t": 3.4949257373809814}
+{"step": 117, "loss": 0.09460821246269585, "grad_norm": 0.13875427842140198, "lr": 7.086315973455981e-05, "epoch": 1, "t": 3.395252227783203}
+{"step": 118, "loss": 0.1139133497538456, "grad_norm": 0.12172689288854599, "lr": 7.035385320678036e-05, "epoch": 1, "t": 4.05577540397644}
+{"step": 119, "loss": 0.10934524434627481, "grad_norm": 0.14775095880031586, "lr": 6.984200260804484e-05, "epoch": 1, "t": 4.173226356506348}
+{"step": 120, "loss": 0.12831438691874314, "grad_norm": 0.1607704758644104, "lr": 6.932767191563703e-05, "epoch": 1, "t": 3.8441884517669678}
+{"step": 121, "loss": 0.13131765064601436, "grad_norm": 0.14566123485565186, "lr": 6.881092541683278e-05, "epoch": 1, "t": 3.574270486831665}
+{"step": 122, "loss": 0.10998609975532249, "grad_norm": 0.1707545816898346, "lr": 6.829182770086474e-05, "epoch": 1, "t": 3.8468105792999268}
+{"step": 123, "loss": 0.09613040288759364, "grad_norm": 0.13486482203006744, "lr": 6.777044365084907e-05, "epoch": 1, "t": 4.131503343582153}
+{"step": 124, "loss": 0.09918517583809516, "grad_norm": 0.16609817743301392, "lr": 6.724683843567568e-05, "epoch": 1, "t": 3.9766712188720703}
+{"step": 125, "loss": 0.1374721993257237, "grad_norm": 0.16386964917182922, "lr": 6.672107750186255e-05, "epoch": 1, "t": 3.9367074966430664}
+{"step": 126, "loss": 0.11830752863571249, "grad_norm": 0.1721578985452652, "lr": 6.619322656537552e-05, "epoch": 1, "t": 3.586796283721924}
+{"step": 127, "loss": 0.14534424316515515, "grad_norm": 0.15439286828041077, "lr": 6.566335160341424e-05, "epoch": 1, "t": 4.047714710235596}
+{"step": 128, "loss": 0.14237466473597976, "grad_norm": 0.13846474885940552, "lr": 6.513151884616556e-05, "epoch": 1, "t": 4.341150760650635}
+{"step": 129, "loss": 0.1283751303468009, "grad_norm": 0.14749395847320557, "lr": 6.459779476852528e-05, "epoch": 1, "t": 3.8643476963043213}
+{"step": 130, "loss": 0.09594976423405839, "grad_norm": 0.13333618640899658, "lr": 6.406224608178932e-05, "epoch": 1, "t": 4.110458850860596}
+{"step": 131, "loss": 0.14402596196068426, "grad_norm": 0.13033701479434967, "lr": 6.352493972531534e-05, "epoch": 1, "t": 4.558952331542969}
+{"step": 132, "loss": 0.11302782565110474, "grad_norm": 0.15420673787593842, "lr": 6.298594285815584e-05, "epoch": 1, "t": 3.471635341644287}
+{"step": 133, "loss": 0.12644427712163678, "grad_norm": 0.14108216762542725, "lr": 6.244532285066382e-05, "epoch": 1, "t": 4.126889228820801}
+{"step": 134, "loss": 0.09498826599937003, "grad_norm": 0.13133059442043304, "lr": 6.190314727607196e-05, "epoch": 1, "t": 3.9983487129211426}
+{"step": 135, "loss": 0.14403155979748453, "grad_norm": 0.1655164211988449, "lr": 6.13594839020466e-05, "epoch": 1, "t": 3.7351326942443848}
+{"step": 136, "loss": 0.11084102319445013, "grad_norm": 0.1655992567539215, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 3.7363123893737793}
+{"step": 137, "loss": 0.0845484026631236, "grad_norm": 0.2001384198665619, "lr": 6.026796574768288e-05, "epoch": 1, "t": 3.1671395301818848}
+{"step": 138, "loss": 0.11909020269824747, "grad_norm": 0.16109265387058258, "lr": 5.972024739849622e-05, "epoch": 1, "t": 3.521636724472046}
+{"step": 139, "loss": 0.11670461510795513, "grad_norm": 0.14545971155166626, "lr": 5.917131409512663e-05, "epoch": 1, "t": 3.649286985397339}
+{"step": 140, "loss": 0.13178009274149544, "grad_norm": 0.13206234574317932, "lr": 5.862123444990318e-05, "epoch": 1, "t": 4.298867225646973}
+{"step": 141, "loss": 0.10886960519268099, "grad_norm": 0.1581381857395172, "lr": 5.807007721843861e-05, "epoch": 1, "t": 3.60166597366333}
+{"step": 142, "loss": 0.11977821110727824, "grad_norm": 0.17158907651901245, "lr": 5.751791129103544e-05, "epoch": 1, "t": 3.6680667400360107}
+{"step": 143, "loss": 0.1580855659045498, "grad_norm": 0.15294139087200165, "lr": 5.696480568407523e-05, "epoch": 1, "t": 3.7718770503997803}
+{"step": 144, "loss": 0.1270797841913195, "grad_norm": 0.14402632415294647, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 3.779677152633667}
+{"step": 145, "loss": 0.1168886546237103, "grad_norm": 0.13393132388591766, "lr": 5.585605207563124e-05, "epoch": 1, "t": 3.82968807220459}
+{"step": 146, "loss": 0.10483957843363935, "grad_norm": 0.15465472638607025, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 3.466855049133301}
+{"step": 147, "loss": 0.1462972526673051, "grad_norm": 0.13039563596248627, "lr": 5.47443707175741e-05, "epoch": 1, "t": 4.224565744400024}
+{"step": 148, "loss": 0.12165614980365425, "grad_norm": 0.16687080264091492, "lr": 5.418760576667071e-05, "epoch": 1, "t": 3.6870906352996826}
+{"step": 149, "loss": 0.09253470844143619, "grad_norm": 0.12571221590042114, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 3.8044228553771973}
+{"step": 150, "loss": 0.11667886796768473, "grad_norm": 0.13059963285923004, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 4.028956413269043}
+{"step": 151, "loss": 0.09435668588199976, "grad_norm": 0.13714224100112915, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 3.578810691833496}
+{"step": 152, "loss": 0.1127393395313701, "grad_norm": 0.16306214034557343, "lr": 5.195600862779421e-05, "epoch": 1, "t": 3.540367841720581}
+{"step": 153, "loss": 0.11398606812778304, "grad_norm": 0.15615014731884003, "lr": 5.139732367864736e-05, "epoch": 1, "t": 3.788630723953247}
+{"step": 154, "loss": 0.10801624425326357, "grad_norm": 0.1353117674589157, "lr": 5.083846407507263e-05, "epoch": 1, "t": 3.871051073074341}
+{"step": 155, "loss": 0.11375422473218225, "grad_norm": 0.16073407232761383, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 3.790372371673584}
+{"step": 156, "loss": 0.12310499710837831, "grad_norm": 0.15290899574756622, "lr": 4.972050032989175e-05, "epoch": 1, "t": 4.007934331893921}
+{"step": 157, "loss": 0.10939468936276263, "grad_norm": 0.15411576628684998, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 3.853719472885132}
+{"step": 158, "loss": 0.11454861571914271, "grad_norm": 0.18326862156391144, "lr": 4.860267632135265e-05, "epoch": 1, "t": 4.670073509216309}
+{"step": 159, "loss": 0.12073157793855013, "grad_norm": 0.14033329486846924, "lr": 4.80439913722058e-05, "epoch": 1, "t": 4.073339223861694}
+{"step": 160, "loss": 0.10222784066698525, "grad_norm": 0.14743591845035553, "lr": 4.748555090869464e-05, "epoch": 1, "t": 3.427717447280884}
+{"step": 161, "loss": 0.10839923629749453, "grad_norm": 0.12882719933986664, "lr": 4.692742473146818e-05, "epoch": 1, "t": 3.675891160964966}
+{"step": 162, "loss": 0.11060722087037009, "grad_norm": 0.1444602906703949, "lr": 4.636968260189214e-05, "epoch": 1, "t": 3.6307687759399414}
+{"step": 163, "loss": 0.09745315570700504, "grad_norm": 0.16120406985282898, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 3.2841203212738037}
+{"step": 164, "loss": 0.13518047790213664, "grad_norm": 0.14629605412483215, "lr": 4.525562928242592e-05, "epoch": 1, "t": 3.8513972759246826}
+{"step": 165, "loss": 0.10768296483248599, "grad_norm": 0.14583884179592133, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 3.702069044113159}
+{"step": 166, "loss": 0.10219340304982348, "grad_norm": 0.12065865844488144, "lr": 4.414394792436877e-05, "epoch": 1, "t": 4.8369481563568115}
+{"step": 167, "loss": 0.11343938582026557, "grad_norm": 0.1462782919406891, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 4.437898874282837}
+{"step": 168, "loss": 0.10913535254705035, "grad_norm": 0.1659395545721054, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 3.6980583667755127}
+{"step": 169, "loss": 0.15457543461879397, "grad_norm": 0.14526551961898804, "lr": 4.248208870896456e-05, "epoch": 1, "t": 4.379364967346191}
+{"step": 170, "loss": 0.13520547747255812, "grad_norm": 0.12755145132541656, "lr": 4.192992278156141e-05, "epoch": 1, "t": 4.213551998138428}
+{"step": 171, "loss": 0.10677907778424307, "grad_norm": 0.14612732827663422, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 3.9786202907562256}
+{"step": 172, "loss": 0.08134608213176242, "grad_norm": 0.16416573524475098, "lr": 4.082868590487339e-05, "epoch": 1, "t": 4.36967396736145}
+{"step": 173, "loss": 0.10075288599863827, "grad_norm": 0.13912546634674072, "lr": 4.027975260150381e-05, "epoch": 1, "t": 3.9529190063476562}
+{"step": 174, "loss": 0.1202054983663767, "grad_norm": 0.12142195552587509, "lr": 3.973203425231715e-05, "epoch": 1, "t": 4.2446608543396}
+{"step": 175, "loss": 0.12757922922625708, "grad_norm": 0.15026652812957764, "lr": 3.918559931778277e-05, "epoch": 1, "t": 3.834829092025757}
+{"step": 176, "loss": 0.12441637836152085, "grad_norm": 0.16532714664936066, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 3.4876058101654053}
+{"step": 177, "loss": 0.10924842050805104, "grad_norm": 0.17287477850914001, "lr": 3.809685272392804e-05, "epoch": 1, "t": 3.4982821941375732}
+{"step": 178, "loss": 0.12224337334594955, "grad_norm": 0.1485215127468109, "lr": 3.755467714933619e-05, "epoch": 1, "t": 3.719533920288086}
+{"step": 179, "loss": 0.11321638461615784, "grad_norm": 0.13741342723369598, "lr": 3.701405714184416e-05, "epoch": 1, "t": 3.72743821144104}
+{"step": 180, "loss": 0.12296512191165171, "grad_norm": 0.12941975891590118, "lr": 3.647506027468467e-05, "epoch": 1, "t": 4.348463296890259}
+{"step": 181, "loss": 0.12444613258298605, "grad_norm": 0.1355736255645752, "lr": 3.59377539182107e-05, "epoch": 1, "t": 3.8641726970672607}
+{"step": 182, "loss": 0.08758931673205744, "grad_norm": 0.12623488903045654, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 4.081145286560059}
+{"step": 183, "loss": 0.15403911797378297, "grad_norm": 0.14874321222305298, "lr": 3.486848115383445e-05, "epoch": 1, "t": 3.8182270526885986}
+{"step": 184, "loss": 0.1450807075188849, "grad_norm": 0.13569097220897675, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 4.235250473022461}
+{"step": 185, "loss": 0.12611287690744075, "grad_norm": 0.1729271560907364, "lr": 3.380677343462447e-05, "epoch": 1, "t": 3.4460654258728027}
+{"step": 186, "loss": 0.11257537519977119, "grad_norm": 0.1563958078622818, "lr": 3.327892249813745e-05, "epoch": 1, "t": 3.7314207553863525}
+{"step": 187, "loss": 0.11521579321433037, "grad_norm": 0.15222899615764618, "lr": 3.275316156432434e-05, "epoch": 1, "t": 3.6132020950317383}
+{"step": 188, "loss": 0.08430357241347072, "grad_norm": 0.15627312660217285, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 3.2945425510406494}
+{"step": 189, "loss": 0.12824636326493385, "grad_norm": 0.16432863473892212, "lr": 3.170817229913526e-05, "epoch": 1, "t": 3.8404314517974854}
+{"step": 190, "loss": 0.10394545823434828, "grad_norm": 0.14150318503379822, "lr": 3.118907458316722e-05, "epoch": 1, "t": 3.777862071990967}
+{"step": 191, "loss": 0.12545840389374358, "grad_norm": 0.12383189797401428, "lr": 3.067232808436299e-05, "epoch": 1, "t": 4.433390855789185}
+{"step": 192, "loss": 0.10462507467578958, "grad_norm": 0.13268303871154785, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 4.511511564254761}
+{"step": 193, "loss": 0.11872306634218263, "grad_norm": 0.15458202362060547, "lr": 2.964614679321966e-05, "epoch": 1, "t": 3.5451276302337646}
+{"step": 194, "loss": 0.10896520743442166, "grad_norm": 0.13386303186416626, "lr": 2.913684026544021e-05, "epoch": 1, "t": 4.118440866470337}
+{"step": 195, "loss": 0.11916484308766793, "grad_norm": 0.1346011906862259, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 3.946871519088745}
+{"step": 196, "loss": 0.11414169330813942, "grad_norm": 0.1404159963130951, "lr": 2.812611373398365e-05, "epoch": 1, "t": 3.625519275665283}
+{"step": 197, "loss": 0.11106807839723279, "grad_norm": 0.14683178067207336, "lr": 2.762482006314324e-05, "epoch": 1, "t": 3.671926975250244}
+{"step": 198, "loss": 0.11121213128370222, "grad_norm": 0.1496427357196808, "lr": 2.712632311314165e-05, "epoch": 1, "t": 3.7366559505462646}
+{"step": 199, "loss": 0.11037936070718377, "grad_norm": 0.16853703558444977, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 3.5477101802825928}
+{"step": 200, "loss": 0.11734546459082401, "grad_norm": 0.1213759183883667, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 4.114522218704224}
+{"step": 201, "loss": 0.12398164435928943, "grad_norm": 0.13922318816184998, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 3.7420241832733154}
+{"step": 202, "loss": 0.10632406866652086, "grad_norm": 0.1393289715051651, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 3.766274929046631}
+{"step": 203, "loss": 0.11007106200658465, "grad_norm": 0.13087406754493713, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 3.950860023498535}
+{"step": 204, "loss": 0.0987709314983562, "grad_norm": 0.13731980323791504, "lr": 2.419753636173379e-05, "epoch": 1, "t": 4.552009344100952}
+{"step": 205, "loss": 0.16526225141843962, "grad_norm": 0.1430990993976593, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 4.154602766036987}
+{"step": 206, "loss": 0.10443879863493756, "grad_norm": 0.10707799345254898, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 4.312841415405273}
+{"step": 207, "loss": 0.10168315772556019, "grad_norm": 0.11449329555034637, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 4.323340654373169}
+{"step": 208, "loss": 0.1468036767899315, "grad_norm": 0.15952178835868835, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 3.6453208923339844}
+{"step": 209, "loss": 0.12779256083676668, "grad_norm": 0.14389540255069733, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 4.031829595565796}
+{"step": 210, "loss": 0.11807253917638404, "grad_norm": 0.12178724259138107, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 4.3989386558532715}
+{"step": 211, "loss": 0.1052134488002936, "grad_norm": 0.16667629778385162, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 3.473600387573242}
+{"step": 212, "loss": 0.11253023582471613, "grad_norm": 0.12076006829738617, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 3.9341483116149902}
+{"step": 213, "loss": 0.11080648036897289, "grad_norm": 0.1520334780216217, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 3.5782368183135986}
+{"step": 214, "loss": 0.0953215759351635, "grad_norm": 0.16387003660202026, "lr": 1.958041044744186e-05, "epoch": 1, "t": 3.2767562866210938}
+{"step": 215, "loss": 0.09419370386310078, "grad_norm": 0.12937988340854645, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 3.636993885040283}
+{"step": 216, "loss": 0.11495178863797341, "grad_norm": 0.1385297328233719, "lr": 1.870079900469392e-05, "epoch": 1, "t": 4.1813507080078125}
+{"step": 217, "loss": 0.0953599008570938, "grad_norm": 0.1450909972190857, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 3.705714464187622}
+{"step": 218, "loss": 0.1405523826585108, "grad_norm": 0.15330204367637634, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 4.076984882354736}
+{"step": 219, "loss": 0.14643278144462535, "grad_norm": 0.14788569509983063, "lr": 1.741085736439031e-05, "epoch": 1, "t": 4.156996726989746}
+{"step": 220, "loss": 0.13124176461971243, "grad_norm": 0.15439791977405548, "lr": 1.698895242929725e-05, "epoch": 1, "t": 4.2245564460754395}
+{"step": 221, "loss": 0.1178461299457403, "grad_norm": 0.12057007104158401, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 4.440277099609375}
+{"step": 222, "loss": 0.12614002252385495, "grad_norm": 0.15780287981033325, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 4.71493935585022}
+{"step": 223, "loss": 0.11372124677189122, "grad_norm": 0.17729029059410095, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 5.187800168991089}
+{"step": 224, "loss": 0.13811774038568073, "grad_norm": 0.12902577221393585, "lr": 1.534311346492381e-05, "epoch": 1, "t": 5.500786066055298}
+{"step": 225, "loss": 0.12619876883226996, "grad_norm": 0.14498120546340942, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 5.161534786224365}
+{"step": 226, "loss": 0.13228675861789074, "grad_norm": 0.12251429259777069, "lr": 1.454598059806609e-05, "epoch": 1, "t": 5.512620210647583}
+{"step": 227, "loss": 0.13730591117704988, "grad_norm": 0.16250741481781006, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 5.199298143386841}
+{"step": 228, "loss": 0.12472689033311403, "grad_norm": 0.13780207931995392, "lr": 1.376657306736453e-05, "epoch": 1, "t": 5.368857145309448}
+{"step": 229, "loss": 0.1239148506510672, "grad_norm": 0.13938894867897034, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 4.838225841522217}
+{"step": 230, "loss": 0.09991920543456334, "grad_norm": 0.11645595729351044, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 4.634552955627441}
+{"step": 231, "loss": 0.13609107479983104, "grad_norm": 0.12388233840465546, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 4.081891775131226}
+{"step": 232, "loss": 0.08958531011415795, "grad_norm": 0.17350469529628754, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 3.238420248031616}
+{"step": 233, "loss": 0.13273630894970176, "grad_norm": 0.1391891986131668, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 3.7956702709198}
+{"step": 234, "loss": 0.13820909570059206, "grad_norm": 0.14823788404464722, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 3.9144842624664307}
+{"step": 235, "loss": 0.11004577230521641, "grad_norm": 0.11503985524177551, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 4.0461766719818115}
+{"step": 236, "loss": 0.10439036991824438, "grad_norm": 0.14346753060817719, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 3.6181647777557373}
+{"step": 237, "loss": 0.08662173568812119, "grad_norm": 0.13368716835975647, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 3.972587823867798}
+{"step": 238, "loss": 0.07688490281431162, "grad_norm": 0.1442182958126068, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 4.237303256988525}
+{"step": 239, "loss": 0.10296622989324716, "grad_norm": 0.1450713872909546, "lr": 9.81362333808718e-06, "epoch": 1, "t": 3.6274020671844482}
+{"step": 240, "loss": 0.11895017203366577, "grad_norm": 0.14547528326511383, "lr": 9.483536801870834e-06, "epoch": 1, "t": 3.604959487915039}
+{"step": 241, "loss": 0.11418216677648194, "grad_norm": 0.17269007861614227, "lr": 9.158514503674543e-06, "epoch": 1, "t": 3.406531810760498}
+{"step": 242, "loss": 0.13363193365779236, "grad_norm": 0.14209501445293427, "lr": 8.838597068719518e-06, "epoch": 1, "t": 3.8660969734191895}
+{"step": 243, "loss": 0.12033202722917166, "grad_norm": 0.13872650265693665, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.8964409828186035}
+{"step": 244, "loss": 0.10058960918612128, "grad_norm": 0.13868655264377594, "lr": 8.21423609408199e-06, "epoch": 1, "t": 3.6160824298858643}
+{"step": 245, "loss": 0.10850093874315761, "grad_norm": 0.16261093318462372, "lr": 7.90987059459195e-06, "epoch": 1, "t": 3.775347948074341}
+{"step": 246, "loss": 0.09171882781896504, "grad_norm": 0.14572811126708984, "lr": 7.610766028973709e-06, "epoch": 1, "t": 3.7401373386383057}
+{"step": 247, "loss": 0.0854891299666945, "grad_norm": 0.1358249932527542, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 3.6188063621520996}
+{"step": 248, "loss": 0.12579634301883863, "grad_norm": 0.1290748566389084, "lr": 7.028488579941506e-06, "epoch": 1, "t": 4.171281337738037}
+{"step": 249, "loss": 0.13838753616704239, "grad_norm": 0.13625887036323547, "lr": 6.745388476611553e-06, "epoch": 1, "t": 4.067877531051636}
+{"step": 250, "loss": 0.10446661840506793, "grad_norm": 0.11313296109437943, "lr": 6.467694858224488e-06, "epoch": 1, "t": 4.208914756774902}
+{"step": 251, "loss": 0.0931022302081748, "grad_norm": 0.12598943710327148, "lr": 6.1954424342902e-06, "epoch": 1, "t": 3.682990789413452}
+{"step": 252, "loss": 0.09719556848400071, "grad_norm": 0.13405030965805054, "lr": 5.928665234212233e-06, "epoch": 1, "t": 3.649326801300049}
+{"step": 253, "loss": 0.11480588525089297, "grad_norm": 0.1170310527086258, "lr": 5.66739660303437e-06, "epoch": 1, "t": 5.035650253295898}
+{"step": 254, "loss": 0.14551311045500265, "grad_norm": 0.1440739631652832, "lr": 5.411669197272795e-06, "epoch": 1, "t": 4.299440860748291}
+{"step": 255, "loss": 0.10512100388567737, "grad_norm": 0.16009369492530823, "lr": 5.161514980834231e-06, "epoch": 1, "t": 3.41422963142395}
+{"step": 256, "loss": 0.09641300468787337, "grad_norm": 0.16746076941490173, "lr": 4.916965221020753e-06, "epoch": 1, "t": 3.675581455230713}
+{"step": 257, "loss": 0.13006946074172873, "grad_norm": 0.13004963099956512, "lr": 4.678050484621615e-06, "epoch": 1, "t": 4.0484936237335205}
+{"step": 258, "loss": 0.11624933826652441, "grad_norm": 0.12923796474933624, "lr": 4.444800634092616e-06, "epoch": 1, "t": 4.318674325942993}
+{"step": 259, "loss": 0.14239484126848145, "grad_norm": 0.1452988088130951, "lr": 4.217244823823546e-06, "epoch": 1, "t": 3.900222063064575}
+{"step": 260, "loss": 0.11137070710612823, "grad_norm": 0.14754535257816315, "lr": 3.995411496494134e-06, "epoch": 1, "t": 3.767199993133545}
+{"step": 261, "loss": 0.1219098773208258, "grad_norm": 0.1505592316389084, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 3.6766936779022217}
+{"step": 262, "loss": 0.1273153463056978, "grad_norm": 0.12352366000413895, "lr": 3.56902248158148e-06, "epoch": 1, "t": 4.187962055206299}
+{"step": 263, "loss": 0.10637813524293752, "grad_norm": 0.13225017488002777, "lr": 3.364520089258727e-06, "epoch": 1, "t": 3.806302070617676}
+{"step": 264, "loss": 0.11944246138898557, "grad_norm": 0.13498401641845703, "lr": 3.165846763735153e-06, "epoch": 1, "t": 3.936896800994873}
+{"step": 265, "loss": 0.13903106421335482, "grad_norm": 0.155850350856781, "lr": 2.973027337607892e-06, "epoch": 1, "t": 3.9133760929107666}
+{"step": 266, "loss": 0.14236460609331456, "grad_norm": 0.15269044041633606, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.7339625358581543}
+{"step": 267, "loss": 0.08511999813101012, "grad_norm": 0.13898096978664398, "lr": 2.605045852462473e-06, "epoch": 1, "t": 3.5182197093963623}
+{"step": 268, "loss": 0.11819427140236691, "grad_norm": 0.15254700183868408, "lr": 2.429929788224722e-06, "epoch": 1, "t": 3.874054193496704}
+{"step": 269, "loss": 0.11059472183397173, "grad_norm": 0.14262066781520844, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 3.7272119522094727}
+{"step": 270, "loss": 0.08498130621101317, "grad_norm": 0.11333472281694412, "lr": 2.097556454310701e-06, "epoch": 1, "t": 4.243751049041748}
+{"step": 271, "loss": 0.11337690163921836, "grad_norm": 0.14444462954998016, "lr": 1.940340728677059e-06, "epoch": 1, "t": 3.66131854057312}
+{"step": 272, "loss": 0.0891804503934128, "grad_norm": 0.11745558679103851, "lr": 1.789132081018674e-06, "epoch": 1, "t": 3.9521219730377197}
+{"step": 273, "loss": 0.10169739746891199, "grad_norm": 0.17165616154670715, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 3.3434834480285645}
+{"step": 274, "loss": 0.108343622336785, "grad_norm": 0.13330884277820587, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 3.6591451168060303}
+{"step": 275, "loss": 0.12467141498144718, "grad_norm": 0.15990428626537323, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 3.7660090923309326}
+{"step": 276, "loss": 0.11941524178974489, "grad_norm": 0.170669287443161, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 3.755234956741333}
+{"step": 277, "loss": 0.10838764875073736, "grad_norm": 0.13164429366588593, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 4.421727895736694}
+{"step": 278, "loss": 0.10467391214734498, "grad_norm": 0.16259969770908356, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 3.7775378227233887}
+{"step": 279, "loss": 0.12416259101643794, "grad_norm": 0.12538431584835052, "lr": 9.003621402309814e-07, "epoch": 1, "t": 4.238988637924194}
+{"step": 280, "loss": 0.0749941657221775, "grad_norm": 0.10618844628334045, "lr": 7.97826969669102e-07, "epoch": 1, "t": 4.841545820236206}
+{"step": 281, "loss": 0.14073634982918937, "grad_norm": 0.13644573092460632, "lr": 7.014416822821556e-07, "epoch": 1, "t": 4.329077959060669}
+{"step": 282, "loss": 0.11568302072682479, "grad_norm": 0.10718589276075363, "lr": 6.112183254700865e-07, "epoch": 1, "t": 4.909542798995972}
+{"step": 283, "loss": 0.1256309869472165, "grad_norm": 0.1507447361946106, "lr": 5.271681764401848e-07, "epoch": 1, "t": 3.656003952026367}
+{"step": 284, "loss": 0.1511873127093355, "grad_norm": 0.15131469070911407, "lr": 4.493017407975086e-07, "epoch": 1, "t": 4.525445222854614}
+{"step": 285, "loss": 0.1058353923804579, "grad_norm": 0.14334936439990997, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 3.7437539100646973}
+{"step": 286, "loss": 0.12282938414467312, "grad_norm": 0.13383422791957855, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 3.8909730911254883}
+{"step": 287, "loss": 0.12590891681669947, "grad_norm": 0.14482846856117249, "lr": 2.528981693106558e-07, "epoch": 1, "t": 4.5848069190979}
+{"step": 288, "loss": 0.12649414153198413, "grad_norm": 0.12033234536647797, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 4.207767486572266}
+{"step": 289, "loss": 0.09800842328651038, "grad_norm": 0.12821905314922333, "lr": 1.530387900802177e-07, "epoch": 1, "t": 3.860168218612671}
+{"step": 290, "loss": 0.12013972534655686, "grad_norm": 0.14479869604110718, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 4.031760215759277}
+{"step": 291, "loss": 0.12322701873351707, "grad_norm": 0.13600681722164154, "lr": 7.81005385163458e-08, "epoch": 1, "t": 3.72101092338562}
+{"step": 292, "loss": 0.1178566493262507, "grad_norm": 0.11361332982778549, "lr": 4.998903085539075e-08, "epoch": 1, "t": 4.333397626876831}
+{"step": 293, "loss": 0.13276658208806064, "grad_norm": 0.15609505772590637, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 4.337067365646362}
+{"step": 294, "loss": 0.13313324704125196, "grad_norm": 0.17982439696788788, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 4.590375185012817}
+{"step": 295, "loss": 0.11335678350625483, "grad_norm": 0.15184786915779114, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 3.7073938846588135}
+{"step": 296, "loss": 0.08455663371587775, "grad_norm": 0.14751861989498138, "lr": 0.0, "epoch": 1, "t": 3.7283072471618652}
diff --git a/qwen3-14b/top-s0/README.md b/qwen3-14b/top-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/top-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/top-s0/adapter_config.json b/qwen3-14b/top-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..bf5ec5d657fd5f30f69df0bbd122d054b9b2d363
--- /dev/null
+++ b/qwen3-14b/top-s0/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "down_proj",
+    "o_proj",
+    "k_proj",
+    "up_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/top-s0/adapter_model.safetensors b/qwen3-14b/top-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e4e9fa308344a7ac840b69197fe5ad57aa86e3d5
--- /dev/null
+++ b/qwen3-14b/top-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4b2eca6ab43520f00552b5a0de2f53cfbb186e5d17b6bb36f56dfcb966f2d62
+size 1027680256
diff --git a/qwen3-14b/top-s0/config.json b/qwen3-14b/top-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..1959879eb89453c20418b790d13147077a51c238
--- /dev/null
+++ b/qwen3-14b/top-s0/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-top-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/top-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 0,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/top-s0/log.jsonl b/qwen3-14b/top-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..7b70e4248f4bdfaf294cc60437e538d731b92dc5
--- /dev/null
+++ b/qwen3-14b/top-s0/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.18078093084155533, "grad_norm": 0.21543461084365845, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 5.670870065689087}
+{"step": 2, "loss": 0.12582288082374668, "grad_norm": 0.19909027218818665, "lr": 2e-05, "epoch": 1, "t": 3.941906213760376}
+{"step": 3, "loss": 0.1541504488598134, "grad_norm": 0.196136474609375, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 3.8562185764312744}
+{"step": 4, "loss": 0.16245576396573166, "grad_norm": 0.20033963024616241, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 4.2135655879974365}
+{"step": 5, "loss": 0.18373874218151484, "grad_norm": 0.20790280401706696, "lr": 4e-05, "epoch": 1, "t": 4.142192840576172}
+{"step": 6, "loss": 0.13798882758752126, "grad_norm": 0.1633693426847458, "lr": 4.666666666666667e-05, "epoch": 1, "t": 5.109975576400757}
+{"step": 7, "loss": 0.13971804125460588, "grad_norm": 0.3595716059207916, "lr": 5.333333333333333e-05, "epoch": 1, "t": 3.713369846343994}
+{"step": 8, "loss": 0.12928648809658377, "grad_norm": 0.2502129375934601, "lr": 6e-05, "epoch": 1, "t": 4.005066633224487}
+{"step": 9, "loss": 0.1475834178628179, "grad_norm": 0.25998467206954956, "lr": 6.666666666666667e-05, "epoch": 1, "t": 4.157970190048218}
+{"step": 10, "loss": 0.1743530355335923, "grad_norm": 0.3950140178203583, "lr": 7.333333333333333e-05, "epoch": 1, "t": 3.7948899269104004}
+{"step": 11, "loss": 0.1291236110835223, "grad_norm": 0.3499937951564789, "lr": 8e-05, "epoch": 1, "t": 3.79697847366333}
+{"step": 12, "loss": 0.12859051067373817, "grad_norm": 0.23681354522705078, "lr": 8.666666666666667e-05, "epoch": 1, "t": 3.749391555786133}
+{"step": 13, "loss": 0.12550311297771102, "grad_norm": 0.26863229274749756, "lr": 9.333333333333334e-05, "epoch": 1, "t": 3.5778677463531494}
+{"step": 14, "loss": 0.09107925962478534, "grad_norm": 0.26592889428138733, "lr": 0.0001, "epoch": 1, "t": 3.2751388549804688}
+{"step": 15, "loss": 0.12067028350735966, "grad_norm": 0.23605218529701233, "lr": 0.0001, "epoch": 1, "t": 3.505476713180542}
+{"step": 16, "loss": 0.13331860667596254, "grad_norm": 0.21001331508159637, "lr": 9.999687519737639e-05, "epoch": 1, "t": 3.4056921005249023}
+{"step": 17, "loss": 0.16839911850569061, "grad_norm": 0.20148932933807373, "lr": 9.998750118008115e-05, "epoch": 1, "t": 3.764092206954956}
+{"step": 18, "loss": 0.12512060935866873, "grad_norm": 0.24095499515533447, "lr": 9.997187911979252e-05, "epoch": 1, "t": 4.054027557373047}
+{"step": 19, "loss": 0.11783227048047464, "grad_norm": 0.19750240445137024, "lr": 9.995001096914461e-05, "epoch": 1, "t": 3.55291485786438}
+{"step": 20, "loss": 0.10038413816285364, "grad_norm": 0.15660704672336578, "lr": 9.992189946148366e-05, "epoch": 1, "t": 3.716048240661621}
+{"step": 21, "loss": 0.14044953940381888, "grad_norm": 0.18374888598918915, "lr": 9.988754811052616e-05, "epoch": 1, "t": 3.993468761444092}
+{"step": 22, "loss": 0.13107693850269914, "grad_norm": 0.16976319253444672, "lr": 9.984696120991978e-05, "epoch": 1, "t": 4.326107740402222}
+{"step": 23, "loss": 0.11059003077070484, "grad_norm": 0.20888294279575348, "lr": 9.980014383270668e-05, "epoch": 1, "t": 3.9496548175811768}
+{"step": 24, "loss": 0.10902174840243771, "grad_norm": 0.23536130785942078, "lr": 9.974710183068935e-05, "epoch": 1, "t": 3.4843668937683105}
+{"step": 25, "loss": 0.12311838003388997, "grad_norm": 0.19582679867744446, "lr": 9.968784183369929e-05, "epoch": 1, "t": 4.1347410678863525}
+{"step": 26, "loss": 0.1087854557883912, "grad_norm": 0.19842907786369324, "lr": 9.962237124876828e-05, "epoch": 1, "t": 3.620652198791504}
+{"step": 27, "loss": 0.11952047203831741, "grad_norm": 0.185745507478714, "lr": 9.955069825920249e-05, "epoch": 1, "t": 3.6010961532592773}
+{"step": 28, "loss": 0.10815487410439246, "grad_norm": 0.20757320523262024, "lr": 9.947283182355982e-05, "epoch": 1, "t": 3.4007794857025146}
+{"step": 29, "loss": 0.10558924799245033, "grad_norm": 0.15870383381843567, "lr": 9.938878167452992e-05, "epoch": 1, "t": 3.633085250854492}
+{"step": 30, "loss": 0.11848456206030301, "grad_norm": 0.19631318747997284, "lr": 9.929855831771786e-05, "epoch": 1, "t": 3.4900362491607666}
+{"step": 31, "loss": 0.11938582143752272, "grad_norm": 0.17121528089046478, "lr": 9.92021730303309e-05, "epoch": 1, "t": 3.5577785968780518}
+{"step": 32, "loss": 0.1100530130914239, "grad_norm": 0.1477796733379364, "lr": 9.909963785976903e-05, "epoch": 1, "t": 3.7378315925598145}
+{"step": 33, "loss": 0.13140434853475816, "grad_norm": 0.1547025889158249, "lr": 9.899096562211902e-05, "epoch": 1, "t": 4.366919040679932}
+{"step": 34, "loss": 0.12498005750234412, "grad_norm": 0.15117643773555756, "lr": 9.887616990055262e-05, "epoch": 1, "t": 4.076546907424927}
+{"step": 35, "loss": 0.1199937956456037, "grad_norm": 0.18078838288784027, "lr": 9.875526504362869e-05, "epoch": 1, "t": 3.544379949569702}
+{"step": 36, "loss": 0.10290357320370365, "grad_norm": 0.16247794032096863, "lr": 9.86282661634998e-05, "epoch": 1, "t": 3.640885829925537}
+{"step": 37, "loss": 0.12317111854844882, "grad_norm": 0.142709419131279, "lr": 9.849518913402334e-05, "epoch": 1, "t": 4.105754852294922}
+{"step": 38, "loss": 0.09888693493445183, "grad_norm": 0.17833088338375092, "lr": 9.835605058877729e-05, "epoch": 1, "t": 3.5036935806274414}
+{"step": 39, "loss": 0.10262923279507825, "grad_norm": 0.19352814555168152, "lr": 9.821086791898134e-05, "epoch": 1, "t": 3.4939422607421875}
+{"step": 40, "loss": 0.11843793615184335, "grad_norm": 0.22154080867767334, "lr": 9.805965927132295e-05, "epoch": 1, "t": 3.4267356395721436}
+{"step": 41, "loss": 0.08981965517075244, "grad_norm": 0.17716406285762787, "lr": 9.79024435456893e-05, "epoch": 1, "t": 3.2837038040161133}
+{"step": 42, "loss": 0.10787966252673283, "grad_norm": 0.15934225916862488, "lr": 9.773924039280487e-05, "epoch": 1, "t": 3.9962387084960938}
+{"step": 43, "loss": 0.12862343112499322, "grad_norm": 0.17343322932720184, "lr": 9.75700702117753e-05, "epoch": 1, "t": 4.001704692840576}
+{"step": 44, "loss": 0.1244880738195358, "grad_norm": 0.1687047928571701, "lr": 9.739495414753753e-05, "epoch": 1, "t": 3.7973380088806152}
+{"step": 45, "loss": 0.14413799185562948, "grad_norm": 0.17575763165950775, "lr": 9.721391408821711e-05, "epoch": 1, "t": 3.6326217651367188}
+{"step": 46, "loss": 0.09852565336650099, "grad_norm": 0.14797110855579376, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.462437629699707}
+{"step": 47, "loss": 0.11137578864015688, "grad_norm": 0.14154157042503357, "lr": 9.683415323626485e-05, "epoch": 1, "t": 3.8369626998901367}
+{"step": 48, "loss": 0.11632150967216717, "grad_norm": 0.1633470505475998, "lr": 9.663547991074127e-05, "epoch": 1, "t": 3.6130459308624268}
+{"step": 49, "loss": 0.12977812556252522, "grad_norm": 0.13827835023403168, "lr": 9.643097751841854e-05, "epoch": 1, "t": 4.361585378646851}
+{"step": 50, "loss": 0.08359471904150335, "grad_norm": 0.12975750863552094, "lr": 9.622067162048112e-05, "epoch": 1, "t": 4.143960237503052}
+{"step": 51, "loss": 0.11336564413506282, "grad_norm": 0.12842722237110138, "lr": 9.600458850350588e-05, "epoch": 1, "t": 4.334429502487183}
+{"step": 52, "loss": 0.12409726028475786, "grad_norm": 0.17467203736305237, "lr": 9.578275517617645e-05, "epoch": 1, "t": 3.6473915576934814}
+{"step": 53, "loss": 0.0948607569355444, "grad_norm": 0.17197443544864655, "lr": 9.555519936590738e-05, "epoch": 1, "t": 3.1997082233428955}
+{"step": 54, "loss": 0.12241539399990099, "grad_norm": 0.1616947054862976, "lr": 9.532194951537838e-05, "epoch": 1, "t": 3.740054130554199}
+{"step": 55, "loss": 0.11414494514465333, "grad_norm": 0.12573809921741486, "lr": 9.508303477897924e-05, "epoch": 1, "t": 3.986130475997925}
+{"step": 56, "loss": 0.10649727000730758, "grad_norm": 0.16972380876541138, "lr": 9.483848501916578e-05, "epoch": 1, "t": 3.6302225589752197}
+{"step": 57, "loss": 0.11437893992378598, "grad_norm": 0.13277684152126312, "lr": 9.458833080272722e-05, "epoch": 1, "t": 4.3771491050720215}
+{"step": 58, "loss": 0.08907954733377742, "grad_norm": 0.17783862352371216, "lr": 9.433260339696563e-05, "epoch": 1, "t": 4.669188976287842}
+{"step": 59, "loss": 0.13647830771020217, "grad_norm": 0.1539842188358307, "lr": 9.407133476578778e-05, "epoch": 1, "t": 4.916244983673096}
+{"step": 60, "loss": 0.11414211137539289, "grad_norm": 0.16534669697284698, "lr": 9.38045575657098e-05, "epoch": 1, "t": 3.917712688446045}
+{"step": 61, "loss": 0.12148131917531801, "grad_norm": 0.1475285440683365, "lr": 9.353230514177552e-05, "epoch": 1, "t": 3.855111837387085}
+{"step": 62, "loss": 0.12488233953752514, "grad_norm": 0.1735742837190628, "lr": 9.325461152338846e-05, "epoch": 1, "t": 3.373034715652466}
+{"step": 63, "loss": 0.12115507730266865, "grad_norm": 0.16001300513744354, "lr": 9.297151142005851e-05, "epoch": 1, "t": 3.6875805854797363}
+{"step": 64, "loss": 0.11033134224873792, "grad_norm": 0.15544933080673218, "lr": 9.268304021706349e-05, "epoch": 1, "t": 3.904444456100464}
+{"step": 65, "loss": 0.1370053753265158, "grad_norm": 0.17626741528511047, "lr": 9.23892339710263e-05, "epoch": 1, "t": 3.437079906463623}
+{"step": 66, "loss": 0.1069775574194845, "grad_norm": 0.15322798490524292, "lr": 9.209012940540805e-05, "epoch": 1, "t": 3.375286817550659}
+{"step": 67, "loss": 0.13238664187391677, "grad_norm": 0.13565783202648163, "lr": 9.178576390591802e-05, "epoch": 1, "t": 4.2164857387542725}
+{"step": 68, "loss": 0.11918927880880746, "grad_norm": 0.1469467580318451, "lr": 9.147617551584066e-05, "epoch": 1, "t": 3.7977542877197266}
+{"step": 69, "loss": 0.1292904541211336, "grad_norm": 0.15517917275428772, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.831500291824341}
+{"step": 70, "loss": 0.08554563612745161, "grad_norm": 0.16317443549633026, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.791607618331909}
+{"step": 71, "loss": 0.10672666617862439, "grad_norm": 0.12530474364757538, "lr": 9.051646319812918e-05, "epoch": 1, "t": 4.066084861755371}
+{"step": 72, "loss": 0.1166960009071896, "grad_norm": 0.15463396906852722, "lr": 9.018637666191283e-05, "epoch": 1, "t": 3.9002726078033447}
+{"step": 73, "loss": 0.1159855733012819, "grad_norm": 0.12475717067718506, "lr": 8.985126714588738e-05, "epoch": 1, "t": 4.196139812469482}
+{"step": 74, "loss": 0.13275711320795952, "grad_norm": 0.15810687839984894, "lr": 8.951117653609666e-05, "epoch": 1, "t": 4.000990152359009}
+{"step": 75, "loss": 0.1083240142235389, "grad_norm": 0.15544044971466064, "lr": 8.916614734118184e-05, "epoch": 1, "t": 3.403935670852661}
+{"step": 76, "loss": 0.11153860755634606, "grad_norm": 0.16254383325576782, "lr": 8.881622268706825e-05, "epoch": 1, "t": 3.493802547454834}
+{"step": 77, "loss": 0.16076388984558743, "grad_norm": 0.1464482545852661, "lr": 8.8461446311575e-05, "epoch": 1, "t": 3.7476203441619873}
+{"step": 78, "loss": 0.1340599476377944, "grad_norm": 0.1564701944589615, "lr": 8.810186255894803e-05, "epoch": 1, "t": 3.625631093978882}
+{"step": 79, "loss": 0.1446214871327865, "grad_norm": 0.14072248339653015, "lr": 8.773751637431748e-05, "epoch": 1, "t": 3.9581570625305176}
+{"step": 80, "loss": 0.10749005277601398, "grad_norm": 0.14363059401512146, "lr": 8.736845329807993e-05, "epoch": 1, "t": 3.545226573944092}
+{"step": 81, "loss": 0.12397643225994368, "grad_norm": 0.120295949280262, "lr": 8.69947194602061e-05, "epoch": 1, "t": 4.155876874923706}
+{"step": 82, "loss": 0.12598336476645955, "grad_norm": 0.14872956275939941, "lr": 8.66163615744751e-05, "epoch": 1, "t": 3.7410964965820312}
+{"step": 83, "loss": 0.14362573278413956, "grad_norm": 0.16827794909477234, "lr": 8.623342693263548e-05, "epoch": 1, "t": 3.8328568935394287}
+{"step": 84, "loss": 0.08981273607443307, "grad_norm": 0.13605117797851562, "lr": 8.584596339849417e-05, "epoch": 1, "t": 3.815539836883545}
+{"step": 85, "loss": 0.09965101982348823, "grad_norm": 0.160883828997612, "lr": 8.545401940193392e-05, "epoch": 1, "t": 3.4498963356018066}
+{"step": 86, "loss": 0.11607775229826696, "grad_norm": 0.13547949492931366, "lr": 8.505764393285984e-05, "epoch": 1, "t": 3.9889557361602783}
+{"step": 87, "loss": 0.10640480537800864, "grad_norm": 0.1333422213792801, "lr": 8.46568865350762e-05, "epoch": 1, "t": 3.8678171634674072}
+{"step": 88, "loss": 0.10641535105226423, "grad_norm": 0.15682487189769745, "lr": 8.425179730009368e-05, "epoch": 1, "t": 3.6293699741363525}
+{"step": 89, "loss": 0.10884914856046367, "grad_norm": 0.1910397857427597, "lr": 8.384242686086848e-05, "epoch": 1, "t": 3.2520365715026855}
+{"step": 90, "loss": 0.1411999141059243, "grad_norm": 0.14669112861156464, "lr": 8.342882638547351e-05, "epoch": 1, "t": 3.854207754135132}
+{"step": 91, "loss": 0.1016362111405462, "grad_norm": 0.12613064050674438, "lr": 8.301104757070274e-05, "epoch": 1, "t": 3.737785816192627}
+{"step": 92, "loss": 0.11298835102832014, "grad_norm": 0.1722400039434433, "lr": 8.258914263560971e-05, "epoch": 1, "t": 3.397721529006958}
+{"step": 93, "loss": 0.10590427705524219, "grad_norm": 0.15405547618865967, "lr": 8.216316431498028e-05, "epoch": 1, "t": 3.6167490482330322}
+{"step": 94, "loss": 0.11041453299991288, "grad_norm": 0.11743570864200592, "lr": 8.173316585274145e-05, "epoch": 1, "t": 4.520207166671753}
+{"step": 95, "loss": 0.1363066214303518, "grad_norm": 0.1539447009563446, "lr": 8.129920099530607e-05, "epoch": 1, "t": 4.09876275062561}
+{"step": 96, "loss": 0.11042544247679299, "grad_norm": 0.1385338306427002, "lr": 8.086132398485524e-05, "epoch": 1, "t": 3.972177028656006}
+{"step": 97, "loss": 0.1019147576647537, "grad_norm": 0.171686589717865, "lr": 8.041958955255814e-05, "epoch": 1, "t": 3.316844940185547}
+{"step": 98, "loss": 0.11752869431273791, "grad_norm": 0.15965993702411652, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.679499387741089}
+{"step": 99, "loss": 0.12090033584589578, "grad_norm": 0.17482811212539673, "lr": 7.952476975093729e-05, "epoch": 1, "t": 3.5313146114349365}
+{"step": 100, "loss": 0.1287459241423825, "grad_norm": 0.1296885907649994, "lr": 7.907179622702408e-05, "epoch": 1, "t": 4.132402420043945}
+{"step": 101, "loss": 0.11878300334301946, "grad_norm": 0.1648198366165161, "lr": 7.861518895810596e-05, "epoch": 1, "t": 4.714941024780273}
+{"step": 102, "loss": 0.12423003789641919, "grad_norm": 0.14597715437412262, "lr": 7.815500501648653e-05, "epoch": 1, "t": 4.240759611129761}
+{"step": 103, "loss": 0.12631553659625444, "grad_norm": 0.15432026982307434, "lr": 7.769130192152538e-05, "epoch": 1, "t": 3.7459630966186523}
+{"step": 104, "loss": 0.12157258438882279, "grad_norm": 0.16133689880371094, "lr": 7.722413763244838e-05, "epoch": 1, "t": 3.6285572052001953}
+{"step": 105, "loss": 0.09997903336346045, "grad_norm": 0.1467406451702118, "lr": 7.675357054110336e-05, "epoch": 1, "t": 3.572279453277588}
+{"step": 106, "loss": 0.13437015235957572, "grad_norm": 0.1462370604276657, "lr": 7.627965946466166e-05, "epoch": 1, "t": 3.7207441329956055}
+{"step": 107, "loss": 0.09419957820158266, "grad_norm": 0.1562517136335373, "lr": 7.580246363826621e-05, "epoch": 1, "t": 4.4763572216033936}
+{"step": 108, "loss": 0.1399215856593495, "grad_norm": 0.155673548579216, "lr": 7.532204270762786e-05, "epoch": 1, "t": 4.720752000808716}
+{"step": 109, "loss": 0.10573507265574497, "grad_norm": 0.1813446283340454, "lr": 7.483845672156998e-05, "epoch": 1, "t": 4.312145471572876}
+{"step": 110, "loss": 0.12933389344600532, "grad_norm": 0.16296401619911194, "lr": 7.435176612452286e-05, "epoch": 1, "t": 3.7811851501464844}
+{"step": 111, "loss": 0.09811320523377468, "grad_norm": 0.14579057693481445, "lr": 7.386203174896872e-05, "epoch": 1, "t": 3.830552816390991}
+{"step": 112, "loss": 0.12977864382791768, "grad_norm": 0.14826302230358124, "lr": 7.336931480783801e-05, "epoch": 1, "t": 3.8837056159973145}
+{"step": 113, "loss": 0.12886856750131256, "grad_norm": 0.13797810673713684, "lr": 7.287367688685835e-05, "epoch": 1, "t": 4.179487466812134}
+{"step": 114, "loss": 0.09840674414832917, "grad_norm": 0.13383975625038147, "lr": 7.237517993685678e-05, "epoch": 1, "t": 3.673002243041992}
+{"step": 115, "loss": 0.1398746154699295, "grad_norm": 0.171783447265625, "lr": 7.187388626601637e-05, "epoch": 1, "t": 3.656867504119873}
+{"step": 116, "loss": 0.12633279269243583, "grad_norm": 0.15717682242393494, "lr": 7.136985853208824e-05, "epoch": 1, "t": 3.6593070030212402}
+{"step": 117, "loss": 0.09159916250187061, "grad_norm": 0.10727085918188095, "lr": 7.086315973455981e-05, "epoch": 1, "t": 4.936774730682373}
+{"step": 118, "loss": 0.14724441923167372, "grad_norm": 0.15063735842704773, "lr": 7.035385320678036e-05, "epoch": 1, "t": 4.8431291580200195}
+{"step": 119, "loss": 0.12029467754883288, "grad_norm": 0.14817851781845093, "lr": 6.984200260804484e-05, "epoch": 1, "t": 4.123239278793335}
+{"step": 120, "loss": 0.10002659185874586, "grad_norm": 0.1318923979997635, "lr": 6.932767191563703e-05, "epoch": 1, "t": 4.000175952911377}
+{"step": 121, "loss": 0.1171912178133907, "grad_norm": 0.1465427726507187, "lr": 6.881092541683278e-05, "epoch": 1, "t": 4.022685766220093}
+{"step": 122, "loss": 0.11841843840947211, "grad_norm": 0.1425304412841797, "lr": 6.829182770086474e-05, "epoch": 1, "t": 4.139328718185425}
+{"step": 123, "loss": 0.11667142473368287, "grad_norm": 0.14699885249137878, "lr": 6.777044365084907e-05, "epoch": 1, "t": 4.0254693031311035}
+{"step": 124, "loss": 0.11514476168103613, "grad_norm": 0.133835107088089, "lr": 6.724683843567568e-05, "epoch": 1, "t": 3.819842576980591}
+{"step": 125, "loss": 0.14367289122081833, "grad_norm": 0.1340443342924118, "lr": 6.672107750186255e-05, "epoch": 1, "t": 3.8937745094299316}
+{"step": 126, "loss": 0.10747240787503488, "grad_norm": 0.15666256844997406, "lr": 6.619322656537552e-05, "epoch": 1, "t": 4.001836061477661}
+{"step": 127, "loss": 0.11833105353984727, "grad_norm": 0.13074466586112976, "lr": 6.566335160341424e-05, "epoch": 1, "t": 4.132088661193848}
+{"step": 128, "loss": 0.15913567512650686, "grad_norm": 0.13294969499111176, "lr": 6.513151884616556e-05, "epoch": 1, "t": 4.374549150466919}
+{"step": 129, "loss": 0.10457521697161477, "grad_norm": 0.1695006787776947, "lr": 6.459779476852528e-05, "epoch": 1, "t": 3.499386787414551}
+{"step": 130, "loss": 0.12252127018412509, "grad_norm": 0.17074526846408844, "lr": 6.406224608178932e-05, "epoch": 1, "t": 3.355849504470825}
+{"step": 131, "loss": 0.1458974173642639, "grad_norm": 0.15384355187416077, "lr": 6.352493972531534e-05, "epoch": 1, "t": 3.75913405418396}
+{"step": 132, "loss": 0.1279706685505216, "grad_norm": 0.14873041212558746, "lr": 6.298594285815584e-05, "epoch": 1, "t": 3.7278811931610107}
+{"step": 133, "loss": 0.12022429512205901, "grad_norm": 0.14457093179225922, "lr": 6.244532285066382e-05, "epoch": 1, "t": 3.824042558670044}
+{"step": 134, "loss": 0.10227480884273675, "grad_norm": 0.1562817394733429, "lr": 6.190314727607196e-05, "epoch": 1, "t": 3.679955244064331}
+{"step": 135, "loss": 0.10766563139056921, "grad_norm": 0.19048047065734863, "lr": 6.13594839020466e-05, "epoch": 1, "t": 3.46612286567688}
+{"step": 136, "loss": 0.12839472438844787, "grad_norm": 0.13402053713798523, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 4.548829793930054}
+{"step": 137, "loss": 0.10790674618804812, "grad_norm": 0.23372948169708252, "lr": 6.026796574768288e-05, "epoch": 1, "t": 3.257747173309326}
+{"step": 138, "loss": 0.11405369279713466, "grad_norm": 0.13311302661895752, "lr": 5.972024739849622e-05, "epoch": 1, "t": 3.9835357666015625}
+{"step": 139, "loss": 0.10701609056493155, "grad_norm": 0.15247826278209686, "lr": 5.917131409512663e-05, "epoch": 1, "t": 3.7933242321014404}
+{"step": 140, "loss": 0.11431209296327247, "grad_norm": 0.13907016813755035, "lr": 5.862123444990318e-05, "epoch": 1, "t": 3.9845409393310547}
+{"step": 141, "loss": 0.14549523225263286, "grad_norm": 0.14027123153209686, "lr": 5.807007721843861e-05, "epoch": 1, "t": 4.074226379394531}
+{"step": 142, "loss": 0.11311370969982992, "grad_norm": 0.13780340552330017, "lr": 5.751791129103544e-05, "epoch": 1, "t": 3.6615114212036133}
+{"step": 143, "loss": 0.12406135482083762, "grad_norm": 0.1386808454990387, "lr": 5.696480568407523e-05, "epoch": 1, "t": 3.6009795665740967}
+{"step": 144, "loss": 0.10797050494188908, "grad_norm": 0.1410973072052002, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 3.7243547439575195}
+{"step": 145, "loss": 0.12399723411346815, "grad_norm": 0.14193448424339294, "lr": 5.585605207563124e-05, "epoch": 1, "t": 3.818485975265503}
+{"step": 146, "loss": 0.11882285128778486, "grad_norm": 0.15566815435886383, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 3.724196672439575}
+{"step": 147, "loss": 0.0895691902516352, "grad_norm": 0.16067829728126526, "lr": 5.47443707175741e-05, "epoch": 1, "t": 3.3457987308502197}
+{"step": 148, "loss": 0.10718928675887077, "grad_norm": 0.15717020630836487, "lr": 5.418760576667071e-05, "epoch": 1, "t": 3.693338632583618}
+{"step": 149, "loss": 0.10463362584873243, "grad_norm": 0.1388978809118271, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 3.9911677837371826}
+{"step": 150, "loss": 0.13628019982710168, "grad_norm": 0.15812093019485474, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 3.7371606826782227}
+{"step": 151, "loss": 0.12222583022605667, "grad_norm": 0.14443819224834442, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 3.7441506385803223}
+{"step": 152, "loss": 0.12971014358312277, "grad_norm": 0.15491265058517456, "lr": 5.195600862779421e-05, "epoch": 1, "t": 3.868527412414551}
+{"step": 153, "loss": 0.11435865951995551, "grad_norm": 0.20258963108062744, "lr": 5.139732367864736e-05, "epoch": 1, "t": 4.167773962020874}
+{"step": 154, "loss": 0.1075460684779957, "grad_norm": 0.1792241483926773, "lr": 5.083846407507263e-05, "epoch": 1, "t": 3.3405580520629883}
+{"step": 155, "loss": 0.11370573677730328, "grad_norm": 0.15837696194648743, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 3.724365472793579}
+{"step": 156, "loss": 0.0938549899157044, "grad_norm": 0.12526878714561462, "lr": 4.972050032989175e-05, "epoch": 1, "t": 3.913439989089966}
+{"step": 157, "loss": 0.14413435619236556, "grad_norm": 0.1548471748828888, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 4.007832050323486}
+{"step": 158, "loss": 0.11357251434454221, "grad_norm": 0.13227373361587524, "lr": 4.860267632135265e-05, "epoch": 1, "t": 3.8738040924072266}
+{"step": 159, "loss": 0.1087706522508101, "grad_norm": 0.14059820771217346, "lr": 4.80439913722058e-05, "epoch": 1, "t": 3.67482852935791}
+{"step": 160, "loss": 0.11988797803588261, "grad_norm": 0.13712352514266968, "lr": 4.748555090869464e-05, "epoch": 1, "t": 3.753204822540283}
+{"step": 161, "loss": 0.11524063148668835, "grad_norm": 0.14776402711868286, "lr": 4.692742473146818e-05, "epoch": 1, "t": 3.655618667602539}
+{"step": 162, "loss": 0.14956962153836217, "grad_norm": 0.15375608205795288, "lr": 4.636968260189214e-05, "epoch": 1, "t": 3.781660556793213}
+{"step": 163, "loss": 0.09825584829397588, "grad_norm": 0.11411170661449432, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 4.606749773025513}
+{"step": 164, "loss": 0.12185469910957944, "grad_norm": 0.13672995567321777, "lr": 4.525562928242592e-05, "epoch": 1, "t": 3.819789171218872}
+{"step": 165, "loss": 0.09286631310175943, "grad_norm": 0.11684384196996689, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 4.116922378540039}
+{"step": 166, "loss": 0.12150597162873752, "grad_norm": 0.12577475607395172, "lr": 4.414394792436877e-05, "epoch": 1, "t": 4.0338521003723145}
+{"step": 167, "loss": 0.09829409735481498, "grad_norm": 0.1458357572555542, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 3.936523675918579}
+{"step": 168, "loss": 0.09888429115114482, "grad_norm": 0.14295373857021332, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 3.7329986095428467}
+{"step": 169, "loss": 0.12043629968625456, "grad_norm": 0.1480800062417984, "lr": 4.248208870896456e-05, "epoch": 1, "t": 3.556403636932373}
+{"step": 170, "loss": 0.1160908742309349, "grad_norm": 0.13095155358314514, "lr": 4.192992278156141e-05, "epoch": 1, "t": 3.679358959197998}
+{"step": 171, "loss": 0.12660413290543385, "grad_norm": 0.140878826379776, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 3.684030055999756}
+{"step": 172, "loss": 0.11499137433940765, "grad_norm": 0.13224995136260986, "lr": 4.082868590487339e-05, "epoch": 1, "t": 4.291258811950684}
+{"step": 173, "loss": 0.09490913574684832, "grad_norm": 0.1406375765800476, "lr": 4.027975260150381e-05, "epoch": 1, "t": 3.587540626525879}
+{"step": 174, "loss": 0.11279438907226448, "grad_norm": 0.1550101339817047, "lr": 3.973203425231715e-05, "epoch": 1, "t": 3.515394926071167}
+{"step": 175, "loss": 0.15934996226622863, "grad_norm": 0.1233309879899025, "lr": 3.918559931778277e-05, "epoch": 1, "t": 4.522766828536987}
+{"step": 176, "loss": 0.08423884463538031, "grad_norm": 0.1321779191493988, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 4.314794063568115}
+{"step": 177, "loss": 0.10201702809602972, "grad_norm": 0.14126354455947876, "lr": 3.809685272392804e-05, "epoch": 1, "t": 3.9226272106170654}
+{"step": 178, "loss": 0.11557401262558577, "grad_norm": 0.14164473116397858, "lr": 3.755467714933619e-05, "epoch": 1, "t": 3.6965043544769287}
+{"step": 179, "loss": 0.15528196712347947, "grad_norm": 0.15970170497894287, "lr": 3.701405714184416e-05, "epoch": 1, "t": 3.909911632537842}
+{"step": 180, "loss": 0.1397888739746396, "grad_norm": 0.18241266906261444, "lr": 3.647506027468467e-05, "epoch": 1, "t": 3.415771007537842}
+{"step": 181, "loss": 0.12829640991378558, "grad_norm": 0.15943564474582672, "lr": 3.59377539182107e-05, "epoch": 1, "t": 3.664119243621826}
+{"step": 182, "loss": 0.11461809487489379, "grad_norm": 0.1422126144170761, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 3.720914840698242}
+{"step": 183, "loss": 0.08553553408349465, "grad_norm": 0.11973387748003006, "lr": 3.486848115383445e-05, "epoch": 1, "t": 4.111364364624023}
+{"step": 184, "loss": 0.10667813302503475, "grad_norm": 0.11677293479442596, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 4.285335540771484}
+{"step": 185, "loss": 0.12617848205091525, "grad_norm": 0.141795814037323, "lr": 3.380677343462447e-05, "epoch": 1, "t": 3.9257261753082275}
+{"step": 186, "loss": 0.09332461867846778, "grad_norm": 0.13392992317676544, "lr": 3.327892249813745e-05, "epoch": 1, "t": 4.102517127990723}
+{"step": 187, "loss": 0.14679910766430934, "grad_norm": 0.13243480026721954, "lr": 3.275316156432434e-05, "epoch": 1, "t": 4.022236347198486}
+{"step": 188, "loss": 0.12036066461781986, "grad_norm": 0.14931295812129974, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 3.8003978729248047}
+{"step": 189, "loss": 0.12167078109837752, "grad_norm": 0.1373678743839264, "lr": 3.170817229913526e-05, "epoch": 1, "t": 3.9012646675109863}
+{"step": 190, "loss": 0.1072719172117352, "grad_norm": 0.11742686480283737, "lr": 3.118907458316722e-05, "epoch": 1, "t": 4.2862842082977295}
+{"step": 191, "loss": 0.08375593344540157, "grad_norm": 0.13339167833328247, "lr": 3.067232808436299e-05, "epoch": 1, "t": 3.653456211090088}
+{"step": 192, "loss": 0.10096689934469975, "grad_norm": 0.14291594922542572, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 3.9348721504211426}
+{"step": 193, "loss": 0.13594844983311183, "grad_norm": 0.12516500055789948, "lr": 2.964614679321966e-05, "epoch": 1, "t": 4.306424379348755}
+{"step": 194, "loss": 0.1306842352294383, "grad_norm": 0.1552329659461975, "lr": 2.913684026544021e-05, "epoch": 1, "t": 4.053359270095825}
+{"step": 195, "loss": 0.11627123112064276, "grad_norm": 0.13674090802669525, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 3.935307502746582}
+{"step": 196, "loss": 0.11920044226938488, "grad_norm": 0.17094795405864716, "lr": 2.812611373398365e-05, "epoch": 1, "t": 3.9484047889709473}
+{"step": 197, "loss": 0.0766151636647346, "grad_norm": 0.11914075165987015, "lr": 2.762482006314324e-05, "epoch": 1, "t": 4.682788848876953}
+{"step": 198, "loss": 0.12958236513589683, "grad_norm": 0.1496438831090927, "lr": 2.712632311314165e-05, "epoch": 1, "t": 4.006940841674805}
+{"step": 199, "loss": 0.09184713697805846, "grad_norm": 0.15052573382854462, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 3.9801559448242188}
+{"step": 200, "loss": 0.12825358005906132, "grad_norm": 0.15824151039123535, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 4.0740132331848145}
+{"step": 201, "loss": 0.12595045235210334, "grad_norm": 0.1473931521177292, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 4.280902624130249}
+{"step": 202, "loss": 0.09369087958485539, "grad_norm": 0.15076716244220734, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 4.748701810836792}
+{"step": 203, "loss": 0.15185879450328632, "grad_norm": 0.15031099319458008, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 5.010256290435791}
+{"step": 204, "loss": 0.1245848155110789, "grad_norm": 0.1478005349636078, "lr": 2.419753636173379e-05, "epoch": 1, "t": 4.97152853012085}
+{"step": 205, "loss": 0.12303665354790108, "grad_norm": 0.13761848211288452, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 4.358026027679443}
+{"step": 206, "loss": 0.10262219146379653, "grad_norm": 0.1390838921070099, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 3.988492012023926}
+{"step": 207, "loss": 0.1288326810106369, "grad_norm": 0.1274680495262146, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 4.200056791305542}
+{"step": 208, "loss": 0.10700027434448404, "grad_norm": 0.1488206535577774, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 4.035815954208374}
+{"step": 209, "loss": 0.09753100396374707, "grad_norm": 0.12700127065181732, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 5.035707473754883}
+{"step": 210, "loss": 0.1040074866387937, "grad_norm": 0.11726857721805573, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 5.113846302032471}
+{"step": 211, "loss": 0.11655322577565762, "grad_norm": 0.15103833377361298, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 4.695590257644653}
+{"step": 212, "loss": 0.12798841111231815, "grad_norm": 0.1448533833026886, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 4.937502861022949}
+{"step": 213, "loss": 0.09533938653112795, "grad_norm": 0.13098298013210297, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 4.02750039100647}
+{"step": 214, "loss": 0.1167240142004924, "grad_norm": 0.15033891797065735, "lr": 1.958041044744186e-05, "epoch": 1, "t": 3.7111308574676514}
+{"step": 215, "loss": 0.13111353684047916, "grad_norm": 0.14265334606170654, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 3.846306562423706}
+{"step": 216, "loss": 0.10622000839629359, "grad_norm": 0.12489472329616547, "lr": 1.870079900469392e-05, "epoch": 1, "t": 3.98367977142334}
+{"step": 217, "loss": 0.11011956695555551, "grad_norm": 0.1353050172328949, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 4.148133993148804}
+{"step": 218, "loss": 0.10432480726444696, "grad_norm": 0.12181513011455536, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 3.8785266876220703}
+{"step": 219, "loss": 0.12997548904969425, "grad_norm": 0.16283245384693146, "lr": 1.741085736439031e-05, "epoch": 1, "t": 3.547520637512207}
+{"step": 220, "loss": 0.10764003020876285, "grad_norm": 0.17088381946086884, "lr": 1.698895242929725e-05, "epoch": 1, "t": 3.364952325820923}
+{"step": 221, "loss": 0.10036377555364759, "grad_norm": 0.1563388556241989, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 3.321061611175537}
+{"step": 222, "loss": 0.11357700002234807, "grad_norm": 0.11825655400753021, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 4.437260866165161}
+{"step": 223, "loss": 0.10780351351631105, "grad_norm": 0.14153169095516205, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 3.4183976650238037}
+{"step": 224, "loss": 0.11067074397339687, "grad_norm": 0.145760640501976, "lr": 1.534311346492381e-05, "epoch": 1, "t": 3.494868278503418}
+{"step": 225, "loss": 0.13485682848789393, "grad_norm": 0.16002051532268524, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 3.5129947662353516}
+{"step": 226, "loss": 0.11011886038719274, "grad_norm": 0.1291523575782776, "lr": 1.454598059806609e-05, "epoch": 1, "t": 3.6210789680480957}
+{"step": 227, "loss": 0.10693234755754853, "grad_norm": 0.150756374001503, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 3.598886728286743}
+{"step": 228, "loss": 0.10516166426960885, "grad_norm": 0.11776912212371826, "lr": 1.376657306736453e-05, "epoch": 1, "t": 4.197521448135376}
+{"step": 229, "loss": 0.1407072312101371, "grad_norm": 0.14439688622951508, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 3.6081395149230957}
+{"step": 230, "loss": 0.11518118446124836, "grad_norm": 0.11391116678714752, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 4.29512619972229}
+{"step": 231, "loss": 0.12709926091016358, "grad_norm": 0.15815281867980957, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 3.89015531539917}
+{"step": 232, "loss": 0.12464843823507465, "grad_norm": 0.14223653078079224, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 3.7703375816345215}
+{"step": 233, "loss": 0.08832229622997095, "grad_norm": 0.25007176399230957, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 3.4372386932373047}
+{"step": 234, "loss": 0.1021679041060137, "grad_norm": 0.1246672049164772, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 3.8089632987976074}
+{"step": 235, "loss": 0.112289905184431, "grad_norm": 0.24810990691184998, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 4.110302686691284}
+{"step": 236, "loss": 0.10785791463124103, "grad_norm": 0.12329133599996567, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 3.920509099960327}
+{"step": 237, "loss": 0.10138124014276459, "grad_norm": 0.1192634105682373, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 4.056989908218384}
+{"step": 238, "loss": 0.08575028924391963, "grad_norm": 0.14694784581661224, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 3.3333706855773926}
+{"step": 239, "loss": 0.13645548691421353, "grad_norm": 0.14239895343780518, "lr": 9.81362333808718e-06, "epoch": 1, "t": 3.8147268295288086}
+{"step": 240, "loss": 0.09247345524539029, "grad_norm": 0.10970944911241531, "lr": 9.483536801870834e-06, "epoch": 1, "t": 4.4236249923706055}
+{"step": 241, "loss": 0.09314704683954345, "grad_norm": 0.1298266053199768, "lr": 9.158514503674543e-06, "epoch": 1, "t": 3.7495830059051514}
+{"step": 242, "loss": 0.09981172469980723, "grad_norm": 0.1517573893070221, "lr": 8.838597068719518e-06, "epoch": 1, "t": 4.198540687561035}
+{"step": 243, "loss": 0.11510658663941431, "grad_norm": 0.13587255775928497, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.8558340072631836}
+{"step": 244, "loss": 0.09087367526692762, "grad_norm": 0.12901368737220764, "lr": 8.21423609408199e-06, "epoch": 1, "t": 3.6269500255584717}
+{"step": 245, "loss": 0.12133481873125654, "grad_norm": 0.15090236067771912, "lr": 7.90987059459195e-06, "epoch": 1, "t": 3.670531749725342}
+{"step": 246, "loss": 0.11570219187528875, "grad_norm": 0.16052192449569702, "lr": 7.610766028973709e-06, "epoch": 1, "t": 3.5546460151672363}
+{"step": 247, "loss": 0.11303498780066643, "grad_norm": 0.12924300134181976, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 3.831899881362915}
+{"step": 248, "loss": 0.124954099259125, "grad_norm": 0.11974083632230759, "lr": 7.028488579941506e-06, "epoch": 1, "t": 4.709738254547119}
+{"step": 249, "loss": 0.13933611023533451, "grad_norm": 0.1666496843099594, "lr": 6.745388476611553e-06, "epoch": 1, "t": 4.06585955619812}
+{"step": 250, "loss": 0.10272121849600495, "grad_norm": 0.14261454343795776, "lr": 6.467694858224488e-06, "epoch": 1, "t": 3.5229015350341797}
+{"step": 251, "loss": 0.12156610535796097, "grad_norm": 0.14119717478752136, "lr": 6.1954424342902e-06, "epoch": 1, "t": 3.7426209449768066}
+{"step": 252, "loss": 0.08723529054446326, "grad_norm": 0.1311442255973816, "lr": 5.928665234212233e-06, "epoch": 1, "t": 4.254141569137573}
+{"step": 253, "loss": 0.146235889827911, "grad_norm": 0.14487206935882568, "lr": 5.66739660303437e-06, "epoch": 1, "t": 4.049948692321777}
+{"step": 254, "loss": 0.12202417389962543, "grad_norm": 0.1263277679681778, "lr": 5.411669197272795e-06, "epoch": 1, "t": 4.146308898925781}
+{"step": 255, "loss": 0.1152144418695227, "grad_norm": 0.13160760700702667, "lr": 5.161514980834231e-06, "epoch": 1, "t": 4.241142988204956}
+{"step": 256, "loss": 0.08598439921954551, "grad_norm": 0.11144998669624329, "lr": 4.916965221020753e-06, "epoch": 1, "t": 4.577575445175171}
+{"step": 257, "loss": 0.09790864901309176, "grad_norm": 0.1261155903339386, "lr": 4.678050484621615e-06, "epoch": 1, "t": 3.7849485874176025}
+{"step": 258, "loss": 0.11497225311706805, "grad_norm": 0.12973695993423462, "lr": 4.444800634092616e-06, "epoch": 1, "t": 4.863637924194336}
+{"step": 259, "loss": 0.09545747448788158, "grad_norm": 0.16509896516799927, "lr": 4.217244823823546e-06, "epoch": 1, "t": 3.3479771614074707}
+{"step": 260, "loss": 0.06801871632672177, "grad_norm": 0.14654715359210968, "lr": 3.995411496494134e-06, "epoch": 1, "t": 4.054205656051636}
+{"step": 261, "loss": 0.14310309870975746, "grad_norm": 0.1418982446193695, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 4.085833787918091}
+{"step": 262, "loss": 0.1021511512838637, "grad_norm": 0.12371320277452469, "lr": 3.56902248158148e-06, "epoch": 1, "t": 4.080917596817017}
+{"step": 263, "loss": 0.1057412025125269, "grad_norm": 0.1703426092863083, "lr": 3.364520089258727e-06, "epoch": 1, "t": 3.2905826568603516}
+{"step": 264, "loss": 0.10168180001860362, "grad_norm": 0.11194133013486862, "lr": 3.165846763735153e-06, "epoch": 1, "t": 4.672393560409546}
+{"step": 265, "loss": 0.1331891062371674, "grad_norm": 0.15342983603477478, "lr": 2.973027337607892e-06, "epoch": 1, "t": 3.9565773010253906}
+{"step": 266, "loss": 0.10697804405181498, "grad_norm": 0.1219903901219368, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.785395622253418}
+{"step": 267, "loss": 0.11834893132003521, "grad_norm": 0.137263223528862, "lr": 2.605045852462473e-06, "epoch": 1, "t": 3.7666523456573486}
+{"step": 268, "loss": 0.11449980942088152, "grad_norm": 0.1390964686870575, "lr": 2.429929788224722e-06, "epoch": 1, "t": 3.6251273155212402}
+{"step": 269, "loss": 0.10350900557039248, "grad_norm": 0.1608605533838272, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 3.8638718128204346}
+{"step": 270, "loss": 0.13297476776026623, "grad_norm": 0.12646310031414032, "lr": 2.097556454310701e-06, "epoch": 1, "t": 4.377230882644653}
+{"step": 271, "loss": 0.09416118486443835, "grad_norm": 0.11981482058763504, "lr": 1.940340728677059e-06, "epoch": 1, "t": 4.394144296646118}
+{"step": 272, "loss": 0.1178974608100363, "grad_norm": 0.11827477067708969, "lr": 1.789132081018674e-06, "epoch": 1, "t": 4.759323596954346}
+{"step": 273, "loss": 0.10933348167958323, "grad_norm": 0.16263115406036377, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 3.752821922302246}
+{"step": 274, "loss": 0.11521070617593689, "grad_norm": 0.12572331726551056, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 4.048858642578125}
+{"step": 275, "loss": 0.10486421274499416, "grad_norm": 0.13279910385608673, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 3.8803489208221436}
+{"step": 276, "loss": 0.10980342466939344, "grad_norm": 0.12758494913578033, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 3.9871888160705566}
+{"step": 277, "loss": 0.111330353081848, "grad_norm": 0.12176986783742905, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 4.025789976119995}
+{"step": 278, "loss": 0.14237364579852665, "grad_norm": 0.12815888226032257, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 4.214742422103882}
+{"step": 279, "loss": 0.11275946979248894, "grad_norm": 0.13668593764305115, "lr": 9.003621402309814e-07, "epoch": 1, "t": 3.9965884685516357}
+{"step": 280, "loss": 0.13870539862870443, "grad_norm": 0.17668040096759796, "lr": 7.97826969669102e-07, "epoch": 1, "t": 3.775920867919922}
+{"step": 281, "loss": 0.1420387237017605, "grad_norm": 0.14728686213493347, "lr": 7.014416822821556e-07, "epoch": 1, "t": 3.9664599895477295}
+{"step": 282, "loss": 0.14208942107578365, "grad_norm": 0.16048476099967957, "lr": 6.112183254700865e-07, "epoch": 1, "t": 3.8097426891326904}
+{"step": 283, "loss": 0.11025185310842445, "grad_norm": 0.12846560776233673, "lr": 5.271681764401848e-07, "epoch": 1, "t": 4.093057870864868}
+{"step": 284, "loss": 0.120536791877059, "grad_norm": 0.1429244726896286, "lr": 4.493017407975086e-07, "epoch": 1, "t": 4.119860410690308}
+{"step": 285, "loss": 0.1268202349529321, "grad_norm": 0.14483031630516052, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 3.6667308807373047}
+{"step": 286, "loss": 0.10915097912032315, "grad_norm": 0.14104314148426056, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 3.736278533935547}
+{"step": 287, "loss": 0.1157974426430203, "grad_norm": 0.13857784867286682, "lr": 2.528981693106558e-07, "epoch": 1, "t": 3.8293545246124268}
+{"step": 288, "loss": 0.1106615249680059, "grad_norm": 0.14568473398685455, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 3.502797842025757}
+{"step": 289, "loss": 0.11443337037151344, "grad_norm": 0.13623696565628052, "lr": 1.530387900802177e-07, "epoch": 1, "t": 3.5356297492980957}
+{"step": 290, "loss": 0.1373133226415373, "grad_norm": 0.14143982529640198, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 3.7768936157226562}
+{"step": 291, "loss": 0.10547937932961336, "grad_norm": 0.1495388150215149, "lr": 7.81005385163458e-08, "epoch": 1, "t": 3.203394889831543}
+{"step": 292, "loss": 0.08913947800592383, "grad_norm": 0.14194656908512115, "lr": 4.998903085539075e-08, "epoch": 1, "t": 3.2934887409210205}
+{"step": 293, "loss": 0.10755857465675864, "grad_norm": 0.15098439157009125, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 3.4686119556427}
+{"step": 294, "loss": 0.12610472874461887, "grad_norm": 0.14134766161441803, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 3.6605780124664307}
+{"step": 295, "loss": 0.11678951880526135, "grad_norm": 0.1266511082649231, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 4.080137014389038}
+{"step": 296, "loss": 0.09181499846148028, "grad_norm": 0.13486066460609436, "lr": 0.0, "epoch": 1, "t": 3.52030611038208}
diff --git a/qwen3-14b/top-s1/README.md b/qwen3-14b/top-s1/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/top-s1/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/top-s1/adapter_config.json b/qwen3-14b/top-s1/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..3eb4bf226eba4ca670253fcafd49241454a4ed66
--- /dev/null
+++ b/qwen3-14b/top-s1/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "up_proj",
+    "down_proj",
+    "gate_proj",
+    "k_proj",
+    "v_proj",
+    "q_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/top-s1/adapter_model.safetensors b/qwen3-14b/top-s1/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..b3f57b09f043851a9d0098a6b7f4218c7a50fc52
--- /dev/null
+++ b/qwen3-14b/top-s1/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0434876ea4eba6ff4b907f69214cf001a8fc9de1f5577685c80fa6bf67764661
+size 1027680256
diff --git a/qwen3-14b/top-s1/config.json b/qwen3-14b/top-s1/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..8b634ffdf61cf3f27d1c02c33125ebcc94a2f344
--- /dev/null
+++ b/qwen3-14b/top-s1/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-top-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/top-s1",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 1,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/top-s1/log.jsonl b/qwen3-14b/top-s1/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..0973dd82ca81f4ded8a96343d205d86c38192a17
--- /dev/null
+++ b/qwen3-14b/top-s1/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.12617697723187932, "grad_norm": 0.17813117802143097, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 4.346640110015869}
+{"step": 2, "loss": 0.13967789549932588, "grad_norm": 0.20436105132102966, "lr": 2e-05, "epoch": 1, "t": 3.452944755554199}
+{"step": 3, "loss": 0.12777951302112692, "grad_norm": 0.1891518235206604, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 3.8093771934509277}
+{"step": 4, "loss": 0.13342494231642385, "grad_norm": 0.20755107700824738, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 3.5580685138702393}
+{"step": 5, "loss": 0.1617757984819126, "grad_norm": 0.2125258892774582, "lr": 4e-05, "epoch": 1, "t": 3.672642469406128}
+{"step": 6, "loss": 0.15452165704407506, "grad_norm": 0.18916922807693481, "lr": 4.666666666666667e-05, "epoch": 1, "t": 4.006139516830444}
+{"step": 7, "loss": 0.15133708443271515, "grad_norm": 0.25525954365730286, "lr": 5.333333333333333e-05, "epoch": 1, "t": 3.8913304805755615}
+{"step": 8, "loss": 0.13791051797124726, "grad_norm": 0.3746955990791321, "lr": 6e-05, "epoch": 1, "t": 3.5093863010406494}
+{"step": 9, "loss": 0.13368880127060298, "grad_norm": 0.37882909178733826, "lr": 6.666666666666667e-05, "epoch": 1, "t": 3.499830961227417}
+{"step": 10, "loss": 0.16171266748270857, "grad_norm": 0.26269984245300293, "lr": 7.333333333333333e-05, "epoch": 1, "t": 3.828583002090454}
+{"step": 11, "loss": 0.09988783597705776, "grad_norm": 0.2627286911010742, "lr": 8e-05, "epoch": 1, "t": 3.502493381500244}
+{"step": 12, "loss": 0.11244590163474391, "grad_norm": 0.198716402053833, "lr": 8.666666666666667e-05, "epoch": 1, "t": 4.043038606643677}
+{"step": 13, "loss": 0.12859856540223286, "grad_norm": 0.16879867017269135, "lr": 9.333333333333334e-05, "epoch": 1, "t": 3.856217622756958}
+{"step": 14, "loss": 0.14503249441778415, "grad_norm": 0.16829434037208557, "lr": 0.0001, "epoch": 1, "t": 3.9850683212280273}
+{"step": 15, "loss": 0.11640409333792952, "grad_norm": 0.17808358371257782, "lr": 0.0001, "epoch": 1, "t": 3.8164761066436768}
+{"step": 16, "loss": 0.11818495682705238, "grad_norm": 0.17896859347820282, "lr": 9.999687519737639e-05, "epoch": 1, "t": 3.711170196533203}
+{"step": 17, "loss": 0.1061926722354531, "grad_norm": 0.2296096235513687, "lr": 9.998750118008115e-05, "epoch": 1, "t": 3.427849054336548}
+{"step": 18, "loss": 0.13692609377766501, "grad_norm": 0.2245275229215622, "lr": 9.997187911979252e-05, "epoch": 1, "t": 3.591865062713623}
+{"step": 19, "loss": 0.10567690409509033, "grad_norm": 0.17905786633491516, "lr": 9.995001096914461e-05, "epoch": 1, "t": 3.6229166984558105}
+{"step": 20, "loss": 0.12212383321877042, "grad_norm": 0.18408314883708954, "lr": 9.992189946148366e-05, "epoch": 1, "t": 3.684929847717285}
+{"step": 21, "loss": 0.11679173279332113, "grad_norm": 0.18877658247947693, "lr": 9.988754811052616e-05, "epoch": 1, "t": 4.104700326919556}
+{"step": 22, "loss": 0.11977183717930215, "grad_norm": 0.21085374057292938, "lr": 9.984696120991978e-05, "epoch": 1, "t": 3.980527639389038}
+{"step": 23, "loss": 0.12119188574410503, "grad_norm": 0.1878778487443924, "lr": 9.980014383270668e-05, "epoch": 1, "t": 3.816519260406494}
+{"step": 24, "loss": 0.1281769426965335, "grad_norm": 0.18422441184520721, "lr": 9.974710183068935e-05, "epoch": 1, "t": 3.5586957931518555}
+{"step": 25, "loss": 0.11002204728063288, "grad_norm": 0.16555893421173096, "lr": 9.968784183369929e-05, "epoch": 1, "t": 3.4558377265930176}
+{"step": 26, "loss": 0.10575913554629188, "grad_norm": 0.16330499947071075, "lr": 9.962237124876828e-05, "epoch": 1, "t": 3.8771142959594727}
+{"step": 27, "loss": 0.11377116936238087, "grad_norm": 0.18165524303913116, "lr": 9.955069825920249e-05, "epoch": 1, "t": 3.6742002964019775}
+{"step": 28, "loss": 0.14377048029978712, "grad_norm": 0.16228368878364563, "lr": 9.947283182355982e-05, "epoch": 1, "t": 3.8753626346588135}
+{"step": 29, "loss": 0.1088600084404849, "grad_norm": 0.17186380922794342, "lr": 9.938878167452992e-05, "epoch": 1, "t": 3.4087300300598145}
+{"step": 30, "loss": 0.1205528750904442, "grad_norm": 0.17793375253677368, "lr": 9.929855831771786e-05, "epoch": 1, "t": 3.631263494491577}
+{"step": 31, "loss": 0.12006650281989056, "grad_norm": 0.14073535799980164, "lr": 9.92021730303309e-05, "epoch": 1, "t": 3.9572231769561768}
+{"step": 32, "loss": 0.11193672696450599, "grad_norm": 0.17888383567333221, "lr": 9.909963785976903e-05, "epoch": 1, "t": 3.375314235687256}
+{"step": 33, "loss": 0.11699088799521094, "grad_norm": 0.15350720286369324, "lr": 9.899096562211902e-05, "epoch": 1, "t": 3.8531696796417236}
+{"step": 34, "loss": 0.12434252147134568, "grad_norm": 0.15298448503017426, "lr": 9.887616990055262e-05, "epoch": 1, "t": 3.585833787918091}
+{"step": 35, "loss": 0.10578599177971086, "grad_norm": 0.13556434214115143, "lr": 9.875526504362869e-05, "epoch": 1, "t": 3.6398258209228516}
+{"step": 36, "loss": 0.12782733016603726, "grad_norm": 0.19202861189842224, "lr": 9.86282661634998e-05, "epoch": 1, "t": 3.3990650177001953}
+{"step": 37, "loss": 0.1438133662762907, "grad_norm": 0.14495648443698883, "lr": 9.849518913402334e-05, "epoch": 1, "t": 4.429717779159546}
+{"step": 38, "loss": 0.09819187750388492, "grad_norm": 0.197117418050766, "lr": 9.835605058877729e-05, "epoch": 1, "t": 3.423982620239258}
+{"step": 39, "loss": 0.12808304561930858, "grad_norm": 0.16620495915412903, "lr": 9.821086791898134e-05, "epoch": 1, "t": 4.021803140640259}
+{"step": 40, "loss": 0.14976499074751634, "grad_norm": 0.1893635243177414, "lr": 9.805965927132295e-05, "epoch": 1, "t": 3.6762332916259766}
+{"step": 41, "loss": 0.09146835024916676, "grad_norm": 0.18207408487796783, "lr": 9.79024435456893e-05, "epoch": 1, "t": 3.3501293659210205}
+{"step": 42, "loss": 0.09629422448639177, "grad_norm": 0.152657613158226, "lr": 9.773924039280487e-05, "epoch": 1, "t": 3.516054391860962}
+{"step": 43, "loss": 0.12300468028029973, "grad_norm": 0.1671159416437149, "lr": 9.75700702117753e-05, "epoch": 1, "t": 3.5735740661621094}
+{"step": 44, "loss": 0.11457848876561876, "grad_norm": 0.17524519562721252, "lr": 9.739495414753753e-05, "epoch": 1, "t": 3.2422616481781006}
+{"step": 45, "loss": 0.13197105155826871, "grad_norm": 0.1605200320482254, "lr": 9.721391408821711e-05, "epoch": 1, "t": 3.988027811050415}
+{"step": 46, "loss": 0.1044743492844449, "grad_norm": 0.15911930799484253, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.934755802154541}
+{"step": 47, "loss": 0.11060346428374672, "grad_norm": 0.1489032804965973, "lr": 9.683415323626485e-05, "epoch": 1, "t": 3.5727455615997314}
+{"step": 48, "loss": 0.12787591203550833, "grad_norm": 0.17437121272087097, "lr": 9.663547991074127e-05, "epoch": 1, "t": 3.529207468032837}
+{"step": 49, "loss": 0.10476903007184551, "grad_norm": 0.17063632607460022, "lr": 9.643097751841854e-05, "epoch": 1, "t": 3.395947217941284}
+{"step": 50, "loss": 0.11559236101643693, "grad_norm": 0.1584615856409073, "lr": 9.622067162048112e-05, "epoch": 1, "t": 3.6691877841949463}
+{"step": 51, "loss": 0.10311131224587501, "grad_norm": 0.13496831059455872, "lr": 9.600458850350588e-05, "epoch": 1, "t": 4.146857023239136}
+{"step": 52, "loss": 0.13594563325506423, "grad_norm": 0.18607966601848602, "lr": 9.578275517617645e-05, "epoch": 1, "t": 3.3366146087646484}
+{"step": 53, "loss": 0.11605016506891218, "grad_norm": 0.183473140001297, "lr": 9.555519936590738e-05, "epoch": 1, "t": 3.5201375484466553}
+{"step": 54, "loss": 0.12449669365464386, "grad_norm": 0.15700986981391907, "lr": 9.532194951537838e-05, "epoch": 1, "t": 3.604322910308838}
+{"step": 55, "loss": 0.11168685911379647, "grad_norm": 0.14813557267189026, "lr": 9.508303477897924e-05, "epoch": 1, "t": 3.6931560039520264}
+{"step": 56, "loss": 0.10140540842607096, "grad_norm": 0.15404844284057617, "lr": 9.483848501916578e-05, "epoch": 1, "t": 3.526390552520752}
+{"step": 57, "loss": 0.09983278448095059, "grad_norm": 0.11487799882888794, "lr": 9.458833080272722e-05, "epoch": 1, "t": 4.796106815338135}
+{"step": 58, "loss": 0.12303032734977447, "grad_norm": 0.17962698638439178, "lr": 9.433260339696563e-05, "epoch": 1, "t": 3.4310760498046875}
+{"step": 59, "loss": 0.08865170816467315, "grad_norm": 0.18519194424152374, "lr": 9.407133476578778e-05, "epoch": 1, "t": 2.9936156272888184}
+{"step": 60, "loss": 0.1413726049214325, "grad_norm": 0.14544999599456787, "lr": 9.38045575657098e-05, "epoch": 1, "t": 4.22357177734375}
+{"step": 61, "loss": 0.10987983937464454, "grad_norm": 0.13499531149864197, "lr": 9.353230514177552e-05, "epoch": 1, "t": 3.8653571605682373}
+{"step": 62, "loss": 0.12817825367671698, "grad_norm": 0.1388777494430542, "lr": 9.325461152338846e-05, "epoch": 1, "t": 4.058272361755371}
+{"step": 63, "loss": 0.1070455249370468, "grad_norm": 0.15302102267742157, "lr": 9.297151142005851e-05, "epoch": 1, "t": 3.4872753620147705}
+{"step": 64, "loss": 0.12868337080215378, "grad_norm": 0.14045396447181702, "lr": 9.268304021706349e-05, "epoch": 1, "t": 3.791433811187744}
+{"step": 65, "loss": 0.11235940025418158, "grad_norm": 0.14888569712638855, "lr": 9.23892339710263e-05, "epoch": 1, "t": 3.5607590675354004}
+{"step": 66, "loss": 0.1256873486494983, "grad_norm": 0.14114518463611603, "lr": 9.209012940540805e-05, "epoch": 1, "t": 3.7805917263031006}
+{"step": 67, "loss": 0.10378119902211907, "grad_norm": 0.16029345989227295, "lr": 9.178576390591802e-05, "epoch": 1, "t": 3.246281862258911}
+{"step": 68, "loss": 0.1216837474301, "grad_norm": 0.153811514377594, "lr": 9.147617551584066e-05, "epoch": 1, "t": 3.6362340450286865}
+{"step": 69, "loss": 0.11043250781767563, "grad_norm": 0.14451350271701813, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.501593589782715}
+{"step": 70, "loss": 0.13497135237627994, "grad_norm": 0.1602874994277954, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.5931949615478516}
+{"step": 71, "loss": 0.09722692632834887, "grad_norm": 0.15740317106246948, "lr": 9.051646319812918e-05, "epoch": 1, "t": 3.4226741790771484}
+{"step": 72, "loss": 0.09790374907315819, "grad_norm": 0.16020715236663818, "lr": 9.018637666191283e-05, "epoch": 1, "t": 3.4990334510803223}
+{"step": 73, "loss": 0.14803199967062625, "grad_norm": 0.15978007018566132, "lr": 8.985126714588738e-05, "epoch": 1, "t": 3.5505919456481934}
+{"step": 74, "loss": 0.1155597930667044, "grad_norm": 0.1583298146724701, "lr": 8.951117653609666e-05, "epoch": 1, "t": 3.800234794616699}
+{"step": 75, "loss": 0.10957571930138177, "grad_norm": 0.14239458739757538, "lr": 8.916614734118184e-05, "epoch": 1, "t": 3.99151611328125}
+{"step": 76, "loss": 0.11006540318636451, "grad_norm": 0.18153788149356842, "lr": 8.881622268706825e-05, "epoch": 1, "t": 3.8811874389648438}
+{"step": 77, "loss": 0.1017583479631397, "grad_norm": 0.15703681111335754, "lr": 8.8461446311575e-05, "epoch": 1, "t": 3.8155720233917236}
+{"step": 78, "loss": 0.14508525815997383, "grad_norm": 0.16783204674720764, "lr": 8.810186255894803e-05, "epoch": 1, "t": 3.9199094772338867}
+{"step": 79, "loss": 0.14189885733331228, "grad_norm": 0.16029906272888184, "lr": 8.773751637431748e-05, "epoch": 1, "t": 3.776341438293457}
+{"step": 80, "loss": 0.11862062740506679, "grad_norm": 0.14433375000953674, "lr": 8.736845329807993e-05, "epoch": 1, "t": 4.121500492095947}
+{"step": 81, "loss": 0.11648394842515145, "grad_norm": 0.14172229170799255, "lr": 8.69947194602061e-05, "epoch": 1, "t": 3.982224464416504}
+{"step": 82, "loss": 0.14382076474505184, "grad_norm": 0.14935703575611115, "lr": 8.66163615744751e-05, "epoch": 1, "t": 4.5979578495025635}
+{"step": 83, "loss": 0.12411033987198897, "grad_norm": 0.1570126712322235, "lr": 8.623342693263548e-05, "epoch": 1, "t": 4.38858437538147}
+{"step": 84, "loss": 0.10168998042169841, "grad_norm": 0.1542520821094513, "lr": 8.584596339849417e-05, "epoch": 1, "t": 3.8072776794433594}
+{"step": 85, "loss": 0.10301344397371759, "grad_norm": 0.20114772021770477, "lr": 8.545401940193392e-05, "epoch": 1, "t": 2.949744701385498}
+{"step": 86, "loss": 0.12391188198358319, "grad_norm": 0.14330871403217316, "lr": 8.505764393285984e-05, "epoch": 1, "t": 3.8098771572113037}
+{"step": 87, "loss": 0.13544891011753513, "grad_norm": 0.16698111593723297, "lr": 8.46568865350762e-05, "epoch": 1, "t": 3.609095573425293}
+{"step": 88, "loss": 0.1253294855020565, "grad_norm": 0.1395493894815445, "lr": 8.425179730009368e-05, "epoch": 1, "t": 3.861459732055664}
+{"step": 89, "loss": 0.1358275218569494, "grad_norm": 0.16607223451137543, "lr": 8.384242686086848e-05, "epoch": 1, "t": 3.9377362728118896}
+{"step": 90, "loss": 0.12201537374805248, "grad_norm": 0.136156365275383, "lr": 8.342882638547351e-05, "epoch": 1, "t": 4.135454416275024}
+{"step": 91, "loss": 0.10596731170808091, "grad_norm": 0.12559491395950317, "lr": 8.301104757070274e-05, "epoch": 1, "t": 4.039834260940552}
+{"step": 92, "loss": 0.11235995745489867, "grad_norm": 0.21693256497383118, "lr": 8.258914263560971e-05, "epoch": 1, "t": 3.578596353530884}
+{"step": 93, "loss": 0.11738971001624124, "grad_norm": 0.18173369765281677, "lr": 8.216316431498028e-05, "epoch": 1, "t": 3.5737931728363037}
+{"step": 94, "loss": 0.10259573527859518, "grad_norm": 0.13276751339435577, "lr": 8.173316585274145e-05, "epoch": 1, "t": 3.725597858428955}
+{"step": 95, "loss": 0.12137065043185806, "grad_norm": 0.16400939226150513, "lr": 8.129920099530607e-05, "epoch": 1, "t": 3.5122134685516357}
+{"step": 96, "loss": 0.11086015578720942, "grad_norm": 0.1350613534450531, "lr": 8.086132398485524e-05, "epoch": 1, "t": 3.899817943572998}
+{"step": 97, "loss": 0.11875248614092002, "grad_norm": 0.15366894006729126, "lr": 8.041958955255814e-05, "epoch": 1, "t": 3.6690890789031982}
+{"step": 98, "loss": 0.11087692391230373, "grad_norm": 0.16304782032966614, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.3700172901153564}
+{"step": 99, "loss": 0.10914460758822364, "grad_norm": 0.15879666805267334, "lr": 7.952476975093729e-05, "epoch": 1, "t": 3.5521280765533447}
+{"step": 100, "loss": 0.12244844047515606, "grad_norm": 0.1741693615913391, "lr": 7.907179622702408e-05, "epoch": 1, "t": 3.5211782455444336}
+{"step": 101, "loss": 0.09923361588185982, "grad_norm": 0.17092706263065338, "lr": 7.861518895810596e-05, "epoch": 1, "t": 3.2267937660217285}
+{"step": 102, "loss": 0.09599852976716058, "grad_norm": 0.14872629940509796, "lr": 7.815500501648653e-05, "epoch": 1, "t": 3.311434745788574}
+{"step": 103, "loss": 0.11346121397593542, "grad_norm": 0.1248384416103363, "lr": 7.769130192152538e-05, "epoch": 1, "t": 3.793327569961548}
+{"step": 104, "loss": 0.1331907953006339, "grad_norm": 0.13637416064739227, "lr": 7.722413763244838e-05, "epoch": 1, "t": 3.6841018199920654}
+{"step": 105, "loss": 0.13123013496533492, "grad_norm": 0.14148768782615662, "lr": 7.675357054110336e-05, "epoch": 1, "t": 3.711219072341919}
+{"step": 106, "loss": 0.10144862798495237, "grad_norm": 0.13383527100086212, "lr": 7.627965946466166e-05, "epoch": 1, "t": 3.7547600269317627}
+{"step": 107, "loss": 0.08842463516314251, "grad_norm": 0.1317060887813568, "lr": 7.580246363826621e-05, "epoch": 1, "t": 3.7979235649108887}
+{"step": 108, "loss": 0.1424757622127872, "grad_norm": 0.15373225510120392, "lr": 7.532204270762786e-05, "epoch": 1, "t": 3.8947222232818604}
+{"step": 109, "loss": 0.09783356644100161, "grad_norm": 0.1271030455827713, "lr": 7.483845672156998e-05, "epoch": 1, "t": 4.047677993774414}
+{"step": 110, "loss": 0.15625576080435916, "grad_norm": 0.16558495163917542, "lr": 7.435176612452286e-05, "epoch": 1, "t": 3.7398111820220947}
+{"step": 111, "loss": 0.060958140714325176, "grad_norm": 0.09934616088867188, "lr": 7.386203174896872e-05, "epoch": 1, "t": 5.033056974411011}
+{"step": 112, "loss": 0.14364481765455747, "grad_norm": 0.16380451619625092, "lr": 7.336931480783801e-05, "epoch": 1, "t": 3.6545825004577637}
+{"step": 113, "loss": 0.09798778755870564, "grad_norm": 0.15225090086460114, "lr": 7.287367688685835e-05, "epoch": 1, "t": 3.813899040222168}
+{"step": 114, "loss": 0.1208999756211953, "grad_norm": 0.1456039994955063, "lr": 7.237517993685678e-05, "epoch": 1, "t": 4.134164571762085}
+{"step": 115, "loss": 0.15594158785763815, "grad_norm": 0.1761215776205063, "lr": 7.187388626601637e-05, "epoch": 1, "t": 3.7116971015930176}
+{"step": 116, "loss": 0.1556412040798022, "grad_norm": 0.187168687582016, "lr": 7.136985853208824e-05, "epoch": 1, "t": 3.4049878120422363}
+{"step": 117, "loss": 0.12004507252368724, "grad_norm": 0.1756347119808197, "lr": 7.086315973455981e-05, "epoch": 1, "t": 3.260153293609619}
+{"step": 118, "loss": 0.10562873026713816, "grad_norm": 0.14095406234264374, "lr": 7.035385320678036e-05, "epoch": 1, "t": 3.5944700241088867}
+{"step": 119, "loss": 0.0955803687391903, "grad_norm": 0.14971613883972168, "lr": 6.984200260804484e-05, "epoch": 1, "t": 3.3246121406555176}
+{"step": 120, "loss": 0.11592486825482597, "grad_norm": 0.14291374385356903, "lr": 6.932767191563703e-05, "epoch": 1, "t": 3.4365973472595215}
+{"step": 121, "loss": 0.12285661407223093, "grad_norm": 0.14880289137363434, "lr": 6.881092541683278e-05, "epoch": 1, "t": 3.4892210960388184}
+{"step": 122, "loss": 0.13448336943188596, "grad_norm": 0.13309575617313385, "lr": 6.829182770086474e-05, "epoch": 1, "t": 3.780925750732422}
+{"step": 123, "loss": 0.10142241036406076, "grad_norm": 0.1549137383699417, "lr": 6.777044365084907e-05, "epoch": 1, "t": 3.2879579067230225}
+{"step": 124, "loss": 0.13075157536609774, "grad_norm": 0.13092055916786194, "lr": 6.724683843567568e-05, "epoch": 1, "t": 4.04414701461792}
+{"step": 125, "loss": 0.12712555972847403, "grad_norm": 0.1429220587015152, "lr": 6.672107750186255e-05, "epoch": 1, "t": 3.646151304244995}
+{"step": 126, "loss": 0.12920168392612774, "grad_norm": 0.1343962848186493, "lr": 6.619322656537552e-05, "epoch": 1, "t": 5.038818359375}
+{"step": 127, "loss": 0.10136577885574173, "grad_norm": 0.12121925503015518, "lr": 6.566335160341424e-05, "epoch": 1, "t": 5.140141487121582}
+{"step": 128, "loss": 0.09307052863413096, "grad_norm": 0.13577862083911896, "lr": 6.513151884616556e-05, "epoch": 1, "t": 4.1879308223724365}
+{"step": 129, "loss": 0.12334084800693415, "grad_norm": 0.12855924665927887, "lr": 6.459779476852528e-05, "epoch": 1, "t": 3.9371771812438965}
+{"step": 130, "loss": 0.11223760366868375, "grad_norm": 0.13784851133823395, "lr": 6.406224608178932e-05, "epoch": 1, "t": 3.7771332263946533}
+{"step": 131, "loss": 0.11287350980118192, "grad_norm": 0.1648254245519638, "lr": 6.352493972531534e-05, "epoch": 1, "t": 3.3348419666290283}
+{"step": 132, "loss": 0.11832260398167344, "grad_norm": 0.1223011463880539, "lr": 6.298594285815584e-05, "epoch": 1, "t": 4.256714582443237}
+{"step": 133, "loss": 0.11724697059787849, "grad_norm": 0.1436534821987152, "lr": 6.244532285066382e-05, "epoch": 1, "t": 3.8852901458740234}
+{"step": 134, "loss": 0.08801555215116899, "grad_norm": 0.1354561150074005, "lr": 6.190314727607196e-05, "epoch": 1, "t": 3.344627618789673}
+{"step": 135, "loss": 0.10998289528294489, "grad_norm": 0.13848434388637543, "lr": 6.13594839020466e-05, "epoch": 1, "t": 3.6931068897247314}
+{"step": 136, "loss": 0.10389392825298893, "grad_norm": 0.13557109236717224, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 3.5528459548950195}
+{"step": 137, "loss": 0.11015337319018649, "grad_norm": 0.15919683873653412, "lr": 6.026796574768288e-05, "epoch": 1, "t": 3.544654130935669}
+{"step": 138, "loss": 0.17025340805432232, "grad_norm": 0.14549554884433746, "lr": 5.972024739849622e-05, "epoch": 1, "t": 3.7516157627105713}
+{"step": 139, "loss": 0.1206067166301217, "grad_norm": 0.14922426640987396, "lr": 5.917131409512663e-05, "epoch": 1, "t": 3.539113759994507}
+{"step": 140, "loss": 0.09171864116218467, "grad_norm": 0.15125498175621033, "lr": 5.862123444990318e-05, "epoch": 1, "t": 4.467451095581055}
+{"step": 141, "loss": 0.13509653512342476, "grad_norm": 0.16121678054332733, "lr": 5.807007721843861e-05, "epoch": 1, "t": 3.6902096271514893}
+{"step": 142, "loss": 0.12958719738723237, "grad_norm": 0.12824058532714844, "lr": 5.751791129103544e-05, "epoch": 1, "t": 4.089444875717163}
+{"step": 143, "loss": 0.0904153858986925, "grad_norm": 0.17429909110069275, "lr": 5.696480568407523e-05, "epoch": 1, "t": 3.3153958320617676}
+{"step": 144, "loss": 0.12444486430231427, "grad_norm": 0.1337614804506302, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 3.944141149520874}
+{"step": 145, "loss": 0.10648673117538839, "grad_norm": 0.15813423693180084, "lr": 5.585605207563124e-05, "epoch": 1, "t": 3.4917004108428955}
+{"step": 146, "loss": 0.12888574645238343, "grad_norm": 0.1726931631565094, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 3.387598752975464}
+{"step": 147, "loss": 0.11660225879969113, "grad_norm": 0.14439034461975098, "lr": 5.47443707175741e-05, "epoch": 1, "t": 3.523622989654541}
+{"step": 148, "loss": 0.12972238279902812, "grad_norm": 0.14789223670959473, "lr": 5.418760576667071e-05, "epoch": 1, "t": 3.531308650970459}
+{"step": 149, "loss": 0.11599285788692681, "grad_norm": 0.16743241250514984, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 3.527240753173828}
+{"step": 150, "loss": 0.10536329533588719, "grad_norm": 0.11961741745471954, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 4.246665000915527}
+{"step": 151, "loss": 0.14691426849451356, "grad_norm": 0.13908380270004272, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 3.7891228199005127}
+{"step": 152, "loss": 0.09125569267584521, "grad_norm": 0.13070951402187347, "lr": 5.195600862779421e-05, "epoch": 1, "t": 3.789931058883667}
+{"step": 153, "loss": 0.13953947879440354, "grad_norm": 0.14746491611003876, "lr": 5.139732367864736e-05, "epoch": 1, "t": 3.7273690700531006}
+{"step": 154, "loss": 0.11148364722441742, "grad_norm": 0.143972709774971, "lr": 5.083846407507263e-05, "epoch": 1, "t": 3.6797475814819336}
+{"step": 155, "loss": 0.11711150210139401, "grad_norm": 0.14206048846244812, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 3.696898937225342}
+{"step": 156, "loss": 0.15427738406982708, "grad_norm": 0.1594860702753067, "lr": 4.972050032989175e-05, "epoch": 1, "t": 3.5698680877685547}
+{"step": 157, "loss": 0.12941604046194416, "grad_norm": 0.14126090705394745, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 3.5420234203338623}
+{"step": 158, "loss": 0.11358545952472371, "grad_norm": 0.14078235626220703, "lr": 4.860267632135265e-05, "epoch": 1, "t": 3.698453903198242}
+{"step": 159, "loss": 0.12307799438795167, "grad_norm": 0.12719698250293732, "lr": 4.80439913722058e-05, "epoch": 1, "t": 4.065964937210083}
+{"step": 160, "loss": 0.12387896494480613, "grad_norm": 0.15634509921073914, "lr": 4.748555090869464e-05, "epoch": 1, "t": 3.57331919670105}
+{"step": 161, "loss": 0.09991615371246224, "grad_norm": 0.1264018416404724, "lr": 4.692742473146818e-05, "epoch": 1, "t": 3.632742404937744}
+{"step": 162, "loss": 0.12465437132188027, "grad_norm": 0.1434398889541626, "lr": 4.636968260189214e-05, "epoch": 1, "t": 3.5576515197753906}
+{"step": 163, "loss": 0.12388255781901172, "grad_norm": 0.1393527388572693, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 3.8255491256713867}
+{"step": 164, "loss": 0.12084446088242522, "grad_norm": 0.17217890918254852, "lr": 4.525562928242592e-05, "epoch": 1, "t": 3.3586931228637695}
+{"step": 165, "loss": 0.1345335028478219, "grad_norm": 0.13619305193424225, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 3.8575119972229004}
+{"step": 166, "loss": 0.1036650218029347, "grad_norm": 0.19194954633712769, "lr": 4.414394792436877e-05, "epoch": 1, "t": 3.160275936126709}
+{"step": 167, "loss": 0.11326826187008518, "grad_norm": 0.14025457203388214, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 3.786343574523926}
+{"step": 168, "loss": 0.12417489809701429, "grad_norm": 0.13368594646453857, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 3.9190316200256348}
+{"step": 169, "loss": 0.12834236412930955, "grad_norm": 0.1635846197605133, "lr": 4.248208870896456e-05, "epoch": 1, "t": 3.3846564292907715}
+{"step": 170, "loss": 0.11016614631457139, "grad_norm": 0.1439928561449051, "lr": 4.192992278156141e-05, "epoch": 1, "t": 3.478168487548828}
+{"step": 171, "loss": 0.1232400970203155, "grad_norm": 0.13067671656608582, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 3.5561423301696777}
+{"step": 172, "loss": 0.10309260972309663, "grad_norm": 0.11806591600179672, "lr": 4.082868590487339e-05, "epoch": 1, "t": 4.4137773513793945}
+{"step": 173, "loss": 0.10267287523915186, "grad_norm": 0.13546214997768402, "lr": 4.027975260150381e-05, "epoch": 1, "t": 3.5982131958007812}
+{"step": 174, "loss": 0.09851670742671544, "grad_norm": 0.15855532884597778, "lr": 3.973203425231715e-05, "epoch": 1, "t": 3.081353187561035}
+{"step": 175, "loss": 0.1051363697270025, "grad_norm": 0.14807681739330292, "lr": 3.918559931778277e-05, "epoch": 1, "t": 3.279660224914551}
+{"step": 176, "loss": 0.12372557784930728, "grad_norm": 0.11470877379179001, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 4.647943019866943}
+{"step": 177, "loss": 0.1283996960689329, "grad_norm": 0.12203007191419601, "lr": 3.809685272392804e-05, "epoch": 1, "t": 4.256687164306641}
+{"step": 178, "loss": 0.12578085667703723, "grad_norm": 0.15542949736118317, "lr": 3.755467714933619e-05, "epoch": 1, "t": 3.3761346340179443}
+{"step": 179, "loss": 0.11705847765581288, "grad_norm": 0.1348995566368103, "lr": 3.701405714184416e-05, "epoch": 1, "t": 3.631040573120117}
+{"step": 180, "loss": 0.14166584991507514, "grad_norm": 0.13458091020584106, "lr": 3.647506027468467e-05, "epoch": 1, "t": 3.945761203765869}
+{"step": 181, "loss": 0.1306085179456594, "grad_norm": 0.13540928065776825, "lr": 3.59377539182107e-05, "epoch": 1, "t": 3.7256736755371094}
+{"step": 182, "loss": 0.10765460822662014, "grad_norm": 0.12800486385822296, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 3.8136699199676514}
+{"step": 183, "loss": 0.11371282885808624, "grad_norm": 0.15802575647830963, "lr": 3.486848115383445e-05, "epoch": 1, "t": 3.2331976890563965}
+{"step": 184, "loss": 0.0981744162632213, "grad_norm": 0.12216585129499435, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 4.3312835693359375}
+{"step": 185, "loss": 0.09754395796285337, "grad_norm": 0.12377235293388367, "lr": 3.380677343462447e-05, "epoch": 1, "t": 3.685432195663452}
+{"step": 186, "loss": 0.10171388364734175, "grad_norm": 0.1388912945985794, "lr": 3.327892249813745e-05, "epoch": 1, "t": 3.4424314498901367}
+{"step": 187, "loss": 0.11433426206265453, "grad_norm": 0.12249286472797394, "lr": 3.275316156432434e-05, "epoch": 1, "t": 3.8866002559661865}
+{"step": 188, "loss": 0.12818598997567252, "grad_norm": 0.14571289718151093, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 3.622340440750122}
+{"step": 189, "loss": 0.11476655278382929, "grad_norm": 0.16208785772323608, "lr": 3.170817229913526e-05, "epoch": 1, "t": 3.9373676776885986}
+{"step": 190, "loss": 0.12519697124724427, "grad_norm": 0.12768830358982086, "lr": 3.118907458316722e-05, "epoch": 1, "t": 4.257965803146362}
+{"step": 191, "loss": 0.12079559410890633, "grad_norm": 0.1425876021385193, "lr": 3.067232808436299e-05, "epoch": 1, "t": 3.6903438568115234}
+{"step": 192, "loss": 0.10877684433039318, "grad_norm": 0.14198388159275055, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 4.07475209236145}
+{"step": 193, "loss": 0.10177911991465753, "grad_norm": 0.14601974189281464, "lr": 2.964614679321966e-05, "epoch": 1, "t": 3.6123125553131104}
+{"step": 194, "loss": 0.12034727400129862, "grad_norm": 0.14945223927497864, "lr": 2.913684026544021e-05, "epoch": 1, "t": 3.6465508937835693}
+{"step": 195, "loss": 0.10917313186498856, "grad_norm": 0.14296695590019226, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 3.62443470954895}
+{"step": 196, "loss": 0.13535631156859323, "grad_norm": 0.12707377970218658, "lr": 2.812611373398365e-05, "epoch": 1, "t": 4.325379848480225}
+{"step": 197, "loss": 0.10377255856641744, "grad_norm": 0.11476591974496841, "lr": 2.762482006314324e-05, "epoch": 1, "t": 4.497923851013184}
+{"step": 198, "loss": 0.09315204227495219, "grad_norm": 0.135996475815773, "lr": 2.712632311314165e-05, "epoch": 1, "t": 3.340762138366699}
+{"step": 199, "loss": 0.10277280397872067, "grad_norm": 0.12450498342514038, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 4.061821699142456}
+{"step": 200, "loss": 0.14604022735465685, "grad_norm": 0.14762699604034424, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 3.9157187938690186}
+{"step": 201, "loss": 0.10821579465528296, "grad_norm": 0.13636647164821625, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 3.563469171524048}
+{"step": 202, "loss": 0.08465742839857676, "grad_norm": 0.11804912984371185, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 4.388417720794678}
+{"step": 203, "loss": 0.10028685424578777, "grad_norm": 0.13766880333423615, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 3.6335527896881104}
+{"step": 204, "loss": 0.14448020692205404, "grad_norm": 0.15696823596954346, "lr": 2.419753636173379e-05, "epoch": 1, "t": 3.720081329345703}
+{"step": 205, "loss": 0.08705586053030567, "grad_norm": 0.13298670947551727, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 3.5988824367523193}
+{"step": 206, "loss": 0.1338730433557052, "grad_norm": 0.14060519635677338, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 3.829575538635254}
+{"step": 207, "loss": 0.12115982575225166, "grad_norm": 0.14780060946941376, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 4.328840494155884}
+{"step": 208, "loss": 0.09969542106584364, "grad_norm": 0.13209664821624756, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 3.957324743270874}
+{"step": 209, "loss": 0.1095413927000518, "grad_norm": 0.147843599319458, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 3.6246933937072754}
+{"step": 210, "loss": 0.12827260363703616, "grad_norm": 0.16618376970291138, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 3.487652063369751}
+{"step": 211, "loss": 0.09563076235182276, "grad_norm": 0.16873812675476074, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 3.27763295173645}
+{"step": 212, "loss": 0.11493629372368257, "grad_norm": 0.14851205050945282, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 3.40578556060791}
+{"step": 213, "loss": 0.10988328247070311, "grad_norm": 0.13131429255008698, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 3.583291530609131}
+{"step": 214, "loss": 0.12714762132230814, "grad_norm": 0.12987689673900604, "lr": 1.958041044744186e-05, "epoch": 1, "t": 4.034169435501099}
+{"step": 215, "loss": 0.13301799405779532, "grad_norm": 0.17417165637016296, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 3.347989559173584}
+{"step": 216, "loss": 0.10980320312516527, "grad_norm": 0.12944453954696655, "lr": 1.870079900469392e-05, "epoch": 1, "t": 3.93153715133667}
+{"step": 217, "loss": 0.11175066650432564, "grad_norm": 0.17680276930332184, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 3.0879065990448}
+{"step": 218, "loss": 0.1070946088177522, "grad_norm": 0.12824957072734833, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 3.8204569816589355}
+{"step": 219, "loss": 0.13382778469529735, "grad_norm": 0.1532336175441742, "lr": 1.741085736439031e-05, "epoch": 1, "t": 3.5585274696350098}
+{"step": 220, "loss": 0.12347208028093738, "grad_norm": 0.13383693993091583, "lr": 1.698895242929725e-05, "epoch": 1, "t": 3.8323400020599365}
+{"step": 221, "loss": 0.09301368938050676, "grad_norm": 0.14995865523815155, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 3.588886260986328}
+{"step": 222, "loss": 0.10457991148088822, "grad_norm": 0.1490771621465683, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 3.399606227874756}
+{"step": 223, "loss": 0.10260286581379038, "grad_norm": 0.14631585776805878, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 3.940944194793701}
+{"step": 224, "loss": 0.09172309889239169, "grad_norm": 0.141727015376091, "lr": 1.534311346492381e-05, "epoch": 1, "t": 3.4267871379852295}
+{"step": 225, "loss": 0.11374407764858398, "grad_norm": 0.146555557847023, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 3.592529296875}
+{"step": 226, "loss": 0.06580967870135651, "grad_norm": 0.11124950647354126, "lr": 1.454598059806609e-05, "epoch": 1, "t": 4.4580628871917725}
+{"step": 227, "loss": 0.09310311438271637, "grad_norm": 0.10735554993152618, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 4.612629413604736}
+{"step": 228, "loss": 0.10898556553197136, "grad_norm": 0.12116745859384537, "lr": 1.376657306736453e-05, "epoch": 1, "t": 4.26071834564209}
+{"step": 229, "loss": 0.08790608545924856, "grad_norm": 0.13979801535606384, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 3.3781697750091553}
+{"step": 230, "loss": 0.11287608131555191, "grad_norm": 0.14813432097434998, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 3.574429750442505}
+{"step": 231, "loss": 0.10926746632679397, "grad_norm": 0.15620684623718262, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 3.2295730113983154}
+{"step": 232, "loss": 0.06341660996172142, "grad_norm": 0.1311025768518448, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 3.8195106983184814}
+{"step": 233, "loss": 0.1346529558094637, "grad_norm": 0.1514120101928711, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 3.6388118267059326}
+{"step": 234, "loss": 0.12326520871898357, "grad_norm": 0.13807372748851776, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 3.495877265930176}
+{"step": 235, "loss": 0.10937284330775818, "grad_norm": 0.14105282723903656, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 4.014347076416016}
+{"step": 236, "loss": 0.08711449902692286, "grad_norm": 0.15301650762557983, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 3.277312755584717}
+{"step": 237, "loss": 0.12649809740102813, "grad_norm": 0.1330740749835968, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 3.691643714904785}
+{"step": 238, "loss": 0.11305359116628583, "grad_norm": 0.16088685393333435, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 3.3242287635803223}
+{"step": 239, "loss": 0.11382307437936587, "grad_norm": 0.1273585706949234, "lr": 9.81362333808718e-06, "epoch": 1, "t": 3.9288711547851562}
+{"step": 240, "loss": 0.08081836175363621, "grad_norm": 0.1453140825033188, "lr": 9.483536801870834e-06, "epoch": 1, "t": 3.0839498043060303}
+{"step": 241, "loss": 0.12736113323212958, "grad_norm": 0.14109191298484802, "lr": 9.158514503674543e-06, "epoch": 1, "t": 3.535029649734497}
+{"step": 242, "loss": 0.0992514305560199, "grad_norm": 0.1593686044216156, "lr": 8.838597068719518e-06, "epoch": 1, "t": 3.1627578735351562}
+{"step": 243, "loss": 0.12921056042787946, "grad_norm": 0.15639816224575043, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.534938335418701}
+{"step": 244, "loss": 0.07484405072272209, "grad_norm": 0.13559651374816895, "lr": 8.21423609408199e-06, "epoch": 1, "t": 3.189039468765259}
+{"step": 245, "loss": 0.08054880225456086, "grad_norm": 0.14232437312602997, "lr": 7.90987059459195e-06, "epoch": 1, "t": 3.3425464630126953}
+{"step": 246, "loss": 0.17963109672933383, "grad_norm": 0.14338843524456024, "lr": 7.610766028973709e-06, "epoch": 1, "t": 3.9058361053466797}
+{"step": 247, "loss": 0.09183027094365563, "grad_norm": 0.12234976887702942, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 4.2730231285095215}
+{"step": 248, "loss": 0.09873497818757208, "grad_norm": 0.14907951653003693, "lr": 7.028488579941506e-06, "epoch": 1, "t": 3.458730936050415}
+{"step": 249, "loss": 0.17370311981546885, "grad_norm": 0.16469648480415344, "lr": 6.745388476611553e-06, "epoch": 1, "t": 3.747774839401245}
+{"step": 250, "loss": 0.09090033047569454, "grad_norm": 0.14682556688785553, "lr": 6.467694858224488e-06, "epoch": 1, "t": 3.3244922161102295}
+{"step": 251, "loss": 0.15026991808063267, "grad_norm": 0.1395793855190277, "lr": 6.1954424342902e-06, "epoch": 1, "t": 4.221449136734009}
+{"step": 252, "loss": 0.08533400387109356, "grad_norm": 0.14570271968841553, "lr": 5.928665234212233e-06, "epoch": 1, "t": 3.2162327766418457}
+{"step": 253, "loss": 0.11475269408648225, "grad_norm": 0.12250617891550064, "lr": 5.66739660303437e-06, "epoch": 1, "t": 4.5142669677734375}
+{"step": 254, "loss": 0.11549933321774004, "grad_norm": 0.1397608369588852, "lr": 5.411669197272795e-06, "epoch": 1, "t": 3.565253257751465}
+{"step": 255, "loss": 0.10628338955520808, "grad_norm": 0.15242017805576324, "lr": 5.161514980834231e-06, "epoch": 1, "t": 3.2479002475738525}
+{"step": 256, "loss": 0.10941664840044635, "grad_norm": 0.15288929641246796, "lr": 4.916965221020753e-06, "epoch": 1, "t": 3.4475889205932617}
+{"step": 257, "loss": 0.11360626193040345, "grad_norm": 0.14277198910713196, "lr": 4.678050484621615e-06, "epoch": 1, "t": 3.7326180934906006}
+{"step": 258, "loss": 0.1273740844508461, "grad_norm": 0.13272571563720703, "lr": 4.444800634092616e-06, "epoch": 1, "t": 4.063765287399292}
+{"step": 259, "loss": 0.09969869257779225, "grad_norm": 0.150743305683136, "lr": 4.217244823823546e-06, "epoch": 1, "t": 3.519660234451294}
+{"step": 260, "loss": 0.09885429833725147, "grad_norm": 0.1521507352590561, "lr": 3.995411496494134e-06, "epoch": 1, "t": 3.3937342166900635}
+{"step": 261, "loss": 0.12922761713404862, "grad_norm": 0.14709627628326416, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 3.4807889461517334}
+{"step": 262, "loss": 0.12014580580113114, "grad_norm": 0.1306675374507904, "lr": 3.56902248158148e-06, "epoch": 1, "t": 3.916518449783325}
+{"step": 263, "loss": 0.11713323430438272, "grad_norm": 0.1347883641719818, "lr": 3.364520089258727e-06, "epoch": 1, "t": 3.5773062705993652}
+{"step": 264, "loss": 0.1438285928228695, "grad_norm": 0.13864943385124207, "lr": 3.165846763735153e-06, "epoch": 1, "t": 4.006881475448608}
+{"step": 265, "loss": 0.1245681589240478, "grad_norm": 0.1318804770708084, "lr": 2.973027337607892e-06, "epoch": 1, "t": 3.8025317192077637}
+{"step": 266, "loss": 0.14530098809782405, "grad_norm": 0.1331164389848709, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.8494632244110107}
+{"step": 267, "loss": 0.1700842685074063, "grad_norm": 0.13135068118572235, "lr": 2.605045852462473e-06, "epoch": 1, "t": 4.168143272399902}
+{"step": 268, "loss": 0.13068006154082787, "grad_norm": 0.13416430354118347, "lr": 2.429929788224722e-06, "epoch": 1, "t": 3.8213531970977783}
+{"step": 269, "loss": 0.08564913835858418, "grad_norm": 0.141933411359787, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 3.738593578338623}
+{"step": 270, "loss": 0.09905013582450904, "grad_norm": 0.12381220608949661, "lr": 2.097556454310701e-06, "epoch": 1, "t": 4.249961614608765}
+{"step": 271, "loss": 0.11759098345033525, "grad_norm": 0.14351508021354675, "lr": 1.940340728677059e-06, "epoch": 1, "t": 3.527235507965088}
+{"step": 272, "loss": 0.12464060810556439, "grad_norm": 0.1532280296087265, "lr": 1.789132081018674e-06, "epoch": 1, "t": 3.5108704566955566}
+{"step": 273, "loss": 0.1359373972792612, "grad_norm": 0.1518130600452423, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 3.826458215713501}
+{"step": 274, "loss": 0.12103388992853831, "grad_norm": 0.14426681399345398, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 3.6588857173919678}
+{"step": 275, "loss": 0.11953243650036094, "grad_norm": 0.14271962642669678, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 3.380645751953125}
+{"step": 276, "loss": 0.12918474836375562, "grad_norm": 0.16321291029453278, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 3.676669120788574}
+{"step": 277, "loss": 0.08142634204359425, "grad_norm": 0.14214715361595154, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 3.3022356033325195}
+{"step": 278, "loss": 0.12297851671776515, "grad_norm": 0.13073664903640747, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 3.934837579727173}
+{"step": 279, "loss": 0.08723520301049784, "grad_norm": 0.14627036452293396, "lr": 9.003621402309814e-07, "epoch": 1, "t": 3.4166531562805176}
+{"step": 280, "loss": 0.08955092498975327, "grad_norm": 0.13474299013614655, "lr": 7.97826969669102e-07, "epoch": 1, "t": 3.5985665321350098}
+{"step": 281, "loss": 0.10957729286173967, "grad_norm": 0.1504601389169693, "lr": 7.014416822821556e-07, "epoch": 1, "t": 3.4060721397399902}
+{"step": 282, "loss": 0.11969655710138577, "grad_norm": 0.14200237393379211, "lr": 6.112183254700865e-07, "epoch": 1, "t": 3.657917022705078}
+{"step": 283, "loss": 0.12179722018396985, "grad_norm": 0.13962052762508392, "lr": 5.271681764401848e-07, "epoch": 1, "t": 3.6534767150878906}
+{"step": 284, "loss": 0.12423437291299357, "grad_norm": 0.1347407102584839, "lr": 4.493017407975086e-07, "epoch": 1, "t": 3.730149507522583}
+{"step": 285, "loss": 0.09884172717709837, "grad_norm": 0.1294063776731491, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 4.321979522705078}
+{"step": 286, "loss": 0.11894170053629595, "grad_norm": 0.16102586686611176, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 3.4120187759399414}
+{"step": 287, "loss": 0.1010571739998631, "grad_norm": 0.12278228998184204, "lr": 2.528981693106558e-07, "epoch": 1, "t": 3.919614553451538}
+{"step": 288, "loss": 0.08412016936738798, "grad_norm": 0.16988617181777954, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 3.295806407928467}
+{"step": 289, "loss": 0.13807679567250433, "grad_norm": 0.15589822828769684, "lr": 1.530387900802177e-07, "epoch": 1, "t": 3.9307994842529297}
+{"step": 290, "loss": 0.10602280350193366, "grad_norm": 0.14829131960868835, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 3.432567834854126}
+{"step": 291, "loss": 0.09269226847600595, "grad_norm": 0.14671681821346283, "lr": 7.81005385163458e-08, "epoch": 1, "t": 3.2810075283050537}
+{"step": 292, "loss": 0.10162069618621868, "grad_norm": 0.1418626457452774, "lr": 4.998903085539075e-08, "epoch": 1, "t": 3.3429131507873535}
+{"step": 293, "loss": 0.12984618940723944, "grad_norm": 0.1609625518321991, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 3.4562764167785645}
+{"step": 294, "loss": 0.10010424084329439, "grad_norm": 0.14943931996822357, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 3.271085500717163}
+{"step": 295, "loss": 0.1370399821600205, "grad_norm": 0.1424897015094757, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 3.9097979068756104}
+{"step": 296, "loss": 0.12105428717271338, "grad_norm": 0.14088213443756104, "lr": 0.0, "epoch": 1, "t": 3.653798818588257}
diff --git a/qwen3-14b/top-s2/README.md b/qwen3-14b/top-s2/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/top-s2/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/top-s2/adapter_config.json b/qwen3-14b/top-s2/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..76a7fab225e29a2afdc8a1aaa6aa7a940b5b91f0
--- /dev/null
+++ b/qwen3-14b/top-s2/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "down_proj",
+    "k_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/top-s2/adapter_model.safetensors b/qwen3-14b/top-s2/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..e8fa1c8e62158dfaa8a6a1d5376c693df5e29dde
--- /dev/null
+++ b/qwen3-14b/top-s2/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0e3220dab9c1d4037e8973099c40a6537b8b05095a7103d333601cad515c23e4
+size 1027680256
diff --git a/qwen3-14b/top-s2/config.json b/qwen3-14b/top-s2/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..8286b4ba2b7d03707303980fc22dc35f1471cac6
--- /dev/null
+++ b/qwen3-14b/top-s2/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-top-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/top-s2",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 2,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/top-s2/log.jsonl b/qwen3-14b/top-s2/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..e4a245141d26a4fc84cf017ac31cf1b85c3f7b07
--- /dev/null
+++ b/qwen3-14b/top-s2/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.13903235479545312, "grad_norm": 0.18691591918468475, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 6.864739418029785}
+{"step": 2, "loss": 0.18642684179521257, "grad_norm": 0.21507515013217926, "lr": 2e-05, "epoch": 1, "t": 5.495803117752075}
+{"step": 3, "loss": 0.11128579936671272, "grad_norm": 0.17726357281208038, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 5.328658580780029}
+{"step": 4, "loss": 0.12394382909098332, "grad_norm": 0.18305252492427826, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 5.441368103027344}
+{"step": 5, "loss": 0.12213580176107608, "grad_norm": 0.19631154835224152, "lr": 4e-05, "epoch": 1, "t": 5.178787469863892}
+{"step": 6, "loss": 0.12199781524670858, "grad_norm": 0.19172978401184082, "lr": 4.666666666666667e-05, "epoch": 1, "t": 5.180421829223633}
+{"step": 7, "loss": 0.1364712164348162, "grad_norm": 0.2786177694797516, "lr": 5.333333333333333e-05, "epoch": 1, "t": 5.311373949050903}
+{"step": 8, "loss": 0.11555326839577275, "grad_norm": 0.25720471143722534, "lr": 6e-05, "epoch": 1, "t": 5.8692872524261475}
+{"step": 9, "loss": 0.1026448891899107, "grad_norm": 0.4983462691307068, "lr": 6.666666666666667e-05, "epoch": 1, "t": 5.104641437530518}
+{"step": 10, "loss": 0.14682528886423368, "grad_norm": 0.2922855317592621, "lr": 7.333333333333333e-05, "epoch": 1, "t": 5.169619083404541}
+{"step": 11, "loss": 0.1148453009314835, "grad_norm": 0.28939303755760193, "lr": 8e-05, "epoch": 1, "t": 5.0628180503845215}
+{"step": 12, "loss": 0.09771283208258691, "grad_norm": 0.23056207597255707, "lr": 8.666666666666667e-05, "epoch": 1, "t": 5.242952346801758}
+{"step": 13, "loss": 0.15001912677989288, "grad_norm": 0.202907532453537, "lr": 9.333333333333334e-05, "epoch": 1, "t": 5.133257627487183}
+{"step": 14, "loss": 0.1387809903577459, "grad_norm": 0.23532724380493164, "lr": 0.0001, "epoch": 1, "t": 5.161443471908569}
+{"step": 15, "loss": 0.06586550699337856, "grad_norm": 0.14754711091518402, "lr": 0.0001, "epoch": 1, "t": 5.511684417724609}
+{"step": 16, "loss": 0.1186719206781346, "grad_norm": 0.21970734000205994, "lr": 9.999687519737639e-05, "epoch": 1, "t": 4.56630539894104}
+{"step": 17, "loss": 0.12690814547967705, "grad_norm": 0.18918105959892273, "lr": 9.998750118008115e-05, "epoch": 1, "t": 4.73888635635376}
+{"step": 18, "loss": 0.10853050770582978, "grad_norm": 0.14272886514663696, "lr": 9.997187911979252e-05, "epoch": 1, "t": 6.093342542648315}
+{"step": 19, "loss": 0.12345775734136803, "grad_norm": 0.1963808834552765, "lr": 9.995001096914461e-05, "epoch": 1, "t": 4.621027231216431}
+{"step": 20, "loss": 0.10624837157890944, "grad_norm": 0.22859708964824677, "lr": 9.992189946148366e-05, "epoch": 1, "t": 4.56150484085083}
+{"step": 21, "loss": 0.1040551171891782, "grad_norm": 0.18460801243782043, "lr": 9.988754811052616e-05, "epoch": 1, "t": 4.663488149642944}
+{"step": 22, "loss": 0.1398770229222542, "grad_norm": 0.2133590430021286, "lr": 9.984696120991978e-05, "epoch": 1, "t": 4.89215350151062}
+{"step": 23, "loss": 0.1369036741142771, "grad_norm": 0.17992180585861206, "lr": 9.980014383270668e-05, "epoch": 1, "t": 4.092624187469482}
+{"step": 24, "loss": 0.11078876366194303, "grad_norm": 0.1668865978717804, "lr": 9.974710183068935e-05, "epoch": 1, "t": 4.052527904510498}
+{"step": 25, "loss": 0.10283707233865906, "grad_norm": 0.17949998378753662, "lr": 9.968784183369929e-05, "epoch": 1, "t": 4.039080381393433}
+{"step": 26, "loss": 0.15775464723656224, "grad_norm": 0.1804666966199875, "lr": 9.962237124876828e-05, "epoch": 1, "t": 4.4249958992004395}
+{"step": 27, "loss": 0.1089669922119915, "grad_norm": 0.1938660442829132, "lr": 9.955069825920249e-05, "epoch": 1, "t": 3.335313558578491}
+{"step": 28, "loss": 0.11441239717264079, "grad_norm": 0.1972186267375946, "lr": 9.947283182355982e-05, "epoch": 1, "t": 3.330704927444458}
+{"step": 29, "loss": 0.10607078026451458, "grad_norm": 0.1655188351869583, "lr": 9.938878167452992e-05, "epoch": 1, "t": 3.6399688720703125}
+{"step": 30, "loss": 0.13462438682388708, "grad_norm": 0.14942032098770142, "lr": 9.929855831771786e-05, "epoch": 1, "t": 4.546204566955566}
+{"step": 31, "loss": 0.14836710208436188, "grad_norm": 0.15340925753116608, "lr": 9.92021730303309e-05, "epoch": 1, "t": 4.276471376419067}
+{"step": 32, "loss": 0.1033451549724333, "grad_norm": 0.15596725046634674, "lr": 9.909963785976903e-05, "epoch": 1, "t": 4.19510293006897}
+{"step": 33, "loss": 0.09271919839616781, "grad_norm": 0.1414995938539505, "lr": 9.899096562211902e-05, "epoch": 1, "t": 3.7739243507385254}
+{"step": 34, "loss": 0.1625036641314465, "grad_norm": 0.1845017522573471, "lr": 9.887616990055262e-05, "epoch": 1, "t": 3.887876033782959}
+{"step": 35, "loss": 0.14665475779888676, "grad_norm": 0.1603679209947586, "lr": 9.875526504362869e-05, "epoch": 1, "t": 4.188788414001465}
+{"step": 36, "loss": 0.1352753917741975, "grad_norm": 0.19020523130893707, "lr": 9.86282661634998e-05, "epoch": 1, "t": 3.9792890548706055}
+{"step": 37, "loss": 0.13594455461109425, "grad_norm": 0.14979976415634155, "lr": 9.849518913402334e-05, "epoch": 1, "t": 4.253630638122559}
+{"step": 38, "loss": 0.13333558052526504, "grad_norm": 0.16773559153079987, "lr": 9.835605058877729e-05, "epoch": 1, "t": 3.575190782546997}
+{"step": 39, "loss": 0.08763617863370786, "grad_norm": 0.14162959158420563, "lr": 9.821086791898134e-05, "epoch": 1, "t": 3.5569889545440674}
+{"step": 40, "loss": 0.10258940749349649, "grad_norm": 0.1467287391424179, "lr": 9.805965927132295e-05, "epoch": 1, "t": 3.7881999015808105}
+{"step": 41, "loss": 0.14698505547299429, "grad_norm": 0.18592801690101624, "lr": 9.79024435456893e-05, "epoch": 1, "t": 3.672590970993042}
+{"step": 42, "loss": 0.1160221549828746, "grad_norm": 0.16695132851600647, "lr": 9.773924039280487e-05, "epoch": 1, "t": 3.4817025661468506}
+{"step": 43, "loss": 0.08964649383127347, "grad_norm": 0.17507533729076385, "lr": 9.75700702117753e-05, "epoch": 1, "t": 3.3905749320983887}
+{"step": 44, "loss": 0.10352311829367271, "grad_norm": 0.16485117375850677, "lr": 9.739495414753753e-05, "epoch": 1, "t": 3.3336985111236572}
+{"step": 45, "loss": 0.10975483155037404, "grad_norm": 0.16509714722633362, "lr": 9.721391408821711e-05, "epoch": 1, "t": 3.499523401260376}
+{"step": 46, "loss": 0.09604111826558676, "grad_norm": 0.15528228878974915, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.602381944656372}
+{"step": 47, "loss": 0.15729760275867674, "grad_norm": 0.15533128380775452, "lr": 9.683415323626485e-05, "epoch": 1, "t": 4.068080186843872}
+{"step": 48, "loss": 0.11879632526644507, "grad_norm": 0.163217693567276, "lr": 9.663547991074127e-05, "epoch": 1, "t": 3.4843509197235107}
+{"step": 49, "loss": 0.13814949473402002, "grad_norm": 0.17024819552898407, "lr": 9.643097751841854e-05, "epoch": 1, "t": 3.4343416690826416}
+{"step": 50, "loss": 0.09578245971839465, "grad_norm": 0.1568632274866104, "lr": 9.622067162048112e-05, "epoch": 1, "t": 3.2537460327148438}
+{"step": 51, "loss": 0.1661884207403463, "grad_norm": 0.16602765023708344, "lr": 9.600458850350588e-05, "epoch": 1, "t": 4.111520767211914}
+{"step": 52, "loss": 0.12818270364768616, "grad_norm": 0.16334186494350433, "lr": 9.578275517617645e-05, "epoch": 1, "t": 3.6875972747802734}
+{"step": 53, "loss": 0.12568404001218303, "grad_norm": 0.17432846128940582, "lr": 9.555519936590738e-05, "epoch": 1, "t": 3.5432868003845215}
+{"step": 54, "loss": 0.1374257833346943, "grad_norm": 0.16003505885601044, "lr": 9.532194951537838e-05, "epoch": 1, "t": 3.797150135040283}
+{"step": 55, "loss": 0.13595183539012123, "grad_norm": 0.16132542490959167, "lr": 9.508303477897924e-05, "epoch": 1, "t": 3.505662441253662}
+{"step": 56, "loss": 0.12508177704107568, "grad_norm": 0.15689826011657715, "lr": 9.483848501916578e-05, "epoch": 1, "t": 3.986131191253662}
+{"step": 57, "loss": 0.09661101080486248, "grad_norm": 0.15625135600566864, "lr": 9.458833080272722e-05, "epoch": 1, "t": 3.664118766784668}
+{"step": 58, "loss": 0.09268714219473367, "grad_norm": 0.1818946897983551, "lr": 9.433260339696563e-05, "epoch": 1, "t": 3.3993043899536133}
+{"step": 59, "loss": 0.12073000745736817, "grad_norm": 0.13196638226509094, "lr": 9.407133476578778e-05, "epoch": 1, "t": 4.410968065261841}
+{"step": 60, "loss": 0.11646890817111, "grad_norm": 0.1672857701778412, "lr": 9.38045575657098e-05, "epoch": 1, "t": 3.5757737159729004}
+{"step": 61, "loss": 0.12169068478878885, "grad_norm": 0.1509135216474533, "lr": 9.353230514177552e-05, "epoch": 1, "t": 3.5008013248443604}
+{"step": 62, "loss": 0.1246174254851176, "grad_norm": 0.1579459011554718, "lr": 9.325461152338846e-05, "epoch": 1, "t": 3.6325533390045166}
+{"step": 63, "loss": 0.10592049717668431, "grad_norm": 0.14569959044456482, "lr": 9.297151142005851e-05, "epoch": 1, "t": 3.4117653369903564}
+{"step": 64, "loss": 0.09376015076694647, "grad_norm": 0.13964180648326874, "lr": 9.268304021706349e-05, "epoch": 1, "t": 3.586988925933838}
+{"step": 65, "loss": 0.12132058226644332, "grad_norm": 0.16833175718784332, "lr": 9.23892339710263e-05, "epoch": 1, "t": 3.478654384613037}
+{"step": 66, "loss": 0.10293037362982478, "grad_norm": 0.15626882016658783, "lr": 9.209012940540805e-05, "epoch": 1, "t": 3.319349765777588}
+{"step": 67, "loss": 0.07490977109647265, "grad_norm": 0.15595267713069916, "lr": 9.178576390591802e-05, "epoch": 1, "t": 3.395246744155884}
+{"step": 68, "loss": 0.13019493175900612, "grad_norm": 0.14851199090480804, "lr": 9.147617551584066e-05, "epoch": 1, "t": 4.0612101554870605}
+{"step": 69, "loss": 0.14147672349345802, "grad_norm": 0.15433450043201447, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.968419313430786}
+{"step": 70, "loss": 0.13425695779062816, "grad_norm": 0.18751974403858185, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.746925115585327}
+{"step": 71, "loss": 0.14047300855881617, "grad_norm": 0.16933654248714447, "lr": 9.051646319812918e-05, "epoch": 1, "t": 3.6014297008514404}
+{"step": 72, "loss": 0.10870128261799715, "grad_norm": 0.18203002214431763, "lr": 9.018637666191283e-05, "epoch": 1, "t": 4.267744064331055}
+{"step": 73, "loss": 0.11368323894820566, "grad_norm": 0.12437428534030914, "lr": 8.985126714588738e-05, "epoch": 1, "t": 3.961292266845703}
+{"step": 74, "loss": 0.11787486099890872, "grad_norm": 0.1750839501619339, "lr": 8.951117653609666e-05, "epoch": 1, "t": 3.319948196411133}
+{"step": 75, "loss": 0.10792613368362788, "grad_norm": 0.14015400409698486, "lr": 8.916614734118184e-05, "epoch": 1, "t": 3.6884305477142334}
+{"step": 76, "loss": 0.08867398826434683, "grad_norm": 0.1677970588207245, "lr": 8.881622268706825e-05, "epoch": 1, "t": 3.4960949420928955}
+{"step": 77, "loss": 0.13048783756706475, "grad_norm": 0.10839230567216873, "lr": 8.8461446311575e-05, "epoch": 1, "t": 5.356718063354492}
+{"step": 78, "loss": 0.12220291914611028, "grad_norm": 0.1625511646270752, "lr": 8.810186255894803e-05, "epoch": 1, "t": 3.906745195388794}
+{"step": 79, "loss": 0.10727703738611928, "grad_norm": 0.14660009741783142, "lr": 8.773751637431748e-05, "epoch": 1, "t": 3.8182528018951416}
+{"step": 80, "loss": 0.1156752438390501, "grad_norm": 0.12983396649360657, "lr": 8.736845329807993e-05, "epoch": 1, "t": 4.508897066116333}
+{"step": 81, "loss": 0.10720597240659924, "grad_norm": 0.17227385938167572, "lr": 8.69947194602061e-05, "epoch": 1, "t": 3.830923080444336}
+{"step": 82, "loss": 0.1379632680957416, "grad_norm": 0.15516285598278046, "lr": 8.66163615744751e-05, "epoch": 1, "t": 4.034547567367554}
+{"step": 83, "loss": 0.1205892566464952, "grad_norm": 0.16279365122318268, "lr": 8.623342693263548e-05, "epoch": 1, "t": 3.9818737506866455}
+{"step": 84, "loss": 0.12185889370552529, "grad_norm": 0.14962351322174072, "lr": 8.584596339849417e-05, "epoch": 1, "t": 4.2080607414245605}
+{"step": 85, "loss": 0.1480261938277827, "grad_norm": 0.15322256088256836, "lr": 8.545401940193392e-05, "epoch": 1, "t": 4.20061993598938}
+{"step": 86, "loss": 0.11446748332210747, "grad_norm": 0.16113662719726562, "lr": 8.505764393285984e-05, "epoch": 1, "t": 3.7204275131225586}
+{"step": 87, "loss": 0.11569758558706951, "grad_norm": 0.1471674144268036, "lr": 8.46568865350762e-05, "epoch": 1, "t": 4.9261462688446045}
+{"step": 88, "loss": 0.10102688809996238, "grad_norm": 0.17423070967197418, "lr": 8.425179730009368e-05, "epoch": 1, "t": 4.902527332305908}
+{"step": 89, "loss": 0.1004634014981185, "grad_norm": 0.1297309547662735, "lr": 8.384242686086848e-05, "epoch": 1, "t": 4.872568368911743}
+{"step": 90, "loss": 0.09690162155897876, "grad_norm": 0.1463904082775116, "lr": 8.342882638547351e-05, "epoch": 1, "t": 5.14177131652832}
+{"step": 91, "loss": 0.12132977440719206, "grad_norm": 0.13613452017307281, "lr": 8.301104757070274e-05, "epoch": 1, "t": 5.179162502288818}
+{"step": 92, "loss": 0.11027849756051122, "grad_norm": 0.13941965997219086, "lr": 8.258914263560971e-05, "epoch": 1, "t": 5.140499830245972}
+{"step": 93, "loss": 0.10921254746550678, "grad_norm": 0.14533889293670654, "lr": 8.216316431498028e-05, "epoch": 1, "t": 5.091244220733643}
+{"step": 94, "loss": 0.09990940598628982, "grad_norm": 0.1502157598733902, "lr": 8.173316585274145e-05, "epoch": 1, "t": 4.544970750808716}
+{"step": 95, "loss": 0.12303229695982551, "grad_norm": 0.12711842358112335, "lr": 8.129920099530607e-05, "epoch": 1, "t": 4.28910756111145}
+{"step": 96, "loss": 0.11684299268070389, "grad_norm": 0.14489944279193878, "lr": 8.086132398485524e-05, "epoch": 1, "t": 3.9226324558258057}
+{"step": 97, "loss": 0.09842703326540589, "grad_norm": 0.1431637704372406, "lr": 8.041958955255814e-05, "epoch": 1, "t": 3.9009037017822266}
+{"step": 98, "loss": 0.11299892828460958, "grad_norm": 0.17163683474063873, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.6051204204559326}
+{"step": 99, "loss": 0.1138883744444146, "grad_norm": 0.1301882266998291, "lr": 7.952476975093729e-05, "epoch": 1, "t": 4.099380016326904}
+{"step": 100, "loss": 0.09696658605533813, "grad_norm": 0.15771359205245972, "lr": 7.907179622702408e-05, "epoch": 1, "t": 3.441956043243408}
+{"step": 101, "loss": 0.10000838070882821, "grad_norm": 0.14419683814048767, "lr": 7.861518895810596e-05, "epoch": 1, "t": 3.736294984817505}
+{"step": 102, "loss": 0.10341655487498634, "grad_norm": 0.14287513494491577, "lr": 7.815500501648653e-05, "epoch": 1, "t": 3.704041004180908}
+{"step": 103, "loss": 0.10354404411580126, "grad_norm": 0.17020243406295776, "lr": 7.769130192152538e-05, "epoch": 1, "t": 4.536968469619751}
+{"step": 104, "loss": 0.12326906849292248, "grad_norm": 0.1542179435491562, "lr": 7.722413763244838e-05, "epoch": 1, "t": 4.603079080581665}
+{"step": 105, "loss": 0.11953143003722523, "grad_norm": 0.14972983300685883, "lr": 7.675357054110336e-05, "epoch": 1, "t": 4.361727476119995}
+{"step": 106, "loss": 0.12147631063064612, "grad_norm": 0.18339772522449493, "lr": 7.627965946466166e-05, "epoch": 1, "t": 3.4952640533447266}
+{"step": 107, "loss": 0.10222580745311009, "grad_norm": 0.14436127245426178, "lr": 7.580246363826621e-05, "epoch": 1, "t": 4.42513632774353}
+{"step": 108, "loss": 0.12040524039556864, "grad_norm": 0.17839989066123962, "lr": 7.532204270762786e-05, "epoch": 1, "t": 3.5947964191436768}
+{"step": 109, "loss": 0.09472360438933819, "grad_norm": 0.1802179217338562, "lr": 7.483845672156998e-05, "epoch": 1, "t": 3.285153388977051}
+{"step": 110, "loss": 0.09446371458542639, "grad_norm": 0.12989963591098785, "lr": 7.435176612452286e-05, "epoch": 1, "t": 3.8462471961975098}
+{"step": 111, "loss": 0.10175636353482136, "grad_norm": 0.16729183495044708, "lr": 7.386203174896872e-05, "epoch": 1, "t": 3.397797107696533}
+{"step": 112, "loss": 0.10568978630568497, "grad_norm": 0.13438227772712708, "lr": 7.336931480783801e-05, "epoch": 1, "t": 3.6507251262664795}
+{"step": 113, "loss": 0.07046771070322094, "grad_norm": 0.19553524255752563, "lr": 7.287367688685835e-05, "epoch": 1, "t": 2.898075580596924}
+{"step": 114, "loss": 0.13684492050896171, "grad_norm": 0.17505201697349548, "lr": 7.237517993685678e-05, "epoch": 1, "t": 3.4407448768615723}
+{"step": 115, "loss": 0.10446413579077454, "grad_norm": 0.1643892228603363, "lr": 7.187388626601637e-05, "epoch": 1, "t": 3.174499750137329}
+{"step": 116, "loss": 0.09990062499841487, "grad_norm": 0.15415890514850616, "lr": 7.136985853208824e-05, "epoch": 1, "t": 3.916940212249756}
+{"step": 117, "loss": 0.09846968869769362, "grad_norm": 0.1610376089811325, "lr": 7.086315973455981e-05, "epoch": 1, "t": 3.262200117111206}
+{"step": 118, "loss": 0.12277775135109033, "grad_norm": 0.16945838928222656, "lr": 7.035385320678036e-05, "epoch": 1, "t": 3.7256360054016113}
+{"step": 119, "loss": 0.13695464896027154, "grad_norm": 0.13485902547836304, "lr": 6.984200260804484e-05, "epoch": 1, "t": 4.107189416885376}
+{"step": 120, "loss": 0.14180696907058385, "grad_norm": 0.17467090487480164, "lr": 6.932767191563703e-05, "epoch": 1, "t": 3.6011698246002197}
+{"step": 121, "loss": 0.11784688548377369, "grad_norm": 0.1612841933965683, "lr": 6.881092541683278e-05, "epoch": 1, "t": 3.5180106163024902}
+{"step": 122, "loss": 0.11324725431324845, "grad_norm": 0.16094669699668884, "lr": 6.829182770086474e-05, "epoch": 1, "t": 3.6547584533691406}
+{"step": 123, "loss": 0.1130481305560694, "grad_norm": 0.13501060009002686, "lr": 6.777044365084907e-05, "epoch": 1, "t": 3.7187747955322266}
+{"step": 124, "loss": 0.10628939167027375, "grad_norm": 0.16607427597045898, "lr": 6.724683843567568e-05, "epoch": 1, "t": 3.4490275382995605}
+{"step": 125, "loss": 0.10047142254594667, "grad_norm": 0.1368291676044464, "lr": 6.672107750186255e-05, "epoch": 1, "t": 3.776353120803833}
+{"step": 126, "loss": 0.12150467341622255, "grad_norm": 0.1366211324930191, "lr": 6.619322656537552e-05, "epoch": 1, "t": 3.7147679328918457}
+{"step": 127, "loss": 0.10394700928649474, "grad_norm": 0.1423984318971634, "lr": 6.566335160341424e-05, "epoch": 1, "t": 3.4270834922790527}
+{"step": 128, "loss": 0.10858605803015295, "grad_norm": 0.14049559831619263, "lr": 6.513151884616556e-05, "epoch": 1, "t": 3.653644323348999}
+{"step": 129, "loss": 0.12351805907365115, "grad_norm": 0.14907100796699524, "lr": 6.459779476852528e-05, "epoch": 1, "t": 3.5609004497528076}
+{"step": 130, "loss": 0.11400704705655636, "grad_norm": 0.11552445590496063, "lr": 6.406224608178932e-05, "epoch": 1, "t": 4.2871363162994385}
+{"step": 131, "loss": 0.12729016067155827, "grad_norm": 0.120306596159935, "lr": 6.352493972531534e-05, "epoch": 1, "t": 4.442035436630249}
+{"step": 132, "loss": 0.11014071847372221, "grad_norm": 0.14029784500598907, "lr": 6.298594285815584e-05, "epoch": 1, "t": 3.8519930839538574}
+{"step": 133, "loss": 0.1126389271436548, "grad_norm": 0.1607287973165512, "lr": 6.244532285066382e-05, "epoch": 1, "t": 3.4949421882629395}
+{"step": 134, "loss": 0.14711026296118204, "grad_norm": 0.16673004627227783, "lr": 6.190314727607196e-05, "epoch": 1, "t": 3.66593074798584}
+{"step": 135, "loss": 0.10317313440454431, "grad_norm": 0.1898699402809143, "lr": 6.13594839020466e-05, "epoch": 1, "t": 3.201864004135132}
+{"step": 136, "loss": 0.10081362742157539, "grad_norm": 0.17306023836135864, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 3.264387845993042}
+{"step": 137, "loss": 0.1371166071298072, "grad_norm": 0.14700132608413696, "lr": 6.026796574768288e-05, "epoch": 1, "t": 4.000636100769043}
+{"step": 138, "loss": 0.09819045987887162, "grad_norm": 0.12261544167995453, "lr": 5.972024739849622e-05, "epoch": 1, "t": 3.8031504154205322}
+{"step": 139, "loss": 0.11227856792668793, "grad_norm": 0.15662407875061035, "lr": 5.917131409512663e-05, "epoch": 1, "t": 3.4284157752990723}
+{"step": 140, "loss": 0.13130800647568772, "grad_norm": 0.16081401705741882, "lr": 5.862123444990318e-05, "epoch": 1, "t": 3.30610728263855}
+{"step": 141, "loss": 0.10549267096520933, "grad_norm": 0.14677999913692474, "lr": 5.807007721843861e-05, "epoch": 1, "t": 3.6190755367279053}
+{"step": 142, "loss": 0.09221887915021897, "grad_norm": 0.1853833943605423, "lr": 5.751791129103544e-05, "epoch": 1, "t": 3.0678720474243164}
+{"step": 143, "loss": 0.10809711013156625, "grad_norm": 0.19103415310382843, "lr": 5.696480568407523e-05, "epoch": 1, "t": 3.2923667430877686}
+{"step": 144, "loss": 0.09200119729099798, "grad_norm": 0.15374243259429932, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 3.324209213256836}
+{"step": 145, "loss": 0.08963247515287423, "grad_norm": 0.14326363801956177, "lr": 5.585605207563124e-05, "epoch": 1, "t": 3.300374746322632}
+{"step": 146, "loss": 0.1271197352077628, "grad_norm": 0.13003697991371155, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 4.254032135009766}
+{"step": 147, "loss": 0.10792646038462586, "grad_norm": 0.13604527711868286, "lr": 5.47443707175741e-05, "epoch": 1, "t": 3.6472439765930176}
+{"step": 148, "loss": 0.10689669943921754, "grad_norm": 0.12806211411952972, "lr": 5.418760576667071e-05, "epoch": 1, "t": 4.904598951339722}
+{"step": 149, "loss": 0.13108099405121523, "grad_norm": 0.14352519810199738, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 4.390496253967285}
+{"step": 150, "loss": 0.11339209117078673, "grad_norm": 0.15391698479652405, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 3.5340001583099365}
+{"step": 151, "loss": 0.1035971251651457, "grad_norm": 0.1408761441707611, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 3.7192673683166504}
+{"step": 152, "loss": 0.11195093427113031, "grad_norm": 0.1513042002916336, "lr": 5.195600862779421e-05, "epoch": 1, "t": 3.5607311725616455}
+{"step": 153, "loss": 0.11917894407527777, "grad_norm": 0.15667138993740082, "lr": 5.139732367864736e-05, "epoch": 1, "t": 3.351611614227295}
+{"step": 154, "loss": 0.09135388231465445, "grad_norm": 0.13934899866580963, "lr": 5.083846407507263e-05, "epoch": 1, "t": 3.4295804500579834}
+{"step": 155, "loss": 0.11992695268941636, "grad_norm": 0.14967402815818787, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 3.532069206237793}
+{"step": 156, "loss": 0.11618499800717058, "grad_norm": 0.15720117092132568, "lr": 4.972050032989175e-05, "epoch": 1, "t": 3.3944740295410156}
+{"step": 157, "loss": 0.10557764194457726, "grad_norm": 0.12787263095378876, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 4.449055194854736}
+{"step": 158, "loss": 0.1124829648128107, "grad_norm": 0.12175461649894714, "lr": 4.860267632135265e-05, "epoch": 1, "t": 4.620234966278076}
+{"step": 159, "loss": 0.11942319013781927, "grad_norm": 0.15093454718589783, "lr": 4.80439913722058e-05, "epoch": 1, "t": 3.915315866470337}
+{"step": 160, "loss": 0.09650375155293786, "grad_norm": 0.1464422047138214, "lr": 4.748555090869464e-05, "epoch": 1, "t": 3.7556653022766113}
+{"step": 161, "loss": 0.1167345025890897, "grad_norm": 0.1474517434835434, "lr": 4.692742473146818e-05, "epoch": 1, "t": 3.687807083129883}
+{"step": 162, "loss": 0.11550987395177406, "grad_norm": 0.14103259146213531, "lr": 4.636968260189214e-05, "epoch": 1, "t": 4.084846258163452}
+{"step": 163, "loss": 0.11559182125085765, "grad_norm": 0.1426371932029724, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 3.9823808670043945}
+{"step": 164, "loss": 0.0943347802739144, "grad_norm": 0.12580768764019012, "lr": 4.525562928242592e-05, "epoch": 1, "t": 4.422297477722168}
+{"step": 165, "loss": 0.1382610732452617, "grad_norm": 0.16197247803211212, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 3.8307621479034424}
+{"step": 166, "loss": 0.1382220540869853, "grad_norm": 0.13612425327301025, "lr": 4.414394792436877e-05, "epoch": 1, "t": 4.48817777633667}
+{"step": 167, "loss": 0.11237072727292084, "grad_norm": 0.16068992018699646, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 3.7255876064300537}
+{"step": 168, "loss": 0.1302426293845069, "grad_norm": 0.1531248539686203, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 4.135506629943848}
+{"step": 169, "loss": 0.09966100582286067, "grad_norm": 0.15213651955127716, "lr": 4.248208870896456e-05, "epoch": 1, "t": 3.688032865524292}
+{"step": 170, "loss": 0.10899822985503213, "grad_norm": 0.16141372919082642, "lr": 4.192992278156141e-05, "epoch": 1, "t": 3.9388372898101807}
+{"step": 171, "loss": 0.14176865956781998, "grad_norm": 0.13430117070674896, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 4.168799877166748}
+{"step": 172, "loss": 0.11407513695002766, "grad_norm": 0.13133546710014343, "lr": 4.082868590487339e-05, "epoch": 1, "t": 4.201272964477539}
+{"step": 173, "loss": 0.08813214408785716, "grad_norm": 0.14218084514141083, "lr": 4.027975260150381e-05, "epoch": 1, "t": 4.790307283401489}
+{"step": 174, "loss": 0.11944543168028694, "grad_norm": 0.16195736825466156, "lr": 3.973203425231715e-05, "epoch": 1, "t": 4.938755750656128}
+{"step": 175, "loss": 0.12177802597208226, "grad_norm": 0.15716810524463654, "lr": 3.918559931778277e-05, "epoch": 1, "t": 4.589896202087402}
+{"step": 176, "loss": 0.12759149464273534, "grad_norm": 0.1523430198431015, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 4.768307447433472}
+{"step": 177, "loss": 0.12117194877202381, "grad_norm": 0.16162808239459991, "lr": 3.809685272392804e-05, "epoch": 1, "t": 4.621376037597656}
+{"step": 178, "loss": 0.07637745691630655, "grad_norm": 0.15107233822345734, "lr": 3.755467714933619e-05, "epoch": 1, "t": 4.5747339725494385}
+{"step": 179, "loss": 0.11611811133310818, "grad_norm": 0.17700903117656708, "lr": 3.701405714184416e-05, "epoch": 1, "t": 4.543384313583374}
+{"step": 180, "loss": 0.13742828814137487, "grad_norm": 0.14458781480789185, "lr": 3.647506027468467e-05, "epoch": 1, "t": 4.233373641967773}
+{"step": 181, "loss": 0.13342492702986167, "grad_norm": 0.1493581384420395, "lr": 3.59377539182107e-05, "epoch": 1, "t": 4.032809019088745}
+{"step": 182, "loss": 0.11744640125589505, "grad_norm": 0.14650888741016388, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 3.877549409866333}
+{"step": 183, "loss": 0.1260729749463701, "grad_norm": 0.15691816806793213, "lr": 3.486848115383445e-05, "epoch": 1, "t": 5.265307426452637}
+{"step": 184, "loss": 0.09119362643533985, "grad_norm": 0.13730184733867645, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 5.163676500320435}
+{"step": 185, "loss": 0.09841817827946997, "grad_norm": 0.14752115309238434, "lr": 3.380677343462447e-05, "epoch": 1, "t": 5.065349340438843}
+{"step": 186, "loss": 0.10569100604999733, "grad_norm": 0.14756996929645538, "lr": 3.327892249813745e-05, "epoch": 1, "t": 4.946471214294434}
+{"step": 187, "loss": 0.09880940406198924, "grad_norm": 0.15062151849269867, "lr": 3.275316156432434e-05, "epoch": 1, "t": 5.127771377563477}
+{"step": 188, "loss": 0.10583417482865162, "grad_norm": 0.1195639818906784, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 5.75699257850647}
+{"step": 189, "loss": 0.1439209506715366, "grad_norm": 0.15688404440879822, "lr": 3.170817229913526e-05, "epoch": 1, "t": 5.054985046386719}
+{"step": 190, "loss": 0.13042229557011642, "grad_norm": 0.149184450507164, "lr": 3.118907458316722e-05, "epoch": 1, "t": 5.004969596862793}
+{"step": 191, "loss": 0.11382201424727205, "grad_norm": 0.13252809643745422, "lr": 3.067232808436299e-05, "epoch": 1, "t": 4.256673097610474}
+{"step": 192, "loss": 0.1341661999843064, "grad_norm": 0.15716558694839478, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 3.522207021713257}
+{"step": 193, "loss": 0.14188155988540366, "grad_norm": 0.12510307133197784, "lr": 2.964614679321966e-05, "epoch": 1, "t": 4.082975149154663}
+{"step": 194, "loss": 0.1186425903608214, "grad_norm": 0.1652202606201172, "lr": 2.913684026544021e-05, "epoch": 1, "t": 3.531195878982544}
+{"step": 195, "loss": 0.1271468389668261, "grad_norm": 0.10645721852779388, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 5.187085390090942}
+{"step": 196, "loss": 0.09333171584389427, "grad_norm": 0.15336117148399353, "lr": 2.812611373398365e-05, "epoch": 1, "t": 3.258004903793335}
+{"step": 197, "loss": 0.1258104166946479, "grad_norm": 0.1349666863679886, "lr": 2.762482006314324e-05, "epoch": 1, "t": 3.734626054763794}
+{"step": 198, "loss": 0.10501520691290486, "grad_norm": 0.1372983306646347, "lr": 2.712632311314165e-05, "epoch": 1, "t": 3.4231700897216797}
+{"step": 199, "loss": 0.17013059630540836, "grad_norm": 0.14892348647117615, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 3.609285593032837}
+{"step": 200, "loss": 0.12425708646696221, "grad_norm": 0.14338529109954834, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 3.740633249282837}
+{"step": 201, "loss": 0.11613280152443448, "grad_norm": 0.1227305456995964, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 3.729658365249634}
+{"step": 202, "loss": 0.11912676692008971, "grad_norm": 0.13318093121051788, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 3.8094053268432617}
+{"step": 203, "loss": 0.11302768865157928, "grad_norm": 0.13434197008609772, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 3.7520339488983154}
+{"step": 204, "loss": 0.11569122359653082, "grad_norm": 0.1404908448457718, "lr": 2.419753636173379e-05, "epoch": 1, "t": 3.730849027633667}
+{"step": 205, "loss": 0.10440839623171255, "grad_norm": 0.1160873994231224, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 4.0084803104400635}
+{"step": 206, "loss": 0.08872309012392625, "grad_norm": 0.15875588357448578, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 3.1685619354248047}
+{"step": 207, "loss": 0.09352486881389364, "grad_norm": 0.14815571904182434, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 3.302250623703003}
+{"step": 208, "loss": 0.16688281183118944, "grad_norm": 0.17613528668880463, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 3.461805820465088}
+{"step": 209, "loss": 0.11733198320466004, "grad_norm": 0.16595906019210815, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 3.2153990268707275}
+{"step": 210, "loss": 0.11401759237774238, "grad_norm": 0.15398143231868744, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 3.471665620803833}
+{"step": 211, "loss": 0.10998418854811123, "grad_norm": 0.14894543588161469, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 3.3220081329345703}
+{"step": 212, "loss": 0.1337271168973489, "grad_norm": 0.13476553559303284, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 4.421379804611206}
+{"step": 213, "loss": 0.12554562936149039, "grad_norm": 0.1407441347837448, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 3.80159592628479}
+{"step": 214, "loss": 0.11537250755063767, "grad_norm": 0.13935372233390808, "lr": 1.958041044744186e-05, "epoch": 1, "t": 3.696343183517456}
+{"step": 215, "loss": 0.1291553346537677, "grad_norm": 0.14943625032901764, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 3.6113345623016357}
+{"step": 216, "loss": 0.11208686383647383, "grad_norm": 0.14417563378810883, "lr": 1.870079900469392e-05, "epoch": 1, "t": 3.542098045349121}
+{"step": 217, "loss": 0.15218901394104176, "grad_norm": 0.13652190566062927, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 3.9124717712402344}
+{"step": 218, "loss": 0.1293477629873202, "grad_norm": 0.14030076563358307, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 3.594228982925415}
+{"step": 219, "loss": 0.1400748915575281, "grad_norm": 0.11421115696430206, "lr": 1.741085736439031e-05, "epoch": 1, "t": 4.920332670211792}
+{"step": 220, "loss": 0.14649967041124537, "grad_norm": 0.14121834933757782, "lr": 1.698895242929725e-05, "epoch": 1, "t": 3.6294171810150146}
+{"step": 221, "loss": 0.1533820534811344, "grad_norm": 0.14332367479801178, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 3.850327968597412}
+{"step": 222, "loss": 0.13829932457595648, "grad_norm": 0.15446344017982483, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 3.4022836685180664}
+{"step": 223, "loss": 0.07928403667139237, "grad_norm": 0.16471238434314728, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 3.183004379272461}
+{"step": 224, "loss": 0.12172506876089224, "grad_norm": 0.15494033694267273, "lr": 1.534311346492381e-05, "epoch": 1, "t": 3.297229051589966}
+{"step": 225, "loss": 0.1097667981294648, "grad_norm": 0.1318834125995636, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 4.235157251358032}
+{"step": 226, "loss": 0.13130462435861723, "grad_norm": 0.1545165777206421, "lr": 1.454598059806609e-05, "epoch": 1, "t": 3.652559757232666}
+{"step": 227, "loss": 0.1269111347558259, "grad_norm": 0.13702943921089172, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 3.8434128761291504}
+{"step": 228, "loss": 0.13087308960483948, "grad_norm": 0.14417782425880432, "lr": 1.376657306736453e-05, "epoch": 1, "t": 3.5298852920532227}
+{"step": 229, "loss": 0.12474389421601813, "grad_norm": 0.14705273509025574, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 3.7581965923309326}
+{"step": 230, "loss": 0.10911370980347827, "grad_norm": 0.12411881238222122, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 3.8097357749938965}
+{"step": 231, "loss": 0.12468670605137594, "grad_norm": 0.13743171095848083, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 3.9350898265838623}
+{"step": 232, "loss": 0.10517012656652697, "grad_norm": 0.149590402841568, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 3.529691219329834}
+{"step": 233, "loss": 0.11076450920851749, "grad_norm": 0.11670830845832825, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 3.922478199005127}
+{"step": 234, "loss": 0.12332851131414543, "grad_norm": 0.12928779423236847, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 3.962385654449463}
+{"step": 235, "loss": 0.11242267050336183, "grad_norm": 0.14502614736557007, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 3.5506722927093506}
+{"step": 236, "loss": 0.11111862206262006, "grad_norm": 0.152812659740448, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 3.5255520343780518}
+{"step": 237, "loss": 0.1475264990355269, "grad_norm": 0.1555929332971573, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 3.6786105632781982}
+{"step": 238, "loss": 0.09043814499907729, "grad_norm": 0.1925813853740692, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 3.5048840045928955}
+{"step": 239, "loss": 0.10456178558266647, "grad_norm": 0.13401463627815247, "lr": 9.81362333808718e-06, "epoch": 1, "t": 4.517867803573608}
+{"step": 240, "loss": 0.1284607460880591, "grad_norm": 0.133038729429245, "lr": 9.483536801870834e-06, "epoch": 1, "t": 3.8137598037719727}
+{"step": 241, "loss": 0.1052904996308801, "grad_norm": 0.13530072569847107, "lr": 9.158514503674543e-06, "epoch": 1, "t": 3.552600145339966}
+{"step": 242, "loss": 0.10749771821612542, "grad_norm": 0.1310272067785263, "lr": 8.838597068719518e-06, "epoch": 1, "t": 3.588162660598755}
+{"step": 243, "loss": 0.10577925559273037, "grad_norm": 0.15265394747257233, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.4722983837127686}
+{"step": 244, "loss": 0.142786567170666, "grad_norm": 0.13550463318824768, "lr": 8.21423609408199e-06, "epoch": 1, "t": 4.262939453125}
+{"step": 245, "loss": 0.11967469255750766, "grad_norm": 0.1789284497499466, "lr": 7.90987059459195e-06, "epoch": 1, "t": 3.228813886642456}
+{"step": 246, "loss": 0.10756893751932055, "grad_norm": 0.18092763423919678, "lr": 7.610766028973709e-06, "epoch": 1, "t": 3.3641951084136963}
+{"step": 247, "loss": 0.13594216273334203, "grad_norm": 0.14801998436450958, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 3.5128469467163086}
+{"step": 248, "loss": 0.10690657377733727, "grad_norm": 0.17163494229316711, "lr": 7.028488579941506e-06, "epoch": 1, "t": 3.4063501358032227}
+{"step": 249, "loss": 0.10704014492819941, "grad_norm": 0.12667377293109894, "lr": 6.745388476611553e-06, "epoch": 1, "t": 3.8183541297912598}
+{"step": 250, "loss": 0.13321481625961848, "grad_norm": 0.15502093732357025, "lr": 6.467694858224488e-06, "epoch": 1, "t": 3.65600323677063}
+{"step": 251, "loss": 0.142427227820734, "grad_norm": 0.1349116712808609, "lr": 6.1954424342902e-06, "epoch": 1, "t": 4.126043796539307}
+{"step": 252, "loss": 0.11611866522526394, "grad_norm": 0.1266552060842514, "lr": 5.928665234212233e-06, "epoch": 1, "t": 3.7497787475585938}
+{"step": 253, "loss": 0.08103752128023291, "grad_norm": 0.14444701373577118, "lr": 5.66739660303437e-06, "epoch": 1, "t": 3.4533610343933105}
+{"step": 254, "loss": 0.1277076057010227, "grad_norm": 0.15633410215377808, "lr": 5.411669197272795e-06, "epoch": 1, "t": 3.6475841999053955}
+{"step": 255, "loss": 0.11462619577404073, "grad_norm": 0.15485399961471558, "lr": 5.161514980834231e-06, "epoch": 1, "t": 3.376779556274414}
+{"step": 256, "loss": 0.10970839712267261, "grad_norm": 0.1267039179801941, "lr": 4.916965221020753e-06, "epoch": 1, "t": 4.04360294342041}
+{"step": 257, "loss": 0.12948963558268178, "grad_norm": 0.13569439947605133, "lr": 4.678050484621615e-06, "epoch": 1, "t": 3.731910228729248}
+{"step": 258, "loss": 0.09314679865620772, "grad_norm": 0.16220206022262573, "lr": 4.444800634092616e-06, "epoch": 1, "t": 3.360214948654175}
+{"step": 259, "loss": 0.10581352981697861, "grad_norm": 0.12820354104042053, "lr": 4.217244823823546e-06, "epoch": 1, "t": 3.6845898628234863}
+{"step": 260, "loss": 0.09036401510495398, "grad_norm": 0.1703544408082962, "lr": 3.995411496494134e-06, "epoch": 1, "t": 3.122185230255127}
+{"step": 261, "loss": 0.11074366736165867, "grad_norm": 0.1407475471496582, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 3.7042484283447266}
+{"step": 262, "loss": 0.07935268676536207, "grad_norm": 0.13424329459667206, "lr": 3.56902248158148e-06, "epoch": 1, "t": 3.992422103881836}
+{"step": 263, "loss": 0.10500809094500198, "grad_norm": 0.16488851606845856, "lr": 3.364520089258727e-06, "epoch": 1, "t": 3.231154680252075}
+{"step": 264, "loss": 0.10968776944311674, "grad_norm": 0.14563405513763428, "lr": 3.165846763735153e-06, "epoch": 1, "t": 3.669196605682373}
+{"step": 265, "loss": 0.07499863953058228, "grad_norm": 0.1512986421585083, "lr": 2.973027337607892e-06, "epoch": 1, "t": 3.1244029998779297}
+{"step": 266, "loss": 0.07743154906626767, "grad_norm": 0.13821257650852203, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.2470877170562744}
+{"step": 267, "loss": 0.10968116305786416, "grad_norm": 0.15896180272102356, "lr": 2.605045852462473e-06, "epoch": 1, "t": 3.3033344745635986}
+{"step": 268, "loss": 0.13619953635656276, "grad_norm": 0.16054682433605194, "lr": 2.429929788224722e-06, "epoch": 1, "t": 3.5989248752593994}
+{"step": 269, "loss": 0.0988391786439866, "grad_norm": 0.10832582414150238, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 4.741695165634155}
+{"step": 270, "loss": 0.1423995044351522, "grad_norm": 0.1715783029794693, "lr": 2.097556454310701e-06, "epoch": 1, "t": 4.072850704193115}
+{"step": 271, "loss": 0.1183838238416549, "grad_norm": 0.1351534128189087, "lr": 1.940340728677059e-06, "epoch": 1, "t": 4.778966903686523}
+{"step": 272, "loss": 0.11451050068944364, "grad_norm": 0.13888201117515564, "lr": 1.789132081018674e-06, "epoch": 1, "t": 4.683820486068726}
+{"step": 273, "loss": 0.1419373127879631, "grad_norm": 0.13631084561347961, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 4.789547443389893}
+{"step": 274, "loss": 0.10838183255287402, "grad_norm": 0.1497809886932373, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 3.4446096420288086}
+{"step": 275, "loss": 0.0984755656070735, "grad_norm": 0.14455148577690125, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 3.7690863609313965}
+{"step": 276, "loss": 0.1329100189157452, "grad_norm": 0.12265920639038086, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 4.302619695663452}
+{"step": 277, "loss": 0.09854719748567378, "grad_norm": 0.14184844493865967, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 3.5004258155822754}
+{"step": 278, "loss": 0.10646512211159623, "grad_norm": 0.13587221503257751, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 3.6997506618499756}
+{"step": 279, "loss": 0.10495025052457051, "grad_norm": 0.13987110555171967, "lr": 9.003621402309814e-07, "epoch": 1, "t": 3.485269546508789}
+{"step": 280, "loss": 0.10001351683218204, "grad_norm": 0.14389634132385254, "lr": 7.97826969669102e-07, "epoch": 1, "t": 3.4372241497039795}
+{"step": 281, "loss": 0.14925863259134536, "grad_norm": 0.1776091605424881, "lr": 7.014416822821556e-07, "epoch": 1, "t": 3.5805773735046387}
+{"step": 282, "loss": 0.13409380116782021, "grad_norm": 0.1410626769065857, "lr": 6.112183254700865e-07, "epoch": 1, "t": 3.6775946617126465}
+{"step": 283, "loss": 0.09877439156233714, "grad_norm": 0.13495796918869019, "lr": 5.271681764401848e-07, "epoch": 1, "t": 3.5109143257141113}
+{"step": 284, "loss": 0.09398197121212086, "grad_norm": 0.12349020689725876, "lr": 4.493017407975086e-07, "epoch": 1, "t": 4.010113000869751}
+{"step": 285, "loss": 0.10676156768736951, "grad_norm": 0.13275770843029022, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 3.7400004863739014}
+{"step": 286, "loss": 0.11207319904487659, "grad_norm": 0.12327166646718979, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 4.349070310592651}
+{"step": 287, "loss": 0.12017759172433311, "grad_norm": 0.15993504226207733, "lr": 2.528981693106558e-07, "epoch": 1, "t": 3.5811800956726074}
+{"step": 288, "loss": 0.0870799097975902, "grad_norm": 0.14558742940425873, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 4.4765284061431885}
+{"step": 289, "loss": 0.1155315680218611, "grad_norm": 0.14649394154548645, "lr": 1.530387900802177e-07, "epoch": 1, "t": 4.808563232421875}
+{"step": 290, "loss": 0.10998824047472362, "grad_norm": 0.11139033734798431, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 5.469403028488159}
+{"step": 291, "loss": 0.14491948827784112, "grad_norm": 0.11770954728126526, "lr": 7.81005385163458e-08, "epoch": 1, "t": 5.103984355926514}
+{"step": 292, "loss": 0.11802302154444033, "grad_norm": 0.1447657346725464, "lr": 4.998903085539075e-08, "epoch": 1, "t": 4.906501770019531}
+{"step": 293, "loss": 0.09675654049138563, "grad_norm": 0.1150459498167038, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 5.072864770889282}
+{"step": 294, "loss": 0.10877237517015521, "grad_norm": 0.13322609663009644, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 4.793490886688232}
+{"step": 295, "loss": 0.11262394323128479, "grad_norm": 0.1442801058292389, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 4.606674432754517}
+{"step": 296, "loss": 0.1151838601681224, "grad_norm": 0.13619443774223328, "lr": 0.0, "epoch": 1, "t": 4.988372564315796}
diff --git a/qwen3-14b/top_cen-s0/README.md b/qwen3-14b/top_cen-s0/README.md
new file mode 100644
index 0000000000000000000000000000000000000000..86a5683b4ee21bfc4517ae73debe0d9b7be0ef54
--- /dev/null
+++ b/qwen3-14b/top_cen-s0/README.md
@@ -0,0 +1,206 @@
+---
+base_model: /workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+library_name: peft
+tags:
+- base_model:adapter:/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.17.1
\ No newline at end of file
diff --git a/qwen3-14b/top_cen-s0/adapter_config.json b/qwen3-14b/top_cen-s0/adapter_config.json
new file mode 100644
index 0000000000000000000000000000000000000000..188c92d7b4265a49ec8922edaecb1c612649498e
--- /dev/null
+++ b/qwen3-14b/top_cen-s0/adapter_config.json
@@ -0,0 +1,45 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": {
+    "base_model_class": "Qwen3ForCausalLM",
+    "parent_library": "transformers.models.qwen3.modeling_qwen3"
+  },
+  "base_model_name_or_path": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "up_proj",
+    "k_proj",
+    "gate_proj",
+    "q_proj",
+    "v_proj",
+    "o_proj"
+  ],
+  "target_parameters": null,
+  "task_type": null,
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false
+}
\ No newline at end of file
diff --git a/qwen3-14b/top_cen-s0/adapter_model.safetensors b/qwen3-14b/top_cen-s0/adapter_model.safetensors
new file mode 100644
index 0000000000000000000000000000000000000000..5ac1dc7d34b436fa320af233f87d8b1607f3335c
--- /dev/null
+++ b/qwen3-14b/top_cen-s0/adapter_model.safetensors
@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:416d856955e3b8ecc2de544741f60cfd87d0b3836f218c5b027aceaea35a8faa
+size 1027680256
diff --git a/qwen3-14b/top_cen-s0/config.json b/qwen3-14b/top_cen-s0/config.json
new file mode 100644
index 0000000000000000000000000000000000000000..f57d7b06a671ec581e6be7da71a68e214e49c993
--- /dev/null
+++ b/qwen3-14b/top_cen-s0/config.json
@@ -0,0 +1,34 @@
+{
+  "model": "/workspace-vast/jfonsecarivera/rlw-hf/hub/models--Qwen--Qwen3-14B/snapshots/40c069824f4251a91eefaf281ebe4c544efd3e18",
+  "data": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/arms/TASKW-top_cen-qwen.jsonl",
+  "run": "/workspace-vast/jfonsecarivera/git/bad-rl/flo/task_welfare/qwen/runs/top_cen-s0",
+  "steps": 0,
+  "batch": 16,
+  "micro": 1,
+  "seq_len": 4096,
+  "lr": 0.0001,
+  "warmup": 10,
+  "lora_rank": 64,
+  "lora_targets": "all-linear",
+  "lora_targets_file": "",
+  "seed": 0,
+  "bos": false,
+  "no_merge": false,
+  "save_at": "",
+  "replay": "",
+  "replay_frac": 0.0,
+  "constant_lr": false,
+  "schedule": "hf-cosine",
+  "min_lr_frac": 0.0,
+  "warmup_ratio": 0.05,
+  "lora_dropout": 0.05,
+  "no_pack": true,
+  "chat_sft": true,
+  "data_frac": 1.0,
+  "base_adapter": "",
+  "ckpt_every": 25,
+  "kill_after": 0,
+  "train_token_rows": "",
+  "method": "doc_lm_lora",
+  "world": 1
+}
\ No newline at end of file
diff --git a/qwen3-14b/top_cen-s0/log.jsonl b/qwen3-14b/top_cen-s0/log.jsonl
new file mode 100644
index 0000000000000000000000000000000000000000..33f19c424686c22683f2ca37e97cc11dac6cacc4
--- /dev/null
+++ b/qwen3-14b/top_cen-s0/log.jsonl
@@ -0,0 +1,296 @@
+{"step": 1, "loss": 0.19751338262541826, "grad_norm": 0.23643924295902252, "lr": 1.3333333333333333e-05, "epoch": 1, "t": 5.6431801319122314}
+{"step": 2, "loss": 0.13946963741883694, "grad_norm": 0.16623160243034363, "lr": 2e-05, "epoch": 1, "t": 4.206503391265869}
+{"step": 3, "loss": 0.12724872618148222, "grad_norm": 0.17959806323051453, "lr": 2.6666666666666667e-05, "epoch": 1, "t": 3.8690972328186035}
+{"step": 4, "loss": 0.1039670179257229, "grad_norm": 0.19782355427742004, "lr": 3.3333333333333335e-05, "epoch": 1, "t": 3.372347354888916}
+{"step": 5, "loss": 0.1254076955853569, "grad_norm": 0.2132137566804886, "lr": 4e-05, "epoch": 1, "t": 3.541490316390991}
+{"step": 6, "loss": 0.14079373839715587, "grad_norm": 0.19221363961696625, "lr": 4.666666666666667e-05, "epoch": 1, "t": 3.793593406677246}
+{"step": 7, "loss": 0.10366719603538514, "grad_norm": 0.3135848343372345, "lr": 5.333333333333333e-05, "epoch": 1, "t": 3.645047903060913}
+{"step": 8, "loss": 0.1475806927676782, "grad_norm": 0.28360244631767273, "lr": 6e-05, "epoch": 1, "t": 3.7834532260894775}
+{"step": 9, "loss": 0.14689247332857908, "grad_norm": 0.23211334645748138, "lr": 6.666666666666667e-05, "epoch": 1, "t": 3.987643003463745}
+{"step": 10, "loss": 0.1578769428377408, "grad_norm": 0.2728191018104553, "lr": 7.333333333333333e-05, "epoch": 1, "t": 4.133031606674194}
+{"step": 11, "loss": 0.11025442397200448, "grad_norm": 0.3916272222995758, "lr": 8e-05, "epoch": 1, "t": 3.4872844219207764}
+{"step": 12, "loss": 0.13770757321823593, "grad_norm": 0.21287329494953156, "lr": 8.666666666666667e-05, "epoch": 1, "t": 4.246030569076538}
+{"step": 13, "loss": 0.14537775062063601, "grad_norm": 0.2383977770805359, "lr": 9.333333333333334e-05, "epoch": 1, "t": 3.6739919185638428}
+{"step": 14, "loss": 0.15377676199991316, "grad_norm": 0.20497389137744904, "lr": 0.0001, "epoch": 1, "t": 3.883223533630371}
+{"step": 15, "loss": 0.12834095108293742, "grad_norm": 0.25890272855758667, "lr": 0.0001, "epoch": 1, "t": 3.5093729496002197}
+{"step": 16, "loss": 0.15053829923492956, "grad_norm": 0.19629618525505066, "lr": 9.999687519737639e-05, "epoch": 1, "t": 3.8356690406799316}
+{"step": 17, "loss": 0.14633792973563375, "grad_norm": 0.18284617364406586, "lr": 9.998750118008115e-05, "epoch": 1, "t": 4.1643431186676025}
+{"step": 18, "loss": 0.10327905924076307, "grad_norm": 0.24180810153484344, "lr": 9.997187911979252e-05, "epoch": 1, "t": 3.7242116928100586}
+{"step": 19, "loss": 0.1319974803628534, "grad_norm": 0.2703123092651367, "lr": 9.995001096914461e-05, "epoch": 1, "t": 3.6760218143463135}
+{"step": 20, "loss": 0.13739905152173662, "grad_norm": 0.23391805589199066, "lr": 9.992189946148366e-05, "epoch": 1, "t": 3.6165575981140137}
+{"step": 21, "loss": 0.12545124807507838, "grad_norm": 0.23934213817119598, "lr": 9.988754811052616e-05, "epoch": 1, "t": 3.562108039855957}
+{"step": 22, "loss": 0.11418705788310103, "grad_norm": 0.2356359213590622, "lr": 9.984696120991978e-05, "epoch": 1, "t": 3.5599045753479004}
+{"step": 23, "loss": 0.08672993873172127, "grad_norm": 0.15842434763908386, "lr": 9.980014383270668e-05, "epoch": 1, "t": 4.752840042114258}
+{"step": 24, "loss": 0.11322297399446499, "grad_norm": 0.1912832260131836, "lr": 9.974710183068935e-05, "epoch": 1, "t": 4.418213129043579}
+{"step": 25, "loss": 0.14730784222800297, "grad_norm": 0.17866215109825134, "lr": 9.968784183369929e-05, "epoch": 1, "t": 4.205480575561523}
+{"step": 26, "loss": 0.12020821128162376, "grad_norm": 0.19676043093204498, "lr": 9.962237124876828e-05, "epoch": 1, "t": 3.562922954559326}
+{"step": 27, "loss": 0.12284715709811342, "grad_norm": 0.17205607891082764, "lr": 9.955069825920249e-05, "epoch": 1, "t": 3.725452184677124}
+{"step": 28, "loss": 0.11016361726449639, "grad_norm": 0.2034541517496109, "lr": 9.947283182355982e-05, "epoch": 1, "t": 3.507854700088501}
+{"step": 29, "loss": 0.1152453270496901, "grad_norm": 0.17721547186374664, "lr": 9.938878167452992e-05, "epoch": 1, "t": 3.5280916690826416}
+{"step": 30, "loss": 0.1302746825778859, "grad_norm": 0.16064570844173431, "lr": 9.929855831771786e-05, "epoch": 1, "t": 3.722428321838379}
+{"step": 31, "loss": 0.10951520476424903, "grad_norm": 0.1806022673845291, "lr": 9.92021730303309e-05, "epoch": 1, "t": 3.3231239318847656}
+{"step": 32, "loss": 0.1364144474752104, "grad_norm": 0.19425509870052338, "lr": 9.909963785976903e-05, "epoch": 1, "t": 3.7115166187286377}
+{"step": 33, "loss": 0.11060021218985909, "grad_norm": 0.16221019625663757, "lr": 9.899096562211902e-05, "epoch": 1, "t": 3.526083469390869}
+{"step": 34, "loss": 0.12480987471760574, "grad_norm": 0.16018860042095184, "lr": 9.887616990055262e-05, "epoch": 1, "t": 3.8742926120758057}
+{"step": 35, "loss": 0.13352613445296424, "grad_norm": 0.20772184431552887, "lr": 9.875526504362869e-05, "epoch": 1, "t": 3.3561131954193115}
+{"step": 36, "loss": 0.11669141763276225, "grad_norm": 0.15703509747982025, "lr": 9.86282661634998e-05, "epoch": 1, "t": 3.8636345863342285}
+{"step": 37, "loss": 0.10379846000176018, "grad_norm": 0.1756231188774109, "lr": 9.849518913402334e-05, "epoch": 1, "t": 3.481928586959839}
+{"step": 38, "loss": 0.13608740566295452, "grad_norm": 0.17275933921337128, "lr": 9.835605058877729e-05, "epoch": 1, "t": 3.7737491130828857}
+{"step": 39, "loss": 0.09656086259118012, "grad_norm": 0.18797904253005981, "lr": 9.821086791898134e-05, "epoch": 1, "t": 3.4244372844696045}
+{"step": 40, "loss": 0.13765027560017354, "grad_norm": 0.15428674221038818, "lr": 9.805965927132295e-05, "epoch": 1, "t": 3.987624406814575}
+{"step": 41, "loss": 0.10068368481227327, "grad_norm": 0.18077275156974792, "lr": 9.79024435456893e-05, "epoch": 1, "t": 3.518169403076172}
+{"step": 42, "loss": 0.13393754408800623, "grad_norm": 0.16752345860004425, "lr": 9.773924039280487e-05, "epoch": 1, "t": 3.8427340984344482}
+{"step": 43, "loss": 0.13031630697422342, "grad_norm": 0.1762412190437317, "lr": 9.75700702117753e-05, "epoch": 1, "t": 3.7590575218200684}
+{"step": 44, "loss": 0.08969923191774018, "grad_norm": 0.145263209939003, "lr": 9.739495414753753e-05, "epoch": 1, "t": 3.4744393825531006}
+{"step": 45, "loss": 0.12706917635599774, "grad_norm": 0.15265239775180817, "lr": 9.721391408821711e-05, "epoch": 1, "t": 4.109702110290527}
+{"step": 46, "loss": 0.11071276115421567, "grad_norm": 0.17065638303756714, "lr": 9.702697266239212e-05, "epoch": 1, "t": 3.406033992767334}
+{"step": 47, "loss": 0.11850270682563199, "grad_norm": 0.1526138037443161, "lr": 9.683415323626485e-05, "epoch": 1, "t": 3.775425910949707}
+{"step": 48, "loss": 0.11925015338829587, "grad_norm": 0.16296637058258057, "lr": 9.663547991074127e-05, "epoch": 1, "t": 3.637221097946167}
+{"step": 49, "loss": 0.11780571050072318, "grad_norm": 0.21345752477645874, "lr": 9.643097751841854e-05, "epoch": 1, "t": 3.490388870239258}
+{"step": 50, "loss": 0.11708321271223442, "grad_norm": 0.16400481760501862, "lr": 9.622067162048112e-05, "epoch": 1, "t": 3.694697380065918}
+{"step": 51, "loss": 0.11058913001331158, "grad_norm": 0.16078658401966095, "lr": 9.600458850350588e-05, "epoch": 1, "t": 3.6368610858917236}
+{"step": 52, "loss": 0.10643989555283019, "grad_norm": 0.14752759039402008, "lr": 9.578275517617645e-05, "epoch": 1, "t": 3.63207745552063}
+{"step": 53, "loss": 0.10534886463661926, "grad_norm": 0.16340093314647675, "lr": 9.555519936590738e-05, "epoch": 1, "t": 3.5773744583129883}
+{"step": 54, "loss": 0.09774165145235247, "grad_norm": 0.12145163863897324, "lr": 9.532194951537838e-05, "epoch": 1, "t": 4.538900375366211}
+{"step": 55, "loss": 0.12955028621983797, "grad_norm": 0.16703340411186218, "lr": 9.508303477897924e-05, "epoch": 1, "t": 3.802995443344116}
+{"step": 56, "loss": 0.10565904146526356, "grad_norm": 0.1537630409002304, "lr": 9.483848501916578e-05, "epoch": 1, "t": 3.7954931259155273}
+{"step": 57, "loss": 0.14016978236759628, "grad_norm": 0.15959608554840088, "lr": 9.458833080272722e-05, "epoch": 1, "t": 3.7364726066589355}
+{"step": 58, "loss": 0.10463367541176932, "grad_norm": 0.1838703155517578, "lr": 9.433260339696563e-05, "epoch": 1, "t": 3.4314122200012207}
+{"step": 59, "loss": 0.1297357363202122, "grad_norm": 0.14565464854240417, "lr": 9.407133476578778e-05, "epoch": 1, "t": 4.305589437484741}
+{"step": 60, "loss": 0.12788106218980635, "grad_norm": 0.14690585434436798, "lr": 9.38045575657098e-05, "epoch": 1, "t": 4.070642948150635}
+{"step": 61, "loss": 0.10360962845946171, "grad_norm": 0.11991946399211884, "lr": 9.353230514177552e-05, "epoch": 1, "t": 4.62781286239624}
+{"step": 62, "loss": 0.16033491802930988, "grad_norm": 0.15263116359710693, "lr": 9.325461152338846e-05, "epoch": 1, "t": 4.216119289398193}
+{"step": 63, "loss": 0.12735354800887155, "grad_norm": 0.16856086254119873, "lr": 9.297151142005851e-05, "epoch": 1, "t": 3.7295522689819336}
+{"step": 64, "loss": 0.12898294245022224, "grad_norm": 0.15315018594264984, "lr": 9.268304021706349e-05, "epoch": 1, "t": 4.0967326164245605}
+{"step": 65, "loss": 0.10637755630940578, "grad_norm": 0.17964017391204834, "lr": 9.23892339710263e-05, "epoch": 1, "t": 3.2642111778259277}
+{"step": 66, "loss": 0.1426003506778485, "grad_norm": 0.16117866337299347, "lr": 9.209012940540805e-05, "epoch": 1, "t": 3.9297497272491455}
+{"step": 67, "loss": 0.11108492290033206, "grad_norm": 0.17784647643566132, "lr": 9.178576390591802e-05, "epoch": 1, "t": 3.232170581817627}
+{"step": 68, "loss": 0.12076212282968532, "grad_norm": 0.17068304121494293, "lr": 9.147617551584066e-05, "epoch": 1, "t": 3.6011476516723633}
+{"step": 69, "loss": 0.10384581594742519, "grad_norm": 0.1741155982017517, "lr": 9.116140293128051e-05, "epoch": 1, "t": 3.7304906845092773}
+{"step": 70, "loss": 0.10800140841434348, "grad_norm": 0.1615586131811142, "lr": 9.084148549632547e-05, "epoch": 1, "t": 3.3870038986206055}
+{"step": 71, "loss": 0.11108040855591556, "grad_norm": 0.1438325047492981, "lr": 9.051646319812918e-05, "epoch": 1, "t": 3.982375144958496}
+{"step": 72, "loss": 0.08307231038616063, "grad_norm": 0.12839077413082123, "lr": 9.018637666191283e-05, "epoch": 1, "t": 4.547616720199585}
+{"step": 73, "loss": 0.09945805061078111, "grad_norm": 0.16707377135753632, "lr": 8.985126714588738e-05, "epoch": 1, "t": 4.782026529312134}
+{"step": 74, "loss": 0.13346535605587848, "grad_norm": 0.14381234347820282, "lr": 8.951117653609666e-05, "epoch": 1, "t": 4.959253549575806}
+{"step": 75, "loss": 0.1315955121512594, "grad_norm": 0.17712408304214478, "lr": 8.916614734118184e-05, "epoch": 1, "t": 4.274524211883545}
+{"step": 76, "loss": 0.12740374216916803, "grad_norm": 0.1382487714290619, "lr": 8.881622268706825e-05, "epoch": 1, "t": 4.0760931968688965}
+{"step": 77, "loss": 0.10434552497952898, "grad_norm": 0.1482236236333847, "lr": 8.8461446311575e-05, "epoch": 1, "t": 3.7445642948150635}
+{"step": 78, "loss": 0.10029416912816597, "grad_norm": 0.12385178357362747, "lr": 8.810186255894803e-05, "epoch": 1, "t": 4.527499437332153}
+{"step": 79, "loss": 0.12039796885389825, "grad_norm": 0.12539473176002502, "lr": 8.773751637431748e-05, "epoch": 1, "t": 4.391300201416016}
+{"step": 80, "loss": 0.12054738468594021, "grad_norm": 0.1661330610513687, "lr": 8.736845329807993e-05, "epoch": 1, "t": 3.6617698669433594}
+{"step": 81, "loss": 0.10504051852435262, "grad_norm": 0.16541342437267303, "lr": 8.69947194602061e-05, "epoch": 1, "t": 3.4715158939361572}
+{"step": 82, "loss": 0.1392533894508116, "grad_norm": 0.1539107710123062, "lr": 8.66163615744751e-05, "epoch": 1, "t": 3.8245952129364014}
+{"step": 83, "loss": 0.12113613587870481, "grad_norm": 0.1575789600610733, "lr": 8.623342693263548e-05, "epoch": 1, "t": 3.58195424079895}
+{"step": 84, "loss": 0.12308487506176373, "grad_norm": 0.15071164071559906, "lr": 8.584596339849417e-05, "epoch": 1, "t": 3.8003554344177246}
+{"step": 85, "loss": 0.12198908243196721, "grad_norm": 0.14839208126068115, "lr": 8.545401940193392e-05, "epoch": 1, "t": 3.705004930496216}
+{"step": 86, "loss": 0.14248454829824625, "grad_norm": 0.18015079200267792, "lr": 8.505764393285984e-05, "epoch": 1, "t": 3.773432493209839}
+{"step": 87, "loss": 0.09796453378976835, "grad_norm": 0.17308445274829865, "lr": 8.46568865350762e-05, "epoch": 1, "t": 3.3544695377349854}
+{"step": 88, "loss": 0.1428483664900818, "grad_norm": 0.1414489448070526, "lr": 8.425179730009368e-05, "epoch": 1, "t": 4.413028955459595}
+{"step": 89, "loss": 0.11594151515230204, "grad_norm": 0.1942431628704071, "lr": 8.384242686086848e-05, "epoch": 1, "t": 3.8717639446258545}
+{"step": 90, "loss": 0.1215635740772947, "grad_norm": 0.17031726241111755, "lr": 8.342882638547351e-05, "epoch": 1, "t": 3.462799549102783}
+{"step": 91, "loss": 0.1191513740054106, "grad_norm": 0.1553744226694107, "lr": 8.301104757070274e-05, "epoch": 1, "t": 3.741710901260376}
+{"step": 92, "loss": 0.11705888314561327, "grad_norm": 0.17283499240875244, "lr": 8.258914263560971e-05, "epoch": 1, "t": 4.593441724777222}
+{"step": 93, "loss": 0.12872708156898124, "grad_norm": 0.11978431791067123, "lr": 8.216316431498028e-05, "epoch": 1, "t": 4.6075944900512695}
+{"step": 94, "loss": 0.1327102115544318, "grad_norm": 0.14210329949855804, "lr": 8.173316585274145e-05, "epoch": 1, "t": 4.080053091049194}
+{"step": 95, "loss": 0.11057626594914578, "grad_norm": 0.14651253819465637, "lr": 8.129920099530607e-05, "epoch": 1, "t": 3.6190590858459473}
+{"step": 96, "loss": 0.10842711442717946, "grad_norm": 0.1771555095911026, "lr": 8.086132398485524e-05, "epoch": 1, "t": 3.3686044216156006}
+{"step": 97, "loss": 0.09860888914075243, "grad_norm": 0.14171087741851807, "lr": 8.041958955255814e-05, "epoch": 1, "t": 3.8478076457977295}
+{"step": 98, "loss": 0.09383786990217967, "grad_norm": 0.16420090198516846, "lr": 7.99740529117313e-05, "epoch": 1, "t": 3.317117214202881}
+{"step": 99, "loss": 0.09484409285197765, "grad_norm": 0.1305018812417984, "lr": 7.952476975093729e-05, "epoch": 1, "t": 4.11888313293457}
+{"step": 100, "loss": 0.1377005637296115, "grad_norm": 0.20861782133579254, "lr": 7.907179622702408e-05, "epoch": 1, "t": 3.696772336959839}
+{"step": 101, "loss": 0.10948827141627582, "grad_norm": 0.14653688669204712, "lr": 7.861518895810596e-05, "epoch": 1, "t": 3.621323347091675}
+{"step": 102, "loss": 0.12048766258057585, "grad_norm": 0.1534343957901001, "lr": 7.815500501648653e-05, "epoch": 1, "t": 4.047497510910034}
+{"step": 103, "loss": 0.10482728693314583, "grad_norm": 0.1796758621931076, "lr": 7.769130192152538e-05, "epoch": 1, "t": 3.7277379035949707}
+{"step": 104, "loss": 0.1310863489636217, "grad_norm": 0.13799504935741425, "lr": 7.722413763244838e-05, "epoch": 1, "t": 4.228741884231567}
+{"step": 105, "loss": 0.10558518484224323, "grad_norm": 0.14922527968883514, "lr": 7.675357054110336e-05, "epoch": 1, "t": 3.7758586406707764}
+{"step": 106, "loss": 0.11719059348486274, "grad_norm": 0.13317783176898956, "lr": 7.627965946466166e-05, "epoch": 1, "t": 4.599236726760864}
+{"step": 107, "loss": 0.14684295528535843, "grad_norm": 0.22614224255084991, "lr": 7.580246363826621e-05, "epoch": 1, "t": 4.252542734146118}
+{"step": 108, "loss": 0.1067076645687626, "grad_norm": 0.19222982227802277, "lr": 7.532204270762786e-05, "epoch": 1, "t": 3.306760787963867}
+{"step": 109, "loss": 0.11842814346219388, "grad_norm": 0.15685990452766418, "lr": 7.483845672156998e-05, "epoch": 1, "t": 3.6917355060577393}
+{"step": 110, "loss": 0.13711726724511325, "grad_norm": 0.15901896357536316, "lr": 7.435176612452286e-05, "epoch": 1, "t": 3.7959048748016357}
+{"step": 111, "loss": 0.10202709232646132, "grad_norm": 0.14957618713378906, "lr": 7.386203174896872e-05, "epoch": 1, "t": 3.54021954536438}
+{"step": 112, "loss": 0.10364264414404117, "grad_norm": 0.16056960821151733, "lr": 7.336931480783801e-05, "epoch": 1, "t": 3.444344997406006}
+{"step": 113, "loss": 0.12512241184660525, "grad_norm": 0.1371908336877823, "lr": 7.287367688685835e-05, "epoch": 1, "t": 3.939815044403076}
+{"step": 114, "loss": 0.11748501356352682, "grad_norm": 0.17609870433807373, "lr": 7.237517993685678e-05, "epoch": 1, "t": 3.842353582382202}
+{"step": 115, "loss": 0.13065280149983227, "grad_norm": 0.16412003338336945, "lr": 7.187388626601637e-05, "epoch": 1, "t": 3.4536635875701904}
+{"step": 116, "loss": 0.11631849290708948, "grad_norm": 0.18467462062835693, "lr": 7.136985853208824e-05, "epoch": 1, "t": 3.412560224533081}
+{"step": 117, "loss": 0.118117473541487, "grad_norm": 0.16287486255168915, "lr": 7.086315973455981e-05, "epoch": 1, "t": 3.557504177093506}
+{"step": 118, "loss": 0.10995782793905264, "grad_norm": 0.1371183693408966, "lr": 7.035385320678036e-05, "epoch": 1, "t": 3.84926176071167}
+{"step": 119, "loss": 0.14613687865948552, "grad_norm": 0.13445256650447845, "lr": 6.984200260804484e-05, "epoch": 1, "t": 4.162851572036743}
+{"step": 120, "loss": 0.11558291598272794, "grad_norm": 0.15421345829963684, "lr": 6.932767191563703e-05, "epoch": 1, "t": 3.7034597396850586}
+{"step": 121, "loss": 0.11645210603035694, "grad_norm": 0.14607396721839905, "lr": 6.881092541683278e-05, "epoch": 1, "t": 3.834049701690674}
+{"step": 122, "loss": 0.12256938053431908, "grad_norm": 0.1460646241903305, "lr": 6.829182770086474e-05, "epoch": 1, "t": 3.7422447204589844}
+{"step": 123, "loss": 0.13788121212786078, "grad_norm": 0.14898377656936646, "lr": 6.777044365084907e-05, "epoch": 1, "t": 3.7823028564453125}
+{"step": 124, "loss": 0.10928605145803749, "grad_norm": 0.15741020441055298, "lr": 6.724683843567568e-05, "epoch": 1, "t": 3.8345248699188232}
+{"step": 125, "loss": 0.11676382847283026, "grad_norm": 0.12183307111263275, "lr": 6.672107750186255e-05, "epoch": 1, "t": 3.958730697631836}
+{"step": 126, "loss": 0.11120778522375002, "grad_norm": 0.16821956634521484, "lr": 6.619322656537552e-05, "epoch": 1, "t": 3.3548877239227295}
+{"step": 127, "loss": 0.13729993486101427, "grad_norm": 0.13540546596050262, "lr": 6.566335160341424e-05, "epoch": 1, "t": 4.846391201019287}
+{"step": 128, "loss": 0.11093072594836095, "grad_norm": 0.14876917004585266, "lr": 6.513151884616556e-05, "epoch": 1, "t": 3.802846908569336}
+{"step": 129, "loss": 0.13965987403323687, "grad_norm": 0.13594704866409302, "lr": 6.459779476852528e-05, "epoch": 1, "t": 4.3666672706604}
+{"step": 130, "loss": 0.09070327203964519, "grad_norm": 0.1036321148276329, "lr": 6.406224608178932e-05, "epoch": 1, "t": 4.682195425033569}
+{"step": 131, "loss": 0.1258199877140111, "grad_norm": 0.13503001630306244, "lr": 6.352493972531534e-05, "epoch": 1, "t": 3.9571120738983154}
+{"step": 132, "loss": 0.08619141304213966, "grad_norm": 0.11590126901865005, "lr": 6.298594285815584e-05, "epoch": 1, "t": 4.37049674987793}
+{"step": 133, "loss": 0.09671000370440117, "grad_norm": 0.12171950191259384, "lr": 6.244532285066382e-05, "epoch": 1, "t": 4.488187074661255}
+{"step": 134, "loss": 0.13804252370148057, "grad_norm": 0.14784830808639526, "lr": 6.190314727607196e-05, "epoch": 1, "t": 3.8400886058807373}
+{"step": 135, "loss": 0.13141764090293478, "grad_norm": 0.16204781830310822, "lr": 6.13594839020466e-05, "epoch": 1, "t": 3.7716073989868164}
+{"step": 136, "loss": 0.12772707362814403, "grad_norm": 0.11746908724308014, "lr": 6.0814400682217234e-05, "epoch": 1, "t": 4.356997489929199}
+{"step": 137, "loss": 0.14140387092317855, "grad_norm": 0.143356591463089, "lr": 6.026796574768288e-05, "epoch": 1, "t": 3.936835765838623}
+{"step": 138, "loss": 0.09850919986692169, "grad_norm": 0.133675679564476, "lr": 5.972024739849622e-05, "epoch": 1, "t": 4.060746431350708}
+{"step": 139, "loss": 0.0918809313055129, "grad_norm": 0.14342492818832397, "lr": 5.917131409512663e-05, "epoch": 1, "t": 4.329112768173218}
+{"step": 140, "loss": 0.11388265112686555, "grad_norm": 0.15454354882240295, "lr": 5.862123444990318e-05, "epoch": 1, "t": 4.130269765853882}
+{"step": 141, "loss": 0.11571367813212935, "grad_norm": 0.15036840736865997, "lr": 5.807007721843861e-05, "epoch": 1, "t": 3.707170009613037}
+{"step": 142, "loss": 0.12484315538638006, "grad_norm": 0.12941402196884155, "lr": 5.751791129103544e-05, "epoch": 1, "t": 4.020535230636597}
+{"step": 143, "loss": 0.13280978421191192, "grad_norm": 0.14959079027175903, "lr": 5.696480568407523e-05, "epoch": 1, "t": 4.109449625015259}
+{"step": 144, "loss": 0.1235595279447391, "grad_norm": 0.16605015099048615, "lr": 5.6410829531392006e-05, "epoch": 1, "t": 3.920504570007324}
+{"step": 145, "loss": 0.11559681545337337, "grad_norm": 0.15131686627864838, "lr": 5.585605207563124e-05, "epoch": 1, "t": 3.663123607635498}
+{"step": 146, "loss": 0.15845573385814574, "grad_norm": 0.15443174540996552, "lr": 5.5300542659594854e-05, "epoch": 1, "t": 3.6651573181152344}
+{"step": 147, "loss": 0.12848125285478504, "grad_norm": 0.15917052328586578, "lr": 5.47443707175741e-05, "epoch": 1, "t": 3.629366397857666}
+{"step": 148, "loss": 0.10528546263287951, "grad_norm": 0.17555207014083862, "lr": 5.418760576667071e-05, "epoch": 1, "t": 3.5238142013549805}
+{"step": 149, "loss": 0.12875179939114104, "grad_norm": 0.13446305692195892, "lr": 5.3630317398107864e-05, "epoch": 1, "t": 4.723986625671387}
+{"step": 150, "loss": 0.1509478544641018, "grad_norm": 0.13347138464450836, "lr": 5.3072575268531835e-05, "epoch": 1, "t": 4.490095376968384}
+{"step": 151, "loss": 0.12530202466880094, "grad_norm": 0.14618465304374695, "lr": 5.2514449091305375e-05, "epoch": 1, "t": 4.730955123901367}
+{"step": 152, "loss": 0.11692930938117567, "grad_norm": 0.164962038397789, "lr": 5.195600862779421e-05, "epoch": 1, "t": 4.710339307785034}
+{"step": 153, "loss": 0.10757210202761774, "grad_norm": 0.15019534528255463, "lr": 5.139732367864736e-05, "epoch": 1, "t": 5.027794599533081}
+{"step": 154, "loss": 0.1179920897759708, "grad_norm": 0.15033726394176483, "lr": 5.083846407507263e-05, "epoch": 1, "t": 4.906826496124268}
+{"step": 155, "loss": 0.11832947406510085, "grad_norm": 0.1623435616493225, "lr": 5.0279499670108245e-05, "epoch": 1, "t": 4.743518590927124}
+{"step": 156, "loss": 0.10203120526956297, "grad_norm": 0.16398712992668152, "lr": 4.972050032989175e-05, "epoch": 1, "t": 3.6121954917907715}
+{"step": 157, "loss": 0.1181907468429137, "grad_norm": 0.11775065213441849, "lr": 4.9161535924927374e-05, "epoch": 1, "t": 4.444746255874634}
+{"step": 158, "loss": 0.13365851477381216, "grad_norm": 0.12783850729465485, "lr": 4.860267632135265e-05, "epoch": 1, "t": 4.111499071121216}
+{"step": 159, "loss": 0.10379785406585987, "grad_norm": 0.1300043910741806, "lr": 4.80439913722058e-05, "epoch": 1, "t": 4.153189659118652}
+{"step": 160, "loss": 0.11476812563142977, "grad_norm": 0.145960733294487, "lr": 4.748555090869464e-05, "epoch": 1, "t": 3.5338592529296875}
+{"step": 161, "loss": 0.1370423346114673, "grad_norm": 0.13007645308971405, "lr": 4.692742473146818e-05, "epoch": 1, "t": 4.521167755126953}
+{"step": 162, "loss": 0.09959102768990358, "grad_norm": 0.15772470831871033, "lr": 4.636968260189214e-05, "epoch": 1, "t": 3.801671266555786}
+{"step": 163, "loss": 0.1193753543538536, "grad_norm": 0.16072018444538116, "lr": 4.5812394233329305e-05, "epoch": 1, "t": 3.414161443710327}
+{"step": 164, "loss": 0.11108459793291152, "grad_norm": 0.15110625326633453, "lr": 4.525562928242592e-05, "epoch": 1, "t": 3.474458694458008}
+{"step": 165, "loss": 0.1114395505633592, "grad_norm": 0.12657447159290314, "lr": 4.4699457340405164e-05, "epoch": 1, "t": 4.358025312423706}
+{"step": 166, "loss": 0.12759973523243853, "grad_norm": 0.1536538153886795, "lr": 4.414394792436877e-05, "epoch": 1, "t": 3.6386351585388184}
+{"step": 167, "loss": 0.11425007120649723, "grad_norm": 0.16074058413505554, "lr": 4.3589170468607985e-05, "epoch": 1, "t": 3.6017074584960938}
+{"step": 168, "loss": 0.12221515120531443, "grad_norm": 0.15506289899349213, "lr": 4.3035194315924785e-05, "epoch": 1, "t": 3.5980899333953857}
+{"step": 169, "loss": 0.09485386312263716, "grad_norm": 0.14748530089855194, "lr": 4.248208870896456e-05, "epoch": 1, "t": 3.7743003368377686}
+{"step": 170, "loss": 0.10053683290423472, "grad_norm": 0.15586788952350616, "lr": 4.192992278156141e-05, "epoch": 1, "t": 3.6582369804382324}
+{"step": 171, "loss": 0.11508096356168351, "grad_norm": 0.1507318764925003, "lr": 4.1378765550096835e-05, "epoch": 1, "t": 3.7490265369415283}
+{"step": 172, "loss": 0.11759738370565735, "grad_norm": 0.15970730781555176, "lr": 4.082868590487339e-05, "epoch": 1, "t": 4.123462677001953}
+{"step": 173, "loss": 0.1146153851326715, "grad_norm": 0.1288793534040451, "lr": 4.027975260150381e-05, "epoch": 1, "t": 4.7843146324157715}
+{"step": 174, "loss": 0.15409434991152868, "grad_norm": 0.1334952414035797, "lr": 3.973203425231715e-05, "epoch": 1, "t": 4.8415586948394775}
+{"step": 175, "loss": 0.10594453158383557, "grad_norm": 0.119917131960392, "lr": 3.918559931778277e-05, "epoch": 1, "t": 4.714414119720459}
+{"step": 176, "loss": 0.11431877921790681, "grad_norm": 0.14694754779338837, "lr": 3.8640516097953405e-05, "epoch": 1, "t": 3.5337812900543213}
+{"step": 177, "loss": 0.10651049256346105, "grad_norm": 0.15320785343647003, "lr": 3.809685272392804e-05, "epoch": 1, "t": 3.591143846511841}
+{"step": 178, "loss": 0.12061873007947065, "grad_norm": 0.15389877557754517, "lr": 3.755467714933619e-05, "epoch": 1, "t": 3.7914445400238037}
+{"step": 179, "loss": 0.12240755889254218, "grad_norm": 0.1502438187599182, "lr": 3.701405714184416e-05, "epoch": 1, "t": 3.692538261413574}
+{"step": 180, "loss": 0.11679360225360348, "grad_norm": 0.16168855130672455, "lr": 3.647506027468467e-05, "epoch": 1, "t": 3.506697177886963}
+{"step": 181, "loss": 0.12102859834586374, "grad_norm": 0.15897907316684723, "lr": 3.59377539182107e-05, "epoch": 1, "t": 3.7180252075195312}
+{"step": 182, "loss": 0.15613014170565376, "grad_norm": 0.14551568031311035, "lr": 3.5402205231474736e-05, "epoch": 1, "t": 4.005361557006836}
+{"step": 183, "loss": 0.13691301535698222, "grad_norm": 0.13993695378303528, "lr": 3.486848115383445e-05, "epoch": 1, "t": 4.022477149963379}
+{"step": 184, "loss": 0.08173359445166482, "grad_norm": 0.1540633589029312, "lr": 3.4336648396585776e-05, "epoch": 1, "t": 3.4598209857940674}
+{"step": 185, "loss": 0.09392102006140504, "grad_norm": 0.14285217225551605, "lr": 3.380677343462447e-05, "epoch": 1, "t": 3.7852771282196045}
+{"step": 186, "loss": 0.10778311657106647, "grad_norm": 0.12833146750926971, "lr": 3.327892249813745e-05, "epoch": 1, "t": 4.180497407913208}
+{"step": 187, "loss": 0.09735368181084837, "grad_norm": 0.14746487140655518, "lr": 3.275316156432434e-05, "epoch": 1, "t": 3.3845295906066895}
+{"step": 188, "loss": 0.1106933924598858, "grad_norm": 0.12444605678319931, "lr": 3.2229556349150945e-05, "epoch": 1, "t": 4.292847633361816}
+{"step": 189, "loss": 0.10506920331084783, "grad_norm": 0.127953439950943, "lr": 3.170817229913526e-05, "epoch": 1, "t": 3.817007541656494}
+{"step": 190, "loss": 0.11478458237402218, "grad_norm": 0.136204794049263, "lr": 3.118907458316722e-05, "epoch": 1, "t": 4.102617979049683}
+{"step": 191, "loss": 0.11260973205862121, "grad_norm": 0.13346047699451447, "lr": 3.067232808436299e-05, "epoch": 1, "t": 3.5292575359344482}
+{"step": 192, "loss": 0.10885947123354774, "grad_norm": 0.14595918357372284, "lr": 3.0157997391955172e-05, "epoch": 1, "t": 4.154806613922119}
+{"step": 193, "loss": 0.12449259257073476, "grad_norm": 0.12871134281158447, "lr": 2.964614679321966e-05, "epoch": 1, "t": 4.435597658157349}
+{"step": 194, "loss": 0.11366290464068629, "grad_norm": 0.12920185923576355, "lr": 2.913684026544021e-05, "epoch": 1, "t": 4.284344673156738}
+{"step": 195, "loss": 0.1358792625485009, "grad_norm": 0.1402549296617508, "lr": 2.8630141467911775e-05, "epoch": 1, "t": 4.37727427482605}
+{"step": 196, "loss": 0.12311273336250185, "grad_norm": 0.15585191547870636, "lr": 2.812611373398365e-05, "epoch": 1, "t": 3.5598506927490234}
+{"step": 197, "loss": 0.09702227086770752, "grad_norm": 0.13163289427757263, "lr": 2.762482006314324e-05, "epoch": 1, "t": 4.1130290031433105}
+{"step": 198, "loss": 0.13112939817202493, "grad_norm": 0.12582525610923767, "lr": 2.712632311314165e-05, "epoch": 1, "t": 4.1001598834991455}
+{"step": 199, "loss": 0.12762115688563683, "grad_norm": 0.1452418714761734, "lr": 2.6630685192161992e-05, "epoch": 1, "t": 3.962661027908325}
+{"step": 200, "loss": 0.08314233686630483, "grad_norm": 0.16718623042106628, "lr": 2.6137968251031287e-05, "epoch": 1, "t": 4.203824043273926}
+{"step": 201, "loss": 0.10167121726593402, "grad_norm": 0.1528662145137787, "lr": 2.5648233875477157e-05, "epoch": 1, "t": 4.317088603973389}
+{"step": 202, "loss": 0.13192389867300114, "grad_norm": 0.1674133837223053, "lr": 2.5161543278430054e-05, "epoch": 1, "t": 3.774893045425415}
+{"step": 203, "loss": 0.12390018401221356, "grad_norm": 0.14074653387069702, "lr": 2.4677957292372167e-05, "epoch": 1, "t": 3.8784403800964355}
+{"step": 204, "loss": 0.13487588211283288, "grad_norm": 0.14306378364562988, "lr": 2.419753636173379e-05, "epoch": 1, "t": 3.640265941619873}
+{"step": 205, "loss": 0.09877316394134687, "grad_norm": 0.13038456439971924, "lr": 2.3720340535338348e-05, "epoch": 1, "t": 4.181823968887329}
+{"step": 206, "loss": 0.10256361521687414, "grad_norm": 0.16352051496505737, "lr": 2.3246429458896634e-05, "epoch": 1, "t": 3.405353546142578}
+{"step": 207, "loss": 0.11882789544504789, "grad_norm": 0.14308398962020874, "lr": 2.2775862367551644e-05, "epoch": 1, "t": 4.011658430099487}
+{"step": 208, "loss": 0.15964151489116996, "grad_norm": 0.13759273290634155, "lr": 2.2308698078474645e-05, "epoch": 1, "t": 4.475231647491455}
+{"step": 209, "loss": 0.12079985053667029, "grad_norm": 0.1137390062212944, "lr": 2.1844994983513467e-05, "epoch": 1, "t": 4.367822647094727}
+{"step": 210, "loss": 0.11297780562986309, "grad_norm": 0.13604016602039337, "lr": 2.1384811041894055e-05, "epoch": 1, "t": 3.7651076316833496}
+{"step": 211, "loss": 0.12601321189250536, "grad_norm": 0.14977240562438965, "lr": 2.0928203772975917e-05, "epoch": 1, "t": 3.599444627761841}
+{"step": 212, "loss": 0.10209279082687803, "grad_norm": 0.16619263589382172, "lr": 2.0475230249062725e-05, "epoch": 1, "t": 3.574302911758423}
+{"step": 213, "loss": 0.13344259216092577, "grad_norm": 0.1424628496170044, "lr": 2.0025947088268717e-05, "epoch": 1, "t": 3.6180953979492188}
+{"step": 214, "loss": 0.09996391243766631, "grad_norm": 0.12776514887809753, "lr": 1.958041044744186e-05, "epoch": 1, "t": 4.7851574420928955}
+{"step": 215, "loss": 0.08142723244143586, "grad_norm": 0.12378152459859848, "lr": 1.9138676015144764e-05, "epoch": 1, "t": 3.8571321964263916}
+{"step": 216, "loss": 0.10119122843335937, "grad_norm": 0.142563596367836, "lr": 1.870079900469392e-05, "epoch": 1, "t": 3.513838291168213}
+{"step": 217, "loss": 0.13182106332891058, "grad_norm": 0.1402498334646225, "lr": 1.8266834147258578e-05, "epoch": 1, "t": 3.7528185844421387}
+{"step": 218, "loss": 0.10698880855517098, "grad_norm": 0.14997121691703796, "lr": 1.7836835685019733e-05, "epoch": 1, "t": 3.463362693786621}
+{"step": 219, "loss": 0.11851168632796404, "grad_norm": 0.1383921056985855, "lr": 1.741085736439031e-05, "epoch": 1, "t": 3.7770984172821045}
+{"step": 220, "loss": 0.10725405567273961, "grad_norm": 0.1630048155784607, "lr": 1.698895242929725e-05, "epoch": 1, "t": 3.8139493465423584}
+{"step": 221, "loss": 0.11349897074294599, "grad_norm": 0.17728491127490997, "lr": 1.6571173614526507e-05, "epoch": 1, "t": 3.40334153175354}
+{"step": 222, "loss": 0.09540789412267545, "grad_norm": 0.12932586669921875, "lr": 1.6157573139131527e-05, "epoch": 1, "t": 4.640115976333618}
+{"step": 223, "loss": 0.11829424488320245, "grad_norm": 0.15388445556163788, "lr": 1.5748202699906335e-05, "epoch": 1, "t": 3.6062557697296143}
+{"step": 224, "loss": 0.12648821234629343, "grad_norm": 0.1398853063583374, "lr": 1.534311346492381e-05, "epoch": 1, "t": 3.8080945014953613}
+{"step": 225, "loss": 0.15609883390599516, "grad_norm": 0.15347224473953247, "lr": 1.4942356067140162e-05, "epoch": 1, "t": 3.9130940437316895}
+{"step": 226, "loss": 0.13415967060089729, "grad_norm": 0.13715052604675293, "lr": 1.454598059806609e-05, "epoch": 1, "t": 3.9020955562591553}
+{"step": 227, "loss": 0.11563435789644994, "grad_norm": 0.14221636950969696, "lr": 1.4154036601505832e-05, "epoch": 1, "t": 4.132835865020752}
+{"step": 228, "loss": 0.08223616757059274, "grad_norm": 0.16633552312850952, "lr": 1.376657306736453e-05, "epoch": 1, "t": 3.187106132507324}
+{"step": 229, "loss": 0.09927764060229238, "grad_norm": 0.11439301818609238, "lr": 1.3383638425524908e-05, "epoch": 1, "t": 4.496842861175537}
+{"step": 230, "loss": 0.11370369518669053, "grad_norm": 0.15841107070446014, "lr": 1.3005280539793907e-05, "epoch": 1, "t": 3.7205324172973633}
+{"step": 231, "loss": 0.13029761580254537, "grad_norm": 0.13881075382232666, "lr": 1.2631546701920071e-05, "epoch": 1, "t": 3.9273955821990967}
+{"step": 232, "loss": 0.17538909194434854, "grad_norm": 0.15010155737400055, "lr": 1.2262483625682513e-05, "epoch": 1, "t": 4.264605522155762}
+{"step": 233, "loss": 0.12532228732823253, "grad_norm": 0.12920606136322021, "lr": 1.1898137441051982e-05, "epoch": 1, "t": 3.922173261642456}
+{"step": 234, "loss": 0.10490559619719997, "grad_norm": 0.14111612737178802, "lr": 1.1538553688425003e-05, "epoch": 1, "t": 3.6933648586273193}
+{"step": 235, "loss": 0.12609147858949485, "grad_norm": 0.13332849740982056, "lr": 1.1183777312931748e-05, "epoch": 1, "t": 3.783797264099121}
+{"step": 236, "loss": 0.11044409572183755, "grad_norm": 0.1406199336051941, "lr": 1.0833852658818166e-05, "epoch": 1, "t": 3.7271251678466797}
+{"step": 237, "loss": 0.11232635000286145, "grad_norm": 0.16482612490653992, "lr": 1.0488823463903342e-05, "epoch": 1, "t": 3.616563081741333}
+{"step": 238, "loss": 0.12694786186116486, "grad_norm": 0.14727739989757538, "lr": 1.0148732854112619e-05, "epoch": 1, "t": 3.5748870372772217}
+{"step": 239, "loss": 0.11935697266932596, "grad_norm": 0.14455722272396088, "lr": 9.81362333808718e-06, "epoch": 1, "t": 3.6619653701782227}
+{"step": 240, "loss": 0.1390664554527734, "grad_norm": 0.1355101615190506, "lr": 9.483536801870834e-06, "epoch": 1, "t": 3.790834426879883}
+{"step": 241, "loss": 0.07628622097361809, "grad_norm": 0.17834733426570892, "lr": 9.158514503674543e-06, "epoch": 1, "t": 3.256704568862915}
+{"step": 242, "loss": 0.12669934513195064, "grad_norm": 0.1178196370601654, "lr": 8.838597068719518e-06, "epoch": 1, "t": 4.258600473403931}
+{"step": 243, "loss": 0.12453458473472875, "grad_norm": 0.15305942296981812, "lr": 8.523824484159349e-06, "epoch": 1, "t": 3.6610565185546875}
+{"step": 244, "loss": 0.10037668471655922, "grad_norm": 0.12658074498176575, "lr": 8.21423609408199e-06, "epoch": 1, "t": 4.66134238243103}
+{"step": 245, "loss": 0.13224196958762482, "grad_norm": 0.14453445374965668, "lr": 7.90987059459195e-06, "epoch": 1, "t": 4.476741552352905}
+{"step": 246, "loss": 0.11854139076453014, "grad_norm": 0.30402347445487976, "lr": 7.610766028973709e-06, "epoch": 1, "t": 3.633941173553467}
+{"step": 247, "loss": 0.1067549350487925, "grad_norm": 0.1502123475074768, "lr": 7.3169597829365165e-06, "epoch": 1, "t": 3.395038604736328}
+{"step": 248, "loss": 0.1352108664982657, "grad_norm": 0.13033181428909302, "lr": 7.028488579941506e-06, "epoch": 1, "t": 4.167044162750244}
+{"step": 249, "loss": 0.10094051416397742, "grad_norm": 0.1356983780860901, "lr": 6.745388476611553e-06, "epoch": 1, "t": 3.649034023284912}
+{"step": 250, "loss": 0.135374080982781, "grad_norm": 0.13638220727443695, "lr": 6.467694858224488e-06, "epoch": 1, "t": 3.8436057567596436}
+{"step": 251, "loss": 0.11666602185815654, "grad_norm": 0.16378286480903625, "lr": 6.1954424342902e-06, "epoch": 1, "t": 3.6649906635284424}
+{"step": 252, "loss": 0.10032906990550024, "grad_norm": 0.12634676694869995, "lr": 5.928665234212233e-06, "epoch": 1, "t": 3.7430708408355713}
+{"step": 253, "loss": 0.11894526571436313, "grad_norm": 0.16291949152946472, "lr": 5.66739660303437e-06, "epoch": 1, "t": 3.537200689315796}
+{"step": 254, "loss": 0.11362934677372613, "grad_norm": 0.1401101052761078, "lr": 5.411669197272795e-06, "epoch": 1, "t": 4.018866062164307}
+{"step": 255, "loss": 0.12557798701497622, "grad_norm": 0.14575357735157013, "lr": 5.161514980834231e-06, "epoch": 1, "t": 3.8143506050109863}
+{"step": 256, "loss": 0.11233742288191656, "grad_norm": 0.14588980376720428, "lr": 4.916965221020753e-06, "epoch": 1, "t": 3.596557140350342}
+{"step": 257, "loss": 0.0902981326032225, "grad_norm": 0.17925389111042023, "lr": 4.678050484621615e-06, "epoch": 1, "t": 3.2069554328918457}
+{"step": 258, "loss": 0.12087348164689377, "grad_norm": 0.15311621129512787, "lr": 4.444800634092616e-06, "epoch": 1, "t": 3.8486878871917725}
+{"step": 259, "loss": 0.09102764636697255, "grad_norm": 0.1572832465171814, "lr": 4.217244823823546e-06, "epoch": 1, "t": 3.3647727966308594}
+{"step": 260, "loss": 0.11781366522145331, "grad_norm": 0.12809409201145172, "lr": 3.995411496494134e-06, "epoch": 1, "t": 4.527900695800781}
+{"step": 261, "loss": 0.11042191457795347, "grad_norm": 0.14428626000881195, "lr": 3.7793283795188984e-06, "epoch": 1, "t": 3.557295560836792}
+{"step": 262, "loss": 0.3091829594925404, "grad_norm": 0.13508212566375732, "lr": 3.56902248158148e-06, "epoch": 1, "t": 3.6631529331207275}
+{"step": 263, "loss": 0.11417226727084936, "grad_norm": 0.1770130693912506, "lr": 3.364520089258727e-06, "epoch": 1, "t": 3.507154941558838}
+{"step": 264, "loss": 0.13347980415140742, "grad_norm": 0.14657466113567352, "lr": 3.165846763735153e-06, "epoch": 1, "t": 3.987422227859497}
+{"step": 265, "loss": 0.0807670794169448, "grad_norm": 0.28558149933815, "lr": 2.973027337607892e-06, "epoch": 1, "t": 4.925394058227539}
+{"step": 266, "loss": 0.10616550390617119, "grad_norm": 0.1513078510761261, "lr": 2.7860859117828987e-06, "epoch": 1, "t": 3.3849599361419678}
+{"step": 267, "loss": 0.0936647256703576, "grad_norm": 0.14811484515666962, "lr": 2.605045852462473e-06, "epoch": 1, "t": 3.49108624458313}
+{"step": 268, "loss": 0.11292326740360294, "grad_norm": 0.15452927350997925, "lr": 2.429929788224722e-06, "epoch": 1, "t": 3.6051175594329834}
+{"step": 269, "loss": 0.15075857057464595, "grad_norm": 0.1656741201877594, "lr": 2.2607596071951286e-06, "epoch": 1, "t": 3.494446039199829}
+{"step": 270, "loss": 0.0967751070121623, "grad_norm": 0.12057405710220337, "lr": 2.097556454310701e-06, "epoch": 1, "t": 3.8424792289733887}
+{"step": 271, "loss": 0.1223120673725397, "grad_norm": 0.14427237212657928, "lr": 1.940340728677059e-06, "epoch": 1, "t": 3.821413040161133}
+{"step": 272, "loss": 0.09642508509264913, "grad_norm": 0.14112572371959686, "lr": 1.789132081018674e-06, "epoch": 1, "t": 3.763314723968506}
+{"step": 273, "loss": 0.1021586982738689, "grad_norm": 0.12266974151134491, "lr": 1.6439494112227172e-06, "epoch": 1, "t": 3.953784465789795}
+{"step": 274, "loss": 0.13481505630456478, "grad_norm": 0.17717137932777405, "lr": 1.5048108659766691e-06, "epoch": 1, "t": 3.386126756668091}
+{"step": 275, "loss": 0.13525642401287985, "grad_norm": 0.13733552396297455, "lr": 1.3717338365001942e-06, "epoch": 1, "t": 3.9524667263031006}
+{"step": 276, "loss": 0.09955735719859671, "grad_norm": 0.1621832400560379, "lr": 1.2447349563713184e-06, "epoch": 1, "t": 3.394646406173706}
+{"step": 277, "loss": 0.11079738014516809, "grad_norm": 0.13838042318820953, "lr": 1.1238300994473983e-06, "epoch": 1, "t": 3.621666431427002}
+{"step": 278, "loss": 0.12100417742200649, "grad_norm": 0.16563449800014496, "lr": 1.0090343778809908e-06, "epoch": 1, "t": 3.509856939315796}
+{"step": 279, "loss": 0.10947836514827758, "grad_norm": 0.1587364375591278, "lr": 9.003621402309814e-07, "epoch": 1, "t": 3.620316505432129}
+{"step": 280, "loss": 0.1164047812580694, "grad_norm": 0.14711160957813263, "lr": 7.97826969669102e-07, "epoch": 1, "t": 3.5458240509033203}
+{"step": 281, "loss": 0.1569018728605597, "grad_norm": 0.15343400835990906, "lr": 7.014416822821556e-07, "epoch": 1, "t": 3.9404256343841553}
+{"step": 282, "loss": 0.11495570472518163, "grad_norm": 0.129398375749588, "lr": 6.112183254700865e-07, "epoch": 1, "t": 3.956192970275879}
+{"step": 283, "loss": 0.1002347370381468, "grad_norm": 0.15354539453983307, "lr": 5.271681764401848e-07, "epoch": 1, "t": 3.448136806488037}
+{"step": 284, "loss": 0.13742754821937742, "grad_norm": 0.13588854670524597, "lr": 4.493017407975086e-07, "epoch": 1, "t": 3.836602210998535}
+{"step": 285, "loss": 0.10330419539409998, "grad_norm": 0.16069848835468292, "lr": 3.7762875123173445e-07, "epoch": 1, "t": 3.3542020320892334}
+{"step": 286, "loss": 0.12895261163211477, "grad_norm": 0.1446145474910736, "lr": 3.1215816630071335e-07, "epoch": 1, "t": 4.222714900970459}
+{"step": 287, "loss": 0.12660529905666593, "grad_norm": 0.13086554408073425, "lr": 2.528981693106558e-07, "epoch": 1, "t": 4.231558322906494}
+{"step": 288, "loss": 0.09218426077603369, "grad_norm": 0.1622740775346756, "lr": 1.9985616729332747e-07, "epoch": 1, "t": 3.656874895095825}
+{"step": 289, "loss": 0.15803322208105147, "grad_norm": 0.1531166434288025, "lr": 1.530387900802177e-07, "epoch": 1, "t": 4.030688524246216}
+{"step": 290, "loss": 0.0967807320836573, "grad_norm": 0.14130055904388428, "lr": 1.1245188947384134e-07, "epoch": 1, "t": 3.5183839797973633}
+{"step": 291, "loss": 0.11862516153327242, "grad_norm": 0.1410289704799652, "lr": 7.81005385163458e-08, "epoch": 1, "t": 3.98561429977417}
+{"step": 292, "loss": 0.1350538710351048, "grad_norm": 0.12859372794628143, "lr": 4.998903085539075e-08, "epoch": 1, "t": 4.319948673248291}
+{"step": 293, "loss": 0.09229648732841635, "grad_norm": 0.1369943767786026, "lr": 2.8120880207493928e-08, "epoch": 1, "t": 3.4407806396484375}
+{"step": 294, "loss": 0.11588857106724462, "grad_norm": 0.12926819920539856, "lr": 1.2498819918843607e-08, "epoch": 1, "t": 4.787572145462036}
+{"step": 295, "loss": 0.09979496542355493, "grad_norm": 0.14342445135116577, "lr": 3.1248026236274652e-09, "epoch": 1, "t": 4.008579969406128}
+{"step": 296, "loss": 0.12066702086379345, "grad_norm": 0.1295050084590912, "lr": 0.0, "epoch": 1, "t": 3.95794939994812}