Upload checkpoint from math_self_distill_INP-OH_u0.001-1.0_gold1_target1_ce0.5

.gitattributes

@@ -37,3 +37,4 @@ math/inp/SD-INP/math_self_distill_INP_gold1_target1_ce0.5/debug_training_example
 math/inp/SD-INP/math_self_distill_INP_u0.0-0.4_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/inp/SD-INP/math_self_distill_INP_u0.6-1.0_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/inp/SD-INP/math_self_distill_INP_u0.8-1.0_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
+math/SFT/inp-onehot_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text

math/SFT/inp-onehot_gold1_target1_ce0.5/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/SFT/inp-onehot_gold1_target1_ce0.5/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:96284485e9de44a9ac3289ee61d486bfc306d90d92d23745450f0fd7ababdaf8
+size 2406624648

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6fbad00a83471321bc98af1bf0b0d3c3f4cdd22eaf4afa6ed11d09169f007739
+size 2406624648

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:497f5c76d864f95f486324379056249800ee3f4f1ebcfddb6f19879dc55baba8
+size 671304442

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cedbb24eb08e412f2b6567529f919723c479356a0b4861fb1f0133d92b4e4aa
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b0d03a2071fbec29bd91c0dd6369cc6159f83e9366dcf2d4966bac1b9db09adc
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d8cdfceac9f7917b978dca661a3b8e04187faea5d5f6bd7b462d61d8234d57f
+size 1064

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/trainer_state.json

@@ -0,0 +1,273 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.21333333333333335,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7671197377738735,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.8686907805610303,
+      "learning_rate": 3e-06,
+      "loss": 1.2408,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5344198682101251,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.1484375,
+      "kd_loss": 0.7096576771870104,
+      "learning_rate": 3e-06,
+      "loss": 0.9455,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5362298497777374,
+      "epoch": 0.064,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.778877005496804,
+      "learning_rate": 3e-06,
+      "loss": 0.9451,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4614376229008258,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.84375,
+      "kd_loss": 0.6962691646146141,
+      "learning_rate": 3e-06,
+      "loss": 0.8619,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.614507899929265,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.5959198616897993,
+      "learning_rate": 3e-06,
+      "loss": 0.9459,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6723507625403272,
+      "epoch": 0.128,
+      "grad_norm": 0.66015625,
+      "kd_loss": 0.7275705483960166,
+      "learning_rate": 3e-06,
+      "loss": 1.143,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6910149530180434,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 1.4765625,
+      "kd_loss": 0.7948297057602758,
+      "learning_rate": 3e-06,
+      "loss": 1.2612,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5694220800869061,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.333984375,
+      "kd_loss": 0.5803848952520638,
+      "learning_rate": 3e-06,
+      "loss": 0.8156,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9436774675735251,
+      "epoch": 0.192,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.9708034214691906,
+      "learning_rate": 3e-06,
+      "loss": 1.3507,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302744172568055,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.74609375,
+      "kd_loss": 0.7968542006539338,
+      "learning_rate": 3e-06,
+      "loss": 1.1755,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04b6dba924441a3d6deb607920bd9c5c280462edbaacc20eb1bdf853287ddf3d
+size 8056

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f1072e7a174f08cb29690143cda82cf15b6b7c80385296a274f12169186fa75f
+size 2406624648

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0723e17744ff441dc778f424ca3957e39dc8ea8bbd1e952a6aeeb0513673d8ed
+size 671304442

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35f0d8cdba91c0873e4ce9bf07f955fed9abee001f34f2ac1f984f19666a371b
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dd53d865b58debfbcc9fd888322bfe451d0ae8651ddb49493b0508f88a0f3e6b
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29d9aa99505fc60c0db1b9cdacaa08b06e8a85c8aaaab4e389667a719fafb9bf
+size 1064

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/trainer_state.json

@@ -0,0 +1,2433 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.1365333333333334,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7671197377738735,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.8686907805610303,
+      "learning_rate": 3e-06,
+      "loss": 1.2408,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5344198682101251,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.1484375,
+      "kd_loss": 0.7096576771870104,
+      "learning_rate": 3e-06,
+      "loss": 0.9455,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5362298497777374,
+      "epoch": 0.064,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.778877005496804,
+      "learning_rate": 3e-06,
+      "loss": 0.9451,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4614376229008258,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.84375,
+      "kd_loss": 0.6962691646146141,
+      "learning_rate": 3e-06,
+      "loss": 0.8619,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.614507899929265,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.5959198616897993,
+      "learning_rate": 3e-06,
+      "loss": 0.9459,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6723507625403272,
+      "epoch": 0.128,
+      "grad_norm": 0.66015625,
+      "kd_loss": 0.7275705483960166,
+      "learning_rate": 3e-06,
+      "loss": 1.143,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6910149530180434,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 1.4765625,
+      "kd_loss": 0.7948297057602758,
+      "learning_rate": 3e-06,
+      "loss": 1.2612,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5694220800869061,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.333984375,
+      "kd_loss": 0.5803848952520638,
+      "learning_rate": 3e-06,
+      "loss": 0.8156,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9436774675735251,
+      "epoch": 0.192,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.9708034214691906,
+      "learning_rate": 3e-06,
+      "loss": 1.3507,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302744172568055,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.74609375,
+      "kd_loss": 0.7968542006539338,
+      "learning_rate": 3e-06,
+      "loss": 1.1755,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    },
+    {
+      "avg_mask_ratio": 0.5232905174256303,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.5232905174256303,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5488719139095337,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 1.0,
+      "kd_loss": 0.8146776424391475,
+      "learning_rate": 3e-06,
+      "loss": 1.1451,
+      "masked_tokens": 106.4375,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 106.4375
+    },
+    {
+      "avg_mask_ratio": 0.4815562474541366,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4815562474541366,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5119639005151612,
+      "epoch": 0.256,
+      "grad_norm": 7.6875,
+      "kd_loss": 0.7391058675566455,
+      "learning_rate": 3e-06,
+      "loss": 0.9956,
+      "masked_tokens": 102.2,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 102.2
+    },
+    {
+      "avg_mask_ratio": 0.47414465841138737,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.47414465841138737,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46758080123779566,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.4977445501957277,
+      "learning_rate": 3e-06,
+      "loss": 0.7473,
+      "masked_tokens": 94.7875,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 94.7875
+    },
+    {
+      "avg_mask_ratio": 0.523321858420968,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.523321858420968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9225109454039966,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 1.75,
+      "kd_loss": 0.9224564624854793,
+      "learning_rate": 3e-06,
+      "loss": 1.3273,
+      "masked_tokens": 135.4,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 135.4
+    },
+    {
+      "avg_mask_ratio": 0.4975809322553687,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4975809322553687,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6314841133786103,
+      "epoch": 0.32,
+      "grad_norm": 0.09375,
+      "kd_loss": 0.802451879998506,
+      "learning_rate": 3e-06,
+      "loss": 1.1868,
+      "masked_tokens": 129.925,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 129.925
+    },
+    {
+      "avg_mask_ratio": 0.5385947977076284,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5385947977076284,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9218708202128709,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.8715213164375939,
+      "learning_rate": 3e-06,
+      "loss": 1.2067,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.5177937666652724,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.5177937666652724,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7012445787927846,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.94140625,
+      "kd_loss": 0.7625857894104684,
+      "learning_rate": 3e-06,
+      "loss": 1.0771,
+      "masked_tokens": 93.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 93.225
+    },
+    {
+      "avg_mask_ratio": 0.4772969324782025,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.4772969324782025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6828591173752898,
+      "epoch": 0.384,
+      "grad_norm": 0.69921875,
+      "kd_loss": 0.6958191808335584,
+      "learning_rate": 3e-06,
+      "loss": 1.0206,
+      "masked_tokens": 108.8375,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 108.8375
+    },
+    {
+      "avg_mask_ratio": 0.5173690344206989,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.5173690344206989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6138432722670132,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.265625,
+      "kd_loss": 0.7333374981938505,
+      "learning_rate": 3e-06,
+      "loss": 1.0175,
+      "masked_tokens": 114.0625,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 114.0625
+    },
+    {
+      "avg_mask_ratio": 0.49981915440876035,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.49981915440876035,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5009475202074555,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.39453125,
+      "kd_loss": 0.6001196937293571,
+      "learning_rate": 3e-06,
+      "loss": 0.8454,
+      "masked_tokens": 101.175,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.484982778178528,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.484982778178528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4791799169369824,
+      "epoch": 0.448,
+      "grad_norm": 0.953125,
+      "kd_loss": 0.5891184500089366,
+      "learning_rate": 3e-06,
+      "loss": 0.8327,
+      "masked_tokens": 101.2,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.5744095016038046,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5744095016038046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7536524894140711,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.9296875,
+      "kd_loss": 0.9245879702670209,
+      "learning_rate": 3e-06,
+      "loss": 1.3423,
+      "masked_tokens": 129.4,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 129.4
+    },
+    {
+      "avg_mask_ratio": 0.4629370831884444,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4629370831884444,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3100870553826326,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.6333749431331853,
+      "learning_rate": 3e-06,
+      "loss": 0.8768,
+      "masked_tokens": 110.5125,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 110.5125
+    },
+    {
+      "avg_mask_ratio": 0.499816512214602,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.499816512214602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44889634368061593,
+      "epoch": 0.512,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.6445640347630445,
+      "learning_rate": 3e-06,
+      "loss": 0.9596,
+      "masked_tokens": 110.075,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.075
+    },
+    {
+      "avg_mask_ratio": 0.4744578254292719,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4744578254292719,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39997816555569443,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.19140625,
+      "kd_loss": 0.5854355251746852,
+      "learning_rate": 3e-06,
+      "loss": 0.8236,
+      "masked_tokens": 117.1125,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 117.1125
+    },
+    {
+      "avg_mask_ratio": 0.4852474880579393,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.4852474880579393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34563268155263815,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 4.8125,
+      "kd_loss": 0.5606092717916908,
+      "learning_rate": 3e-06,
+      "loss": 0.7208,
+      "masked_tokens": 113.725,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 113.725
+    },
+    {
+      "avg_mask_ratio": 0.565397203550674,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.565397203550674,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6026960281743186,
+      "epoch": 0.576,
+      "grad_norm": 1.0078125,
+      "kd_loss": 0.8927684382426377,
+      "learning_rate": 3e-06,
+      "loss": 1.2617,
+      "masked_tokens": 124.7125,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 124.7125
+    },
+    {
+      "avg_mask_ratio": 0.4814051762456074,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4814051762456074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4806147089428293,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 6.65625,
+      "kd_loss": 0.6031759152804284,
+      "learning_rate": 3e-06,
+      "loss": 0.8716,
+      "masked_tokens": 129.975,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 129.975
+    },
+    {
+      "avg_mask_ratio": 0.4164489531540312,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.4164489531540312,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1550224335986968,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.0869140625,
+      "kd_loss": 0.4830638362604759,
+      "learning_rate": 3e-06,
+      "loss": 0.5862,
+      "masked_tokens": 100.625,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.625
+    },
+    {
+      "avg_mask_ratio": 0.47973727830685675,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.47973727830685675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4442484440705357,
+      "epoch": 0.64,
+      "grad_norm": 1.140625,
+      "kd_loss": 0.7006052142764929,
+      "learning_rate": 3e-06,
+      "loss": 0.9131,
+      "masked_tokens": 107.2375,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 107.2375
+    },
+    {
+      "avg_mask_ratio": 0.514206234831363,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.514206234831363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5049073612585289,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.51171875,
+      "kd_loss": 0.7227865120981732,
+      "learning_rate": 3e-06,
+      "loss": 1.0107,
+      "masked_tokens": 88.925,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 88.925
+    },
+    {
+      "avg_mask_ratio": 0.5238390378654003,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.5238390378654003,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4860030581583942,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.8063735463714693,
+      "learning_rate": 3e-06,
+      "loss": 1.1637,
+      "masked_tokens": 123.25,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 123.25
+    },
+    {
+      "avg_mask_ratio": 0.5409158666618168,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.5409158666618168,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45924132662039485,
+      "epoch": 0.704,
+      "grad_norm": 0.58203125,
+      "kd_loss": 0.7391011167788519,
+      "learning_rate": 3e-06,
+      "loss": 1.0546,
+      "masked_tokens": 132.2625,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 132.2625
+    },
+    {
+      "avg_mask_ratio": 0.47903697268920953,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.47903697268920953,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5926188694903601,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 1.359375,
+      "kd_loss": 0.8297885791466342,
+      "learning_rate": 3e-06,
+      "loss": 1.0715,
+      "masked_tokens": 114.6375,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 114.6375
+    },
+    {
+      "avg_mask_ratio": 0.5254506973840762,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.5254506973840762,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6182753879609549,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 1.203125,
+      "kd_loss": 0.8253819732506245,
+      "learning_rate": 3e-06,
+      "loss": 1.1773,
+      "masked_tokens": 129.7,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 129.7
+    },
+    {
+      "avg_mask_ratio": 0.5038800648180768,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.5038800648180768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3779912759518879,
+      "epoch": 0.768,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.8277858792208462,
+      "learning_rate": 3e-06,
+      "loss": 0.9585,
+      "masked_tokens": 118.8375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 118.8375
+    },
+    {
+      "avg_mask_ratio": 0.5092529703164473,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.5092529703164473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5031921155097961,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.7001321792347881,
+      "learning_rate": 3e-06,
+      "loss": 0.923,
+      "masked_tokens": 130.4375,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 130.4375
+    },
+    {
+      "avg_mask_ratio": 0.47521690553985535,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.47521690553985535,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3017320279206615,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.6370899313044902,
+      "learning_rate": 3e-06,
+      "loss": 0.8137,
+      "masked_tokens": 99.7125,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 99.7125
+    },
+    {
+      "avg_mask_ratio": 0.5089340912294574,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.5089340912294574,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43493460873024786,
+      "epoch": 0.832,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.7282625613909545,
+      "learning_rate": 3e-06,
+      "loss": 1.0052,
+      "masked_tokens": 115.925,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 115.925
+    },
+    {
+      "avg_mask_ratio": 0.5041010878514498,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.5041010878514498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5107963937724207,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.7805601076866878,
+      "learning_rate": 3e-06,
+      "loss": 1.0557,
+      "masked_tokens": 124.875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 124.875
+    },
+    {
+      "avg_mask_ratio": 0.5127229066158179,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.5127229066158179,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7406563252751311,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.625,
+      "kd_loss": 0.9257289324105245,
+      "learning_rate": 3e-06,
+      "loss": 1.1941,
+      "masked_tokens": 123.575,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 123.575
+    },
+    {
+      "avg_mask_ratio": 0.47257317856419834,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.47257317856419834,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2641133719835068,
+      "epoch": 0.896,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.5586602845531161,
+      "learning_rate": 3e-06,
+      "loss": 0.6794,
+      "masked_tokens": 90.175,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 90.175
+    },
+    {
+      "avg_mask_ratio": 0.49090774822980165,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.49090774822980165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4790991306209548,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.484375,
+      "kd_loss": 0.6454372880304617,
+      "learning_rate": 3e-06,
+      "loss": 0.9157,
+      "masked_tokens": 108.85,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4731982925441116,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.4731982925441116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5319532209085537,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.3984375,
+      "kd_loss": 0.7658510596184896,
+      "learning_rate": 3e-06,
+      "loss": 0.9988,
+      "masked_tokens": 111.5125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 111.5125
+    },
+    {
+      "avg_mask_ratio": 0.4575169428717345,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4575169428717345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40062239499485486,
+      "epoch": 0.96,
+      "grad_norm": 0.62890625,
+      "kd_loss": 0.8030378437517811,
+      "learning_rate": 3e-06,
+      "loss": 0.9794,
+      "masked_tokens": 107.8875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 107.8875
+    },
+    {
+      "avg_mask_ratio": 0.5099512930959463,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.5099512930959463,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3675635530332329,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.134765625,
+      "kd_loss": 0.6000972521935182,
+      "learning_rate": 3e-06,
+      "loss": 0.8352,
+      "masked_tokens": 109.275,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 109.275
+    },
+    {
+      "avg_mask_ratio": 0.5108432768334058,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.5108432768334058,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4013952974987552,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 1.03125,
+      "kd_loss": 0.8058514126374532,
+      "learning_rate": 3e-06,
+      "loss": 1.06,
+      "masked_tokens": 111.75,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 111.75
+    },
+    {
+      "avg_mask_ratio": 0.49879020540975033,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.49879020540975033,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4040452508418184,
+      "epoch": 1.0256,
+      "grad_norm": 0.64453125,
+      "kd_loss": 0.7641570946838329,
+      "learning_rate": 3e-06,
+      "loss": 0.9387,
+      "masked_tokens": 121.6875,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 121.6875
+    },
+    {
+      "avg_mask_ratio": 0.48607371354009954,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.48607371354009954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44693371437709006,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.8984375,
+      "kd_loss": 0.6808075895191905,
+      "learning_rate": 3e-06,
+      "loss": 0.9264,
+      "masked_tokens": 102.1625,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 102.1625
+    },
+    {
+      "avg_mask_ratio": 0.5385718538891524,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5385718538891524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.445710831214069,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 1.8984375,
+      "kd_loss": 0.7960160556252959,
+      "learning_rate": 3e-06,
+      "loss": 1.0089,
+      "masked_tokens": 127.6125,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.6125
+    },
+    {
+      "avg_mask_ratio": 0.5356179510476068,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5356179510476068,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5134360113543494,
+      "epoch": 1.0896,
+      "grad_norm": 3.484375,
+      "kd_loss": 0.8251110358912228,
+      "learning_rate": 3e-06,
+      "loss": 1.001,
+      "masked_tokens": 136.725,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 136.725
+    },
+    {
+      "avg_mask_ratio": 0.4930020817089826,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4930020817089826,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4553626166405934,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7196989472281075,
+      "learning_rate": 3e-06,
+      "loss": 0.9774,
+      "masked_tokens": 91.975,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 91.975
+    },
+    {
+      "avg_mask_ratio": 0.4998604157241061,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4998604157241061,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5219662474520191,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.95703125,
+      "kd_loss": 0.8503037900029083,
+      "learning_rate": 3e-06,
+      "loss": 1.0856,
+      "masked_tokens": 103.4125,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 103.4125
+    },
+    {
+      "avg_mask_ratio": 0.5236943962518126,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.5236943962518126,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6011495636476297,
+      "epoch": 1.1536,
+      "grad_norm": 0.6171875,
+      "kd_loss": 0.7388030910891757,
+      "learning_rate": 3e-06,
+      "loss": 1.0347,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.4774137590778992,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4774137590778992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33609242954775026,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.419921875,
+      "kd_loss": 0.6285939413004143,
+      "learning_rate": 3e-06,
+      "loss": 0.7996,
+      "masked_tokens": 101.425,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 101.425
+    },
+    {
+      "avg_mask_ratio": 0.41173738130601123,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.41173738130601123,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3657617368780734,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.8828125,
+      "kd_loss": 0.6714434385379491,
+      "learning_rate": 3e-06,
+      "loss": 0.8279,
+      "masked_tokens": 102.0375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 102.0375
+    },
+    {
+      "avg_mask_ratio": 0.4797614786075428,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.4797614786075428,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37769897556100884,
+      "epoch": 1.2176,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.6094748291181077,
+      "learning_rate": 3e-06,
+      "loss": 0.8231,
+      "masked_tokens": 112.25,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 112.25
+    },
+    {
+      "avg_mask_ratio": 0.4974610014585778,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4974610014585778,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46419010059532867,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 1.2265625,
+      "kd_loss": 0.820088501922146,
+      "learning_rate": 3e-06,
+      "loss": 0.9708,
+      "masked_tokens": 134.025,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 134.025
+    },
+    {
+      "avg_mask_ratio": 0.5565119812032208,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5565119812032208,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4556695409415738,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 1.046875,
+      "kd_loss": 0.848517366728629,
+      "learning_rate": 3e-06,
+      "loss": 1.0779,
+      "masked_tokens": 126.1375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 126.1375
+    },
+    {
+      "avg_mask_ratio": 0.4784870075061917,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.4784870075061917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42650491216649017,
+      "epoch": 1.2816,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.7230841763311446,
+      "learning_rate": 3e-06,
+      "loss": 0.983,
+      "masked_tokens": 113.875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 113.875
+    },
+    {
+      "avg_mask_ratio": 0.5459770569577813,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5459770569577813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46574052337223293,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.21484375,
+      "kd_loss": 0.9031681247121014,
+      "learning_rate": 3e-06,
+      "loss": 1.1601,
+      "masked_tokens": 115.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 115.85
+    },
+    {
+      "avg_mask_ratio": 0.5268841385375709,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.5268841385375709,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5097857009053428,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.44140625,
+      "kd_loss": 0.826706444665524,
+      "learning_rate": 3e-06,
+      "loss": 1.0892,
+      "masked_tokens": 114.6625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 114.6625
+    },
+    {
+      "avg_mask_ratio": 0.5629246362368576,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5629246362368576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5826418710530561,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.5703125,
+      "kd_loss": 0.89890192824449,
+      "learning_rate": 3e-06,
+      "loss": 1.3331,
+      "masked_tokens": 130.675,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.675
+    },
+    {
+      "avg_mask_ratio": 0.5119291188195347,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.5119291188195347,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40580563298177597,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.435546875,
+      "kd_loss": 0.6370190013494721,
+      "learning_rate": 3e-06,
+      "loss": 0.8205,
+      "masked_tokens": 125.9,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 125.9
+    },
+    {
+      "avg_mask_ratio": 0.5539714884362184,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5539714884362184,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.694471138650897,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.9244145819217892,
+      "learning_rate": 3e-06,
+      "loss": 1.2334,
+      "masked_tokens": 131.7625,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 131.7625
+    },
+    {
+      "avg_mask_ratio": 0.5141558598377742,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.5141558598377742,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43524807556412953,
+      "epoch": 1.4096,
+      "grad_norm": 2.375,
+      "kd_loss": 0.7787983914435245,
+      "learning_rate": 3e-06,
+      "loss": 1.0634,
+      "masked_tokens": 133.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 133.35
+    },
+    {
+      "avg_mask_ratio": 0.4895282822311856,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.4895282822311856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40460901753227174,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 1.203125,
+      "kd_loss": 0.5940112132494051,
+      "learning_rate": 3e-06,
+      "loss": 0.8149,
+      "masked_tokens": 123.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 123.125
+    },
+    {
+      "avg_mask_ratio": 0.4951617428450845,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4951617428450845,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48473086243019453,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.44140625,
+      "kd_loss": 0.6884326858420409,
+      "learning_rate": 3e-06,
+      "loss": 0.9258,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.5100495176156983,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.5100495176156983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.519521524004017,
+      "epoch": 1.4736,
+      "grad_norm": 0.59375,
+      "kd_loss": 0.7857662321038787,
+      "learning_rate": 3e-06,
+      "loss": 0.9692,
+      "masked_tokens": 115.8875,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 115.8875
+    },
+    {
+      "avg_mask_ratio": 0.5639110118616373,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5639110118616373,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46224736819546025,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.59375,
+      "kd_loss": 1.0577162121335277,
+      "learning_rate": 3e-06,
+      "loss": 1.2682,
+      "masked_tokens": 138.2,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 138.2
+    },
+    {
+      "avg_mask_ratio": 0.5292218026472255,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.5292218026472255,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35752006234570216,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.6908905010689239,
+      "learning_rate": 3e-06,
+      "loss": 0.8571,
+      "masked_tokens": 113.375,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 113.375
+    },
+    {
+      "avg_mask_ratio": 0.5125403102487326,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.5125403102487326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5403474027357873,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 1.1796875,
+      "kd_loss": 0.8581615810285712,
+      "learning_rate": 3e-06,
+      "loss": 1.09,
+      "masked_tokens": 115.675,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 115.675
+    },
+    {
+      "avg_mask_ratio": 0.48811948703369124,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.48811948703369124,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5603859513967677,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.7109375,
+      "kd_loss": 0.7485213522588197,
+      "learning_rate": 3e-06,
+      "loss": 1.0393,
+      "masked_tokens": 106.65,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.5547609420493245,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5547609420493245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6015421481137537,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.9012988628433959,
+      "learning_rate": 3e-06,
+      "loss": 1.226,
+      "masked_tokens": 100.775,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 100.775
+    },
+    {
+      "avg_mask_ratio": 0.44697874613921157,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.44697874613921157,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45085387741235083,
+      "epoch": 1.6016,
+      "grad_norm": 0.76171875,
+      "kd_loss": 0.771520164485878,
+      "learning_rate": 3e-06,
+      "loss": 0.9446,
+      "masked_tokens": 99.5,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 99.5
+    },
+    {
+      "avg_mask_ratio": 0.49905171967693607,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.49905171967693607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5226021331908157,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.76953125,
+      "kd_loss": 0.9288661203041159,
+      "learning_rate": 3e-06,
+      "loss": 1.0794,
+      "masked_tokens": 111.525,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 111.525
+    },
+    {
+      "avg_mask_ratio": 0.4734679562970996,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4734679562970996,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33050077693034724,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.73828125,
+      "kd_loss": 0.6156658631806067,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 124.1625,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 124.1625
+    },
+    {
+      "avg_mask_ratio": 0.45589545626135075,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.45589545626135075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41272709482695974,
+      "epoch": 1.6656,
+      "grad_norm": 0.4765625,
+      "kd_loss": 0.6095967918252938,
+      "learning_rate": 3e-06,
+      "loss": 0.7507,
+      "masked_tokens": 120.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 120.2
+    },
+    {
+      "avg_mask_ratio": 0.4975356309209019,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4975356309209019,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4011998525083527,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.15625,
+      "kd_loss": 0.6194601121176675,
+      "learning_rate": 3e-06,
+      "loss": 0.8021,
+      "masked_tokens": 107.35,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 107.35
+    },
+    {
+      "avg_mask_ratio": 0.4948011673986912,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.4948011673986912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3284698034103485,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.5971616579688088,
+      "learning_rate": 3e-06,
+      "loss": 0.8092,
+      "masked_tokens": 109.1875,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 109.1875
+    },
+    {
+      "avg_mask_ratio": 0.5321399106411263,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5321399106411263,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5248136481198913,
+      "epoch": 1.7296,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.7927273895948019,
+      "learning_rate": 3e-06,
+      "loss": 1.0943,
+      "masked_tokens": 123.0375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 123.0375
+    },
+    {
+      "avg_mask_ratio": 0.5357416228158399,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5357416228158399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5000895128354841,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.859375,
+      "kd_loss": 0.9356607880370575,
+      "learning_rate": 3e-06,
+      "loss": 1.1976,
+      "masked_tokens": 121.5625,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 121.5625
+    },
+    {
+      "avg_mask_ratio": 0.5232944375369698,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.5232944375369698,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48456703309973365,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.8498503854701539,
+      "learning_rate": 3e-06,
+      "loss": 1.0467,
+      "masked_tokens": 138.675,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 138.675
+    },
+    {
+      "avg_mask_ratio": 0.5344608084415086,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5344608084415086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39900637990784843,
+      "epoch": 1.7936,
+      "grad_norm": 0.1962890625,
+      "kd_loss": 0.6959655691830562,
+      "learning_rate": 3e-06,
+      "loss": 0.8985,
+      "masked_tokens": 119.225,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 119.225
+    },
+    {
+      "avg_mask_ratio": 0.5352845921181142,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5352845921181142,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3846706166316153,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.458984375,
+      "kd_loss": 0.6893469515551714,
+      "learning_rate": 3e-06,
+      "loss": 0.8883,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4979630701942369,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4979630701942369,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.49622775785310863,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.73828125,
+      "kd_loss": 0.784965463258402,
+      "learning_rate": 3e-06,
+      "loss": 0.964,
+      "masked_tokens": 111.275,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 111.275
+    },
+    {
+      "avg_mask_ratio": 0.5208624298567883,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.5208624298567883,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3778860895960065,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.609375,
+      "kd_loss": 0.7243039658023435,
+      "learning_rate": 3e-06,
+      "loss": 1.0455,
+      "masked_tokens": 119.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 119.8875
+    },
+    {
+      "avg_mask_ratio": 0.4884064760175534,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.4884064760175534,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3462603269857141,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.7865955847492956,
+      "learning_rate": 3e-06,
+      "loss": 0.9653,
+      "masked_tokens": 97.0,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 97.0
+    },
+    {
+      "avg_mask_ratio": 0.47601241993543225,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.47601241993543225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2950649654762401,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.1845703125,
+      "kd_loss": 0.5946491838043585,
+      "learning_rate": 3e-06,
+      "loss": 0.6996,
+      "masked_tokens": 107.1375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 107.1375
+    },
+    {
+      "avg_mask_ratio": 0.4820589871611446,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4820589871611446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41851851929281453,
+      "epoch": 1.9216,
+      "grad_norm": 0.67578125,
+      "kd_loss": 0.7024738637371911,
+      "learning_rate": 3e-06,
+      "loss": 0.9338,
+      "masked_tokens": 106.675,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 106.675
+    },
+    {
+      "avg_mask_ratio": 0.5009820312960074,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.5009820312960074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44660618857540724,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.447265625,
+      "kd_loss": 0.6575563041935993,
+      "learning_rate": 3e-06,
+      "loss": 0.8679,
+      "masked_tokens": 129.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 129.1625
+    },
+    {
+      "avg_mask_ratio": 0.4952817424898967,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4952817424898967,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4072961182277595,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 1.65625,
+      "kd_loss": 0.773787010011074,
+      "learning_rate": 3e-06,
+      "loss": 0.9519,
+      "masked_tokens": 114.2625,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 114.2625
+    },
+    {
+      "avg_mask_ratio": 0.5025755434762686,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.5025755434762686,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44203572303481453,
+      "epoch": 1.9856,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.6455665581320773,
+      "learning_rate": 3e-06,
+      "loss": 0.8321,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5328231096001608,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5328231096001608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34336739452088033,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.7452835773230098,
+      "learning_rate": 3e-06,
+      "loss": 1.0129,
+      "masked_tokens": 126.51190476190476,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 126.51190476190476
+    },
+    {
+      "avg_mask_ratio": 0.46634063599049114,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.46634063599049114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.345527906726322,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 1.8203125,
+      "kd_loss": 0.6856312883097416,
+      "learning_rate": 3e-06,
+      "loss": 0.8718,
+      "masked_tokens": 111.15,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 111.15
+    },
+    {
+      "avg_mask_ratio": 0.5202614731155336,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.5202614731155336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4029362733661742,
+      "epoch": 2.0512,
+      "grad_norm": 0.404296875,
+      "kd_loss": 0.8637022192546169,
+      "learning_rate": 3e-06,
+      "loss": 1.0614,
+      "masked_tokens": 146.275,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 146.275
+    },
+    {
+      "avg_mask_ratio": 0.4732307325524744,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4732307325524744,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41734947142567763,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 2.015625,
+      "kd_loss": 0.6341307566849423,
+      "learning_rate": 3e-06,
+      "loss": 0.8378,
+      "masked_tokens": 111.6375,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 111.6375
+    },
+    {
+      "avg_mask_ratio": 0.45015103057958183,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.45015103057958183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2503517944936732,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.546875,
+      "kd_loss": 0.5644539449379409,
+      "learning_rate": 3e-06,
+      "loss": 0.7301,
+      "masked_tokens": 102.2875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 102.2875
+    },
+    {
+      "avg_mask_ratio": 0.48529006402241065,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.48529006402241065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24893513410114565,
+      "epoch": 2.1152,
+      "grad_norm": 0.345703125,
+      "kd_loss": 0.5718885382049848,
+      "learning_rate": 3e-06,
+      "loss": 0.6848,
+      "masked_tokens": 123.075,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 123.075
+    },
+    {
+      "avg_mask_ratio": 0.4696127205621451,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4696127205621451,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35570654946394314,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.5947819571083528,
+      "learning_rate": 3e-06,
+      "loss": 0.7695,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 103.0875
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04b6dba924441a3d6deb607920bd9c5c280462edbaacc20eb1bdf853287ddf3d
+size 8056

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3abdc19eea4b0fd5e0fa80bc607e5c77877e6f2878ae04aaa9b385342066c68e
+size 2406624648

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d8538f3c63711ffe0ab0f8c4fd6700045e9106735570ec000f46a23c681bd71
+size 671304442

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4d951b3681768d0a5bb5b4a429126b0d534a20a49c0499d63f2afab759f4fb3
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b99783a29a620816bb0a632d5d2cf8313aa70711e5da7dbe41120d24b53c799f
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25795e3b7374d0f6abdd7ab4b34fbf7ab0447ba73c04014500c2ab8b5acec5b4
+size 1064

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/trainer_state.json

@@ -0,0 +1,2673 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.3498666666666668,
+  "eval_steps": 500,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7671197377738735,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.8686907805610303,
+      "learning_rate": 3e-06,
+      "loss": 1.2408,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5344198682101251,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.1484375,
+      "kd_loss": 0.7096576771870104,
+      "learning_rate": 3e-06,
+      "loss": 0.9455,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5362298497777374,
+      "epoch": 0.064,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.778877005496804,
+      "learning_rate": 3e-06,
+      "loss": 0.9451,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4614376229008258,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.84375,
+      "kd_loss": 0.6962691646146141,
+      "learning_rate": 3e-06,
+      "loss": 0.8619,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.614507899929265,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.5959198616897993,
+      "learning_rate": 3e-06,
+      "loss": 0.9459,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6723507625403272,
+      "epoch": 0.128,
+      "grad_norm": 0.66015625,
+      "kd_loss": 0.7275705483960166,
+      "learning_rate": 3e-06,
+      "loss": 1.143,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6910149530180434,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 1.4765625,
+      "kd_loss": 0.7948297057602758,
+      "learning_rate": 3e-06,
+      "loss": 1.2612,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5694220800869061,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.333984375,
+      "kd_loss": 0.5803848952520638,
+      "learning_rate": 3e-06,
+      "loss": 0.8156,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9436774675735251,
+      "epoch": 0.192,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.9708034214691906,
+      "learning_rate": 3e-06,
+      "loss": 1.3507,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302744172568055,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.74609375,
+      "kd_loss": 0.7968542006539338,
+      "learning_rate": 3e-06,
+      "loss": 1.1755,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    },
+    {
+      "avg_mask_ratio": 0.5232905174256303,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.5232905174256303,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5488719139095337,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 1.0,
+      "kd_loss": 0.8146776424391475,
+      "learning_rate": 3e-06,
+      "loss": 1.1451,
+      "masked_tokens": 106.4375,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 106.4375
+    },
+    {
+      "avg_mask_ratio": 0.4815562474541366,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4815562474541366,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5119639005151612,
+      "epoch": 0.256,
+      "grad_norm": 7.6875,
+      "kd_loss": 0.7391058675566455,
+      "learning_rate": 3e-06,
+      "loss": 0.9956,
+      "masked_tokens": 102.2,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 102.2
+    },
+    {
+      "avg_mask_ratio": 0.47414465841138737,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.47414465841138737,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46758080123779566,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.4977445501957277,
+      "learning_rate": 3e-06,
+      "loss": 0.7473,
+      "masked_tokens": 94.7875,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 94.7875
+    },
+    {
+      "avg_mask_ratio": 0.523321858420968,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.523321858420968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9225109454039966,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 1.75,
+      "kd_loss": 0.9224564624854793,
+      "learning_rate": 3e-06,
+      "loss": 1.3273,
+      "masked_tokens": 135.4,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 135.4
+    },
+    {
+      "avg_mask_ratio": 0.4975809322553687,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4975809322553687,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6314841133786103,
+      "epoch": 0.32,
+      "grad_norm": 0.09375,
+      "kd_loss": 0.802451879998506,
+      "learning_rate": 3e-06,
+      "loss": 1.1868,
+      "masked_tokens": 129.925,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 129.925
+    },
+    {
+      "avg_mask_ratio": 0.5385947977076284,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5385947977076284,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9218708202128709,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.8715213164375939,
+      "learning_rate": 3e-06,
+      "loss": 1.2067,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.5177937666652724,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.5177937666652724,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7012445787927846,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.94140625,
+      "kd_loss": 0.7625857894104684,
+      "learning_rate": 3e-06,
+      "loss": 1.0771,
+      "masked_tokens": 93.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 93.225
+    },
+    {
+      "avg_mask_ratio": 0.4772969324782025,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.4772969324782025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6828591173752898,
+      "epoch": 0.384,
+      "grad_norm": 0.69921875,
+      "kd_loss": 0.6958191808335584,
+      "learning_rate": 3e-06,
+      "loss": 1.0206,
+      "masked_tokens": 108.8375,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 108.8375
+    },
+    {
+      "avg_mask_ratio": 0.5173690344206989,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.5173690344206989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6138432722670132,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.265625,
+      "kd_loss": 0.7333374981938505,
+      "learning_rate": 3e-06,
+      "loss": 1.0175,
+      "masked_tokens": 114.0625,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 114.0625
+    },
+    {
+      "avg_mask_ratio": 0.49981915440876035,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.49981915440876035,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5009475202074555,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.39453125,
+      "kd_loss": 0.6001196937293571,
+      "learning_rate": 3e-06,
+      "loss": 0.8454,
+      "masked_tokens": 101.175,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.484982778178528,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.484982778178528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4791799169369824,
+      "epoch": 0.448,
+      "grad_norm": 0.953125,
+      "kd_loss": 0.5891184500089366,
+      "learning_rate": 3e-06,
+      "loss": 0.8327,
+      "masked_tokens": 101.2,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.5744095016038046,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5744095016038046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7536524894140711,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.9296875,
+      "kd_loss": 0.9245879702670209,
+      "learning_rate": 3e-06,
+      "loss": 1.3423,
+      "masked_tokens": 129.4,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 129.4
+    },
+    {
+      "avg_mask_ratio": 0.4629370831884444,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4629370831884444,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3100870553826326,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.6333749431331853,
+      "learning_rate": 3e-06,
+      "loss": 0.8768,
+      "masked_tokens": 110.5125,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 110.5125
+    },
+    {
+      "avg_mask_ratio": 0.499816512214602,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.499816512214602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44889634368061593,
+      "epoch": 0.512,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.6445640347630445,
+      "learning_rate": 3e-06,
+      "loss": 0.9596,
+      "masked_tokens": 110.075,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.075
+    },
+    {
+      "avg_mask_ratio": 0.4744578254292719,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4744578254292719,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39997816555569443,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.19140625,
+      "kd_loss": 0.5854355251746852,
+      "learning_rate": 3e-06,
+      "loss": 0.8236,
+      "masked_tokens": 117.1125,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 117.1125
+    },
+    {
+      "avg_mask_ratio": 0.4852474880579393,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.4852474880579393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34563268155263815,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 4.8125,
+      "kd_loss": 0.5606092717916908,
+      "learning_rate": 3e-06,
+      "loss": 0.7208,
+      "masked_tokens": 113.725,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 113.725
+    },
+    {
+      "avg_mask_ratio": 0.565397203550674,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.565397203550674,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6026960281743186,
+      "epoch": 0.576,
+      "grad_norm": 1.0078125,
+      "kd_loss": 0.8927684382426377,
+      "learning_rate": 3e-06,
+      "loss": 1.2617,
+      "masked_tokens": 124.7125,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 124.7125
+    },
+    {
+      "avg_mask_ratio": 0.4814051762456074,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4814051762456074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4806147089428293,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 6.65625,
+      "kd_loss": 0.6031759152804284,
+      "learning_rate": 3e-06,
+      "loss": 0.8716,
+      "masked_tokens": 129.975,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 129.975
+    },
+    {
+      "avg_mask_ratio": 0.4164489531540312,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.4164489531540312,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1550224335986968,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.0869140625,
+      "kd_loss": 0.4830638362604759,
+      "learning_rate": 3e-06,
+      "loss": 0.5862,
+      "masked_tokens": 100.625,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.625
+    },
+    {
+      "avg_mask_ratio": 0.47973727830685675,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.47973727830685675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4442484440705357,
+      "epoch": 0.64,
+      "grad_norm": 1.140625,
+      "kd_loss": 0.7006052142764929,
+      "learning_rate": 3e-06,
+      "loss": 0.9131,
+      "masked_tokens": 107.2375,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 107.2375
+    },
+    {
+      "avg_mask_ratio": 0.514206234831363,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.514206234831363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5049073612585289,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.51171875,
+      "kd_loss": 0.7227865120981732,
+      "learning_rate": 3e-06,
+      "loss": 1.0107,
+      "masked_tokens": 88.925,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 88.925
+    },
+    {
+      "avg_mask_ratio": 0.5238390378654003,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.5238390378654003,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4860030581583942,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.8063735463714693,
+      "learning_rate": 3e-06,
+      "loss": 1.1637,
+      "masked_tokens": 123.25,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 123.25
+    },
+    {
+      "avg_mask_ratio": 0.5409158666618168,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.5409158666618168,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45924132662039485,
+      "epoch": 0.704,
+      "grad_norm": 0.58203125,
+      "kd_loss": 0.7391011167788519,
+      "learning_rate": 3e-06,
+      "loss": 1.0546,
+      "masked_tokens": 132.2625,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 132.2625
+    },
+    {
+      "avg_mask_ratio": 0.47903697268920953,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.47903697268920953,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5926188694903601,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 1.359375,
+      "kd_loss": 0.8297885791466342,
+      "learning_rate": 3e-06,
+      "loss": 1.0715,
+      "masked_tokens": 114.6375,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 114.6375
+    },
+    {
+      "avg_mask_ratio": 0.5254506973840762,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.5254506973840762,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6182753879609549,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 1.203125,
+      "kd_loss": 0.8253819732506245,
+      "learning_rate": 3e-06,
+      "loss": 1.1773,
+      "masked_tokens": 129.7,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 129.7
+    },
+    {
+      "avg_mask_ratio": 0.5038800648180768,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.5038800648180768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3779912759518879,
+      "epoch": 0.768,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.8277858792208462,
+      "learning_rate": 3e-06,
+      "loss": 0.9585,
+      "masked_tokens": 118.8375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 118.8375
+    },
+    {
+      "avg_mask_ratio": 0.5092529703164473,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.5092529703164473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5031921155097961,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.7001321792347881,
+      "learning_rate": 3e-06,
+      "loss": 0.923,
+      "masked_tokens": 130.4375,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 130.4375
+    },
+    {
+      "avg_mask_ratio": 0.47521690553985535,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.47521690553985535,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3017320279206615,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.6370899313044902,
+      "learning_rate": 3e-06,
+      "loss": 0.8137,
+      "masked_tokens": 99.7125,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 99.7125
+    },
+    {
+      "avg_mask_ratio": 0.5089340912294574,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.5089340912294574,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43493460873024786,
+      "epoch": 0.832,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.7282625613909545,
+      "learning_rate": 3e-06,
+      "loss": 1.0052,
+      "masked_tokens": 115.925,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 115.925
+    },
+    {
+      "avg_mask_ratio": 0.5041010878514498,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.5041010878514498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5107963937724207,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.7805601076866878,
+      "learning_rate": 3e-06,
+      "loss": 1.0557,
+      "masked_tokens": 124.875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 124.875
+    },
+    {
+      "avg_mask_ratio": 0.5127229066158179,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.5127229066158179,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7406563252751311,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.625,
+      "kd_loss": 0.9257289324105245,
+      "learning_rate": 3e-06,
+      "loss": 1.1941,
+      "masked_tokens": 123.575,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 123.575
+    },
+    {
+      "avg_mask_ratio": 0.47257317856419834,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.47257317856419834,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2641133719835068,
+      "epoch": 0.896,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.5586602845531161,
+      "learning_rate": 3e-06,
+      "loss": 0.6794,
+      "masked_tokens": 90.175,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 90.175
+    },
+    {
+      "avg_mask_ratio": 0.49090774822980165,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.49090774822980165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4790991306209548,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.484375,
+      "kd_loss": 0.6454372880304617,
+      "learning_rate": 3e-06,
+      "loss": 0.9157,
+      "masked_tokens": 108.85,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4731982925441116,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.4731982925441116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5319532209085537,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.3984375,
+      "kd_loss": 0.7658510596184896,
+      "learning_rate": 3e-06,
+      "loss": 0.9988,
+      "masked_tokens": 111.5125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 111.5125
+    },
+    {
+      "avg_mask_ratio": 0.4575169428717345,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4575169428717345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40062239499485486,
+      "epoch": 0.96,
+      "grad_norm": 0.62890625,
+      "kd_loss": 0.8030378437517811,
+      "learning_rate": 3e-06,
+      "loss": 0.9794,
+      "masked_tokens": 107.8875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 107.8875
+    },
+    {
+      "avg_mask_ratio": 0.5099512930959463,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.5099512930959463,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3675635530332329,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.134765625,
+      "kd_loss": 0.6000972521935182,
+      "learning_rate": 3e-06,
+      "loss": 0.8352,
+      "masked_tokens": 109.275,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 109.275
+    },
+    {
+      "avg_mask_ratio": 0.5108432768334058,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.5108432768334058,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4013952974987552,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 1.03125,
+      "kd_loss": 0.8058514126374532,
+      "learning_rate": 3e-06,
+      "loss": 1.06,
+      "masked_tokens": 111.75,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 111.75
+    },
+    {
+      "avg_mask_ratio": 0.49879020540975033,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.49879020540975033,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4040452508418184,
+      "epoch": 1.0256,
+      "grad_norm": 0.64453125,
+      "kd_loss": 0.7641570946838329,
+      "learning_rate": 3e-06,
+      "loss": 0.9387,
+      "masked_tokens": 121.6875,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 121.6875
+    },
+    {
+      "avg_mask_ratio": 0.48607371354009954,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.48607371354009954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44693371437709006,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.8984375,
+      "kd_loss": 0.6808075895191905,
+      "learning_rate": 3e-06,
+      "loss": 0.9264,
+      "masked_tokens": 102.1625,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 102.1625
+    },
+    {
+      "avg_mask_ratio": 0.5385718538891524,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5385718538891524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.445710831214069,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 1.8984375,
+      "kd_loss": 0.7960160556252959,
+      "learning_rate": 3e-06,
+      "loss": 1.0089,
+      "masked_tokens": 127.6125,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.6125
+    },
+    {
+      "avg_mask_ratio": 0.5356179510476068,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5356179510476068,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5134360113543494,
+      "epoch": 1.0896,
+      "grad_norm": 3.484375,
+      "kd_loss": 0.8251110358912228,
+      "learning_rate": 3e-06,
+      "loss": 1.001,
+      "masked_tokens": 136.725,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 136.725
+    },
+    {
+      "avg_mask_ratio": 0.4930020817089826,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4930020817089826,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4553626166405934,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7196989472281075,
+      "learning_rate": 3e-06,
+      "loss": 0.9774,
+      "masked_tokens": 91.975,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 91.975
+    },
+    {
+      "avg_mask_ratio": 0.4998604157241061,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4998604157241061,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5219662474520191,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.95703125,
+      "kd_loss": 0.8503037900029083,
+      "learning_rate": 3e-06,
+      "loss": 1.0856,
+      "masked_tokens": 103.4125,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 103.4125
+    },
+    {
+      "avg_mask_ratio": 0.5236943962518126,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.5236943962518126,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6011495636476297,
+      "epoch": 1.1536,
+      "grad_norm": 0.6171875,
+      "kd_loss": 0.7388030910891757,
+      "learning_rate": 3e-06,
+      "loss": 1.0347,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.4774137590778992,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4774137590778992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33609242954775026,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.419921875,
+      "kd_loss": 0.6285939413004143,
+      "learning_rate": 3e-06,
+      "loss": 0.7996,
+      "masked_tokens": 101.425,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 101.425
+    },
+    {
+      "avg_mask_ratio": 0.41173738130601123,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.41173738130601123,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3657617368780734,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.8828125,
+      "kd_loss": 0.6714434385379491,
+      "learning_rate": 3e-06,
+      "loss": 0.8279,
+      "masked_tokens": 102.0375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 102.0375
+    },
+    {
+      "avg_mask_ratio": 0.4797614786075428,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.4797614786075428,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37769897556100884,
+      "epoch": 1.2176,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.6094748291181077,
+      "learning_rate": 3e-06,
+      "loss": 0.8231,
+      "masked_tokens": 112.25,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 112.25
+    },
+    {
+      "avg_mask_ratio": 0.4974610014585778,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4974610014585778,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46419010059532867,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 1.2265625,
+      "kd_loss": 0.820088501922146,
+      "learning_rate": 3e-06,
+      "loss": 0.9708,
+      "masked_tokens": 134.025,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 134.025
+    },
+    {
+      "avg_mask_ratio": 0.5565119812032208,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5565119812032208,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4556695409415738,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 1.046875,
+      "kd_loss": 0.848517366728629,
+      "learning_rate": 3e-06,
+      "loss": 1.0779,
+      "masked_tokens": 126.1375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 126.1375
+    },
+    {
+      "avg_mask_ratio": 0.4784870075061917,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.4784870075061917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42650491216649017,
+      "epoch": 1.2816,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.7230841763311446,
+      "learning_rate": 3e-06,
+      "loss": 0.983,
+      "masked_tokens": 113.875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 113.875
+    },
+    {
+      "avg_mask_ratio": 0.5459770569577813,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5459770569577813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46574052337223293,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.21484375,
+      "kd_loss": 0.9031681247121014,
+      "learning_rate": 3e-06,
+      "loss": 1.1601,
+      "masked_tokens": 115.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 115.85
+    },
+    {
+      "avg_mask_ratio": 0.5268841385375709,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.5268841385375709,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5097857009053428,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.44140625,
+      "kd_loss": 0.826706444665524,
+      "learning_rate": 3e-06,
+      "loss": 1.0892,
+      "masked_tokens": 114.6625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 114.6625
+    },
+    {
+      "avg_mask_ratio": 0.5629246362368576,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5629246362368576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5826418710530561,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.5703125,
+      "kd_loss": 0.89890192824449,
+      "learning_rate": 3e-06,
+      "loss": 1.3331,
+      "masked_tokens": 130.675,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.675
+    },
+    {
+      "avg_mask_ratio": 0.5119291188195347,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.5119291188195347,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40580563298177597,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.435546875,
+      "kd_loss": 0.6370190013494721,
+      "learning_rate": 3e-06,
+      "loss": 0.8205,
+      "masked_tokens": 125.9,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 125.9
+    },
+    {
+      "avg_mask_ratio": 0.5539714884362184,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5539714884362184,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.694471138650897,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.9244145819217892,
+      "learning_rate": 3e-06,
+      "loss": 1.2334,
+      "masked_tokens": 131.7625,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 131.7625
+    },
+    {
+      "avg_mask_ratio": 0.5141558598377742,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.5141558598377742,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43524807556412953,
+      "epoch": 1.4096,
+      "grad_norm": 2.375,
+      "kd_loss": 0.7787983914435245,
+      "learning_rate": 3e-06,
+      "loss": 1.0634,
+      "masked_tokens": 133.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 133.35
+    },
+    {
+      "avg_mask_ratio": 0.4895282822311856,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.4895282822311856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40460901753227174,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 1.203125,
+      "kd_loss": 0.5940112132494051,
+      "learning_rate": 3e-06,
+      "loss": 0.8149,
+      "masked_tokens": 123.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 123.125
+    },
+    {
+      "avg_mask_ratio": 0.4951617428450845,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4951617428450845,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48473086243019453,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.44140625,
+      "kd_loss": 0.6884326858420409,
+      "learning_rate": 3e-06,
+      "loss": 0.9258,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.5100495176156983,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.5100495176156983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.519521524004017,
+      "epoch": 1.4736,
+      "grad_norm": 0.59375,
+      "kd_loss": 0.7857662321038787,
+      "learning_rate": 3e-06,
+      "loss": 0.9692,
+      "masked_tokens": 115.8875,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 115.8875
+    },
+    {
+      "avg_mask_ratio": 0.5639110118616373,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5639110118616373,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46224736819546025,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.59375,
+      "kd_loss": 1.0577162121335277,
+      "learning_rate": 3e-06,
+      "loss": 1.2682,
+      "masked_tokens": 138.2,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 138.2
+    },
+    {
+      "avg_mask_ratio": 0.5292218026472255,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.5292218026472255,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35752006234570216,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.6908905010689239,
+      "learning_rate": 3e-06,
+      "loss": 0.8571,
+      "masked_tokens": 113.375,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 113.375
+    },
+    {
+      "avg_mask_ratio": 0.5125403102487326,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.5125403102487326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5403474027357873,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 1.1796875,
+      "kd_loss": 0.8581615810285712,
+      "learning_rate": 3e-06,
+      "loss": 1.09,
+      "masked_tokens": 115.675,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 115.675
+    },
+    {
+      "avg_mask_ratio": 0.48811948703369124,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.48811948703369124,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5603859513967677,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.7109375,
+      "kd_loss": 0.7485213522588197,
+      "learning_rate": 3e-06,
+      "loss": 1.0393,
+      "masked_tokens": 106.65,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.5547609420493245,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5547609420493245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6015421481137537,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.9012988628433959,
+      "learning_rate": 3e-06,
+      "loss": 1.226,
+      "masked_tokens": 100.775,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 100.775
+    },
+    {
+      "avg_mask_ratio": 0.44697874613921157,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.44697874613921157,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45085387741235083,
+      "epoch": 1.6016,
+      "grad_norm": 0.76171875,
+      "kd_loss": 0.771520164485878,
+      "learning_rate": 3e-06,
+      "loss": 0.9446,
+      "masked_tokens": 99.5,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 99.5
+    },
+    {
+      "avg_mask_ratio": 0.49905171967693607,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.49905171967693607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5226021331908157,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.76953125,
+      "kd_loss": 0.9288661203041159,
+      "learning_rate": 3e-06,
+      "loss": 1.0794,
+      "masked_tokens": 111.525,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 111.525
+    },
+    {
+      "avg_mask_ratio": 0.4734679562970996,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4734679562970996,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33050077693034724,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.73828125,
+      "kd_loss": 0.6156658631806067,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 124.1625,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 124.1625
+    },
+    {
+      "avg_mask_ratio": 0.45589545626135075,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.45589545626135075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41272709482695974,
+      "epoch": 1.6656,
+      "grad_norm": 0.4765625,
+      "kd_loss": 0.6095967918252938,
+      "learning_rate": 3e-06,
+      "loss": 0.7507,
+      "masked_tokens": 120.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 120.2
+    },
+    {
+      "avg_mask_ratio": 0.4975356309209019,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4975356309209019,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4011998525083527,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.15625,
+      "kd_loss": 0.6194601121176675,
+      "learning_rate": 3e-06,
+      "loss": 0.8021,
+      "masked_tokens": 107.35,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 107.35
+    },
+    {
+      "avg_mask_ratio": 0.4948011673986912,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.4948011673986912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3284698034103485,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.5971616579688088,
+      "learning_rate": 3e-06,
+      "loss": 0.8092,
+      "masked_tokens": 109.1875,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 109.1875
+    },
+    {
+      "avg_mask_ratio": 0.5321399106411263,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5321399106411263,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5248136481198913,
+      "epoch": 1.7296,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.7927273895948019,
+      "learning_rate": 3e-06,
+      "loss": 1.0943,
+      "masked_tokens": 123.0375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 123.0375
+    },
+    {
+      "avg_mask_ratio": 0.5357416228158399,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5357416228158399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5000895128354841,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.859375,
+      "kd_loss": 0.9356607880370575,
+      "learning_rate": 3e-06,
+      "loss": 1.1976,
+      "masked_tokens": 121.5625,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 121.5625
+    },
+    {
+      "avg_mask_ratio": 0.5232944375369698,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.5232944375369698,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48456703309973365,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.8498503854701539,
+      "learning_rate": 3e-06,
+      "loss": 1.0467,
+      "masked_tokens": 138.675,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 138.675
+    },
+    {
+      "avg_mask_ratio": 0.5344608084415086,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5344608084415086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39900637990784843,
+      "epoch": 1.7936,
+      "grad_norm": 0.1962890625,
+      "kd_loss": 0.6959655691830562,
+      "learning_rate": 3e-06,
+      "loss": 0.8985,
+      "masked_tokens": 119.225,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 119.225
+    },
+    {
+      "avg_mask_ratio": 0.5352845921181142,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5352845921181142,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3846706166316153,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.458984375,
+      "kd_loss": 0.6893469515551714,
+      "learning_rate": 3e-06,
+      "loss": 0.8883,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4979630701942369,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4979630701942369,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.49622775785310863,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.73828125,
+      "kd_loss": 0.784965463258402,
+      "learning_rate": 3e-06,
+      "loss": 0.964,
+      "masked_tokens": 111.275,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 111.275
+    },
+    {
+      "avg_mask_ratio": 0.5208624298567883,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.5208624298567883,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3778860895960065,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.609375,
+      "kd_loss": 0.7243039658023435,
+      "learning_rate": 3e-06,
+      "loss": 1.0455,
+      "masked_tokens": 119.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 119.8875
+    },
+    {
+      "avg_mask_ratio": 0.4884064760175534,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.4884064760175534,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3462603269857141,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.7865955847492956,
+      "learning_rate": 3e-06,
+      "loss": 0.9653,
+      "masked_tokens": 97.0,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 97.0
+    },
+    {
+      "avg_mask_ratio": 0.47601241993543225,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.47601241993543225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2950649654762401,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.1845703125,
+      "kd_loss": 0.5946491838043585,
+      "learning_rate": 3e-06,
+      "loss": 0.6996,
+      "masked_tokens": 107.1375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 107.1375
+    },
+    {
+      "avg_mask_ratio": 0.4820589871611446,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4820589871611446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41851851929281453,
+      "epoch": 1.9216,
+      "grad_norm": 0.67578125,
+      "kd_loss": 0.7024738637371911,
+      "learning_rate": 3e-06,
+      "loss": 0.9338,
+      "masked_tokens": 106.675,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 106.675
+    },
+    {
+      "avg_mask_ratio": 0.5009820312960074,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.5009820312960074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44660618857540724,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.447265625,
+      "kd_loss": 0.6575563041935993,
+      "learning_rate": 3e-06,
+      "loss": 0.8679,
+      "masked_tokens": 129.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 129.1625
+    },
+    {
+      "avg_mask_ratio": 0.4952817424898967,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4952817424898967,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4072961182277595,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 1.65625,
+      "kd_loss": 0.773787010011074,
+      "learning_rate": 3e-06,
+      "loss": 0.9519,
+      "masked_tokens": 114.2625,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 114.2625
+    },
+    {
+      "avg_mask_ratio": 0.5025755434762686,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.5025755434762686,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44203572303481453,
+      "epoch": 1.9856,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.6455665581320773,
+      "learning_rate": 3e-06,
+      "loss": 0.8321,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5328231096001608,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5328231096001608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34336739452088033,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.7452835773230098,
+      "learning_rate": 3e-06,
+      "loss": 1.0129,
+      "masked_tokens": 126.51190476190476,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 126.51190476190476
+    },
+    {
+      "avg_mask_ratio": 0.46634063599049114,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.46634063599049114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.345527906726322,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 1.8203125,
+      "kd_loss": 0.6856312883097416,
+      "learning_rate": 3e-06,
+      "loss": 0.8718,
+      "masked_tokens": 111.15,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 111.15
+    },
+    {
+      "avg_mask_ratio": 0.5202614731155336,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.5202614731155336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4029362733661742,
+      "epoch": 2.0512,
+      "grad_norm": 0.404296875,
+      "kd_loss": 0.8637022192546169,
+      "learning_rate": 3e-06,
+      "loss": 1.0614,
+      "masked_tokens": 146.275,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 146.275
+    },
+    {
+      "avg_mask_ratio": 0.4732307325524744,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4732307325524744,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41734947142567763,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 2.015625,
+      "kd_loss": 0.6341307566849423,
+      "learning_rate": 3e-06,
+      "loss": 0.8378,
+      "masked_tokens": 111.6375,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 111.6375
+    },
+    {
+      "avg_mask_ratio": 0.45015103057958183,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.45015103057958183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2503517944936732,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.546875,
+      "kd_loss": 0.5644539449379409,
+      "learning_rate": 3e-06,
+      "loss": 0.7301,
+      "masked_tokens": 102.2875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 102.2875
+    },
+    {
+      "avg_mask_ratio": 0.48529006402241065,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.48529006402241065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24893513410114565,
+      "epoch": 2.1152,
+      "grad_norm": 0.345703125,
+      "kd_loss": 0.5718885382049848,
+      "learning_rate": 3e-06,
+      "loss": 0.6848,
+      "masked_tokens": 123.075,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 123.075
+    },
+    {
+      "avg_mask_ratio": 0.4696127205621451,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4696127205621451,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35570654946394314,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.5947819571083528,
+      "learning_rate": 3e-06,
+      "loss": 0.7695,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 103.0875
+    },
+    {
+      "avg_mask_ratio": 0.46368037317879496,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.46368037317879496,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33185927524032194,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.267578125,
+      "kd_loss": 0.6457533754415123,
+      "learning_rate": 3e-06,
+      "loss": 0.8234,
+      "masked_tokens": 93.1375,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 93.1375
+    },
+    {
+      "avg_mask_ratio": 0.5379365492146462,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5379365492146462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45867338509913225,
+      "epoch": 2.1792,
+      "grad_norm": 0.55859375,
+      "kd_loss": 0.8188646811875515,
+      "learning_rate": 3e-06,
+      "loss": 1.0556,
+      "masked_tokens": 114.975,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 114.975
+    },
+    {
+      "avg_mask_ratio": 0.5036081655998714,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.5036081655998714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4625989968056842,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 1.6484375,
+      "kd_loss": 0.8334748067945263,
+      "learning_rate": 3e-06,
+      "loss": 1.039,
+      "masked_tokens": 109.9125,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 109.9125
+    },
+    {
+      "avg_mask_ratio": 0.529415801318828,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.529415801318828,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3988730081591484,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.7416239527323342,
+      "learning_rate": 3e-06,
+      "loss": 0.912,
+      "masked_tokens": 104.3125,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 104.3125
+    },
+    {
+      "avg_mask_ratio": 0.5512922222726047,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.5512922222726047,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4180156662756417,
+      "epoch": 2.2432,
+      "grad_norm": 0.625,
+      "kd_loss": 0.8845789112904413,
+      "learning_rate": 3e-06,
+      "loss": 1.0177,
+      "masked_tokens": 127.425,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 127.425
+    },
+    {
+      "avg_mask_ratio": 0.480971388152102,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.480971388152102,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35645183491433274,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.5820196808959907,
+      "learning_rate": 3e-06,
+      "loss": 0.7404,
+      "masked_tokens": 125.65,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 125.65
+    },
+    {
+      "avg_mask_ratio": 0.5030692228931002,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.5030692228931002,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38549644878142997,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.2734375,
+      "kd_loss": 0.6196052623042988,
+      "learning_rate": 3e-06,
+      "loss": 0.8827,
+      "masked_tokens": 139.2,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 139.2
+    },
+    {
+      "avg_mask_ratio": 0.4997857674607076,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4997857674607076,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25885673743827625,
+      "epoch": 2.3072,
+      "grad_norm": 0.1513671875,
+      "kd_loss": 0.5832488962907576,
+      "learning_rate": 3e-06,
+      "loss": 0.7719,
+      "masked_tokens": 102.5125,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 102.5125
+    },
+    {
+      "avg_mask_ratio": 0.4668914210633375,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.4668914210633375,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2831251597374546,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.6004543000809491,
+      "learning_rate": 3e-06,
+      "loss": 0.7469,
+      "masked_tokens": 94.85,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 94.85
+    },
+    {
+      "avg_mask_ratio": 0.561556038632989,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.561556038632989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5443290839097472,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.7766849096638907,
+      "learning_rate": 3e-06,
+      "loss": 1.1417,
+      "masked_tokens": 139.1375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 139.1375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04b6dba924441a3d6deb607920bd9c5c280462edbaacc20eb1bdf853287ddf3d
+size 8056

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d56b4b2f8859a6d27166222b99bc5d43356a19a06d20d68d38db6ddd7b648a8
+size 2406624648

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9efc7956d3377209e1ba5b0978d9d777ce6cb946e9a757e86e81e199afe05188
+size 671304442

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbbd0e8c1bfbd7ba8c634ca07b1d8702578d8a5068f2536ae69c20a51bf959b7
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:76f6abf5ed464ad05ce07fc3eaa3005f1e7bc064355635524d65b9082829c58d
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f8c95a6d9085dfcee1e6620c88ede526366d3a02c5018932b1bc04809c0e0c7
+size 1064

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/trainer_state.json

@@ -0,0 +1,2913 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5632,
+  "eval_steps": 500,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.5237232760176994,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.5237232760176994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7671197377738735,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.8686907805610303,
+      "learning_rate": 3e-06,
+      "loss": 1.2408,
+      "masked_tokens": 116.45,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 116.45
+    },
+    {
+      "avg_mask_ratio": 0.44560358227463437,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.44560358227463437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5344198682101251,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 1.1484375,
+      "kd_loss": 0.7096576771870104,
+      "learning_rate": 3e-06,
+      "loss": 0.9455,
+      "masked_tokens": 98.5375,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 98.5375
+    },
+    {
+      "avg_mask_ratio": 0.4828839812951628,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4828839812951628,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5362298497777374,
+      "epoch": 0.064,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.778877005496804,
+      "learning_rate": 3e-06,
+      "loss": 0.9451,
+      "masked_tokens": 115.35,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 115.35
+    },
+    {
+      "avg_mask_ratio": 0.4496018341596937,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.4496018341596937,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4614376229008258,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.84375,
+      "kd_loss": 0.6962691646146141,
+      "learning_rate": 3e-06,
+      "loss": 0.8619,
+      "masked_tokens": 98.025,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.025
+    },
+    {
+      "avg_mask_ratio": 0.46073982657690066,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.46073982657690066,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.614507899929265,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.5959198616897993,
+      "learning_rate": 3e-06,
+      "loss": 0.9459,
+      "masked_tokens": 89.0125,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 89.0125
+    },
+    {
+      "avg_mask_ratio": 0.4842382468283176,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4842382468283176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6723507625403272,
+      "epoch": 0.128,
+      "grad_norm": 0.66015625,
+      "kd_loss": 0.7275705483960166,
+      "learning_rate": 3e-06,
+      "loss": 1.143,
+      "masked_tokens": 122.8875,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 122.8875
+    },
+    {
+      "avg_mask_ratio": 0.5495844878954813,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5495844878954813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6910149530180434,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 1.4765625,
+      "kd_loss": 0.7948297057602758,
+      "learning_rate": 3e-06,
+      "loss": 1.2612,
+      "masked_tokens": 110.0,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 110.0
+    },
+    {
+      "avg_mask_ratio": 0.40544593064114454,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.40544593064114454,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5694220800869061,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.333984375,
+      "kd_loss": 0.5803848952520638,
+      "learning_rate": 3e-06,
+      "loss": 0.8156,
+      "masked_tokens": 90.1875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 90.1875
+    },
+    {
+      "avg_mask_ratio": 0.5312973088817671,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.5312973088817671,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9436774675735251,
+      "epoch": 0.192,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.9708034214691906,
+      "learning_rate": 3e-06,
+      "loss": 1.3507,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4958431267237756,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.4958431267237756,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5302744172568055,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.74609375,
+      "kd_loss": 0.7968542006539338,
+      "learning_rate": 3e-06,
+      "loss": 1.1755,
+      "masked_tokens": 109.0375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 109.0375
+    },
+    {
+      "avg_mask_ratio": 0.5232905174256303,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.5232905174256303,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5488719139095337,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 1.0,
+      "kd_loss": 0.8146776424391475,
+      "learning_rate": 3e-06,
+      "loss": 1.1451,
+      "masked_tokens": 106.4375,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 106.4375
+    },
+    {
+      "avg_mask_ratio": 0.4815562474541366,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4815562474541366,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5119639005151612,
+      "epoch": 0.256,
+      "grad_norm": 7.6875,
+      "kd_loss": 0.7391058675566455,
+      "learning_rate": 3e-06,
+      "loss": 0.9956,
+      "masked_tokens": 102.2,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 102.2
+    },
+    {
+      "avg_mask_ratio": 0.47414465841138737,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.47414465841138737,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46758080123779566,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.4977445501957277,
+      "learning_rate": 3e-06,
+      "loss": 0.7473,
+      "masked_tokens": 94.7875,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 94.7875
+    },
+    {
+      "avg_mask_ratio": 0.523321858420968,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.523321858420968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9225109454039966,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 1.75,
+      "kd_loss": 0.9224564624854793,
+      "learning_rate": 3e-06,
+      "loss": 1.3273,
+      "masked_tokens": 135.4,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 135.4
+    },
+    {
+      "avg_mask_ratio": 0.4975809322553687,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4975809322553687,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6314841133786103,
+      "epoch": 0.32,
+      "grad_norm": 0.09375,
+      "kd_loss": 0.802451879998506,
+      "learning_rate": 3e-06,
+      "loss": 1.1868,
+      "masked_tokens": 129.925,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 129.925
+    },
+    {
+      "avg_mask_ratio": 0.5385947977076284,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5385947977076284,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.9218708202128709,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.8715213164375939,
+      "learning_rate": 3e-06,
+      "loss": 1.2067,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.5177937666652724,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.5177937666652724,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7012445787927846,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.94140625,
+      "kd_loss": 0.7625857894104684,
+      "learning_rate": 3e-06,
+      "loss": 1.0771,
+      "masked_tokens": 93.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 93.225
+    },
+    {
+      "avg_mask_ratio": 0.4772969324782025,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.4772969324782025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6828591173752898,
+      "epoch": 0.384,
+      "grad_norm": 0.69921875,
+      "kd_loss": 0.6958191808335584,
+      "learning_rate": 3e-06,
+      "loss": 1.0206,
+      "masked_tokens": 108.8375,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 108.8375
+    },
+    {
+      "avg_mask_ratio": 0.5173690344206989,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.5173690344206989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6138432722670132,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.265625,
+      "kd_loss": 0.7333374981938505,
+      "learning_rate": 3e-06,
+      "loss": 1.0175,
+      "masked_tokens": 114.0625,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 114.0625
+    },
+    {
+      "avg_mask_ratio": 0.49981915440876035,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.49981915440876035,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5009475202074555,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.39453125,
+      "kd_loss": 0.6001196937293571,
+      "learning_rate": 3e-06,
+      "loss": 0.8454,
+      "masked_tokens": 101.175,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.484982778178528,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.484982778178528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4791799169369824,
+      "epoch": 0.448,
+      "grad_norm": 0.953125,
+      "kd_loss": 0.5891184500089366,
+      "learning_rate": 3e-06,
+      "loss": 0.8327,
+      "masked_tokens": 101.2,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.5744095016038046,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5744095016038046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7536524894140711,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.9296875,
+      "kd_loss": 0.9245879702670209,
+      "learning_rate": 3e-06,
+      "loss": 1.3423,
+      "masked_tokens": 129.4,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 129.4
+    },
+    {
+      "avg_mask_ratio": 0.4629370831884444,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4629370831884444,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3100870553826326,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.6333749431331853,
+      "learning_rate": 3e-06,
+      "loss": 0.8768,
+      "masked_tokens": 110.5125,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 110.5125
+    },
+    {
+      "avg_mask_ratio": 0.499816512214602,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.499816512214602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44889634368061593,
+      "epoch": 0.512,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.6445640347630445,
+      "learning_rate": 3e-06,
+      "loss": 0.9596,
+      "masked_tokens": 110.075,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.075
+    },
+    {
+      "avg_mask_ratio": 0.4744578254292719,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4744578254292719,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39997816555569443,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.19140625,
+      "kd_loss": 0.5854355251746852,
+      "learning_rate": 3e-06,
+      "loss": 0.8236,
+      "masked_tokens": 117.1125,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 117.1125
+    },
+    {
+      "avg_mask_ratio": 0.4852474880579393,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.4852474880579393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34563268155263815,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 4.8125,
+      "kd_loss": 0.5606092717916908,
+      "learning_rate": 3e-06,
+      "loss": 0.7208,
+      "masked_tokens": 113.725,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 113.725
+    },
+    {
+      "avg_mask_ratio": 0.565397203550674,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.565397203550674,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6026960281743186,
+      "epoch": 0.576,
+      "grad_norm": 1.0078125,
+      "kd_loss": 0.8927684382426377,
+      "learning_rate": 3e-06,
+      "loss": 1.2617,
+      "masked_tokens": 124.7125,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 124.7125
+    },
+    {
+      "avg_mask_ratio": 0.4814051762456074,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4814051762456074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4806147089428293,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 6.65625,
+      "kd_loss": 0.6031759152804284,
+      "learning_rate": 3e-06,
+      "loss": 0.8716,
+      "masked_tokens": 129.975,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 129.975
+    },
+    {
+      "avg_mask_ratio": 0.4164489531540312,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.4164489531540312,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1550224335986968,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.0869140625,
+      "kd_loss": 0.4830638362604759,
+      "learning_rate": 3e-06,
+      "loss": 0.5862,
+      "masked_tokens": 100.625,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.625
+    },
+    {
+      "avg_mask_ratio": 0.47973727830685675,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.47973727830685675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4442484440705357,
+      "epoch": 0.64,
+      "grad_norm": 1.140625,
+      "kd_loss": 0.7006052142764929,
+      "learning_rate": 3e-06,
+      "loss": 0.9131,
+      "masked_tokens": 107.2375,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 107.2375
+    },
+    {
+      "avg_mask_ratio": 0.514206234831363,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.514206234831363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5049073612585289,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.51171875,
+      "kd_loss": 0.7227865120981732,
+      "learning_rate": 3e-06,
+      "loss": 1.0107,
+      "masked_tokens": 88.925,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 88.925
+    },
+    {
+      "avg_mask_ratio": 0.5238390378654003,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.5238390378654003,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4860030581583942,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.8063735463714693,
+      "learning_rate": 3e-06,
+      "loss": 1.1637,
+      "masked_tokens": 123.25,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 123.25
+    },
+    {
+      "avg_mask_ratio": 0.5409158666618168,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.5409158666618168,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45924132662039485,
+      "epoch": 0.704,
+      "grad_norm": 0.58203125,
+      "kd_loss": 0.7391011167788519,
+      "learning_rate": 3e-06,
+      "loss": 1.0546,
+      "masked_tokens": 132.2625,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 132.2625
+    },
+    {
+      "avg_mask_ratio": 0.47903697268920953,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.47903697268920953,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5926188694903601,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 1.359375,
+      "kd_loss": 0.8297885791466342,
+      "learning_rate": 3e-06,
+      "loss": 1.0715,
+      "masked_tokens": 114.6375,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 114.6375
+    },
+    {
+      "avg_mask_ratio": 0.5254506973840762,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.5254506973840762,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6182753879609549,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 1.203125,
+      "kd_loss": 0.8253819732506245,
+      "learning_rate": 3e-06,
+      "loss": 1.1773,
+      "masked_tokens": 129.7,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 129.7
+    },
+    {
+      "avg_mask_ratio": 0.5038800648180768,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.5038800648180768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3779912759518879,
+      "epoch": 0.768,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.8277858792208462,
+      "learning_rate": 3e-06,
+      "loss": 0.9585,
+      "masked_tokens": 118.8375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 118.8375
+    },
+    {
+      "avg_mask_ratio": 0.5092529703164473,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.5092529703164473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5031921155097961,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.7001321792347881,
+      "learning_rate": 3e-06,
+      "loss": 0.923,
+      "masked_tokens": 130.4375,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 130.4375
+    },
+    {
+      "avg_mask_ratio": 0.47521690553985535,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.47521690553985535,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3017320279206615,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.6370899313044902,
+      "learning_rate": 3e-06,
+      "loss": 0.8137,
+      "masked_tokens": 99.7125,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 99.7125
+    },
+    {
+      "avg_mask_ratio": 0.5089340912294574,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.5089340912294574,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43493460873024786,
+      "epoch": 0.832,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.7282625613909545,
+      "learning_rate": 3e-06,
+      "loss": 1.0052,
+      "masked_tokens": 115.925,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 115.925
+    },
+    {
+      "avg_mask_ratio": 0.5041010878514498,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.5041010878514498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5107963937724207,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.7805601076866878,
+      "learning_rate": 3e-06,
+      "loss": 1.0557,
+      "masked_tokens": 124.875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 124.875
+    },
+    {
+      "avg_mask_ratio": 0.5127229066158179,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.5127229066158179,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.7406563252751311,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.625,
+      "kd_loss": 0.9257289324105245,
+      "learning_rate": 3e-06,
+      "loss": 1.1941,
+      "masked_tokens": 123.575,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 123.575
+    },
+    {
+      "avg_mask_ratio": 0.47257317856419834,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.47257317856419834,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2641133719835068,
+      "epoch": 0.896,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.5586602845531161,
+      "learning_rate": 3e-06,
+      "loss": 0.6794,
+      "masked_tokens": 90.175,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 90.175
+    },
+    {
+      "avg_mask_ratio": 0.49090774822980165,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.49090774822980165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4790991306209548,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.484375,
+      "kd_loss": 0.6454372880304617,
+      "learning_rate": 3e-06,
+      "loss": 0.9157,
+      "masked_tokens": 108.85,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4731982925441116,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.4731982925441116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5319532209085537,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.3984375,
+      "kd_loss": 0.7658510596184896,
+      "learning_rate": 3e-06,
+      "loss": 0.9988,
+      "masked_tokens": 111.5125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 111.5125
+    },
+    {
+      "avg_mask_ratio": 0.4575169428717345,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4575169428717345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40062239499485486,
+      "epoch": 0.96,
+      "grad_norm": 0.62890625,
+      "kd_loss": 0.8030378437517811,
+      "learning_rate": 3e-06,
+      "loss": 0.9794,
+      "masked_tokens": 107.8875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 107.8875
+    },
+    {
+      "avg_mask_ratio": 0.5099512930959463,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.5099512930959463,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3675635530332329,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.134765625,
+      "kd_loss": 0.6000972521935182,
+      "learning_rate": 3e-06,
+      "loss": 0.8352,
+      "masked_tokens": 109.275,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 109.275
+    },
+    {
+      "avg_mask_ratio": 0.5108432768334058,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.5108432768334058,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4013952974987552,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 1.03125,
+      "kd_loss": 0.8058514126374532,
+      "learning_rate": 3e-06,
+      "loss": 1.06,
+      "masked_tokens": 111.75,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 111.75
+    },
+    {
+      "avg_mask_ratio": 0.49879020540975033,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.49879020540975033,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4040452508418184,
+      "epoch": 1.0256,
+      "grad_norm": 0.64453125,
+      "kd_loss": 0.7641570946838329,
+      "learning_rate": 3e-06,
+      "loss": 0.9387,
+      "masked_tokens": 121.6875,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 121.6875
+    },
+    {
+      "avg_mask_ratio": 0.48607371354009954,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.48607371354009954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44693371437709006,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.8984375,
+      "kd_loss": 0.6808075895191905,
+      "learning_rate": 3e-06,
+      "loss": 0.9264,
+      "masked_tokens": 102.1625,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 102.1625
+    },
+    {
+      "avg_mask_ratio": 0.5385718538891524,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5385718538891524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.445710831214069,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 1.8984375,
+      "kd_loss": 0.7960160556252959,
+      "learning_rate": 3e-06,
+      "loss": 1.0089,
+      "masked_tokens": 127.6125,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.6125
+    },
+    {
+      "avg_mask_ratio": 0.5356179510476068,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5356179510476068,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5134360113543494,
+      "epoch": 1.0896,
+      "grad_norm": 3.484375,
+      "kd_loss": 0.8251110358912228,
+      "learning_rate": 3e-06,
+      "loss": 1.001,
+      "masked_tokens": 136.725,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 136.725
+    },
+    {
+      "avg_mask_ratio": 0.4930020817089826,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4930020817089826,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4553626166405934,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.7196989472281075,
+      "learning_rate": 3e-06,
+      "loss": 0.9774,
+      "masked_tokens": 91.975,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 91.975
+    },
+    {
+      "avg_mask_ratio": 0.4998604157241061,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4998604157241061,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5219662474520191,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.95703125,
+      "kd_loss": 0.8503037900029083,
+      "learning_rate": 3e-06,
+      "loss": 1.0856,
+      "masked_tokens": 103.4125,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 103.4125
+    },
+    {
+      "avg_mask_ratio": 0.5236943962518126,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.5236943962518126,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6011495636476297,
+      "epoch": 1.1536,
+      "grad_norm": 0.6171875,
+      "kd_loss": 0.7388030910891757,
+      "learning_rate": 3e-06,
+      "loss": 1.0347,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.4774137590778992,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4774137590778992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33609242954775026,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.419921875,
+      "kd_loss": 0.6285939413004143,
+      "learning_rate": 3e-06,
+      "loss": 0.7996,
+      "masked_tokens": 101.425,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 101.425
+    },
+    {
+      "avg_mask_ratio": 0.41173738130601123,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.41173738130601123,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3657617368780734,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.8828125,
+      "kd_loss": 0.6714434385379491,
+      "learning_rate": 3e-06,
+      "loss": 0.8279,
+      "masked_tokens": 102.0375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 102.0375
+    },
+    {
+      "avg_mask_ratio": 0.4797614786075428,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.4797614786075428,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37769897556100884,
+      "epoch": 1.2176,
+      "grad_norm": 0.69140625,
+      "kd_loss": 0.6094748291181077,
+      "learning_rate": 3e-06,
+      "loss": 0.8231,
+      "masked_tokens": 112.25,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 112.25
+    },
+    {
+      "avg_mask_ratio": 0.4974610014585778,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4974610014585778,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46419010059532867,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 1.2265625,
+      "kd_loss": 0.820088501922146,
+      "learning_rate": 3e-06,
+      "loss": 0.9708,
+      "masked_tokens": 134.025,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 134.025
+    },
+    {
+      "avg_mask_ratio": 0.5565119812032208,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5565119812032208,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4556695409415738,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 1.046875,
+      "kd_loss": 0.848517366728629,
+      "learning_rate": 3e-06,
+      "loss": 1.0779,
+      "masked_tokens": 126.1375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 126.1375
+    },
+    {
+      "avg_mask_ratio": 0.4784870075061917,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.4784870075061917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42650491216649017,
+      "epoch": 1.2816,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.7230841763311446,
+      "learning_rate": 3e-06,
+      "loss": 0.983,
+      "masked_tokens": 113.875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 113.875
+    },
+    {
+      "avg_mask_ratio": 0.5459770569577813,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5459770569577813,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46574052337223293,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.21484375,
+      "kd_loss": 0.9031681247121014,
+      "learning_rate": 3e-06,
+      "loss": 1.1601,
+      "masked_tokens": 115.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 115.85
+    },
+    {
+      "avg_mask_ratio": 0.5268841385375709,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.5268841385375709,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5097857009053428,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.44140625,
+      "kd_loss": 0.826706444665524,
+      "learning_rate": 3e-06,
+      "loss": 1.0892,
+      "masked_tokens": 114.6625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 114.6625
+    },
+    {
+      "avg_mask_ratio": 0.5629246362368576,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5629246362368576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5826418710530561,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.5703125,
+      "kd_loss": 0.89890192824449,
+      "learning_rate": 3e-06,
+      "loss": 1.3331,
+      "masked_tokens": 130.675,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.675
+    },
+    {
+      "avg_mask_ratio": 0.5119291188195347,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.5119291188195347,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40580563298177597,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.435546875,
+      "kd_loss": 0.6370190013494721,
+      "learning_rate": 3e-06,
+      "loss": 0.8205,
+      "masked_tokens": 125.9,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 125.9
+    },
+    {
+      "avg_mask_ratio": 0.5539714884362184,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5539714884362184,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.694471138650897,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.78125,
+      "kd_loss": 0.9244145819217892,
+      "learning_rate": 3e-06,
+      "loss": 1.2334,
+      "masked_tokens": 131.7625,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 131.7625
+    },
+    {
+      "avg_mask_ratio": 0.5141558598377742,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.5141558598377742,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43524807556412953,
+      "epoch": 1.4096,
+      "grad_norm": 2.375,
+      "kd_loss": 0.7787983914435245,
+      "learning_rate": 3e-06,
+      "loss": 1.0634,
+      "masked_tokens": 133.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 133.35
+    },
+    {
+      "avg_mask_ratio": 0.4895282822311856,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.4895282822311856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40460901753227174,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 1.203125,
+      "kd_loss": 0.5940112132494051,
+      "learning_rate": 3e-06,
+      "loss": 0.8149,
+      "masked_tokens": 123.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 123.125
+    },
+    {
+      "avg_mask_ratio": 0.4951617428450845,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4951617428450845,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48473086243019453,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.44140625,
+      "kd_loss": 0.6884326858420409,
+      "learning_rate": 3e-06,
+      "loss": 0.9258,
+      "masked_tokens": 111.9375,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 111.9375
+    },
+    {
+      "avg_mask_ratio": 0.5100495176156983,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.5100495176156983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.519521524004017,
+      "epoch": 1.4736,
+      "grad_norm": 0.59375,
+      "kd_loss": 0.7857662321038787,
+      "learning_rate": 3e-06,
+      "loss": 0.9692,
+      "masked_tokens": 115.8875,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 115.8875
+    },
+    {
+      "avg_mask_ratio": 0.5639110118616373,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5639110118616373,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46224736819546025,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.59375,
+      "kd_loss": 1.0577162121335277,
+      "learning_rate": 3e-06,
+      "loss": 1.2682,
+      "masked_tokens": 138.2,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 138.2
+    },
+    {
+      "avg_mask_ratio": 0.5292218026472255,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.5292218026472255,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35752006234570216,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.6908905010689239,
+      "learning_rate": 3e-06,
+      "loss": 0.8571,
+      "masked_tokens": 113.375,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 113.375
+    },
+    {
+      "avg_mask_ratio": 0.5125403102487326,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.5125403102487326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5403474027357873,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 1.1796875,
+      "kd_loss": 0.8581615810285712,
+      "learning_rate": 3e-06,
+      "loss": 1.09,
+      "masked_tokens": 115.675,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 115.675
+    },
+    {
+      "avg_mask_ratio": 0.48811948703369124,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.48811948703369124,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5603859513967677,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.7109375,
+      "kd_loss": 0.7485213522588197,
+      "learning_rate": 3e-06,
+      "loss": 1.0393,
+      "masked_tokens": 106.65,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.5547609420493245,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5547609420493245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6015421481137537,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.9012988628433959,
+      "learning_rate": 3e-06,
+      "loss": 1.226,
+      "masked_tokens": 100.775,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 100.775
+    },
+    {
+      "avg_mask_ratio": 0.44697874613921157,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.44697874613921157,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45085387741235083,
+      "epoch": 1.6016,
+      "grad_norm": 0.76171875,
+      "kd_loss": 0.771520164485878,
+      "learning_rate": 3e-06,
+      "loss": 0.9446,
+      "masked_tokens": 99.5,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 99.5
+    },
+    {
+      "avg_mask_ratio": 0.49905171967693607,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.49905171967693607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5226021331908157,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.76953125,
+      "kd_loss": 0.9288661203041159,
+      "learning_rate": 3e-06,
+      "loss": 1.0794,
+      "masked_tokens": 111.525,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 111.525
+    },
+    {
+      "avg_mask_ratio": 0.4734679562970996,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4734679562970996,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33050077693034724,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.73828125,
+      "kd_loss": 0.6156658631806067,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 124.1625,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 124.1625
+    },
+    {
+      "avg_mask_ratio": 0.45589545626135075,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.45589545626135075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41272709482695974,
+      "epoch": 1.6656,
+      "grad_norm": 0.4765625,
+      "kd_loss": 0.6095967918252938,
+      "learning_rate": 3e-06,
+      "loss": 0.7507,
+      "masked_tokens": 120.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 120.2
+    },
+    {
+      "avg_mask_ratio": 0.4975356309209019,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4975356309209019,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4011998525083527,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.15625,
+      "kd_loss": 0.6194601121176675,
+      "learning_rate": 3e-06,
+      "loss": 0.8021,
+      "masked_tokens": 107.35,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 107.35
+    },
+    {
+      "avg_mask_ratio": 0.4948011673986912,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.4948011673986912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3284698034103485,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.5971616579688088,
+      "learning_rate": 3e-06,
+      "loss": 0.8092,
+      "masked_tokens": 109.1875,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 109.1875
+    },
+    {
+      "avg_mask_ratio": 0.5321399106411263,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5321399106411263,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5248136481198913,
+      "epoch": 1.7296,
+      "grad_norm": 0.85546875,
+      "kd_loss": 0.7927273895948019,
+      "learning_rate": 3e-06,
+      "loss": 1.0943,
+      "masked_tokens": 123.0375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 123.0375
+    },
+    {
+      "avg_mask_ratio": 0.5357416228158399,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5357416228158399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5000895128354841,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.859375,
+      "kd_loss": 0.9356607880370575,
+      "learning_rate": 3e-06,
+      "loss": 1.1976,
+      "masked_tokens": 121.5625,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 121.5625
+    },
+    {
+      "avg_mask_ratio": 0.5232944375369698,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.5232944375369698,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.48456703309973365,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 1.171875,
+      "kd_loss": 0.8498503854701539,
+      "learning_rate": 3e-06,
+      "loss": 1.0467,
+      "masked_tokens": 138.675,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 138.675
+    },
+    {
+      "avg_mask_ratio": 0.5344608084415086,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5344608084415086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39900637990784843,
+      "epoch": 1.7936,
+      "grad_norm": 0.1962890625,
+      "kd_loss": 0.6959655691830562,
+      "learning_rate": 3e-06,
+      "loss": 0.8985,
+      "masked_tokens": 119.225,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 119.225
+    },
+    {
+      "avg_mask_ratio": 0.5352845921181142,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5352845921181142,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3846706166316153,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.458984375,
+      "kd_loss": 0.6893469515551714,
+      "learning_rate": 3e-06,
+      "loss": 0.8883,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4979630701942369,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4979630701942369,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.49622775785310863,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.73828125,
+      "kd_loss": 0.784965463258402,
+      "learning_rate": 3e-06,
+      "loss": 0.964,
+      "masked_tokens": 111.275,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 111.275
+    },
+    {
+      "avg_mask_ratio": 0.5208624298567883,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.5208624298567883,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3778860895960065,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.609375,
+      "kd_loss": 0.7243039658023435,
+      "learning_rate": 3e-06,
+      "loss": 1.0455,
+      "masked_tokens": 119.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 119.8875
+    },
+    {
+      "avg_mask_ratio": 0.4884064760175534,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.4884064760175534,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3462603269857141,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.7865955847492956,
+      "learning_rate": 3e-06,
+      "loss": 0.9653,
+      "masked_tokens": 97.0,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 97.0
+    },
+    {
+      "avg_mask_ratio": 0.47601241993543225,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.47601241993543225,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2950649654762401,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.1845703125,
+      "kd_loss": 0.5946491838043585,
+      "learning_rate": 3e-06,
+      "loss": 0.6996,
+      "masked_tokens": 107.1375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 107.1375
+    },
+    {
+      "avg_mask_ratio": 0.4820589871611446,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4820589871611446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41851851929281453,
+      "epoch": 1.9216,
+      "grad_norm": 0.67578125,
+      "kd_loss": 0.7024738637371911,
+      "learning_rate": 3e-06,
+      "loss": 0.9338,
+      "masked_tokens": 106.675,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 106.675
+    },
+    {
+      "avg_mask_ratio": 0.5009820312960074,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.5009820312960074,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44660618857540724,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.447265625,
+      "kd_loss": 0.6575563041935993,
+      "learning_rate": 3e-06,
+      "loss": 0.8679,
+      "masked_tokens": 129.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 129.1625
+    },
+    {
+      "avg_mask_ratio": 0.4952817424898967,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4952817424898967,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4072961182277595,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 1.65625,
+      "kd_loss": 0.773787010011074,
+      "learning_rate": 3e-06,
+      "loss": 0.9519,
+      "masked_tokens": 114.2625,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 114.2625
+    },
+    {
+      "avg_mask_ratio": 0.5025755434762686,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.5025755434762686,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44203572303481453,
+      "epoch": 1.9856,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.6455665581320773,
+      "learning_rate": 3e-06,
+      "loss": 0.8321,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5328231096001608,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5328231096001608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34336739452088033,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.6796875,
+      "kd_loss": 0.7452835773230098,
+      "learning_rate": 3e-06,
+      "loss": 1.0129,
+      "masked_tokens": 126.51190476190476,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 126.51190476190476
+    },
+    {
+      "avg_mask_ratio": 0.46634063599049114,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.46634063599049114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.345527906726322,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 1.8203125,
+      "kd_loss": 0.6856312883097416,
+      "learning_rate": 3e-06,
+      "loss": 0.8718,
+      "masked_tokens": 111.15,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 111.15
+    },
+    {
+      "avg_mask_ratio": 0.5202614731155336,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.5202614731155336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4029362733661742,
+      "epoch": 2.0512,
+      "grad_norm": 0.404296875,
+      "kd_loss": 0.8637022192546169,
+      "learning_rate": 3e-06,
+      "loss": 1.0614,
+      "masked_tokens": 146.275,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 146.275
+    },
+    {
+      "avg_mask_ratio": 0.4732307325524744,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4732307325524744,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.41734947142567763,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 2.015625,
+      "kd_loss": 0.6341307566849423,
+      "learning_rate": 3e-06,
+      "loss": 0.8378,
+      "masked_tokens": 111.6375,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 111.6375
+    },
+    {
+      "avg_mask_ratio": 0.45015103057958183,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.45015103057958183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2503517944936732,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.546875,
+      "kd_loss": 0.5644539449379409,
+      "learning_rate": 3e-06,
+      "loss": 0.7301,
+      "masked_tokens": 102.2875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 102.2875
+    },
+    {
+      "avg_mask_ratio": 0.48529006402241065,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.48529006402241065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24893513410114565,
+      "epoch": 2.1152,
+      "grad_norm": 0.345703125,
+      "kd_loss": 0.5718885382049848,
+      "learning_rate": 3e-06,
+      "loss": 0.6848,
+      "masked_tokens": 123.075,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 123.075
+    },
+    {
+      "avg_mask_ratio": 0.4696127205621451,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4696127205621451,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35570654946394314,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.6640625,
+      "kd_loss": 0.5947819571083528,
+      "learning_rate": 3e-06,
+      "loss": 0.7695,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 103.0875
+    },
+    {
+      "avg_mask_ratio": 0.46368037317879496,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.46368037317879496,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33185927524032194,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.267578125,
+      "kd_loss": 0.6457533754415123,
+      "learning_rate": 3e-06,
+      "loss": 0.8234,
+      "masked_tokens": 93.1375,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 93.1375
+    },
+    {
+      "avg_mask_ratio": 0.5379365492146462,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5379365492146462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.45867338509913225,
+      "epoch": 2.1792,
+      "grad_norm": 0.55859375,
+      "kd_loss": 0.8188646811875515,
+      "learning_rate": 3e-06,
+      "loss": 1.0556,
+      "masked_tokens": 114.975,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 114.975
+    },
+    {
+      "avg_mask_ratio": 0.5036081655998714,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.5036081655998714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4625989968056842,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 1.6484375,
+      "kd_loss": 0.8334748067945263,
+      "learning_rate": 3e-06,
+      "loss": 1.039,
+      "masked_tokens": 109.9125,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 109.9125
+    },
+    {
+      "avg_mask_ratio": 0.529415801318828,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.529415801318828,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3988730081591484,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.7416239527323342,
+      "learning_rate": 3e-06,
+      "loss": 0.912,
+      "masked_tokens": 104.3125,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 104.3125
+    },
+    {
+      "avg_mask_ratio": 0.5512922222726047,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.5512922222726047,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4180156662756417,
+      "epoch": 2.2432,
+      "grad_norm": 0.625,
+      "kd_loss": 0.8845789112904413,
+      "learning_rate": 3e-06,
+      "loss": 1.0177,
+      "masked_tokens": 127.425,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 127.425
+    },
+    {
+      "avg_mask_ratio": 0.480971388152102,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.480971388152102,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35645183491433274,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.5820196808959907,
+      "learning_rate": 3e-06,
+      "loss": 0.7404,
+      "masked_tokens": 125.65,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 125.65
+    },
+    {
+      "avg_mask_ratio": 0.5030692228931002,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.5030692228931002,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38549644878142997,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.2734375,
+      "kd_loss": 0.6196052623042988,
+      "learning_rate": 3e-06,
+      "loss": 0.8827,
+      "masked_tokens": 139.2,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 139.2
+    },
+    {
+      "avg_mask_ratio": 0.4997857674607076,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4997857674607076,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25885673743827625,
+      "epoch": 2.3072,
+      "grad_norm": 0.1513671875,
+      "kd_loss": 0.5832488962907576,
+      "learning_rate": 3e-06,
+      "loss": 0.7719,
+      "masked_tokens": 102.5125,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 102.5125
+    },
+    {
+      "avg_mask_ratio": 0.4668914210633375,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.4668914210633375,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2831251597374546,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.6004543000809491,
+      "learning_rate": 3e-06,
+      "loss": 0.7469,
+      "masked_tokens": 94.85,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 94.85
+    },
+    {
+      "avg_mask_ratio": 0.561556038632989,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.561556038632989,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5443290839097472,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.7766849096638907,
+      "learning_rate": 3e-06,
+      "loss": 1.1417,
+      "masked_tokens": 139.1375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 139.1375
+    },
+    {
+      "avg_mask_ratio": 0.47325256096664814,
+      "avg_response_length": 226.6375,
+      "avg_student_mask_ratio": 0.47325256096664814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3305117641903962,
+      "epoch": 2.3712,
+      "grad_norm": 1.3203125,
+      "kd_loss": 0.4907656863284501,
+      "learning_rate": 3e-06,
+      "loss": 0.7383,
+      "masked_tokens": 107.3,
+      "mean_t": 0.4757364276825683,
+      "step": 1110,
+      "student_masked_tokens": 107.3
+    },
+    {
+      "avg_mask_ratio": 0.5052445781533607,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.5052445781533607,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34653769246153276,
+      "epoch": 2.392533333333333,
+      "grad_norm": 0.38671875,
+      "kd_loss": 0.6271119887123178,
+      "learning_rate": 3e-06,
+      "loss": 0.9278,
+      "masked_tokens": 117.325,
+      "mean_t": 0.5013068238971755,
+      "step": 1120,
+      "student_masked_tokens": 117.325
+    },
+    {
+      "avg_mask_ratio": 0.5352560582570731,
+      "avg_response_length": 262.25,
+      "avg_student_mask_ratio": 0.5352560582570731,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.49558473878678344,
+      "epoch": 2.413866666666667,
+      "grad_norm": 0.97265625,
+      "kd_loss": 0.7805616922649279,
+      "learning_rate": 3e-06,
+      "loss": 1.0163,
+      "masked_tokens": 140.5375,
+      "mean_t": 0.5303254407714121,
+      "step": 1130,
+      "student_masked_tokens": 140.5375
+    },
+    {
+      "avg_mask_ratio": 0.4938803721917793,
+      "avg_response_length": 217.05,
+      "avg_student_mask_ratio": 0.4938803721917793,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3766478347863654,
+      "epoch": 2.4352,
+      "grad_norm": 0.6171875,
+      "kd_loss": 0.5812560225293055,
+      "learning_rate": 3e-06,
+      "loss": 0.7576,
+      "masked_tokens": 107.6125,
+      "mean_t": 0.4845335395424627,
+      "step": 1140,
+      "student_masked_tokens": 107.6125
+    },
+    {
+      "avg_mask_ratio": 0.5653773612109945,
+      "avg_response_length": 212.5375,
+      "avg_student_mask_ratio": 0.5653773612109945,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.79658860497957,
+      "epoch": 2.4565333333333332,
+      "grad_norm": 1.609375,
+      "kd_loss": 0.9014413897515624,
+      "learning_rate": 3e-06,
+      "loss": 1.253,
+      "masked_tokens": 114.6,
+      "mean_t": 0.5690932425903157,
+      "step": 1150,
+      "student_masked_tokens": 114.6
+    },
+    {
+      "avg_mask_ratio": 0.49925965811125933,
+      "avg_response_length": 225.6125,
+      "avg_student_mask_ratio": 0.49925965811125933,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4565864244541672,
+      "epoch": 2.4778666666666664,
+      "grad_norm": 0.98828125,
+      "kd_loss": 0.6585768764591193,
+      "learning_rate": 3e-06,
+      "loss": 0.8381,
+      "masked_tokens": 106.125,
+      "mean_t": 0.5040684466948733,
+      "step": 1160,
+      "student_masked_tokens": 106.125
+    },
+    {
+      "avg_mask_ratio": 0.5130727548676077,
+      "avg_response_length": 247.7625,
+      "avg_student_mask_ratio": 0.5130727548676077,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43352378975719147,
+      "epoch": 2.4992,
+      "grad_norm": 0.36328125,
+      "kd_loss": 0.7219950402087534,
+      "learning_rate": 3e-06,
+      "loss": 0.8737,
+      "masked_tokens": 128.05,
+      "mean_t": 0.5114516971167177,
+      "step": 1170,
+      "student_masked_tokens": 128.05
+    },
+    {
+      "avg_mask_ratio": 0.4515186986711342,
+      "avg_response_length": 214.725,
+      "avg_student_mask_ratio": 0.4515186986711342,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2465131887897769,
+      "epoch": 2.5205333333333333,
+      "grad_norm": 0.30078125,
+      "kd_loss": 0.5771227813067525,
+      "learning_rate": 3e-06,
+      "loss": 0.7516,
+      "masked_tokens": 89.6375,
+      "mean_t": 0.4491677140351385,
+      "step": 1180,
+      "student_masked_tokens": 89.6375
+    },
+    {
+      "avg_mask_ratio": 0.5575610842439346,
+      "avg_response_length": 220.3375,
+      "avg_student_mask_ratio": 0.5575610842439346,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5411728262092481,
+      "epoch": 2.5418666666666665,
+      "grad_norm": 0.93359375,
+      "kd_loss": 1.0418927980632133,
+      "learning_rate": 3e-06,
+      "loss": 1.2353,
+      "masked_tokens": 129.425,
+      "mean_t": 0.5590635397238657,
+      "step": 1190,
+      "student_masked_tokens": 129.425
+    },
+    {
+      "avg_mask_ratio": 0.5073940187954576,
+      "avg_response_length": 215.675,
+      "avg_student_mask_ratio": 0.5073940187954576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 1.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3380157720294562,
+      "epoch": 2.5632,
+      "grad_norm": 0.515625,
+      "kd_loss": 0.6177925667685031,
+      "learning_rate": 3e-06,
+      "loss": 0.8089,
+      "masked_tokens": 103.8375,
+      "mean_t": 0.506370971655997,
+      "step": 1200,
+      "student_masked_tokens": 103.8375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04b6dba924441a3d6deb607920bd9c5c280462edbaacc20eb1bdf853287ddf3d
+size 8056

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "k_proj",
+    "up_proj",
+    "down_proj",
+    "o_proj",
+    "q_proj",
+    "v_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93960a7925e7b725c9e8a456c390b87c8927475c297bf70e6dd72eca5fbbc359
+size 2406624648

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9955268cbe2eebd599c398cfc51d5306bced969d1b8fa6696a68c446298ec271
+size 671304442

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2de0073388a2b514a6f97ca23626cb552641aaab9fcf4308111c0ad94ee7e712
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f34ca8ba860d1ab11666737e7bfb3827b2c85a30d364d9cff93ea785a42427ce
+size 14512

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:368502bb8b2f2d6f452bda7249e88ca57b330ec2ba407ec248613b4300c99d0d
+size 1064

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04b6dba924441a3d6deb607920bd9c5c280462edbaacc20eb1bdf853287ddf3d
+size 8056

math/SFT/inp-onehot_gold1_target1_ce0.5/checkpoint-1400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1