Upload checkpoint from unmask_tags_math_self_distill_INP_u0.001-1.0_gold1_target1_ce0.0

.gitattributes

@@ -42,3 +42,4 @@ math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/debug_training_examples.jso
 math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.0/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/INP-PAR/unmask_tags_gold1_target1_ce0.0/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/INP-OH/unmask_tags_gold1_target1_ce0.0/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
+math/INP/unmask_tags_gold1_target1_ce0.0/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text

math/INP/unmask_tags_gold1_target1_ce0.0/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP/unmask_tags_gold1_target1_ce0.0/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP/unmask_tags_gold1_target1_ce0.0/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e90943f8bc59dea6fbbaf9d3389725f5a577a23ea69a3d881412d5a1cc4cbfc
+size 2406624648

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d11747f5cbf2a4eefe8d043656d723de60d8607ce7b1140b8efc13dd14bb621f
+size 2406624648

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:112817507f1d4e0f5f9dcfdb5c23e12f595d97944016f843df499b8314d3031a
+size 671304442

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6603180b4f7b4f23eb2fefc470a07c3ec6223e2d309190662f43f30d96be9ce5
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:deb2c756bcf75c1e6a028e8158963c3a49cc6f9bf5c39ee87ab65bad3d767bd0
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d8cdfceac9f7917b978dca661a3b8e04187faea5d5f6bd7b462d61d8234d57f
+size 1064

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/trainer_state.json

@@ -0,0 +1,273 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.21333333333333335,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.49314909732202067,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.49314909732202067,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.8524778203239294,
+      "learning_rate": 3e-06,
+      "loss": 0.8897,
+      "masked_tokens": 111.775,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 111.775
+    },
+    {
+      "avg_mask_ratio": 0.42081435878062623,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.42081435878062623,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.6601746417798836,
+      "learning_rate": 3e-06,
+      "loss": 0.6494,
+      "masked_tokens": 94.7,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.7
+    },
+    {
+      "avg_mask_ratio": 0.4538542575784959,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4538542575784959,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.064,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.7305509156625248,
+      "learning_rate": 3e-06,
+      "loss": 0.6735,
+      "masked_tokens": 110.2,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.42071006570186,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.42071006570186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.6237847281259746,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 94.5125,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.5125
+    },
+    {
+      "avg_mask_ratio": 0.43260439952719026,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.43260439952719026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.6509197178371323,
+      "learning_rate": 3e-06,
+      "loss": 0.6726,
+      "masked_tokens": 85.05,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.05
+    },
+    {
+      "avg_mask_ratio": 0.4602369442814961,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4602369442814961,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.128,
+      "grad_norm": 0.1455078125,
+      "kd_loss": 0.6954449985298851,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 119.8625,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.8625
+    },
+    {
+      "avg_mask_ratio": 0.5111442144960165,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5111442144960165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.7338119231904784,
+      "learning_rate": 3e-06,
+      "loss": 0.837,
+      "masked_tokens": 104.325,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.37548826879356056,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.37548826879356056,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.5029199755957052,
+      "learning_rate": 3e-06,
+      "loss": 0.536,
+      "masked_tokens": 85.0625,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 85.0625
+    },
+    {
+      "avg_mask_ratio": 0.4998963106656447,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.4998963106656447,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.192,
+      "grad_norm": 0.3359375,
+      "kd_loss": 0.8495752438985221,
+      "learning_rate": 3e-06,
+      "loss": 0.8143,
+      "masked_tokens": 106.025,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 106.025
+    },
+    {
+      "avg_mask_ratio": 0.457616981890169,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.457616981890169,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.6096036813853857,
+      "learning_rate": 3e-06,
+      "loss": 0.6649,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 103.3875
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2611bced1337375831cc344da28d849ea16c0f2f5a6aeca004865b84ca7216e0
+size 7992

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5a08c264601aa973660886c640fa4d706960dc441eaf09bca6ce825e60d0766d
+size 2406624648

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:133c5ce74e93a4222ed19bb64e5001b2f46084f5909f5136f43ee6030315403c
+size 671304442

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35f0d8cdba91c0873e4ce9bf07f955fed9abee001f34f2ac1f984f19666a371b
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9b22b8ef77ceca4de67915ad7623dbd0d1bebe55b72c5fc618cd9e1f83bdcfc6
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29d9aa99505fc60c0db1b9cdacaa08b06e8a85c8aaaab4e389667a719fafb9bf
+size 1064

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/trainer_state.json

@@ -0,0 +1,2433 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.1365333333333334,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.49314909732202067,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.49314909732202067,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.8524778203239294,
+      "learning_rate": 3e-06,
+      "loss": 0.8897,
+      "masked_tokens": 111.775,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 111.775
+    },
+    {
+      "avg_mask_ratio": 0.42081435878062623,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.42081435878062623,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.6601746417798836,
+      "learning_rate": 3e-06,
+      "loss": 0.6494,
+      "masked_tokens": 94.7,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.7
+    },
+    {
+      "avg_mask_ratio": 0.4538542575784959,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4538542575784959,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.064,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.7305509156625248,
+      "learning_rate": 3e-06,
+      "loss": 0.6735,
+      "masked_tokens": 110.2,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.42071006570186,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.42071006570186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.6237847281259746,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 94.5125,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.5125
+    },
+    {
+      "avg_mask_ratio": 0.43260439952719026,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.43260439952719026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.6509197178371323,
+      "learning_rate": 3e-06,
+      "loss": 0.6726,
+      "masked_tokens": 85.05,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.05
+    },
+    {
+      "avg_mask_ratio": 0.4602369442814961,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4602369442814961,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.128,
+      "grad_norm": 0.1455078125,
+      "kd_loss": 0.6954449985298851,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 119.8625,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.8625
+    },
+    {
+      "avg_mask_ratio": 0.5111442144960165,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5111442144960165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.7338119231904784,
+      "learning_rate": 3e-06,
+      "loss": 0.837,
+      "masked_tokens": 104.325,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.37548826879356056,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.37548826879356056,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.5029199755957052,
+      "learning_rate": 3e-06,
+      "loss": 0.536,
+      "masked_tokens": 85.0625,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 85.0625
+    },
+    {
+      "avg_mask_ratio": 0.4998963106656447,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.4998963106656447,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.192,
+      "grad_norm": 0.3359375,
+      "kd_loss": 0.8495752438985221,
+      "learning_rate": 3e-06,
+      "loss": 0.8143,
+      "masked_tokens": 106.025,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 106.025
+    },
+    {
+      "avg_mask_ratio": 0.457616981890169,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.457616981890169,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.6096036813853857,
+      "learning_rate": 3e-06,
+      "loss": 0.6649,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 103.3875
+    },
+    {
+      "avg_mask_ratio": 0.48775083787040785,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.48775083787040785,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.8403033299919912,
+      "learning_rate": 3e-06,
+      "loss": 0.8646,
+      "masked_tokens": 101.725,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 101.725
+    },
+    {
+      "avg_mask_ratio": 0.4505724890856072,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4505724890856072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.256,
+      "grad_norm": 0.2734375,
+      "kd_loss": 0.7449374303294463,
+      "learning_rate": 3e-06,
+      "loss": 0.6985,
+      "masked_tokens": 97.3125,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 97.3125
+    },
+    {
+      "avg_mask_ratio": 0.43574556948733517,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.43574556948733517,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.392578125,
+      "kd_loss": 0.4566404212248244,
+      "learning_rate": 3e-06,
+      "loss": 0.4885,
+      "masked_tokens": 89.325,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 89.325
+    },
+    {
+      "avg_mask_ratio": 0.49070783695206044,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.49070783695206044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.474609375,
+      "kd_loss": 0.6905622516694165,
+      "learning_rate": 3e-06,
+      "loss": 0.7061,
+      "masked_tokens": 130.2625,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 130.2625
+    },
+    {
+      "avg_mask_ratio": 0.4736109555349685,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4736109555349685,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.32,
+      "grad_norm": 0.08056640625,
+      "kd_loss": 0.7106425042155138,
+      "learning_rate": 3e-06,
+      "loss": 0.7643,
+      "masked_tokens": 126.2,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 126.2
+    },
+    {
+      "avg_mask_ratio": 0.5032523009285796,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5032523009285796,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.408203125,
+      "kd_loss": 0.8267615023827602,
+      "learning_rate": 3e-06,
+      "loss": 0.8409,
+      "masked_tokens": 99.425,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 99.425
+    },
+    {
+      "avg_mask_ratio": 0.4807244518771768,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.4807244518771768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.7329186423268765,
+      "learning_rate": 3e-06,
+      "loss": 0.6767,
+      "masked_tokens": 88.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 88.225
+    },
+    {
+      "avg_mask_ratio": 0.45808705932286103,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.45808705932286103,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.384,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5571840626302332,
+      "learning_rate": 3e-06,
+      "loss": 0.5724,
+      "masked_tokens": 105.8625,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 105.8625
+    },
+    {
+      "avg_mask_ratio": 0.4842760307248682,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.4842760307248682,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6595436891710733,
+      "learning_rate": 3e-06,
+      "loss": 0.6415,
+      "masked_tokens": 109.5875,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 109.5875
+    },
+    {
+      "avg_mask_ratio": 0.4665210062637925,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4665210062637925,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.5610411517063085,
+      "learning_rate": 3e-06,
+      "loss": 0.5866,
+      "masked_tokens": 97.0125,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 97.0125
+    },
+    {
+      "avg_mask_ratio": 0.45183838629163803,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.45183838629163803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.448,
+      "grad_norm": 0.29296875,
+      "kd_loss": 0.5228309052588884,
+      "learning_rate": 3e-06,
+      "loss": 0.5326,
+      "masked_tokens": 96.0125,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 96.0125
+    },
+    {
+      "avg_mask_ratio": 0.5396745702251792,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5396745702251792,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.7454893081907358,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 124.0125,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 124.0125
+    },
+    {
+      "avg_mask_ratio": 0.4404038088978268,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4404038088978268,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.5848532651940331,
+      "learning_rate": 3e-06,
+      "loss": 0.6239,
+      "masked_tokens": 107.1875,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 107.1875
+    },
+    {
+      "avg_mask_ratio": 0.47433349161874505,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.47433349161874505,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.512,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.7448974770184804,
+      "learning_rate": 3e-06,
+      "loss": 0.7729,
+      "masked_tokens": 106.275,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 106.275
+    },
+    {
+      "avg_mask_ratio": 0.4465038388385437,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4465038388385437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.1328125,
+      "kd_loss": 0.5912455287105786,
+      "learning_rate": 3e-06,
+      "loss": 0.5836,
+      "masked_tokens": 112.725,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 112.725
+    },
+    {
+      "avg_mask_ratio": 0.45215590375009923,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.45215590375009923,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.2451171875,
+      "kd_loss": 0.5663141592655847,
+      "learning_rate": 3e-06,
+      "loss": 0.531,
+      "masked_tokens": 109.1,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 109.1
+    },
+    {
+      "avg_mask_ratio": 0.5305180630879477,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.5305180630879477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.576,
+      "grad_norm": 0.4453125,
+      "kd_loss": 0.8847639085633545,
+      "learning_rate": 3e-06,
+      "loss": 0.9006,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4644554515252821,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4644554515252821,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.6207698138765181,
+      "learning_rate": 3e-06,
+      "loss": 0.5834,
+      "masked_tokens": 126.2125,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 126.2125
+    },
+    {
+      "avg_mask_ratio": 0.3939544238965027,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.3939544238965027,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.483729143582201,
+      "learning_rate": 3e-06,
+      "loss": 0.471,
+      "masked_tokens": 96.975,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 96.975
+    },
+    {
+      "avg_mask_ratio": 0.4442457370925695,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.4442457370925695,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.64,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.5351485910949123,
+      "learning_rate": 3e-06,
+      "loss": 0.607,
+      "masked_tokens": 101.525,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 101.525
+    },
+    {
+      "avg_mask_ratio": 0.48191855432232844,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.48191855432232844,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.2890625,
+      "kd_loss": 0.7135523370927593,
+      "learning_rate": 3e-06,
+      "loss": 0.695,
+      "masked_tokens": 84.875,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 84.875
+    },
+    {
+      "avg_mask_ratio": 0.49743892522528765,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.49743892522528765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.7042804263782273,
+      "learning_rate": 3e-06,
+      "loss": 0.7858,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.511213600449264,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.511213600449264,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.704,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.6977765330942021,
+      "learning_rate": 3e-06,
+      "loss": 0.7303,
+      "masked_tokens": 128.1,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 128.1
+    },
+    {
+      "avg_mask_ratio": 0.44370225080056114,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.44370225080056114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.4375,
+      "kd_loss": 0.714193849134108,
+      "learning_rate": 3e-06,
+      "loss": 0.6445,
+      "masked_tokens": 108.775,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 108.775
+    },
+    {
+      "avg_mask_ratio": 0.49959173843380994,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.49959173843380994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.7770639997791452,
+      "learning_rate": 3e-06,
+      "loss": 0.7806,
+      "masked_tokens": 125.625,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 125.625
+    },
+    {
+      "avg_mask_ratio": 0.47441557042766364,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.47441557042766364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.768,
+      "grad_norm": 0.15234375,
+      "kd_loss": 0.762115379356667,
+      "learning_rate": 3e-06,
+      "loss": 0.7428,
+      "masked_tokens": 114.3375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 114.3375
+    },
+    {
+      "avg_mask_ratio": 0.4738660019589588,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.4738660019589588,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.0859375,
+      "kd_loss": 0.7563776895753108,
+      "learning_rate": 3e-06,
+      "loss": 0.6971,
+      "masked_tokens": 126.3875,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 126.3875
+    },
+    {
+      "avg_mask_ratio": 0.4487657839199528,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.4487657839199528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.259765625,
+      "kd_loss": 0.5462394427451727,
+      "learning_rate": 3e-06,
+      "loss": 0.4745,
+      "masked_tokens": 96.725,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 96.725
+    },
+    {
+      "avg_mask_ratio": 0.4751023279619403,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.4751023279619403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.832,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.685858413620528,
+      "learning_rate": 3e-06,
+      "loss": 0.6922,
+      "masked_tokens": 110.5625,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 110.5625
+    },
+    {
+      "avg_mask_ratio": 0.4746393185690977,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.4746393185690977,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.7327228688028754,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 120.3875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 120.3875
+    },
+    {
+      "avg_mask_ratio": 0.4831854701798875,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.4831854701798875,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.265625,
+      "kd_loss": 0.8579687503222686,
+      "learning_rate": 3e-06,
+      "loss": 0.7988,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.4379329536575824,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4379329536575824,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.896,
+      "grad_norm": 0.21875,
+      "kd_loss": 0.45460584598010884,
+      "learning_rate": 3e-06,
+      "loss": 0.4832,
+      "masked_tokens": 85.525,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 85.525
+    },
+    {
+      "avg_mask_ratio": 0.4674084897618741,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4674084897618741,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.2216796875,
+      "kd_loss": 0.653534199286878,
+      "learning_rate": 3e-06,
+      "loss": 0.6388,
+      "masked_tokens": 104.5625,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 104.5625
+    },
+    {
+      "avg_mask_ratio": 0.44310826995642855,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.44310826995642855,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.6742433619177973,
+      "learning_rate": 3e-06,
+      "loss": 0.6468,
+      "masked_tokens": 106.1875,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 106.1875
+    },
+    {
+      "avg_mask_ratio": 0.4248724762815982,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4248724762815982,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.96,
+      "grad_norm": 0.25390625,
+      "kd_loss": 0.6201432387717432,
+      "learning_rate": 3e-06,
+      "loss": 0.6546,
+      "masked_tokens": 102.7375,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 102.7375
+    },
+    {
+      "avg_mask_ratio": 0.47578654896933587,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.47578654896933587,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.16015625,
+      "kd_loss": 0.6602298976420571,
+      "learning_rate": 3e-06,
+      "loss": 0.6259,
+      "masked_tokens": 105.3125,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 105.3125
+    },
+    {
+      "avg_mask_ratio": 0.4781565316176663,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.4781565316176663,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.361328125,
+      "kd_loss": 0.7762555406697461,
+      "learning_rate": 3e-06,
+      "loss": 0.7824,
+      "masked_tokens": 106.6547619047619,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 106.6547619047619
+    },
+    {
+      "avg_mask_ratio": 0.4757596247596666,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.4757596247596666,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0256,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7768732401565984,
+      "learning_rate": 3e-06,
+      "loss": 0.7435,
+      "masked_tokens": 118.225,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 118.225
+    },
+    {
+      "avg_mask_ratio": 0.4564065972692333,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.4564065972692333,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.6187377570489844,
+      "learning_rate": 3e-06,
+      "loss": 0.5998,
+      "masked_tokens": 97.6375,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 97.6375
+    },
+    {
+      "avg_mask_ratio": 0.5074894667137414,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5074894667137414,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.712813160381853,
+      "learning_rate": 3e-06,
+      "loss": 0.7089,
+      "masked_tokens": 122.9,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 122.9
+    },
+    {
+      "avg_mask_ratio": 0.5111216343124397,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5111216343124397,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0896,
+      "grad_norm": 0.30078125,
+      "kd_loss": 0.7725131734614138,
+      "learning_rate": 3e-06,
+      "loss": 0.7319,
+      "masked_tokens": 132.4625,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 132.4625
+    },
+    {
+      "avg_mask_ratio": 0.45396183808334173,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.45396183808334173,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.396484375,
+      "kd_loss": 0.6740453685139528,
+      "learning_rate": 3e-06,
+      "loss": 0.6974,
+      "masked_tokens": 87.0625,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 87.0625
+    },
+    {
+      "avg_mask_ratio": 0.4701436776900664,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4701436776900664,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.37890625,
+      "kd_loss": 0.7894933511876161,
+      "learning_rate": 3e-06,
+      "loss": 0.8161,
+      "masked_tokens": 99.2875,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 99.2875
+    },
+    {
+      "avg_mask_ratio": 0.48931564125232396,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.48931564125232396,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1536,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.8131277295872679,
+      "learning_rate": 3e-06,
+      "loss": 0.7558,
+      "masked_tokens": 107.3625,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 107.3625
+    },
+    {
+      "avg_mask_ratio": 0.4541942774085328,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4541942774085328,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.1767578125,
+      "kd_loss": 0.5594845790216368,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 97.8875,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 97.8875
+    },
+    {
+      "avg_mask_ratio": 0.3928302472108044,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.3928302472108044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.6713891216638501,
+      "learning_rate": 3e-06,
+      "loss": 0.6598,
+      "masked_tokens": 99.175,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 99.175
+    },
+    {
+      "avg_mask_ratio": 0.44847587069962175,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.44847587069962175,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2176,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.6333371287628324,
+      "learning_rate": 3e-06,
+      "loss": 0.6153,
+      "masked_tokens": 108.375,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 108.375
+    },
+    {
+      "avg_mask_ratio": 0.4655611292924732,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4655611292924732,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.7399741232418364,
+      "learning_rate": 3e-06,
+      "loss": 0.7193,
+      "masked_tokens": 127.4125,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 127.4125
+    },
+    {
+      "avg_mask_ratio": 0.5226378521183506,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5226378521183506,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.7912517642864259,
+      "learning_rate": 3e-06,
+      "loss": 0.7556,
+      "masked_tokens": 121.5375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 121.5375
+    },
+    {
+      "avg_mask_ratio": 0.45001045021926983,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.45001045021926983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2816,
+      "grad_norm": 0.2333984375,
+      "kd_loss": 0.5968554831010096,
+      "learning_rate": 3e-06,
+      "loss": 0.622,
+      "masked_tokens": 109.9,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 109.9
+    },
+    {
+      "avg_mask_ratio": 0.5040990226087161,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5040990226087161,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.1435546875,
+      "kd_loss": 0.7809002916160352,
+      "learning_rate": 3e-06,
+      "loss": 0.8354,
+      "masked_tokens": 110.375,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 110.375
+    },
+    {
+      "avg_mask_ratio": 0.4963218850665726,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.4963218850665726,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.19921875,
+      "kd_loss": 0.7623143148503232,
+      "learning_rate": 3e-06,
+      "loss": 0.7637,
+      "masked_tokens": 110.675,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 110.675
+    },
+    {
+      "avg_mask_ratio": 0.5259752279380336,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5259752279380336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 0.55859375,
+      "kd_loss": 0.9169201381674611,
+      "learning_rate": 3e-06,
+      "loss": 0.9908,
+      "masked_tokens": 125.525,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 125.525
+    },
+    {
+      "avg_mask_ratio": 0.49199640220031143,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.49199640220031143,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.658431020938042,
+      "learning_rate": 3e-06,
+      "loss": 0.6105,
+      "masked_tokens": 121.8625,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 121.8625
+    },
+    {
+      "avg_mask_ratio": 0.5331356228562072,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5331356228562072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.205078125,
+      "kd_loss": 0.8636023813149223,
+      "learning_rate": 3e-06,
+      "loss": 0.7956,
+      "masked_tokens": 128.65,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 128.65
+    },
+    {
+      "avg_mask_ratio": 0.4887094351579435,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.4887094351579435,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4096,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.7148105578169635,
+      "learning_rate": 3e-06,
+      "loss": 0.7531,
+      "masked_tokens": 129.8,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 129.8
+    },
+    {
+      "avg_mask_ratio": 0.46143888669321315,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.46143888669321315,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.451171875,
+      "kd_loss": 0.6032656382115874,
+      "learning_rate": 3e-06,
+      "loss": 0.6039,
+      "masked_tokens": 118.7375,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 118.7375
+    },
+    {
+      "avg_mask_ratio": 0.46107456521131096,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.46107456521131096,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.314453125,
+      "kd_loss": 0.6940350630834473,
+      "learning_rate": 3e-06,
+      "loss": 0.6841,
+      "masked_tokens": 106.4875,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 106.4875
+    },
+    {
+      "avg_mask_ratio": 0.4799915248062462,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.4799915248062462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4736,
+      "grad_norm": 0.189453125,
+      "kd_loss": 0.6958799545635429,
+      "learning_rate": 3e-06,
+      "loss": 0.684,
+      "masked_tokens": 111.5375,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 111.5375
+    },
+    {
+      "avg_mask_ratio": 0.5273691712645814,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5273691712645814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.9459125808391036,
+      "learning_rate": 3e-06,
+      "loss": 0.8879,
+      "masked_tokens": 132.525,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 132.525
+    },
+    {
+      "avg_mask_ratio": 0.48835311776492746,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.48835311776492746,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.1650390625,
+      "kd_loss": 0.6531627567595478,
+      "learning_rate": 3e-06,
+      "loss": 0.6323,
+      "masked_tokens": 108.125,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 108.125
+    },
+    {
+      "avg_mask_ratio": 0.48280774110462515,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.48280774110462515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.8419079009756387,
+      "learning_rate": 3e-06,
+      "loss": 0.7928,
+      "masked_tokens": 110.2,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.46588142022956164,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.46588142022956164,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.244140625,
+      "kd_loss": 0.6997778600025854,
+      "learning_rate": 3e-06,
+      "loss": 0.7623,
+      "masked_tokens": 103.3375,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 103.3375
+    },
+    {
+      "avg_mask_ratio": 0.5192104739835486,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5192104739835486,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.1708984375,
+      "kd_loss": 0.8950252983398969,
+      "learning_rate": 3e-06,
+      "loss": 0.8392,
+      "masked_tokens": 96.55,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 96.55
+    },
+    {
+      "avg_mask_ratio": 0.41821987080038525,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.41821987080038525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6016,
+      "grad_norm": 0.2099609375,
+      "kd_loss": 0.6870280907321394,
+      "learning_rate": 3e-06,
+      "loss": 0.6255,
+      "masked_tokens": 95.2,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 95.2
+    },
+    {
+      "avg_mask_ratio": 0.4688679567188956,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.4688679567188956,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 1.390625,
+      "kd_loss": 0.8148217237806421,
+      "learning_rate": 3e-06,
+      "loss": 0.7954,
+      "masked_tokens": 107.55,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 107.55
+    },
+    {
+      "avg_mask_ratio": 0.4429742675391026,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4429742675391026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.1826171875,
+      "kd_loss": 0.5454595529316066,
+      "learning_rate": 3e-06,
+      "loss": 0.5506,
+      "masked_tokens": 117.825,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 117.825
+    },
+    {
+      "avg_mask_ratio": 0.43331936674949245,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.43331936674949245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6656,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.4808090373327854,
+      "learning_rate": 3e-06,
+      "loss": 0.5158,
+      "masked_tokens": 117.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 117.2
+    },
+    {
+      "avg_mask_ratio": 0.4646148854692001,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4646148854692001,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.140625,
+      "kd_loss": 0.6250457387332972,
+      "learning_rate": 3e-06,
+      "loss": 0.6097,
+      "masked_tokens": 102.15,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 102.15
+    },
+    {
+      "avg_mask_ratio": 0.46852388482075186,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.46852388482075186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.1552734375,
+      "kd_loss": 0.5451920869360265,
+      "learning_rate": 3e-06,
+      "loss": 0.6197,
+      "masked_tokens": 105.6375,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 105.6375
+    },
+    {
+      "avg_mask_ratio": 0.5052226510597393,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5052226510597393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7296,
+      "grad_norm": 0.27734375,
+      "kd_loss": 0.7552159034145006,
+      "learning_rate": 3e-06,
+      "loss": 0.7562,
+      "masked_tokens": 119.5375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 119.5375
+    },
+    {
+      "avg_mask_ratio": 0.5101147621171549,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5101147621171549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.8518789664900283,
+      "learning_rate": 3e-06,
+      "loss": 0.851,
+      "masked_tokens": 117.525,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 117.525
+    },
+    {
+      "avg_mask_ratio": 0.4939760136883706,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.4939760136883706,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.8243849514963074,
+      "learning_rate": 3e-06,
+      "loss": 0.7816,
+      "masked_tokens": 134.1625,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 134.1625
+    },
+    {
+      "avg_mask_ratio": 0.5015010182803963,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5015010182803963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7936,
+      "grad_norm": 0.1279296875,
+      "kd_loss": 0.6244754713046745,
+      "learning_rate": 3e-06,
+      "loss": 0.6667,
+      "masked_tokens": 114.4875,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 114.4875
+    },
+    {
+      "avg_mask_ratio": 0.5062767470255494,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5062767470255494,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.236328125,
+      "kd_loss": 0.6353196922647157,
+      "learning_rate": 3e-06,
+      "loss": 0.6607,
+      "masked_tokens": 115.7125,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 115.7125
+    },
+    {
+      "avg_mask_ratio": 0.4643575747264549,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4643575747264549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.7141597053345781,
+      "learning_rate": 3e-06,
+      "loss": 0.6556,
+      "masked_tokens": 106.9375,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 106.9375
+    },
+    {
+      "avg_mask_ratio": 0.487393151212018,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.487393151212018,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.6699687715730555,
+      "learning_rate": 3e-06,
+      "loss": 0.787,
+      "masked_tokens": 114.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 114.8875
+    },
+    {
+      "avg_mask_ratio": 0.45142438664333895,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.45142438664333895,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7402904870072688,
+      "learning_rate": 3e-06,
+      "loss": 0.6762,
+      "masked_tokens": 92.25,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 92.25
+    },
+    {
+      "avg_mask_ratio": 0.4500263270718278,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.4500263270718278,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.0947265625,
+      "kd_loss": 0.5816223171608726,
+      "learning_rate": 3e-06,
+      "loss": 0.5512,
+      "masked_tokens": 102.8125,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 102.8125
+    },
+    {
+      "avg_mask_ratio": 0.4509512555785477,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4509512555785477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9216,
+      "grad_norm": 0.234375,
+      "kd_loss": 0.6806100918192441,
+      "learning_rate": 3e-06,
+      "loss": 0.6938,
+      "masked_tokens": 102.025,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 102.025
+    },
+    {
+      "avg_mask_ratio": 0.47468002026434986,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.47468002026434986,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.185546875,
+      "kd_loss": 0.6469848818184118,
+      "learning_rate": 3e-06,
+      "loss": 0.6391,
+      "masked_tokens": 124.3,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 124.3
+    },
+    {
+      "avg_mask_ratio": 0.4673476937925443,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4673476937925443,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.357421875,
+      "kd_loss": 0.7518777136654535,
+      "learning_rate": 3e-06,
+      "loss": 0.7765,
+      "masked_tokens": 109.7875,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 109.7875
+    },
+    {
+      "avg_mask_ratio": 0.4756378314923495,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.4756378314923495,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9856,
+      "grad_norm": 0.17578125,
+      "kd_loss": 0.6069021332950797,
+      "learning_rate": 3e-06,
+      "loss": 0.6412,
+      "masked_tokens": 119.8125,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 119.8125
+    },
+    {
+      "avg_mask_ratio": 0.5000089073021498,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5000089073021498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.7108402555168141,
+      "learning_rate": 3e-06,
+      "loss": 0.7752,
+      "masked_tokens": 121.98809523809524,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 121.98809523809524
+    },
+    {
+      "avg_mask_ratio": 0.4385950181807857,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.4385950181807857,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.2158203125,
+      "kd_loss": 0.652561020819121,
+      "learning_rate": 3e-06,
+      "loss": 0.6515,
+      "masked_tokens": 107.025,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 107.025
+    },
+    {
+      "avg_mask_ratio": 0.49335200865752993,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.49335200865752993,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0512,
+      "grad_norm": 0.2021484375,
+      "kd_loss": 0.8505055773921185,
+      "learning_rate": 3e-06,
+      "loss": 0.8085,
+      "masked_tokens": 141.5,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 141.5
+    },
+    {
+      "avg_mask_ratio": 0.4434558014094364,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4434558014094364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.564091070059294,
+      "learning_rate": 3e-06,
+      "loss": 0.5634,
+      "masked_tokens": 107.3,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 107.3
+    },
+    {
+      "avg_mask_ratio": 0.42445047190412877,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.42445047190412877,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.5745988191337801,
+      "learning_rate": 3e-06,
+      "loss": 0.5537,
+      "masked_tokens": 98.0875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 98.0875
+    },
+    {
+      "avg_mask_ratio": 0.4553717203787528,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.4553717203787528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1152,
+      "grad_norm": 0.14453125,
+      "kd_loss": 0.5502215233629159,
+      "learning_rate": 3e-06,
+      "loss": 0.5637,
+      "masked_tokens": 118.75,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 118.75
+    },
+    {
+      "avg_mask_ratio": 0.4424907588865608,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4424907588865608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5585700995494335,
+      "learning_rate": 3e-06,
+      "loss": 0.5795,
+      "masked_tokens": 98.9125,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 98.9125
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2611bced1337375831cc344da28d849ea16c0f2f5a6aeca004865b84ca7216e0
+size 7992

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0b734b32812998339231bdafaf2ce94d91a3763cd9bdfb08ee8598f3c4063f51
+size 2406624648

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a32b18ae9f3ea0aed6b12b649f6f42dce701f4d73a0a484b0ec6ca849d607cbe
+size 671304442

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e4d951b3681768d0a5bb5b4a429126b0d534a20a49c0499d63f2afab759f4fb3
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:989f865c0724dd4eec83cd129cfa791de296a1df44cc52c29a74928519e0d01c
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25795e3b7374d0f6abdd7ab4b34fbf7ab0447ba73c04014500c2ab8b5acec5b4
+size 1064

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/trainer_state.json

@@ -0,0 +1,2673 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.3498666666666668,
+  "eval_steps": 500,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.49314909732202067,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.49314909732202067,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.8524778203239294,
+      "learning_rate": 3e-06,
+      "loss": 0.8897,
+      "masked_tokens": 111.775,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 111.775
+    },
+    {
+      "avg_mask_ratio": 0.42081435878062623,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.42081435878062623,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.6601746417798836,
+      "learning_rate": 3e-06,
+      "loss": 0.6494,
+      "masked_tokens": 94.7,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.7
+    },
+    {
+      "avg_mask_ratio": 0.4538542575784959,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4538542575784959,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.064,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.7305509156625248,
+      "learning_rate": 3e-06,
+      "loss": 0.6735,
+      "masked_tokens": 110.2,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.42071006570186,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.42071006570186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.6237847281259746,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 94.5125,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.5125
+    },
+    {
+      "avg_mask_ratio": 0.43260439952719026,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.43260439952719026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.6509197178371323,
+      "learning_rate": 3e-06,
+      "loss": 0.6726,
+      "masked_tokens": 85.05,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.05
+    },
+    {
+      "avg_mask_ratio": 0.4602369442814961,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4602369442814961,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.128,
+      "grad_norm": 0.1455078125,
+      "kd_loss": 0.6954449985298851,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 119.8625,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.8625
+    },
+    {
+      "avg_mask_ratio": 0.5111442144960165,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5111442144960165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.7338119231904784,
+      "learning_rate": 3e-06,
+      "loss": 0.837,
+      "masked_tokens": 104.325,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.37548826879356056,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.37548826879356056,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.5029199755957052,
+      "learning_rate": 3e-06,
+      "loss": 0.536,
+      "masked_tokens": 85.0625,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 85.0625
+    },
+    {
+      "avg_mask_ratio": 0.4998963106656447,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.4998963106656447,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.192,
+      "grad_norm": 0.3359375,
+      "kd_loss": 0.8495752438985221,
+      "learning_rate": 3e-06,
+      "loss": 0.8143,
+      "masked_tokens": 106.025,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 106.025
+    },
+    {
+      "avg_mask_ratio": 0.457616981890169,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.457616981890169,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.6096036813853857,
+      "learning_rate": 3e-06,
+      "loss": 0.6649,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 103.3875
+    },
+    {
+      "avg_mask_ratio": 0.48775083787040785,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.48775083787040785,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.8403033299919912,
+      "learning_rate": 3e-06,
+      "loss": 0.8646,
+      "masked_tokens": 101.725,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 101.725
+    },
+    {
+      "avg_mask_ratio": 0.4505724890856072,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4505724890856072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.256,
+      "grad_norm": 0.2734375,
+      "kd_loss": 0.7449374303294463,
+      "learning_rate": 3e-06,
+      "loss": 0.6985,
+      "masked_tokens": 97.3125,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 97.3125
+    },
+    {
+      "avg_mask_ratio": 0.43574556948733517,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.43574556948733517,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.392578125,
+      "kd_loss": 0.4566404212248244,
+      "learning_rate": 3e-06,
+      "loss": 0.4885,
+      "masked_tokens": 89.325,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 89.325
+    },
+    {
+      "avg_mask_ratio": 0.49070783695206044,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.49070783695206044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.474609375,
+      "kd_loss": 0.6905622516694165,
+      "learning_rate": 3e-06,
+      "loss": 0.7061,
+      "masked_tokens": 130.2625,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 130.2625
+    },
+    {
+      "avg_mask_ratio": 0.4736109555349685,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4736109555349685,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.32,
+      "grad_norm": 0.08056640625,
+      "kd_loss": 0.7106425042155138,
+      "learning_rate": 3e-06,
+      "loss": 0.7643,
+      "masked_tokens": 126.2,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 126.2
+    },
+    {
+      "avg_mask_ratio": 0.5032523009285796,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5032523009285796,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.408203125,
+      "kd_loss": 0.8267615023827602,
+      "learning_rate": 3e-06,
+      "loss": 0.8409,
+      "masked_tokens": 99.425,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 99.425
+    },
+    {
+      "avg_mask_ratio": 0.4807244518771768,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.4807244518771768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.7329186423268765,
+      "learning_rate": 3e-06,
+      "loss": 0.6767,
+      "masked_tokens": 88.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 88.225
+    },
+    {
+      "avg_mask_ratio": 0.45808705932286103,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.45808705932286103,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.384,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5571840626302332,
+      "learning_rate": 3e-06,
+      "loss": 0.5724,
+      "masked_tokens": 105.8625,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 105.8625
+    },
+    {
+      "avg_mask_ratio": 0.4842760307248682,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.4842760307248682,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6595436891710733,
+      "learning_rate": 3e-06,
+      "loss": 0.6415,
+      "masked_tokens": 109.5875,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 109.5875
+    },
+    {
+      "avg_mask_ratio": 0.4665210062637925,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4665210062637925,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.5610411517063085,
+      "learning_rate": 3e-06,
+      "loss": 0.5866,
+      "masked_tokens": 97.0125,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 97.0125
+    },
+    {
+      "avg_mask_ratio": 0.45183838629163803,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.45183838629163803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.448,
+      "grad_norm": 0.29296875,
+      "kd_loss": 0.5228309052588884,
+      "learning_rate": 3e-06,
+      "loss": 0.5326,
+      "masked_tokens": 96.0125,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 96.0125
+    },
+    {
+      "avg_mask_ratio": 0.5396745702251792,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5396745702251792,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.7454893081907358,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 124.0125,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 124.0125
+    },
+    {
+      "avg_mask_ratio": 0.4404038088978268,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4404038088978268,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.5848532651940331,
+      "learning_rate": 3e-06,
+      "loss": 0.6239,
+      "masked_tokens": 107.1875,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 107.1875
+    },
+    {
+      "avg_mask_ratio": 0.47433349161874505,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.47433349161874505,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.512,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.7448974770184804,
+      "learning_rate": 3e-06,
+      "loss": 0.7729,
+      "masked_tokens": 106.275,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 106.275
+    },
+    {
+      "avg_mask_ratio": 0.4465038388385437,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4465038388385437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.1328125,
+      "kd_loss": 0.5912455287105786,
+      "learning_rate": 3e-06,
+      "loss": 0.5836,
+      "masked_tokens": 112.725,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 112.725
+    },
+    {
+      "avg_mask_ratio": 0.45215590375009923,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.45215590375009923,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.2451171875,
+      "kd_loss": 0.5663141592655847,
+      "learning_rate": 3e-06,
+      "loss": 0.531,
+      "masked_tokens": 109.1,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 109.1
+    },
+    {
+      "avg_mask_ratio": 0.5305180630879477,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.5305180630879477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.576,
+      "grad_norm": 0.4453125,
+      "kd_loss": 0.8847639085633545,
+      "learning_rate": 3e-06,
+      "loss": 0.9006,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4644554515252821,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4644554515252821,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.6207698138765181,
+      "learning_rate": 3e-06,
+      "loss": 0.5834,
+      "masked_tokens": 126.2125,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 126.2125
+    },
+    {
+      "avg_mask_ratio": 0.3939544238965027,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.3939544238965027,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.483729143582201,
+      "learning_rate": 3e-06,
+      "loss": 0.471,
+      "masked_tokens": 96.975,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 96.975
+    },
+    {
+      "avg_mask_ratio": 0.4442457370925695,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.4442457370925695,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.64,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.5351485910949123,
+      "learning_rate": 3e-06,
+      "loss": 0.607,
+      "masked_tokens": 101.525,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 101.525
+    },
+    {
+      "avg_mask_ratio": 0.48191855432232844,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.48191855432232844,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.2890625,
+      "kd_loss": 0.7135523370927593,
+      "learning_rate": 3e-06,
+      "loss": 0.695,
+      "masked_tokens": 84.875,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 84.875
+    },
+    {
+      "avg_mask_ratio": 0.49743892522528765,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.49743892522528765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.7042804263782273,
+      "learning_rate": 3e-06,
+      "loss": 0.7858,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.511213600449264,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.511213600449264,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.704,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.6977765330942021,
+      "learning_rate": 3e-06,
+      "loss": 0.7303,
+      "masked_tokens": 128.1,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 128.1
+    },
+    {
+      "avg_mask_ratio": 0.44370225080056114,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.44370225080056114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.4375,
+      "kd_loss": 0.714193849134108,
+      "learning_rate": 3e-06,
+      "loss": 0.6445,
+      "masked_tokens": 108.775,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 108.775
+    },
+    {
+      "avg_mask_ratio": 0.49959173843380994,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.49959173843380994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.7770639997791452,
+      "learning_rate": 3e-06,
+      "loss": 0.7806,
+      "masked_tokens": 125.625,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 125.625
+    },
+    {
+      "avg_mask_ratio": 0.47441557042766364,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.47441557042766364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.768,
+      "grad_norm": 0.15234375,
+      "kd_loss": 0.762115379356667,
+      "learning_rate": 3e-06,
+      "loss": 0.7428,
+      "masked_tokens": 114.3375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 114.3375
+    },
+    {
+      "avg_mask_ratio": 0.4738660019589588,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.4738660019589588,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.0859375,
+      "kd_loss": 0.7563776895753108,
+      "learning_rate": 3e-06,
+      "loss": 0.6971,
+      "masked_tokens": 126.3875,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 126.3875
+    },
+    {
+      "avg_mask_ratio": 0.4487657839199528,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.4487657839199528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.259765625,
+      "kd_loss": 0.5462394427451727,
+      "learning_rate": 3e-06,
+      "loss": 0.4745,
+      "masked_tokens": 96.725,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 96.725
+    },
+    {
+      "avg_mask_ratio": 0.4751023279619403,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.4751023279619403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.832,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.685858413620528,
+      "learning_rate": 3e-06,
+      "loss": 0.6922,
+      "masked_tokens": 110.5625,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 110.5625
+    },
+    {
+      "avg_mask_ratio": 0.4746393185690977,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.4746393185690977,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.7327228688028754,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 120.3875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 120.3875
+    },
+    {
+      "avg_mask_ratio": 0.4831854701798875,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.4831854701798875,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.265625,
+      "kd_loss": 0.8579687503222686,
+      "learning_rate": 3e-06,
+      "loss": 0.7988,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.4379329536575824,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4379329536575824,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.896,
+      "grad_norm": 0.21875,
+      "kd_loss": 0.45460584598010884,
+      "learning_rate": 3e-06,
+      "loss": 0.4832,
+      "masked_tokens": 85.525,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 85.525
+    },
+    {
+      "avg_mask_ratio": 0.4674084897618741,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4674084897618741,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.2216796875,
+      "kd_loss": 0.653534199286878,
+      "learning_rate": 3e-06,
+      "loss": 0.6388,
+      "masked_tokens": 104.5625,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 104.5625
+    },
+    {
+      "avg_mask_ratio": 0.44310826995642855,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.44310826995642855,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.6742433619177973,
+      "learning_rate": 3e-06,
+      "loss": 0.6468,
+      "masked_tokens": 106.1875,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 106.1875
+    },
+    {
+      "avg_mask_ratio": 0.4248724762815982,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4248724762815982,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.96,
+      "grad_norm": 0.25390625,
+      "kd_loss": 0.6201432387717432,
+      "learning_rate": 3e-06,
+      "loss": 0.6546,
+      "masked_tokens": 102.7375,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 102.7375
+    },
+    {
+      "avg_mask_ratio": 0.47578654896933587,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.47578654896933587,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.16015625,
+      "kd_loss": 0.6602298976420571,
+      "learning_rate": 3e-06,
+      "loss": 0.6259,
+      "masked_tokens": 105.3125,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 105.3125
+    },
+    {
+      "avg_mask_ratio": 0.4781565316176663,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.4781565316176663,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.361328125,
+      "kd_loss": 0.7762555406697461,
+      "learning_rate": 3e-06,
+      "loss": 0.7824,
+      "masked_tokens": 106.6547619047619,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 106.6547619047619
+    },
+    {
+      "avg_mask_ratio": 0.4757596247596666,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.4757596247596666,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0256,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7768732401565984,
+      "learning_rate": 3e-06,
+      "loss": 0.7435,
+      "masked_tokens": 118.225,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 118.225
+    },
+    {
+      "avg_mask_ratio": 0.4564065972692333,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.4564065972692333,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.6187377570489844,
+      "learning_rate": 3e-06,
+      "loss": 0.5998,
+      "masked_tokens": 97.6375,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 97.6375
+    },
+    {
+      "avg_mask_ratio": 0.5074894667137414,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5074894667137414,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.712813160381853,
+      "learning_rate": 3e-06,
+      "loss": 0.7089,
+      "masked_tokens": 122.9,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 122.9
+    },
+    {
+      "avg_mask_ratio": 0.5111216343124397,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5111216343124397,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0896,
+      "grad_norm": 0.30078125,
+      "kd_loss": 0.7725131734614138,
+      "learning_rate": 3e-06,
+      "loss": 0.7319,
+      "masked_tokens": 132.4625,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 132.4625
+    },
+    {
+      "avg_mask_ratio": 0.45396183808334173,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.45396183808334173,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.396484375,
+      "kd_loss": 0.6740453685139528,
+      "learning_rate": 3e-06,
+      "loss": 0.6974,
+      "masked_tokens": 87.0625,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 87.0625
+    },
+    {
+      "avg_mask_ratio": 0.4701436776900664,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4701436776900664,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.37890625,
+      "kd_loss": 0.7894933511876161,
+      "learning_rate": 3e-06,
+      "loss": 0.8161,
+      "masked_tokens": 99.2875,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 99.2875
+    },
+    {
+      "avg_mask_ratio": 0.48931564125232396,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.48931564125232396,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1536,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.8131277295872679,
+      "learning_rate": 3e-06,
+      "loss": 0.7558,
+      "masked_tokens": 107.3625,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 107.3625
+    },
+    {
+      "avg_mask_ratio": 0.4541942774085328,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4541942774085328,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.1767578125,
+      "kd_loss": 0.5594845790216368,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 97.8875,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 97.8875
+    },
+    {
+      "avg_mask_ratio": 0.3928302472108044,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.3928302472108044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.6713891216638501,
+      "learning_rate": 3e-06,
+      "loss": 0.6598,
+      "masked_tokens": 99.175,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 99.175
+    },
+    {
+      "avg_mask_ratio": 0.44847587069962175,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.44847587069962175,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2176,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.6333371287628324,
+      "learning_rate": 3e-06,
+      "loss": 0.6153,
+      "masked_tokens": 108.375,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 108.375
+    },
+    {
+      "avg_mask_ratio": 0.4655611292924732,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4655611292924732,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.7399741232418364,
+      "learning_rate": 3e-06,
+      "loss": 0.7193,
+      "masked_tokens": 127.4125,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 127.4125
+    },
+    {
+      "avg_mask_ratio": 0.5226378521183506,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5226378521183506,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.7912517642864259,
+      "learning_rate": 3e-06,
+      "loss": 0.7556,
+      "masked_tokens": 121.5375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 121.5375
+    },
+    {
+      "avg_mask_ratio": 0.45001045021926983,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.45001045021926983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2816,
+      "grad_norm": 0.2333984375,
+      "kd_loss": 0.5968554831010096,
+      "learning_rate": 3e-06,
+      "loss": 0.622,
+      "masked_tokens": 109.9,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 109.9
+    },
+    {
+      "avg_mask_ratio": 0.5040990226087161,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5040990226087161,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.1435546875,
+      "kd_loss": 0.7809002916160352,
+      "learning_rate": 3e-06,
+      "loss": 0.8354,
+      "masked_tokens": 110.375,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 110.375
+    },
+    {
+      "avg_mask_ratio": 0.4963218850665726,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.4963218850665726,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.19921875,
+      "kd_loss": 0.7623143148503232,
+      "learning_rate": 3e-06,
+      "loss": 0.7637,
+      "masked_tokens": 110.675,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 110.675
+    },
+    {
+      "avg_mask_ratio": 0.5259752279380336,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5259752279380336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 0.55859375,
+      "kd_loss": 0.9169201381674611,
+      "learning_rate": 3e-06,
+      "loss": 0.9908,
+      "masked_tokens": 125.525,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 125.525
+    },
+    {
+      "avg_mask_ratio": 0.49199640220031143,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.49199640220031143,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.658431020938042,
+      "learning_rate": 3e-06,
+      "loss": 0.6105,
+      "masked_tokens": 121.8625,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 121.8625
+    },
+    {
+      "avg_mask_ratio": 0.5331356228562072,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5331356228562072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.205078125,
+      "kd_loss": 0.8636023813149223,
+      "learning_rate": 3e-06,
+      "loss": 0.7956,
+      "masked_tokens": 128.65,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 128.65
+    },
+    {
+      "avg_mask_ratio": 0.4887094351579435,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.4887094351579435,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4096,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.7148105578169635,
+      "learning_rate": 3e-06,
+      "loss": 0.7531,
+      "masked_tokens": 129.8,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 129.8
+    },
+    {
+      "avg_mask_ratio": 0.46143888669321315,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.46143888669321315,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.451171875,
+      "kd_loss": 0.6032656382115874,
+      "learning_rate": 3e-06,
+      "loss": 0.6039,
+      "masked_tokens": 118.7375,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 118.7375
+    },
+    {
+      "avg_mask_ratio": 0.46107456521131096,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.46107456521131096,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.314453125,
+      "kd_loss": 0.6940350630834473,
+      "learning_rate": 3e-06,
+      "loss": 0.6841,
+      "masked_tokens": 106.4875,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 106.4875
+    },
+    {
+      "avg_mask_ratio": 0.4799915248062462,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.4799915248062462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4736,
+      "grad_norm": 0.189453125,
+      "kd_loss": 0.6958799545635429,
+      "learning_rate": 3e-06,
+      "loss": 0.684,
+      "masked_tokens": 111.5375,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 111.5375
+    },
+    {
+      "avg_mask_ratio": 0.5273691712645814,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5273691712645814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.9459125808391036,
+      "learning_rate": 3e-06,
+      "loss": 0.8879,
+      "masked_tokens": 132.525,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 132.525
+    },
+    {
+      "avg_mask_ratio": 0.48835311776492746,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.48835311776492746,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.1650390625,
+      "kd_loss": 0.6531627567595478,
+      "learning_rate": 3e-06,
+      "loss": 0.6323,
+      "masked_tokens": 108.125,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 108.125
+    },
+    {
+      "avg_mask_ratio": 0.48280774110462515,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.48280774110462515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.8419079009756387,
+      "learning_rate": 3e-06,
+      "loss": 0.7928,
+      "masked_tokens": 110.2,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.46588142022956164,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.46588142022956164,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.244140625,
+      "kd_loss": 0.6997778600025854,
+      "learning_rate": 3e-06,
+      "loss": 0.7623,
+      "masked_tokens": 103.3375,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 103.3375
+    },
+    {
+      "avg_mask_ratio": 0.5192104739835486,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5192104739835486,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.1708984375,
+      "kd_loss": 0.8950252983398969,
+      "learning_rate": 3e-06,
+      "loss": 0.8392,
+      "masked_tokens": 96.55,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 96.55
+    },
+    {
+      "avg_mask_ratio": 0.41821987080038525,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.41821987080038525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6016,
+      "grad_norm": 0.2099609375,
+      "kd_loss": 0.6870280907321394,
+      "learning_rate": 3e-06,
+      "loss": 0.6255,
+      "masked_tokens": 95.2,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 95.2
+    },
+    {
+      "avg_mask_ratio": 0.4688679567188956,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.4688679567188956,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 1.390625,
+      "kd_loss": 0.8148217237806421,
+      "learning_rate": 3e-06,
+      "loss": 0.7954,
+      "masked_tokens": 107.55,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 107.55
+    },
+    {
+      "avg_mask_ratio": 0.4429742675391026,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4429742675391026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.1826171875,
+      "kd_loss": 0.5454595529316066,
+      "learning_rate": 3e-06,
+      "loss": 0.5506,
+      "masked_tokens": 117.825,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 117.825
+    },
+    {
+      "avg_mask_ratio": 0.43331936674949245,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.43331936674949245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6656,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.4808090373327854,
+      "learning_rate": 3e-06,
+      "loss": 0.5158,
+      "masked_tokens": 117.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 117.2
+    },
+    {
+      "avg_mask_ratio": 0.4646148854692001,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4646148854692001,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.140625,
+      "kd_loss": 0.6250457387332972,
+      "learning_rate": 3e-06,
+      "loss": 0.6097,
+      "masked_tokens": 102.15,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 102.15
+    },
+    {
+      "avg_mask_ratio": 0.46852388482075186,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.46852388482075186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.1552734375,
+      "kd_loss": 0.5451920869360265,
+      "learning_rate": 3e-06,
+      "loss": 0.6197,
+      "masked_tokens": 105.6375,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 105.6375
+    },
+    {
+      "avg_mask_ratio": 0.5052226510597393,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5052226510597393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7296,
+      "grad_norm": 0.27734375,
+      "kd_loss": 0.7552159034145006,
+      "learning_rate": 3e-06,
+      "loss": 0.7562,
+      "masked_tokens": 119.5375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 119.5375
+    },
+    {
+      "avg_mask_ratio": 0.5101147621171549,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5101147621171549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.8518789664900283,
+      "learning_rate": 3e-06,
+      "loss": 0.851,
+      "masked_tokens": 117.525,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 117.525
+    },
+    {
+      "avg_mask_ratio": 0.4939760136883706,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.4939760136883706,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.8243849514963074,
+      "learning_rate": 3e-06,
+      "loss": 0.7816,
+      "masked_tokens": 134.1625,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 134.1625
+    },
+    {
+      "avg_mask_ratio": 0.5015010182803963,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5015010182803963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7936,
+      "grad_norm": 0.1279296875,
+      "kd_loss": 0.6244754713046745,
+      "learning_rate": 3e-06,
+      "loss": 0.6667,
+      "masked_tokens": 114.4875,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 114.4875
+    },
+    {
+      "avg_mask_ratio": 0.5062767470255494,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5062767470255494,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.236328125,
+      "kd_loss": 0.6353196922647157,
+      "learning_rate": 3e-06,
+      "loss": 0.6607,
+      "masked_tokens": 115.7125,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 115.7125
+    },
+    {
+      "avg_mask_ratio": 0.4643575747264549,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4643575747264549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.7141597053345781,
+      "learning_rate": 3e-06,
+      "loss": 0.6556,
+      "masked_tokens": 106.9375,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 106.9375
+    },
+    {
+      "avg_mask_ratio": 0.487393151212018,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.487393151212018,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.6699687715730555,
+      "learning_rate": 3e-06,
+      "loss": 0.787,
+      "masked_tokens": 114.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 114.8875
+    },
+    {
+      "avg_mask_ratio": 0.45142438664333895,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.45142438664333895,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7402904870072688,
+      "learning_rate": 3e-06,
+      "loss": 0.6762,
+      "masked_tokens": 92.25,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 92.25
+    },
+    {
+      "avg_mask_ratio": 0.4500263270718278,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.4500263270718278,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.0947265625,
+      "kd_loss": 0.5816223171608726,
+      "learning_rate": 3e-06,
+      "loss": 0.5512,
+      "masked_tokens": 102.8125,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 102.8125
+    },
+    {
+      "avg_mask_ratio": 0.4509512555785477,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4509512555785477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9216,
+      "grad_norm": 0.234375,
+      "kd_loss": 0.6806100918192441,
+      "learning_rate": 3e-06,
+      "loss": 0.6938,
+      "masked_tokens": 102.025,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 102.025
+    },
+    {
+      "avg_mask_ratio": 0.47468002026434986,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.47468002026434986,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.185546875,
+      "kd_loss": 0.6469848818184118,
+      "learning_rate": 3e-06,
+      "loss": 0.6391,
+      "masked_tokens": 124.3,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 124.3
+    },
+    {
+      "avg_mask_ratio": 0.4673476937925443,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4673476937925443,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.357421875,
+      "kd_loss": 0.7518777136654535,
+      "learning_rate": 3e-06,
+      "loss": 0.7765,
+      "masked_tokens": 109.7875,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 109.7875
+    },
+    {
+      "avg_mask_ratio": 0.4756378314923495,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.4756378314923495,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9856,
+      "grad_norm": 0.17578125,
+      "kd_loss": 0.6069021332950797,
+      "learning_rate": 3e-06,
+      "loss": 0.6412,
+      "masked_tokens": 119.8125,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 119.8125
+    },
+    {
+      "avg_mask_ratio": 0.5000089073021498,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5000089073021498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.7108402555168141,
+      "learning_rate": 3e-06,
+      "loss": 0.7752,
+      "masked_tokens": 121.98809523809524,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 121.98809523809524
+    },
+    {
+      "avg_mask_ratio": 0.4385950181807857,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.4385950181807857,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.2158203125,
+      "kd_loss": 0.652561020819121,
+      "learning_rate": 3e-06,
+      "loss": 0.6515,
+      "masked_tokens": 107.025,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 107.025
+    },
+    {
+      "avg_mask_ratio": 0.49335200865752993,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.49335200865752993,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0512,
+      "grad_norm": 0.2021484375,
+      "kd_loss": 0.8505055773921185,
+      "learning_rate": 3e-06,
+      "loss": 0.8085,
+      "masked_tokens": 141.5,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 141.5
+    },
+    {
+      "avg_mask_ratio": 0.4434558014094364,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4434558014094364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.564091070059294,
+      "learning_rate": 3e-06,
+      "loss": 0.5634,
+      "masked_tokens": 107.3,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 107.3
+    },
+    {
+      "avg_mask_ratio": 0.42445047190412877,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.42445047190412877,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.5745988191337801,
+      "learning_rate": 3e-06,
+      "loss": 0.5537,
+      "masked_tokens": 98.0875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 98.0875
+    },
+    {
+      "avg_mask_ratio": 0.4553717203787528,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.4553717203787528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1152,
+      "grad_norm": 0.14453125,
+      "kd_loss": 0.5502215233629159,
+      "learning_rate": 3e-06,
+      "loss": 0.5637,
+      "masked_tokens": 118.75,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 118.75
+    },
+    {
+      "avg_mask_ratio": 0.4424907588865608,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4424907588865608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5585700995494335,
+      "learning_rate": 3e-06,
+      "loss": 0.5795,
+      "masked_tokens": 98.9125,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 98.9125
+    },
+    {
+      "avg_mask_ratio": 0.43784486907534304,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.43784486907534304,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.111328125,
+      "kd_loss": 0.5065722677004316,
+      "learning_rate": 3e-06,
+      "loss": 0.5536,
+      "masked_tokens": 89.1625,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 89.1625
+    },
+    {
+      "avg_mask_ratio": 0.5077433207072317,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5077433207072317,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1792,
+      "grad_norm": 0.134765625,
+      "kd_loss": 0.7864730670612972,
+      "learning_rate": 3e-06,
+      "loss": 0.7616,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4700460441526957,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.4700460441526957,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.7690424034725968,
+      "learning_rate": 3e-06,
+      "loss": 0.7476,
+      "masked_tokens": 105.35,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 105.35
+    },
+    {
+      "avg_mask_ratio": 0.49483722454169765,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.49483722454169765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.7194663421146288,
+      "learning_rate": 3e-06,
+      "loss": 0.7147,
+      "masked_tokens": 99.85,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 99.85
+    },
+    {
+      "avg_mask_ratio": 0.515152573143132,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.515152573143132,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2432,
+      "grad_norm": 0.3046875,
+      "kd_loss": 0.8700408121391092,
+      "learning_rate": 3e-06,
+      "loss": 0.7471,
+      "masked_tokens": 122.6125,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 122.6125
+    },
+    {
+      "avg_mask_ratio": 0.4577924039273057,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.4577924039273057,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.565913611318291,
+      "learning_rate": 3e-06,
+      "loss": 0.5762,
+      "masked_tokens": 121.45,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 121.45
+    },
+    {
+      "avg_mask_ratio": 0.47600019703386354,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.47600019703386354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.5862566602218522,
+      "learning_rate": 3e-06,
+      "loss": 0.6313,
+      "masked_tokens": 134.6,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 134.6
+    },
+    {
+      "avg_mask_ratio": 0.4636200438835658,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4636200438835658,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3072,
+      "grad_norm": 0.12060546875,
+      "kd_loss": 0.5776349333416817,
+      "learning_rate": 3e-06,
+      "loss": 0.5714,
+      "masked_tokens": 97.875,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 97.875
+    },
+    {
+      "avg_mask_ratio": 0.43436883635586127,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.43436883635586127,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.11328125,
+      "kd_loss": 0.5290679484073877,
+      "learning_rate": 3e-06,
+      "loss": 0.5147,
+      "masked_tokens": 90.375,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 90.375
+    },
+    {
+      "avg_mask_ratio": 0.5262026051990688,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.5262026051990688,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.322265625,
+      "kd_loss": 0.662759238135186,
+      "learning_rate": 3e-06,
+      "loss": 0.7141,
+      "masked_tokens": 133.7375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 133.7375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2611bced1337375831cc344da28d849ea16c0f2f5a6aeca004865b84ca7216e0
+size 7992

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5f39ef5d2084fc7c84bc5a80e580394030c2b375d1a2840a64a2e1d1e83fa1dc
+size 2406624648

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b55114cd459b830c787261045b12c2bc19a0b7de5db04aeb26aa204aa829798
+size 671304442

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fbbd0e8c1bfbd7ba8c634ca07b1d8702578d8a5068f2536ae69c20a51bf959b7
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4c8dcfaaa26518a9158534ae7671344dd035fcb11f3b40b193e3c3bd47123883
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f8c95a6d9085dfcee1e6620c88ede526366d3a02c5018932b1bc04809c0e0c7
+size 1064

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/trainer_state.json

@@ -0,0 +1,2913 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5632,
+  "eval_steps": 500,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.49314909732202067,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.49314909732202067,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.8524778203239294,
+      "learning_rate": 3e-06,
+      "loss": 0.8897,
+      "masked_tokens": 111.775,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 111.775
+    },
+    {
+      "avg_mask_ratio": 0.42081435878062623,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.42081435878062623,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.6601746417798836,
+      "learning_rate": 3e-06,
+      "loss": 0.6494,
+      "masked_tokens": 94.7,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.7
+    },
+    {
+      "avg_mask_ratio": 0.4538542575784959,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4538542575784959,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.064,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.7305509156625248,
+      "learning_rate": 3e-06,
+      "loss": 0.6735,
+      "masked_tokens": 110.2,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.42071006570186,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.42071006570186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.6237847281259746,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 94.5125,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.5125
+    },
+    {
+      "avg_mask_ratio": 0.43260439952719026,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.43260439952719026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.6509197178371323,
+      "learning_rate": 3e-06,
+      "loss": 0.6726,
+      "masked_tokens": 85.05,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.05
+    },
+    {
+      "avg_mask_ratio": 0.4602369442814961,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4602369442814961,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.128,
+      "grad_norm": 0.1455078125,
+      "kd_loss": 0.6954449985298851,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 119.8625,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.8625
+    },
+    {
+      "avg_mask_ratio": 0.5111442144960165,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5111442144960165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.7338119231904784,
+      "learning_rate": 3e-06,
+      "loss": 0.837,
+      "masked_tokens": 104.325,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.37548826879356056,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.37548826879356056,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.5029199755957052,
+      "learning_rate": 3e-06,
+      "loss": 0.536,
+      "masked_tokens": 85.0625,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 85.0625
+    },
+    {
+      "avg_mask_ratio": 0.4998963106656447,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.4998963106656447,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.192,
+      "grad_norm": 0.3359375,
+      "kd_loss": 0.8495752438985221,
+      "learning_rate": 3e-06,
+      "loss": 0.8143,
+      "masked_tokens": 106.025,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 106.025
+    },
+    {
+      "avg_mask_ratio": 0.457616981890169,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.457616981890169,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.6096036813853857,
+      "learning_rate": 3e-06,
+      "loss": 0.6649,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 103.3875
+    },
+    {
+      "avg_mask_ratio": 0.48775083787040785,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.48775083787040785,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.8403033299919912,
+      "learning_rate": 3e-06,
+      "loss": 0.8646,
+      "masked_tokens": 101.725,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 101.725
+    },
+    {
+      "avg_mask_ratio": 0.4505724890856072,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4505724890856072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.256,
+      "grad_norm": 0.2734375,
+      "kd_loss": 0.7449374303294463,
+      "learning_rate": 3e-06,
+      "loss": 0.6985,
+      "masked_tokens": 97.3125,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 97.3125
+    },
+    {
+      "avg_mask_ratio": 0.43574556948733517,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.43574556948733517,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.392578125,
+      "kd_loss": 0.4566404212248244,
+      "learning_rate": 3e-06,
+      "loss": 0.4885,
+      "masked_tokens": 89.325,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 89.325
+    },
+    {
+      "avg_mask_ratio": 0.49070783695206044,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.49070783695206044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.474609375,
+      "kd_loss": 0.6905622516694165,
+      "learning_rate": 3e-06,
+      "loss": 0.7061,
+      "masked_tokens": 130.2625,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 130.2625
+    },
+    {
+      "avg_mask_ratio": 0.4736109555349685,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4736109555349685,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.32,
+      "grad_norm": 0.08056640625,
+      "kd_loss": 0.7106425042155138,
+      "learning_rate": 3e-06,
+      "loss": 0.7643,
+      "masked_tokens": 126.2,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 126.2
+    },
+    {
+      "avg_mask_ratio": 0.5032523009285796,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5032523009285796,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.408203125,
+      "kd_loss": 0.8267615023827602,
+      "learning_rate": 3e-06,
+      "loss": 0.8409,
+      "masked_tokens": 99.425,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 99.425
+    },
+    {
+      "avg_mask_ratio": 0.4807244518771768,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.4807244518771768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.7329186423268765,
+      "learning_rate": 3e-06,
+      "loss": 0.6767,
+      "masked_tokens": 88.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 88.225
+    },
+    {
+      "avg_mask_ratio": 0.45808705932286103,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.45808705932286103,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.384,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5571840626302332,
+      "learning_rate": 3e-06,
+      "loss": 0.5724,
+      "masked_tokens": 105.8625,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 105.8625
+    },
+    {
+      "avg_mask_ratio": 0.4842760307248682,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.4842760307248682,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6595436891710733,
+      "learning_rate": 3e-06,
+      "loss": 0.6415,
+      "masked_tokens": 109.5875,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 109.5875
+    },
+    {
+      "avg_mask_ratio": 0.4665210062637925,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4665210062637925,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.5610411517063085,
+      "learning_rate": 3e-06,
+      "loss": 0.5866,
+      "masked_tokens": 97.0125,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 97.0125
+    },
+    {
+      "avg_mask_ratio": 0.45183838629163803,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.45183838629163803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.448,
+      "grad_norm": 0.29296875,
+      "kd_loss": 0.5228309052588884,
+      "learning_rate": 3e-06,
+      "loss": 0.5326,
+      "masked_tokens": 96.0125,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 96.0125
+    },
+    {
+      "avg_mask_ratio": 0.5396745702251792,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5396745702251792,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.7454893081907358,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 124.0125,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 124.0125
+    },
+    {
+      "avg_mask_ratio": 0.4404038088978268,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4404038088978268,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.5848532651940331,
+      "learning_rate": 3e-06,
+      "loss": 0.6239,
+      "masked_tokens": 107.1875,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 107.1875
+    },
+    {
+      "avg_mask_ratio": 0.47433349161874505,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.47433349161874505,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.512,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.7448974770184804,
+      "learning_rate": 3e-06,
+      "loss": 0.7729,
+      "masked_tokens": 106.275,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 106.275
+    },
+    {
+      "avg_mask_ratio": 0.4465038388385437,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4465038388385437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.1328125,
+      "kd_loss": 0.5912455287105786,
+      "learning_rate": 3e-06,
+      "loss": 0.5836,
+      "masked_tokens": 112.725,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 112.725
+    },
+    {
+      "avg_mask_ratio": 0.45215590375009923,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.45215590375009923,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.2451171875,
+      "kd_loss": 0.5663141592655847,
+      "learning_rate": 3e-06,
+      "loss": 0.531,
+      "masked_tokens": 109.1,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 109.1
+    },
+    {
+      "avg_mask_ratio": 0.5305180630879477,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.5305180630879477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.576,
+      "grad_norm": 0.4453125,
+      "kd_loss": 0.8847639085633545,
+      "learning_rate": 3e-06,
+      "loss": 0.9006,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4644554515252821,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4644554515252821,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.6207698138765181,
+      "learning_rate": 3e-06,
+      "loss": 0.5834,
+      "masked_tokens": 126.2125,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 126.2125
+    },
+    {
+      "avg_mask_ratio": 0.3939544238965027,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.3939544238965027,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.483729143582201,
+      "learning_rate": 3e-06,
+      "loss": 0.471,
+      "masked_tokens": 96.975,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 96.975
+    },
+    {
+      "avg_mask_ratio": 0.4442457370925695,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.4442457370925695,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.64,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.5351485910949123,
+      "learning_rate": 3e-06,
+      "loss": 0.607,
+      "masked_tokens": 101.525,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 101.525
+    },
+    {
+      "avg_mask_ratio": 0.48191855432232844,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.48191855432232844,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.2890625,
+      "kd_loss": 0.7135523370927593,
+      "learning_rate": 3e-06,
+      "loss": 0.695,
+      "masked_tokens": 84.875,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 84.875
+    },
+    {
+      "avg_mask_ratio": 0.49743892522528765,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.49743892522528765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.7042804263782273,
+      "learning_rate": 3e-06,
+      "loss": 0.7858,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.511213600449264,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.511213600449264,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.704,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.6977765330942021,
+      "learning_rate": 3e-06,
+      "loss": 0.7303,
+      "masked_tokens": 128.1,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 128.1
+    },
+    {
+      "avg_mask_ratio": 0.44370225080056114,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.44370225080056114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.4375,
+      "kd_loss": 0.714193849134108,
+      "learning_rate": 3e-06,
+      "loss": 0.6445,
+      "masked_tokens": 108.775,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 108.775
+    },
+    {
+      "avg_mask_ratio": 0.49959173843380994,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.49959173843380994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.7770639997791452,
+      "learning_rate": 3e-06,
+      "loss": 0.7806,
+      "masked_tokens": 125.625,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 125.625
+    },
+    {
+      "avg_mask_ratio": 0.47441557042766364,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.47441557042766364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.768,
+      "grad_norm": 0.15234375,
+      "kd_loss": 0.762115379356667,
+      "learning_rate": 3e-06,
+      "loss": 0.7428,
+      "masked_tokens": 114.3375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 114.3375
+    },
+    {
+      "avg_mask_ratio": 0.4738660019589588,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.4738660019589588,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.0859375,
+      "kd_loss": 0.7563776895753108,
+      "learning_rate": 3e-06,
+      "loss": 0.6971,
+      "masked_tokens": 126.3875,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 126.3875
+    },
+    {
+      "avg_mask_ratio": 0.4487657839199528,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.4487657839199528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.259765625,
+      "kd_loss": 0.5462394427451727,
+      "learning_rate": 3e-06,
+      "loss": 0.4745,
+      "masked_tokens": 96.725,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 96.725
+    },
+    {
+      "avg_mask_ratio": 0.4751023279619403,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.4751023279619403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.832,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.685858413620528,
+      "learning_rate": 3e-06,
+      "loss": 0.6922,
+      "masked_tokens": 110.5625,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 110.5625
+    },
+    {
+      "avg_mask_ratio": 0.4746393185690977,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.4746393185690977,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.7327228688028754,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 120.3875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 120.3875
+    },
+    {
+      "avg_mask_ratio": 0.4831854701798875,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.4831854701798875,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.265625,
+      "kd_loss": 0.8579687503222686,
+      "learning_rate": 3e-06,
+      "loss": 0.7988,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.4379329536575824,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4379329536575824,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.896,
+      "grad_norm": 0.21875,
+      "kd_loss": 0.45460584598010884,
+      "learning_rate": 3e-06,
+      "loss": 0.4832,
+      "masked_tokens": 85.525,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 85.525
+    },
+    {
+      "avg_mask_ratio": 0.4674084897618741,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4674084897618741,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.2216796875,
+      "kd_loss": 0.653534199286878,
+      "learning_rate": 3e-06,
+      "loss": 0.6388,
+      "masked_tokens": 104.5625,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 104.5625
+    },
+    {
+      "avg_mask_ratio": 0.44310826995642855,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.44310826995642855,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.6742433619177973,
+      "learning_rate": 3e-06,
+      "loss": 0.6468,
+      "masked_tokens": 106.1875,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 106.1875
+    },
+    {
+      "avg_mask_ratio": 0.4248724762815982,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4248724762815982,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.96,
+      "grad_norm": 0.25390625,
+      "kd_loss": 0.6201432387717432,
+      "learning_rate": 3e-06,
+      "loss": 0.6546,
+      "masked_tokens": 102.7375,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 102.7375
+    },
+    {
+      "avg_mask_ratio": 0.47578654896933587,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.47578654896933587,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.16015625,
+      "kd_loss": 0.6602298976420571,
+      "learning_rate": 3e-06,
+      "loss": 0.6259,
+      "masked_tokens": 105.3125,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 105.3125
+    },
+    {
+      "avg_mask_ratio": 0.4781565316176663,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.4781565316176663,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.361328125,
+      "kd_loss": 0.7762555406697461,
+      "learning_rate": 3e-06,
+      "loss": 0.7824,
+      "masked_tokens": 106.6547619047619,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 106.6547619047619
+    },
+    {
+      "avg_mask_ratio": 0.4757596247596666,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.4757596247596666,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0256,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7768732401565984,
+      "learning_rate": 3e-06,
+      "loss": 0.7435,
+      "masked_tokens": 118.225,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 118.225
+    },
+    {
+      "avg_mask_ratio": 0.4564065972692333,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.4564065972692333,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.6187377570489844,
+      "learning_rate": 3e-06,
+      "loss": 0.5998,
+      "masked_tokens": 97.6375,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 97.6375
+    },
+    {
+      "avg_mask_ratio": 0.5074894667137414,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5074894667137414,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.712813160381853,
+      "learning_rate": 3e-06,
+      "loss": 0.7089,
+      "masked_tokens": 122.9,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 122.9
+    },
+    {
+      "avg_mask_ratio": 0.5111216343124397,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5111216343124397,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0896,
+      "grad_norm": 0.30078125,
+      "kd_loss": 0.7725131734614138,
+      "learning_rate": 3e-06,
+      "loss": 0.7319,
+      "masked_tokens": 132.4625,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 132.4625
+    },
+    {
+      "avg_mask_ratio": 0.45396183808334173,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.45396183808334173,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.396484375,
+      "kd_loss": 0.6740453685139528,
+      "learning_rate": 3e-06,
+      "loss": 0.6974,
+      "masked_tokens": 87.0625,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 87.0625
+    },
+    {
+      "avg_mask_ratio": 0.4701436776900664,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4701436776900664,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.37890625,
+      "kd_loss": 0.7894933511876161,
+      "learning_rate": 3e-06,
+      "loss": 0.8161,
+      "masked_tokens": 99.2875,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 99.2875
+    },
+    {
+      "avg_mask_ratio": 0.48931564125232396,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.48931564125232396,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1536,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.8131277295872679,
+      "learning_rate": 3e-06,
+      "loss": 0.7558,
+      "masked_tokens": 107.3625,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 107.3625
+    },
+    {
+      "avg_mask_ratio": 0.4541942774085328,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4541942774085328,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.1767578125,
+      "kd_loss": 0.5594845790216368,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 97.8875,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 97.8875
+    },
+    {
+      "avg_mask_ratio": 0.3928302472108044,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.3928302472108044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.6713891216638501,
+      "learning_rate": 3e-06,
+      "loss": 0.6598,
+      "masked_tokens": 99.175,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 99.175
+    },
+    {
+      "avg_mask_ratio": 0.44847587069962175,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.44847587069962175,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2176,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.6333371287628324,
+      "learning_rate": 3e-06,
+      "loss": 0.6153,
+      "masked_tokens": 108.375,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 108.375
+    },
+    {
+      "avg_mask_ratio": 0.4655611292924732,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4655611292924732,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.7399741232418364,
+      "learning_rate": 3e-06,
+      "loss": 0.7193,
+      "masked_tokens": 127.4125,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 127.4125
+    },
+    {
+      "avg_mask_ratio": 0.5226378521183506,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5226378521183506,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.7912517642864259,
+      "learning_rate": 3e-06,
+      "loss": 0.7556,
+      "masked_tokens": 121.5375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 121.5375
+    },
+    {
+      "avg_mask_ratio": 0.45001045021926983,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.45001045021926983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2816,
+      "grad_norm": 0.2333984375,
+      "kd_loss": 0.5968554831010096,
+      "learning_rate": 3e-06,
+      "loss": 0.622,
+      "masked_tokens": 109.9,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 109.9
+    },
+    {
+      "avg_mask_ratio": 0.5040990226087161,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5040990226087161,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.1435546875,
+      "kd_loss": 0.7809002916160352,
+      "learning_rate": 3e-06,
+      "loss": 0.8354,
+      "masked_tokens": 110.375,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 110.375
+    },
+    {
+      "avg_mask_ratio": 0.4963218850665726,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.4963218850665726,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.19921875,
+      "kd_loss": 0.7623143148503232,
+      "learning_rate": 3e-06,
+      "loss": 0.7637,
+      "masked_tokens": 110.675,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 110.675
+    },
+    {
+      "avg_mask_ratio": 0.5259752279380336,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5259752279380336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 0.55859375,
+      "kd_loss": 0.9169201381674611,
+      "learning_rate": 3e-06,
+      "loss": 0.9908,
+      "masked_tokens": 125.525,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 125.525
+    },
+    {
+      "avg_mask_ratio": 0.49199640220031143,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.49199640220031143,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.658431020938042,
+      "learning_rate": 3e-06,
+      "loss": 0.6105,
+      "masked_tokens": 121.8625,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 121.8625
+    },
+    {
+      "avg_mask_ratio": 0.5331356228562072,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5331356228562072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.205078125,
+      "kd_loss": 0.8636023813149223,
+      "learning_rate": 3e-06,
+      "loss": 0.7956,
+      "masked_tokens": 128.65,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 128.65
+    },
+    {
+      "avg_mask_ratio": 0.4887094351579435,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.4887094351579435,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4096,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.7148105578169635,
+      "learning_rate": 3e-06,
+      "loss": 0.7531,
+      "masked_tokens": 129.8,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 129.8
+    },
+    {
+      "avg_mask_ratio": 0.46143888669321315,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.46143888669321315,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.451171875,
+      "kd_loss": 0.6032656382115874,
+      "learning_rate": 3e-06,
+      "loss": 0.6039,
+      "masked_tokens": 118.7375,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 118.7375
+    },
+    {
+      "avg_mask_ratio": 0.46107456521131096,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.46107456521131096,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.314453125,
+      "kd_loss": 0.6940350630834473,
+      "learning_rate": 3e-06,
+      "loss": 0.6841,
+      "masked_tokens": 106.4875,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 106.4875
+    },
+    {
+      "avg_mask_ratio": 0.4799915248062462,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.4799915248062462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4736,
+      "grad_norm": 0.189453125,
+      "kd_loss": 0.6958799545635429,
+      "learning_rate": 3e-06,
+      "loss": 0.684,
+      "masked_tokens": 111.5375,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 111.5375
+    },
+    {
+      "avg_mask_ratio": 0.5273691712645814,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5273691712645814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.9459125808391036,
+      "learning_rate": 3e-06,
+      "loss": 0.8879,
+      "masked_tokens": 132.525,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 132.525
+    },
+    {
+      "avg_mask_ratio": 0.48835311776492746,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.48835311776492746,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.1650390625,
+      "kd_loss": 0.6531627567595478,
+      "learning_rate": 3e-06,
+      "loss": 0.6323,
+      "masked_tokens": 108.125,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 108.125
+    },
+    {
+      "avg_mask_ratio": 0.48280774110462515,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.48280774110462515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.8419079009756387,
+      "learning_rate": 3e-06,
+      "loss": 0.7928,
+      "masked_tokens": 110.2,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.46588142022956164,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.46588142022956164,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.244140625,
+      "kd_loss": 0.6997778600025854,
+      "learning_rate": 3e-06,
+      "loss": 0.7623,
+      "masked_tokens": 103.3375,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 103.3375
+    },
+    {
+      "avg_mask_ratio": 0.5192104739835486,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5192104739835486,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.1708984375,
+      "kd_loss": 0.8950252983398969,
+      "learning_rate": 3e-06,
+      "loss": 0.8392,
+      "masked_tokens": 96.55,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 96.55
+    },
+    {
+      "avg_mask_ratio": 0.41821987080038525,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.41821987080038525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6016,
+      "grad_norm": 0.2099609375,
+      "kd_loss": 0.6870280907321394,
+      "learning_rate": 3e-06,
+      "loss": 0.6255,
+      "masked_tokens": 95.2,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 95.2
+    },
+    {
+      "avg_mask_ratio": 0.4688679567188956,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.4688679567188956,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 1.390625,
+      "kd_loss": 0.8148217237806421,
+      "learning_rate": 3e-06,
+      "loss": 0.7954,
+      "masked_tokens": 107.55,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 107.55
+    },
+    {
+      "avg_mask_ratio": 0.4429742675391026,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4429742675391026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.1826171875,
+      "kd_loss": 0.5454595529316066,
+      "learning_rate": 3e-06,
+      "loss": 0.5506,
+      "masked_tokens": 117.825,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 117.825
+    },
+    {
+      "avg_mask_ratio": 0.43331936674949245,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.43331936674949245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6656,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.4808090373327854,
+      "learning_rate": 3e-06,
+      "loss": 0.5158,
+      "masked_tokens": 117.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 117.2
+    },
+    {
+      "avg_mask_ratio": 0.4646148854692001,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4646148854692001,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.140625,
+      "kd_loss": 0.6250457387332972,
+      "learning_rate": 3e-06,
+      "loss": 0.6097,
+      "masked_tokens": 102.15,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 102.15
+    },
+    {
+      "avg_mask_ratio": 0.46852388482075186,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.46852388482075186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.1552734375,
+      "kd_loss": 0.5451920869360265,
+      "learning_rate": 3e-06,
+      "loss": 0.6197,
+      "masked_tokens": 105.6375,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 105.6375
+    },
+    {
+      "avg_mask_ratio": 0.5052226510597393,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5052226510597393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7296,
+      "grad_norm": 0.27734375,
+      "kd_loss": 0.7552159034145006,
+      "learning_rate": 3e-06,
+      "loss": 0.7562,
+      "masked_tokens": 119.5375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 119.5375
+    },
+    {
+      "avg_mask_ratio": 0.5101147621171549,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5101147621171549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.8518789664900283,
+      "learning_rate": 3e-06,
+      "loss": 0.851,
+      "masked_tokens": 117.525,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 117.525
+    },
+    {
+      "avg_mask_ratio": 0.4939760136883706,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.4939760136883706,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.8243849514963074,
+      "learning_rate": 3e-06,
+      "loss": 0.7816,
+      "masked_tokens": 134.1625,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 134.1625
+    },
+    {
+      "avg_mask_ratio": 0.5015010182803963,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5015010182803963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7936,
+      "grad_norm": 0.1279296875,
+      "kd_loss": 0.6244754713046745,
+      "learning_rate": 3e-06,
+      "loss": 0.6667,
+      "masked_tokens": 114.4875,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 114.4875
+    },
+    {
+      "avg_mask_ratio": 0.5062767470255494,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5062767470255494,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.236328125,
+      "kd_loss": 0.6353196922647157,
+      "learning_rate": 3e-06,
+      "loss": 0.6607,
+      "masked_tokens": 115.7125,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 115.7125
+    },
+    {
+      "avg_mask_ratio": 0.4643575747264549,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4643575747264549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.7141597053345781,
+      "learning_rate": 3e-06,
+      "loss": 0.6556,
+      "masked_tokens": 106.9375,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 106.9375
+    },
+    {
+      "avg_mask_ratio": 0.487393151212018,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.487393151212018,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.6699687715730555,
+      "learning_rate": 3e-06,
+      "loss": 0.787,
+      "masked_tokens": 114.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 114.8875
+    },
+    {
+      "avg_mask_ratio": 0.45142438664333895,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.45142438664333895,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7402904870072688,
+      "learning_rate": 3e-06,
+      "loss": 0.6762,
+      "masked_tokens": 92.25,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 92.25
+    },
+    {
+      "avg_mask_ratio": 0.4500263270718278,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.4500263270718278,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.0947265625,
+      "kd_loss": 0.5816223171608726,
+      "learning_rate": 3e-06,
+      "loss": 0.5512,
+      "masked_tokens": 102.8125,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 102.8125
+    },
+    {
+      "avg_mask_ratio": 0.4509512555785477,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4509512555785477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9216,
+      "grad_norm": 0.234375,
+      "kd_loss": 0.6806100918192441,
+      "learning_rate": 3e-06,
+      "loss": 0.6938,
+      "masked_tokens": 102.025,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 102.025
+    },
+    {
+      "avg_mask_ratio": 0.47468002026434986,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.47468002026434986,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.185546875,
+      "kd_loss": 0.6469848818184118,
+      "learning_rate": 3e-06,
+      "loss": 0.6391,
+      "masked_tokens": 124.3,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 124.3
+    },
+    {
+      "avg_mask_ratio": 0.4673476937925443,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4673476937925443,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.357421875,
+      "kd_loss": 0.7518777136654535,
+      "learning_rate": 3e-06,
+      "loss": 0.7765,
+      "masked_tokens": 109.7875,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 109.7875
+    },
+    {
+      "avg_mask_ratio": 0.4756378314923495,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.4756378314923495,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9856,
+      "grad_norm": 0.17578125,
+      "kd_loss": 0.6069021332950797,
+      "learning_rate": 3e-06,
+      "loss": 0.6412,
+      "masked_tokens": 119.8125,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 119.8125
+    },
+    {
+      "avg_mask_ratio": 0.5000089073021498,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5000089073021498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.7108402555168141,
+      "learning_rate": 3e-06,
+      "loss": 0.7752,
+      "masked_tokens": 121.98809523809524,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 121.98809523809524
+    },
+    {
+      "avg_mask_ratio": 0.4385950181807857,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.4385950181807857,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.2158203125,
+      "kd_loss": 0.652561020819121,
+      "learning_rate": 3e-06,
+      "loss": 0.6515,
+      "masked_tokens": 107.025,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 107.025
+    },
+    {
+      "avg_mask_ratio": 0.49335200865752993,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.49335200865752993,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0512,
+      "grad_norm": 0.2021484375,
+      "kd_loss": 0.8505055773921185,
+      "learning_rate": 3e-06,
+      "loss": 0.8085,
+      "masked_tokens": 141.5,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 141.5
+    },
+    {
+      "avg_mask_ratio": 0.4434558014094364,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4434558014094364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.564091070059294,
+      "learning_rate": 3e-06,
+      "loss": 0.5634,
+      "masked_tokens": 107.3,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 107.3
+    },
+    {
+      "avg_mask_ratio": 0.42445047190412877,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.42445047190412877,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.5745988191337801,
+      "learning_rate": 3e-06,
+      "loss": 0.5537,
+      "masked_tokens": 98.0875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 98.0875
+    },
+    {
+      "avg_mask_ratio": 0.4553717203787528,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.4553717203787528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1152,
+      "grad_norm": 0.14453125,
+      "kd_loss": 0.5502215233629159,
+      "learning_rate": 3e-06,
+      "loss": 0.5637,
+      "masked_tokens": 118.75,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 118.75
+    },
+    {
+      "avg_mask_ratio": 0.4424907588865608,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4424907588865608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5585700995494335,
+      "learning_rate": 3e-06,
+      "loss": 0.5795,
+      "masked_tokens": 98.9125,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 98.9125
+    },
+    {
+      "avg_mask_ratio": 0.43784486907534304,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.43784486907534304,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.111328125,
+      "kd_loss": 0.5065722677004316,
+      "learning_rate": 3e-06,
+      "loss": 0.5536,
+      "masked_tokens": 89.1625,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 89.1625
+    },
+    {
+      "avg_mask_ratio": 0.5077433207072317,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5077433207072317,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1792,
+      "grad_norm": 0.134765625,
+      "kd_loss": 0.7864730670612972,
+      "learning_rate": 3e-06,
+      "loss": 0.7616,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4700460441526957,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.4700460441526957,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.7690424034725968,
+      "learning_rate": 3e-06,
+      "loss": 0.7476,
+      "masked_tokens": 105.35,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 105.35
+    },
+    {
+      "avg_mask_ratio": 0.49483722454169765,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.49483722454169765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.7194663421146288,
+      "learning_rate": 3e-06,
+      "loss": 0.7147,
+      "masked_tokens": 99.85,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 99.85
+    },
+    {
+      "avg_mask_ratio": 0.515152573143132,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.515152573143132,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2432,
+      "grad_norm": 0.3046875,
+      "kd_loss": 0.8700408121391092,
+      "learning_rate": 3e-06,
+      "loss": 0.7471,
+      "masked_tokens": 122.6125,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 122.6125
+    },
+    {
+      "avg_mask_ratio": 0.4577924039273057,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.4577924039273057,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.565913611318291,
+      "learning_rate": 3e-06,
+      "loss": 0.5762,
+      "masked_tokens": 121.45,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 121.45
+    },
+    {
+      "avg_mask_ratio": 0.47600019703386354,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.47600019703386354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.5862566602218522,
+      "learning_rate": 3e-06,
+      "loss": 0.6313,
+      "masked_tokens": 134.6,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 134.6
+    },
+    {
+      "avg_mask_ratio": 0.4636200438835658,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4636200438835658,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3072,
+      "grad_norm": 0.12060546875,
+      "kd_loss": 0.5776349333416817,
+      "learning_rate": 3e-06,
+      "loss": 0.5714,
+      "masked_tokens": 97.875,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 97.875
+    },
+    {
+      "avg_mask_ratio": 0.43436883635586127,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.43436883635586127,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.11328125,
+      "kd_loss": 0.5290679484073877,
+      "learning_rate": 3e-06,
+      "loss": 0.5147,
+      "masked_tokens": 90.375,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 90.375
+    },
+    {
+      "avg_mask_ratio": 0.5262026051990688,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.5262026051990688,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.322265625,
+      "kd_loss": 0.662759238135186,
+      "learning_rate": 3e-06,
+      "loss": 0.7141,
+      "masked_tokens": 133.7375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 133.7375
+    },
+    {
+      "avg_mask_ratio": 0.44306369486730546,
+      "avg_response_length": 226.6375,
+      "avg_student_mask_ratio": 0.44306369486730546,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3712,
+      "grad_norm": 0.6484375,
+      "kd_loss": 0.42938064339505216,
+      "learning_rate": 3e-06,
+      "loss": 0.4673,
+      "masked_tokens": 102.675,
+      "mean_t": 0.4757364276825683,
+      "step": 1110,
+      "student_masked_tokens": 102.675
+    },
+    {
+      "avg_mask_ratio": 0.47274903577053917,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.47274903577053917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.392533333333333,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.5081101807423011,
+      "learning_rate": 3e-06,
+      "loss": 0.5923,
+      "masked_tokens": 112.65,
+      "mean_t": 0.5013068238971755,
+      "step": 1120,
+      "student_masked_tokens": 112.65
+    },
+    {
+      "avg_mask_ratio": 0.5020543755264952,
+      "avg_response_length": 262.25,
+      "avg_student_mask_ratio": 0.5020543755264952,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.413866666666667,
+      "grad_norm": 0.431640625,
+      "kd_loss": 0.7412553383448994,
+      "learning_rate": 3e-06,
+      "loss": 0.7351,
+      "masked_tokens": 135.7,
+      "mean_t": 0.5303254407714121,
+      "step": 1130,
+      "student_masked_tokens": 135.7
+    },
+    {
+      "avg_mask_ratio": 0.46305053916294125,
+      "avg_response_length": 217.05,
+      "avg_student_mask_ratio": 0.46305053916294125,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4352,
+      "grad_norm": 0.49609375,
+      "kd_loss": 0.5668171994801924,
+      "learning_rate": 3e-06,
+      "loss": 0.5938,
+      "masked_tokens": 103.45,
+      "mean_t": 0.4845335395424627,
+      "step": 1140,
+      "student_masked_tokens": 103.45
+    },
+    {
+      "avg_mask_ratio": 0.5285213771509006,
+      "avg_response_length": 212.5375,
+      "avg_student_mask_ratio": 0.5285213771509006,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4565333333333332,
+      "grad_norm": 0.30078125,
+      "kd_loss": 0.7812401379895164,
+      "learning_rate": 3e-06,
+      "loss": 0.8148,
+      "masked_tokens": 109.75,
+      "mean_t": 0.5690932425903157,
+      "step": 1150,
+      "student_masked_tokens": 109.75
+    },
+    {
+      "avg_mask_ratio": 0.46531253699213265,
+      "avg_response_length": 225.6125,
+      "avg_student_mask_ratio": 0.46531253699213265,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4778666666666664,
+      "grad_norm": 0.2578125,
+      "kd_loss": 0.5916059602122914,
+      "learning_rate": 3e-06,
+      "loss": 0.5667,
+      "masked_tokens": 101.65,
+      "mean_t": 0.5040684466948733,
+      "step": 1160,
+      "student_masked_tokens": 101.65
+    },
+    {
+      "avg_mask_ratio": 0.48369049807661213,
+      "avg_response_length": 247.7625,
+      "avg_student_mask_ratio": 0.48369049807661213,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4992,
+      "grad_norm": 0.232421875,
+      "kd_loss": 0.6494532859223682,
+      "learning_rate": 3e-06,
+      "loss": 0.691,
+      "masked_tokens": 123.7625,
+      "mean_t": 0.5114516971167177,
+      "step": 1170,
+      "student_masked_tokens": 123.7625
+    },
+    {
+      "avg_mask_ratio": 0.4209623454895336,
+      "avg_response_length": 214.725,
+      "avg_student_mask_ratio": 0.4209623454895336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.5205333333333333,
+      "grad_norm": 0.267578125,
+      "kd_loss": 0.5382059471070534,
+      "learning_rate": 3e-06,
+      "loss": 0.5929,
+      "masked_tokens": 85.35,
+      "mean_t": 0.4491677140351385,
+      "step": 1180,
+      "student_masked_tokens": 85.35
+    },
+    {
+      "avg_mask_ratio": 0.5209485858562403,
+      "avg_response_length": 220.3375,
+      "avg_student_mask_ratio": 0.5209485858562403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.5418666666666665,
+      "grad_norm": 0.5078125,
+      "kd_loss": 0.9205318759757916,
+      "learning_rate": 3e-06,
+      "loss": 0.9099,
+      "masked_tokens": 124.3375,
+      "mean_t": 0.5590635397238657,
+      "step": 1190,
+      "student_masked_tokens": 124.3375
+    },
+    {
+      "avg_mask_ratio": 0.47338692427438217,
+      "avg_response_length": 215.675,
+      "avg_student_mask_ratio": 0.47338692427438217,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.5632,
+      "grad_norm": 0.29296875,
+      "kd_loss": 0.598471246479835,
+      "learning_rate": 3e-06,
+      "loss": 0.6085,
+      "masked_tokens": 99.6125,
+      "mean_t": 0.506370971655997,
+      "step": 1200,
+      "student_masked_tokens": 99.6125
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2611bced1337375831cc344da28d849ea16c0f2f5a6aeca004865b84ca7216e0
+size 7992

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "v_proj",
+    "k_proj",
+    "down_proj",
+    "up_proj",
+    "q_proj",
+    "gate_proj",
+    "o_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:763c2385c94712e8ce11b4a0c0e79d48476efbae4ef4340fc2cf5d8d2ecbad58
+size 2406624648

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e05e0cf7c3e6e22d45e8510bbb5cbd36bfa01bac144ef64139feae96a18e2d1
+size 671304442

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ee1b2ec0b2aff8a8fd97fb1bfda21d46200ad13359eed2946e8811381e785529
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a37caa81ca3b37bcfae996e60a1ec762e4a75d1e14211368b059806d5e594b2b
+size 14512

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:368502bb8b2f2d6f452bda7249e88ca57b330ec2ba407ec248613b4300c99d0d
+size 1064

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/trainer_state.json

@@ -0,0 +1,3153 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.776533333333333,
+  "eval_steps": 500,
+  "global_step": 1300,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.49314909732202067,
+      "avg_response_length": 225.725,
+      "avg_student_mask_ratio": 0.49314909732202067,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.8524778203239294,
+      "learning_rate": 3e-06,
+      "loss": 0.8897,
+      "masked_tokens": 111.775,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 111.775
+    },
+    {
+      "avg_mask_ratio": 0.42081435878062623,
+      "avg_response_length": 251.6,
+      "avg_student_mask_ratio": 0.42081435878062623,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.353515625,
+      "kd_loss": 0.6601746417798836,
+      "learning_rate": 3e-06,
+      "loss": 0.6494,
+      "masked_tokens": 94.7,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.7
+    },
+    {
+      "avg_mask_ratio": 0.4538542575784959,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4538542575784959,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.064,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.7305509156625248,
+      "learning_rate": 3e-06,
+      "loss": 0.6735,
+      "masked_tokens": 110.2,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.42071006570186,
+      "avg_response_length": 218.825,
+      "avg_student_mask_ratio": 0.42071006570186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.6237847281259746,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 94.5125,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.5125
+    },
+    {
+      "avg_mask_ratio": 0.43260439952719026,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.43260439952719026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.6509197178371323,
+      "learning_rate": 3e-06,
+      "loss": 0.6726,
+      "masked_tokens": 85.05,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.05
+    },
+    {
+      "avg_mask_ratio": 0.4602369442814961,
+      "avg_response_length": 248.3,
+      "avg_student_mask_ratio": 0.4602369442814961,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.128,
+      "grad_norm": 0.1455078125,
+      "kd_loss": 0.6954449985298851,
+      "learning_rate": 3e-06,
+      "loss": 0.7222,
+      "masked_tokens": 119.8625,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.8625
+    },
+    {
+      "avg_mask_ratio": 0.5111442144960165,
+      "avg_response_length": 201.6375,
+      "avg_student_mask_ratio": 0.5111442144960165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.359375,
+      "kd_loss": 0.7338119231904784,
+      "learning_rate": 3e-06,
+      "loss": 0.837,
+      "masked_tokens": 104.325,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.37548826879356056,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.37548826879356056,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.5029199755957052,
+      "learning_rate": 3e-06,
+      "loss": 0.536,
+      "masked_tokens": 85.0625,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 85.0625
+    },
+    {
+      "avg_mask_ratio": 0.4998963106656447,
+      "avg_response_length": 222.7,
+      "avg_student_mask_ratio": 0.4998963106656447,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.192,
+      "grad_norm": 0.3359375,
+      "kd_loss": 0.8495752438985221,
+      "learning_rate": 3e-06,
+      "loss": 0.8143,
+      "masked_tokens": 106.025,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 106.025
+    },
+    {
+      "avg_mask_ratio": 0.457616981890169,
+      "avg_response_length": 207.2,
+      "avg_student_mask_ratio": 0.457616981890169,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.6096036813853857,
+      "learning_rate": 3e-06,
+      "loss": 0.6649,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 103.3875
+    },
+    {
+      "avg_mask_ratio": 0.48775083787040785,
+      "avg_response_length": 212.225,
+      "avg_student_mask_ratio": 0.48775083787040785,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.380859375,
+      "kd_loss": 0.8403033299919912,
+      "learning_rate": 3e-06,
+      "loss": 0.8646,
+      "masked_tokens": 101.725,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 101.725
+    },
+    {
+      "avg_mask_ratio": 0.4505724890856072,
+      "avg_response_length": 220.6375,
+      "avg_student_mask_ratio": 0.4505724890856072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.256,
+      "grad_norm": 0.2734375,
+      "kd_loss": 0.7449374303294463,
+      "learning_rate": 3e-06,
+      "loss": 0.6985,
+      "masked_tokens": 97.3125,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 97.3125
+    },
+    {
+      "avg_mask_ratio": 0.43574556948733517,
+      "avg_response_length": 201.8125,
+      "avg_student_mask_ratio": 0.43574556948733517,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.392578125,
+      "kd_loss": 0.4566404212248244,
+      "learning_rate": 3e-06,
+      "loss": 0.4885,
+      "masked_tokens": 89.325,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 89.325
+    },
+    {
+      "avg_mask_ratio": 0.49070783695206044,
+      "avg_response_length": 249.175,
+      "avg_student_mask_ratio": 0.49070783695206044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.474609375,
+      "kd_loss": 0.6905622516694165,
+      "learning_rate": 3e-06,
+      "loss": 0.7061,
+      "masked_tokens": 130.2625,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 130.2625
+    },
+    {
+      "avg_mask_ratio": 0.4736109555349685,
+      "avg_response_length": 254.6875,
+      "avg_student_mask_ratio": 0.4736109555349685,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.32,
+      "grad_norm": 0.08056640625,
+      "kd_loss": 0.7106425042155138,
+      "learning_rate": 3e-06,
+      "loss": 0.7643,
+      "masked_tokens": 126.2,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 126.2
+    },
+    {
+      "avg_mask_ratio": 0.5032523009285796,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.5032523009285796,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.408203125,
+      "kd_loss": 0.8267615023827602,
+      "learning_rate": 3e-06,
+      "loss": 0.8409,
+      "masked_tokens": 99.425,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 99.425
+    },
+    {
+      "avg_mask_ratio": 0.4807244518771768,
+      "avg_response_length": 184.65,
+      "avg_student_mask_ratio": 0.4807244518771768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.7329186423268765,
+      "learning_rate": 3e-06,
+      "loss": 0.6767,
+      "masked_tokens": 88.225,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 88.225
+    },
+    {
+      "avg_mask_ratio": 0.45808705932286103,
+      "avg_response_length": 230.875,
+      "avg_student_mask_ratio": 0.45808705932286103,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.384,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5571840626302332,
+      "learning_rate": 3e-06,
+      "loss": 0.5724,
+      "masked_tokens": 105.8625,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 105.8625
+    },
+    {
+      "avg_mask_ratio": 0.4842760307248682,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.4842760307248682,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 0.7421875,
+      "kd_loss": 0.6595436891710733,
+      "learning_rate": 3e-06,
+      "loss": 0.6415,
+      "masked_tokens": 109.5875,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 109.5875
+    },
+    {
+      "avg_mask_ratio": 0.4665210062637925,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4665210062637925,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.5610411517063085,
+      "learning_rate": 3e-06,
+      "loss": 0.5866,
+      "masked_tokens": 97.0125,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 97.0125
+    },
+    {
+      "avg_mask_ratio": 0.45183838629163803,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.45183838629163803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.448,
+      "grad_norm": 0.29296875,
+      "kd_loss": 0.5228309052588884,
+      "learning_rate": 3e-06,
+      "loss": 0.5326,
+      "masked_tokens": 96.0125,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 96.0125
+    },
+    {
+      "avg_mask_ratio": 0.5396745702251792,
+      "avg_response_length": 234.05,
+      "avg_student_mask_ratio": 0.5396745702251792,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.7454893081907358,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 124.0125,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 124.0125
+    },
+    {
+      "avg_mask_ratio": 0.4404038088978268,
+      "avg_response_length": 252.025,
+      "avg_student_mask_ratio": 0.4404038088978268,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.5848532651940331,
+      "learning_rate": 3e-06,
+      "loss": 0.6239,
+      "masked_tokens": 107.1875,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 107.1875
+    },
+    {
+      "avg_mask_ratio": 0.47433349161874505,
+      "avg_response_length": 211.175,
+      "avg_student_mask_ratio": 0.47433349161874505,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.512,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.7448974770184804,
+      "learning_rate": 3e-06,
+      "loss": 0.7729,
+      "masked_tokens": 106.275,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 106.275
+    },
+    {
+      "avg_mask_ratio": 0.4465038388385437,
+      "avg_response_length": 243.225,
+      "avg_student_mask_ratio": 0.4465038388385437,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.1328125,
+      "kd_loss": 0.5912455287105786,
+      "learning_rate": 3e-06,
+      "loss": 0.5836,
+      "masked_tokens": 112.725,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 112.725
+    },
+    {
+      "avg_mask_ratio": 0.45215590375009923,
+      "avg_response_length": 244.7375,
+      "avg_student_mask_ratio": 0.45215590375009923,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.2451171875,
+      "kd_loss": 0.5663141592655847,
+      "learning_rate": 3e-06,
+      "loss": 0.531,
+      "masked_tokens": 109.1,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 109.1
+    },
+    {
+      "avg_mask_ratio": 0.5305180630879477,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.5305180630879477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.576,
+      "grad_norm": 0.4453125,
+      "kd_loss": 0.8847639085633545,
+      "learning_rate": 3e-06,
+      "loss": 0.9006,
+      "masked_tokens": 120.475,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 120.475
+    },
+    {
+      "avg_mask_ratio": 0.4644554515252821,
+      "avg_response_length": 250.75,
+      "avg_student_mask_ratio": 0.4644554515252821,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.1474609375,
+      "kd_loss": 0.6207698138765181,
+      "learning_rate": 3e-06,
+      "loss": 0.5834,
+      "masked_tokens": 126.2125,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 126.2125
+    },
+    {
+      "avg_mask_ratio": 0.3939544238965027,
+      "avg_response_length": 238.475,
+      "avg_student_mask_ratio": 0.3939544238965027,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.138671875,
+      "kd_loss": 0.483729143582201,
+      "learning_rate": 3e-06,
+      "loss": 0.471,
+      "masked_tokens": 96.975,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 96.975
+    },
+    {
+      "avg_mask_ratio": 0.4442457370925695,
+      "avg_response_length": 213.4125,
+      "avg_student_mask_ratio": 0.4442457370925695,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.64,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.5351485910949123,
+      "learning_rate": 3e-06,
+      "loss": 0.607,
+      "masked_tokens": 101.525,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 101.525
+    },
+    {
+      "avg_mask_ratio": 0.48191855432232844,
+      "avg_response_length": 175.3375,
+      "avg_student_mask_ratio": 0.48191855432232844,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.2890625,
+      "kd_loss": 0.7135523370927593,
+      "learning_rate": 3e-06,
+      "loss": 0.695,
+      "masked_tokens": 84.875,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 84.875
+    },
+    {
+      "avg_mask_ratio": 0.49743892522528765,
+      "avg_response_length": 232.85,
+      "avg_student_mask_ratio": 0.49743892522528765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.7042804263782273,
+      "learning_rate": 3e-06,
+      "loss": 0.7858,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.511213600449264,
+      "avg_response_length": 234.3625,
+      "avg_student_mask_ratio": 0.511213600449264,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.704,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.6977765330942021,
+      "learning_rate": 3e-06,
+      "loss": 0.7303,
+      "masked_tokens": 128.1,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 128.1
+    },
+    {
+      "avg_mask_ratio": 0.44370225080056114,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.44370225080056114,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.4375,
+      "kd_loss": 0.714193849134108,
+      "learning_rate": 3e-06,
+      "loss": 0.6445,
+      "masked_tokens": 108.775,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 108.775
+    },
+    {
+      "avg_mask_ratio": 0.49959173843380994,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.49959173843380994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.7770639997791452,
+      "learning_rate": 3e-06,
+      "loss": 0.7806,
+      "masked_tokens": 125.625,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 125.625
+    },
+    {
+      "avg_mask_ratio": 0.47441557042766364,
+      "avg_response_length": 241.6875,
+      "avg_student_mask_ratio": 0.47441557042766364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.768,
+      "grad_norm": 0.15234375,
+      "kd_loss": 0.762115379356667,
+      "learning_rate": 3e-06,
+      "loss": 0.7428,
+      "masked_tokens": 114.3375,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 114.3375
+    },
+    {
+      "avg_mask_ratio": 0.4738660019589588,
+      "avg_response_length": 254.05,
+      "avg_student_mask_ratio": 0.4738660019589588,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.0859375,
+      "kd_loss": 0.7563776895753108,
+      "learning_rate": 3e-06,
+      "loss": 0.6971,
+      "masked_tokens": 126.3875,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 126.3875
+    },
+    {
+      "avg_mask_ratio": 0.4487657839199528,
+      "avg_response_length": 203.9875,
+      "avg_student_mask_ratio": 0.4487657839199528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.259765625,
+      "kd_loss": 0.5462394427451727,
+      "learning_rate": 3e-06,
+      "loss": 0.4745,
+      "masked_tokens": 96.725,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 96.725
+    },
+    {
+      "avg_mask_ratio": 0.4751023279619403,
+      "avg_response_length": 217.0,
+      "avg_student_mask_ratio": 0.4751023279619403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.832,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.685858413620528,
+      "learning_rate": 3e-06,
+      "loss": 0.6922,
+      "masked_tokens": 110.5625,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 110.5625
+    },
+    {
+      "avg_mask_ratio": 0.4746393185690977,
+      "avg_response_length": 242.5125,
+      "avg_student_mask_ratio": 0.4746393185690977,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.7327228688028754,
+      "learning_rate": 3e-06,
+      "loss": 0.7636,
+      "masked_tokens": 120.3875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 120.3875
+    },
+    {
+      "avg_mask_ratio": 0.4831854701798875,
+      "avg_response_length": 227.6375,
+      "avg_student_mask_ratio": 0.4831854701798875,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.265625,
+      "kd_loss": 0.8579687503222686,
+      "learning_rate": 3e-06,
+      "loss": 0.7988,
+      "masked_tokens": 118.825,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 118.825
+    },
+    {
+      "avg_mask_ratio": 0.4379329536575824,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4379329536575824,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.896,
+      "grad_norm": 0.21875,
+      "kd_loss": 0.45460584598010884,
+      "learning_rate": 3e-06,
+      "loss": 0.4832,
+      "masked_tokens": 85.525,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 85.525
+    },
+    {
+      "avg_mask_ratio": 0.4674084897618741,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4674084897618741,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.2216796875,
+      "kd_loss": 0.653534199286878,
+      "learning_rate": 3e-06,
+      "loss": 0.6388,
+      "masked_tokens": 104.5625,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 104.5625
+    },
+    {
+      "avg_mask_ratio": 0.44310826995642855,
+      "avg_response_length": 233.2,
+      "avg_student_mask_ratio": 0.44310826995642855,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.6742433619177973,
+      "learning_rate": 3e-06,
+      "loss": 0.6468,
+      "masked_tokens": 106.1875,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 106.1875
+    },
+    {
+      "avg_mask_ratio": 0.4248724762815982,
+      "avg_response_length": 230.75,
+      "avg_student_mask_ratio": 0.4248724762815982,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.96,
+      "grad_norm": 0.25390625,
+      "kd_loss": 0.6201432387717432,
+      "learning_rate": 3e-06,
+      "loss": 0.6546,
+      "masked_tokens": 102.7375,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 102.7375
+    },
+    {
+      "avg_mask_ratio": 0.47578654896933587,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.47578654896933587,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.16015625,
+      "kd_loss": 0.6602298976420571,
+      "learning_rate": 3e-06,
+      "loss": 0.6259,
+      "masked_tokens": 105.3125,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 105.3125
+    },
+    {
+      "avg_mask_ratio": 0.4781565316176663,
+      "avg_response_length": 223.33333333333334,
+      "avg_student_mask_ratio": 0.4781565316176663,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.361328125,
+      "kd_loss": 0.7762555406697461,
+      "learning_rate": 3e-06,
+      "loss": 0.7824,
+      "masked_tokens": 106.6547619047619,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 106.6547619047619
+    },
+    {
+      "avg_mask_ratio": 0.4757596247596666,
+      "avg_response_length": 249.1875,
+      "avg_student_mask_ratio": 0.4757596247596666,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0256,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7768732401565984,
+      "learning_rate": 3e-06,
+      "loss": 0.7435,
+      "masked_tokens": 118.225,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 118.225
+    },
+    {
+      "avg_mask_ratio": 0.4564065972692333,
+      "avg_response_length": 228.025,
+      "avg_student_mask_ratio": 0.4564065972692333,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.6187377570489844,
+      "learning_rate": 3e-06,
+      "loss": 0.5998,
+      "masked_tokens": 97.6375,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 97.6375
+    },
+    {
+      "avg_mask_ratio": 0.5074894667137414,
+      "avg_response_length": 244.5625,
+      "avg_student_mask_ratio": 0.5074894667137414,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.712813160381853,
+      "learning_rate": 3e-06,
+      "loss": 0.7089,
+      "masked_tokens": 122.9,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 122.9
+    },
+    {
+      "avg_mask_ratio": 0.5111216343124397,
+      "avg_response_length": 245.5125,
+      "avg_student_mask_ratio": 0.5111216343124397,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.0896,
+      "grad_norm": 0.30078125,
+      "kd_loss": 0.7725131734614138,
+      "learning_rate": 3e-06,
+      "loss": 0.7319,
+      "masked_tokens": 132.4625,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 132.4625
+    },
+    {
+      "avg_mask_ratio": 0.45396183808334173,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.45396183808334173,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.396484375,
+      "kd_loss": 0.6740453685139528,
+      "learning_rate": 3e-06,
+      "loss": 0.6974,
+      "masked_tokens": 87.0625,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 87.0625
+    },
+    {
+      "avg_mask_ratio": 0.4701436776900664,
+      "avg_response_length": 212.7125,
+      "avg_student_mask_ratio": 0.4701436776900664,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.37890625,
+      "kd_loss": 0.7894933511876161,
+      "learning_rate": 3e-06,
+      "loss": 0.8161,
+      "masked_tokens": 99.2875,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 99.2875
+    },
+    {
+      "avg_mask_ratio": 0.48931564125232396,
+      "avg_response_length": 231.2625,
+      "avg_student_mask_ratio": 0.48931564125232396,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1536,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.8131277295872679,
+      "learning_rate": 3e-06,
+      "loss": 0.7558,
+      "masked_tokens": 107.3625,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 107.3625
+    },
+    {
+      "avg_mask_ratio": 0.4541942774085328,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4541942774085328,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.1767578125,
+      "kd_loss": 0.5594845790216368,
+      "learning_rate": 3e-06,
+      "loss": 0.5624,
+      "masked_tokens": 97.8875,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 97.8875
+    },
+    {
+      "avg_mask_ratio": 0.3928302472108044,
+      "avg_response_length": 230.5125,
+      "avg_student_mask_ratio": 0.3928302472108044,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.4609375,
+      "kd_loss": 0.6713891216638501,
+      "learning_rate": 3e-06,
+      "loss": 0.6598,
+      "masked_tokens": 99.175,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 99.175
+    },
+    {
+      "avg_mask_ratio": 0.44847587069962175,
+      "avg_response_length": 229.2875,
+      "avg_student_mask_ratio": 0.44847587069962175,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2176,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.6333371287628324,
+      "learning_rate": 3e-06,
+      "loss": 0.6153,
+      "masked_tokens": 108.375,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 108.375
+    },
+    {
+      "avg_mask_ratio": 0.4655611292924732,
+      "avg_response_length": 264.6375,
+      "avg_student_mask_ratio": 0.4655611292924732,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.7399741232418364,
+      "learning_rate": 3e-06,
+      "loss": 0.7193,
+      "masked_tokens": 127.4125,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 127.4125
+    },
+    {
+      "avg_mask_ratio": 0.5226378521183506,
+      "avg_response_length": 227.8875,
+      "avg_student_mask_ratio": 0.5226378521183506,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.279296875,
+      "kd_loss": 0.7912517642864259,
+      "learning_rate": 3e-06,
+      "loss": 0.7556,
+      "masked_tokens": 121.5375,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 121.5375
+    },
+    {
+      "avg_mask_ratio": 0.45001045021926983,
+      "avg_response_length": 235.8125,
+      "avg_student_mask_ratio": 0.45001045021926983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.2816,
+      "grad_norm": 0.2333984375,
+      "kd_loss": 0.5968554831010096,
+      "learning_rate": 3e-06,
+      "loss": 0.622,
+      "masked_tokens": 109.9,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 109.9
+    },
+    {
+      "avg_mask_ratio": 0.5040990226087161,
+      "avg_response_length": 226.9125,
+      "avg_student_mask_ratio": 0.5040990226087161,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.1435546875,
+      "kd_loss": 0.7809002916160352,
+      "learning_rate": 3e-06,
+      "loss": 0.8354,
+      "masked_tokens": 110.375,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 110.375
+    },
+    {
+      "avg_mask_ratio": 0.4963218850665726,
+      "avg_response_length": 231.7,
+      "avg_student_mask_ratio": 0.4963218850665726,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.19921875,
+      "kd_loss": 0.7623143148503232,
+      "learning_rate": 3e-06,
+      "loss": 0.7637,
+      "masked_tokens": 110.675,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 110.675
+    },
+    {
+      "avg_mask_ratio": 0.5259752279380336,
+      "avg_response_length": 249.325,
+      "avg_student_mask_ratio": 0.5259752279380336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 0.55859375,
+      "kd_loss": 0.9169201381674611,
+      "learning_rate": 3e-06,
+      "loss": 0.9908,
+      "masked_tokens": 125.525,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 125.525
+    },
+    {
+      "avg_mask_ratio": 0.49199640220031143,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.49199640220031143,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.658431020938042,
+      "learning_rate": 3e-06,
+      "loss": 0.6105,
+      "masked_tokens": 121.8625,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 121.8625
+    },
+    {
+      "avg_mask_ratio": 0.5331356228562072,
+      "avg_response_length": 230.15,
+      "avg_student_mask_ratio": 0.5331356228562072,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.205078125,
+      "kd_loss": 0.8636023813149223,
+      "learning_rate": 3e-06,
+      "loss": 0.7956,
+      "masked_tokens": 128.65,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 128.65
+    },
+    {
+      "avg_mask_ratio": 0.4887094351579435,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.4887094351579435,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4096,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.7148105578169635,
+      "learning_rate": 3e-06,
+      "loss": 0.7531,
+      "masked_tokens": 129.8,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 129.8
+    },
+    {
+      "avg_mask_ratio": 0.46143888669321315,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.46143888669321315,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.451171875,
+      "kd_loss": 0.6032656382115874,
+      "learning_rate": 3e-06,
+      "loss": 0.6039,
+      "masked_tokens": 118.7375,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 118.7375
+    },
+    {
+      "avg_mask_ratio": 0.46107456521131096,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.46107456521131096,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.314453125,
+      "kd_loss": 0.6940350630834473,
+      "learning_rate": 3e-06,
+      "loss": 0.6841,
+      "masked_tokens": 106.4875,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 106.4875
+    },
+    {
+      "avg_mask_ratio": 0.4799915248062462,
+      "avg_response_length": 201.375,
+      "avg_student_mask_ratio": 0.4799915248062462,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4736,
+      "grad_norm": 0.189453125,
+      "kd_loss": 0.6958799545635429,
+      "learning_rate": 3e-06,
+      "loss": 0.684,
+      "masked_tokens": 111.5375,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 111.5375
+    },
+    {
+      "avg_mask_ratio": 0.5273691712645814,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5273691712645814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.9459125808391036,
+      "learning_rate": 3e-06,
+      "loss": 0.8879,
+      "masked_tokens": 132.525,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 132.525
+    },
+    {
+      "avg_mask_ratio": 0.48835311776492746,
+      "avg_response_length": 210.4875,
+      "avg_student_mask_ratio": 0.48835311776492746,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.1650390625,
+      "kd_loss": 0.6531627567595478,
+      "learning_rate": 3e-06,
+      "loss": 0.6323,
+      "masked_tokens": 108.125,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 108.125
+    },
+    {
+      "avg_mask_ratio": 0.48280774110462515,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.48280774110462515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.8419079009756387,
+      "learning_rate": 3e-06,
+      "loss": 0.7928,
+      "masked_tokens": 110.2,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 110.2
+    },
+    {
+      "avg_mask_ratio": 0.46588142022956164,
+      "avg_response_length": 227.0625,
+      "avg_student_mask_ratio": 0.46588142022956164,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.244140625,
+      "kd_loss": 0.6997778600025854,
+      "learning_rate": 3e-06,
+      "loss": 0.7623,
+      "masked_tokens": 103.3375,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 103.3375
+    },
+    {
+      "avg_mask_ratio": 0.5192104739835486,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5192104739835486,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.1708984375,
+      "kd_loss": 0.8950252983398969,
+      "learning_rate": 3e-06,
+      "loss": 0.8392,
+      "masked_tokens": 96.55,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 96.55
+    },
+    {
+      "avg_mask_ratio": 0.41821987080038525,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.41821987080038525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6016,
+      "grad_norm": 0.2099609375,
+      "kd_loss": 0.6870280907321394,
+      "learning_rate": 3e-06,
+      "loss": 0.6255,
+      "masked_tokens": 95.2,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 95.2
+    },
+    {
+      "avg_mask_ratio": 0.4688679567188956,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.4688679567188956,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 1.390625,
+      "kd_loss": 0.8148217237806421,
+      "learning_rate": 3e-06,
+      "loss": 0.7954,
+      "masked_tokens": 107.55,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 107.55
+    },
+    {
+      "avg_mask_ratio": 0.4429742675391026,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4429742675391026,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.1826171875,
+      "kd_loss": 0.5454595529316066,
+      "learning_rate": 3e-06,
+      "loss": 0.5506,
+      "masked_tokens": 117.825,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 117.825
+    },
+    {
+      "avg_mask_ratio": 0.43331936674949245,
+      "avg_response_length": 251.275,
+      "avg_student_mask_ratio": 0.43331936674949245,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6656,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.4808090373327854,
+      "learning_rate": 3e-06,
+      "loss": 0.5158,
+      "masked_tokens": 117.2,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 117.2
+    },
+    {
+      "avg_mask_ratio": 0.4646148854692001,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.4646148854692001,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.140625,
+      "kd_loss": 0.6250457387332972,
+      "learning_rate": 3e-06,
+      "loss": 0.6097,
+      "masked_tokens": 102.15,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 102.15
+    },
+    {
+      "avg_mask_ratio": 0.46852388482075186,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.46852388482075186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.1552734375,
+      "kd_loss": 0.5451920869360265,
+      "learning_rate": 3e-06,
+      "loss": 0.6197,
+      "masked_tokens": 105.6375,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 105.6375
+    },
+    {
+      "avg_mask_ratio": 0.5052226510597393,
+      "avg_response_length": 236.5625,
+      "avg_student_mask_ratio": 0.5052226510597393,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7296,
+      "grad_norm": 0.27734375,
+      "kd_loss": 0.7552159034145006,
+      "learning_rate": 3e-06,
+      "loss": 0.7562,
+      "masked_tokens": 119.5375,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 119.5375
+    },
+    {
+      "avg_mask_ratio": 0.5101147621171549,
+      "avg_response_length": 202.5625,
+      "avg_student_mask_ratio": 0.5101147621171549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.8518789664900283,
+      "learning_rate": 3e-06,
+      "loss": 0.851,
+      "masked_tokens": 117.525,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 117.525
+    },
+    {
+      "avg_mask_ratio": 0.4939760136883706,
+      "avg_response_length": 257.0125,
+      "avg_student_mask_ratio": 0.4939760136883706,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.150390625,
+      "kd_loss": 0.8243849514963074,
+      "learning_rate": 3e-06,
+      "loss": 0.7816,
+      "masked_tokens": 134.1625,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 134.1625
+    },
+    {
+      "avg_mask_ratio": 0.5015010182803963,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.5015010182803963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.7936,
+      "grad_norm": 0.1279296875,
+      "kd_loss": 0.6244754713046745,
+      "learning_rate": 3e-06,
+      "loss": 0.6667,
+      "masked_tokens": 114.4875,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 114.4875
+    },
+    {
+      "avg_mask_ratio": 0.5062767470255494,
+      "avg_response_length": 224.025,
+      "avg_student_mask_ratio": 0.5062767470255494,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.236328125,
+      "kd_loss": 0.6353196922647157,
+      "learning_rate": 3e-06,
+      "loss": 0.6607,
+      "masked_tokens": 115.7125,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 115.7125
+    },
+    {
+      "avg_mask_ratio": 0.4643575747264549,
+      "avg_response_length": 224.225,
+      "avg_student_mask_ratio": 0.4643575747264549,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.7141597053345781,
+      "learning_rate": 3e-06,
+      "loss": 0.6556,
+      "masked_tokens": 106.9375,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 106.9375
+    },
+    {
+      "avg_mask_ratio": 0.487393151212018,
+      "avg_response_length": 228.2625,
+      "avg_student_mask_ratio": 0.487393151212018,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.6699687715730555,
+      "learning_rate": 3e-06,
+      "loss": 0.787,
+      "masked_tokens": 114.8875,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 114.8875
+    },
+    {
+      "avg_mask_ratio": 0.45142438664333895,
+      "avg_response_length": 197.925,
+      "avg_student_mask_ratio": 0.45142438664333895,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.310546875,
+      "kd_loss": 0.7402904870072688,
+      "learning_rate": 3e-06,
+      "loss": 0.6762,
+      "masked_tokens": 92.25,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 92.25
+    },
+    {
+      "avg_mask_ratio": 0.4500263270718278,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.4500263270718278,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.0947265625,
+      "kd_loss": 0.5816223171608726,
+      "learning_rate": 3e-06,
+      "loss": 0.5512,
+      "masked_tokens": 102.8125,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 102.8125
+    },
+    {
+      "avg_mask_ratio": 0.4509512555785477,
+      "avg_response_length": 224.5375,
+      "avg_student_mask_ratio": 0.4509512555785477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9216,
+      "grad_norm": 0.234375,
+      "kd_loss": 0.6806100918192441,
+      "learning_rate": 3e-06,
+      "loss": 0.6938,
+      "masked_tokens": 102.025,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 102.025
+    },
+    {
+      "avg_mask_ratio": 0.47468002026434986,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.47468002026434986,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.185546875,
+      "kd_loss": 0.6469848818184118,
+      "learning_rate": 3e-06,
+      "loss": 0.6391,
+      "masked_tokens": 124.3,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 124.3
+    },
+    {
+      "avg_mask_ratio": 0.4673476937925443,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4673476937925443,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.357421875,
+      "kd_loss": 0.7518777136654535,
+      "learning_rate": 3e-06,
+      "loss": 0.7765,
+      "masked_tokens": 109.7875,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 109.7875
+    },
+    {
+      "avg_mask_ratio": 0.4756378314923495,
+      "avg_response_length": 236.45,
+      "avg_student_mask_ratio": 0.4756378314923495,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 1.9856,
+      "grad_norm": 0.17578125,
+      "kd_loss": 0.6069021332950797,
+      "learning_rate": 3e-06,
+      "loss": 0.6412,
+      "masked_tokens": 119.8125,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 119.8125
+    },
+    {
+      "avg_mask_ratio": 0.5000089073021498,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.5000089073021498,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.7108402555168141,
+      "learning_rate": 3e-06,
+      "loss": 0.7752,
+      "masked_tokens": 121.98809523809524,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 121.98809523809524
+    },
+    {
+      "avg_mask_ratio": 0.4385950181807857,
+      "avg_response_length": 232.1875,
+      "avg_student_mask_ratio": 0.4385950181807857,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.2158203125,
+      "kd_loss": 0.652561020819121,
+      "learning_rate": 3e-06,
+      "loss": 0.6515,
+      "masked_tokens": 107.025,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 107.025
+    },
+    {
+      "avg_mask_ratio": 0.49335200865752993,
+      "avg_response_length": 273.6625,
+      "avg_student_mask_ratio": 0.49335200865752993,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0512,
+      "grad_norm": 0.2021484375,
+      "kd_loss": 0.8505055773921185,
+      "learning_rate": 3e-06,
+      "loss": 0.8085,
+      "masked_tokens": 141.5,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 141.5
+    },
+    {
+      "avg_mask_ratio": 0.4434558014094364,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4434558014094364,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.341796875,
+      "kd_loss": 0.564091070059294,
+      "learning_rate": 3e-06,
+      "loss": 0.5634,
+      "masked_tokens": 107.3,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 107.3
+    },
+    {
+      "avg_mask_ratio": 0.42445047190412877,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.42445047190412877,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.5745988191337801,
+      "learning_rate": 3e-06,
+      "loss": 0.5537,
+      "masked_tokens": 98.0875,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 98.0875
+    },
+    {
+      "avg_mask_ratio": 0.4553717203787528,
+      "avg_response_length": 256.175,
+      "avg_student_mask_ratio": 0.4553717203787528,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1152,
+      "grad_norm": 0.14453125,
+      "kd_loss": 0.5502215233629159,
+      "learning_rate": 3e-06,
+      "loss": 0.5637,
+      "masked_tokens": 118.75,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 118.75
+    },
+    {
+      "avg_mask_ratio": 0.4424907588865608,
+      "avg_response_length": 214.875,
+      "avg_student_mask_ratio": 0.4424907588865608,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5585700995494335,
+      "learning_rate": 3e-06,
+      "loss": 0.5795,
+      "masked_tokens": 98.9125,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 98.9125
+    },
+    {
+      "avg_mask_ratio": 0.43784486907534304,
+      "avg_response_length": 213.175,
+      "avg_student_mask_ratio": 0.43784486907534304,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.111328125,
+      "kd_loss": 0.5065722677004316,
+      "learning_rate": 3e-06,
+      "loss": 0.5536,
+      "masked_tokens": 89.1625,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 89.1625
+    },
+    {
+      "avg_mask_ratio": 0.5077433207072317,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5077433207072317,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.1792,
+      "grad_norm": 0.134765625,
+      "kd_loss": 0.7864730670612972,
+      "learning_rate": 3e-06,
+      "loss": 0.7616,
+      "masked_tokens": 110.475,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 110.475
+    },
+    {
+      "avg_mask_ratio": 0.4700460441526957,
+      "avg_response_length": 219.175,
+      "avg_student_mask_ratio": 0.4700460441526957,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.7690424034725968,
+      "learning_rate": 3e-06,
+      "loss": 0.7476,
+      "masked_tokens": 105.35,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 105.35
+    },
+    {
+      "avg_mask_ratio": 0.49483722454169765,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.49483722454169765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.7194663421146288,
+      "learning_rate": 3e-06,
+      "loss": 0.7147,
+      "masked_tokens": 99.85,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 99.85
+    },
+    {
+      "avg_mask_ratio": 0.515152573143132,
+      "avg_response_length": 237.875,
+      "avg_student_mask_ratio": 0.515152573143132,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2432,
+      "grad_norm": 0.3046875,
+      "kd_loss": 0.8700408121391092,
+      "learning_rate": 3e-06,
+      "loss": 0.7471,
+      "masked_tokens": 122.6125,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 122.6125
+    },
+    {
+      "avg_mask_ratio": 0.4577924039273057,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.4577924039273057,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.4140625,
+      "kd_loss": 0.565913611318291,
+      "learning_rate": 3e-06,
+      "loss": 0.5762,
+      "masked_tokens": 121.45,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 121.45
+    },
+    {
+      "avg_mask_ratio": 0.47600019703386354,
+      "avg_response_length": 253.8375,
+      "avg_student_mask_ratio": 0.47600019703386354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.5862566602218522,
+      "learning_rate": 3e-06,
+      "loss": 0.6313,
+      "masked_tokens": 134.6,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 134.6
+    },
+    {
+      "avg_mask_ratio": 0.4636200438835658,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4636200438835658,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3072,
+      "grad_norm": 0.12060546875,
+      "kd_loss": 0.5776349333416817,
+      "learning_rate": 3e-06,
+      "loss": 0.5714,
+      "masked_tokens": 97.875,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 97.875
+    },
+    {
+      "avg_mask_ratio": 0.43436883635586127,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.43436883635586127,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.11328125,
+      "kd_loss": 0.5290679484073877,
+      "learning_rate": 3e-06,
+      "loss": 0.5147,
+      "masked_tokens": 90.375,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 90.375
+    },
+    {
+      "avg_mask_ratio": 0.5262026051990688,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.5262026051990688,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.322265625,
+      "kd_loss": 0.662759238135186,
+      "learning_rate": 3e-06,
+      "loss": 0.7141,
+      "masked_tokens": 133.7375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 133.7375
+    },
+    {
+      "avg_mask_ratio": 0.44306369486730546,
+      "avg_response_length": 226.6375,
+      "avg_student_mask_ratio": 0.44306369486730546,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.3712,
+      "grad_norm": 0.6484375,
+      "kd_loss": 0.42938064339505216,
+      "learning_rate": 3e-06,
+      "loss": 0.4673,
+      "masked_tokens": 102.675,
+      "mean_t": 0.4757364276825683,
+      "step": 1110,
+      "student_masked_tokens": 102.675
+    },
+    {
+      "avg_mask_ratio": 0.47274903577053917,
+      "avg_response_length": 239.0375,
+      "avg_student_mask_ratio": 0.47274903577053917,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.392533333333333,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.5081101807423011,
+      "learning_rate": 3e-06,
+      "loss": 0.5923,
+      "masked_tokens": 112.65,
+      "mean_t": 0.5013068238971755,
+      "step": 1120,
+      "student_masked_tokens": 112.65
+    },
+    {
+      "avg_mask_ratio": 0.5020543755264952,
+      "avg_response_length": 262.25,
+      "avg_student_mask_ratio": 0.5020543755264952,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.413866666666667,
+      "grad_norm": 0.431640625,
+      "kd_loss": 0.7412553383448994,
+      "learning_rate": 3e-06,
+      "loss": 0.7351,
+      "masked_tokens": 135.7,
+      "mean_t": 0.5303254407714121,
+      "step": 1130,
+      "student_masked_tokens": 135.7
+    },
+    {
+      "avg_mask_ratio": 0.46305053916294125,
+      "avg_response_length": 217.05,
+      "avg_student_mask_ratio": 0.46305053916294125,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4352,
+      "grad_norm": 0.49609375,
+      "kd_loss": 0.5668171994801924,
+      "learning_rate": 3e-06,
+      "loss": 0.5938,
+      "masked_tokens": 103.45,
+      "mean_t": 0.4845335395424627,
+      "step": 1140,
+      "student_masked_tokens": 103.45
+    },
+    {
+      "avg_mask_ratio": 0.5285213771509006,
+      "avg_response_length": 212.5375,
+      "avg_student_mask_ratio": 0.5285213771509006,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4565333333333332,
+      "grad_norm": 0.30078125,
+      "kd_loss": 0.7812401379895164,
+      "learning_rate": 3e-06,
+      "loss": 0.8148,
+      "masked_tokens": 109.75,
+      "mean_t": 0.5690932425903157,
+      "step": 1150,
+      "student_masked_tokens": 109.75
+    },
+    {
+      "avg_mask_ratio": 0.46531253699213265,
+      "avg_response_length": 225.6125,
+      "avg_student_mask_ratio": 0.46531253699213265,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4778666666666664,
+      "grad_norm": 0.2578125,
+      "kd_loss": 0.5916059602122914,
+      "learning_rate": 3e-06,
+      "loss": 0.5667,
+      "masked_tokens": 101.65,
+      "mean_t": 0.5040684466948733,
+      "step": 1160,
+      "student_masked_tokens": 101.65
+    },
+    {
+      "avg_mask_ratio": 0.48369049807661213,
+      "avg_response_length": 247.7625,
+      "avg_student_mask_ratio": 0.48369049807661213,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.4992,
+      "grad_norm": 0.232421875,
+      "kd_loss": 0.6494532859223682,
+      "learning_rate": 3e-06,
+      "loss": 0.691,
+      "masked_tokens": 123.7625,
+      "mean_t": 0.5114516971167177,
+      "step": 1170,
+      "student_masked_tokens": 123.7625
+    },
+    {
+      "avg_mask_ratio": 0.4209623454895336,
+      "avg_response_length": 214.725,
+      "avg_student_mask_ratio": 0.4209623454895336,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.5205333333333333,
+      "grad_norm": 0.267578125,
+      "kd_loss": 0.5382059471070534,
+      "learning_rate": 3e-06,
+      "loss": 0.5929,
+      "masked_tokens": 85.35,
+      "mean_t": 0.4491677140351385,
+      "step": 1180,
+      "student_masked_tokens": 85.35
+    },
+    {
+      "avg_mask_ratio": 0.5209485858562403,
+      "avg_response_length": 220.3375,
+      "avg_student_mask_ratio": 0.5209485858562403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.5418666666666665,
+      "grad_norm": 0.5078125,
+      "kd_loss": 0.9205318759757916,
+      "learning_rate": 3e-06,
+      "loss": 0.9099,
+      "masked_tokens": 124.3375,
+      "mean_t": 0.5590635397238657,
+      "step": 1190,
+      "student_masked_tokens": 124.3375
+    },
+    {
+      "avg_mask_ratio": 0.47338692427438217,
+      "avg_response_length": 215.675,
+      "avg_student_mask_ratio": 0.47338692427438217,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.5632,
+      "grad_norm": 0.29296875,
+      "kd_loss": 0.598471246479835,
+      "learning_rate": 3e-06,
+      "loss": 0.6085,
+      "masked_tokens": 99.6125,
+      "mean_t": 0.506370971655997,
+      "step": 1200,
+      "student_masked_tokens": 99.6125
+    },
+    {
+      "avg_mask_ratio": 0.46816018712706864,
+      "avg_response_length": 223.8875,
+      "avg_student_mask_ratio": 0.46816018712706864,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.5845333333333333,
+      "grad_norm": 0.296875,
+      "kd_loss": 0.7455215450305331,
+      "learning_rate": 3e-06,
+      "loss": 0.7056,
+      "masked_tokens": 102.9375,
+      "mean_t": 0.5015621181708412,
+      "step": 1210,
+      "student_masked_tokens": 102.9375
+    },
+    {
+      "avg_mask_ratio": 0.4605732822965365,
+      "avg_response_length": 235.95,
+      "avg_student_mask_ratio": 0.4605732822965365,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.6058666666666666,
+      "grad_norm": 0.1611328125,
+      "kd_loss": 0.6978286715412485,
+      "learning_rate": 3e-06,
+      "loss": 0.7079,
+      "masked_tokens": 107.9375,
+      "mean_t": 0.48838805715786293,
+      "step": 1220,
+      "student_masked_tokens": 107.9375
+    },
+    {
+      "avg_mask_ratio": 0.5095087783643976,
+      "avg_response_length": 213.425,
+      "avg_student_mask_ratio": 0.5095087783643976,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.6272,
+      "grad_norm": 0.349609375,
+      "kd_loss": 0.7659635735792107,
+      "learning_rate": 3e-06,
+      "loss": 0.7962,
+      "masked_tokens": 113.9375,
+      "mean_t": 0.5434774641471449,
+      "step": 1230,
+      "student_masked_tokens": 113.9375
+    },
+    {
+      "avg_mask_ratio": 0.45200805182103065,
+      "avg_response_length": 233.4875,
+      "avg_student_mask_ratio": 0.45200805182103065,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.6485333333333334,
+      "grad_norm": 0.08154296875,
+      "kd_loss": 0.5647351608607352,
+      "learning_rate": 3e-06,
+      "loss": 0.5456,
+      "masked_tokens": 98.3375,
+      "mean_t": 0.48289845581166446,
+      "step": 1240,
+      "student_masked_tokens": 98.3375
+    },
+    {
+      "avg_mask_ratio": 0.45047278017154896,
+      "avg_response_length": 250.5875,
+      "avg_student_mask_ratio": 0.45047278017154896,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.6698666666666666,
+      "grad_norm": 0.55859375,
+      "kd_loss": 0.6626386449781194,
+      "learning_rate": 3e-06,
+      "loss": 0.6958,
+      "masked_tokens": 117.4625,
+      "mean_t": 0.4824625514564104,
+      "step": 1250,
+      "student_masked_tokens": 117.4625
+    },
+    {
+      "avg_mask_ratio": 0.42361471222830005,
+      "avg_response_length": 237.1875,
+      "avg_student_mask_ratio": 0.42361471222830005,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.6912000000000003,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.44570972973237416,
+      "learning_rate": 3e-06,
+      "loss": 0.4479,
+      "masked_tokens": 95.6375,
+      "mean_t": 0.44945613203162793,
+      "step": 1260,
+      "student_masked_tokens": 95.6375
+    },
+    {
+      "avg_mask_ratio": 0.4967681970912963,
+      "avg_response_length": 226.925,
+      "avg_student_mask_ratio": 0.4967681970912963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.7125333333333335,
+      "grad_norm": 0.314453125,
+      "kd_loss": 0.891427778175057,
+      "learning_rate": 3e-06,
+      "loss": 0.8392,
+      "masked_tokens": 121.1625,
+      "mean_t": 0.5232691798126325,
+      "step": 1270,
+      "student_masked_tokens": 121.1625
+    },
+    {
+      "avg_mask_ratio": 0.41789044241886586,
+      "avg_response_length": 236.0625,
+      "avg_student_mask_ratio": 0.41789044241886586,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.7338666666666667,
+      "grad_norm": 0.240234375,
+      "kd_loss": 0.5116885483056183,
+      "learning_rate": 3e-06,
+      "loss": 0.5405,
+      "masked_tokens": 95.1375,
+      "mean_t": 0.4489673391333781,
+      "step": 1280,
+      "student_masked_tokens": 95.1375
+    },
+    {
+      "avg_mask_ratio": 0.451448042539414,
+      "avg_response_length": 204.1625,
+      "avg_student_mask_ratio": 0.451448042539414,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.7552,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.5385229101943821,
+      "learning_rate": 3e-06,
+      "loss": 0.5534,
+      "masked_tokens": 92.475,
+      "mean_t": 0.4806917709764093,
+      "step": 1290,
+      "student_masked_tokens": 92.475
+    },
+    {
+      "avg_mask_ratio": 0.43493172535672786,
+      "avg_response_length": 244.0375,
+      "avg_student_mask_ratio": 0.43493172535672786,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 1.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0,
+      "epoch": 2.776533333333333,
+      "grad_norm": 0.2890625,
+      "kd_loss": 0.8396390925825244,
+      "learning_rate": 3e-06,
+      "loss": 0.7886,
+      "masked_tokens": 102.4375,
+      "mean_t": 0.4616577463573776,
+      "step": 1300,
+      "student_masked_tokens": 102.4375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2611bced1337375831cc344da28d849ea16c0f2f5a6aeca004865b84ca7216e0
+size 7992

math/INP/unmask_tags_gold1_target1_ce0.0/checkpoint-1400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1