Upload checkpoint from unmask_tags_math_self_distill_INP-PAR-REVERSE_u0.001-1.0_gold1_target1_ce0.5

.gitattributes

@@ -38,3 +38,4 @@ math/inp/SD-INP/math_self_distill_INP_u0.0-0.4_gold1_target1_ce0.5/debug_trainin
 math/inp/SD-INP/math_self_distill_INP_u0.6-1.0_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/inp/SD-INP/math_self_distill_INP_u0.8-1.0_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/SFT/inp-onehot_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
+math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf4b9d210a3391c22f1df9d549fcdd748462cf9c030a4fa30d4ec0f752bef170
+size 2406624648

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a57426daccd1cc2bbb0924939202b317e3fd57700e00cb47d8ca0e81250ca521
+size 2406624648

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d8ba50cf0aeb608baa3b74fb50a95f62f3a09ccef1f9b3ce5a0b1b0e01d11fb7
+size 671304442

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cedbb24eb08e412f2b6567529f919723c479356a0b4861fb1f0133d92b4e4aa
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27f1021fb57112918a3a6091b09b0ccd50cb071a2324c12ae9afcc9851ee8bd3
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d8cdfceac9f7917b978dca661a3b8e04187faea5d5f6bd7b462d61d8234d57f
+size 1064

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/trainer_state.json

@@ -0,0 +1,273 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.21333333333333335,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.4930951670394279,
+      "avg_response_length": 224.7375,
+      "avg_student_mask_ratio": 0.4930951670394279,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.560375437145467,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.404296875,
+      "kd_loss": 0.25131759785651864,
+      "learning_rate": 3e-06,
+      "loss": 0.511,
+      "masked_tokens": 110.925,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 110.925
+    },
+    {
+      "avg_mask_ratio": 0.4127206720062532,
+      "avg_response_length": 277.15,
+      "avg_student_mask_ratio": 0.4127206720062532,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4417972476620662,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.65234375,
+      "kd_loss": 0.14074679291339579,
+      "learning_rate": 3e-06,
+      "loss": 0.3708,
+      "masked_tokens": 108.8625,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 108.8625
+    },
+    {
+      "avg_mask_ratio": 0.4616696212324314,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4616696212324314,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4375385328372886,
+      "epoch": 0.064,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.19322810963351497,
+      "learning_rate": 3e-06,
+      "loss": 0.4217,
+      "masked_tokens": 111.375,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 111.375
+    },
+    {
+      "avg_mask_ratio": 0.42360913623997476,
+      "avg_response_length": 224.6125,
+      "avg_student_mask_ratio": 0.42360913623997476,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39910652460303025,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.15960182610455148,
+      "learning_rate": 3e-06,
+      "loss": 0.367,
+      "masked_tokens": 98.3,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.3
+    },
+    {
+      "avg_mask_ratio": 0.4330951495358022,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.4330951495358022,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6348971565400916,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.15276786715631374,
+      "learning_rate": 3e-06,
+      "loss": 0.4348,
+      "masked_tokens": 85.075,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.075
+    },
+    {
+      "avg_mask_ratio": 0.4555395155097358,
+      "avg_response_length": 254.2125,
+      "avg_student_mask_ratio": 0.4555395155097358,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5031439974316981,
+      "epoch": 0.128,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.195508924105161,
+      "learning_rate": 3e-06,
+      "loss": 0.4481,
+      "masked_tokens": 119.1125,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.1125
+    },
+    {
+      "avg_mask_ratio": 0.5148372989846394,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.5148372989846394,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913914184368878,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.94921875,
+      "kd_loss": 0.17476431281479562,
+      "learning_rate": 3e-06,
+      "loss": 0.5543,
+      "masked_tokens": 105.45,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 105.45
+    },
+    {
+      "avg_mask_ratio": 0.3827478863298893,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.3827478863298893,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40601554353645497,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.1317249322714872,
+      "learning_rate": 3e-06,
+      "loss": 0.2945,
+      "masked_tokens": 86.675,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 86.675
+    },
+    {
+      "avg_mask_ratio": 0.5017695252550766,
+      "avg_response_length": 234.25,
+      "avg_student_mask_ratio": 0.5017695252550766,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6200478935968476,
+      "epoch": 0.192,
+      "grad_norm": 1.0,
+      "kd_loss": 0.23770100035361139,
+      "learning_rate": 3e-06,
+      "loss": 0.6467,
+      "masked_tokens": 108.4125,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 108.4125
+    },
+    {
+      "avg_mask_ratio": 0.4637213449750561,
+      "avg_response_length": 210.175,
+      "avg_student_mask_ratio": 0.4637213449750561,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4699021781670581,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 1.4296875,
+      "kd_loss": 0.17425421165266286,
+      "learning_rate": 3e-06,
+      "loss": 0.4513,
+      "masked_tokens": 107.375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 107.375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b968dbbf06e2017aeda9d45021ff20554875b30a6340f05bc88d8864eccb6012
+size 8120

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:eb11bb91ba0a6deb7346d831437367a289bcc4864e96e5903e2b51b1bc8a0d2a
+size 2406624648

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:760fb3a4c295ac0d8ff87a3e4fb0b31048c29c45d6e97461b621844181013299
+size 671304442

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a2658504939ac737621bd8a2b154858cd4e206cff2216ea3f98af5ed6e476c37
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9ccd67cb51eba4d5d9beb26859701364d21cc26cf86c5a531d92547a5efdbdcb
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29d9aa99505fc60c0db1b9cdacaa08b06e8a85c8aaaab4e389667a719fafb9bf
+size 1064

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/trainer_state.json

@@ -0,0 +1,2433 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.1365333333333334,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.4930951670394279,
+      "avg_response_length": 224.7375,
+      "avg_student_mask_ratio": 0.4930951670394279,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.560375437145467,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.404296875,
+      "kd_loss": 0.25131759785651864,
+      "learning_rate": 3e-06,
+      "loss": 0.511,
+      "masked_tokens": 110.925,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 110.925
+    },
+    {
+      "avg_mask_ratio": 0.4127206720062532,
+      "avg_response_length": 277.15,
+      "avg_student_mask_ratio": 0.4127206720062532,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4417972476620662,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.65234375,
+      "kd_loss": 0.14074679291339579,
+      "learning_rate": 3e-06,
+      "loss": 0.3708,
+      "masked_tokens": 108.8625,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 108.8625
+    },
+    {
+      "avg_mask_ratio": 0.4616696212324314,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4616696212324314,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4375385328372886,
+      "epoch": 0.064,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.19322810963351497,
+      "learning_rate": 3e-06,
+      "loss": 0.4217,
+      "masked_tokens": 111.375,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 111.375
+    },
+    {
+      "avg_mask_ratio": 0.42360913623997476,
+      "avg_response_length": 224.6125,
+      "avg_student_mask_ratio": 0.42360913623997476,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39910652460303025,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.15960182610455148,
+      "learning_rate": 3e-06,
+      "loss": 0.367,
+      "masked_tokens": 98.3,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.3
+    },
+    {
+      "avg_mask_ratio": 0.4330951495358022,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.4330951495358022,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6348971565400916,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.15276786715631374,
+      "learning_rate": 3e-06,
+      "loss": 0.4348,
+      "masked_tokens": 85.075,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.075
+    },
+    {
+      "avg_mask_ratio": 0.4555395155097358,
+      "avg_response_length": 254.2125,
+      "avg_student_mask_ratio": 0.4555395155097358,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5031439974316981,
+      "epoch": 0.128,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.195508924105161,
+      "learning_rate": 3e-06,
+      "loss": 0.4481,
+      "masked_tokens": 119.1125,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.1125
+    },
+    {
+      "avg_mask_ratio": 0.5148372989846394,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.5148372989846394,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913914184368878,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.94921875,
+      "kd_loss": 0.17476431281479562,
+      "learning_rate": 3e-06,
+      "loss": 0.5543,
+      "masked_tokens": 105.45,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 105.45
+    },
+    {
+      "avg_mask_ratio": 0.3827478863298893,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.3827478863298893,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40601554353645497,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.1317249322714872,
+      "learning_rate": 3e-06,
+      "loss": 0.2945,
+      "masked_tokens": 86.675,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 86.675
+    },
+    {
+      "avg_mask_ratio": 0.5017695252550766,
+      "avg_response_length": 234.25,
+      "avg_student_mask_ratio": 0.5017695252550766,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6200478935968476,
+      "epoch": 0.192,
+      "grad_norm": 1.0,
+      "kd_loss": 0.23770100035361139,
+      "learning_rate": 3e-06,
+      "loss": 0.6467,
+      "masked_tokens": 108.4125,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 108.4125
+    },
+    {
+      "avg_mask_ratio": 0.4637213449750561,
+      "avg_response_length": 210.175,
+      "avg_student_mask_ratio": 0.4637213449750561,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4699021781670581,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 1.4296875,
+      "kd_loss": 0.17425421165266286,
+      "learning_rate": 3e-06,
+      "loss": 0.4513,
+      "masked_tokens": 107.375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 107.375
+    },
+    {
+      "avg_mask_ratio": 0.48738867897773164,
+      "avg_response_length": 213.0875,
+      "avg_student_mask_ratio": 0.48738867897773164,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2935672491260675,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.63671875,
+      "kd_loss": 0.21233718555377265,
+      "learning_rate": 3e-06,
+      "loss": 0.4251,
+      "masked_tokens": 101.7875,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 101.7875
+    },
+    {
+      "avg_mask_ratio": 0.4451883323024958,
+      "avg_response_length": 232.3,
+      "avg_student_mask_ratio": 0.4451883323024958,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3356981491514944,
+      "epoch": 0.256,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.23394757619298617,
+      "learning_rate": 3e-06,
+      "loss": 0.3743,
+      "masked_tokens": 107.7,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 107.7
+    },
+    {
+      "avg_mask_ratio": 0.43939279407495635,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.43939279407495635,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2916025636156746,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.734375,
+      "kd_loss": 0.14590738382988774,
+      "learning_rate": 3e-06,
+      "loss": 0.2778,
+      "masked_tokens": 89.95,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 89.95
+    },
+    {
+      "avg_mask_ratio": 0.4922366282902658,
+      "avg_response_length": 264.5375,
+      "avg_student_mask_ratio": 0.4922366282902658,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44275596315469556,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.1741957870854094,
+      "learning_rate": 3e-06,
+      "loss": 0.4423,
+      "masked_tokens": 144.9,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 144.9
+    },
+    {
+      "avg_mask_ratio": 0.4724786171223968,
+      "avg_response_length": 258.1125,
+      "avg_student_mask_ratio": 0.4724786171223968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6182238667762021,
+      "epoch": 0.32,
+      "grad_norm": 0.051513671875,
+      "kd_loss": 0.2385995153230624,
+      "learning_rate": 3e-06,
+      "loss": 0.4876,
+      "masked_tokens": 127.4125,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 127.4125
+    },
+    {
+      "avg_mask_ratio": 0.49717973986989816,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.49717973986989816,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44498473816368006,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.216546824422403,
+      "learning_rate": 3e-06,
+      "loss": 0.4246,
+      "masked_tokens": 98.4875,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 98.4875
+    },
+    {
+      "avg_mask_ratio": 0.48284467663615943,
+      "avg_response_length": 188.65,
+      "avg_student_mask_ratio": 0.48284467663615943,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4632640206855626,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.19741544655316828,
+      "learning_rate": 3e-06,
+      "loss": 0.4259,
+      "masked_tokens": 89.3125,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 89.3125
+    },
+    {
+      "avg_mask_ratio": 0.44864035704231353,
+      "avg_response_length": 246.6875,
+      "avg_student_mask_ratio": 0.44864035704231353,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38090254231135245,
+      "epoch": 0.384,
+      "grad_norm": 0.35546875,
+      "kd_loss": 0.18000799598165002,
+      "learning_rate": 3e-06,
+      "loss": 0.4605,
+      "masked_tokens": 110.0125,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 110.0125
+    },
+    {
+      "avg_mask_ratio": 0.47850618849042803,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.47850618849042803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.524182743717688,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.4140625,
+      "kd_loss": 0.19971058982482645,
+      "learning_rate": 3e-06,
+      "loss": 0.5185,
+      "masked_tokens": 109.575,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 109.575
+    },
+    {
+      "avg_mask_ratio": 0.4662990250624716,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4662990250624716,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2655160516187379,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 2.65625,
+      "kd_loss": 0.11893262579505973,
+      "learning_rate": 3e-06,
+      "loss": 0.272,
+      "masked_tokens": 97.5,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 97.5
+    },
+    {
+      "avg_mask_ratio": 0.451080821454525,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.451080821454525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3671989720950819,
+      "epoch": 0.448,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.15058198695817282,
+      "learning_rate": 3e-06,
+      "loss": 0.3553,
+      "masked_tokens": 96.6375,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 96.6375
+    },
+    {
+      "avg_mask_ratio": 0.5438536155037582,
+      "avg_response_length": 236.5125,
+      "avg_student_mask_ratio": 0.5438536155037582,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5652186098743186,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 2.21875,
+      "kd_loss": 0.2394665865747811,
+      "learning_rate": 3e-06,
+      "loss": 0.5566,
+      "masked_tokens": 126.4375,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 126.4375
+    },
+    {
+      "avg_mask_ratio": 0.43992503273766487,
+      "avg_response_length": 255.875,
+      "avg_student_mask_ratio": 0.43992503273766487,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6014562642475084,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.14900631055388658,
+      "learning_rate": 3e-06,
+      "loss": 0.4491,
+      "masked_tokens": 107.3875,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 107.3875
+    },
+    {
+      "avg_mask_ratio": 0.46683448635449165,
+      "avg_response_length": 224.125,
+      "avg_student_mask_ratio": 0.46683448635449165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3313195034900218,
+      "epoch": 0.512,
+      "grad_norm": 0.384765625,
+      "kd_loss": 0.21447069352492037,
+      "learning_rate": 3e-06,
+      "loss": 0.3762,
+      "masked_tokens": 110.2875,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.2875
+    },
+    {
+      "avg_mask_ratio": 0.4476269483449869,
+      "avg_response_length": 243.2375,
+      "avg_student_mask_ratio": 0.4476269483449869,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2578068570326423,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.1669921875,
+      "kd_loss": 0.1731222485599604,
+      "learning_rate": 3e-06,
+      "loss": 0.2859,
+      "masked_tokens": 112.6375,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 112.6375
+    },
+    {
+      "avg_mask_ratio": 0.45657019784557634,
+      "avg_response_length": 245.55,
+      "avg_student_mask_ratio": 0.45657019784557634,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23127953946784602,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.1577436012762064,
+      "learning_rate": 3e-06,
+      "loss": 0.2823,
+      "masked_tokens": 110.0375,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 110.0375
+    },
+    {
+      "avg_mask_ratio": 0.5293830037582665,
+      "avg_response_length": 223.975,
+      "avg_student_mask_ratio": 0.5293830037582665,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3770375440454927,
+      "epoch": 0.576,
+      "grad_norm": 0.8359375,
+      "kd_loss": 0.2504498413263718,
+      "learning_rate": 3e-06,
+      "loss": 0.4807,
+      "masked_tokens": 119.6,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 119.6
+    },
+    {
+      "avg_mask_ratio": 0.4577330934116617,
+      "avg_response_length": 257.2125,
+      "avg_student_mask_ratio": 0.4577330934116617,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5647278043467623,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.421875,
+      "kd_loss": 0.18768687424256,
+      "learning_rate": 3e-06,
+      "loss": 0.3942,
+      "masked_tokens": 130.475,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 130.475
+    },
+    {
+      "avg_mask_ratio": 0.39295024327002465,
+      "avg_response_length": 246.6375,
+      "avg_student_mask_ratio": 0.39295024327002465,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.22343009332175826,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.1796875,
+      "kd_loss": 0.10545187053209588,
+      "learning_rate": 3e-06,
+      "loss": 0.2961,
+      "masked_tokens": 100.9,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.9
+    },
+    {
+      "avg_mask_ratio": 0.4409991275751963,
+      "avg_response_length": 217.9125,
+      "avg_student_mask_ratio": 0.4409991275751963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2520033180570181,
+      "epoch": 0.64,
+      "grad_norm": 0.1904296875,
+      "kd_loss": 0.1336840357888832,
+      "learning_rate": 3e-06,
+      "loss": 0.3042,
+      "masked_tokens": 106.925,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 106.925
+    },
+    {
+      "avg_mask_ratio": 0.47207197032403203,
+      "avg_response_length": 188.9125,
+      "avg_student_mask_ratio": 0.47207197032403203,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3869200809263475,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.19366624409758515,
+      "learning_rate": 3e-06,
+      "loss": 0.4395,
+      "masked_tokens": 85.125,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 85.125
+    },
+    {
+      "avg_mask_ratio": 0.4926959708333015,
+      "avg_response_length": 248.4,
+      "avg_student_mask_ratio": 0.4926959708333015,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4449735159470379,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.2666319968902826,
+      "learning_rate": 3e-06,
+      "loss": 0.5082,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5075328870676458,
+      "avg_response_length": 235.075,
+      "avg_student_mask_ratio": 0.5075328870676458,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4397935895413411,
+      "epoch": 0.704,
+      "grad_norm": 0.46484375,
+      "kd_loss": 0.21343186177154222,
+      "learning_rate": 3e-06,
+      "loss": 0.4382,
+      "masked_tokens": 127.9,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 127.9
+    },
+    {
+      "avg_mask_ratio": 0.44940012450679206,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.44940012450679206,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.508160601661291,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.89453125,
+      "kd_loss": 0.18509022962782815,
+      "learning_rate": 3e-06,
+      "loss": 0.4375,
+      "masked_tokens": 110.15,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 110.15
+    },
+    {
+      "avg_mask_ratio": 0.4945301389612723,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.4945301389612723,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6931099190221858,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.89453125,
+      "kd_loss": 0.2149010855906642,
+      "learning_rate": 3e-06,
+      "loss": 0.4849,
+      "masked_tokens": 125.175,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 125.175
+    },
+    {
+      "avg_mask_ratio": 0.4749954905593768,
+      "avg_response_length": 243.575,
+      "avg_student_mask_ratio": 0.4749954905593768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.27926406990906116,
+      "epoch": 0.768,
+      "grad_norm": 0.1376953125,
+      "kd_loss": 0.16614201006656232,
+      "learning_rate": 3e-06,
+      "loss": 0.3252,
+      "masked_tokens": 115.3875,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 115.3875
+    },
+    {
+      "avg_mask_ratio": 0.47621052770409733,
+      "avg_response_length": 257.575,
+      "avg_student_mask_ratio": 0.47621052770409733,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5629604070229106,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.12060546875,
+      "kd_loss": 0.21382154841430748,
+      "learning_rate": 3e-06,
+      "loss": 0.4245,
+      "masked_tokens": 126.85,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 126.85
+    },
+    {
+      "avg_mask_ratio": 0.4449806016869843,
+      "avg_response_length": 226.3625,
+      "avg_student_mask_ratio": 0.4449806016869843,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.19371982016177752,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.91796875,
+      "kd_loss": 0.13640525953642282,
+      "learning_rate": 3e-06,
+      "loss": 0.3082,
+      "masked_tokens": 109.35,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 109.35
+    },
+    {
+      "avg_mask_ratio": 0.47845896739745514,
+      "avg_response_length": 218.1125,
+      "avg_student_mask_ratio": 0.47845896739745514,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46541876822730044,
+      "epoch": 0.832,
+      "grad_norm": 0.2578125,
+      "kd_loss": 0.14829268052657757,
+      "learning_rate": 3e-06,
+      "loss": 0.395,
+      "masked_tokens": 111.85,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 111.85
+    },
+    {
+      "avg_mask_ratio": 0.4727763219270855,
+      "avg_response_length": 242.6125,
+      "avg_student_mask_ratio": 0.4727763219270855,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42413156108468686,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 1.6796875,
+      "kd_loss": 0.21946758147294076,
+      "learning_rate": 3e-06,
+      "loss": 0.4026,
+      "masked_tokens": 119.775,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 119.775
+    },
+    {
+      "avg_mask_ratio": 0.4756184325611684,
+      "avg_response_length": 239.5375,
+      "avg_student_mask_ratio": 0.4756184325611684,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.606393156799436,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.322265625,
+      "kd_loss": 0.20679433524416027,
+      "learning_rate": 3e-06,
+      "loss": 0.4871,
+      "masked_tokens": 122.1875,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 122.1875
+    },
+    {
+      "avg_mask_ratio": 0.4428858984610997,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4428858984610997,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1625635328691999,
+      "epoch": 0.896,
+      "grad_norm": 0.92578125,
+      "kd_loss": 0.14000170415542926,
+      "learning_rate": 3e-06,
+      "loss": 0.236,
+      "masked_tokens": 86.0125,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 86.0125
+    },
+    {
+      "avg_mask_ratio": 0.4625907339621335,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4625907339621335,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2754062189423848,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.15317172392096837,
+      "learning_rate": 3e-06,
+      "loss": 0.3358,
+      "masked_tokens": 103.675,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 103.675
+    },
+    {
+      "avg_mask_ratio": 0.4465438393759541,
+      "avg_response_length": 241.9625,
+      "avg_student_mask_ratio": 0.4465438393759541,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35291280565822947,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.3046875,
+      "kd_loss": 0.19183106588135387,
+      "learning_rate": 3e-06,
+      "loss": 0.3552,
+      "masked_tokens": 114.9125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 114.9125
+    },
+    {
+      "avg_mask_ratio": 0.42805201532319187,
+      "avg_response_length": 231.9875,
+      "avg_student_mask_ratio": 0.42805201532319187,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24969172771297962,
+      "epoch": 0.96,
+      "grad_norm": 0.63671875,
+      "kd_loss": 0.14995061269372628,
+      "learning_rate": 3e-06,
+      "loss": 0.3279,
+      "masked_tokens": 103.875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 103.875
+    },
+    {
+      "avg_mask_ratio": 0.4651826085988432,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.4651826085988432,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4396149708533358,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.10009765625,
+      "kd_loss": 0.15930728618550347,
+      "learning_rate": 3e-06,
+      "loss": 0.4374,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.47693050167124185,
+      "avg_response_length": 226.16666666666666,
+      "avg_student_mask_ratio": 0.47693050167124185,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4381092349216964,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.19419911270465343,
+      "learning_rate": 3e-06,
+      "loss": 0.5193,
+      "masked_tokens": 109.10714285714286,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 109.10714285714286
+    },
+    {
+      "avg_mask_ratio": 0.47416816898621617,
+      "avg_response_length": 250.25,
+      "avg_student_mask_ratio": 0.47416816898621617,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5465742808028722,
+      "epoch": 1.0256,
+      "grad_norm": 0.4921875,
+      "kd_loss": 0.2201738972554949,
+      "learning_rate": 3e-06,
+      "loss": 0.4372,
+      "masked_tokens": 117.9125,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 117.9125
+    },
+    {
+      "avg_mask_ratio": 0.45927587888436394,
+      "avg_response_length": 233.05,
+      "avg_student_mask_ratio": 0.45927587888436394,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3881588882651158,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.671875,
+      "kd_loss": 0.1297203417185301,
+      "learning_rate": 3e-06,
+      "loss": 0.3693,
+      "masked_tokens": 98.8375,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 98.8375
+    },
+    {
+      "avg_mask_ratio": 0.5047377114649862,
+      "avg_response_length": 260.225,
+      "avg_student_mask_ratio": 0.5047377114649862,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2485250894494584,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.15780083275045698,
+      "learning_rate": 3e-06,
+      "loss": 0.2985,
+      "masked_tokens": 127.4,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.4
+    },
+    {
+      "avg_mask_ratio": 0.5005011082510464,
+      "avg_response_length": 252.05,
+      "avg_student_mask_ratio": 0.5005011082510464,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3350205140219941,
+      "epoch": 1.0896,
+      "grad_norm": 2.953125,
+      "kd_loss": 0.21016031454611267,
+      "learning_rate": 3e-06,
+      "loss": 0.4012,
+      "masked_tokens": 133.5,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 133.5
+    },
+    {
+      "avg_mask_ratio": 0.4552151845302433,
+      "avg_response_length": 200.7625,
+      "avg_student_mask_ratio": 0.4552151845302433,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5980809078201788,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.70703125,
+      "kd_loss": 0.1575855805382389,
+      "learning_rate": 3e-06,
+      "loss": 0.4353,
+      "masked_tokens": 86.475,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 86.475
+    },
+    {
+      "avg_mask_ratio": 0.46727682640776036,
+      "avg_response_length": 214.5375,
+      "avg_student_mask_ratio": 0.46727682640776036,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44713975176116494,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.5625,
+      "kd_loss": 0.2050198341205757,
+      "learning_rate": 3e-06,
+      "loss": 0.4469,
+      "masked_tokens": 99.8375,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 99.8375
+    },
+    {
+      "avg_mask_ratio": 0.48785575344227256,
+      "avg_response_length": 231.3125,
+      "avg_student_mask_ratio": 0.48785575344227256,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35778319709847894,
+      "epoch": 1.1536,
+      "grad_norm": 0.318359375,
+      "kd_loss": 0.2095373813806191,
+      "learning_rate": 3e-06,
+      "loss": 0.4302,
+      "masked_tokens": 106.625,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 106.625
+    },
+    {
+      "avg_mask_ratio": 0.4477671392261982,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4477671392261982,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3559980413245398,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.283203125,
+      "kd_loss": 0.16553757517219764,
+      "learning_rate": 3e-06,
+      "loss": 0.3392,
+      "masked_tokens": 97.7,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 97.7
+    },
+    {
+      "avg_mask_ratio": 0.3861591775319539,
+      "avg_response_length": 238.5375,
+      "avg_student_mask_ratio": 0.3861591775319539,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25838428690098,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.365234375,
+      "kd_loss": 0.19571233854148887,
+      "learning_rate": 3e-06,
+      "loss": 0.3186,
+      "masked_tokens": 99.625,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 99.625
+    },
+    {
+      "avg_mask_ratio": 0.44424078196752814,
+      "avg_response_length": 230.1625,
+      "avg_student_mask_ratio": 0.44424078196752814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.30152245728992283,
+      "epoch": 1.2176,
+      "grad_norm": 0.703125,
+      "kd_loss": 0.16761301296596684,
+      "learning_rate": 3e-06,
+      "loss": 0.3934,
+      "masked_tokens": 108.0125,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 108.0125
+    },
+    {
+      "avg_mask_ratio": 0.4715048542013392,
+      "avg_response_length": 269.1375,
+      "avg_student_mask_ratio": 0.4715048542013392,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3189941981676611,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.93359375,
+      "kd_loss": 0.2090823985024258,
+      "learning_rate": 3e-06,
+      "loss": 0.3805,
+      "masked_tokens": 129.4375,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 129.4375
+    },
+    {
+      "avg_mask_ratio": 0.520072100055404,
+      "avg_response_length": 228.2875,
+      "avg_student_mask_ratio": 0.520072100055404,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4312982382521682,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.2421910996934457,
+      "learning_rate": 3e-06,
+      "loss": 0.4936,
+      "masked_tokens": 121.1625,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 121.1625
+    },
+    {
+      "avg_mask_ratio": 0.4505112706683576,
+      "avg_response_length": 237.1625,
+      "avg_student_mask_ratio": 0.4505112706683576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3104078326705803,
+      "epoch": 1.2816,
+      "grad_norm": 0.62890625,
+      "kd_loss": 0.17020997749766878,
+      "learning_rate": 3e-06,
+      "loss": 0.3534,
+      "masked_tokens": 110.0875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 110.0875
+    },
+    {
+      "avg_mask_ratio": 0.5064190638251602,
+      "avg_response_length": 244.3,
+      "avg_student_mask_ratio": 0.5064190638251602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.402807544786765,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.1942155301481648,
+      "learning_rate": 3e-06,
+      "loss": 0.3868,
+      "masked_tokens": 119.825,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 119.825
+    },
+    {
+      "avg_mask_ratio": 0.4891548154759221,
+      "avg_response_length": 234.6625,
+      "avg_student_mask_ratio": 0.4891548154759221,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3629302057381551,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.178446790360789,
+      "learning_rate": 3e-06,
+      "loss": 0.3871,
+      "masked_tokens": 110.5625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 110.5625
+    },
+    {
+      "avg_mask_ratio": 0.5235460251918994,
+      "avg_response_length": 258.0875,
+      "avg_student_mask_ratio": 0.5235460251918994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5023660743239304,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.2248805959525349,
+      "learning_rate": 3e-06,
+      "loss": 0.4892,
+      "masked_tokens": 130.725,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.725
+    },
+    {
+      "avg_mask_ratio": 0.4871393243782222,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.4871393243782222,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.29940326604166784,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.18196311790991332,
+      "learning_rate": 3e-06,
+      "loss": 0.2959,
+      "masked_tokens": 121.4625,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 121.4625
+    },
+    {
+      "avg_mask_ratio": 0.5254402696969919,
+      "avg_response_length": 240.575,
+      "avg_student_mask_ratio": 0.5254402696969919,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6097493719300247,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.265625,
+      "kd_loss": 0.20292491851313663,
+      "learning_rate": 3e-06,
+      "loss": 0.3965,
+      "masked_tokens": 133.725,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 133.725
+    },
+    {
+      "avg_mask_ratio": 0.48242234602803363,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.48242234602803363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35620763409436335,
+      "epoch": 1.4096,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.1878819163746357,
+      "learning_rate": 3e-06,
+      "loss": 0.4048,
+      "masked_tokens": 128.4625,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 128.4625
+    },
+    {
+      "avg_mask_ratio": 0.4551548367831856,
+      "avg_response_length": 247.3,
+      "avg_student_mask_ratio": 0.4551548367831856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3581841780794434,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.19171390956483023,
+      "learning_rate": 3e-06,
+      "loss": 0.3241,
+      "masked_tokens": 127.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 127.125
+    },
+    {
+      "avg_mask_ratio": 0.4658544249658007,
+      "avg_response_length": 224.7375,
+      "avg_student_mask_ratio": 0.4658544249658007,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3943864138858316,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.40625,
+      "kd_loss": 0.2207362408495353,
+      "learning_rate": 3e-06,
+      "loss": 0.3831,
+      "masked_tokens": 107.325,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 107.325
+    },
+    {
+      "avg_mask_ratio": 0.4795732157537714,
+      "avg_response_length": 200.975,
+      "avg_student_mask_ratio": 0.4795732157537714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38489766512459483,
+      "epoch": 1.4736,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.18907199591662902,
+      "learning_rate": 3e-06,
+      "loss": 0.3422,
+      "masked_tokens": 110.9375,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 110.9375
+    },
+    {
+      "avg_mask_ratio": 0.5250519359949977,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5250519359949977,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23998604974298132,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.22733612624851957,
+      "learning_rate": 3e-06,
+      "loss": 0.4221,
+      "masked_tokens": 132.55,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 132.55
+    },
+    {
+      "avg_mask_ratio": 0.4788092178525403,
+      "avg_response_length": 215.325,
+      "avg_student_mask_ratio": 0.4788092178525403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24203067874498174,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.259765625,
+      "kd_loss": 0.17745301808752317,
+      "learning_rate": 3e-06,
+      "loss": 0.3234,
+      "masked_tokens": 108.7125,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 108.7125
+    },
+    {
+      "avg_mask_ratio": 0.47950589570682495,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.47950589570682495,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3623171389520394,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.22090139569954773,
+      "learning_rate": 3e-06,
+      "loss": 0.3885,
+      "masked_tokens": 110.8,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 110.8
+    },
+    {
+      "avg_mask_ratio": 0.4604924251558259,
+      "avg_response_length": 232.925,
+      "avg_student_mask_ratio": 0.4604924251558259,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.644551625979841,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.455078125,
+      "kd_loss": 0.1767749028022223,
+      "learning_rate": 3e-06,
+      "loss": 0.4317,
+      "masked_tokens": 104.9625,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 104.9625
+    },
+    {
+      "avg_mask_ratio": 0.5185885130194947,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5185885130194947,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3369603527123218,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.32421875,
+      "kd_loss": 0.2044303039108854,
+      "learning_rate": 3e-06,
+      "loss": 0.4135,
+      "masked_tokens": 97.125,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 97.125
+    },
+    {
+      "avg_mask_ratio": 0.4191439319110941,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.4191439319110941,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37243218952012286,
+      "epoch": 1.6016,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.20017539288881814,
+      "learning_rate": 3e-06,
+      "loss": 0.3757,
+      "masked_tokens": 95.3125,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 95.3125
+    },
+    {
+      "avg_mask_ratio": 0.46666589792585,
+      "avg_response_length": 228.5125,
+      "avg_student_mask_ratio": 0.46666589792585,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3059532491540608,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.248046875,
+      "kd_loss": 0.21255611882733944,
+      "learning_rate": 3e-06,
+      "loss": 0.4021,
+      "masked_tokens": 117.2625,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 117.2625
+    },
+    {
+      "avg_mask_ratio": 0.4440126782981679,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4440126782981679,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28341264147752554,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.396484375,
+      "kd_loss": 0.15398789559246354,
+      "learning_rate": 3e-06,
+      "loss": 0.3165,
+      "masked_tokens": 117.8,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 117.8
+    },
+    {
+      "avg_mask_ratio": 0.42723098206624854,
+      "avg_response_length": 258.0125,
+      "avg_student_mask_ratio": 0.42723098206624854,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.413171638951826,
+      "epoch": 1.6656,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.12727623267242053,
+      "learning_rate": 3e-06,
+      "loss": 0.3069,
+      "masked_tokens": 118.575,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 118.575
+    },
+    {
+      "avg_mask_ratio": 0.46588709874195045,
+      "avg_response_length": 220.7,
+      "avg_student_mask_ratio": 0.46588709874195045,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.21302388652923127,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.1611328125,
+      "kd_loss": 0.164701623401076,
+      "learning_rate": 3e-06,
+      "loss": 0.2692,
+      "masked_tokens": 102.7625,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 102.7625
+    },
+    {
+      "avg_mask_ratio": 0.46366424662992356,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.46366424662992356,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28800638252976113,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 1.046875,
+      "kd_loss": 0.1414074915684978,
+      "learning_rate": 3e-06,
+      "loss": 0.2934,
+      "masked_tokens": 104.5,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 104.5
+    },
+    {
+      "avg_mask_ratio": 0.5037169002695009,
+      "avg_response_length": 250.2875,
+      "avg_student_mask_ratio": 0.5037169002695009,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6385474821537969,
+      "epoch": 1.7296,
+      "grad_norm": 0.76953125,
+      "kd_loss": 0.18596106451021796,
+      "learning_rate": 3e-06,
+      "loss": 0.534,
+      "masked_tokens": 119.575,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 119.575
+    },
+    {
+      "avg_mask_ratio": 0.5109186505898833,
+      "avg_response_length": 225.95,
+      "avg_student_mask_ratio": 0.5109186505898833,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46484450603404637,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.859375,
+      "kd_loss": 0.24066244094453565,
+      "learning_rate": 3e-06,
+      "loss": 0.4932,
+      "masked_tokens": 129.25,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 129.25
+    },
+    {
+      "avg_mask_ratio": 0.4988811274059117,
+      "avg_response_length": 263.7875,
+      "avg_student_mask_ratio": 0.4988811274059117,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4519748693252268,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.1826171875,
+      "kd_loss": 0.19858365579191287,
+      "learning_rate": 3e-06,
+      "loss": 0.3708,
+      "masked_tokens": 137.0,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 137.0
+    },
+    {
+      "avg_mask_ratio": 0.4997270987310912,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.4997270987310912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46417846596852996,
+      "epoch": 1.7936,
+      "grad_norm": 0.365234375,
+      "kd_loss": 0.17983142291315063,
+      "learning_rate": 3e-06,
+      "loss": 0.3971,
+      "masked_tokens": 114.525,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 114.525
+    },
+    {
+      "avg_mask_ratio": 0.4988076956477016,
+      "avg_response_length": 225.5,
+      "avg_student_mask_ratio": 0.4988076956477016,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3025185387264401,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.17459867448646946,
+      "learning_rate": 3e-06,
+      "loss": 0.3211,
+      "masked_tokens": 116.9125,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 116.9125
+    },
+    {
+      "avg_mask_ratio": 0.4497753610135987,
+      "avg_response_length": 260.15,
+      "avg_student_mask_ratio": 0.4497753610135987,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43239108170809004,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.1513671875,
+      "kd_loss": 0.21549869684477016,
+      "learning_rate": 3e-06,
+      "loss": 0.3711,
+      "masked_tokens": 121.7625,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 121.7625
+    },
+    {
+      "avg_mask_ratio": 0.48591957957251,
+      "avg_response_length": 231.0875,
+      "avg_student_mask_ratio": 0.48591957957251,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3273810862849018,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.41796875,
+      "kd_loss": 0.18414456472100937,
+      "learning_rate": 3e-06,
+      "loss": 0.362,
+      "masked_tokens": 116.7625,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 116.7625
+    },
+    {
+      "avg_mask_ratio": 0.44369487821822984,
+      "avg_response_length": 197.9125,
+      "avg_student_mask_ratio": 0.44369487821822984,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23855479693941106,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.16760400208100634,
+      "learning_rate": 3e-06,
+      "loss": 0.3392,
+      "masked_tokens": 91.4125,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 91.4125
+    },
+    {
+      "avg_mask_ratio": 0.44944015803339427,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.44944015803339427,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.21236650576719285,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.357421875,
+      "kd_loss": 0.15185958203733207,
+      "learning_rate": 3e-06,
+      "loss": 0.2372,
+      "masked_tokens": 103.2375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 103.2375
+    },
+    {
+      "avg_mask_ratio": 0.45069065956631676,
+      "avg_response_length": 230.175,
+      "avg_student_mask_ratio": 0.45069065956631676,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2341734012890015,
+      "epoch": 1.9216,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.18844677137070107,
+      "learning_rate": 3e-06,
+      "loss": 0.3624,
+      "masked_tokens": 104.325,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.475881968671456,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.475881968671456,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24741036754367088,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.3125,
+      "kd_loss": 0.15667265115065448,
+      "learning_rate": 3e-06,
+      "loss": 0.3189,
+      "masked_tokens": 125.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 125.1625
+    },
+    {
+      "avg_mask_ratio": 0.4633113604504615,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4633113604504615,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23597910864411914,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.17416997326051825,
+      "learning_rate": 3e-06,
+      "loss": 0.3294,
+      "masked_tokens": 109.5375,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 109.5375
+    },
+    {
+      "avg_mask_ratio": 0.47329409609083084,
+      "avg_response_length": 244.875,
+      "avg_student_mask_ratio": 0.47329409609083084,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2694289070637751,
+      "epoch": 1.9856,
+      "grad_norm": 0.2255859375,
+      "kd_loss": 0.16790816846706277,
+      "learning_rate": 3e-06,
+      "loss": 0.3419,
+      "masked_tokens": 120.525,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 120.525
+    },
+    {
+      "avg_mask_ratio": 0.4973435569776311,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.4973435569776311,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34412962615628895,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.18454486661500727,
+      "learning_rate": 3e-06,
+      "loss": 0.3697,
+      "masked_tokens": 120.63095238095238,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 120.63095238095238
+    },
+    {
+      "avg_mask_ratio": 0.4365456592233386,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4365456592233386,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28184777884216033,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.6875,
+      "kd_loss": 0.15645663726530615,
+      "learning_rate": 3e-06,
+      "loss": 0.2725,
+      "masked_tokens": 118.8125,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 118.8125
+    },
+    {
+      "avg_mask_ratio": 0.4914003949146718,
+      "avg_response_length": 275.3,
+      "avg_student_mask_ratio": 0.4914003949146718,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43313308064230116,
+      "epoch": 2.0512,
+      "grad_norm": 0.5625,
+      "kd_loss": 0.2300663744529629,
+      "learning_rate": 3e-06,
+      "loss": 0.4182,
+      "masked_tokens": 143.075,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 143.075
+    },
+    {
+      "avg_mask_ratio": 0.4381961932755075,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4381961932755075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34196077592644086,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.1847320883095051,
+      "learning_rate": 3e-06,
+      "loss": 0.3923,
+      "masked_tokens": 106.5125,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 106.5125
+    },
+    {
+      "avg_mask_ratio": 0.42702240714570505,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.42702240714570505,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4801598512272108,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.14269168120167705,
+      "learning_rate": 3e-06,
+      "loss": 0.3004,
+      "masked_tokens": 98.525,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 98.525
+    },
+    {
+      "avg_mask_ratio": 0.4583221158827655,
+      "avg_response_length": 262.0375,
+      "avg_student_mask_ratio": 0.4583221158827655,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5193568647300936,
+      "epoch": 2.1152,
+      "grad_norm": 0.5234375,
+      "kd_loss": 0.15391424604512166,
+      "learning_rate": 3e-06,
+      "loss": 0.3657,
+      "masked_tokens": 121.2875,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 121.2875
+    },
+    {
+      "avg_mask_ratio": 0.45086776099633424,
+      "avg_response_length": 214.925,
+      "avg_student_mask_ratio": 0.45086776099633424,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.20815674337951579,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.470703125,
+      "kd_loss": 0.1478596617299445,
+      "learning_rate": 3e-06,
+      "loss": 0.3149,
+      "masked_tokens": 100.4875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 100.4875
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b968dbbf06e2017aeda9d45021ff20554875b30a6340f05bc88d8864eccb6012
+size 8120

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7305085d281f41ee9f5299f9d5ca387a68fa153b1985bcbbf66a53b313f282a7
+size 2406624648

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:df84ab5bd1ece3984c49ec853679765c499043c85741d4afb2ac818a43362300
+size 671304442

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:907e39dff0cf7ad1a1affaa1e7047653794ab16e25c6977ce7b5524769fdf799
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93b99032f4d9f420b20d654e1d4d712556184b2955893a2ffe42549865e9d1b4
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25795e3b7374d0f6abdd7ab4b34fbf7ab0447ba73c04014500c2ab8b5acec5b4
+size 1064

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/trainer_state.json

@@ -0,0 +1,2673 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.3498666666666668,
+  "eval_steps": 500,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.4930951670394279,
+      "avg_response_length": 224.7375,
+      "avg_student_mask_ratio": 0.4930951670394279,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.560375437145467,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.404296875,
+      "kd_loss": 0.25131759785651864,
+      "learning_rate": 3e-06,
+      "loss": 0.511,
+      "masked_tokens": 110.925,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 110.925
+    },
+    {
+      "avg_mask_ratio": 0.4127206720062532,
+      "avg_response_length": 277.15,
+      "avg_student_mask_ratio": 0.4127206720062532,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4417972476620662,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.65234375,
+      "kd_loss": 0.14074679291339579,
+      "learning_rate": 3e-06,
+      "loss": 0.3708,
+      "masked_tokens": 108.8625,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 108.8625
+    },
+    {
+      "avg_mask_ratio": 0.4616696212324314,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4616696212324314,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4375385328372886,
+      "epoch": 0.064,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.19322810963351497,
+      "learning_rate": 3e-06,
+      "loss": 0.4217,
+      "masked_tokens": 111.375,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 111.375
+    },
+    {
+      "avg_mask_ratio": 0.42360913623997476,
+      "avg_response_length": 224.6125,
+      "avg_student_mask_ratio": 0.42360913623997476,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39910652460303025,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.15960182610455148,
+      "learning_rate": 3e-06,
+      "loss": 0.367,
+      "masked_tokens": 98.3,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.3
+    },
+    {
+      "avg_mask_ratio": 0.4330951495358022,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.4330951495358022,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6348971565400916,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.15276786715631374,
+      "learning_rate": 3e-06,
+      "loss": 0.4348,
+      "masked_tokens": 85.075,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.075
+    },
+    {
+      "avg_mask_ratio": 0.4555395155097358,
+      "avg_response_length": 254.2125,
+      "avg_student_mask_ratio": 0.4555395155097358,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5031439974316981,
+      "epoch": 0.128,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.195508924105161,
+      "learning_rate": 3e-06,
+      "loss": 0.4481,
+      "masked_tokens": 119.1125,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.1125
+    },
+    {
+      "avg_mask_ratio": 0.5148372989846394,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.5148372989846394,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913914184368878,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.94921875,
+      "kd_loss": 0.17476431281479562,
+      "learning_rate": 3e-06,
+      "loss": 0.5543,
+      "masked_tokens": 105.45,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 105.45
+    },
+    {
+      "avg_mask_ratio": 0.3827478863298893,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.3827478863298893,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40601554353645497,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.1317249322714872,
+      "learning_rate": 3e-06,
+      "loss": 0.2945,
+      "masked_tokens": 86.675,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 86.675
+    },
+    {
+      "avg_mask_ratio": 0.5017695252550766,
+      "avg_response_length": 234.25,
+      "avg_student_mask_ratio": 0.5017695252550766,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6200478935968476,
+      "epoch": 0.192,
+      "grad_norm": 1.0,
+      "kd_loss": 0.23770100035361139,
+      "learning_rate": 3e-06,
+      "loss": 0.6467,
+      "masked_tokens": 108.4125,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 108.4125
+    },
+    {
+      "avg_mask_ratio": 0.4637213449750561,
+      "avg_response_length": 210.175,
+      "avg_student_mask_ratio": 0.4637213449750561,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4699021781670581,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 1.4296875,
+      "kd_loss": 0.17425421165266286,
+      "learning_rate": 3e-06,
+      "loss": 0.4513,
+      "masked_tokens": 107.375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 107.375
+    },
+    {
+      "avg_mask_ratio": 0.48738867897773164,
+      "avg_response_length": 213.0875,
+      "avg_student_mask_ratio": 0.48738867897773164,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2935672491260675,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.63671875,
+      "kd_loss": 0.21233718555377265,
+      "learning_rate": 3e-06,
+      "loss": 0.4251,
+      "masked_tokens": 101.7875,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 101.7875
+    },
+    {
+      "avg_mask_ratio": 0.4451883323024958,
+      "avg_response_length": 232.3,
+      "avg_student_mask_ratio": 0.4451883323024958,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3356981491514944,
+      "epoch": 0.256,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.23394757619298617,
+      "learning_rate": 3e-06,
+      "loss": 0.3743,
+      "masked_tokens": 107.7,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 107.7
+    },
+    {
+      "avg_mask_ratio": 0.43939279407495635,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.43939279407495635,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2916025636156746,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.734375,
+      "kd_loss": 0.14590738382988774,
+      "learning_rate": 3e-06,
+      "loss": 0.2778,
+      "masked_tokens": 89.95,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 89.95
+    },
+    {
+      "avg_mask_ratio": 0.4922366282902658,
+      "avg_response_length": 264.5375,
+      "avg_student_mask_ratio": 0.4922366282902658,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44275596315469556,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.1741957870854094,
+      "learning_rate": 3e-06,
+      "loss": 0.4423,
+      "masked_tokens": 144.9,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 144.9
+    },
+    {
+      "avg_mask_ratio": 0.4724786171223968,
+      "avg_response_length": 258.1125,
+      "avg_student_mask_ratio": 0.4724786171223968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6182238667762021,
+      "epoch": 0.32,
+      "grad_norm": 0.051513671875,
+      "kd_loss": 0.2385995153230624,
+      "learning_rate": 3e-06,
+      "loss": 0.4876,
+      "masked_tokens": 127.4125,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 127.4125
+    },
+    {
+      "avg_mask_ratio": 0.49717973986989816,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.49717973986989816,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44498473816368006,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.216546824422403,
+      "learning_rate": 3e-06,
+      "loss": 0.4246,
+      "masked_tokens": 98.4875,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 98.4875
+    },
+    {
+      "avg_mask_ratio": 0.48284467663615943,
+      "avg_response_length": 188.65,
+      "avg_student_mask_ratio": 0.48284467663615943,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4632640206855626,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.19741544655316828,
+      "learning_rate": 3e-06,
+      "loss": 0.4259,
+      "masked_tokens": 89.3125,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 89.3125
+    },
+    {
+      "avg_mask_ratio": 0.44864035704231353,
+      "avg_response_length": 246.6875,
+      "avg_student_mask_ratio": 0.44864035704231353,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38090254231135245,
+      "epoch": 0.384,
+      "grad_norm": 0.35546875,
+      "kd_loss": 0.18000799598165002,
+      "learning_rate": 3e-06,
+      "loss": 0.4605,
+      "masked_tokens": 110.0125,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 110.0125
+    },
+    {
+      "avg_mask_ratio": 0.47850618849042803,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.47850618849042803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.524182743717688,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.4140625,
+      "kd_loss": 0.19971058982482645,
+      "learning_rate": 3e-06,
+      "loss": 0.5185,
+      "masked_tokens": 109.575,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 109.575
+    },
+    {
+      "avg_mask_ratio": 0.4662990250624716,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4662990250624716,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2655160516187379,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 2.65625,
+      "kd_loss": 0.11893262579505973,
+      "learning_rate": 3e-06,
+      "loss": 0.272,
+      "masked_tokens": 97.5,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 97.5
+    },
+    {
+      "avg_mask_ratio": 0.451080821454525,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.451080821454525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3671989720950819,
+      "epoch": 0.448,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.15058198695817282,
+      "learning_rate": 3e-06,
+      "loss": 0.3553,
+      "masked_tokens": 96.6375,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 96.6375
+    },
+    {
+      "avg_mask_ratio": 0.5438536155037582,
+      "avg_response_length": 236.5125,
+      "avg_student_mask_ratio": 0.5438536155037582,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5652186098743186,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 2.21875,
+      "kd_loss": 0.2394665865747811,
+      "learning_rate": 3e-06,
+      "loss": 0.5566,
+      "masked_tokens": 126.4375,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 126.4375
+    },
+    {
+      "avg_mask_ratio": 0.43992503273766487,
+      "avg_response_length": 255.875,
+      "avg_student_mask_ratio": 0.43992503273766487,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6014562642475084,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.14900631055388658,
+      "learning_rate": 3e-06,
+      "loss": 0.4491,
+      "masked_tokens": 107.3875,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 107.3875
+    },
+    {
+      "avg_mask_ratio": 0.46683448635449165,
+      "avg_response_length": 224.125,
+      "avg_student_mask_ratio": 0.46683448635449165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3313195034900218,
+      "epoch": 0.512,
+      "grad_norm": 0.384765625,
+      "kd_loss": 0.21447069352492037,
+      "learning_rate": 3e-06,
+      "loss": 0.3762,
+      "masked_tokens": 110.2875,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.2875
+    },
+    {
+      "avg_mask_ratio": 0.4476269483449869,
+      "avg_response_length": 243.2375,
+      "avg_student_mask_ratio": 0.4476269483449869,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2578068570326423,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.1669921875,
+      "kd_loss": 0.1731222485599604,
+      "learning_rate": 3e-06,
+      "loss": 0.2859,
+      "masked_tokens": 112.6375,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 112.6375
+    },
+    {
+      "avg_mask_ratio": 0.45657019784557634,
+      "avg_response_length": 245.55,
+      "avg_student_mask_ratio": 0.45657019784557634,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23127953946784602,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.1577436012762064,
+      "learning_rate": 3e-06,
+      "loss": 0.2823,
+      "masked_tokens": 110.0375,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 110.0375
+    },
+    {
+      "avg_mask_ratio": 0.5293830037582665,
+      "avg_response_length": 223.975,
+      "avg_student_mask_ratio": 0.5293830037582665,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3770375440454927,
+      "epoch": 0.576,
+      "grad_norm": 0.8359375,
+      "kd_loss": 0.2504498413263718,
+      "learning_rate": 3e-06,
+      "loss": 0.4807,
+      "masked_tokens": 119.6,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 119.6
+    },
+    {
+      "avg_mask_ratio": 0.4577330934116617,
+      "avg_response_length": 257.2125,
+      "avg_student_mask_ratio": 0.4577330934116617,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5647278043467623,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.421875,
+      "kd_loss": 0.18768687424256,
+      "learning_rate": 3e-06,
+      "loss": 0.3942,
+      "masked_tokens": 130.475,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 130.475
+    },
+    {
+      "avg_mask_ratio": 0.39295024327002465,
+      "avg_response_length": 246.6375,
+      "avg_student_mask_ratio": 0.39295024327002465,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.22343009332175826,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.1796875,
+      "kd_loss": 0.10545187053209588,
+      "learning_rate": 3e-06,
+      "loss": 0.2961,
+      "masked_tokens": 100.9,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.9
+    },
+    {
+      "avg_mask_ratio": 0.4409991275751963,
+      "avg_response_length": 217.9125,
+      "avg_student_mask_ratio": 0.4409991275751963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2520033180570181,
+      "epoch": 0.64,
+      "grad_norm": 0.1904296875,
+      "kd_loss": 0.1336840357888832,
+      "learning_rate": 3e-06,
+      "loss": 0.3042,
+      "masked_tokens": 106.925,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 106.925
+    },
+    {
+      "avg_mask_ratio": 0.47207197032403203,
+      "avg_response_length": 188.9125,
+      "avg_student_mask_ratio": 0.47207197032403203,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3869200809263475,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.19366624409758515,
+      "learning_rate": 3e-06,
+      "loss": 0.4395,
+      "masked_tokens": 85.125,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 85.125
+    },
+    {
+      "avg_mask_ratio": 0.4926959708333015,
+      "avg_response_length": 248.4,
+      "avg_student_mask_ratio": 0.4926959708333015,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4449735159470379,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.2666319968902826,
+      "learning_rate": 3e-06,
+      "loss": 0.5082,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5075328870676458,
+      "avg_response_length": 235.075,
+      "avg_student_mask_ratio": 0.5075328870676458,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4397935895413411,
+      "epoch": 0.704,
+      "grad_norm": 0.46484375,
+      "kd_loss": 0.21343186177154222,
+      "learning_rate": 3e-06,
+      "loss": 0.4382,
+      "masked_tokens": 127.9,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 127.9
+    },
+    {
+      "avg_mask_ratio": 0.44940012450679206,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.44940012450679206,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.508160601661291,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.89453125,
+      "kd_loss": 0.18509022962782815,
+      "learning_rate": 3e-06,
+      "loss": 0.4375,
+      "masked_tokens": 110.15,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 110.15
+    },
+    {
+      "avg_mask_ratio": 0.4945301389612723,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.4945301389612723,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6931099190221858,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.89453125,
+      "kd_loss": 0.2149010855906642,
+      "learning_rate": 3e-06,
+      "loss": 0.4849,
+      "masked_tokens": 125.175,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 125.175
+    },
+    {
+      "avg_mask_ratio": 0.4749954905593768,
+      "avg_response_length": 243.575,
+      "avg_student_mask_ratio": 0.4749954905593768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.27926406990906116,
+      "epoch": 0.768,
+      "grad_norm": 0.1376953125,
+      "kd_loss": 0.16614201006656232,
+      "learning_rate": 3e-06,
+      "loss": 0.3252,
+      "masked_tokens": 115.3875,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 115.3875
+    },
+    {
+      "avg_mask_ratio": 0.47621052770409733,
+      "avg_response_length": 257.575,
+      "avg_student_mask_ratio": 0.47621052770409733,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5629604070229106,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.12060546875,
+      "kd_loss": 0.21382154841430748,
+      "learning_rate": 3e-06,
+      "loss": 0.4245,
+      "masked_tokens": 126.85,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 126.85
+    },
+    {
+      "avg_mask_ratio": 0.4449806016869843,
+      "avg_response_length": 226.3625,
+      "avg_student_mask_ratio": 0.4449806016869843,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.19371982016177752,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.91796875,
+      "kd_loss": 0.13640525953642282,
+      "learning_rate": 3e-06,
+      "loss": 0.3082,
+      "masked_tokens": 109.35,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 109.35
+    },
+    {
+      "avg_mask_ratio": 0.47845896739745514,
+      "avg_response_length": 218.1125,
+      "avg_student_mask_ratio": 0.47845896739745514,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46541876822730044,
+      "epoch": 0.832,
+      "grad_norm": 0.2578125,
+      "kd_loss": 0.14829268052657757,
+      "learning_rate": 3e-06,
+      "loss": 0.395,
+      "masked_tokens": 111.85,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 111.85
+    },
+    {
+      "avg_mask_ratio": 0.4727763219270855,
+      "avg_response_length": 242.6125,
+      "avg_student_mask_ratio": 0.4727763219270855,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42413156108468686,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 1.6796875,
+      "kd_loss": 0.21946758147294076,
+      "learning_rate": 3e-06,
+      "loss": 0.4026,
+      "masked_tokens": 119.775,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 119.775
+    },
+    {
+      "avg_mask_ratio": 0.4756184325611684,
+      "avg_response_length": 239.5375,
+      "avg_student_mask_ratio": 0.4756184325611684,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.606393156799436,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.322265625,
+      "kd_loss": 0.20679433524416027,
+      "learning_rate": 3e-06,
+      "loss": 0.4871,
+      "masked_tokens": 122.1875,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 122.1875
+    },
+    {
+      "avg_mask_ratio": 0.4428858984610997,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4428858984610997,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1625635328691999,
+      "epoch": 0.896,
+      "grad_norm": 0.92578125,
+      "kd_loss": 0.14000170415542926,
+      "learning_rate": 3e-06,
+      "loss": 0.236,
+      "masked_tokens": 86.0125,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 86.0125
+    },
+    {
+      "avg_mask_ratio": 0.4625907339621335,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4625907339621335,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2754062189423848,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.15317172392096837,
+      "learning_rate": 3e-06,
+      "loss": 0.3358,
+      "masked_tokens": 103.675,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 103.675
+    },
+    {
+      "avg_mask_ratio": 0.4465438393759541,
+      "avg_response_length": 241.9625,
+      "avg_student_mask_ratio": 0.4465438393759541,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35291280565822947,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.3046875,
+      "kd_loss": 0.19183106588135387,
+      "learning_rate": 3e-06,
+      "loss": 0.3552,
+      "masked_tokens": 114.9125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 114.9125
+    },
+    {
+      "avg_mask_ratio": 0.42805201532319187,
+      "avg_response_length": 231.9875,
+      "avg_student_mask_ratio": 0.42805201532319187,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24969172771297962,
+      "epoch": 0.96,
+      "grad_norm": 0.63671875,
+      "kd_loss": 0.14995061269372628,
+      "learning_rate": 3e-06,
+      "loss": 0.3279,
+      "masked_tokens": 103.875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 103.875
+    },
+    {
+      "avg_mask_ratio": 0.4651826085988432,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.4651826085988432,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4396149708533358,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.10009765625,
+      "kd_loss": 0.15930728618550347,
+      "learning_rate": 3e-06,
+      "loss": 0.4374,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.47693050167124185,
+      "avg_response_length": 226.16666666666666,
+      "avg_student_mask_ratio": 0.47693050167124185,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4381092349216964,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.19419911270465343,
+      "learning_rate": 3e-06,
+      "loss": 0.5193,
+      "masked_tokens": 109.10714285714286,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 109.10714285714286
+    },
+    {
+      "avg_mask_ratio": 0.47416816898621617,
+      "avg_response_length": 250.25,
+      "avg_student_mask_ratio": 0.47416816898621617,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5465742808028722,
+      "epoch": 1.0256,
+      "grad_norm": 0.4921875,
+      "kd_loss": 0.2201738972554949,
+      "learning_rate": 3e-06,
+      "loss": 0.4372,
+      "masked_tokens": 117.9125,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 117.9125
+    },
+    {
+      "avg_mask_ratio": 0.45927587888436394,
+      "avg_response_length": 233.05,
+      "avg_student_mask_ratio": 0.45927587888436394,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3881588882651158,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.671875,
+      "kd_loss": 0.1297203417185301,
+      "learning_rate": 3e-06,
+      "loss": 0.3693,
+      "masked_tokens": 98.8375,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 98.8375
+    },
+    {
+      "avg_mask_ratio": 0.5047377114649862,
+      "avg_response_length": 260.225,
+      "avg_student_mask_ratio": 0.5047377114649862,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2485250894494584,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.15780083275045698,
+      "learning_rate": 3e-06,
+      "loss": 0.2985,
+      "masked_tokens": 127.4,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.4
+    },
+    {
+      "avg_mask_ratio": 0.5005011082510464,
+      "avg_response_length": 252.05,
+      "avg_student_mask_ratio": 0.5005011082510464,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3350205140219941,
+      "epoch": 1.0896,
+      "grad_norm": 2.953125,
+      "kd_loss": 0.21016031454611267,
+      "learning_rate": 3e-06,
+      "loss": 0.4012,
+      "masked_tokens": 133.5,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 133.5
+    },
+    {
+      "avg_mask_ratio": 0.4552151845302433,
+      "avg_response_length": 200.7625,
+      "avg_student_mask_ratio": 0.4552151845302433,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5980809078201788,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.70703125,
+      "kd_loss": 0.1575855805382389,
+      "learning_rate": 3e-06,
+      "loss": 0.4353,
+      "masked_tokens": 86.475,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 86.475
+    },
+    {
+      "avg_mask_ratio": 0.46727682640776036,
+      "avg_response_length": 214.5375,
+      "avg_student_mask_ratio": 0.46727682640776036,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44713975176116494,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.5625,
+      "kd_loss": 0.2050198341205757,
+      "learning_rate": 3e-06,
+      "loss": 0.4469,
+      "masked_tokens": 99.8375,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 99.8375
+    },
+    {
+      "avg_mask_ratio": 0.48785575344227256,
+      "avg_response_length": 231.3125,
+      "avg_student_mask_ratio": 0.48785575344227256,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35778319709847894,
+      "epoch": 1.1536,
+      "grad_norm": 0.318359375,
+      "kd_loss": 0.2095373813806191,
+      "learning_rate": 3e-06,
+      "loss": 0.4302,
+      "masked_tokens": 106.625,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 106.625
+    },
+    {
+      "avg_mask_ratio": 0.4477671392261982,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4477671392261982,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3559980413245398,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.283203125,
+      "kd_loss": 0.16553757517219764,
+      "learning_rate": 3e-06,
+      "loss": 0.3392,
+      "masked_tokens": 97.7,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 97.7
+    },
+    {
+      "avg_mask_ratio": 0.3861591775319539,
+      "avg_response_length": 238.5375,
+      "avg_student_mask_ratio": 0.3861591775319539,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25838428690098,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.365234375,
+      "kd_loss": 0.19571233854148887,
+      "learning_rate": 3e-06,
+      "loss": 0.3186,
+      "masked_tokens": 99.625,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 99.625
+    },
+    {
+      "avg_mask_ratio": 0.44424078196752814,
+      "avg_response_length": 230.1625,
+      "avg_student_mask_ratio": 0.44424078196752814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.30152245728992283,
+      "epoch": 1.2176,
+      "grad_norm": 0.703125,
+      "kd_loss": 0.16761301296596684,
+      "learning_rate": 3e-06,
+      "loss": 0.3934,
+      "masked_tokens": 108.0125,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 108.0125
+    },
+    {
+      "avg_mask_ratio": 0.4715048542013392,
+      "avg_response_length": 269.1375,
+      "avg_student_mask_ratio": 0.4715048542013392,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3189941981676611,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.93359375,
+      "kd_loss": 0.2090823985024258,
+      "learning_rate": 3e-06,
+      "loss": 0.3805,
+      "masked_tokens": 129.4375,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 129.4375
+    },
+    {
+      "avg_mask_ratio": 0.520072100055404,
+      "avg_response_length": 228.2875,
+      "avg_student_mask_ratio": 0.520072100055404,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4312982382521682,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.2421910996934457,
+      "learning_rate": 3e-06,
+      "loss": 0.4936,
+      "masked_tokens": 121.1625,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 121.1625
+    },
+    {
+      "avg_mask_ratio": 0.4505112706683576,
+      "avg_response_length": 237.1625,
+      "avg_student_mask_ratio": 0.4505112706683576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3104078326705803,
+      "epoch": 1.2816,
+      "grad_norm": 0.62890625,
+      "kd_loss": 0.17020997749766878,
+      "learning_rate": 3e-06,
+      "loss": 0.3534,
+      "masked_tokens": 110.0875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 110.0875
+    },
+    {
+      "avg_mask_ratio": 0.5064190638251602,
+      "avg_response_length": 244.3,
+      "avg_student_mask_ratio": 0.5064190638251602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.402807544786765,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.1942155301481648,
+      "learning_rate": 3e-06,
+      "loss": 0.3868,
+      "masked_tokens": 119.825,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 119.825
+    },
+    {
+      "avg_mask_ratio": 0.4891548154759221,
+      "avg_response_length": 234.6625,
+      "avg_student_mask_ratio": 0.4891548154759221,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3629302057381551,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.178446790360789,
+      "learning_rate": 3e-06,
+      "loss": 0.3871,
+      "masked_tokens": 110.5625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 110.5625
+    },
+    {
+      "avg_mask_ratio": 0.5235460251918994,
+      "avg_response_length": 258.0875,
+      "avg_student_mask_ratio": 0.5235460251918994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5023660743239304,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.2248805959525349,
+      "learning_rate": 3e-06,
+      "loss": 0.4892,
+      "masked_tokens": 130.725,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.725
+    },
+    {
+      "avg_mask_ratio": 0.4871393243782222,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.4871393243782222,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.29940326604166784,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.18196311790991332,
+      "learning_rate": 3e-06,
+      "loss": 0.2959,
+      "masked_tokens": 121.4625,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 121.4625
+    },
+    {
+      "avg_mask_ratio": 0.5254402696969919,
+      "avg_response_length": 240.575,
+      "avg_student_mask_ratio": 0.5254402696969919,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6097493719300247,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.265625,
+      "kd_loss": 0.20292491851313663,
+      "learning_rate": 3e-06,
+      "loss": 0.3965,
+      "masked_tokens": 133.725,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 133.725
+    },
+    {
+      "avg_mask_ratio": 0.48242234602803363,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.48242234602803363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35620763409436335,
+      "epoch": 1.4096,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.1878819163746357,
+      "learning_rate": 3e-06,
+      "loss": 0.4048,
+      "masked_tokens": 128.4625,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 128.4625
+    },
+    {
+      "avg_mask_ratio": 0.4551548367831856,
+      "avg_response_length": 247.3,
+      "avg_student_mask_ratio": 0.4551548367831856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3581841780794434,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.19171390956483023,
+      "learning_rate": 3e-06,
+      "loss": 0.3241,
+      "masked_tokens": 127.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 127.125
+    },
+    {
+      "avg_mask_ratio": 0.4658544249658007,
+      "avg_response_length": 224.7375,
+      "avg_student_mask_ratio": 0.4658544249658007,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3943864138858316,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.40625,
+      "kd_loss": 0.2207362408495353,
+      "learning_rate": 3e-06,
+      "loss": 0.3831,
+      "masked_tokens": 107.325,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 107.325
+    },
+    {
+      "avg_mask_ratio": 0.4795732157537714,
+      "avg_response_length": 200.975,
+      "avg_student_mask_ratio": 0.4795732157537714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38489766512459483,
+      "epoch": 1.4736,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.18907199591662902,
+      "learning_rate": 3e-06,
+      "loss": 0.3422,
+      "masked_tokens": 110.9375,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 110.9375
+    },
+    {
+      "avg_mask_ratio": 0.5250519359949977,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5250519359949977,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23998604974298132,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.22733612624851957,
+      "learning_rate": 3e-06,
+      "loss": 0.4221,
+      "masked_tokens": 132.55,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 132.55
+    },
+    {
+      "avg_mask_ratio": 0.4788092178525403,
+      "avg_response_length": 215.325,
+      "avg_student_mask_ratio": 0.4788092178525403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24203067874498174,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.259765625,
+      "kd_loss": 0.17745301808752317,
+      "learning_rate": 3e-06,
+      "loss": 0.3234,
+      "masked_tokens": 108.7125,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 108.7125
+    },
+    {
+      "avg_mask_ratio": 0.47950589570682495,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.47950589570682495,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3623171389520394,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.22090139569954773,
+      "learning_rate": 3e-06,
+      "loss": 0.3885,
+      "masked_tokens": 110.8,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 110.8
+    },
+    {
+      "avg_mask_ratio": 0.4604924251558259,
+      "avg_response_length": 232.925,
+      "avg_student_mask_ratio": 0.4604924251558259,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.644551625979841,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.455078125,
+      "kd_loss": 0.1767749028022223,
+      "learning_rate": 3e-06,
+      "loss": 0.4317,
+      "masked_tokens": 104.9625,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 104.9625
+    },
+    {
+      "avg_mask_ratio": 0.5185885130194947,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5185885130194947,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3369603527123218,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.32421875,
+      "kd_loss": 0.2044303039108854,
+      "learning_rate": 3e-06,
+      "loss": 0.4135,
+      "masked_tokens": 97.125,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 97.125
+    },
+    {
+      "avg_mask_ratio": 0.4191439319110941,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.4191439319110941,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37243218952012286,
+      "epoch": 1.6016,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.20017539288881814,
+      "learning_rate": 3e-06,
+      "loss": 0.3757,
+      "masked_tokens": 95.3125,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 95.3125
+    },
+    {
+      "avg_mask_ratio": 0.46666589792585,
+      "avg_response_length": 228.5125,
+      "avg_student_mask_ratio": 0.46666589792585,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3059532491540608,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.248046875,
+      "kd_loss": 0.21255611882733944,
+      "learning_rate": 3e-06,
+      "loss": 0.4021,
+      "masked_tokens": 117.2625,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 117.2625
+    },
+    {
+      "avg_mask_ratio": 0.4440126782981679,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4440126782981679,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28341264147752554,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.396484375,
+      "kd_loss": 0.15398789559246354,
+      "learning_rate": 3e-06,
+      "loss": 0.3165,
+      "masked_tokens": 117.8,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 117.8
+    },
+    {
+      "avg_mask_ratio": 0.42723098206624854,
+      "avg_response_length": 258.0125,
+      "avg_student_mask_ratio": 0.42723098206624854,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.413171638951826,
+      "epoch": 1.6656,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.12727623267242053,
+      "learning_rate": 3e-06,
+      "loss": 0.3069,
+      "masked_tokens": 118.575,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 118.575
+    },
+    {
+      "avg_mask_ratio": 0.46588709874195045,
+      "avg_response_length": 220.7,
+      "avg_student_mask_ratio": 0.46588709874195045,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.21302388652923127,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.1611328125,
+      "kd_loss": 0.164701623401076,
+      "learning_rate": 3e-06,
+      "loss": 0.2692,
+      "masked_tokens": 102.7625,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 102.7625
+    },
+    {
+      "avg_mask_ratio": 0.46366424662992356,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.46366424662992356,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28800638252976113,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 1.046875,
+      "kd_loss": 0.1414074915684978,
+      "learning_rate": 3e-06,
+      "loss": 0.2934,
+      "masked_tokens": 104.5,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 104.5
+    },
+    {
+      "avg_mask_ratio": 0.5037169002695009,
+      "avg_response_length": 250.2875,
+      "avg_student_mask_ratio": 0.5037169002695009,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6385474821537969,
+      "epoch": 1.7296,
+      "grad_norm": 0.76953125,
+      "kd_loss": 0.18596106451021796,
+      "learning_rate": 3e-06,
+      "loss": 0.534,
+      "masked_tokens": 119.575,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 119.575
+    },
+    {
+      "avg_mask_ratio": 0.5109186505898833,
+      "avg_response_length": 225.95,
+      "avg_student_mask_ratio": 0.5109186505898833,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46484450603404637,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.859375,
+      "kd_loss": 0.24066244094453565,
+      "learning_rate": 3e-06,
+      "loss": 0.4932,
+      "masked_tokens": 129.25,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 129.25
+    },
+    {
+      "avg_mask_ratio": 0.4988811274059117,
+      "avg_response_length": 263.7875,
+      "avg_student_mask_ratio": 0.4988811274059117,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4519748693252268,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.1826171875,
+      "kd_loss": 0.19858365579191287,
+      "learning_rate": 3e-06,
+      "loss": 0.3708,
+      "masked_tokens": 137.0,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 137.0
+    },
+    {
+      "avg_mask_ratio": 0.4997270987310912,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.4997270987310912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46417846596852996,
+      "epoch": 1.7936,
+      "grad_norm": 0.365234375,
+      "kd_loss": 0.17983142291315063,
+      "learning_rate": 3e-06,
+      "loss": 0.3971,
+      "masked_tokens": 114.525,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 114.525
+    },
+    {
+      "avg_mask_ratio": 0.4988076956477016,
+      "avg_response_length": 225.5,
+      "avg_student_mask_ratio": 0.4988076956477016,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3025185387264401,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.17459867448646946,
+      "learning_rate": 3e-06,
+      "loss": 0.3211,
+      "masked_tokens": 116.9125,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 116.9125
+    },
+    {
+      "avg_mask_ratio": 0.4497753610135987,
+      "avg_response_length": 260.15,
+      "avg_student_mask_ratio": 0.4497753610135987,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43239108170809004,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.1513671875,
+      "kd_loss": 0.21549869684477016,
+      "learning_rate": 3e-06,
+      "loss": 0.3711,
+      "masked_tokens": 121.7625,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 121.7625
+    },
+    {
+      "avg_mask_ratio": 0.48591957957251,
+      "avg_response_length": 231.0875,
+      "avg_student_mask_ratio": 0.48591957957251,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3273810862849018,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.41796875,
+      "kd_loss": 0.18414456472100937,
+      "learning_rate": 3e-06,
+      "loss": 0.362,
+      "masked_tokens": 116.7625,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 116.7625
+    },
+    {
+      "avg_mask_ratio": 0.44369487821822984,
+      "avg_response_length": 197.9125,
+      "avg_student_mask_ratio": 0.44369487821822984,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23855479693941106,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.16760400208100634,
+      "learning_rate": 3e-06,
+      "loss": 0.3392,
+      "masked_tokens": 91.4125,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 91.4125
+    },
+    {
+      "avg_mask_ratio": 0.44944015803339427,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.44944015803339427,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.21236650576719285,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.357421875,
+      "kd_loss": 0.15185958203733207,
+      "learning_rate": 3e-06,
+      "loss": 0.2372,
+      "masked_tokens": 103.2375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 103.2375
+    },
+    {
+      "avg_mask_ratio": 0.45069065956631676,
+      "avg_response_length": 230.175,
+      "avg_student_mask_ratio": 0.45069065956631676,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2341734012890015,
+      "epoch": 1.9216,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.18844677137070107,
+      "learning_rate": 3e-06,
+      "loss": 0.3624,
+      "masked_tokens": 104.325,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.475881968671456,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.475881968671456,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24741036754367088,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.3125,
+      "kd_loss": 0.15667265115065448,
+      "learning_rate": 3e-06,
+      "loss": 0.3189,
+      "masked_tokens": 125.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 125.1625
+    },
+    {
+      "avg_mask_ratio": 0.4633113604504615,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4633113604504615,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23597910864411914,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.17416997326051825,
+      "learning_rate": 3e-06,
+      "loss": 0.3294,
+      "masked_tokens": 109.5375,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 109.5375
+    },
+    {
+      "avg_mask_ratio": 0.47329409609083084,
+      "avg_response_length": 244.875,
+      "avg_student_mask_ratio": 0.47329409609083084,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2694289070637751,
+      "epoch": 1.9856,
+      "grad_norm": 0.2255859375,
+      "kd_loss": 0.16790816846706277,
+      "learning_rate": 3e-06,
+      "loss": 0.3419,
+      "masked_tokens": 120.525,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 120.525
+    },
+    {
+      "avg_mask_ratio": 0.4973435569776311,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.4973435569776311,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34412962615628895,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.18454486661500727,
+      "learning_rate": 3e-06,
+      "loss": 0.3697,
+      "masked_tokens": 120.63095238095238,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 120.63095238095238
+    },
+    {
+      "avg_mask_ratio": 0.4365456592233386,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4365456592233386,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28184777884216033,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.6875,
+      "kd_loss": 0.15645663726530615,
+      "learning_rate": 3e-06,
+      "loss": 0.2725,
+      "masked_tokens": 118.8125,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 118.8125
+    },
+    {
+      "avg_mask_ratio": 0.4914003949146718,
+      "avg_response_length": 275.3,
+      "avg_student_mask_ratio": 0.4914003949146718,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43313308064230116,
+      "epoch": 2.0512,
+      "grad_norm": 0.5625,
+      "kd_loss": 0.2300663744529629,
+      "learning_rate": 3e-06,
+      "loss": 0.4182,
+      "masked_tokens": 143.075,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 143.075
+    },
+    {
+      "avg_mask_ratio": 0.4381961932755075,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4381961932755075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34196077592644086,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.1847320883095051,
+      "learning_rate": 3e-06,
+      "loss": 0.3923,
+      "masked_tokens": 106.5125,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 106.5125
+    },
+    {
+      "avg_mask_ratio": 0.42702240714570505,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.42702240714570505,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4801598512272108,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.14269168120167705,
+      "learning_rate": 3e-06,
+      "loss": 0.3004,
+      "masked_tokens": 98.525,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 98.525
+    },
+    {
+      "avg_mask_ratio": 0.4583221158827655,
+      "avg_response_length": 262.0375,
+      "avg_student_mask_ratio": 0.4583221158827655,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5193568647300936,
+      "epoch": 2.1152,
+      "grad_norm": 0.5234375,
+      "kd_loss": 0.15391424604512166,
+      "learning_rate": 3e-06,
+      "loss": 0.3657,
+      "masked_tokens": 121.2875,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 121.2875
+    },
+    {
+      "avg_mask_ratio": 0.45086776099633424,
+      "avg_response_length": 214.925,
+      "avg_student_mask_ratio": 0.45086776099633424,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.20815674337951579,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.470703125,
+      "kd_loss": 0.1478596617299445,
+      "learning_rate": 3e-06,
+      "loss": 0.3149,
+      "masked_tokens": 100.4875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 100.4875
+    },
+    {
+      "avg_mask_ratio": 0.4363243154773954,
+      "avg_response_length": 224.3,
+      "avg_student_mask_ratio": 0.4363243154773954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.47673265045309565,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.07421875,
+      "kd_loss": 0.14714571952575567,
+      "learning_rate": 3e-06,
+      "loss": 0.3653,
+      "masked_tokens": 89.5,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 89.5
+    },
+    {
+      "avg_mask_ratio": 0.5063220548443497,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5063220548443497,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38743265072380384,
+      "epoch": 2.1792,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.18382307874209686,
+      "learning_rate": 3e-06,
+      "loss": 0.3938,
+      "masked_tokens": 110.25,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 110.25
+    },
+    {
+      "avg_mask_ratio": 0.46985941788880153,
+      "avg_response_length": 220.05,
+      "avg_student_mask_ratio": 0.46985941788880153,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43328518526819837,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.19683520471950544,
+      "learning_rate": 3e-06,
+      "loss": 0.395,
+      "masked_tokens": 104.9,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 104.9
+    },
+    {
+      "avg_mask_ratio": 0.49566771630197765,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.49566771630197765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2710993955800006,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.94140625,
+      "kd_loss": 0.18894708146090125,
+      "learning_rate": 3e-06,
+      "loss": 0.4122,
+      "masked_tokens": 100.35,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 100.35
+    },
+    {
+      "avg_mask_ratio": 0.5123685836791992,
+      "avg_response_length": 238.8,
+      "avg_student_mask_ratio": 0.5123685836791992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.30231127230786115,
+      "epoch": 2.2432,
+      "grad_norm": 0.64453125,
+      "kd_loss": 0.21115153960363386,
+      "learning_rate": 3e-06,
+      "loss": 0.3488,
+      "masked_tokens": 122.875,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 122.875
+    },
+    {
+      "avg_mask_ratio": 0.46218636581033934,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.46218636581033934,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33772741691473984,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.15173700823205394,
+      "learning_rate": 3e-06,
+      "loss": 0.2989,
+      "masked_tokens": 122.125,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 122.125
+    },
+    {
+      "avg_mask_ratio": 0.485661978519056,
+      "avg_response_length": 275.075,
+      "avg_student_mask_ratio": 0.485661978519056,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4382721236863517,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.375,
+      "kd_loss": 0.195586445067174,
+      "learning_rate": 3e-06,
+      "loss": 0.365,
+      "masked_tokens": 142.6375,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 142.6375
+    },
+    {
+      "avg_mask_ratio": 0.4626998565625399,
+      "avg_response_length": 214.45,
+      "avg_student_mask_ratio": 0.4626998565625399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.22534348728113401,
+      "epoch": 2.3072,
+      "grad_norm": 0.2109375,
+      "kd_loss": 0.16271676397591364,
+      "learning_rate": 3e-06,
+      "loss": 0.2785,
+      "masked_tokens": 98.4,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 98.4
+    },
+    {
+      "avg_mask_ratio": 0.44248262273031286,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.44248262273031286,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2362390007866111,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.05126953125,
+      "kd_loss": 0.12846805150115514,
+      "learning_rate": 3e-06,
+      "loss": 0.3103,
+      "masked_tokens": 91.9125,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 91.9125
+    },
+    {
+      "avg_mask_ratio": 0.5204601250356063,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.5204601250356063,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4075917797143461,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.22264093817780123,
+      "learning_rate": 3e-06,
+      "loss": 0.4293,
+      "masked_tokens": 133.1875,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 133.1875
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b968dbbf06e2017aeda9d45021ff20554875b30a6340f05bc88d8864eccb6012
+size 8120

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60317dd72c65e83828106c352049f31623517ac3ec671c6322447603e7648168
+size 2406624648

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9eea88f0147c5b78f4ae98528ec9300c217b28c207cb21d3113d8d92e1d5b1d0
+size 671304442

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1394463a46489e6dce7c0369a296b9effad20c6a87b30dbb892b34b73b5d6365
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d76e37c34ff78ced346b4f015037d8f7548792061c1d4914c31c56700f59f65f
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f8c95a6d9085dfcee1e6620c88ede526366d3a02c5018932b1bc04809c0e0c7
+size 1064

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/trainer_state.json

@@ -0,0 +1,2913 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.5632,
+  "eval_steps": 500,
+  "global_step": 1200,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.4930951670394279,
+      "avg_response_length": 224.7375,
+      "avg_student_mask_ratio": 0.4930951670394279,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.560375437145467,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.404296875,
+      "kd_loss": 0.25131759785651864,
+      "learning_rate": 3e-06,
+      "loss": 0.511,
+      "masked_tokens": 110.925,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 110.925
+    },
+    {
+      "avg_mask_ratio": 0.4127206720062532,
+      "avg_response_length": 277.15,
+      "avg_student_mask_ratio": 0.4127206720062532,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4417972476620662,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.65234375,
+      "kd_loss": 0.14074679291339579,
+      "learning_rate": 3e-06,
+      "loss": 0.3708,
+      "masked_tokens": 108.8625,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 108.8625
+    },
+    {
+      "avg_mask_ratio": 0.4616696212324314,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.4616696212324314,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4375385328372886,
+      "epoch": 0.064,
+      "grad_norm": 0.8671875,
+      "kd_loss": 0.19322810963351497,
+      "learning_rate": 3e-06,
+      "loss": 0.4217,
+      "masked_tokens": 111.375,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 111.375
+    },
+    {
+      "avg_mask_ratio": 0.42360913623997476,
+      "avg_response_length": 224.6125,
+      "avg_student_mask_ratio": 0.42360913623997476,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.39910652460303025,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 1.1328125,
+      "kd_loss": 0.15960182610455148,
+      "learning_rate": 3e-06,
+      "loss": 0.367,
+      "masked_tokens": 98.3,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 98.3
+    },
+    {
+      "avg_mask_ratio": 0.4330951495358022,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.4330951495358022,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6348971565400916,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.65625,
+      "kd_loss": 0.15276786715631374,
+      "learning_rate": 3e-06,
+      "loss": 0.4348,
+      "masked_tokens": 85.075,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 85.075
+    },
+    {
+      "avg_mask_ratio": 0.4555395155097358,
+      "avg_response_length": 254.2125,
+      "avg_student_mask_ratio": 0.4555395155097358,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5031439974316981,
+      "epoch": 0.128,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.195508924105161,
+      "learning_rate": 3e-06,
+      "loss": 0.4481,
+      "masked_tokens": 119.1125,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 119.1125
+    },
+    {
+      "avg_mask_ratio": 0.5148372989846394,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.5148372989846394,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6913914184368878,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.94921875,
+      "kd_loss": 0.17476431281479562,
+      "learning_rate": 3e-06,
+      "loss": 0.5543,
+      "masked_tokens": 105.45,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 105.45
+    },
+    {
+      "avg_mask_ratio": 0.3827478863298893,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.3827478863298893,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.40601554353645497,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.1317249322714872,
+      "learning_rate": 3e-06,
+      "loss": 0.2945,
+      "masked_tokens": 86.675,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 86.675
+    },
+    {
+      "avg_mask_ratio": 0.5017695252550766,
+      "avg_response_length": 234.25,
+      "avg_student_mask_ratio": 0.5017695252550766,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6200478935968476,
+      "epoch": 0.192,
+      "grad_norm": 1.0,
+      "kd_loss": 0.23770100035361139,
+      "learning_rate": 3e-06,
+      "loss": 0.6467,
+      "masked_tokens": 108.4125,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 108.4125
+    },
+    {
+      "avg_mask_ratio": 0.4637213449750561,
+      "avg_response_length": 210.175,
+      "avg_student_mask_ratio": 0.4637213449750561,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4699021781670581,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 1.4296875,
+      "kd_loss": 0.17425421165266286,
+      "learning_rate": 3e-06,
+      "loss": 0.4513,
+      "masked_tokens": 107.375,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 107.375
+    },
+    {
+      "avg_mask_ratio": 0.48738867897773164,
+      "avg_response_length": 213.0875,
+      "avg_student_mask_ratio": 0.48738867897773164,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2935672491260675,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.63671875,
+      "kd_loss": 0.21233718555377265,
+      "learning_rate": 3e-06,
+      "loss": 0.4251,
+      "masked_tokens": 101.7875,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 101.7875
+    },
+    {
+      "avg_mask_ratio": 0.4451883323024958,
+      "avg_response_length": 232.3,
+      "avg_student_mask_ratio": 0.4451883323024958,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3356981491514944,
+      "epoch": 0.256,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.23394757619298617,
+      "learning_rate": 3e-06,
+      "loss": 0.3743,
+      "masked_tokens": 107.7,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 107.7
+    },
+    {
+      "avg_mask_ratio": 0.43939279407495635,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.43939279407495635,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2916025636156746,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.734375,
+      "kd_loss": 0.14590738382988774,
+      "learning_rate": 3e-06,
+      "loss": 0.2778,
+      "masked_tokens": 89.95,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 89.95
+    },
+    {
+      "avg_mask_ratio": 0.4922366282902658,
+      "avg_response_length": 264.5375,
+      "avg_student_mask_ratio": 0.4922366282902658,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44275596315469556,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.1741957870854094,
+      "learning_rate": 3e-06,
+      "loss": 0.4423,
+      "masked_tokens": 144.9,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 144.9
+    },
+    {
+      "avg_mask_ratio": 0.4724786171223968,
+      "avg_response_length": 258.1125,
+      "avg_student_mask_ratio": 0.4724786171223968,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6182238667762021,
+      "epoch": 0.32,
+      "grad_norm": 0.051513671875,
+      "kd_loss": 0.2385995153230624,
+      "learning_rate": 3e-06,
+      "loss": 0.4876,
+      "masked_tokens": 127.4125,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 127.4125
+    },
+    {
+      "avg_mask_ratio": 0.49717973986989816,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.49717973986989816,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44498473816368006,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.216546824422403,
+      "learning_rate": 3e-06,
+      "loss": 0.4246,
+      "masked_tokens": 98.4875,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 98.4875
+    },
+    {
+      "avg_mask_ratio": 0.48284467663615943,
+      "avg_response_length": 188.65,
+      "avg_student_mask_ratio": 0.48284467663615943,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4632640206855626,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.19741544655316828,
+      "learning_rate": 3e-06,
+      "loss": 0.4259,
+      "masked_tokens": 89.3125,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 89.3125
+    },
+    {
+      "avg_mask_ratio": 0.44864035704231353,
+      "avg_response_length": 246.6875,
+      "avg_student_mask_ratio": 0.44864035704231353,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38090254231135245,
+      "epoch": 0.384,
+      "grad_norm": 0.35546875,
+      "kd_loss": 0.18000799598165002,
+      "learning_rate": 3e-06,
+      "loss": 0.4605,
+      "masked_tokens": 110.0125,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 110.0125
+    },
+    {
+      "avg_mask_ratio": 0.47850618849042803,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.47850618849042803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.524182743717688,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 1.4140625,
+      "kd_loss": 0.19971058982482645,
+      "learning_rate": 3e-06,
+      "loss": 0.5185,
+      "masked_tokens": 109.575,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 109.575
+    },
+    {
+      "avg_mask_ratio": 0.4662990250624716,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4662990250624716,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2655160516187379,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 2.65625,
+      "kd_loss": 0.11893262579505973,
+      "learning_rate": 3e-06,
+      "loss": 0.272,
+      "masked_tokens": 97.5,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 97.5
+    },
+    {
+      "avg_mask_ratio": 0.451080821454525,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.451080821454525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3671989720950819,
+      "epoch": 0.448,
+      "grad_norm": 2.53125,
+      "kd_loss": 0.15058198695817282,
+      "learning_rate": 3e-06,
+      "loss": 0.3553,
+      "masked_tokens": 96.6375,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 96.6375
+    },
+    {
+      "avg_mask_ratio": 0.5438536155037582,
+      "avg_response_length": 236.5125,
+      "avg_student_mask_ratio": 0.5438536155037582,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5652186098743186,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 2.21875,
+      "kd_loss": 0.2394665865747811,
+      "learning_rate": 3e-06,
+      "loss": 0.5566,
+      "masked_tokens": 126.4375,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 126.4375
+    },
+    {
+      "avg_mask_ratio": 0.43992503273766487,
+      "avg_response_length": 255.875,
+      "avg_student_mask_ratio": 0.43992503273766487,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6014562642475084,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.14900631055388658,
+      "learning_rate": 3e-06,
+      "loss": 0.4491,
+      "masked_tokens": 107.3875,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 107.3875
+    },
+    {
+      "avg_mask_ratio": 0.46683448635449165,
+      "avg_response_length": 224.125,
+      "avg_student_mask_ratio": 0.46683448635449165,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3313195034900218,
+      "epoch": 0.512,
+      "grad_norm": 0.384765625,
+      "kd_loss": 0.21447069352492037,
+      "learning_rate": 3e-06,
+      "loss": 0.3762,
+      "masked_tokens": 110.2875,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 110.2875
+    },
+    {
+      "avg_mask_ratio": 0.4476269483449869,
+      "avg_response_length": 243.2375,
+      "avg_student_mask_ratio": 0.4476269483449869,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2578068570326423,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.1669921875,
+      "kd_loss": 0.1731222485599604,
+      "learning_rate": 3e-06,
+      "loss": 0.2859,
+      "masked_tokens": 112.6375,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 112.6375
+    },
+    {
+      "avg_mask_ratio": 0.45657019784557634,
+      "avg_response_length": 245.55,
+      "avg_student_mask_ratio": 0.45657019784557634,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23127953946784602,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.828125,
+      "kd_loss": 0.1577436012762064,
+      "learning_rate": 3e-06,
+      "loss": 0.2823,
+      "masked_tokens": 110.0375,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 110.0375
+    },
+    {
+      "avg_mask_ratio": 0.5293830037582665,
+      "avg_response_length": 223.975,
+      "avg_student_mask_ratio": 0.5293830037582665,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3770375440454927,
+      "epoch": 0.576,
+      "grad_norm": 0.8359375,
+      "kd_loss": 0.2504498413263718,
+      "learning_rate": 3e-06,
+      "loss": 0.4807,
+      "masked_tokens": 119.6,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 119.6
+    },
+    {
+      "avg_mask_ratio": 0.4577330934116617,
+      "avg_response_length": 257.2125,
+      "avg_student_mask_ratio": 0.4577330934116617,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5647278043467623,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.421875,
+      "kd_loss": 0.18768687424256,
+      "learning_rate": 3e-06,
+      "loss": 0.3942,
+      "masked_tokens": 130.475,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 130.475
+    },
+    {
+      "avg_mask_ratio": 0.39295024327002465,
+      "avg_response_length": 246.6375,
+      "avg_student_mask_ratio": 0.39295024327002465,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.22343009332175826,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.1796875,
+      "kd_loss": 0.10545187053209588,
+      "learning_rate": 3e-06,
+      "loss": 0.2961,
+      "masked_tokens": 100.9,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 100.9
+    },
+    {
+      "avg_mask_ratio": 0.4409991275751963,
+      "avg_response_length": 217.9125,
+      "avg_student_mask_ratio": 0.4409991275751963,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2520033180570181,
+      "epoch": 0.64,
+      "grad_norm": 0.1904296875,
+      "kd_loss": 0.1336840357888832,
+      "learning_rate": 3e-06,
+      "loss": 0.3042,
+      "masked_tokens": 106.925,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 106.925
+    },
+    {
+      "avg_mask_ratio": 0.47207197032403203,
+      "avg_response_length": 188.9125,
+      "avg_student_mask_ratio": 0.47207197032403203,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3869200809263475,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.19366624409758515,
+      "learning_rate": 3e-06,
+      "loss": 0.4395,
+      "masked_tokens": 85.125,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 85.125
+    },
+    {
+      "avg_mask_ratio": 0.4926959708333015,
+      "avg_response_length": 248.4,
+      "avg_student_mask_ratio": 0.4926959708333015,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4449735159470379,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.2666319968902826,
+      "learning_rate": 3e-06,
+      "loss": 0.5082,
+      "masked_tokens": 124.5625,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 124.5625
+    },
+    {
+      "avg_mask_ratio": 0.5075328870676458,
+      "avg_response_length": 235.075,
+      "avg_student_mask_ratio": 0.5075328870676458,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4397935895413411,
+      "epoch": 0.704,
+      "grad_norm": 0.46484375,
+      "kd_loss": 0.21343186177154222,
+      "learning_rate": 3e-06,
+      "loss": 0.4382,
+      "masked_tokens": 127.9,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 127.9
+    },
+    {
+      "avg_mask_ratio": 0.44940012450679206,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.44940012450679206,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.508160601661291,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.89453125,
+      "kd_loss": 0.18509022962782815,
+      "learning_rate": 3e-06,
+      "loss": 0.4375,
+      "masked_tokens": 110.15,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 110.15
+    },
+    {
+      "avg_mask_ratio": 0.4945301389612723,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.4945301389612723,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6931099190221858,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.89453125,
+      "kd_loss": 0.2149010855906642,
+      "learning_rate": 3e-06,
+      "loss": 0.4849,
+      "masked_tokens": 125.175,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 125.175
+    },
+    {
+      "avg_mask_ratio": 0.4749954905593768,
+      "avg_response_length": 243.575,
+      "avg_student_mask_ratio": 0.4749954905593768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.27926406990906116,
+      "epoch": 0.768,
+      "grad_norm": 0.1376953125,
+      "kd_loss": 0.16614201006656232,
+      "learning_rate": 3e-06,
+      "loss": 0.3252,
+      "masked_tokens": 115.3875,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 115.3875
+    },
+    {
+      "avg_mask_ratio": 0.47621052770409733,
+      "avg_response_length": 257.575,
+      "avg_student_mask_ratio": 0.47621052770409733,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5629604070229106,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.12060546875,
+      "kd_loss": 0.21382154841430748,
+      "learning_rate": 3e-06,
+      "loss": 0.4245,
+      "masked_tokens": 126.85,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 126.85
+    },
+    {
+      "avg_mask_ratio": 0.4449806016869843,
+      "avg_response_length": 226.3625,
+      "avg_student_mask_ratio": 0.4449806016869843,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.19371982016177752,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.91796875,
+      "kd_loss": 0.13640525953642282,
+      "learning_rate": 3e-06,
+      "loss": 0.3082,
+      "masked_tokens": 109.35,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 109.35
+    },
+    {
+      "avg_mask_ratio": 0.47845896739745514,
+      "avg_response_length": 218.1125,
+      "avg_student_mask_ratio": 0.47845896739745514,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46541876822730044,
+      "epoch": 0.832,
+      "grad_norm": 0.2578125,
+      "kd_loss": 0.14829268052657757,
+      "learning_rate": 3e-06,
+      "loss": 0.395,
+      "masked_tokens": 111.85,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 111.85
+    },
+    {
+      "avg_mask_ratio": 0.4727763219270855,
+      "avg_response_length": 242.6125,
+      "avg_student_mask_ratio": 0.4727763219270855,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.42413156108468686,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 1.6796875,
+      "kd_loss": 0.21946758147294076,
+      "learning_rate": 3e-06,
+      "loss": 0.4026,
+      "masked_tokens": 119.775,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 119.775
+    },
+    {
+      "avg_mask_ratio": 0.4756184325611684,
+      "avg_response_length": 239.5375,
+      "avg_student_mask_ratio": 0.4756184325611684,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.606393156799436,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.322265625,
+      "kd_loss": 0.20679433524416027,
+      "learning_rate": 3e-06,
+      "loss": 0.4871,
+      "masked_tokens": 122.1875,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 122.1875
+    },
+    {
+      "avg_mask_ratio": 0.4428858984610997,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4428858984610997,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1625635328691999,
+      "epoch": 0.896,
+      "grad_norm": 0.92578125,
+      "kd_loss": 0.14000170415542926,
+      "learning_rate": 3e-06,
+      "loss": 0.236,
+      "masked_tokens": 86.0125,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 86.0125
+    },
+    {
+      "avg_mask_ratio": 0.4625907339621335,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4625907339621335,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2754062189423848,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.33984375,
+      "kd_loss": 0.15317172392096837,
+      "learning_rate": 3e-06,
+      "loss": 0.3358,
+      "masked_tokens": 103.675,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 103.675
+    },
+    {
+      "avg_mask_ratio": 0.4465438393759541,
+      "avg_response_length": 241.9625,
+      "avg_student_mask_ratio": 0.4465438393759541,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35291280565822947,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 1.3046875,
+      "kd_loss": 0.19183106588135387,
+      "learning_rate": 3e-06,
+      "loss": 0.3552,
+      "masked_tokens": 114.9125,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 114.9125
+    },
+    {
+      "avg_mask_ratio": 0.42805201532319187,
+      "avg_response_length": 231.9875,
+      "avg_student_mask_ratio": 0.42805201532319187,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24969172771297962,
+      "epoch": 0.96,
+      "grad_norm": 0.63671875,
+      "kd_loss": 0.14995061269372628,
+      "learning_rate": 3e-06,
+      "loss": 0.3279,
+      "masked_tokens": 103.875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 103.875
+    },
+    {
+      "avg_mask_ratio": 0.4651826085988432,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.4651826085988432,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4396149708533358,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.10009765625,
+      "kd_loss": 0.15930728618550347,
+      "learning_rate": 3e-06,
+      "loss": 0.4374,
+      "masked_tokens": 104.125,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 104.125
+    },
+    {
+      "avg_mask_ratio": 0.47693050167124185,
+      "avg_response_length": 226.16666666666666,
+      "avg_student_mask_ratio": 0.47693050167124185,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4381092349216964,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.90625,
+      "kd_loss": 0.19419911270465343,
+      "learning_rate": 3e-06,
+      "loss": 0.5193,
+      "masked_tokens": 109.10714285714286,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 109.10714285714286
+    },
+    {
+      "avg_mask_ratio": 0.47416816898621617,
+      "avg_response_length": 250.25,
+      "avg_student_mask_ratio": 0.47416816898621617,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5465742808028722,
+      "epoch": 1.0256,
+      "grad_norm": 0.4921875,
+      "kd_loss": 0.2201738972554949,
+      "learning_rate": 3e-06,
+      "loss": 0.4372,
+      "masked_tokens": 117.9125,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 117.9125
+    },
+    {
+      "avg_mask_ratio": 0.45927587888436394,
+      "avg_response_length": 233.05,
+      "avg_student_mask_ratio": 0.45927587888436394,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3881588882651158,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.671875,
+      "kd_loss": 0.1297203417185301,
+      "learning_rate": 3e-06,
+      "loss": 0.3693,
+      "masked_tokens": 98.8375,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 98.8375
+    },
+    {
+      "avg_mask_ratio": 0.5047377114649862,
+      "avg_response_length": 260.225,
+      "avg_student_mask_ratio": 0.5047377114649862,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2485250894494584,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.15780083275045698,
+      "learning_rate": 3e-06,
+      "loss": 0.2985,
+      "masked_tokens": 127.4,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 127.4
+    },
+    {
+      "avg_mask_ratio": 0.5005011082510464,
+      "avg_response_length": 252.05,
+      "avg_student_mask_ratio": 0.5005011082510464,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3350205140219941,
+      "epoch": 1.0896,
+      "grad_norm": 2.953125,
+      "kd_loss": 0.21016031454611267,
+      "learning_rate": 3e-06,
+      "loss": 0.4012,
+      "masked_tokens": 133.5,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 133.5
+    },
+    {
+      "avg_mask_ratio": 0.4552151845302433,
+      "avg_response_length": 200.7625,
+      "avg_student_mask_ratio": 0.4552151845302433,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5980809078201788,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.70703125,
+      "kd_loss": 0.1575855805382389,
+      "learning_rate": 3e-06,
+      "loss": 0.4353,
+      "masked_tokens": 86.475,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 86.475
+    },
+    {
+      "avg_mask_ratio": 0.46727682640776036,
+      "avg_response_length": 214.5375,
+      "avg_student_mask_ratio": 0.46727682640776036,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.44713975176116494,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.5625,
+      "kd_loss": 0.2050198341205757,
+      "learning_rate": 3e-06,
+      "loss": 0.4469,
+      "masked_tokens": 99.8375,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 99.8375
+    },
+    {
+      "avg_mask_ratio": 0.48785575344227256,
+      "avg_response_length": 231.3125,
+      "avg_student_mask_ratio": 0.48785575344227256,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35778319709847894,
+      "epoch": 1.1536,
+      "grad_norm": 0.318359375,
+      "kd_loss": 0.2095373813806191,
+      "learning_rate": 3e-06,
+      "loss": 0.4302,
+      "masked_tokens": 106.625,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 106.625
+    },
+    {
+      "avg_mask_ratio": 0.4477671392261982,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4477671392261982,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3559980413245398,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.283203125,
+      "kd_loss": 0.16553757517219764,
+      "learning_rate": 3e-06,
+      "loss": 0.3392,
+      "masked_tokens": 97.7,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 97.7
+    },
+    {
+      "avg_mask_ratio": 0.3861591775319539,
+      "avg_response_length": 238.5375,
+      "avg_student_mask_ratio": 0.3861591775319539,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.25838428690098,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.365234375,
+      "kd_loss": 0.19571233854148887,
+      "learning_rate": 3e-06,
+      "loss": 0.3186,
+      "masked_tokens": 99.625,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 99.625
+    },
+    {
+      "avg_mask_ratio": 0.44424078196752814,
+      "avg_response_length": 230.1625,
+      "avg_student_mask_ratio": 0.44424078196752814,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.30152245728992283,
+      "epoch": 1.2176,
+      "grad_norm": 0.703125,
+      "kd_loss": 0.16761301296596684,
+      "learning_rate": 3e-06,
+      "loss": 0.3934,
+      "masked_tokens": 108.0125,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 108.0125
+    },
+    {
+      "avg_mask_ratio": 0.4715048542013392,
+      "avg_response_length": 269.1375,
+      "avg_student_mask_ratio": 0.4715048542013392,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3189941981676611,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.93359375,
+      "kd_loss": 0.2090823985024258,
+      "learning_rate": 3e-06,
+      "loss": 0.3805,
+      "masked_tokens": 129.4375,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 129.4375
+    },
+    {
+      "avg_mask_ratio": 0.520072100055404,
+      "avg_response_length": 228.2875,
+      "avg_student_mask_ratio": 0.520072100055404,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4312982382521682,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 1.2734375,
+      "kd_loss": 0.2421910996934457,
+      "learning_rate": 3e-06,
+      "loss": 0.4936,
+      "masked_tokens": 121.1625,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 121.1625
+    },
+    {
+      "avg_mask_ratio": 0.4505112706683576,
+      "avg_response_length": 237.1625,
+      "avg_student_mask_ratio": 0.4505112706683576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3104078326705803,
+      "epoch": 1.2816,
+      "grad_norm": 0.62890625,
+      "kd_loss": 0.17020997749766878,
+      "learning_rate": 3e-06,
+      "loss": 0.3534,
+      "masked_tokens": 110.0875,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 110.0875
+    },
+    {
+      "avg_mask_ratio": 0.5064190638251602,
+      "avg_response_length": 244.3,
+      "avg_student_mask_ratio": 0.5064190638251602,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.402807544786765,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.1942155301481648,
+      "learning_rate": 3e-06,
+      "loss": 0.3868,
+      "masked_tokens": 119.825,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 119.825
+    },
+    {
+      "avg_mask_ratio": 0.4891548154759221,
+      "avg_response_length": 234.6625,
+      "avg_student_mask_ratio": 0.4891548154759221,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3629302057381551,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.178446790360789,
+      "learning_rate": 3e-06,
+      "loss": 0.3871,
+      "masked_tokens": 110.5625,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 110.5625
+    },
+    {
+      "avg_mask_ratio": 0.5235460251918994,
+      "avg_response_length": 258.0875,
+      "avg_student_mask_ratio": 0.5235460251918994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5023660743239304,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 1.015625,
+      "kd_loss": 0.2248805959525349,
+      "learning_rate": 3e-06,
+      "loss": 0.4892,
+      "masked_tokens": 130.725,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 130.725
+    },
+    {
+      "avg_mask_ratio": 0.4871393243782222,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.4871393243782222,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.29940326604166784,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.18196311790991332,
+      "learning_rate": 3e-06,
+      "loss": 0.2959,
+      "masked_tokens": 121.4625,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 121.4625
+    },
+    {
+      "avg_mask_ratio": 0.5254402696969919,
+      "avg_response_length": 240.575,
+      "avg_student_mask_ratio": 0.5254402696969919,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6097493719300247,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.265625,
+      "kd_loss": 0.20292491851313663,
+      "learning_rate": 3e-06,
+      "loss": 0.3965,
+      "masked_tokens": 133.725,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 133.725
+    },
+    {
+      "avg_mask_ratio": 0.48242234602803363,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.48242234602803363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.35620763409436335,
+      "epoch": 1.4096,
+      "grad_norm": 0.68359375,
+      "kd_loss": 0.1878819163746357,
+      "learning_rate": 3e-06,
+      "loss": 0.4048,
+      "masked_tokens": 128.4625,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 128.4625
+    },
+    {
+      "avg_mask_ratio": 0.4551548367831856,
+      "avg_response_length": 247.3,
+      "avg_student_mask_ratio": 0.4551548367831856,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3581841780794434,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.19171390956483023,
+      "learning_rate": 3e-06,
+      "loss": 0.3241,
+      "masked_tokens": 127.125,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 127.125
+    },
+    {
+      "avg_mask_ratio": 0.4658544249658007,
+      "avg_response_length": 224.7375,
+      "avg_student_mask_ratio": 0.4658544249658007,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3943864138858316,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.40625,
+      "kd_loss": 0.2207362408495353,
+      "learning_rate": 3e-06,
+      "loss": 0.3831,
+      "masked_tokens": 107.325,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 107.325
+    },
+    {
+      "avg_mask_ratio": 0.4795732157537714,
+      "avg_response_length": 200.975,
+      "avg_student_mask_ratio": 0.4795732157537714,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38489766512459483,
+      "epoch": 1.4736,
+      "grad_norm": 0.337890625,
+      "kd_loss": 0.18907199591662902,
+      "learning_rate": 3e-06,
+      "loss": 0.3422,
+      "masked_tokens": 110.9375,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 110.9375
+    },
+    {
+      "avg_mask_ratio": 0.5250519359949977,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.5250519359949977,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23998604974298132,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.24609375,
+      "kd_loss": 0.22733612624851957,
+      "learning_rate": 3e-06,
+      "loss": 0.4221,
+      "masked_tokens": 132.55,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 132.55
+    },
+    {
+      "avg_mask_ratio": 0.4788092178525403,
+      "avg_response_length": 215.325,
+      "avg_student_mask_ratio": 0.4788092178525403,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24203067874498174,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.259765625,
+      "kd_loss": 0.17745301808752317,
+      "learning_rate": 3e-06,
+      "loss": 0.3234,
+      "masked_tokens": 108.7125,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 108.7125
+    },
+    {
+      "avg_mask_ratio": 0.47950589570682495,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.47950589570682495,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3623171389520394,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.494140625,
+      "kd_loss": 0.22090139569954773,
+      "learning_rate": 3e-06,
+      "loss": 0.3885,
+      "masked_tokens": 110.8,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 110.8
+    },
+    {
+      "avg_mask_ratio": 0.4604924251558259,
+      "avg_response_length": 232.925,
+      "avg_student_mask_ratio": 0.4604924251558259,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.644551625979841,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.455078125,
+      "kd_loss": 0.1767749028022223,
+      "learning_rate": 3e-06,
+      "loss": 0.4317,
+      "masked_tokens": 104.9625,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 104.9625
+    },
+    {
+      "avg_mask_ratio": 0.5185885130194947,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.5185885130194947,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3369603527123218,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.32421875,
+      "kd_loss": 0.2044303039108854,
+      "learning_rate": 3e-06,
+      "loss": 0.4135,
+      "masked_tokens": 97.125,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 97.125
+    },
+    {
+      "avg_mask_ratio": 0.4191439319110941,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.4191439319110941,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.37243218952012286,
+      "epoch": 1.6016,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.20017539288881814,
+      "learning_rate": 3e-06,
+      "loss": 0.3757,
+      "masked_tokens": 95.3125,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 95.3125
+    },
+    {
+      "avg_mask_ratio": 0.46666589792585,
+      "avg_response_length": 228.5125,
+      "avg_student_mask_ratio": 0.46666589792585,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3059532491540608,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.248046875,
+      "kd_loss": 0.21255611882733944,
+      "learning_rate": 3e-06,
+      "loss": 0.4021,
+      "masked_tokens": 117.2625,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 117.2625
+    },
+    {
+      "avg_mask_ratio": 0.4440126782981679,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.4440126782981679,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28341264147752554,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.396484375,
+      "kd_loss": 0.15398789559246354,
+      "learning_rate": 3e-06,
+      "loss": 0.3165,
+      "masked_tokens": 117.8,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 117.8
+    },
+    {
+      "avg_mask_ratio": 0.42723098206624854,
+      "avg_response_length": 258.0125,
+      "avg_student_mask_ratio": 0.42723098206624854,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.413171638951826,
+      "epoch": 1.6656,
+      "grad_norm": 0.53515625,
+      "kd_loss": 0.12727623267242053,
+      "learning_rate": 3e-06,
+      "loss": 0.3069,
+      "masked_tokens": 118.575,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 118.575
+    },
+    {
+      "avg_mask_ratio": 0.46588709874195045,
+      "avg_response_length": 220.7,
+      "avg_student_mask_ratio": 0.46588709874195045,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.21302388652923127,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.1611328125,
+      "kd_loss": 0.164701623401076,
+      "learning_rate": 3e-06,
+      "loss": 0.2692,
+      "masked_tokens": 102.7625,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 102.7625
+    },
+    {
+      "avg_mask_ratio": 0.46366424662992356,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.46366424662992356,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28800638252976113,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 1.046875,
+      "kd_loss": 0.1414074915684978,
+      "learning_rate": 3e-06,
+      "loss": 0.2934,
+      "masked_tokens": 104.5,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 104.5
+    },
+    {
+      "avg_mask_ratio": 0.5037169002695009,
+      "avg_response_length": 250.2875,
+      "avg_student_mask_ratio": 0.5037169002695009,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6385474821537969,
+      "epoch": 1.7296,
+      "grad_norm": 0.76953125,
+      "kd_loss": 0.18596106451021796,
+      "learning_rate": 3e-06,
+      "loss": 0.534,
+      "masked_tokens": 119.575,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 119.575
+    },
+    {
+      "avg_mask_ratio": 0.5109186505898833,
+      "avg_response_length": 225.95,
+      "avg_student_mask_ratio": 0.5109186505898833,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46484450603404637,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.859375,
+      "kd_loss": 0.24066244094453565,
+      "learning_rate": 3e-06,
+      "loss": 0.4932,
+      "masked_tokens": 129.25,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 129.25
+    },
+    {
+      "avg_mask_ratio": 0.4988811274059117,
+      "avg_response_length": 263.7875,
+      "avg_student_mask_ratio": 0.4988811274059117,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4519748693252268,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.1826171875,
+      "kd_loss": 0.19858365579191287,
+      "learning_rate": 3e-06,
+      "loss": 0.3708,
+      "masked_tokens": 137.0,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 137.0
+    },
+    {
+      "avg_mask_ratio": 0.4997270987310912,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.4997270987310912,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.46417846596852996,
+      "epoch": 1.7936,
+      "grad_norm": 0.365234375,
+      "kd_loss": 0.17983142291315063,
+      "learning_rate": 3e-06,
+      "loss": 0.3971,
+      "masked_tokens": 114.525,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 114.525
+    },
+    {
+      "avg_mask_ratio": 0.4988076956477016,
+      "avg_response_length": 225.5,
+      "avg_student_mask_ratio": 0.4988076956477016,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3025185387264401,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.17459867448646946,
+      "learning_rate": 3e-06,
+      "loss": 0.3211,
+      "masked_tokens": 116.9125,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 116.9125
+    },
+    {
+      "avg_mask_ratio": 0.4497753610135987,
+      "avg_response_length": 260.15,
+      "avg_student_mask_ratio": 0.4497753610135987,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43239108170809004,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.1513671875,
+      "kd_loss": 0.21549869684477016,
+      "learning_rate": 3e-06,
+      "loss": 0.3711,
+      "masked_tokens": 121.7625,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 121.7625
+    },
+    {
+      "avg_mask_ratio": 0.48591957957251,
+      "avg_response_length": 231.0875,
+      "avg_student_mask_ratio": 0.48591957957251,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3273810862849018,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.41796875,
+      "kd_loss": 0.18414456472100937,
+      "learning_rate": 3e-06,
+      "loss": 0.362,
+      "masked_tokens": 116.7625,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 116.7625
+    },
+    {
+      "avg_mask_ratio": 0.44369487821822984,
+      "avg_response_length": 197.9125,
+      "avg_student_mask_ratio": 0.44369487821822984,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23855479693941106,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.16760400208100634,
+      "learning_rate": 3e-06,
+      "loss": 0.3392,
+      "masked_tokens": 91.4125,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 91.4125
+    },
+    {
+      "avg_mask_ratio": 0.44944015803339427,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.44944015803339427,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.21236650576719285,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.357421875,
+      "kd_loss": 0.15185958203733207,
+      "learning_rate": 3e-06,
+      "loss": 0.2372,
+      "masked_tokens": 103.2375,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 103.2375
+    },
+    {
+      "avg_mask_ratio": 0.45069065956631676,
+      "avg_response_length": 230.175,
+      "avg_student_mask_ratio": 0.45069065956631676,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2341734012890015,
+      "epoch": 1.9216,
+      "grad_norm": 0.50390625,
+      "kd_loss": 0.18844677137070107,
+      "learning_rate": 3e-06,
+      "loss": 0.3624,
+      "masked_tokens": 104.325,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 104.325
+    },
+    {
+      "avg_mask_ratio": 0.475881968671456,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.475881968671456,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.24741036754367088,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.3125,
+      "kd_loss": 0.15667265115065448,
+      "learning_rate": 3e-06,
+      "loss": 0.3189,
+      "masked_tokens": 125.1625,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 125.1625
+    },
+    {
+      "avg_mask_ratio": 0.4633113604504615,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.4633113604504615,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.23597910864411914,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.17416997326051825,
+      "learning_rate": 3e-06,
+      "loss": 0.3294,
+      "masked_tokens": 109.5375,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 109.5375
+    },
+    {
+      "avg_mask_ratio": 0.47329409609083084,
+      "avg_response_length": 244.875,
+      "avg_student_mask_ratio": 0.47329409609083084,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2694289070637751,
+      "epoch": 1.9856,
+      "grad_norm": 0.2255859375,
+      "kd_loss": 0.16790816846706277,
+      "learning_rate": 3e-06,
+      "loss": 0.3419,
+      "masked_tokens": 120.525,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 120.525
+    },
+    {
+      "avg_mask_ratio": 0.4973435569776311,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.4973435569776311,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34412962615628895,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.18454486661500727,
+      "learning_rate": 3e-06,
+      "loss": 0.3697,
+      "masked_tokens": 120.63095238095238,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 120.63095238095238
+    },
+    {
+      "avg_mask_ratio": 0.4365456592233386,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.4365456592233386,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.28184777884216033,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.6875,
+      "kd_loss": 0.15645663726530615,
+      "learning_rate": 3e-06,
+      "loss": 0.2725,
+      "masked_tokens": 118.8125,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 118.8125
+    },
+    {
+      "avg_mask_ratio": 0.4914003949146718,
+      "avg_response_length": 275.3,
+      "avg_student_mask_ratio": 0.4914003949146718,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43313308064230116,
+      "epoch": 2.0512,
+      "grad_norm": 0.5625,
+      "kd_loss": 0.2300663744529629,
+      "learning_rate": 3e-06,
+      "loss": 0.4182,
+      "masked_tokens": 143.075,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 143.075
+    },
+    {
+      "avg_mask_ratio": 0.4381961932755075,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4381961932755075,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.34196077592644086,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.57421875,
+      "kd_loss": 0.1847320883095051,
+      "learning_rate": 3e-06,
+      "loss": 0.3923,
+      "masked_tokens": 106.5125,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 106.5125
+    },
+    {
+      "avg_mask_ratio": 0.42702240714570505,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.42702240714570505,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4801598512272108,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.14269168120167705,
+      "learning_rate": 3e-06,
+      "loss": 0.3004,
+      "masked_tokens": 98.525,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 98.525
+    },
+    {
+      "avg_mask_ratio": 0.4583221158827655,
+      "avg_response_length": 262.0375,
+      "avg_student_mask_ratio": 0.4583221158827655,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.5193568647300936,
+      "epoch": 2.1152,
+      "grad_norm": 0.5234375,
+      "kd_loss": 0.15391424604512166,
+      "learning_rate": 3e-06,
+      "loss": 0.3657,
+      "masked_tokens": 121.2875,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 121.2875
+    },
+    {
+      "avg_mask_ratio": 0.45086776099633424,
+      "avg_response_length": 214.925,
+      "avg_student_mask_ratio": 0.45086776099633424,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.20815674337951579,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.470703125,
+      "kd_loss": 0.1478596617299445,
+      "learning_rate": 3e-06,
+      "loss": 0.3149,
+      "masked_tokens": 100.4875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 100.4875
+    },
+    {
+      "avg_mask_ratio": 0.4363243154773954,
+      "avg_response_length": 224.3,
+      "avg_student_mask_ratio": 0.4363243154773954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.47673265045309565,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.07421875,
+      "kd_loss": 0.14714571952575567,
+      "learning_rate": 3e-06,
+      "loss": 0.3653,
+      "masked_tokens": 89.5,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 89.5
+    },
+    {
+      "avg_mask_ratio": 0.5063220548443497,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.5063220548443497,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.38743265072380384,
+      "epoch": 2.1792,
+      "grad_norm": 0.6953125,
+      "kd_loss": 0.18382307874209686,
+      "learning_rate": 3e-06,
+      "loss": 0.3938,
+      "masked_tokens": 110.25,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 110.25
+    },
+    {
+      "avg_mask_ratio": 0.46985941788880153,
+      "avg_response_length": 220.05,
+      "avg_student_mask_ratio": 0.46985941788880153,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.43328518526819837,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.19683520471950544,
+      "learning_rate": 3e-06,
+      "loss": 0.395,
+      "masked_tokens": 104.9,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 104.9
+    },
+    {
+      "avg_mask_ratio": 0.49566771630197765,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.49566771630197765,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2710993955800006,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.94140625,
+      "kd_loss": 0.18894708146090125,
+      "learning_rate": 3e-06,
+      "loss": 0.4122,
+      "masked_tokens": 100.35,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 100.35
+    },
+    {
+      "avg_mask_ratio": 0.5123685836791992,
+      "avg_response_length": 238.8,
+      "avg_student_mask_ratio": 0.5123685836791992,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.30231127230786115,
+      "epoch": 2.2432,
+      "grad_norm": 0.64453125,
+      "kd_loss": 0.21115153960363386,
+      "learning_rate": 3e-06,
+      "loss": 0.3488,
+      "masked_tokens": 122.875,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 122.875
+    },
+    {
+      "avg_mask_ratio": 0.46218636581033934,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.46218636581033934,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.33772741691473984,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.369140625,
+      "kd_loss": 0.15173700823205394,
+      "learning_rate": 3e-06,
+      "loss": 0.2989,
+      "masked_tokens": 122.125,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 122.125
+    },
+    {
+      "avg_mask_ratio": 0.485661978519056,
+      "avg_response_length": 275.075,
+      "avg_student_mask_ratio": 0.485661978519056,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4382721236863517,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.375,
+      "kd_loss": 0.195586445067174,
+      "learning_rate": 3e-06,
+      "loss": 0.365,
+      "masked_tokens": 142.6375,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 142.6375
+    },
+    {
+      "avg_mask_ratio": 0.4626998565625399,
+      "avg_response_length": 214.45,
+      "avg_student_mask_ratio": 0.4626998565625399,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.22534348728113401,
+      "epoch": 2.3072,
+      "grad_norm": 0.2109375,
+      "kd_loss": 0.16271676397591364,
+      "learning_rate": 3e-06,
+      "loss": 0.2785,
+      "masked_tokens": 98.4,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 98.4
+    },
+    {
+      "avg_mask_ratio": 0.44248262273031286,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.44248262273031286,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.2362390007866111,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.05126953125,
+      "kd_loss": 0.12846805150115514,
+      "learning_rate": 3e-06,
+      "loss": 0.3103,
+      "masked_tokens": 91.9125,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 91.9125
+    },
+    {
+      "avg_mask_ratio": 0.5204601250356063,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.5204601250356063,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.4075917797143461,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.22264093817780123,
+      "learning_rate": 3e-06,
+      "loss": 0.4293,
+      "masked_tokens": 133.1875,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 133.1875
+    },
+    {
+      "avg_mask_ratio": 0.447697223268915,
+      "avg_response_length": 226.6375,
+      "avg_student_mask_ratio": 0.447697223268915,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.32370925757547864,
+      "epoch": 2.3712,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.1342358343013629,
+      "learning_rate": 3e-06,
+      "loss": 0.3495,
+      "masked_tokens": 103.475,
+      "mean_t": 0.4757364276825683,
+      "step": 1110,
+      "student_masked_tokens": 103.475
+    },
+    {
+      "avg_mask_ratio": 0.4671802403172478,
+      "avg_response_length": 250.8,
+      "avg_student_mask_ratio": 0.4671802403172478,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.22600486942956194,
+      "epoch": 2.392533333333333,
+      "grad_norm": 1.0234375,
+      "kd_loss": 0.14021195607333042,
+      "learning_rate": 3e-06,
+      "loss": 0.2938,
+      "masked_tokens": 117.775,
+      "mean_t": 0.5013068238971755,
+      "step": 1120,
+      "student_masked_tokens": 117.775
+    },
+    {
+      "avg_mask_ratio": 0.5009213570854627,
+      "avg_response_length": 266.3125,
+      "avg_student_mask_ratio": 0.5009213570854627,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.6386108998961845,
+      "epoch": 2.413866666666667,
+      "grad_norm": 1.296875,
+      "kd_loss": 0.19046277637947923,
+      "learning_rate": 3e-06,
+      "loss": 0.4746,
+      "masked_tokens": 138.875,
+      "mean_t": 0.5303254407714121,
+      "step": 1130,
+      "student_masked_tokens": 138.875
+    },
+    {
+      "avg_mask_ratio": 0.44728723394218833,
+      "avg_response_length": 234.075,
+      "avg_student_mask_ratio": 0.44728723394218833,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.20888477643130726,
+      "epoch": 2.4352,
+      "grad_norm": 0.1357421875,
+      "kd_loss": 0.12818265949827037,
+      "learning_rate": 3e-06,
+      "loss": 0.2181,
+      "masked_tokens": 107.9125,
+      "mean_t": 0.4845335395424627,
+      "step": 1140,
+      "student_masked_tokens": 107.9125
+    },
+    {
+      "avg_mask_ratio": 0.5291026248247362,
+      "avg_response_length": 212.5875,
+      "avg_student_mask_ratio": 0.5291026248247362,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.36574477101037245,
+      "epoch": 2.4565333333333332,
+      "grad_norm": 0.796875,
+      "kd_loss": 0.22556910791559517,
+      "learning_rate": 3e-06,
+      "loss": 0.3874,
+      "masked_tokens": 110.025,
+      "mean_t": 0.5690932425903157,
+      "step": 1150,
+      "student_masked_tokens": 110.025
+    },
+    {
+      "avg_mask_ratio": 0.46978622819297017,
+      "avg_response_length": 225.15,
+      "avg_student_mask_ratio": 0.46978622819297017,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.3836005017641583,
+      "epoch": 2.4778666666666664,
+      "grad_norm": 0.97265625,
+      "kd_loss": 0.18042389544071114,
+      "learning_rate": 3e-06,
+      "loss": 0.3104,
+      "masked_tokens": 101.7875,
+      "mean_t": 0.5040684466948733,
+      "step": 1160,
+      "student_masked_tokens": 101.7875
+    },
+    {
+      "avg_mask_ratio": 0.480710746452678,
+      "avg_response_length": 247.7625,
+      "avg_student_mask_ratio": 0.480710746452678,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.32349593787396314,
+      "epoch": 2.4992,
+      "grad_norm": 0.294921875,
+      "kd_loss": 0.18438192660469213,
+      "learning_rate": 3e-06,
+      "loss": 0.3192,
+      "masked_tokens": 123.9125,
+      "mean_t": 0.5114516971167177,
+      "step": 1170,
+      "student_masked_tokens": 123.9125
+    },
+    {
+      "avg_mask_ratio": 0.41955015211715363,
+      "avg_response_length": 213.775,
+      "avg_student_mask_ratio": 0.41955015211715363,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.310194344992226,
+      "epoch": 2.5205333333333333,
+      "grad_norm": 0.7109375,
+      "kd_loss": 0.1178153028772158,
+      "learning_rate": 3e-06,
+      "loss": 0.2991,
+      "masked_tokens": 85.15,
+      "mean_t": 0.4491677140351385,
+      "step": 1180,
+      "student_masked_tokens": 85.15
+    },
+    {
+      "avg_mask_ratio": 0.522994744987227,
+      "avg_response_length": 220.3375,
+      "avg_student_mask_ratio": 0.522994744987227,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.47026978965000127,
+      "epoch": 2.5418666666666665,
+      "grad_norm": 0.8046875,
+      "kd_loss": 0.2796627198796557,
+      "learning_rate": 3e-06,
+      "loss": 0.4353,
+      "masked_tokens": 124.2875,
+      "mean_t": 0.5590635397238657,
+      "step": 1190,
+      "student_masked_tokens": 124.2875
+    },
+    {
+      "avg_mask_ratio": 0.4730891800048994,
+      "avg_response_length": 215.675,
+      "avg_student_mask_ratio": 0.4730891800048994,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.251052744397839,
+      "epoch": 2.5632,
+      "grad_norm": 0.388671875,
+      "kd_loss": 0.16112432569007068,
+      "learning_rate": 3e-06,
+      "loss": 0.3037,
+      "masked_tokens": 98.775,
+      "mean_t": 0.506370971655997,
+      "step": 1200,
+      "student_masked_tokens": 98.775
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b968dbbf06e2017aeda9d45021ff20554875b30a6340f05bc88d8864eccb6012
+size 8120

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "down_proj",
+    "o_proj",
+    "v_proj",
+    "q_proj",
+    "up_proj",
+    "k_proj",
+    "gate_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a5d74ee7632509b3ac7f3084a2fc117ade8675e43defc09f7828430826cd234f
+size 2406624648

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ffd171200874cce4c3cd023caef37246ed4a786fe743fa902b60dfc07deda36d
+size 671304442

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fed4fb76d2cb63b43a8b71459a39a6ff6dd752f3acf64d0dd3af2ae385d4295
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc205b61cbef56c14d086bf49dab681dbab5c8a2eb46a8969323a3a488b84d35
+size 14512

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:368502bb8b2f2d6f452bda7249e88ca57b330ec2ba407ec248613b4300c99d0d
+size 1064

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b968dbbf06e2017aeda9d45021ff20554875b30a6340f05bc88d8864eccb6012
+size 8120

math/INP-PAR-REVERSE/unmask_tags_gold1_target1_ce0.5/checkpoint-1400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1