Upload checkpoint from unmask_tags_math_self_distill_INP_RKD_ACE_trefleave_last_step_u0.001-1.0_gold1_target1_ce0.5

.gitattributes

@@ -45,3 +45,4 @@ math/INP-OH/unmask_tags_gold1_target1_ce0.0/debug_training_examples.jsonl filter
 math/INP/unmask_tags_gold1_target1_ce0.0/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/INP/unmask_tags_leave_last_step_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
 math/INP/unmask_tags_leave_last_step_gold1_target1_ce0.0/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text
+math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/debug_training_examples.jsonl filter=lfs diff=lfs merge=lfs -text

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6881f6cead2cfcc4e8b2369f7a703a417f7788648870d0d4edd997ad4baaa8a0
+size 2406624648

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27f1f640d4b9d8817bc0bf3c001775e097ba39da25c71ffcda6d6247695c0e56
+size 2406624648

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e9beb2129fe556f52f251ec3c7428ea46608c130fe4fb3fbe7cddc85d256903b
+size 671304442

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3cedbb24eb08e412f2b6567529f919723c479356a0b4861fb1f0133d92b4e4aa
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:27f1021fb57112918a3a6091b09b0ccd50cb071a2324c12ae9afcc9851ee8bd3
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3d8cdfceac9f7917b978dca661a3b8e04187faea5d5f6bd7b462d61d8234d57f
+size 1064

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/trainer_state.json

@@ -0,0 +1,293 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.21333333333333335,
+  "eval_steps": 500,
+  "global_step": 100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.4625345868640579,
+      "avg_response_length": 225.975,
+      "avg_student_mask_ratio": 0.4625345868640579,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.09143845530088583,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.561622231673482,
+      "learning_rate": 3e-06,
+      "loss": 0.5986,
+      "masked_tokens": 107.275,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 107.275
+    },
+    {
+      "avg_mask_ratio": 0.38337777109118176,
+      "avg_response_length": 255.2625,
+      "avg_student_mask_ratio": 0.38337777109118176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.15581396042414325,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.255859375,
+      "kd_loss": 0.4017359749782827,
+      "learning_rate": 3e-06,
+      "loss": 0.4266,
+      "masked_tokens": 94.775,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.775
+    },
+    {
+      "avg_mask_ratio": 0.42985178013332187,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.42985178013332187,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.14054332938953848,
+      "epoch": 0.064,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.45842150248379454,
+      "learning_rate": 3e-06,
+      "loss": 0.4834,
+      "masked_tokens": 106.6625,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 106.6625
+    },
+    {
+      "avg_mask_ratio": 0.39618103006796446,
+      "avg_response_length": 224.125,
+      "avg_student_mask_ratio": 0.39618103006796446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0865171173807994,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.4022788017454332,
+      "learning_rate": 3e-06,
+      "loss": 0.4169,
+      "masked_tokens": 94.05,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.05
+    },
+    {
+      "avg_mask_ratio": 0.40298989725124557,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.40298989725124557,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1259558707769884,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.34293031186681444,
+      "learning_rate": 3e-06,
+      "loss": 0.4529,
+      "masked_tokens": 80.725,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 80.725
+    },
+    {
+      "avg_mask_ratio": 0.42760977511061354,
+      "avg_response_length": 251.0875,
+      "avg_student_mask_ratio": 0.42760977511061354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0689788069453698,
+      "epoch": 0.128,
+      "grad_norm": 0.486328125,
+      "kd_loss": 0.4250469338211474,
+      "learning_rate": 3e-06,
+      "loss": 0.4874,
+      "masked_tokens": 114.85,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 114.85
+    },
+    {
+      "avg_mask_ratio": 0.4766521197976544,
+      "avg_response_length": 202.5875,
+      "avg_student_mask_ratio": 0.4766521197976544,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04920884784860391,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.4921875,
+      "kd_loss": 0.4804954072886403,
+      "learning_rate": 3e-06,
+      "loss": 0.5452,
+      "masked_tokens": 100.2625,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 100.2625
+    },
+    {
+      "avg_mask_ratio": 0.3576235310174525,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.3576235310174525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.10345090993349686,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.32782310033885553,
+      "learning_rate": 3e-06,
+      "loss": 0.3698,
+      "masked_tokens": 82.9875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 82.9875
+    },
+    {
+      "avg_mask_ratio": 0.4745737922610715,
+      "avg_response_length": 229.75,
+      "avg_student_mask_ratio": 0.4745737922610715,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02389885471989146,
+      "epoch": 0.192,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.6274212263749901,
+      "learning_rate": 3e-06,
+      "loss": 0.6216,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 103.3875
+    },
+    {
+      "avg_mask_ratio": 0.43199558680062183,
+      "avg_response_length": 208.0,
+      "avg_student_mask_ratio": 0.43199558680062183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.08216186909899079,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.5703125,
+      "kd_loss": 0.512328494097926,
+      "learning_rate": 3e-06,
+      "loss": 0.5188,
+      "masked_tokens": 101.175,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 101.175
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac6ad8e89a9dc564b7ec50d42ce5557e5bda1f33609e4fc24c3968508afb13fa
+size 8312

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:60ff4d6ceba92cdd4131ec72dd58c654b4cebe0b659e5364740baf54e72f1149
+size 2406624648

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7a01939fbcf066fead8dbe9d243e0cfaec6056aa357c2b1eeb0761e916f31141
+size 671304442

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b4ecc755e04a1de7d395a9f8674949eeb6ec1c48a524c8ff3f90d2e79673723
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b6d7553a00b2b7aca1d94ffd0db817c97e76a72ace1483d876c988b636753962
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:29d9aa99505fc60c0db1b9cdacaa08b06e8a85c8aaaab4e389667a719fafb9bf
+size 1064

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/trainer_state.json

@@ -0,0 +1,2633 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.1365333333333334,
+  "eval_steps": 500,
+  "global_step": 1000,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.4625345868640579,
+      "avg_response_length": 225.975,
+      "avg_student_mask_ratio": 0.4625345868640579,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.09143845530088583,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.561622231673482,
+      "learning_rate": 3e-06,
+      "loss": 0.5986,
+      "masked_tokens": 107.275,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 107.275
+    },
+    {
+      "avg_mask_ratio": 0.38337777109118176,
+      "avg_response_length": 255.2625,
+      "avg_student_mask_ratio": 0.38337777109118176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.15581396042414325,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.255859375,
+      "kd_loss": 0.4017359749782827,
+      "learning_rate": 3e-06,
+      "loss": 0.4266,
+      "masked_tokens": 94.775,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.775
+    },
+    {
+      "avg_mask_ratio": 0.42985178013332187,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.42985178013332187,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.14054332938953848,
+      "epoch": 0.064,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.45842150248379454,
+      "learning_rate": 3e-06,
+      "loss": 0.4834,
+      "masked_tokens": 106.6625,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 106.6625
+    },
+    {
+      "avg_mask_ratio": 0.39618103006796446,
+      "avg_response_length": 224.125,
+      "avg_student_mask_ratio": 0.39618103006796446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0865171173807994,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.4022788017454332,
+      "learning_rate": 3e-06,
+      "loss": 0.4169,
+      "masked_tokens": 94.05,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.05
+    },
+    {
+      "avg_mask_ratio": 0.40298989725124557,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.40298989725124557,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1259558707769884,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.34293031186681444,
+      "learning_rate": 3e-06,
+      "loss": 0.4529,
+      "masked_tokens": 80.725,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 80.725
+    },
+    {
+      "avg_mask_ratio": 0.42760977511061354,
+      "avg_response_length": 251.0875,
+      "avg_student_mask_ratio": 0.42760977511061354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0689788069453698,
+      "epoch": 0.128,
+      "grad_norm": 0.486328125,
+      "kd_loss": 0.4250469338211474,
+      "learning_rate": 3e-06,
+      "loss": 0.4874,
+      "masked_tokens": 114.85,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 114.85
+    },
+    {
+      "avg_mask_ratio": 0.4766521197976544,
+      "avg_response_length": 202.5875,
+      "avg_student_mask_ratio": 0.4766521197976544,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04920884784860391,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.4921875,
+      "kd_loss": 0.4804954072886403,
+      "learning_rate": 3e-06,
+      "loss": 0.5452,
+      "masked_tokens": 100.2625,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 100.2625
+    },
+    {
+      "avg_mask_ratio": 0.3576235310174525,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.3576235310174525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.10345090993349686,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.32782310033885553,
+      "learning_rate": 3e-06,
+      "loss": 0.3698,
+      "masked_tokens": 82.9875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 82.9875
+    },
+    {
+      "avg_mask_ratio": 0.4745737922610715,
+      "avg_response_length": 229.75,
+      "avg_student_mask_ratio": 0.4745737922610715,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02389885471989146,
+      "epoch": 0.192,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.6274212263749901,
+      "learning_rate": 3e-06,
+      "loss": 0.6216,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 103.3875
+    },
+    {
+      "avg_mask_ratio": 0.43199558680062183,
+      "avg_response_length": 208.0,
+      "avg_student_mask_ratio": 0.43199558680062183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.08216186909899079,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.5703125,
+      "kd_loss": 0.512328494097926,
+      "learning_rate": 3e-06,
+      "loss": 0.5188,
+      "masked_tokens": 101.175,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.46170439083361997,
+      "avg_response_length": 213.0875,
+      "avg_student_mask_ratio": 0.46170439083361997,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.013703654134837961,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.302734375,
+      "kd_loss": 0.5333612294962734,
+      "learning_rate": 3e-06,
+      "loss": 0.5228,
+      "masked_tokens": 98.05,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 98.05
+    },
+    {
+      "avg_mask_ratio": 0.41981868548318746,
+      "avg_response_length": 224.1875,
+      "avg_student_mask_ratio": 0.41981868548318746,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04153112161757235,
+      "epoch": 0.256,
+      "grad_norm": 0.1904296875,
+      "kd_loss": 0.4523675209566136,
+      "learning_rate": 3e-06,
+      "loss": 0.4657,
+      "masked_tokens": 96.75,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 96.75
+    },
+    {
+      "avg_mask_ratio": 0.41246208412339913,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.41246208412339913,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04344684002453505,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.2729826127932597,
+      "learning_rate": 3e-06,
+      "loss": 0.3442,
+      "masked_tokens": 86.4625,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 86.4625
+    },
+    {
+      "avg_mask_ratio": 0.46814797970000654,
+      "avg_response_length": 255.625,
+      "avg_student_mask_ratio": 0.46814797970000654,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0643174976140017,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.84765625,
+      "kd_loss": 0.5402939972886088,
+      "learning_rate": 3e-06,
+      "loss": 0.5102,
+      "masked_tokens": 131.9375,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 131.9375
+    },
+    {
+      "avg_mask_ratio": 0.45036806432763116,
+      "avg_response_length": 255.375,
+      "avg_student_mask_ratio": 0.45036806432763116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03358945442510049,
+      "epoch": 0.32,
+      "grad_norm": 0.4765625,
+      "kd_loss": 0.50907573059975,
+      "learning_rate": 3e-06,
+      "loss": 0.5057,
+      "masked_tokens": 121.8625,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 121.8625
+    },
+    {
+      "avg_mask_ratio": 0.46546199796139265,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.46546199796139265,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.11014544108202244,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.734375,
+      "kd_loss": 0.5193568798754313,
+      "learning_rate": 3e-06,
+      "loss": 0.6247,
+      "masked_tokens": 93.85,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 93.85
+    },
+    {
+      "avg_mask_ratio": 0.4556341585703194,
+      "avg_response_length": 190.325,
+      "avg_student_mask_ratio": 0.4556341585703194,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.09658154292699236,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 1.65625,
+      "kd_loss": 0.38418297848565996,
+      "learning_rate": 3e-06,
+      "loss": 0.4504,
+      "masked_tokens": 85.8375,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 85.8375
+    },
+    {
+      "avg_mask_ratio": 0.4315772625413956,
+      "avg_response_length": 244.0,
+      "avg_student_mask_ratio": 0.4315772625413956,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.033778612267508376,
+      "epoch": 0.384,
+      "grad_norm": 0.294921875,
+      "kd_loss": 0.36942618340425903,
+      "learning_rate": 3e-06,
+      "loss": 0.3988,
+      "masked_tokens": 105.8125,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 105.8125
+    },
+    {
+      "avg_mask_ratio": 0.44514731492381543,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.44514731492381543,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.041242919112346496,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 0.294921875,
+      "kd_loss": 0.43466754313005823,
+      "learning_rate": 3e-06,
+      "loss": 0.4758,
+      "masked_tokens": 104.5,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 104.5
+    },
+    {
+      "avg_mask_ratio": 0.4382028553634882,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4382028553634882,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02168446985060655,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 1.234375,
+      "kd_loss": 0.3671490783934132,
+      "learning_rate": 3e-06,
+      "loss": 0.3728,
+      "masked_tokens": 92.8125,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 92.8125
+    },
+    {
+      "avg_mask_ratio": 0.42423906628973784,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.42423906628973784,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03828413697524411,
+      "epoch": 0.448,
+      "grad_norm": 0.2314453125,
+      "kd_loss": 0.3992176352882552,
+      "learning_rate": 3e-06,
+      "loss": 0.4291,
+      "masked_tokens": 91.8875,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 91.8875
+    },
+    {
+      "avg_mask_ratio": 0.5066629054490477,
+      "avg_response_length": 236.5125,
+      "avg_student_mask_ratio": 0.5066629054490477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01518030167494544,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.4646616537815135,
+      "learning_rate": 3e-06,
+      "loss": 0.5837,
+      "masked_tokens": 120.1375,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 120.1375
+    },
+    {
+      "avg_mask_ratio": 0.41337928114226086,
+      "avg_response_length": 252.7,
+      "avg_student_mask_ratio": 0.41337928114226086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04124281868480466,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.2109375,
+      "kd_loss": 0.4556285015354157,
+      "learning_rate": 3e-06,
+      "loss": 0.4488,
+      "masked_tokens": 102.9625,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 102.9625
+    },
+    {
+      "avg_mask_ratio": 0.4432992546178866,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4432992546178866,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02707661109639048,
+      "epoch": 0.512,
+      "grad_norm": 0.10009765625,
+      "kd_loss": 0.5192753246924667,
+      "learning_rate": 3e-06,
+      "loss": 0.5058,
+      "masked_tokens": 102.4125,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 102.4125
+    },
+    {
+      "avg_mask_ratio": 0.4213961783214472,
+      "avg_response_length": 243.475,
+      "avg_student_mask_ratio": 0.4213961783214472,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03464133263869371,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.21875,
+      "kd_loss": 0.42494537351371947,
+      "learning_rate": 3e-06,
+      "loss": 0.4432,
+      "masked_tokens": 108.525,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 108.525
+    },
+    {
+      "avg_mask_ratio": 0.4247272064210847,
+      "avg_response_length": 245.55,
+      "avg_student_mask_ratio": 0.4247272064210847,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.027336416684272537,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.32382538581124154,
+      "learning_rate": 3e-06,
+      "loss": 0.3319,
+      "masked_tokens": 104.9625,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 104.9625
+    },
+    {
+      "avg_mask_ratio": 0.49350170011166483,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.49350170011166483,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.024796622712074168,
+      "epoch": 0.576,
+      "grad_norm": 0.236328125,
+      "kd_loss": 0.5767332979444746,
+      "learning_rate": 3e-06,
+      "loss": 0.5914,
+      "masked_tokens": 114.9875,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 114.9875
+    },
+    {
+      "avg_mask_ratio": 0.4354009481729008,
+      "avg_response_length": 254.825,
+      "avg_student_mask_ratio": 0.4354009481729008,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010312629349800772,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.4453125,
+      "kd_loss": 0.39940525426293905,
+      "learning_rate": 3e-06,
+      "loss": 0.4235,
+      "masked_tokens": 124.0875,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 124.0875
+    },
+    {
+      "avg_mask_ratio": 0.3683109345962293,
+      "avg_response_length": 245.9125,
+      "avg_student_mask_ratio": 0.3683109345962293,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.012720032719425945,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.1162109375,
+      "kd_loss": 0.2857841577428431,
+      "learning_rate": 3e-06,
+      "loss": 0.294,
+      "masked_tokens": 96.475,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 96.475
+    },
+    {
+      "avg_mask_ratio": 0.4097130674403161,
+      "avg_response_length": 217.0375,
+      "avg_student_mask_ratio": 0.4097130674403161,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.023793277296908853,
+      "epoch": 0.64,
+      "grad_norm": 0.1630859375,
+      "kd_loss": 0.33583841863110137,
+      "learning_rate": 3e-06,
+      "loss": 0.4441,
+      "masked_tokens": 100.1,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 100.1
+    },
+    {
+      "avg_mask_ratio": 0.43538314661709593,
+      "avg_response_length": 175.45,
+      "avg_student_mask_ratio": 0.43538314661709593,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.015725749519765485,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.5370311794263898,
+      "learning_rate": 3e-06,
+      "loss": 0.5061,
+      "masked_tokens": 80.1,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 80.1
+    },
+    {
+      "avg_mask_ratio": 0.46665120031684637,
+      "avg_response_length": 234.7875,
+      "avg_student_mask_ratio": 0.46665120031684637,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.014960732916938469,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.46875,
+      "kd_loss": 0.39295422226678056,
+      "learning_rate": 3e-06,
+      "loss": 0.496,
+      "masked_tokens": 115.0875,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 115.0875
+    },
+    {
+      "avg_mask_ratio": 0.47865987645927816,
+      "avg_response_length": 236.0625,
+      "avg_student_mask_ratio": 0.47865987645927816,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.015041795306922267,
+      "epoch": 0.704,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.4789525415606931,
+      "learning_rate": 3e-06,
+      "loss": 0.5297,
+      "masked_tokens": 124.5,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 124.5
+    },
+    {
+      "avg_mask_ratio": 0.4224522379576229,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.4224522379576229,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01793750617994192,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.37109375,
+      "kd_loss": 0.46617110954375673,
+      "learning_rate": 3e-06,
+      "loss": 0.4758,
+      "masked_tokens": 106.125,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 106.125
+    },
+    {
+      "avg_mask_ratio": 0.46417169462656604,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.46417169462656604,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02235494569315506,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.470703125,
+      "kd_loss": 0.517485610961802,
+      "learning_rate": 3e-06,
+      "loss": 0.5454,
+      "masked_tokens": 119.8625,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 119.8625
+    },
+    {
+      "avg_mask_ratio": 0.43942091512144543,
+      "avg_response_length": 246.1625,
+      "avg_student_mask_ratio": 0.43942091512144543,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007548657262009329,
+      "epoch": 0.768,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.3857669980663559,
+      "learning_rate": 3e-06,
+      "loss": 0.3864,
+      "masked_tokens": 113.2875,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 113.2875
+    },
+    {
+      "avg_mask_ratio": 0.4481156505411491,
+      "avg_response_length": 257.575,
+      "avg_student_mask_ratio": 0.4481156505411491,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.018972885205766944,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.14453125,
+      "kd_loss": 0.6020083630461841,
+      "learning_rate": 3e-06,
+      "loss": 0.5037,
+      "masked_tokens": 123.3,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 123.3
+    },
+    {
+      "avg_mask_ratio": 0.42134943113196643,
+      "avg_response_length": 209.8,
+      "avg_student_mask_ratio": 0.42134943113196643,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.008798050403879643,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.8046875,
+      "kd_loss": 0.41509318549498175,
+      "learning_rate": 3e-06,
+      "loss": 0.371,
+      "masked_tokens": 96.8375,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 96.8375
+    },
+    {
+      "avg_mask_ratio": 0.45006616330938415,
+      "avg_response_length": 219.5625,
+      "avg_student_mask_ratio": 0.45006616330938415,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010152369248544346,
+      "epoch": 0.832,
+      "grad_norm": 0.375,
+      "kd_loss": 0.4801266926267914,
+      "learning_rate": 3e-06,
+      "loss": 0.4572,
+      "masked_tokens": 108.4125,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 108.4125
+    },
+    {
+      "avg_mask_ratio": 0.45018406328745186,
+      "avg_response_length": 242.6125,
+      "avg_student_mask_ratio": 0.45018406328745186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.012864228464172811,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.5373568097349561,
+      "learning_rate": 3e-06,
+      "loss": 0.4948,
+      "masked_tokens": 116.4875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 116.4875
+    },
+    {
+      "avg_mask_ratio": 0.44001492017996496,
+      "avg_response_length": 228.15,
+      "avg_student_mask_ratio": 0.44001492017996496,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03768795826176756,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.3984375,
+      "kd_loss": 0.4913158558696722,
+      "learning_rate": 3e-06,
+      "loss": 0.5277,
+      "masked_tokens": 113.8,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 113.8
+    },
+    {
+      "avg_mask_ratio": 0.4093534855055623,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4093534855055623,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004924662637654365,
+      "epoch": 0.896,
+      "grad_norm": 0.3046875,
+      "kd_loss": 0.27524946155799623,
+      "learning_rate": 3e-06,
+      "loss": 0.3288,
+      "masked_tokens": 81.2375,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 81.2375
+    },
+    {
+      "avg_mask_ratio": 0.43686744165606795,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.43686744165606795,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.016146755228146504,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.1494140625,
+      "kd_loss": 0.48660410012041383,
+      "learning_rate": 3e-06,
+      "loss": 0.4854,
+      "masked_tokens": 101.0125,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 101.0125
+    },
+    {
+      "avg_mask_ratio": 0.4180427666055039,
+      "avg_response_length": 241.0875,
+      "avg_student_mask_ratio": 0.4180427666055039,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.029632186345816082,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 0.201171875,
+      "kd_loss": 0.5406854634089313,
+      "learning_rate": 3e-06,
+      "loss": 0.5413,
+      "masked_tokens": 109.7375,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 109.7375
+    },
+    {
+      "avg_mask_ratio": 0.40543155584018675,
+      "avg_response_length": 231.9875,
+      "avg_student_mask_ratio": 0.40543155584018675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.006896732788118243,
+      "epoch": 0.96,
+      "grad_norm": 0.333984375,
+      "kd_loss": 0.5245532327341607,
+      "learning_rate": 3e-06,
+      "loss": 0.4648,
+      "masked_tokens": 100.1875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 100.1875
+    },
+    {
+      "avg_mask_ratio": 0.4476535278838128,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.4476535278838128,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02515205367649287,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.3874450994949029,
+      "learning_rate": 3e-06,
+      "loss": 0.4217,
+      "masked_tokens": 101.2,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.4478310279415122,
+      "avg_response_length": 224.0952380952381,
+      "avg_student_mask_ratio": 0.4478310279415122,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03232636066334072,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.46496453467603416,
+      "learning_rate": 3e-06,
+      "loss": 0.5059,
+      "masked_tokens": 103.03571428571429,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 103.03571428571429
+    },
+    {
+      "avg_mask_ratio": 0.44247635449282824,
+      "avg_response_length": 249.4125,
+      "avg_student_mask_ratio": 0.44247635449282824,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04671022014210848,
+      "epoch": 1.0256,
+      "grad_norm": 0.1630859375,
+      "kd_loss": 0.5139326051962939,
+      "learning_rate": 3e-06,
+      "loss": 0.5147,
+      "masked_tokens": 112.8125,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 112.8125
+    },
+    {
+      "avg_mask_ratio": 0.42829811533447354,
+      "avg_response_length": 232.0375,
+      "avg_student_mask_ratio": 0.42829811533447354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005066035011736858,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.1630859375,
+      "kd_loss": 0.4333162663751821,
+      "learning_rate": 3e-06,
+      "loss": 0.4199,
+      "masked_tokens": 94.7125,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 94.7125
+    },
+    {
+      "avg_mask_ratio": 0.48132704342715443,
+      "avg_response_length": 253.7875,
+      "avg_student_mask_ratio": 0.48132704342715443,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01742940873630232,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.169921875,
+      "kd_loss": 0.5254742230891452,
+      "learning_rate": 3e-06,
+      "loss": 0.508,
+      "masked_tokens": 124.275,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 124.275
+    },
+    {
+      "avg_mask_ratio": 0.46568706033285706,
+      "avg_response_length": 254.2,
+      "avg_student_mask_ratio": 0.46568706033285706,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.014650851830197098,
+      "epoch": 1.0896,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5521236633266653,
+      "learning_rate": 3e-06,
+      "loss": 0.5066,
+      "masked_tokens": 133.0,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 133.0
+    },
+    {
+      "avg_mask_ratio": 0.4205072323558852,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4205072323558852,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010721822535629144,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.4538552779665597,
+      "learning_rate": 3e-06,
+      "loss": 0.4843,
+      "masked_tokens": 83.1875,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 83.1875
+    },
+    {
+      "avg_mask_ratio": 0.44365208947565404,
+      "avg_response_length": 214.275,
+      "avg_student_mask_ratio": 0.44365208947565404,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.012283863066681989,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.31640625,
+      "kd_loss": 0.5291487223667846,
+      "learning_rate": 3e-06,
+      "loss": 0.6198,
+      "masked_tokens": 95.475,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 95.475
+    },
+    {
+      "avg_mask_ratio": 0.4564033471979201,
+      "avg_response_length": 231.3125,
+      "avg_student_mask_ratio": 0.4564033471979201,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02264503003948448,
+      "epoch": 1.1536,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.43877944423244913,
+      "learning_rate": 3e-06,
+      "loss": 0.434,
+      "masked_tokens": 102.1125,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 102.1125
+    },
+    {
+      "avg_mask_ratio": 0.4085368953179568,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4085368953179568,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.011074162583918223,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.41639115329383997,
+      "learning_rate": 3e-06,
+      "loss": 0.4289,
+      "masked_tokens": 92.7125,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 92.7125
+    },
+    {
+      "avg_mask_ratio": 0.36651339489617385,
+      "avg_response_length": 231.4125,
+      "avg_student_mask_ratio": 0.36651339489617385,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010163615908072642,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.47027518333145935,
+      "learning_rate": 3e-06,
+      "loss": 0.4476,
+      "masked_tokens": 95.2375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 95.2375
+    },
+    {
+      "avg_mask_ratio": 0.4171164375031367,
+      "avg_response_length": 230.1625,
+      "avg_student_mask_ratio": 0.4171164375031367,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004821049512975151,
+      "epoch": 1.2176,
+      "grad_norm": 0.2890625,
+      "kd_loss": 0.37785149019453,
+      "learning_rate": 3e-06,
+      "loss": 0.4009,
+      "masked_tokens": 103.925,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 103.925
+    },
+    {
+      "avg_mask_ratio": 0.4461298058507964,
+      "avg_response_length": 267.4375,
+      "avg_student_mask_ratio": 0.4461298058507964,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03442675815317671,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.5179231762156633,
+      "learning_rate": 3e-06,
+      "loss": 0.5121,
+      "masked_tokens": 126.825,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 126.825
+    },
+    {
+      "avg_mask_ratio": 0.4900112466188148,
+      "avg_response_length": 228.45,
+      "avg_student_mask_ratio": 0.4900112466188148,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0073458934444261105,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.6004035866773847,
+      "learning_rate": 3e-06,
+      "loss": 0.5784,
+      "masked_tokens": 116.15,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 116.15
+    },
+    {
+      "avg_mask_ratio": 0.42360459361225367,
+      "avg_response_length": 236.7,
+      "avg_student_mask_ratio": 0.42360459361225367,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005112273251961596,
+      "epoch": 1.2816,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.40553972261057536,
+      "learning_rate": 3e-06,
+      "loss": 0.3902,
+      "masked_tokens": 105.8375,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 105.8375
+    },
+    {
+      "avg_mask_ratio": 0.47793483664281666,
+      "avg_response_length": 230.3375,
+      "avg_student_mask_ratio": 0.47793483664281666,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.013057019122277325,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.15234375,
+      "kd_loss": 0.6178492149250815,
+      "learning_rate": 3e-06,
+      "loss": 0.5852,
+      "masked_tokens": 108.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4574313384189736,
+      "avg_response_length": 233.0625,
+      "avg_student_mask_ratio": 0.4574313384189736,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02562547706791065,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.1728515625,
+      "kd_loss": 0.5067496699713558,
+      "learning_rate": 3e-06,
+      "loss": 0.4958,
+      "masked_tokens": 105.0,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 105.0
+    },
+    {
+      "avg_mask_ratio": 0.4954570603906177,
+      "avg_response_length": 257.2125,
+      "avg_student_mask_ratio": 0.4954570603906177,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.009351487243975498,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.425488157782544,
+      "learning_rate": 3e-06,
+      "loss": 0.5267,
+      "masked_tokens": 125.5875,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 125.5875
+    },
+    {
+      "avg_mask_ratio": 0.45939837526530025,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.45939837526530025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.018259476403090957,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.1328125,
+      "kd_loss": 0.38472498857565823,
+      "learning_rate": 3e-06,
+      "loss": 0.3739,
+      "masked_tokens": 116.825,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 116.825
+    },
+    {
+      "avg_mask_ratio": 0.49772885923739524,
+      "avg_response_length": 238.0125,
+      "avg_student_mask_ratio": 0.49772885923739524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.013440046917298786,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.216796875,
+      "kd_loss": 0.5762969084854006,
+      "learning_rate": 3e-06,
+      "loss": 0.5345,
+      "masked_tokens": 127.6875,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 127.6875
+    },
+    {
+      "avg_mask_ratio": 0.4574655814212747,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.4574655814212747,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.027927246921797176,
+      "epoch": 1.4096,
+      "grad_norm": 0.640625,
+      "kd_loss": 0.4334344154445716,
+      "learning_rate": 3e-06,
+      "loss": 0.5146,
+      "masked_tokens": 124.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 124.35
+    },
+    {
+      "avg_mask_ratio": 0.4189265530381817,
+      "avg_response_length": 240.4375,
+      "avg_student_mask_ratio": 0.4189265530381817,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007356403671803946,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.3046875,
+      "kd_loss": 0.41319570616201984,
+      "learning_rate": 3e-06,
+      "loss": 0.3941,
+      "masked_tokens": 115.0875,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 115.0875
+    },
+    {
+      "avg_mask_ratio": 0.4405042894242797,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4405042894242797,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02681735201608717,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.212890625,
+      "kd_loss": 0.5227603636610183,
+      "learning_rate": 3e-06,
+      "loss": 0.4925,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 103.0875
+    },
+    {
+      "avg_mask_ratio": 0.45782678785035386,
+      "avg_response_length": 202.05,
+      "avg_student_mask_ratio": 0.45782678785035386,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005062625911909891,
+      "epoch": 1.4736,
+      "grad_norm": 0.1201171875,
+      "kd_loss": 0.45889621370954503,
+      "learning_rate": 3e-06,
+      "loss": 0.4633,
+      "masked_tokens": 108.025,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 108.025
+    },
+    {
+      "avg_mask_ratio": 0.4901438815984875,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.4901438815984875,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.017818167519862982,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.2119140625,
+      "kd_loss": 0.6823521373159338,
+      "learning_rate": 3e-06,
+      "loss": 0.5921,
+      "masked_tokens": 126.525,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 126.525
+    },
+    {
+      "avg_mask_ratio": 0.44477287216577677,
+      "avg_response_length": 212.5,
+      "avg_student_mask_ratio": 0.44477287216577677,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005565753509768001,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.1923828125,
+      "kd_loss": 0.4292566096216433,
+      "learning_rate": 3e-06,
+      "loss": 0.4259,
+      "masked_tokens": 104.525,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 104.525
+    },
+    {
+      "avg_mask_ratio": 0.4493113035801798,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.4493113035801798,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007549421283908941,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.482421875,
+      "kd_loss": 0.508985077529897,
+      "learning_rate": 3e-06,
+      "loss": 0.5035,
+      "masked_tokens": 106.65,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.44044942427426576,
+      "avg_response_length": 232.925,
+      "avg_student_mask_ratio": 0.44044942427426576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004612203660190062,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.46282253071470336,
+      "learning_rate": 3e-06,
+      "loss": 0.4677,
+      "masked_tokens": 100.2875,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 100.2875
+    },
+    {
+      "avg_mask_ratio": 0.48421492378693076,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.48421492378693076,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0072626265307462745,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.1943359375,
+      "kd_loss": 0.5804909225802021,
+      "learning_rate": 3e-06,
+      "loss": 0.5679,
+      "masked_tokens": 91.6875,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 91.6875
+    },
+    {
+      "avg_mask_ratio": 0.3897687786899041,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.3897687786899041,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005423597515937217,
+      "epoch": 1.6016,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.46322623784844835,
+      "learning_rate": 3e-06,
+      "loss": 0.4614,
+      "masked_tokens": 90.4625,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 90.4625
+    },
+    {
+      "avg_mask_ratio": 0.43048173217976,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.43048173217976,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004789848048892509,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.1708984375,
+      "kd_loss": 0.5197787190380041,
+      "learning_rate": 3e-06,
+      "loss": 0.4946,
+      "masked_tokens": 102.5875,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 102.5875
+    },
+    {
+      "avg_mask_ratio": 0.41878489739028735,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.41878489739028735,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.006974586636010827,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.19921875,
+      "kd_loss": 0.4032763403680336,
+      "learning_rate": 3e-06,
+      "loss": 0.3384,
+      "masked_tokens": 113.825,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 113.825
+    },
+    {
+      "avg_mask_ratio": 0.39619951823260635,
+      "avg_response_length": 258.5125,
+      "avg_student_mask_ratio": 0.39619951823260635,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007122849350025717,
+      "epoch": 1.6656,
+      "grad_norm": 0.2412109375,
+      "kd_loss": 0.38649988778331124,
+      "learning_rate": 3e-06,
+      "loss": 0.3997,
+      "masked_tokens": 113.9,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 113.9
+    },
+    {
+      "avg_mask_ratio": 0.43434095685952345,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.43434095685952345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.009959504639243732,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.146484375,
+      "kd_loss": 0.31865618440902316,
+      "learning_rate": 3e-06,
+      "loss": 0.3552,
+      "masked_tokens": 98.775,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 98.775
+    },
+    {
+      "avg_mask_ratio": 0.42826264500617983,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.42826264500617983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.002781124600801377,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.1416015625,
+      "kd_loss": 0.3458431469396146,
+      "learning_rate": 3e-06,
+      "loss": 0.4078,
+      "masked_tokens": 99.1125,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 99.1125
+    },
+    {
+      "avg_mask_ratio": 0.45382156379055233,
+      "avg_response_length": 237.85,
+      "avg_student_mask_ratio": 0.45382156379055233,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007012360586395516,
+      "epoch": 1.7296,
+      "grad_norm": 0.3125,
+      "kd_loss": 0.4872051531401212,
+      "learning_rate": 3e-06,
+      "loss": 0.5518,
+      "masked_tokens": 113.4625,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 113.4625
+    },
+    {
+      "avg_mask_ratio": 0.4765406621154398,
+      "avg_response_length": 209.0875,
+      "avg_student_mask_ratio": 0.4765406621154398,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.008087280945187559,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.291015625,
+      "kd_loss": 0.582653931081586,
+      "learning_rate": 3e-06,
+      "loss": 0.568,
+      "masked_tokens": 116.15,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 116.15
+    },
+    {
+      "avg_mask_ratio": 0.4660664959577844,
+      "avg_response_length": 263.975,
+      "avg_student_mask_ratio": 0.4660664959577844,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004976632978514317,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.361328125,
+      "kd_loss": 0.6404682535923711,
+      "learning_rate": 3e-06,
+      "loss": 0.6111,
+      "masked_tokens": 130.85,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 130.85
+    },
+    {
+      "avg_mask_ratio": 0.46782192731043326,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.46782192731043326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004557234377546138,
+      "epoch": 1.7936,
+      "grad_norm": 0.0908203125,
+      "kd_loss": 0.4399012086084646,
+      "learning_rate": 3e-06,
+      "loss": 0.4424,
+      "masked_tokens": 110.3625,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 110.3625
+    },
+    {
+      "avg_mask_ratio": 0.4748796411789954,
+      "avg_response_length": 225.5,
+      "avg_student_mask_ratio": 0.4748796411789954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.008075677036322304,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.1806640625,
+      "kd_loss": 0.34330057573731665,
+      "learning_rate": 3e-06,
+      "loss": 0.4022,
+      "masked_tokens": 112.0625,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 112.0625
+    },
+    {
+      "avg_mask_ratio": 0.4120468311710283,
+      "avg_response_length": 229.125,
+      "avg_student_mask_ratio": 0.4120468311710283,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0052631380136517695,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.41644778757299716,
+      "learning_rate": 3e-06,
+      "loss": 0.4304,
+      "masked_tokens": 106.2125,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 106.2125
+    },
+    {
+      "avg_mask_ratio": 0.4631627084803768,
+      "avg_response_length": 229.1,
+      "avg_student_mask_ratio": 0.4631627084803768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.011806862431546961,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.498046875,
+      "kd_loss": 0.5520484269072569,
+      "learning_rate": 3e-06,
+      "loss": 0.5795,
+      "masked_tokens": 111.6,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 111.6
+    },
+    {
+      "avg_mask_ratio": 0.42117048120126127,
+      "avg_response_length": 198.5625,
+      "avg_student_mask_ratio": 0.42117048120126127,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010472421555880374,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 1.390625,
+      "kd_loss": 0.5054981482934636,
+      "learning_rate": 3e-06,
+      "loss": 0.4881,
+      "masked_tokens": 87.725,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 87.725
+    },
+    {
+      "avg_mask_ratio": 0.4137770858884323,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.4137770858884323,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.00998285402735064,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.126953125,
+      "kd_loss": 0.4549323421288364,
+      "learning_rate": 3e-06,
+      "loss": 0.4288,
+      "masked_tokens": 96.85,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 96.85
+    },
+    {
+      "avg_mask_ratio": 0.4260695957229473,
+      "avg_response_length": 227.7875,
+      "avg_student_mask_ratio": 0.4260695957229473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02715508515102556,
+      "epoch": 1.9216,
+      "grad_norm": 0.23828125,
+      "kd_loss": 0.44619655802651437,
+      "learning_rate": 3e-06,
+      "loss": 0.4721,
+      "masked_tokens": 99.1,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 99.1
+    },
+    {
+      "avg_mask_ratio": 0.45544130907510405,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.45544130907510405,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010313108547143202,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.435546875,
+      "kd_loss": 0.47350807999975475,
+      "learning_rate": 3e-06,
+      "loss": 0.4303,
+      "masked_tokens": 120.1875,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 120.1875
+    },
+    {
+      "avg_mask_ratio": 0.43869577182922515,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.43869577182922515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0427373267434632,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.23046875,
+      "kd_loss": 0.4529029980097164,
+      "learning_rate": 3e-06,
+      "loss": 0.4814,
+      "masked_tokens": 104.7375,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 104.7375
+    },
+    {
+      "avg_mask_ratio": 0.4477143040392548,
+      "avg_response_length": 237.05,
+      "avg_student_mask_ratio": 0.4477143040392548,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01040498305607116,
+      "epoch": 1.9856,
+      "grad_norm": 0.2197265625,
+      "kd_loss": 0.4587304208488604,
+      "learning_rate": 3e-06,
+      "loss": 0.4397,
+      "masked_tokens": 115.1125,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 115.1125
+    },
+    {
+      "avg_mask_ratio": 0.46240209431077045,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.46240209431077045,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0032689166722602316,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.5169763166982302,
+      "learning_rate": 3e-06,
+      "loss": 0.5523,
+      "masked_tokens": 117.0,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 117.0
+    },
+    {
+      "avg_mask_ratio": 0.4168280088924803,
+      "avg_response_length": 240.9125,
+      "avg_student_mask_ratio": 0.4168280088924803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.021347658315528407,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.5328356460729651,
+      "learning_rate": 3e-06,
+      "loss": 0.4641,
+      "masked_tokens": 108.0125,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 108.0125
+    },
+    {
+      "avg_mask_ratio": 0.4622723014326766,
+      "avg_response_length": 275.3,
+      "avg_student_mask_ratio": 0.4622723014326766,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0057922464925923125,
+      "epoch": 2.0512,
+      "grad_norm": 0.1064453125,
+      "kd_loss": 0.6181535236281206,
+      "learning_rate": 3e-06,
+      "loss": 0.5813,
+      "masked_tokens": 137.475,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 137.475
+    },
+    {
+      "avg_mask_ratio": 0.4141947620024439,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4141947620024439,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0025189679287393575,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.58984375,
+      "kd_loss": 0.44248166069168204,
+      "learning_rate": 3e-06,
+      "loss": 0.4533,
+      "masked_tokens": 103.0625,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 103.0625
+    },
+    {
+      "avg_mask_ratio": 0.3987298493855633,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.3987298493855633,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004547219713845152,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.173828125,
+      "kd_loss": 0.31845395617929173,
+      "learning_rate": 3e-06,
+      "loss": 0.3591,
+      "masked_tokens": 94.65,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 94.65
+    },
+    {
+      "avg_mask_ratio": 0.4348693466046825,
+      "avg_response_length": 262.0375,
+      "avg_student_mask_ratio": 0.4348693466046825,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.006510952173971418,
+      "epoch": 2.1152,
+      "grad_norm": 0.11669921875,
+      "kd_loss": 0.391171305554158,
+      "learning_rate": 3e-06,
+      "loss": 0.4005,
+      "masked_tokens": 116.6375,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 116.6375
+    },
+    {
+      "avg_mask_ratio": 0.42609867608407515,
+      "avg_response_length": 215.3,
+      "avg_student_mask_ratio": 0.42609867608407515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010184290214397151,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.166015625,
+      "kd_loss": 0.35061151300810706,
+      "learning_rate": 3e-06,
+      "loss": 0.3837,
+      "masked_tokens": 96.2875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 96.2875
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1000/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac6ad8e89a9dc564b7ec50d42ce5557e5bda1f33609e4fc24c3968508afb13fa
+size 8312

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ae90d0459a4ed7c1edcb2de05a767ec3d6592988530e3a13980f8bf506d6c8c2
+size 2406624648

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fd51f4786bb9a7db90ca2080e5d0d440ffe0d25c63a948e0f4b4b14e30970c1f
+size 671304442

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba5dd92515be5edfdf3c0caa345a3b40a7f72a99c4da2cae28c78c94c73f88b7
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9d62574bce0ea79a560c8f378c14c05b500f716e6cbb1d9d956399b061410bfc
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:25795e3b7374d0f6abdd7ab4b34fbf7ab0447ba73c04014500c2ab8b5acec5b4
+size 1064

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/trainer_state.json

@@ -0,0 +1,2893 @@
+{
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 2.3498666666666668,
+  "eval_steps": 500,
+  "global_step": 1100,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "avg_mask_ratio": 0.4625345868640579,
+      "avg_response_length": 225.975,
+      "avg_student_mask_ratio": 0.4625345868640579,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.09143845530088583,
+      "epoch": 0.021333333333333333,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.561622231673482,
+      "learning_rate": 3e-06,
+      "loss": 0.5986,
+      "masked_tokens": 107.275,
+      "mean_t": 0.5145528071501758,
+      "step": 10,
+      "student_masked_tokens": 107.275
+    },
+    {
+      "avg_mask_ratio": 0.38337777109118176,
+      "avg_response_length": 255.2625,
+      "avg_student_mask_ratio": 0.38337777109118176,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.15581396042414325,
+      "epoch": 0.042666666666666665,
+      "grad_norm": 0.255859375,
+      "kd_loss": 0.4017359749782827,
+      "learning_rate": 3e-06,
+      "loss": 0.4266,
+      "masked_tokens": 94.775,
+      "mean_t": 0.43874448732240123,
+      "step": 20,
+      "student_masked_tokens": 94.775
+    },
+    {
+      "avg_mask_ratio": 0.42985178013332187,
+      "avg_response_length": 211.7625,
+      "avg_student_mask_ratio": 0.42985178013332187,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.14054332938953848,
+      "epoch": 0.064,
+      "grad_norm": 0.4296875,
+      "kd_loss": 0.45842150248379454,
+      "learning_rate": 3e-06,
+      "loss": 0.4834,
+      "masked_tokens": 106.6625,
+      "mean_t": 0.4803953981841914,
+      "step": 30,
+      "student_masked_tokens": 106.6625
+    },
+    {
+      "avg_mask_ratio": 0.39618103006796446,
+      "avg_response_length": 224.125,
+      "avg_student_mask_ratio": 0.39618103006796446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0865171173807994,
+      "epoch": 0.08533333333333333,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.4022788017454332,
+      "learning_rate": 3e-06,
+      "loss": 0.4169,
+      "masked_tokens": 94.05,
+      "mean_t": 0.4569831106782658,
+      "step": 40,
+      "student_masked_tokens": 94.05
+    },
+    {
+      "avg_mask_ratio": 0.40298989725124557,
+      "avg_response_length": 207.125,
+      "avg_student_mask_ratio": 0.40298989725124557,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.1259558707769884,
+      "epoch": 0.10666666666666667,
+      "grad_norm": 0.6328125,
+      "kd_loss": 0.34293031186681444,
+      "learning_rate": 3e-06,
+      "loss": 0.4529,
+      "masked_tokens": 80.725,
+      "mean_t": 0.4612453707959503,
+      "step": 50,
+      "student_masked_tokens": 80.725
+    },
+    {
+      "avg_mask_ratio": 0.42760977511061354,
+      "avg_response_length": 251.0875,
+      "avg_student_mask_ratio": 0.42760977511061354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0689788069453698,
+      "epoch": 0.128,
+      "grad_norm": 0.486328125,
+      "kd_loss": 0.4250469338211474,
+      "learning_rate": 3e-06,
+      "loss": 0.4874,
+      "masked_tokens": 114.85,
+      "mean_t": 0.48597636765334756,
+      "step": 60,
+      "student_masked_tokens": 114.85
+    },
+    {
+      "avg_mask_ratio": 0.4766521197976544,
+      "avg_response_length": 202.5875,
+      "avg_student_mask_ratio": 0.4766521197976544,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04920884784860391,
+      "epoch": 0.14933333333333335,
+      "grad_norm": 0.4921875,
+      "kd_loss": 0.4804954072886403,
+      "learning_rate": 3e-06,
+      "loss": 0.5452,
+      "masked_tokens": 100.2625,
+      "mean_t": 0.5459650319069624,
+      "step": 70,
+      "student_masked_tokens": 100.2625
+    },
+    {
+      "avg_mask_ratio": 0.3576235310174525,
+      "avg_response_length": 225.85,
+      "avg_student_mask_ratio": 0.3576235310174525,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.10345090993349686,
+      "epoch": 0.17066666666666666,
+      "grad_norm": 0.60546875,
+      "kd_loss": 0.32782310033885553,
+      "learning_rate": 3e-06,
+      "loss": 0.3698,
+      "masked_tokens": 82.9875,
+      "mean_t": 0.40758824030635876,
+      "step": 80,
+      "student_masked_tokens": 82.9875
+    },
+    {
+      "avg_mask_ratio": 0.4745737922610715,
+      "avg_response_length": 229.75,
+      "avg_student_mask_ratio": 0.4745737922610715,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02389885471989146,
+      "epoch": 0.192,
+      "grad_norm": 0.56640625,
+      "kd_loss": 0.6274212263749901,
+      "learning_rate": 3e-06,
+      "loss": 0.6216,
+      "masked_tokens": 103.3875,
+      "mean_t": 0.5297661645396147,
+      "step": 90,
+      "student_masked_tokens": 103.3875
+    },
+    {
+      "avg_mask_ratio": 0.43199558680062183,
+      "avg_response_length": 208.0,
+      "avg_student_mask_ratio": 0.43199558680062183,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.08216186909899079,
+      "epoch": 0.21333333333333335,
+      "grad_norm": 0.5703125,
+      "kd_loss": 0.512328494097926,
+      "learning_rate": 3e-06,
+      "loss": 0.5188,
+      "masked_tokens": 101.175,
+      "mean_t": 0.4886587227345444,
+      "step": 100,
+      "student_masked_tokens": 101.175
+    },
+    {
+      "avg_mask_ratio": 0.46170439083361997,
+      "avg_response_length": 213.0875,
+      "avg_student_mask_ratio": 0.46170439083361997,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.013703654134837961,
+      "epoch": 0.23466666666666666,
+      "grad_norm": 0.302734375,
+      "kd_loss": 0.5333612294962734,
+      "learning_rate": 3e-06,
+      "loss": 0.5228,
+      "masked_tokens": 98.05,
+      "mean_t": 0.5246987929102034,
+      "step": 110,
+      "student_masked_tokens": 98.05
+    },
+    {
+      "avg_mask_ratio": 0.41981868548318746,
+      "avg_response_length": 224.1875,
+      "avg_student_mask_ratio": 0.41981868548318746,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04153112161757235,
+      "epoch": 0.256,
+      "grad_norm": 0.1904296875,
+      "kd_loss": 0.4523675209566136,
+      "learning_rate": 3e-06,
+      "loss": 0.4657,
+      "masked_tokens": 96.75,
+      "mean_t": 0.4805434140143916,
+      "step": 120,
+      "student_masked_tokens": 96.75
+    },
+    {
+      "avg_mask_ratio": 0.41246208412339913,
+      "avg_response_length": 202.7,
+      "avg_student_mask_ratio": 0.41246208412339913,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04344684002453505,
+      "epoch": 0.2773333333333333,
+      "grad_norm": 0.328125,
+      "kd_loss": 0.2729826127932597,
+      "learning_rate": 3e-06,
+      "loss": 0.3442,
+      "masked_tokens": 86.4625,
+      "mean_t": 0.47522516988683494,
+      "step": 130,
+      "student_masked_tokens": 86.4625
+    },
+    {
+      "avg_mask_ratio": 0.46814797970000654,
+      "avg_response_length": 255.625,
+      "avg_student_mask_ratio": 0.46814797970000654,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0643174976140017,
+      "epoch": 0.2986666666666667,
+      "grad_norm": 0.84765625,
+      "kd_loss": 0.5402939972886088,
+      "learning_rate": 3e-06,
+      "loss": 0.5102,
+      "masked_tokens": 131.9375,
+      "mean_t": 0.5204090005659964,
+      "step": 140,
+      "student_masked_tokens": 131.9375
+    },
+    {
+      "avg_mask_ratio": 0.45036806432763116,
+      "avg_response_length": 255.375,
+      "avg_student_mask_ratio": 0.45036806432763116,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03358945442510049,
+      "epoch": 0.32,
+      "grad_norm": 0.4765625,
+      "kd_loss": 0.50907573059975,
+      "learning_rate": 3e-06,
+      "loss": 0.5057,
+      "masked_tokens": 121.8625,
+      "mean_t": 0.5012552456930279,
+      "step": 150,
+      "student_masked_tokens": 121.8625
+    },
+    {
+      "avg_mask_ratio": 0.46546199796139265,
+      "avg_response_length": 209.325,
+      "avg_student_mask_ratio": 0.46546199796139265,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.11014544108202244,
+      "epoch": 0.3413333333333333,
+      "grad_norm": 0.734375,
+      "kd_loss": 0.5193568798754313,
+      "learning_rate": 3e-06,
+      "loss": 0.6247,
+      "masked_tokens": 93.85,
+      "mean_t": 0.5408745193795766,
+      "step": 160,
+      "student_masked_tokens": 93.85
+    },
+    {
+      "avg_mask_ratio": 0.4556341585703194,
+      "avg_response_length": 190.325,
+      "avg_student_mask_ratio": 0.4556341585703194,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.09658154292699236,
+      "epoch": 0.3626666666666667,
+      "grad_norm": 1.65625,
+      "kd_loss": 0.38418297848565996,
+      "learning_rate": 3e-06,
+      "loss": 0.4504,
+      "masked_tokens": 85.8375,
+      "mean_t": 0.5134547733236104,
+      "step": 170,
+      "student_masked_tokens": 85.8375
+    },
+    {
+      "avg_mask_ratio": 0.4315772625413956,
+      "avg_response_length": 244.0,
+      "avg_student_mask_ratio": 0.4315772625413956,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.033778612267508376,
+      "epoch": 0.384,
+      "grad_norm": 0.294921875,
+      "kd_loss": 0.36942618340425903,
+      "learning_rate": 3e-06,
+      "loss": 0.3988,
+      "masked_tokens": 105.8125,
+      "mean_t": 0.48226988823735156,
+      "step": 180,
+      "student_masked_tokens": 105.8125
+    },
+    {
+      "avg_mask_ratio": 0.44514731492381543,
+      "avg_response_length": 233.675,
+      "avg_student_mask_ratio": 0.44514731492381543,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.041242919112346496,
+      "epoch": 0.4053333333333333,
+      "grad_norm": 0.294921875,
+      "kd_loss": 0.43466754313005823,
+      "learning_rate": 3e-06,
+      "loss": 0.4758,
+      "masked_tokens": 104.5,
+      "mean_t": 0.5165087037021294,
+      "step": 190,
+      "student_masked_tokens": 104.5
+    },
+    {
+      "avg_mask_ratio": 0.4382028553634882,
+      "avg_response_length": 197.8,
+      "avg_student_mask_ratio": 0.4382028553634882,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02168446985060655,
+      "epoch": 0.4266666666666667,
+      "grad_norm": 1.234375,
+      "kd_loss": 0.3671490783934132,
+      "learning_rate": 3e-06,
+      "loss": 0.3728,
+      "masked_tokens": 92.8125,
+      "mean_t": 0.5073627714533359,
+      "step": 200,
+      "student_masked_tokens": 92.8125
+    },
+    {
+      "avg_mask_ratio": 0.42423906628973784,
+      "avg_response_length": 213.7875,
+      "avg_student_mask_ratio": 0.42423906628973784,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03828413697524411,
+      "epoch": 0.448,
+      "grad_norm": 0.2314453125,
+      "kd_loss": 0.3992176352882552,
+      "learning_rate": 3e-06,
+      "loss": 0.4291,
+      "masked_tokens": 91.8875,
+      "mean_t": 0.48430291628465055,
+      "step": 210,
+      "student_masked_tokens": 91.8875
+    },
+    {
+      "avg_mask_ratio": 0.5066629054490477,
+      "avg_response_length": 236.5125,
+      "avg_student_mask_ratio": 0.5066629054490477,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01518030167494544,
+      "epoch": 0.4693333333333333,
+      "grad_norm": 0.2392578125,
+      "kd_loss": 0.4646616537815135,
+      "learning_rate": 3e-06,
+      "loss": 0.5837,
+      "masked_tokens": 120.1375,
+      "mean_t": 0.570199209311977,
+      "step": 220,
+      "student_masked_tokens": 120.1375
+    },
+    {
+      "avg_mask_ratio": 0.41337928114226086,
+      "avg_response_length": 252.7,
+      "avg_student_mask_ratio": 0.41337928114226086,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04124281868480466,
+      "epoch": 0.49066666666666664,
+      "grad_norm": 0.2109375,
+      "kd_loss": 0.4556285015354157,
+      "learning_rate": 3e-06,
+      "loss": 0.4488,
+      "masked_tokens": 102.9625,
+      "mean_t": 0.46891279935371133,
+      "step": 230,
+      "student_masked_tokens": 102.9625
+    },
+    {
+      "avg_mask_ratio": 0.4432992546178866,
+      "avg_response_length": 212.85,
+      "avg_student_mask_ratio": 0.4432992546178866,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02707661109639048,
+      "epoch": 0.512,
+      "grad_norm": 0.10009765625,
+      "kd_loss": 0.5192753246924667,
+      "learning_rate": 3e-06,
+      "loss": 0.5058,
+      "masked_tokens": 102.4125,
+      "mean_t": 0.502228345896583,
+      "step": 240,
+      "student_masked_tokens": 102.4125
+    },
+    {
+      "avg_mask_ratio": 0.4213961783214472,
+      "avg_response_length": 243.475,
+      "avg_student_mask_ratio": 0.4213961783214472,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03464133263869371,
+      "epoch": 0.5333333333333333,
+      "grad_norm": 0.21875,
+      "kd_loss": 0.42494537351371947,
+      "learning_rate": 3e-06,
+      "loss": 0.4432,
+      "masked_tokens": 108.525,
+      "mean_t": 0.4733429416548461,
+      "step": 250,
+      "student_masked_tokens": 108.525
+    },
+    {
+      "avg_mask_ratio": 0.4247272064210847,
+      "avg_response_length": 245.55,
+      "avg_student_mask_ratio": 0.4247272064210847,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.027336416684272537,
+      "epoch": 0.5546666666666666,
+      "grad_norm": 0.34375,
+      "kd_loss": 0.32382538581124154,
+      "learning_rate": 3e-06,
+      "loss": 0.3319,
+      "masked_tokens": 104.9625,
+      "mean_t": 0.4843149524240289,
+      "step": 260,
+      "student_masked_tokens": 104.9625
+    },
+    {
+      "avg_mask_ratio": 0.49350170011166483,
+      "avg_response_length": 224.45,
+      "avg_student_mask_ratio": 0.49350170011166483,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.024796622712074168,
+      "epoch": 0.576,
+      "grad_norm": 0.236328125,
+      "kd_loss": 0.5767332979444746,
+      "learning_rate": 3e-06,
+      "loss": 0.5914,
+      "masked_tokens": 114.9875,
+      "mean_t": 0.5643589949700981,
+      "step": 270,
+      "student_masked_tokens": 114.9875
+    },
+    {
+      "avg_mask_ratio": 0.4354009481729008,
+      "avg_response_length": 254.825,
+      "avg_student_mask_ratio": 0.4354009481729008,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010312629349800772,
+      "epoch": 0.5973333333333334,
+      "grad_norm": 0.4453125,
+      "kd_loss": 0.39940525426293905,
+      "learning_rate": 3e-06,
+      "loss": 0.4235,
+      "masked_tokens": 124.0875,
+      "mean_t": 0.47818811538163575,
+      "step": 280,
+      "student_masked_tokens": 124.0875
+    },
+    {
+      "avg_mask_ratio": 0.3683109345962293,
+      "avg_response_length": 245.9125,
+      "avg_student_mask_ratio": 0.3683109345962293,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.012720032719425945,
+      "epoch": 0.6186666666666667,
+      "grad_norm": 0.1162109375,
+      "kd_loss": 0.2857841577428431,
+      "learning_rate": 3e-06,
+      "loss": 0.294,
+      "masked_tokens": 96.475,
+      "mean_t": 0.4088635521940887,
+      "step": 290,
+      "student_masked_tokens": 96.475
+    },
+    {
+      "avg_mask_ratio": 0.4097130674403161,
+      "avg_response_length": 217.0375,
+      "avg_student_mask_ratio": 0.4097130674403161,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.023793277296908853,
+      "epoch": 0.64,
+      "grad_norm": 0.1630859375,
+      "kd_loss": 0.33583841863110137,
+      "learning_rate": 3e-06,
+      "loss": 0.4441,
+      "masked_tokens": 100.1,
+      "mean_t": 0.47984200695063917,
+      "step": 300,
+      "student_masked_tokens": 100.1
+    },
+    {
+      "avg_mask_ratio": 0.43538314661709593,
+      "avg_response_length": 175.45,
+      "avg_student_mask_ratio": 0.43538314661709593,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.015725749519765485,
+      "epoch": 0.6613333333333333,
+      "grad_norm": 0.28515625,
+      "kd_loss": 0.5370311794263898,
+      "learning_rate": 3e-06,
+      "loss": 0.5061,
+      "masked_tokens": 80.1,
+      "mean_t": 0.5026606284547597,
+      "step": 310,
+      "student_masked_tokens": 80.1
+    },
+    {
+      "avg_mask_ratio": 0.46665120031684637,
+      "avg_response_length": 234.7875,
+      "avg_student_mask_ratio": 0.46665120031684637,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.014960732916938469,
+      "epoch": 0.6826666666666666,
+      "grad_norm": 0.46875,
+      "kd_loss": 0.39295422226678056,
+      "learning_rate": 3e-06,
+      "loss": 0.496,
+      "masked_tokens": 115.0875,
+      "mean_t": 0.5293499688967132,
+      "step": 320,
+      "student_masked_tokens": 115.0875
+    },
+    {
+      "avg_mask_ratio": 0.47865987645927816,
+      "avg_response_length": 236.0625,
+      "avg_student_mask_ratio": 0.47865987645927816,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.015041795306922267,
+      "epoch": 0.704,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.4789525415606931,
+      "learning_rate": 3e-06,
+      "loss": 0.5297,
+      "masked_tokens": 124.5,
+      "mean_t": 0.5426030711154454,
+      "step": 330,
+      "student_masked_tokens": 124.5
+    },
+    {
+      "avg_mask_ratio": 0.4224522379576229,
+      "avg_response_length": 241.4875,
+      "avg_student_mask_ratio": 0.4224522379576229,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01793750617994192,
+      "epoch": 0.7253333333333334,
+      "grad_norm": 0.37109375,
+      "kd_loss": 0.46617110954375673,
+      "learning_rate": 3e-06,
+      "loss": 0.4758,
+      "masked_tokens": 106.125,
+      "mean_t": 0.47635243807453664,
+      "step": 340,
+      "student_masked_tokens": 106.125
+    },
+    {
+      "avg_mask_ratio": 0.46417169462656604,
+      "avg_response_length": 235.6375,
+      "avg_student_mask_ratio": 0.46417169462656604,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02235494569315506,
+      "epoch": 0.7466666666666667,
+      "grad_norm": 0.470703125,
+      "kd_loss": 0.517485610961802,
+      "learning_rate": 3e-06,
+      "loss": 0.5454,
+      "masked_tokens": 119.8625,
+      "mean_t": 0.5268881446914747,
+      "step": 350,
+      "student_masked_tokens": 119.8625
+    },
+    {
+      "avg_mask_ratio": 0.43942091512144543,
+      "avg_response_length": 246.1625,
+      "avg_student_mask_ratio": 0.43942091512144543,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007548657262009329,
+      "epoch": 0.768,
+      "grad_norm": 0.2060546875,
+      "kd_loss": 0.3857669980663559,
+      "learning_rate": 3e-06,
+      "loss": 0.3864,
+      "masked_tokens": 113.2875,
+      "mean_t": 0.5040419134311378,
+      "step": 360,
+      "student_masked_tokens": 113.2875
+    },
+    {
+      "avg_mask_ratio": 0.4481156505411491,
+      "avg_response_length": 257.575,
+      "avg_student_mask_ratio": 0.4481156505411491,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.018972885205766944,
+      "epoch": 0.7893333333333333,
+      "grad_norm": 0.14453125,
+      "kd_loss": 0.6020083630461841,
+      "learning_rate": 3e-06,
+      "loss": 0.5037,
+      "masked_tokens": 123.3,
+      "mean_t": 0.5127181728370488,
+      "step": 370,
+      "student_masked_tokens": 123.3
+    },
+    {
+      "avg_mask_ratio": 0.42134943113196643,
+      "avg_response_length": 209.8,
+      "avg_student_mask_ratio": 0.42134943113196643,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.008798050403879643,
+      "epoch": 0.8106666666666666,
+      "grad_norm": 0.8046875,
+      "kd_loss": 0.41509318549498175,
+      "learning_rate": 3e-06,
+      "loss": 0.371,
+      "masked_tokens": 96.8375,
+      "mean_t": 0.4825185665744357,
+      "step": 380,
+      "student_masked_tokens": 96.8375
+    },
+    {
+      "avg_mask_ratio": 0.45006616330938415,
+      "avg_response_length": 219.5625,
+      "avg_student_mask_ratio": 0.45006616330938415,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010152369248544346,
+      "epoch": 0.832,
+      "grad_norm": 0.375,
+      "kd_loss": 0.4801266926267914,
+      "learning_rate": 3e-06,
+      "loss": 0.4572,
+      "masked_tokens": 108.4125,
+      "mean_t": 0.5053101469413377,
+      "step": 390,
+      "student_masked_tokens": 108.4125
+    },
+    {
+      "avg_mask_ratio": 0.45018406328745186,
+      "avg_response_length": 242.6125,
+      "avg_student_mask_ratio": 0.45018406328745186,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.012864228464172811,
+      "epoch": 0.8533333333333334,
+      "grad_norm": 0.3515625,
+      "kd_loss": 0.5373568097349561,
+      "learning_rate": 3e-06,
+      "loss": 0.4948,
+      "masked_tokens": 116.4875,
+      "mean_t": 0.5052250675857067,
+      "step": 400,
+      "student_masked_tokens": 116.4875
+    },
+    {
+      "avg_mask_ratio": 0.44001492017996496,
+      "avg_response_length": 228.15,
+      "avg_student_mask_ratio": 0.44001492017996496,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03768795826176756,
+      "epoch": 0.8746666666666667,
+      "grad_norm": 0.3984375,
+      "kd_loss": 0.4913158558696722,
+      "learning_rate": 3e-06,
+      "loss": 0.5277,
+      "masked_tokens": 113.8,
+      "mean_t": 0.5050956419203431,
+      "step": 410,
+      "student_masked_tokens": 113.8
+    },
+    {
+      "avg_mask_ratio": 0.4093534855055623,
+      "avg_response_length": 220.225,
+      "avg_student_mask_ratio": 0.4093534855055623,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004924662637654365,
+      "epoch": 0.896,
+      "grad_norm": 0.3046875,
+      "kd_loss": 0.27524946155799623,
+      "learning_rate": 3e-06,
+      "loss": 0.3288,
+      "masked_tokens": 81.2375,
+      "mean_t": 0.4769687672611326,
+      "step": 420,
+      "student_masked_tokens": 81.2375
+    },
+    {
+      "avg_mask_ratio": 0.43686744165606795,
+      "avg_response_length": 249.2125,
+      "avg_student_mask_ratio": 0.43686744165606795,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.016146755228146504,
+      "epoch": 0.9173333333333333,
+      "grad_norm": 0.1494140625,
+      "kd_loss": 0.48660410012041383,
+      "learning_rate": 3e-06,
+      "loss": 0.4854,
+      "masked_tokens": 101.0125,
+      "mean_t": 0.49262027950026094,
+      "step": 430,
+      "student_masked_tokens": 101.0125
+    },
+    {
+      "avg_mask_ratio": 0.4180427666055039,
+      "avg_response_length": 241.0875,
+      "avg_student_mask_ratio": 0.4180427666055039,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.029632186345816082,
+      "epoch": 0.9386666666666666,
+      "grad_norm": 0.201171875,
+      "kd_loss": 0.5406854634089313,
+      "learning_rate": 3e-06,
+      "loss": 0.5413,
+      "masked_tokens": 109.7375,
+      "mean_t": 0.47046207524836064,
+      "step": 440,
+      "student_masked_tokens": 109.7375
+    },
+    {
+      "avg_mask_ratio": 0.40543155584018675,
+      "avg_response_length": 231.9875,
+      "avg_student_mask_ratio": 0.40543155584018675,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.006896732788118243,
+      "epoch": 0.96,
+      "grad_norm": 0.333984375,
+      "kd_loss": 0.5245532327341607,
+      "learning_rate": 3e-06,
+      "loss": 0.4648,
+      "masked_tokens": 100.1875,
+      "mean_t": 0.45781184462830427,
+      "step": 450,
+      "student_masked_tokens": 100.1875
+    },
+    {
+      "avg_mask_ratio": 0.4476535278838128,
+      "avg_response_length": 214.6125,
+      "avg_student_mask_ratio": 0.4476535278838128,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02515205367649287,
+      "epoch": 0.9813333333333333,
+      "grad_norm": 0.62109375,
+      "kd_loss": 0.3874450994949029,
+      "learning_rate": 3e-06,
+      "loss": 0.4217,
+      "masked_tokens": 101.2,
+      "mean_t": 0.5075790266972036,
+      "step": 460,
+      "student_masked_tokens": 101.2
+    },
+    {
+      "avg_mask_ratio": 0.4478310279415122,
+      "avg_response_length": 224.0952380952381,
+      "avg_student_mask_ratio": 0.4478310279415122,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03232636066334072,
+      "epoch": 1.0042666666666666,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.46496453467603416,
+      "learning_rate": 3e-06,
+      "loss": 0.5059,
+      "masked_tokens": 103.03571428571429,
+      "mean_t": 0.5031429776822084,
+      "step": 470,
+      "student_masked_tokens": 103.03571428571429
+    },
+    {
+      "avg_mask_ratio": 0.44247635449282824,
+      "avg_response_length": 249.4125,
+      "avg_student_mask_ratio": 0.44247635449282824,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04671022014210848,
+      "epoch": 1.0256,
+      "grad_norm": 0.1630859375,
+      "kd_loss": 0.5139326051962939,
+      "learning_rate": 3e-06,
+      "loss": 0.5147,
+      "masked_tokens": 112.8125,
+      "mean_t": 0.504472183593316,
+      "step": 480,
+      "student_masked_tokens": 112.8125
+    },
+    {
+      "avg_mask_ratio": 0.42829811533447354,
+      "avg_response_length": 232.0375,
+      "avg_student_mask_ratio": 0.42829811533447354,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005066035011736858,
+      "epoch": 1.0469333333333333,
+      "grad_norm": 0.1630859375,
+      "kd_loss": 0.4333162663751821,
+      "learning_rate": 3e-06,
+      "loss": 0.4199,
+      "masked_tokens": 94.7125,
+      "mean_t": 0.4888980514719151,
+      "step": 490,
+      "student_masked_tokens": 94.7125
+    },
+    {
+      "avg_mask_ratio": 0.48132704342715443,
+      "avg_response_length": 253.7875,
+      "avg_student_mask_ratio": 0.48132704342715443,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01742940873630232,
+      "epoch": 1.0682666666666667,
+      "grad_norm": 0.169921875,
+      "kd_loss": 0.5254742230891452,
+      "learning_rate": 3e-06,
+      "loss": 0.508,
+      "masked_tokens": 124.275,
+      "mean_t": 0.5469163245841628,
+      "step": 500,
+      "student_masked_tokens": 124.275
+    },
+    {
+      "avg_mask_ratio": 0.46568706033285706,
+      "avg_response_length": 254.2,
+      "avg_student_mask_ratio": 0.46568706033285706,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.014650851830197098,
+      "epoch": 1.0896,
+      "grad_norm": 0.2353515625,
+      "kd_loss": 0.5521236633266653,
+      "learning_rate": 3e-06,
+      "loss": 0.5066,
+      "masked_tokens": 133.0,
+      "mean_t": 0.5275314710394013,
+      "step": 510,
+      "student_masked_tokens": 133.0
+    },
+    {
+      "avg_mask_ratio": 0.4205072323558852,
+      "avg_response_length": 202.7625,
+      "avg_student_mask_ratio": 0.4205072323558852,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010721822535629144,
+      "epoch": 1.1109333333333333,
+      "grad_norm": 0.5546875,
+      "kd_loss": 0.4538552779665597,
+      "learning_rate": 3e-06,
+      "loss": 0.4843,
+      "masked_tokens": 83.1875,
+      "mean_t": 0.49193521235138177,
+      "step": 520,
+      "student_masked_tokens": 83.1875
+    },
+    {
+      "avg_mask_ratio": 0.44365208947565404,
+      "avg_response_length": 214.275,
+      "avg_student_mask_ratio": 0.44365208947565404,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.012283863066681989,
+      "epoch": 1.1322666666666668,
+      "grad_norm": 0.31640625,
+      "kd_loss": 0.5291487223667846,
+      "learning_rate": 3e-06,
+      "loss": 0.6198,
+      "masked_tokens": 95.475,
+      "mean_t": 0.49621942077938,
+      "step": 530,
+      "student_masked_tokens": 95.475
+    },
+    {
+      "avg_mask_ratio": 0.4564033471979201,
+      "avg_response_length": 231.3125,
+      "avg_student_mask_ratio": 0.4564033471979201,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02264503003948448,
+      "epoch": 1.1536,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.43877944423244913,
+      "learning_rate": 3e-06,
+      "loss": 0.434,
+      "masked_tokens": 102.1125,
+      "mean_t": 0.5208023569080978,
+      "step": 540,
+      "student_masked_tokens": 102.1125
+    },
+    {
+      "avg_mask_ratio": 0.4085368953179568,
+      "avg_response_length": 213.525,
+      "avg_student_mask_ratio": 0.4085368953179568,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.011074162583918223,
+      "epoch": 1.1749333333333334,
+      "grad_norm": 0.1953125,
+      "kd_loss": 0.41639115329383997,
+      "learning_rate": 3e-06,
+      "loss": 0.4289,
+      "masked_tokens": 92.7125,
+      "mean_t": 0.4767197913257405,
+      "step": 550,
+      "student_masked_tokens": 92.7125
+    },
+    {
+      "avg_mask_ratio": 0.36651339489617385,
+      "avg_response_length": 231.4125,
+      "avg_student_mask_ratio": 0.36651339489617385,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010163615908072642,
+      "epoch": 1.1962666666666666,
+      "grad_norm": 0.3671875,
+      "kd_loss": 0.47027518333145935,
+      "learning_rate": 3e-06,
+      "loss": 0.4476,
+      "masked_tokens": 95.2375,
+      "mean_t": 0.4111072298779618,
+      "step": 560,
+      "student_masked_tokens": 95.2375
+    },
+    {
+      "avg_mask_ratio": 0.4171164375031367,
+      "avg_response_length": 230.1625,
+      "avg_student_mask_ratio": 0.4171164375031367,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004821049512975151,
+      "epoch": 1.2176,
+      "grad_norm": 0.2890625,
+      "kd_loss": 0.37785149019453,
+      "learning_rate": 3e-06,
+      "loss": 0.4009,
+      "masked_tokens": 103.925,
+      "mean_t": 0.48533305872697385,
+      "step": 570,
+      "student_masked_tokens": 103.925
+    },
+    {
+      "avg_mask_ratio": 0.4461298058507964,
+      "avg_response_length": 267.4375,
+      "avg_student_mask_ratio": 0.4461298058507964,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.03442675815317671,
+      "epoch": 1.2389333333333332,
+      "grad_norm": 0.61328125,
+      "kd_loss": 0.5179231762156633,
+      "learning_rate": 3e-06,
+      "loss": 0.5121,
+      "masked_tokens": 126.825,
+      "mean_t": 0.49976949762785805,
+      "step": 580,
+      "student_masked_tokens": 126.825
+    },
+    {
+      "avg_mask_ratio": 0.4900112466188148,
+      "avg_response_length": 228.45,
+      "avg_student_mask_ratio": 0.4900112466188148,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0073458934444261105,
+      "epoch": 1.2602666666666666,
+      "grad_norm": 0.251953125,
+      "kd_loss": 0.6004035866773847,
+      "learning_rate": 3e-06,
+      "loss": 0.5784,
+      "masked_tokens": 116.15,
+      "mean_t": 0.5521843038732186,
+      "step": 590,
+      "student_masked_tokens": 116.15
+    },
+    {
+      "avg_mask_ratio": 0.42360459361225367,
+      "avg_response_length": 236.7,
+      "avg_student_mask_ratio": 0.42360459361225367,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005112273251961596,
+      "epoch": 1.2816,
+      "grad_norm": 0.263671875,
+      "kd_loss": 0.40553972261057536,
+      "learning_rate": 3e-06,
+      "loss": 0.3902,
+      "masked_tokens": 105.8375,
+      "mean_t": 0.4788527532829903,
+      "step": 600,
+      "student_masked_tokens": 105.8375
+    },
+    {
+      "avg_mask_ratio": 0.47793483664281666,
+      "avg_response_length": 230.3375,
+      "avg_student_mask_ratio": 0.47793483664281666,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.013057019122277325,
+      "epoch": 1.3029333333333333,
+      "grad_norm": 0.15234375,
+      "kd_loss": 0.6178492149250815,
+      "learning_rate": 3e-06,
+      "loss": 0.5852,
+      "masked_tokens": 108.85,
+      "mean_t": 0.5445419924799353,
+      "step": 610,
+      "student_masked_tokens": 108.85
+    },
+    {
+      "avg_mask_ratio": 0.4574313384189736,
+      "avg_response_length": 233.0625,
+      "avg_student_mask_ratio": 0.4574313384189736,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02562547706791065,
+      "epoch": 1.3242666666666667,
+      "grad_norm": 0.1728515625,
+      "kd_loss": 0.5067496699713558,
+      "learning_rate": 3e-06,
+      "loss": 0.4958,
+      "masked_tokens": 105.0,
+      "mean_t": 0.52490478400141,
+      "step": 620,
+      "student_masked_tokens": 105.0
+    },
+    {
+      "avg_mask_ratio": 0.4954570603906177,
+      "avg_response_length": 257.2125,
+      "avg_student_mask_ratio": 0.4954570603906177,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.009351487243975498,
+      "epoch": 1.3456000000000001,
+      "grad_norm": 0.41015625,
+      "kd_loss": 0.425488157782544,
+      "learning_rate": 3e-06,
+      "loss": 0.5267,
+      "masked_tokens": 125.5875,
+      "mean_t": 0.5564947265549562,
+      "step": 630,
+      "student_masked_tokens": 125.5875
+    },
+    {
+      "avg_mask_ratio": 0.45939837526530025,
+      "avg_response_length": 237.7125,
+      "avg_student_mask_ratio": 0.45939837526530025,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.018259476403090957,
+      "epoch": 1.3669333333333333,
+      "grad_norm": 0.1328125,
+      "kd_loss": 0.38472498857565823,
+      "learning_rate": 3e-06,
+      "loss": 0.3739,
+      "masked_tokens": 116.825,
+      "mean_t": 0.5093393943971023,
+      "step": 640,
+      "student_masked_tokens": 116.825
+    },
+    {
+      "avg_mask_ratio": 0.49772885923739524,
+      "avg_response_length": 238.0125,
+      "avg_student_mask_ratio": 0.49772885923739524,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.013440046917298786,
+      "epoch": 1.3882666666666665,
+      "grad_norm": 0.216796875,
+      "kd_loss": 0.5762969084854006,
+      "learning_rate": 3e-06,
+      "loss": 0.5345,
+      "masked_tokens": 127.6875,
+      "mean_t": 0.5558586571365595,
+      "step": 650,
+      "student_masked_tokens": 127.6875
+    },
+    {
+      "avg_mask_ratio": 0.4574655814212747,
+      "avg_response_length": 247.775,
+      "avg_student_mask_ratio": 0.4574655814212747,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.027927246921797176,
+      "epoch": 1.4096,
+      "grad_norm": 0.640625,
+      "kd_loss": 0.4334344154445716,
+      "learning_rate": 3e-06,
+      "loss": 0.5146,
+      "masked_tokens": 124.35,
+      "mean_t": 0.51307404555846,
+      "step": 660,
+      "student_masked_tokens": 124.35
+    },
+    {
+      "avg_mask_ratio": 0.4189265530381817,
+      "avg_response_length": 240.4375,
+      "avg_student_mask_ratio": 0.4189265530381817,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007356403671803946,
+      "epoch": 1.4309333333333334,
+      "grad_norm": 0.3046875,
+      "kd_loss": 0.41319570616201984,
+      "learning_rate": 3e-06,
+      "loss": 0.3941,
+      "masked_tokens": 115.0875,
+      "mean_t": 0.4907285622088239,
+      "step": 670,
+      "student_masked_tokens": 115.0875
+    },
+    {
+      "avg_mask_ratio": 0.4405042894242797,
+      "avg_response_length": 226.7375,
+      "avg_student_mask_ratio": 0.4405042894242797,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02681735201608717,
+      "epoch": 1.4522666666666666,
+      "grad_norm": 0.212890625,
+      "kd_loss": 0.5227603636610183,
+      "learning_rate": 3e-06,
+      "loss": 0.4925,
+      "masked_tokens": 103.0875,
+      "mean_t": 0.4913603452499956,
+      "step": 680,
+      "student_masked_tokens": 103.0875
+    },
+    {
+      "avg_mask_ratio": 0.45782678785035386,
+      "avg_response_length": 202.05,
+      "avg_student_mask_ratio": 0.45782678785035386,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005062625911909891,
+      "epoch": 1.4736,
+      "grad_norm": 0.1201171875,
+      "kd_loss": 0.45889621370954503,
+      "learning_rate": 3e-06,
+      "loss": 0.4633,
+      "masked_tokens": 108.025,
+      "mean_t": 0.5133644798654131,
+      "step": 690,
+      "student_masked_tokens": 108.025
+    },
+    {
+      "avg_mask_ratio": 0.4901438815984875,
+      "avg_response_length": 228.125,
+      "avg_student_mask_ratio": 0.4901438815984875,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.017818167519862982,
+      "epoch": 1.4949333333333334,
+      "grad_norm": 0.2119140625,
+      "kd_loss": 0.6823521373159338,
+      "learning_rate": 3e-06,
+      "loss": 0.5921,
+      "masked_tokens": 126.525,
+      "mean_t": 0.5625698395539075,
+      "step": 700,
+      "student_masked_tokens": 126.525
+    },
+    {
+      "avg_mask_ratio": 0.44477287216577677,
+      "avg_response_length": 212.5,
+      "avg_student_mask_ratio": 0.44477287216577677,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005565753509768001,
+      "epoch": 1.5162666666666667,
+      "grad_norm": 0.1923828125,
+      "kd_loss": 0.4292566096216433,
+      "learning_rate": 3e-06,
+      "loss": 0.4259,
+      "masked_tokens": 104.525,
+      "mean_t": 0.5135623761918395,
+      "step": 710,
+      "student_masked_tokens": 104.525
+    },
+    {
+      "avg_mask_ratio": 0.4493113035801798,
+      "avg_response_length": 227.075,
+      "avg_student_mask_ratio": 0.4493113035801798,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007549421283908941,
+      "epoch": 1.5375999999999999,
+      "grad_norm": 0.482421875,
+      "kd_loss": 0.508985077529897,
+      "learning_rate": 3e-06,
+      "loss": 0.5035,
+      "masked_tokens": 106.65,
+      "mean_t": 0.5117021896177902,
+      "step": 720,
+      "student_masked_tokens": 106.65
+    },
+    {
+      "avg_mask_ratio": 0.44044942427426576,
+      "avg_response_length": 232.925,
+      "avg_student_mask_ratio": 0.44044942427426576,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004612203660190062,
+      "epoch": 1.5589333333333333,
+      "grad_norm": 0.26171875,
+      "kd_loss": 0.46282253071470336,
+      "learning_rate": 3e-06,
+      "loss": 0.4677,
+      "masked_tokens": 100.2875,
+      "mean_t": 0.49050743713742123,
+      "step": 730,
+      "student_masked_tokens": 100.2875
+    },
+    {
+      "avg_mask_ratio": 0.48421492378693076,
+      "avg_response_length": 183.325,
+      "avg_student_mask_ratio": 0.48421492378693076,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0072626265307462745,
+      "epoch": 1.5802666666666667,
+      "grad_norm": 0.1943359375,
+      "kd_loss": 0.5804909225802021,
+      "learning_rate": 3e-06,
+      "loss": 0.5679,
+      "masked_tokens": 91.6875,
+      "mean_t": 0.5505168779753149,
+      "step": 740,
+      "student_masked_tokens": 91.6875
+    },
+    {
+      "avg_mask_ratio": 0.3897687786899041,
+      "avg_response_length": 223.65,
+      "avg_student_mask_ratio": 0.3897687786899041,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.005423597515937217,
+      "epoch": 1.6016,
+      "grad_norm": 0.275390625,
+      "kd_loss": 0.46322623784844835,
+      "learning_rate": 3e-06,
+      "loss": 0.4614,
+      "masked_tokens": 90.4625,
+      "mean_t": 0.4437690361432033,
+      "step": 750,
+      "student_masked_tokens": 90.4625
+    },
+    {
+      "avg_mask_ratio": 0.43048173217976,
+      "avg_response_length": 216.0625,
+      "avg_student_mask_ratio": 0.43048173217976,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004789848048892509,
+      "epoch": 1.6229333333333333,
+      "grad_norm": 0.1708984375,
+      "kd_loss": 0.5197787190380041,
+      "learning_rate": 3e-06,
+      "loss": 0.4946,
+      "masked_tokens": 102.5875,
+      "mean_t": 0.49132869170280175,
+      "step": 760,
+      "student_masked_tokens": 102.5875
+    },
+    {
+      "avg_mask_ratio": 0.41878489739028735,
+      "avg_response_length": 259.675,
+      "avg_student_mask_ratio": 0.41878489739028735,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.006974586636010827,
+      "epoch": 1.6442666666666668,
+      "grad_norm": 0.19921875,
+      "kd_loss": 0.4032763403680336,
+      "learning_rate": 3e-06,
+      "loss": 0.3384,
+      "masked_tokens": 113.825,
+      "mean_t": 0.4667695587326307,
+      "step": 770,
+      "student_masked_tokens": 113.825
+    },
+    {
+      "avg_mask_ratio": 0.39619951823260635,
+      "avg_response_length": 258.5125,
+      "avg_student_mask_ratio": 0.39619951823260635,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007122849350025717,
+      "epoch": 1.6656,
+      "grad_norm": 0.2412109375,
+      "kd_loss": 0.38649988778331124,
+      "learning_rate": 3e-06,
+      "loss": 0.3997,
+      "masked_tokens": 113.9,
+      "mean_t": 0.44942845597106496,
+      "step": 780,
+      "student_masked_tokens": 113.9
+    },
+    {
+      "avg_mask_ratio": 0.43434095685952345,
+      "avg_response_length": 222.3125,
+      "avg_student_mask_ratio": 0.43434095685952345,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.009959504639243732,
+      "epoch": 1.6869333333333332,
+      "grad_norm": 0.146484375,
+      "kd_loss": 0.31865618440902316,
+      "learning_rate": 3e-06,
+      "loss": 0.3552,
+      "masked_tokens": 98.775,
+      "mean_t": 0.4993515375303105,
+      "step": 790,
+      "student_masked_tokens": 98.775
+    },
+    {
+      "avg_mask_ratio": 0.42826264500617983,
+      "avg_response_length": 219.6875,
+      "avg_student_mask_ratio": 0.42826264500617983,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.002781124600801377,
+      "epoch": 1.7082666666666668,
+      "grad_norm": 0.1416015625,
+      "kd_loss": 0.3458431469396146,
+      "learning_rate": 3e-06,
+      "loss": 0.4078,
+      "masked_tokens": 99.1125,
+      "mean_t": 0.500370389316231,
+      "step": 800,
+      "student_masked_tokens": 99.1125
+    },
+    {
+      "avg_mask_ratio": 0.45382156379055233,
+      "avg_response_length": 237.85,
+      "avg_student_mask_ratio": 0.45382156379055233,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007012360586395516,
+      "epoch": 1.7296,
+      "grad_norm": 0.3125,
+      "kd_loss": 0.4872051531401212,
+      "learning_rate": 3e-06,
+      "loss": 0.5518,
+      "masked_tokens": 113.4625,
+      "mean_t": 0.5317009104182944,
+      "step": 810,
+      "student_masked_tokens": 113.4625
+    },
+    {
+      "avg_mask_ratio": 0.4765406621154398,
+      "avg_response_length": 209.0875,
+      "avg_student_mask_ratio": 0.4765406621154398,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.008087280945187559,
+      "epoch": 1.7509333333333332,
+      "grad_norm": 0.291015625,
+      "kd_loss": 0.582653931081586,
+      "learning_rate": 3e-06,
+      "loss": 0.568,
+      "masked_tokens": 116.15,
+      "mean_t": 0.5392061032878701,
+      "step": 820,
+      "student_masked_tokens": 116.15
+    },
+    {
+      "avg_mask_ratio": 0.4660664959577844,
+      "avg_response_length": 263.975,
+      "avg_student_mask_ratio": 0.4660664959577844,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004976632978514317,
+      "epoch": 1.7722666666666667,
+      "grad_norm": 0.361328125,
+      "kd_loss": 0.6404682535923711,
+      "learning_rate": 3e-06,
+      "loss": 0.6111,
+      "masked_tokens": 130.85,
+      "mean_t": 0.5238314627087675,
+      "step": 830,
+      "student_masked_tokens": 130.85
+    },
+    {
+      "avg_mask_ratio": 0.46782192731043326,
+      "avg_response_length": 221.9,
+      "avg_student_mask_ratio": 0.46782192731043326,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004557234377546138,
+      "epoch": 1.7936,
+      "grad_norm": 0.0908203125,
+      "kd_loss": 0.4399012086084646,
+      "learning_rate": 3e-06,
+      "loss": 0.4424,
+      "masked_tokens": 110.3625,
+      "mean_t": 0.5301066277665086,
+      "step": 840,
+      "student_masked_tokens": 110.3625
+    },
+    {
+      "avg_mask_ratio": 0.4748796411789954,
+      "avg_response_length": 225.5,
+      "avg_student_mask_ratio": 0.4748796411789954,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.008075677036322304,
+      "epoch": 1.8149333333333333,
+      "grad_norm": 0.1806640625,
+      "kd_loss": 0.34330057573731665,
+      "learning_rate": 3e-06,
+      "loss": 0.4022,
+      "masked_tokens": 112.0625,
+      "mean_t": 0.5343429344706238,
+      "step": 850,
+      "student_masked_tokens": 112.0625
+    },
+    {
+      "avg_mask_ratio": 0.4120468311710283,
+      "avg_response_length": 229.125,
+      "avg_student_mask_ratio": 0.4120468311710283,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0052631380136517695,
+      "epoch": 1.8362666666666667,
+      "grad_norm": 0.1884765625,
+      "kd_loss": 0.41644778757299716,
+      "learning_rate": 3e-06,
+      "loss": 0.4304,
+      "masked_tokens": 106.2125,
+      "mean_t": 0.4791536889737472,
+      "step": 860,
+      "student_masked_tokens": 106.2125
+    },
+    {
+      "avg_mask_ratio": 0.4631627084803768,
+      "avg_response_length": 229.1,
+      "avg_student_mask_ratio": 0.4631627084803768,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.011806862431546961,
+      "epoch": 1.8576000000000001,
+      "grad_norm": 0.498046875,
+      "kd_loss": 0.5520484269072569,
+      "learning_rate": 3e-06,
+      "loss": 0.5795,
+      "masked_tokens": 111.6,
+      "mean_t": 0.5203817339061061,
+      "step": 870,
+      "student_masked_tokens": 111.6
+    },
+    {
+      "avg_mask_ratio": 0.42117048120126127,
+      "avg_response_length": 198.5625,
+      "avg_student_mask_ratio": 0.42117048120126127,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010472421555880374,
+      "epoch": 1.8789333333333333,
+      "grad_norm": 1.390625,
+      "kd_loss": 0.5054981482934636,
+      "learning_rate": 3e-06,
+      "loss": 0.4881,
+      "masked_tokens": 87.725,
+      "mean_t": 0.4875184997683391,
+      "step": 880,
+      "student_masked_tokens": 87.725
+    },
+    {
+      "avg_mask_ratio": 0.4137770858884323,
+      "avg_response_length": 225.8375,
+      "avg_student_mask_ratio": 0.4137770858884323,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.00998285402735064,
+      "epoch": 1.9002666666666665,
+      "grad_norm": 0.126953125,
+      "kd_loss": 0.4549323421288364,
+      "learning_rate": 3e-06,
+      "loss": 0.4288,
+      "masked_tokens": 96.85,
+      "mean_t": 0.4766692223958671,
+      "step": 890,
+      "student_masked_tokens": 96.85
+    },
+    {
+      "avg_mask_ratio": 0.4260695957229473,
+      "avg_response_length": 227.7875,
+      "avg_student_mask_ratio": 0.4260695957229473,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02715508515102556,
+      "epoch": 1.9216,
+      "grad_norm": 0.23828125,
+      "kd_loss": 0.44619655802651437,
+      "learning_rate": 3e-06,
+      "loss": 0.4721,
+      "masked_tokens": 99.1,
+      "mean_t": 0.487134758150205,
+      "step": 900,
+      "student_masked_tokens": 99.1
+    },
+    {
+      "avg_mask_ratio": 0.45544130907510405,
+      "avg_response_length": 245.1625,
+      "avg_student_mask_ratio": 0.45544130907510405,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010313108547143202,
+      "epoch": 1.9429333333333334,
+      "grad_norm": 0.435546875,
+      "kd_loss": 0.47350807999975475,
+      "learning_rate": 3e-06,
+      "loss": 0.4303,
+      "masked_tokens": 120.1875,
+      "mean_t": 0.5027793228859082,
+      "step": 910,
+      "student_masked_tokens": 120.1875
+    },
+    {
+      "avg_mask_ratio": 0.43869577182922515,
+      "avg_response_length": 226.2875,
+      "avg_student_mask_ratio": 0.43869577182922515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0427373267434632,
+      "epoch": 1.9642666666666666,
+      "grad_norm": 0.23046875,
+      "kd_loss": 0.4529029980097164,
+      "learning_rate": 3e-06,
+      "loss": 0.4814,
+      "masked_tokens": 104.7375,
+      "mean_t": 0.49417946098838,
+      "step": 920,
+      "student_masked_tokens": 104.7375
+    },
+    {
+      "avg_mask_ratio": 0.4477143040392548,
+      "avg_response_length": 237.05,
+      "avg_student_mask_ratio": 0.4477143040392548,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.01040498305607116,
+      "epoch": 1.9856,
+      "grad_norm": 0.2197265625,
+      "kd_loss": 0.4587304208488604,
+      "learning_rate": 3e-06,
+      "loss": 0.4397,
+      "masked_tokens": 115.1125,
+      "mean_t": 0.5045580042526125,
+      "step": 930,
+      "student_masked_tokens": 115.1125
+    },
+    {
+      "avg_mask_ratio": 0.46240209431077045,
+      "avg_response_length": 224.79761904761904,
+      "avg_student_mask_ratio": 0.46240209431077045,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0032689166722602316,
+      "epoch": 2.0085333333333333,
+      "grad_norm": 0.3828125,
+      "kd_loss": 0.5169763166982302,
+      "learning_rate": 3e-06,
+      "loss": 0.5523,
+      "masked_tokens": 117.0,
+      "mean_t": 0.5321138524893849,
+      "step": 940,
+      "student_masked_tokens": 117.0
+    },
+    {
+      "avg_mask_ratio": 0.4168280088924803,
+      "avg_response_length": 240.9125,
+      "avg_student_mask_ratio": 0.4168280088924803,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.021347658315528407,
+      "epoch": 2.0298666666666665,
+      "grad_norm": 0.28125,
+      "kd_loss": 0.5328356460729651,
+      "learning_rate": 3e-06,
+      "loss": 0.4641,
+      "masked_tokens": 108.0125,
+      "mean_t": 0.4632946296595037,
+      "step": 950,
+      "student_masked_tokens": 108.0125
+    },
+    {
+      "avg_mask_ratio": 0.4622723014326766,
+      "avg_response_length": 275.3,
+      "avg_student_mask_ratio": 0.4622723014326766,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0057922464925923125,
+      "epoch": 2.0512,
+      "grad_norm": 0.1064453125,
+      "kd_loss": 0.6181535236281206,
+      "learning_rate": 3e-06,
+      "loss": 0.5813,
+      "masked_tokens": 137.475,
+      "mean_t": 0.5198000721400604,
+      "step": 960,
+      "student_masked_tokens": 137.475
+    },
+    {
+      "avg_mask_ratio": 0.4141947620024439,
+      "avg_response_length": 236.2375,
+      "avg_student_mask_ratio": 0.4141947620024439,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0025189679287393575,
+      "epoch": 2.0725333333333333,
+      "grad_norm": 0.58984375,
+      "kd_loss": 0.44248166069168204,
+      "learning_rate": 3e-06,
+      "loss": 0.4533,
+      "masked_tokens": 103.0625,
+      "mean_t": 0.4703940597362816,
+      "step": 970,
+      "student_masked_tokens": 103.0625
+    },
+    {
+      "avg_mask_ratio": 0.3987298493855633,
+      "avg_response_length": 230.8625,
+      "avg_student_mask_ratio": 0.3987298493855633,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004547219713845152,
+      "epoch": 2.0938666666666665,
+      "grad_norm": 0.173828125,
+      "kd_loss": 0.31845395617929173,
+      "learning_rate": 3e-06,
+      "loss": 0.3591,
+      "masked_tokens": 94.65,
+      "mean_t": 0.4511947895749472,
+      "step": 980,
+      "student_masked_tokens": 94.65
+    },
+    {
+      "avg_mask_ratio": 0.4348693466046825,
+      "avg_response_length": 262.0375,
+      "avg_student_mask_ratio": 0.4348693466046825,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.006510952173971418,
+      "epoch": 2.1152,
+      "grad_norm": 0.11669921875,
+      "kd_loss": 0.391171305554158,
+      "learning_rate": 3e-06,
+      "loss": 0.4005,
+      "masked_tokens": 116.6375,
+      "mean_t": 0.4923786667350214,
+      "step": 990,
+      "student_masked_tokens": 116.6375
+    },
+    {
+      "avg_mask_ratio": 0.42609867608407515,
+      "avg_response_length": 215.3,
+      "avg_student_mask_ratio": 0.42609867608407515,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.010184290214397151,
+      "epoch": 2.1365333333333334,
+      "grad_norm": 0.166015625,
+      "kd_loss": 0.35061151300810706,
+      "learning_rate": 3e-06,
+      "loss": 0.3837,
+      "masked_tokens": 96.2875,
+      "mean_t": 0.4773523230338469,
+      "step": 1000,
+      "student_masked_tokens": 96.2875
+    },
+    {
+      "avg_mask_ratio": 0.4110122739803046,
+      "avg_response_length": 215.0,
+      "avg_student_mask_ratio": 0.4110122739803046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.0055731275298512625,
+      "epoch": 2.1578666666666666,
+      "grad_norm": 0.15625,
+      "kd_loss": 0.3826096343551285,
+      "learning_rate": 3e-06,
+      "loss": 0.4,
+      "masked_tokens": 85.1,
+      "mean_t": 0.4648138735938119,
+      "step": 1010,
+      "student_masked_tokens": 85.1
+    },
+    {
+      "avg_mask_ratio": 0.4742612716276199,
+      "avg_response_length": 206.9125,
+      "avg_student_mask_ratio": 0.4742612716276199,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.009519673228305691,
+      "epoch": 2.1792,
+      "grad_norm": 0.384765625,
+      "kd_loss": 0.4247023156226078,
+      "learning_rate": 3e-06,
+      "loss": 0.4927,
+      "masked_tokens": 105.725,
+      "mean_t": 0.5327763411332853,
+      "step": 1020,
+      "student_masked_tokens": 105.725
+    },
+    {
+      "avg_mask_ratio": 0.4356532185454853,
+      "avg_response_length": 220.05,
+      "avg_student_mask_ratio": 0.4356532185454853,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004953916401620973,
+      "epoch": 2.2005333333333335,
+      "grad_norm": 0.1640625,
+      "kd_loss": 0.46281327859967336,
+      "learning_rate": 3e-06,
+      "loss": 0.4761,
+      "masked_tokens": 100.425,
+      "mean_t": 0.5033508580760099,
+      "step": 1030,
+      "student_masked_tokens": 100.425
+    },
+    {
+      "avg_mask_ratio": 0.4579320074350107,
+      "avg_response_length": 213.7,
+      "avg_student_mask_ratio": 0.4579320074350107,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.004975110809772332,
+      "epoch": 2.2218666666666667,
+      "grad_norm": 0.25,
+      "kd_loss": 0.5819871042260786,
+      "learning_rate": 3e-06,
+      "loss": 0.5189,
+      "masked_tokens": 94.825,
+      "mean_t": 0.5349024560535327,
+      "step": 1040,
+      "student_masked_tokens": 94.825
+    },
+    {
+      "avg_mask_ratio": 0.47517456728965046,
+      "avg_response_length": 239.1125,
+      "avg_student_mask_ratio": 0.47517456728965046,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.02202778593809853,
+      "epoch": 2.2432,
+      "grad_norm": 0.2333984375,
+      "kd_loss": 0.5699342514281994,
+      "learning_rate": 3e-06,
+      "loss": 0.5176,
+      "masked_tokens": 117.05,
+      "mean_t": 0.5457118917722255,
+      "step": 1050,
+      "student_masked_tokens": 117.05
+    },
+    {
+      "avg_mask_ratio": 0.4347405159147456,
+      "avg_response_length": 273.7875,
+      "avg_student_mask_ratio": 0.4347405159147456,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.002644515588644936,
+      "epoch": 2.2645333333333335,
+      "grad_norm": 0.384765625,
+      "kd_loss": 0.4309727882229254,
+      "learning_rate": 3e-06,
+      "loss": 0.4662,
+      "masked_tokens": 116.4375,
+      "mean_t": 0.48194136443780733,
+      "step": 1060,
+      "student_masked_tokens": 116.4375
+    },
+    {
+      "avg_mask_ratio": 0.45182205125456676,
+      "avg_response_length": 260.8125,
+      "avg_student_mask_ratio": 0.45182205125456676,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.019521321286731563,
+      "epoch": 2.2858666666666667,
+      "grad_norm": 0.1669921875,
+      "kd_loss": 0.43615129943457304,
+      "learning_rate": 3e-06,
+      "loss": 0.4436,
+      "masked_tokens": 131.9375,
+      "mean_t": 0.5015889146190602,
+      "step": 1070,
+      "student_masked_tokens": 131.9375
+    },
+    {
+      "avg_mask_ratio": 0.4326194179826416,
+      "avg_response_length": 219.8875,
+      "avg_student_mask_ratio": 0.4326194179826416,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.04336916068134542,
+      "epoch": 2.3072,
+      "grad_norm": 0.10107421875,
+      "kd_loss": 0.38963948390632624,
+      "learning_rate": 3e-06,
+      "loss": 0.3892,
+      "masked_tokens": 96.375,
+      "mean_t": 0.4983203248586506,
+      "step": 1080,
+      "student_masked_tokens": 96.375
+    },
+    {
+      "avg_mask_ratio": 0.41299611830036154,
+      "avg_response_length": 213.55,
+      "avg_student_mask_ratio": 0.41299611830036154,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.007154606581798362,
+      "epoch": 2.3285333333333336,
+      "grad_norm": 0.244140625,
+      "kd_loss": 0.3930492476621577,
+      "learning_rate": 3e-06,
+      "loss": 0.356,
+      "masked_tokens": 86.925,
+      "mean_t": 0.47094749807147307,
+      "step": 1090,
+      "student_masked_tokens": 86.925
+    },
+    {
+      "avg_mask_ratio": 0.49129993114620446,
+      "avg_response_length": 246.1125,
+      "avg_student_mask_ratio": 0.49129993114620446,
+      "batch_ainp_frac": 0.0,
+      "batch_inp_frac": 0.0,
+      "batch_inp_oh_frac": 0.0,
+      "batch_inp_par_frac": 0.0,
+      "batch_inp_par_par_frac": 0.0,
+      "batch_inp_par_reverse_frac": 0.0,
+      "batch_inp_rkd_ace_frac": 1.0,
+      "batch_rl_frac": 0.0,
+      "batch_sft_frac": 0.0,
+      "batch_soft_sft_frac": 0.0,
+      "batch_tf_frac": 0.0,
+      "ce_loss": 0.015378171362749526,
+      "epoch": 2.3498666666666668,
+      "grad_norm": 0.314453125,
+      "kd_loss": 0.46861715018949324,
+      "learning_rate": 3e-06,
+      "loss": 0.5143,
+      "masked_tokens": 128.6375,
+      "mean_t": 0.5531192034482956,
+      "step": 1100,
+      "student_masked_tokens": 128.6375
+    }
+  ],
+  "logging_steps": 10,
+  "max_steps": 1404,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 100,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1100/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac6ad8e89a9dc564b7ec50d42ce5557e5bda1f33609e4fc24c3968508afb13fa
+size 8312

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:de27683b23fed29849d9865ad25e9bda86a5fb86209dc0eaf869e8dcacb175cd
+size 2406624648

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce14246d65d949439df23b9505af8f663520e209089c5bba67d499529265fcb2
+size 671304442

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc2c8fea75e2aecacc04a093bcd4c703c8c46f258988f5f072e4b3101e3e4924
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f560737eefb4d05df766dfc245439bfc749ee327e6b93f630defabec21a5222e
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7f8c95a6d9085dfcee1e6620c88ede526366d3a02c5018932b1bc04809c0e0c7
+size 1064

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1200/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac6ad8e89a9dc564b7ec50d42ce5557e5bda1f33609e4fc24c3968508afb13fa
+size 8312

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/adapter_config.json

@@ -0,0 +1,39 @@
+{
+  "alpha_pattern": {},
+  "auto_mapping": null,
+  "base_model_name_or_path": "GSAI-ML/LLaDA-8B-Instruct",
+  "bias": "none",
+  "corda_config": null,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 64,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "peft_type": "LORA",
+  "r": 128,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "gate_proj",
+    "q_proj",
+    "up_proj",
+    "o_proj",
+    "down_proj",
+    "v_proj",
+    "k_proj"
+  ],
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_dora": false,
+  "use_rslora": false
+}

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c00a7874c9a4e21e5e4334fc692a869f6794a54d0f3ccb76de4a78494cd247e7
+size 2406624648

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:524a4e4afcd420221d9b682a0a33ebed3055ca6d47c4906de94856f638d0636c
+size 671304442

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/rng_state_0.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9baaa48c7a1c79647d102bde8b4ca080750a0fe5f4428e2fa4d882dd7592f3d
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/rng_state_1.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2392914836e1df95f5ffbb59eef9664f5a98300d7adb76338916a3a02543647d
+size 14512

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:368502bb8b2f2d6f452bda7249e88ca57b330ec2ba407ec248613b4300c99d0d
+size 1064

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1300/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ac6ad8e89a9dc564b7ec50d42ce5557e5bda1f33609e4fc24c3968508afb13fa
+size 8312

math/INP_RKD_ACE/unmask_tags_leave_last_step_gold1_target1_ce0.5/checkpoint-1400/README.md

@@ -0,0 +1,202 @@
+---
+base_model: GSAI-ML/LLaDA-8B-Instruct
+library_name: peft
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.15.1